Sign in to follow this  
Followers 0
mitchelwb

MouseClick doesn't seem to always be in the right place

16 posts in this topic

#1 ·  Posted (edited)

I've got a simple little script that is trying to step through data in a datagrid.  I have written and rewritten this thing to try and keep it running, but it just keeps coming up with values I just don't think it can get.  I've played with the timing and it seems to run a little better when slowed down, but setting my $delay higher and higher gets me less and less improvement, but starts to really drag the process to a crawl.  Removing the sleeps entirely isn't even that bad. 

Every time I send a ^C to copy a value, I verify it, and then when I'm done with it, I reset the Clipboard with a known value that will never show in my data and reverify the clipboard.  So either my reset/verification is not doing it's job or my mouse double click is not getting the coordinates updated.  Neither of these options are great because I'm running out of ideas how to make sure I get what I want.  About all I can do at this point is attempt my copy, then validate that the clipboard isn't my reset string, isn't zero length, and THEN match it to a date REGEX to see if it's the data from the correct column.  That would be triple checking something that shouldn't need to be checked once.  Scrip and output below:

 

$desiredDate = "2/9/2015"
$yCoord = 290
$yStep = 15
$recDisplay = 9 ;number of rows to read before starting to scroll
$totalRec = 0
$PONList = ""
$delay = 250
$dcDelay = 12
$count = 0
$clipReset = "chicken"


$oCoastal = WinActivate("Find FOC")
WinWaitActive($oCoastal)

if @error Then
    msgBox(1, "winactivate error", @error)
    Exit
endif

AutoItSetOption("MouseCoordMode", 0)
AutoItSetOption("SendKeyDownDelay", 100)

ClipPut("")

While 1
    ;get date
    ;MouseMove(284, $yCoord, 1)
    MouseClick("left", 284, $yCoord, 2, $dcDelay)
    Sleep($delay)

    while 1
        MouseClick("left", 284, $yCoord, 2, $dcDelay)
        ;Sleep($delay)
        Send("^c")
        ;Sleep($delay)
        $thisDate = clipGet()

        if StringCompare($thisDate, $clipReset) <> 0 and StringLen($thisDate) > 0 Then
            resetClip($clipReset)
            ExitLoop
        EndIf
    WEnd

    ConsoleWrite($count & ": Date: " & $thisDate)

    ;check date matches what we want
    ;if not StringCompare($desiredDate, $thisDate) = 0 Then  ;with some date functions, this could be improved to a date range
    if StringRegExp($thisDate, $desiredDate) = 0 Then ;just check to see if the desired date exists inside what was copied
        ;we have reached the end of our records
        MsgBox(1, "Found " & $totalRec & " total records", $PONList)
        Exit ;or exitloop
    endif

    ;get pon
    ;MouseMove(81, $yCoord, 1)
    MouseClick("left", 81, $yCoord, 2, $dcDelay)
    Sleep($delay)

    While 1
        MouseClick("left", 81, $yCoord, 2, $dcDelay)
        ;Sleep($delay)
        Send("^c")
        ;Sleep($delay)
        $thisPON = clipGet()

        if StringCompare($thisPON, $clipReset) <> 0 and StringLen($thisPON) > 0 Then
            resetClip($clipReset)
            ExitLoop
        EndIf
    WEnd

    ConsoleWrite(" PON:" & $thisPON & @CRLF)

    ;Check if we want this record
    if StringCompare(StringMid($thisPON, 10, 1), "D") = 0 Then
        ;We want this record!
        $totalRec += 1
        $PONList = $PONList & ", " & $thisPon
    EndIf

    if $recDisplay >= 1 Then
        ;still in the first 10 records and need to move the mouseclick position down by a record
        $yCoord += $yStep
        $recDisplay -= 1
    Else
        ;reached the bottom of the data grid and need to click the down arrow in the bottom right of the grid to advance one record
        MouseClick("left", 650, 438,1)
        ;Sleep($delay)
    endIf
    ;consolewrite($recDisplay & @CRLF)
    $count += 1
Wend


func resetClip($val = "")
    Do
        ClipPut($val)
    Until StringCompare(ClipGet(), $val) = 0
EndFunc

And here's a sample output. Everything looks great until the last row.  This was not the last row in the data and should not have stopped here, but you can see the PON is either still on the clipboard, or the mouse click did not update the coordinates and copied the MouseClick commands didn't move to the coordinates:

0: Date: 2/9/2015  PON:173714357INN0001
1: Date: 2/9/2015  PON:173708683DID0001
2: Date: 2/9/2015  PON:173708683DID0003
3: Date: 2/9/2015  PON:173546121INN0001
4: Date: 2/9/2015  PON:173701848INN0001
5: Date: 2/9/2015  PON:173454114INN0001
6: Date: 2/9/2015  PON:173708683DID0002
7: Date: 2/9/2015  PON:173714373DID0001
8: Date: 2/9/2015  PON:173709684INN0001
9: Date: 2/9/2015  PON:173621964INN0001
10: Date: 2/9/2015  PON:173710242CHC0001
11: Date: 2/9/2015  PON:855440C    
12: Date: 2/9/2015  PON:173531625INN0001
13: Date: 2/9/2015  PON:173717963DID0001
14: Date: 2/9/2015  PON:173717560DID0001
15: Date: 2/9/2015  PON:173713978DID0001
16: Date: 2/9/2015  PON:173710221INN0001
17: Date: 2/9/2015  PON:173713002INN0001
18: Date: 2/9/2015  PON:173456352RGN0001
19: Date: 2/9/2015  PON:173712598DID0001
20: Date: 2/9/2015  PON:173709807DID0001
21: Date: 2/9/2015  PON:173616984INN0001
22: Date: 2/9/2015  PON:173646606INN0001
23: Date: 2/9/2015  PON:173649988INN0002
24: Date: 2/9/2015  PON:173712524INR0001
25: Date: 2/9/2015  PON:173710904DID0001
26: Date: 2/9/2015  PON:173717004INN0001
27: Date: 2/9/2015  PON:173712871INN0001
28: Date: 2/9/2015  PON:173426776RGN0001
29: Date: 2/9/2015  PON:173418924RGN0001
30: Date: 2/9/2015  PON:173712465INR0001
31: Date: 2/9/2015  PON:173712465INR0002
32: Date: 2/9/2015  PON:173712524INR0002
33: Date: 2/9/2015  PON:173710788INR0001
34: Date: 2/9/2015  PON:173710788INR0002
35: Date: 2/9/2015  PON:173712737DID0001
36: Date: 2/9/2015  PON:173717129INN0001
37: Date: 2/9/2015  PON:173718061DID0001
38: Date: 2/9/2015  PON:173717663INN0001
39: Date: 2/9/2015  PON:173718181DID0001
40: Date: 2/9/2015  PON:173714071DID0001
41: Date: 2/9/2015  PON:173692375INN0002
42: Date: 173692375INN0002

 

Edited by mitchelwb

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

If you want any sort of accuracy, you will stay away from the generic functions like MouseClick, or ClipGet.

Use, ControlGetText instead...or if you are using a browser, use a browser specific function.

As always, specifics help, tell us what you are attempting to do, and on what.

Side note...winwaitactive, with no timeout param, will never return an @error value.  In fact, it doesn't return any @error.  It will also never exit that function until a window matching is active.  Make sure to read the help file for each function to see what they do.

Anyways, you ControlClick, or if it's a browser, _IeAction with action=click.

Edited by jdelaney

IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites

I wish this was a browser.  The data grid I'm trying to pull data from is in a custom built proprietary app that I think was built using Power Builder.  There are two of us working on this and I'm forced to access the app in Citrix (The other guy has a desktop version of it but isn't having much more success).

So if the problem is in using generic version of mouseclick and clipget, what's the alternative? 

Share this post


Link to post
Share on other sites

Use the AutoIt spy tool.  Focus on the control you are manipulating, and then go to the summary tab and post that.  (delete any sensitive info)


IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites

I'm using Citrix to access the app.  I only get the title, handle, and coordinates of the window.  the other guy using a desktop version gets a few more values when clicking on the datagrid, but nothing of use.  I know that's not a very helpful answer, but we've tried using the spy tool to find any advantage we can and there just isn't anything more than a single control handle.  There are no controls or handles inside the data grid. 

If I can get a copy of the Summary tab, I will, but I've got to find a solution ASAP and it may be sometime tomorrow at best before I can get a copy of what he's getting.  I honestly don't think it'll be of any help though.

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

Do keyboard commands do anything?  Can you ControlSend copy commands to grab row data or cell data?  Do arrows keys do anything?

Edited by jdelaney

IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites

I can navigate up and down the data grid one row at a time with the up and down arrows, but I can't navigate through the data.  if I double click to select a value and press down, it moves to the next row, but does not stay focused on the column I was on.  So I have to do the double click on every cell that I want to read a value from.  That's the only way to get the cell I need selected.

Once the cell is selected, double clicking again will highlight a word portion of the value..  In both cases, this is ok.  On the date field, there is a time stamp, but I don't need it and the space between the date and timestamp means that when I double click that coordinate, I only highlight the date.  On the PON field, that's the only value and it's a solid string, so double clicking it highlights the entire PON.  Sending a ^a does not work.  I've tried using arrow keys and shiftdown and up to try and collect it, but it runs too much risk of losing focus and highlighting the whole row and causing it to scroll wildly horizontally.

I did some more testing and basically came to the conclusion that my mouseclicks aren't always getting done.  Even though I send two sets of double clicks (one to select the field, one to highlight the data) I still often don't get a new value.  I'm outputting the value of my clipboard before getting the new value and it's definitely getting cleared, so it's a matter of my mouse clicks just aren't getting placed correctly by Autoit, or that the application is not registering them (or not fast enough)

This is the most recent version that is running and actually seems fairly bulletproof, but it sure seems like I'm having to do a lot of work just to get it to do what it's supposed to do.  You may be able to sense my frustration in the code.

Global $desiredDate = "2/9/2015"
Global $yCoord = 290
Global $yStep = 15
Global $recDisplay = 9  ;number of rows to read before starting to scroll
Global $totalRec = 0
Global $PONList = ""
Global $delay = 250
Global $dcDelay = 12
Global $count = 0
Global $clipReset = "chicken"
Global $pattern = "\d+\/\d+\/\d+"


$oCoastal = WinActivate("Find FOC")
WinWaitActive($oCoastal)

if @error Then
    msgBox(1, "winactivate error", @error)
    Exit
endif

AutoItSetOption("MouseCoordMode", 0)
AutoItSetOption("SendKeyDownDelay", 100)

ClipPut("")

While 1
    ;get date
    MouseClick("left", 284, $yCoord, 2, $dcDelay)
    sleep($delay)

    $thisDate = getMouseDCVal(284, $yCoord, $dcDelay, $pattern)

    ConsoleWrite($count & ": Date: " & $thisDate)

    ;check date matches what we want
    if StringRegExp($thisDate, $desiredDate) = 0 Then ;just check to see if the desired date exists inside what was copied
        ;we have reached the end of our records
        MsgBox(1, "Found " & $totalRec & " total records", $PONList)
        Exit ;or exitloop
    endif

    ;get pon
    MouseClick("left", 81, $yCoord, 2, $dcDelay)
    Sleep($delay)

    $thisPON = getMouseDCVal(81, $yCoord, $dcDelay, $pattern, false)

    ConsoleWrite(" PON:" & $thisPON & @CRLF)

    ;Check if we want this record
    if StringCompare(StringMid($thisPON, 10, 1), "D") = 0 Then
        ;We want this record!
        $totalRec += 1
        $PONList = $PONList & ", " & $thisPon
    EndIf

    if $recDisplay >= 1 Then
        ;still in the first 10 records and need to move the mouseclick position down by a record
        $yCoord += $yStep
        $recDisplay -= 1
    Else
        ;reached the bottom of the data grid and need to click the down arrow in the bottom right of the grid to advance one record
        MouseClick("left", 650, 438,1)
    endIf
    $count += 1
Wend


func resetClip($val = "")
    Do
        ClipPut($val)
    Until StringCompare(ClipGet(), $val) = 0
    Do
        ClipPut($val)
    Until StringCompare(ClipGet(), $val) = 0
    Do
        ClipPut($val)
    Until StringCompare(ClipGet(), $val) = 0
    Local $damnyou = ClipGet()
    ConsoleWrite("  ~" & $damnyou & "~  ")
EndFunc

Func getMouseDCVal($xPos, $yPos, $mcDelay, $matchPattern, $equalsPattern = true)
    Local $copyVal = ""

    while 1
        MouseClick("left", $xPos, $yCoord, 2, $mcDelay)
        Send("^c")
        $copyVal = clipGet()

        $matchResult = StringRegExp($copyVal, $matchPattern)
        if not  then
            if $matchResult = 1 Then
                $matchResult = 0
            Else
                $matchResult = 1
            EndIf

        if StringCompare($copyVal, $clipReset) <> 0 and StringLen($copyVal) > 7 and $matchResult The
        endifn
            resetClip($clipReset)
            return $copyVal
        EndIf
    WEnd

EndFunc

 

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

func resetClip($val = "")
    Do
        ClipPut($val)
    Until StringCompare(ClipGet(), $val) = 0
    Do
        ClipPut($val)
    Until StringCompare(ClipGet(), $val) = 0
    Do
        ClipPut($val)
    Until StringCompare(ClipGet(), $val) = 0
    Local $damnyou = ClipGet()
    ConsoleWrite("  ~" & $damnyou & "~  ")
EndFunc

That is just ridiculous.

func resetClip($val = "")
    Do
        ClipPut($val)
        Sleep(10)
    Until ClipGet() == $val
EndFunc

Should suffice.

If you are having trouble with that not doing what it is supposed to then it suggests some other app is using the clipboard

Edited by JohnOne

AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt   ipify 

Monkey's are, like, natures humans.

Share this post


Link to post
Share on other sites

When you are in a cell, you can do a Control+A to select all the text, right?  If you can do that, then you can fairly accurately grab data.  The less you have to use mouseclicks, the better.  Use the Arrow keys to navigate.


IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites
func resetClip($val = "")
    Do
        ClipPut($val)
    Until StringCompare(ClipGet(), $val) = 0
    Do
        ClipPut($val)
    Until StringCompare(ClipGet(), $val) = 0
    Do
        ClipPut($val)
    Until StringCompare(ClipGet(), $val) = 0
    Local $damnyou = ClipGet()
    ConsoleWrite("  ~" & $damnyou & "~  ")
EndFunc

That is just ridiculous.

func resetClip($val = "")
    Do
        ClipPut($val)
        Sleep(10)
    Until ClipGet() == $val
EndFunc

Should suffice.

If you are having trouble with that not doing what it is supposed to then it suggests some other app is using the clipboard

You're darn right it's ridiculous.  But I was at my wits end trying to make it work like I expected.  And I didn't know if it was a clipboard issue or if it was a click issue.  I knew it wasn't something else using the clipboard though because the value that kept coming up was the previous value I had copied.  I knew that block of code was just dumb, but at least I felt it was pretty safe to assume I had reset the clipboard.  And outputting the value to the console proved it.  I was definitely getting the clipboard reset.  I think the problem was with the clicks.

Share this post


Link to post
Share on other sites

When you are in a cell, you can do a Control+A to select all the text, right?  If you can do that, then you can fairly accurately grab data.  The less you have to use mouseclicks, the better.  Use the Arrow keys to navigate.

that's just it.  Ctrl-A didn't work.  a double click would set focus on the cell, but ctrl-A did not select all.  I could right click on it and Select All was an option, but even on the right click menu, there were no hot keys.  So the option was either to double click a second time to highlight, or to right click and then calculate a second location for another left click to get Select All. 

Believe me, we tried absolutely every key combo we could think of to get it to highlight and let us select fields without having to do the mouse clicks but it just wasn't gonna happen.

 

When I added the regex match to verify that we were pulling the right data, that seemed to finally make it solid.  What I think was happening was that the app UI just wasn't keeping up.  It acted like it had a bad memory leak.  After a fresh reboot, it would run pretty snappy, but within 15 minutes, it would drag.  Popping up the next windows that we needed to go in to when we came across records that needed to be inspected would get slower and slower.  To the point that they sometimes took up to 30 seconds before they would pop up.  And it would take an entire reboot, not just restarting the app to get it flowing again.  And so I think what was happening was that the script we were running was sending the double-click and the UI just wasn't updating.  Even after the second set of double clicks, if the UI still hadn't updated and the copy command was sent, it copied what had previously been highlighted.  That's why I was working so hard to clear the clipboard, I had to prove it was either the clicks or the clipboard, and I couldn't otherwise prove it was the clicks.

 

Share this post


Link to post
Share on other sites

If I were you, I'd write a completely new app to do whatever it is that this lousy app is doing.

You are using it so you obviously know what it does.

if only that were an option. I'm working as a consultant for another company and we're trying to extract data using this terrible system because we can't get direct access to the data.  It would be much easier to do this with a SQL query.

Share this post


Link to post
Share on other sites

we watched the task manager, but being under the gun, we only did it briefly.  The app itself didn't grow, but there was one generic process that did seem to keep growing.  It never really seemed like it grew to a critical size though. 

There was more to it than just this piece though.  In the script that I've posted, we were just looking for the records.  When we found them there was a whole series of steps that had to be done in this app, one of which included another data grid just like the one we were fighting with here.  the deeper we would get in to that process, the more it slowed down as well.  I only posted asking about this portion because the rest of the process had actual control handles we could work with except for the other data grid, but then it was just the same as this logic.

Share this post


Link to post
Share on other sites

Well it all sounds like complete madness, consultant or not, I'd consult them that they're being stupid in pursuing this course.

Take a look into https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/


AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt   ipify 

Monkey's are, like, natures humans.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Similar Content

    • chacoya121
      By chacoya121
      can anyone help me plz
      how can you put mouseclick in if statement or is it possible
      example:
      if Mouseclick = "left" then Tooltip(" you click left mouse") thank you
    • Sandy89
      By Sandy89
      I am trying to drag and drop an item from listview on one side to the parent element in workspace. From list view I selected the text which I want to drag, but  I need to get the coordinates of the selected text.
      I thought of using MouseClickDrag command. is there any other way to do this?
       
    • Mr_Was_geht_sie_das_an
      By Mr_Was_geht_sie_das_an
      Hi,
      i wanna creat an _Imagesearch / Mousemove script for an 3D/FPS Game and have Problems with the mousemove.
       
      In the game like Curveball(2D) its still works but in games like Battlefield (3D) are the absolute coordinates not really helpfull.
      Someone have an idea or an Example for me ?
       
       
       
    • xgjengen
      By xgjengen
      Hey guys!
      Im new when it comes to scripting and i follow some videos on how to make clicks and such. So i will say i know the basics alrady.
      However, im having a really big problem
      When i run the script is works fine with the clicks and all. But if i run it one or two more times it just moves to very inaccurate positions. YES, i have tried mulitple times to change positions and such but no luck. My script is working its just so inaccurate that im unable to use it. Are there any solution for this?
      I have also set the camera to a specific position. I have also tried other camera angles, no luck there either. Making more smoother macro movements does not seem to work aswell.
      I have also tried to have clicks away from the minimap, since i know some youtuber had problem with inaccuracy because of this.
      I came here because i was googling and did not see anyone else having this problem, or mentioning it. So if someone have a solution, i would be happy!! All i have seen are people trying to change the resolution. But it has nothing to do with that. I heard someone mention controlmouseclick. However are there any differences? I have also A window selection in my script so it changes to the window the macro will be running in. Resolution is not changing or anything. I have also tried other macros and same problem there.
       
      Also does anyone know or can teach me how pixel search works? I want the macro to be able to click at the correct character instead of waiting for the character to get in the position its searching for.
    • Gordi300
      By Gordi300
      Hey everyone!! 
      I'm trying to make a program in VB.NET that click inside a picturebox of an app without moving the mouse (using ControlClick).
      My code is this:
      1) ControlClick("Form1", "", "[CLASS:PictureBox; INSTANCE:12]", "left", 1, 5, 5) 2) ControlClick("Form1", "", "[CLASS:PictureBox; INSTANCE:1]", "left", 1, 5, 5) The code is correct  but the problem is that 1) works perfectly and 2) doesn't work. 
      '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
      Here I leave you the summary of each picturebox:
      >>>> Window <<<<
      Title:    Form1
      Class:    Form
      Position:    168, 0
      Size:    1030, 780
      Style:    0x16CA0000
      ExStyle:    0x00040100
      Handle:    0x00000000000B0092
       
      >>>> Control <<<<                PICTUREBOX 12
      Class:    PictureBox
      Instance:    12
      ClassnameNN:    PictureBox12
      Name:    
      Advanced (Class):    [CLASS:PictureBox; INSTANCE:12]
      ID:    28
      Text:    
      Position:    305, 721
      Size:    40, 40
      Style:    0x56010000
      ExStyle:    0x00000004
      Handle:    0x0000000000040520
       
      >>>> Control <<<<                PICTUREBOX 1
      Class:    PictureBox
      Instance:    1
      ClassnameNN:    PictureBox1
      Name:    
      Advanced (Class):    [CLASS:PictureBox; INSTANCE:1]
      ID:    4
      Text:    
      Position:    4, 176
      Size:    700, 533
      Style:    0x56010000
      ExStyle:    0x00000004
      Handle:    0x00000000000E02BE
       
      Thank you for you help!!!