Jump to content

Script that uses a lot of mouse click coordinates

Go to solution Solved by Nine,

Recommended Posts

I'm writing a script that uses a lot of specific mouse clicks to automate tasks on my browser, server system, and email software for work.
Sometimes the system gets updated or someone drags a handle to the left or right and I have to put in new mouse click coordinates.

It would be a lot more easy to label them and put them in some sort of ini file, but then i have to load every X and Y (and sometimes mouse speed) into a variable
when loading the script.

How would you guys go about such a script, any ideas on how to store and load lot's of click coordinates from a file?
There are some pixel color checks, sleeps and other code between the mouse clicks too..

Ideas i had so far:
- string replace new coordinates in the main script with another autoit script that uses a crosshair to show every mouse position.
- manually write an ini file and load every X, Y, Speed into a huge number of variables: $Searchinput_X = iniread...., $Searchinput_Yiniread...., $Searchinput_Speed = iniread....
- split a textfile line by line into an array and ignore labels that begin with ";" for example (a bit like a custom ini file)

These all seem somewhat tedious i guess..

Link to post
Share on other sites
  • Solution

How about extracting all MouseClick from the au3 file and convert them into a ini file (assuming you can associate each MouseClick to a unique name : maybe by having a comment after the MouseClick naming the action).  Once this is done, you could read the ini file into the new Map var type. Each entry would be a string of three elements (e.g. X|Y|Speed).

Link to post
Share on other sites
20 minutes ago, Nine said:

How about extracting all MouseClick from the au3 file and convert them into a ini file (assuming you can associate each MouseClick to a unique name : maybe by having a comment after the MouseClick naming the action).  Once this is done, you could read the ini file into the new Map var type. Each entry would be a string of three elements (e.g. X|Y|Speed).

You mean like an array of strings "x|y|s" and then split them every time by "|"?

Link to post
Share on other sites

Yes.  You can create a MouseClickEx function that would do that for each action.  You would only need to pass the name of the MouseClick, and it would do all (search map, get the 3 param and click the mouse accordingly).

Link to post
Share on other sites
Link to post
Share on other sites
2 hours ago, Nine said:

You could add more info into the ini file : button, sleep time after click, number of click, etc...  It would be a very general clicker that way.

unfortunately "maps" are still in beta and unstable..
i have to make some sort of multidimensional array instead, the thing is.. i use filereadtoarray to get an array of elements from a file, those lines need to be split and after that i need to put them all together into a multidimensional array. it's funny, i need a 2D array and a normal array in a loop to fill up the final multidimensional array :p

Local $n = 0, $x = 1, $y = 2, $s = 3
Local $file = FileReadToArray('nxys.txt')
Local $map[@extended][4]
$map_max = @extended-1

For $i = 0 To $map_max
    $split = StringSplit($file[$i], '|', 3)
    $map[$i][$n] = $split[$n]
    $map[$i][$x] = $split[$x]
    $map[$i][$y] = $split[$y]
    $map[$i][$s] = $split[$s]

Func map_mouse_move($name)
    Local $i
    For $i = 0 To $map_max
        If $map[$i][$n] = $name Then
            MouseMove($map[$i][$x], $map[$i][$y], $map[$i][$s])
            Return True




Link to post
Share on other sites
55 minutes ago, TheAutomator said:

unfortunately "maps" are still in beta and unstable.

Wrong, it is prod in and it is stable.

But if array is what you understand more, i guess you should go ahead with it, but you are missing a good opportunity to learn a new approach.

Link to post
Share on other sites
Posted (edited)
Global $position[]
Func position($label, $x, $y, $speed = 10)
    Local $sub[]
    $sub.x = $x
    $sub.y = $y
    $sub.s = $speed
    $position[$label] = $sub



position('refresh'          , 95, 65)
position('downloads'        , 1823, 63)
position('pdf drag'         , 1608, 121)
position('pdf drop'         , 1368, 505)


Func mouse_move($name)
    Local $key = $position[$name]
    MouseMove($key.x, $key.y, $key.s)

Func mouse_click($name)
    Local $key = $position[$name]
    MouseClick('main', $key.x, $key.y, 1, $key.s)


I included 4 positions as an example, would it be a good idea to read the keys to a variable?
or should I do "mousemove($position[$name].x, $position[$name].y....)"?

If i think correctly, the example code that stores a temp "key" variable would be less work for the cpu right?

Edited by TheAutomator
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By dejhost
      I am trying to automate a software called "LabelImg" (https://pypi.org/project/labelImg/).
      My autoit-script is started once I selected a folder with images within LabelImg. Pressing the button "Next Image" or pressing the shortcut "d" (https://github.com/tzutalin/labelImg#Hotkeys) jumps to the next image in the selected folder. This shall happen once per second.
      #include <Misc.au3> #include <MsgBoxConstants.au3> #include <AutoItConstants.au3> Opt("WinTitleMatchMode", 1) Local $hDLL = DllOpen("user32.dll") While 1 If _IsPressed("1B", $hDLL) Then ExitLoop Else Local $temp = WinActivate("labelImg") ConsoleWrite($temp & @CRLF) If WinActivate("labelImg") Then ConsoleWrite("All Set!" & @CRLF) EndIf ;Send("d") Local $temp = MouseClick($MOUSE_CLICK_RIGHT, 50, 200) If $temp <> 1 Then MsgBox(1, "$temp", $temp) ExitLoop EndIf Sleep(1000) EndIf WEnd DllClose($hDLL) So the Send ("d")-command and the MouseClick are alternative methods to jump to the next image. Both fail.
      Both ConsoleWrite's deliver proper feedback (I continiously get the handle and "All set" ).

      Could you tell me what I'm doing wrong? 
      Thank you.
    • By Rskm
      Hi, recently i created a GUI for some calculations in AutoIT.  The GUI has 2 tabs and on the first tab, it has few inputboxes where i use to click with mouse and start entering the inputs. i recently made some changes in the position of these textboxes. I made this by changing the autoit code file instead of making changes in KODA. Now i cannot click inside these textboxes with mouse. However, i can use TAB key to cycle through/enter input in these input boxes. what could have made the inputboxes not responding to mouseclicks. thanks
    • By lunarlattice
      Hi All,
      I am trying to login into our application using AutoIT. When I launch the application a splash screen appears and the login screen goes behind it. At this point the script pauses and I loose control of everything. I have literally tried everything for the past two days but nothing has worked yet. Any suggestions would be appreciated.
      RunWait("\\mchsv411.siplaceworld.net\splmshare\int00\conf\start_apps\windows\start_nx110.bat en portal_client int00 tc112", "", @SW_MAXIMIZE)
      AutoItSetOption("SendKeyDelay", 200)
      Local $hWnd = WinWaitActive("Teamcenter Login")
      MouseClick("left", @DesktopWidth / 2, @DesktopHeight / 2)
      The splash screen has title "Teamcenter" and is static.
      I tried to click on the center to get control of login window.
      I have tried WinActivate, WinSetState, ControlCommand and a lot of other things.
    • By lonardd
      I have a very strange problem concerning MouseClick function.
      I need to start Control Panel, navigate it on the Display Section (Adjust screen resolution link), click on it, and from the next Dialog choose Intel Graphic tool tab and navigate into it when it opens. 
      I wasn't using MouseClick() at first when I tried to use Control IDs, but I was fed up with the Autoit Window Info poor and inaccurate info (It flickers and the moment I click on the control the control ID and class disappear) so I ended up choosing the easiest way.
      The code I'm posting worked OK until two weeks ago, the mouse clicks were accurately performed and the Script reached the end with no errors...and I was happy.
      All of a sudden, between one try and the other, I noticed the cursor not flying exactly where it was supposed to, namely to the Control Panel ->Display->Adjust screen resolution link   but it clicked some 30 pixels below and some 30 pixels to the left, choosing obviously and undesired function and from that point it screwed the whole thing up. And from that moment onward, it seems I can no longer regain the mouse to click on that sequence.
      Could it be because my Control Panel ->Display form moved slightly from one test to another and therefore I got that small offside?
      If you believe this is the reason, I should then re position the Control Panel ->Display window to 0,0 and recalculate all the clicks. 
      do you have a suggestion?
      Thanks a lot
    • By c1one
      Something simple enough, and I am sure it's an oversight, but I have not been able to track this down. The entire script is attached, but here is the point of failure.
      Note: I am getting the "Error: subscript used on non-accessible variable" but I thought the initial line of:
      AutoItSetOption('MouseCoordMode', 0)
      Should address that issue? 
      ; Install updates (minimized and updating for many minutes)
      ; Updates Not Installed Screen
         ; Wait for updates to complete
      WinWait("ProSeries Update")
         ; Wait just over 9 minutes to ensure popup is ready (test alternative to WinWait)
         ; Set focus on Product Licenses window
      WinActivate("ProSeries Update")
         ; Send Mouse Click to Install Now button
      MouseClick ( "left" [, 581, 362 [, clicks = 1 [, speed = 10]]] )
      ; end Installing Updates screen
      Thanks for any insights!
  • Create New...