Jump to content
brad3260

ControlClick in apps

Recommended Posts

brad3260

Hello everyone, I have been beating my head against the wall all day today over this and am hoping someone can help. I work for a company who makes assistive technology and I have multiple consumers who want to use the Mail app included in Windows 10. Most of these consumers are using eye tracking technology so the MouseClick is not an option since their eyes are constantly controlling the mouse cursor so it would be a battle between that function and their eyes. And hence I need to use ControlClick. I'm having problems with it though, I can't get it to click anything in the app. I've been left clicking on things with no apparent luck. However, when I changed it to right click, something very interesting happens, the window menu (the one you get when you hit ALT + SPACE) opens every time the ControlClick runs, regardless of the X,Y coordinates as if it is right clicking at the top left of the window every time.

Based on the Window Info, the entire app is one big control whose position is 0,40 and the size is 1440,838.

Does anybody have any ideas why I can apparently only click on the top left corner of the app? Any direction is greatly appreciated!

 

Opt('MouseClickDownDelay', 100)
Opt('WinTitleMatchMode', 2)
Local $hWnd = WinActivate('Gmail')
$inc = 30
For $hi = 1 To 10 Step 1
    $hadj = $hi * $inc
    For $i = 1 To 10 Step 1
        $adj = $i * $inc
        $j = ControlClick($hWnd, "", "[CLASS:ApplicationFrameInputSinkWindow; INSTANCE:1]", "Right", 1, $hadj, $adj)
        MouseMove($hadj, $adj, 3)
    Next
Next

 

Share this post


Link to post
Share on other sites
Subz

You'll need to use something like IUIAutomation for newer Windows 10 Apps like Mail, Calculator etc...

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • autotin
      By autotin
      Hello I am a first time user of AutoIT with little coding experience and I would REALLY really appreciate your help.
      I work in a warehouse where I am responsible for data entry along with a slew of other different tasks such as auditing and managing paperwork. Most of my time is tied up entering info to close receipts of product orders (8-12+ hours for thousands of products). The quantity actually produced by the warehouse is sometimes different from the quantity listed on the reciept (in this case 20). I need to change all the build qty for sublocation "1-WIP" from 20 to the actual quantity 22 and set all other numbers as 0. The form is an iframe coded with jQuery and freezes for 1-2 seconds after every entry and refreshes. As such, it doesn't allow me to use tab and I need to click and type every incorrect field.
      I have attached an image as an example from the webapp, FinaleInventory.
      I'm thinking that I probably need a macro that has a message box entry where I can type the corrected number entry I want. Then it uses image/character recognition (Tesseract?) to find "WIP" sublocation =,   Move the mouse over to the right to "Quantity" relative to location of WIP If quantity number = entry, then move to next "WIP" image (Not quite sure how to make Tesseract "find next") Elseif quantity number ≠ entry then click quantity number of "WIP" and type corrected entry Scroll down Loop until bottom of page I would greatly appreciate any help, ideas or pointers for this project. Thank you!
      -A newbie scripter

    • Rammanan
      By Rammanan
      Hi all, Below is my script,
      Pls advise me if any wrong on that....File exit in location but the return -1.
      FileOpen($file, 0)
         $line = FileReadLine($file)
         MsgBox(0,'',$line)
      FileClose($file)
      Local $aArray = _FileListToArrayRec($serverpath, "*|*.txt", $FLTAR_FILES, $FLTAR_RECUR, $FLTAR_SORT, $FLTAR_NOPATH)
      If @error Then
              MsgBox($MB_SYSTEMMODAL, "Ooops!", "No files found")
          Else
          ;_ArrayDisplay($aArray, "Sorted tree")
                 Local $hSearch = FileFindFirstFile($line)
               MsgBox(0,'',$hSearch)
       
    • JNutt
      By JNutt
      I'm trying ot use controlClick to click on button. So I'm using the Window Info Finder tool. But I noticed that the control ID in windowInfo changes each time I restart the app.  
      Am I doing this wrong.
    • Theodoor
      By Theodoor
      Hi,
      I am trying to use the ControlClick in a "save as"-window to select the path of where to save the document.
      I found out that when in press the icon in front of the path, i select the path. So I am trying to press that button (i have attached a print screen of this window).
      While trying to figure out how i had to use the ControlClick, it said somewhere that i could us Au3Info to find out what title,... i should use with the ControlClick.
      The result of the Au3Info I have attached aswel.
      The ControlClick that i tried are these: (and none of these even move the mouse)
      ControlClick("[CLASS:ToolbarWindow32; INSTANCE:4]", "", "", "Left" ,1 ,10,10)
      ControlClick("[CLASS:ToolbarWindow32; INSTANCE:4]", "", "1001", "Left" ,1 ,10,10)
      ControlClick("[CLASS:ToolbarWindow32]", "", "", "Left" ,1 ,10,10)
      ControlClick("Save project as", "", "", "Left" ,1 ,10,10)
      I don't know what I am doing wrong.
      Should I use a controlID? is my windowtitle wrong?
      Can anyone help me?
      Thanks in advance,
      Theo


    • Trisha
      By Trisha
      I tried (using AutoIt) to automate a scenario where i  have to open IE with specific URL on my extranet(Xen Desktop or Citrix ) ,from there i need to download  file and then through WinSCP; I  have to move that file to another location that can be access from my Local system using VMWare.
      I did as far as the above step. Now i want to navigate from my extranet to my local desktop system. But i am not able to move  from there.Mouse click is also not working.Please find the below screenshot highlighted in YELLOW.
       


×