brad3260

ControlClick in apps

2 posts in this topic

Hello everyone, I have been beating my head against the wall all day today over this and am hoping someone can help. I work for a company who makes assistive technology and I have multiple consumers who want to use the Mail app included in Windows 10. Most of these consumers are using eye tracking technology so the MouseClick is not an option since their eyes are constantly controlling the mouse cursor so it would be a battle between that function and their eyes. And hence I need to use ControlClick. I'm having problems with it though, I can't get it to click anything in the app. I've been left clicking on things with no apparent luck. However, when I changed it to right click, something very interesting happens, the window menu (the one you get when you hit ALT + SPACE) opens every time the ControlClick runs, regardless of the X,Y coordinates as if it is right clicking at the top left of the window every time.

Based on the Window Info, the entire app is one big control whose position is 0,40 and the size is 1440,838.

Does anybody have any ideas why I can apparently only click on the top left corner of the app? Any direction is greatly appreciated!

 

Opt('MouseClickDownDelay', 100)
Opt('WinTitleMatchMode', 2)
Local $hWnd = WinActivate('Gmail')
$inc = 30
For $hi = 1 To 10 Step 1
    $hadj = $hi * $inc
    For $i = 1 To 10 Step 1
        $adj = $i * $inc
        $j = ControlClick($hWnd, "", "[CLASS:ApplicationFrameInputSinkWindow; INSTANCE:1]", "Right", 1, $hadj, $adj)
        MouseMove($hadj, $adj, 3)
    Next
Next

 

Share this post


Link to post
Share on other sites

You'll need to use something like IUIAutomation for newer Windows 10 Apps like Mail, Calculator etc...

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • Trisha
      By Trisha
      I tried (using AutoIt) to automate a scenario where i  have to open IE with specific URL on my extranet(Xen Desktop or Citrix ) ,from there i need to download  file and then through WinSCP; I  have to move that file to another location that can be access from my Local system using VMWare.
      I did as far as the above step. Now i want to navigate from my extranet to my local desktop system. But i am not able to move  from there.Mouse click is also not working.Please find the below screenshot highlighted in YELLOW.
       


    • CYCho
      By CYCho
      Browsers seem to have a built-in feature to insert a check box (Prevent this page from creating additional dialogues) into the message box created by an web site. This check box seems to appear when the same message box is triggered in succession within a short span of time. My problem is that AutoIt cannot read the text of the IE11 message box when this check box is included. I started to experence this after my Windows 10 was updated to Anniversay Update. I would like to know if anyone has experienced the same thing and if there is a solution to let the AutoIt read the text.
    • PauloRodrigues
      By PauloRodrigues
      I need to select an item from a SAP combobox, could anyone help me or did something like that? I tried to use some AutoIt functions but without success.
      When i try to identify the combobox with au3Info, this is the return:


      Could anyone help me with this challange?
    • breakbadsp
      By breakbadsp
      AutoIT AU3info doeas not detect all gui objects uniquely for .NET GUIs developed in C#.
      this is not working now i am using COM windows approach for this, But its very difficult.
      Please let me know if anyone has done it before.
    • KB505
      By KB505
      Hello,
       
      I'm starting a script in AutoIT to automize a task in a software. I need to click a several Controls. 
      I'm just starting to learn AutoIT, and discovered that we can use the Window Info tool (au3info) to identify controls precisely and targetting safely a specific button.
      However, when I try it, I realize that most of the controls in my toolbar have the same Control Info (No ID, same class, same instance, ...). The only thing that changes is the ControlClicks Coord but I don't think that it's interesting, I don't see the difference with the Mouse Coordinates. Besides, I was aiming to write a stable script that would work even if the button changes a little bit of place for example. 
      Should I use another tool that Window Info tool to detect this Controls ? Is there another way to identify a Control ? 
      Thank you in advance for your help