junkew

IUIAutomation MS framework automate chrome, FF, IE, ....

615 posts in this topic

#561 ·  Posted

one impediment at a time.  

UIAutomation is recognizing your object:ILA2:

  • If its slow then it means properties are not good enough and UIAwrappers try to find in a dynamic way which can mean scnanning 100's of objects and take the best guess.
  • Advice is to look at the treewalker examples as the wrappers are only creating an algorithm around that (which can have issues)
    #32770 windows is normally child of the desktop
  • So walk the tree and highlight all elements.

I will check later for you if i can make a small coding example (but those are based on the treewalker examples anyway so please try yourself)

 

Share this post


Link to post
Share on other sites



#562 ·  Posted

Your screenshots are not reachable
Another test
Start chrome, menu tools, save as. Then you have a #32770 on screen
In my W10 system its a child of Chrome_WidgetWin_1 and then below works

_UIA_setVar("oP1","title:=Nieuw tabblad;class:=Chrome_WidgetWin_1")
_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")

_UIA_setVar("oP2","class:=#32770")
_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")

If I do not have the #32770 with right parent structure I see 

  • deep find in subtree class:=#32770;RTI.SEARCHCONTEXT

This deep finding you can prevent by adding a return in uiawrappers just after (but offcourse then it will not try to search thru all objects of the system)

If Not _UIA_IsElement($oElement) Then

                consolewrite("  deep find in subtree " & $tPhysical & ";" & $strStartElement & @CRLF)
                _UIA_LOG("  deep find in subtree " & $tPhysical & @CRLF, $UIA_Log_Wrapper)
;~              return
                $oElement = _UIA_getObjectByFindAll($oStart, $tPhysical, $treescope_subtree)
            EndIf

Try this with some trial/error and maybe even remove title property if you are sure its the only Qt application
As in each step it will just search childs you can start at highest level (below desktop) for first QWidget

_UIA_setVar("oP1","title:=CLO.*;class:=QWidget")
_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")

_UIA_setVar("oP2","class:=QWidget")
_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")

by counting / index number you can also reach your window but all a little more work

_UIA_setVar("oP1","class:=QWidget;index:=1")

Run and try to understand these

  • ex1_Treewalker.au3
  • Dump childs to a log file (see LOGS folder thats created in folder of your script) and inspect the details 
  • _UIA_DumpThemAll($UIA_oDesktop,$treescope_children)

     

 

Share this post


Link to post
Share on other sites

#563 ·  Posted

Hi...

This is a screenshot of the window I'm trying to control: https://www.dropbox.com/s/xmnarbil99ddtqw/screenshot.png?dl=0

There's a big button there (with the label "Scan Sample") which I can control just fine, if I do "UIA_Action" - "Click" on it, it clicks properly.

My problem is with the radio buttons in the middle. If I use simplespy, I get the following code. I can see that it finds the radio button properly, highlights it and everything, but then the click doesn't change the selection of the radio button...  (of course that manually clicking on this UI element does work). There's no errors reported in the logs, it just says that it's clicking on the UI element.

This is the code I got from simplespy. Any tips would be highly appreciated. Thanks.

;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=NIRscan Nano GUI v2.0.2;controltype:=UIA_WindowControlTypeId;class:=Qt5QWindowIcon;instance:=1")  ;NIRscan Nano GUI v2.0.2
_UIA_setVar("oP2","Title:=;controltype:=UIA_CustomControlTypeId;class:=")   ;
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=") ;
_UIA_setVar("oP4","Title:=;controltype:=UIA_CustomControlTypeId;class:=")   ;
_UIA_setVar("oP5","Title:=;controltype:=UIA_CustomControlTypeId;class:=")   ;
_UIA_setVar("oP6","Title:=;controltype:=UIA_CustomControlTypeId;class:=")   ;
_UIA_setVar("oP7","Title:=;controltype:=UIA_PaneControlTypeId;class:=") ;
_UIA_setVar("oP8","Title:=;controltype:=UIA_CustomControlTypeId;class:=")   ;
_UIA_setVar("oP9","Title:=Scan Config Select;controltype:=UIA_CustomControlTypeId;class:=") ;Scan Config Select
_UIA_setVar("oP10","Title:=Scan Config Select;controltype:=UIA_CustomControlTypeId;class:=")    ;Scan Config Select
_UIA_setVar("oP11","Title:=Scan Reference Select;controltype:=UIA_CustomControlTypeId;class:=") ;Scan Reference Select

;~ $oUIElement=_UIA_getObjectByFindAll("Previous.mainwindow", "title:=Previous;ControlType:=UIA_RadioButtonControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=Previous;controltype:=UIA_RadioButtonControlTypeId;class:=") ;ControlType:=UIA_RadioButtonControlTypeId;classname:=")

;~ Actions split away from logical/technical definition above can come from configfiles

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")
;~_UIA_Action("oP4","highlight")
_UIA_Action("oP4","setfocus")
;~_UIA_Action("oP5","highlight")
_UIA_Action("oP5","setfocus")
;~_UIA_Action("oP6","highlight")
_UIA_Action("oP6","setfocus")
;~_UIA_Action("oP7","highlight")
_UIA_Action("oP7","setfocus")
;~_UIA_Action("oP8","highlight")
_UIA_Action("oP8","setfocus")
;~_UIA_Action("oP9","highlight")
_UIA_Action("oP9","setfocus")
;~_UIA_Action("oP10","highlight")
_UIA_Action("oP10","setfocus")
;~_UIA_Action("oP11","highlight")
_UIA_Action("oP11","setfocus")

;~_UIA_action($oUIElement","highlight")
;~_UIA_action($oUIElement,"click")
;~_UIA_action("oUIElement","highlight")
_UIA_action("oUIElement","click")

 

screenshot.png

Share this post


Link to post
Share on other sites

#564 ·  Posted

To hard to say from a distance. Most likely you only need reference to p1 p11 and uielement. Check also findall treescope descendants to search all levels.

The algorithm i made is not perfect so could be you have to fallback to treewalker examples to get to right element. Spy suggests you are 10 or more levels deep but some you can skip. Try first highligt p1 then p2 and so onwards. Things not highlighting you leave out.

You could try to play with index or instance property to be 1 or 2 or 3 as sometimes same element exists hidden.

Share this post


Link to post
Share on other sites

#565 ·  Posted

I have a button that I need to click on from time to time, it makes no sense to search for it every time. 
what I want to do is to have a script that finds the oUIElement object, stores it somewhere, and then I'll have my main script that I run, which will directly access that object.
So I want my pseudo code to look like:

$obj1 = fetchObject([object 1])
_UIA_action($obj1,"click")

$obj2 = fetchObject([object 2])
_UIA_action($obj2,"click")

 

Can that be done?

Share this post


Link to post
Share on other sites

#566 ·  Posted

yes, simplespy shows you the examples (there are multiple ways)

With the wrappers

_UIA_setVar("oP1","Title:=<yourtitle>;controltype:=UIA_WindowControlTypeId;class:=<yourclass>)  

$myObject=_UIA_action("oP1","object")

Or

with the findall / treewalker examples as given in the zip of post#1

Share this post


Link to post
Share on other sites

#567 ·  Posted

We are using the most recent version of UIAWrappers.au3 (6.3 10/07/2016)

In the function _UIA_getObjectByFindAll() line 1058

;- If not an object given directly then
    If $iMatch = 0 Then
;~ Get the exceptional properties with special meaning
        For $i = 1 To $arrSize
            $propName = $properties2Match[$i][3]

Should $propName be set to $properties2Match[$i][0] instead?

It appears to be the case and fixes some issues for us.

Thank you

Share this post


Link to post
Share on other sites

#568 ·  Posted

I think you are right as the definition of those 4 is as below. Which behavior did you have

$asProperties2Match[$i][0] = $propName
            $asProperties2Match[$i][1] = $propValue
            $asProperties2Match[$i][2] = $iMatch
            $asProperties2Match[$i][3] = $bSkipSpecialProperty

 

and array is incorrect dimension  change 2nd dimension from 2 to 4 but unsure why that does not give compile or runtime errors

Local $properties2Match[1][2] ;~ All properties of the expression to match in a normalized form

 

Share this post


Link to post
Share on other sites

#569 ·  Posted

Our script would find the item it was looking for, but we could not get any properties of the object returned.

Share this post


Link to post
Share on other sites

#570 ·  Posted

Updated first post with latest version V0.64 (many small essential tweaks and fixes)

 

@HudsonKane 

1. Fixed the issues you gave also in post 1
and
2 Not sure what you mean but  
This gives all properties

_UIA_getAllPropertyValues($UIA_oUIElement)

and this to retrieve a single property

$UIA_oUIElement.GetCurrentPropertyValue($id, $tmpValue)

and you can only get all Javascript/html by using bookmarklets thru addressbar (type javascript:......  in your addressbar and sky will be the limit)

Share this post


Link to post
Share on other sites

#571 ·  Posted

Thank you very much, we will start working with this version right away and let you know what we find out.

 

Share this post


Link to post
Share on other sites

#572 ·  Posted

On 9/6/2017 at 1:32 PM, junkew said:

yes, simplespy shows you the examples (there are multiple ways)

With the wrappers

_UIA_setVar("oP1","Title:=<yourtitle>;controltype:=UIA_WindowControlTypeId;class:=<yourclass>)  

$myObject=_UIA_action("oP1","object")

Or

with the findall / treewalker examples as given in the zip of post#1

Thank you, but I guess my problem is not exactly that. If I to a "setVar" and store the control in a var that's all well and good, but I then need to be able to call a script, a while later, and this script is supposed to be able to access this var (after the first script that found the control already exited). So I need this setvar to be stored somewhere so I can retrieve it later. Is that possible? Or do I need to keep the first script running all the time?

 

Share this post


Link to post
Share on other sites

#574 ·  Posted

20 hours ago, junkew said:

@Yoavi do not understand why refinding it would be an issue in the 2nd script. First script could remind the automationid which 2nd script uses to find it.

I will try to better explain myself.

I have an "initialization" script, which first finds the button. That script can successfully use:

_UIA_setVar("oP1","Title:=<yourtitle>;controltype:=UIA_WindowControlTypeId;class:=<yourclass>)  
$myObject=_UIA_action("oP1","object")

That same initialization script can even do (just for the sake of testing), either

_UIA_action($myObject,"highlight")

or

_UIA_action("oP1","highlight")

And both would work.

That initialization script then terminates. At a later time, I would like to call a new script. The sole purpose of this new script is to press that button I previously found.

That second script needs to somehow restore $myObject from somewhere. Is the value stored somewhere when I'm using _UIA_action("oP1","object") ? How do I retrieve it then?

 

I appreciate the patience. I realize it's probably something very basic here that I'm missing.

 

Thank you.

 

Share this post


Link to post
Share on other sites

#575 ·  Posted

No, the found object in script1 does not stay in memory ready for scrip2.

  • When first script ends nothing is there to refer to in 2nd script so 
    you have to refind it. you can find it either with same description as used in script1 or
    you make sure you save the automationid property (Global Const $UIA_AutomationIdPropertyId=30011) in script 1 in an ini file and read that value back in script 2 and use then 
_UIA_SetVar("oP1","AutomationId=" & $retrievedIniValue)
  • Both ways leads to a refind where I am not sure if the speeddifference justifies finding by automationid.
  • Only when you are in need of n scripts you could build an advanced mainscript that keeps references in memory and the other scripts communicate with first script doing all the actions (although I do not see a need to make it that complicated). Why you are not handling it iall n 1 script?

 

 

 

Share this post


Link to post
Share on other sites

#576 ·  Posted

15 hours ago, junkew said:

No, the found object in script1 does not stay in memory ready for scrip2.

  • When first script ends nothing is there to refer to in 2nd script so 
    you have to refind it. you can find it either with same description as used in script1 or
    you make sure you save the automationid property (Global Const $UIA_AutomationIdPropertyId=30011) in script 1 in an ini file and read that value back in script 2 and use then 
_UIA_SetVar("oP1","AutomationId=" & $retrievedIniValue)
  • Both ways leads to a refind where I am not sure if the speeddifference justifies finding by automationid.
  • Only when you are in need of n scripts you could build an advanced mainscript that keeps references in memory and the other scripts communicate with first script doing all the actions (although I do not see a need to make it that complicated). Why you are not handling it iall n 1 script?

 

Thank you, that's what I was afraid of... I would do it all in one script, but I don't know if it's possible in autoit to have a script running and then waiting to be called again without creating another instance and in such a way that the same instance is simply invoked with a possibility to access the previously defined and populated variables. We're getting a little off-topic, perhaps I'll proceed in a more appropriate forum category...

 

Thanks.

Share this post


Link to post
Share on other sites

#577 ·  Posted

Do you have something for Windows 10 that will look for Steam icon on the hidden part of system tray, right click it and select Exit, without using ProcessClose()?


-itisallinthemind-

Share this post


Link to post
Share on other sites

#578 ·  Posted

I do not know you (as i do not have steam) should just try with Simplespy and inspect.exe if you can see the elements highlight. If you can you can scan them with the treewalkers. Within examples there are examples on finding and clicking clock in the systemtray.

Share this post


Link to post
Share on other sites

#579 ·  Posted

Hello again. I am playing around with UI Automation again. Maybe I found a potential bug in this UDF :unsure:

When I use LegacyIAccessibleName with _UIA_getObjectByFindAll it is not working, but when I try to iterate through the objects I can find the correct element using the Legacy Name.

Here is an example:

#include "UIA\UIAWrappers.au3"

Local $oChrome=_UIA_getObjectByFindAll($UIA_oDesktop, "controltype:=UIA_WindowControlTypeId;class:=Chrome_WidgetWin_1", $treescope_children)
Local $oUIElement=_UIA_getObjectByFindAll($oChrome, "title:=Play;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"highlight")

$oMuteButton = _UIA_getObjectByFindAll($oChrome, 'LegacyIAccessibleName:=Mute;ControlType:=UIA_ButtonControlTypeId')
MsgBox(0, "Highlight", "Using _UIA_getObjectByFindAll to find object and highlight")
_UIA_action($oMuteButton,"highlight")
MsgBox(0, "Highlight", "Using GetSiblingObjectByLegacyName to find object and highlight")
$oMuteButton = GetSiblingObjectByLegacyName($oUIElement, "Mute")
_UIA_action($oMuteButton,"highlight")

Func GetSiblingObjectByLegacyName($oSiblingObject, $sName)
    $UIA_oUIAutomation.RawViewWalker($UIA_pTW)

    $oTW = ObjCreateInterface($UIA_pTW, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker)

    While IsObj($oSiblingObject) = True
        $oTW.GetNextSiblingElement($oSiblingObject, $UIA_pUIElement)
        $oSiblingObject = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
        If _UIA_getPropertyValue($oSiblingObject,  $UIA_LegacyIAccessibleNamePropertyId) = $sName Then Return $oSiblingObject
    WEnd
EndFunc

In this example I made a custom function which iterates through all of the sibling objects for a given object and finds the correct object by the Legacy Name... and guess what? _UIA_getObjectByFindAll cannot find it, I wonder why? You can run this example by opening a youtube video in chrome, it should highlight the play button and the mute button.

Thanks for the help in advance, TD :)


AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Spoiler

My contributions to the AutoIt Community

Some messages & Apologizes:

If I hurt you, Please accept my apologies, I never (regardless of the situation) mean to hurt anybody!!!

Also, I am very busy with my project so I will appear in the last row of the online list, if you want to contact me: Email@TheDcoder.xyz

Or you can have a nice chat with me in freenode, I use the same nick on freenode too!

3fHNZJ.gif

PLEASE JOIN ##AutoIt AND HELP THE IRC AUTOIT COMMUNITY!

Share this post


Link to post
Share on other sites

#580 ·  Posted

Seems not to be in my wrappers. Seems to be a difference between findall and treewalkers and implementation of Google Chrome.

Interesting to study later how this goes for IE and FireFox on YouTube

  • This works also for the play button using LegacyIAccessibleName but not for the mute button
;~ Local $oUIElement=_UIA_getObjectByFindAll($oChrome, "title:=Play;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oChrome, "LegacyIAccessibleName:=Play;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree)
  • When I try with simplespy
    • on calc.exe all seems to function fine also with LegacyIAccessibleName
    • When I try with youtube and simplespy the LegacyIAccessibleName is NOT coming for mute but it is filled for play button
    • When I try your example with the treewalker (all 3 different ones tried) it finds it
  • Debugging in uiaWrappers with some consolewrite does not give me a clue either
    $tmpStr = _UIA_getBasePropertyInfo($itUIA_oUIElement)
                _UIA_LOG(" Element: " & $i & " " & $tmpStr & @CRLF, $UIA_Log_Wrapper)
                consolewrite(" Element: " & $i & " " & $tmpStr & @CRLF, $UIA_Log_Wrapper)

    extending this function to show the legacy name

    Func _UIA_getBasePropertyInfo($oUIElement)
        Local $title = _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId)
        Local $class = _UIA_getPropertyValue($oUIElement, $uia_classnamepropertyid)
        Local $controltypeName = _UIA_getControlName(_UIA_getPropertyValue($oUIElement, $UIA_ControlTypePropertyId))
        Local $controltypeId = _UIA_getPropertyValue($oUIElement, $UIA_ControlTypePropertyId)
        Local $nativeWindow = _UIA_getPropertyValue($oUIElement, $UIA_NativeWindowHandlePropertyId)
        Local $controlRect = _UIA_getPropertyValue($oUIElement, $UIA_BoundingRectanglePropertyId)
        Local $acceleratorkey = _UIA_getPropertyValue($oUIElement, $UIA_AcceleratorKeyPropertyId)
        Local $automationid = _UIA_getPropertyValue($oUIElement, $UIA_AutomationIdPropertyId)
        Local $LegacyIAccessibleName= _UIA_getPropertyValue($oUIElement,  $UIA_LegacyIAccessibleNamePropertyId)
        Return "Title is: <" & $title & ">" & @TAB _
                 & "Class   := <" & $class & ">" & @TAB _
                 & "controltype:= " & "<" & $controltypeName & ">" & @TAB _
                 & ",<" & $controltypeId & ">" & @TAB _
                 & ", (" & Hex($controltypeId) & ")" & @TAB _
                 & "rect := < " & $controlRect & ">" & @TAB _
                 & "hwnd := < " & $nativeWindow & ">" & @TAB _
                 & "acceleratorkey := < " & $acceleratorkey & ">" & @TAB _
                 & "LegacyIAccessibleName := <" & $LegacyIAccessibleName & ">" & @TAB _
                 & "automationid := <" & $automationid & ">" & @CRLF
    
    EndFunc   ;==>_UIA_getBasePropertyInfo

     

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • bobmcrae
      By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • jackchen
      By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object. Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?