Jump to content

IUIAutomation MS framework automate chrome, FF, IE, ....


Recommended Posts

I GOT IT :) That was the right line of thinking.

To reiterate, a specific popup window from an uninstall that could not be suppressed was also inaccessible with AutoIT and Inspect.exe. AutoIT would stop processing more commands, IUIAutomation would crash. Inspect.exe shows:
"
Cannot get object from point:
[Error: FromPoint: hr=0xFFFFFFFF80070005 - Access is denied.]" when clicking or dragging the window in question.

MSIExec runs the particular window as as "SYSTEM" and not "Administrator" and thus the access denied ssue.. My solution to get Inspect.exe working:

Run CMD.exe as Administrator (so I can do the next step)
Run CMD.exe using "psexec -i -s cmd.exe" which will run cmd.exe as the "SYSTEM" user
Ran inspect.exe from here.

Now Inspect.exe can see the window and the button. I am sure if I run these tools as SYSTEM things will start to work correctly :) Thanks for your help.

Link to post
Share on other sites
  • Replies 848
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Automate all windows and browser applications with one UDF function library. Based on the microsoft automation API this library high level supports Recognition of conttrols from EDGE, Chrome, FF

Quickstart  Unzip the 2 zips (UIA*.zip and examples*.zip) of post #1 Start simplespy.au3 Hover your mouse over the element you want to handle Press ctrl+w Copy / past

@TheSaint no problem, bear in mind I am working (slowly) on a new spy to integrate all different spying tools so far seen. Will have different tabs with more/les information, more different ways

Posted Images

Check what narrator from windows is speaking to you when you hover over it. Check and search @larsj threads for catching uia events to see if it tells you more. He wrote some threads and deep information on usage of uia in examples section.

Edited by junkew
Link to post
Share on other sites
  • 1 month later...

Do I need to have a special build of autoit to use it? I downloaded UIA_V0_70.zip, unpacked to a new  folder and try to run SimpleSpy.au3

I got error messages about missing includes (WinAPISys.au3, StringConstants.au3, APIShPathConstants.au3 and more). Found them all on GitHub (like this) and put them all in my \Autoit\Includes. Clearly it doesn't went very well since I'm now get a bunch of "previously declared as a 'Const'" errors

Quote

C:\Portable\AutoIt3\Include\constants.au3(18,76) : ERROR: $OPT_COORDSRELATIVE previously declared as a 'Const'.
Global Const $OPT_COORDSRELATIVE = 0 ; Relative coords to the active window
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
C:\Portable\AutoIt3\Include\constants.au3(19,77) : ERROR: $OPT_COORDSABSOLUTE previously declared as a 'Const'.
Global Const $OPT_COORDSABSOLUTE = 1 ; Absolute screen coordinates (default)

Tried on my 3.3.8.0 and fresh install of 3.3.8.1, Win7 x64.

May be someone who get it working can share his Includes dir?

Link to post
Share on other sites
  • 2 months later...

@junkew

When I try to run the example script number 5 for chrome I get this error:

>"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "C:\Users\a\Desktop\General help\AutoIt_Stuff\UIA\examples\ex5_chrome.au3" /UserParams    
+>08:30:28 Starting AutoIt3Wrapper v.18.708.1148.0 SciTE v.4.1.0.0   Keyboard:00000409  OS:WIN_10/  CPU:X64 OS:X64  Environment(Language:0409)  CodePage:0  utf8.auto.check:4
+>         SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE   UserDir => C:\Users\a\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper   SCITE_USERHOME => C:\Users\acook\AppData\Local\AutoIt v3\SciTE 
>Running AU3Check (3.3.14.5)  from:C:\Program Files (x86)\AutoIt3  input:C:\Users\a\Desktop\General help\AutoIt_Stuff\UIA\examples\ex5_chrome.au3
+>08:30:28 AU3Check ended.rc:0

>Running:(3.3.14.5):C:\Program Files (x86)\AutoIt3\autoit3_x64.exe "C:\Users\a\Desktop\General help\AutoIt_Stuff\UIA\examples\ex5_chrome.au3"    
--> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop

"C:\Users\a\Desktop\General help\AutoIt_Stuff\AU3 include files\UIAWrappers.au3" (1726) : ==> Variable must be of type "Object".:
$t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement)
$t = $obj^ ERROR

->08:30:30 AutoIt3.exe ended.rc:1
+>08:30:30 AutoIt3Wrapper Finished.
>Exit code: 1    Time: 2.613

It happens when it gets to this line;

;~  get the addressbar
;~  $oChromeAddressBar=_UIA_getFirstObjectOfElement($oChrome,"class:=Chrome_OmniboxView", $treescope_children) ;worked in chrome 28
$oChromeAddressBar =_UIA_getFirstObjectOfElement($oChromeToolbar,"controltype:=" & $UIA_EditControlTypeId , $treescope_subtree) ;works in chrome 29
;~  $oChromeAddressBar=_UIA_getFirstObjectOfElement($oChrome,"name:=Adres- en zoekbalk"  , $treescope_children) ;works in chrome 29

I am using chrome version 83.0.4103

Edited by nooneclose
Link to post
Share on other sites
  • 1 month later...

Great UDF junkew!

I'm trying to click on an icon on the TrayNotifyWnd. The problem is this item does not have a Name nor a class etc

It is the icon of ExpressVPN program.

So I did this way:

Local $oP3=_UIA_getFirstObjectOfElement($UIA_oDesktop, "class:=Shell_TrayWnd", $treescope_children) 
Local $oP2=_UIA_getFirstObjectOfElement($oP3, "class:=TrayNotifyWnd", $treescope_children)  
Local $oP1=_UIA_getFirstObjectOfElement($oP2, "class:=SysPager", $treescope_children)   
Local $oP0=_UIA_getFirstObjectOfElement($oP1, "class:=ToolbarWindow32", $treescope_children)

; --- FindALL     ****************   looking for NamePropertyId = ''     *****************

Local $pCondition,  $pElements
$UIA_oUIAutomation.CreateTrueCondition( $pCondition )
$oP0.FindAll( $TreeScope_Descendants, $pCondition, $pElements )
Local $oUIElementArray1, $iLength
$oUIElementArray1 = ObjCreateInterFace( $pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
$oUIElementArray1.Length( $iLength )
Local $pElement, $oElement, $sValue
For $i = 0 To $iLength - 1
  $oUIElementArray1.GetElement( $i, $pElement )
  $oElement = ObjCreateInterface( $pElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  $oElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $sValue )
  ConsoleWrite($i &  " $sValue = " & $sValue & @CRLF )
  if ($sValue = "") Then 
    _UIA_action($oElement,"left")
  endif
next

but this does not work as the $sValue duplicates an old Name of previous element found!

These are the first entries of Tray in Scite output:

0 $sValue = Telegram Desktop
1 $sValue = AutoIt - gui_test2.au3
2 $sValue = Telegram Desktop
3 $sValue = PDF24 Creator
4 $sValue = Rimozione sicura dell'hardware ed espulsione supporti

The 2° index should be instead "" and it's the one I'm looking for, instead it duplicates the name of the 0° index "Telegram Desktop".

So, after many tries,  I found this one is working:

Local $oP3=_UIA_getFirstObjectOfElement($UIA_oDesktop, "class:=Shell_TrayWnd", $treescope_children) 
Local $oP2=_UIA_getFirstObjectOfElement($oP3, "class:=TrayNotifyWnd", $treescope_children)  
Local $oP1=_UIA_getFirstObjectOfElement($oP2, "class:=SysPager", $treescope_children)   

Local $pCondition3, $pP0, $oP0
$UIA_oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "ToolbarWindow32", $pCondition3 )
$oP1.FindFirst( $TreeScope_Descendants, $pCondition3, $pP0 )
$oP0 = ObjCreateInterface( $pP0, $sIID_IUIAutomationElement, $dtagIUIAutomationElement ) 
    
; --- FindALL     ****************   looking for NamePropertyId = ''     *****************

Local $pCondition,  $pElements
$UIA_oUIAutomation.CreateTrueCondition( $pCondition )
$oP0.FindAll( $TreeScope_Descendants, $pCondition, $pElements )
Local $oUIElementArray1, $iLength
$oUIElementArray1 = ObjCreateInterFace( $pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
$oUIElementArray1.Length( $iLength )
Local $pElement, $oElement, $sValue
For $i = 0 To $iLength - 1
  $oUIElementArray1.GetElement( $i, $pElement )
  $oElement = ObjCreateInterface( $pElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  $oElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $sValue )
  ConsoleWrite($i &  " $sValue = " & $sValue & @CRLF )
  if ($sValue = "") Then 
    _UIA_action($oElement,"left")
  endif
next

With the correct output:

0 $sValue = Telegram Desktop
1 $sValue = AutoIt - gui_test2.au3
2 $sValue = 
3 $sValue = PDF24 Creator
4 $sValue = Rimozione sicura dell'hardware ed espulsione supporti

 

So, why the first code does not work? It seems the same equal logic to me... something wrong in _UIA_getFirstObjectOfElement or FindAll funcs?

EDIT

Now I also check the processID of the icon on tray to see if it corresponds to the correct PID .exe I'm searching, using  _WinAPI_ReadProcessMemory.

Edited by frank10
Link to post
Share on other sites

I attach the hierarchy:hierarchy.jpg

 

Button without name is my item.

But I don't understand what's the difference writing:

Local $oP0=_UIA_getFirstObjectOfElement($oP1, "class:=ToolbarWindow32", $treescope_children)

vs

Local $pCondition3, $pP0, $oP0
$UIA_oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "ToolbarWindow32", $pCondition3 )
$oP1.FindFirst( $TreeScope_Descendants, $pCondition3, $pP0 )
$oP0 = ObjCreateInterface( $pP0, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

All the rest of code is the same... but it works only using the 2° form of $oP0.

Edited by frank10
Link to post
Share on other sites

@junkew,

is it possible to intercept notifications that goes on the button TrayNotify inside the  trayNotifyWnd on the bottom right of the taskbar (without opening the relative Window clicking on it),  with IUIA?

Notification.jpg.c95d192e70d344019cb57068ebb90a4c.jpg

Edited by frank10
Link to post
Share on other sites

Here what I'm using :

#include <Constants.au3>
#include <Array.au3>
#include "Includes\CUIAutomation2.au3"

Global Const $sCLSID_CUIAutomation8 = "{e22ad333-b25f-460c-83d0-0581107395c9}"
Global Const $sIID_IUIAutomation2 = "{34723AFF-0C9D-49D0-9896-7AB52DF8CD8A}" ; Windows 8
Global Const $dtag_IUIAutomation2 = $dtagIUIAutomation & _
    "get_AutoSetFocus hresult(bool*);" & _
    "put_AutoSetFocus hresult(bool);" & _
    "get_ConnectionTimeout hresult(dword*);" & _
    "put_ConnectionTimeout hresult(dword);" & _
    "get_TransactionTimeout hresult(dword*);" & _
    "put_TransactionTimeout hresult(dword);"


Opt("MustDeclareVars", 1)

While True
  If _WinExits() Then ExitLoop
WEnd

Func _WinExits()
  ;Local $hWnd = WinWait("[CLASS:ApplicationFrameWindow]")
  ;ConsoleWrite($hWnd & @CRLF)

  Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
  If Not IsObj($oUIAutomation) Then Exit ConsoleWrite("$oUIAutomation ERR" & @CRLF)
  ConsoleWrite("$oUIAutomation OK" & @CRLF)

;  Local $oUIAutomation2 = ObjCreateInterface($sCLSID_CUIAutomation8, $sIID_IUIAutomation2, $dtag_IUIAutomation2)
;  If Not IsObj($oUIAutomation2) Then Exit ConsoleWrite("$oUIAutomation2 ERR" & @CRLF)
;  ConsoleWrite("$oUIAutomation2 OK" & @CRLF)

; $oUIAutomation2.put_ConnectionTimeout (0)

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement($pDesktop)
  $oDesktop = ObjCreateInterface($pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
  If Not IsObj($oDesktop) Then Exit ConsoleWrite("$oDesktop ERR" & @CRLF)
  ConsoleWrite("$oDesktop OK" & @CRLF)

  ;Get application window
  Local $pCondition, $pElement, $oElement
  $oUIAutomation.CreatePropertyCondition($UIA_ClassNamePropertyId, "Windows.UI.Core.CoreWindow", $pCondition)
  If Not $pCondition Then Exit ConsoleWrite("$pCondition ERR" & @CRLF)
  ConsoleWrite("$pCondition OK" & @CRLF)

  $oDesktop.FindFirst($TreeScope_Descendants, $pCondition, $pElement)
  $oElement = ObjCreateInterface($pElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
  If Not IsObj($oElement) Then Return False
  ConsoleWrite("Frame Window OK" & @CRLF)

  Local $vValue
  $oElement.GetCurrentPropertyValue($UIA_NamePropertyId, $vValue)
  Return $vValue == "Centre de contrôle"
EndFunc

There is a bit of a delay, that I haven't got the time to resolve so far. 

Link to post
Share on other sites

Thank you Nine, it works, but only if you have notifications popup that appears as windows. If they don't appear, the notifications go directly on the tray icon (also without a descriptive text on the tooltip, you can read them only clicking to make window appear)... so maybe we must use an event catcher to get those hidden notifications.

I looked at Larsj events, but I don't see an event to detect such notifications. Maybe I'll look deeper.

 

With this code all is fine if it finds the element oMenu:

_UIA_setVar("oMenu","Title:=Connesso a.*;controltype:=UIA_MenuItemControlTypeId;class:=MenuItem") 
local $oMenu=_UIA_action("oMenu", "getobject")
But if it cannot find the element "oMenu", the program blocks at the line: action getObject... for a lot of time...
Shouldn't it be possible to get a faster response without blocking the script? (i.e in my pc it lasts for 67 seconds!)

BTW even if the obj exists it's very slow: 34 seconds...

Edited by frank10
Link to post
Share on other sites

I found out the problem with ScrollViewer element...

This is the hierarchy:

hierarchy.thumb.jpg.b3d9dddfc9bb4da313c62a35519ee94c.jpg

 

This works:

_UIA_setVar("oPopup","Title:=;controltype:=UIA_WindowControlTypeId;class:=Popup")   
    _UIA_setVar("oContextMenu","Title:=;controltype:=UIA_MenuControlTypeId;class:=ContextMenu") 
    _UIA_setVar("oScrollViewer","Title:=;controltype:=UIA_PaneControlTypeId;class:=ScrollViewer")   
    _UIA_setVar("oMenuItem","Title:=Connesso a.*;controltype:=UIA_MenuItemControlTypeId;class:=MenuItem")
    local $oMenuItem=_UIA_action("oMenuItem", "getobject")

If there is the menuItem "Connesso a ...", it works in about 3 seconds.

But if it doesn't find the menuItem because the menu is changed, it lasts more than 1 minute to find out the object does not exist.

So, I thought getting the object oScrollViewer and using findAll to list all menuItems to check which is there, with:

_UIA_setVar("oPopup","Title:=;controltype:=UIA_WindowControlTypeId;class:=Popup")   ;
    _UIA_setVar("oContextMenu","Title:=;controltype:=UIA_MenuControlTypeId;class:=ContextMenu") ;
    _UIA_setVar("oScrollViewer","Title:=;controltype:=UIA_PaneControlTypeId;class:=ScrollViewer")   ;
    local $oScrollViewer=_UIA_action("oScrollViewer", "getobject")

    ; --- FindALL
    Local $pCondition,  $pElements
    $UIA_oUIAutomation.CreateTrueCondition( $pCondition )
    $oScrollViewer.FindAll( $TreeScope_Descendants, $pCondition, $pElements )
    Local $oUIElementArray1, $iLength1 ; $pElements is a pointer to an UI Automation element array
    $oUIElementArray1 = ObjCreateInterFace( $pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
    $oUIElementArray1.Length( $iLength1 )
    Local $pElement1, $oElement1, $sValue

    For $i = 0 To $iLength1 - 1
      $oUIElementArray1.GetElement( $i, $pElement1 )
      $oElement1 = ObjCreateInterface( $pElement1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
      if not isobj($oElement1) Then ConsoleWrite( "no oElement1")
        ;$oElement1.GetCurrentPropertyValue( $UIA_ClassNamePropertyId, $sValue ) ; $UIA_ClassNamePropertyId is used as example
      ;ConsoleWrite( "$sValue1 = " & $sValue & @CRLF )
      $oElement1.GetCurrentPropertyValue( $UIA_NamePropertyId, $sValue )
      ConsoleWrite($i &  " $sValue = " & $sValue & @CRLF )

    Next

But it is very slow in finding ScrollViewer, and more, it does not find the correct one!

In fact in the list there are other object names different from the menu I'm analyzing.

 

But I found out if I use the FindAll on the oContextMenu, all is good, I get the menuItems...

So, even if the simplespy and UIAspy say there is the ScrollViewer element, it must be ignored!!

 

EDIT:

there is some problem with the forum layout... I cannot align correctly the text...

Edited by frank10
Link to post
Share on other sites
  • 2 weeks later...

Hi @junkew,

I've used this UDF a few years ago and it was nice.  I'm trying to run the Example #5 "Chrome example" as is and I'm seeing a unique error in the UDF:

Here is the resulting output and error:

address: 
We have 2 tabs
Title is: <New Tab>    Class   := <>    controltype:= <50019>    
Title is: <New Tab>    Class   := <>    controltype:= <50000>    
"T:\Cool\Autoit-T\Scripts\STATScraper\ScrapingOptionsWebDriverOrIUI\UIA_V0_70\UIAWrappers.au3" (1673) : ==> Array variable has incorrect number of subscripts or subscript dimension range exceeded.:
$x = Int($t[1] + ($t[3] / 2))
$x = Int($t[1] + (^ ERROR

I see the UDF highlight the address bar in red then the program crashes with the above error.  

I'm running the following Chrome:

Google Chrome 
Version 85.0.4183.102 (Official Build) (64-bit)
 
Any tips, tricks, ideas or suggestions would be great.
 
Thanks for all ! :-)
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object.Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?
×
×
  • Create New...