Jump to content

IUIAutomation MS framework automate chrome, FF, IE, ....


Recommended Posts

  • Developers
6 hours ago, poisonkiller said:

I am hopping here from AHK forums

...and you are used to crosspost your question at multiple plac s in a forum?...  Why exactly?

SciTE4AutoIt3 Full installer Download page   - Beta files       Read before posting     How to post scriptsource   Forum etiquette  Forum Rules 
 
Live for the present,
Dream of the future,
Learn from the past.
  :)

Link to post
Share on other sites
  • Replies 870
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Automate all windows and browser applications with one UDF function library. Based on the microsoft automation API this library high level supports Recognition of conttrols from EDGE, Chrome, FF

Quickstart  Unzip the 2 zips (UIA*.zip and examples*.zip) of post #1 Start simplespy.au3 Hover your mouse over the element you want to handle Press ctrl+w Copy / past

@TheSaint no problem, bear in mind I am working (slowly) on a new spy to integrate all different spying tools so far seen. Will have different tabs with more/les information, more different ways

Posted Images

9 hours ago, Jos said:

...and you are used to crosspost your question at multiple plac s in a forum?...  Why exactly?

Although this discussion feels more like belonging to a PM, I'll answer publicly (since you did):

1) I first contacted the author of this thread and he recommended I post my question here on the forums

2) "are used to" is very incorrect, since this is the first time.

3) The why... UIAutomation is a rather niche topic, and interacting with it through COM even moreso. Most of the Google results about UIAutomation contain at least some links to this forum, and the userbase for UIAutomation is much larger here, and has years more experience with it. My question is also quite niche, and hours of Googling hasn't resulted in anything. If there is any place to at least hope to get an answer, I think it would be here.

 

To not get totally offtopic, here is my current workaround to get access to Skype and other Chromium-based apps: after activating the window with WinActivate(), get an element from inside the content with either UIA GetFocusedElement or ElementFromPoint. Then use a condition for ControlType:=Document to create a TreeWalker and use that TreeWalker's NormalizeElement method to get the Document element, which contains all the Chromium content. The other elements of the window (for example minmax buttons) you can get with the regular ElementFromHandle. Although this is a hacky workaround, at least it works... But a ElementFromHandle approach would be preferrable, and the question still remains that why is there a disconnect between Skype window element and the Chromium Document element.

Edited by poisonkiller
Link to post
Share on other sites

I am not using skype so hard to say but use the raw uia examples and not the wrapper.

But as it seems you have more then enough knowledge on how to run this lets see the basic steps

Me and @LarsJ https://www.autoitscript.com/forum/search/?&q=iuiautomation&type=forums_topic&author=LarsJ&nodes=9&search_and_or=and&sortby=relevancy

have written a lot of stuff and it boils down when you have questions "I cannot find my object"

* whats inspect.exe giving you back

* whats simplespy giving you back

* whats uiaspy giving you back https://www.autoitscript.com/forum/topic/196833-uiaspy-ui-automation-spy-tool/?tab=comments#comment-1411655

If any of above shows your object we should be able to find it 😉 

if inspect.exe is not showing it it makes sense to report an issue to the chromium team they have fixed issues for me in the past. 

Looking at chrome://accessibility/ there is some configuration possible but I would not know how to do that on skype/electron apps

maybe FindContainerWindowElement gives a different answer 

I can only guess but advice is just to use your workaround there are so many glitches anyway in automation that sometimes its not worth the effort to find the rootcause of an issue if the workaround is fine

Link to post
Share on other sites

I am not using uia, but to get all child elements with iaccessible for some chrominium applications like vivaldi or skype You need first call accName of iaccessible object, after that wait some time for appearing childs.

As example autohotkey code:

if !init
{
   SendMessage, WM_GETOBJECT := 0x003D, 0, 1, Chrome_RenderWidgetHostHWND1, A
   Acc := Acc_ObjectFromPoint(child)
   Acc.accName(child)
   sleep 300
   init := 1
}
Acc := Acc_ObjectFromPoint(child)
MsgBox, % Acc.accRole(child)
Return

 

Link to post
Share on other sites

@junkew, thanks, I'll slowly start going through all the links you have provided (especially the chromium accessibility source seems quite interesting). But it it seeming more and more likely that this is just a bug in Chromium or UIA... I think I'll read the Chromium accessibility source code and if it doesn't reveal anything, then I'll report a bug.

@malcev, thank you for the suggestion, but in this case it doesn't seem to be relevant. After using MSAA ObjectFromPoint doesn't change the UIA tree, neither after your example nor inspecting with inspect.exe in MSAA mode (which should be calling both ObjectFromPoint and getting roles/names).

As a side-note, the Sleep 300 might sometimes not be enough, since my tests have shown that some windows take a very short time to enable MSAA/UIA trees, but others might take >300ms (especially windows with thousands of elements, like Chrome with a very complex website open). In UIA you can dynamically wait for the tree be constructed by sending the SendMessage signal, then find the element in the window for ClassName:=Chrome_RendererWidgetHostHWND, get the CurrentName (this part is important, otherwise ValuePattern ValueValueProperty is not accessible) and then wait for ValueValueProperty to have a value (which is usually an URL).

Link to post
Share on other sites

300ms - it is just an example.

You can test like this.

Exit all uia inspectors, exit skype, run skype, run AccExplorer32.

https://github.com/blackrosezy/gui-inspect-tool/blob/master/AccExplorer32.exe

Choose skype window, build tree with this utility -> as result You will see that Your tree will not have all child elements.

Then run ahk code, activate skype, press f11.

f11::
SendMessage, WM_GETOBJECT := 0x003D, 0, 1, Chrome_RenderWidgetHostHWND1, A
Acc := Acc_ObjectFromWindow(WinExist("A"))
Acc.accName(0)
Return


; http://www.autohotkey.com/board/topic/77303-acc-library-ahk-l-updated-09272012/
; https://dl.dropbox.com/u/47573473/Web%20Server/AHK_L/Acc.ahk
;------------------------------------------------------------------------------
; Acc.ahk Standard Library
; by Sean
; Updated by jethrow:
;   Modified ComObjEnwrap params from (9,pacc) --> (9,pacc,1)
;   Changed ComObjUnwrap to ComObjValue in order to avoid AddRef (thanks fincs)
;   Added Acc_GetRoleText & Acc_GetStateText
;   Added additional functions - commented below
;   Removed original Acc_Children function
; last updated 2/25/2010
;------------------------------------------------------------------------------

Acc_Init()
{
    Static  h
    If Not  h
        h:=DllCall("LoadLibrary","Str","oleacc","Ptr")
}
Acc_ObjectFromEvent(ByRef _idChild_, hWnd, idObject, idChild)
{
    Acc_Init()
    If  DllCall("oleacc\AccessibleObjectFromEvent", "Ptr", hWnd, "UInt", idObject, "UInt", idChild, "Ptr*", pacc, "Ptr", VarSetCapacity(varChild,8+2*A_PtrSize,0)*0+&varChild)=0
    Return  ComObjEnwrap(9,pacc,1), _idChild_:=NumGet(varChild,8,"UInt")
}

Acc_ObjectFromPoint(ByRef _idChild_ = "", x = "", y = "")
{
    Acc_Init()
    If  DllCall("oleacc\AccessibleObjectFromPoint", "Int64", x==""||y==""?0*DllCall("GetCursorPos","Int64*",pt)+pt:x&0xFFFFFFFF|y<<32, "Ptr*", pacc, "Ptr", VarSetCapacity(varChild,8+2*A_PtrSize,0)*0+&varChild)=0
    Return  ComObjEnwrap(9,pacc,1), _idChild_:=NumGet(varChild,8,"UInt")
}

Acc_ObjectFromWindow(hWnd, idObject = -4)
{
    Acc_Init()
    If  DllCall("oleacc\AccessibleObjectFromWindow", "Ptr", hWnd, "UInt", idObject&=0xFFFFFFFF, "Ptr", -VarSetCapacity(IID,16)+NumPut(idObject==0xFFFFFFF0?0x46000000000000C0:0x719B3800AA000C81,NumPut(idObject==0xFFFFFFF0?0x0000000000020400:0x11CF3C3D618736E0,IID,"Int64"),"Int64"), "Ptr*", pacc)=0
    Return  ComObjEnwrap(9,pacc,1)
}

Acc_WindowFromObject(pacc)
{
    If  DllCall("oleacc\WindowFromAccessibleObject", "Ptr", IsObject(pacc)?ComObjValue(pacc):pacc, "Ptr*", hWnd)=0
    Return  hWnd
}

Acc_GetRoleText(nRole)
{
    nSize := DllCall("oleacc\GetRoleText", "Uint", nRole, "Ptr", 0, "Uint", 0)
    VarSetCapacity(sRole, (A_IsUnicode?2:1)*nSize)
    DllCall("oleacc\GetRoleText", "Uint", nRole, "str", sRole, "Uint", nSize+1)
    Return  sRole
}

Acc_GetStateText(nState)
{
    nSize := DllCall("oleacc\GetStateText", "Uint", nState, "Ptr", 0, "Uint", 0)
    VarSetCapacity(sState, (A_IsUnicode?2:1)*nSize)
    DllCall("oleacc\GetStateText", "Uint", nState, "str", sState, "Uint", nSize+1)
    Return  sState
}

Acc_SetWinEventHook(eventMin, eventMax, pCallback)
{
    Return  DllCall("SetWinEventHook", "Uint", eventMin, "Uint", eventMax, "Uint", 0, "Ptr", pCallback, "Uint", 0, "Uint", 0, "Uint", 0)
}

Acc_UnhookWinEvent(hHook)
{
    Return  DllCall("UnhookWinEvent", "Ptr", hHook)
}
/*  Win Events:
    pCallback := RegisterCallback("WinEventProc")
    WinEventProc(hHook, event, hWnd, idObject, idChild, eventThread, eventTime)
    {
        Critical
        Acc := Acc_ObjectFromEvent(_idChild_, hWnd, idObject, idChild)
        ; Code Here:
    }
*/

; Written by jethrow
Acc_Role(Acc, ChildId=0) {
    try return ComObjType(Acc,"Name")="IAccessible"?Acc_GetRoleText(Acc.accRole(ChildId)):"invalid object"
}
Acc_State(Acc, ChildId=0) {
    try return ComObjType(Acc,"Name")="IAccessible"?Acc_GetStateText(Acc.accState(ChildId)):"invalid object"
}
Acc_Location(Acc, ChildId=0, byref Position="") { ; adapted from Sean's code
    try Acc.accLocation(ComObj(0x4003,&x:=0), ComObj(0x4003,&y:=0), ComObj(0x4003,&w:=0), ComObj(0x4003,&h:=0), ChildId)
    catch
        return
    Position := "x" NumGet(x,0,"int") " y" NumGet(y,0,"int") " w" NumGet(w,0,"int") " h" NumGet(h,0,"int")
    return  {x:NumGet(x,0,"int"), y:NumGet(y,0,"int"), w:NumGet(w,0,"int"), h:NumGet(h,0,"int")}
}
Acc_Parent(Acc) { 
    try parent:=Acc.accParent
    return parent?Acc_Query(parent):
}
Acc_Child(Acc, ChildId=0) {
    try child:=Acc.accChild(ChildId)
    return child?Acc_Query(child):
}
Acc_Query(Acc) { ; thanks Lexikos - www.autohotkey.com/forum/viewtopic.php?t=81731&p=509530#509530
    try return ComObj(9, ComObjQuery(Acc,"{618736e0-3c3d-11cf-810c-00aa00389b71}"), 1)
}
Acc_Error(p="") {
    static setting:=0
    return p=""?setting:setting:=p
}
Acc_Children(Acc) {
    if ComObjType(Acc,"Name") != "IAccessible"
        ErrorLevel := "Invalid IAccessible Object"
    else {
        Acc_Init(), cChildren:=Acc.accChildCount, Children:=[]
        if DllCall("oleacc\AccessibleChildren", "Ptr",ComObjValue(Acc), "Int",0, "Int",cChildren, "Ptr",VarSetCapacity(varChildren,cChildren*(8+2*A_PtrSize),0)*0+&varChildren, "Int*",cChildren)=0 {
            Loop %cChildren%
                i:=(A_Index-1)*(A_PtrSize*2+8)+8, child:=NumGet(varChildren,i), Children.Insert(NumGet(varChildren,i-8)=9?Acc_Query(child):child), NumGet(varChildren,i-8)=9?ObjRelease(child):
            return Children.MaxIndex()?Children:
        } else
            ErrorLevel := "AccessibleChildren DllCall Failed"
    }
    if Acc_Error()
        throw Exception(ErrorLevel,-1)
}
Acc_ChildrenByRole(Acc, Role) {
    if ComObjType(Acc,"Name")!="IAccessible"
        ErrorLevel := "Invalid IAccessible Object"
    else {
        Acc_Init(), cChildren:=Acc.accChildCount, Children:=[]
        if DllCall("oleacc\AccessibleChildren", "Ptr",ComObjValue(Acc), "Int",0, "Int",cChildren, "Ptr",VarSetCapacity(varChildren,cChildren*(8+2*A_PtrSize),0)*0+&varChildren, "Int*",cChildren)=0 {
            Loop %cChildren% {
                i:=(A_Index-1)*(A_PtrSize*2+8)+8, child:=NumGet(varChildren,i)
                if NumGet(varChildren,i-8)=9
                    AccChild:=Acc_Query(child), ObjRelease(child), Acc_Role(AccChild)=Role?Children.Insert(AccChild):
                else
                    Acc_Role(Acc, child)=Role?Children.Insert(child):
            }
            return Children.MaxIndex()?Children:, ErrorLevel:=0
        } else
            ErrorLevel := "AccessibleChildren DllCall Failed"
    }
    if Acc_Error()
        throw Exception(ErrorLevel,-1)
}
Acc_Get(Cmd, ChildPath="", ChildID=0, WinTitle="", WinText="", ExcludeTitle="", ExcludeText="") {
    static properties := {Action:"DefaultAction", DoAction:"DoDefaultAction", Keyboard:"KeyboardShortcut"}
    AccObj :=   IsObject(WinTitle)? WinTitle
            :   Acc_ObjectFromWindow( WinExist(WinTitle, WinText, ExcludeTitle, ExcludeText), 0 )
    if ComObjType(AccObj, "Name") != "IAccessible"
        ErrorLevel := "Could not access an IAccessible Object"
    else {
        StringReplace, ChildPath, ChildPath, _, %A_Space%, All
        AccError:=Acc_Error(), Acc_Error(true)
        Loop Parse, ChildPath, ., %A_Space%
            try {
                if A_LoopField is digit
                    Children:=Acc_Children(AccObj), m2:=A_LoopField ; mimic "m2" output in else-statement
                else
                    RegExMatch(A_LoopField, "(\D*)(\d*)", m), Children:=Acc_ChildrenByRole(AccObj, m1), m2:=(m2?m2:1)
                if Not Children.HasKey(m2)
                    throw
                AccObj := Children[m2]
            } catch {
                ErrorLevel:="Cannot access ChildPath Item #" A_Index " -> " A_LoopField, Acc_Error(AccError)
                if Acc_Error()
                    throw Exception("Cannot access ChildPath Item", -1, "Item #" A_Index " -> " A_LoopField)
                return
            }
        Acc_Error(AccError)
        StringReplace, Cmd, Cmd, %A_Space%, , All
        properties.HasKey(Cmd)? Cmd:=properties[Cmd]:
        try {
            if (Cmd = "Location")
                AccObj.accLocation(ComObj(0x4003,&x:=0), ComObj(0x4003,&y:=0), ComObj(0x4003,&w:=0), ComObj(0x4003,&h:=0), ChildId)
              , ret_val := "x" NumGet(x,0,"int") " y" NumGet(y,0,"int") " w" NumGet(w,0,"int") " h" NumGet(h,0,"int")
            else if (Cmd = "Object")
                ret_val := AccObj
            else if Cmd in Role,State
                ret_val := Acc_%Cmd%(AccObj, ChildID+0)
            else if Cmd in ChildCount,Selection,Focus
                ret_val := AccObj["acc" Cmd]
            else
                ret_val := AccObj["acc" Cmd](ChildID+0)
        } catch {
            ErrorLevel := """" Cmd """ Cmd Not Implemented"
            if Acc_Error()
                throw Exception("Cmd Not Implemented", -1, Cmd)
            return
        }
        return ret_val, ErrorLevel:=0
    }
    if Acc_Error()
        throw Exception(ErrorLevel,-1)
}

Activate AccExplorer32, rebuild tree.

As result You will see that tree has all child elements.

Link to post
Share on other sites

Hi, I have 2 chrome windows/tabs that have the same title but different adresses.

I want to get this 2 tabs and iterate to find who has the correct addressbar to do some stuff in it.

There's a way to get 2 or more windows and find things inside them?

 

I thought in using something like:

Local $pid = Run(<chrome stuff>)

and finding using:

Local $oElement =  _UIA_getObjectByFindAll($UIA_oDesktop,  "ProcessId:=" &  $pid,  $treescope_children)

 

But this is only working if I have no prior chrome windows opened, but I need to not fiddle with the other chrome tabs, and find this new instance and do stuff there.

Edited by Onolox
Link to post
Share on other sites

First step is to see what the spying tools are spying.

Find the main windows

Iterate the childcontrols or do a findall and iterate them and dump your proporties.

Please make a reproducer script if you want help. Unfortunately it can be yoa have to send refresh Keys or send specific action before your accessibility tree is shown. If not you have to fallback to pixel or ocr recognition.

Link to post
Share on other sites
  • 4 weeks later...

I'm having troubles in a dual monitor setup, if the window open in the second monitor I can find it but I can't operate over it, focus() for example trows an error "requested action with this object failed".

For some reason after many hours now it started working 🤷‍♂️ ,  only after I posted here...

 

This post can be deleted.

Edited by Onolox
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object.Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?
×
×
  • Create New...