Jump to content
junkew

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

Hi, junkew
 
Thanks for your great scripts and examples!  :D
I would like to ask you under what license can I use these scripts?
 
Thanks.

Share this post


Link to post
Share on other sites

I tested using mainly this func:

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf, $pSender, $iEventId ) ; Ret: long  Par: ptr;int
  ConsoleWrite( "oIUIAutomationEventHandler_HandleAutomationEvent: " & $iEventId & @CRLF )
  If $iEventId <> $UIA_Window_WindowClosedEventId Then
    Local $oSender = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $oSender.AddRef()
    if _UIA_getPropertyValue( $oSender, $UIA_NamePropertyId ) = "Shiny.Chat.ViewModels.PrivateChatSessionViewModel" Then
        ConsoleWrite(_UIA_getAllPropertyValues($oSender) & @CRLF)
        MsgBox(0,"","");
    EndIf
    ConsoleWrite( "Title     = " & _UIA_getPropertyValue( $oSender, $UIA_NamePropertyId ) & @CRLF & _
                  "Class     = " & _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId ) & @CRLF & _
                  "Ctrl type = " & _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId ) & @CRLF & _
                  "Ctrl name = " & _UIA_getPropertyValue( $oSender, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                  "Value     = " & _UIA_getPropertyValue( $oSender, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                  "Handle    = " & Hex( _UIA_getPropertyValue( $oSender, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )
  EndIf
  Return $S_OK
EndFunc

 

In the nested IF i check if the pop-up is what i'm looking for (and I suppose I need to check with its $UIA_NamePropertyId).

Once I get it, I need to close it.

I looked trough all the _UIA_getAllPropertyValues($oSender) but couldn't find the absolute position of the pop-up neither the close button coords.

These datas are available if che pop-up is clicked (as shown in simplespy.au3).

How can I retrieve this information and mouseclick it? (or how can I close the popup trough code?)

Thanks a lot,

Marco

Share this post


Link to post
Share on other sites

Also different solutions are possible

a. Combine it with example 1 a treewalker to see the components of the window itself

use properties like UIA_BoundingRectanglePropertyId on those components to find the area for clicking

b. easiest I think is to retrieve the native hwnd: UIA_NativeWindowHandlePropertyId and with other autoit commands just send a WinClose(hWnd)

Share this post


Link to post
Share on other sites

got the handle but, probably since it isn't a standard window, winclose(_UIA_getPropertyValue($oSender, $UIA_NativeWindowHandlePropertyId)) doesn't close it.

I worked around the ex1 and added controls for position:

Func sampleWinList()
    Local $var = WinList()
    $pos = WinGetPos("Network Messages")
    For $i = 1 To $var[0][0]
        ; Only display visble windows
        If $var[$i][0] <> "" And IsVisible($var[$i][1]) Then
            If $var[$i][0] = "Network Messages" Then
                ConsoleWrite("Title=" & $var[$i][0] & @TAB & "Handle=" & $var[$i][1] & @TAB & "Pos: " &$pos[0] & "-" & $pos[1] & "-" & $pos[2] & "-" & $pos[3] & @CRLF)
            Else
                ConsoleWrite("Title=" & $var[$i][0] & @TAB & "Handle=" & $var[$i][1] & @CRLF)
            EndIf
        EndIf
    Next
EndFunc   ;==>sampleWinList

 

in this case, simple winlist()  the popup (name is "Network Messages") appears in list with its coords)

Func sampleTW($t)
    ConsoleWrite("initializing tw " & $t & @CRLF)
;~ ' Lets show all the items of the desktop with a treewalker
    If $t = 1 Then $UIA_oUIAutomation.RawViewWalker($UIA_pTW)
    If $t = 2 Then $UIA_oUIAutomation.ControlViewWalker($UIA_pTW)
    If $t = 3 Then $UIA_oUIAutomation.ContentViewWalker($UIA_pTW)

    $oTW = ObjCreateInterface($UIA_pTW, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker)
    If IsObj($oTW) = 0 Then
        MsgBox(1, "UI automation treewalker failed", "UI Automation failed failed", 10)
    EndIf

    $oTW.GetFirstChildElement($UIA_oDesktop, $UIA_pUIElement)
    $oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

    While IsObj($oUIElement) = True
        ConsoleWrite("Title is: " & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & @TAB & "Handle=" & Hex(_UIA_getPropertyValue($oUIElement, $UIA_NativeWindowHandlePropertyId)) & @TAB & "Class=" & _UIA_getPropertyValue($oUIElement, $uia_classnamepropertyid) & @TAB & "Rect: " & _UIA_getPropertyValue($oUIElement, $UIA_BoundingRectanglePropertyId) & @CRLF)
        $oTW.GetNextSiblingElement($oUIElement, $UIA_pUIElement)
        $oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    WEnd
EndFunc   ;==>sampleTW

 

in this case the window doesn't appear in list except in "All childs of taskbar". 

Probably because this is child of the main program ?

Edited by marko001

Share this post


Link to post
Share on other sites

no winlist ignores the hierarchy

whereas a treewalker gives you options to walk thru the tree

with getparent, getfirstchild, getnextsibling

as in the example you only walk thru the childs of the desktop but not the grandchildren

Share this post


Link to post
Share on other sites

and how do I get, for example, the grandchildren?

I mean, winlist() give me the 2 windows needed:

Medical Center (main of MedicalCenter.exe)

Network Messages (child of MedicalCenter.exe)

and _UIA_getPropertyValue() gives me only the first

[EDIT]:

I used

findThemAll($UIA_oDesktop, $treescope_subtree)
...
...
    For $i = 0 To $iLength - 1; it's zero based
        $oAutomationElementArray.GetElement($i, $UIA_pUIElement)
        $oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
        If StringInStr(_UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId),"Network Message") <> 0 Then
            ConsoleWrite("Title is: " & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & @TAB & "Class=" & _UIA_getPropertyValue($oUIElement, $uia_classnamepropertyid) & @TAB & "Rect: " & _UIA_getPropertyValue($oUIElement, $UIA_BoundingRectanglePropertyId) & @CRLF)
        EndIf

    Next
 

 

This way is a bit slow, if I put no conditions I can see hundreds results.

Edited by marko001

Share this post


Link to post
Share on other sites

The logic in a treewalker in a tree of depth 2 is

A ---> B    ---> D

|         +-------->E

+----->C ------>F

          +------->G

So from A you getFirstChild which is B and if you want to go to C you ask for nextSibling if you want to go to D you ask for firstChild (and as its a tree it can be done more recursively deep in the tree)

 

This is a combination of example 19 and example 1

It checks for the event and sees if its class #32770 and then will print the items of that dialogbox by making a treewalker on the #32770 popupped window

 
;~ Example 19a events
#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"

Opt( "MustDeclareVars", 1 )

;~ Global Const $S_OK = 0x00000000
;~ Global Const $E_NOINTERFACE = 0x80004002
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"

Global $tIUIAutomationEventHandler, $oIUIAutomationEventHandler

Global $oUIAutomation

MainFunc()



Func MainFunc()

  $oIUIAutomationEventHandler = ObjectFromTag( "oIUIAutomationEventHandler_", $dtagIUIAutomationEventHandler, $tIUIAutomationEventHandler, True )
  If Not IsObj( $oIUIAutomationEventHandler ) Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pUIElement
  $oUIAutomation.GetRootElement( $pUIElement ) ; Desktop
  If Not $pUIElement Then Return

  ; Window open/close events
  If $oUIAutomation.AddAutomationEventHandler( $UIA_Window_WindowOpenedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit
  If $oUIAutomation.AddAutomationEventHandler( $UIA_Window_WindowClosedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit

  ; Menu open/close events
  If $oUIAutomation.AddAutomationEventHandler( $UIA_MenuOpenedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit
  If $oUIAutomation.AddAutomationEventHandler( $UIA_MenuClosedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit

  HotKeySet( "{ESC}", "Quit" )

  While Sleep(10)
  WEnd

EndFunc

Func Quit()
  $oIUIAutomationEventHandler = 0
  DeleteObjectFromTag( $tIUIAutomationEventHandler )
  Exit
EndFunc



;~ Func _UIA_getPropertyValue( $obj, $id )
;~   Local $tVal
;~   $obj.GetCurrentPropertyValue( $id, $tVal )
;~   Return $tVal
;~ EndFunc

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf, $pSender, $iEventId ) ; Ret: long  Par: ptr;int
    dim $t
;~   ConsoleWrite( "oIUIAutomationEventHandler_HandleAutomationEvent: " & $iEventId & @CRLF )
  If $iEventId <> $UIA_Window_WindowClosedEventId Then
    Local $oSender = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $oSender.AddRef()

        ;~catch popup
        if _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId )= "#32770" then

        ConsoleWrite( "Title     = " & _UIA_getPropertyValue( $oSender, $UIA_NamePropertyId ) & @CRLF & _
                      "Class     = " & _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId ) & @CRLF & _
                      "Ctrl type = " & _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId ) & @CRLF & _
                      "Ctrl name = " & _UIA_getPropertyValue( $oSender, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                      "Value     = " & _UIA_getPropertyValue( $oSender, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                      "Handle    = " & Hex( _UIA_getPropertyValue( $oSender, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )

    ;~  javascript:alert("hello");

        ConsoleWrite("initializing tw " & $t & @CRLF)
    ;~ ' Lets show all the items of the desktop with a treewalker
    ;~  If $t = 1 Then $UIA_oUIAutomation.RawViewWalker($UIA_pTW)
    ;~  If $t = 2 Then $UIA_oUIAutomation.ControlViewWalker($UIA_pTW)

    $UIA_oUIAutomation.ControlViewWalker($UIA_pTW)

    ;~  If $t = 3 Then $UIA_oUIAutomation.ContentViewWalker($UIA_pTW)

        $UIA_oTW = ObjCreateInterface($UIA_pTW, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker)
        If IsObj($UIA_oTW) = 0 Then
            MsgBox(1, "UI automation treewalker failed", "UI Automation failed failed", 10)
        EndIf

        $UIA_oTW.GetFirstChildElement($oSender, $UIA_pUIElement)
        $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

        While IsObj($UIA_oUIElement) = True
            ConsoleWrite("Title is: " & _UIA_getPropertyValue($UIA_oUIElement, $UIA_NamePropertyId) & @TAB & "Handle=" & Hex(_UIA_getPropertyValue($UIA_oUIElement, $UIA_NativeWindowHandlePropertyId)) & @TAB & "Class=" & _UIA_getPropertyValue($UIA_oUIElement, $uia_classnamepropertyid) & @CRLF)
            $UIA_oTW.GetNextSiblingElement($UIA_oUIElement, $UIA_pUIElement)
            $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
        WEnd

      EndIf
  EndIf

  Return $S_OK
EndFunc

Func oIUIAutomationEventHandler_QueryInterface( $pSelf, $pRIID, $pObj ) ; Ret: long  Par: ptr;ptr*
  Local $sIID = StringFromGUID( $pRIID )
  If $sIID = $sIID_IUnknown Then
;~     ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUnknown" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  ElseIf $sIID = $sIID_IUIAutomationEventHandler Then
;~     ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  Else
;~     ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: " & $sIID & @CRLF )
    Return $E_NOINTERFACE
  EndIf
EndFunc

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: ulong
;~   ConsoleWrite( "oIUIAutomationEventHandler_AddRef" & @CRLF )
  Return 1
EndFunc

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: ulong
;~   ConsoleWrite( "oIUIAutomationEventHandler_Release" & @CRLF )
  Return 1
EndFunc



Func StringFromGUID( $pGUID )
  Local $aResult = DllCall( "ole32.dll", "int", "StringFromGUID2", "struct*", $pGUID, "wstr", "", "int", 40 )
  If @error Then Return SetError( @error, @extended, "" )
  Return SetExtended( $aResult[0], $aResult[2] )
EndFunc



Func ObjectFromTag($sFunctionPrefix, $tagInterface, ByRef $tInterface, $fPrint = False, $bIsUnknown = Default, $sIID = "{00000000-0000-0000-C000-000000000046}") ; last param is IID_IUnknown by default
    If $bIsUnknown = Default Then $bIsUnknown = True
    Local $sInterface = $tagInterface ; copy interface description
    Local $tagIUnknown = "QueryInterface hresult(ptr;ptr*);" & _
            "AddRef dword();" & _
            "Release dword();"
    ; Adding IUnknown methods
    If $bIsUnknown Then $tagInterface = $tagIUnknown & $tagInterface
    ; Below line is really simple even though it looks super complex. It's just written weird to fit in one line, not to steal your attention
    Local $aMethods = StringSplit(StringReplace(StringReplace(StringReplace(StringReplace(StringTrimRight(StringReplace(StringRegExpReplace(StringRegExpReplace($tagInterface, "\w+\*", "ptr"), "\h*(\w+)\h*(\w+\*?)\h*(\((.*?)\))\h*(;|;*\z)", "$1\|$2;$4" & @LF), ";" & @LF, @LF), 1), "object", "idispatch"), "hresult", "long"), "bstr", "ptr"), "variant", "ptr"), @LF, 3)
    Local $iUbound = UBound($aMethods)
    Local $sMethod, $aSplit, $sNamePart, $aTagPart, $sTagPart, $sRet, $sParams, $hCallback
    ; Allocation
    $tInterface = DllStructCreate("int RefCount;int Size;ptr Object;ptr Methods[" & $iUbound & "];int_ptr Callbacks[" & $iUbound & "];ulong_ptr Slots[16]") ; 16 pointer sized elements more to create space for possible private props
    If @error Then Return SetError(1, 0, 0)
    For $i = 0 To $iUbound - 1
        $aSplit = StringSplit($aMethods[$i], "|", 2)
        If UBound($aSplit) <> 2 Then ReDim $aSplit[2]
        $sNamePart = $aSplit[0]
        $sTagPart = $aSplit[1]
        $sMethod = $sFunctionPrefix & $sNamePart
        If $fPrint Then
            Local $iPar = StringInStr( $sTagPart, ";", 2 ), $t
            If $iPar Then
                $t = "Ret: " & StringLeft( $sTagPart, $iPar - 1 ) & "  " & _
                     "Par: " & StringRight( $sTagPart, StringLen( $sTagPart ) - $iPar )
            Else
                $t = "Ret: " & $sTagPart
            EndIf
            Local $s = "Func " & $sMethod & _
                "( $pSelf ) ; " & $t & @CRLF & _
                "EndFunc" & @CRLF
;~             ConsoleWrite( $s )
        EndIf
        $aTagPart = StringSplit($sTagPart, ";", 2)
        $sRet = $aTagPart[0]
        $sParams = StringReplace($sTagPart, $sRet, "", 1)
        $sParams = "ptr" & $sParams
        $hCallback = DllCallbackRegister($sMethod, $sRet, $sParams)
;~         ConsoleWrite(@error & @CRLF & @CRLF)
        DllStructSetData($tInterface, "Methods", DllCallbackGetPtr($hCallback), $i + 1) ; save callback pointer
        DllStructSetData($tInterface, "Callbacks", $hCallback, $i + 1) ; save callback handle
    Next
    DllStructSetData($tInterface, "RefCount", 1) ; initial ref count is 1
    DllStructSetData($tInterface, "Size", $iUbound) ; number of interface methods
    DllStructSetData($tInterface, "Object", DllStructGetPtr($tInterface, "Methods")) ; Interface method pointers
    Return ObjCreateInterface(DllStructGetPtr($tInterface, "Object"), $sIID, $sInterface, $bIsUnknown) ; pointer that's wrapped into object
EndFunc

Func DeleteObjectFromTag(ByRef $tInterface)
    For $i = 1 To DllStructGetData($tInterface, "Size")
        DllCallbackFree(DllStructGetData($tInterface, "Callbacks", $i))
    Next
    $tInterface = 0
EndFunc

I test this with

1. run script

2. go to IE and drop bookmarklet in address bar

javascript:alert("hello");

Edited by junkew

Share this post


Link to post
Share on other sites

I got it and using that example I found the popup:

I changed the IF to _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId) = "50032" to fullfil my request

If _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId) = "50032" Then



        ConsoleWrite( "Title     = " & _UIA_getPropertyValue( $oSender, $UIA_NamePropertyId ) & @CRLF & _

                      "Class     = " & _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId ) & @CRLF & _

                      "Ctrl type = " & _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId ) & @CRLF & _

                      "Ctrl name = " & _UIA_getPropertyValue( $oSender, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _

                      "Value     = " & _UIA_getPropertyValue( $oSender, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _

                      "Pos       = " & _UIA_getPropertyValue( $oSender, $UIA_BoundingRectanglePropertyId ) & @CRLF & _

                      "Handle    = " & Hex( _UIA_getPropertyValue( $oSender, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )

        ConsoleWrite("**** Desktop windows ****" & @CRLF)

        ConsoleWrite(_UIA_getAllPropertyValues($oSender) & @CRLF)

For the pop-up now I can get:

Title     = Shiny.Chat.ViewModels.PrivateChatSessionViewModel

Pos         = 454;311;400;550
Handle    = 00950CE8

This is interesting since I can calculate position of the close button.

Anyway Winclose(00950CE8) doesn't work.

How can I now get all the contents of the pop-up? (close button, name,...)

This could be interesting since I could ctrlclick it or at least having exact coords of close button to close (if I use simplespy.au3 clicking on the pop-up items I have all these infos but I'd like to have them in the program itself)

Thanks for you patience,

Marco

Share this post


Link to post
Share on other sites

Well I got it now using the following:

findThemAll($oSender, $TreeScope_subtree)
...
 
Func findThemAll($oElementStart, $TreeScope)
;~  Get result with findall function alternative could be the treewalker
    Local $oCondition, $oAutomationElementArray,$oUIElement
    Dim $pCondition, $pTrueCondition
    Dim $pElements, $iLength

    $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition)
    $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition)
;~  $oCondition1 = _AutoItObject_WrapperCreate($aCall[1], $dtagIUIAutomationCondition)
;~ Tricky to search all descendants on html objects or from desktop/root element
    $oElementStart.FindAll($TreeScope, $oCondition, $pElements)

    $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray)

    $oAutomationElementArray.Length($iLength)
    For $i = 0 To $iLength - 1; it's zero based
        $oAutomationElementArray.GetElement($i, $UIA_pUIElement)
        $oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
        ConsoleWrite("Title is: " & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & @TAB & "Pos: " & _UIA_getPropertyValue( $oUIElement, $UIA_BoundingRectanglePropertyId ) & @TAB & "Class=" & _UIA_getPropertyValue($oUIElement, $uia_classnamepropertyid) & @CRLF)
    Next

EndFunc   ;==>findThemAll
 

 

In this way I have everything I need:

Title is: Close    Pos: 828;312;24;24    Class=Button

Which is the correct command to parse to close now the window without using the usual mouseclick("left",828+12,312-2,1,0)?

Thanks again,

M.

Share this post


Link to post
Share on other sites

You can click the button with code like this:

Local $pInvokeButton, $oInvokeButton
$oUIButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokeButton )
$oInvokeButton = ObjCreateInterface( $pInvokeButton, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
$oInvokeButton.Invoke()

Share this post


Link to post
Share on other sites

Before reading your message I tried with

        if _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) = "Close" Then
            ConsoleWrite("Closing now" & @CRLF)
            Sleep(5000)
            dim $t2
            ;_UIA_action($oUIElement,"leftclick")
            $t2=stringsplit(_UIA_getPropertyValue($oUIElement, $UIA_ClickablePointPropertyId),";")
            MouseClick("left",$t2[1],$t2[2],1,0)
        EndIf

_UIA_action($oUIElement,"leftclick") bring mouse to 0,0

and the loop you see doesn't work because he iterate the for 2 times, 1st one brings mouse to 0,0 and clicks (losing focus to popup) and 2nd time goes to correct location and clicks (but popup is now under another window)

GOing to check your solution now.

M.

[EDIT]

ERROR: $oUIButton: undeclared global variable.

        if _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) = "Close" Then
            ConsoleWrite("chiudo!!!" & @CRLF)
            Sleep(5000)
            Local $pInvokeButton, $oInvokeButton
            $oUIElement.GetCurrentPattern( $UIA_InvokePatternId, $pInvokeButton )
            $oInvokeButton = ObjCreateInterface( $pInvokeButton, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
            $oInvokeButton.Invoke()
;~             dim $t2
;~             $t2=stringsplit(_UIA_getPropertyValue($oUIElement, $UIA_ClickablePointPropertyId),";")
;~             MouseClick("left",$t2[1],$t2[2],1,0)
        EndIf

it doesn't click and bring main window at front.

Edited by marko001

Share this post


Link to post
Share on other sites

 

this works for me (its an OK button on my example and not CLOSE) no issue with x,y beeing 0,0

 

within the loop

if _UIA_getPropertyValue($UIA_oUIElement, $UIA_NamePropertyId)="OK" Then
                _UIA_Action($UIA_oUIElement,"leftclick")
            EndIf
uncomment this in UIWrappers.au3

 

;~  _UIA_DEBUG("Title is: <" &  _UIA_getPropertyValue($UIA_oUIElement,$UIA_NamePropertyId) &  ">" & $clickcount & ":" & $clickaction & ":" & $x & ":" & $y & ":" & @CRLF, $UIA_Log_Wrapper)
then you can see the details in log.txt (within the same folder if you did not make a UIA.CFG configuration file)

and this

 

$oUIButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokeButton )
$oInvokeButton = ObjCreateInterface( $pInvokeButton, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
$oInvokeButton.Invoke()

can also be written with the UIAWrappers.au3 to

_UIA_Action($UIA_oUIElement,"invoke") 
offcourse in the library it does the same as LarsJ has written out Edited by junkew

Share this post


Link to post
Share on other sites

infact I changed it in $oUIElement.GetCurrentPattern( $UIA_InvokePatternId, $pInvokeButton )

Button is just the [X] close button upright near minimize,maximize. There's no other button and Title is "Close".

MsgBox(0,"Coords: ",_UIA_getPropertyValue($oUIElement, $UIA_ClickablePointPropertyId),5)

works fine, I could just use that.

Share this post


Link to post
Share on other sites

This way works fine:

    For $i = 0 To $iLength - 1; it's zero based
        $oAutomationElementArray.GetElement($i, $UIA_pUIElement)
        $oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
        ConsoleWrite("Title is: " & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & @TAB & "Pos: " & _UIA_getPropertyValue( $oUIElement, $UIA_BoundingRectanglePropertyId ) & @TAB & "Center =" & _UIA_getPropertyValue($oUIElement, $UIA_ClickablePointPropertyId) & @TAB & "Class=" & _UIA_getPropertyValue($oUIElement, $uia_classnamepropertyid) & @CRLF)
        if _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) = "Close" Then
             MsgBox(0,"Coords: ",_UIA_getPropertyValue($oUIElement, $UIA_ClickablePointPropertyId),2)
            ConsoleWrite("Closing Now" & @CRLF)
            Sleep(5000)
            dim $t2
            $t2=stringsplit(_UIA_getPropertyValue($oUIElement, $UIA_ClickablePointPropertyId),";")
            MouseClick("left",$t2[1],$t2[2],1,0)
            ExitLoop
            ;_UIA_action($oUIElement,"leftclick")
;~             Local $pInvokeButton, $oInvokeButton
;~             $oUIElement.GetCurrentPattern( $UIA_InvokePatternId, $pInvokeButton )
;~             $oInvokeButton = ObjCreateInterface( $pInvokeButton, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
;~             $oInvokeButton.Invoke()
;~             dim $t2
;~             $t2=stringsplit(_UIA_getPropertyValue($oUIElement, $UIA_ClickablePointPropertyId),";")
;~             MouseClick("left",$t2[1],$t2[2],1,0)
        EndIf

    Next

 

I see, pausing the code, that after closing the popup it reiterate the loop another time, look here:

Closing Now

Title     = Shiny.Chat.ViewModels.PrivateChatSessionViewModel
Class     = ToolWindow
Ctrl type = 50032
Ctrl name = window
Value     =
Pos       = 249;423;250;550
Handle    = 002B0472

Title is: Shiny.Chat.ViewModels.PrivateChatSessionViewModel    Pos: 249;423;250;550    Centro =374;698    Class=ToolWindow
Title is: Minimize    Pos: 425;424;24;24    Center =437;436    Class=Button
Title is: Maximize    Pos: 449;424;24;24    Center =461;436    Class=Button
Title is: Close    Pos: 473;424;24;24    Center =485;436    Class=Button

AND HERE IT'S CLOSED

?? It loops again??

Closing Now
Title     =
Class     = ToolWindow
Ctrl type = 50032
Ctrl name = finestra
Value     =
Pos       = 249;423;250;550
Handle    = 002B0472

Title is:     Pos: 249;423;250;550    Center =374;698    Class=ToolWindow
Title is: Minimize    Pos: 0;0;0;0    Center =-2147483648;-2147483648    Class=Button
Title is: Maximize    Pos: 0;0;0;0    Center =-2147483648;-2147483648    Class=Button
Title is: Close    Pos: 0;0;0;0    Center =-2147483648;-2147483648    Class=Button
Closing Now
 

Any idea?

Share this post


Link to post
Share on other sites

Got it. It just need some time to reset. If I put a sleep(5000) after the mouseclick() it works perfectly.

This is the final code:

Func findThemAll($oElementStart, $TreeScope)
    Local $oCondition, $oAutomationElementArray,$oUIElement
    Dim $pCondition, $pTrueCondition
    Dim $pElements, $iLength
    $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition)
    $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition)
    $oElementStart.FindAll($TreeScope, $oCondition, $pElements)
    $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray)
    $oAutomationElementArray.Length($iLength)
    For $i = 0 To $iLength - 1; it's zero based
        $oAutomationElementArray.GetElement($i, $UIA_pUIElement)
        $oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
        if _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) = "Close" Then
            ConsoleWrite("Closing Now" & @CRLF)
            Sleep(500)
            dim $t2
            $t2=stringsplit(_UIA_getPropertyValue($oUIElement, $UIA_ClickablePointPropertyId),";")
            MouseClick("left",$t2[1],$t2[2],1,0)
            Sleep(5000)
            ExitLoop
        EndIf
    Next
EndFunc   ;==>findThemAll

If you want to try to explain me where and how to use:

Local $pInvokeButton, $oInvokeButton
$oUIButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokeButton )
$oInvokeButton = ObjCreateInterface( $pInvokeButton, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
$oInvokeButton.Invoke()

would be really appreciated, in any case I can use it now as shown. Really thanks anyway for the great help.

And the last question:

which is the best way to include in the program these functions? trough an

AdlibRegister("_checkpopups", 5000) ;

or it's enough to run the function before the while cycle?

Marco

Edited by marko001

Share this post


Link to post
Share on other sites

There is an issue on your code as you seem to be falling thru to the 2nd window closing and as such

_UIA_Action($UIA_oUIElement,"leftclick")

is not working

 

You should start reading here

http://msdn.microsoft.com/en-us/library/windows/desktop/ee684009(v=vs.85).aspx

UIAutomation framework is huge and I only abstracted so far a small subset

Controls you can detect in multiple ways in order of complexity but the more complex the more you can do

1. based on mouseposition

2. by iterating thru a tree with a treewalker

3. act on events that happen like opening/closing of windows or changing properties

You do not have to use adlibregister as event functions are automatically called.

In your example you can see that actually 2 windows are closing the 2nd doesn't have a name. You should have an if statement to get the correct one.

When you have an IUIElement you can do depending on the class different things if defined. Each thing you can do is hidden in a pattern and 1 control can have n patterns with m methods/functions

LarsJ gave you the details on getting a pattern

Here you have all the patterns

;module UIA_PatternIds
Global Const $UIA_InvokePatternId=10000
Global Const $UIA_SelectionPatternId=10001
Global Const $UIA_ValuePatternId=10002
Global Const $UIA_RangeValuePatternId=10003
Global Const $UIA_ScrollPatternId=10004
Global Const $UIA_ExpandCollapsePatternId=10005
Global Const $UIA_GridPatternId=10006
Global Const $UIA_GridItemPatternId=10007
Global Const $UIA_MultipleViewPatternId=10008
Global Const $UIA_WindowPatternId=10009
Global Const $UIA_SelectionItemPatternId=10010
Global Const $UIA_DockPatternId=10011
Global Const $UIA_TablePatternId=10012
Global Const $UIA_TableItemPatternId=10013
Global Const $UIA_TextPatternId=10014
Global Const $UIA_TogglePatternId=10015
Global Const $UIA_TransformPatternId=10016
Global Const $UIA_ScrollItemPatternId=10017
Global Const $UIA_LegacyIAccessiblePatternId=10018
Global Const $UIA_ItemContainerPatternId=10019
Global Const $UIA_VirtualizedItemPatternId=10020
Global Const $UIA_SynchronizedInputPatternId=10021
so if you have an element

you can first check the patternproperties like

$UIA_IsWindowPatternAvailablePropertyId

 

if its available you can thru this pattern call methods like

 

Global Const $sIID_IUIAutomationWindowPattern="{0FAEF453-9208-43EF-BBB2-3B485177864F}"
Global $dtagIUIAutomationWindowPattern = "Close hresult();" & _
"WaitForInputIdle hresult(int;long*);" & _
"SetWindowVisualState hresult(long);" & _
"CurrentCanMaximize hresult(long*);" & _
"CurrentCanMinimize hresult(long*);" & _
"CurrentIsModal hresult(long*);" & _
"CurrentIsTopmost hresult(long*);" & _
"CurrentWindowVisualState hresult(long*);" & _
"CurrentWindowInteractionState hresult(long*);" & _
"CachedCanMaximize hresult(long*);" & _
"CachedCanMinimize hresult(long*);" & _
"CachedIsModal hresult(long*);" & _
"CachedIsTopmost hresult(long*);" & _
"CachedWindowVisualState hresult(long*);" & _
"CachedWindowInteractionState hresult(long*);"
getting a pattern is abstracted in 

func _UIA_getPattern($obj,$patternID)
used like this see example 9 for mediaplayer

$oValueP=_UIA_getpattern($oVolumeButton,$UIA_ValuePatternId)
if isobj($oValueP) Then
dim $volume
    $oValuep.currentvalue($volume)
    _UIA_Debug("Volume is " & $volume & @CRLF)
EndIf

Share this post


Link to post
Share on other sites

 

Hi, junkew
 
Thanks for your great scripts and examples!  :D
I would like to ask you under what license can I use these scripts?
 
Thanks.

 

 

I found the line in UIAWrappers.au3

; License .......: GPL

Share this post


Link to post
Share on other sites

http://www.autoitscript.com/site/donate/

Just help a great tool like autoit by referencing it. The IUIAutomation and wrappers are just as it says wrappers around UIAUTOMATIONCORE.DLL which is shipped as part of the windows operating system.

you can hire me for EUR 300,-- per hour for onsite dedicated support

Share this post


Link to post
Share on other sites

I solved most parts. It remains unknown why it calls function 2 times once pop-up is visible:

Func oIUIAutomationEventHandler_HandleAutomationEvent($pSelf, $pSender, $iEventId) ; Ret: long  Par: ptr;int
    Dim $t
    If $iEventId <> $UIA_Window_WindowClosedEventId Then
        Local $oSender = ObjCreateInterface($pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
        $oSender.AddRef()
        If _UIA_getPropertyValue($oSender, $UIA_ControlTypePropertyId) = "50032" Then ;50032 popup
            ConsoleWrite("PopUp Visible - Windows Name: " & _UIA_getPropertyValue($oSender, $UIA_NamePropertyId) & @CRLF)
            findThemAll($oSender, $TreeScope_Subtree)
            ; sleep(5000) ; used to avoid 2nd loop
        EndIf
    EndIf
    Return $S_OK
EndFunc   ;==>oIUIAutomationEventHandler_HandleAutomationEvent
 
[...]
 
Func findThemAll($oElementStart, $TreeScope)
    Local $oCondition, $oAutomationElementArray, $oUIElement
    Dim $pCondition, $pTrueCondition
    Dim $pElements, $iLength
    $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition)
    $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition)
    $oElementStart.FindAll($TreeScope, $oCondition, $pElements)
    $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray)
    $oAutomationElementArray.Length($iLength)
    For $i = 0 To $iLength - 1; it's zero based
        $oAutomationElementArray.GetElement($i, $UIA_pUIElement)
        $oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
        ConsoleWrite("Title: " & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & @TAB & "Pos: " & _UIA_getPropertyValue($oUIElement, $UIA_BoundingRectanglePropertyId) & @TAB & "Center =" & _UIA_getPropertyValue($oUIElement, $UIA_ClickablePointPropertyId) & @TAB & "Class=" & _UIA_getPropertyValue($oUIElement, $uia_classnamepropertyid) & @CRLF)
    Next
EndFunc   ;==>findThemAll
 

 

I tried to put a sleep(5000) but it doesn't work.

I can I stop this reiteration but not the function itself? (I need to be informed about pop-ups but just one time not twice).

Thanks,

M.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object. Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?
×
×
  • Create New...