Jump to content
junkew

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

Melba23

D4RKON3,

Thanks. :)

M23


Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______Open Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites
dlow

Greetingz All

First off Thanks to everyone for their contribution.

@ junkew You are brilliant & thanks for sharing your code, knowledge, & time.

I have been working with your FF & IE examples. When I run the code I get a error in the UIAWrapper.

"C:\Program Files (x86)\AutoIt3\Include\UIAWrappers.au3" (363) : ==> Variable must be of type "Object".:
$obj.GetCurrentPropertyValue($Id,$tVal)
$obj^ ERROR

Well I am new to AutoIt & though I have tried to figure this out I don't know how to fix this. Could someone please point me in the right direction.

Thanks & all the best,

d ;)

Share this post


Link to post
Share on other sites
junkew

In an update I will put this function as a fix

replace the function in UIA_Wrappers with below and script will at least continue (but still not do what you want as the object itself is not found)

func _UIA_getPropertyValue($obj, $id)
    local $tval
    local $tStr

    if not isobj($obj) Then
        seterror(1,0,0)
        return "** NO PROPERTYVALUE DUE TO NONEXISTING OBJECT"
    EndIf
    
    $obj.GetCurrentPropertyValue($Id,$tVal)
    $tStr=""
    if isarray($tVal) Then
        for $i=0 to ubound($tval)-1
            $tStr=$tStr & $tVal[$i]
            if $i <> ubound($tVal)-1 Then
                $tStr=$tStr & ";"
            endif
        Next
        return $tStr
    endIf
    return $tVal
EndFunc

Share this post


Link to post
Share on other sites
dlow

Okay thanks.

In my quest to use the JAB I came across this post. The answer seems logical from my limited programing knowledge. I am not sure if you are doing anything like this in your code. Or know/don't know, I just wanted to put it out there in my effort to further help the development of the project.

Thanks again. Now I am off to try & put this wonderful code you wrote to work.

All the best,

dirty :P

Share this post


Link to post
Share on other sites
topten

Strange thing: When I run the script with admin rights, I get error "The requested action with this object has failed". But when I run it in normal mode- everything works fine. What can be the reason for that? Thanx and best regards, and Happy New Year!!!

Share this post


Link to post
Share on other sites
topten

I've checked it. I started simple spy with admin rights- and it works ok. You are right- the problem is different: IE 10 doesnt allow to create IE object with admin rights!!!!

 

Thanx a lot!!!!

Share this post


Link to post
Share on other sites
topten

Hi!

1. Possible bug: I found that in some cases ^a - simply won't work (for example I have multi language keyboard layout- and I was having something like a

avarvarvar instead of varvarvar, so I made a small change

case "setvalue using keys"
            $obj.setfocus()
            ;send("^a")
          send($p1)
          sleep($UIA_DefaultWaitTime)

Now works ok -perhaps will be usefull for anyone-  but again it can be different on different machines of course

2.  Is it possible to use Controlsend() function instead of send () -  so that I could work with application in background

case "setvalue using keys"
            $obj.setfocus()
            ;send("^a")
           Controlsend($p1)
            sleep($UIA_DefaultWaitTime)

If the answer "yes" how can I find controlID?

ControlSend ( "title", "text", controlID, "string" [, flag = 0] )

Thanx in advance!!!!!!

Share this post


Link to post
Share on other sites
junkew

not sure what in multi language the shortcut is for select all (Ctrl+A keycombination)

you could try to replace with 

a. send("^(a)")

b. send("^{a}")

c. make a small testapplication against notepad and see how it behaves with setvalue actions

try to use the action setvalue instead of setvalue using keys. Setvalue using keys is already a workaround for those textboxes that do not deal well with setvalue action itself
 

controlsend is the native AutoIT way of doing it so check within the helpfile of autoit the controlsend syntax

in general the simple spy automationid will be equal to the ID property of the AU3INF.EXE result(s) but not all controls are identifiable by a controlid.

Use the AU3INF.exe files to see your result(s)

This could be an alternate viewer for controls '?do=embed' frameborder='0' data-embedContent>>

Share this post


Link to post
Share on other sites
topten

Thank you very much for your help and support!!!! 

if IsObj($IE) Then
   $sText="search"; indexrelative:=1"    ;Is equal to
   $oButton=_UIA_getFirstObjectOfElement($IE,"name:=" & $sText, $treescope_subtree)
EndIf

Is there any method to calculate the x,y position of found element on the window or on the screen?

 

Thanx in advance!!!

Share this post


Link to post
Share on other sites
LarsJ

I have made some tests with event handler interfaces. For a start just to verify whether I could get some events at all.

I succeeded to get events for IUIAutomationEventHandler, IUIAutomationFocusChangedEventHandler and UIAutomationStructureChangedEventHandler but not for IUIAutomationPropertyChangedEventHandler.

Update 2014-01-13

Added the proper function to create custom interface methods to all 4 examples. See post #115. This means that all examples are working including the example with IUIAutomationPropertyChangedEventHandler.

IUIAutomationEventHandler

Run this code in Scite and you'll see window and menu open/close events in the console when you open/close a program e.g. Calculator and when you open/close a menu.

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global Const $S_OK = 0x00000000
Global Const $E_NOINTERFACE = 0x80004002
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"

Global $tIUIAutomationEventHandler, $oIUIAutomationEventHandler

Global $oUIAutomation

MainFunc()



Func MainFunc()

  $oIUIAutomationEventHandler = ObjectFromTag( "oIUIAutomationEventHandler_", $dtagIUIAutomationEventHandler, $tIUIAutomationEventHandler, True )
  If Not IsObj( $oIUIAutomationEventHandler ) Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pUIElement
  $oUIAutomation.GetRootElement( $pUIElement ) ; Desktop
  If Not $pUIElement Then Return

  ; Window open/close events
  If $oUIAutomation.AddAutomationEventHandler( $UIA_Window_WindowOpenedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit
  If $oUIAutomation.AddAutomationEventHandler( $UIA_Window_WindowClosedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit

  ; Menu open/close events
  If $oUIAutomation.AddAutomationEventHandler( $UIA_MenuOpenedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit
  If $oUIAutomation.AddAutomationEventHandler( $UIA_MenuClosedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit

  HotKeySet( "{ESC}", "Quit" )

  While Sleep(10)
  WEnd

EndFunc

Func Quit()
  $oIUIAutomationEventHandler = 0
  DeleteObjectFromTag( $tIUIAutomationEventHandler )
  Exit
EndFunc



Func _UIA_getPropertyValue( $obj, $id )
  Local $tVal
  $obj.GetCurrentPropertyValue( $id, $tVal )
  Return $tVal
EndFunc

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf, $pSender, $iEventId ) ; Ret: long  Par: ptr;int
  ConsoleWrite( "oIUIAutomationEventHandler_HandleAutomationEvent: " & $iEventId & @CRLF )
  If $iEventId <> $UIA_Window_WindowClosedEventId Then
    Local $oSender = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $oSender.AddRef()
    ConsoleWrite( "Title     = " & _UIA_getPropertyValue( $oSender, $UIA_NamePropertyId ) & @CRLF & _
                  "Class     = " & _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId ) & @CRLF & _
                  "Ctrl type = " & _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId ) & @CRLF & _
                  "Ctrl name = " & _UIA_getPropertyValue( $oSender, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                  "Value     = " & _UIA_getPropertyValue( $oSender, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                  "Handle    = " & Hex( _UIA_getPropertyValue( $oSender, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )
  EndIf
  Return $S_OK
EndFunc

Func oIUIAutomationEventHandler_QueryInterface( $pSelf, $pRIID, $pObj ) ; Ret: long  Par: ptr;ptr*
  Local $sIID = StringFromGUID( $pRIID )
  If $sIID = $sIID_IUnknown Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUnknown" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  ElseIf $sIID = $sIID_IUIAutomationEventHandler Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  Else
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: " & $sIID & @CRLF )
    Return $E_NOINTERFACE
  EndIf
EndFunc

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_AddRef" & @CRLF )
  Return 1
EndFunc

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_Release" & @CRLF )
  Return 1
EndFunc



Func StringFromGUID( $pGUID )
  Local $aResult = DllCall( "ole32.dll", "int", "StringFromGUID2", "struct*", $pGUID, "wstr", "", "int", 40 )
  If @error Then Return SetError( @error, @extended, "" )
  Return SetExtended( $aResult[0], $aResult[2] )
EndFunc



Func ObjectFromTag($sFunctionPrefix, $tagInterface, ByRef $tInterface, $fPrint = False, $bIsUnknown = Default, $sIID = "{00000000-0000-0000-C000-000000000046}") ; last param is IID_IUnknown by default
    If $bIsUnknown = Default Then $bIsUnknown = True
    Local $sInterface = $tagInterface ; copy interface description
    Local $tagIUnknown = "QueryInterface hresult(ptr;ptr*);" & _
            "AddRef dword();" & _
            "Release dword();"
    ; Adding IUnknown methods
    If $bIsUnknown Then $tagInterface = $tagIUnknown & $tagInterface
    ; Below line is really simple even though it looks super complex. It's just written weird to fit in one line, not to steal your attention
    Local $aMethods = StringSplit(StringReplace(StringReplace(StringReplace(StringReplace(StringTrimRight(StringReplace(StringRegExpReplace(StringRegExpReplace($tagInterface, "\w+\*", "ptr"), "\h*(\w+)\h*(\w+\*?)\h*(\((.*?)\))\h*(;|;*\z)", "$1\|$2;$4" & @LF), ";" & @LF, @LF), 1), "object", "idispatch"), "hresult", "long"), "bstr", "ptr"), "variant", "ptr"), @LF, 3)
    Local $iUbound = UBound($aMethods)
    Local $sMethod, $aSplit, $sNamePart, $aTagPart, $sTagPart, $sRet, $sParams, $hCallback
    ; Allocation
    $tInterface = DllStructCreate("int RefCount;int Size;ptr Object;ptr Methods[" & $iUbound & "];int_ptr Callbacks[" & $iUbound & "];ulong_ptr Slots[16]") ; 16 pointer sized elements more to create space for possible private props
    If @error Then Return SetError(1, 0, 0)
    For $i = 0 To $iUbound - 1
        $aSplit = StringSplit($aMethods[$i], "|", 2)
        If UBound($aSplit) <> 2 Then ReDim $aSplit[2]
        $sNamePart = $aSplit[0]
        $sTagPart = $aSplit[1]
        $sMethod = $sFunctionPrefix & $sNamePart
        If $fPrint Then
            Local $iPar = StringInStr( $sTagPart, ";", 2 ), $t
            If $iPar Then
                $t = "Ret: " & StringLeft( $sTagPart, $iPar - 1 ) & "  " & _
                     "Par: " & StringRight( $sTagPart, StringLen( $sTagPart ) - $iPar )
            Else
                $t = "Ret: " & $sTagPart
            EndIf
            Local $s = "Func " & $sMethod & _
                "( $pSelf ) ; " & $t & @CRLF & _
                "EndFunc" & @CRLF
            ConsoleWrite( $s )
        EndIf
        $aTagPart = StringSplit($sTagPart, ";", 2)
        $sRet = $aTagPart[0]
        $sParams = StringReplace($sTagPart, $sRet, "", 1)
        $sParams = "ptr" & $sParams
        $hCallback = DllCallbackRegister($sMethod, $sRet, $sParams)
        ConsoleWrite(@error & @CRLF & @CRLF)
        DllStructSetData($tInterface, "Methods", DllCallbackGetPtr($hCallback), $i + 1) ; save callback pointer
        DllStructSetData($tInterface, "Callbacks", $hCallback, $i + 1) ; save callback handle
    Next
    DllStructSetData($tInterface, "RefCount", 1) ; initial ref count is 1
    DllStructSetData($tInterface, "Size", $iUbound) ; number of interface methods
    DllStructSetData($tInterface, "Object", DllStructGetPtr($tInterface, "Methods")) ; Interface method pointers
    Return ObjCreateInterface(DllStructGetPtr($tInterface, "Object"), $sIID, $sInterface, $bIsUnknown) ; pointer that's wrapped into object
EndFunc

Func DeleteObjectFromTag(ByRef $tInterface)
    For $i = 1 To DllStructGetData($tInterface, "Size")
        DllCallbackFree(DllStructGetData($tInterface, "Callbacks", $i))
    Next
    $tInterface = 0
EndFunc

IUIAutomationFocusChangedEventHandler

If you test with Calculator you'll get events when you click a number button and the focus changes between the button and the edit box.

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global Const $S_OK = 0x00000000
Global Const $E_NOINTERFACE = 0x80004002
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"

Global $tIUIAutomationFocusChangedEventHandler, $oIUIAutomationFocusChangedEventHandler

Global $oUIAutomation

MainFunc()



Func MainFunc()

  $oIUIAutomationFocusChangedEventHandler = ObjectFromTag( "oIUIAutomationFocusChangedEventHandler_", $dtagIUIAutomationFocusChangedEventHandler, $tIUIAutomationFocusChangedEventHandler, True )
  If Not IsObj( $oIUIAutomationFocusChangedEventHandler ) Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pUIElement
  $oUIAutomation.GetRootElement( $pUIElement ) ; Desktop
  If Not $pUIElement Then Return

  If $oUIAutomation.AddFocusChangedEventHandler( 0, $oIUIAutomationFocusChangedEventHandler ) Then Exit

  HotKeySet( "{ESC}", "Quit" )

  While Sleep(10)
  WEnd

EndFunc

Func Quit()
  $oIUIAutomationFocusChangedEventHandler = 0
  DeleteObjectFromTag( $tIUIAutomationFocusChangedEventHandler )
  Exit
EndFunc



Func _UIA_getPropertyValue( $obj, $id )
  Local $tVal
  $obj.GetCurrentPropertyValue( $id, $tVal )
  Return $tVal
EndFunc

Func oIUIAutomationFocusChangedEventHandler_HandleFocusChangedEvent( $pSelf, $pSender ) ; Ret: long  Par: ptr
  ConsoleWrite( "oIUIAutomationFocusChangedEventHandler_HandleFocusChangedEvent: " & $pSender & @CRLF )
  Local $oSender = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  $oSender.AddRef()
  ConsoleWrite( "Title     = " & _UIA_getPropertyValue( $oSender, $UIA_NamePropertyId ) & @CRLF & _
                "Class     = " & _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId ) & @CRLF & _
                "Ctrl type = " & _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId ) & @CRLF & _
                "Ctrl name = " & _UIA_getPropertyValue( $oSender, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                "Value     = " & _UIA_getPropertyValue( $oSender, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                "Handle    = " & Hex( _UIA_getPropertyValue( $oSender, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )
  Return $S_OK
EndFunc

Func oIUIAutomationFocusChangedEventHandler_QueryInterface( $pSelf, $pRIID, $pObj ) ; Ret: long  Par: ptr;ptr*
  Local $sIID = StringFromGUID( $pRIID )
  If $sIID = $sIID_IUnknown Then
    ConsoleWrite( "oIUIAutomationFocusChangedEventHandler_QueryInterface: IUnknown" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationFocusChangedEventHandler_AddRef( $pSelf )
    Return $S_OK
  ElseIf $sIID = $sIID_IUIAutomationFocusChangedEventHandler Then
    ConsoleWrite( "oIUIAutomationFocusChangedEventHandler_QueryInterface: IUIAutomationFocusChangedEventHandler" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationFocusChangedEventHandler_AddRef( $pSelf )
    Return $S_OK
  Else
    ConsoleWrite( "oIUIAutomationFocusChangedEventHandler_QueryInterface: " & $sIID & @CRLF )
    Return $E_NOINTERFACE
  EndIf
EndFunc

Func oIUIAutomationFocusChangedEventHandler_AddRef( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationFocusChangedEventHandler_AddRef" & @CRLF )
  Return 1
EndFunc

Func oIUIAutomationFocusChangedEventHandler_Release( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationFocusChangedEventHandler_Release" & @CRLF )
  Return 1
EndFunc



Func StringFromGUID( $pGUID )
  Local $aResult = DllCall( "ole32.dll", "int", "StringFromGUID2", "struct*", $pGUID, "wstr", "", "int", 40 )
  If @error Then Return SetError( @error, @extended, "" )
  Return SetExtended( $aResult[0], $aResult[2] )
EndFunc



Func ObjectFromTag($sFunctionPrefix, $tagInterface, ByRef $tInterface, $fPrint = False, $bIsUnknown = Default, $sIID = "{00000000-0000-0000-C000-000000000046}") ; last param is IID_IUnknown by default
    If $bIsUnknown = Default Then $bIsUnknown = True
    Local $sInterface = $tagInterface ; copy interface description
    Local $tagIUnknown = "QueryInterface hresult(ptr;ptr*);" & _
            "AddRef dword();" & _
            "Release dword();"
    ; Adding IUnknown methods
    If $bIsUnknown Then $tagInterface = $tagIUnknown & $tagInterface
    ; Below line is really simple even though it looks super complex. It's just written weird to fit in one line, not to steal your attention
    Local $aMethods = StringSplit(StringReplace(StringReplace(StringReplace(StringReplace(StringTrimRight(StringReplace(StringRegExpReplace(StringRegExpReplace($tagInterface, "\w+\*", "ptr"), "\h*(\w+)\h*(\w+\*?)\h*(\((.*?)\))\h*(;|;*\z)", "$1\|$2;$4" & @LF), ";" & @LF, @LF), 1), "object", "idispatch"), "hresult", "long"), "bstr", "ptr"), "variant", "ptr"), @LF, 3)
    Local $iUbound = UBound($aMethods)
    Local $sMethod, $aSplit, $sNamePart, $aTagPart, $sTagPart, $sRet, $sParams, $hCallback
    ; Allocation
    $tInterface = DllStructCreate("int RefCount;int Size;ptr Object;ptr Methods[" & $iUbound & "];int_ptr Callbacks[" & $iUbound & "];ulong_ptr Slots[16]") ; 16 pointer sized elements more to create space for possible private props
    If @error Then Return SetError(1, 0, 0)
    For $i = 0 To $iUbound - 1
        $aSplit = StringSplit($aMethods[$i], "|", 2)
        If UBound($aSplit) <> 2 Then ReDim $aSplit[2]
        $sNamePart = $aSplit[0]
        $sTagPart = $aSplit[1]
        $sMethod = $sFunctionPrefix & $sNamePart
        If $fPrint Then
            Local $iPar = StringInStr( $sTagPart, ";", 2 ), $t
            If $iPar Then
                $t = "Ret: " & StringLeft( $sTagPart, $iPar - 1 ) & "  " & _
                     "Par: " & StringRight( $sTagPart, StringLen( $sTagPart ) - $iPar )
            Else
                $t = "Ret: " & $sTagPart
            EndIf
            Local $s = "Func " & $sMethod & _
                "( $pSelf ) ; " & $t & @CRLF & _
                "EndFunc" & @CRLF
            ConsoleWrite( $s )
        EndIf
        $aTagPart = StringSplit($sTagPart, ";", 2)
        $sRet = $aTagPart[0]
        $sParams = StringReplace($sTagPart, $sRet, "", 1)
        $sParams = "ptr" & $sParams
        $hCallback = DllCallbackRegister($sMethod, $sRet, $sParams)
        ConsoleWrite(@error & @CRLF & @CRLF)
        DllStructSetData($tInterface, "Methods", DllCallbackGetPtr($hCallback), $i + 1) ; save callback pointer
        DllStructSetData($tInterface, "Callbacks", $hCallback, $i + 1) ; save callback handle
    Next
    DllStructSetData($tInterface, "RefCount", 1) ; initial ref count is 1
    DllStructSetData($tInterface, "Size", $iUbound) ; number of interface methods
    DllStructSetData($tInterface, "Object", DllStructGetPtr($tInterface, "Methods")) ; Interface method pointers
    Return ObjCreateInterface(DllStructGetPtr($tInterface, "Object"), $sIID, $sInterface, $bIsUnknown) ; pointer that's wrapped into object
EndFunc

Func DeleteObjectFromTag(ByRef $tInterface)
    For $i = 1 To DllStructGetData($tInterface, "Size")
        DllCallbackFree(DllStructGetData($tInterface, "Callbacks", $i))
    Next
    $tInterface = 0
EndFunc

IUIAutomationStructureChangedEventHandler

Switch between Standard and Scientific view in Calculator or switch folder in Windows Explorer and you'll get lots of events.

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global Const $S_OK = 0x00000000
Global Const $E_NOINTERFACE = 0x80004002
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"

Global $tIUIAutomationStructureChangedEventHandler, $oIUIAutomationStructureChangedEventHandler

Global $oUIAutomation

MainFunc()



Func MainFunc()

  $oIUIAutomationStructureChangedEventHandler = ObjectFromTag( "oIUIAutomationStructureChangedEventHandler_", $dtagIUIAutomationStructureChangedEventHandler, $tIUIAutomationStructureChangedEventHandler, True )
  If Not IsObj( $oIUIAutomationStructureChangedEventHandler ) Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pUIElement
  $oUIAutomation.GetRootElement( $pUIElement ) ; Desktop
  If Not $pUIElement Then Return

  If $oUIAutomation.AddStructureChangedEventHandler( $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationStructureChangedEventHandler ) Then Exit

  HotKeySet( "{ESC}", "Quit" )

  While Sleep(10)
  WEnd

EndFunc

Func Quit()
  $oIUIAutomationStructureChangedEventHandler = 0
  DeleteObjectFromTag( $tIUIAutomationStructureChangedEventHandler )
  Exit
EndFunc



Func _UIA_getPropertyValue( $obj, $id )
  Local $tVal
  $obj.GetCurrentPropertyValue( $id, $tVal )
  Return $tVal
EndFunc

Func oIUIAutomationStructureChangedEventHandler_HandleStructureChangedEvent( $pSelf, $pSender, $iChangeType, $pRuntimeId ) ; Ret: long  Par: ptr;long;ptr
  ConsoleWrite( "oIUIAutomationStructureChangedEventHandler_HandleStructureChangedEvent: " & $pSender & " " & $iChangeType & @CRLF )
  Local $oSender = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  $oSender.AddRef()
  ConsoleWrite( "Title     = " & _UIA_getPropertyValue( $oSender, $UIA_NamePropertyId ) & @CRLF & _
                "Class     = " & _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId ) & @CRLF & _
                "Ctrl type = " & _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId ) & @CRLF & _
                "Ctrl name = " & _UIA_getPropertyValue( $oSender, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                "Value     = " & _UIA_getPropertyValue( $oSender, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                "Handle    = " & Hex( _UIA_getPropertyValue( $oSender, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )
  Return $S_OK
EndFunc

Func oIUIAutomationStructureChangedEventHandler_QueryInterface( $pSelf, $pRIID, $pObj ) ; Ret: long  Par: ptr;ptr*
  Local $sIID = StringFromGUID( $pRIID )
  If $sIID = $sIID_IUnknown Then
    ConsoleWrite( "oIUIAutomationStructureChangedEventHandler_QueryInterface: IUnknown" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationStructureChangedEventHandler_AddRef( $pSelf )
    Return $S_OK
  ElseIf $sIID = $sIID_IUIAutomationStructureChangedEventHandler Then
    ConsoleWrite( "oIUIAutomationStructureChangedEventHandler_QueryInterface: IUIAutomationStructureChangedEventHandler" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationStructureChangedEventHandler_AddRef( $pSelf )
    Return $S_OK
  Else
    ConsoleWrite( "oIUIAutomationStructureChangedEventHandler_QueryInterface: " & $sIID & @CRLF )
    Return $E_NOINTERFACE
  EndIf
EndFunc

Func oIUIAutomationStructureChangedEventHandler_AddRef( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationStructureChangedEventHandler_AddRef" & @CRLF )
  Return 1
EndFunc

Func oIUIAutomationStructureChangedEventHandler_Release( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationStructureChangedEventHandler_Release" & @CRLF )
  Return 1
EndFunc



Func StringFromGUID( $pGUID )
  Local $aResult = DllCall( "ole32.dll", "int", "StringFromGUID2", "struct*", $pGUID, "wstr", "", "int", 40 )
  If @error Then Return SetError( @error, @extended, "" )
  Return SetExtended( $aResult[0], $aResult[2] )
EndFunc



Func ObjectFromTag($sFunctionPrefix, $tagInterface, ByRef $tInterface, $fPrint = False, $bIsUnknown = Default, $sIID = "{00000000-0000-0000-C000-000000000046}") ; last param is IID_IUnknown by default
    If $bIsUnknown = Default Then $bIsUnknown = True
    Local $sInterface = $tagInterface ; copy interface description
    Local $tagIUnknown = "QueryInterface hresult(ptr;ptr*);" & _
            "AddRef dword();" & _
            "Release dword();"
    ; Adding IUnknown methods
    If $bIsUnknown Then $tagInterface = $tagIUnknown & $tagInterface
    ; Below line is really simple even though it looks super complex. It's just written weird to fit in one line, not to steal your attention
    Local $aMethods = StringSplit(StringReplace(StringReplace(StringReplace(StringReplace(StringTrimRight(StringReplace(StringRegExpReplace(StringRegExpReplace($tagInterface, "\w+\*", "ptr"), "\h*(\w+)\h*(\w+\*?)\h*(\((.*?)\))\h*(;|;*\z)", "$1\|$2;$4" & @LF), ";" & @LF, @LF), 1), "object", "idispatch"), "hresult", "long"), "bstr", "ptr"), "variant", "ptr"), @LF, 3)
    Local $iUbound = UBound($aMethods)
    Local $sMethod, $aSplit, $sNamePart, $aTagPart, $sTagPart, $sRet, $sParams, $hCallback
    ; Allocation
    $tInterface = DllStructCreate("int RefCount;int Size;ptr Object;ptr Methods[" & $iUbound & "];int_ptr Callbacks[" & $iUbound & "];ulong_ptr Slots[16]") ; 16 pointer sized elements more to create space for possible private props
    If @error Then Return SetError(1, 0, 0)
    For $i = 0 To $iUbound - 1
        $aSplit = StringSplit($aMethods[$i], "|", 2)
        If UBound($aSplit) <> 2 Then ReDim $aSplit[2]
        $sNamePart = $aSplit[0]
        $sTagPart = $aSplit[1]
        $sMethod = $sFunctionPrefix & $sNamePart
        If $fPrint Then
            Local $iPar = StringInStr( $sTagPart, ";", 2 ), $t
            If $iPar Then
                $t = "Ret: " & StringLeft( $sTagPart, $iPar - 1 ) & "  " & _
                     "Par: " & StringRight( $sTagPart, StringLen( $sTagPart ) - $iPar )
            Else
                $t = "Ret: " & $sTagPart
            EndIf
            Local $s = "Func " & $sMethod & _
                "( $pSelf ) ; " & $t & @CRLF & _
                "EndFunc" & @CRLF
            ConsoleWrite( $s )
        EndIf
        $aTagPart = StringSplit($sTagPart, ";", 2)
        $sRet = $aTagPart[0]
        $sParams = StringReplace($sTagPart, $sRet, "", 1)
        $sParams = "ptr" & $sParams
        $hCallback = DllCallbackRegister($sMethod, $sRet, $sParams)
        ConsoleWrite(@error & @CRLF & @CRLF)
        DllStructSetData($tInterface, "Methods", DllCallbackGetPtr($hCallback), $i + 1) ; save callback pointer
        DllStructSetData($tInterface, "Callbacks", $hCallback, $i + 1) ; save callback handle
    Next
    DllStructSetData($tInterface, "RefCount", 1) ; initial ref count is 1
    DllStructSetData($tInterface, "Size", $iUbound) ; number of interface methods
    DllStructSetData($tInterface, "Object", DllStructGetPtr($tInterface, "Methods")) ; Interface method pointers
    Return ObjCreateInterface(DllStructGetPtr($tInterface, "Object"), $sIID, $sInterface, $bIsUnknown) ; pointer that's wrapped into object
EndFunc

Func DeleteObjectFromTag(ByRef $tInterface)
    For $i = 1 To DllStructGetData($tInterface, "Size")
        DllCallbackFree(DllStructGetData($tInterface, "Callbacks", $i))
    Next
    $tInterface = 0
EndFunc

IUIAutomationPropertyChangedEventHandler

I have tested the code below with Internet Explorer and I have expected to get $UIA_ValueValuePropertyId change events when I switch between different URLs. But I get no events. I have tried with several other programs but I simply get no events.

Is there anyone who has managed to record these events? In that case, I'd like to see the code.

To test the code you must change the description for AddPropertyChangedEventHandlerNativeArray in CUIAutomation2.au3 to this:

"AddPropertyChangedEventHandlerNativeArray hresult(ptr;long;ptr;ptr;struct*;int);"
#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global Const $S_OK = 0x00000000
Global Const $E_NOINTERFACE = 0x80004002
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"

Global $tIUIAutomationPropertyChangedEventHandler, $oIUIAutomationPropertyChangedEventHandler

Global $oUIAutomation

MainFunc()



Func MainFunc()

  $oIUIAutomationPropertyChangedEventHandler = ObjectFromTag( "oIUIAutomationPropertyChangedEventHandler_", $dtagIUIAutomationPropertyChangedEventHandler, $tIUIAutomationPropertyChangedEventHandler, True )
  If Not IsObj( $oIUIAutomationPropertyChangedEventHandler ) Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pUIElement
  $oUIAutomation.GetRootElement( $pUIElement ) ; Desktop
  If Not $pUIElement Then Return

  Local $tPropertyArray = DllStructCreate( "int[2]" )
  DllStructSetData( $tPropertyArray, 1, $UIA_ValueValuePropertyId, 1 )
  DllStructSetData( $tPropertyArray, 1, $UIA_ToggleToggleStatePropertyId, 2 )
  $oUIAutomation.AddPropertyChangedEventHandlerNativeArray( $pUIElement, $TreeScope_Descendants, 0, $oIUIAutomationPropertyChangedEventHandler, $tPropertyArray, 2 )

  HotKeySet( "{ESC}", "Quit" )

  While Sleep(100)
  WEnd

EndFunc

Func Quit()
  $oIUIAutomationPropertyChangedEventHandler = 0
  DeleteObjectFromTag( $tIUIAutomationPropertyChangedEventHandler )
  Exit
EndFunc



Func _UIA_getPropertyValue( $obj, $id )
  Local $vVal
  $obj.GetCurrentPropertyValue( $id, $vVal )
  Return $vVal
EndFunc

Func oIUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent( $pSelf, $pSender, $iPropertyId, $newValue ) ; Ret: long  Par: ptr;int;variant
  ConsoleWrite( @CRLF & "oIUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent: $iPropertyId = " & $iPropertyId & ", $newValue = " & $newValue & @CRLF )

  Local $oSender = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  $oSender.AddRef(  )

  ConsoleWrite( "Handle    " & _UIA_getPropertyValue( $oSender, $UIA_NativeWindowHandlePropertyId ) & @CRLF )
  ConsoleWrite( "Name      " & _UIA_getPropertyValue( $oSender, $UIA_NamePropertyId ) & @CRLF )
  ConsoleWrite( "Class     " & _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId ) & @CRLF )
  ConsoleWrite( "Ctrl type " & _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId ) & @CRLF )
  ConsoleWrite( "Ctrl name " & _UIA_getPropertyValue( $oSender, $UIA_LocalizedControlTypePropertyId ) & @CRLF )
  ConsoleWrite( "Value     " & _UIA_getPropertyValue( $oSender, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF )

  Return $S_OK
EndFunc

Func oIUIAutomationPropertyChangedEventHandler_QueryInterface( $pSelf, $pRIID, $pObj ) ; Ret: long  Par: ptr;ptr*
  Local $sIID = StringFromGUID( $pRIID )
  If $sIID = $sIID_IUnknown Then
    ConsoleWrite( "oIUIAutomationPropertyChangedEventHandler_QueryInterface: IUnknown" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    Return $S_OK
  ElseIf $sIID = $sIID_IUIAutomationPropertyChangedEventHandler Then
    ConsoleWrite( "oIUIAutomationPropertyChangedEventHandler_QueryInterface: IUIAutomationPropertyChangedEventHandler" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    Return $S_OK
  Else
    ConsoleWrite( "oIUIAutomationPropertyChangedEventHandler_QueryInterface: " & $sIID & @CRLF )
    Return $E_NOINTERFACE
  EndIf
EndFunc

Func oIUIAutomationPropertyChangedEventHandler_AddRef( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationPropertyChangedEventHandler_AddRef" & @CRLF )
  Return 1
EndFunc

Func oIUIAutomationPropertyChangedEventHandler_Release( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationPropertyChangedEventHandler_Release" & @CRLF )
  Return 1
EndFunc



Func StringFromGUID( $pGUID )
  Local $aResult = DllCall( "ole32.dll", "int", "StringFromGUID2", "struct*", $pGUID, "wstr", "", "int", 40 )
  If @error Then Return SetError( @error, @extended, "" )
  Return SetExtended( $aResult[0], $aResult[2] )
EndFunc



Func ObjectFromTag($sFunctionPrefix, $tagInterface, ByRef $tInterface, $fPrint = False, $bIsUnknown = Default, $sIID = "{00000000-0000-0000-C000-000000000046}") ; last param is IID_IUnknown by default
    If $bIsUnknown = Default Then $bIsUnknown = True
    Local $sInterface = $tagInterface ; copy interface description
    Local $tagIUnknown = "QueryInterface hresult(ptr;ptr*);" & _
            "AddRef dword();" & _
            "Release dword();"
    ; Adding IUnknown methods
    If $bIsUnknown Then $tagInterface = $tagIUnknown & $tagInterface
    ; Below line is really simple even though it looks super complex. It's just written weird to fit in one line, not to steal your attention
    Local $aMethods = StringSplit(StringReplace(StringReplace(StringReplace(StringReplace(StringTrimRight(StringReplace(StringRegExpReplace(StringRegExpReplace($tagInterface, "\w+\*", "ptr"), "\h*(\w+)\h*(\w+\*?)\h*(\((.*?)\))\h*(;|;*\z)", "$1\|$2;$4" & @LF), ";" & @LF, @LF), 1), "object", "idispatch"), "hresult", "long"), "bstr", "ptr"), "variant", "ptr"), @LF, 3)
    Local $iUbound = UBound($aMethods)
    Local $sMethod, $aSplit, $sNamePart, $aTagPart, $sTagPart, $sRet, $sParams, $hCallback
    ; Allocation
    $tInterface = DllStructCreate("int RefCount;int Size;ptr Object;ptr Methods[" & $iUbound & "];int_ptr Callbacks[" & $iUbound & "];ulong_ptr Slots[16]") ; 16 pointer sized elements more to create space for possible private props
    If @error Then Return SetError(1, 0, 0)
    For $i = 0 To $iUbound - 1
        $aSplit = StringSplit($aMethods[$i], "|", 2)
        If UBound($aSplit) <> 2 Then ReDim $aSplit[2]
        $sNamePart = $aSplit[0]
        $sTagPart = $aSplit[1]
        $sMethod = $sFunctionPrefix & $sNamePart
        If $fPrint Then
            Local $iPar = StringInStr( $sTagPart, ";", 2 ), $t
            If $iPar Then
                $t = "Ret: " & StringLeft( $sTagPart, $iPar - 1 ) & "  " & _
                     "Par: " & StringRight( $sTagPart, StringLen( $sTagPart ) - $iPar )
            Else
                $t = "Ret: " & $sTagPart
            EndIf
            Local $s = "Func " & $sMethod & _
                "( $pSelf ) ; " & $t & @CRLF & _
                "EndFunc" & @CRLF
            ConsoleWrite( $s )
        EndIf
        $aTagPart = StringSplit($sTagPart, ";", 2)
        $sRet = $aTagPart[0]
        $sParams = StringReplace($sTagPart, $sRet, "", 1)
        $sParams = "ptr" & $sParams
        $hCallback = DllCallbackRegister($sMethod, $sRet, $sParams)
        ConsoleWrite(@error & @CRLF & @CRLF)
        DllStructSetData($tInterface, "Methods", DllCallbackGetPtr($hCallback), $i + 1) ; save callback pointer
        DllStructSetData($tInterface, "Callbacks", $hCallback, $i + 1) ; save callback handle
    Next
    DllStructSetData($tInterface, "RefCount", 1) ; initial ref count is 1
    DllStructSetData($tInterface, "Size", $iUbound) ; number of interface methods
    DllStructSetData($tInterface, "Object", DllStructGetPtr($tInterface, "Methods")) ; Interface method pointers
    Return ObjCreateInterface(DllStructGetPtr($tInterface, "Object"), $sIID, $sInterface, $bIsUnknown) ; pointer that's wrapped into object
EndFunc

Func DeleteObjectFromTag(ByRef $tInterface)
    For $i = 1 To DllStructGetData($tInterface, "Size")
        DllCallbackFree(DllStructGetData($tInterface, "Callbacks", $i))
    Next
    $tInterface = 0
EndFunc

Update 2014-01-13

Added the proper function to create custom interface methods to all 4 examples. See post #115. This means that all examples are working including the example with IUIAutomationPropertyChangedEventHandler.

Edited by LarsJ

Share this post


Link to post
Share on other sites
LarsJ

I didn't succeed to get events for the IUIAutomationPropertyChangedEventHandler in the post above. As a quick test I have put the code in a DLL. And now I get events (printed in a text file).

This is the code in the DLL:

#define WIN32_LEAN_AND_MEAN

#include <stdio.h>
#include <windows.h>
#include <oleauto.h>
#include <UIAutomation.h>

class CEventHandler : public IUIAutomationPropertyChangedEventHandler
{
  public: 

    CEventHandler();
    ~CEventHandler();

    HRESULT RegisterEventHandler();

    IFACEMETHODIMP QueryInterface(__in REFIID riid, __out PVOID *ppvObj);
    IFACEMETHODIMP_(ULONG) AddRef();
    IFACEMETHODIMP_(ULONG) Release();

    IFACEMETHODIMP HandlePropertyChangedEvent(IUIAutomationElement *sender, PROPERTYID propertyId, VARIANT newValue);

  private:

    void  UIAPrintProperty( IUIAutomationElement *pUIAutomationElement, PROPERTYID propertyId, char *text );
    ULONG _cRef;
    FILE  *f;
};

CEventHandler  *pEventHandler = NULL;


extern "C" __declspec(dllexport) int __cdecl TstFunc()
{
  pEventHandler = new CEventHandler();
  if ( pEventHandler == NULL ) return 1;

  HRESULT hr = pEventHandler->RegisterEventHandler();
  if ( SUCCEEDED(hr) < 0 ) return 1;

  return 0;
}


CEventHandler::CEventHandler() : _cRef(1)
{
  CoInitialize( NULL );
  f = fopen( "UIAEventHandler.txt", "w" );
}

CEventHandler::~CEventHandler()
{
  fclose(f);
  CoUninitialize();
}

HRESULT CEventHandler::RegisterEventHandler()
{
  HRESULT              hr = E_FAIL;
  IUIAutomation        *pUIAutomation;
  IUIAutomationElement *pUIAutomationElement;
  PROPERTYID           properties[] = {UIA_ValueValuePropertyId,
                                       UIA_ToggleToggleStatePropertyId};

  hr = CoCreateInstance(CLSID_CUIAutomation, NULL, CLSCTX_INPROC_SERVER, IID_PPV_ARGS(&pUIAutomation));
  if (SUCCEEDED(hr))
  {
    hr = pUIAutomation->GetRootElement( &pUIAutomationElement );
    if (SUCCEEDED(hr))
    {
      hr = pUIAutomation->AddPropertyChangedEventHandlerNativeArray( pUIAutomationElement, TreeScope_Descendants, NULL, this, properties, sizeof(properties)/sizeof(properties[0]) );
      if (SUCCEEDED(hr)) {
        fprintf( f, "AddPropertyChangedEventHandlerNativeArray\n" );
      }
    }
  }

  return hr;
}

IFACEMETHODIMP CEventHandler::HandlePropertyChangedEvent(IUIAutomationElement *sender, PROPERTYID propertyId, VARIANT newValue)
{
  fprintf( f, "\nHandlePropertyChangedEvent\n" );
  UIAPrintProperty( sender, UIA_NativeWindowHandlePropertyId,     "Handle   " );
  UIAPrintProperty( sender, UIA_NamePropertyId,                   "Name     " );
  UIAPrintProperty( sender, UIA_ClassNamePropertyId,              "Class    " );
  UIAPrintProperty( sender, UIA_ControlTypePropertyId,            "Ctrl type" );
  UIAPrintProperty( sender, UIA_LocalizedControlTypePropertyId,   "Ctrl name" );
  UIAPrintProperty( sender, UIA_LegacyIAccessibleValuePropertyId, "Value    " );
  return S_OK;
}

IFACEMETHODIMP CEventHandler::QueryInterface(__in REFIID riid, __out PVOID *ppvObj)
{
  if (ppvObj == NULL)
  {
    return E_POINTER;
  }

  *ppvObj = NULL;

  HRESULT hr = E_NOINTERFACE;

  if ( riid == __uuidof(IUnknown) )
  {
    fprintf( f, "QueryInterface: IUnknown\n" );
    *ppvObj = static_cast<IUIAutomationPropertyChangedEventHandler*>(this);
    AddRef();
    hr = S_OK;
  }
  else if ( riid == __uuidof(IUIAutomationPropertyChangedEventHandler) )
  {
    fprintf( f, "QueryInterface: IUIAutomationPropertyChangedEventHandler\n" );
    *ppvObj = static_cast<IUIAutomationPropertyChangedEventHandler*>(this);
    AddRef();
    hr = S_OK;
  }
  else
  {
    LPOLESTR lpolestr;
    StringFromCLSID( riid, &lpolestr );
    fwprintf( f, L"QueryInterface: %s\n", lpolestr );
    CoTaskMemFree( lpolestr );
  }

  return hr;
}

IFACEMETHODIMP_(ULONG) CEventHandler::AddRef()
{
  return InterlockedIncrement(&_cRef);
}

IFACEMETHODIMP_(ULONG) CEventHandler::Release()
{
  ULONG cRef = InterlockedDecrement(&_cRef);
  if (cRef == 0)
  {
    delete this;
  }

  return cRef;
}

void CEventHandler::UIAPrintProperty( IUIAutomationElement *pUIAutomationElement, PROPERTYID propertyId, char *text )
{
  VARIANT v;
  pUIAutomationElement->GetCurrentPropertyValue( propertyId, &v );
  if ( v.vt == VT_I4 )
    if ( strcmp( text, "Handle   " ) == 0 )
      fprintf( f, "%s: 0x%p\n", text, v.lVal);
    else
      fprintf( f, "%s: %d\n", text, v.lVal);
  else if ( v.vt == VT_BSTR ) {
    fprintf( f, "%s: ", text );
    fwprintf( f, L"%s\n", v.bstrVal ); }
}

The AutoIt GUI:

#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>

Opt( "MustDeclareVars", 1 )

MainFunc()

Func MainFunc()
  Local $dllTest, $aRet
  If @AutoItX64 Then
    $dllTest = DllOpen( "UIAEventHandler-x64.dll" )
  Else
    $dllTest = DllOpen( "UIAEventHandler.dll" )
  EndIf
  GUICreate( "UIAEventHandler", 200, 100, 200, 100 )
  GUISetState( @SW_SHOW )
  $aRet = DllCall( $dllTest, "int:cdecl", "TstFunc" )
  ;ConsoleWrite( "@error = " & @error & @CRLF )
  ;ConsoleWrite( "TstFunc = " & $aRet[0] & @CRLF )
  While GUIGetMsg() <> $GUI_EVENT_CLOSE
  WEnd
  GUIDelete()
  DllClose( $dllTest )
  Exit
EndFunc

And this is the events when I open Internet Explorer, change the URL to AutoIt home, open a text file with Wordpad and types "Test":

QueryInterface: {0000001B-0000-0000-C000-000000000046}
QueryInterface: {00000003-0000-0000-C000-000000000046}
AddPropertyChangedEventHandlerNativeArray
QueryInterface: {0000001B-0000-0000-C000-000000000046}
QueryInterface: IUnknown
QueryInterface: {00000018-0000-0000-C000-000000000046}
QueryInterface: {00000019-0000-0000-C000-000000000046}
QueryInterface: {4C1E39E1-E3E3-4296-AA86-EC938D896E92}
QueryInterface: IUIAutomationPropertyChangedEventHandler
QueryInterface: {1C733A30-2A1C-11CE-ADE5-00AA0044773D}
QueryInterface: IUIAutomationPropertyChangedEventHandler

HandlePropertyChangedEvent
Handle   : 0x00000000000D0506
Name     : Address and search using Google
Class    : Edit
Ctrl type: 50004
Ctrl name: edit
Value    : about:blank

HandlePropertyChangedEvent
Handle   : 0x00000000000D0506
Name     : Address and search using Google
Class    : Edit
Ctrl type: 50004
Ctrl name: edit
Value    : about:blank

HandlePropertyChangedEvent
Handle   : 0x00000000000D0506
Name     : Address and search using Google
Class    : Edit
Ctrl type: 50004
Ctrl name: edit
Value    : http://www.autoitscript.com/

HandlePropertyChangedEvent
Handle   : 0x00000000000D0506
Name     : Address and search using Google
Class    : Edit
Ctrl type: 50004
Ctrl name: edit
Value    : http://www.autoitscript.com/

HandlePropertyChangedEvent
Handle   : 0x00000000000D0506
Name     : Address and search using Google
Class    : Edit
Ctrl type: 50004
Ctrl name: edit
Value    : http://www.autoitscript.com/site/

HandlePropertyChangedEvent
Handle   : 0x00000000000D0506
Name     : Address and search using Google
Class    : Edit
Ctrl type: 50004
Ctrl name: edit
Value    : http://www.autoitscript.com/site/

HandlePropertyChangedEvent
Handle   : 0x00000000000C0432
Name     : Rich Text Window
Class    : RICHEDIT50W
Ctrl type: 50004
Ctrl name: edit
Value    :  
HandlePropertyChangedEvent
Handle   : 0x00000000000C0432
Name     : Rich Text Window
Class    : RICHEDIT50W
Ctrl type: 50004
Ctrl name: edit
Value    :  
HandlePropertyChangedEvent
Handle   : 0x00000000000C0432
Name     : Rich Text Window
Class    : RICHEDIT50W
Ctrl type: 50004
Ctrl name: edit
Value    :  
HandlePropertyChangedEvent
Handle   : 0x00000000000C0432
Name     : Rich Text Window
Class    : RICHEDIT50W
Ctrl type: 50004
Ctrl name: edit
Value    :  
HandlePropertyChangedEvent
Handle   : 0x00000000000C0432
Name     : Rich Text Window
Class    : RICHEDIT50W
Ctrl type: 50004
Ctrl name: edit
Value    :  
HandlePropertyChangedEvent
Handle   : 0x00000000000C0432
Name     : Rich Text Window
Class    : RICHEDIT50W
Ctrl type: 50004
Ctrl name: edit
Value    :  
HandlePropertyChangedEvent
Handle   : 0x00000000000C0432
Name     : Rich Text Window
Class    : RICHEDIT50W
Ctrl type: 50004
Ctrl name: edit
Value    : T 
HandlePropertyChangedEvent
Handle   : 0x00000000000C0432
Name     : Rich Text Window
Class    : RICHEDIT50W
Ctrl type: 50004
Ctrl name: edit
Value    : Te 
HandlePropertyChangedEvent
Handle   : 0x00000000000C0432
Name     : Rich Text Window
Class    : RICHEDIT50W
Ctrl type: 50004
Ctrl name: edit
Value    : Tes 
HandlePropertyChangedEvent
Handle   : 0x00000000000C0432
Name     : Rich Text Window
Class    : RICHEDIT50W
Ctrl type: 50004
Ctrl name: edit
Value    : Test

DLL files:

DLLfiles.7z

Share this post


Link to post
Share on other sites
junkew

@topten:

if IsObj($IE) Then

   $sText="search"; indexrelative:=1"    ;Is equal to

   $oButton=_UIA_getFirstObjectOfElement($IE,"name:=" & $sText, $treescope_subtree)

EndIf

above is functionally equal but technically differently implemented

_UIA_getFirstObjectOfElement will use the MS UIA library straightforward to find on a single property the first object and as such syntax with indexrelative will not work

The UIA_Wrappers add some functionality to walk thru all elements and add enhancements like

a. indexrelative, index, instance properties

b. match on regular expression

c. match easier on multiple properties combined (opposed to the complex logica of and/or conditions the MS UIA has)

 

Is there any method to calculate the x,y position of found element on the window or on the screen?

_UIA_getPropertyValue($oElement, $UIA_BoundingRectanglePropertyId)

 

 

@LarsJ: Regarding eventhandling I will post another example (without dll)

Edited by junkew

Share this post


Link to post
Share on other sites
trancexx

LarsJ, I have rewritten that event handling function (several times LOL), so your non-working example should look more like this:

#include "CUIAutomation2.au3"

Opt("MustDeclareVars", 1)

Global Const $S_OK = 0x00000000
Global Const $E_NOINTERFACE = 0x80004002
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"

Global $tIUIAutomationPropertyChangedEventHandler, $oIUIAutomationPropertyChangedEventHandler

Global $oUIAutomation

MainFunc()


Func MainFunc()

    $oIUIAutomationPropertyChangedEventHandler = ObjectFromTag("oIUIAutomationPropertyChangedEventHandler_", $dtagIUIAutomationPropertyChangedEventHandler, $tIUIAutomationPropertyChangedEventHandler)
    If Not IsObj($oIUIAutomationPropertyChangedEventHandler) Then Return

    $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
    If Not IsObj($oUIAutomation) Then Return

    Local $pUIElement
    $oUIAutomation.GetRootElement($pUIElement) ; Desktop
    If Not $pUIElement Then Return

    Local $tPropertyArray = DllStructCreate("int[2]")
    DllStructSetData($tPropertyArray, 1, $UIA_ValueValuePropertyId, 1)
    DllStructSetData($tPropertyArray, 1, $UIA_ToggleToggleStatePropertyId, 2)
    Local $x = $oUIAutomation.AddPropertyChangedEventHandlerNativeArray($pUIElement, $TreeScope_Descendants, 0, $oIUIAutomationPropertyChangedEventHandler, $tPropertyArray, 2)


    HotKeySet("{ESC}", "Quit")

    While Sleep(100)
    WEnd

EndFunc   ;==>MainFunc

Func Quit()
    Exit
EndFunc   ;==>Quit



Func _UIA_getPropertyValue($obj, $id)
    Local $vVal
    $obj.GetCurrentPropertyValue($id, $vVal)
    Return $vVal
EndFunc

Func oIUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent($pSelf, $pSender, $iPropertyId, $newValue) ; Ret: long  Par: ptr;int;variant
    ConsoleWrite(@CRLF & "oIUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent: $iPropertyId = " & $iPropertyId & ", $newValue = " & $newValue & @CRLF)

    Local $oSender = ObjCreateInterface($pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    $oSender.AddRef()

    ConsoleWrite("Handle    " & _UIA_getPropertyValue($oSender, $UIA_NativeWindowHandlePropertyId) & @CRLF)
    ConsoleWrite("Name      " & _UIA_getPropertyValue($oSender, $UIA_NamePropertyId) & @CRLF)
    ConsoleWrite("Class     " & _UIA_getPropertyValue($oSender, $UIA_ClassNamePropertyId) & @CRLF)
    ConsoleWrite("Ctrl type " & _UIA_getPropertyValue($oSender, $UIA_ControlTypePropertyId) & @CRLF)
    ConsoleWrite("Ctrl name " & _UIA_getPropertyValue($oSender, $UIA_LocalizedControlTypePropertyId) & @CRLF)
    ConsoleWrite("Value     " & _UIA_getPropertyValue($oSender, $UIA_LegacyIAccessibleValuePropertyId) & @CRLF)

    Return $S_OK
EndFunc

Func oIUIAutomationPropertyChangedEventHandler_QueryInterface($pSelf, $pRIID, $pObj) ; Ret: long  Par: ptr;ptr*
    Local $sIID = StringFromGUID($pRIID)
    If $sIID = $sIID_IUnknown Then
        ConsoleWrite("oIUIAutomationPropertyChangedEventHandler_QueryInterface: IUnknown" & @CRLF)
        DllStructSetData(DllStructCreate("ptr", $pObj), 1, $pSelf)
        Return $S_OK
    ElseIf $sIID = $sIID_IUIAutomationPropertyChangedEventHandler Then
        ConsoleWrite("oIUIAutomationPropertyChangedEventHandler_QueryInterface: IUIAutomationPropertyChangedEventHandler" & @CRLF)
        DllStructSetData(DllStructCreate("ptr", $pObj), 1, $pSelf)
        Return $S_OK
    Else
        ConsoleWrite("oIUIAutomationPropertyChangedEventHandler_QueryInterface: " & $sIID & @CRLF)
        Return $E_NOINTERFACE
    EndIf
EndFunc

Func oIUIAutomationPropertyChangedEventHandler_AddRef($pSelf) ; Ret: ulong
    ConsoleWrite("oIUIAutomationPropertyChangedEventHandler_AddRef" & @CRLF)
    Return 1
EndFunc

Func oIUIAutomationPropertyChangedEventHandler_Release($pSelf) ; Ret: ulong
    ConsoleWrite("oIUIAutomationPropertyChangedEventHandler_Release" & @CRLF)
    Return 1
EndFunc



Func StringFromGUID($pGUID)
    Local $aResult = DllCall("ole32.dll", "int", "StringFromGUID2", "struct*", $pGUID, "wstr", "", "int", 40)
    If @error Then Return SetError(@error, @extended, "")
    Return SetExtended($aResult[0], $aResult[2])
EndFunc



Func ObjectFromTag($sFunctionPrefix, $tagInterface, ByRef $tInterface, $bIsUnknown = Default, $sIID = "{00000000-0000-0000-C000-000000000046}") ; last param is IID_IUnknown by default
    If $bIsUnknown = Default Then $bIsUnknown = True
    Local $sInterface = $tagInterface ; copy interface description
    Local $tagIUnknown = "QueryInterface hresult(ptr;ptr*);" & _
            "AddRef dword();" & _
            "Release dword();"
    ; Adding IUnknown methods
    If $bIsUnknown Then $tagInterface = $tagIUnknown & $tagInterface
    ; Below line is really simple even though it looks super complex. It's just written weird to fit in one line, not to steal your attention
    Local $aMethods = StringSplit(StringReplace(StringReplace(StringReplace(StringReplace(StringTrimRight(StringReplace(StringRegExpReplace(StringRegExpReplace($tagInterface, "\w+\*", "ptr"), "\h*(\w+)\h*(\w+\*?)\h*(\((.*?)\))\h*(;|;*\z)", "$1\|$2;$4" & @LF), ";" & @LF, @LF), 1), "object", "idispatch"), "hresult", "long"), "bstr", "ptr"), "variant", "ptr"), @LF, 3)
    Local $iUbound = UBound($aMethods)
    Local $sMethod, $aSplit, $sNamePart, $aTagPart, $sTagPart, $sRet, $sParams, $hCallback
    ; Allocation
    $tInterface = DllStructCreate("int RefCount;int Size;ptr Object;ptr Methods[" & $iUbound & "];int_ptr Callbacks[" & $iUbound & "];ulong_ptr Slots[16]") ; 16 pointer sized elements more to create space for possible private props
    If @error Then Return SetError(1, 0, 0)
    For $i = 0 To $iUbound - 1
        $aSplit = StringSplit($aMethods[$i], "|", 2)
        If UBound($aSplit) <> 2 Then ReDim $aSplit[2]
        $sNamePart = $aSplit[0]
        $sTagPart = $aSplit[1]
        $sMethod = $sFunctionPrefix & $sNamePart
        $aTagPart = StringSplit($sTagPart, ";", 2)
        $sRet = $aTagPart[0]
        $sParams = StringReplace($sTagPart, $sRet, "", 1)
        $sParams = "ptr" & $sParams
        $hCallback = DllCallbackRegister($sMethod, $sRet, $sParams)
        ConsoleWrite(@error & @CRLF)
        DllStructSetData($tInterface, "Methods", DllCallbackGetPtr($hCallback), $i + 1) ; save callback pointer
        DllStructSetData($tInterface, "Callbacks", $hCallback, $i + 1) ; save callback handle
    Next
    DllStructSetData($tInterface, "RefCount", 1) ; initial ref count is 1
    DllStructSetData($tInterface, "Size", $iUbound) ; number of interface methods
    DllStructSetData($tInterface, "Object", DllStructGetPtr($tInterface, "Methods")) ; Interface method pointers
    Return ObjCreateInterface(DllStructGetPtr($tInterface, "Object"), $sIID, $sInterface, $bIsUnknown) ; pointer that's wrapped into object
EndFunc

Func DeleteObjectFromTag(ByRef $tInterface)
    For $i = 1 To DllStructGetData($tInterface, "Size")
        DllCallbackFree(DllStructGetData($tInterface, "Callbacks", $i))
    Next
    $tInterface = 0
EndFunc
...That's almost exact replica of c++ dll code you posted. Edited by trancexx
  • Like 1

♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites
junkew

>"C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.exe" /run /prod /ErrorStdOut /in "C:\Users\Elwin\Documents\UIA\ex12_propertyevent.au3" /UserParams    
+>19:12:27 Starting AutoIt3Wrapper v.2.1.0.33    Environment(Language:0413  Keyboard:00020409  OS:WIN_7/Service Pack 1  CPU:X64 OS:X64)
>Running AU3Check (3.3.10.2)  from:C:\Program Files (x86)\AutoIt3
+>19:12:27 AU3Check ended.rc:0
>Running:(3.3.10.2):C:\Program Files (x86)\AutoIt3\autoit3.exe "C:\Users\Elwin\Documents\UIA\ex12_propertyevent.au3"    
--> Press Ctrl+Alt+F5 to Restart or Ctrl+Break to Stop
0
0
0
0
oIUIAutomationPropertyChangedEventHandler_QueryInterface: IUnknown
oIUIAutomationPropertyChangedEventHandler_Release
oIUIAutomationPropertyChangedEventHandler_AddRef
oIUIAutomationPropertyChangedEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationPropertyChangedEventHandler_QueryInterface: {00000003-0000-0000-C000-000000000046}
oIUIAutomationPropertyChangedEventHandler_AddRef
oIUIAutomationPropertyChangedEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationPropertyChangedEventHandler_QueryInterface: IUnknown
oIUIAutomationPropertyChangedEventHandler_AddRef
oIUIAutomationPropertyChangedEventHandler_QueryInterface: {00000018-0000-0000-C000-000000000046}
oIUIAutomationPropertyChangedEventHandler_QueryInterface: {00000019-0000-0000-C000-000000000046}
oIUIAutomationPropertyChangedEventHandler_QueryInterface: {4C1E39E1-E3E3-4296-AA86-EC938D896E92}
oIUIAutomationPropertyChangedEventHandler_Release
oIUIAutomationPropertyChangedEventHandler_QueryInterface: IUIAutomationPropertyChangedEventHandler
oIUIAutomationPropertyChangedEventHandler_AddRef
oIUIAutomationPropertyChangedEventHandler_QueryInterface: {1C733A30-2A1C-11CE-ADE5-00AA0044773D}
oIUIAutomationPropertyChangedEventHandler_QueryInterface: IUIAutomationPropertyChangedEventHandler

oIUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent: $iPropertyId = 30045, $newValue = 0x001D0000
Handle    1901574
Name      Adres en zoeken met SecureSearch
Class     Edit
Ctrl type 50004
Ctrl name bewerken
Value     http://www.google.nl/ 

On the example of trancexx I only get one time output (and all I can do is press escape and program exits) 

steps i did

1. start example in Scite

2. open Internet Explorer and see above output coming

3. clicking around, closing IE and reopening gives no additonal output (not expected)

4. press escape and program exits (as expected)

 

Share this post


Link to post
Share on other sites
trancexx

Yes, I forgot to add reference for the sender inside event function.

But I have edited the post 18 minutes ago.


♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites
trancexx

So it works for you correctly now?

I also left consolewrite line inside ObjectFromTag() in case there are more types that aren't converted from COM types to callback type correctly because I'm not sure if all the types inside your UDF are covered. It should always print zeroes.

You had some invalid types last time I checked the UDF, but it was really long time ago.

Edited by trancexx

♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites
junkew

yes it works now correctly.

It still could be there are invalid types in the UDF but I think most are fixed (only complaints came from LarsJ since november)

  • none definitions changed to correct type
  • LarsJ identified this to be fixed in CUIAutomation2.au3 (lines 532 - 537 in the current version):
    "TreeScope hresult(long*);" & _
    "TreeScope hresult(long);" & _
    "TreeFilter hresult(ptr*);" & _
    "TreeFilter hresult(ptr);" & _
    "AutomationElementMode hresult(long*);" & _
    "AutomationElementMode hresult(long);"

    with these lines:
    "get_TreeScope hresult(long*);" & _
    "put_TreeScope hresult(long);" & _
    "get_TreeFilter hresult(ptr*);" & _
    "put_TreeFilter hresult(ptr);" & _
    "get_AutomationElementMode hresult(long*);" & _
    "put_AutomationElementMode hresult(long);"
     
  • and LarsJ gave now back some issue on SAFEARRAY currently its in the UDF probably an int and should be a struct*
    AddPropertyChangedEventHandlerNativeArray  is an example of where safearray was defined incorrectly

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • bobmcrae
      By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • jackchen
      By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object. Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?
×