UIA Need to speed this code up

LarsJ · December 15, 2017

I've done some debugging myself. And I've been able to reproduce your console output. I can reproduce your console output in the specific situation where the "Open with ..." dialog box from a previous run isn't closed. When I run the program again with an already existing "Open with ..." dialog box, I get exactly your console output (last line is "$oIUIAutomationEventHandler OK"), the program is hanging, and now there are two "Open with ..." dialog boxes. When I close a dialog box, the program continues.

The "Set Associations" window is the owner of the "Open with ..." dialog box and the "Set Associations" window is non-responsive until the "Open with ..." dialog box is closed.

This means that if the program is run once more when an "Open with ..." dialog box already exists, it's absolutely necessary that a new "Set Associations" window is created with the ShellExecute command, and that the code in HandlePopupWindow function is working on this new window.

The problem is that ShellExecute seems to be reusing the existing "Set Associations" window instead of creating a new. Because "Set Associations" window is non-responsive (because of the already existing "Open with ..." dialog box) the program hangs.

I've been able to reproduce your console output in this specific situation. If there is a need to open multiple "Set Associations" windows and "Open with ..." dialog boxes at the same time, it must be taken into account in the code. The existing code can only handle a single "Set Associations" window and "Open with ..." dialog box at a time. Before the code is run another time at least the "Open with ..." dialog box must be closed.

If this situation is not the reason for your errors, you must do some debugging in the same way as I've done above.

careca · December 15, 2017

Results are random, i close all windows, and sometimes it works fine, others, doesn't.

Im gonna take a look to see if i can even understand the code.

Thanks

It's stopping here:

While Not $bShellFlyoutFound And Sleep(10)
  WEnd

Edited December 15, 2017 by careca

careca · December 16, 2017

Any idea why it's stopping? I find it very weird that it works everytime i tested when i comment

ControlClick($hWnd, "", "[CLASS:Button; INSTANCE:1]")

And click the button myself.

If i let the script click, fails 90% of the times, by fail, i mean it gets stuck on the loop i pointed in previous post.

$bShellFlyoutFound = False

Forever, unless i click to the side of the popup to allow it to close, and then click this button to bring it up again, then it follows its course.

Any idea?

Edited December 16, 2017 by careca

LarsJ · December 17, 2017

Because the code with the window open event handler didn't work I've changed strategy. This is straight forward UI Automation code (you need CUIAutomation2.au3):

#AutoIt3Wrapper_UseX64=y
#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6

#include "CUIAutomation2.au3" ; junkew

AutoItSetOption("MustDeclareVars", 1)

SetAssociations()


Func SetAssociations()
  Local $sText1, $Vis
  ShellExecute(@WindowsDir & '\system32\control.exe', '/name Microsoft.DefaultPrograms /page pageFileAssoc')
  WinWaitActive("Set Associations", 'Set Associations', 10)
  Local $hWnd = WinGetHandle("[CLASS:CabinetWClass]", "Set Associations")
  If $hWnd <> 0 Or @error <> 0 Then
    Do
      $Vis = ControlCommand($hWnd, "", "[CLASS:Button; INSTANCE:1]", 'IsVisible')
    Until $Vis = 1
    Sleep(100)
    Local $iCount = ControlListView($hWnd, "", "SysListView321", "GetItemCount")
    For $k = 0 To $iCount - 1
      $sText1 = ControlListView($hWnd, "", "SysListView321", "GetText", $k, 0)
      If $sText1 = '.mp3' Then
        ControlListView($hWnd, "", "SysListView321", "Select", $k, $k)
        ConsoleWrite( "SysListView321, Item " & $k & ": " & $sText1 & @CRLF & @CRLF )
        ControlClick($hWnd, "", "[CLASS:Button; INSTANCE:1]")
        HandlePopupWindow( 250 )
      EndIf
    Next
  EndIf
EndFunc

Func HandlePopupWindow( $iSleep )
  Sleep($iSleep) ; Wait for Shell_Flyout window to open

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Desktop
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; Condition to find Shell_Flyout window
  Local $pCondition, $pCondition1, $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_WindowControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Shell_Flyout", $pCondition2 )
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  ; Find Shell_Flyout window
  Local $pWindow, $oWindow
  $oDesktop.FindFirst( $TreeScope_Children, $pCondition, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR (close existing ""Open with..."" window or increase $iSleep)" & @CRLF )
  ConsoleWrite( "$oWindow OK" & @CRLF )

  ; --- More apps ---

  ConsoleWrite( "--- More apps ---" & @CRLF )

  ; Condition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  ; Find link
  Local $pLink, $oLink
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pLink )
  $oLink = ObjCreateInterface( $pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oLink ) Then Return ConsoleWrite( "$oLink ERR" & @CRLF )
  ConsoleWrite( "$oLink OK" & @CRLF )

  ; Click (invoke) link
  Local $pInvoke, $oInvoke
  $oLink.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()

  ; --- Look for another app on this PC ---

  ConsoleWrite( "--- Look for another app on this PC ---" & @CRLF )

  ; Condition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  ; Find link
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pLink )
  $oLink = ObjCreateInterface( $pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oLink ) Then Return ConsoleWrite( "$oLink ERR" & @CRLF )
  ConsoleWrite( "$oLink OK" & @CRLF )

  ; Click (invoke) link
  $oLink.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()
EndFunc

I will return to the COM event handler solution later today. It turned out to be quite interesting (or what to call it).

careca · December 17, 2017

Hi, thanks for the help, this is perfect.

There's no winwaitactive for the popup, is there? That way we could bypass the sleep.

LarsJ · December 17, 2017

I agree. The Sleep command is annoying. That's why I preferred the solution with the COM event handler. But I think I've found out to combine the two solutions without a Sleep. I'll post new code in a little while.

LarsJ · December 17, 2017

This post has nothing to do with the interesting things I wrote about at the bottom of post #44. I'll still return to this in another post.

Comments on the code in post #34.
The code in this post implements a custom COM event handler to detect window open events. Because UI Automation code is based on COM interfaces and the objects are created with ObjCreateInterface, such a custom event handler must be created with ObjectFromTag function. A large part of the code is about implementing this event handler.

Because the event handler detects window open events and nothing else (it doesn't detect if the window is already open) it's crucial that the event handler starts up and is ready to listen to window open events before the Shell_Flyout window opens.

If the Shell_Flyout window opens before the event handler is ready, the event handler will wait for the open event in the loop. Because the Shell_Flyout window is already open it'll wait forever. Unless you open the Shell_Flyout window manually so that it can detect the window open event. The last sentence is an explanation to post #43.

On my PC the event handler always starts up before the Shell_Flyout window opens. And the code works. On careca's PC it seems to be the opposite most of the time. The Shell_Flyout window opens before the event handler is ready. The event handler is unable to detect the window open event and hangs in the loop. The code doesn't work.

Comments on the code in post #44.
This is standard UI Automation code. Very often you need a Sleep to wait for a window to open. This is also the case here.

The first time you run the code eg. after a reboot the code always run slower. The next time when files are stored in the disk cache (maybe also because of other reasons) the code runs faster.

To make sure that the code works on my PC the first time after a reboot I need to set the Sleep time as high as 250 ms. But the next time I run the code I can see the Shell_Flyout window flashing because of the long Sleep. That's pretty annoying.

New code.
The new code is a combination of the code in post #34 and #44. There is no Sleep. The code works beautifully on my PC. Hopefully it also works on careca's PC.

Now there are two purposes of the loop:

If the COM event handler starts up before the Shell_Flyout window the loop waits for the window to open and the event handler can detect the open event.
If the Shell_Flyout window opens before the COM event handler is ready the event handler is unable to detect the window open event. In this case the (already open) Shell_Flyout window is detected in the While loop.

#AutoIt3Wrapper_UseX64=y
#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6

#include "CUIAutomation2.au3" ; junkew
#include "ObjectFromTag.au3"  ; trancexx

AutoItSetOption("MustDeclareVars", 1)

Global Const $S_OK = 0x00000000
Global Const $E_NOINTERFACE = 0x80004002
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"

Global $bShellFlyoutFound = False

SetAssociations()


Func SetAssociations()
  Local $sText1, $Vis
  ShellExecute(@WindowsDir & '\system32\control.exe', '/name Microsoft.DefaultPrograms /page pageFileAssoc')
  WinWaitActive("Set Associations", 'Set Associations', 10)
  Local $hWnd = WinGetHandle("[CLASS:CabinetWClass]", "Set Associations")
  If $hWnd <> 0 Or @error <> 0 Then
    Do
      $Vis = ControlCommand($hWnd, "", "[CLASS:Button; INSTANCE:1]", 'IsVisible')
    Until $Vis = 1
    Sleep(100)
    Local $iCount = ControlListView($hWnd, "", "SysListView321", "GetItemCount")
    For $k = 0 To $iCount - 1
      $sText1 = ControlListView($hWnd, "", "SysListView321", "GetText", $k, 0)
      If $sText1 = '.mp3' Then
        ControlListView($hWnd, "", "SysListView321", "Select", $k, $k)
        ConsoleWrite( "SysListView321, Item " & $k & ": " & $sText1 & @CRLF & @CRLF )
        ControlClick($hWnd, "", "[CLASS:Button; INSTANCE:1]")
        HandlePopupWindow()
      EndIf
    Next
  EndIf
EndFunc

Func HandlePopupWindow()
  ; Create custom UI Automation event handler
  Local $tIUIAutomationEventHandler, $oIUIAutomationEventHandler
  $oIUIAutomationEventHandler = ObjectFromTag( "oIUIAutomationEventHandler_", $dtagIUIAutomationEventHandler, $tIUIAutomationEventHandler, True )
  If Not IsObj( $oIUIAutomationEventHandler ) Then Return ConsoleWrite( "$oIUIAutomationEventHandler ERR" & @CRLF )
  ConsoleWrite( "$oIUIAutomationEventHandler OK" & @CRLF )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Desktop
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; Window open events
  ; This COM event handler detects the Shell_Flyout window open event and clicks the links in the window
  If $oUIAutomation.AddAutomationEventHandler( $UIA_Window_WindowOpenedEventId, $pDesktop, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Return ConsoleWrite( "$pIUIAutomationEventHandler ERR" & @CRLF )
  ConsoleWrite( "$oIUIAutomationEventHandler OK" & @CRLF )

  ; There are two purposes of the While loop below:
  ; 1) If the COM event handler starts up before the Shell_Flyout window the loop waits for the window.
  ; 2) If the Shell_Flyout window opens before the COM event handler is ready the event handler is unable
  ;    to detect the window open event and therefore unable to detect the Shell_Flyout window. In this case
  ;    the Shell_Flyout window is detected in the while loop.
  Local $pCondition = 0, $pCondition1, $pCondition2, $i
  While Not $bShellFlyoutFound And Sleep(10)

    ; Do this only once
    If Not $pCondition Then
      ; Condition to find Shell_Flyout window
      $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_WindowControlTypeId, $pCondition1 )
      $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Shell_Flyout", $pCondition2 )
      $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
      $i = $pCondition ? ConsoleWrite( "$pCondition OK" & @CRLF ) : ConsoleWrite( "$pCondition ERR" & @CRLF )
      #forceref $i
    EndIf

    ; Find Shell_Flyout window
    Local $pWindow, $oWindow
    $oDesktop.FindFirst( $TreeScope_Children, $pCondition, $pWindow )
    $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $i = IsObj( $oWindow ) ? ConsoleWrite( "$oWindow OK" & @CRLF ) _
                           : ConsoleWrite( "$oWindow ERR (Shell_Flyout window is not open)" & @CRLF )

    If IsObj( $oWindow ) Then
      ; --- More apps ---

      ConsoleWrite( "--- More apps ---" & @CRLF )

      ; Condition
      $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition )
      If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
      ConsoleWrite( "$pCondition OK" & @CRLF )

      ; Find link
      Local $pLink, $oLink
      $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pLink )
      $oLink = ObjCreateInterface( $pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
      If Not IsObj( $oLink ) Then Return ConsoleWrite( "$oLink ERR" & @CRLF )
      ConsoleWrite( "$oLink OK" & @CRLF )

      ; Click (invoke) link
      Local $pInvoke, $oInvoke
      $oLink.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
      $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
      If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
      ConsoleWrite( "$oInvoke OK" & @CRLF )
      $oInvoke.Invoke()

      ; --- Look for another app on this PC ---

      ConsoleWrite( "--- Look for another app on this PC ---" & @CRLF )

      ; Condition
      $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition )
      If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
      ConsoleWrite( "$pCondition OK" & @CRLF )

      ; Find link
      $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pLink )
      $oLink = ObjCreateInterface( $pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
      If Not IsObj( $oLink ) Then Return ConsoleWrite( "$oLink ERR" & @CRLF )
      ConsoleWrite( "$oLink OK" & @CRLF )

      ; Click (invoke) link
      $oLink.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
      $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
      If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
      ConsoleWrite( "$oInvoke OK" & @CRLF )
      $oInvoke.Invoke()

      $bShellFlyoutFound = True
    EndIf
  WEnd

  $oUIAutomation.RemoveAutomationEventHandler( $UIA_Window_WindowOpenedEventId, $pDesktop, $oIUIAutomationEventHandler )
EndFunc

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf, $pSender, $iEventId ) ; Ret: long  Par: ptr;int
  ConsoleWrite( "oIUIAutomationEventHandler_HandleAutomationEvent: " & $iEventId & @CRLF )

  If $iEventId = $UIA_Window_WindowOpenedEventId Then
    Local $oSender = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $oSender.AddRef()
   
    ConsoleWrite( "Handle    " & Ptr( _UIA_getPropertyValue( $oSender, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF )
    ConsoleWrite( "Name      " & _UIA_getPropertyValue( $oSender, $UIA_NamePropertyId ) & @CRLF )
    ConsoleWrite( "Class     " & _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId ) & @CRLF )
    ConsoleWrite( "Ctrl type " & _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId ) & @CRLF )
    ConsoleWrite( "Ctrl name " & _UIA_getPropertyValue( $oSender, $UIA_LocalizedControlTypePropertyId ) & @CRLF )
    ConsoleWrite( "Value     " & _UIA_getPropertyValue( $oSender, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF )

    ; Shell_Flyout popup?
    Local $sClassName, $hWindow
    $oSender.GetCurrentPropertyValue( $UIA_ClassNamePropertyId, $sClassName )
    If $sClassName = "Shell_Flyout" Then
      $oSender.GetCurrentPropertyValue( $UIA_NativeWindowHandlePropertyId, $hWindow )

      ; Create UI Automation object
      Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
      If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
      ConsoleWrite( "$oUIAutomation OK" & @CRLF )

      ; Get UI Automation element from window handle
      Local $pWindow, $oWindow
      $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
      $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
      If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
      ConsoleWrite( "$oWindow OK" & @CRLF )

      ; --- More apps ---

      ConsoleWrite( "--- More apps ---" & @CRLF )

      ; Condition
      Local $pCondition
      $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition )
      If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
      ConsoleWrite( "$pCondition OK" & @CRLF )

      ; Find link
      Local $pLink, $oLink
      $oSender.FindFirst( $TreeScope_Descendants, $pCondition, $pLink )
      $oLink = ObjCreateInterface( $pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
      If Not IsObj( $oLink ) Then Return ConsoleWrite( "$oLink ERR" & @CRLF )
      ConsoleWrite( "$oLink OK" & @CRLF )

      ; Click (invoke) link
      Local $pInvoke, $oInvoke
      $oLink.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
      $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
      If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
      ConsoleWrite( "$oInvoke OK" & @CRLF )
      $oInvoke.Invoke()

      ; --- Look for another app on this PC ---

      ConsoleWrite( "--- Look for another app on this PC ---" & @CRLF )

      ; Condition
      $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition )
      If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
      ConsoleWrite( "$pCondition OK" & @CRLF )

      ; Find link
      $oSender.FindFirst( $TreeScope_Descendants, $pCondition, $pLink )
      $oLink = ObjCreateInterface( $pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
      If Not IsObj( $oLink ) Then Return ConsoleWrite( "$oLink ERR" & @CRLF )
      ConsoleWrite( "$oLink OK" & @CRLF )

      ; Click (invoke) link
      $oLink.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
      $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
      If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
      ConsoleWrite( "$oInvoke OK" & @CRLF )
      $oInvoke.Invoke()

      $bShellFlyoutFound = True
    EndIf
  EndIf

  Return $S_OK
  #forceref $pSelf
EndFunc

Func _UIA_getPropertyValue( $obj, $id )
  Local $vVal
  $obj.GetCurrentPropertyValue( $id, $vVal )
  Return $vVal
EndFunc

Func oIUIAutomationEventHandler_QueryInterface( $pSelf, $pRIID, $pObj ) ; Ret: long  Par: ptr;ptr*
  Local $sIID = StringFromGUID( $pRIID )
  If $sIID = $sIID_IUnknown Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUnknown" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  ElseIf $sIID = $sIID_IUIAutomationEventHandler Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  Else
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: " & $sIID & @CRLF )
    Return $E_NOINTERFACE
  EndIf
  #forceref $pSelf
EndFunc

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_AddRef" & @CRLF )
  Return 1
  #forceref $pSelf
EndFunc

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_Release" & @CRLF )
  Return 1
  #forceref $pSelf
EndFunc

Func StringFromGUID( $pGUID )
  Local $aResult = DllCall( "ole32.dll", "int", "StringFromGUID2", "struct*", $pGUID, "wstr", "", "int", 40 )
  If @error Then Return SetError( @error, @extended, "" )
  Return SetExtended( $aResult[0], $aResult[2] )
EndFunc

You must close the "Open with ..." dialog box after each run of the code. But you can leave the "Set Associations" window open. This is a little bit faster.

AllCode2.7z

Edited December 17, 2017 by LarsJ

junkew · December 17, 2017

Nice example and shows how speedy things can be with UIA. Just a little harder to make the same speed in the UIAwrappers.

Struggled a lot initially as I was running from ISN Auto It studio (application just dissapears after a $sText1 = ControlListView($hWnd, "", "SysListView321", "GetText", $k, 0) )

Running from SCITE went fine after changing some parts to dutch

careca · December 17, 2017

This also works perfect! Many thanks.

Is there any benefit from using a method over the other? The one with the sleep has considerably less code..

junkew · December 17, 2017

Speed, stability, effort in coding, maintenance, understandability, time of development,...... are all kinds of factors you could consider.

I would have been happy with the 30 seconds initial solution as long as its not on the critical path in your team/project.

If you really want speed you would follow the registry approach. So basically it depends on your requirements.

Its now to fast that I am not even sure if in between the UI Windows are there (should make a screenshot after each window appears)

This was a nice excercise in all possibilities of UIA and will add these ones to the examples of uiawrappers.

careca · December 18, 2017

I was looking at the code from the "first" post, the one with the sleep.

And i can't figure it out, how would i go about clicking an item from the list, that's not the more apps hyperlink?

ConsoleWrite( "--- More apps ---" & @CRLF )

  ; Condition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  ; Find link
  Local $pLink, $oLink
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pLink )
  $oLink = ObjCreateInterface( $pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oLink ) Then Return ConsoleWrite( "$oLink ERR" & @CRLF )
  ConsoleWrite( "$oLink OK" & @CRLF )

  ; Click (invoke) link
  Local $pInvoke, $oInvoke
  $oLink.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()

Doesn't seem to be anything identifying the item, i was expecting a label or something, but it's all variables, how does this work?

junkew · December 18, 2017

Item selecting and clicking is this part of coding

If $sText1 = '.mp3' Then
        ControlListView($hWnd, "", "SysListView321", "Select", $k, $k)
        ConsoleWrite( "SysListView321, Item " & $k & ": " & $sText1 & @CRLF & @CRLF )
        ControlClick($hWnd, "", "[CLASS:Button; INSTANCE:1]")
        HandlePopupWindow()
      EndIf

@LarsJ made this a language independent as possible so instead of selection / condition on $UIA_NamePropertyID he was using the class hyperlink for more apps ... and as that's the only / first one it will match. Besides findFirst you have findAll and then you get multiple other ones. To understand this logic my advice is try to understand the UIAutomation thread the main first 10-20 examples explaining findfirst, findall, treewalkers.

The whole documentation on uia you can find over here with Microsoft https://msdn.microsoft.com/en-us/library/windows/desktop/ee684021(v=vs.85).aspx

UIA Need to speed this code up

Recommended Posts

Link to comment

Share on other sites

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Danyfirex

junkew

LarsJ

IUIAutomation - Topic with framework and examples

Link to comment

Share on other sites

IUIAutomation - Topic with framework and examples

Link to comment

Share on other sites

Link to comment

Share on other sites

IUIAutomation - Topic with framework and examples

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

IUIAutomation - Topic with framework and examples

Link to comment

Share on other sites

Link to comment

Share on other sites

IUIAutomation - Topic with framework and examples

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members