Unable to activate popup window using ui automation

rmckay · November 9, 2019

Hello,

I'm trying to use ui automation to control a 3rd party program called Ninja Trader. It's a stock charting program. It creates child charts that can be modified individually. Each chart has a popup window to enable modifications to the respective chart. I've nearly completed the automation project I've been working on for months (includes attempting to understand the ui automation process). The last challenge is to activate the popup window. It's accessed from the respective chart by right clicking on the chart. I've tried the combination of each Element Properties (has/is info), Control Patterns (element actions), and Control Pattern Properties with all the Control Pattern Methods. Most of those combinations give an error. I assumed the combination of $UIA_IsLegacyIAccessiblePatternAvailablePropertyId True (LegacyIAccessiblePattern) and DoDefaultAction() would do something. It doesn't get flagged as an error but it also doesn't do anything. I've done the same drill with both "Menu: ContextMenu" and " Pane:ScrollIViewer" as the base for ObjCreateInterface. I've included screen shots of the UIASpy results and a screen shot of the window with popup. I assumed that I could modify code that worked with menus activated by right clicking on task bar icons. I've not been able to locate code (maybe recognize it) that addresses the problem. Any pointers as to what I'm missing will be greatly appreciated.

Thanks in advance.

junkew · November 9, 2019

Try it with simplespy as then you can see/catch the popup and identify the hierarchy. Another way could be to catch the events of newwindow. There are event examples in th iuiautomation thread that show how to do that.

rmckay · November 9, 2019

Thanks @junkew I've downloaded and printed out all of the examples from your forum page. I'll get started. Is the logic the same for popups as other UI elements in that you first identify it and the send it an action?

junkew · November 9, 2019

Yes but popups tend to have not allways intuitive parents and dissapear when you are activating other windows like spytooling.. Frequently a popup is child of desktop. That makes it sometimes harder to inspect the details of your popup.

Edited November 9, 2019 by junkew

rmckay · November 10, 2019

Thanks @junkew I’ve seen that problem when using Inspect, SimpleSpy, and UIASpy. The idea occurred to me that if I created an element from the Popup window and then changed the state (@SW_SHOW) the popup would appear. I can’t find any examples anywhere and none of the descriptions on the MS website about window state goes into any detail. Does that seem like a reasonable approach?

LarsJ · November 12, 2019

ContextMenus
This black window with the mouse hovering over Indicators... is not really a popup window. It's a context menu. Indicators... is a menu item. Windows commands are usually not used to manipulate context menus. You do not activate a context menu with eg. WinActivate or similar functions. The only action you take in connection with a context menu is to open the menu with a mouse right click. When the menu is open you can click a menu item to execute the corresponding function and the menu closes. The menu also closes as soon as it loses focus. Next time you need to use the context menu, you must open it again with a new mouse right click.

ContextMenus and UIASpy
Open UIASpy. Delete all top windows. Open Notepad. Right click in the Edit control to open the context menu. It should look like this:

Hover the mouse over the Undo menu point and press F4. It should look like this:

Switch to UIASpy. It closes the context menu. Because the menu is closed, all UI elements are red:

The menu item "Right to left Reading order" will be used to demonstrate point 3) below. This is UIASpy information:

Treeview Element                                    MenuItem: Right to left Reading order
                                                    
Element Properties (identification)                 
$UIA_AccessKeyPropertyId                            r
$UIA_AutomationIdPropertyId                         32768
$UIA_ControlTypePropertyId                          $UIA_MenuItemControlTypeId
$UIA_NamePropertyId                                 Right to left Reading order
                                                    
Element Properties (session unique)                 
$UIA_ProcessIdPropertyId                            4688
$UIA_RuntimeIdPropertyId                            42,722022,2,-2147483646,4688,360907857,10
                                                    
Element Properties (information)                    
$UIA_BoundingRectanglePropertyId                    l=551,t=593,w=258,h=22
$UIA_LocalizedControlTypePropertyId                 menu item
$UIA_ProviderDescriptionPropertyId                  [pid:4688,hwnd:0x0 Annotation:Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]
                                                    
Element Properties (has/is info)                    
$UIA_HasKeyboardFocusPropertyId                     True
$UIA_IsContentElementPropertyId                     True
$UIA_IsControlElementPropertyId                     True
$UIA_IsDataValidForFormPropertyId                   False
$UIA_IsEnabledPropertyId                            True
$UIA_IsKeyboardFocusablePropertyId                  False
$UIA_IsOffscreenPropertyId                          False
$UIA_IsPasswordPropertyId                           False
$UIA_IsRequiredForFormPropertyId                    False
                                                    
Control Patterns (element actions)                  
$UIA_IsInvokePatternAvailablePropertyId             True (InvokePattern)
$UIA_IsLegacyIAccessiblePatternAvailablePropertyId  True (LegacyIAccessiblePattern)
                                                    
Control Pattern Properties                          
$UIA_LegacyIAccessibleChildIdPropertyId             10
$UIA_LegacyIAccessibleDefaultActionPropertyId       Execute
$UIA_LegacyIAccessibleDescriptionPropertyId         
$UIA_LegacyIAccessibleHelpPropertyId                
$UIA_LegacyIAccessibleKeyboardShortcutPropertyId    r
$UIA_LegacyIAccessibleNamePropertyId                Right to left Reading order
$UIA_LegacyIAccessibleRolePropertyId                12 = $ROLE_SYSTEM_MENUITEM
$UIA_LegacyIAccessibleStatePropertyId               132 = $STATE_SYSTEM_FOCUSED+$STATE_SYSTEM_HOTTRACKED
$UIA_LegacyIAccessibleValuePropertyId               
                                                    
Control Pattern Methods                             
Invoke Pattern Methods                              
                                                    Invoke()
LegacyIAccessible Pattern Methods                   
                                                    DoDefaultAction()
                                                    Select(long)
                                                    SetValue(wstr)
                                                    GetIAccessible(idispatch*)
                                                    CurrentChildId(int*)
                                                    CurrentDefaultAction(bstr*)
                                                    CurrentDescription(bstr*)
                                                    CurrentHelp(bstr*)
                                                    CurrentKeyboardShortcut(bstr*)
                                                    CurrentName(bstr*)
                                                    CurrentRole(uint*)
                                                    CurrentState(uint*)
                                                    CurrentValue(bstr*)
                                                    GetCurrentSelection(ptr*)
                                                    
Parents from Desktop                                Pane: Desktop
                                                    Menu: Context
                                                    
Parent to child index

Note that the Invoke Pattern is supported so that the menu item can be invoked (Invoke()).

Your issues
As far as I understand from your first post, you want to automate the following three tasks:
1) Open the context menu
2) Detect the open context menu
3) Click Indicators... menu item
Point 3) also closes the menu.

Solutions
I'll demonstrate how the three tasks are solved with the context menu in Notepad. First try the Notepad examples and remember to correct language specific texts for your own language. Then you can customize the examples and try them out with your own program. All code is in the zip-file at the bottom.

2) Detect the open context menu
We'll start with task 2). The code in tst02.au3 is copied directly from the zip-file in this post. Run tst02.au3 in SciTE. It prints information in console. Right click in the Edit control in Notepad to open the context menu. Watch output in SciTE console. tst02.au3 keeps running until you stop it with Esc.

You can test tst02.au3 with your own program without changing any code at all. tst02.au3 should be able to identify your context menu.

tst02.au3:

#include "CUIAutomation2.au3"
#include "ObjectFromTag.au3"

Opt( "MustDeclareVars", 1 )

Global Const $S_OK = 0x00000000
Global Const $E_NOINTERFACE = 0x80004002
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"

Global $tIUIAutomationEventHandler, $oIUIAutomationEventHandler

Global $oUIAutomation

Example()



Func Example()
  $oIUIAutomationEventHandler = ObjectFromTag( "oIUIAutomationEventHandler_", $dtagIUIAutomationEventHandler, $tIUIAutomationEventHandler, True )
  If Not IsObj( $oIUIAutomationEventHandler ) Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pUIElement
  $oUIAutomation.GetRootElement( $pUIElement ) ; Desktop
  If Not $pUIElement Then Return

  ; Window open/close events
  ;If $oUIAutomation.AddAutomationEventHandler( $UIA_Window_WindowOpenedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit
  ;If $oUIAutomation.AddAutomationEventHandler( $UIA_Window_WindowClosedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit

  ; Menu open/close events
  If $oUIAutomation.AddAutomationEventHandler( $UIA_MenuOpenedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit
  ;If $oUIAutomation.AddAutomationEventHandler( $UIA_MenuClosedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit

  HotKeySet( "{ESC}", "Quit" )

  While Sleep(10)
  WEnd
EndFunc

Func Quit()
  $oIUIAutomationEventHandler = 0
  DeleteObjectFromTag( $tIUIAutomationEventHandler )
  Exit
EndFunc



Func _UIA_getPropertyValue( $obj, $id )
  Local $tVal
  $obj.GetCurrentPropertyValue( $id, $tVal )
  Return $tVal
EndFunc

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf, $pSender, $iEventId ) ; Ret: long  Par: ptr;int
  ConsoleWrite( "oIUIAutomationEventHandler_HandleAutomationEvent: " & $iEventId & @CRLF )
  If $iEventId <> $UIA_Window_WindowClosedEventId Then
    Local $oSender = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $oSender.AddRef()
    Local $hWindow = _UIA_getPropertyValue( $oSender, $UIA_NativeWindowHandlePropertyId )
    ConsoleWrite( "Title     = " & _UIA_getPropertyValue( $oSender, $UIA_NamePropertyId ) & @CRLF & _
                  "Class     = " & _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId ) & @CRLF & _
                  "Ctrl type = " & _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId ) & @CRLF & _
                  "Ctrl name = " & _UIA_getPropertyValue( $oSender, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                  "Value     = " & _UIA_getPropertyValue( $oSender, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                  "Handle    = " & Hex( $hWindow ) & @CRLF & @CRLF )

    ; Get UI Automation element from window handle
    Local $pWindow, $oWindow
    $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
    $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window error" & @CRLF )

    ; List all elements of window
    ListDescendants( $oWindow, 0, 0 )
  EndIf
  Return $S_OK
EndFunc

; List all child elements of parent
Func ListDescendants( $oParent, $iLevel, $iLevels = 0 )

  If Not IsObj( $oParent ) Then Return
  If $iLevels And $iLevel = $iLevels Then Return

  ; Create RawViewWalker object
  Local $pRawViewWalker, $oRawViewWalker
  $oUIAutomation.RawViewWalker( $pRawViewWalker )
  $oRawViewWalker = ObjCreateInterface( $pRawViewWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker )
  If Not IsObj( $oRawViewWalker ) Then Return ConsoleWrite( "RawViewWalker object error" & @CRLF )

  ; Get first child element
  Local $pUIElement, $oUIElement
  $oRawViewWalker.GetFirstChildElement( $oParent, $pUIElement )
  $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

  Local $sIndent = ""
  For $i = 0 To $iLevel - 1
    $sIndent &= "    "
  Next

  While IsObj( $oUIElement )
    ConsoleWrite( $sIndent & "Title     = " & _UIA_getPropertyValue( $oUIElement, $UIA_NamePropertyId ) & @CRLF & _
                  $sIndent & "Class     = " & _UIA_getPropertyValue( $oUIElement, $UIA_ClassNamePropertyId ) & @CRLF & _
                  $sIndent & "Ctrl type = " & _UIA_getPropertyValue( $oUIElement, $UIA_ControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Ctrl name = " & _UIA_getPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Value     = " & _UIA_getPropertyValue( $oUIElement, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                  $sIndent & "Handle    = " & Hex( _UIA_getPropertyValue( $oUIElement, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )

    ListDescendants( $oUIElement, $iLevel + 1, $iLevels )

    $oRawViewWalker.GetNextSiblingElement( $oUIElement, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  WEnd

EndFunc



Func oIUIAutomationEventHandler_QueryInterface( $pSelf, $pRIID, $pObj ) ; Ret: long  Par: ptr;ptr*
  Local $sIID = StringFromGUID( $pRIID )
  If $sIID = $sIID_IUnknown Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUnknown" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  ElseIf $sIID = $sIID_IUIAutomationEventHandler Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  Else
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: " & $sIID & @CRLF )
    Return $E_NOINTERFACE
  EndIf
EndFunc

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_AddRef" & @CRLF )
  Return 1
EndFunc

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_Release" & @CRLF )
  Return 1
EndFunc



Func StringFromGUID( $pGUID )
  Local $aResult = DllCall( "ole32.dll", "int", "StringFromGUID2", "struct*", $pGUID, "wstr", "", "int", 40 )
  If @error Then Return SetError( @error, @extended, "" )
  Return SetExtended( $aResult[0], $aResult[2] )
EndFunc

Console output:

Func oIUIAutomationEventHandler_QueryInterface( $pSelf ) ; Ret: long  Par: ptr;ptr
EndFunc
0

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: dword
EndFunc
0

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: dword
EndFunc
0

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf ) ; Ret: long  Par: ptr;int
EndFunc
0

oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {00000003-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {00000018-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {00000019-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {4C1E39E1-E3E3-4296-AA86-EC938D896E92}
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {1C733A30-2A1C-11CE-ADE5-00AA0044773D}
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_HandleAutomationEvent: 20003
Title     = Context
Class     = #32768
Ctrl type = 50009
Ctrl name = menu
Value     = 
Handle    = 000E039C

Title     = Undo
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = 
Class     = 
Ctrl type = 50038
Ctrl name = separator
Value     = 
Handle    = 00000000

Title     = Cut
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = Copy
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = Paste
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = Delete
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = 
Class     = 
Ctrl type = 50038
Ctrl name = separator
Value     = 
Handle    = 00000000

Title     = Select All
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = 
Class     = 
Ctrl type = 50038
Ctrl name = separator
Value     = 
Handle    = 00000000

Title     = Right to left Reading order
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = Show Unicode control characters
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = Insert Unicode control character
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = 
Class     = 
Ctrl type = 50038
Ctrl name = separator
Value     = 
Handle    = 00000000

Title     = Open IME
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = Reconversion
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

oIUIAutomationEventHandler_Release

1) Open the context menu
So far, there is no UI Automation method to open a context menu. You open a context menu by mimicking the manual method in programming: Set focus to window and control and perform a mouse right click.

Notepad must be open beforehand. Run tst01.au3 in SciTE. tst01.au3 opens the context menu in Notepad edit control when you click the buttons in the GUI. But no menu items are executed.

To customize tst01.au3 to your own program, the UI Automation code in particular needs to be corrected. You can also delete one of the buttons and correct the text on the other.

tst01.au3:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

Opt( "MustDeclareVars", 1 )

#include "UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
#include <GUIConstantsEx.au3>

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pNotepad, $oNotepad
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pNotepad )
  $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF )
  ConsoleWrite( "$oNotepad OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_EditControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pEdit1, $oEdit1
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition1, $pEdit1 )
  $oEdit1 = ObjCreateInterface( $pEdit1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oEdit1 ) Then Return ConsoleWrite( "$oEdit1 ERR" & @CRLF )
  ConsoleWrite( "$oEdit1 OK" & @CRLF )

  ; --- Create GUI ---

  Local $hGui = GUICreate( "Automate Notepad Context Menu", 330, 40, 100, 100 )
  Local $idRight = GUICtrlCreateButton( "Right to left Reading order",  10, 10, 150, 20 )
  Local $idLeft  = GUICtrlCreateButton( "Left to right Reading order", 170, 10, 150, 20 )
  GUICtrlSetState( $idLeft, $GUI_DISABLE )

  GUISetState( @SW_SHOW )

  Local $iMsg
  While 1
    $iMsg = GUIGetMsg()
    Switch $iMsg
      Case $idRight, $idLeft
        ; --- Element Properties (information) ---
        ConsoleWrite( "--- Element Properties (information) ---" & @CRLF )
        Local $asBoundingRectangle1
        $oEdit1.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $asBoundingRectangle1 )
        UIA_GetArrayPropertyValueAsString( $asBoundingRectangle1 )
        ConsoleWrite( "$asBoundingRectangle1 = " & $asBoundingRectangle1 & @CRLF )
        Local $aBoundingRectangle1 = StringSplit( $asBoundingRectangle1, ",", 2 ) ; 2 = $STR_NOCOUNT
        ; --- Secondary mouse click ---
        UIA_WinActivate( $oNotepad )
        Sleep( 100 )
        Local $aPos = MouseGetPos()
        DllCall( "user32.dll", "int", "ShowCursor", "bool", False )
        MouseClick( "secondary", $aBoundingRectangle1[0]+100, $aBoundingRectangle1[1]+100, 1, 0 )
        GUICtrlSetState( $iMsg = $idRight ? $idRight : $idLeft, $GUI_DISABLE )
        GUICtrlSetState( $iMsg = $idRight ? $idLeft : $idRight, $GUI_ENABLE )
        MouseMove( $aPos[0], $aPos[1], 0 )
        DllCall( "user32.dll", "int", "ShowCursor", "bool", True )

      Case $GUI_EVENT_CLOSE
        ExitLoop
    EndSwitch
  WEnd

  GUIDelete( $hGui )
EndFunc

Console output:

$oUIAutomation OK
$oDesktop OK
--- Find window/control ---
$pCondition0 OK
$oNotepad OK
--- Find window/control ---
$pCondition1 OK
$oEdit1 OK
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842

3) Click Indicators... menu item (your program)
3) Click "Right to left Reading order" menu item (Notepad)
Type a text eg. "Test" in Notepad. tst03.au3 is a combination of tst01.au3 and tst02.au3. And then code is added to execute a menu item in the context menu. It's the "Case $idContextMenu" section of the main loop. Run tst03.au3 in SciTE in the same way as tst01.au3.

To test your own program, make the same corrections in tst03.au3 as in tst01.au3. And then you have to correct the "Case $idContextMenu" section.

tst03.au3:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

Opt( "MustDeclareVars", 1 )

#include "UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
#include "ObjectFromTag.au3"
#include <GUIConstantsEx.au3>

Global Const $S_OK = 0x00000000
Global Const $E_NOINTERFACE = 0x80004002
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"

Global $oUIAutomation, $tIUIAutomationEventHandler, $oIUIAutomationEventHandler

Global $idContextMenu, $oContextMenu

Example()

Func Example()
  ; Create UI Automation object
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Create UI Automation event handler ---

  ConsoleWrite( "--- Create UI Automation event handler ---" & @CRLF )

  ; Create UI Automation event handler
  $oIUIAutomationEventHandler = ObjectFromTag( "oIUIAutomationEventHandler_", $dtag_IUIAutomationEventHandler, $tIUIAutomationEventHandler, True )
  If Not IsObj( $oIUIAutomationEventHandler ) Then Return ConsoleWrite( "$oIUIAutomationEventHandler ERR" & @CRLF )
  ConsoleWrite( "$oIUIAutomationEventHandler OK" & @CRLF )

  ; Menu open/close events
  If $oUIAutomation.AddAutomationEventHandler( $UIA_MenuOpenedEventId, $pDesktop, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Return ConsoleWrite( "AddAutomationEventHandler ERR" & @CRLF )
  ConsoleWrite( "AddAutomationEventHandler OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pNotepad, $oNotepad
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pNotepad )
  $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF )
  ConsoleWrite( "$oNotepad OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_EditControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pEdit1, $oEdit1
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition1, $pEdit1 )
  $oEdit1 = ObjCreateInterface( $pEdit1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oEdit1 ) Then Return ConsoleWrite( "$oEdit1 ERR" & @CRLF )
  ConsoleWrite( "$oEdit1 OK" & @CRLF )

  Local $hGui = GUICreate( "Automate Notepad Context Menu", 330, 40, 100, 100 )
  Local $idRight = GUICtrlCreateButton( "Right to left Reading order",  10, 10, 150, 20 )
  Local $idLeft  = GUICtrlCreateButton( "Left to right Reading order", 170, 10, 150, 20 )
  GUICtrlSetState( $idLeft, $GUI_DISABLE )
  $idContextMenu = GUICtrlCreateDummy()

  GUISetState( @SW_SHOW )

  Local $iMsg
  While 1
    $iMsg = GUIGetMsg()
    Switch $iMsg
      Case $idRight, $idLeft
        ; --- Element Properties (information) ---
        ConsoleWrite( "--- Element Properties (information) ---" & @CRLF )
        Local $asBoundingRectangle1
        $oEdit1.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $asBoundingRectangle1 )
        UIA_GetArrayPropertyValueAsString( $asBoundingRectangle1 )
        ConsoleWrite( "$asBoundingRectangle1 = " & $asBoundingRectangle1 & @CRLF )
        Local $aBoundingRectangle1 = StringSplit( $asBoundingRectangle1, ",", 2 ) ; 2 = $STR_NOCOUNT
        ; --- Secondary mouse click ---
        UIA_WinActivate( $oNotepad )
        Sleep( 100 )
        Local $aPos = MouseGetPos()
        DllCall( "user32.dll", "int", "ShowCursor", "bool", False )
        MouseClick( "secondary", $aBoundingRectangle1[0]+100, $aBoundingRectangle1[1]+100, 1, 0 )
        GUICtrlSetState( $iMsg = $idRight ? $idRight : $idLeft, $GUI_DISABLE )
        GUICtrlSetState( $iMsg = $idRight ? $idLeft : $idRight, $GUI_ENABLE )
        MouseMove( $aPos[0], $aPos[1], 0 )
        DllCall( "user32.dll", "int", "ShowCursor", "bool", True )

      Case $idContextMenu
        ConsoleWrite( "ContextMenu handler" & @CRLF )
        Local $pAndCondition1
        $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition0 )
        $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Right to left Reading order", $pCondition1 )
        $oUIAutomation.CreateAndCondition( $pCondition0, $pCondition1, $pAndCondition1 )
        If Not $pAndCondition1 Then Return ConsoleWrite( "$pAndCondition1 ERR" & @CRLF )
        ConsoleWrite( "$pAndCondition1 OK" & @CRLF )

        Local $pMenuItem1, $oMenuItem1
        $oContextMenu.FindFirst( $TreeScope_Descendants, $pAndCondition1, $pMenuItem1 )
        $oMenuItem1 = ObjCreateInterface( $pMenuItem1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
        If Not IsObj( $oMenuItem1 ) Then Return ConsoleWrite( "$oMenuItem1 ERR" & @CRLF )
        ConsoleWrite( "$oMenuItem1 OK" & @CRLF )

        Local $pInvokePattern1, $oInvokePattern1
        $oMenuItem1.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern1 )
        $oInvokePattern1 = ObjCreateInterface( $pInvokePattern1, $sIID_IUIAutomationInvokePattern, $dtag_IUIAutomationInvokePattern )
        If Not IsObj( $oInvokePattern1 ) Then Return ConsoleWrite( "$oInvokePattern1 ERR" & @CRLF )
        ConsoleWrite( "$oInvokePattern1 OK" & @CRLF )
        $oInvokePattern1.Invoke()
        ConsoleWrite( "$oInvokePattern1.Invoke()" & @CRLF )

      Case $GUI_EVENT_CLOSE
        ExitLoop
    EndSwitch
  WEnd

  GUIDelete( $hGui )
  $oIUIAutomationEventHandler = 0
  DeleteObjectFromTag( $tIUIAutomationEventHandler )
EndFunc

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf, $pSender, $iEventId ) ; Ret: long  Par: ptr;int
  ConsoleWrite( "oIUIAutomationEventHandler_HandleAutomationEvent: " & $iEventId & @CRLF )
  $oContextMenu = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  $oContextMenu.AddRef()
  If Not IsObj( $oContextMenu ) Then Return ConsoleWrite( "$oContextMenu ERR" & @CRLF )
  ConsoleWrite( "$oContextMenu OK" & @CRLF )
  GUICtrlSendToDummy( $idContextMenu )
  Return $S_OK
  #forceref $pSelf
EndFunc

Func oIUIAutomationEventHandler_QueryInterface( $pSelf, $pRIID, $pObj ) ; Ret: long  Par: ptr;ptr*
  Local $sIID = StringFromGUID( $pRIID )
  If $sIID = $sIID_IUnknown Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUnknown" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  ElseIf $sIID = $sIID_IUIAutomationEventHandler Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  Else
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: " & $sIID & @CRLF )
    Return $E_NOINTERFACE
  EndIf
  #forceref $pSelf
EndFunc

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_AddRef" & @CRLF )
  Return 1
  #forceref $pSelf
EndFunc

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_Release" & @CRLF )
  Return 1
  #forceref $pSelf
EndFunc

Func StringFromGUID( $pGUID )
  Local $aResult = DllCall( "ole32.dll", "int", "StringFromGUID2", "struct*", $pGUID, "wstr", "", "int", 40 )
  If @error Then Return SetError( @error, @extended, "" )
  Return SetExtended( $aResult[0], $aResult[2] )
EndFunc

Console output:

$oUIAutomation OK
$oDesktop OK
--- Create UI Automation event handler ---
Func oIUIAutomationEventHandler_QueryInterface( $pSelf ) ; Ret: long  Par: ptr;ptr
EndFunc
0

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: dword
EndFunc
0

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: dword
EndFunc
0

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf ) ; Ret: long  Par: ptr;int
EndFunc
0

oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_Release
$oIUIAutomationEventHandler OK
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {00000003-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_AddRef
AddAutomationEventHandler OK
--- Find window/control ---
$pCondition0 OK
$oNotepad OK
--- Find window/control ---
$pCondition1 OK
$oEdit1 OK
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
oIUIAutomationEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {00000018-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {00000019-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {4C1E39E1-E3E3-4296-AA86-EC938D896E92}
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {1C733A30-2A1C-11CE-ADE5-00AA0044773D}
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_HandleAutomationEvent: 20003
$oContextMenu OK
ContextMenu handler
$pAndCondition1 OK
$oMenuItem1 OK
$oInvokePattern1 OK
$oInvokePattern1.Invoke()
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
oIUIAutomationEventHandler_HandleAutomationEvent: 20003
$oContextMenu OK
ContextMenu handler
$pAndCondition1 OK
$oMenuItem1 OK
$oInvokePattern1 OK
$oInvokePattern1.Invoke()
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
oIUIAutomationEventHandler_HandleAutomationEvent: 20003
$oContextMenu OK
ContextMenu handler
$pAndCondition1 OK
$oMenuItem1 OK
$oInvokePattern1 OK
$oInvokePattern1.Invoke()
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
oIUIAutomationEventHandler_HandleAutomationEvent: 20003
$oContextMenu OK
ContextMenu handler
$pAndCondition1 OK
$oMenuItem1 OK
$oInvokePattern1 OK
$oInvokePattern1.Invoke()
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
oIUIAutomationEventHandler_HandleAutomationEvent: 20003
$oContextMenu OK
ContextMenu handler
$pAndCondition1 OK
$oMenuItem1 OK
$oInvokePattern1 OK
$oInvokePattern1.Invoke()
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
oIUIAutomationEventHandler_HandleAutomationEvent: 20003
$oContextMenu OK
ContextMenu handler
$pAndCondition1 OK
$oMenuItem1 OK
$oInvokePattern1 OK
$oInvokePattern1.Invoke()
oIUIAutomationEventHandler_Release

ContextMenu.7z

Edited November 12, 2019 by LarsJ

rmckay · November 12, 2019

Hi @LarsJ,

Can't tell you how much I appreciate this. I've been stumped by this for a couple of weeks. I contacted the developer and got a "We don't support this" response. I'll have this implemented tonight and definitely breath a sigh of relief.

Thanks,

Rod

Jonatas · November 13, 2019

Hello rmckay,
I don't know if I understand what you're trying to do, but it seems to me that you're trying to automate your brokerage operations,
if so did you know that there are specific bots for this purpose some offered by the Broker itself

Earthshine · November 13, 2019

@Jonatas please don’t add useless comments to threads.

rmckay · November 13, 2019

Hello @Jonatas

Quote

Hello rmckay,
I don't know if I understand what you're trying to do, but it seems to me that you're trying to automate your brokerage operations,
if so did you know that there are specific bots for this purpose some offered by the Broker itself

What I'm scripting is the shutdown of a stock charting program, reboot of computer, and restart charting program. Due to some glitch in one of the software packages (charting program or custom indicators) some of the charts do not get repopulated correctly. I've contacted both developers and neither of their solutions work. Reloading the price data solves the problem for some reason. Process can take up to 30 min. My workaround is to automate the shutdown, restart and reload of the price data. That's why the context menu needs to be accessed.

Thanks

Jonatas · November 14, 2019

@rmckay

I understand, I usually do some operations in the stock market and use a software called Rico - MetaTrader 5,
I have no problem with it, maybe it can be useful to you

@Earthshine

Sorry for not being as smart as you

rmckay · November 14, 2019

@Jonatas

Thanks for the heads up. I'll give it a look.

Fiinguer · September 10, 2020

On 11/11/2019 at 9:41 PM, LarsJ said:

ContextMenus
This black window with the mouse hovering over Indicators... is not really a popup window. It's a context menu. Indicators... is a menu item. Windows commands are usually not used to manipulate context menus. You do not activate a context menu with eg. WinActivate or similar functions. The only action you take in connection with a context menu is to open the menu with a mouse right click. When the menu is open you can click a menu item to execute the corresponding function and the menu closes. The menu also closes as soon as it loses focus. Next time you need to use the context menu, you must open it again with a new mouse right click.

ContextMenus and UIASpy
Open UIASpy. Delete all top windows. Open Notepad. Right click in the Edit control to open the context menu. It should look like this:

Hover the mouse over the Undo menu point and press F4. It should look like this:

Switch to UIASpy. It closes the context menu. Because the menu is closed, all UI elements are red:

The menu item "Right to left Reading order" will be used to demonstrate point 3) below. This is UIASpy information:

Treeview Element                                    MenuItem: Right to left Reading order
                                                    
Element Properties (identification)                 
$UIA_AccessKeyPropertyId                            r
$UIA_AutomationIdPropertyId                         32768
$UIA_ControlTypePropertyId                          $UIA_MenuItemControlTypeId
$UIA_NamePropertyId                                 Right to left Reading order
                                                    
Element Properties (session unique)                 
$UIA_ProcessIdPropertyId                            4688
$UIA_RuntimeIdPropertyId                            42,722022,2,-2147483646,4688,360907857,10
                                                    
Element Properties (information)                    
$UIA_BoundingRectanglePropertyId                    l=551,t=593,w=258,h=22
$UIA_LocalizedControlTypePropertyId                 menu item
$UIA_ProviderDescriptionPropertyId                  [pid:4688,hwnd:0x0 Annotation:Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]
                                                    
Element Properties (has/is info)                    
$UIA_HasKeyboardFocusPropertyId                     True
$UIA_IsContentElementPropertyId                     True
$UIA_IsControlElementPropertyId                     True
$UIA_IsDataValidForFormPropertyId                   False
$UIA_IsEnabledPropertyId                            True
$UIA_IsKeyboardFocusablePropertyId                  False
$UIA_IsOffscreenPropertyId                          False
$UIA_IsPasswordPropertyId                           False
$UIA_IsRequiredForFormPropertyId                    False
                                                    
Control Patterns (element actions)                  
$UIA_IsInvokePatternAvailablePropertyId             True (InvokePattern)
$UIA_IsLegacyIAccessiblePatternAvailablePropertyId  True (LegacyIAccessiblePattern)
                                                    
Control Pattern Properties                          
$UIA_LegacyIAccessibleChildIdPropertyId             10
$UIA_LegacyIAccessibleDefaultActionPropertyId       Execute
$UIA_LegacyIAccessibleDescriptionPropertyId         
$UIA_LegacyIAccessibleHelpPropertyId                
$UIA_LegacyIAccessibleKeyboardShortcutPropertyId    r
$UIA_LegacyIAccessibleNamePropertyId                Right to left Reading order
$UIA_LegacyIAccessibleRolePropertyId                12 = $ROLE_SYSTEM_MENUITEM
$UIA_LegacyIAccessibleStatePropertyId               132 = $STATE_SYSTEM_FOCUSED+$STATE_SYSTEM_HOTTRACKED
$UIA_LegacyIAccessibleValuePropertyId               
                                                    
Control Pattern Methods                             
Invoke Pattern Methods                              
                                                    Invoke()
LegacyIAccessible Pattern Methods                   
                                                    DoDefaultAction()
                                                    Select(long)
                                                    SetValue(wstr)
                                                    GetIAccessible(idispatch*)
                                                    CurrentChildId(int*)
                                                    CurrentDefaultAction(bstr*)
                                                    CurrentDescription(bstr*)
                                                    CurrentHelp(bstr*)
                                                    CurrentKeyboardShortcut(bstr*)
                                                    CurrentName(bstr*)
                                                    CurrentRole(uint*)
                                                    CurrentState(uint*)
                                                    CurrentValue(bstr*)
                                                    GetCurrentSelection(ptr*)
                                                    
Parents from Desktop                                Pane: Desktop
                                                    Menu: Context
                                                    
Parent to child index

Note that the Invoke Pattern is supported so that the menu item can be invoked (Invoke()).

Your issues
As far as I understand from your first post, you want to automate the following three tasks:
1) Open the context menu
2) Detect the open context menu
3) Click Indicators... menu item
Point 3) also closes the menu.

Solutions
I'll demonstrate how the three tasks are solved with the context menu in Notepad. First try the Notepad examples and remember to correct language specific texts for your own language. Then you can customize the examples and try them out with your own program. All code is in the zip-file at the bottom.

2) Detect the open context menu
We'll start with task 2). The code in tst02.au3 is copied directly from the zip-file in this post. Run tst02.au3 in SciTE. It prints information in console. Right click in the Edit control in Notepad to open the context menu. Watch output in SciTE console. tst02.au3 keeps running until you stop it with Esc.

You can test tst02.au3 with your own program without changing any code at all. tst02.au3 should be able to identify your context menu.

tst02.au3:

#include "CUIAutomation2.au3"
#include "ObjectFromTag.au3"

Opt( "MustDeclareVars", 1 )

Global Const $S_OK = 0x00000000
Global Const $E_NOINTERFACE = 0x80004002
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"

Global $tIUIAutomationEventHandler, $oIUIAutomationEventHandler

Global $oUIAutomation

Example()



Func Example()
  $oIUIAutomationEventHandler = ObjectFromTag( "oIUIAutomationEventHandler_", $dtagIUIAutomationEventHandler, $tIUIAutomationEventHandler, True )
  If Not IsObj( $oIUIAutomationEventHandler ) Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pUIElement
  $oUIAutomation.GetRootElement( $pUIElement ) ; Desktop
  If Not $pUIElement Then Return

  ; Window open/close events
  ;If $oUIAutomation.AddAutomationEventHandler( $UIA_Window_WindowOpenedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit
  ;If $oUIAutomation.AddAutomationEventHandler( $UIA_Window_WindowClosedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit

  ; Menu open/close events
  If $oUIAutomation.AddAutomationEventHandler( $UIA_MenuOpenedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit
  ;If $oUIAutomation.AddAutomationEventHandler( $UIA_MenuClosedEventId, $pUIElement, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Exit

  HotKeySet( "{ESC}", "Quit" )

  While Sleep(10)
  WEnd
EndFunc

Func Quit()
  $oIUIAutomationEventHandler = 0
  DeleteObjectFromTag( $tIUIAutomationEventHandler )
  Exit
EndFunc



Func _UIA_getPropertyValue( $obj, $id )
  Local $tVal
  $obj.GetCurrentPropertyValue( $id, $tVal )
  Return $tVal
EndFunc

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf, $pSender, $iEventId ) ; Ret: long  Par: ptr;int
  ConsoleWrite( "oIUIAutomationEventHandler_HandleAutomationEvent: " & $iEventId & @CRLF )
  If $iEventId <> $UIA_Window_WindowClosedEventId Then
    Local $oSender = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $oSender.AddRef()
    Local $hWindow = _UIA_getPropertyValue( $oSender, $UIA_NativeWindowHandlePropertyId )
    ConsoleWrite( "Title     = " & _UIA_getPropertyValue( $oSender, $UIA_NamePropertyId ) & @CRLF & _
                  "Class     = " & _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId ) & @CRLF & _
                  "Ctrl type = " & _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId ) & @CRLF & _
                  "Ctrl name = " & _UIA_getPropertyValue( $oSender, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                  "Value     = " & _UIA_getPropertyValue( $oSender, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                  "Handle    = " & Hex( $hWindow ) & @CRLF & @CRLF )

    ; Get UI Automation element from window handle
    Local $pWindow, $oWindow
    $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
    $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window error" & @CRLF )

    ; List all elements of window
    ListDescendants( $oWindow, 0, 0 )
  EndIf
  Return $S_OK
EndFunc

; List all child elements of parent
Func ListDescendants( $oParent, $iLevel, $iLevels = 0 )

  If Not IsObj( $oParent ) Then Return
  If $iLevels And $iLevel = $iLevels Then Return

  ; Create RawViewWalker object
  Local $pRawViewWalker, $oRawViewWalker
  $oUIAutomation.RawViewWalker( $pRawViewWalker )
  $oRawViewWalker = ObjCreateInterface( $pRawViewWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker )
  If Not IsObj( $oRawViewWalker ) Then Return ConsoleWrite( "RawViewWalker object error" & @CRLF )

  ; Get first child element
  Local $pUIElement, $oUIElement
  $oRawViewWalker.GetFirstChildElement( $oParent, $pUIElement )
  $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

  Local $sIndent = ""
  For $i = 0 To $iLevel - 1
    $sIndent &= "    "
  Next

  While IsObj( $oUIElement )
    ConsoleWrite( $sIndent & "Title     = " & _UIA_getPropertyValue( $oUIElement, $UIA_NamePropertyId ) & @CRLF & _
                  $sIndent & "Class     = " & _UIA_getPropertyValue( $oUIElement, $UIA_ClassNamePropertyId ) & @CRLF & _
                  $sIndent & "Ctrl type = " & _UIA_getPropertyValue( $oUIElement, $UIA_ControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Ctrl name = " & _UIA_getPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Value     = " & _UIA_getPropertyValue( $oUIElement, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                  $sIndent & "Handle    = " & Hex( _UIA_getPropertyValue( $oUIElement, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )

    ListDescendants( $oUIElement, $iLevel + 1, $iLevels )

    $oRawViewWalker.GetNextSiblingElement( $oUIElement, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  WEnd

EndFunc



Func oIUIAutomationEventHandler_QueryInterface( $pSelf, $pRIID, $pObj ) ; Ret: long  Par: ptr;ptr*
  Local $sIID = StringFromGUID( $pRIID )
  If $sIID = $sIID_IUnknown Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUnknown" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  ElseIf $sIID = $sIID_IUIAutomationEventHandler Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  Else
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: " & $sIID & @CRLF )
    Return $E_NOINTERFACE
  EndIf
EndFunc

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_AddRef" & @CRLF )
  Return 1
EndFunc

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_Release" & @CRLF )
  Return 1
EndFunc



Func StringFromGUID( $pGUID )
  Local $aResult = DllCall( "ole32.dll", "int", "StringFromGUID2", "struct*", $pGUID, "wstr", "", "int", 40 )
  If @error Then Return SetError( @error, @extended, "" )
  Return SetExtended( $aResult[0], $aResult[2] )
EndFunc

Console output:

Func oIUIAutomationEventHandler_QueryInterface( $pSelf ) ; Ret: long  Par: ptr;ptr
EndFunc
0

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: dword
EndFunc
0

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: dword
EndFunc
0

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf ) ; Ret: long  Par: ptr;int
EndFunc
0

oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {00000003-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {00000018-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {00000019-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {4C1E39E1-E3E3-4296-AA86-EC938D896E92}
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {1C733A30-2A1C-11CE-ADE5-00AA0044773D}
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_HandleAutomationEvent: 20003
Title     = Context
Class     = #32768
Ctrl type = 50009
Ctrl name = menu
Value     = 
Handle    = 000E039C

Title     = Undo
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = 
Class     = 
Ctrl type = 50038
Ctrl name = separator
Value     = 
Handle    = 00000000

Title     = Cut
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = Copy
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = Paste
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = Delete
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = 
Class     = 
Ctrl type = 50038
Ctrl name = separator
Value     = 
Handle    = 00000000

Title     = Select All
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = 
Class     = 
Ctrl type = 50038
Ctrl name = separator
Value     = 
Handle    = 00000000

Title     = Right to left Reading order
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = Show Unicode control characters
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = Insert Unicode control character
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = 
Class     = 
Ctrl type = 50038
Ctrl name = separator
Value     = 
Handle    = 00000000

Title     = Open IME
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

Title     = Reconversion
Class     = 
Ctrl type = 50011
Ctrl name = menu item
Value     = 
Handle    = 00000000

oIUIAutomationEventHandler_Release

1) Open the context menu
So far, there is no UI Automation method to open a context menu. You open a context menu by mimicking the manual method in programming: Set focus to window and control and perform a mouse right click.

Notepad must be open beforehand. Run tst01.au3 in SciTE. tst01.au3 opens the context menu in Notepad edit control when you click the buttons in the GUI. But no menu items are executed.

To customize tst01.au3 to your own program, the UI Automation code in particular needs to be corrected. You can also delete one of the buttons and correct the text on the other.

tst01.au3:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

Opt( "MustDeclareVars", 1 )

#include "UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
#include <GUIConstantsEx.au3>

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pNotepad, $oNotepad
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pNotepad )
  $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF )
  ConsoleWrite( "$oNotepad OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_EditControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pEdit1, $oEdit1
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition1, $pEdit1 )
  $oEdit1 = ObjCreateInterface( $pEdit1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oEdit1 ) Then Return ConsoleWrite( "$oEdit1 ERR" & @CRLF )
  ConsoleWrite( "$oEdit1 OK" & @CRLF )

  ; --- Create GUI ---

  Local $hGui = GUICreate( "Automate Notepad Context Menu", 330, 40, 100, 100 )
  Local $idRight = GUICtrlCreateButton( "Right to left Reading order",  10, 10, 150, 20 )
  Local $idLeft  = GUICtrlCreateButton( "Left to right Reading order", 170, 10, 150, 20 )
  GUICtrlSetState( $idLeft, $GUI_DISABLE )

  GUISetState( @SW_SHOW )

  Local $iMsg
  While 1
    $iMsg = GUIGetMsg()
    Switch $iMsg
      Case $idRight, $idLeft
        ; --- Element Properties (information) ---
        ConsoleWrite( "--- Element Properties (information) ---" & @CRLF )
        Local $asBoundingRectangle1
        $oEdit1.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $asBoundingRectangle1 )
        UIA_GetArrayPropertyValueAsString( $asBoundingRectangle1 )
        ConsoleWrite( "$asBoundingRectangle1 = " & $asBoundingRectangle1 & @CRLF )
        Local $aBoundingRectangle1 = StringSplit( $asBoundingRectangle1, ",", 2 ) ; 2 = $STR_NOCOUNT
        ; --- Secondary mouse click ---
        UIA_WinActivate( $oNotepad )
        Sleep( 100 )
        Local $aPos = MouseGetPos()
        DllCall( "user32.dll", "int", "ShowCursor", "bool", False )
        MouseClick( "secondary", $aBoundingRectangle1[0]+100, $aBoundingRectangle1[1]+100, 1, 0 )
        GUICtrlSetState( $iMsg = $idRight ? $idRight : $idLeft, $GUI_DISABLE )
        GUICtrlSetState( $iMsg = $idRight ? $idLeft : $idRight, $GUI_ENABLE )
        MouseMove( $aPos[0], $aPos[1], 0 )
        DllCall( "user32.dll", "int", "ShowCursor", "bool", True )

      Case $GUI_EVENT_CLOSE
        ExitLoop
    EndSwitch
  WEnd

  GUIDelete( $hGui )
EndFunc

Console output:

$oUIAutomation OK
$oDesktop OK
--- Find window/control ---
$pCondition0 OK
$oNotepad OK
--- Find window/control ---
$pCondition1 OK
$oEdit1 OK
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842

3) Click Indicators... menu item (your program)
3) Click "Right to left Reading order" menu item (Notepad)
Type a text eg. "Test" in Notepad. tst03.au3 is a combination of tst01.au3 and tst02.au3. And then code is added to execute a menu item in the context menu. It's the "Case $idContextMenu" section of the main loop. Run tst03.au3 in SciTE in the same way as tst01.au3.

To test your own program, make the same corrections in tst03.au3 as in tst01.au3. And then you have to correct the "Case $idContextMenu" section.

tst03.au3:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

Opt( "MustDeclareVars", 1 )

#include "UIA_Constants.au3" ; Can be copied from UIASpy Includes folder
#include "UIA_Functions.au3" ; Can be copied from UIASpy Includes folder
#include "ObjectFromTag.au3"
#include <GUIConstantsEx.au3>

Global Const $S_OK = 0x00000000
Global Const $E_NOINTERFACE = 0x80004002
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"

Global $oUIAutomation, $tIUIAutomationEventHandler, $oIUIAutomationEventHandler

Global $idContextMenu, $oContextMenu

Example()

Func Example()
  ; Create UI Automation object
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Create UI Automation event handler ---

  ConsoleWrite( "--- Create UI Automation event handler ---" & @CRLF )

  ; Create UI Automation event handler
  $oIUIAutomationEventHandler = ObjectFromTag( "oIUIAutomationEventHandler_", $dtag_IUIAutomationEventHandler, $tIUIAutomationEventHandler, True )
  If Not IsObj( $oIUIAutomationEventHandler ) Then Return ConsoleWrite( "$oIUIAutomationEventHandler ERR" & @CRLF )
  ConsoleWrite( "$oIUIAutomationEventHandler OK" & @CRLF )

  ; Menu open/close events
  If $oUIAutomation.AddAutomationEventHandler( $UIA_MenuOpenedEventId, $pDesktop, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Return ConsoleWrite( "AddAutomationEventHandler ERR" & @CRLF )
  ConsoleWrite( "AddAutomationEventHandler OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pNotepad, $oNotepad
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pNotepad )
  $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF )
  ConsoleWrite( "$oNotepad OK" & @CRLF )

  ; --- Find window/control ---

  ConsoleWrite( "--- Find window/control ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_EditControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pEdit1, $oEdit1
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition1, $pEdit1 )
  $oEdit1 = ObjCreateInterface( $pEdit1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  If Not IsObj( $oEdit1 ) Then Return ConsoleWrite( "$oEdit1 ERR" & @CRLF )
  ConsoleWrite( "$oEdit1 OK" & @CRLF )

  Local $hGui = GUICreate( "Automate Notepad Context Menu", 330, 40, 100, 100 )
  Local $idRight = GUICtrlCreateButton( "Right to left Reading order",  10, 10, 150, 20 )
  Local $idLeft  = GUICtrlCreateButton( "Left to right Reading order", 170, 10, 150, 20 )
  GUICtrlSetState( $idLeft, $GUI_DISABLE )
  $idContextMenu = GUICtrlCreateDummy()

  GUISetState( @SW_SHOW )

  Local $iMsg
  While 1
    $iMsg = GUIGetMsg()
    Switch $iMsg
      Case $idRight, $idLeft
        ; --- Element Properties (information) ---
        ConsoleWrite( "--- Element Properties (information) ---" & @CRLF )
        Local $asBoundingRectangle1
        $oEdit1.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $asBoundingRectangle1 )
        UIA_GetArrayPropertyValueAsString( $asBoundingRectangle1 )
        ConsoleWrite( "$asBoundingRectangle1 = " & $asBoundingRectangle1 & @CRLF )
        Local $aBoundingRectangle1 = StringSplit( $asBoundingRectangle1, ",", 2 ) ; 2 = $STR_NOCOUNT
        ; --- Secondary mouse click ---
        UIA_WinActivate( $oNotepad )
        Sleep( 100 )
        Local $aPos = MouseGetPos()
        DllCall( "user32.dll", "int", "ShowCursor", "bool", False )
        MouseClick( "secondary", $aBoundingRectangle1[0]+100, $aBoundingRectangle1[1]+100, 1, 0 )
        GUICtrlSetState( $iMsg = $idRight ? $idRight : $idLeft, $GUI_DISABLE )
        GUICtrlSetState( $iMsg = $idRight ? $idLeft : $idRight, $GUI_ENABLE )
        MouseMove( $aPos[0], $aPos[1], 0 )
        DllCall( "user32.dll", "int", "ShowCursor", "bool", True )

      Case $idContextMenu
        ConsoleWrite( "ContextMenu handler" & @CRLF )
        Local $pAndCondition1
        $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition0 )
        $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Right to left Reading order", $pCondition1 )
        $oUIAutomation.CreateAndCondition( $pCondition0, $pCondition1, $pAndCondition1 )
        If Not $pAndCondition1 Then Return ConsoleWrite( "$pAndCondition1 ERR" & @CRLF )
        ConsoleWrite( "$pAndCondition1 OK" & @CRLF )

        Local $pMenuItem1, $oMenuItem1
        $oContextMenu.FindFirst( $TreeScope_Descendants, $pAndCondition1, $pMenuItem1 )
        $oMenuItem1 = ObjCreateInterface( $pMenuItem1, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
        If Not IsObj( $oMenuItem1 ) Then Return ConsoleWrite( "$oMenuItem1 ERR" & @CRLF )
        ConsoleWrite( "$oMenuItem1 OK" & @CRLF )

        Local $pInvokePattern1, $oInvokePattern1
        $oMenuItem1.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern1 )
        $oInvokePattern1 = ObjCreateInterface( $pInvokePattern1, $sIID_IUIAutomationInvokePattern, $dtag_IUIAutomationInvokePattern )
        If Not IsObj( $oInvokePattern1 ) Then Return ConsoleWrite( "$oInvokePattern1 ERR" & @CRLF )
        ConsoleWrite( "$oInvokePattern1 OK" & @CRLF )
        $oInvokePattern1.Invoke()
        ConsoleWrite( "$oInvokePattern1.Invoke()" & @CRLF )

      Case $GUI_EVENT_CLOSE
        ExitLoop
    EndSwitch
  WEnd

  GUIDelete( $hGui )
  $oIUIAutomationEventHandler = 0
  DeleteObjectFromTag( $tIUIAutomationEventHandler )
EndFunc

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf, $pSender, $iEventId ) ; Ret: long  Par: ptr;int
  ConsoleWrite( "oIUIAutomationEventHandler_HandleAutomationEvent: " & $iEventId & @CRLF )
  $oContextMenu = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement )
  $oContextMenu.AddRef()
  If Not IsObj( $oContextMenu ) Then Return ConsoleWrite( "$oContextMenu ERR" & @CRLF )
  ConsoleWrite( "$oContextMenu OK" & @CRLF )
  GUICtrlSendToDummy( $idContextMenu )
  Return $S_OK
  #forceref $pSelf
EndFunc

Func oIUIAutomationEventHandler_QueryInterface( $pSelf, $pRIID, $pObj ) ; Ret: long  Par: ptr;ptr*
  Local $sIID = StringFromGUID( $pRIID )
  If $sIID = $sIID_IUnknown Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUnknown" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  ElseIf $sIID = $sIID_IUIAutomationEventHandler Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  Else
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: " & $sIID & @CRLF )
    Return $E_NOINTERFACE
  EndIf
  #forceref $pSelf
EndFunc

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_AddRef" & @CRLF )
  Return 1
  #forceref $pSelf
EndFunc

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_Release" & @CRLF )
  Return 1
  #forceref $pSelf
EndFunc

Func StringFromGUID( $pGUID )
  Local $aResult = DllCall( "ole32.dll", "int", "StringFromGUID2", "struct*", $pGUID, "wstr", "", "int", 40 )
  If @error Then Return SetError( @error, @extended, "" )
  Return SetExtended( $aResult[0], $aResult[2] )
EndFunc

Console output:

$oUIAutomation OK
$oDesktop OK
--- Create UI Automation event handler ---
Func oIUIAutomationEventHandler_QueryInterface( $pSelf ) ; Ret: long  Par: ptr;ptr
EndFunc
0

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: dword
EndFunc
0

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: dword
EndFunc
0

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf ) ; Ret: long  Par: ptr;int
EndFunc
0

oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_Release
$oIUIAutomationEventHandler OK
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {00000003-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_AddRef
AddAutomationEventHandler OK
--- Find window/control ---
$pCondition0 OK
$oNotepad OK
--- Find window/control ---
$pCondition1 OK
$oEdit1 OK
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
oIUIAutomationEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {00000018-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {00000019-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {4C1E39E1-E3E3-4296-AA86-EC938D896E92}
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {1C733A30-2A1C-11CE-ADE5-00AA0044773D}
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_HandleAutomationEvent: 20003
$oContextMenu OK
ContextMenu handler
$pAndCondition1 OK
$oMenuItem1 OK
$oInvokePattern1 OK
$oInvokePattern1.Invoke()
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
oIUIAutomationEventHandler_HandleAutomationEvent: 20003
$oContextMenu OK
ContextMenu handler
$pAndCondition1 OK
$oMenuItem1 OK
$oInvokePattern1 OK
$oInvokePattern1.Invoke()
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
oIUIAutomationEventHandler_HandleAutomationEvent: 20003
$oContextMenu OK
ContextMenu handler
$pAndCondition1 OK
$oMenuItem1 OK
$oInvokePattern1 OK
$oInvokePattern1.Invoke()
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
oIUIAutomationEventHandler_HandleAutomationEvent: 20003
$oContextMenu OK
ContextMenu handler
$pAndCondition1 OK
$oMenuItem1 OK
$oInvokePattern1 OK
$oInvokePattern1.Invoke()
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
oIUIAutomationEventHandler_HandleAutomationEvent: 20003
$oContextMenu OK
ContextMenu handler
$pAndCondition1 OK
$oMenuItem1 OK
$oInvokePattern1 OK
$oInvokePattern1.Invoke()
--- Element Properties (information) ---
$asBoundingRectangle1 = 258,100,984,842
oIUIAutomationEventHandler_HandleAutomationEvent: 20003
$oContextMenu OK
ContextMenu handler
$pAndCondition1 OK
$oMenuItem1 OK
$oInvokePattern1 OK
$oInvokePattern1.Invoke()
oIUIAutomationEventHandler_Release

ContextMenu.7z 19.15 kB · 16 downloads

You are a Legend!

I am trying to execute the script to select a submenu item but without the creating of a GUI.

Any help, please?

Thank you!!!

Edited September 14, 2020 by Fiinguer

Sign In

Advert

Unable to activate popup window using ui automation

Recommended Posts

rmckay

Advert

junkew

rmckay

junkew

rmckay

LarsJ

rmckay

Jonatas

Earthshine

rmckay

Jonatas

rmckay

Fiinguer

Create an account or sign in to comment

Create an account

Sign in

Similar Content

UI-SimpleWrappers UDF

Change text in text block

UIAutomation check if OBJECT exists

UI Automation Help

AutoIT & UI Automation

Browse

AutoIt Resources

Release

Beta