UIA Invoke click for a UIA_TextControlTypeId - Is that possible?

[souldjer777]

Good Morning AutoIT Gurus

How is everyone this fine Tuesday? 

I have been using SimpleSpy and inspect.exe to get data from various program windows - but this one, I'm not sure about since it's a button - but it's also text.

Could I treat UIA_TextControlTypeID (TextBlock) as a button and click on it somehow? -  If so, how would I do that? FYI - I have successfully been using code examples from LarsJ and Junkew in the past.

The reason I have to treat that text as a button is that the button itself has no data. (At least we have an element [][button]) and the text within the button does... (At least we have an element [Run Threat Analysis Scan][TextBlock])

 SimpleSpy info:

Mouse position is retrieved 486-377

At least we have an element [Run Threat Analysis Scan][TextBlock]

 

Having the following values for all properties: 

Title is: <Run Threat Analysis Scan> Class   := <TextBlock> controltype:= <UIA_TextControlTypeId> ,<50020> , (0000C364) 397;367;154;16

*** Parent Information top down ***

7: Title is: <Desktop> Class   := <#32769> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) 0;0;1920;1080

"Title:=Desktop;controltype:=UIA_PaneControlTypeId;class:=#32769"

6: Title is: <Symantec Help v2.1> Class   := <Window> controltype:= <UIA_WindowControlTypeId> ,<50032> , (0000C370) 322;65;1024;680

"Title:=Symantec Help v2.1;controltype:=UIA_WindowControlTypeId;class:=Window"

5: Title is: <> Class   := <SharedWindow> controltype:= <UIA_CustomControlTypeId> ,<50025> , (0000C369) 330;143;1008;572

"Title:=;controltype:=UIA_CustomControlTypeId;class:=SharedWindow"

4: Title is: <> Class   := <Home2> controltype:= <UIA_CustomControlTypeId> ,<50025> , (0000C369) 342;193;996;522

"Title:=;controltype:=UIA_CustomControlTypeId;class:=Home2"

3: Title is: <> Class   := <ScrollViewer> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) 342;193;996;522

"Title:=;controltype:=UIA_PaneControlTypeId;class:=ScrollViewer"

2: Title is: <> Class   := <Ui2Widget> controltype:= <UIA_CustomControlTypeId> ,<50025> , (0000C369) 357;250;263;169

"Title:=;controltype:=UIA_CustomControlTypeId;class:=Ui2Widget"

1: Title is: <> Class   := <UiActionButton> controltype:= <UIA_CustomControlTypeId> ,<50025> , (0000C369) 357;355;263;40

"Title:=;controltype:=UIA_CustomControlTypeId;class:=UiActionButton"

0: Title is: <> Class   := <Button> controltype:= <UIA_ButtonControlTypeId> ,<50000> , (0000C350) 357;355;263;40

"Title:=;controltype:=UIA_ButtonControlTypeId;class:=Button"

 

#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP6=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Symantec Help v2.1;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children)  
_UIA_Action($oP6,"setfocus")
Local $oP5=_UIA_getObjectByFindAll($oP6, "Title:=;controltype:=UIA_CustomControlTypeId;class:=SharedWindow", $treescope_children)   
Local $oP4=_UIA_getObjectByFindAll($oP5, "Title:=;controltype:=UIA_CustomControlTypeId;class:=Home2", $treescope_children)  
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=ScrollViewer", $treescope_children) 
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_CustomControlTypeId;class:=Ui2Widget", $treescope_children)  
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_CustomControlTypeId;class:=UiActionButton", $treescope_children) 
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_ButtonControlTypeId;class:=Button", $treescope_children) 
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("RunThreatAnalysisScan.mainwindow", "title:=Run Threat Analysis Scan;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Run Threat Analysis Scan;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"click")

This fails as well...

Window handle OK

UI Automation object OK

Automation element from window OK

Run Threat Analysis Scancondition OK

Find button ERR

 

expandcollapse popup

Local $hWindow = WinGetHandle("Symantec Help v2.1")
    If Not $hWindow Then Return ConsoleWrite("Window handle ERR" & @CRLF)
    ConsoleWrite("Window handle OK" & @CRLF)

    ; Create UI Automation object
    Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
    If Not IsObj($oUIAutomation) Then Return ConsoleWrite("UI Automation object ERR" & @CRLF)
    ConsoleWrite("UI Automation object OK" & @CRLF)

    ; Get UI Automation element from window handle
    Local $pWindow, $oWindow
    $oUIAutomation.ElementFromHandle($hWindow, $pWindow)
    $oWindow = ObjCreateInterface($pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    If Not IsObj($oWindow) Then Return ConsoleWrite("Automation element from window ERR" & @CRLF)
    ConsoleWrite("Automation element from window OK" & @CRLF)

    ; Condition to find "Run Threat Analysis Scan" button
    Local $pCondition, $pCondition1, $pCondition2
    $oUIAutomation.CreatePropertyCondition($UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1)
    $oUIAutomation.CreatePropertyCondition($UIA_NamePropertyId, "Run Threat Analysis Scan", $pCondition2)
    $oUIAutomation.CreateAndCondition($pCondition1, $pCondition2, $pCondition)
    If Not $pCondition Then Return ConsoleWrite("Run Threat Analysis Scan condition ERR" & @CRLF)
    ConsoleWrite("Run Threat Analysis Scancondition OK" & @CRLF)

    ; Find "Run Threat Analysis Scan" button
    Local $pButton, $oButton
    $oWindow.FindFirst($TreeScope_Descendants, $pCondition, $pButton)
    $oButton = ObjCreateInterface($pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    If Not IsObj($oButton) Then Return ConsoleWrite("Find button ERR" & @CRLF)
    ConsoleWrite("Find button OK" & @CRLF)

    ; Click (invoke) "Run Threat Analysis Scan" button
    Local $pInvoke, $oInvoke
    $oButton.GetCurrentPattern($UIA_InvokePatternId, $pInvoke)
    $oInvoke = ObjCreateInterface($pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern)
    If Not IsObj($oInvoke) Then Return ConsoleWrite("Invoke pattern ERR" & @CRLF)
    ConsoleWrite("Invoke pattern OK" & @CRLF)
    $oInvoke.Invoke()
    Sleep(2000)

Thank you all very much - I do appreciate your help as always. 

 

[junkew]

try what happens with the other commands of _uia_action

"move","setposition"

"highlight"

"screenshot"

"getobject"

if it moves you just can send a normal click thru autoitcommand

[LarsJ]

Note that the button is the parent of the text control as you can see from line "0: Title is: ..." above. Try this:

 

expandcollapse popup

#include <CUIAutomation2.au3>

Example()

Func Example()

  ; Get window handle
  Local $hWindow = WinGetHandle( "Symantec Help v2.1" )
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object ERR" & @CRLF )
  ConsoleWrite( "UI Automation object OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  ; Condition to find text control
  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_TextControlTypeId, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Find condition ERR" & @CRLF )
  ConsoleWrite( "Find condition OK" & @CRLF )

  ; Find all text controls
  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return ConsoleWrite( "No objects found ERR" & @CRLF )
  ConsoleWrite( $iElements & " objects found OK" & @CRLF )

  ; Identify text control by name
  Local $pUIElement, $oUIElement, $fIdentified = False
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    If _UIA_getPropertyValue( $oUIElement, $UIA_NamePropertyId ) = "Run Threat Analysis Scan" Then
      ConsoleWrite( "Object identified OK" & @CRLF )
      $fIdentified = True
      ExitLoop
    EndIf
  Next
  If Not $fIdentified Then Return ConsoleWrite( "No object identified ERR" & @CRLF )

  ; Find parent control
  Local $pRawWalker, $oRawWalker, $pUIParent, $oUIParent
  $oUIAutomation.RawViewWalker( $pRawWalker )
  $oRawWalker = ObjCreateInterface( $pRawWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker )
  $oRawWalker.GetParentElement( $pUIElement, $pUIParent )
  If Not $pUIParent Then Return ConsoleWrite( "Find parent control ERR" & @CRLF )
  ConsoleWrite( "Parent control found OK" & @CRLF )
  $oUIParent = ObjCreateInterface( $pUIParent, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

  ; Click (invoke) parent button
  Local $pInvoke, $oInvoke
  $oUIParent.GetCurrentPattern($UIA_InvokePatternId, $pInvoke)
  $oInvoke = ObjCreateInterface($pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern)
  If Not IsObj($oInvoke) Then Return ConsoleWrite("Invoke pattern ERR" & @CRLF)
  ConsoleWrite("Invoke pattern OK" & @CRLF)
  $oInvoke.Invoke()

EndFunc


Func _UIA_getPropertyValue( $obj, $id )
  Local $tVal
  $obj.GetCurrentPropertyValue( $id, $tVal )
  If Not IsArray( $tVal ) Then Return $tVal
  Local $tStr = $tVal[0]
  For $i = 1 To UBound( $tVal ) - 1
    $tStr &= "; " & $tVal[$i]
  Next
  Return $tStr
EndFunc

[mycloutier]

I am facing a similar challenge using the same controltype. I want my script to store a certain property (title) of a UIA_TextControlTypeId into a variable so I can then perform certain actions depending on what is stored in this variable. 

This is the code I have so far. You should be able to understand my logic in terms of what it is I am trying to accomplish. Any ideas on how to go about this or point me into a certain direction would be greatly appreciated. I am learning lots from theses forums. Still a newbie, just trying to figure out one challenge at a time :-) Thank you.

expandcollapse popup

#include "UIAWrappers1.au3"
#include <CUIAutomation2.au3>
AutoItSetOption("MustDeclareVars", 1)

#AutoIt3Wrapper_UseX64=N  ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os

; Run
ShellExecute("mobsync.exe")

; Wait 2 seconds for windows explorer to appear.
WinWait("Sync Center")

IF Not WinActive("{CLASS:CabinetWClass}", "Sync Center") Then WinActivate("{CLASS:CabinetWClass}", "Sync Center")

Local $oP7=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Sync Center;controltype:=UIA_WindowControlTypeId;class:=CabinetWClass", $treescope_children)
_UIA_Action($oP7,"setfocus")
Local $oP6=_UIA_getObjectByFindAll($oP7, "Title:=Sync Center;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass", $treescope_children)
_UIA_Action($oP6,"setfocus")
Local $oP5=_UIA_getObjectByFindAll($oP6, "Title:=;controltype:=UIA_PaneControlTypeId;class:=DUIViewWndClassName", $treescope_children)
_UIA_Action($oP5,"setfocus")
Local $oP4=_UIA_getObjectByFindAll($oP5, "Title:=Shell Folder View;controltype:=UIA_PaneControlTypeId;class:=DUIListView", $treescope_children)
_UIA_Action($oP4,"setfocus")
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=Folder View;controltype:=UIA_ListControlTypeId;class:=SysListView32", $treescope_children)
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=Folders (1);controltype:=UIA_GroupControlTypeId;class:=", $treescope_children)
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "title:=Offline Files;ControlType:=UIA_ListItemControlTypeId", $treescope_subtree)
_UIA_action($oP1,"setfocus")
Local $oUIElement=_UIA_getPropertyValue($oP1, "title:=:=UIA_TextControlTypeId", $treescope_subtree)


If Local $oUIElement=_UIA_getObjectByFindAll($oP1, "title:=Disconnected;ControlType:=UIA_TextControlTypeId", $treescope_subtree) Then
   _UIA_action($oUIElement,"setfocus")
   MsgBox(0, "Warning!", "Disconnected from the network!", 10)

ElseIf Local $oUIElement2=_UIA_getObjectByFindAll($oP1, "title:=Sync in progress;ControlType:=UIA_TextControlTypeId", $treescope_subtree) Then
   _UIA_action($oUIElement,"setfocus")
   MsgBox(0, "Warning!", "Sync is already in progress!", 10)

Elseif

Local $oP10=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Sync Center;controltype:=UIA_WindowControlTypeId;class:=CabinetWClass", $treescope_children)
_UIA_Action($oP10,"setfocus")
Local $oP9=_UIA_getObjectByFindAll($oP10, "Title:=Sync Center;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass", $treescope_children)
Local $oP8=_UIA_getObjectByFindAll($oP9, "title:=Sync All;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree)
_UIA_Action($oP8,"invoke")

EndIF

 

 

Edited April 10, 2015 by mycloutier

[junkew]

Something like this retrieves a propertyid

local $name=_UIA_getPropertyValue($oP8, $UIA_NamePropertyId)

[junkew]

Its allways difficult to exactly give an answer as there are so many different controls and different ways of tackling the problem

1. The hard way as shown in example 1 but then you need to study microsoft IUIAutomation and understand the model

2. The wrapped way with IUIWrappers but then you have the disadvantage of Work In Progress and not final version

3. The mixed way with SimpleSpy that gives a starting point like

$oP2=_UIA_getObjectByFindAll($oP3, "Title:=Folders (1);controltype:=UIA_GroupControlTypeId;class:=", $treescope_children)

but still then its just a basic example as the whole library supports many properties to identify a control

Its allways good if your basic code is highlighting the control then the other parts become more easy

Those 2 lines 

$oUIElement=_UIA_getObjectByFindAll($oP1, "title:=Disconnected;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
$oUIElement=_UIA_getObjectByFindAll($oP1, "title:=Sync in progress;ControlType:=UIA_TextControlTypeId", $treescope_subtree) 

you could write as

$oUIElement=_UIA_getObjectByFindAll($oP1, "title:=((Disconnected)|(Sync in progress));ControlType:=UIA_TextControlTypeId", $treescope_subtree)

and to setfocus of a textbox you can choose one of the 3 below

1. _UIA_Action($oUIElement,"setfocus")   ~; https://msdn.microsoft.com/en-us/library/windows/desktop/ee696043(v=vs.85).aspx
2. _UIA_Action($oUIElement,"click")         ~;  is actually a mousemove and a click on that location
3. _UIA_Action($oUIElement,"invoke")      ~; is doing implicitly also the first setfocus command in the wrappers

and if above 3 are not working you can only fallback to sending keyboardcommands like n times <TAB> from a certain closely related textbox which you can setfocus to

and potentially this can work directly for focus setting

$oUIElement.setfocus()

hopefully above gives you a solution direction

 

[mycloutier]

@junkew Thank you very much for your help. I was able to accomplish my task with your input :-)