souldjer777

UIA Invoke click for a UIA_TextControlTypeId - Is that possible?

7 posts in this topic

Good Morning AutoIT Gurus :)

How is everyone this fine Tuesday?  :thumbsup:

I have been using SimpleSpy and inspect.exe to get data from various program windows - but this one, I'm not sure about since it's a button - but it's also text.

Could I treat UIA_TextControlTypeID (TextBlock) as a button and click on it somehow? -  If so, how would I do that? FYI - I have successfully been using code examples from LarsJ and Junkew in the past.

The reason I have to treat that text as a button is that the button itself has no data. (At least we have an element [][button]) and the text within the button does... (At least we have an element [Run Threat Analysis Scan][TextBlock])

:idea:  SimpleSpy info:

Mouse position is retrieved 486-377
At least we have an element [Run Threat Analysis Scan][TextBlock]
 
Having the following values for all properties: 
Title is: <Run Threat Analysis Scan> Class   := <TextBlock> controltype:= <UIA_TextControlTypeId> ,<50020> , (0000C364) 397;367;154;16
*** Parent Information top down ***
7: Title is: <Desktop> Class   := <#32769> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) 0;0;1920;1080
"Title:=Desktop;controltype:=UIA_PaneControlTypeId;class:=#32769"
6: Title is: <Symantec Help v2.1> Class   := <Window> controltype:= <UIA_WindowControlTypeId> ,<50032> , (0000C370) 322;65;1024;680
"Title:=Symantec Help v2.1;controltype:=UIA_WindowControlTypeId;class:=Window"
5: Title is: <> Class   := <SharedWindow> controltype:= <UIA_CustomControlTypeId> ,<50025> , (0000C369) 330;143;1008;572
"Title:=;controltype:=UIA_CustomControlTypeId;class:=SharedWindow"
4: Title is: <> Class   := <Home2> controltype:= <UIA_CustomControlTypeId> ,<50025> , (0000C369) 342;193;996;522
"Title:=;controltype:=UIA_CustomControlTypeId;class:=Home2"
3: Title is: <> Class   := <ScrollViewer> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) 342;193;996;522
"Title:=;controltype:=UIA_PaneControlTypeId;class:=ScrollViewer"
2: Title is: <> Class   := <Ui2Widget> controltype:= <UIA_CustomControlTypeId> ,<50025> , (0000C369) 357;250;263;169
"Title:=;controltype:=UIA_CustomControlTypeId;class:=Ui2Widget"
1: Title is: <> Class   := <UiActionButton> controltype:= <UIA_CustomControlTypeId> ,<50025> , (0000C369) 357;355;263;40
"Title:=;controltype:=UIA_CustomControlTypeId;class:=UiActionButton"
0: Title is: <> Class   := <Button> controltype:= <UIA_ButtonControlTypeId> ,<50000> , (0000C350) 357;355;263;40
"Title:=;controltype:=UIA_ButtonControlTypeId;class:=Button"
 
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP6=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Symantec Help v2.1;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children)  
_UIA_Action($oP6,"setfocus")
Local $oP5=_UIA_getObjectByFindAll($oP6, "Title:=;controltype:=UIA_CustomControlTypeId;class:=SharedWindow", $treescope_children)   
Local $oP4=_UIA_getObjectByFindAll($oP5, "Title:=;controltype:=UIA_CustomControlTypeId;class:=Home2", $treescope_children)  
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=ScrollViewer", $treescope_children) 
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_CustomControlTypeId;class:=Ui2Widget", $treescope_children)  
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_CustomControlTypeId;class:=UiActionButton", $treescope_children) 
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_ButtonControlTypeId;class:=Button", $treescope_children) 
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("RunThreatAnalysisScan.mainwindow", "title:=Run Threat Analysis Scan;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Run Threat Analysis Scan;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"click")

This fails as well...

Window handle OK
UI Automation object OK
Automation element from window OK
Run Threat Analysis Scancondition OK
Find button ERR
 
Local $hWindow = WinGetHandle("Symantec Help v2.1")
    If Not $hWindow Then Return ConsoleWrite("Window handle ERR" & @CRLF)
    ConsoleWrite("Window handle OK" & @CRLF)

    ; Create UI Automation object
    Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
    If Not IsObj($oUIAutomation) Then Return ConsoleWrite("UI Automation object ERR" & @CRLF)
    ConsoleWrite("UI Automation object OK" & @CRLF)

    ; Get UI Automation element from window handle
    Local $pWindow, $oWindow
    $oUIAutomation.ElementFromHandle($hWindow, $pWindow)
    $oWindow = ObjCreateInterface($pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    If Not IsObj($oWindow) Then Return ConsoleWrite("Automation element from window ERR" & @CRLF)
    ConsoleWrite("Automation element from window OK" & @CRLF)

    ; Condition to find "Run Threat Analysis Scan" button
    Local $pCondition, $pCondition1, $pCondition2
    $oUIAutomation.CreatePropertyCondition($UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1)
    $oUIAutomation.CreatePropertyCondition($UIA_NamePropertyId, "Run Threat Analysis Scan", $pCondition2)
    $oUIAutomation.CreateAndCondition($pCondition1, $pCondition2, $pCondition)
    If Not $pCondition Then Return ConsoleWrite("Run Threat Analysis Scan condition ERR" & @CRLF)
    ConsoleWrite("Run Threat Analysis Scancondition OK" & @CRLF)

    ; Find "Run Threat Analysis Scan" button
    Local $pButton, $oButton
    $oWindow.FindFirst($TreeScope_Descendants, $pCondition, $pButton)
    $oButton = ObjCreateInterface($pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    If Not IsObj($oButton) Then Return ConsoleWrite("Find button ERR" & @CRLF)
    ConsoleWrite("Find button OK" & @CRLF)

    ; Click (invoke) "Run Threat Analysis Scan" button
    Local $pInvoke, $oInvoke
    $oButton.GetCurrentPattern($UIA_InvokePatternId, $pInvoke)
    $oInvoke = ObjCreateInterface($pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern)
    If Not IsObj($oInvoke) Then Return ConsoleWrite("Invoke pattern ERR" & @CRLF)
    ConsoleWrite("Invoke pattern OK" & @CRLF)
    $oInvoke.Invoke()
    Sleep(2000)

Thank you all very much - I do appreciate your help as always.  :ILA:

 


"Maybe I'm on a road that ain't been paved yet. And maybe I see a sign that ain't been made yet"
Song Title: I guess you could say
Artist: Middle Class Rut

Share this post


Link to post
Share on other sites



Note that the button is the parent of the text control as you can see from line "0: Title is: ..." above. Try this:

 

#include <CUIAutomation2.au3>

Example()

Func Example()

  ; Get window handle
  Local $hWindow = WinGetHandle( "Symantec Help v2.1" )
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object ERR" & @CRLF )
  ConsoleWrite( "UI Automation object OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  ; Condition to find text control
  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_TextControlTypeId, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Find condition ERR" & @CRLF )
  ConsoleWrite( "Find condition OK" & @CRLF )

  ; Find all text controls
  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return ConsoleWrite( "No objects found ERR" & @CRLF )
  ConsoleWrite( $iElements & " objects found OK" & @CRLF )

  ; Identify text control by name
  Local $pUIElement, $oUIElement, $fIdentified = False
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    If _UIA_getPropertyValue( $oUIElement, $UIA_NamePropertyId ) = "Run Threat Analysis Scan" Then
      ConsoleWrite( "Object identified OK" & @CRLF )
      $fIdentified = True
      ExitLoop
    EndIf
  Next
  If Not $fIdentified Then Return ConsoleWrite( "No object identified ERR" & @CRLF )

  ; Find parent control
  Local $pRawWalker, $oRawWalker, $pUIParent, $oUIParent
  $oUIAutomation.RawViewWalker( $pRawWalker )
  $oRawWalker = ObjCreateInterface( $pRawWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker )
  $oRawWalker.GetParentElement( $pUIElement, $pUIParent )
  If Not $pUIParent Then Return ConsoleWrite( "Find parent control ERR" & @CRLF )
  ConsoleWrite( "Parent control found OK" & @CRLF )
  $oUIParent = ObjCreateInterface( $pUIParent, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

  ; Click (invoke) parent button
  Local $pInvoke, $oInvoke
  $oUIParent.GetCurrentPattern($UIA_InvokePatternId, $pInvoke)
  $oInvoke = ObjCreateInterface($pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern)
  If Not IsObj($oInvoke) Then Return ConsoleWrite("Invoke pattern ERR" & @CRLF)
  ConsoleWrite("Invoke pattern OK" & @CRLF)
  $oInvoke.Invoke()

EndFunc


Func _UIA_getPropertyValue( $obj, $id )
  Local $tVal
  $obj.GetCurrentPropertyValue( $id, $tVal )
  If Not IsArray( $tVal ) Then Return $tVal
  Local $tStr = $tVal[0]
  For $i = 1 To UBound( $tVal ) - 1
    $tStr &= "; " & $tVal[$i]
  Next
  Return $tStr
EndFunc

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

I am facing a similar challenge using the same controltype. I want my script to store a certain property (title) of a UIA_TextControlTypeId into a variable so I can then perform certain actions depending on what is stored in this variable. 

This is the code I have so far. You should be able to understand my logic in terms of what it is I am trying to accomplish. Any ideas on how to go about this or point me into a certain direction would be greatly appreciated. I am learning lots from theses forums. Still a newbie, just trying to figure out one challenge at a time :-) Thank you.

#include "UIAWrappers1.au3"
#include <CUIAutomation2.au3>
AutoItSetOption("MustDeclareVars", 1)

#AutoIt3Wrapper_UseX64=N  ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os

; Run
ShellExecute("mobsync.exe")

; Wait 2 seconds for windows explorer to appear.
WinWait("Sync Center")

IF Not WinActive("{CLASS:CabinetWClass}", "Sync Center") Then WinActivate("{CLASS:CabinetWClass}", "Sync Center")

Local $oP7=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Sync Center;controltype:=UIA_WindowControlTypeId;class:=CabinetWClass", $treescope_children)
_UIA_Action($oP7,"setfocus")
Local $oP6=_UIA_getObjectByFindAll($oP7, "Title:=Sync Center;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass", $treescope_children)
_UIA_Action($oP6,"setfocus")
Local $oP5=_UIA_getObjectByFindAll($oP6, "Title:=;controltype:=UIA_PaneControlTypeId;class:=DUIViewWndClassName", $treescope_children)
_UIA_Action($oP5,"setfocus")
Local $oP4=_UIA_getObjectByFindAll($oP5, "Title:=Shell Folder View;controltype:=UIA_PaneControlTypeId;class:=DUIListView", $treescope_children)
_UIA_Action($oP4,"setfocus")
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=Folder View;controltype:=UIA_ListControlTypeId;class:=SysListView32", $treescope_children)
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=Folders (1);controltype:=UIA_GroupControlTypeId;class:=", $treescope_children)
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "title:=Offline Files;ControlType:=UIA_ListItemControlTypeId", $treescope_subtree)
_UIA_action($oP1,"setfocus")
Local $oUIElement=_UIA_getPropertyValue($oP1, "title:=:=UIA_TextControlTypeId", $treescope_subtree)


If Local $oUIElement=_UIA_getObjectByFindAll($oP1, "title:=Disconnected;ControlType:=UIA_TextControlTypeId", $treescope_subtree) Then
   _UIA_action($oUIElement,"setfocus")
   MsgBox(0, "Warning!", "Disconnected from the network!", 10)

ElseIf Local $oUIElement2=_UIA_getObjectByFindAll($oP1, "title:=Sync in progress;ControlType:=UIA_TextControlTypeId", $treescope_subtree) Then
   _UIA_action($oUIElement,"setfocus")
   MsgBox(0, "Warning!", "Sync is already in progress!", 10)

Elseif

Local $oP10=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Sync Center;controltype:=UIA_WindowControlTypeId;class:=CabinetWClass", $treescope_children)
_UIA_Action($oP10,"setfocus")
Local $oP9=_UIA_getObjectByFindAll($oP10, "Title:=Sync Center;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass", $treescope_children)
Local $oP8=_UIA_getObjectByFindAll($oP9, "title:=Sync All;ControlType:=UIA_ButtonControlTypeId", $treescope_subtree)
_UIA_Action($oP8,"invoke")

EndIF

 

 

Edited by mycloutier

Share this post


Link to post
Share on other sites

Its allways difficult to exactly give an answer as there are so many different controls and different ways of tackling the problem

1. The hard way as shown in example 1 but then you need to study microsoft IUIAutomation and understand the model

2. The wrapped way with IUIWrappers but then you have the disadvantage of Work In Progress and not final version

3. The mixed way with SimpleSpy that gives a starting point like

$oP2=_UIA_getObjectByFindAll($oP3, "Title:=Folders (1);controltype:=UIA_GroupControlTypeId;class:=", $treescope_children)

but still then its just a basic example as the whole library supports many properties to identify a control

Its allways good if your basic code is highlighting the control then the other parts become more easy

Those 2 lines 

$oUIElement=_UIA_getObjectByFindAll($oP1, "title:=Disconnected;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
$oUIElement=_UIA_getObjectByFindAll($oP1, "title:=Sync in progress;ControlType:=UIA_TextControlTypeId", $treescope_subtree) 

you could write as

$oUIElement=_UIA_getObjectByFindAll($oP1, "title:=((Disconnected)|(Sync in progress));ControlType:=UIA_TextControlTypeId", $treescope_subtree)

and to setfocus of a textbox you can choose one of the 3 below

1. _UIA_Action($oUIElement,"setfocus")   ~; https://msdn.microsoft.com/en-us/library/windows/desktop/ee696043(v=vs.85).aspx
2. _UIA_Action($oUIElement,"click")         ~;  is actually a mousemove and a click on that location
3. _UIA_Action($oUIElement,"invoke")      ~; is doing implicitly also the first setfocus command in the wrappers

and if above 3 are not working you can only fallback to sending keyboardcommands like n times <TAB> from a certain closely related textbox which you can setfocus to

and potentially this can work directly for focus setting

$oUIElement.setfocus()

hopefully above gives you a solution direction

 
1 person likes this

Share this post


Link to post
Share on other sites

@junkew Thank you very much for your help. I was able to accomplish my task with your input :-)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now