IUIAutomation MS framework automate chrome, FF, IE, ....

Inververs · December 19, 2013

How to execute DoDefaultAction on oUIElement?

Please show in example.

$objUIAutomation.RawViewWalker($pTW)
$oTW=ObjCreateInterface($pTW, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker)
$oTW.GetPreviousSiblingElement($oElement, $pUIElement)
$oUIElement = ObjCreateInterface($pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)

$oUIElement.????

junkew · December 19, 2013

see example 3

dim $pLegacy
;~ $oClock.getCurrentPattern($UIA_InvokePatternId, $pInvoke)
$oLegacyP=_UIA_getPattern($oClock,$UIA_LegacyIAccessiblePatternId)
$oLegacyP.dodefaultaction()

topten · December 20, 2013

Thank you very much for this great library. To tell the truith it takes creating things onto higher level. What was taking me days of job- now it takes only several hours

Could you please give any additional example of using the library in IE, finding the element

not by title

For example if I want to obtain the Search form on this forum, using the spy I am getting this infromation:

Title is: <Search...> Class := <> controltype:= <UIA_EditControlTypeId> ,<50004> , (0000C354)
*** Parent Information ***
Title is: <> Class := <> controltype:= <UIA_CustomControlTypeId> ,<50025> , (0000C369)
*** Detailed properties of the highlighted element ***
UIA_AcceleratorKeyPropertyId :=
UIA_AccessKeyPropertyId :=
UIA_AriaPropertiesPropertyId :=
UIA_AriaRolePropertyId :=
UIA_AutomationIdPropertyId :=
UIA_BoundingRectanglePropertyId :=1050;126;130;15
UIA_ClassNamePropertyId :=
UIA_ClickablePointPropertyId :=
UIA_ControllerForPropertyId :=
UIA_ControlTypePropertyId :=50004
UIA_CulturePropertyId :=0
UIA_DescribedByPropertyId :=
UIA_DockDockPositionPropertyId :=5
UIA_ExpandCollapseExpandCollapseStatePropertyId :=3
UIA_FlowsToPropertyId :=
UIA_FrameworkIdPropertyId :=
UIA_GridColumnCountPropertyId :=0
UIA_GridItemColumnPropertyId :=0
UIA_GridItemColumnSpanPropertyId :=1
UIA_GridItemContainingGridPropertyId :=
UIA_GridItemRowPropertyId :=0
UIA_GridItemRowSpanPropertyId :=1
UIA_GridRowCountPropertyId :=0
UIA_HasKeyboardFocusPropertyId :=False
UIA_HelpTextPropertyId :=
UIA_IsContentElementPropertyId :=True
UIA_IsControlElementPropertyId :=True
UIA_IsDataValidForFormPropertyId :=False
UIA_IsDockPatternAvailablePropertyId :=False
UIA_IsEnabledPropertyId :=True
UIA_IsExpandCollapsePatternAvailablePropertyId :=False
UIA_IsGridItemPatternAvailablePropertyId :=False
UIA_IsGridPatternAvailablePropertyId :=False
UIA_IsInvokePatternAvailablePropertyId :=True
UIA_IsItemContainerPatternAvailablePropertyId :=False
UIA_IsKeyboardFocusablePropertyId :=True
UIA_IsLegacyIAccessiblePatternAvailablePropertyId :=True
UIA_IsMultipleViewPatternAvailablePropertyId :=False
UIA_IsOffscreenPropertyId :=False
UIA_IsPasswordPropertyId :=False
UIA_IsRangeValuePatternAvailablePropertyId :=False
UIA_IsRequiredForFormPropertyId :=False
UIA_IsScrollItemPatternAvailablePropertyId :=False
UIA_IsScrollPatternAvailablePropertyId :=False
UIA_IsSelectionItemPatternAvailablePropertyId :=False
UIA_IsSelectionPatternAvailablePropertyId :=False
UIA_IsSynchronizedInputPatternAvailablePropertyId :=False
UIA_IsTableItemPatternAvailablePropertyId :=False
UIA_IsTablePatternAvailablePropertyId :=False
UIA_IsTextPatternAvailablePropertyId :=False
UIA_IsTogglePatternAvailablePropertyId :=False
UIA_IsTransformPatternAvailablePropertyId :=False
UIA_IsValuePatternAvailablePropertyId :=True
UIA_IsVirtualizedItemPatternAvailablePropertyId :=False
UIA_IsWindowPatternAvailablePropertyId :=False
UIA_ItemStatusPropertyId :=
UIA_ItemTypePropertyId :=
UIA_LabeledByPropertyId :=
UIA_LegacyIAccessibleChildIdPropertyId :=0
UIA_LegacyIAccessibleDefaultActionPropertyId :=activate
UIA_LegacyIAccessibleDescriptionPropertyId :=
UIA_LegacyIAccessibleHelpPropertyId :=
UIA_LegacyIAccessibleKeyboardShortcutPropertyId :=
UIA_LegacyIAccessibleNamePropertyId :=Search...
UIA_LegacyIAccessibleRolePropertyId :=42
UIA_LegacyIAccessibleSelectionPropertyId :=
UIA_LegacyIAccessibleStatePropertyId :=1048576
UIA_LegacyIAccessibleValuePropertyId :=
UIA_LocalizedControlTypePropertyId :=поле
UIA_MultipleViewCurrentViewPropertyId :=0
UIA_MultipleViewSupportedViewsPropertyId :=
UIA_NamePropertyId :=Search...
UIA_NativeWindowHandlePropertyId :=0
UIA_OrientationPropertyId :=0
UIA_ProcessIdPropertyId :=7164
UIA_ProviderDescriptionPropertyId :=[pid:7164,hwnd:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]
UIA_RangeValueIsReadOnlyPropertyId :=True
UIA_RangeValueLargeChangePropertyId :=0
UIA_RangeValueMaximumPropertyId :=0
UIA_RangeValueMinimumPropertyId :=0
UIA_RangeValueSmallChangePropertyId :=0
UIA_RangeValueValuePropertyId :=0
UIA_RuntimeIdPropertyId :=
UIA_ScrollHorizontallyScrollablePropertyId :=False
UIA_ScrollHorizontalScrollPercentPropertyId :=0
UIA_ScrollHorizontalViewSizePropertyId :=100
UIA_ScrollVerticallyScrollablePropertyId :=False
UIA_ScrollVerticalScrollPercentPropertyId :=0
UIA_ScrollVerticalViewSizePropertyId :=100
UIA_SelectionCanSelectMultiplePropertyId :=False
UIA_SelectionIsSelectionRequiredPropertyId :=False
UIA_SelectionselectionPropertyId :=
UIA_SelectionItemIsSelectedPropertyId :=False
UIA_SelectionItemSelectionContainerPropertyId :=
UIA_TableColumnHeadersPropertyId :=
UIA_TableItemColumnHeaderItemsPropertyId :=
UIA_TableRowHeadersPropertyId :=
UIA_TableRowOrColumnMajorPropertyId :=2
UIA_TableItemRowHeaderItemsPropertyId :=
UIA_ToggleToggleStatePropertyId :=2
UIA_TransformCanMovePropertyId :=False
UIA_TransformCanResizePropertyId :=False
UIA_TransformCanRotatePropertyId :=False
UIA_ValueIsReadOnlyPropertyId :=False
UIA_ValueValuePropertyId :=
UIA_WindowCanMaximizePropertyId :=False
UIA_WindowCanMinimizePropertyId :=False
UIA_WindowIsModalPropertyId :=False
UIA_WindowIsTopmostPropertyId :=False
UIA_WindowWindowInteractionStatePropertyId :=0
UIA_WindowWindowVisualStatePropertyId :=0

To obtain it in IE by title I would do it this way

$IE=_UIA_getFirstObjectOfElement($oDesktop,"class:=IEFrame", $treescope_children)

if IsObj($IE) Then
   $sText="Search..."
    $oButton=_UIA_getFirstObjectOfElement($IE,"name:=" & $sText, $treescope_subtree)
   _UIA_action($oButton,"leftclick")
   _UIA_action($oButton,"setvalue using keys","autoit")

endif

Can I do it in some other way, not by title, because- the title can be different in other languages (German, Russian, Chinese etc)

Thanx in advance!!!!!

junkew · December 20, 2013

frequently this helps in your code (at least while you are debugging your script)

_UIA_Dumpthemall($IE, $treescope_subtree)

It dumps to a log.txt or to the terminaloutput

then you could count in that file or look for the properties

$oSearchbox=_UIA_getFirstObjectOfElement($IE,"automationid:=main_search" , $treescope_subtree)

and another function that can just iterate thru the array of elements (by using findall) and checking by using multiple properties syntax

$oSearchbox=_UIA_getObjectByFindAll($IE,"controltype:=Edit;index:=3" , $treescope_subtree)

ofcourse counting is always tricky to use especially in above scenario where you use IE as the root starting point (instead of the html tab window itself) in combination with $treescope_subtree which means scan all elements below the starting point recursively

Edited December 20, 2013 by junkew

Inververs · December 21, 2013

junkew

For the FindAll method can I specify 2 condition and combine them through AND?

for example, want to find only $ UIA_EditControlTypeId which $ UIA_IsEnabledPropertyId = TRUE

$objUIAutomation.createPropertyCondition($UIA_IsEnabledPropertyId, True, $pCondition_1)
$objUIAutomation.createPropertyCondition($UIA_ControlTypePropertyId, $UIA_EditControlTypeId, $pCondition_2)

whether you can now combine them?

I try this, but hav error:

$objUIAutomation.createAndCondition($pCondition_1, $pCondition_2, $pCondition)
$oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition)

LarsJ · December 21, 2013

Inververs, I have had the same issues with $UIA_SelectionItemIsSelectedPropertyId. It was not possible to change the default False value to True. It seems to be a problem with the variant boolean data type. No problems with strings or integers. I don't think it has anything to do with CreateAndCondition or FindAll.

This issue seems to be fixed in the newest beta version of AutoIt (AutoIt v3.3.9.25 Beta). You should try to run your code with this version.

Inververs · December 21, 2013

LarsJ, i think that $objUIAutomation.createAndCondition return pointer that not suitable for $obj.FindAll method.

I try add two identical Condition like this:

Local $pCondition_1
    Local $pCondition_2
    $objUIAutomation.createPropertyCondition($UIA_ControlTypePropertyId, $UIA_EditControlTypeId, $pCondition_1)
    $objUIAutomation.createPropertyCondition($UIA_ControlTypePropertyId, $UIA_EditControlTypeId, $pCondition_2)
    Local $pCondition
    $objUIAutomation.createAndCondition($pCondition_1, $pCondition_2, $pCondition)
    $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition)
    Local $pElements
    $oFrame.FindAll($treescope_subtree, $oCondition, $pElements)

And have autoit crash in $oFrame.FindAll($treescope_subtree, $oCondition, $pElements)

AutoIt v3.3.9.25 Beta

LarsJ · December 21, 2013

Try with $pCondition in FindAll in stead of $oCondition.

This code works for me:

Local $pCondition1
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $iCtrlType, $pCondition1 )
If Not $pCondition1 Then Return

Local $pCondition2
$oUIAutomation.CreatePropertyCondition( $UIA_SelectionItemIsSelectedPropertyId, True, $pCondition2 )
If Not $pCondition2 Then Return

Local $pAndCondition
$oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pAndCondition )
If Not $pAndCondition Then Return

Local $pUIElementArray, $oUIElementArray, $iSelItems
$oWindow.FindAll( $TreeScope_Descendants, $pAndCondition, $pUIElementArray )
$oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
$oUIElementArray.Length( $iSelItems )
If Not $iSelItems Then Return

Edited December 21, 2013 by LarsJ

Inververs · December 21, 2013

LarsJ, Thanks, it worked :beer:

topten · December 23, 2013

I am trying to use alternative names, but I guess I am using incorrect syntax

$IEr=_UIA_getFirstObjectOfElement($oDesktop,"class:=IEFrame", $treescope_children)
if IsObj($IEr) Then

$sText=(Search|Search the text)
    $oButton=_UIA_getFirstObjectOfElement($IEr,"name:=" & $sText, $treescope_subtree)
    _UIA_action($oButton,"setvalue using keys","autoit")
    
 EndIf

Please explain, how can I use several variants of titles?

Thanx in advance!!!!

junkew · December 23, 2013

$sText=(Search|Search the text)

should become something like

$sText="((Search)|(Search the text))"

or just do

$sText="Search.*"

read some manuals on regular expressions to understand grouping mechanism with parenthesis ()

LarsJ · December 23, 2013

This example shows how to use caching.

To run the example you must replace the following lines in the description string for the IUIAutomationCacheRequest interface in CUIAutomation2.au3 (lines 532 - 537 in the current version):

"TreeScope hresult(long*);" & _
"TreeScope hresult(long);" & _
"TreeFilter hresult(ptr*);" & _
"TreeFilter hresult(ptr);" & _
"AutomationElementMode hresult(long*);" & _
"AutomationElementMode hresult(long);"

with these lines:

"get_TreeScope hresult(long*);" & _
"put_TreeScope hresult(long);" & _
"get_TreeFilter hresult(ptr*);" & _
"put_TreeFilter hresult(ptr);" & _
"get_AutomationElementMode hresult(long*);" & _
"put_AutomationElementMode hresult(long);"

This issue where get/put methods for read/write properties have the same names is also the case for the interfaces IAccessible and IUIAutomationProxyFactoryEntry.

This code prints the items in Windows Explorer listview with caching:

#include "..\Include\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

MainFunc()


Func MainFunc()

  ; Be sure to use the right class if you are on Vista or Windows 8
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]", "" )  ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]", "" ) ; Windows Explorer, Windows XP
  If Not $hWindow Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  If Not $pWindow Then Return

  Local $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return

  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ListControlTypeId, $pCondition )
  If Not $pCondition Then Return

  Local $pUIList, $oUIList
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pUIList )
  $oUIList = ObjCreateInterface( $pUIList, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oUIList ) Then Return

  ListAllItemsCached( $oUIList )

EndFunc


Func ListAllItemsCached( $oUIList )

  ConsoleWrite( "With CACHING" & @CRLF )

  Local $pIUIAutomationCacheRequest, $oIUIAutomationCacheRequest
  $oUIAutomation.CreateCacheRequest( $pIUIAutomationCacheRequest )
  $oIUIAutomationCacheRequest = ObjCreateInterface( $pIUIAutomationCacheRequest, $sIID_IUIAutomationCacheRequest, $dtagIUIAutomationCacheRequest )
  If Not IsObj( $oIUIAutomationCacheRequest ) Then Return

  Local $iAutomationElementMode
  If $oIUIAutomationCacheRequest.AddProperty( $UIA_NamePropertyId ) Then Return                       ; Method returns non-zero value on error
  If $oIUIAutomationCacheRequest.AddProperty( $UIA_SelectionItemIsSelectedPropertyId ) Then Return
  If $oIUIAutomationCacheRequest.put_AutomationElementMode( $AutomationElementMode_None ) Then Return ; Set Mode = None = 0 ; Mode = Full = 1 is default
  If $oIUIAutomationCacheRequest.get_AutomationElementMode( $iAutomationElementMode ) Then Return     ; Get the Mode we have just set
  ConsoleWrite( "AutomationElementMode = " & $iAutomationElementMode & @CRLF )                        ; This should print Mode = 0

  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ListItemControlTypeId, $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oUIList.FindAllBuildCache( $TreeScope_Children, $pCondition, $pIUIAutomationCacheRequest, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $sel
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCachedPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCachedPropertyValue( $UIA_SelectionItemIsSelectedPropertyId, $sel )
    ConsoleWrite( $name & "  " & $sel & @CRLF )
  Next

EndFunc

This code prints the items without caching:

#include "..\Include\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

MainFunc()


Func MainFunc()

  ; Be sure to use the right class if you are on Vista or Windows 8
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]", "" )  ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]", "" ) ; Windows Explorer, Windows XP
  If Not $hWindow Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  If Not $pWindow Then Return

  Local $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return

  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ListControlTypeId, $pCondition )
  If Not $pCondition Then Return

  Local $pUIList, $oUIList
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pUIList )
  $oUIList = ObjCreateInterface( $pUIList, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oUIList ) Then Return

  ListAllItems( $oUIList )

EndFunc


Func ListAllItems( $oUIList )

  ConsoleWrite( "No caching" & @CRLF )

  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ListItemControlTypeId, $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oUIList.FindAll( $TreeScope_Children, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $sel
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_SelectionItemIsSelectedPropertyId, $sel )
    ConsoleWrite( $name & "  " & $sel & @CRLF )
  Next

EndFunc

topten · December 24, 2013

Thank you very much, this works!!!!

$sText="((Search)|(Search the text))"

I have another question, will try to explain it:

I have this code

$IEr=_UIA_getFirstObjectOfElement($oDesktop,"class:=IEFrame", $treescope_children) if IsObj($IEr) Then $sText=(Serch|Sarch the text)     $oButton=_UIA_getFirstObjectOfElement($IEr,"name:=" & $sText, $treescope_subtree)     _UIA_action($oButton,"setvalue using keys","autoit")      EndIf

And I am putting here incorrect title- so that I could explain the situation

Lets assume, I have a wrong title, and of course, while executing the code I will receive the mistake. Usually the program will stop after that. How can I catch the mistake? For example catch it- but let the program continue? I think it should be a little something like this

$sText=(Serch|Sarch the text)

if $oButton=_UIA_getFirstObjectOfElement($IEr,"name:=" & $sText, $treescope_subtree) then

_UIA_action($oButton,"setvalue using keys","autoit")

EndIf

Is this correct way?

LarsJ · December 25, 2013

The listview in Windows Explorer on Win 7 is a virtual listview: The items are stored in an array and the listview only contains the visible items. That's the reason why you only get the visible items when you run the code in post #92.

This example shows how to get virtual items.

#include "..\Include\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

MainFunc()


Func MainFunc()

  ; Get window handle for Windows Explorer on Windows 7, 8
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]", "" )
  If Not $hWindow Then Return ConsoleWrite( "Could not get window handle" & @CRLF )

  ; Create UI Automation object
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "Could not create UI Automation object" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Could not get UI Automation element from window handle" & @CRLF )

  ; Condition to find listview
  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ListControlTypeId, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Could not create property condition" & @CRLF )

  ; Find listview
  Local $pList, $oList
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pList )
  $oList = ObjCreateInterface( $pList, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oList ) Then Return ConsoleWrite( "Could not find listview" & @CRLF )

  ; List all items
  ListAllItems( $oList )

EndFunc


Func ListAllItems( $oList )

  ; ItemContainer object
  ; A virtual listview must support the ItemContainer object
  Local $pItemContainer, $oItemContainer
  $oList.GetCurrentPattern( $UIA_ItemContainerPatternId, $pItemContainer )
  $oItemContainer = ObjCreateInterface( $pItemContainer, $sIID_IUIAutomationItemContainerPattern, $dtagIUIAutomationItemContainerPattern )
  If Not IsObj( $oItemContainer ) Then Return ConsoleWrite( "Could not create ItemContainer object" & @CRLF )

  ; Find first item
  Local $pItem, $oItem
  ; The following command works for both real and virtual items
  $oItemContainer.FindItemByProperty( 0, $UIA_NamePropertyId, 0, $pItem )
  $oItem = ObjCreateInterface( $pItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oItem ) Then Return ConsoleWrite( "Could not find first item. Empty folder?" & @CRLF ) ; Empty folder?

  ; Is first item a virtual item?
  Local $pVirtualItem, $oVirtualItem
  $oItem.GetCurrentPattern( $UIA_VirtualizedItemPatternId, $pVirtualItem )
  If $pVirtualItem Then ; First item is a virtual item, realize it
    $oVirtualItem = ObjCreateInterface( $pVirtualItem, $sIID_IUIAutomationVirtualizedItemPattern, $dtagIUIAutomationVirtualizedItemPattern )
    If $oVirtualItem.Realize() Then Return ConsoleWrite( "Could not realize first item" & @CRLF ) ; No error: $oItem is now a real item
  EndIf

  Local $fSel, $sName
  While $pItem
    $oItem.GetCurrentPropertyValue( $UIA_SelectionItemIsSelectedPropertyId, $fSel )
    $oItem.GetCurrentPropertyValue( $UIA_NamePropertyId, $sName )
    ConsoleWrite( $sName & "  " & $fSel & @CRLF )

    ; Find next item
    $oItemContainer.FindItemByProperty( $pItem, $UIA_NamePropertyId, 0, $pItem )
    $oItem = ObjCreateInterface( $pItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    If Not IsObj( $oItem ) Then ExitLoop ; No more items

    ; Virtual item?
    $oItem.GetCurrentPattern( $UIA_VirtualizedItemPatternId, $pVirtualItem )
    If $pVirtualItem Then ; The item is a virtual item, realize it
      $oVirtualItem = ObjCreateInterface( $pVirtualItem, $sIID_IUIAutomationVirtualizedItemPattern, $dtagIUIAutomationVirtualizedItemPattern )
      $oVirtualItem.Realize() ; $oItem is now a real item
    EndIf
  WEnd

EndFunc

Last edit: Code update.

Edited July 21, 2014 by LarsJ

junkew · December 28, 2013

Any advice on how to best synchronize with applications under test as there seems not to be a generic way?

Thoughts I have

1. Sleep() the way it is now

2. GetGUIThreadInfo and check hwndActive and hwndFocus in combination with a send paint message and wait for it to return,SendMessageW($hWND,$WM_PAINT,0,0)

3. _WinAPI_SendMessageTimeout($hWnd, 0x00, 0, 0, $SMTO_ABORTIFHUNG,0) seems to work well with notepad but not with browsers (as when one tab is loading its not meaning I cannot click)

4. check the systemcursor

5. outdated/not adviced (at least when I google and read the results) is get _WinApi_WaitForInputIdle

'?do=embed' frameborder='0' data-embedContent>>

6. Count elements (if stable for more then n milliseconds then assume everything is loaded)

for the moment I live with option 1 but for sure that will be tricky when scripts are larger

some trial code I have with IE (probably change $UIA_desktop to $oDesktop)

#include <WinAPISys.au3>
#include "UIAWrappers.au3"

;~ _UIA_setVar("global.debug.file",false)

const $cBrowser="classname:=IEFrame"
const $cTabBar="classname:=ReBarWindow32"
const $closeText=".*((sluiten)|(close))" ;Handle dutch and english text

$oIE=_UIA_getFirstObjectOfElement($UIA_oDesktop, $cBrowser, $treescope_children)
$hWND=_UIA_getPropertyValue($oIE, $UIA_NativeWindowHandlePropertyId)

$oIETabBar=_UIA_getFirstObjectOfElement($oIE, $cTabBar, $treescope_children)

$oCloseTab=_UIA_getObjectByFindAll($oIETabbar, $closeText , $treescope_subtree)
;~ _UIA_Action($oCloseTab,"invoke") ;click will only work when you set focus first


_UIA_Action($oIE,"sendkeys","{F5}")
$ret1=_WinAPI_SendMessageTimeout($hWND, 0x00, 0, 0, $SMTO_ABORTIFHUNG,0)
$ret=$ret1
consolewrite($ret & @crlf)
while ($ret1<>0)
    $ret1=_WinAPI_SendMessageTimeout($hWND, 0x00, 0, 0, $SMTO_ABORTIFHUNG,0)
    $ret=$ret & $ret1
    sleep(1)
    consolewrite($ret & @crlf)
wend
_SendMessage($hWnd, 0x000F, 0, 0) ; $WM_PAINT = 0x000F
consolewrite($ret & @crlf)

local $pElements, $iLength, $start, $ret

$iLength=-1
    $oIE.FindAll($treescope_subtree, $UIA_oTRUECondition, $pElements)
    $oAutomationElementArray = ObjCreateInterFace($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray)
    ;~ All elements to inspect are in this array
    $oAutomationElementArray.Length($start)

while $start<>$iLength
    $start=$iLength
    $oIE.FindAll($treescope_subtree, $UIA_oTRUECondition, $pElements)
    $oAutomationElementArray = ObjCreateInterFace($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray)
    ;~ All elements to inspect are in this array
    $oAutomationElementArray.Length($iLength)
    $ret=$ret & $iLength & ";"
    sleep(100)
WEnd
consolewrite($ret & @crlf)

Edited December 28, 2013 by junkew

FaridAgl · December 29, 2013

I don't know so much about this UDF, but seems like it's a great job.

I have a window, which AutoIt Window Info doesn't recognize any of its controls, do you think it could be done with this UDF?

Here is what I have captured:

As you see, it's just a window, with a title and a class and no controls at all. However checkbox, inputbox, button, etc seems to be standard controls.

It's Garena Plus and could be downloaded from here. I have multiple Garena accounts and just want to have multiple scripts which runnig each of them automatically log me in to Garena Plus.

Any help and/or tips are greatly welcomed.

Cheers

Jos · December 29, 2013

... and you feel Garena isn't falling under the "no games related threads/posts" rules we have in the forums?

Jos

FaridAgl · December 29, 2013

Is automating Skype's login form against forum rules? Or Yahoo Messenger's login form?

It's just a login form, I want to just automate typing of Username and Password and then clicking Login button. Is there any problem?

junkew · December 29, 2013

@Darkon3: Its on the boundary but anyway I just tried with the simple spy and although it sees something its far below what you need

userbox

At least we have an element [][ATL:5B8E4D08]

Having the following values for all properties:

Title is: <> Class := <ATL:5B8E4D08> controltype:= <UIA_EditControlTypeId> ,<50004> , (0000C354)

*** Parent Information ***

Title is: <Garena Plus (Beta)> Class := <UIHOST> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371)

password box

At least we have an element [][ATL:5B8E2660]

Having the following values for all properties:

Title is: <> Class := <ATL:5B8E2660> controltype:= <UIA_EditControlTypeId> ,<50004> , (0000C354)

*** Parent Information ***

Title is: <Garena Plus (Beta)> Class := <UIHOST> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371)

all other elements are not recognized and even above 2 are not reachable in a normal way with the treewalkers.

Easiest is probably by navigating with relative coordinates from the top of the UIHOST window (but please start a different thread for that in the support forum)

Melba23 · December 29, 2013

D4RKON3,

Quote
Is automating Skype's login form against forum rules? Or Yahoo Messenger's login form?

No, because they are not designed solely for gaming.

Quote
please start a different thread for that in the support forum

Please do NOT do that because it will be locked and a very dim view will be taken of you having done it.

M23

Sign In

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

Inververs

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

junkew

junkew

junkew

Posted Images

junkew

topten

junkew

Inververs

LarsJ

Inververs

LarsJ

Inververs

topten

junkew

LarsJ

topten

LarsJ

junkew

FaridAgl

Jos

FaridAgl

junkew

Melba23

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members

Browse

AutoIt Resources

Release

Beta