Jump to content
junkew

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

This example prints the items of the Windows Explorer listview in Scite console. The selected state of the items is printed too.

#include "..\Include\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

MainFunc()


Func MainFunc()

  ; Be sure to use the right class if you are on Vista or Windows 8
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]", "" )  ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]", "" ) ; Windows Explorer, Windows XP
  If Not $hWindow Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  If Not $pWindow Then Return

  Local $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return

  ListAllItems( $oWindow, 50007 ) ; 50007 = List view element
  ; Run the code in post #71 to get this value

EndFunc


Func ListAllItems( $oWindow, $iCtrlType )

  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $iCtrlType, $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $sel
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_SelectionItemIsSelectedPropertyId, $sel )
    ConsoleWrite( $name & "  " & $sel & @CRLF )
  Next

EndFunc

Hi,

I used this code to list all the files in a window. But I also want to list corresponding details like ModifiedTime, Size etc along with this. Can you let me know how I can retrieve those information?

Share this post


Link to post
Share on other sites

I certainly can. The subitems with the detail information are children of the row (parent). This function prints the name and value of all children to a parent:

Func ListChildren( $oParent )

  Local $pCondition
  $oUIAutomation.CreateTrueCondition( $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oParent.FindAll( $TreeScope_Children, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $value
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_ValueValuePropertyId, $value )
    If $name <> "" Then ConsoleWrite( "    " & $name & ": " & $value & @CRLF )
  Next

EndFunc
Note that I have copied most of the code.

Call the function like this just before the Next statement

ListChildren( $oUIElement )
Regards Lars.

Share this post


Link to post
Share on other sites

I certainly can. The subitems with the detail information are children of the row (parent). This function prints the name and value of all children to a parent:

Func ListChildren( $oParent )

  Local $pCondition
  $oUIAutomation.CreateTrueCondition( $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oParent.FindAll( $TreeScope_Children, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $value
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_ValueValuePropertyId, $value )
    If $name <> "" Then ConsoleWrite( "    " & $name & ": " & $value & @CRLF )
  Next

EndFunc
Note that I have copied most of the code.

Call the function like this just before the Next statement

ListChildren( $oUIElement )
Regards Lars.

 

Hi LarsJ,

I tried what you said but it just prints the name of the file and nothing else. Is there anything like UIA_NamePropertyId field for modifiedDate and size?

Edited by dilipped

Share this post


Link to post
Share on other sites

It works for me on Windows 7. Try the full code:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

MainFunc()


Func MainFunc()

  ; Be sure to use the right class if you are on Vista or Windows 8
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]", "" ) ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]", "" )  ; Windows Explorer, Windows XP
  If Not $hWindow Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  If Not $pWindow Then Return

  Local $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return

  ListAllItems( $oWindow, 50007 ) ; 50007 = List view element
  ; Run the code in post #71 to get this value

EndFunc


Func ListAllItems( $oWindow, $iCtrlType )

  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $iCtrlType, $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $sel
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_SelectionItemIsSelectedPropertyId, $sel )
    ConsoleWrite( $name & "  " & $sel & @CRLF )

    ListChildren( $oUIElement )
  Next

EndFunc


Func ListChildren( $oParent )

  Local $pCondition
  $oUIAutomation.CreateTrueCondition( $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oParent.FindAll( $TreeScope_Children, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $value
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_ValueValuePropertyId, $value )
    If $name <> "" Then ConsoleWrite( "    " & $name & ": " & $value & @CRLF )
  Next

EndFunc

Share this post


Link to post
Share on other sites

It works for me on Windows 7. Try the full code:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

MainFunc()


Func MainFunc()

  ; Be sure to use the right class if you are on Vista or Windows 8
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]", "" ) ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]", "" )  ; Windows Explorer, Windows XP
  If Not $hWindow Then Return

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  If Not $pWindow Then Return

  Local $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return

  ListAllItems( $oWindow, 50007 ) ; 50007 = List view element
  ; Run the code in post #71 to get this value

EndFunc


Func ListAllItems( $oWindow, $iCtrlType )

  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $iCtrlType, $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $sel
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_SelectionItemIsSelectedPropertyId, $sel )
    ConsoleWrite( $name & "  " & $sel & @CRLF )

    ListChildren( $oUIElement )
  Next

EndFunc


Func ListChildren( $oParent )

  Local $pCondition
  $oUIAutomation.CreateTrueCondition( $pCondition )
  If Not $pCondition Then Return

  Local $pUIElementArray, $oUIElementArray, $iElements
  $oParent.FindAll( $TreeScope_Children, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return

  Local $pUIElement, $oUIElement, $name, $value
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

    $oUIElement.GetCurrentPropertyValue( $UIA_NamePropertyId, $name )
    $oUIElement.GetCurrentPropertyValue( $UIA_ValueValuePropertyId, $value )
    If $name <> "" Then ConsoleWrite( "    " & $name & ": " & $value & @CRLF )
  Next

EndFunc

Great!! It works. Thanks a ton. I had made some changes while trying something out and forgot to change it back. That's the reason it did not work in the first go.

Share this post


Link to post
Share on other sites

@LarsJ.  Actually I started recently with Autoit and though I am comfortable with autoit I struggle a lot with coding using UIAutomation components. It would be helpful if you can show me a roadmap on how to start learning it. Some resources or links to tutorials would be helpful

Share this post


Link to post
Share on other sites

Share this post


Link to post
Share on other sites

Hello guys, i need someone to help me for accomplish a little task

I need to extract text ( just that, not automate ) from a "standard" Windows but this Windows as class DirectUIHWND, i have try many method but nothing work i can't read the text. Seaching for an alternative to Autoit Info Window i have found the "obsolete" UISpy and that was the only one that give me information about that control with the list of text i need to extract

I have use the "Simple spy demo" and i can access the "listview" but only the single "element", this is is what i get:

At least we have an element [name][Element]
Having the following values for all properties: 
Title is: <name>    Class   := <Element>    controltype:= <UIA_TextControlTypeId>   ,<50020>    , (0000C364)    
*** Parent Information ***
Title is: <listitem>    Class   := <Element>    controltype:= <UIA_ListItemControlTypeId>   ,<50007>    , (0000C357)    
*** Detailed properties of the highlighted element ***

The difference between the first item and the second are ( at left the first, at right the second )

UIA_BoundingRectanglePropertyId :=287;279;211;15 --- UIA_BoundingRectanglePropertyId :=287;327;64;15
UIA_ClickablePointPropertyId :=392;286 --- UIA_ClickablePointPropertyId :=319;334
UIA_HelpTextPropertyId :=Name of the item1 --- UIA_HelpTextPropertyId :=Name of the item2
UIA_LegacyIAccessibleHelpPropertyId :=Name of the item 1 --- UIA_LegacyIAccessibleHelpPropertyId :=Name of the item 2
UIA_LegacyIAccessibleValuePropertyId :=Name of the item 1 --- UIA_LegacyIAccessibleValuePropertyId :=Name of the item 2
UIA_RuntimeIdPropertyId :=1336;138542016;0 --- UIA_RuntimeIdPropertyId :=1336;138543168;0
UIA_ValueValuePropertyId :=Name of the item 1 --- UIA_ValueValuePropertyId :=Name of the item 2

Ovb i need to extract all the name list like:

Name of the item 1

Name of the item 2

etc.

The name can be different and i don't know the number of the elements of the list

A complete list of the first element:

Mouse position is retrieved 416-289
At least we have an element [name][Element]
Having the following values for all properties: 
Title is: <name>    Class   := <Element>    controltype:= <UIA_TextControlTypeId>   ,<50020>    , (0000C364)    
*** Parent Information ***
Title is: <listitem>    Class   := <Element>    controltype:= <UIA_ListItemControlTypeId>   ,<50007>    , (0000C357)    
*** Detailed properties of the highlighted element ***
UIA_AcceleratorKeyPropertyId :=
UIA_AccessKeyPropertyId :=
UIA_AriaPropertiesPropertyId :=
UIA_AriaRolePropertyId :=
UIA_AutomationIdPropertyId :=listitem_name
UIA_BoundingRectanglePropertyId :=287;279;211;15
UIA_ClassNamePropertyId :=Element
UIA_ClickablePointPropertyId :=392;286
UIA_ControllerForPropertyId :=
UIA_ControlTypePropertyId :=50020
UIA_CulturePropertyId :=0
UIA_DescribedByPropertyId :=
UIA_DockDockPositionPropertyId :=5
UIA_ExpandCollapseExpandCollapseStatePropertyId :=3
UIA_FlowsToPropertyId :=
UIA_FrameworkIdPropertyId :=DirectUI
UIA_GridColumnCountPropertyId :=0
UIA_GridItemColumnPropertyId :=0
UIA_GridItemColumnSpanPropertyId :=1
UIA_GridItemContainingGridPropertyId :=
UIA_GridItemRowPropertyId :=0
UIA_GridItemRowSpanPropertyId :=1
UIA_GridRowCountPropertyId :=0
UIA_HasKeyboardFocusPropertyId :=False
UIA_HelpTextPropertyId :=Name of the item 1
UIA_IsContentElementPropertyId :=True
UIA_IsControlElementPropertyId :=True
UIA_IsDataValidForFormPropertyId :=False
UIA_IsDockPatternAvailablePropertyId :=False
UIA_IsEnabledPropertyId :=True
UIA_IsExpandCollapsePatternAvailablePropertyId :=False
UIA_IsGridItemPatternAvailablePropertyId :=False
UIA_IsGridPatternAvailablePropertyId :=False
UIA_IsInvokePatternAvailablePropertyId :=False
UIA_IsItemContainerPatternAvailablePropertyId :=False
UIA_IsKeyboardFocusablePropertyId :=False
UIA_IsLegacyIAccessiblePatternAvailablePropertyId :=True
UIA_IsMultipleViewPatternAvailablePropertyId :=False
UIA_IsOffscreenPropertyId :=False
UIA_IsPasswordPropertyId :=False
UIA_IsRangeValuePatternAvailablePropertyId :=False
UIA_IsRequiredForFormPropertyId :=False
UIA_IsScrollItemPatternAvailablePropertyId :=True
UIA_IsScrollPatternAvailablePropertyId :=False
UIA_IsSelectionItemPatternAvailablePropertyId :=False
UIA_IsSelectionPatternAvailablePropertyId :=False
UIA_IsSynchronizedInputPatternAvailablePropertyId :=False
UIA_IsTableItemPatternAvailablePropertyId :=False
UIA_IsTablePatternAvailablePropertyId :=False
UIA_IsTextPatternAvailablePropertyId :=False
UIA_IsTogglePatternAvailablePropertyId :=False
UIA_IsTransformPatternAvailablePropertyId :=False
UIA_IsValuePatternAvailablePropertyId :=True
UIA_IsVirtualizedItemPatternAvailablePropertyId :=False
UIA_IsWindowPatternAvailablePropertyId :=False
UIA_ItemStatusPropertyId :=
UIA_ItemTypePropertyId :=
UIA_LabeledByPropertyId :=
UIA_LegacyIAccessibleChildIdPropertyId :=0
UIA_LegacyIAccessibleDefaultActionPropertyId :=
UIA_LegacyIAccessibleDescriptionPropertyId :=
UIA_LegacyIAccessibleHelpPropertyId :=Name of the item 1
UIA_LegacyIAccessibleKeyboardShortcutPropertyId :=
UIA_LegacyIAccessibleNamePropertyId :=name
UIA_LegacyIAccessibleRolePropertyId :=41
UIA_LegacyIAccessibleSelectionPropertyId :=
UIA_LegacyIAccessibleStatePropertyId :=64
UIA_LegacyIAccessibleValuePropertyId :=Name of the item 1
UIA_LocalizedControlTypePropertyId :=testo
UIA_MultipleViewCurrentViewPropertyId :=0
UIA_MultipleViewSupportedViewsPropertyId :=
UIA_NamePropertyId :=name
UIA_NativeWindowHandlePropertyId :=0
UIA_OrientationPropertyId :=0
UIA_ProcessIdPropertyId :=1336
UIA_ProviderDescriptionPropertyId :=[pid:1336,hwnd:0x0 Main(parent link):Unidentified Provider (unmanaged:DUI70.dll)]
UIA_RangeValueIsReadOnlyPropertyId :=True
UIA_RangeValueLargeChangePropertyId :=0
UIA_RangeValueMaximumPropertyId :=0
UIA_RangeValueMinimumPropertyId :=0
UIA_RangeValueSmallChangePropertyId :=0
UIA_RangeValueValuePropertyId :=0
UIA_RuntimeIdPropertyId :=1336;138542016;0
UIA_ScrollHorizontallyScrollablePropertyId :=False
UIA_ScrollHorizontalScrollPercentPropertyId :=0
UIA_ScrollHorizontalViewSizePropertyId :=100
UIA_ScrollVerticallyScrollablePropertyId :=False
UIA_ScrollVerticalScrollPercentPropertyId :=0
UIA_ScrollVerticalViewSizePropertyId :=100
UIA_SelectionCanSelectMultiplePropertyId :=False
UIA_SelectionIsSelectionRequiredPropertyId :=False
UIA_SelectionselectionPropertyId :=
UIA_SelectionItemIsSelectedPropertyId :=False
UIA_SelectionItemSelectionContainerPropertyId :=
UIA_TableColumnHeadersPropertyId :=
UIA_TableItemColumnHeaderItemsPropertyId :=
UIA_TableRowHeadersPropertyId :=
UIA_TableRowOrColumnMajorPropertyId :=2
UIA_TableItemRowHeaderItemsPropertyId :=
UIA_ToggleToggleStatePropertyId :=2
UIA_TransformCanMovePropertyId :=False
UIA_TransformCanResizePropertyId :=False
UIA_TransformCanRotatePropertyId :=False
UIA_ValueIsReadOnlyPropertyId :=True
UIA_ValueValuePropertyId :=Name of the item 1
UIA_WindowCanMaximizePropertyId :=False
UIA_WindowCanMinimizePropertyId :=False
UIA_WindowIsModalPropertyId :=False
UIA_WindowIsTopmostPropertyId :=False
UIA_WindowWindowInteractionStatePropertyId :=0
UIA_WindowWindowVisualStatePropertyId :=0

I think i have say it all but if need more information i'll post it. Thanks for the big hand :D

P.S. Just for know, the UISPy tool say there is a:

  1. list "scrollviewer"

Under that two element:

  • scrollbar "Vertical"
  • list "listview"

Under the "listview" there are items called "listitem", so pratically all the list i need

Edited by MyEarth

Share this post


Link to post
Share on other sites

Try this. The code should print all listview items:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

MainFunc()


Func MainFunc()

  ; Get window handle
  Local $hWindow = WinGetHandle( "Write something here to identify your window" )
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object ERR" & @CRLF )
  ConsoleWrite( "UI Automation object OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  ; Condition to find listview items
  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ListItemControlTypeId, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find listview items
  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return ConsoleWrite( "Find listview items ERR" & @CRLF )
  ConsoleWrite( "Find listview items OK" & @CRLF )

  ; Print listview items
  Local $pUIElement, $oUIElement, $value
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $oUIElement.GetCurrentPropertyValue( $UIA_ValueValuePropertyId, $value )
    ConsoleWrite( $value & @CRLF )
  Next

EndFunc

Share this post


Link to post
Share on other sites

Thanks Lars, i'll try it very soon. I have a question, is true that the "capture" using UIFramework work only if the window is visible? Long time ago i have try one of the example but if the windows was @SW_HIDE or minimized nothing happened. If is true, there is a "workaround" for have the window not visible to the user and "capture" the information? Thanks again ;)

Share this post


Link to post
Share on other sites

It's true, that there are some issues with minimized or hidden windows. If you test the code in post 212 with the listview in Windows Explorer, it'll not print any files/folders, if the window is minimized. I don't know if there is a workaround. May be junkew has a solution.

You can try to do a test with Inspect.exe (in Windows SDK). If Inspect.exe can get the information, it should also be possible to get the information with this UDF.

Share this post


Link to post
Share on other sites

This UDF is absolutely amazing...

Is there anyway to extract the HTML document into a variable with this?


Things that I've done..

Icon Resource Editor: icon resource editor 

AutoIt Piano: a piano

AutoIt Unlocker: unlocks files when you want to delete them

Colorful tooltips: a wrapper for the tool tips UDF

Rouge GoogleBot: a full screen animation

ASciTE text editor: a text editor written in autoit

Warning: Posts by this user are subject to change or may disappear without notice.

Share this post


Link to post
Share on other sites

no, the only way to do that is to switch to a native component for the specific browsers

yes, indirectly by opening the menu/view to view the html source or navigate thru the development tooling menu with UIA actions

Share this post


Link to post
Share on other sites

@junkew , @LarsJ

Great work, in years of work with AutoIt still never tried using IUIAutomation.

I tried to mesh around the code but have still some difficulties on it.

Mainly i'm getting too many errors like:

F:\AUTO-IT Projects\___2.0 __\ex\ex2_Taskbar.au3 (26) : ==> Variable must be of type "Object".:

$oStart.getCurrentPattern($UIA_InvokePatternId, $pInvoke)

$oStart^ ERROR

->11:29:39 AutoIT3.exe ended.rc:1

anyway, I started looking at the code of you first example, spy_demo.au3 and understood what I need to find and where to find.

I need just a "kickstart":

In the spy demo your starting point is the mouse position used as starting point to find/evaluate controls.

    $x = MouseGetPos(0)

    $y = MouseGetPos(1)

    DllStructSetData($tStruct, "x", $x)

    DllStructSetData($tStruct, "y", $y)

    ConsoleWrite(DllStructGetData($tStruct, "x") & DllStructGetData($tStruct, "y"))



;~ consolewrite("Mouse position is retrieved " & @crlf)

    $UIA_oUIAutomation.ElementFromPoint($tStruct, $UIA_pUIElement)

Instead I need to know whether a condition occurs or not and "do things" in case it occurs.

I.e. I want to dismiss a pop-up if it occurs but I don't know where or when it occurs.

Thanks to your spy demo i realized that the control is given by

Mouse position is retrieved 813-427

At least we have an element [Close][Button]

Having the following values for all properties:

Title is: <Close>    Class   := <Button>    controltype:= <UIA_ButtonControlTypeId>    ,<50000>    , (0000C350)    

*** Parent Information ***

Title is: <Shiny.Chat.ViewModels.PrivateChatSessionViewModel>    Class   := <ToolWindow>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    

*** Detailed properties of the highlighted element ***

UIA_AcceleratorKeyPropertyId :=Close

UIA_AccessKeyPropertyId :=

UIA_AriaPropertiesPropertyId :=

UIA_AriaRolePropertyId :=

UIA_AutomationIdPropertyId :=CloseButtom

UIA_BoundingRectanglePropertyId :=802;414;24;24

UIA_ClassNamePropertyId :=Button

UIA_ClickablePointPropertyId :=814;426

what I would like to do is create a function

AdlibRegister("_dismiss", 1000)

that checks the following:

if Class = Button and Title = Close and Title is <Shiny.Chat.ViewModels.PrivateChatSessionViewModel>   then
  close the popup
endif

To close popup i could use

mouseclick("left",UIA_ClickablePointPropertyId :=814;426)

or maybe you can suggest me a solution without mouseclicks.

Thanks, hope you can help me with this.

Best regards,

Marco

Edited by marko001

Share this post


Link to post
Share on other sites

you have to download the zip with the examples. example code in the thread will not work due to some renaming of variables.

 

Variable must be of type "Object" message you just get when you did not have a valid object based on the search criteria given

 

There are different ways of doing this (easy and hard)

a. you just have a loop and every nnn milliseconds you check for your popup which is more or less what you are doing

b. use the event examples that tell you when there is a new window appearing examples 19-24

Share this post


Link to post
Share on other sites

I always worked with loops, I know it's the easiest way, but honestly not the best from a programmer's point of view.

Can you give me some hints on B) way?

Thanks a lot mate, meanwhile I'll study the codes.

Marco

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object. Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?
×
×
  • Create New...