junkew

IUIAutomation MS framework automate chrome, FF, IE, ....

563 posts in this topic

It starts with spying on the windows to get the basic information. See first thread on a basic way of spying

I was unable to install this AV program as it gives an installation error so only can help you high level

0. It helps if you install inspect.exe (in windows SDK), not fully needed

1. Make a const reference to your main ADAware window

;~ ClassName: "GuiFramework::HtmlView" Name: "Ad-Aware Antivirus Installation"
$cADName="name:=Ad*"

2. Get the Element reference

$oAD=_UIA_getFirstObjectOfElement($oDesktop, $cADName, $treescope_children)
if not isobj($oAD) Then
findThemAll($oDesktop,$treescope_children)
EndIf

3. Debug, if it is not fully working but you get the main window you could do this to get all elements in the console window shown

findThemAll($oAD,$treescope_subtree)

Share this post


Link to post
Share on other sites



#42 ·  Posted (edited)

Example 9 Windows media player

 

Updated UIAWrappers extensively to abstract more from the (complex) MS UIA framework

#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <debug.au3>
#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"
HotKeySet("{ESC}", "Terminate")

;~ Turn debugging UIA on by default, dumps most details to consolewindow for the moment, later will use a logfile
;~ _UIA_setVar("Global.Debug",true)
;~ _UIA_setVar("Global.Highlight",true) ;- Highlights object when found

;~ Set the system under test variables
_UIA_setVar("SUT1.Folder","C:\Program Files (x86)\Windows Media Player")
_UIA_setVar("SUT1.Workingdir","C:\Program Files (x86)\Windows Media Player\")
_UIA_setVar("SUT1.EXE","wmplayer.exe")
_UIA_setVar("SUT1.Fullname", _UIA_getVar("SUT1.Folder") & "\"& _UIA_getVar("SUT1.EXE"))
_UIA_setVar("SUT1.Parameters","")
_UIA_setVar("SUT1.Processname","wmplayer.exe")
_UIA_setVar("SUT1.Windowstate",@SW_RESTORE)

_UIA_setVar("SUT1.Fullname", _UIA_getVar("SUT1.Folder") & "\"& _UIA_getVar("SUT1.EXE"))
_UIA_setVar("SUT1.Parameters","")
_UIA_setVar("SUT1.Processname","wmplayer.exe")

;~ Set the system under test objects to recognize
_UIA_setVar("WMP.mainwindow","classname:=WMPlayerApp")
_UIA_setVar("WMP.Playlists","classname:=SysTreeView32")
_UIA_setVar("WMP.Firstlist","name:=(Afspeellijst.*)|(Playlist.*)")
_UIA_setVar("WMP.Playbutton","name:=playGlyph")
_UIA_setVar("WMP.Volumebutton","name:=Volume")

;~ Start system under test
_UIA_DEBUG("Starting system under test" & @CRLF)
_UIA_StartSUT("SUT1")

;~ Get the main WMP element
;~ _UIA_DEBUG("Action 1 Finding main window" & @CRLF)
;~ $oWMP=_UIA_getFirstObjectOfElement($oDesktop, _UIA_getVar("WMP.mainwindow"), $treescope_children)
;~ $oWMP.setfocus()

;~ Action on the logical object as defined
_UIA_action("WMP.MAINWINDOW","setfocus")
;~ Retrieve the actual object from the runtime type information
$oWMP=_UIA_getVar("RTI.WMP.mainwindow")

;~ Get the playlists
;~ _UIA_DEBUG("Action 2 Finding playlists" & @CRLF)
$oPL=_UIA_getFirstObjectOfElement($oWMP, _UIA_getVar("WMP.playlists"), $treescope_subtree)
$oFirstPL=_UIA_getObjectByFindAll($oPL, _UIA_getVar("WMP.Firstlist"), $treescope_subtree)

;~ See if its expanded/collapsed
_UIA_DEBUG("Action 3 Expanding if needed" & @CRLF)
dim $state
;~ Get the pattern
$oExpandP=_UIA_getpattern($oFirstPL,$UIA_ExpandCollapsePatternId)
if isobj($oExpandP) Then
;~ Get the state
;~ Global Const $ExpandCollapseState_Collapsed=0
;~ Global Const $ExpandCollapseState_Expanded=1
;~ Global Const $ExpandCollapseState_PartiallyExpanded=2
;~ Global Const $ExpandCollapseState_LeafNode=3
    $oExpandP.CurrentExpandCollapseState($state)
    if $state=$ExpandCollapseState_Collapsed Then
        _UIA_action("WMP.Firstlist","leftdoubleclick")
    EndIf
endif

;~ _UIA_DEBUG("Action 4 Starting with the playbutton" & @CRLF)
;~ $oPlayButton=_UIA_getObjectByFindAll($oWMP, _UIA_getVar("WMP.Playbutton"), $treescope_subtree)
_UIA_action("WMP.Playbutton","left")

_UIA_DEBUG("Action 5 And finding the volume" & @CRLF)
$oVolumeButton=_UIA_getObjectByFindAll($oWMP, _UIA_getVar("WMP.Volumebutton"), $treescope_subtree)
$oValueP=_UIA_getpattern($oVolumeButton,$UIA_ValuePatternId)

if isobj($oValueP) Then
dim $volume
    $oValuep.currentvalue($volume)
    _UIA_Debug("Volume is " & $volume & @CRLF)
EndIf

; The End
Func Terminate()
    consolewrite("Exiting")
    $running=false
;~  Exit 0
EndFunc   ;==>Terminate
Edited by junkew

Share this post


Link to post
Share on other sites

junkew,

You are  a wizzard! :sorcerer:

I just started exploring this, but it is AWESOME :thumbsup:

Share this post


Link to post
Share on other sites

#44 ·  Posted (edited)

I know this is not much, but I am trying to contribute what i can to this.

Modified the _UIA_action function as follows:

  • Merged all the mouse clicking cases into one and added a right click option
  • Removed "MouseMove" commands since "MouseClick" can move AND click mouse
  • made $p1 = number of clicks, $p2 = speed of mousemove and $p3 = number of ms to sleep at end of func
Func _UIA_action($obj, $strAction, $p1 = 0, $p2 = 0, $p3 = 0)
    Local $tPattern, $t, $x, $y, $mouseButton = "left"
    $strAction = StringLower($strAction)
    Switch $strAction
        Case "leftclick", "left", "rightclick", "right"
            $t = StringSplit(_UIA_getPropertyValue($obj, $UIA_BoundingRectanglePropertyId),";")
            $x = $t[1] + ($t[3] / 2)
            $y = $t[2] + ($t[4] / 2)
            If StringInStr($strAction, "right") Then $mouseButton = "right"
            If $p1 = 0 Then $p1 = 1
            MouseClick($mouseButton, $x, $y, $p1, $p2)
            If $p3 > 0 Then Sleep($p3)
        Case "setvalue using keys"
            $obj.setfocus
            Send("^a")
            Send($p1)
            If $p2 > 0 Then Sleep($p2)
        Case "invoke"
            $tPattern=_UIA_getPattern($obj,$UIA_InvokePatternId)
            $tPattern.invoke()
        Case "focus"
            $obj.setfocus()
        Case Else
    EndSwitch
EndFunc

edit: improvements

Edited by Shrapnel

Share this post


Link to post
Share on other sites

#45 ·  Posted (edited)

Based on the request in this thread I enhanced the UIAwrappers more extensively to make life much easier

'?do=embed' frameborder='0' data-embedContent>>

http://www.machsupport.com/software/mach3/

Example 10 Automating mach 3 (AFX windows and other hard to get recognized by AutoIT)

#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <debug.au3>
#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"
HotKeySet("{ESC}", "Terminate")

;~ Turn debugging UIA on by default, dumps most details to consolewindow for the moment, later will use a logfile
;~ _UIA_setVar("Global.Debug",true)
;~ _UIA_setVar("Global.Highlight",true) ;- Highlights object when found

;~ Set the system under test variables
_UIA_setVar("SUT1.Folder","C:\Mach3")
_UIA_setVar("SUT1.Workingdir","C:\Mach3\")
_UIA_setVar("SUT1.EXE","Mach3.exe")
_UIA_setVar("SUT1.Fullname", _UIA_getVar("SUT1.Folder") & "\"& _UIA_getVar("SUT1.EXE"))
_UIA_setVar("SUT1.Parameters","/p Mach3Mill")
_UIA_setVar("SUT1.Processname","mach3.exe")
_UIA_setVar("SUT1.Windowstate",@SW_RESTORE)

_UIA_setVar("SUT1.Fullname", _UIA_getVar("SUT1.Folder") & "\"& _UIA_getVar("SUT1.EXE"))

;~ Set the system under test objects to recognize and abstract logical and physical names for readability in main script

_UIA_setVar("MACH3.mainwindow","classname:=Afx:00400000:b:00010003:00000006.*")
;~ _UIA_setVar("MACH3.mainwindow","name:=Mach3 CNC  Demo index:=1")

_UIA_setVar("MACH3.editgcode","name:=Edit G-Code")
_UIA_setVar("MACH3.btnCyclestart","automationid:=8142")
_UIA_setVar("MACH3.btnStop","automationid:=8140")
_UIA_setVar("MACH3.btnRewind","automationid:=8295")

_UIA_setVar("MACH3.mnuFile","name:=((File)|(Bestand)); index:=2")
_UIA_setVar("MACH3.mnuOpen","name:=Load.*")

_UIA_setVar("MACH3.dlgOpen","name:=Open*")
_UIA_setVar("MACH3.dlgOpen.edtFilename","name:=((Filename:)|(Bestandsnaam:)); indexrelative:=2")
_UIA_setVar("MACH3.dlgOpen.btnOpen","name:=Open* index:=6")

_UIA_setVar("NOTEPAD.mainwindow","classname:=Notepad")
_UIA_setVar("NOTEPAD.edit","classname:=Edit")
_UIA_setVar("NOTEPAD.mnuFile","name:=((File)|(Bestand))")
_UIA_setVar("NOTEPAD.mnuSave","name:=((Save.*)|(Opslaan.*))")
_UIA_setVar("NOTEPAD.btnClose","name:=((Close)|(Sluiten))")

_UIA_setVar("NOTEPAD.dlgSave","name:=((Save as.*)|(Opslaan als.*))")
_UIA_setVar("NOTEPAD.dlgSave.edtFilename","name:=((Filename:)|(Bestandsnaam:)); indexrelative:=1")
_UIA_setVar("NOTEPAD.dlgSave.btnSave","name:=((Save)|(Opslaan)) index:=4")
_UIA_setVar("NOTEPAD.dlgSave.btnYes","name:=((Yes)|(Ja))")

;~ Start system under test
_UIA_DEBUG("Starting system under test" & @CRLF)
_UIA_StartSUT("SUT1")
_UIA_DEBUG("SUT is started" & @CRLF)

_UIA_DEBUG("PID of SUT1 is " & _UIA_getVar("SUT1.PID") & @CRLF)

;~ Main script
_UIA_DEBUG("*** Main script started ***" & @CRLF)
_UIA_action("MACH3.mainwindow","focus")
_UIA_action("MACH3.editgCode", "left")

_UIA_action("NOTEPAD.mainwindow","focus")
_UIA_action("NOTEPAD.edit","sendkeys","f600{ENTER}g1 Z-2{ENTER}g1 y-13.5{ENTER}g1 z-14{ENTER}g1 z-26{ENTER}g1 z-38{ENTER}g1 y-35{ENTER}")
_UIA_action("NOTEPAD.mnuFile","left")
_UIA_action("NOTEPAD.mnuSave","left")
_UIA_action("NOTEPAD.dlgSave.edtFilename","sendkeys","demofile.txt")
_UIA_action("NOTEPAD.dlgSave.btnSave","left")
_UIA_action("NOTEPAD.dlgSave.btnYes","left")
_UIA_action("NOTEPAD.btnClose","left")

_UIA_action("MACH3.mnuFile","left")
_UIA_action("MACH3.mnuOpen","left")
_UIA_action("MACH3.dlgOpen.edtFilename","sendkeys","demofile.txt")
_UIA_action("MACH3.dlgOpen.btnOpen","left")

sleep(3000) ;~ Just 3 seconds to be able to load the script

_UIA_action("MACH3.btnStop","left")
_UIA_action("MACH3.btnRewind","left")
_UIA_action("MACH3.btnCycleStart","left")

; The End
Func Terminate()
    consolewrite("Exiting")
    $running=false
;~  Exit 0
EndFunc   ;==>Terminate
Edited by junkew

Share this post


Link to post
Share on other sites

junkew,

Excellent work. Thanks for all the effort you have put (and are still putting) into this. :thumbsup:

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

The application I am trying to automate has Data Grid View controls and your spy tool is able to identify them, but I am unsure how to loop through all of the rows/columns and put the data into a 2d array.

I will be attempting to code this myself, but I was hoping someone may have already figured this out, or at least have a good idea on how to go about accomplishing this.

Anyways, here is the spy information of the Data Grid View control itself, and the spy information of one of the cells of the Data Grid view control (items that had no value were removed)

Data Grid View:

Mouse position is retrieved 900-402
At least we have an element [DataGridView][WindowsForms10.Window.8.app.0.1a6f050_r13_ad1]
Having the following values for all properties: 
Title is: <DataGridView>    Class   := <WindowsForms10.Window.8.app.0.1a6f050_r13_ad1>  controltype:= <UIA_TableControlTypeId>  ,<50036>    , (0000C374)    

UIA_AutomationIdPropertyId :=dgvWIP
UIA_BoundingRectanglePropertyId :=283;402;1354;372
UIA_ClassNamePropertyId :=WindowsForms10.Window.8.app.0.1a6f050_r13_ad1
UIA_ControlTypePropertyId :=50036
UIA_CulturePropertyId :=0
UIA_DockDockPositionPropertyId :=5
UIA_ExpandCollapseExpandCollapseStatePropertyId :=3
UIA_FrameworkIdPropertyId :=WinForm
UIA_GridColumnCountPropertyId :=0
UIA_GridItemColumnPropertyId :=0
UIA_GridItemColumnSpanPropertyId :=1
UIA_GridItemRowPropertyId :=0
UIA_GridItemRowSpanPropertyId :=1
UIA_GridRowCountPropertyId :=0
UIA_HasKeyboardFocusPropertyId :=True
UIA_IsContentElementPropertyId :=True
UIA_IsControlElementPropertyId :=True
UIA_IsDataValidForFormPropertyId :=False
UIA_IsDockPatternAvailablePropertyId :=False
UIA_IsEnabledPropertyId :=True
UIA_IsExpandCollapsePatternAvailablePropertyId :=False
UIA_IsGridItemPatternAvailablePropertyId :=False
UIA_IsGridPatternAvailablePropertyId :=False
UIA_IsInvokePatternAvailablePropertyId :=False
UIA_IsItemContainerPatternAvailablePropertyId :=False
UIA_IsKeyboardFocusablePropertyId :=True
UIA_IsLegacyIAccessiblePatternAvailablePropertyId :=True
UIA_IsMultipleViewPatternAvailablePropertyId :=False
UIA_IsOffscreenPropertyId :=False
UIA_IsPasswordPropertyId :=False
UIA_IsRangeValuePatternAvailablePropertyId :=False
UIA_IsRequiredForFormPropertyId :=False
UIA_IsScrollItemPatternAvailablePropertyId :=False
UIA_IsScrollPatternAvailablePropertyId :=False
UIA_IsSelectionItemPatternAvailablePropertyId :=False
UIA_IsSelectionPatternAvailablePropertyId :=False
UIA_IsSynchronizedInputPatternAvailablePropertyId :=False
UIA_IsTableItemPatternAvailablePropertyId :=False
UIA_IsTablePatternAvailablePropertyId :=False
UIA_IsTextPatternAvailablePropertyId :=False
UIA_IsTogglePatternAvailablePropertyId :=False
UIA_IsTransformPatternAvailablePropertyId :=False
UIA_IsValuePatternAvailablePropertyId :=False
UIA_IsVirtualizedItemPatternAvailablePropertyId :=False
UIA_IsWindowPatternAvailablePropertyId :=False
UIA_LegacyIAccessibleChildIdPropertyId :=0
UIA_LegacyIAccessibleNamePropertyId :=DataGridView
UIA_LegacyIAccessibleRolePropertyId :=24
UIA_LegacyIAccessibleStatePropertyId :=1048580
UIA_LocalizedControlTypePropertyId :=table
UIA_MultipleViewCurrentViewPropertyId :=0
UIA_NamePropertyId :=DataGridView
UIA_NativeWindowHandlePropertyId :=1179694
UIA_OrientationPropertyId :=0
UIA_ProcessIdPropertyId :=8912
UIA_ProviderDescriptionPropertyId :=[pid:148,hwnd:0x12002E Main:Nested [pid:8912,hwnd:0x12002E Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]
UIA_RangeValueIsReadOnlyPropertyId :=True
UIA_RangeValueLargeChangePropertyId :=0
UIA_RangeValueMaximumPropertyId :=0
UIA_RangeValueMinimumPropertyId :=0
UIA_RangeValueSmallChangePropertyId :=0
UIA_RangeValueValuePropertyId :=0
UIA_RuntimeIdPropertyId :=42;1179694
UIA_ScrollHorizontallyScrollablePropertyId :=False
UIA_ScrollHorizontalScrollPercentPropertyId :=0
UIA_ScrollHorizontalViewSizePropertyId :=100
UIA_ScrollVerticallyScrollablePropertyId :=False
UIA_ScrollVerticalScrollPercentPropertyId :=0
UIA_ScrollVerticalViewSizePropertyId :=100
UIA_SelectionCanSelectMultiplePropertyId :=False
UIA_SelectionIsSelectionRequiredPropertyId :=False
UIA_SelectionItemIsSelectedPropertyId :=False
UIA_TableRowOrColumnMajorPropertyId :=2
UIA_ToggleToggleStatePropertyId :=2
UIA_TransformCanMovePropertyId :=False
UIA_TransformCanResizePropertyId :=False
UIA_TransformCanRotatePropertyId :=False
UIA_ValueIsReadOnlyPropertyId :=True
UIA_WindowCanMaximizePropertyId :=False
UIA_WindowCanMinimizePropertyId :=False
UIA_WindowIsModalPropertyId :=False
UIA_WindowIsTopmostPropertyId :=False
UIA_WindowWindowInteractionStatePropertyId :=0
UIA_WindowWindowVisualStatePropertyId :=0

Data Grid View Cell:

Mouse position is retrieved 400-461
At least we have an element [OrderNo Row 0][]
Having the following values for all properties: 
Title is: <OrderNo Row 0>   Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    

UIA_BoundingRectanglePropertyId :=361;448;70;22
UIA_ControlTypePropertyId :=50025
UIA_CulturePropertyId :=0
UIA_DockDockPositionPropertyId :=5
UIA_ExpandCollapseExpandCollapseStatePropertyId :=3
UIA_GridColumnCountPropertyId :=0
UIA_GridItemColumnPropertyId :=0
UIA_GridItemColumnSpanPropertyId :=1
UIA_GridItemRowPropertyId :=0
UIA_GridItemRowSpanPropertyId :=1
UIA_GridRowCountPropertyId :=0
UIA_HasKeyboardFocusPropertyId :=True
UIA_HelpTextPropertyId :=DataGridViewTextBoxCell(DataGridViewCell)
UIA_IsContentElementPropertyId :=True
UIA_IsControlElementPropertyId :=True
UIA_IsDataValidForFormPropertyId :=False
UIA_IsDockPatternAvailablePropertyId :=False
UIA_IsEnabledPropertyId :=True
UIA_IsExpandCollapsePatternAvailablePropertyId :=False
UIA_IsGridItemPatternAvailablePropertyId :=False
UIA_IsGridPatternAvailablePropertyId :=False
UIA_IsInvokePatternAvailablePropertyId :=False
UIA_IsItemContainerPatternAvailablePropertyId :=False
UIA_IsKeyboardFocusablePropertyId :=True
UIA_IsLegacyIAccessiblePatternAvailablePropertyId :=True
UIA_IsMultipleViewPatternAvailablePropertyId :=False
UIA_IsOffscreenPropertyId :=False
UIA_IsPasswordPropertyId :=False
UIA_IsRangeValuePatternAvailablePropertyId :=False
UIA_IsRequiredForFormPropertyId :=False
UIA_IsScrollItemPatternAvailablePropertyId :=False
UIA_IsScrollPatternAvailablePropertyId :=False
UIA_IsSelectionItemPatternAvailablePropertyId :=False
UIA_IsSelectionPatternAvailablePropertyId :=False
UIA_IsSynchronizedInputPatternAvailablePropertyId :=False
UIA_IsTableItemPatternAvailablePropertyId :=False
UIA_IsTablePatternAvailablePropertyId :=False
UIA_IsTextPatternAvailablePropertyId :=False
UIA_IsTogglePatternAvailablePropertyId :=False
UIA_IsTransformPatternAvailablePropertyId :=False
UIA_IsValuePatternAvailablePropertyId :=False
UIA_IsVirtualizedItemPatternAvailablePropertyId :=False
UIA_IsWindowPatternAvailablePropertyId :=False
UIA_LegacyIAccessibleChildIdPropertyId :=0
UIA_LegacyIAccessibleHelpPropertyId :=DataGridViewTextBoxCell(DataGridViewCell)
UIA_LegacyIAccessibleNamePropertyId :=OrderNo Row 0
UIA_LegacyIAccessibleRolePropertyId :=29
UIA_LegacyIAccessibleStatePropertyId :=3145734
UIA_LegacyIAccessibleValuePropertyId :=580902
UIA_MultipleViewCurrentViewPropertyId :=0
UIA_NamePropertyId :=OrderNo Row 0
UIA_NativeWindowHandlePropertyId :=0
UIA_OrientationPropertyId :=0
UIA_ProcessIdPropertyId :=8012
UIA_ProviderDescriptionPropertyId :=[pid:8012,hwnd:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]
UIA_RangeValueIsReadOnlyPropertyId :=True
UIA_RangeValueLargeChangePropertyId :=0
UIA_RangeValueMaximumPropertyId :=0
UIA_RangeValueMinimumPropertyId :=0
UIA_RangeValueSmallChangePropertyId :=0
UIA_RangeValueValuePropertyId :=0
UIA_ScrollHorizontallyScrollablePropertyId :=False
UIA_ScrollHorizontalScrollPercentPropertyId :=0
UIA_ScrollHorizontalViewSizePropertyId :=100
UIA_ScrollVerticallyScrollablePropertyId :=False
UIA_ScrollVerticalScrollPercentPropertyId :=0
UIA_ScrollVerticalViewSizePropertyId :=100
UIA_SelectionCanSelectMultiplePropertyId :=False
UIA_SelectionIsSelectionRequiredPropertyId :=False
UIA_SelectionItemIsSelectedPropertyId :=False
UIA_TableRowOrColumnMajorPropertyId :=2
UIA_ToggleToggleStatePropertyId :=2
UIA_TransformCanMovePropertyId :=False
UIA_TransformCanResizePropertyId :=False
UIA_TransformCanRotatePropertyId :=False
UIA_ValueIsReadOnlyPropertyId :=True
UIA_WindowCanMaximizePropertyId :=False
UIA_WindowCanMinimizePropertyId :=False
UIA_WindowIsModalPropertyId :=False
UIA_WindowIsTopmostPropertyId :=False
UIA_WindowWindowInteractionStatePropertyId :=0
UIA_WindowWindowVisualStatePropertyId :=0

Share this post


Link to post
Share on other sites

#48 ·  Posted (edited)

First of all within Windows UI Automation documentation is stated that those patterns are available for windows explorer

Grid Used for controls that support grid functionality such as sizing and moving to a specified cell, without header information. The “large icon view” in Windows® Explorer is an example of a control that follows the Grid control pattern. GridItem Used for controls within grids. For example, each cell in Explorer’s “details view” could follow the GridItem pattern.

So steps are

1. Get a UI Element object

2. Check the Is.....Pattern property

Seems to be false for your objects so no detailed pattern available that abstracts rows/cells (weird anyway as it seems just to be a WPF control)

UIA_IsGridItemPatternAvailablePropertyId :=False 
UIA_IsGridPatternAvailablePropertyId :=False

So to determine the way to handle your grid I suggest do do the following

1. Get your main UI element that contains the grid

2. _UIA_DumpThemAll(<UI Reference>, $treescope_subtree)

Then you can see the logic of the whole tree and build your own logic to iterate over the row's

see example 1 on the treewalkers on how to iterate thru the three

to work with /get  the gridpattern replace the _UIA_getPattern with this function (will be put later in a new version of UIAWrapper UDF)

func _UIA_getPattern($obj,$patternID)
local $patternArray[21][3]=[ _
    [$UIA_ValuePatternId    ,           $sIID_IUIAutomationValuePattern    ,        $dtagIUIAutomationValuePattern], _
    [$UIA_InvokePatternId   ,           $sIID_IUIAutomationInvokePattern   ,        $dtagIUIAutomationInvokePattern], _
    [$UIA_SelectionPatternId,           $sIID_IUIAutomationSelectionPattern,        $dtagIUIAutomationSelectionPattern], _
    [$UIA_LegacyIAccessiblePatternId,   $sIID_IUIAutomationLegacyIAccessiblePattern,$dtagIUIAutomationLegacyIAccessiblePattern], _
    [$UIA_SelectionItemPatternId,       $sIID_IUIAutomationSelectionItemPattern,    $dtagIUIAutomationSelectionItemPattern], _
    [$UIA_RangeValuePatternId,          $sIID_IUIAutomationRangeValuePattern,       $dtagIUIAutomationRangeValuePattern], _
    [$UIA_ScrollPatternId,              $sIID_IUIAutomationScrollPattern,           $dtagIUIAutomationScrollPattern], _
    [$UIA_GridPatternId,                $sIID_IUIAutomationGridPattern,             $dtagIUIAutomationGridPattern], _
    [$UIA_GridItemPatternId,            $sIID_IUIAutomationGridItemPattern,         $dtagIUIAutomationGridItemPattern], _
    [$UIA_MultipleViewPatternId,        $sIID_IUIAutomationMultipleViewPattern,     $dtagIUIAutomationMultipleViewPattern], _
    [$UIA_WindowPatternId,              $sIID_IUIAutomationWindowPattern,           $dtagIUIAutomationWindowPattern], _
    [$UIA_DockPatternId,                $sIID_IUIAutomationDockPattern,             $dtagIUIAutomationDockPattern], _
    [$UIA_TablePatternId,               $sIID_IUIAutomationTablePattern,            $dtagIUIAutomationTablePattern], _
    [$UIA_TextPatternId,                $sIID_IUIAutomationTextPattern,             $dtagIUIAutomationTextPattern], _
    [$UIA_TogglePatternId,              $sIID_IUIAutomationTogglePattern,           $dtagIUIAutomationTogglePattern], _
    [$UIA_TransformPatternId,           $sIID_IUIAutomationTransformPattern,        $dtagIUIAutomationTransformPattern], _
    [$UIA_ScrollItemPatternId,          $sIID_IUIAutomationScrollItemPattern,       $dtagIUIAutomationScrollItemPattern], _
    [$UIA_ItemContainerPatternId,       $sIID_IUIAutomationItemContainerPattern,    $dtagIUIAutomationItemContainerPattern], _
    [$UIA_VirtualizedItemPatternId,     $sIID_IUIAutomationVirtualizedItemPattern,  $dtagIUIAutomationVirtualizedItemPattern], _
    [$UIA_SynchronizedInputPatternId,   $sIID_IUIAutomationSynchronizedInputPattern,$dtagIUIAutomationSynchronizedInputPattern], _
    [$UIA_ExpandCollapsePatternId,      $sIID_IUIAutomationExpandCollapsePattern,   $dtagIUIAutomationExpandCollapsePattern] _
        ]

    local $pPattern
    local $sIID_Pattern
    local $sdTagPattern
    local $i

    for $i=0 to ubound($patternArray)-1
        if $patternArray[$i][0]=$patternId Then
;~          consolewrite("Pattern identified " & @crlf)
            $sIID_Pattern=$patternArray[$i][1]
            $sdTagPattern=$patternArray[$i][2]
        EndIf
    next
;~  consolewrite($patternid & $sIID_Pattern & $sdTagPattern & @CRLF)

    $obj.getCurrentPattern($PatternId, $pPattern)
    $oPattern=objCreateInterface($pPattern, $sIID_Pattern, $sdtagPattern)
    if isobj($oPattern) Then
;~      consolewrite("UIA found the pattern" & @CRLF)
        return $oPattern
    Else
        _UIA_Debug("UIA WARNING ** NOT ** found the pattern" & @CRLF)
    endif
EndFunc
Edited by junkew
1 person likes this

Share this post


Link to post
Share on other sites

Thanks junkew. I will study the information you provided. Thanks again for the great work!

Share this post


Link to post
Share on other sites

I am still playing with this trying to figure out how to click on found items and on example #2 to click the start menu button I get the following error:

**** All childs of the taskbar ****
Title is:   Class=Shell_TrayWnd
Title is: Start Class=Button
Title is:   Class=TrayNotifyWnd
Title is: Clock Class=TrayClockWClass
Title is: Show desktop  Class=TrayShowDesktopButtonWClass
Title is:   Class=SysPager
Title is: User Promoted Notification Area   Class=ToolbarWindow32
Title is: Fully charged (100%)  Class=
Title is: Unidentified network No network access ryanco.com Internet access Class=
Title is: Solve PC issues: 1 important message
5 total messages    Class=
Title is: Speakers / Headphones: 32%    Class=
Title is: Dropbox 2.0.22
Up to date  Class=
Title is: Connectify: Share your Internet
Hotspot disabled.   Class=
Title is: NotificationChevron   Class=Button
Title is:   Class=ReBarWindow32
Title is: Running applications  Class=MSTaskSwWClass
Title is: Running applications  Class=MSTaskListWClass
Title is: UIAutomation  Class=
Title is: Microsoft Excel 2010  Class=
Title is: Windows Virtual PC    Class=
Title is: Internet Explorer Class=
Title is: Microsoft Word    Class=
Title is: Mozilla Firefox   Class=
Title is: AutoIt Form Designer  Class=
Title is: Microsoft Outlook 2010    Class=
Title is: C:\Users\RC01712\Desktop\UIAutomation\startbtn.au3 - SciTE [7 of 7]   Class=
**** try to click on the start button of the taskbar ****
C:\Users\RC01712\Desktop\UIAutomation\UIAWrappers.au3 (475) : ==> Variable used without being declared.:
$objUIAutomation.createPropertyCondition($PropertyId, $tVal, $pCondition)
$objUIAutomation.createPropertyCondition($PropertyId, ^ ERROR
->11:15:54 AutoIt3.exe ended.rc:1
>Exit code: 1    Time: 0.728

Any thoughts on why?


Build your own poker game with AutoIt: pokerlogic.au3 | Learn To Program Using FREE Tools with AutoIt

Share this post


Link to post
Share on other sites

#52 ·  Posted (edited)

I wanted to get everyone's feed back on my idea to improve the _UIA_StartSUT function. Please let me know what you think!

Edit: Any suggestions/improvements are very welcome!

Modified _UIA_StartSUT function:

Func _UIA_StartSUT($SUT_VAR, $Return_hWnd = True)
    local $fullName = _UIA_getVar( $SUT_VAR & ".Fullname")
    local $processName = _UIA_getVar($SUT_VAR & ".processname")
    local $app2Start = $fullName & " " & _UIA_getVar($SUT_VAR & ".Parameters")
    local $workingDir =  _UIA_getVar($SUT_VAR & ".Workingdir")
    local $windowState = _UIA_getVar($SUT_VAR & ".Windowstate")
    local $result

    if fileexists($fullName) Then
        $result = processexists($processName)
        if $result = 0 Then
            _UIA_Debug("Starting : " & $app2Start & " from " & $workingDir)
            $result = Run($app2Start,$workingDir, $windowState )
            ProcessWait($processName,60)
            Sleep(500)

            _UIA_setVar($SUT_VAR & ".PID", $result)
        Else
            _UIA_setVar($SUT_VAR & ".PID", $result)
        EndIf
    Else
        _UIA_Debug("No clue where to find the system under test (SUT) on your system, please start manually:" & @CRLF )
        _UIA_Debug($app2Start & @CRLF)
    EndIf

    If $Return_hWnd Then
        Return _UIA_getSUThWnd($SUT_VAR)
    EndIf
EndFunc

Added _UIA_getSUThWnd function

Func _UIA_getSUThWnd($SUT_VAR)
    If Not ProcessExists(_UIA_getVar($SUT_VAR & ".PID")) Then
        Return _UIA_Debug("System Under Test Process ID does not exist" & @CRLF )
    Else
        _UIA_setVar($SUT_VAR & ".hWnd", _GetHwndFromPID(_UIA_getVar($SUT_VAR & ".PID")))
        Return HWnd(_UIA_getVar($SUT_VAR & ".hWnd")) ; must be converted to HWnd to work correctly
    EndIf
EndFunc

Added _GetHwndFromPID function from

Func _GetHwndFromPID($PID)
    ;http://www.autoitscript.com/forum/topic/86680-pid-window-handle-hwnd-conversion/#entry621521
    $hWnd = 0
    $stPID = DllStructCreate("int")
    Do
        $winlist2 = WinList()
        For $i = 1 To $winlist2[0][0]
            If $winlist2[$i][0] <> "" Then
                DllCall("user32.dll", "int", "GetWindowThreadProcessId", "hwnd", $winlist2[$i][1], "ptr", DllStructGetPtr($stPID))
                If DllStructGetData($stPID, 1) = $PID Then
                    $hWnd = $winlist2[$i][1]
                    ExitLoop
                EndIf
            EndIf
        Next
        Sleep(100)
    Until $hWnd <> 0
    Return $hWnd
EndFunc ;==>_GetHwndFromPID

small very simple example of use:

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_Version=Beta
#AutoIt3Wrapper_Change2CUI=y
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <debug.au3>
#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"

_UIA_setVar("SUT1.Folder", @WindowsDir & "\System32")
_UIA_setVar("SUT1.Workingdir",@WindowsDir & "\System32\")
_UIA_setVar("SUT1.EXE","calc.exe")
_UIA_setVar("SUT1.Fullname", _UIA_getVar("SUT1.Folder") & "\"& _UIA_getVar("SUT1.EXE"))
_UIA_setVar("SUT1.Parameters","")
_UIA_setVar("SUT1.Processname","calc.exe")
_UIA_setVar("SUT1.Windowstate",@SW_RESTORE)

_UIA_setVar("SUT1.Fullname", _UIA_getVar("SUT1.Folder") & "\"& _UIA_getVar("SUT1.EXE"))


;~ Start system under test
MsgBox(0,"", "Get winow title of SUT from _UIA_StartSUT = " & WinGetTitle(_UIA_StartSUT("SUT1", True)))

WinSetTitle(_UIA_getSUThWnd("SUT1"), "", "New Title") ; Change the title of the calculator window

MsgBox(0,"", "The window title of the calculator was changed from 'Calculator' to 'New Title'")

MsgBox(0,"", "Now you can get the updated window title by using _UIA_getSUThWnd = " & WinGetTitle(_UIA_getSUThWnd("SUT1")))

Exit
Edited by Shrapnel

Share this post


Link to post
Share on other sites

@JFish: Examples fixed, download examples and UIWrappers from first post and retry.

 

@Shrapnel: Nice suggestion, I implemented in a little different way see updated UIWrappers

;~ Wait for the window to be there
        $oSUT=_UIA_getObjectByFindAll($oDesktop, "processid:=" & $result2, $treescope_children)
        if not isobj($oSUT) Then
            _UIA_Debug("No window found in SUT : " & $app2Start & " from " & $workingDir)
        Else
        ;~ Add it to the Runtime Type Information
            _UIA_setVar("RTI." & $SUT_VAR & ".PID", $result2)
            _UIA_setVar("RTI." & $SUT_VAR & ".HWND", hex(_UIA_getPropertyValue($oSUT, $UIA_NativeWindowHandlePropertyId)))
;~          _UIA_DumpThemAll($oSUT,$treescope_subtree)
        EndIf

Tricky part is with splashscreens in that case you could have the wrong HWND in RTI.SUTx.HWND

Share this post


Link to post
Share on other sites

#54 ·  Posted (edited)

Fo the RTI.SUT.HWND, you should use the hwnd() function instead of hex()

It seems to be a limitation to the data dictionary to not save the hwnd() value, or not save it in the correct way, so it must be converted each time you call it. see examples below

Edit _UIA_StartSUT function as follows:

;~ Wait for the window to be there
        $oSUT=_UIA_getObjectByFindAll($oDesktop, "processid:=" & $result2, $treescope_children)
        if not isobj($oSUT) Then
            _UIA_Debug("No window found in SUT : " & $app2Start & " from " & $workingDir)
        Else
        ;~ Add it to the Runtime Type Information
            _UIA_setVar("RTI." & $SUT_VAR & ".PID", $result2)
            _UIA_setVar("RTI." & $SUT_VAR & ".HWND", HWnd(_UIA_getPropertyValue($oSUT, $UIA_NativeWindowHandlePropertyId)))
                _UIA_Debug("just get HWND = " & _UIA_getVar("RTI.SUT1.HWND") & @CRLF)
                _UIA_Debug("convert HWND with HWND function = " & hwnd(_UIA_getVar("RTI.SUT1.HWND")) & @CRLF)

            _UIA_setVar("RTI." & $SUT_VAR & ".HWND", Hex(_UIA_getPropertyValue($oSUT, $UIA_NativeWindowHandlePropertyId)))
                _UIA_Debug("just hex = " & _UIA_getVar("RTI.SUT1.HWND") & @CRLF)
                _UIA_Debug("Hex converted to HWND with HWND function= " & HWnd(_UIA_getVar("RTI.SUT1.HWND")) & @CRLF)

;~          _UIA_DumpThemAll($oSUT,$treescope_subtree)
        EndIf

Run this script

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_Version=Beta
#AutoIt3Wrapper_Change2CUI=y
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <debug.au3>
#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"

_UIA_setVar("SUT1.Folder", @WindowsDir & "\System32")
_UIA_setVar("SUT1.Workingdir",@WindowsDir & "\System32\")
_UIA_setVar("SUT1.EXE","calc.exe")
_UIA_setVar("SUT1.Fullname", _UIA_getVar("SUT1.Folder") & "\"& _UIA_getVar("SUT1.EXE"))
_UIA_setVar("SUT1.Parameters","")
_UIA_setVar("SUT1.Processname","calc.exe")
_UIA_setVar("SUT1.Windowstate",@SW_RESTORE)

_UIA_setVar("SUT1.Fullname", _UIA_getVar("SUT1.Folder") & "\"& _UIA_getVar("SUT1.EXE"))

_UIA_StartSUT("SUT1")

Exit

And view the console output (red text is the only hwnd autoit will recognize):

just get HWND = 590240
convert HWND with HWND function = 0x000901A0
just hex = 000901A0
Hex converted to HWND with HWND function= 0x00000000

Edited by Shrapnel

Share this post


Link to post
Share on other sites

#55 ·  Posted (edited)

So maybe the solution to this is to setVar as hwnd() and in the getVar() function to return hwnd() if the SUT begins with RTI and ends with hwnd

 

Edit: something like this:

Modified _UIA_getVar() func:

func _UIA_getVar($varName)
    if $UIA_VARS.exists($varName) Then
        If StringRight($varname, 5) = ".HWND" Then
            If StringLeft($varname, 4) = "RTI." Then
                Return HWnd($UIA_Vars($varName))
            Else
                Return HWnd($UIA_Vars("RTI." & $varName))
            EndIf
        Else
            Return $UIA_Vars($varName)
        EndIf
    Else
        SetError(1) ;~ Not defined in repository
        return "*** ERROR *** " & $varname & " not found!"
    endif
EndFunc

Modified _UIA_StartSUT() so it sets mainwindow in RTI. (side note: $result2 is redundant variable. could just use $result)

Func _UIA_StartSUT($SUT_VAR, $hasWindow = True)
    Local $fullName=_UIA_getVar( $SUT_VAR & ".Fullname")
    Local $processName=_UIA_getVar($SUT_VAR & ".processname")
    Local $app2Start=$fullName & " " & _UIA_getVar($SUT_VAR & ".Parameters")
    Local $workingDir= _UIA_getVar($SUT_VAR & ".Workingdir")
    Local $windowState=_UIA_getVar($SUT_VAR & ".Windowstate")
    Local $result, $result2   ; Holds the process id's

    If FileExists($fullName) Then
;~      Only start new instance when not found
        $result2 = processexists($processName)
        If $result2 = 0 Then
            _UIA_Debug("Starting : " & $app2Start & " from " & $workingDir & @CRLF)
            $result = run($app2Start,$workingDir, $windowState )
            $result2 = ProcessWait($processName, 60)
;~          Sleep(500) ;~ Just to give the system some time to show everything
        EndIf

#cs ### Old Section ###
        $oSUT = _UIA_getObjectByFindAll($oDesktop, "processid:=" & $result2, $treescope_children)

        If Not IsObj($oSUT) Then
            _UIA_Debug("No window found in SUT : " & $app2Start & " from " & $workingDir & @CRLF)
        Else
        ;~ Add it to the Runtime Type Information
            _UIA_setVar("RTI." & $SUT_VAR & ".PID", $result2)
            _UIA_setVar("RTI." & $SUT_VAR & ".HWND", HWnd(_UIA_getPropertyValue($oSUT, $UIA_NativeWindowHandlePropertyId)))
        EndIf
#ce; ### Old Section ###

#Region ### New Section ###
        If $hasWindow Then
            Do
                Sleep(100)
                $oSUT = _UIA_getObjectByFindAll($oDesktop, "processid:=" & $result2, $treescope_children)
            Until IsObj($oSUT)

            ;~ Add it to the Runtime Type Information
            _UIA_setVar("RTI." & $SUT_VAR & ".PID", $result2)
            _UIA_setVar("RTI." & $SUT_VAR & ".HWND", HWnd(_UIA_getPropertyValue($oSUT, $UIA_NativeWindowHandlePropertyId)))
            _UIA_setVar("RTI." & $SUT_VAR & ".mainwindow", $oSUT)
        EndIf
#EndRegion ### New Section  ###

    Else
        _UIA_Debug("No clue where to find the system under test (SUT) on your system, please start manually:" & @CRLF )
        _UIA_Debug($app2Start & @CRLF)
    EndIf
EndFunc
Edited by Shrapnel

Share this post


Link to post
Share on other sites

It is hard for me to contribute to any meaningful part of this code as I am still trying to understand it all (though I love it and have used it for some things already).  I think the original simple Spy was awesome.  My tiny contribution is shrinking it down and adding some functionality with some additional hotkeys to copy to the clipboard, open and paste into Notepad, and toggle the mouse capture on and off.  I find it easier to spy things this way.  Hope it is helpful to someone.

#include <GuiEdit.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <WinAPI.au3>
#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"
#include <Misc.au3>

#AutoIt3Wrapper_UseX64=Y  ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os

dim $oldUIElement ; To keep track of latest referenced element

;~ Some references for reading
;~ [url=http://support.microsoft.com/kb/138518/nl]http://support.microsoft.com/kb/138518/nl[/url]  tagpoint structures attention point
;~ [url=http://www.autoitscript.com/forum/topic/128406-interface-autoitobject-iuiautomation/]http://www.autoitscript.com/forum/topic/128406-interface-autoitobject-iuiautomation/[/url]
;~ [url=http://msdn.microsoft.com/en-us/library/windows/desktop/ff625914(v=vs.85).aspx]http://msdn.microsoft.com/en-us/library/windows/desktop/ff625914(v=vs.85).aspx[/url]

HotKeySet("{ESC}", "Close") ; Set ESC as a hotkey to exit the script.
HotKeySet("^w", "GetElementInfo") ; Set Hotkey Ctrl+M to get some basic information in the GUI
HotKeySet("^c", "_copyedit")
HotKeySet("^p", "_pasteedit")
HotKeySet("^t", "_togglemouse")

#Region ### START Koda GUI section ### Form=c:\users\rc01712\desktop\uiautomation\spy2.kxf
$Form1_1 = GUICreate("Spy 2", 630, 479, 192, 132)
$Edit1 = GUICtrlCreateEdit("", 8, 16, 385, 441)
GUICtrlSetData(-1, "Edit1")
$Label1 = GUICtrlCreateLabel("Control+w to capture information", 400, 48, 201, 20)
$Label2 = GUICtrlCreateLabel("Control+p to open Notepad and paste", 400, 100, 221, 20)
$Label3 = GUICtrlCreateLabel("Control+c to copy Info to Clipboard", 400, 74, 211, 20)
$Label4 = GUICtrlCreateLabel("Esc to exit", 400, 146, 64, 20)
$Label5 = GUICtrlCreateLabel("Control+t to toggle mouse capture", 400, 125, 209, 20)
$Clear = GUICtrlCreateButton("Clear edit", 448, 192, 121, 25)
GUISetState(@SW_SHOW)
#EndRegion ### END Koda GUI section ###

_UIA_Init()

dim $content=""
dim $toggle=0

; Run the GUI until the dialog is closed
While true
$msg = GUIGetMsg()
sleep(100)
if _ispressed(01) AND $toggle=1 Then
getelementinfo()
endif
if $msg=$clear Then
    GUICtrlSetData($Edit1,"")
EndIf
If $msg = $GUI_EVENT_CLOSE Then ExitLoop
WEnd

func _copyEdit()
    $content=guictrlread($Edit1)
    ClipPut($content)
EndFunc

func _pasteedit()
    run("notepad.exe")
    _UIA_setVar("Notepad.mainwindow","classname:=Notepad") ; works
    _UIA_setVar("Notepad.document","classname:=Edit")
    _UIA_action("Notepad.Mainwindow","setfocus"); works
    send("^v")
EndFunc

func _togglemouse()
    if $toggle=0 then
        $toggle=1
    Else
        $toggle=0
    EndIf
    ;MsgBox("","",$toggle)
EndFunc


Func GetElementInfo()
Local $hWnd
Local $tStruct = DllStructCreate($tagPOINT) ; Create a structure that defines the point to be checked.
;~ Local $tStruct = DllStructCreate("INT64,INT64")
;~ Local $tStruct =_AutoItObject_DllStructCreate($tagPoint)

ToolTip("")

$x=MouseGetPos(0)
$y=MouseGetPos(1)
DllStructSetData($tStruct, "x", $x)
DllStructSetData($tStruct, "y", $y)
consolewrite(DllStructGetData($tStruct,"x") & DllStructGetData($tStruct,"y"))

;~ consolewrite("Mouse position is retrieved " & @crlf)
$objUIAutomation.ElementFromPoint($tStruct,$pUIElement)
;~ $objUIAutomation.ElementFromPoint(DllStructGetPtr($tStruct),$pUIElement)
;~ consolewrite("Element from point is passed, trying to convert to object ")
$oUIElement = objcreateinterface($pUIElement,$sIID_IUIAutomationElement, $dtagIUIAutomationElement)

if isobj($oldUIElement) Then
if $oldUIElement=$oUIElement then
return
endif
endif
_WinAPI_RedrawWindow(_WinAPI_GetDesktopWindow(), 0, 0, $RDW_INVALIDATE + $RDW_ALLCHILDREN) ; Clears Red outline graphics.
GUICtrlSetData($Edit1, "Mouse position is retrieved " & $x & "-" & $y & @CRLF)
$oldElement=$oUIElement

If IsObj($oUIElement) Then
;~  ConsoleWrite("At least we have an element "  & "[" & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & "][" & _UIA_getPropertyValue($oUIElement, $UIA_ClassNamePropertyId) & "]" & @CRLF)
GUICtrlSetData($Edit1, "At least we have an element "  & "[" & _UIA_getPropertyValue($oUIElement, $UIA_NamePropertyId) & "][" & _UIA_getPropertyValue($oUIElement, $UIA_ClassNamePropertyId) & "]" & @CRLF,1)
    $text1="Title is: <" &  _UIA_getPropertyValue($oUIElement,$UIA_NamePropertyId) &  ">" & @TAB _
   & "Class   := <" & _UIA_getPropertyValue($oUIElement,$uia_classnamepropertyid) &  ">" & @TAB _
& "controltype:= " _
& "<" &  _UIA_getControlName(_UIA_getPropertyValue($oUIElement,$UIA_ControlTypePropertyId)) &  ">" & @TAB  _
& ",<" &  _UIA_getPropertyValue($oUIElement,$UIA_ControlTypePropertyId) &  ">" & @TAB  _
& ", (" &  hex(_UIA_getPropertyValue($oUIElement,$UIA_ControlTypePropertyId)) &  ")" & @TAB & @CRLF
$text1= $text1 & @CRLF & getAllPropertyValues($oUIElement)
ConsoleWrite($text1)
GUICtrlSetData($Edit1, "Having the following values for all properties: " & @crlf & $text1 & @CRLF, 1)


_GUICtrlEdit_LineScroll($Edit1, 0, 0 - _GUICtrlEdit_GetLineCount($Edit1))

$t=stringsplit(_UIA_getPropertyValue($oUIElement, $UIA_BoundingRectanglePropertyId),";")
_DrawRect($t[1],$t[3]+$t[1],$t[2],$t[4]+$t[2])
EndIf

EndFunc   ;==>Example

Func Close()
Exit
EndFunc   ;==>Close
1 person likes this

Build your own poker game with AutoIt: pokerlogic.au3 | Learn To Program Using FREE Tools with AutoIt

Share this post


Link to post
Share on other sites

The changes I made in post #55 caused a "bug" in _UIA_Action() because it was not checking if the main window already existed in RTI.

To account for this, I modified the _UIA_Action() function as follows:

Func _UIA_action($obj_or_string, $strAction, $p1=0, $p2=0, $p3=0)
    Local $tPattern
    Local $x, $y
;~  Local $objElement
    Local $oElement

;~ If we are giving a description then try to make an object first by looking from repository
;~ Otherwise assume an advanced description we should search under one of the previously referenced elements at runtime
    If isobj($obj_or_string) Then
        $oElement=$obj_or_string
        $obj=$obj_or_string
    Else
        $tPhysical=_UIA_getVar($obj_or_string)
;~ If not found in repository assume its a physical description
        If @error=1 Then
            _UIA_DEBUG("Finding object (bypassing repository) with physical description " & $tPhysical & @CRLF)
            $tPhysical=$obj_or_string
        EndIf

        _UIA_DEBUG("Finding object with physical description " & $tPhysical & @CRLF)

#Region ###; Updated If Statement ###
            If stringinstr($obj_or_string,".mainwindow") Then ;~ if its a mainwindow reference find it under the desktop
                If IsObj(_UIA_getVar("RTI." & $obj_or_string)) Then ; If object already exists in RTI, then use it
                    $oElement = _UIA_getVar("RTI." & $obj_or_string)
;~                  _UIA_setVar("RTI.MAINWINDOW", $oElement) ; <-------------------------------------------------------### Not sure if this should be here or not. junkew, please advise ###
                Else ; Else, find the object
                    $startElement = "Desktop"
                    $oStart = $oDesktop
                    $oElement =_UIA_getObjectByFindAll($oStart, $tPhysical, $treescope_subtree,$obj_or_string)
                    _UIA_setVar("RTI.MAINWINDOW",$oElement)
                EndIf
            Else ;~     Find the object under the last referenced mainwindow / parent window
                $startElement = "RTI." & stringleft($obj_or_string,stringinstr($obj_or_string,".")) & "mainwindow"
                $oStart = _UIA_getVar($startElement)
                $oElement = _UIA_getObjectByFindAll($oStart, $tPhysical, $treescope_subtree)
            EndIf
#EndRegion ###; Updated If Statement ###

        If IsObj($oElement) Then ; Continue the action by setting the $obj value to an UIA element
            $obj = $oElement
        Else
            Return SetError(1)
        Endif
    EndIf

    _UIA_setVar("RTI.ACTIONCOUNT", _UIA_getVar("RTI.ACTIONCOUNT") + 1)
    _UIA_DEBUG("Action " & _UIA_getVar("RTI.ACTIONCOUNT") & " " & $strAction & " on " & $obj_or_string & @CRLF & "   ")

    $strAction = StringLower($strAction)
    Switch $strAction
        Case "leftclick", "left", "rightclick", "right"
            Local $mouseButton = "left"
            $t = StringSplit(_UIA_getPropertyValue($obj, $UIA_BoundingRectanglePropertyId),";")
            $x = $t[1] + ($t[3] / 2)
            $y = $t[2] + ($t[4] / 2)
            If StringInStr($strAction, "right") Then $mouseButton = "right"
            If $p1 = 0 Then $p1 = 1
            MouseClick($mouseButton, $x, $y, $p1, $p2)
            If $p3 > 0 Then Sleep($p3)
        Case "setvalue using keys"
            $obj.setfocus
            Send("^a")
            Send($p1)
            If $p2 > 0 Then Sleep($p2)
        Case "invoke"
            $tPattern=_UIA_getPattern($obj,$UIA_InvokePatternId)
            $tPattern.invoke()
        Case "focus"
            $obj.setfocus()
        Case Else
    EndSwitch
EndFunc

Share this post


Link to post
Share on other sites

#58 ·  Posted (edited)

 

It is hard for me to contribute to any meaningful part of this code as I am still trying to understand it all (though I love it and have used it for some things already).  I think the original simple Spy was awesome.  My tiny contribution is shrinking it down and adding some functionality with some additional hotkeys to copy to the clipboard, open and paste into Notepad, and toggle the mouse capture on and off.  I find it easier to spy things this way.  Hope it is helpful to someone.

 

Very nice Jfish!

Edited by Shrapnel
1 person likes this

Share this post


Link to post
Share on other sites

#59 ·  Posted (edited)

Ok, i kinda feel like im spamming this thread :> , but I'm hoping some find it useful.

I added two functions to UIAWrappers.au3, _UIA_setVarsFromArray and _UIA_getVarsFromArray

Func _UIA_setVarsFromArray(ByRef $_array)
    If Not IsArray($_array) Then Return 0
    For $x = 1 To $_array[0][0] Step 1
        _UIA_setVar($_array[$x][0], $_array[$x][1])
    Next
EndFunc


Func _UIA_getVarsFromArray(ByRef $_array)
    If Not IsArray($_array) Then Return 0
    Local $returnArray = $_array
    For $x = 1 To $returnArray[0][0] Step 1
        $returnArray[$x][1] = _UIA_getVar($returnArray[$x][0])
    Next
    Return $returnArray
EndFunc

Here is an example of their use:

#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <Array.au3>
#include <debug.au3>
#include "CUIAutomation2.au3"
#include "UIAWrappers.au3"

Local $CalcFolder = @WindowsDir & "\system32\"

Local $SUT_Array [8][2]= [ _
    [7,2], _
    ["Calculator.Folder",       $CalcFolder], _
    ["Calculator.Workingdir",   StringTrimRight($CalcFolder, 1)], _
    ["Calculator.exe",          "calc.exe"], _
    ["Calculator.Fullname",     $CalcFolder & "calc.exe"], _
    ["Calculator.Parameters",   ""], _
    ["Calculator.Processname",  "calc.exe"], _
    ["Calculator.Windowstate",  @SW_RESTORE] _
]

_ArrayDisplay($SUT_Array, "$SUT_Array")

_UIA_setVarsFromArray($SUT_Array)

ConsoleWrite("_UIA_getVar(""Calculator.Folder"") = " & _UIA_getVar("Calculator.Folder") & @CRLF)
ConsoleWrite("_UIA_getVar(""Calculator.Workingdir"") = " & _UIA_getVar("Calculator.Workingdir") & @CRLF)
ConsoleWrite("_UIA_getVar(""Calculator.exe"") = " & _UIA_getVar("Calculator.exe") & @CRLF)
ConsoleWrite("_UIA_getVar(""Calculator.Fullname"") = " & _UIA_getVar("Calculator.Fullname") & @CRLF)
ConsoleWrite("_UIA_getVar(""Calculator.Parameters"") = " & _UIA_getVar("Calculator.Parameters") & @CRLF)
ConsoleWrite("_UIA_getVar(""Calculator.Processname"") = " & _UIA_getVar("Calculator.Processname") & @CRLF)
ConsoleWrite("_UIA_getVar(""Calculator.Windowstate"") = " & _UIA_getVar("Calculator.Windowstate") & @CRLF)

Local $tempArray = _UIA_getVarsFromArray($SUT_Array)

_ArrayDisplay($tempArray, "$tempArray")

_UIA_StartSUT("Calculator")

Exit

This could be useful because now you can create arrays that contain all the controls, RTI, etc of an object and easily set/get them in the data dictionary.

Edited by Shrapnel

Share this post


Link to post
Share on other sites

It makes indeed sense to have the definitions of the elements split from the actions.

My intention is to get those from definition files (still in doubt if I should do it in INI files or XML files) but will include your array method logic too as then you can compile everything without dependencies on external files when you have a small script.

Not sure If I will switch to your convention for starting the SUT. Within the framework its a little easier to use SUT1 thru SUT9, SUTnn as reference and only once the main SUT references are there use the

calculator.button1, calculator.button2 convention for the elements but I am open for suggestions

the mainwindow should be defined upfront and if detected nonexisting in UIA action function it should not be added (but maybe I have to add logic to recover/search from another starting point in the tree)

@jfish: I will be working on a more extended spy with logical tree and screenshots and other shortcut keys so please post more suggestions/features you would like in a UIA based spy

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • bobmcrae
      By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • jackchen
      By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object. Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?