Jump to content
junkew

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

Having some issue with this UDF working correctly on multiple monitors.


My UDFs are generally for me. If they aren't updated for a while, it means I'm not using them myself. As soon as I start using them again, they'll get updated.

My Projects
Cisco FinesseGithubIRC UDFWindowEx UDF

 

Share this post


Link to post
Share on other sites

Windows Start menu

Over the past year there have been a number of questions related to Windows 10 Start menu, for example this.

The code below prints the controls in the Start menu in a hierarchical structure like a treeview. Output in SciTE console. The code is tested on Windows 10, 7 and XP. On Windows 10 I've tested version 1511 and 1607. There are some differences in the Start menu between these versions.

(Windows 10 was released in july 2015. So far there have been two major updates. An update in november 2015 and an update in juli 2016 (the anniversary update). In Windows 10 major updates are provided with a version number. The november 2015 update is version 1511 (yymm) and the juli 2016 update is version 1607. The first release is version 1507. You can see the version number in Start menu | Settings | Update & security when updates are downloaded and installed.)

On Windows 10 the class name of the Start menu is "Windows.UI.Core.CoreWindow". The class name of the Search menu is also "Windows.UI.Core.CoreWindow". The code works for both the Start menu and the Search Menu. (In a standard configuration the Search button is located in the taskbar just to the right of the Start button.)

On Windows 7 and XP the class name of the Start menu is "DV2ControlHost".

Output in SciTE console depends on which windows and menus you have activated in the Start menu (or Search menu on Windows 10). You can have very different output. And generally you will see a lot of output.

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

ConsoleWrite( "5 seconds to open Start menu..." & @CRLF )
Sleep( 5000 )

Example()

Func Example()
  ; Vindows version
  Local $sOS = @OSVersion, $f10 = ( $sOS = "WIN_10" ), $f7 = ( $sOS = "WIN_7" ), $fXP = ( $sOS = "WIN_XP" ), $fOS = $f10?1:$f7?1:$fXP?1:0
  If Not $fOS Then Return ConsoleWrite( "@OSVersion ERR" & @CRLF )
  ConsoleWrite( "@OSVersion OK" & @CRLF )

  ; Automation object
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Desktop object
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; Find condition
  Local $pCondition ; Use Inspect.exe (Inspect Objects) or Simple spy demo to find the class name
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, $f10?"Windows.UI.Core.CoreWindow":"DV2ControlHost", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  ; Find Start menu
  Local $pStartMenu, $oStartMenu
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pStartMenu )
  $oStartMenu = ObjCreateInterface( $pStartMenu, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oStartMenu ) Then Return ConsoleWrite( "$oStartMenu ERR" & @CRLF )
  ConsoleWrite( "$oStartMenu OK" & @CRLF )

  ; Print info
  ConsoleWrite( @CRLF )
  ConsoleWrite( "Title     = " & GetCurrentPropertyValue( $oStartMenu, $UIA_NamePropertyId ) & @CRLF & _
                "Class     = " & GetCurrentPropertyValue( $oStartMenu, $UIA_ClassNamePropertyId ) & @CRLF & _
                "Ctrl type = " & GetCurrentPropertyValue( $oStartMenu, $UIA_ControlTypePropertyId ) & @CRLF & _
                "Ctrl name = " & GetCurrentPropertyValue( $oStartMenu, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                "Value     = " & GetCurrentPropertyValue( $oStartMenu, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                "Handle    = " & Hex( GetCurrentPropertyValue( $oStartMenu, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )

  ; List all child elements
  ListDescendants( $oStartMenu, 0, 0, "    " )

  ; On Windows 7 and XP popup menus are not accessible directly from the Start menu. They
  ; are only accessible from the desktop. Therefore, it's necessary with a separate search.

  Switch True
    Case $f10
      Return
    Case $f7, $fXP
      ; Find condition
      $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuControlTypeId, $pCondition )
      If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
      ConsoleWrite( "$pCondition OK" & @CRLF )

      ; Find Popup menu
      Local $pPopupMenu, $oPopupMenu
      $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pPopupMenu )
      $oPopupMenu = ObjCreateInterface( $pPopupMenu, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
      If Not IsObj( $oPopupMenu ) Then Return ConsoleWrite( "$oPopupMenu ERR" & @CRLF )
      ConsoleWrite( "$oPopupMenu OK" & @CRLF )

      ; Print info
      ConsoleWrite( @CRLF )
      ConsoleWrite( "Title     = " & GetCurrentPropertyValue( $oPopupMenu, $UIA_NamePropertyId ) & @CRLF & _
                    "Class     = " & GetCurrentPropertyValue( $oPopupMenu, $UIA_ClassNamePropertyId ) & @CRLF & _
                    "Ctrl type = " & GetCurrentPropertyValue( $oPopupMenu, $UIA_ControlTypePropertyId ) & @CRLF & _
                    "Ctrl name = " & GetCurrentPropertyValue( $oPopupMenu, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                    "Value     = " & GetCurrentPropertyValue( $oPopupMenu, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                    "Handle    = " & Hex( GetCurrentPropertyValue( $oPopupMenu, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )

      ; List all child elements
      ListDescendants( $oPopupMenu, 0, 0, "    " )
  EndSwitch
EndFunc

; List all child elements of parent
Func ListDescendants( $oParent, $iLevel, $iLevels = 0, $sIndent = "" )
  If Not IsObj( $oParent ) Then Return
  If $iLevels And $iLevel = $iLevels Then Return

  ; Create RawViewWalker object
  Local $pRawViewWalker, $oRawViewWalker
  $oUIAutomation.RawViewWalker( $pRawViewWalker )
  $oRawViewWalker = ObjCreateInterface( $pRawViewWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker )
  If Not IsObj( $oRawViewWalker ) Then Return ConsoleWrite( "RawViewWalker object error" & @CRLF )

  ; Get first child element
  Local $pUIElement, $oUIElement
  $oRawViewWalker.GetFirstChildElement( $oParent, $pUIElement )
  $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

  For $i = 0 To $iLevel - 1
    $sIndent &= "    "
  Next

  While IsObj( $oUIElement )
    ConsoleWrite( $sIndent & "Title     = " & GetCurrentPropertyValue( $oUIElement, $UIA_NamePropertyId ) & @CRLF & _
                  $sIndent & "Class     = " & GetCurrentPropertyValue( $oUIElement, $UIA_ClassNamePropertyId ) & @CRLF & _
                  $sIndent & "Ctrl type = " & GetCurrentPropertyValue( $oUIElement, $UIA_ControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Ctrl name = " & GetCurrentPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Value     = " & GetCurrentPropertyValue( $oUIElement, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                  $sIndent & "Handle    = " & Hex( GetCurrentPropertyValue( $oUIElement, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )

    ListDescendants( $oUIElement, $iLevel + 1, $iLevels )

    $oRawViewWalker.GetNextSiblingElement( $oUIElement, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  WEnd
EndFunc

Func GetCurrentPropertyValue( $oObject, $iPropertyId )
  Local $vValue, $sString
  $oObject.GetCurrentPropertyValue( $iPropertyId, $vValue )
  If Not IsArray( $vValue ) Then Return $vValue
  $sString = $vValue[0]
  For $i = 1 To UBound( $vValue ) - 1
    $sString &= "; " & $vValue[$i]
  Next
  Return $sString
EndFunc

 

I'm testing the Windows version (10, 7, XP) with code like this:

Local $sOS = @OSVersion, $f10 = ( $sOS = "WIN_10" )

If you are using Windows 10 with an old version of AutoIt eg. 3.3.10 the "WIN_10" string is not registered in the @OSVersion macro and the Windows 10 version test will not work. If this is the case just replace

$f10 = ( $sOS = "WIN_10" )

with

$f10 = 1

 

Edited by LarsJ
IUIAutomation::GetRootElement

Share this post


Link to post
Share on other sites

Share this post


Link to post
Share on other sites

Hello,

 

I go the process to click on the file button in outlook, now trying to figure out how to click on account settings drop down box , and

then select account settings from the dropdown box

 

I have tried the below and have had no luck, I'm trying to learn this process so i can use it

 

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

Example()


Func Example()
  Local $hWindow = WinGetHandle( "Mail to Read" )
  If Not $hWindow Then Return ConsoleWrite( "$hWindow ERR" & @CRLF )
  ConsoleWrite( "$hWindow OK" & @CRLF )

  ; Create UI Automation object
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
  ConsoleWrite( "$oWindow OK" & @CRLF )

  ; Condition to find split button
  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  ; Condition to find "Lists"
  ; "Lists" is a localized word
  ; Use the word in your own language
  Local $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Account Settings", $pCondition2 )
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  ; Both conditions
  Local $pCondition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  ; Find "Lists" split button
  Local $pButton, $oButton
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pButton )
  $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF )
  ConsoleWrite( "$oButton OK" & @CRLF )

  ; Click (Invoke) button
  Local $pInvoke, $oInvoke
  $oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()
EndFunc

and in the console i get

 

$hWindow OK
$oUIAutomation OK
$oWindow OK
$pCondition1 OK
$pCondition2 OK
$pCondition OK
$oButton ERR

 

 

 

Edited by HighlanderSword

Share this post


Link to post
Share on other sites

Are you sure that this dropdown box is a button and not a combo box? Have you checked with the spy tool or Inspect.exe?

You can also print information with code like this:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Global $oUIAutomation

Example()


; List all automation elements of a window
; in a hierarchical structure like a treeview.
Func Example()
  ; Get window handle
  ;Local $hWindow = 0x0006045C                                                             ; Infragistics
  ;Local $hWindow = WinGetHandle( "[CLASS:WindowsForms10.Window.8.app.0.bb8560_r19_ad1]" ) ; Infragistics
  ;Local $hWindow = WinGetHandle( "[REGEXPCLASS:WindowsForms10.Window.8.app.*]" )          ; Windows Updates Downloader
  ;Local $hWindow = WinGetHandle( "BCGPVisualStudioGUIDemo - Start Page" )                 ; BCGSoft BCGPVisualStudioGUIDemo
  ;Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]" )                                ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]" )                                ; Windows Explorer, Windows XP
  ;Local $hWindow = WinGetHandle( "Windows Explorer right pane" )                          ; Windows Explorer right pane
  ;Local $hWindow = WinGetHandle( "[TITLE:My GUI Checkbox; CLASS:AutoIt v3 GUI]" )         ; AutoIt GUI window
  ;Local $hWindow = WinGetHandle( "[CLASS:AutoIt v3 GUI]" )                                ; AutoIt script
  ;Local $hWindow = WinGetHandle( "[CLASS:Chrome_WidgetWin_1]" )                           ; Chrome
  ;Local $hWindow = WinGetHandle( "[CLASS:IEFrame]" )                                      ; Internet Explorer
  Local $hWindow = WinGetHandle( "[CLASS:WordPadClass]" )                                 ; WordPad
  ;Local $hWindow = WinGetHandle( "Calculator" )                                           ; Calculator
  If Not $hWindow Then Return ConsoleWrite( "$hWindow ERR" & @CRLF )
  ConsoleWrite( "$hWindow OK" & @CRLF )

  ; Create UI Automation object
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
  ConsoleWrite( "$oWindow OK" & @CRLF & @CRLF )

  ; List all elements of window
  ListDescendants( $oWindow, 0 )
EndFunc

; List all child elements of parent
Func ListDescendants( $oParent, $iLevel, $iLevels = 0, $sIndent = "" )
  If Not IsObj( $oParent ) Then Return
  If $iLevels And $iLevel = $iLevels Then Return

  ; Create RawViewWalker object
  Local $pRawViewWalker, $oRawViewWalker
  $oUIAutomation.RawViewWalker( $pRawViewWalker )
  $oRawViewWalker = ObjCreateInterface( $pRawViewWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker )
  If Not IsObj( $oRawViewWalker ) Then Return ConsoleWrite( "RawViewWalker object error" & @CRLF )

  ; Get first child element
  Local $pUIElement, $oUIElement
  $oRawViewWalker.GetFirstChildElement( $oParent, $pUIElement )
  $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

  For $i = 0 To $iLevel - 1
    $sIndent &= "    "
  Next

  While IsObj( $oUIElement )
    ConsoleWrite( $sIndent & "Title     = " & GetCurrentPropertyValue( $oUIElement, $UIA_NamePropertyId ) & @CRLF & _
                  $sIndent & "Class     = " & GetCurrentPropertyValue( $oUIElement, $UIA_ClassNamePropertyId ) & @CRLF & _
                  $sIndent & "Ctrl type = " & GetCurrentPropertyValue( $oUIElement, $UIA_ControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Ctrl name = " & GetCurrentPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
                  $sIndent & "Value     = " & GetCurrentPropertyValue( $oUIElement, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF & _
                  $sIndent & "Handle    = " & Hex( GetCurrentPropertyValue( $oUIElement, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )

    ListDescendants( $oUIElement, $iLevel + 1, $iLevels )

    $oRawViewWalker.GetNextSiblingElement( $oUIElement, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  WEnd
EndFunc

Func GetCurrentPropertyValue( $oObject, $iPropertyId )
  Local $vValue, $sString
  $oObject.GetCurrentPropertyValue( $iPropertyId, $vValue )
  If Not IsArray( $vValue ) Then Return $vValue
  $sString = $vValue[0]
  For $i = 1 To UBound( $vValue ) - 1
    $sString &= "; " & $vValue[$i]
  Next
  Return $sString
EndFunc

Use the proper window. You can add a few seconds Sleep as first line in Example function if you need time to open a menu or child window.

Edited by LarsJ

Share this post


Link to post
Share on other sites

Hello,

 

Below is what I got from Inspector.

 

And Simplespy

 

Regards

Paul

 

Inspector :

How found:    Focus [o:0x60200AA,c:0x0]
    hwnd=0x00000000000206D0 64bit class="NetUIHWND" style=0x56030000 ex=0x0
ChildId:    0
Interfaces:    
Impl:    Remote native IAccessible
Name:    "Account Settings..."
Value:    [null]
Role:    drop down button (0x38)
State:    focused,pressed,focusable,has popup (0x4010000C)
Location:    {l:180, t:207, w:97, h:86}
Selection:    [Error: calling getter for this property: hr=0xFFFFFFFF80020003 - Member not found.]
Description:    [null]
Kbshortcut:    "Alt, F, I, S"
DefAction:    "Open"
Help:    [null]
HelpTopic:    ""
ChildCount:    0
Window:    0x206D0
FirstChild:    [null]
LastChild:    [null]
Next:    "Account Settings" : text : read only
Previous:    [null]
Left:    [null]
Up:    [null]
Right:    "Account Settings" : text : read only
Down:    [null]
Other Props:    Object has no additional properties
Children:    Container has no children
Ancestors:    "Account Settings" : grouping : normal
    "Info" : pane : normal
    "" : client : normal
    "Backstage view" : pane : normal
    "" : pane : normal
    "" : window : focused,focusable
    none : client : focusable
    none : window : focusable
    "Mail to Read - Paul Schrader 2012 - Outlook" : client : focusable
    "Mail to Read - Paul Schrader 2012 - Outlook" : window : sizeable,moveable,focusable
    "Desktop" : client : focusable
    "Desktop" : window : focusable
    [ No Parent ]

==================================

SimpleSpy

Mouse position is retrieved 217-247
At least we have an element [Account Settings...][NetUIAnchor]
Having the following values for all properties:
Title is: <Account Settings...>    Class   := <NetUIAnchor>    controltype:= <UIA_MenuItemControlTypeId>    ,<50011>    , (0000C35B)    
*** Parent Information ***
Title is: <Account Settings>    Class   := <NetUISlabContainer>    controltype:= <UIA_GroupControlTypeId>    ,<50026>    , (0000C36A)    
*** Detailed properties of the highlighted element ***
UIA_title:= <Account Settings...>
UIA_text:= <Account Settings...>
UIA_regexptitle:= <Account Settings...>
UIA_class:= <NetUIAnchor>
UIA_regexpclass:= <NetUIAnchor>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <0>
UIA_RuntimeId:= <42;1639720;2;963468816;522>
UIA_BoundingRectangle:= <180;207;114;86>
UIA_ProcessId:= <3792>
UIA_ControlType:= <50011>
UIA_LocalizedControlType:= <Menu Item>
UIA_Name:= <Account Settings...>
UIA_AccessKey:= <Alt, F, I, S>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_ClassName:= <NetUIAnchor>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <0>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <True>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <False>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <True>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <0>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Account Settings...>
UIA_LegacyIAccessibleRole:= <12>
UIA_LegacyIAccessibleState:= <1074791424>
UIA_LegacyIAccessibleKeyboardShortcut:= <Alt, F, I, S>
UIA_LegacyIAccessibleDefaultAction:= <Open>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:3792,hwnd:0x0 Main(parent link):Unidentified Provider (unmanaged:mso.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

 

 

Edited by HighlanderSword

Share this post


Link to post
Share on other sites

It's a menu item. So I think you just need to replace $UIA_ButtonControlTypeId with $UIA_MenuItemControlTypeId (you can find these constants in CUIAutomation2.au3) in the code in post 507. And maybe "Button" with "MenuItem" in $pButton and $oButton.

Share this post


Link to post
Share on other sites

Hello,

 

Yes , below is the code I'm using.......

 

 

 

 Local $hWindow = WinGetHandle( "Mail to Read" )
  If Not $hWindow Then Return ConsoleWrite( "$hWindow ERR" & @CRLF )
  ConsoleWrite( "$hWindow OK" & @CRLF )

  ; Create UI Automation object
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
  ConsoleWrite( "$oWindow OK" & @CRLF )

  ; Condition to find split button

 


  ;find the account settings and click it


  ; Condition to find split button
  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId,  $UIA_MenuItemControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  ; Condition to find "Lists"
  ; "Lists" is a localized word
  ; Use the word in your own language
  Local $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Account Settings...", $pCondition2 )
  msgbox(0,"66666",$pcondition2)
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  ; Both conditions
  Local $pCondition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  msgbox(0,"pcon2",$pcondition)
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  ; Find "Lists" split button
  Local $pButton, $oButton
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pButton )
  msgbox(0,"77777",$pbutton)
  $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF )
  ConsoleWrite( "$oButton OK" & @CRLF )

  ; Click (Invoke) button
  Local $pInvoke, $oInvoke
  $oButton.GetCurrentPattern( $UIA_LegacyIAccessibleDefaultActionPropertyId, $pInvoke )

  MsgBox(0,"pinvoke",$pInvoke)
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()

Share this post


Link to post
Share on other sites

You're still getting the "$oButton ERR" which means that it can't find the button or menu item?

Is the file menu open while you're running the code? To click an item ("Account Settings...") in a menu the menu must be open.

In post 509 I can see that UIA_IsInvokePatternAvailable = <False> which probably means that the Invoke method is not working. If Invoke is not working the easiest is to use the MouseClick command.

I can also see that UIA_AccessKey = <Alt, F, I, S>. Using the AccessKey to open the "Account Settings" window could be an easy alternative solution.

The GetCurrentPattern method can only be used with pattern IDs (in top of CUIAutomation2.au3). $oButton.GetCurrentPattern( $UIA_LegacyIAccessibleDefaultActionPropertyId, $pInvoke ) will not work.

I'm not using Office/Outlook so I can't test anything myself. I've Office 2003 installed on my old XP, but I don't think that'll help.

(You can use the "<>"-button to post code or output from the spy tool.)

Share this post


Link to post
Share on other sites

Which outlook version? which windows version? which autoit version?

Can you try with the example code that the simplespy gives you and post it here.

At first sight it looks a straightforward hierarchy of 2-3 levels deep. Shouldn't be that hard.

Tip: Highlight first before the actual action you want to do so you know your recognition pattern is correct

  • indeed frequently invoke doesn't work in the way you expect

its this function in uiawrappers and calls the _UIA_DrawRect function that draws a rectangle on the desktop in the specified area

Func _UIA_Highlight($oElement)
    Local $t
    $t = StringSplit(_UIA_getPropertyValue($oElement, $UIA_BoundingRectanglePropertyId), ";")
    _UIA_DrawRect($t[1], $t[3] + $t[1], $t[2], $t[4] + $t[2])
EndFunc   ;==>_UIA_Highlight

 

Share this post


Link to post
Share on other sites

Start simplespy

Move mouse over your object

Ctlr w

Now you should see output including example source that you copy paste to new autoit script. post then the source and result so we can enhance where needed.

Share this post


Link to post
Share on other sites

Hello,

Below is what I get from simplespy, I don't get any Autoit Code

Can you post the SimpleSpy Au3 Code you are using to get code as well ?

 

 

=======================================================================

Mouse position is retrieved 232-238
At least we have an element [Account Settings...][NetUIAnchor]
Having the following values for all properties:
Title is: <Account Settings...>    Class   := <NetUIAnchor>    controltype:= <UIA_MenuItemControlTypeId>    ,<50011>    , (0000C35B)    
*** Parent Information ***
Title is: <Account Settings>    Class   := <NetUISlabContainer>    controltype:= <UIA_GroupControlTypeId>    ,<50026>    , (0000C36A)    
*** Detailed properties of the highlighted element ***
UIA_title:= <Account Settings...>
UIA_text:= <Account Settings...>
UIA_regexptitle:= <Account Settings...>
UIA_class:= <NetUIAnchor>
UIA_regexpclass:= <NetUIAnchor>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <0>
UIA_RuntimeId:= <42;8259664;2;1087324544;543>
UIA_BoundingRectangle:= <180;207;114;86>
UIA_ProcessId:= <10264>
UIA_ControlType:= <50011>
UIA_LocalizedControlType:= <Menu Item>
UIA_Name:= <Account Settings...>
UIA_AccessKey:= <Alt, F, I, S>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_ClassName:= <NetUIAnchor>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <0>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <True>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <False>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <True>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <0>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Account Settings...>
UIA_LegacyIAccessibleRole:= <12>
UIA_LegacyIAccessibleState:= <1074791424>
UIA_LegacyIAccessibleKeyboardShortcut:= <Alt, F, I, S>
UIA_LegacyIAccessibleDefaultAction:= <Open>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:10264,hwnd:0x0 Main(parent link):Unidentified Provider (unmanaged:mso.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

 

Share this post


Link to post
Share on other sites

I am a beginner on Autoit and wanted  to understand UI automation code so the question is where do I start? I already went through some of examples but I am not able to understand the use of functions that are part of UIAutomation is there any documentation where I can start with some basics and using those basic to automate advanced topic. some times it happens that looking at code one cannot understand the arguments passed and the returned object from a function so is there any doc or link to get overcome such problems?

Small examples in UIAutomation (not auto it example code) to perform basic operations like reading data from a list view or tree nodes will be helpful.

After looking at the code and some examples executed by me I have following questions :

What are the steps followed in order to extract rows from a table like view?(for e.g in outlook we have mail list for which UDFs are defined so similarly using UIautomation)

Where should I look for understanding the different UIA properties and their purpose defined in the UIAWrappers?

How do you get the text value from a input textbox using UIAutomation and not autoit example code?

How to set the textbox using UIAutomation(not auto it example code)?

How to identify a table from  a window and get the rows of tables and extract the icons or text in contents of that row?

How do you go to the parent container of that table as when one uses simple spy it directly gives you the autogenerated code to click on that particular element and it does not help to extract the contents of complete rows present in that table?

Sometimes the simplespy code does not identify the element correctly and starts random highlighting and selection so how to correctly identify the UI element?

Thanks in advance.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object. Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?
×
×
  • Create New...