Jump to content
junkew

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

This is not my code you are running. Have you tried just to run my code as it is in Scite editor. I am pretty sure my code is working.

Before you run the code you must open HOBOnode Viewer Utility, select the "Data Search" tab, and make sure the "Save Data..." or "Get/Refresh Device List" button is enabled.

I can see, that your code fails on the "Find" section. To figure out what's wrong, I need to see the complete code.

Share this post


Link to post
Share on other sites

Sorry about that LarsJ. Yeah the previous error code came out from my code. If I run only your code I get this error. I'm glad that you are able to have it working. That gives me hope. I know I am pretty close on this. 

>"C:\Program Files (x86)\AutoIt3\SciTE\..\autoit3.exe" /ErrorStdOut "C:\Users\richi\Desktop\HOBO_MAIN2.au3"    
Window handle OK
UI Automation object OK
Automation element from window OK
Property condition OK
Find button ERR
>Exit code: 0    Time: 1.280

Share this post


Link to post
Share on other sites

I think I have found the error. When I use the Spy tool on the "Save Data..." button, I get this output:

 

Mouse position is retrieved 1200-706
At least we have an element [Save Data...][WindowsForms10.BUTTON.app.0.141b42a_r12_ad1]
Having the following values for all properties: 
Title is: <Save Data...>    Class   := <WindowsForms10.BUTTON.app.0.141b42a_r12_ad1>    controltype:= <UIA_ButtonControlTypeId> ,<50000>    , (00010502)    
*** Parent Information ***
Title is: <Data Search> Class   := <WindowsForms10.Window.8.app.0.141b42a_r12_ad1>  controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (000104E4)    
*** Detailed properties of the highlighted element ***
UIA_AcceleratorKeyPropertyId :=
UIA_AccessKeyPropertyId :=
UIA_AriaPropertiesPropertyId :=
UIA_AriaRolePropertyId :=
UIA_AutomationIdPropertyId :=dataSearchB
UIA_BoundingRectanglePropertyId :=1151;691;102;31
UIA_ClassNamePropertyId :=WindowsForms10.BUTTON.app.0.141b42a_r12_ad1
UIA_ClickablePointPropertyId :=
UIA_ControllerForPropertyId :=
UIA_ControlTypePropertyId :=50000
UIA_CulturePropertyId :=0
UIA_DescribedByPropertyId :=
UIA_DockDockPositionPropertyId :=5
UIA_ExpandCollapseExpandCollapseStatePropertyId :=3
UIA_FlowsToPropertyId :=
UIA_FrameworkIdPropertyId :=WinForm
UIA_GridColumnCountPropertyId :=0
UIA_GridItemColumnPropertyId :=0
UIA_GridItemColumnSpanPropertyId :=1
UIA_GridItemContainingGridPropertyId :=
UIA_GridItemRowPropertyId :=0
UIA_GridItemRowSpanPropertyId :=1
UIA_GridRowCountPropertyId :=0
UIA_HasKeyboardFocusPropertyId :=False
UIA_HelpTextPropertyId :=
UIA_IsContentElementPropertyId :=True
UIA_IsControlElementPropertyId :=True
UIA_IsDataValidForFormPropertyId :=False
UIA_IsDockPatternAvailablePropertyId :=False
UIA_IsEnabledPropertyId :=False
UIA_IsExpandCollapsePatternAvailablePropertyId :=False
UIA_IsGridItemPatternAvailablePropertyId :=False
UIA_IsGridPatternAvailablePropertyId :=False
UIA_IsInvokePatternAvailablePropertyId :=True
UIA_IsItemContainerPatternAvailablePropertyId :=False
UIA_IsKeyboardFocusablePropertyId :=True
UIA_IsLegacyIAccessiblePatternAvailablePropertyId :=True
UIA_IsMultipleViewPatternAvailablePropertyId :=False
UIA_IsOffscreenPropertyId :=False
UIA_IsPasswordPropertyId :=False
UIA_IsRangeValuePatternAvailablePropertyId :=False
UIA_IsRequiredForFormPropertyId :=False
UIA_IsScrollItemPatternAvailablePropertyId :=False
UIA_IsScrollPatternAvailablePropertyId :=False
UIA_IsSelectionItemPatternAvailablePropertyId :=False
UIA_IsSelectionPatternAvailablePropertyId :=False
UIA_IsSynchronizedInputPatternAvailablePropertyId :=False
UIA_IsTableItemPatternAvailablePropertyId :=False
UIA_IsTablePatternAvailablePropertyId :=False
UIA_IsTextPatternAvailablePropertyId :=False
UIA_IsTogglePatternAvailablePropertyId :=False
UIA_IsTransformPatternAvailablePropertyId :=False
UIA_IsValuePatternAvailablePropertyId :=False
UIA_IsVirtualizedItemPatternAvailablePropertyId :=False
UIA_IsWindowPatternAvailablePropertyId :=False
UIA_ItemStatusPropertyId :=
UIA_ItemTypePropertyId :=
UIA_LabeledByPropertyId :=
UIA_LegacyIAccessibleChildIdPropertyId :=0
UIA_LegacyIAccessibleDefaultActionPropertyId :=
UIA_LegacyIAccessibleDescriptionPropertyId :=
UIA_LegacyIAccessibleHelpPropertyId :=
UIA_LegacyIAccessibleKeyboardShortcutPropertyId :=
UIA_LegacyIAccessibleNamePropertyId :=Save Data...
UIA_LegacyIAccessibleRolePropertyId :=43
UIA_LegacyIAccessibleSelectionPropertyId :=
UIA_LegacyIAccessibleStatePropertyId :=00100001
UIA_LegacyIAccessibleValuePropertyId :=
UIA_LocalizedControlTypePropertyId :=button
UIA_MultipleViewCurrentViewPropertyId :=0
UIA_MultipleViewSupportedViewsPropertyId :=
UIA_NamePropertyId :=Save Data...
UIA_NativeWindowHandlePropertyId :=00010502
UIA_OrientationPropertyId :=0
UIA_ProcessIdPropertyId :=4584
UIA_ProviderDescriptionPropertyId :=[pid:2960,hwnd:0x10502 Main:Nested [pid:4584,hwnd:0x10502 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]
UIA_RangeValueIsReadOnlyPropertyId :=True
UIA_RangeValueLargeChangePropertyId :=0
UIA_RangeValueMaximumPropertyId :=0
UIA_RangeValueMinimumPropertyId :=0
UIA_RangeValueSmallChangePropertyId :=0
UIA_RangeValueValuePropertyId :=0
UIA_RuntimeIdPropertyId :=42;66818
UIA_ScrollHorizontallyScrollablePropertyId :=False
UIA_ScrollHorizontalScrollPercentPropertyId :=0
UIA_ScrollHorizontalViewSizePropertyId :=100
UIA_ScrollVerticallyScrollablePropertyId :=False
UIA_ScrollVerticalScrollPercentPropertyId :=0
UIA_ScrollVerticalViewSizePropertyId :=100
UIA_SelectionCanSelectMultiplePropertyId :=False
UIA_SelectionIsSelectionRequiredPropertyId :=False
UIA_SelectionselectionPropertyId :=
UIA_SelectionItemIsSelectedPropertyId :=False
UIA_SelectionItemSelectionContainerPropertyId :=
UIA_TableColumnHeadersPropertyId :=
UIA_TableItemColumnHeaderItemsPropertyId :=
UIA_TableRowHeadersPropertyId :=
UIA_TableRowOrColumnMajorPropertyId :=2
UIA_TableItemRowHeaderItemsPropertyId :=
UIA_ToggleToggleStatePropertyId :=2
UIA_TransformCanMovePropertyId :=False
UIA_TransformCanResizePropertyId :=False
UIA_TransformCanRotatePropertyId :=False
UIA_ValueIsReadOnlyPropertyId :=True
UIA_ValueValuePropertyId :=
UIA_WindowCanMaximizePropertyId :=False
UIA_WindowCanMinimizePropertyId :=False
UIA_WindowIsModalPropertyId :=False
UIA_WindowIsTopmostPropertyId :=False
UIA_WindowWindowInteractionStatePropertyId :=0
UIA_WindowWindowVisualStatePropertyId :=0

The button is correctly reported with a type id of UIA_ButtonControlTypeId.

In your text box in post #176 the button is reported with a type id of UIA_PaneControlTypeId. This is wrong. The proper type is UIA_ButtonControlTypeId. Why is your button reported with a wrong type id?

Is the UI Automation API on your PC uptodate? Version 3.0. Are you using the newest versions of the UDFs in first post?

Edited by LarsJ

Share this post


Link to post
Share on other sites

Good eye LarsJ,

I have no idea why is reported with a wrong type id. 

I downloaded all files from post#1 by Junkew. There are 2 files with the same name, but Junkew says the smallest one is the most updated one with your corrections to the code. That is the one I downloaded. Is there any other update in a different post? I went trhough all pages before and don't recall seeing one. 

How can I check the UI Automation API version?

The Script Editor SciTE-Lite version is: Version 3.3.6 Dec.30 2013

Thanks for all your help.

Share this post


Link to post
Share on other sites

If you are on Windows 7 or 8 the API is OK. If you are on XP or Vista there may be a risk that the API isn't uptodate. See http://support.microsoft.com/kb/971513.

It's very important that you use AutoIt 3.3.10.2. Version 3.3.8 is flawed e.g. according to parameter transfer for some parameter types for the interface methods.

If you are using Scite (I'm not), I would recommend the full version.

You can use Inspect.exe from Windows SDK to check the output of the Spy tool. Inspect.exe also reports the "Save Data..." button with a type id of UIA_ButtonControlTypeId.

It's very very important that you figure out, why you are getting wrong control type ids. This means, that nothing will work.

If you have any questions, you can use this forum.

Regards Lars.

Share this post


Link to post
Share on other sites

Thanks a lot for your help LarsJ

I am on Windows 7. I don't think we have a problem there.

I downloaded the full version of Scite after your recomendation. I've been using that because I thougth that was the best option to write the script. What do you use? Should I use a different editor to write the script?

I found the inspect.exe tool from Windows and compare the output of the spy tool. The inspect.exe gives me the control type as a UIA_ButtonControlTypeId. Just like you are saying. I have inserted a picture of the output. 

 

I don't understand how the spy tool is giving a different control type than the inspect tool.

inspect.jpg

Share this post


Link to post
Share on other sites

LarsJ

Thanks for all your help. I want to apologize for the time invested. I am new in this automation world and very new in Autoit. I believe is powerful and amazing. I don't know if it matters the editor, and I will test later, but I copied your code to notepad++, save it, then right clicked on the file and compile it. This created the exe file, which runs as administrator and clicks the save button from the HOBO utility when is enabeled. 

I will follow your example to create the whole automation process for the utility, which is selecting the device, clicking the Get/Refresh button, then the Generate Data Streams, and finally the Save button.

Thanks a lot.

Share this post


Link to post
Share on other sites

casma, You're welcome. I'm only answering if I have the time for it. Good to see that you are able to go on with your project.

This has absolutely nothing to do with the editor. I'm using Scite myself to run code with F5 to see ConsoleWrite output. But I'm not using Scite for editing. For editing I'm using Notepad++. (This means that often I have both editors open.)

The Spy tool is definitely reliable. This is simply a consequence of the implementation. The Spy tool is just doing some kind of reverse lookup of the control ids and pattern ids (to get the names of the equivalent variable names) returned directly by the interface methods in the UI Automation DLL files.

You should download the UDFs and the Spy tool in the first post once more. Something must have happened with these files.

Regards Lars.

Share this post


Link to post
Share on other sites

I'll do that LarsJ

I've been working on the project and everything is running much smoother. I've been following the same logic as you did in your code. Find the buttons and creating a condition property, then find the button and then invoke the instance of the button. At least that's how I've tried to think of this. This is working perfect for all buttons.

But I run into another problem when trying to automate other elements of the application just like the type:

UIA_ListItemControlTypeId to select the devices  detected by the utility.

And once I select the "Save Data..." button the utility displays a "Save As" window where I can select the location and name of the file which contains the data. I am going to be overwritting the same file everytime I export the data and the field to type in the name of the file is: UIA_EditControlTypeId.

Any ideas how to work on this or a good example to reference this to. 

Thanks a lot for all your help. 

Share this post


Link to post
Share on other sites

Check what you want to use

The UIAWrappers will make life a little easier by abstracting stuff in a generic way

for example see example 11 

Example 9 is more flexible (in between low level use of the interfaces or fully abstracted as in example 11)

Edited by junkew

Share this post


Link to post
Share on other sites

Thanks a lot for your reply Junkew.

I've been trying 2 different approachs with no luck yet. I was hoping you guys could help me determine which is the best for this scenario.

The first one I tried is based on examples 11 and 9 as suggested by Junkew. Although if I try this I have to comment out Opt("MustDeclareVars",1) otherwise I get errors of variables not being declared in the UIAWrappers file. I have downloaded this one several times from post #1. Other errors are like:

"C:\Users\richi\Desktop\UIAWrappers.au3" (657) : ==> Variable must be of type "Object".:
$obj.FindAll($treescope, $oCondition, $pElements)
$obj^ ERROR
;Opt( "MustDeclareVars", 1 )

....

 Local $oSensor1
  _UIA_setVar("HOBO.sensor1","name:=416 SN:10485416; controltype:=ListItem" )
  $oSensor1=_UIA_getFirstObjectOfElement($hWindow,_UIA_getVar("HOBO.sensor2"),$treescope_subtree)
  _UIA_action("HOBO.sensor1","leftclick")

The other code I have tried is this one:

;Condition to find "416 SN: 10485416" sensor
  Local $pConditionSensor1, $pCondition1Sensor1, $pCondition2Sensor1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ListItemControlTypeId, $pCondition1Sensor1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "416 SN: 10485416; controltype:=ListItem", $pCondition2Sensor1 )
  $oUIAutomation.CreateAndCondition( $pCondition1Sensor1, $pCondition2Sensor1, $pConditionSensor1 )
  If Not $pConditionSensor1 Then Return ConsoleWrite( "Property condition ERR Sensor1" & @CRLF )
  ConsoleWrite( "Property condition OK Sensor1" & @CRLF )


; Find "416 SN: 10485416" sensor
  Local $pListSensor1, $oListSensor1
  $oWindow.FindFirst( $TreeScope_Descendants, $pConditionSensor1, $pListSensor1 )
  $oListSensor1 = ObjCreateInterface( $pListSensor1, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
 ; If Not IsObj( $oListSensor1 ) Then Return ConsoleWrite( "Find List ERR Sensor 1" & @CRLF )
 ; ConsoleWrite( "Find List OK Sensor 1" & @CRLF )


  ; Click (invoke) "416 SN: 10485416"
  Local $pInvokeSensor1, $oInvokeSensor1
  $oListSensor1.GetCurrentPattern( $UIA_InvokePatternId, $pInvokeSensor1 )
  $oInvokeSensor1 = ObjCreateInterface( $pInvokeSensor1, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvokeSensor1) Then Return ConsoleWrite( "Invoke pattern ERR Sensor 1" & @CRLF )
  ConsoleWrite( "Invoke pattern OK Sensor 1" & @CRLF )
  $oInvokeSensor1.Invoke()

And the console returns this:

Window handle OK
UI Automation object OK
Automation element from window OK
Property condition OK Save
Property condition OK Time
Property condition OK Generate
Property condition OK Sensor1
"C:\Users\richi\Desktop\test_notepad.au3" (103) : ==> Variable must be of type "Object".:
$oListSensor1.GetCurrentPattern( $UIA_InvokePatternId, $pInvokeSensor1 )
$oListSensor1^ ERROR

or if I don't comment out condition for sensor:

Property condition OK Time
Property condition OK Generate
Property condition OK Sensor1
Find List ERR Sensor 1

Thanks a lot for taking a look at this. 

Share this post


Link to post
Share on other sites

Try this to select an item:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

MainFunc()


Func MainFunc()

  ; Get window handle
  Local $hWindow = WinGetHandle( "HOBOnode Viewer Utility" )
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object ERR" & @CRLF )
  ConsoleWrite( "UI Automation object OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  ; --- Find list control ---

  ; There are two list controls, we need the proper one

  ; Condition to find list controls
  Local $pCondition
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ListControlTypeId, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find list controls
  Local $pUIElementArray, $oUIElementArray, $iElements
  $oWindow.FindAll( $TreeScope_Descendants, $pCondition, $pUIElementArray )
  $oUIElementArray = ObjCreateInterface( $pUIElementArray, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray )
  $oUIElementArray.Length( $iElements )
  If Not $iElements Then Return ConsoleWrite( "Find list controls ERR" & @CRLF )
  ConsoleWrite( "Find list controls OK" & @CRLF )

  ; Find the proper list control
  Local $pList, $oList, $sName
  For $i = 0 To $iElements - 1
    $oUIElementArray.GetElement( $i, $pList )
    $oList = ObjCreateInterface( $pList, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $oList.GetCurrentPropertyValue( $UIA_ClassNamePropertyId, $sName )
    ConsoleWrite( "List control class name: " & $sName & @CRLF )
    If StringInStr( $sName, "LISTBOX" ) Then
      ConsoleWrite( "This is the proper list" & @CRLF )
      ExitLoop
    EndIf
  Next

  ; --- Select list item ---

  ; Condition to find list item
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "416 SN: 10485416", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find list item
  Local $pItem, $oItem
  $oList.FindFirst( $TreeScope_Descendants, $pCondition, $pItem )
  $oItem = ObjCreateInterface( $pItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oItem ) Then Return ConsoleWrite( "Find item ERR" & @CRLF )
  ConsoleWrite( "Find item OK" & @CRLF )

  ; Add item to selection
  Local $pSelection, $oSelection
  $oItem.GetCurrentPattern( $UIA_SelectionItemPatternId, $pSelection )
  $oSelection = ObjCreateInterface( $pSelection, $sIID_IUIAutomationSelectionItemPattern, $dtagIUIAutomationSelectionItemPattern )
  $oSelection.AddToSelection()

EndFunc

Share this post


Link to post
Share on other sites

@Casma: Will fix coming week the examples in combination with the last uiawrappers.au3 and cuiautomation2.au3. You could try to run the examples with the biggest cuiautomation2.au3 but not sure if that will fix your issues.

Share this post


Link to post
Share on other sites

Hi.

This looks amazing.  I'm trying to get the state of a radio button in a dialog and nothing in default AutoIt is working.  Here's the Spy output of the control:

Mouse position is retrieved 386-359
At least we have an element [Items][WindowsForms10.BUTTON.app.0.165f26b_r59_ad1]
Having the following values for all properties: 
Title is: <Items>   Class   := <WindowsForms10.BUTTON.app.0.165f26b_r59_ad1>    controltype:= <UIA_RadioButtonControlTypeId>    ,<50013>    , (0000C35D)    
*** Parent Information ***
Title is: <>    Class   := <WindowsForms10.Window.8.app.0.165f26b_r59_ad1>  controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    
*** Detailed properties of the highlighted element ***
UIA_AcceleratorKeyPropertyId :=
UIA_AccessKeyPropertyId :=
UIA_AriaPropertiesPropertyId :=
UIA_AriaRolePropertyId :=
UIA_AutomationIdPropertyId :=radioButton_Output_Items
UIA_BoundingRectanglePropertyId :=365;356;50;18
UIA_ClassNamePropertyId :=WindowsForms10.BUTTON.app.0.165f26b_r59_ad1
UIA_ClickablePointPropertyId :=
UIA_ControllerForPropertyId :=
UIA_ControlTypePropertyId :=50013
UIA_CulturePropertyId :=0
UIA_DescribedByPropertyId :=
UIA_DockDockPositionPropertyId :=5
UIA_ExpandCollapseExpandCollapseStatePropertyId :=3
UIA_FlowsToPropertyId :=
UIA_FrameworkIdPropertyId :=WinForm
UIA_GridColumnCountPropertyId :=0
UIA_GridItemColumnPropertyId :=0
UIA_GridItemColumnSpanPropertyId :=1
UIA_GridItemContainingGridPropertyId :=
UIA_GridItemRowPropertyId :=0
UIA_GridItemRowSpanPropertyId :=1
UIA_GridRowCountPropertyId :=0
UIA_HasKeyboardFocusPropertyId :=True
UIA_HelpTextPropertyId :=
UIA_IsContentElementPropertyId :=True
UIA_IsControlElementPropertyId :=True
UIA_IsDataValidForFormPropertyId :=False
UIA_IsDockPatternAvailablePropertyId :=False
UIA_IsEnabledPropertyId :=True
UIA_IsExpandCollapsePatternAvailablePropertyId :=False
UIA_IsGridItemPatternAvailablePropertyId :=False
UIA_IsGridPatternAvailablePropertyId :=False
UIA_IsInvokePatternAvailablePropertyId :=True
UIA_IsItemContainerPatternAvailablePropertyId :=False
UIA_IsKeyboardFocusablePropertyId :=True
UIA_IsLegacyIAccessiblePatternAvailablePropertyId :=True
UIA_IsMultipleViewPatternAvailablePropertyId :=False
UIA_IsOffscreenPropertyId :=False
UIA_IsPasswordPropertyId :=False
UIA_IsRangeValuePatternAvailablePropertyId :=False
UIA_IsRequiredForFormPropertyId :=False
UIA_IsScrollItemPatternAvailablePropertyId :=False
UIA_IsScrollPatternAvailablePropertyId :=False
UIA_IsSelectionItemPatternAvailablePropertyId :=True
UIA_IsSelectionPatternAvailablePropertyId :=False
UIA_IsSynchronizedInputPatternAvailablePropertyId :=False
UIA_IsTableItemPatternAvailablePropertyId :=False
UIA_IsTablePatternAvailablePropertyId :=False
UIA_IsTextPatternAvailablePropertyId :=False
UIA_IsTogglePatternAvailablePropertyId :=False
UIA_IsTransformPatternAvailablePropertyId :=False
UIA_IsValuePatternAvailablePropertyId :=False
UIA_IsVirtualizedItemPatternAvailablePropertyId :=False
UIA_IsWindowPatternAvailablePropertyId :=False
UIA_ItemStatusPropertyId :=
UIA_ItemTypePropertyId :=
UIA_LabeledByPropertyId :=
UIA_LegacyIAccessibleChildIdPropertyId :=0
UIA_LegacyIAccessibleDefaultActionPropertyId :=Check
UIA_LegacyIAccessibleDescriptionPropertyId :=
UIA_LegacyIAccessibleHelpPropertyId :=
UIA_LegacyIAccessibleKeyboardShortcutPropertyId :=
UIA_LegacyIAccessibleNamePropertyId :=Items
UIA_LegacyIAccessibleRolePropertyId :=45
UIA_LegacyIAccessibleSelectionPropertyId :=
UIA_LegacyIAccessibleStatePropertyId :=1048596
UIA_LegacyIAccessibleValuePropertyId :=
UIA_LocalizedControlTypePropertyId :=radio button
UIA_MultipleViewCurrentViewPropertyId :=0
UIA_MultipleViewSupportedViewsPropertyId :=
UIA_NamePropertyId :=Items
UIA_NativeWindowHandlePropertyId :=66296
UIA_OrientationPropertyId :=0
UIA_ProcessIdPropertyId :=1568
UIA_ProviderDescriptionPropertyId :=[pid:5360,hwnd:0x102F8 Main:Nested [pid:1568,hwnd:0x102F8 Main(parent link):Microsoft: MSAA Proxy (unmanaged:UIAutomationCore.DLL)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]
UIA_RangeValueIsReadOnlyPropertyId :=True
UIA_RangeValueLargeChangePropertyId :=0
UIA_RangeValueMaximumPropertyId :=0
UIA_RangeValueMinimumPropertyId :=0
UIA_RangeValueSmallChangePropertyId :=0
UIA_RangeValueValuePropertyId :=0
UIA_RuntimeIdPropertyId :=42;66296
UIA_ScrollHorizontallyScrollablePropertyId :=False
UIA_ScrollHorizontalScrollPercentPropertyId :=0
UIA_ScrollHorizontalViewSizePropertyId :=100
UIA_ScrollVerticallyScrollablePropertyId :=False
UIA_ScrollVerticalScrollPercentPropertyId :=0
UIA_ScrollVerticalViewSizePropertyId :=100
UIA_SelectionCanSelectMultiplePropertyId :=False
UIA_SelectionIsSelectionRequiredPropertyId :=False
UIA_SelectionselectionPropertyId :=
UIA_SelectionItemIsSelectedPropertyId :=True
UIA_SelectionItemSelectionContainerPropertyId :=
UIA_TableColumnHeadersPropertyId :=
UIA_TableItemColumnHeaderItemsPropertyId :=
UIA_TableRowHeadersPropertyId :=
UIA_TableRowOrColumnMajorPropertyId :=2
UIA_TableItemRowHeaderItemsPropertyId :=
UIA_ToggleToggleStatePropertyId :=2
UIA_TransformCanMovePropertyId :=False
UIA_TransformCanResizePropertyId :=False
UIA_TransformCanRotatePropertyId :=False
UIA_ValueIsReadOnlyPropertyId :=True
UIA_ValueValuePropertyId :=
UIA_WindowCanMaximizePropertyId :=False
UIA_WindowCanMinimizePropertyId :=False
UIA_WindowIsModalPropertyId :=False
UIA_WindowIsTopmostPropertyId :=False
UIA_WindowWindowInteractionStatePropertyId :=0
UIA_WindowWindowVisualStatePropertyId :=0

Can someone show me simple code to check whether this radio button is checked or not?

Many thanks.

Share this post


Link to post
Share on other sites

Try this:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

MainFunc()


Func MainFunc()

  Local $hWindow = WinGetHandle(  ) ; <<<< Fill in window info <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object ERR" & @CRLF )
  ConsoleWrite( "UI Automation object OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  ; Condition to find radio button
  Local $pCondition, $pCondition1, $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_RadioButtonControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Items", $pCondition2 ) ; <<<< Name <<<<<<<<<<<<<<<<
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find radio button
  Local $pButton, $oButton
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pButton )
  $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oButton ) Then Return ConsoleWrite( "Find button ERR" & @CRLF )
  ConsoleWrite( "Find button OK" & @CRLF )

  ; Get selected state
  Local $fSelected
  $oButton.GetCurrentPropertyValue( $UIA_SelectionItemIsSelectedPropertyId, $fSelected )
  ConsoleWrite( "Selected: " & $fSelected & @CRLF )

EndFunc

Share this post


Link to post
Share on other sites

I'm trying to open Action Center so I can set the state of the checkboxes inside.

It seems I can get the control ID with the below but the "_UIA_action($control, "Left", 0 , 0, 0) ; leftclick" always results in an error.

I've not modified the includes, and the error I see is about an array which is malformed. MustDeclareVars has to be set to 0 for the below to work.

If someone could point out an example that shows how to get a handle on the control and click it I might be able to figure it out from there.

#include "CUIAutomation2.au3"
#include <uiawrappers.au3>

Opt( "MustDeclareVars", 0 )

Global $oUIAutomation
Run("rundll32.exe shell32.dll,Control_RunDLL wscui.cpl", "", @SW_HIDE)

WinWait("Action Center")
WinActivate("Action Center")
WinWaitActive("Action Center")
MainFunc()


Func MainFunc()

;~   Local $hWindow = WinGetHandle( "[CLASS:Element]" )              ;
  Local $hWindow = WinGetHandle( "[CLASS:CabinetWClass]" )              ; Windows Explorer, Windows 7
  ;Local $hWindow = WinGetHandle( "[CLASS:ExploreWClass]" )             ; Windows Explorer, Windows XP
  ;Local $hWindow = WinGetHandle( "Windows Explorer right pane" )       ; Windows Explorer right pane
  ;Local $hWindow = WinGetHandle( "[TITLE:Test; CLASS:AutoIt v3 GUI]" ) ; AutoIt GUI window
  ;Local $hWindow = WinGetHandle( "[CLASS:IEFrame]" )                   ; Internet Explorer
  ;Local $hWindow = WinGetHandle( "Calculator" )                        ; Calculator
  If Not $hWindow Then
    MsgBox(4096,"","")
    Return
  EndIf

  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return

  Local $pWindow
  ;$oUIAutomation.GetRootElement( $pWindow )             ; Desktop
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow ) ; Window
  If Not $pWindow Then Return

  Local $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return

  ;ListDescendants( $oWindow, 0, 1 ) ; Desktop
  ListDescendants( $oWindow, 0, 0 )  ; Window

EndFunc


Func ListDescendants( $oParent, $iLevel, $iLevels = 0 )

  If Not IsObj( $oParent ) Then Return
  If $iLevels And $iLevel = $iLevels Then Return

  Local $pRawWalker, $oRawWalker
  $oUIAutomation.RawViewWalker( $pRawWalker )
  $oRawWalker = ObjCreateInterface( $pRawWalker, $sIID_IUIAutomationTreeWalker, $dtagIUIAutomationTreeWalker )

  Local $pUIElement, $oUIElement
  $oRawWalker.GetFirstChildElement( $oParent, $pUIElement )
  $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

  Local $sIndent = ""
  For $i = 0 To $iLevel - 1
    $sIndent &= "    "
  Next

  While IsObj( $oUIElement )
;~     ConsoleWrite( $sIndent & "Title     = " & _UIA_getPropertyValue( $oUIElement, $UIA_NamePropertyId ) & @CRLF & _
;~                   $sIndent & "Class     = " & _UIA_getPropertyValue( $oUIElement, $UIA_ClassNamePropertyId ) & @CRLF & _
;~                   $sIndent & "Ctrl type = " & _UIA_getPropertyValue( $oUIElement, $UIA_ControlTypePropertyId ) & @CRLF & _
;~                   $sIndent & "Ctrl name = " & _UIA_getPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ) & @CRLF & _
;~                   $sIndent & "Selected  = " & _UIA_getPropertyValue( $oUIElement, $UIA_SelectionItemIsSelectedPropertyId ) & @CRLF & _
;~                   $sIndent & "Handle    = " & Hex( _UIA_getPropertyValue( $oUIElement, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF & @CRLF )
;~ ControlClick(_UIA_getPropertyValue( $oUIElement, $UIA_NamePropertyId ),"", _UIA_getPropertyValue( $oUIElement, $UIA_LocalizedControlTypePropertyId ))
;~ ControlClick ( "Address: Control Panel\All Control Panel Items\Action Center", "Action Center", "00000000")


Local $title = _UIA_getPropertyValue( $oUIElement, $UIA_NamePropertyId )
If $title = "Action Center" Then
;~ MsgBox(4096,"",$title)
Local $control = hex(_UIA_getPropertyValue($oUIElement,$UIA_ControlTypePropertyId))
;~ MsgBox(4096,"",$control, 1)
_UIA_action($control, "Left", 0 , 0, 0) ; leftclick
;~ ControlClick("Action Center","",$control)
EndIf

    ListDescendants( $oUIElement, $iLevel + 1, $iLevels )

    $oRawWalker.GetNextSiblingElement( $oUIElement, $pUIElement )
    $oUIElement = ObjCreateInterface( $pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  WEnd

EndFunc

Share this post


Link to post
Share on other sites

Try this:

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

MainFunc()


Func MainFunc()

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "UI Automation object ERR" & @CRLF )
  ConsoleWrite( "UI Automation object OK" & @CRLF )

  ; --- Change Action Center settings ---

  ; Get window handle
  Local $hWindow = WinGetHandle( "[TITLE:Action Center; CLASS:CabinetWClass]", "")
  If Not $hWindow Then      ; With this If-statement, you can run the script several
    ; Open "Action Center"  ; times without the need to close "Action Center".
    Run( "rundll32.exe shell32.dll,Control_RunDLL wscui.cpl" )
    WinWait( "Action Center" )
    WinActivate( "Action Center" )
    WinWaitActive( "Action Center" )
    $hWindow = WinGetHandle( "[TITLE:Action Center; CLASS:CabinetWClass]", "")
  EndIf
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  ; Condition to find link
  Local $pCondition, $pCondition1, $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Change Action Center settings", $pCondition2 )
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find link
  Local $pLink, $oLink
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pLink )
  $oLink = ObjCreateInterface( $pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oLink ) Then Return ConsoleWrite( "Find Link ERR" & @CRLF )
  ConsoleWrite( "Find Link OK" & @CRLF )

  ; Click (invoke) link
  Local $pInvoke, $oInvoke
  $oLink.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "Invoke pattern ERR" & @CRLF )
  ConsoleWrite( "Invoke pattern OK" & @CRLF )
  $oInvoke.Invoke()
  Sleep( 1000 )

  ; --- Click "Windows Update" checkbox ---

  ; Get window handle
  Local $hWindow = WinGetHandle( "[TITLE:Change Action Center settings; CLASS:CabinetWClass]", "")
  If Not $hWindow Then Return ConsoleWrite( "Window handle ERR" & @CRLF )
  ConsoleWrite( "Window handle OK" & @CRLF )

  ; Get UI Automation element from window handle
  Local $pWindow, $oWindow
  $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
  $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oWindow ) Then Return ConsoleWrite( "Automation element from window ERR" & @CRLF )
  ConsoleWrite( "Automation element from window OK" & @CRLF )

  ; Condition to find checkbox
  Local $pCondition, $pCondition1, $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_CheckBoxControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Windows Update", $pCondition2 )
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find checkbox
  Local $pCheckbox, $oCheckbox
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pCheckbox )
  $oCheckbox = ObjCreateInterface( $pCheckbox, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oCheckbox ) Then Return ConsoleWrite( "Find Checkbox ERR" & @CRLF )
  ConsoleWrite( "Find Checkbox OK" & @CRLF )

  ; Toggle checkbox
  Local $pToggle, $oToggle
  $oCheckbox.GetCurrentPattern( $UIA_TogglePatternId, $pToggle )
  $oToggle = ObjCreateInterface( $pToggle, $sIID_IUIAutomationTogglePattern, $dtagIUIAutomationTogglePattern )
  If Not IsObj( $oToggle ) Then Return ConsoleWrite( "Toggle pattern ERR" & @CRLF )
  ConsoleWrite( "Toggle pattern OK" & @CRLF )
  $oToggle.Toggle()

  ; Get toggle state
  Local $fToggle
  $oCheckbox.GetCurrentPropertyValue( $UIA_ToggleToggleStatePropertyId, $fToggle )
  ConsoleWrite( "Toggle: " & $fToggle & " <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<" & @CRLF )

  ; --- Click OK button ---

  ; Condition to find OK button
  Local $pCondition, $pCondition1, $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "OK", $pCondition2 )
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "Property condition ERR" & @CRLF )
  ConsoleWrite( "Property condition OK" & @CRLF )

  ; Find OK button
  Local $pButton, $oButton
  $oWindow.FindFirst( $TreeScope_Descendants, $pCondition, $pButton )
  $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oButton ) Then Return ConsoleWrite( "Find button ERR" & @CRLF )
  ConsoleWrite( "Find button OK" & @CRLF )

  ; Click (invoke) OK button
  Local $pInvoke, $oInvoke
  $oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "Invoke pattern ERR" & @CRLF )
  ConsoleWrite( "Invoke pattern OK" & @CRLF )
  $oInvoke.Invoke()

EndFunc
Edited by LarsJ

Share this post


Link to post
Share on other sites

Try this:

 

Wow, that worked perfectly, and is more than I could have figured out from a hint. I will study this to see if I can use it on other windows. Thank you!

I made a hybrid from it with a fixed function to check the box instead of toggle, it's less code for all the items and they are accessible to ControlFocus/ControlCommand.

#include "CUIAutomation2.au3"

Opt("MustDeclareVars", 1)

MainFunc()

Func MainFunc()

    ; Create UI Automation object
    Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
    If Not IsObj($oUIAutomation) Then Return ConsoleWrite("UI Automation object ERR" & @CRLF)
    ConsoleWrite("UI Automation object OK" & @CRLF)

    ; --- Change Action Center settings ---

    ; Get window handle
    Local $hWindow = WinGetHandle("[TITLE:Action Center; CLASS:CabinetWClass]", "")
    If Not $hWindow Then ; With this If-statement, you can run the script several
        ; Open "Action Center"  ; times without the need to close "Action Center".
        Run("rundll32.exe shell32.dll,Control_RunDLL wscui.cpl")
        WinWait("Action Center")
        WinActivate("Action Center")
        WinWaitActive("Action Center")
        $hWindow = WinGetHandle("[TITLE:Action Center; CLASS:CabinetWClass]", "")
    EndIf
    If Not $hWindow Then Return ConsoleWrite("Window handle ERR" & @CRLF)
    ConsoleWrite("Window handle OK" & @CRLF)

    ; Get UI Automation element from window handle
    Local $pWindow, $oWindow
    $oUIAutomation.ElementFromHandle($hWindow, $pWindow)
    $oWindow = ObjCreateInterface($pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    If Not IsObj($oWindow) Then Return ConsoleWrite("Automation element from window ERR" & @CRLF)
    ConsoleWrite("Automation element from window OK" & @CRLF)

    ; Condition to find link
    Local $pCondition, $pCondition1, $pCondition2
    $oUIAutomation.CreatePropertyCondition($UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition1)
    $oUIAutomation.CreatePropertyCondition($UIA_NamePropertyId, "Change Action Center settings", $pCondition2)
    $oUIAutomation.CreateAndCondition($pCondition1, $pCondition2, $pCondition)
    If Not $pCondition Then Return ConsoleWrite("Property condition ERR" & @CRLF)
    ConsoleWrite("Property condition OK" & @CRLF)

    ; Find link
    Local $pLink, $oLink
    $oWindow.FindFirst($TreeScope_Descendants, $pCondition, $pLink)
    $oLink = ObjCreateInterface($pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
    If Not IsObj($oLink) Then Return ConsoleWrite("Find Link ERR" & @CRLF)
    ConsoleWrite("Find Link OK" & @CRLF)

    ; Click (invoke) link
    Local $pInvoke, $oInvoke
    $oLink.GetCurrentPattern($UIA_InvokePatternId, $pInvoke)
    $oInvoke = ObjCreateInterface($pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern)
    If Not IsObj($oInvoke) Then Return ConsoleWrite("Invoke pattern ERR" & @CRLF)
    ConsoleWrite("Invoke pattern OK" & @CRLF)
    $oInvoke.Invoke()
    Sleep(1000)

    ControlFocus("Change Action Center settings", "", "Button1")
    ControlCommand("Change Action Center settings", "", "Button1", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button2")
    ControlCommand("Change Action Center settings", "", "Button2", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button3")
    ControlCommand("Change Action Center settings", "", "Button3", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button4")
    ControlCommand("Change Action Center settings", "", "Button4", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button5")
    ControlCommand("Change Action Center settings", "", "Button5", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button6")
    ControlCommand("Change Action Center settings", "", "Button6", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button7")
    ControlCommand("Change Action Center settings", "", "Button7", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button8")
    ControlCommand("Change Action Center settings", "", "Button8", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button9")
    ControlCommand("Change Action Center settings", "", "Button9", "Check", "")
    ControlFocus("Change Action Center settings", "", "Button10")
    ControlClick("Change Action Center settings", "", "Button10")

    Sleep(100)
    WinClose($hWindow, "")

;~  CabinetWClass
EndFunc   ;==>MainFunc

Share this post


Link to post
Share on other sites

Updated first post with references to all (working) examples in thread

Added 2 zip files to first post. 

UIA_V0_4.ZIP with all au3 modules to handle IUIAutomation, IAccessible, IAccessible2 and ISimpleDom

EXAMPLES.ZIP containing about 30 examples as are discussed in this whole thread 

Enhanced the UIAWrappers.AU3 with some logic (for easier testautomation and some abstraction of interfaces)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object. Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?
×
×
  • Create New...