Jump to content
LarsJ

Using UI Automation Code in AutoIt

Recommended Posts

This example is about using Microsoft UI Automation APIs in AutoIt. Ie. it's about using interfaces, objects, methods, properties and constants as they are defined directly by Microsoft.

In AutoIt, the definitions are contained in CUIAutomation2.au3 by junkew. The definitions are copied from corresponding C/C++ header files and customized the AutoIt language.

This example is not about the functions in UIAWrappers.au3 by junkew.

The purpose of the example is to show how to use Microsoft UI Automation APIs directly without any intermediate AutoIt functions.

A major advantage of this approach is that the Microsoft documentation directly can be used as AutoIt documentation.

Other major advantages are that there is no need for documenting and maintaining a set of intermediate AutoIt functions. Without intermediate AutoIt functions the code will be very fast. All Microsoft functions can be used this way eg. functions for event handling. New interfaces, objects, methods, properties and constants can easily be added simply by adding the definitions to CUIAutomation2.au3.

Is this approach difficult? Not at all as the examples will show.

The example is inspired by this comment of mLipok. Including documentation and pictures, even a simple Notepad automation can be somewhat comprehensive. Therefore a new example.

First post will be a list of examples. The examples themselves are reviewed in the following posts.

In all examples UIASpy is used to get element information. UIASpy is included in the zip-file below.

 

Thread topics

Using UIASpy

Other spy tools

How to topics

  • If UI Automation or the UIASpy tool is new to you, then you should start reading the
    How to topics

Examples

 

Zip-file
You need AutoIt 3.3.12 or later. Tested on Windows XP, Windows 7 and Windows 10.

Comments and questions about using UIA code are welcome. Let me know if there are any issues.

Questions about the functions in UIAWrappers.au3 should be asked in junkew's thread.

UIAExamples.7z

Edited by LarsJ
New zip-file

Share this post


Link to post
Share on other sites

@LarsJ

Why don't you create a condition object from a pointer and use the pointer directly?

Local $pCondition
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition ) ; get pointer
Local $pNotepad, $oNotepad
$oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad ) ; use pointer

What's the difference between your code and this?

Local $pCondition, $oCondition
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition ) ; get pointer
$oCondition = ObjCreateInterface( $pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition ) ; create object
Local $pNotepad, $oNotepad
$oDesktop.FindFirst( $TreeScope_Descendants, $oCondition, $pNotepad ) ; use object

 

Share this post


Link to post
Share on other sites

This is what I was looking for.

BIG THANKS @LarsJ

Edited by mLipok

Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API Forum Rules *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST APIErrorLog.au3 UDF - A logging Library *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 UDF * ADO.au3 UDF SMTP Mailer UDF * Dual Monitor resolution detection * * 2GUI on Dual Monitor System * _SciLexer.au3 UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Good coding practices in AutoIt * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * IE in TaskScheduler

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

11

Signature last update: 2018-10-31

Share this post


Link to post
Share on other sites

Nice detailed example and yes uiautomation of ms is in general fast when you can do exact matching.

Similar examples are given in examples.zip of uiautomation thread. Main reason for the uiawrappers is to have more flexibility on

  • Index, indexrelative
  • Small wrappers around pointer and objcreateinterface logic
  • Named properties and easy combining them
  • At some cost of speed
  • Setvalue, settext interface on many objects not working
  • Invoke sometimes is a click and sometimes a setfocus.
  • Match on regular expressions

 

Share this post


Link to post
Share on other sites

InnI, Because these 2 ConsoleWrites prints the same value

ConsoleWrite( "Ptr( $pCondition ) = " & Ptr( $pCondition ) & @CRLF )
ConsoleWrite( "Ptr( $oCondition() ) = " & Ptr( $oCondition() ) & @CRLF )

then I don't think there is much difference between these lines:

$oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad )
$oDesktop.FindFirst( $TreeScope_Descendants, $oCondition, $pNotepad )
$oDesktop.FindFirst( $TreeScope_Descendants, $oCondition(), $pNotepad )

But you are using an additional line to create the $oCondition object. Since you are not calling any methods of the object, it seems to be an unnecessary additional line.

 

junkew, There are none of these points that cannot be implemented even if one uses a direct approach as demonstrated here. Using the Microsoft code in a direct way is simply not a limitation. You can make an example or code snippet covering each situation. I have already made a number of examples based on this technique, most of them in your own thread. There is nothing that can't be done.

Edited by LarsJ

Share this post


Link to post
Share on other sites
1 hour ago, LarsJ said:

it seems to be an unnecessary additional line

Could you please explain, why there is a constant increase in memory usage in this code?

#include "CUIAutomation2.au3"

HotKeySet("{esc}", "Quit")
Func Quit()
  Exit
EndFunc

Global $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
Global $pCondition, $pDesktop, $oDesktop, $pProgman, $oProgman
$oUIAutomation.GetRootElement( $pDesktop )
$oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

While 1
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Progman", $pCondition )
  $oDesktop.FindFirst( $TreeScope_Children, $pCondition, $pProgman )
  $oProgman = ObjCreateInterface( $pProgman, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj($oProgman) Then Exit
WEnd

But in this code, memory usage increases slightly and remains constant

#include "CUIAutomation2.au3"

HotKeySet("{esc}", "Quit")
Func Quit()
  Exit
EndFunc

Global $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
Global $pCondition, $oCondition, $pDesktop, $oDesktop, $pProgman, $oProgman
$oUIAutomation.GetRootElement( $pDesktop )
$oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )

While 1
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Progman", $pCondition )
  $oCondition = ObjCreateInterface( $pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition )
  $oDesktop.FindFirst( $TreeScope_Children, $oCondition, $pProgman )
  $oProgman = ObjCreateInterface( $pProgman, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj($oProgman) Then Exit
WEnd

It looks like a memory leak occurs when to use the pointer directly.

Share this post


Link to post
Share on other sites

What do you mean with UIA friendly?

Automating SCITE works fine for me (copy, pasted with simplespy merged and some small fixes on patterns for recognition)

;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP1=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=SciTE;controltype:=UIA_WindowControlTypeId;class:=SciTEWindow", $treescope_children)
;~ _UIA_Action($oP1,"setfocus")
_UIA_action($oP1,"highlight")

Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Toepassing;controltype:=UIA_MenuBarControlTypeId;class:=", $treescope_children)
;~ First find the object in the parent before you can do something
;~ $oUIElement=_UIA_getObjectByFindAll("Tools.mainwindow", "title:=Tools;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Tools;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")

Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Tools;controltype:=UIA_MenuControlTypeId;class:=#32768", $treescope_children)
_UIA_Action($oP0,"setfocus")
_UIA_Action($oP0,"highlight")

;~ First find the object in the parent before you can do something
;~ $oUIElement=_UIA_getObjectByFindAll("Go  F5.mainwindow", "title:=Go  F5;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Go  F5;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"highlight")
;~ _UIA_action($oUIElement,"click")

 

Share this post


Link to post
Share on other sites
On 12/23/2018 at 3:33 PM, junkew said:

What do you mean with UIA friendly? Automating SCITE works fine for me

hmmm, I'm trying to replicate the crashes I experienced before, ...but it wont crash. Tho my PC rebooted due to windows update. Maybe all it needed was a reboot.
Sowy, disregard my question and thanks for the answer  =) 

PS: image.png.647be33c8def5390045aed7ecb203647.png

...I'll look into it.

PS2: in SciTE v3.6.6 there is no crash but in SciTE v4.1.0 it crashes with the above Pic.
Be aware that I am not talking about running code. The crash is after I use, any tool really, to highlight the ID/Text/etc., and press F5 to run any code from the editor, is that the editor crashes.

PS3: use "SampleControls.au3" from your hard drive to replicate.

 

Edited by argumentum
deep, ( or not ), investigation result

Share this post


Link to post
Share on other sites

InnI, I can't explain that. But in relation to UI Automation code, it is not really a problem because we're dealing with a very limited number of objects.

Share this post


Link to post
Share on other sites
Posted (edited)

Based on something you wrote in 2014 in UIAutomation thread I am curious on if this still is a valid statement? why?
Any example where this has to be done on a propertycondition?

Quote

 

If you are running a 32 bit script, you must change the CreatePropertyCondition description in this way:


"CreatePropertyCondition hresult(int;uint64;uint64;ptr*);"

If you are running a 64 bit script, you must change the CreatePropertyCondition description in this way:


"CreatePropertyCondition hresult(int;ptr;ptr*);"

You need this description for an advanced data type like a structure (created with DllStructCreate). For simple data types like numbers and strings, you still use the old description with a variant. The reason for two new defitions are caused by differences in parameter transfer on 32 and 64 bit systems.

 

 I am working on (MIDL to AU3 converter) instead of CUIAutomation2.au3 we will have UIAutomationClient.au3 based on the IDL UIAutomationClient.IDL and then I will generate

;~ HRESULT CreatePropertyCondition ([in] PROPERTYID propertyId, [in] VARIANT value, [out, retval] IUIAutomationCondition ** newCondition );
 "CreatePropertyCondition hresult(struct;variant;ptr*);"

 

Edited by junkew

Share this post


Link to post
Share on other sites

This is still valid. It must be taken into account in situations where a method takes an array as input parameter and the array is not an array of variants.

If you search for VT_ARRAY in this list you'll get 13 matches. When you use any of these PropertyIds as input parameters you have to handle the arrays properly.

The problem arises because the internal AutoIt COM conversions that converts native AutoIt variables and arrays to COM compatible data types always convert a native AutoIt array into a safearray of variants.

But in these 13 situations the arrays are safearrays of integers, doubles and IUnknown pointers and thus not safearrays of variants. Here we have to handle the COM conversions ourselves.

When a safearray is passed as a parameter to a COM method it's usually passed as a variant ($tagVariant structure) where the variant data field contains a pointer to the safearray.

In 32 bit code parameters are passed through the stack. In 32 bit code a variant takes up 16 bytes. The purpose of "uint64;uint64;" is to pass the 16 bytes variant structure onto the stack wherefrom it's read by the internal method code.

In 64 bit code the first 4 parameters are passed through CPU registers. If one of the first 4 parameters is a structure and the structure isn't larger than 8 bytes (64 bit) it's passed directly through the CPU register. If it's larger than 8 bytes a pointer to the structure is passed through the CPU register.

In 64 bit code a variant takes up 24 bytes. The purpose of "ptr;" is to pass a pointer to the variant structure through a CPU register wherefrom the pointer and then the variant structure is read by the internal method code.


If you read the posts around the quoted post, you'll see a working example.


junkew, Let's make a little deal. I'll code a function to handle arrays as input parameters and post it here. Then you can copy the function directly into your new UDF. In return, I'll use your UDF in my code.

Share this post


Link to post
Share on other sites

thx for the explanation.

Deal is great :)

The new UIA UDF will be based on

  1. the UIA IDL files of Win10 SDK, (So I think 2-3 files)  generated by the converter
    https://www.autoitscript.com/forum/topic/197177-idl-to-au3-generator/ 
    Little in doubt if the generator is just good enough and some manual changes to be done afterward on generated au3 files (alternative is to build IDL lexer/parser but  more work or use OAIdl.Idl to make an .au3 file and use the typelib info to generate then the au3 files)
  2. UIAWrappers will be then based on
    • definitions of step 1
    • improved on finding on single propertyconditions (input you gave in this thread and UIA thread) where no regex finding is needed
  3. Extended simplespy with some addition coding examples when spied on object
  4. fixed/cleaned examples

 

 

 

Share this post


Link to post
Share on other sites

Automating Notepad with Sample code - step by step

This is a repetition of the Notepad example above. But this time, the possibilities for creating Sample code in UIASpy will be exploited.

Only the first part of the example will be reviewed once more in detail. But the pictures will not be repeated. So please open the previous example in a new tab in the browser.

Open "Examples\Notepad\Windows 7, 10\2) Sample code - step by step\Notepad.au3" (empty) in SciTE. The file is included in the zip-file in bottom of first post.

Open UIASpy (also included in zip-file) and keep UIASpy open throughout the entire example. Open Left pane main menu and click Delete top windows to delete all treeview windows except Desktop.


First step is to add initial code to create the UI Automation object and the Desktop element. Open Sample code main menu in UIASpy and click Initial code.

Open the menu once more and click Corrections to get an idea of the corrections that have to be made to this generic code.

Paste the code from clipboard into SciTE (Shift+Insert). The code should look like this:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )
EndFunc

; Code corrections:
; CUIAutomation2.au3 path
; Code to open target appl
; Create undeclared variables
; Delete double declared variables
; Correct $pWindow, $pParent and $pControl names
; Correct $oWindow, $oParent and $oControl names

Correct the path to CUIAutomation2.au3. Add code to open Notepad. Delete correction comments. Now the code should look like this:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\..\Includes\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Open Notepad
  Run( "Notepad" )
  Sleep( 1000 )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )
EndFunc

Run the code with F5 and leave Notepad open. Check that SciTE output is OK.
 

Next step is to get the Notepad window as an UI Automation element. Place mouse cursor over Notepad title bar and press F2.

Select the row in the Notepad detail information listview page in UIASpy that matches the row with the red arrow in the first picture in the previous example. Right-click and click Clear sample code in the context menu. Right-click once more and click Create sample code.

Open Sample code main menu and click Corrections to get an idea of the corrections that have to be made to the code.

Paste the code from clipboard into SciTE as the very last code (after EndFunc). The code should look like this:

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition0
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
ConsoleWrite( "$pCondition0 OK" & @CRLF )

Local $pWindow, $oWindow
$oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pWindow )
$oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
ConsoleWrite( "$oWindow OK" & @CRLF )

; Code corrections:
; CUIAutomation2.au3 path
; Code to open target appl
; Create undeclared variables
; Delete double declared variables
; Correct $pWindow, $pParent and $pControl names
; Correct $oWindow, $oParent and $oControl names

Replace "window/control" with "Notepad window". Replace "Window" with "Notepad". Delete correction comments. Move code into function:

; --- Find Notepad window ---

ConsoleWrite( "--- Find Notepad window ---" & @CRLF )

Local $pCondition0
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
ConsoleWrite( "$pCondition0 OK" & @CRLF )

Local $pNotepad, $oNotepad
$oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pNotepad )
$oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF )
ConsoleWrite( "$oNotepad OK" & @CRLF )

Close the previous Notepad, run the code and leave Notepad open. Check that SciTE output is OK.
 

Third step is to fill out the Edit control with "HelloWorld".

An UI Automation task is almost always done in two or three steps:
1. Create condition to find element
2. Find the UI Automation element
3. Extract info or perform action

Each of these steps can usually be done in 1 - 3 lines of UI Automation code. All three steps can usually be done in less that 10 lines of UI Automation code (common AutoIt debug/info code not included).

To get the Notepad window above we only used two steps.

Place mouse cursor in the Edit control in Notepad and press F1.

Select the row in the Edit detail information listview page in UIASpy that matches the row with the red arrow in the second picture in the previous example. Right-click and click Clear sample code in the context menu. Right-click once more and click Create sample code.

Open Sample code main menu and click Corrections to get an idea of the corrections that have to be made to the code.

Paste the code from clipboard into SciTE as the very last code (after EndFunc). The code should look like this:

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition0
$oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition0 )
If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
ConsoleWrite( "$pCondition0 OK" & @CRLF )

Local $pEdit, $oEdit
$oParent.FindFirst( $TreeScope_Descendants, $pCondition0, $pEdit )
$oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
ConsoleWrite( "$oEdit OK" & @CRLF )

; Code corrections:
; CUIAutomation2.au3 path
; Code to open target appl
; Create undeclared variables
; Delete double declared variables
; Correct $pWindow, $pParent and $pControl names
; Correct $oWindow, $oParent and $oControl names

Replace "Find window/control" with "Fill Edit control". Delete the line "Local $pCondition0". Replace "$oParent" with "$oNotepad". Delete correction comments. Move code into function:

; --- Fill Edit control ---

ConsoleWrite( "--- Fill Edit control ---" & @CRLF )

$oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition0 )
If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
ConsoleWrite( "$pCondition0 OK" & @CRLF )

Local $pEdit, $oEdit
$oNotepad.FindFirst( $TreeScope_Descendants, $pCondition0, $pEdit )
$oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
ConsoleWrite( "$oEdit OK" & @CRLF )

Close the previous Notepad, run the code and leave Notepad open. Check that SciTE output is OK.
 

Place mouse cursor in the Edit control in Notepad and press F1. If there is a red (invalid) Notepad window in the treeview then delete it (Right-click the red element and click Delete element and all childs).

Scroll down to the Control Patterns (element actions) section in the Edit detail information listview page in UIASpy.

Select the row "$UIA_IsValuePatternAvailablePropertyId" that matches the row with the red arrow in the third picture in the previous example. Right-click and click Clear sample code in the context menu. Right-click once more and click Create sample code.

Click the Edit item in the treeview to return to the Edit detail information listview page. Scroll down to the bottom of the listview page. The Value Pattern Methods subsection should be visible.

Select the row with the SetValue(bstr) method in second column:

rFYVAs0.png

Right-click and click Create sample code (don't clear the code this time).

Paste the code from clipboard into SciTE as the very last code (after EndFunc). The code should look like this:

; --- Value Pattern (action) Object ---

ConsoleWrite( "--- Value Pattern (action) Object ---" & @CRLF )

Local $pValuePattern, $oValuePattern
$oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
$oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
ConsoleWrite( "$oValuePattern OK" & @CRLF )

; --- Value Pattern (action) Methods ---

ConsoleWrite( "--- Value Pattern (action) Methods ---" & @CRLF )

$oValuePattern.SetValue(bstr)

Delete the two comments, two ConsoleWrites and empty lines. Replace "bstr" with "HelloWorld". Move code into function:

Local $pValuePattern, $oValuePattern
$oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
$oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
ConsoleWrite( "$oValuePattern OK" & @CRLF )
$oValuePattern.SetValue( "HelloWorld" )

Close the previous Notepad, run the code, check that SciTE output is OK, check "HelloWorld" in Notepad Edit control.
 

All code after the third step:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\..\Includes\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Open Notepad
  Run( "Notepad" )
  Sleep( 1000 )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find Notepad window ---

  ConsoleWrite( "--- Find Notepad window ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pNotepad, $oNotepad
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pNotepad )
  $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF )
  ConsoleWrite( "$oNotepad OK" & @CRLF )

  ; --- Fill Edit control ---

  ConsoleWrite( "--- Fill Edit control ---" & @CRLF )

  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pEdit, $oEdit
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition0, $pEdit )
  $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
  ConsoleWrite( "$oEdit OK" & @CRLF )

  Local $pValuePattern, $oValuePattern
  $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
  $oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
  If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
  ConsoleWrite( "$oValuePattern OK" & @CRLF )
  $oValuePattern.SetValue( "HelloWorld" )
EndFunc

SciTE output:

$oUIAutomation OK
$oDesktop OK
--- Find Notepad window ---
$pCondition0 OK
$oNotepad OK
--- Fill Edit control ---
$pCondition0 OK
$oEdit OK
$oValuePattern OK

 

Edited by LarsJ

Share this post


Link to post
Share on other sites

Automating Notepad with Sample code - all at once

In an automation task like the Notepad example above, it's much faster to create all sample code at once than it is to create the sample code in a step-by-step procedure.

For pattern methods, it's also faster to create sample code by selecting the method through the Pattern... item in the Sample code menu in UIASpy than it is to create pattern method sample code through the detail information listview page.

xXnvTys.png

But to use the pattern listview page you need to know which pattern to use. The advantage of using the detail information listview page is that it only shows the patterns and pattern methods that are applicable to the specific control.
 

These are all the steps in the Notepad example:

  • Add initial code
  • Open Notepad
    • Sleep( 1000 )
  • Get Notepad window
  • Fill Edit control
    • Identify Edit control
    • Execute SetValue pattern method
  • Open File menu
    • Identify File menu
    • Execute Invoke pattern method
    • Sleep( 100 )
  • Click Save As... menu item
    • Identify Save As... menu item
    • Execute Invoke pattern method
    • Sleep( 1000 )
  • Get Save As window
  • Fill Edit control (File name)
    • Identify Edit control
    • Execute SetValue pattern method
  • Click Save button
    • Identify Save button
    • Execute Invoke pattern method
  • Corrections

And this is sample code for all the steps in the entire Notepad example.

Examples\Notepad\Windows 7, 10\3) Sample code - all at once\1) Before corrections.au3:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )
EndFunc

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition0
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
ConsoleWrite( "$pCondition0 OK" & @CRLF )

Local $pWindow, $oWindow
$oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pWindow )
$oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
ConsoleWrite( "$oWindow OK" & @CRLF )

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition1
$oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition1 )
If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
ConsoleWrite( "$pCondition1 OK" & @CRLF )

Local $pEdit, $oEdit
$oParent.FindFirst( $TreeScope_Descendants, $pCondition1, $pEdit )
$oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
ConsoleWrite( "$oEdit OK" & @CRLF )

; --- Value Pattern (action) Object ---

ConsoleWrite( "--- Value Pattern (action) Object ---" & @CRLF )

Local $pValuePattern, $oValuePattern
$oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
$oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
ConsoleWrite( "$oValuePattern OK" & @CRLF )
$oValuePattern.SetValue(bstr)

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition2, $pCondition3, $pAndCondition3
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition2 )
$oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition3 )
$oUIAutomation.CreateAndCondition( $pCondition2, $pCondition3, $pAndCondition3 )
If Not $pAndCondition3 Then Return ConsoleWrite( "$pAndCondition3 ERR" & @CRLF )
ConsoleWrite( "$pAndCondition3 OK" & @CRLF )

Local $pMenuItem, $oMenuItem
$oParent.FindFirst( $TreeScope_Descendants, $pAndCondition3, $pMenuItem )
$oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF )
ConsoleWrite( "$oMenuItem OK" & @CRLF )

; --- Invoke Pattern (action) Object ---

ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

Local $pInvokePattern, $oInvokePattern
$oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
$oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
ConsoleWrite( "$oInvokePattern OK" & @CRLF )
$oInvokePattern.Invoke()

Sleep( 1000 )

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition4, $pCondition5, $pAndCondition5
$oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition4 )
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition5 )
$oUIAutomation.CreateAndCondition( $pCondition4, $pCondition5, $pAndCondition5 )
If Not $pAndCondition5 Then Return ConsoleWrite( "$pAndCondition5 ERR" & @CRLF )
ConsoleWrite( "$pAndCondition5 OK" & @CRLF )

Local $pMenuItem, $oMenuItem
$oParent.FindFirst( $TreeScope_Descendants, $pAndCondition5, $pMenuItem )
$oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF )
ConsoleWrite( "$oMenuItem OK" & @CRLF )

; --- Invoke Pattern (action) Object ---

ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

Local $pInvokePattern, $oInvokePattern
$oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
$oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
ConsoleWrite( "$oInvokePattern OK" & @CRLF )
$oInvokePattern.Invoke()

Sleep( 1000 )

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition6
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition6 )
If Not $pCondition6 Then Return ConsoleWrite( "$pCondition6 ERR" & @CRLF )
ConsoleWrite( "$pCondition6 OK" & @CRLF )

Local $pWindow, $oWindow
$oParent.FindFirst( $TreeScope_Descendants, $pCondition6, $pWindow )
$oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
ConsoleWrite( "$oWindow OK" & @CRLF )

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition7
$oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1001", $pCondition7 )
If Not $pCondition7 Then Return ConsoleWrite( "$pCondition7 ERR" & @CRLF )
ConsoleWrite( "$pCondition7 OK" & @CRLF )

Local $pEdit, $oEdit
$oParent.FindFirst( $TreeScope_Descendants, $pCondition7, $pEdit )
$oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
ConsoleWrite( "$oEdit OK" & @CRLF )

; --- Value Pattern (action) Object ---

ConsoleWrite( "--- Value Pattern (action) Object ---" & @CRLF )

Local $pValuePattern, $oValuePattern
$oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
$oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
ConsoleWrite( "$oValuePattern OK" & @CRLF )
$oValuePattern.SetValue(bstr)

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition8, $pCondition9, $pAndCondition9
$oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition8 )
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition9 )
$oUIAutomation.CreateAndCondition( $pCondition8, $pCondition9, $pAndCondition9 )
If Not $pAndCondition9 Then Return ConsoleWrite( "$pAndCondition9 ERR" & @CRLF )
ConsoleWrite( "$pAndCondition9 OK" & @CRLF )

Local $pButton, $oButton
$oParent.FindFirst( $TreeScope_Descendants, $pAndCondition9, $pButton )
$oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF )
ConsoleWrite( "$oButton OK" & @CRLF )

; --- Invoke Pattern (action) Object ---

ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

Local $pInvokePattern, $oInvokePattern
$oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
$oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
ConsoleWrite( "$oInvokePattern OK" & @CRLF )
$oInvokePattern.Invoke()

; Code corrections:
; CUIAutomation2.au3 path
; Code to open target appl
; Create undeclared variables
; Delete double declared variables
; Correct $pWindow, $pParent and $pControl names
; Correct $oWindow, $oParent and $oControl names

The code isn't immediately executable. It's necessary to make some corrections. Most corrections are listed at the bottom of the code as comments.

Note that all condition variables are created properly. They need no corrections.

This is the code after corrections.

Examples\Notepad\Windows 7, 10\3) Sample code - all at once\2) After corrections.au3:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\..\Includes\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Open Notepad
  Run( "Notepad" )
  Sleep( 1000 )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Get Notepad window ---

  ConsoleWrite( "--- Get Notepad window ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pNotepad, $oNotepad
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pNotepad )
  $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF )
  ConsoleWrite( "$oNotepad OK" & @CRLF )

  ; --- Fill Edit control ---

  ConsoleWrite( "--- Fill Edit control ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pEdit, $oEdit
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition1, $pEdit )
  $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
  ConsoleWrite( "$oEdit OK" & @CRLF )

  Local $pValuePattern, $oValuePattern
  $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
  $oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
  If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
  ConsoleWrite( "$oValuePattern OK" & @CRLF )
  $oValuePattern.SetValue( "HelloWorld" )

  ; --- Open File menu ---

  ConsoleWrite( "--- Open File menu ---" & @CRLF )

  Local $pCondition2, $pCondition3, $pAndCondition3
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition2 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition3 ) ; <<<<<<<<<<<<<<<<<<<<<<<<
  $oUIAutomation.CreateAndCondition( $pCondition2, $pCondition3, $pAndCondition3 )
  If Not $pAndCondition3 Then Return ConsoleWrite( "$pAndCondition3 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition3 OK" & @CRLF )

  Local $pMenuItem, $oMenuItem
  $oNotepad.FindFirst( $TreeScope_Descendants, $pAndCondition3, $pMenuItem )
  $oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF )
  ConsoleWrite( "$oMenuItem OK" & @CRLF )

  Local $pInvokePattern, $oInvokePattern
  $oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
  $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern OK" & @CRLF )
  $oInvokePattern.Invoke()
  Sleep( 100 )

  ; --- Click Save As... menu item ---

  ConsoleWrite( "--- Click Save As... menu item ---" & @CRLF )

  Local $pCondition4, $pCondition5, $pAndCondition5
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition4 )
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition5 )
  $oUIAutomation.CreateAndCondition( $pCondition4, $pCondition5, $pAndCondition5 )
  If Not $pAndCondition5 Then Return ConsoleWrite( "$pAndCondition5 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition5 OK" & @CRLF )

  $oNotepad.FindFirst( $TreeScope_Descendants, $pAndCondition5, $pMenuItem )
  $oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF )
  ConsoleWrite( "$oMenuItem OK" & @CRLF )

  $oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
  $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern OK" & @CRLF )
  $oInvokePattern.Invoke()
  Sleep( 1000 )

  ; --- Get Save As window ---

  ConsoleWrite( "--- Get Save As window ---" & @CRLF )

  Local $pCondition6
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition6 )
  If Not $pCondition6 Then Return ConsoleWrite( "$pCondition6 ERR" & @CRLF )
  ConsoleWrite( "$pCondition6 OK" & @CRLF )

  Local $pSaveAsWin, $oSaveAsWin
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition6, $pSaveAsWin )
  $oSaveAsWin = ObjCreateInterface( $pSaveAsWin, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oSaveAsWin ) Then Return ConsoleWrite( "$oSaveAsWin ERR" & @CRLF )
  ConsoleWrite( "$oSaveAsWin OK" & @CRLF )

  ; --- Fill Edit control (File name) ---

  ConsoleWrite( "--- Fill Edit control (File name) ---" & @CRLF )

  Local $pCondition7
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1001", $pCondition7 )
  If Not $pCondition7 Then Return ConsoleWrite( "$pCondition7 ERR" & @CRLF )
  ConsoleWrite( "$pCondition7 OK" & @CRLF )

  $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition7, $pEdit )
  $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
  ConsoleWrite( "$oEdit OK" & @CRLF )

  $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValuePattern )
  $oValuePattern = ObjCreateInterface( $pValuePattern, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
  If Not IsObj( $oValuePattern ) Then Return ConsoleWrite( "$oValuePattern ERR" & @CRLF )
  ConsoleWrite( "$oValuePattern OK" & @CRLF )
  $oValuePattern.SetValue( "HelloWorld.txt" )

  ; --- Click Save button ---

  ConsoleWrite( "--- Click Save button ---" & @CRLF )

  Local $pCondition8, $pCondition9, $pAndCondition9
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition8 )
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition9 )
  $oUIAutomation.CreateAndCondition( $pCondition8, $pCondition9, $pAndCondition9 )
  If Not $pAndCondition9 Then Return ConsoleWrite( "$pAndCondition9 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition9 OK" & @CRLF )

  Local $pButton, $oButton
  $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pAndCondition9, $pButton )
  $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF )
  ConsoleWrite( "$oButton OK" & @CRLF )

  $oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
  $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern OK" & @CRLF )
  $oInvokePattern.Invoke()
EndFunc

When executing the code, it should perform all Notepad automation tasks and generate this output in SciTE.

SciTE output:

$oUIAutomation OK
$oDesktop OK
--- Get Notepad window ---
$pCondition0 OK
$oNotepad OK
--- Fill Edit control ---
$pCondition1 OK
$oEdit OK
$oValuePattern OK
--- Open File menu ---
$pAndCondition3 OK
$oMenuItem OK
$oInvokePattern OK
--- Click Save As... menu item ---
$pAndCondition5 OK
$oMenuItem OK
$oInvokePattern OK
--- Get Save As window ---
$pCondition6 OK
$oSaveAsWin OK
--- Fill Edit control (File name) ---
$pCondition7 OK
$oEdit OK
$oValuePattern OK
--- Click Save button ---
$pAndCondition9 OK
$oButton OK
$oInvokePattern OK

 

Edited by LarsJ

Share this post


Link to post
Share on other sites

Automating Notepad - Windows XP
This is a repetition of the Notepad example above. But this time under Windows XP.

I'll not review the whole example once more but only focus on the challenges of Windows XP. These challenges are about not all pattern methods working under Windows XP.

There are two cases where the pattern methods does not work.

The first case is to open the File menu by clicking the menu item. Here, the Invoke method does not work. The only thing you can do is use the bounding rectangle to position the mouse over the File menu and perform a mouse click:

; --- Open File menu ---

ConsoleWrite( "--- Open File menu ---" & @CRLF )

Local $pCondition1
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 )
If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
ConsoleWrite( "$pCondition1 OK" & @CRLF )

Local $pCondition2 ; $UIA_NamePropertyId is LOCALIZED and maybe CASE SENSITIVE
$oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition2 ) ; File <<<<<<<<<<<<<<<<<<<<
If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
ConsoleWrite( "$pCondition2 OK" & @CRLF )

; And condition
$oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
ConsoleWrite( "$pCondition OK" & @CRLF )

Local $pFile, $oFile
$oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pFile )
$oFile = ObjCreateInterface( $pFile, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oFile ) Then Return ConsoleWrite( "$oFile ERR" & @CRLF )
ConsoleWrite( "$oFile OK" & @CRLF )

; Rectangle
Local $aRect ; l, t, w, h
$oFile.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $aRect )
If Not IsArray( $aRect ) Then Return ConsoleWrite( "$aRect ERR" & @CRLF )
ConsoleWrite( "$aRect OK" & @CRLF )

; Click File menu
MouseClick( "primary", $aRect[0]+$aRect[2]/2, $aRect[1]+$aRect[3]/2, 1, 0 )
Sleep( 100 )

 

The second case is to fill out the File name (an Edit control) in the "Save As ..." dialog. The SetValue pattern method does not work in this Edit control. The only thing that works is the Send command.

Note that the File name control gets focus when the "Save As ..." dialog opens and that the text in Edit control is selected:

; --- Set File name ---

ConsoleWrite( "--- Set File name ---" & @CRLF )

ConsoleWrite( "Send()" & @CRLF )
Send( "HelloWorld.txt", 1 )

 

All Windows XP code.

Examples\Notepad\Windows XP\1) Create code\NotepadAll.au3:

#include "..\..\..\..\Includes\CUIAutomation2.au3"

Example()

Func Example()
  ; Open Notepad
  Run( "Notepad" )
  Sleep( 1000 )

  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Notepad window ---

  ConsoleWrite( "--- Notepad window ---" & @CRLF )

  Local $pCondition ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Notepad", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pNotepad, $oNotepad
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition, $pNotepad )
  $oNotepad = ObjCreateInterface( $pNotepad, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oNotepad ) Then Return ConsoleWrite( "$oNotepad ERR" & @CRLF )
  ConsoleWrite( "$oNotepad OK" & @CRLF )

  ; --- Fill Edit element ---

  ConsoleWrite( "--- Fill Edit element ---" & @CRLF )

  ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "15", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pEdit, $oEdit
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pEdit )
  $oEdit = ObjCreateInterface( $pEdit, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oEdit ) Then Return ConsoleWrite( "$oEdit ERR" & @CRLF )
  ConsoleWrite( "$oEdit OK" & @CRLF )

  Local $pValue, $oValue
  $oEdit.GetCurrentPattern( $UIA_ValuePatternId, $pValue )
  $oValue = ObjCreateInterface( $pValue, $sIID_IUIAutomationValuePattern, $dtagIUIAutomationValuePattern )
  If Not IsObj( $oValue ) Then Return ConsoleWrite( "$oValue ERR" & @CRLF )
  ConsoleWrite( "$oValue OK" & @CRLF )
  $oValue.SetValue( "HelloWorld" )

  ; --- Open File menu ---

  ConsoleWrite( "--- Open File menu ---" & @CRLF )

  Local $pCondition1
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  Local $pCondition2 ; $UIA_NamePropertyId is LOCALIZED and maybe CASE SENSITIVE
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "File", $pCondition2 ) ; File <<<<<<<<<<<<<<<<<<<<
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pFile, $oFile
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pFile )
  $oFile = ObjCreateInterface( $pFile, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oFile ) Then Return ConsoleWrite( "$oFile ERR" & @CRLF )
  ConsoleWrite( "$oFile OK" & @CRLF )

  ; Rectangle
  Local $aRect ; l, t, w, h
  $oFile.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $aRect )
  If Not IsArray( $aRect ) Then Return ConsoleWrite( "$aRect ERR" & @CRLF )
  ConsoleWrite( "$aRect OK" & @CRLF )

  ; Click File menu
  MouseClick( "primary", $aRect[0]+$aRect[2]/2, $aRect[1]+$aRect[3]/2, 1, 0 )
  Sleep( 100 )

  ; --- Click "Save As..." menu ---

  ConsoleWrite( "--- Click ""Save As..."" menu ---" & @CRLF )

  ; Reuse $pCondition1 above

  ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "4", $pCondition2 )
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pSaveAs, $oSaveAs
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAs )
  $oSaveAs = ObjCreateInterface( $pSaveAs, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oSaveAs ) Then Return ConsoleWrite( "$oSaveAs ERR" & @CRLF )
  ConsoleWrite( "$oSaveAs OK" & @CRLF )

  Local $pInvoke, $oInvoke
  $oSaveAs.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()
  Sleep( 1000 )

  ; --- Save As window ---

  ConsoleWrite( "--- Save As window ---" & @CRLF )

  ; Note that $UIA_ClassNamePropertyId maybe ia a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "#32770", $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pSaveAsWin, $oSaveAsWin, $hSaveAsWin
  $oNotepad.FindFirst( $TreeScope_Descendants, $pCondition, $pSaveAsWin )
  $oSaveAsWin = ObjCreateInterface( $pSaveAsWin, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  $oSaveAsWin.GetCurrentPropertyValue( $UIA_NativeWindowHandlePropertyId, $hSaveAsWin )
  If Not IsObj( $oSaveAsWin ) Then Return ConsoleWrite( "$oSaveAsWin ERR" & @CRLF )

  ConsoleWrite( "$oSaveAsWin OK" & @CRLF )

  ; --- Set File name ---

  ConsoleWrite( "--- Set File name ---" & @CRLF )

  ConsoleWrite( "Send()" & @CRLF )
  Send( "HelloWorld.txt", 1 )

  ; --- Click Save button ---

  ConsoleWrite( "--- Click Save button ---" & @CRLF )

  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 )
  If Not $pCondition1 Then Return ConsoleWrite( "$pCondition1 ERR" & @CRLF )
  ConsoleWrite( "$pCondition1 OK" & @CRLF )

  ; Note that $UIA_AutomationIdPropertyId is a STRING and maybe a CASE SENSITIVE condition
  $oUIAutomation.CreatePropertyCondition( $UIA_AutomationIdPropertyId, "1", $pCondition2 )
  If Not $pCondition2 Then Return ConsoleWrite( "$pCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pCondition2 OK" & @CRLF )

  ; And condition
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
  ConsoleWrite( "$pCondition OK" & @CRLF )

  Local $pSave, $oSave
  $oSaveAsWin.FindFirst( $TreeScope_Descendants, $pCondition, $pSave )
  $oSave = ObjCreateInterface( $pSave, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oSave ) Then Return ConsoleWrite( "$oSave ERR" & @CRLF )
  ConsoleWrite( "$oSave OK" & @CRLF )

  $oSave.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
  $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
  ConsoleWrite( "$oInvoke OK" & @CRLF )
  $oInvoke.Invoke()
EndFunc

 

SciTE output:

$oUIAutomation OK
$oDesktop OK
--- Notepad window ---
$pCondition OK
$oNotepad OK
--- Fill Edit element ---
$pCondition OK
$oEdit OK
$oValue OK
--- Open File menu ---
$pCondition1 OK
$pCondition2 OK
$pCondition OK
$oFile OK
$aRect OK
--- Click "Save As..." menu ---
$pCondition2 OK
$pCondition OK
$oSaveAs OK
$oInvoke OK
--- Save As window ---
$pCondition OK
$oSaveAsWin OK
--- Set File name ---
Send()
--- Click Save button ---
$pCondition1 OK
$pCondition2 OK
$pCondition OK
$oSave OK
$oInvoke OK

 

Edited by LarsJ

Share this post


Link to post
Share on other sites

Chrome - Clicking an extension
A day ago, there was a question of clicking a Chrome extension.

This is a demonstration of how it can be done with UIASpy. Here the "Blank New Tab Page" button is clicked. Chrome is already open and displaying the AutoIt start page.

Because this is about the outer elements of Chrome it's important that accessibility is not enabled.

Open UIASpy, turn off Only visible elements in the Left pane menu, place the mouse cursor over the "Blank New Tab Page" button and press F1:
 

wLJCppY.png

XPUhYCz.png

The butten is located below a pane control. That's why Only visible elements has to be turned off. This is also the reason why the button cannot be identified with the AutoIt Window Info tool. And junkew's simplespy.au3 will identify the pane control instead of the button.


So we are not at all interested in the pane control. We are interested in the Chrome top window:

fEwsaPD.png

Open Sample code main menu, click Initial code, click the selected treeview item to return to the detail info page, right click the selected row and click Create sample code.


Then we are interested in the "Blank New Tab Page" button:

RrLGUQb.png

Repeat this for each group of selected row(s): Right click a selected row and click Create sample code.


Click the "Blank New Tab Page" button in Chrome to open a menu (see the picture above). Place the mouse cursor over the "Blank New Tab Page" menu item and press F1. The menu is closed when UIASpy gets focus. That's why the treeview items are red:

gCINU9U.png

Repeat this for each group of selected row(s): Right click a selected row and click Create sample code.


Because I've tested the code, I know that the Invoke method doesn't work for the "Blank New Tab Page" menu item. We have to use a mouse click:

5m4WcW2.png

Right click the selected row and click Create sample code.


Open Sample code main menu, click Corrections. Paste the code into an editor.


All sample code (Examples\Chrome\Click extension button\1) Before corrections.au3) should look like this:

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )
EndFunc

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition0
$oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Chrome_WidgetWin_1", $pCondition0 )
If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
ConsoleWrite( "$pCondition0 OK" & @CRLF )

Local $pPane, $oPane
$oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pPane )
$oPane = ObjCreateInterface( $pPane, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oPane ) Then Return ConsoleWrite( "$oPane ERR" & @CRLF )
ConsoleWrite( "$oPane OK" & @CRLF )

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition1, $pCondition2, $pAndCondition2
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 )
$oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Blank New Tab Page", $pCondition2 )
$oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pAndCondition2 )
If Not $pAndCondition2 Then Return ConsoleWrite( "$pAndCondition2 ERR" & @CRLF )
ConsoleWrite( "$pAndCondition2 OK" & @CRLF )

Local $pButton, $oButton
$oParent.FindFirst( $TreeScope_Descendants, $pAndCondition2, $pButton )
$oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF )
ConsoleWrite( "$oButton OK" & @CRLF )

; --- Invoke Pattern (action) Object ---

ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

Local $pInvokePattern, $oInvokePattern
$oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
$oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
ConsoleWrite( "$oInvokePattern OK" & @CRLF )

; --- Invoke Pattern (action) Methods ---

ConsoleWrite( "--- Invoke Pattern (action) Methods ---" & @CRLF )

$oInvokePattern.Invoke()

; --- Find window/control ---

ConsoleWrite( "--- Find window/control ---" & @CRLF )

Local $pCondition3, $pCondition4, $pAndCondition4
$oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition3 )
$oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Blank New Tab Page", $pCondition4 )
$oUIAutomation.CreateAndCondition( $pCondition3, $pCondition4, $pAndCondition4 )
If Not $pAndCondition4 Then Return ConsoleWrite( "$pAndCondition4 ERR" & @CRLF )
ConsoleWrite( "$pAndCondition4 OK" & @CRLF )

Local $pMenuItem, $oMenuItem
$oParent.FindFirst( $TreeScope_Descendants, $pAndCondition4, $pMenuItem )
$oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF )
ConsoleWrite( "$oMenuItem OK" & @CRLF )

; --- Invoke Pattern (action) Object ---

ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

Local $pInvokePattern, $oInvokePattern
$oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
$oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
ConsoleWrite( "$oInvokePattern OK" & @CRLF )
$oInvokePattern.Invoke()

; --- Element Properties (information) ---

ConsoleWrite( "--- Element Properties (information) ---" & @CRLF )

Local $vValue
$oMenuItem.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $vValue )

; Code corrections:
; CUIAutomation2.au3 path
; Code to open target appl
; Create undeclared variables
; Delete double declared variables
; Correct $pWindow, $pParent and $pControl names
; Correct $oWindow, $oParent and $oControl names


Code after corrections (Examples\Chrome\Click extension button\2) After corrections.au3):

#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7

;#AutoIt3Wrapper_UseX64=n ; If target application is running as 32 bit code
;#AutoIt3Wrapper_UseX64=y ; If target application is running as 64 bit code

#include "..\..\..\Includes\CUIAutomation2.au3"

Opt( "MustDeclareVars", 1 )

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  $oDesktop = ObjCreateInterface( $pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oDesktop ) Then Return ConsoleWrite( "$oDesktop ERR" & @CRLF )
  ConsoleWrite( "$oDesktop OK" & @CRLF )

  ; --- Find Chrome window ---

  ConsoleWrite( "--- Find Chrome window ---" & @CRLF )

  Local $pCondition0
  $oUIAutomation.CreatePropertyCondition( $UIA_ClassNamePropertyId, "Chrome_WidgetWin_1", $pCondition0 )
  If Not $pCondition0 Then Return ConsoleWrite( "$pCondition0 ERR" & @CRLF )
  ConsoleWrite( "$pCondition0 OK" & @CRLF )

  Local $pChrome, $oChrome
  $oDesktop.FindFirst( $TreeScope_Descendants, $pCondition0, $pChrome )
  $oChrome = ObjCreateInterface( $pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oChrome ) Then Return ConsoleWrite( "$oChrome ERR" & @CRLF )
  ConsoleWrite( "$oChrome OK" & @CRLF )

  ; --- Find "Blank New Tab Page" button ---

  ConsoleWrite( "--- Find ""Blank New Tab Page"" button ---" & @CRLF )

  Local $pCondition1, $pCondition2, $pAndCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Blank New Tab Page", $pCondition2 )
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pAndCondition2 )
  If Not $pAndCondition2 Then Return ConsoleWrite( "$pAndCondition2 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition2 OK" & @CRLF )

  Local $pButton, $oButton
  $oChrome.FindFirst( $TreeScope_Descendants, $pAndCondition2, $pButton )
  $oButton = ObjCreateInterface( $pButton, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oButton ) Then Return ConsoleWrite( "$oButton ERR" & @CRLF )
  ConsoleWrite( "$oButton OK" & @CRLF )

  ; --- Invoke Pattern (action) Object ---

  ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

  Local $pInvokePattern, $oInvokePattern
  $oButton.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
  $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern OK" & @CRLF )

  ; --- Invoke Pattern (action) Methods ---

  ConsoleWrite( "--- Invoke Pattern (action) Methods ---" & @CRLF )

  $oInvokePattern.Invoke()
  Sleep( 100 )

  ; --- Find "Blank New Tab Page" menu item ---

  ConsoleWrite( "--- Find ""Blank New Tab Page"" menu item ---" & @CRLF )

  Local $pCondition3, $pCondition4, $pAndCondition4
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition3 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "Blank New Tab Page", $pCondition4 )
  $oUIAutomation.CreateAndCondition( $pCondition3, $pCondition4, $pAndCondition4 )
  If Not $pAndCondition4 Then Return ConsoleWrite( "$pAndCondition4 ERR" & @CRLF )
  ConsoleWrite( "$pAndCondition4 OK" & @CRLF )

  Local $pMenuItem, $oMenuItem
  $oChrome.FindFirst( $TreeScope_Descendants, $pAndCondition4, $pMenuItem )
  $oMenuItem = ObjCreateInterface( $pMenuItem, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
  If Not IsObj( $oMenuItem ) Then Return ConsoleWrite( "$oMenuItem ERR" & @CRLF )
  ConsoleWrite( "$oMenuItem OK" & @CRLF )

  #cs
  ; --- Invoke Pattern (action) Object ---

  ConsoleWrite( "--- Invoke Pattern (action) Object ---" & @CRLF )

  $oMenuItem.GetCurrentPattern( $UIA_InvokePatternId, $pInvokePattern )
  $oInvokePattern = ObjCreateInterface( $pInvokePattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
  If Not IsObj( $oInvokePattern ) Then Return ConsoleWrite( "$oInvokePattern ERR" & @CRLF )
  ConsoleWrite( "$oInvokePattern OK" & @CRLF )
  $oInvokePattern.Invoke()
  #ce

  ; Rectangle
  Local $aRect ; l, t, w, h
  $oMenuItem.GetCurrentPropertyValue( $UIA_BoundingRectanglePropertyId, $aRect )
  If Not IsArray( $aRect ) Then Return ConsoleWrite( "$aRect ERR" & @CRLF )
  ConsoleWrite( "$aRect OK" & @CRLF )

  ; Click menu item
  MouseClick( "primary", $aRect[0]+$aRect[2]/2, $aRect[1]+$aRect[3]/2, 1, 0 )
EndFunc


SciTE output:

$oUIAutomation OK
$oDesktop OK
--- Find Chrome window ---
$pCondition0 OK
$oChrome OK
--- Find "Blank New Tab Page" button ---
$pAndCondition2 OK
$oButton OK
--- Invoke Pattern (action) Object ---
$oInvokePattern OK
--- Invoke Pattern (action) Methods ---
--- Find "Blank New Tab Page" menu item ---
$pAndCondition4 OK
$oMenuItem OK
$aRect OK


2zfqOmI.png


New zip-file at bottom of first post.

Edited by LarsJ
A few text updates

Share this post


Link to post
Share on other sites

Nice examples. Simplespy just uses elementfrompoint and you could argue pane as child from button is not a logical gui tree. Similar behavior you can see when automating ms access forms. An improved spy i tried scan hierarchy elementfrompoint up to desktop and identify object with lowest squared size height times width but also that leads to issues.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...