UIA Need to speed this code up

junkew · December 12, 2017

You should understand the tree hierarchy. First find (grand)parents before you act on leafchilds of the tree.

;Do the actions in the popup
_UIA_Action("oP1Popup.oUIElement2","setfocus") ;Focus on the "button"
_UIA_Action("oP1Popup.oUIElement2","click") ;Click the button

As you only act on the leafchild the uiawrappers tries to search from desktop, all windows, all subsubsubchilds

1. So add a setfocus on the popupwindow and you will be much better of

_UIA_Action("oP1popup.mainwindow","setfocus"))

2. Maybe needed but normally not when you act on a .mainwindow (as that implicitly sets the searchcontext)

_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oP1popup.mainwindow","getobject"))

could be that 1 and 2 is needed
May suggestion is to make your full script either with UIA or with AutoIt commands. I would not blend them

junkew · December 12, 2017

registry logic to make alternative direction complete (google and you will find which entries to manipulate)

#include <MsgBoxConstants.au3>

Local $sVar = RegRead("HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.mp3\OpenWithList", "a")
MsgBox($MB_SYSTEMMODAL, "Your MP3 player is:", $sVar)

If its empty its the default of your OS

use the HKEY_CURRENT_USER\Software\Classes or HKEY_LOCAL_MACHINE\Software\Classes

manipulating registry can be more tricky but is fast ;-)

You have to edit / change multiple ones

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.mp3\UserChoice

careca · December 12, 2017

Registry isn't tricky, it just doesn't do it for me, even tried the "assoc" and "ftype" embedded in cmd, but the application was never set.

Even changed the entries in ROOT and still it's like it doesn't refresh the change, even after restart.

That registry key you show right there, adds the application to the list in the popup, so that's one way to go about it, add to the list, and then allow user to select it, but i would prefer to make it all automatic.

The lines provided didn't speed it up, takes about the same time.

Quote

" As you only act on the leafchild the uiawrappers tries to search from desktop, all windows, all subsubsubchilds "

I thought this line was to set preciselly where i need it to search?

_UIA_setVar("oP1Popup.oUIElement2","Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink")

After all, the title is provided, why will it search all windows and all subsubchilds?

As i said before, i find this very confusing. Once i thought it was/could be a bit like controlclick or something, i provide the title and ID, bam. Done.

As i can't find a fast way to automate this the way i need, im going for the registry tweak to add the application to the list and let user select.

I thank you very much for your patience, and hope, for the future, this method is simplified and improved, if possible.

Maybe search only the window with the title provided.

Best regards.

junkew · December 12, 2017

This line _UIA_setVar is doing only an add to a translation table logical name

so the script later onwards is more readable with "logical names"

It does not execute any action

_UIA_setVar("oP1Popup.oUIElement2","Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink")

Then the logic is just like an enduser would do

Set focus to the main window
Do an action on child element in the main window

So the shortest way of writing the action only for the popup is

_UIA_Action("Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "setfocus")
_UIA_Action("Title:=((Meer apps)|(More apps));controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink", "click")

Post the log then i can see where it goes slow

Edited December 12, 2017 by junkew

careca · December 12, 2017

"C:\Users\careca\Desktop\123\IUA Assoc.au3"(28,97) : error: _UIAction(): undefined function.
_UIAction("Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "setfocus")
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
C:\Users\careca\Desktop\123\IUA Assoc.au3 - 1 error(s), 0 warning(s)

Seems like it doesn't even run.

junkew · December 12, 2017

typo you should change it to (fixed previous post)

_UIA_Action

Could be that you have to set the context to search in

_UIA_Action("Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "setfocus")
_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "getobject"))
_UIA_Action("Title:=((Meer apps)|(More apps));controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink", "click")

Edited December 12, 2017 by junkew

careca · December 12, 2017

Right, so both codes work, but they still take some time to do so.

2line code log

Spoiler

<?xml version="1.0" encoding="UTF-8"?><log space="preserve"><logline level="5" timestamp="20171212-152039118"> <information> Information _UIA_VersionInfo version: T0.6-5; Release date: 20160909; OS Version: WIN_10</information></logline>
<logline level="0" timestamp="20171212-152041085"> </logline>
<logline level="0" timestamp="20171212-152048608"> Finding object (bypassing repository) with physical description *** WARNING: not in repository *** reference:=Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout:Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_action: No RTI.SEARCHCONTEXT used for Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_action: try for RTI.TITLE:=.MAINWINDOW
_UIA_action: No RTI.MAINWINDOW used for Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
Key: [RTI.ACTIONCOUNT] Value: [C:\Users\careca\Desktop\123\LOG\20171212-152039117.XML] Variable Type: String
Key: [DESKTOP] Value: [] Variable Type: Object
Key: [RTI.MAINWINDOW] Value: [] Variable Type: Object
_UIA_action: No RTI.PARENT used for Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_action: Finding object Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout object a:=0 under Desktop object b:=1
looking for Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_normalizeExpression Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=.*open.*
name:[Title] value:[.*open.*] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_WindowControlTypeId
name:[controltype] value:[50032] having index 15
_UIA_getObjectByFindAll property 3 class:=Shell_Flyout
name:[class] value:[Shell_Flyout] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 9<propertymatching></propertymatching> deep find in subtree Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_normalizeExpression Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=.*open.*
name:[Title] value:[.*open.*] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_WindowControlTypeId
name:[controltype] value:[50032] having index 15
_UIA_getObjectByFindAll property 3 class:=Shell_Flyout
name:[class] value:[Shell_Flyout] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 4217Found match with element: 2 the Name is: <Title is: <How do you want to open .mp3 files from now on?> Class := <Shell_Flyout> controltype:= <UIA_WindowControlTypeId> ,<50032> , (0000C370) rect := < 1080;270;392;522> hwnd := < 329272> acceleratorkey := < > automationid := <Popup Window>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT Desktop 1
<action>Action 1 setfocus on Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171212-152048818"> </logline>
<logline level="0" timestamp="20171212-152049087"> Finding object (bypassing repository) with physical description *** WARNING: not in repository *** reference:=Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink:Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_action: Finding object Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink object a:=0 under RTI.SEARCHCONTEXT object b:=1
looking for Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=More apps
name:[Title] value:[More apps] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
name:[controltype] value:[50005] having index 15
_UIA_getObjectByFindAll property 3 class:=TouchHyperLink
name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 4<propertymatching></propertymatching> deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=More apps
name:[Title] value:[More apps] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
name:[controltype] value:[50005] having index 15
_UIA_getObjectByFindAll property 3 class:=TouchHyperLink
name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 31Found match with element: 29 the Name is: <Title is: <More apps> Class := <TouchHyperLink> controltype:= <UIA_HyperlinkControlTypeId> ,<50005> , (0000C355) rect := < 1081;682;390;50> hwnd := < 0> acceleratorkey := < > automationid := <OptionsButton>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT
<action>Action 2 click on Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
</log>

3line code log

Spoiler

<?xml version="1.0" encoding="UTF-8"?><log space="preserve"><logline level="5" timestamp="20171212-150834889"> <information> Information _UIA_VersionInfo version: T0.6-5; Release date: 20160909; OS Version: WIN_10</information></logline>
<logline level="0" timestamp="20171212-150836827"> </logline>
<logline level="0" timestamp="20171212-150844902"> Finding object (bypassing repository) with physical description *** WARNING: not in repository *** reference:=Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout:Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_action: No RTI.SEARCHCONTEXT used for Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_action: try for RTI.MAINWINDOW
_UIA_action: Finding object Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout object a:=0 under RTI.MAINWINDOW object b:=1
looking for Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_normalizeExpression Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=How do you want to open *
name:[Title] value:[How do you want to open *] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_WindowControlTypeId
name:[controltype] value:[50032] having index 15
_UIA_getObjectByFindAll property 3 class:=Shell_Flyout
name:[class] value:[Shell_Flyout] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 8<propertymatching></propertymatching> deep find in subtree Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_normalizeExpression Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=How do you want to open *
name:[Title] value:[How do you want to open *] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_WindowControlTypeId
name:[controltype] value:[50032] having index 15
_UIA_getObjectByFindAll property 3 class:=Shell_Flyout
name:[class] value:[Shell_Flyout] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 4249Found match with element: 2 the Name is: <Title is: <How do you want to open .mp3 files from now on?> Class := <Shell_Flyout> controltype:= <UIA_WindowControlTypeId> ,<50032> , (0000C370) rect := < 1080;270;392;522> hwnd := < 525854> acceleratorkey := < > automationid := <Popup Window>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT Desktop 1
<action>Action 1 setfocus on Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171212-150845104"> </logline>
<logline level="0" timestamp="20171212-150845286"> Finding object (bypassing repository) with physical description *** WARNING: not in repository *** reference:=Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout:Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_action: Finding object Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout object a:=0 under RTI.SEARCHCONTEXT object b:=1
looking for Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_normalizeExpression Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=How do you want to open *
name:[Title] value:[How do you want to open *] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_WindowControlTypeId
name:[controltype] value:[50032] having index 15
_UIA_getObjectByFindAll property 3 class:=Shell_Flyout
name:[class] value:[Shell_Flyout] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 4<propertymatching></propertymatching> deep find in subtree Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_normalizeExpression Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=How do you want to open *
name:[Title] value:[How do you want to open *] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_WindowControlTypeId
name:[controltype] value:[50032] having index 15
_UIA_getObjectByFindAll property 3 class:=Shell_Flyout
name:[class] value:[Shell_Flyout] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 31Found match with element: 0 the Name is: <Title is: <How do you want to open .mp3 files from now on?> Class := <Shell_Flyout> controltype:= <UIA_WindowControlTypeId> ,<50032> , (0000C370) rect := < 1080;270;392;522> hwnd := < 525854> acceleratorkey := < > automationid := <Popup Window>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT Desktop 1
<action>Action 2 getobject on Title:=How do you want to open *;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171212-150845287"> </logline>
<logline level="0" timestamp="20171212-150845535"> Finding object (bypassing repository) with physical description *** WARNING: not in repository *** reference:=Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink:Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_action: Finding object Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink object a:=0 under RTI.SEARCHCONTEXT object b:=1
looking for Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=More apps
name:[Title] value:[More apps] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
name:[controltype] value:[50005] having index 15
_UIA_getObjectByFindAll property 3 class:=TouchHyperLink
name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 4<propertymatching></propertymatching> deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=More apps
name:[Title] value:[More apps] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
name:[controltype] value:[50005] having index 15
_UIA_getObjectByFindAll property 3 class:=TouchHyperLink
name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 31Found match with element: 29 the Name is: <Title is: <More apps> Class := <TouchHyperLink> controltype:= <UIA_HyperlinkControlTypeId> ,<50005> , (0000C355) rect := < 1081;682;390;50> hwnd := < 0> acceleratorkey := < > automationid := <OptionsButton>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT
<action>Action 3 click on Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
</log>

It's only the first line that's taking a long time

_UIA_action("Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "setfocus")

Result is

deep find in subtree Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout;Desktop

Even tried with a more sutable title, and the complete title, but the result is the same, didn't help.

Any way to speed the first line up?

junkew · December 12, 2017

as you can see in the log with n elements where n equals 4249 still means not finding in right hierarchy.

I see you use: Title:=How do you want to open *

It has to be a .* (dot followed by star)

And as said its important to search for an object in a searchcontext otherwise it scans all grandchilds. Not having AutoIt with me around so best guess is like below

;~ Search main window under the desktop
_UIA_setVar("RTI.SEARCHCONTEXT", $UIA_oDesktop)
_UIA_Action("Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "setfocus")
;~ Search for the element in the main window
_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "getobject"))
_UIA_Action("Title:=((Meer apps)|(More apps));controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink", "click")

Its same as given in the other examples

although above is short

it bypasses the hierarchy of finding in childs in favor of finding in subtree.

This hierarchy you get from simplespy and inspect.exe spying tools.

So if Shell_Flyout window is NOT a child of the desktop it will start finding in subtree and as such will search the full extensionslist.

Edited December 12, 2017 by junkew
fixed $UIA_desktop

careca · December 12, 2017

Quote

It has to be a .* (dot followed by star)

Ah yes, thanks.

_UIA_setVar("RTI.SEARCHCONTEXT", .....??? $UIA_Desktop ???........)

This line gives undeclared variable and syntax error, what should i do with that?

This is what simplespy gives me for the desktop

Spoiler

;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=Program Manager;controltype:=UIA_PaneControlTypeId;class:=Progman") ;Program Manager
_UIA_setVar("oP2","Title:=;controltype:=UIA_PaneControlTypeId;class:=SHELLDLL_DefView") ;

_UIA_setVar("Desktop.mainwindow","title:=Desktop;classname:=SysListView32")

;~ Actions split away from logical/technical definition above can come from configfiles

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")

_UIA_action("Desktop.mainwindow","setfocus")

;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP1=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Program Manager;controltype:=UIA_PaneControlTypeId;class:=Progman", $treescope_children)
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=SHELLDLL_DefView", $treescope_children)
_UIA_Action($oP0,"setfocus")
_UIA_setVar("Desktop.mainwindow","title:=Desktop;classname:=SysListView32")
_UIA_action("Desktop.mainwindow","setfocus")

junkew · December 12, 2017

To get it fully stable working especially in case a certain element is not coming on screen is not possible I feel without programming a treewalker over the childs for that you have to read in the examples on how to create a treewalker

$UIA_Desktop you had to use, fixed previous post.

I will add on todo list in the uiawrappers.au3

Fallback search algorithm turn on/off
- Turn on/off deep find
- Turn on/off walking back to mainwindow (as that really is searching thousands of elements)
Find a way on how to stop when the element is not found like More apps links will not allways be there
- For that you have to incorparate a treewalker over the childs of the flyout popupwindow
Deal with simplespy and flyout (although it handles it better then inspect or visualuiaverify)
Have simple spy also generate the searchcontext for easy copy/paste
_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP2","getobject"))

A more structured example with the different styles of programming in it

Please try it out with the english names and post your code back so we have a multi language example
With use
- of uia_setvar
- of functions to group more logically the intention
Without user of uia_setvar

#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

local $oListView
local $oListGroup

Local $iPID = ShellExecute(@WindowsDir & '\system32\control.exe', '/name Microsoft.DefaultPrograms /page pageFileAssoc')
Sleep(3000) ;Give some time 10 seconds to start
;~ consolewrite($ipid)

;~ Base flow
;~ 1. Set main window context, find it in childs of desktop
;~ 2. Select an extension
;~ 3. Associate the new program
;~ 4. Find popupwindow
;~ 5. Click more

Set_main_window_context()
Select_an_extension(".mp3")
;~ Select_an_extension(".123")
;~ Associate_the_new_program()      ;- Not needed when you use action invoke you do not need to click the change program button
Show_more_apps()


;~ 1. Set main window context, find it in childs of desktop
func Set_main_window_context()
    _UIA_setVar("RTI.SEARCHCONTEXT", $UIA_oDesktop)
EndFunc

func Select_an_extension($strExtension)
    _UIA_setVar("oScreen1_P1.mainwindow","Title:=((Koppelingen instellen)|(Set Associations));ControlType:=UIA_WindowControlTypeId;class:=CabinetWClass")   ;Koppelingen instellen
    _UIA_setVar("oScreen1_oP2","Title:=((Koppelingen instellen)|(Set Associations));controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass") ;Koppelingen instellen
    _UIA_setVar("oScreen1_oP3","controltype:=UIA_PaneControlTypeId;class:=DUIViewWndClassName") ;
    _UIA_setVar("oScreen1_oP4","Title:=Deelvenster Verkenner;controltype:=UIA_PaneControlTypeId;class:=HWNDView")   ;Deelvenster Verkenner
    _UIA_setVar("oScreen1_oP5","Title:=Deelvenster Mapindeling;controltype:=UIA_PaneControlTypeId;class:=Element")  ;Deelvenster Mapindeling
    _UIA_setVar("oScreen1_oP6","controltype:=UIA_PaneControlTypeId;class:=XElement")    ;
    _UIA_setVar("oScreen1_oP7","Title:=;controltype:=UIA_PaneControlTypeId;class:=XBaby")   ;
    _UIA_setVar("oScreen1_oP8","Title:=Een bestandstype of protocol aan een specifiek programma koppelen;controltype:=UIA_PaneControlTypeId;class:=SUDCplPage") ;Een bestandstype of protocol aan een specifiek programma koppelen
    _UIA_setVar("oScreen1_oP9","Title:=Een bestandstype of protocol aan een specifiek programma koppelen;controltype:=UIA_PaneControlTypeId;class:=Element")    ;Een bestandstype of protocol aan een specifiek programma koppelen
    _UIA_setVar("oScreen1_oP10","Title:=((Lijst met extensies)|(List with extensions));ControlType:=UIA_ListControlTypeId;class:=CCListViewEx") ;Lijst met extensies
    _UIA_setVar("oScreen1_oP11","Title:=((Extensies)|(Extensions));controltype:=UIA_GroupControlTypeId") ;class:=") ;Extensies
    _UIA_setVar("oScreen1_oP12","Title:=" & $strExtension & ";controltype:=UIA_ListItemControlTypeId;class:=")  ;.123

    _UIA_setVar("oScreen1_oUIExtensionElement","Title:=" & $strExtension & ";controltype:=UIA_ListItemControlTypeId;class:=")   ;.123
;~  _UIA_setVar("oScreen1_oUIExtensionElement","Title:=" & $strExtension & ";controltype:=UIA_TextControlTypeId;class:=") ;ControlType:=UIA_TextControlTypeId;classname:=")

    ;~ Actions split away from logical/technical definition above can come from configfiles

    _UIA_Action("oScreen1_oP1.mainwindow","setfocus")
    _UIA_Action("oScreen1_oP1.mainwindow","highlight")
    sleep(500)

    ;~ _UIA_Action("oP2","highlight")
    ;~ _UIA_Action("oP2","setfocus")
    ;~ sleep(500)
    _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP2","getobject"))
    _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP3","getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP4","getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP5","getobject"))
    _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP6","getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP7","getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP8","getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP9","getobject"))
    sleep(500)

    $oListView=_UIA_Action("oScreen1_oP10","getobject")
    _UIA_Action($oListView,"setfocus")
    _UIA_Action($oListView,"highlight")
    sleep(250)

    _UIA_setVar("RTI.SEARCHCONTEXT", $oListView)

    $oListGroup=_UIA_Action("oScreen1_oP11","getobject")
    _UIA_Action($oListGroup,"setfocus")
    _UIA_Action($oListGroup,"highlight")
    sleep(250)

;~  _UIA_setVar("RTI.SEARCHCONTEXT", $oListGroup)
;~  _UIA_Action("oScreen1_oUIExtensionElement","setfocus")

;~  _UIA_setVar("RTI.SEARCHCONTEXT", $oListGroup)
;~  _UIA_Action("oScreen1_oUIExtensionElement","highlight")

    _UIA_setVar("RTI.SEARCHCONTEXT", $oListGroup)
    _UIA_Action("oScreen1_oUIExtensionElement","invoke")

EndFunc

Func Associate_the_new_program()
    clickTheChangeProgramButton()
    Show_more_apps()
EndFunc

;- Seems not needed when invoke is used on an extension line
func clickTheChangeProgramButton()
    ;- Click the button
    ;~ Based on spy you can create this hierarchy (but some of the parents of simplespy are not needed, VisualUIAVerifyNative.exe of win sdk 8.1 seems to be best to read hierarchy
    _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Title:=((Koppelingen instellen)|(Set Associations));ControlType:=UIA_WindowControlTypeId;class:=CabinetWClass", "getobject"))
    _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Title:=((Koppelingen instellen)|(Set Associations));controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass", "getobject"))
    _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("controltype:=UIA_PaneControlTypeId;class:=DUIViewWndClassName", "getobject"))
    _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("controltype:=UIA_PaneControlTypeId;class:=XElement", "getobject"))

    sleep(250)
    _UIA_Action("title:=((Ander.*)|(Change program.*));classname:=CCPushButton","setfocus")
    _UIA_Action("title:=((Ander.*)|(Change program.*));classname:=CCPushButton","click")
    sleep(250)
EndFunc

func Show_more_apps()

;~  _UIA_setVar("oP1","Title:=Hoe wilt u .123-bestanden vanaf nu openen?;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout") ;Hoe wilt u .123-bestanden vanaf nu openen?
    _UIA_setVar("Popup_oP1.mainwindow","Title:=((Hoe wilt u.*)|(How.*));controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout")  ;Hoe wilt u .123-bestanden vanaf nu openen?
    _UIA_setVar("Popup_oP2","Title:=Flyout.*;controltype:=UIA_PaneControlTypeId;class:=FlyoutElement")  ;Flyoutvenster
    _UIA_setVar("Popup_oP3","controltype:=UIA_PaneControlTypeId;class:=Element")    ;
    _UIA_setVar("Popup_oP4","Title:=Immersive Openwith Flyout;controltype:=UIA_PaneControlTypeId;class:=Element")   ;Immersive Openwith Flyout
    _UIA_setVar("Popup_oP5","controltype:=UIA_PaneControlTypeId;class:=TouchScrollViewer")  ;
    _UIA_setVar("Popup_oP6","controltype:=UIA_ListControlTypeId;class:=AppItemListContainer")   ;
    _UIA_setVar("Popup_oP7","controltype:=UIA_ListControlTypeId;class:=AppItemList")    ;

    ;~ $oUIElement=_UIA_getObjectByFindAll("Meerapps.mainwindow", "title:=Meer apps;ControlType:=UIA_HyperlinkControlTypeId", $treescope_subtree)
    _UIA_setVar("Popup_oUIElement","Title:=Meer apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink") ;ControlType:=UIA_HyperlinkControlTypeId;classname:=TouchHyperLink")

    ;- Restart the searchcontext to desktop
    ;~ Search popup window under the desktop and setfocus to it
    _UIA_setVar("RTI.SEARCHCONTEXT", $UIA_oDesktop)
    _UIA_Action("Popup_oP1.mainwindow", "setfocus")

    ;~ Change the searchcontext to the popup window
    _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Popup_oP1.mainwindow", "getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Popup_oP2", "getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Popup_oP3", "getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Popup_oP4", "getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Popup_oP5", "getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Popup_oP6", "getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Popup_oP7", "getobject"))

    ;~ Search for the element in the popup window and setfocus and click on it
;~  _UIA_Action("Popup_oUIElement", "setfocus")
    _UIA_Action("Popup_oUIElement", "click")
;~  _UIA_Action("Popup_oUIElement", "highlight")

endFunc

Exit

Edited December 12, 2017 by junkew

careca · December 14, 2017

Well, that code took 167 secs to run. But i had to comment out some stuff i have no idea what it is:

; _UIA_setVar("oScreen1_oP4","Title:=Deelvenster Verkenner;controltype:=UIA_PaneControlTypeId;class:=HWNDView")   ;Deelvenster Verkenner
   ; _UIA_setVar("oScreen1_oP5","Title:=Deelvenster Mapindeling;controltype:=UIA_PaneControlTypeId;class:=Element")  ;Deelvenster Mapindeling
    _UIA_setVar("oScreen1_oP6","controltype:=UIA_PaneControlTypeId;class:=XElement")    ;
    _UIA_setVar("oScreen1_oP7","Title:=;controltype:=UIA_PaneControlTypeId;class:=XBaby")   ;
   ; _UIA_setVar("oScreen1_oP8","Title:=Een bestandstype of protocol aan een specifiek programma koppelen;controltype:=UIA_PaneControlTypeId;class:=SUDCplPage") ;Een bestandstype of protocol aan een specifiek programma koppelen
   ; _UIA_setVar("oScreen1_oP9","Title:=Een bestandstype of protocol aan een specifiek programma koppelen;controltype:=UIA_PaneControlTypeId;class:=Element")    ;Een bestandstype of protocol aan een specifiek programma koppelen

Those titles mean nothing to me.

Spoiler

<?xml version="1.0" encoding="UTF-8"?><log space="preserve"><logline level="5" timestamp="20171214-040721189"> <information> Information _UIA_VersionInfo version: T0.6-5; Release date: 20160909; OS Version: WIN_10</information></logline>
<logline level="0" timestamp="20171214-040724221"> </logline>
<logline level="0" timestamp="20171214-040724887"> </logline>
<logline level="0" timestamp="20171214-040725607"> Fallback finding 1 object under Desktop
_UIA_normalizeExpression ;elements 1-1in properties array
_UIA_getObjectByFindAll property 1
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 10Found match with element: 0 the Name is: <Title is: <> Class := <Shell_TrayWnd> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) rect := < 0;1050;2560;30> hwnd := < 65766> acceleratorkey := < > automationid := <>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT Desktop 1
Storing in RTI as RTI.oScreen1_oP1.mainwindow
<action>Action 1 highlight on oScreen1_oP1.mainwindow _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171214-040726237"> </logline>
<logline level="0" timestamp="20171214-040738369"> _UIA_action: Finding object oScreen1_oP2 object a:=0 under RTI.SEARCHCONTEXT object b:=1
looking for Title:=Set Associations;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass
_UIA_normalizeExpression Title:=Set Associations;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=Set Associations
name:[Title] value:[Set Associations] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_PaneControlTypeId
name:[controltype] value:[50033] having index 15
_UIA_getObjectByFindAll property 3 class:=ShellTabWindowClass
name:[class] value:[ShellTabWindowClass] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 4<propertymatching></propertymatching> deep find in subtree Title:=Set Associations;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass
_UIA_normalizeExpression Title:=Set Associations;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=Set Associations
name:[Title] value:[Set Associations] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_PaneControlTypeId
name:[controltype] value:[50033] having index 15
_UIA_getObjectByFindAll property 3 class:=ShellTabWindowClass
name:[class] value:[ShellTabWindowClass] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 31<propertymatching></propertymatching> walking back to mainwindow and deep find in subtree Title:=Set Associations;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass
Desktop 1 parent0_UIA_normalizeExpression Title:=Set Associations;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=Set Associations
name:[Title] value:[Set Associations] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_PaneControlTypeId
name:[controltype] value:[50033] having index 15
_UIA_getObjectByFindAll property 3 class:=ShellTabWindowClass
name:[class] value:[ShellTabWindowClass] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 5831Found match with element: 85 the Name is: <Title is: <Set Associations> Class := <ShellTabWindowClass> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) rect := < 53;147;1009;629> hwnd := < 2885280> acceleratorkey := < > automationid := <>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT Set Associations
<action>Action 2 getobject on oScreen1_oP2 _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171214-040738370"> </logline>
<logline level="0" timestamp="20171214-040738518"> _UIA_action: Finding object oScreen1_oP3 object a:=0 under RTI.SEARCHCONTEXT object b:=1
looking for controltype:=UIA_PaneControlTypeId;class:=DUIViewWndClassName
_UIA_normalizeExpression controltype:=UIA_PaneControlTypeId;class:=DUIViewWndClassName;elements 1-2in properties array
_UIA_getObjectByFindAll property 1 controltype:=UIA_PaneControlTypeId
name:[controltype] value:[50033] having index 15
_UIA_getObjectByFindAll property 2 class:=DUIViewWndClassName
name:[class] value:[DUIViewWndClassName] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 2Found match with element: 0 the Name is: <Title is: <> Class := <DUIViewWndClassName> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) rect := < 53;168;1009;608> hwnd := < 1836718> acceleratorkey := < > automationid := <>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT Set Associations
<action>Action 3 getobject on oScreen1_oP3 _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171214-040738518"> </logline>
<logline level="0" timestamp="20171214-040738668"> _UIA_action: Finding object oScreen1_oP6 object a:=0 under RTI.SEARCHCONTEXT object b:=1
looking for controltype:=UIA_PaneControlTypeId;class:=XElement
_UIA_normalizeExpression controltype:=UIA_PaneControlTypeId;class:=XElement;elements 1-2in properties array
_UIA_getObjectByFindAll property 1 controltype:=UIA_PaneControlTypeId
name:[controltype] value:[50033] having index 15
_UIA_getObjectByFindAll property 2 class:=XElement
name:[class] value:[XElement] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 1Found match with element: 0 the Name is: <Title is: <> Class := <XElement> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) rect := < 53;168;1009;608> hwnd := < 1443256> acceleratorkey := < > automationid := <>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT Folder Layout Pane
<action>Action 4 getobject on oScreen1_oP6 _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171214-040739176"> </logline>
<logline level="0" timestamp="20171214-040909335"> </logline>
<logline level="0" timestamp="20171214-040943838"> Finding object (bypassing repository) with physical description *** WARNING: not in repository *** reference:=False:False
_UIA_action: Finding object False object a:=0 under RTI.SEARCHCONTEXT object b:=1
looking for False
_UIA_normalizeExpression False;elements 1-1in properties array
_UIA_getObjectByFindAll property 1 False
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 9<propertymatching></propertymatching> deep find in subtree False
_UIA_normalizeExpression False;elements 1-1in properties array
_UIA_getObjectByFindAll property 1 False
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 2685<propertymatching></propertymatching> walking back to mainwindow and deep find in subtree False
Folder Layout Pane parent0Explorer Pane parent1 parent2Set Associations parent3Set Associations parent4Desktop 1 parent5_UIA_normalizeExpression False;elements 1-1in properties array
_UIA_getObjectByFindAll property 1 False
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 5831Found match with element: 2810 the Name is: <Title is: <>"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "C:\Users\careca\Desktop\123\IUA Assoc3.au3" /UserParams
+>04:07:20 Starting AutoIt3Wrapper v.16.612.1119.0 SciTE v.3.6.6.0   Keyboard:00000816 OS:WIN_10/ CPU:X64 OS:X64 Environment(Language:0809) CodePage:0 utf8.auto.check:4
+>         SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE   UserDir => C:\Users\careca\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper   SCITE_USERHOME => C:\Users\careca\AppData\Local\AutoIt v3\SciTE
>Running AU3Check (3.3.14.2) from:C:\Program Files (x86)\AutoIt3 input:C:\Users\careca\Desktop\123\IUA Assoc3.au3
+>04:07:20 AU3Check ended.rc:0
>Running:(3.3.14.2):C:\Program Files (x86)\AutoIt3\autoit3_x64.exe "C:\Users\careca\Desktop\123\IUA Assoc3.au3"
--> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop
deep find in subtree Title:=Set Associations;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass;RTI.SEARCHCONTEXT
deep find in subtree Title:=List with extensions;ControlType:=UIA_ListControlTypeId;class:=CCListViewEx;RTI.SEARCHCONTEXT
deep find in subtree False;RTI.SEARCHCONTEXT
>   Class   := <Scintilla>   controltype:= <UIA_PaneControlTypeId>   ,<50033>   , (0000C371)   rect := < 924;922;1493;100>   hwnd := < 656866>   acceleratorkey := < >   automationid := <351>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT Source
<action>Action 5 setfocus on False _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171214-040944045"> </logline>
<logline level="0" timestamp="20171214-040944195"> Finding object (bypassing repository) with physical description *** WARNING: not in repository *** reference:=False:False
_UIA_action: Finding object False object a:=0 under RTI.SEARCHCONTEXT object b:=1
looking for False
_UIA_normalizeExpression False;elements 1-1in properties array
_UIA_getObjectByFindAll property 1 False
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 3<propertymatching></propertymatching> deep find in subtree False
_UIA_normalizeExpression False;elements 1-1in properties array
_UIA_getObjectByFindAll property 1 False
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 12Found match with element: 0 the Name is: <Title is: <>"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "C:\Users\careca\Desktop\123\IUA Assoc3.au3" /UserParams
+>04:07:20 Starting AutoIt3Wrapper v.16.612.1119.0 SciTE v.3.6.6.0   Keyboard:00000816 OS:WIN_10/ CPU:X64 OS:X64 Environment(Language:0809) CodePage:0 utf8.auto.check:4
+>         SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE   UserDir => C:\Users\careca\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper   SCITE_USERHOME => C:\Users\careca\AppData\Local\AutoIt v3\SciTE
>Running AU3Check (3.3.14.2) from:C:\Program Files (x86)\AutoIt3 input:C:\Users\careca\Desktop\123\IUA Assoc3.au3
+>04:07:20 AU3Check ended.rc:0
>Running:(3.3.14.2):C:\Program Files (x86)\AutoIt3\autoit3_x64.exe "C:\Users\careca\Desktop\123\IUA Assoc3.au3"
--> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop
deep find in subtree Title:=Set Associations;controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass;RTI.SEARCHCONTEXT
deep find in subtree Title:=List with extensions;ControlType:=UIA_ListControlTypeId;class:=CCListViewEx;RTI.SEARCHCONTEXT
deep find in subtree False;RTI.SEARCHCONTEXT
>   Class   := <Scintilla>   controltype:= <UIA_PaneControlTypeId>   ,<50033>   , (0000C371)   rect := < 924;922;1493;100>   hwnd := < 656866>   acceleratorkey := < >   automationid := <351>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT Source
<action>Action 6 highlight on False _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171214-040944582"> </logline>
<logline level="0" timestamp="20171214-040956179"> _UIA_action: No RTI.SEARCHCONTEXT used for oScreen1_oP11
_UIA_action: try for RTI.MAINWINDOW
_UIA_action: Finding object oScreen1_oP11 object a:=0 under RTI.MAINWINDOW object b:=1
looking for Title:=Extensions;controltype:=UIA_GroupControlTypeId
_UIA_normalizeExpression Title:=Extensions;controltype:=UIA_GroupControlTypeId;elements 1-2in properties array
_UIA_getObjectByFindAll property 1 Title:=Extensions
name:[Title] value:[Extensions] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_GroupControlTypeId
name:[controltype] value:[50026] having index 15
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 4<propertymatching></propertymatching> deep find in subtree Title:=Extensions;controltype:=UIA_GroupControlTypeId
_UIA_normalizeExpression Title:=Extensions;controltype:=UIA_GroupControlTypeId;elements 1-2in properties array
_UIA_getObjectByFindAll property 1 Title:=Extensions
name:[Title] value:[Extensions] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_GroupControlTypeId
name:[controltype] value:[50026] having index 15
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 31<propertymatching></propertymatching> walking back to mainwindow and deep find in subtree Title:=Extensions;controltype:=UIA_GroupControlTypeId
Desktop 1 parent0_UIA_normalizeExpression Title:=Extensions;controltype:=UIA_GroupControlTypeId;elements 1-2in properties array
_UIA_getObjectByFindAll property 1 Title:=Extensions
name:[Title] value:[Extensions] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_GroupControlTypeId
name:[controltype] value:[50026] having index 15
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 5831Found match with element: 170 the Name is: <Title is: <Extensions> Class := <> controltype:= <UIA_GroupControlTypeId> ,<50026> , (0000C36A) rect := < 74;322;950;390> hwnd := < 0> acceleratorkey := < > automationid := <>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT Extensions List
<action>Action 7 getobject on oScreen1_oP11 _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171214-040956179"> </logline>
<logline level="0" timestamp="20171214-040956181"> Quickly referenced object :
<action>Action 8 setfocus on _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171214-040956411"> </logline>
<logline level="0" timestamp="20171214-040956412"> Quickly referenced object :
<action>Action 9 highlight on _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171214-040956799"> </logline>
<logline level="0" timestamp="20171214-040958202"> _UIA_action: Finding object oScreen1_oUIExtensionElement object a:=0 under RTI.SEARCHCONTEXT object b:=1
looking for Title:=.mp3;controltype:=UIA_ListItemControlTypeId;class:=
_UIA_normalizeExpression Title:=.mp3;controltype:=UIA_ListItemControlTypeId;class:=;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=.mp3
name:[Title] value:[.mp3] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_ListItemControlTypeId
name:[controltype] value:[50007] having index 15
_UIA_getObjectByFindAll property 3 class:=
name:[class] value:[] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 630Found match with element: 286 the Name is: <Title is: <.mp3> Class := <> controltype:= <UIA_ListItemControlTypeId> ,<50007> , (0000C357) rect := < 0;0;0;0> hwnd := < 0> acceleratorkey := < > automationid := <>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT Extensions
<action>Action 10 invoke on oScreen1_oUIExtensionElement _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171214-040958429"> </logline>
<logline level="0" timestamp="20171214-040959143"> </logline>
<logline level="0" timestamp="20171214-040959695"> Fallback finding 1 object under Desktop
_UIA_normalizeExpression ;elements 1-1in properties array
_UIA_getObjectByFindAll property 1
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 12Found match with element: 0 the Name is: <Title is: <Close> Class := <Shell_LightDismissOverlay> controltype:= <UIA_ButtonControlTypeId> ,<50000> , (0000C350) rect := < 0;0;2560;2560> hwnd := < 1443246> acceleratorkey := < > automationid := <Light Dismiss>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT Desktop 1
Storing in RTI as RTI.Popup_oP1.mainwindow
<action>Action 11 getobject on Popup_oP1.mainwindow _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171214-040959696"> </logline>
<logline level="0" timestamp="20171214-041011172"> _UIA_action: Finding object Popup_oUIElement object a:=0 under RTI.SEARCHCONTEXT object b:=1
looking for Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=More apps
name:[Title] value:[More apps] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
name:[controltype] value:[50005] having index 15
_UIA_getObjectByFindAll property 3 class:=TouchHyperLink
name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 0<propertymatching></propertymatching> deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=More apps
name:[Title] value:[More apps] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
name:[controltype] value:[50005] having index 15
_UIA_getObjectByFindAll property 3 class:=TouchHyperLink
name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 1<propertymatching></propertymatching> walking back to mainwindow and deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
Desktop 1 parent0_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=More apps
name:[Title] value:[More apps] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
name:[controltype] value:[50005] having index 15
_UIA_getObjectByFindAll property 3 class:=TouchHyperLink
name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 5858Found match with element: 31 the Name is: <Title is: <More apps> Class := <TouchHyperLink> controltype:= <UIA_HyperlinkControlTypeId> ,<50005> , (0000C355) rect := < 1081;682;390;50> hwnd := < 0> acceleratorkey := < > automationid := <OptionsButton>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT
<action>Action 12 click on Popup_oUIElement _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
</log>

The fastest code is the last you posted, this one, takes "only" 15 secs to find the more apps hyperlink and do the click.

Local $UIA_Desktop
_UIA_setVar("RTI.SEARCHCONTEXT", $UIA_Desktop)
_UIA_Action("Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "setfocus")
_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "getobject"))
_UIA_Action("Title:=((Meer apps)|(More apps));controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink", "click")

Although it complains about the variable not being set before, so maybe that screws it up a bit.

Spoiler

<?xml version="1.0" encoding="UTF-8"?><log space="preserve"><logline level="5" timestamp="20171214-042007596"> <information> Information _UIA_VersionInfo version: T0.6-5; Release date: 20160909; OS Version: WIN_10</information></logline>
<logline level="0" timestamp="20171214-042009841"> </logline>
<logline level="0" timestamp="20171214-042022473"> Finding object (bypassing repository) with physical description *** WARNING: not in repository *** reference:=Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout:Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_action: No RTI.SEARCHCONTEXT used for Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_action: try for RTI.TITLE:=HOW DO YOU WANT TO OPEN.MAINWINDOW
_UIA_action: No RTI.MAINWINDOW used for Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
Key: [RTI.ACTIONCOUNT] Value: [C:\Users\careca\Desktop\123\LOG\20171214-042007595.XML] Variable Type: String
Key: [DESKTOP] Value: [] Variable Type: Object
Key: [RTI.MAINWINDOW] Value: [] Variable Type: Object
Key: [RTI.SEARCHCONTEXT] Value: [] Variable Type: String
_UIA_action: No RTI.PARENT used for Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_action: Finding object Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout object a:=0 under Desktop object b:=1
looking for Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_normalizeExpression Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=How do you want to open.*
name:[Title] value:[How do you want to open.*] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_WindowControlTypeId
name:[controltype] value:[50032] having index 15
_UIA_getObjectByFindAll property 3 class:=Shell_Flyout
name:[class] value:[Shell_Flyout] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 9<propertymatching></propertymatching> deep find in subtree Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_normalizeExpression Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=How do you want to open.*
name:[Title] value:[How do you want to open.*] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_WindowControlTypeId
name:[controltype] value:[50032] having index 15
_UIA_getObjectByFindAll property 3 class:=Shell_Flyout
name:[class] value:[Shell_Flyout] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 5940Found match with element: 2 the Name is: <Title is: <How do you want to open .mp3 files from now on?> Class := <Shell_Flyout> controltype:= <UIA_WindowControlTypeId> ,<50032> , (0000C370) rect := < 1080;270;392;522> hwnd := < 4589116> acceleratorkey := < > automationid := <Popup Window>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT Desktop 1
<action>Action 1 setfocus on Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171214-042022681"> </logline>
<logline level="0" timestamp="20171214-042022865"> Finding object (bypassing repository) with physical description *** WARNING: not in repository *** reference:=Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout:Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_action: Finding object Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout object a:=0 under RTI.SEARCHCONTEXT object b:=1
looking for Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_normalizeExpression Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=How do you want to open.*
name:[Title] value:[How do you want to open.*] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_WindowControlTypeId
name:[controltype] value:[50032] having index 15
_UIA_getObjectByFindAll property 3 class:=Shell_Flyout
name:[class] value:[Shell_Flyout] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 4<propertymatching></propertymatching> deep find in subtree Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout
_UIA_normalizeExpression Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=How do you want to open.*
name:[Title] value:[How do you want to open.*] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_WindowControlTypeId
name:[controltype] value:[50032] having index 15
_UIA_getObjectByFindAll property 3 class:=Shell_Flyout
name:[class] value:[Shell_Flyout] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 31Found match with element: 0 the Name is: <Title is: <How do you want to open .mp3 files from now on?> Class := <Shell_Flyout> controltype:= <UIA_WindowControlTypeId> ,<50032> , (0000C370) rect := < 1080;270;392;522> hwnd := < 4589116> acceleratorkey := < > automationid := <Popup Window>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT Desktop 1
<action>Action 2 getobject on Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
<logline level="0" timestamp="20171214-042022866"> </logline>
<logline level="0" timestamp="20171214-042023130"> Finding object (bypassing repository) with physical description *** WARNING: not in repository *** reference:=Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink:Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_action: Finding object Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink object a:=0 under RTI.SEARCHCONTEXT object b:=1
looking for Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=More apps
name:[Title] value:[More apps] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
name:[controltype] value:[50005] having index 15
_UIA_getObjectByFindAll property 3 class:=TouchHyperLink
name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=2
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 4<propertymatching></propertymatching> deep find in subtree Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink
_UIA_normalizeExpression Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink;elements 1-3in properties array
_UIA_getObjectByFindAll property 1 Title:=More apps
name:[Title] value:[More apps] having index 3
_UIA_getObjectByFindAll property 2 controltype:=UIA_HyperlinkControlTypeId
name:[controltype] value:[50005] having index 15
_UIA_getObjectByFindAll property 3 class:=TouchHyperLink
name:[class] value:[TouchHyperLink] having index 6
*** Try to get a list of elements *** treescopetype:=7
_UIA_getObjectByFindAll walk thru the tree with n elements where n equals 31Found match with element: 29 the Name is: <Title is: <More apps> Class := <TouchHyperLink> controltype:= <UIA_HyperlinkControlTypeId> ,<50005> , (0000C355) rect := < 1081;682;390;50> hwnd := < 0> acceleratorkey := < > automationid := <OptionsButton>

<propertymatching></propertymatching>Storing parent for found object in RTI as RTI.PARENT
<action>Action 3 click on Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink _UIA_IsElement:=1 Parameters 1:=0 2:=0 3:=0 4:=0</action></logline>
</log>

Edited December 14, 2017 by careca

junkew · December 14, 2017

With the UIA wrappers you will not get much more speed as it will search in the whole extensionlist with a findall which returns 5000+ elements where the actual element that matches is at position before 50.

However the UI AutomionLibrary in itself can get fast to your objects with either

1. findfirst
https://msdn.microsoft.com/en-us/library/windows/desktop/ee696029(v=vs.85).aspx

and in the examples.zip there are some examples that make use of findfirst

2. treewalker

and in the examples.zip there are some examples that make use of treewalker

But above 2 options are more difficult to learn. I am having not much time to write these examples but will put it on my TODO list as this application is a nice one to show all stuff.

careca · December 14, 2017

Ok, thank you.

LarsJ · December 15, 2017

This is based on an UI Automation event handler:

#AutoIt3Wrapper_UseX64=y
#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6

#include "CUIAutomation2.au3" ; junkew
#include "ObjectFromTag.au3"  ; trancexx

AutoItSetOption("MustDeclareVars", 1)

Global Const $S_OK = 0x00000000
Global Const $E_NOINTERFACE = 0x80004002
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"

Global $oUIAutomation
Global $bShellFlyoutFound = False

SetAssociations()


Func SetAssociations()
  Local $sText1, $Vis
  ShellExecute(@WindowsDir & '\system32\control.exe', '/name Microsoft.DefaultPrograms /page pageFileAssoc')
  WinWaitActive("Set Associations", 'Set Associations', 10)
  Local $hWnd = WinGetHandle("[CLASS:CabinetWClass]", "Set Associations")
  If $hWnd <> 0 Or @error <> 0 Then
    Do
      $Vis = ControlCommand($hWnd, "", "[CLASS:Button; INSTANCE:1]", 'IsVisible')
    Until $Vis = 1
    Sleep(100)
    Local $iCount = ControlListView($hWnd, "", "SysListView321", "GetItemCount")
    For $k = 0 To $iCount - 1
      $sText1 = ControlListView($hWnd, "", "SysListView321", "GetText", $k, 0)
      If $sText1 = '.mp3' Then
        ControlListView($hWnd, "", "SysListView321", "Select", $k, $k)
        ConsoleWrite( "SysListView321, Item " & $k & ": " & $sText1 & @CRLF & @CRLF )
        ControlClick($hWnd, "", "[CLASS:Button; INSTANCE:1]")
        HandlePopupWindow()
      EndIf
    Next
  EndIf
EndFunc

Func HandlePopupWindow()
  ; Create custom UI Automation event handler
  Local $tIUIAutomationEventHandler, $oIUIAutomationEventHandler
  $oIUIAutomationEventHandler = ObjectFromTag( "oIUIAutomationEventHandler_", $dtagIUIAutomationEventHandler, $tIUIAutomationEventHandler, True )
  If Not IsObj( $oIUIAutomationEventHandler ) Then Return ConsoleWrite( "$oIUIAutomationEventHandler ERR" & @CRLF )
  ConsoleWrite( "$oIUIAutomationEventHandler OK" & @CRLF )

  ; Create UI Automation object
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Desktop
  Local $pDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  If Not $pDesktop Then Return ConsoleWrite( "$pDesktop ERR" & @CRLF )
  ConsoleWrite( "$pDesktop OK" & @CRLF )

  ; Window open events
  If $oUIAutomation.AddAutomationEventHandler( $UIA_Window_WindowOpenedEventId, $pDesktop, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Return ConsoleWrite( "$pIUIAutomationEventHandler ERR" & @CRLF )
  ConsoleWrite( "$oIUIAutomationEventHandler OK" & @CRLF )

  While Not $bShellFlyoutFound And Sleep(10)
  WEnd

  $oUIAutomation.RemoveAutomationEventHandler( $UIA_Window_WindowOpenedEventId, $pDesktop, $oIUIAutomationEventHandler )
EndFunc

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf, $pSender, $iEventId ) ; Ret: long  Par: ptr;int
  ConsoleWrite( "oIUIAutomationEventHandler_HandleAutomationEvent: " & $iEventId & @CRLF )

  If $iEventId = $UIA_Window_WindowOpenedEventId Then
    Local $oSender = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $oSender.AddRef()
   
    ConsoleWrite( "Handle    " & Ptr( _UIA_getPropertyValue( $oSender, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF )
    ConsoleWrite( "Name      " & _UIA_getPropertyValue( $oSender, $UIA_NamePropertyId ) & @CRLF )
    ConsoleWrite( "Class     " & _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId ) & @CRLF )
    ConsoleWrite( "Ctrl type " & _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId ) & @CRLF )
    ConsoleWrite( "Ctrl name " & _UIA_getPropertyValue( $oSender, $UIA_LocalizedControlTypePropertyId ) & @CRLF )
    ConsoleWrite( "Value     " & _UIA_getPropertyValue( $oSender, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF )

    ; Shell_Flyout popup?
    Local $sClassName, $hWindow
    $oSender.GetCurrentPropertyValue( $UIA_ClassNamePropertyId, $sClassName )
    If $sClassName = "Shell_Flyout" Then
      $oSender.GetCurrentPropertyValue( $UIA_NativeWindowHandlePropertyId, $hWindow )

      ; Get UI Automation element from window handle
      Local $pWindow, $oWindow
      $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
      $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
      If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
      ConsoleWrite( "$oWindow OK" & @CRLF )

      ; --- More apps ---

      ConsoleWrite( "--- More apps ---" & @CRLF )

      ; Condition
      Local $pCondition
      $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition )
      If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
      ConsoleWrite( "$pCondition OK" & @CRLF )

      ; Find link
      Local $pLink, $oLink
      $oSender.FindFirst( $TreeScope_Descendants, $pCondition, $pLink )
      $oLink = ObjCreateInterface( $pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
      If Not IsObj( $oLink ) Then Return ConsoleWrite( "$oLink ERR" & @CRLF )
      ConsoleWrite( "$oLink OK" & @CRLF )

      ; Click (invoke) link
      Local $pInvoke, $oInvoke
      $oLink.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
      $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
      If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
      ConsoleWrite( "$oInvoke OK" & @CRLF )
      $oInvoke.Invoke()

      ; --- Look for another app on this PC ---

      ConsoleWrite( "--- Look for another app on this PC ---" & @CRLF )

      ; Condition
      $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition )
      If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
      ConsoleWrite( "$pCondition OK" & @CRLF )

      ; Find link
      $oSender.FindFirst( $TreeScope_Descendants, $pCondition, $pLink )
      $oLink = ObjCreateInterface( $pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
      If Not IsObj( $oLink ) Then Return ConsoleWrite( "$oLink ERR" & @CRLF )
      ConsoleWrite( "$oLink OK" & @CRLF )

      ; Click (invoke) link
      $oLink.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
      $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
      If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
      ConsoleWrite( "$oInvoke OK" & @CRLF )
      $oInvoke.Invoke()

      $bShellFlyoutFound = True
    EndIf
  EndIf

  Return $S_OK
  #forceref $pSelf
EndFunc

Func _UIA_getPropertyValue( $obj, $id )
  Local $vVal
  $obj.GetCurrentPropertyValue( $id, $vVal )
  Return $vVal
EndFunc

Func oIUIAutomationEventHandler_QueryInterface( $pSelf, $pRIID, $pObj ) ; Ret: long  Par: ptr;ptr*
  Local $sIID = StringFromGUID( $pRIID )
  If $sIID = $sIID_IUnknown Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUnknown" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  ElseIf $sIID = $sIID_IUIAutomationEventHandler Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  Else
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: " & $sIID & @CRLF )
    Return $E_NOINTERFACE
  EndIf
  #forceref $pSelf
EndFunc

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_AddRef" & @CRLF )
  Return 1
  #forceref $pSelf
EndFunc

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_Release" & @CRLF )
  Return 1
  #forceref $pSelf
EndFunc

Func StringFromGUID( $pGUID )
  Local $aResult = DllCall( "ole32.dll", "int", "StringFromGUID2", "struct*", $pGUID, "wstr", "", "int", 40 )
  If @error Then Return SetError( @error, @extended, "" )
  Return SetExtended( $aResult[0], $aResult[2] )
EndFunc

Console output:

SysListView321, Item 246: .mp3

Func oIUIAutomationEventHandler_QueryInterface( $pSelf ) ; Ret: long  Par: ptr;ptr
EndFunc
0

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: dword
EndFunc
0

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: dword
EndFunc
0

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf ) ; Ret: long  Par: ptr;int
EndFunc
0

oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_Release
$oIUIAutomationEventHandler OK
$oUIAutomation OK
$pDesktop OK
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {ECC8691B-C1DB-4DC0-855E-65F6C551AF49}
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {00000039-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {94EA2B94-E9CC-49E0-C0FF-EE64CA8F5B90}
oIUIAutomationEventHandler_QueryInterface: {00000003-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
$oIUIAutomationEventHandler OK
oIUIAutomationEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {00000018-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {334D391F-0E79-3B15-C9FF-EAC65DD07C42}
oIUIAutomationEventHandler_QueryInterface: {00000040-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {334D391F-0E79-3B15-C9FF-EAC65DD07C42}
oIUIAutomationEventHandler_QueryInterface: {94EA2B94-E9CC-49E0-C0FF-EE64CA8F5B90}
oIUIAutomationEventHandler_QueryInterface: {334D391F-0E79-3B15-C9FF-EAC65DD07C42}
oIUIAutomationEventHandler_QueryInterface: {77DD1250-139C-2BC3-BD95-900ACED61BE5}
oIUIAutomationEventHandler_QueryInterface: {334D391F-0E79-3B15-C9FF-EAC65DD07C42}
oIUIAutomationEventHandler_QueryInterface: {BFD60505-5A1F-4E41-88BA-A6FB07202DA9}
oIUIAutomationEventHandler_QueryInterface: {334D391F-0E79-3B15-C9FF-EAC65DD07C42}
oIUIAutomationEventHandler_QueryInterface: {03FB5C57-D534-45F5-A1F4-D39556983875}
oIUIAutomationEventHandler_QueryInterface: {334D391F-0E79-3B15-C9FF-EAC65DD07C42}
oIUIAutomationEventHandler_QueryInterface: {2C258AE7-50DC-49FF-9D1D-2ECB9A52CDD7}
oIUIAutomationEventHandler_QueryInterface: {00000019-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {4C1E39E1-E3E3-4296-AA86-EC938D896E92}
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {1C733A30-2A1C-11CE-ADE5-00AA0044773D}
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_HandleAutomationEvent: 20016
Handle    0x0000000000330084
Name      How do you want to open .mp3 files from now on?
Class     Shell_Flyout
Ctrl type 50032
Ctrl name window
Value     
$oWindow OK
--- More apps ---
$pCondition OK
$oLink OK
$oInvoke OK
--- Look for another app on this PC ---
$pCondition OK
$oLink OK
$oInvoke OK
oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release

AllCode.7z

careca · December 15, 2017

This is very interesting! Sometimes it opens the file select dialog right away, which was the thing i was looking for.

But other times opens the popup and does nothing.

what kind of sorcery is this?

LarsJ · December 15, 2017

This is pure UI Automation code. Show the console output when it doesn't work.

careca · December 15, 2017

It seems to hang, does nothing, have to close it.

Spoiler

>"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "C:\Users\careca\Desktop\ztst01.au3" /UserParams
+>15:38:46 Starting AutoIt3Wrapper v.16.612.1119.0 SciTE v.3.6.6.0   Keyboard:00000816 OS:WIN_10/ CPU:X64 OS:X64 Environment(Language:0809) CodePage:0 utf8.auto.check:4
+>         SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE   UserDir => C:\Users\careca\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper   SCITE_USERHOME => C:\Users\careca\AppData\Local\AutoIt v3\SciTE
>Running AU3Check (3.3.14.2) params:-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 from:C:\Program Files (x86)\AutoIt3 input:C:\Users\careca\Desktop\ztst01.au3
+>15:38:46 AU3Check ended.rc:0
>Running:(3.3.14.2):C:\Program Files (x86)\AutoIt3\autoit3_x64.exe "C:\Users\careca\Desktop\ztst01.au3"
--> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop
SysListView321, Item 286: .mp3

Func oIUIAutomationEventHandler_QueryInterface( $pSelf ) ; Ret: long Par: ptr;ptr
EndFunc
0

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: dword
EndFunc
0

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: dword
EndFunc
0

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf ) ; Ret: long Par: ptr;int
EndFunc
0

oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_Release
$oIUIAutomationEventHandler OK
$oUIAutomation OK
$pDesktop OK
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {ECC8691B-C1DB-4DC0-855E-65F6C551AF49}
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {00000039-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {94EA2B94-E9CC-49E0-C0FF-EE64CA8F5B90}
oIUIAutomationEventHandler_QueryInterface: {00000003-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
$oIUIAutomationEventHandler OK
+>15:38:58 AutoIt3.exe ended.rc:0
+>15:38:58 AutoIt3Wrapper Finished.
>Exit code: 0 Time: 13.09

BTW it seems even more complex than UIA udf from previous posts.

Edited December 15, 2017 by careca

LarsJ · December 15, 2017

It appears that you are running the code as 32 bit code. If your CPU and operating system are 64 bit, you must also run the code as 64 bit.

careca · December 15, 2017

Didnt touch the code:

#AutoIt3Wrapper_UseX64=y

LarsJ · December 15, 2017

I saw that you actually ran the code as 64 bit. I have not had any errors when I have tested the code so I cannot really say anything. You must try to debug the code in the usual way. Is there any difference between the circumstances the times where it goes well and wrong?

UIA Need to speed this code up

Recommended Posts

Link to comment

Share on other sites

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Danyfirex

junkew

LarsJ

Link to comment

Share on other sites

IUIAutomation - Topic with framework and examples

Link to comment

Share on other sites

Link to comment

Share on other sites

IUIAutomation - Topic with framework and examples

Link to comment

Share on other sites

Link to comment

Share on other sites

IUIAutomation - Topic with framework and examples

Link to comment

Share on other sites

Link to comment

Share on other sites

IUIAutomation - Topic with framework and examples

Link to comment

Share on other sites

Link to comment

Share on other sites

IUIAutomation - Topic with framework and examples

Link to comment

Share on other sites

Link to comment

Share on other sites

IUIAutomation - Topic with framework and examples

Link to comment

Share on other sites

Link to comment

Share on other sites

IUIAutomation - Topic with framework and examples

Link to comment

Share on other sites

Link to comment

Share on other sites

IUIAutomation - Topic with framework and examples

Link to comment

Share on other sites

Link to comment

Share on other sites

IUIAutomation - Topic with framework and examples

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members