Jump to content

Recommended Posts

Posted
  • You should understand the tree hierarchy. First find (grand)parents before you act on leafchilds of the tree.
;Do the actions in the popup
_UIA_Action("oP1Popup.oUIElement2","setfocus") ;Focus on the "button"
_UIA_Action("oP1Popup.oUIElement2","click") ;Click the button

As you only act on the leafchild the uiawrappers tries to search from desktop, all windows, all subsubsubchilds

1. So add a setfocus on the popupwindow and you will be much better of

_UIA_Action("oP1popup.mainwindow","setfocus"))

2. Maybe needed but normally not when you act on a .mainwindow (as that implicitly sets the searchcontext)

_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oP1popup.mainwindow","getobject"))
  • could be that 1 and 2 is needed
  • May suggestion is to make your full script either with UIA or with AutoIt commands. I would not blend them

 

 

Posted

registry logic to make alternative direction complete (google and you will find which entries to manipulate)

#include <MsgBoxConstants.au3>

Local $sVar = RegRead("HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.mp3\OpenWithList", "a")
MsgBox($MB_SYSTEMMODAL, "Your MP3 player is:", $sVar)

If its empty its the default of your OS

use the  HKEY_CURRENT_USER\Software\Classes or HKEY_LOCAL_MACHINE\Software\Classes

manipulating registry can be more tricky but is fast ;-)

You have to edit / change multiple ones

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.mp3\UserChoice

Posted

Registry isn't tricky, it just doesn't do it for me, even tried the "assoc" and "ftype" embedded in cmd, but the application was never set.

Even changed the entries in ROOT and still it's like it doesn't refresh the change, even after restart.

That registry key you show right there, adds the application to the list in the popup, so that's one way to go about it, add to the list, and then allow user to select it, but i  would prefer to make it all automatic.

The lines provided didn't speed it up, takes about the same time.

  Quote

" As you only act on the leafchild the uiawrappers tries to search from desktop, all windows, all subsubsubchilds "

Expand  

I thought this line was to set preciselly where i need it to search?

_UIA_setVar("oP1Popup.oUIElement2","Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink")

After all, the title is provided, why will it search all windows and all subsubchilds?

As i said before, i find this very confusing. Once i thought it was/could be a bit like controlclick or something, i provide the title and ID, bam. Done.

As i can't find a fast way to automate this the way i need, im going for the registry tweak to add the application to the list and let user select.

I thank you very much for your patience, and hope, for the future, this method is simplified and improved, if possible.

Maybe search only the window with the title provided.

Best regards.

  Reveal hidden contents

IUIAutomation - Topic with framework and examples

Au3Record.exe

Posted (edited)

This line  _UIA_setVar is doing only an add to a translation table logical name 

so the script later onwards is more readable with "logical names"

It does not execute any action

_UIA_setVar("oP1Popup.oUIElement2","Title:=More apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink")

Then the logic is just like an enduser would do

  1.  Set focus to the main window
  2.  Do an action on child element in the main window

So the shortest way of writing the action only for the popup is

_UIA_Action("Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "setfocus")
_UIA_Action("Title:=((Meer apps)|(More apps));controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink", "click")

Post the log then i can see where it goes slow

Edited by junkew
Posted

"C:\Users\careca\Desktop\123\IUA Assoc.au3"(28,97) : error: _UIAction(): undefined function.
_UIAction("Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "setfocus")
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
C:\Users\careca\Desktop\123\IUA Assoc.au3 - 1 error(s), 0 warning(s)

Seems like it doesn't even run.

  Reveal hidden contents

IUIAutomation - Topic with framework and examples

Au3Record.exe

Posted (edited)

typo you should change it to (fixed previous post)

_UIA_Action

Could be that you have to set the context to search in

 

_UIA_Action("Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "setfocus")
_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "getobject"))
_UIA_Action("Title:=((Meer apps)|(More apps));controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink", "click")

 

Edited by junkew
Posted

Right, so both codes work, but they still take some time to do so.

2line code log

  Reveal hidden contents

3line code log

  Reveal hidden contents

It's only the first line that's taking a long time

_UIA_action("Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "setfocus")

Result is

deep find in subtree Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout;Desktop

Even tried with a more sutable title, and the complete title, but the result is the same, didn't help.

Any way to speed the first line up?

 

  Reveal hidden contents

IUIAutomation - Topic with framework and examples

Au3Record.exe

Posted (edited)

as you can see in the log   with n elements where n equals 4249  still means not finding in right hierarchy.

I see you use:   Title:=How do you want to open *

It has to be a .* (dot followed by star)

And as said its important to search for an object in a searchcontext otherwise it scans all grandchilds. Not having AutoIt with me around so best guess is like below

;~ Search main window under the desktop
_UIA_setVar("RTI.SEARCHCONTEXT", $UIA_oDesktop)
_UIA_Action("Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "setfocus")
;~ Search for the element in the main window
_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("Title:=.*open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "getobject"))
_UIA_Action("Title:=((Meer apps)|(More apps));controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink", "click")

Its same as given in the other examples

although above is short

it bypasses the hierarchy of finding in childs in favor of finding in subtree.

This hierarchy you get from simplespy and inspect.exe spying tools.

So if Shell_Flyout window is NOT a child of the desktop it will start finding in subtree and as such will search the full extensionslist. 

 

Edited by junkew
fixed $UIA_desktop
Posted
  Quote

It has to be a .* (dot followed by star)

Expand  

Ah yes, thanks.

_UIA_setVar("RTI.SEARCHCONTEXT", .....??? $UIA_Desktop ???........)

This line gives undeclared variable and syntax error, what should i do with that?

This is what simplespy gives me for the desktop

  Reveal hidden contents

 

  Reveal hidden contents

IUIAutomation - Topic with framework and examples

Au3Record.exe

Posted (edited)

To get it fully stable working especially in case a certain element is not coming on screen is not possible I feel without programming a treewalker over the childs for that you have to read in the examples on how to create a treewalker

  • $UIA_Desktop you had to use, fixed previous post.

I will add on todo list in the uiawrappers.au3

  • Fallback search algorithm turn on/off
    • Turn on/off deep find
    • Turn on/off walking back to mainwindow (as that really is searching thousands of elements)
  • Find a way on how to stop when the element is not found like More apps links will not allways be there
    • For that you have to incorparate a treewalker over the childs of the flyout popupwindow
  • Deal with simplespy and flyout (although it handles it better then inspect or visualuiaverify)
  • Have simple spy also generate the searchcontext for easy copy/paste
        _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP2","getobject"))

A more structured example with the different styles of programming in it

  • Please try it out with the english names and post your code back so we have a multi language example
  • With use
    • of uia_setvar
    • of functions to group more logically the intention
  • Without user of uia_setvar
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

local $oListView
local $oListGroup

Local $iPID = ShellExecute(@WindowsDir & '\system32\control.exe', '/name Microsoft.DefaultPrograms /page pageFileAssoc')
Sleep(3000) ;Give some time 10 seconds to start
;~ consolewrite($ipid)

;~ Base flow
;~ 1. Set main window context, find it in childs of desktop
;~ 2. Select an extension
;~ 3. Associate the new program
;~ 4. Find popupwindow
;~ 5. Click more

Set_main_window_context()
Select_an_extension(".mp3")
;~ Select_an_extension(".123")
;~ Associate_the_new_program()      ;- Not needed when you use action invoke you do not need to click the change program button
Show_more_apps()


;~ 1. Set main window context, find it in childs of desktop
func Set_main_window_context()
    _UIA_setVar("RTI.SEARCHCONTEXT", $UIA_oDesktop)
EndFunc

func Select_an_extension($strExtension)
    _UIA_setVar("oScreen1_P1.mainwindow","Title:=((Koppelingen instellen)|(Set Associations));ControlType:=UIA_WindowControlTypeId;class:=CabinetWClass")   ;Koppelingen instellen
    _UIA_setVar("oScreen1_oP2","Title:=((Koppelingen instellen)|(Set Associations));controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass") ;Koppelingen instellen
    _UIA_setVar("oScreen1_oP3","controltype:=UIA_PaneControlTypeId;class:=DUIViewWndClassName") ;
    _UIA_setVar("oScreen1_oP4","Title:=Deelvenster Verkenner;controltype:=UIA_PaneControlTypeId;class:=HWNDView")   ;Deelvenster Verkenner
    _UIA_setVar("oScreen1_oP5","Title:=Deelvenster Mapindeling;controltype:=UIA_PaneControlTypeId;class:=Element")  ;Deelvenster Mapindeling
    _UIA_setVar("oScreen1_oP6","controltype:=UIA_PaneControlTypeId;class:=XElement")    ;
    _UIA_setVar("oScreen1_oP7","Title:=;controltype:=UIA_PaneControlTypeId;class:=XBaby")   ;
    _UIA_setVar("oScreen1_oP8","Title:=Een bestandstype of protocol aan een specifiek programma koppelen;controltype:=UIA_PaneControlTypeId;class:=SUDCplPage") ;Een bestandstype of protocol aan een specifiek programma koppelen
    _UIA_setVar("oScreen1_oP9","Title:=Een bestandstype of protocol aan een specifiek programma koppelen;controltype:=UIA_PaneControlTypeId;class:=Element")    ;Een bestandstype of protocol aan een specifiek programma koppelen
    _UIA_setVar("oScreen1_oP10","Title:=((Lijst met extensies)|(List with extensions));ControlType:=UIA_ListControlTypeId;class:=CCListViewEx") ;Lijst met extensies
    _UIA_setVar("oScreen1_oP11","Title:=((Extensies)|(Extensions));controltype:=UIA_GroupControlTypeId") ;class:=") ;Extensies
    _UIA_setVar("oScreen1_oP12","Title:=" & $strExtension & ";controltype:=UIA_ListItemControlTypeId;class:=")  ;.123

    _UIA_setVar("oScreen1_oUIExtensionElement","Title:=" & $strExtension & ";controltype:=UIA_ListItemControlTypeId;class:=")   ;.123
;~  _UIA_setVar("oScreen1_oUIExtensionElement","Title:=" & $strExtension & ";controltype:=UIA_TextControlTypeId;class:=") ;ControlType:=UIA_TextControlTypeId;classname:=")

    ;~ Actions split away from logical/technical definition above can come from configfiles

    _UIA_Action("oScreen1_oP1.mainwindow","setfocus")
    _UIA_Action("oScreen1_oP1.mainwindow","highlight")
    sleep(500)

    ;~ _UIA_Action("oP2","highlight")
    ;~ _UIA_Action("oP2","setfocus")
    ;~ sleep(500)
    _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP2","getobject"))
    _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP3","getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP4","getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP5","getobject"))
    _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP6","getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP7","getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP8","getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("oScreen1_oP9","getobject"))
    sleep(500)

    $oListView=_UIA_Action("oScreen1_oP10","getobject")
    _UIA_Action($oListView,"setfocus")
    _UIA_Action($oListView,"highlight")
    sleep(250)

    _UIA_setVar("RTI.SEARCHCONTEXT", $oListView)

    $oListGroup=_UIA_Action("oScreen1_oP11","getobject")
    _UIA_Action($oListGroup,"setfocus")
    _UIA_Action($oListGroup,"highlight")
    sleep(250)

;~  _UIA_setVar("RTI.SEARCHCONTEXT", $oListGroup)
;~  _UIA_Action("oScreen1_oUIExtensionElement","setfocus")

;~  _UIA_setVar("RTI.SEARCHCONTEXT", $oListGroup)
;~  _UIA_Action("oScreen1_oUIExtensionElement","highlight")

    _UIA_setVar("RTI.SEARCHCONTEXT", $oListGroup)
    _UIA_Action("oScreen1_oUIExtensionElement","invoke")

EndFunc

Func Associate_the_new_program()
    clickTheChangeProgramButton()
    Show_more_apps()
EndFunc

;- Seems not needed when invoke is used on an extension line
func clickTheChangeProgramButton()
    ;- Click the button
    ;~ Based on spy you can create this hierarchy (but some of the parents of simplespy are not needed, VisualUIAVerifyNative.exe of win sdk 8.1 seems to be best to read hierarchy
    _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Title:=((Koppelingen instellen)|(Set Associations));ControlType:=UIA_WindowControlTypeId;class:=CabinetWClass", "getobject"))
    _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Title:=((Koppelingen instellen)|(Set Associations));controltype:=UIA_PaneControlTypeId;class:=ShellTabWindowClass", "getobject"))
    _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("controltype:=UIA_PaneControlTypeId;class:=DUIViewWndClassName", "getobject"))
    _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("controltype:=UIA_PaneControlTypeId;class:=XElement", "getobject"))

    sleep(250)
    _UIA_Action("title:=((Ander.*)|(Change program.*));classname:=CCPushButton","setfocus")
    _UIA_Action("title:=((Ander.*)|(Change program.*));classname:=CCPushButton","click")
    sleep(250)
EndFunc

func Show_more_apps()

;~  _UIA_setVar("oP1","Title:=Hoe wilt u .123-bestanden vanaf nu openen?;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout") ;Hoe wilt u .123-bestanden vanaf nu openen?
    _UIA_setVar("Popup_oP1.mainwindow","Title:=((Hoe wilt u.*)|(How.*));controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout")  ;Hoe wilt u .123-bestanden vanaf nu openen?
    _UIA_setVar("Popup_oP2","Title:=Flyout.*;controltype:=UIA_PaneControlTypeId;class:=FlyoutElement")  ;Flyoutvenster
    _UIA_setVar("Popup_oP3","controltype:=UIA_PaneControlTypeId;class:=Element")    ;
    _UIA_setVar("Popup_oP4","Title:=Immersive Openwith Flyout;controltype:=UIA_PaneControlTypeId;class:=Element")   ;Immersive Openwith Flyout
    _UIA_setVar("Popup_oP5","controltype:=UIA_PaneControlTypeId;class:=TouchScrollViewer")  ;
    _UIA_setVar("Popup_oP6","controltype:=UIA_ListControlTypeId;class:=AppItemListContainer")   ;
    _UIA_setVar("Popup_oP7","controltype:=UIA_ListControlTypeId;class:=AppItemList")    ;

    ;~ $oUIElement=_UIA_getObjectByFindAll("Meerapps.mainwindow", "title:=Meer apps;ControlType:=UIA_HyperlinkControlTypeId", $treescope_subtree)
    _UIA_setVar("Popup_oUIElement","Title:=Meer apps;controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink") ;ControlType:=UIA_HyperlinkControlTypeId;classname:=TouchHyperLink")

    ;- Restart the searchcontext to desktop
    ;~ Search popup window under the desktop and setfocus to it
    _UIA_setVar("RTI.SEARCHCONTEXT", $UIA_oDesktop)
    _UIA_Action("Popup_oP1.mainwindow", "setfocus")

    ;~ Change the searchcontext to the popup window
    _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Popup_oP1.mainwindow", "getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Popup_oP2", "getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Popup_oP3", "getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Popup_oP4", "getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Popup_oP5", "getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Popup_oP6", "getobject"))
;~  _UIA_setVar("RTI.SEARCHCONTEXT", _UIA_Action("Popup_oP7", "getobject"))

    ;~ Search for the element in the popup window and setfocus and click on it
;~  _UIA_Action("Popup_oUIElement", "setfocus")
    _UIA_Action("Popup_oUIElement", "click")
;~  _UIA_Action("Popup_oUIElement", "highlight")

endFunc

Exit

 

Edited by junkew
Posted (edited)

Well, that code took 167 secs to run. But i had to comment out some stuff i have no idea what it is:

; _UIA_setVar("oScreen1_oP4","Title:=Deelvenster Verkenner;controltype:=UIA_PaneControlTypeId;class:=HWNDView")   ;Deelvenster Verkenner
   ; _UIA_setVar("oScreen1_oP5","Title:=Deelvenster Mapindeling;controltype:=UIA_PaneControlTypeId;class:=Element")  ;Deelvenster Mapindeling
    _UIA_setVar("oScreen1_oP6","controltype:=UIA_PaneControlTypeId;class:=XElement")    ;
    _UIA_setVar("oScreen1_oP7","Title:=;controltype:=UIA_PaneControlTypeId;class:=XBaby")   ;
   ; _UIA_setVar("oScreen1_oP8","Title:=Een bestandstype of protocol aan een specifiek programma koppelen;controltype:=UIA_PaneControlTypeId;class:=SUDCplPage") ;Een bestandstype of protocol aan een specifiek programma koppelen
   ; _UIA_setVar("oScreen1_oP9","Title:=Een bestandstype of protocol aan een specifiek programma koppelen;controltype:=UIA_PaneControlTypeId;class:=Element")    ;Een bestandstype of protocol aan een specifiek programma koppelen

Those titles mean nothing to me.

  Reveal hidden contents

The fastest code is the last you posted, this one, takes "only" 15 secs to find the more apps hyperlink and do the click.

Local $UIA_Desktop
_UIA_setVar("RTI.SEARCHCONTEXT", $UIA_Desktop)
_UIA_Action("Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "setfocus")
_UIA_setVar("RTI.SEARCHCONTEXT",_UIA_Action("Title:=How do you want to open.*;controltype:=UIA_WindowControlTypeId;class:=Shell_Flyout", "getobject"))
_UIA_Action("Title:=((Meer apps)|(More apps));controltype:=UIA_HyperlinkControlTypeId;class:=TouchHyperLink", "click")

Although it complains about the variable not being set before, so maybe that screws it up a bit.

  Reveal hidden contents

 

Edited by careca
  Reveal hidden contents

IUIAutomation - Topic with framework and examples

Au3Record.exe

Posted

With the UIA wrappers you will not get much more speed as it will search in the whole extensionlist with a findall which returns 5000+ elements where the actual element that matches is at position before 50.

However the UI AutomionLibrary in itself can get fast to your objects with either

1. findfirst
https://msdn.microsoft.com/en-us/library/windows/desktop/ee696029(v=vs.85).aspx

and in the examples.zip there are some examples that make use of findfirst

2. treewalker

and in the examples.zip there are some examples that make use of treewalker

But above 2 options are more difficult to learn.  I am having not much time to write these examples but will put it on my TODO list as this application is a nice one to show all stuff.

Posted

Ok, thank you.

  Reveal hidden contents

IUIAutomation - Topic with framework and examples

Au3Record.exe

Posted

This is based on an UI Automation event handler:

#AutoIt3Wrapper_UseX64=y
#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6

#include "CUIAutomation2.au3" ; junkew
#include "ObjectFromTag.au3"  ; trancexx

AutoItSetOption("MustDeclareVars", 1)

Global Const $S_OK = 0x00000000
Global Const $E_NOINTERFACE = 0x80004002
Global Const $sIID_IUnknown = "{00000000-0000-0000-C000-000000000046}"

Global $oUIAutomation
Global $bShellFlyoutFound = False

SetAssociations()


Func SetAssociations()
  Local $sText1, $Vis
  ShellExecute(@WindowsDir & '\system32\control.exe', '/name Microsoft.DefaultPrograms /page pageFileAssoc')
  WinWaitActive("Set Associations", 'Set Associations', 10)
  Local $hWnd = WinGetHandle("[CLASS:CabinetWClass]", "Set Associations")
  If $hWnd <> 0 Or @error <> 0 Then
    Do
      $Vis = ControlCommand($hWnd, "", "[CLASS:Button; INSTANCE:1]", 'IsVisible')
    Until $Vis = 1
    Sleep(100)
    Local $iCount = ControlListView($hWnd, "", "SysListView321", "GetItemCount")
    For $k = 0 To $iCount - 1
      $sText1 = ControlListView($hWnd, "", "SysListView321", "GetText", $k, 0)
      If $sText1 = '.mp3' Then
        ControlListView($hWnd, "", "SysListView321", "Select", $k, $k)
        ConsoleWrite( "SysListView321, Item " & $k & ": " & $sText1 & @CRLF & @CRLF )
        ControlClick($hWnd, "", "[CLASS:Button; INSTANCE:1]")
        HandlePopupWindow()
      EndIf
    Next
  EndIf
EndFunc

Func HandlePopupWindow()
  ; Create custom UI Automation event handler
  Local $tIUIAutomationEventHandler, $oIUIAutomationEventHandler
  $oIUIAutomationEventHandler = ObjectFromTag( "oIUIAutomationEventHandler_", $dtagIUIAutomationEventHandler, $tIUIAutomationEventHandler, True )
  If Not IsObj( $oIUIAutomationEventHandler ) Then Return ConsoleWrite( "$oIUIAutomationEventHandler ERR" & @CRLF )
  ConsoleWrite( "$oIUIAutomationEventHandler OK" & @CRLF )

  ; Create UI Automation object
  $oUIAutomation = ObjCreateInterface( $sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation )
  If Not IsObj( $oUIAutomation ) Then Return ConsoleWrite( "$oUIAutomation ERR" & @CRLF )
  ConsoleWrite( "$oUIAutomation OK" & @CRLF )

  ; Desktop
  Local $pDesktop
  $oUIAutomation.GetRootElement( $pDesktop )
  If Not $pDesktop Then Return ConsoleWrite( "$pDesktop ERR" & @CRLF )
  ConsoleWrite( "$pDesktop OK" & @CRLF )

  ; Window open events
  If $oUIAutomation.AddAutomationEventHandler( $UIA_Window_WindowOpenedEventId, $pDesktop, $TreeScope_Subtree, 0, $oIUIAutomationEventHandler ) Then Return ConsoleWrite( "$pIUIAutomationEventHandler ERR" & @CRLF )
  ConsoleWrite( "$oIUIAutomationEventHandler OK" & @CRLF )

  While Not $bShellFlyoutFound And Sleep(10)
  WEnd

  $oUIAutomation.RemoveAutomationEventHandler( $UIA_Window_WindowOpenedEventId, $pDesktop, $oIUIAutomationEventHandler )
EndFunc

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf, $pSender, $iEventId ) ; Ret: long  Par: ptr;int
  ConsoleWrite( "oIUIAutomationEventHandler_HandleAutomationEvent: " & $iEventId & @CRLF )

  If $iEventId = $UIA_Window_WindowOpenedEventId Then
    Local $oSender = ObjCreateInterface( $pSender, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
    $oSender.AddRef()
   
    ConsoleWrite( "Handle    " & Ptr( _UIA_getPropertyValue( $oSender, $UIA_NativeWindowHandlePropertyId ) ) & @CRLF )
    ConsoleWrite( "Name      " & _UIA_getPropertyValue( $oSender, $UIA_NamePropertyId ) & @CRLF )
    ConsoleWrite( "Class     " & _UIA_getPropertyValue( $oSender, $UIA_ClassNamePropertyId ) & @CRLF )
    ConsoleWrite( "Ctrl type " & _UIA_getPropertyValue( $oSender, $UIA_ControlTypePropertyId ) & @CRLF )
    ConsoleWrite( "Ctrl name " & _UIA_getPropertyValue( $oSender, $UIA_LocalizedControlTypePropertyId ) & @CRLF )
    ConsoleWrite( "Value     " & _UIA_getPropertyValue( $oSender, $UIA_LegacyIAccessibleValuePropertyId ) & @CRLF )

    ; Shell_Flyout popup?
    Local $sClassName, $hWindow
    $oSender.GetCurrentPropertyValue( $UIA_ClassNamePropertyId, $sClassName )
    If $sClassName = "Shell_Flyout" Then
      $oSender.GetCurrentPropertyValue( $UIA_NativeWindowHandlePropertyId, $hWindow )

      ; Get UI Automation element from window handle
      Local $pWindow, $oWindow
      $oUIAutomation.ElementFromHandle( $hWindow, $pWindow )
      $oWindow = ObjCreateInterface( $pWindow, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
      If Not IsObj( $oWindow ) Then Return ConsoleWrite( "$oWindow ERR" & @CRLF )
      ConsoleWrite( "$oWindow OK" & @CRLF )

      ; --- More apps ---

      ConsoleWrite( "--- More apps ---" & @CRLF )

      ; Condition
      Local $pCondition
      $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition )
      If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
      ConsoleWrite( "$pCondition OK" & @CRLF )

      ; Find link
      Local $pLink, $oLink
      $oSender.FindFirst( $TreeScope_Descendants, $pCondition, $pLink )
      $oLink = ObjCreateInterface( $pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
      If Not IsObj( $oLink ) Then Return ConsoleWrite( "$oLink ERR" & @CRLF )
      ConsoleWrite( "$oLink OK" & @CRLF )

      ; Click (invoke) link
      Local $pInvoke, $oInvoke
      $oLink.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
      $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
      If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
      ConsoleWrite( "$oInvoke OK" & @CRLF )
      $oInvoke.Invoke()

      ; --- Look for another app on this PC ---

      ConsoleWrite( "--- Look for another app on this PC ---" & @CRLF )

      ; Condition
      $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_HyperlinkControlTypeId, $pCondition )
      If Not $pCondition Then Return ConsoleWrite( "$pCondition ERR" & @CRLF )
      ConsoleWrite( "$pCondition OK" & @CRLF )

      ; Find link
      $oSender.FindFirst( $TreeScope_Descendants, $pCondition, $pLink )
      $oLink = ObjCreateInterface( $pLink, $sIID_IUIAutomationElement, $dtagIUIAutomationElement )
      If Not IsObj( $oLink ) Then Return ConsoleWrite( "$oLink ERR" & @CRLF )
      ConsoleWrite( "$oLink OK" & @CRLF )

      ; Click (invoke) link
      $oLink.GetCurrentPattern( $UIA_InvokePatternId, $pInvoke )
      $oInvoke = ObjCreateInterface( $pInvoke, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern )
      If Not IsObj( $oInvoke ) Then Return ConsoleWrite( "$oInvoke ERR" & @CRLF )
      ConsoleWrite( "$oInvoke OK" & @CRLF )
      $oInvoke.Invoke()

      $bShellFlyoutFound = True
    EndIf
  EndIf

  Return $S_OK
  #forceref $pSelf
EndFunc

Func _UIA_getPropertyValue( $obj, $id )
  Local $vVal
  $obj.GetCurrentPropertyValue( $id, $vVal )
  Return $vVal
EndFunc

Func oIUIAutomationEventHandler_QueryInterface( $pSelf, $pRIID, $pObj ) ; Ret: long  Par: ptr;ptr*
  Local $sIID = StringFromGUID( $pRIID )
  If $sIID = $sIID_IUnknown Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUnknown" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  ElseIf $sIID = $sIID_IUIAutomationEventHandler Then
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler" & @CRLF )
    DllStructSetData( DllStructCreate( "ptr", $pObj ), 1, $pSelf )
    oIUIAutomationEventHandler_AddRef( $pSelf )
    Return $S_OK
  Else
    ConsoleWrite( "oIUIAutomationEventHandler_QueryInterface: " & $sIID & @CRLF )
    Return $E_NOINTERFACE
  EndIf
  #forceref $pSelf
EndFunc

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_AddRef" & @CRLF )
  Return 1
  #forceref $pSelf
EndFunc

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: ulong
  ConsoleWrite( "oIUIAutomationEventHandler_Release" & @CRLF )
  Return 1
  #forceref $pSelf
EndFunc

Func StringFromGUID( $pGUID )
  Local $aResult = DllCall( "ole32.dll", "int", "StringFromGUID2", "struct*", $pGUID, "wstr", "", "int", 40 )
  If @error Then Return SetError( @error, @extended, "" )
  Return SetExtended( $aResult[0], $aResult[2] )
EndFunc

 

Console output:

SysListView321, Item 246: .mp3

Func oIUIAutomationEventHandler_QueryInterface( $pSelf ) ; Ret: long  Par: ptr;ptr
EndFunc
0

Func oIUIAutomationEventHandler_AddRef( $pSelf ) ; Ret: dword
EndFunc
0

Func oIUIAutomationEventHandler_Release( $pSelf ) ; Ret: dword
EndFunc
0

Func oIUIAutomationEventHandler_HandleAutomationEvent( $pSelf ) ; Ret: long  Par: ptr;int
EndFunc
0

oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_Release
$oIUIAutomationEventHandler OK
$oUIAutomation OK
$pDesktop OK
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {ECC8691B-C1DB-4DC0-855E-65F6C551AF49}
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {00000039-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {94EA2B94-E9CC-49E0-C0FF-EE64CA8F5B90}
oIUIAutomationEventHandler_QueryInterface: {00000003-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
$oIUIAutomationEventHandler OK
oIUIAutomationEventHandler_QueryInterface: {0000001B-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {00000018-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {334D391F-0E79-3B15-C9FF-EAC65DD07C42}
oIUIAutomationEventHandler_QueryInterface: {00000040-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {334D391F-0E79-3B15-C9FF-EAC65DD07C42}
oIUIAutomationEventHandler_QueryInterface: {94EA2B94-E9CC-49E0-C0FF-EE64CA8F5B90}
oIUIAutomationEventHandler_QueryInterface: {334D391F-0E79-3B15-C9FF-EAC65DD07C42}
oIUIAutomationEventHandler_QueryInterface: {77DD1250-139C-2BC3-BD95-900ACED61BE5}
oIUIAutomationEventHandler_QueryInterface: {334D391F-0E79-3B15-C9FF-EAC65DD07C42}
oIUIAutomationEventHandler_QueryInterface: {BFD60505-5A1F-4E41-88BA-A6FB07202DA9}
oIUIAutomationEventHandler_QueryInterface: {334D391F-0E79-3B15-C9FF-EAC65DD07C42}
oIUIAutomationEventHandler_QueryInterface: {03FB5C57-D534-45F5-A1F4-D39556983875}
oIUIAutomationEventHandler_QueryInterface: {334D391F-0E79-3B15-C9FF-EAC65DD07C42}
oIUIAutomationEventHandler_QueryInterface: {2C258AE7-50DC-49FF-9D1D-2ECB9A52CDD7}
oIUIAutomationEventHandler_QueryInterface: {00000019-0000-0000-C000-000000000046}
oIUIAutomationEventHandler_QueryInterface: {4C1E39E1-E3E3-4296-AA86-EC938D896E92}
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: {1C733A30-2A1C-11CE-ADE5-00AA0044773D}
oIUIAutomationEventHandler_QueryInterface: IUIAutomationEventHandler
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_HandleAutomationEvent: 20016
Handle    0x0000000000330084
Name      How do you want to open .mp3 files from now on?
Class     Shell_Flyout
Ctrl type 50032
Ctrl name window
Value     
$oWindow OK
--- More apps ---
$pCondition OK
$oLink OK
$oInvoke OK
--- Look for another app on this PC ---
$pCondition OK
$oLink OK
$oInvoke OK
oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_QueryInterface: IUnknown
oIUIAutomationEventHandler_AddRef
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release
oIUIAutomationEventHandler_Release

 

AllCode.7z

Posted

This is very interesting! Sometimes it opens the file select dialog right away, which was the thing i was looking for.

But other times opens the popup and does nothing.

what kind of sorcery is this?

  Reveal hidden contents

IUIAutomation - Topic with framework and examples

Au3Record.exe

Posted
Posted (edited)

It seems to hang, does nothing, have to close it.

  Reveal hidden contents

BTW it seems even more complex than UIA udf from previous posts.

Edited by careca
  Reveal hidden contents

IUIAutomation - Topic with framework and examples

Au3Record.exe

Posted

It appears that you are running the code as 32 bit code. If your CPU and operating system are 64 bit, you must also run the code as 64 bit.

Posted

Didnt touch the code:

#AutoIt3Wrapper_UseX64=y

 

  Reveal hidden contents

IUIAutomation - Topic with framework and examples

Au3Record.exe

Posted

I saw that you actually ran the code as 64 bit. I have not had any errors when I have tested the code so I cannot really say anything. You must try to debug the code in the usual way. Is there any difference between the circumstances the times where it goes well and wrong?

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...