IUIAutomation MS framework automate chrome, FF, IE, ....

[u547899]

I have a textbox [classname:=WindowsForms10.EDIT.app.0.202c666] on my page that I am trying to set the value of like this:

_UIA_action($element, 'setvalue', 'foo')

That works, and if I do this afterwords I get the expected value:

_UIA_action($element, 'getvalue')

If I send '' (empty string) instead of 'foo' in my 'setvalue' call, the text box clears itself out correctly but the subsequent 'getvalue' call returns the value prior to the input being set to empty string. The same holds true if I prefill a value, sleep for 5 seconds and clear out the field with my keyboard, then call 'getvalue' again; it retains the original value.

Is this a bug? Attached recording of this happening.

 

Edited January 16, 2018 by u547899
uploaded better recording

[junkew]

I think that is related to uiautomation caching, you should refind the element with getobjectbyfindall. Its not done by the wrappers as such its in the ms dll which i cannot look into.

Alternative could be sendkeys ctrl c and gettext from clipboard. Uncertain if i made an action getvalue by keys similar to setvalue using keys

 

 

[u547899]

14 hours ago, junkew said:

you should refind the element with getobjectbyfindall

I modified my code to this:

$element = _UIA_getObjectByFindAll($section, 'title:=Timeframe:;classname:=WindowsForms10.EDIT.app.0.202c666')
ConsoleWrite(_UIA_action($element, 'getvalue') & @CRLF)
_UIA_action($element, 'setvalue', '')
Sleep(1000)
$element2 = _UIA_getObjectByFindAll($section, 'title:=Timeframe:;classname:=WindowsForms10.EDIT.app.0.202c666')
ConsoleWrite(_UIA_action($element2, 'getvalue') & @CRLF)

The output was still the same. I'll probably go ahead and just use ^a^c to copy the text for verification as I prefer using a different method than what was used to set the data to verify. Thanks!

[junkew]

It could help to setfocus to this element and then setfocus to another element and setfocus again as maybe microsoft refreshes when events are sent around.

Another one could be to setfocus and with core uia api getfocusedelement.

[BatMan22]

So I tried to customize the IUI to run without a program using the Calculator example as my starting point.. I'm ALMOST there but not all the way.. It FINDS the name of the radio button but apparently it's not the correct one because our customized version of Microsoft Access gives me the error.. "MicrosoftAccess(basically) can't move the focus to the control Option17". I think I'm toggling the wrong thing.. So I tried to add the ControlType in there but then it finds nothing..

I also posted what simplespy gave me as an output.. Help?  

 

#AutoIt3Wrapper_Au3Check_Parameters=-q -d -w 1 -w 2 -w 3 -w- 4 -w 5 -w 6 -w- 7
#include "UIAWrappers.au3"

local $oButton

#forceref $oButton
#AutoIt3Wrapper_UseX64=Y  ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os

local $program="MicrosoftAccess(basically)"

local $cButton1
$cButton1="Analytical"

Local $omega=_UIA_getFirstObjectOfElement($UIA_oDesktop,$program, $treescope_children)

if isobj($omega) Then

    $oButton=_UIA_getFirstObjectOfElement($omega,"name:=Analytical", $treescope_subtree) ; Works but chooses wrong item apparently?
;~  $oButton=_UIA_getFirstObjectOfElement($omega,"name:=Analytical;ControlType:=UIA_RadioButtonControlTypeId", $treescope_subtree) ; Works but chooses wrong item apparently?

    local $oInvokeP=_UIA_getpattern($oButton,$UIA_InvokePatternID)
    $oInvokeP.Invoke
EndIf

expandcollapse popup

https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/?do=findComment&comment=1156373 
At least we have an element title: [Analytical] class: []

Having the following values for all properties: 
Title is: <Analytical>  Class   := <>   controltype:= <UIA_RadioButtonControlTypeId>    ,<50013>    , (0000C35D)    359;465;17;16
*** Parent Information top down ***
3: Title is: <McCampbell Omega_Me - [ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Offi]> Class   := <OMain>  controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    333;125;1544;831
"Title:=McCampbell Omega_Me - [ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Offi];controltype:=UIA_WindowControlTypeId;class:=OMain""    
2: Title is: <Workspace>    Class   := <MDIClient>  controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    341;270;1528;656
"Title:=Workspace;controltype:=UIA_PaneControlTypeId;class:=MDIClient"" 
1: Title is: <ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Office 2010;DATABASE=LIMS_BACKEND;>   Class   := <OForm>  controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    333;240;1544;694
"Title:=ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Office 2010;DATABASE=LIMS_BACKEND;;controltype:=UIA_WindowControlTypeId;class:=OForm""  
0: Title is: <SelectTopic>  Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    351;372;172;236
"Title:=SelectTopic;controltype:=UIA_CustomControlTypeId;class:=""  


;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=McCampbell Omega_Me - [ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Offi];controltype:=UIA_WindowControlTypeId;class:=OMain")  ;McCampbell Omega_Me - [ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Offi]
_UIA_setVar("oP2","Title:=Workspace;controltype:=UIA_PaneControlTypeId;class:=MDIClient")   ;Workspace
_UIA_setVar("oP3","Title:=ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Office 2010;DATABASE=LIMS_BACKEND;;controltype:=UIA_WindowControlTypeId;class:=OForm")    ;ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Office 2010;DATABASE=LIMS_BACKEND;
_UIA_setVar("oP4","Title:=SelectTopic;controltype:=UIA_CustomControlTypeId;class:=")    ;SelectTopic

;~ $oUIElement=_UIA_getObjectByFindAll("Analytical.mainwindow", "title:=Analytical;ControlType:=UIA_RadioButtonControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=Analytical;controltype:=UIA_RadioButtonControlTypeId;class:=") ;ControlType:=UIA_RadioButtonControlTypeId;classname:=")

;~ Actions split away from logical/technical definition above can come from configfiles 

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")
;~_UIA_Action("oP4","highlight")
_UIA_Action("oP4","setfocus")

_UIA_action("oUIElement","highlight")
;~_UIA_action("oUIElement","click")


;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP3=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=McCampbell Omega_Me - [ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Offi];controltype:=UIA_WindowControlTypeId;class:=OMain", $treescope_children) 
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=Workspace;controltype:=UIA_PaneControlTypeId;class:=MDIClient", $treescope_children)   
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Office 2010;DATABASE=LIMS_BACKEND;;controltype:=UIA_WindowControlTypeId;class:=OForm", $treescope_children)    
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=SelectTopic;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)    
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("Analytical.mainwindow", "title:=Analytical;ControlType:=UIA_RadioButtonControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Analytical;ControlType:=UIA_RadioButtonControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")


*** Detailed properties of the highlighted element ***
UIA_title:= <Analytical>
UIA_text:= <Analytical>
UIA_regexptitle:= <Analytical>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <0>
UIA_BoundingRectangle:= <359;465;17;16>
UIA_ProcessId:= <2840>
UIA_ControlType:= <50013>
UIA_LocalizedControlType:= <radio button>
UIA_Name:= <Analytical>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <0>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <True>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <True>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Analytical>
UIA_LegacyIAccessibleRole:= <45>
UIA_LegacyIAccessibleState:= <1048576>
UIA_LegacyIAccessibleDefaultAction:= <Check>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:2840,hwnd:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

 

Edited January 24, 2018 by BatMan22

[fxg4758]

Hi.

Thanks a lot for your work.

Can someone tell me what i made wrong with this code? I need the click in a submenu but i need to test if is enabled before. Also, this code is working but it put a red line in the desktop. Thanks.

ControlClick("Window", "", "[NAME:Edit1]","right")
WinWait("[CLASS:WindowsForms10.Window.20808.app.0.13965fa_r26_ad1]")
Local $oP1=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Valid;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.13965fa_r26_ad1", $treescope_children)
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=DropDown;controltype:=UIA_MenuControlTypeId;class:=WindowsForms10.Window.20808.app.0.13965fa_r26_ad1", $treescope_children)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Join all;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"click")

 

[TheSaint]

@junkew - Thanks for your UDF.

I am coming to grips with it for the first time, not used it before.

One thing that held me up a bit at the start, was waiting for something to show while hovering the mouse over a control.
It would have been better, if you had written - Hover your mouse over the element you want to handle & Press ctrl+w in one line.
Just a little thing, but it flummoxed me for a bit.

I hope you don't mind that I made some basic changes to your simplespy.au3 script. I wanted a narrower window (200 less) and the window always set ON TOP, plus I wanted elements listed one under the other rather than tabbed across the edit field, and a blank line between each grouping. Hopefully I have not done anything I shouldn't have.

Dcoder said I should share so I am ... so he's the one to blame, not me if something is amiss.

Here it is folks. Backup your copy of junkew's original, and use my modified version instead ... if you dare.

simplespy.au3

I've not read all 32 pages of this topic, so have no idea if anyone else has made any improvements.

Thanks again junkew.

Gawd, the pest Dcoder also wants a screenshot ... is there no escape from his demands.

Edited January 25, 2018 by TheSaint

[TheDcoder]

32 minutes ago, TheSaint said:

Dcoder said I should share so I am ... so he's the one to blame, not me if something is amiss.

Don't blame me, my mind made me do it

31 minutes ago, TheSaint said:

Gawd, the pest Dcoder also wants a screenshot ... is there no escape from his demands.

Edited January 25, 2018 by TheDcoder

[junkew]

@TheSaint no problem, bear in mind I am working (slowly) on a new spy to integrate all different spying tools so far seen.

Will have different tabs with more/les information, more different ways of example coding (plain MS UIA, wrapped, pageobjects in separate files) a certain control.

 

[BatMan22]

I'm having a REALLY hard time learning through these examples/microsoft's site.. Can anyone recommend something for me to read to try and get this all? I don't even know what functions this UDF adds, or syntax for using them. I've tried going through the examples/CUIAutomation2.au3 file... And going through Microsoft's site is like going through sand paper, it's a bunch of single paragraph explanations with no examples, and the examples you find are all in c++ (I think?). Still stuck on post #626 as well. 

 

Any youtube videos or video tutorials? (they really make new concepts easier for me, sometimes) 

Edited January 26, 2018 by BatMan22

[junkew]

• Read the first few posts of this thread
• Read faq 31 and install the different spying tools
• Understand what an element hierarchy is
• Understand what identifies an element with 1,n properties
• Read the uiawrappers to see which actions you have but they are also mentioned in the beginning of this thread
• Some people feel it helps to read hp uft or selenium manuals as it uses similar concepts however with a different implementation.

Your post 626 is about ms access i understand. Please post that in support forum with more details.

And maybe start first with simplespy on calculator so you learn the base concept of identifying, higlighting and clicking elements.

[fxg4758]

Hi.

Can someone tell me why the screen stays with red lines, after i use this code? Thanks for any help possible.

ControlClick("Window", "", "[NAME:Edit1]","right")
WinWait("[CLASS:WindowsForms10.Window.20808.app.0.13965fa_r26_ad1]")
Local $oP1=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Valid;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.13965fa_r26_ad1", $treescope_children)
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=DropDown;controltype:=UIA_MenuControlTypeId;class:=WindowsForms10.Window.20808.app.0.13965fa_r26_ad1", $treescope_children)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Join all;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"click")

 

 

[junkew]

Its higlighting stuff. You can turn it on and off. Default is on.

[fxg4758]

13 hours ago, junkew said:

Its higlighting stuff. You can turn it on and off. Default is on.

Hi junkew.

Thanks a lot for your help. I changed this values in the UIAWrappers.au3 to get the highlight off.

_UIA_setVar("Global.Highlight", False)

;~      If (_UIA_getVar("Global.Highlight") = True) Then
;~          consolewrite("its h value " & isstring(_UIA_getVar("Global.Highlight")) & isbool(_UIA_getVar("Global.Highlight"))  & (_UIA_getVar("Global.Highlight") =true ) & (_UIA_getVar("Global.Highlight") ="False" ) & @CRLF)
;~          _UIA_Highlight($UIA_oUIElementFound)
;~      EndIf

 

[fxg4758]

Hi junkew.

I use the simplespy to get the properties of a EVRVideoHandler (media player), but it don´t show any properties about the position of the file open. Is possible to the use function _UIA_getPropertyValue to get the information?

Regards.

[junkew]

Then check with VisualUIAVerifyNative.exe  and inspect.exe as sometimes elements are after each other. If app is written in java uia will not help. Search then for jabsimplespy that uses java accessibility bridge.

[fxg4758]

1 hour ago, junkew said:

Then check with VisualUIAVerifyNative.exe  and inspect.exe as sometimes elements are after each other. If app is written in java uia will not help. Search then for jabsimplespy that uses java accessibility bridge.

Hi junkew.

Thanks for your post. With the inspect.exe, i got the same results. Where i can get the VisualUIAVerifyNative.exe?

Regards.

[junkew]

See faq31 or search with google its in windows sdk

[fxg4758]

Hi junkew.

Thanks for your help.

With the inspect, i obtain the follow information, but i cannot access to the field Value.Value that it reports. I want to save the value and then send back again after a previous operation.

Can you help?

Regards.

 

How found:  Focus
BoundingRectangle:  {l:153 t:645 r:671 b:657}
ProcessId:  4376
ControlType:    UIA_SliderControlTypeId (0xC35F)
LocalizedControlType:   "controlo de deslize"
Name:   "seek"
HasKeyboardFocus:   true
IsKeyboardFocusable:    true
IsEnabled:  true
HelpText:   ""
IsPassword: false
IsOffscreen:    false
ProviderDescription:    "[pid:4376,hwnd:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]"
Value.IsReadOnly:   false
Value.Value:    "211,2"


Local $oP1=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Test;controltype:=UIA_WindowControlTypeId;class:=", $treescope_children)
Local $oUIElement=_UIA_getObjectByFindAll($oP1, "controltype:=UIA_SliderControlTypeId", $treescope_subtree)
ConsoleWrite(_UIA_action($oUIElement,"getvalue"))

 

[junkew]

_UIA_getPropertyValue

Should be able to use value.value as that seems the propertyname. 

If you know you have the object you can dump all propertyvalues to console just like simplespy is doing.