Jump to content
junkew

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

junkew

Study post 3 example1 thats the raw interface with treewalkers and base condition filters.

This is the uiabase

  1. Create a filter condition.
    Search in examples for CreatePropertyCondition or in this thread
    A true condition is the simplest (will not filter/give back all elements)
    see https://msdn.microsoft.com/en-us/library/ms752331(v=vs.85).aspx#Using_Conditions
     
  2. Do a findall with treescope, filter condition and the reference for the result
    As said then it will only find on full matching and not on partial (which you frequently need if you deal with titles, multi language etc)
$UIA_oUIAutomation.CreateTrueCondition($pTrueCondition)
    $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition)
;~  $oCondition1 = _AutoItObject_WrapperCreate($aCall[1], $dtagIUIAutomationCondition)
;~ Tricky to search all descendants on html objects or from desktop/root element
    $oElementStart.FindAll($TreeScope, $oCondition, $pElements)
; Condition to find View menu item
  Local $pCondition1, $pCondition2
  $oUIAutomation.CreatePropertyCondition( $UIA_ControlTypePropertyId, $UIA_MenuItemControlTypeId, $pCondition1 )
  $oUIAutomation.CreatePropertyCondition( $UIA_NamePropertyId, "View", $pCondition2 )
  $oUIAutomation.CreateAndCondition( $pCondition1, $pCondition2, $pCondition )
  If Not $pCondition Then Return

Thoughts for UIAwrappers (will put it on TODO list)

  • Keep the initial elements in array found of a certain element in memory and scan that first as most likely you will handle n controls under same parent in sequence
    so UIA_SETVAR("RTI.SEARCHCONTEXT", $window)  should set the window and the initial array and
    UIA_Action("checkbox1","click"), UIA_Action("checkbox2","click") should reuse the array thats set as a searching context
  • Like 1

Share this post


Link to post
Share on other sites
u547899

I have a textbox [classname:=WindowsForms10.EDIT.app.0.202c666] on my page that I am trying to set the value of like this:

_UIA_action($element, 'setvalue', 'foo')

That works, and if I do this afterwords I get the expected value:

_UIA_action($element, 'getvalue')

If I send '' (empty string) instead of 'foo' in my 'setvalue' call, the text box clears itself out correctly but the subsequent 'getvalue' call returns the value prior to the input being set to empty string. The same holds true if I prefill a value, sleep for 5 seconds and clear out the field with my keyboard, then call 'getvalue' again; it retains the original value.

Is this a bug? Attached recording of this happening.

 

autoit2.gif

Edited by u547899
uploaded better recording

Share this post


Link to post
Share on other sites
junkew

I think that is related to uiautomation caching, you should refind the element with getobjectbyfindall. Its not done by the wrappers as such its in the ms dll which i cannot look into.

Alternative could be sendkeys ctrl c and gettext from clipboard. Uncertain if i made an action getvalue by keys similar to setvalue using keys

 

 

Share this post


Link to post
Share on other sites
u547899
14 hours ago, junkew said:

you should refind the element with getobjectbyfindall

I modified my code to this:

$element = _UIA_getObjectByFindAll($section, 'title:=Timeframe:;classname:=WindowsForms10.EDIT.app.0.202c666')
ConsoleWrite(_UIA_action($element, 'getvalue') & @CRLF)
_UIA_action($element, 'setvalue', '')
Sleep(1000)
$element2 = _UIA_getObjectByFindAll($section, 'title:=Timeframe:;classname:=WindowsForms10.EDIT.app.0.202c666')
ConsoleWrite(_UIA_action($element2, 'getvalue') & @CRLF)

The output was still the same. I'll probably go ahead and just use ^a^c to copy the text for verification as I prefer using a different method than what was used to set the data to verify. Thanks!

Share this post


Link to post
Share on other sites
junkew

It could help to setfocus to this element and then setfocus to another element and setfocus again as maybe microsoft refreshes when events are sent around.

Another one could be to setfocus and with core uia api getfocusedelement.

Share this post


Link to post
Share on other sites
BatMan22

So I tried to customize the IUI to run without a program using the Calculator example as my starting point.. I'm ALMOST there but not all the way.. It FINDS the name of the radio button but apparently it's not the correct one because our customized version of Microsoft Access gives me the error.. "MicrosoftAccess(basically) can't move the focus to the control Option17". I think I'm toggling the wrong thing.. So I tried to add the ControlType in there but then it finds nothing..

I also posted what simplespy gave me as an output.. Help? :) 

 

#AutoIt3Wrapper_Au3Check_Parameters=-q -d -w 1 -w 2 -w 3 -w- 4 -w 5 -w 6 -w- 7
#include "UIAWrappers.au3"

local $oButton

#forceref $oButton
#AutoIt3Wrapper_UseX64=Y  ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os

local $program="MicrosoftAccess(basically)"

local $cButton1
$cButton1="Analytical"

Local $omega=_UIA_getFirstObjectOfElement($UIA_oDesktop,$program, $treescope_children)

if isobj($omega) Then

    $oButton=_UIA_getFirstObjectOfElement($omega,"name:=Analytical", $treescope_subtree) ; Works but chooses wrong item apparently?
;~  $oButton=_UIA_getFirstObjectOfElement($omega,"name:=Analytical;ControlType:=UIA_RadioButtonControlTypeId", $treescope_subtree) ; Works but chooses wrong item apparently?

    local $oInvokeP=_UIA_getpattern($oButton,$UIA_InvokePatternID)
    $oInvokeP.Invoke
EndIf
https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/?do=findComment&comment=1156373 
At least we have an element title: [Analytical] class: []

Having the following values for all properties: 
Title is: <Analytical>  Class   := <>   controltype:= <UIA_RadioButtonControlTypeId>    ,<50013>    , (0000C35D)    359;465;17;16
*** Parent Information top down ***
3: Title is: <McCampbell Omega_Me - [ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Offi]> Class   := <OMain>  controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    333;125;1544;831
"Title:=McCampbell Omega_Me - [ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Offi];controltype:=UIA_WindowControlTypeId;class:=OMain""    
2: Title is: <Workspace>    Class   := <MDIClient>  controltype:= <UIA_PaneControlTypeId>   ,<50033>    , (0000C371)    341;270;1528;656
"Title:=Workspace;controltype:=UIA_PaneControlTypeId;class:=MDIClient"" 
1: Title is: <ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Office 2010;DATABASE=LIMS_BACKEND;>   Class   := <OForm>  controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    333;240;1544;694
"Title:=ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Office 2010;DATABASE=LIMS_BACKEND;;controltype:=UIA_WindowControlTypeId;class:=OForm""  
0: Title is: <SelectTopic>  Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    351;372;172;236
"Title:=SelectTopic;controltype:=UIA_CustomControlTypeId;class:=""  


;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=McCampbell Omega_Me - [ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Offi];controltype:=UIA_WindowControlTypeId;class:=OMain")  ;McCampbell Omega_Me - [ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Offi]
_UIA_setVar("oP2","Title:=Workspace;controltype:=UIA_PaneControlTypeId;class:=MDIClient")   ;Workspace
_UIA_setVar("oP3","Title:=ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Office 2010;DATABASE=LIMS_BACKEND;;controltype:=UIA_WindowControlTypeId;class:=OForm")    ;ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Office 2010;DATABASE=LIMS_BACKEND;
_UIA_setVar("oP4","Title:=SelectTopic;controltype:=UIA_CustomControlTypeId;class:=")    ;SelectTopic

;~ $oUIElement=_UIA_getObjectByFindAll("Analytical.mainwindow", "title:=Analytical;ControlType:=UIA_RadioButtonControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=Analytical;controltype:=UIA_RadioButtonControlTypeId;class:=") ;ControlType:=UIA_RadioButtonControlTypeId;classname:=")

;~ Actions split away from logical/technical definition above can come from configfiles 

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")
;~_UIA_Action("oP4","highlight")
_UIA_Action("oP4","setfocus")

_UIA_action("oUIElement","highlight")
;~_UIA_action("oUIElement","click")


;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP3=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=McCampbell Omega_Me - [ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Offi];controltype:=UIA_WindowControlTypeId;class:=OMain", $treescope_children) 
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=Workspace;controltype:=UIA_PaneControlTypeId;class:=MDIClient", $treescope_children)   
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=ODBC;DRIVER=SQL Server Native Client 10.0;SERVER=tcp:MAI-SQL\MAI_BACKEND;UID=huan;PWD=;Trusted_Connection=Yes;APP=Microsoft Office 2010;DATABASE=LIMS_BACKEND;;controltype:=UIA_WindowControlTypeId;class:=OForm", $treescope_children)    
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=SelectTopic;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)    
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("Analytical.mainwindow", "title:=Analytical;ControlType:=UIA_RadioButtonControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Analytical;ControlType:=UIA_RadioButtonControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")


*** Detailed properties of the highlighted element ***
UIA_title:= <Analytical>
UIA_text:= <Analytical>
UIA_regexptitle:= <Analytical>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <0>
UIA_BoundingRectangle:= <359;465;17;16>
UIA_ProcessId:= <2840>
UIA_ControlType:= <50013>
UIA_LocalizedControlType:= <radio button>
UIA_Name:= <Analytical>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <0>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <True>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <True>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Analytical>
UIA_LegacyIAccessibleRole:= <45>
UIA_LegacyIAccessibleState:= <1048576>
UIA_LegacyIAccessibleDefaultAction:= <Check>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:2840,hwnd:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

 

Edited by BatMan22

Share this post


Link to post
Share on other sites
fxg4758

Hi.

Thanks a lot for your work.

Can someone tell me what i made wrong with this code? I need the click in a submenu but i need to test if is enabled before. Also, this code is working but it put a red line in the desktop. Thanks.

ControlClick("Window", "", "[NAME:Edit1]","right")
WinWait("[CLASS:WindowsForms10.Window.20808.app.0.13965fa_r26_ad1]")
Local $oP1=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Valid;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.13965fa_r26_ad1", $treescope_children)
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=DropDown;controltype:=UIA_MenuControlTypeId;class:=WindowsForms10.Window.20808.app.0.13965fa_r26_ad1", $treescope_children)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Join all;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"click")

 

Share this post


Link to post
Share on other sites
TheSaint

@junkew - Thanks for your UDF. :D

I am coming to grips with it for the first time, not used it before.

One thing that held me up a bit at the start, was waiting for something to show while hovering the mouse over a control.
It would have been better, if you had written - Hover your mouse over the element you want to handle & Press ctrl+w in one line.
Just a little thing, but it flummoxed me for a bit. :)

I hope you don't mind that I made some basic changes to your simplespy.au3 script. I wanted a narrower window (200 less) and the window always set ON TOP, plus I wanted elements listed one under the other rather than tabbed across the edit field, and a blank line between each grouping. Hopefully I have not done anything I shouldn't have. :blink:

Dcoder said I should share so I am ... so he's the one to blame, not me if something is amiss. :muttley:

Here it is folks. Backup your copy of junkew's original, and use my modified version instead ... if you dare. :P

simplespy.au3

I've not read all 32 pages of this topic, so have no idea if anyone else has made any improvements.

Thanks again junkew.

Gawd, the pest Dcoder also wants a screenshot ... is there no escape from his demands. :lol:

Spy window.png

Edited by TheSaint
  • Like 1

AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Make sure brain is in gear before opening mouth!
Remember, what is not said, can be just as important as what is said.

Spoiler

What is the Secret Key? Life is like a Donut

If I put effort into communication, I expect you to read properly & fully, or just not comment.
Ignoring those who try to divert conversation with irrelevancies.
If I'm intent on insulting you or being rude, I will be obvious, not ambiguous about it.
I'm only big and bad, to those who have an over-active imagination.

I may have the Artistic Liesense ;) to disagree with you. TheSaint's Toolbox (be advised many downloads are not working due to ISP screwup with my storage)

userbar.png

Share this post


Link to post
Share on other sites
TheDcoder
32 minutes ago, TheSaint said:

Dcoder said I should share so I am ... so he's the one to blame, not me if something is amiss. :muttley:

Don't blame me, my mind made me do it :P

31 minutes ago, TheSaint said:

Gawd, the pest Dcoder also wants a screenshot ... is there no escape from his demands. :lol:

:muttley:

Edited by TheDcoder

AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Spoiler

My contributions to the AutoIt Community

Some messages & Apologizes:

If I hurt you, Please accept my apologies, I never (regardless of the situation) mean to hurt anybody!!!

Also, I am very busy with my project so I will appear in the last row of the online list, if you want to contact me: Email@TheDcoder.xyz

Or you can have a nice chat with me in freenode, I use the same nick on freenode too!

3fHNZJ.gif

PLEASE JOIN ##AutoIt AND HELP THE IRC AUTOIT COMMUNITY!

Share this post


Link to post
Share on other sites
junkew

@TheSaint no problem, bear in mind I am working (slowly) on a new spy to integrate all different spying tools so far seen.

Will have different tabs with more/les information, more different ways of example coding (plain MS UIA, wrapped, pageobjects in separate files) a certain control.

 

  • Like 3

Share this post


Link to post
Share on other sites
BatMan22

I'm having a REALLY hard time learning through these examples/microsoft's site.. Can anyone recommend something for me to read to try and get this all? I don't even know what functions this UDF adds, or syntax for using them. I've tried going through the examples/CUIAutomation2.au3 file... And going through Microsoft's site is like going through sand paper, it's a bunch of single paragraph explanations with no examples, and the examples you find are all in c++ (I think?). Still stuck on post #626 as well. 

 

Any youtube videos or video tutorials? (they really make new concepts easier for me, sometimes) 

Edited by BatMan22

Share this post


Link to post
Share on other sites
junkew
  • Read the first few posts of this thread
  • Read faq 31 and install the different spying tools
  • Understand what an element hierarchy is
  • Understand what identifies an element with 1,n properties
  • Read the uiawrappers to see which actions you have but they are also mentioned in the beginning of this thread
  • Some people feel it helps to read hp uft or selenium manuals as it uses similar concepts however with a different implementation.

Your post 626 is about ms access i understand. Please post that in support forum with more details.

And maybe start first with simplespy on calculator so you learn the base concept of identifying, higlighting and clicking elements.

Share this post


Link to post
Share on other sites
fxg4758

Hi.

Can someone tell me why the screen stays with red lines, after i use this code? Thanks for any help possible.

Screen.JPG

ControlClick("Window", "", "[NAME:Edit1]","right")
WinWait("[CLASS:WindowsForms10.Window.20808.app.0.13965fa_r26_ad1]")
Local $oP1=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Valid;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.13965fa_r26_ad1", $treescope_children)
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=DropDown;controltype:=UIA_MenuControlTypeId;class:=WindowsForms10.Window.20808.app.0.13965fa_r26_ad1", $treescope_children)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Join all;ControlType:=UIA_MenuItemControlTypeId", $treescope_subtree)
_UIA_action($oUIElement,"click")

 

 

Share this post


Link to post
Share on other sites
fxg4758
13 hours ago, junkew said:

Its higlighting stuff. You can turn it on and off. Default is on.

Hi junkew.

Thanks a lot for your help. I changed this values in the UIAWrappers.au3 to get the highlight off.

_UIA_setVar("Global.Highlight", False)

;~      If (_UIA_getVar("Global.Highlight") = True) Then
;~          consolewrite("its h value " & isstring(_UIA_getVar("Global.Highlight")) & isbool(_UIA_getVar("Global.Highlight"))  & (_UIA_getVar("Global.Highlight") =true ) & (_UIA_getVar("Global.Highlight") ="False" ) & @CRLF)
;~          _UIA_Highlight($UIA_oUIElementFound)
;~      EndIf

 

Share this post


Link to post
Share on other sites
fxg4758

Hi junkew.

I use the simplespy to get the properties of a EVRVideoHandler (media player), but it don´t show any properties about the position of the file open. Is possible to the use function _UIA_getPropertyValue to get the information?

Regards.

Share this post


Link to post
Share on other sites
fxg4758
1 hour ago, junkew said:

Then check with VisualUIAVerifyNative.exe  and inspect.exe as sometimes elements are after each other. If app is written in java uia will not help. Search then for jabsimplespy that uses java accessibility bridge.

Hi junkew.

Thanks for your post. With the inspect.exe, i got the same results. Where i can get the VisualUIAVerifyNative.exe?

Regards.

Share this post


Link to post
Share on other sites
fxg4758

Hi junkew.

Thanks for your help.

With the inspect, i obtain the follow information, but i cannot access to the field Value.Value that it reports. I want to save the value and then send back again after a previous operation.

Can you help?

Regards.

 

How found:  Focus
BoundingRectangle:  {l:153 t:645 r:671 b:657}
ProcessId:  4376
ControlType:    UIA_SliderControlTypeId (0xC35F)
LocalizedControlType:   "controlo de deslize"
Name:   "seek"
HasKeyboardFocus:   true
IsKeyboardFocusable:    true
IsEnabled:  true
HelpText:   ""
IsPassword: false
IsOffscreen:    false
ProviderDescription:    "[pid:4376,hwnd:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]"
Value.IsReadOnly:   false
Value.Value:    "211,2"


Local $oP1=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Test;controltype:=UIA_WindowControlTypeId;class:=", $treescope_children)
Local $oUIElement=_UIA_getObjectByFindAll($oP1, "controltype:=UIA_SliderControlTypeId", $treescope_subtree)
ConsoleWrite(_UIA_action($oUIElement,"getvalue"))

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • bobmcrae
      By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • jackchen
      By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object. Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?
×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.