Jump to content
junkew

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

rvicta

Okay, here's what SimpleSpy gives me.

In my script, I can read the contents of an edit control fine, it's the text control that I'm having an issue with. In SimpleSpy it shows that I should be able to get the value.

https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/?do=findComment&comment=1156373 
At least we have an element title: [Part Number Entry] class: []

Having the following values for all properties: 
Title is:= <Part Number Entry>
Class:= <>
Control Type:= <UIA_TextControlTypeId>
ID:= <50020>    Hex ID (0000C364)
Position & Size:= 468;300;197;24

*** Parent Information top down ***
6: Title is: <BS1-My Favorites>
Class:= <Chrome_WidgetWin_1>
Control Type:= <UIA_WindowControlTypeId>
ID:= <50032>    Hex ID (0000C370)
Position & Size:= 21;20;1663;1056

"Title:=BS1-My Favorites;
controltype:=UIA_WindowControlTypeId;
class:=Chrome_WidgetWin_1"" 
5: Title is: <BS1-My Favorites>
Class:= <Chrome_RenderWidgetHostHWND>
Control Type:= <UIA_DocumentControlTypeId>
ID:= <50030>    Hex ID (0000C36E)
Position & Size:= 29;105;1647;963

"Title:=BS1-My Favorites;
controltype:=UIA_DocumentControlTypeId;
class:=Chrome_RenderWidgetHostHWND""    
4: Title is: <>
Class:= <>
Control Type:= <UIA_CustomControlTypeId>
ID:= <50025>    Hex ID (0000C369)
Position & Size:= 29;105;1647;808

"Title:=;
controltype:=UIA_CustomControlTypeId;
class:=""   
3: Title is: <>
Class:= <>
Control Type:= <UIA_CustomControlTypeId>
ID:= <50025>    Hex ID (0000C369)
Position & Size:= 29;140;1647;773

"Title:=;
controltype:=UIA_CustomControlTypeId;
class:=""   
2: Title is: <>
Class:= <>
Control Type:= <UIA_CustomControlTypeId>
ID:= <50025>    Hex ID (0000C369)
Position & Size:= 372;263;961;688

"Title:=;
controltype:=UIA_CustomControlTypeId;
class:=""   
1: Title is: <Part Number Entry GM :AD1000                     Description GEAR OIL Non-Stocked                                                                                            On Hand 0 Last Sale 10/01/13 In Invent     Stock Group* 203 Phased Out 3/07/15 Group Cost Stock Status                      Trade        Order Status                                List 27.68 Minimum Flat Price Maximum Comp 1 Pack Quantity 1 Comp 2 Low Year Update* Y High Year Prog Code   Reman Ind New             Return Code Price Guide Promo Code*  Taxable*   Y Remarks/Dsp Function* Ok Exit Cancel Bin/Shelf* >
Class:= <>
Control Type:= <UIA_CustomControlTypeId>
ID:= <50025>    Hex ID (0000C369)
Position & Size:= 382;289;1023;616

"Title:=Part Number Entry GM :AD1000                     Description GEAR OIL Non-Stocked                                                                                            On Hand 0 Last Sale 10/01/13 In Invent     Stock Group* 203 Phased Out 3/07/15 Group Cost Stock Status                      Trade        Order Status                                List 27.68 Minimum Flat Price Maximum Comp 1 Pack Quantity 1 Comp 2 Low Year Update* Y High Year Prog Code   Reman Ind New             Return Code Price Guide Promo Code*  Taxable*   Y Remarks/Dsp Function* Ok Exit Cancel Bin/Shelf* ;
controltype:=UIA_CustomControlTypeId;
class:=""   
0: Title is: <>
Class:= <>
Control Type:= <UIA_CustomControlTypeId>
ID:= <50025>    Hex ID (0000C369)
Position & Size:= 467;299;199;26

"Title:=;
controltype:=UIA_CustomControlTypeId;
class:=""   


;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=BS1-My Favorites;
controltype:=UIA_WindowControlTypeId;
class:=Chrome_WidgetWin_1") ;BS1-My Favorites
_UIA_setVar("oP2","Title:=BS1-My Favorites;
controltype:=UIA_DocumentControlTypeId;
class:=Chrome_RenderWidgetHostHWND")    ;BS1-My Favorites
_UIA_setVar("oP3","Title:=;
controltype:=UIA_CustomControlTypeId;
class:=")   ;
_UIA_setVar("oP4","Title:=;
controltype:=UIA_CustomControlTypeId;
class:=")   ;
_UIA_setVar("oP5","Title:=;
controltype:=UIA_CustomControlTypeId;
class:=")   ;
_UIA_setVar("oP6","Title:=Part Number Entry GM :AD1000                     Description GEAR OIL Non-Stocked                                                                                            On Hand 0 Last Sale 10/01/13 In Invent     Stock Group* 203 Phased Out 3/07/15 Group Cost Stock Status                      Trade        Order Status                                List 27.68 Minimum Flat Price Maximum Comp 1 Pack Quantity 1 Comp 2 Low Year Update* Y High Year Prog Code   Reman Ind New             Return Code Price Guide Promo Code*  Taxable*   Y Remarks/Dsp Function* Ok Exit Cancel Bin/Shelf* ;
controltype:=UIA_CustomControlTypeId;
class:=")   ;Part Number Entry GM :AD1000                     Description GEAR OIL Non-Stocked                                                                                            On Hand 0 Last Sale 10/01/13 In Invent     Stock Group* 203 Phased Out 3/07/15 Group Cost Stock Status                      Trade        Order Status                                List 27.68 Minimum Flat Price Maximum Comp 1 Pack Quantity 1 Comp 2 Low Year Update* Y High Year Prog Code   Reman Ind New             Return Code Price Guide Promo Code*  Taxable*   Y Remarks/Dsp Function* Ok Exit Cancel Bin/Shelf* 
_UIA_setVar("oP7","Title:=;
controltype:=UIA_CustomControlTypeId;
class:=")   ;

;~ $oUIElement=_UIA_getObjectByFindAll("Part Number Entry.mainwindow", "title:=Part Number Entry
;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=Part Number Entry;controltype:=UIA_TextControlTypeId;class:="
) ;ControlType:=UIA_TextControlTypeId
;classname:=")

;~ Actions split away from logical/technical definition above can come from configfiles 

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")
;~_UIA_Action("oP4","highlight")
_UIA_Action("oP4","setfocus")
;~_UIA_Action("oP5","highlight")
_UIA_Action("oP5","setfocus")
;~_UIA_Action("oP6","highlight")
_UIA_Action("oP6","setfocus")
;~_UIA_Action("oP7","highlight")
_UIA_Action("oP7","setfocus")

_UIA_action("oUIElement","highlight")
;~_UIA_action("oUIElement","click")


;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP6=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=BS1-My Favorites;
controltype:=UIA_WindowControlTypeId;
class:=Chrome_WidgetWin_1", $treescope_children)    
_UIA_Action($oP6,"setfocus")
Local $oP5=_UIA_getObjectByFindAll($oP6, "Title:=BS1-My Favorites;
controltype:=UIA_DocumentControlTypeId;
class:=Chrome_RenderWidgetHostHWND", $treescope_children)   
_UIA_Action($oP5,"setfocus")
Local $oP4=_UIA_getObjectByFindAll($oP5, "Title:=;
controltype:=UIA_CustomControlTypeId;
class:=", $treescope_children)  
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;
controltype:=UIA_CustomControlTypeId;
class:=", $treescope_children)  
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;
controltype:=UIA_CustomControlTypeId;
class:=", $treescope_children)  
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Part Number Entry GM :AD1000                     Description GEAR OIL Non-Stocked                                                                                            On Hand 0 Last Sale 10/01/13 In Invent     Stock Group* 203 Phased Out 3/07/15 Group Cost Stock Status                      Trade        Order Status                                List 27.68 Minimum Flat Price Maximum Comp 1 Pack Quantity 1 Comp 2 Low Year Update* Y High Year Prog Code   Reman Ind New             Return Code Price Guide Promo Code*  Taxable*   Y Remarks/Dsp Function* Ok Exit Cancel Bin/Shelf* ;
controltype:=UIA_CustomControlTypeId;
class:=", $treescope_children)  
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;
controltype:=UIA_CustomControlTypeId;
class:=", $treescope_children)  
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll("Part Number Entry.mainwindow", "title:=Part Number Entry
;ControlType:=UIA_TextControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=Part Number Entry;ControlType:=
UIA_TextControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")


*** Detailed properties of the highlighted element ***
UIA_title:= <Part Number Entry>
UIA_text:= <Part Number Entry>
UIA_regexptitle:= <Part Number Entry>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <0>
UIA_BoundingRectangle:= <468;300;197;24>
UIA_ProcessId:= <7016>
UIA_ControlType:= <50020>
UIA_LocalizedControlType:= <text>
UIA_Name:= <Part Number Entry>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <False>
UIA_IsEnabled:= <True>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <0>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <True>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleName:= <Part Number Entry>
UIA_LegacyIAccessibleRole:= <41>
UIA_LegacyIAccessibleState:= <0>
UIA_LegacyIAccessibleDefaultAction:= <click ancestor>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:7016,providerId:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

 

Share this post


Link to post
Share on other sites
junkew

released in first post as attachment version 0.66 + examplesV0.66

@rvicta download in first post v0.66 and see example 6 where in last step 10 I just reference a piece of text by name only.

Example could be

;~ Find the chrome window
local $oChrome = _UIA_getFirstObjectOfElement($UIA_oDesktop, "class:=Chrome_WidgetWin_1", $treescope_children)
If Not IsObj($oChrome) Then
    _UIA_DumpThemAll($UIA_oDesktop, $treescope_subtree)
    Exit
EndIf
;~ Get the document reference
$oDocument = _UIA_getFirstObjectOfElement($oChrome, "controltype:=" & $UIA_DocumentControlTypeId, $treescope_subtree)
If Not IsObj($oDocument) Then
    _UIA_DumpThemAll($oDocument, $treescope_subtree)
EndIf
    
Local $oUIElement=_UIA_getObjectByFindAll($oDocument, "title:=Part Number Entry;ControlType:=Text", $treescope_subtree)

_UIA_Action($oUIElement,"highlight")
consolewrite(_UIA_action($oUIElement, "property", "title"))

otherwise search in examples for indexrelative:=(+/-)n where you can reference relative n position back or forward close to another object. You can determine n by looking into the result of _UIA_DumpThemAll($oDocument,$treescope_subtree)

Share this post


Link to post
Share on other sites
rvicta

Thank you, junkew, for your suggestions. This worked great for what I was trying to do. I can now read the text controls properly.

But now, I have come upon an odd behavior. This is still the same Chrome session for a customer. I have 3 (actually there are more and they exhibit this same behavior but I'm using these three as an example) edit controls on the screen. The first and third controls have different values in them. When I write their values to the console they showed the correct values. The second edit control, if it is empty on the screen, will show me the same value as the first edit control when I write it to the console. I know the index is correct for this edit control because I commented out the code to read from all other edit controls on the page and the correct one gets highlighted when running the script.

Local $oPhasedOut = _UIA_getObjectByFindAll($oDocument, "controltype:=" & $UIA_TextControlTypeId & ";Index:=15", $treescope_subtree)
   Local $sPhasedOut = _UIA_action($oPhasedOut, "property", "title")
   ConsoleWrite("Phased Out: " & $sPhasedOut & @CRLF)

   Local $oCost = _UIA_getObjectByFindAll($oDocument, "controltype:=" & $UIA_EditControlTypeId & ";Index:=6", $treescope_subtree)
   Local $sCost = _UIA_Action($oCost, "getValue")
   ConsoleWrite("Cost: " & $sCost & @CRLF)

   Local $oTrade = _UIA_getObjectByFindAll($oDocument, "controltype:=" & $UIA_EditControlTypeId & ";Index:=7", $treescope_subtree)
   Local $sTrade = _UIA_Action($oTrade, "getValue")
   ConsoleWrite("Trade: " & $sTrade & @CRLF)

   Local $oList = _UIA_getObjectByFindAll($oDocument, "controltype:=" & $UIA_EditControlTypeId & ";Index:=8", $treescope_subtree)
   Local $sList = _UIA_Action($oList, "getValue")
   ConsoleWrite("List: " & $sList & @CRLF)

The output from SimplySpy for this empty edit control is:

Having the following values for all properties: 
Title is: <>    Class   := <>   controltype:= <UIA_EditControlTypeId>   ,<50004>    , (0000C354)    610;551;87;20
*** Parent Information top down ***
5: Title is: <BS1-My Favorites> Class   := <Chrome_WidgetWin_1> controltype:= <UIA_WindowControlTypeId> ,<50032>    , (0000C370)    14;16;1663;1056
"Title:=BS1-My Favorites;controltype:=UIA_WindowControlTypeId;class:=Chrome_WidgetWin_1""   
4: Title is: <BS1-My Favorites> Class   := <Chrome_RenderWidgetHostHWND>    controltype:= <UIA_DocumentControlTypeId>   ,<50030>    , (0000C36E)    22;101;1647;963
"Title:=BS1-My Favorites;controltype:=UIA_DocumentControlTypeId;class:=Chrome_RenderWidgetHostHWND""    
3: Title is: <> Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    22;101;1647;808
"Title:=;controltype:=UIA_CustomControlTypeId;class:="" 
2: Title is: <> Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    22;136;1647;773
"Title:=;controltype:=UIA_CustomControlTypeId;class:="" 
1: Title is: <> Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    365;259;961;688
"Title:=;controltype:=UIA_CustomControlTypeId;class:="" 
0: Title is: <Part Number Entry GM :AD1000                     Description GEAR OIL Non-Stocked                                                                                            On Hand 0 Last Sale 10/01/13 In Invent     Stock Group* 203 Phased Out 3/07/15 Group Cost Stock Status                      Trade        Order Status                                List 27.68 Minimum Flat Price Maximum Comp 1 Pack Quantity 1 Comp 2 Low Year Update* Y High Year Prog Code   Reman Ind New             Return Code Price Guide Promo Code*  Taxable*   Y Remarks/Dsp Function* Ok Exit Cancel Bin/Shelf* >  Class   := <>   controltype:= <UIA_CustomControlTypeId> ,<50025>    , (0000C369)    375;285;1023;616
"Title:=Part Number Entry GM :AD1000                     Description GEAR OIL Non-Stocked                                                                                            On Hand 0 Last Sale 10/01/13 In Invent     Stock Group* 203 Phased Out 3/07/15 Group Cost Stock Status                      Trade        Order Status                                List 27.68 Minimum Flat Price Maximum Comp 1 Pack Quantity 1 Comp 2 Low Year Update* Y High Year Prog Code   Reman Ind New             Return Code Price Guide Promo Code*  Taxable*   Y Remarks/Dsp Function* Ok Exit Cancel Bin/Shelf* ;controltype:=UIA_CustomControlTypeId;class:=""  


;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=BS1-My Favorites;controltype:=UIA_WindowControlTypeId;class:=Chrome_WidgetWin_1") ;BS1-My Favorites
_UIA_setVar("oP2","Title:=BS1-My Favorites;controltype:=UIA_DocumentControlTypeId;class:=Chrome_RenderWidgetHostHWND")  ;BS1-My Favorites
_UIA_setVar("oP3","Title:=;controltype:=UIA_CustomControlTypeId;class:=")   ;
_UIA_setVar("oP4","Title:=;controltype:=UIA_CustomControlTypeId;class:=")   ;
_UIA_setVar("oP5","Title:=;controltype:=UIA_CustomControlTypeId;class:=")   ;
_UIA_setVar("oP6","Title:=Part Number Entry GM :AD1000                     Description GEAR OIL Non-Stocked                                                                                            On Hand 0 Last Sale 10/01/13 In Invent     Stock Group* 203 Phased Out 3/07/15 Group Cost Stock Status                      Trade        Order Status                                List 27.68 Minimum Flat Price Maximum Comp 1 Pack Quantity 1 Comp 2 Low Year Update* Y High Year Prog Code   Reman Ind New             Return Code Price Guide Promo Code*  Taxable*   Y Remarks/Dsp Function* Ok Exit Cancel Bin/Shelf* ;controltype:=UIA_CustomControlTypeId;class:=")    ;Part Number Entry GM :AD1000                     Description GEAR OIL Non-Stocked                                                                                            On Hand 0 Last Sale 10/01/13 In Invent     Stock Group* 203 Phased Out 3/07/15 Group Cost Stock Status                      Trade        Order Status                                List 27.68 Minimum Flat Price Maximum Comp 1 Pack Quantity 1 Comp 2 Low Year Update* Y High Year Prog Code   Reman Ind New             Return Code Price Guide Promo Code*  Taxable*   Y Remarks/Dsp Function* Ok Exit Cancel Bin/Shelf* 

;~ $oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=;controltype:=UIA_EditControlTypeId;class:=") ;ControlType:=UIA_EditControlTypeId;classname:=")

;~ Actions split away from logical/technical definition above can come from configfiles 

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")
;~_UIA_Action("oP4","highlight")
_UIA_Action("oP4","setfocus")
;~_UIA_Action("oP5","highlight")
_UIA_Action("oP5","setfocus")
;~_UIA_Action("oP6","highlight")
_UIA_Action("oP6","setfocus")

_UIA_action("oUIElement","highlight")
;~_UIA_action("oUIElement","click")


;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP5=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=BS1-My Favorites;controltype:=UIA_WindowControlTypeId;class:=Chrome_WidgetWin_1", $treescope_children)    
_UIA_Action($oP5,"setfocus")
Local $oP4=_UIA_getObjectByFindAll($oP5, "Title:=BS1-My Favorites;controltype:=UIA_DocumentControlTypeId;class:=Chrome_RenderWidgetHostHWND", $treescope_children)  
_UIA_Action($oP4,"setfocus")
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)   
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)   
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)   
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Part Number Entry GM :AD1000                     Description GEAR OIL Non-Stocked                                                                                            On Hand 0 Last Sale 10/01/13 In Invent     Stock Group* 203 Phased Out 3/07/15 Group Cost Stock Status                      Trade        Order Status                                List 27.68 Minimum Flat Price Maximum Comp 1 Pack Quantity 1 Comp 2 Low Year Update* Y High Year Prog Code   Reman Ind New             Return Code Price Guide Promo Code*  Taxable*   Y Remarks/Dsp Function* Ok Exit Cancel Bin/Shelf* ;controltype:=UIA_CustomControlTypeId;class:=", $treescope_children)    
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")


*** Detailed properties of the highlighted element ***
UIA_iaccessiblechildId:= <0>
UIA_handle:= <0>
UIA_BoundingRectangle:= <610;551;87;20>
UIA_ProcessId:= <2964>
UIA_ControlType:= <50004>
UIA_LocalizedControlType:= <edit>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <0>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <True>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <True>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <True>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <False>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleRole:= <42>
UIA_LegacyIAccessibleState:= <1048576>
UIA_LegacyIAccessibleDefaultAction:= <activate>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:2964,providerId:0x0 Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

I noticed that UIA_iaccessiblevalue is not present for empty edit controls. Is there a way to check for this? Then I would probably know that there is no value.

Edited by rvicta

Share this post


Link to post
Share on other sites
junkew

Within simplespy  or uiawrappers I filter out empty values.
To be 100% sure my advice use

  • inspect as part of windows SDK
  • VisualUIAVerifyNative.exe as part of windows SDK (8.1)

I see you use index (which is probably fine for your purpose) but understand clearly also the indexrelative property as that could be usefull to get the textlabel normally nn positions before your editbox if your developers program nicely its the same distance for all textlabel controls.

Share this post


Link to post
Share on other sites
rvicta

Thank you for the help, junkew.

Using $UIA_LegacyIAccessibleValuePropertyId for edit controls I am able to determine if it is empty. Using Inspect.exe I was able to see that ValuePropertId would either contain the value or an empty string.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • bobmcrae
      By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • jackchen
      By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object. Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?
×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.