Jump to content
junkew

IUIAutomation MS framework automate chrome, FF, IE, ....

Recommended Posts

32 minutes ago, LarsJ said:

TheDcoder, This is not the correct link to documentation.

Oops, sorry, I should have been more careful. Pardon my knowledge of Automation and WinAPI, I suck at both :>


AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Spoiler

My contributions to the AutoIt Community

If I have hurt or offended you in anyway, Please accept my apologies, I never (regardless of the situation) mean to do that to anybody!!!

3fHNZJ.gif

PLEASE JOIN ##AutoIt AND HELP THE IRC AUTOIT COMMUNITY!

Share this post


Link to post
Share on other sites

ttf16.ocx is something very old. Any reference you have to the supplier of this controls

vsflexgrid is similar question but you probably will have more luck on that one if you can find the supplier.

Whats the application you are trying to automate and is it inhouse developed or no access to the sourcecode / developers of the application.

Share this post


Link to post
Share on other sites

Mouse position is retrieved 556-234
At least we have an element title: [] class: [TTF16.WndClass]Having the following values for all properties: 

Title is: <>    Class   := <TTF16.WndClass>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    204;69;1236;777
*** Parent Information top down ***
7: Title is: <User333-中联医院信息系统(旗舰版)-检验信息系统>    Class   := <ThunderRT6FormDC>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    -4;-4;1448;878
"Title:=User333-中联医院信息系统(旗舰版)-检验信息系统;controltype:=UIA_WindowControlTypeId;class:=ThunderRT6FormDC""    
6: Title is: <新版住院护士工作站(当前用户:User333)>    Class   := <ThunderRT6FormDC>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    -4;-4;1448;878
"Title:=新版住院护士工作站(当前用户:User333);controltype:=UIA_WindowControlTypeId;class:=ThunderRT6FormDC""    
5: Title is: <病人事务处理 - 护理病历(当前用户:User333)>    Class   := <ThunderRT6FormDC>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    -4;-4;1448;878
"Title:=病人事务处理 - 护理病历(当前用户:User333);controltype:=UIA_WindowControlTypeId;class:=ThunderRT6FormDC""    
4: Title is: <>    Class   := <AfxOleControl42>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    0;153;1440;693
"Title:=;controltype:=UIA_PaneControlTypeId;class:=AfxOleControl42""    
3: Title is: <Form1(当前用户:User333)>    Class   := <ThunderRT6FormDC>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    0;177;1440;669
"Title:=Form1(当前用户:User333);controltype:=UIA_PaneControlTypeId;class:=ThunderRT6FormDC""    
2: Title is: <(NOSIE)评定记录>    Class   := <ThunderRT6FormDC>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    -4;-4;1448;878
"Title:=(NOSIE)评定记录;controltype:=UIA_WindowControlTypeId;class:=ThunderRT6FormDC""    
1: Title is: <>    Class   := <Afx:2180000:8:10011:0:0>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    204;69;1236;777
"Title:=;controltype:=UIA_PaneControlTypeId;class:=Afx:2180000:8:10011:0:0""    
0: Title is: <>    Class   := <ThunderRT6PictureBoxDC>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    204;69;1236;777
"Title:=;controltype:=UIA_PaneControlTypeId;class:=ThunderRT6PictureBoxDC""    


;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=User333-中联医院信息系统(旗舰版)-检验信息系统;controltype:=UIA_WindowControlTypeId;class:=ThunderRT6FormDC")    ;User333-中联医院信息系统(旗舰版)-检验信息系统
_UIA_setVar("oP2","Title:=新版住院护士工作站(当前用户:User333);controltype:=UIA_WindowControlTypeId;class:=ThunderRT6FormDC")    ;新版住院护士工作站(当前用户:User333)
_UIA_setVar("oP3","Title:=病人事务处理 - 护理病历(当前用户:User333);controltype:=UIA_WindowControlTypeId;class:=ThunderRT6FormDC")    ;病人事务处理 - 护理病历(当前用户:User333)
_UIA_setVar("oP4","Title:=;controltype:=UIA_PaneControlTypeId;class:=AfxOleControl42")    ;
_UIA_setVar("oP5","Title:=Form1(当前用户:User333);controltype:=UIA_PaneControlTypeId;class:=ThunderRT6FormDC")    ;Form1(当前用户:User333)
_UIA_setVar("oP6","Title:=(NOSIE)评定记录;controltype:=UIA_WindowControlTypeId;class:=ThunderRT6FormDC")    ;(NOSIE)评定记录
_UIA_setVar("oP7","Title:=;controltype:=UIA_PaneControlTypeId;class:=Afx:2180000:8:10011:0:0")    ;
_UIA_setVar("oP8","Title:=;controltype:=UIA_PaneControlTypeId;class:=ThunderRT6PictureBoxDC")    ;

_UIA_setVar(".mainwindow","title:=;classname:=TTF16.WndClass")

;~ Actions split away from logical/technical definition above can come from configfiles 

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")
;~_UIA_Action("oP4","highlight")
_UIA_Action("oP4","setfocus")
;~_UIA_Action("oP5","highlight")
_UIA_Action("oP5","setfocus")
;~_UIA_Action("oP6","highlight")
_UIA_Action("oP6","setfocus")
;~_UIA_Action("oP7","highlight")
_UIA_Action("oP7","setfocus")
;~_UIA_Action("oP8","highlight")
_UIA_Action("oP8","setfocus")

_UIA_action(".mainwindow","setfocus")


;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP7=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=User333-中联医院信息系统(旗舰版)-检验信息系统;controltype:=UIA_WindowControlTypeId;class:=ThunderRT6FormDC", $treescope_children)    
_UIA_Action($oP7,"setfocus")
Local $oP6=_UIA_getObjectByFindAll($oP7, "Title:=新版住院护士工作站(当前用户:User333);controltype:=UIA_WindowControlTypeId;class:=ThunderRT6FormDC", $treescope_children)    
_UIA_Action($oP6,"setfocus")
Local $oP5=_UIA_getObjectByFindAll($oP6, "Title:=病人事务处理 - 护理病历(当前用户:User333);controltype:=UIA_WindowControlTypeId;class:=ThunderRT6FormDC", $treescope_children)    
_UIA_Action($oP5,"setfocus")
Local $oP4=_UIA_getObjectByFindAll($oP5, "Title:=;controltype:=UIA_PaneControlTypeId;class:=AfxOleControl42", $treescope_children)    
_UIA_Action($oP4,"setfocus")
Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=Form1(当前用户:User333);controltype:=UIA_PaneControlTypeId;class:=ThunderRT6FormDC", $treescope_children)    
_UIA_Action($oP3,"setfocus")
Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=(NOSIE)评定记录;controltype:=UIA_WindowControlTypeId;class:=ThunderRT6FormDC", $treescope_children)    
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Afx:2180000:8:10011:0:0", $treescope_children)    
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=ThunderRT6PictureBoxDC", $treescope_children)    
_UIA_Action($oP0,"setfocus")
_UIA_setVar(".mainwindow","title:=;classname:=TTF16.WndClass")
_UIA_action(".mainwindow","setfocus")


*** Detailed properties of the highlighted element ***
UIA_class:= <TTF16.WndClass>
UIA_regexpclass:= <TTF16.WndClass>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <67808>
UIA_RuntimeId:= <42;67808>
UIA_BoundingRectangle:= <204;69;1236;777>
; Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Nonclient:Microsoft: Non-Client Proxy (unmanaged:uiautomationcore.dll); Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

Ttf16 control in the window
On the right side of the picture

ttf16.jpg

I need to fill in a lot of data every day
I want to use OBJGET ()
But always shows an error
What is the third parameter of OBJGET?
Thank you

Edited by JLogan3o13

Share this post


Link to post
Share on other sites

Hello again, sorry for my newbie question but am I doing it right?:

_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_ButtonControlTypeId;UIA_HelpText:=MODIFY SETTINGS;UIA_IsOffscreen:=False", $treescope_subtree)

I am trying to target a specific button (it acts more like a tab) but the default code generated by simple spy is not enough so I added 2 new properties, "UIA_HelpText" and "UIA_IsOffscreen". I got both of those values from simple spy.

The first property targets the help text (which is displayed if you hover your mouse over the button) and the second one targets a control which is not off-screen... The first property is unique but when I tried it without the IsOffscreen property, it didn't work :(. It was highlighting a "invisible" control, I couldn't see it but it is in the middle of the window... obviously off screen so I decided to add the Offscreen property and tried it again. Still no luck, now I am doubting myself, am I doing it right?

Thanks for the help in advance :)


AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Spoiler

My contributions to the AutoIt Community

If I have hurt or offended you in anyway, Please accept my apologies, I never (regardless of the situation) mean to do that to anybody!!!

3fHNZJ.gif

PLEASE JOIN ##AutoIt AND HELP THE IRC AUTOIT COMMUNITY!

Share this post


Link to post
Share on other sites

@eqjtlm these controls are from VB6 previous century build. Only if you see a small area / cell highlighting with simplespy you could try with a combination of sendkeys but otherwise its hard to tell how to deal with this exactly

 

@TheDcoder Am I doing it right? is a weird question if its not working  :D title:= with nothing behind it then you can leave it away.  
From all properties you must leave away the UIA_ prefix.  see uiawrappers $UIA_propertiesSupportedArray

["HelpText", $UIA_HelpTextPropertyId], _
["IsOffscreen", $UIA_IsOffscreenPropertyId], _

 

Share this post


Link to post
Share on other sites
49 minutes ago, junkew said:

Am I doing it right? is a weird question if its not working  :D

:>

49 minutes ago, junkew said:

title:= with nothing behind it then you can leave it away.

:o! I didn't notice it, thanks :D

49 minutes ago, junkew said:

From all properties you must leave away the UIA_ prefix.  see uiawrappers $UIA_propertiesSupportedArray

$UIA_propertiesSupportedArray is really useful, thanks for the information, now I can automate things properly :)

Update: It worked! :thumbsup:

Edited by TheDcoder
Added update

AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Spoiler

My contributions to the AutoIt Community

If I have hurt or offended you in anyway, Please accept my apologies, I never (regardless of the situation) mean to do that to anybody!!!

3fHNZJ.gif

PLEASE JOIN ##AutoIt AND HELP THE IRC AUTOIT COMMUNITY!

Share this post


Link to post
Share on other sites
1 hour ago, junkew said:

@eqjtlm  这些控件来自上个世纪VB6版本。只有当你看到一个小区域/单元格突出显示simplespy,你可以尝试与sendkeys的组合,但否则很难说如何处理这个

 

@TheDcoder  我做的对吗? 是一个奇怪的问题,如果它不工作的   :D 标题:=没有任何背后,那么你可以离开它。  
从所有的属性,你必须  离开UIA_前缀。看uiawrappers $ UIA_propertiesSupportedArray

[ “HelpText” , $ UIA_HelpTextPropertyId ]_ 
[ “IsOffscreen” , $ UIA_IsOffscreenPropertyId ]_

 

With sendkey I have long been doing, but one by one input data, very slow, although not use your code, I am still very grateful to your answer! :)

Share this post


Link to post
Share on other sites

Hello, I have another noob question, I think I am being lazy here but I am really confused. Can you point me to a example which show how I can loop through the objects/elements/controls inside a parent object/element/control? I am trying to detect a piece of text inside a "pane", I was planning to match every element's title with a set of predefined title texts and if the text matches one of the titles, I will check if its off-screen or not and if it is not off-screen I need to capture the title and return it.

Thank you for the answer in advance :)


AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Spoiler

My contributions to the AutoIt Community

If I have hurt or offended you in anyway, Please accept my apologies, I never (regardless of the situation) mean to do that to anybody!!!

3fHNZJ.gif

PLEASE JOIN ##AutoIt AND HELP THE IRC AUTOIT COMMUNITY!

Share this post


Link to post
Share on other sites

example 1 where you can see with treewalkers how to walk thru the tree from a certain object (root is desktop)

In the UIAWrappers in functions with find you can see how it works

Be aware on the parameter that controls the number of childs you get and how many levels deep.
All descendants inculding all grandgrandgrand childs is a time consuming job.

Share this post


Link to post
Share on other sites

Thanks, I will look into the example :).

5 minutes ago, junkew said:

Be aware on the parameter that controls the number of childs you get and how many levels deep.
All descendants inculding all grandgrandgrand childs is a time consuming job.

I only need to go 1 level deep, I don't need any grand children :P


AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Spoiler

My contributions to the AutoIt Community

If I have hurt or offended you in anyway, Please accept my apologies, I never (regardless of the situation) mean to do that to anybody!!!

3fHNZJ.gif

PLEASE JOIN ##AutoIt AND HELP THE IRC AUTOIT COMMUNITY!

Share this post


Link to post
Share on other sites

then reading thru this function in uiawrappers

;~ Just dumps all information under a certain object
Func _UIA_DumpThemAll($oElementStart, $treeScope)

together with example 1 would give you the solution

The array of elements is for most people the easiest as they then just can loop with a for/next construction

Share this post


Link to post
Share on other sites

Hello, can anyone have a look at this question that I posted at StackOverflow?: https://stackoverflow.com/questions/43841450/uiautomation-wont-work-in-a-vps-if-i-am-not-connected-via-rdp

I know that it is not encouraged to link to questions on other sites but I thought it would be more suited there than here at the AutoIt Forum... I have tried many things since I asked that question but none worked, I am very desperate to get to the bottom of this hence I wrote this post :).

P.S I asked that at 8th of May


AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Spoiler

My contributions to the AutoIt Community

If I have hurt or offended you in anyway, Please accept my apologies, I never (regardless of the situation) mean to do that to anybody!!!

3fHNZJ.gif

PLEASE JOIN ##AutoIt AND HELP THE IRC AUTOIT COMMUNITY!

Share this post


Link to post
Share on other sites

Hi all,

I'm having hard time trying to simply pass text or button clicks to a Qwidget, I'm not a programmer but I know how to use Autoit

I've found this thread and I'm using SimpleSpy to get a simple code to interact with my Qwidget

Unfortunately I can't focus the windows with the code provided, this is strange, the script is just hanging...

In this example I have open an "Open File" window inside the Qwidget this is why I have #32770 class window
 

#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=CLO Enterprise OnlineAuth;controltype:=UIA_PaneControlTypeId;class:=QWidget")    ;CLO Enterprise OnlineAuth
_UIA_setVar("oP2","Title:=Open File;controltype:=UIA_WindowControlTypeId;class:=#32770")    ;Open File

;~ $oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_TitleBarControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=;controltype:=UIA_TitleBarControlTypeId;class:=") ;ControlType:=UIA_TitleBarControlTypeId;classname:=")

;~ Actions split away from logical/technical definition above can come from configfiles

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")

WinActivate ("[class:QWidget]","") or WinActivate ("[class:#32770]","") will work to get the focus..

I'm running windows 10 x64

Thanks for your help

 

Here's wat spy gives me :

Quote

Mouse position is retrieved 1134-141
At least we have an element title: [] class: []

Having the following values for all properties:
Title is: <>    Class   := <>    controltype:= <UIA_TitleBarControlTypeId>    ,<50037>    , (0000C375)    271;119;1286;28
*** Parent Information top down ***
1: Title is: <CLO Enterprise OnlineAuth>    Class   := <QWidget>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    0;0;2560;1439
"Title:=CLO Enterprise OnlineAuth;controltype:=UIA_PaneControlTypeId;class:=QWidget""    
0: Title is: <Open File>    Class   := <#32770>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    247;116;1318;1136
"Title:=Open File;controltype:=UIA_WindowControlTypeId;class:=#32770""    


;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=CLO Enterprise OnlineAuth;controltype:=UIA_PaneControlTypeId;class:=QWidget")    ;CLO Enterprise OnlineAuth
_UIA_setVar("oP2","Title:=Open File;controltype:=UIA_WindowControlTypeId;class:=#32770")    ;Open File

;~ $oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_TitleBarControlTypeId", $treescope_subtree)
_UIA_setVar("oUIElement","Title:=;controltype:=UIA_TitleBarControlTypeId;class:=") ;ControlType:=UIA_TitleBarControlTypeId;classname:=")

;~ Actions split away from logical/technical definition above can come from configfiles

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")

;~_UIA_action($oUIElement","highlight")
;~_UIA_action($oUIElement,"click")
;~_UIA_action("oUIElement","highlight")
;~_UIA_action("oUIElement","click")


;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP1=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=CLO Enterprise OnlineAuth;controltype:=UIA_PaneControlTypeId;class:=QWidget", $treescope_children)    
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=Open File;controltype:=UIA_WindowControlTypeId;class:=#32770", $treescope_children)    
_UIA_Action($oP0,"setfocus")
;~ First find the object in the parent before you can do something
;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_TitleBarControlTypeId", $treescope_subtree)
Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_TitleBarControlTypeId", $treescope_subtree)
;~_UIA_action($oUIElement,"highlight")
_UIA_action($oUIElement,"click")


*** Detailed properties of the highlighted element ***
UIA_iaccessiblevalue:= <Open File>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <0>
UIA_RuntimeId:= <42;6036874;1;-2147483647;6036874;-2;0>
UIA_BoundingRectangle:= <271;119;1286;28>
UIA_ProcessId:= <15260>
UIA_ControlType:= <50037>
UIA_LocalizedControlType:= <barre de titre>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <False>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <0>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <False>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <True>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueValue:= <Open File>
UIA_ValueIsReadOnly:= <False>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleValue:= <Open File>
UIA_LegacyIAccessibleDescription:= <Affiche le nom de la fenêtre et contient des contrôles pour la manipuler>
UIA_LegacyIAccessibleRole:= <1>
UIA_LegacyIAccessibleState:= <1048576>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:16468,hwnd:0x0 Annotation:Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

 

 

 

Edited by ratus69

Share this post


Link to post
Share on other sites

It helps if you tell if the simplespy puts a highlighting rectangle around your object(s).

Check also with inspect.exe from MS WIN 10 SDK if your elements can be found.

you could try to skip $op1.setfocus as frequently not all elements in hierarchy have to be set focus to and as its a popup very likely its not needed it all.

I assume after your window has focus you want to interact with the elements in it. As I see QtWidget in your hierarchy it really depends on which version and how nicely your developers have build the application  with accessibilityattributes.

Share this post


Link to post
Share on other sites

thanks for your help junkew

I don't want to waste your time and pollute this thread so perhaps I will need to open an other thread

I have installed window inspector who seems to be as good as inspector and don't need 2Gb of installation

here it is what I get for the frame containing the menu File Edit ... of the program I want to dialog with

2017-06-13_05361.png

What I would like to do is to select file > import > alembic or to send shift - A which is the shortcut for this command, to the program

I have tried Send, ControlSend with IDs, class etc but nothing seems to work so I thought UIA could make it but I don't get how to send text or command or clicks to this type of Qwidget

anyway thanks again

Edited by ratus69

Share this post


Link to post
Share on other sites

Window Detective will determine far less then inspect and simplespy will do (and ranorex spy will do better I think on QT stuff).
Try yourself on the tabsheets of scite. Detective will see it as one large area whereas inspect will recognize them (and simplespy the same as its both based on IUIAutomation). Why try with Inspect.exe? The simplespy cannot show layers that have a "weird" hierarchy which can be seen with the treewalker(s) that inspect.exe is using.

Anyway from a distance its hard if you do not attach a screenshot of your window that comes up and if you see the window/element in the treeview of your spy.

This example is normally enough to highlight your popup and setfocus

_UIA_setVar("oP2","Title:=Open File;controltype:=UIA_WindowControlTypeId;class:=#32770")    ;Open File 
_UIA_Action("oP2","highlight") 
_UIA_Action("oP2","setfocus")

1.  Is simplespy drawing a rectangle around your window or around your element of interest?

2, What is detective telling you? highlight window or is element of interest highlighted?

If the element of interest is not highlighted then solution is most likely with sendkeys (or try with Ranorex tooling)

If the element of interest is highlighted then you should determine all (grand)parents and start at the root with

_UIA_setVar("oP1",...)    ;Open File 
_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")

and when that works at only oP2 in a similar way it that then does not work on P2 you know you can skip that one and try with P3 and so till you are at leaf level.

Share this post


Link to post
Share on other sites

hello junkew,

1. simplespy draw a red rectangle around the element of interest without problem

2. detective seems to have been improved since it tells a lot than a simple block, here it is what I get for my application CLO and for Scite :

window detective CLO

window detective Scite

detective draw a red rectangle too around the elements like this and focus in the hierarchy list the item selected  :

window detective target element

Now if ranorex spy or inspect gives more element I will install them, just tell me if window detective gives enough info or not

There's one thing I don't understand : if I focus in my window using  WinActivate ("CLO Enterprise OnlineAuth","") and I try to move the mouse using mousemove nothing happen, the mouse is not moving at all, it's very strange...

I will try your examples

thanks a bunch for your help junkew !

 

Share this post


Link to post
Share on other sites

ok I have tried your example and the script is hanging but nothing happen

I have tried it with an other application with a #32770 window class and it takes up to 25 seconds before highlighting the window

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • By bobmcrae
      I am experimenting with UIAWrappers.au3 from junkew to complete an application which presents absolutely no control information using AutoIT Window Info.  While I am able to complete the form successfully, I am not happy with the speed.  As a benchmark, the simple Send method occurs in far less than 1-second, but the UIAutomation approach takes 3-seconds.  I am wondering whether performance gains may be achieved by specifying the controls more precisely; but I am unsure how to do that.  I was able to speed things up a bit by setting $UIA_DefaultWaitTime=0.  The controls of interest are 5-levels deep, as show in the simplespy output below.  It seems I do get faster response by specifying the target/top-level window, as show in the code below.  Any ideas?
      #include "UIAWrappers.au3" _UIA_setVar("Global.Debug", False) _UIA_setVar("Global.Debug.File", False) _UIA_setVar("Global.Highlight", False) _UIA_setVar("DPN","Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window") _UIA_action('DPN','setFocus') _UIA_setVar("DPN.firstName","AutomationId:=txtFirstName") _UIA_setVar("DPN.lastName", "AutomationId:=txtLastName") _UIA_Action('DPN.lastName','setvalue','last name') _UIA_setVar("DPN.ID", "AutomationId:=txtSubjectId") _UIA_setVar("DPN.DOB", "AutomationId:=PART_TextBox") _UIA_setVar("DPN.Ft", "AutomationId:=txtSubjectHeight") _UIA_setVar("DPN.In", "AutomationId:=txtSubjectHeight2") _UIA_Action('DPN.firstName','setvalue','first name') _UIA_Action('DPN.ID','setvalue','ID012345') _UIA_Action('DPN.DOB','setvalue','1/31/1932') _UIA_Action('DPN.Ft','setvalue','6') _UIA_Action('DPN.In','setvalue','1') SimpleSpy output:
      ;~ *** Standard code *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP4=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=NC-stat DPNCheck Communicator;controltype:=UIA_WindowControlTypeId;class:=Window", $treescope_children) _UIA_Action($oP4,"setfocus") Local $oP3=_UIA_getObjectByFindAll($oP4, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) Local $oP2=_UIA_getObjectByFindAll($oP3, "Title:=;controltype:=UIA_TabControlTypeId;class:=TabControl", $treescope_children) Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=Patient;controltype:=UIA_TabItemControlTypeId;class:=TabItem", $treescope_children) Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=Frame", $treescope_children) ;~ First find the object in the parent before you can do something ;~$oUIElement=_UIA_getObjectByFindAll(".mainwindow", "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) Local $oUIElement=_UIA_getObjectByFindAll($oP0, "title:=;ControlType:=UIA_EditControlTypeId", $treescope_subtree) _UIA_action($oUIElement,"click")  
    • By jackchen
      OK,I've made my "Chrome_KeepLastTab.au3" work.This script adds some features to Chrome browser:
      1. Double click on a tab to close the tab.
      2. Keep last tab:This script monitors your mouse clicks and hotkeys,if you are about to close the last tab within Chrome(click close button on the last tab, middle click/double click on the last tab or press Ctrl + w or Ctrl + {F4}), a new tab will be open and then the old tab be closed.
      #include <WindowsConstants.au3> #include <WinAPI.au3> #include "CUIAutomation2.au3" #AutoIt3Wrapper_UseX64=Y ;Should be used for stuff like tagpoint having right struct etc. when running on a 64 bits os ConsoleWrite("@OSArch: " & @OSArch & ", @AutoItX64: " & @AutoItX64 & @CRLF) Global $DoubleClickTime = 500 Global $UIA_oUIAutomation ; The main library core CUI automation reference Global $hMouseEvent, $hMouseHook Global $aMouseEvent[2] Global $KeepLastTab = True ; settings from ini file If $KeepLastTab Then ;The main object with acces to the windows automation api 3.0 $UIA_oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation) If IsObj($UIA_oUIAutomation) Then HotKeySet("^w", "Hotkey_Event") ; Ctrl + w to close tab HotKeySet("^{F4}", "Hotkey_Event") ; Ctrl + {F4} to close tab $DoubleClickTime = DllCall("user32.dll", "uint", "GetDoubleClickTime")[0] OnAutoItExitRegister("UnhookMouse") ; Register mouse events callback $hMouseEvent = DllCallbackRegister("Mouse_Event", "int", "int;ptr;ptr") $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) EndIf EndIf While 1 Sleep(100) WEnd ToolTip("") ; https://www.autoitscript.com/forum/topic/103362-monitoring-mouse-events/ Func Mouse_Event($nCode, $wParam, $lParam) Local $info, $mouseData, $time, $timeDiff If $nCode < 0 Then Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf $tPoint = _WinAPI_GetMousePos() $hWnd = _WinAPI_WindowFromPoint($tPoint) ; if mouse is on the widget window(class: Chrome_RenderWidgetHostHWND), ; use $hWnd = _WinAPI_GetParent($hWnd) to get the parent Chrome window If Not StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_") Then ; Chrome_WidgetWin_1: Chrome window ; Chrome_WidgetWin_2: Chrome menu ; ignore non Chrome window Return _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) EndIf ToolTip($hWnd & " - " & _WinAPI_GetClassName($hWnd)) ;$tagPOINT = "struct;long X;long Y;endstruct" Local Const $MSLLHOOKSTRUCT = $tagPOINT & ";dword mouseData;dword flags;dword time;ulong_ptr dwExtraInfo" $info = DllStructCreate($MSLLHOOKSTRUCT, $lParam) $mouseData = DllStructGetData($info, 3) $time = DllStructGetData($info, 5) $timeDiff = $time - $aMouseEvent[1] Local $block Switch $wParam Case $WM_LBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "LClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "LDClick" Else $aMouseEvent[0] = "LClick" EndIf $block = KeepLastTab($hWnd, $aMouseEvent[0]) Case $WM_MBUTTONUP $aMouseEvent[1] = $time If $aMouseEvent[0] = "MClick" And ($timeDiff) < $DoubleClickTime Then $aMouseEvent[0] = "MDClick" Else $aMouseEvent[0] = "MClick" $block = KeepLastTab($hWnd, $aMouseEvent[0]) EndIf EndSwitch If Not $block Then _WinAPI_CallNextHookEx($hMouseHook, $nCode, $wParam, $lParam) ; Continue processing EndIf EndFunc ;==>Mouse_Event Func UnhookMouse() _WinAPI_UnhookWindowsHookEx($hMouseHook) $hMouseHook = 0 DllCallbackFree($hMouseEvent) $hMouseEvent = 0 EndFunc ;==>UnhookMouse Func Hotkey_Event() Local $block Local $hWnd = WinGetHandle(WinGetTitle("[ACTIVE]")) If StringInStr(_WinAPI_GetClassName($hWnd), "Chrome_WidgetWin_1") Then ; Chrome_WidgetWin_1, Chrome window $block = KeepLastTab($hWnd, "Hotkey") EndIf If Not $block Then HotKeySet("^w") Send(@HotKeyPressed) HotKeySet("^w", "Hotkey_Event") EndIf EndFunc Func KeepLastTab($hWnd, $action = "LClick") ; Possible $action value: LClick, LDClick, MClick, Hotkey ConsoleWrite(@CRLF & $action & " on a Chrome window: " & $hWnd & @CRLF) Local $aMousePos = MouseGetPos() Local $pChrome, $oChrome $UIA_oUIAutomation.ElementFromHandle($hWnd, $pChrome) ; Window $oChrome = ObjCreateInterface($pChrome, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oChrome) Then ConsoleWrite("Failed to get Chrome object from hWnd." & @CRLF) Return EndIf ;$UIA_ControlTypePropertyId = 30003 $oChromeTabs = UIA_getFirstElement($oChrome, $UIA_ControlTypePropertyId, $UIA_TabControlTypeId, $treescope_subtree) If Not IsObj($oChromeTabs) Then ConsoleWrite("Failed to get Chrome tab bar object." & @CRLF) Return EndIf Local $t $oChromeTabs.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1]) Then ; Mouse not on tabs bar ConsoleWrite("Mouse is not on the tab bar. Ignore and return..." & @CRLF) Return EndIf Local $pTrueCondition, $pElements, $iLength, $oAutomationElementArray $UIA_oUIAutomation.CreateTrueCondition($pTrueCondition) $oCondition = ObjCreateInterface($pTrueCondition, $sIID_IUIAutomationCondition, $dtagIUIAutomationCondition) If Not IsObj($oCondition) Then Return $oChromeTabs.FindAll($treescope_children, $oCondition, $pElements) $oAutomationElementArray = ObjCreateInterface($pElements, $sIID_IUIAutomationElementArray, $dtagIUIAutomationElementArray) If Not IsObj($oAutomationElementArray) Then ConsoleWrite("Failed to find all Chrome tabs. " & @CRLF) Return EndIf $oAutomationElementArray.Length($iLength) Local $UIA_pUIElement, $oTab2 Local $iTabs = $iLength - 1 If $iTabs > 1 Then ; more than one tab If $action = "LDClick" Then ConsoleWrite("There are " & $iTabs & " tabs within Chrome window. " & @CRLF) For $i = 1 To $iTabs $oAutomationElementArray.GetElement($i, $UIA_pUIElement) $oTab2 = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab2) Then ContinueLoop $oTab2.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then ContinueLoop If $aMousePos[0] >= $t[0] And $aMousePos[0] <= $t[2]+$t[0] And $aMousePos[1] >= $t[1] And $aMousePos[1] <= $t[3]+$t[1] Then ConsoleWrite("You double clicked on one of " & $iTabs & " tabs. Close the tab and return..." & @CRLF) HotKeySet("^w") Send("^w") HotKeySet("^w", "Hotkey_Event") Return True EndIf Next EndIf Return EndIf ConsoleWrite("There is ONLY one tab within Chrome window. " & @CRLF) $oAutomationElementArray.GetElement(1, $UIA_pUIElement) $oTab = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If Not IsObj($oTab) Then ConsoleWrite("Failed to get the last tab object." & @CRLF) Return EndIf Local $rtTab $oTab.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $rtTab) If UBound($rtTab) < 4 Then Return If $action <> "Hotkey" And ($aMousePos[0] < $rtTab[0] Or $aMousePos[0] > $rtTab[2]+$rtTab[0] Or $aMousePos[1] < $rtTab[1] Or $aMousePos[1] > $rtTab[3]+$rtTab[1]) Then ; Mouse not on the last tab ConsoleWrite("Mouse is not on the last tab. Ignore and return..." & @CRLF) Return EndIf If $action = "LClick" Then $oTabClose = UIA_getFirstElement($oTab, $UIA_ControlTypePropertyId, $UIA_ButtonControlTypeId, $treescope_subtree) If Not IsObj($oTabClose) Then ConsoleWrite("Failed to get the last tab close object." & @CRLF) Return EndIf $oTabClose.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $t) If UBound($t) < 4 Then Return If $aMousePos[0] < $t[0] Or $aMousePos[0] > $t[2]+$t[0] Or $aMousePos[1] < $t[1] Or $aMousePos[1] > $t[3]+$t[1] Then ; Mouse not on the tab close button Return EndIf EndIf ; open a new tab within chrome ConsoleWrite("The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing." & @CRLF) Send("^t") ConsoleWrite("A new tab created!" & @CRLF) Sleep(100) ConsoleWrite("Close the old tab and return..." & @CRLF) ;~ Local $pPattern ;~ $oTabClose.GetCurrentPattern($UIA_InvokePatternId, $pPattern) ;~ $oPattern = ObjCreateInterface($pPattern, $sIID_IUIAutomationInvokePattern, $dtagIUIAutomationInvokePattern) ;~ If IsObj($oPattern) Then ;~ ConsoleWrite("Invoke to close the tab..." & @CRLF) ;~ $oTabClose.SetFocus() ;~ $oPattern.Invoke() ;~ EndIf _WinAPI_UnhookWindowsHookEx($hMouseHook) $aMousePos = MouseGetPos() If $aMousePos[0] >= $rtTab[0] And $aMousePos[0] <= $rtTab[2]+$rtTab[0] And $aMousePos[1] >= $rtTab[1] And $aMousePos[1] <= $rtTab[3]+$rtTab[1] Then MouseClick("middle", $aMousePos[0], $aMousePos[1], 1, 0) Else MouseClick("middle", $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2, 1, 0) ; close the tab MouseMove($aMousePos[0], $aMousePos[1], 0) ; move mouse back to previous position EndIf ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEDOWN, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) ;~ Sleep(100) ;~ _WinAPI_Mouse_Event($MOUSEEVENTF_MIDDLEUP, $rtTab[0]+$rtTab[2]/2, $rtTab[1]+$rtTab[3]/2) $hMouseHook = _WinAPI_SetWindowsHookEx($WH_MOUSE_LL, DllCallbackGetPtr($hMouseEvent), _WinAPI_GetModuleHandle(0)) Return True ; to block mouse click/hot key EndFunc Func UIA_getFirstElement($obj, $propertyID, $tval, $treeScope) Local $pCondition, $oCondition $UIA_oUIAutomation.CreatePropertyCondition($propertyID, $tval, $pCondition) $oCondition = ObjCreateInterface($pCondition, $sIID_IUIAutomationPropertyCondition, $dtagIUIAutomationPropertyCondition) Local $UIA_oUIElement, $UIA_pUIElement $t = $obj.Findfirst($treeScope, $oCondition, $UIA_pUIElement) $UIA_oUIElement = ObjCreateInterface($UIA_pUIElement, $sIID_IUIAutomationElement, $dtagIUIAutomationElement) If IsObj($UIA_oUIElement) Then Return $UIA_oUIElement Return SetError(1, 0, "") EndFunc ;==>UIA_getFirstElement My question is related to #AutoIt3Wrapper_UseX64 when run this script on 64-bit Win 7.
      No mater I set #AutoIt3Wrapper_UseX64=Y or #AutoIt3Wrapper_UseX64=N, this script works very well on hotkey event, while mouse clicks sometimes works if #AutoIt3Wrapper_UseX64=N and sometimes works on Y. Can some one test this and finger out what's wrong?
      Info from SciTe if #AutoIt3Wrapper_UseX64=Y :
      @OSArch: X64, @AutoItX64: 1 Hotkey on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x0000000000140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... Info if #AutoIt3Wrapper_UseX64=N or comment out this line:
      @OSArch: X64, @AutoItX64: 0 Hotkey on a Chrome window: 0x00140330 There is ONLY one tab within Chrome window. The last tab is about to be closed, so we have to open a new tab to prevent Chrome window from closing. A new tab created! Close the old tab and return... LClick on a Chrome window: 0x00140330 Failed to get Chrome tab bar object. Hotkey events and mouse click events share the same function KeepLastTab($hWnd, $action = "LClick"),Why this function triggered by hotkey works on both 32-bit and 64-bit while that triggered by mouse events failed on 32-bit autoit?
×
×
  • Create New...