Jump to content
chibill

Hooking other applications custom controls?

Recommended Posts

chibill

In an application that I am automating (And writing an API for) I have came across the problem of being unable to hook into elements that are custom to the application and not a generic type of element. Any idea how I would hook into them?

 

For example on is an image display that shows an image.

Share this post


Link to post
Share on other sites
FrancescoDiMuro

Hi @chibill:)
Have you already tried Control* functions?

Francesco


Click here to see my signature:

Spoiler

I will always thank you for the time you spent for me.
I'm here to ask, and from your response, I'd like to learn.
By my knowledge, I can help someone else, and "that someone" could help in turn another, and so on.

Share this post


Link to post
Share on other sites
chibill

Yes I have. Because they are custom control classes those functions don't seem to return anything useful. Which is why I have asked this question

Share this post


Link to post
Share on other sites
chibill

I have used most of the tools there. My problem is the object is a custom Class one that autoit doesn’t know how to deal with. I will give the class name when I get home

Share this post


Link to post
Share on other sites
Earthshine

Use Inspect.exe


My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
chibill

Where can i find that? 

Share this post


Link to post
Share on other sites
chibill

In the FAQ Inspect.exe just says in the windows SDK. I don't know what SDK its talking about. And the problem is not that I can't select the image/other control.

 

The problem is once I have it I can't do anything useful with it. Because it is a custom control.

Share this post


Link to post
Share on other sites
chibill

According to Au3Info the Control has a class of WindowsForms10.Window.8.app.0.3553390 which doesn't help much

Share this post


Link to post
Share on other sites
chibill

Output from SimpleSky


 

Quote

 

    https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/?do=findComment&comment=1156373
At least we have an element title: [] class: [WindowsForms10.Window.8.app.0.3553390]

Having the following values for all properties: 
Title is: <>    Class   := <WindowsForms10.Window.8.app.0.3553390>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    1057;306;85;32
*** Parent Information top down ***
2: Title is: < EOS DIGITAL REBEL XS>    Class   := <WindowsForms10.Window.8.app.0.3553390>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    898;116;260;641
"Title:= EOS DIGITAL REBEL XS;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390""    
1: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.3553390>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    901;255;254;161
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390""    
0: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.3553390>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    911;267;234;143
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390""    


;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:= EOS DIGITAL REBEL XS;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390")    ; EOS DIGITAL REBEL XS
_UIA_setVar("oP2","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390")    ;
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390")    ;

_UIA_setVar(".mainwindow","title:=;classname:=WindowsForms10.Window.8.app.0.3553390")

;~ Actions split away from logical/technical definition above can come from configfiles 

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")

_UIA_action(".mainwindow","setfocus")


;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP2=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:= EOS DIGITAL REBEL XS;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390", $treescope_children)    
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390", $treescope_children)    
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390", $treescope_children)    
_UIA_Action($oP0,"setfocus")
_UIA_setVar(".mainwindow","title:=;classname:=WindowsForms10.Window.8.app.0.3553390")
_UIA_action(".mainwindow","setfocus")


*** Detailed properties of the highlighted element ***
UIA_class:= <WindowsForms10.Window.8.app.0.3553390>
UIA_regexpclass:= <WindowsForms10.Window.8.app.0.3553390>
UIA_iaccessiblechildId:= <0>
UIA_id:= <olcIso>
UIA_handle:= <591566>
UIA_RuntimeId:= <42;591566>
UIA_BoundingRectangle:= <1057;306;85;32>
UIA_ProcessId:= <36828>
UIA_ControlType:= <50033>
UIA_LocalizedControlType:= <pane>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_AutomationId:= <olcIso>
UIA_ClassName:= <WindowsForms10.Window.8.app.0.3553390>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <591566>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_FrameworkId:= <WinForm>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <False>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleRole:= <10>
UIA_LegacyIAccessibleState:= <1048576>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:35524,providerId:0x906CE Main:Nested [pid:36828,providerId:0x906CE Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

 

 

Share this post


Link to post
Share on other sites
chibill

clicking on the control will not let me see what it is currently set to.

Share this post


Link to post
Share on other sites
junkew

So you let me guess what you want? Ocr, snapshot, gettext, ......

If its a textbox and you can activate it most likely you can get text with sending copy paste to clipboard

If its an image with decent readable text tesseract can help out if you just want the image you can capture it as you have the dimensions

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×