Jump to content

Hooking other applications custom controls?


Recommended Posts

In an application that I am automating (And writing an API for) I have came across the problem of being unable to hook into elements that are custom to the application and not a generic type of element. Any idea how I would hook into them?

 

For example on is an image display that shows an image.

Link to comment
Share on other sites

In the FAQ Inspect.exe just says in the windows SDK. I don't know what SDK its talking about. And the problem is not that I can't select the image/other control.

 

The problem is once I have it I can't do anything useful with it. Because it is a custom control.

Link to comment
Share on other sites

Output from SimpleSky


 

Quote

 

    https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/?do=findComment&comment=1156373
At least we have an element title: [] class: [WindowsForms10.Window.8.app.0.3553390]

Having the following values for all properties: 
Title is: <>    Class   := <WindowsForms10.Window.8.app.0.3553390>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    1057;306;85;32
*** Parent Information top down ***
2: Title is: < EOS DIGITAL REBEL XS>    Class   := <WindowsForms10.Window.8.app.0.3553390>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    898;116;260;641
"Title:= EOS DIGITAL REBEL XS;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390""    
1: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.3553390>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    901;255;254;161
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390""    
0: Title is: <>    Class   := <WindowsForms10.Window.8.app.0.3553390>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    911;267;234;143
"Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390""    


;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:= EOS DIGITAL REBEL XS;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390")    ; EOS DIGITAL REBEL XS
_UIA_setVar("oP2","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390")    ;
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390")    ;

_UIA_setVar(".mainwindow","title:=;classname:=WindowsForms10.Window.8.app.0.3553390")

;~ Actions split away from logical/technical definition above can come from configfiles 

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")

_UIA_action(".mainwindow","setfocus")


;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP2=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:= EOS DIGITAL REBEL XS;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390", $treescope_children)    
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390", $treescope_children)    
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390", $treescope_children)    
_UIA_Action($oP0,"setfocus")
_UIA_setVar(".mainwindow","title:=;classname:=WindowsForms10.Window.8.app.0.3553390")
_UIA_action(".mainwindow","setfocus")


*** Detailed properties of the highlighted element ***
UIA_class:= <WindowsForms10.Window.8.app.0.3553390>
UIA_regexpclass:= <WindowsForms10.Window.8.app.0.3553390>
UIA_iaccessiblechildId:= <0>
UIA_id:= <olcIso>
UIA_handle:= <591566>
UIA_RuntimeId:= <42;591566>
UIA_BoundingRectangle:= <1057;306;85;32>
UIA_ProcessId:= <36828>
UIA_ControlType:= <50033>
UIA_LocalizedControlType:= <pane>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_AutomationId:= <olcIso>
UIA_ClassName:= <WindowsForms10.Window.8.app.0.3553390>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <591566>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_FrameworkId:= <WinForm>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <False>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleRole:= <10>
UIA_LegacyIAccessibleState:= <1048576>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:35524,providerId:0x906CE Main:Nested [pid:36828,providerId:0x906CE Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

 

 

Link to comment
Share on other sites

So you let me guess what you want? Ocr, snapshot, gettext, ......

If its a textbox and you can activate it most likely you can get text with sending copy paste to clipboard

If its an image with decent readable text tesseract can help out if you just want the image you can capture it as you have the dimensions

 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...