chibill Posted April 26, 2018 Share Posted April 26, 2018 In an application that I am automating (And writing an API for) I have came across the problem of being unable to hook into elements that are custom to the application and not a generic type of element. Any idea how I would hook into them? For example on is an image display that shows an image. Link to comment Share on other sites More sharing options...
FrancescoDiMuro Posted April 27, 2018 Share Posted April 27, 2018 Hi @chibillHave you already tried Control* functions? Francesco Click here to see my signature: Spoiler ALWAYS GOOD TO READ: Forum Rules Forum Etiquette Link to comment Share on other sites More sharing options...
chibill Posted May 4, 2018 Author Share Posted May 4, 2018 Yes I have. Because they are custom control classes those functions don't seem to return anything useful. Which is why I have asked this question Link to comment Share on other sites More sharing options...
junkew Posted May 5, 2018 Share Posted May 5, 2018 Check within FAQ 31 the different spying tools. If it are images still it helps to make the area to search smaller before using imagesearch-findbmp solutions )also desc ribed in frequently asked questions' FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets Link to comment Share on other sites More sharing options...
chibill Posted May 8, 2018 Author Share Posted May 8, 2018 I have used most of the tools there. My problem is the object is a custom Class one that autoit doesn’t know how to deal with. I will give the class name when I get home Link to comment Share on other sites More sharing options...
Earthshine Posted May 8, 2018 Share Posted May 8, 2018 Use Inspect.exe My resources are limited. You must ask the right questions Link to comment Share on other sites More sharing options...
chibill Posted May 8, 2018 Author Share Posted May 8, 2018 Where can i find that? Link to comment Share on other sites More sharing options...
junkew Posted May 8, 2018 Share Posted May 8, 2018 FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets Link to comment Share on other sites More sharing options...
chibill Posted May 10, 2018 Author Share Posted May 10, 2018 In the FAQ Inspect.exe just says in the windows SDK. I don't know what SDK its talking about. And the problem is not that I can't select the image/other control. The problem is once I have it I can't do anything useful with it. Because it is a custom control. Link to comment Share on other sites More sharing options...
chibill Posted May 10, 2018 Author Share Posted May 10, 2018 According to Au3Info the Control has a class of WindowsForms10.Window.8.app.0.3553390 which doesn't help much Link to comment Share on other sites More sharing options...
chibill Posted May 10, 2018 Author Share Posted May 10, 2018 Output from SimpleSky Quote https://www.autoitscript.com/forum/topic/153520-iuiautomation-ms-framework-automate-chrome-ff-ie/?do=findComment&comment=1156373 At least we have an element title: [] class: [WindowsForms10.Window.8.app.0.3553390] Having the following values for all properties: Title is: <> Class := <WindowsForms10.Window.8.app.0.3553390> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) 1057;306;85;32 *** Parent Information top down *** 2: Title is: < EOS DIGITAL REBEL XS> Class := <WindowsForms10.Window.8.app.0.3553390> controltype:= <UIA_WindowControlTypeId> ,<50032> , (0000C370) 898;116;260;641 "Title:= EOS DIGITAL REBEL XS;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390"" 1: Title is: <> Class := <WindowsForms10.Window.8.app.0.3553390> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) 901;255;254;161 "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390"" 0: Title is: <> Class := <WindowsForms10.Window.8.app.0.3553390> controltype:= <UIA_PaneControlTypeId> ,<50033> , (0000C371) 911;267;234;143 "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390"" ;~ *** Standard code maintainable *** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) _UIA_setVar("oP1","Title:= EOS DIGITAL REBEL XS;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390") ; EOS DIGITAL REBEL XS _UIA_setVar("oP2","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390") ; _UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390") ; _UIA_setVar(".mainwindow","title:=;classname:=WindowsForms10.Window.8.app.0.3553390") ;~ Actions split away from logical/technical definition above can come from configfiles ;~_UIA_Action("oP1","highlight") _UIA_Action("oP1","setfocus") ;~_UIA_Action("oP2","highlight") _UIA_Action("oP2","setfocus") ;~_UIA_Action("oP3","highlight") _UIA_Action("oP3","setfocus") _UIA_action(".mainwindow","setfocus") ;~ *** Standard code Flexible*** #include "UIAWrappers.au3" AutoItSetOption("MustDeclareVars", 1) Local $oP2=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:= EOS DIGITAL REBEL XS;controltype:=UIA_WindowControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390", $treescope_children) _UIA_Action($oP2,"setfocus") Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390", $treescope_children) _UIA_Action($oP1,"setfocus") Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=WindowsForms10.Window.8.app.0.3553390", $treescope_children) _UIA_Action($oP0,"setfocus") _UIA_setVar(".mainwindow","title:=;classname:=WindowsForms10.Window.8.app.0.3553390") _UIA_action(".mainwindow","setfocus") *** Detailed properties of the highlighted element *** UIA_class:= <WindowsForms10.Window.8.app.0.3553390> UIA_regexpclass:= <WindowsForms10.Window.8.app.0.3553390> UIA_iaccessiblechildId:= <0> UIA_id:= <olcIso> UIA_handle:= <591566> UIA_RuntimeId:= <42;591566> UIA_BoundingRectangle:= <1057;306;85;32> UIA_ProcessId:= <36828> UIA_ControlType:= <50033> UIA_LocalizedControlType:= <pane> UIA_HasKeyboardFocus:= <False> UIA_IsKeyboardFocusable:= <True> UIA_IsEnabled:= <True> UIA_AutomationId:= <olcIso> UIA_ClassName:= <WindowsForms10.Window.8.app.0.3553390> UIA_Culture:= <0> UIA_IsControlElement:= <True> UIA_IsContentElement:= <True> UIA_IsPassword:= <False> UIA_NativeWindowHandle:= <591566> UIA_IsOffscreen:= <False> UIA_Orientation:= <0> UIA_FrameworkId:= <WinForm> UIA_IsRequiredForForm:= <False> UIA_IsDockPatternAvailable:= <False> UIA_IsExpandCollapsePatternAvailable:= <False> UIA_IsGridItemPatternAvailable:= <False> UIA_IsGridPatternAvailable:= <False> UIA_IsInvokePatternAvailable:= <False> UIA_IsMultipleViewPatternAvailable:= <False> UIA_IsRangeValuePatternAvailable:= <False> UIA_IsScrollPatternAvailable:= <False> UIA_IsScrollItemPatternAvailable:= <False> UIA_IsSelectionItemPatternAvailable:= <False> UIA_IsSelectionPatternAvailable:= <False> UIA_IsTablePatternAvailable:= <False> UIA_IsTableItemPatternAvailable:= <False> UIA_IsTextPatternAvailable:= <False> UIA_IsTogglePatternAvailable:= <False> UIA_IsTransformPatternAvailable:= <False> UIA_IsValuePatternAvailable:= <False> UIA_IsWindowPatternAvailable:= <False> UIA_ValueIsReadOnly:= <True> UIA_RangeValueValue:= <0> UIA_RangeValueIsReadOnly:= <True> UIA_RangeValueMinimum:= <0> UIA_RangeValueMaximum:= <0> UIA_RangeValueLargeChange:= <0> UIA_RangeValueSmallChange:= <0> UIA_ScrollHorizontalScrollPercent:= <0> UIA_ScrollHorizontalViewSize:= <100> UIA_ScrollVerticalScrollPercent:= <0> UIA_ScrollVerticalViewSize:= <100> UIA_ScrollHorizontallyScrollable:= <False> UIA_ScrollVerticallyScrollable:= <False> UIA_SelectionCanSelectMultiple:= <False> UIA_SelectionIsSelectionRequired:= <False> UIA_GridRowCount:= <0> UIA_GridColumnCount:= <0> UIA_GridItemRow:= <0> UIA_GridItemColumn:= <0> UIA_GridItemRowSpan:= <1> UIA_GridItemColumnSpan:= <1> UIA_DockDockPosition:= <5> UIA_ExpandCollapseExpandCollapseState:= <3> UIA_MultipleViewCurrentView:= <0> UIA_WindowCanMaximize:= <False> UIA_WindowCanMinimize:= <False> UIA_WindowWindowVisualState:= <0> UIA_WindowWindowInteractionState:= <0> UIA_WindowIsModal:= <False> UIA_WindowIsTopmost:= <False> UIA_SelectionItemIsSelected:= <False> UIA_TableRowOrColumnMajor:= <2> UIA_ToggleToggleState:= <2> UIA_TransformCanMove:= <False> UIA_TransformCanResize:= <False> UIA_TransformCanRotate:= <False> UIA_IsLegacyIAccessiblePatternAvailable:= <True> UIA_LegacyIAccessibleChildId:= <0> UIA_LegacyIAccessibleRole:= <10> UIA_LegacyIAccessibleState:= <1048576> UIA_IsDataValidForForm:= <False> UIA_ProviderDescription:= <[pid:35524,providerId:0x906CE Main:Nested [pid:36828,providerId:0x906CE Main(parent link):Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]> UIA_IsItemContainerPatternAvailable:= <False> UIA_IsVirtualizedItemPatternAvailable:= <False> UIA_IsSynchronizedInputPatternAvailable:= <False> Link to comment Share on other sites More sharing options...
junkew Posted May 10, 2018 Share Posted May 10, 2018 So you have bounding rectangle with x y so you can move mouse and click mouse in the rectangle. FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets Link to comment Share on other sites More sharing options...
chibill Posted May 11, 2018 Author Share Posted May 11, 2018 clicking on the control will not let me see what it is currently set to. Link to comment Share on other sites More sharing options...
junkew Posted May 16, 2018 Share Posted May 16, 2018 So you let me guess what you want? Ocr, snapshot, gettext, ...... If its a textbox and you can activate it most likely you can get text with sending copy paste to clipboard If its an image with decent readable text tesseract can help out if you just want the image you can capture it as you have the dimensions FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now