Jump to content
SlackerAl

ControlClick to Tk button control

Recommended Posts

Hi,

I am experimenting with the automation of Altair HyperWorks - HyperView.

I used the Au3Info_x64.exe tool to obtain information on a button like control within the application ClassnameNN TkChild130 as a test. I have left the application window open on my desktop.

I then run the following code (this is the entire au3 file):

Opt('MustDeclareVars', 1)
Local $hWnd = WinWaitActive("Untitled - HyperWorks v14.0")
ControlClick ( $hWnd, "", "TkChild130")

I then click on the Altair application (forcing it to become active and tripping the WinWaitActive). As soon as I click the title bar of the application (there are no associated events with this action) I see the control I previously identified as TkChild130 produce a mouse click animation, but the menu system change associated with clicking that control does not happen.

So I know the window handle is correct, and I know the control is the correct ID, but the click event seems to be only partially recognised. I have tried repeating the last line to send 2 consecutive clicks, but there is no change. I can immediately click the control and it provides the menu system update I was expecting.

Is there an issue with interacting with Tk controls? 

I am using this on Win 7 Enterprise, I am not a privileged user (and cannot be).

Thank you in advance.


Problem solving step 1: Write a simple, self-contained, running, replicator of your problem.

Share this post


Link to post
Share on other sites

Additional Information:

I see the mouse-click animation is no different to the mouse-over animation... perhaps only the latter is occurring.


Problem solving step 1: Write a simple, self-contained, running, replicator of your problem.

Share this post


Link to post
Share on other sites

Apologies for the gentle bump of this. Perhaps I could add to this by asking if anyone has successfully used AutoIt to interact with Tk based controls? i.e. should I give up or invest a little more effort?


Problem solving step 1: Write a simple, self-contained, running, replicator of your problem.

Share this post


Link to post
Share on other sites

Can you provide a little more information? I'm fairly sure I have already correctly identified the component. I don't wish to use image based selection or click coordinates (although I accept they may be an answer). Are you suggesting I need to use IUIAutomation?


Problem solving step 1: Write a simple, self-contained, running, replicator of your problem.

Share this post


Link to post
Share on other sites

I tried with simplespy and got the following:

Spoiler

Mouse position is retrieved 2694-939
At least we have an element title: [] class: [TkChild]

Having the following values for all properties: 
Title is: <>    Class   := <TkChild>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    2677;931;28;28
*** Parent Information top down ***
2: Title is: <Untitled - HyperWorks v14.0>    Class   := <TkTopLevel>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    1911;-9;1938;1218
"Title:=Untitled - HyperWorks v14.0;controltype:=UIA_WindowControlTypeId;class:=TkTopLevel""    
1: Title is: <>    Class   := <TkChild>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    1920;52;1920;1148
"Title:=;controltype:=UIA_PaneControlTypeId;class:=TkChild""    
0: Title is: <>    Class   := <TkChild>    controltype:= <UIA_PaneControlTypeId>    ,<50033>    , (0000C371)    2666;929;349;32
"Title:=;controltype:=UIA_PaneControlTypeId;class:=TkChild""    


;~ *** Standard code maintainable ***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

_UIA_setVar("oP1","Title:=Untitled - HyperWorks v14.0;controltype:=UIA_WindowControlTypeId;class:=TkTopLevel")    ;Untitled - HyperWorks v14.0
_UIA_setVar("oP2","Title:=;controltype:=UIA_PaneControlTypeId;class:=TkChild")    ;
_UIA_setVar("oP3","Title:=;controltype:=UIA_PaneControlTypeId;class:=TkChild")    ;

_UIA_setVar(".mainwindow","title:=;classname:=TkChild")

;~ Actions split away from logical/technical definition above can come from configfiles 

;~_UIA_Action("oP1","highlight")
_UIA_Action("oP1","setfocus")
;~_UIA_Action("oP2","highlight")
_UIA_Action("oP2","setfocus")
;~_UIA_Action("oP3","highlight")
_UIA_Action("oP3","setfocus")

_UIA_action(".mainwindow","setfocus")


;~ *** Standard code Flexible***
#include "UIAWrappers.au3"
AutoItSetOption("MustDeclareVars", 1)

Local $oP2=_UIA_getObjectByFindAll($UIA_oDesktop, "Title:=Untitled - HyperWorks v14.0;controltype:=UIA_WindowControlTypeId;class:=TkTopLevel", $treescope_children)    
_UIA_Action($oP2,"setfocus")
Local $oP1=_UIA_getObjectByFindAll($oP2, "Title:=;controltype:=UIA_PaneControlTypeId;class:=TkChild", $treescope_children)    
_UIA_Action($oP1,"setfocus")
Local $oP0=_UIA_getObjectByFindAll($oP1, "Title:=;controltype:=UIA_PaneControlTypeId;class:=TkChild", $treescope_children)    
_UIA_Action($oP0,"setfocus")
_UIA_setVar(".mainwindow","title:=;classname:=TkChild")
_UIA_action(".mainwindow","setfocus")


*** Detailed properties of the highlighted element ***
UIA_class:= <TkChild>
UIA_regexpclass:= <TkChild>
UIA_iaccessiblechildId:= <0>
UIA_handle:= <6227794>
UIA_RuntimeId:= <42;6227794>
UIA_BoundingRectangle:= <2677;931;28;28>
UIA_ProcessId:= <13028>
UIA_ControlType:= <50033>
UIA_LocalizedControlType:= <pane>
UIA_HasKeyboardFocus:= <False>
UIA_IsKeyboardFocusable:= <True>
UIA_IsEnabled:= <True>
UIA_ClassName:= <TkChild>
UIA_Culture:= <0>
UIA_IsControlElement:= <True>
UIA_IsContentElement:= <True>
UIA_IsPassword:= <False>
UIA_NativeWindowHandle:= <6227794>
UIA_IsOffscreen:= <False>
UIA_Orientation:= <0>
UIA_FrameworkId:= <Win32>
UIA_IsRequiredForForm:= <False>
UIA_IsDockPatternAvailable:= <False>
UIA_IsExpandCollapsePatternAvailable:= <False>
UIA_IsGridItemPatternAvailable:= <False>
UIA_IsGridPatternAvailable:= <False>
UIA_IsInvokePatternAvailable:= <False>
UIA_IsMultipleViewPatternAvailable:= <False>
UIA_IsRangeValuePatternAvailable:= <False>
UIA_IsScrollPatternAvailable:= <False>
UIA_IsScrollItemPatternAvailable:= <False>
UIA_IsSelectionItemPatternAvailable:= <False>
UIA_IsSelectionPatternAvailable:= <False>
UIA_IsTablePatternAvailable:= <False>
UIA_IsTableItemPatternAvailable:= <False>
UIA_IsTextPatternAvailable:= <False>
UIA_IsTogglePatternAvailable:= <False>
UIA_IsTransformPatternAvailable:= <False>
UIA_IsValuePatternAvailable:= <False>
UIA_IsWindowPatternAvailable:= <False>
UIA_ValueIsReadOnly:= <True>
UIA_RangeValueValue:= <0>
UIA_RangeValueIsReadOnly:= <True>
UIA_RangeValueMinimum:= <0>
UIA_RangeValueMaximum:= <0>
UIA_RangeValueLargeChange:= <0>
UIA_RangeValueSmallChange:= <0>
UIA_ScrollHorizontalScrollPercent:= <0>
UIA_ScrollHorizontalViewSize:= <100>
UIA_ScrollVerticalScrollPercent:= <0>
UIA_ScrollVerticalViewSize:= <100>
UIA_ScrollHorizontallyScrollable:= <False>
UIA_ScrollVerticallyScrollable:= <False>
UIA_SelectionCanSelectMultiple:= <False>
UIA_SelectionIsSelectionRequired:= <False>
UIA_GridRowCount:= <0>
UIA_GridColumnCount:= <0>
UIA_GridItemRow:= <0>
UIA_GridItemColumn:= <0>
UIA_GridItemRowSpan:= <1>
UIA_GridItemColumnSpan:= <1>
UIA_DockDockPosition:= <5>
UIA_ExpandCollapseExpandCollapseState:= <3>
UIA_MultipleViewCurrentView:= <0>
UIA_WindowCanMaximize:= <False>
UIA_WindowCanMinimize:= <False>
UIA_WindowWindowVisualState:= <0>
UIA_WindowWindowInteractionState:= <0>
UIA_WindowIsModal:= <False>
UIA_WindowIsTopmost:= <False>
UIA_SelectionItemIsSelected:= <False>
UIA_TableRowOrColumnMajor:= <2>
UIA_ToggleToggleState:= <2>
UIA_TransformCanMove:= <False>
UIA_TransformCanResize:= <False>
UIA_TransformCanRotate:= <False>
UIA_IsLegacyIAccessiblePatternAvailable:= <True>
UIA_LegacyIAccessibleChildId:= <0>
UIA_LegacyIAccessibleRole:= <10>
UIA_LegacyIAccessibleState:= <1048576>
UIA_IsDataValidForForm:= <False>
UIA_ProviderDescription:= <[pid:9936,hwnd:0x5F0752 Main:Nested [pid:13028,hwnd:0x5F0752 Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]>
UIA_IsItemContainerPatternAvailable:= <False>
UIA_IsVirtualizedItemPatternAvailable:= <False>
UIA_IsSynchronizedInputPatternAvailable:= <False>

Which did not seem to work (notice how the instance of the TkChild is not identified). I tried to run the 'Standard code maintainable' but it failed to parse the line:

_UIA_setVar("oP1","Title:=Untitled - HyperWorks v14.0;controltype:=UIA_WindowControlTypeId;class:=TkTopLevel")

SciTE gives: error: Statement cannot be just an expression. (illegal character)

I did not pursue this further.

A quick search showed me some other unresolved posts regarding working with TkChild widgets... perhaps it is not going to work with AutoIt. After my initial posts I noticed a few things:

1) I was wrong, the button click animation is subtly different to the mouseover animation. ControlClick() is causing a mouse over event.

2) If I MouseClick() at the location (returned by Au3Info_x64.exe) it does work, click animation played and menu updates.

3) If I use ControlGetPos(), it returns some values (correct size), but not correct position (the error is not a scalar of control size), so I can't use ControlGetPos() to drive MouseClick(), which I'm sort of glad about as it seems so ugly. 

Any thoughts?


Problem solving step 1: Write a simple, self-contained, running, replicator of your problem.

Share this post


Link to post
Share on other sites

Is the proper rectangle highlighted?

If not try inspect.exe or ranorex spy,

If still issues try trial version of testcomplete from smartbear.

Try orca screenreader

Read http://ocemp.sourceforge.net/papidown.html

But seems atk and spi interfaces which are hard to reach when using windows. Tools fot tkwidgets you probably have to write in python

Edited by junkew

Share this post


Link to post
Share on other sites

Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest beginning - communication with GitHub REST API Forum Rules *
Include Dependency Tree (Tool for analyzing script relations)
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST APIErrorLog.au3 UDF - A logging Library *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 UDF * ADO.au3 UDF SMTP Mailer UDF * Dual Monitor resolution detection * * 2GUI on Dual Monitor System * _SciLexer.au3 UDF * SciTE - Lexer for console pane

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Good coding practices in AutoIt * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * IE in TaskScheduler

PDF Related:https://www.autoitscript.com/forum/topic/177368-how-to-get-reference-to-pdf-object-embeded-in-ie/ *

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2020-09-20

Share this post


Link to post
Share on other sites

@mLipok I never had any success with it, I did manage to highlight the correct widget, but did not have any success activating it from AutoIt: I think I got as far as being able to trip the widget mouseover event, but that was it. 

In the end it wasn't worth it for the time I was spending on it. I didn't reply because of the final line of "But seems atk and spi interfaces which are hard to reach when using windows. Tools fot tkwidgets you probably have to write in python", pretty much summed up where I got to.


Problem solving step 1: Write a simple, self-contained, running, replicator of your problem.

Share this post


Link to post
Share on other sites

Sorry, this was from 3 years ago, I've long moved on from that.


Problem solving step 1: Write a simple, self-contained, running, replicator of your problem.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...