Sign in to follow this  
Followers 0

wxWindowClass etc, widgets problems

8 posts in this topic

Hello everyone. 
I am trying to automate a program that's written using "widgets". It has controls that don't work well with the AutoIt Window Info app, or with spy. For example, one control is of class wxWindowClassNR28, and its text property is "wxlistctrlcolumntitles". The actual displayed text is empty.
It seems to be drawn inside of a control of class "wxWindowClassNR" with text property "CNOwnerDrawSplitterWindow". There are many layered controls. When I use the _WinAPI_GetWindow command to get the children, I have no idea what child control I have from the text or the class name, and since not all of the children are visible or on top, the best I have been able to do is get their coordinates, do a screen capture at the coordinates, use OCR and hope to have it match something I expect. I need a better way. I have tried Junkew's IUIAutomation framework, but did not get anywhere.
I need new ideas.
Has anyone been successful using Autoit on this type of controls?
In case someone is familiar with this, the program is a printing application called CWS.exe, "command work station".

Share this post

Link to post
Share on other sites

Not sure but it looks like it supports accessibility

wxAccessible *  GetAccessible ()  

Returns the accessible object for this window, if any.

as such it should be possible to get working with IUIAutomation

Share this post

Link to post
Share on other sites

OK, I looked at the pages, and you're right, it seems to support accessibility. But I have no idea where to begin to use wxaccessible with IUIAutomation. Any thoughts?

Share this post

Link to post
Share on other sites

OK, I attached spy It has 2 screenshots and a text file. 

1- cws tabs.jpg shows the app UI, and 2 tabs are circled. I want to be able to select one of those tabs.

2- "cws tabs seen by spy.jpg" shows what is selected when I park the mouse on one of the tabs.

Both tabs are drawn inside the same rectangle, and the rectangle extends farther to the right.

3- spy capture.txt contains all the text that spy demo displayed when it captured all the data about what was selected in "cws tabs seen by spy.jpg"

I work with languages. I'd like to automate capturing strings and comparing them to spreadsheets to make sure the correct translated strings were used in the right place.

So this is what I'd like to do for the beginning:

1- click on the Device Center tab. Expected result: the Device Center tab should be selected, and sub tabs should be displayed.

2- capture the text for each of the sub tabs so I can do my comparisons etc. 



Share this post

Link to post
Share on other sites

In the 2-picture it seems as if you have selected the entire tab control with the Spy tool.

Are you able to select a single tab e.g. Device Center? You should see a red border only around the "Device Center" text. If you are able to do that, will you copy the information from the Spy tool to a code/text box.

Share this post

Link to post
Share on other sites

I can't select anything smaller than the selection you see in the screenshot. It's the same with the autoit tool.

Share this post

Link to post
Share on other sites

I looked on the internet for some wxWidget demo.

I used that to see what the simple spy is recognizing and apparently it detects all wxWidget.

See this link

maybe AxTK is a direction: AxTk could stimulate work and ideas on a better accessibility approach within wxWidgets itself!

I also tried Audacity and in general I can recognize a lot (although not everything) with the simple spy

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  
Followers 0

  • Similar Content

    • Earthshine
      By Earthshine
      This is pretty useful to me at least, I hope to others. Whenever you need to poke a button or such on a GUI, you can just make calls to this UDF and instruct it how long to wait, 0 is forever, anyway, let the code to the explaining. Feel free to help expand this UDF. I plan to create new ones as I go along that support all the Win32 and .NET Controls. Ultimately, I would like to be able to fully control any type of control, this works well with most. Just an example of what you can do.
      The test installs and can uninstall 7-Zip 15.14 (x64) Setup.exe
      NOTE: Logger Author(s) .....: Michael Mims (zorphnog)
      ;                    the logging script can be obtained here
    • FrancescoDiMuro
      By FrancescoDiMuro
      Good morning everyone
      I worked a bit on a little project of mine, and I managed to do what I was trying to do, or, automate a quiz in AutoIt.
      Now, everything works fine but, in order to achieve what I'm trying to achive, I'd like to make this script run faster.
      I'll attach a portion of the code in order to let you understand what my script does.
      $objIE_h1 = _IETagNameGetCollection($objIE, "h1") ; Take the text of the answer If @error Then ConsoleWrite("Errore durante la lettura degli h1 nella pagina. - Errore: " & @error & @CRLF) Else For $objElement In $objIE_h1 $strInnerText = $objElement.innerText Next Select Case StringInStr($strInnerText, "QUESTION") ; Here is the question ; PS4 PRO $objSpans = _IETagNameGetCollection($objIE, "span") ; Answers are stored here For $objElement In $objSpans If $i >= 4 Then $strAnswer = $objElement.innerText ; Take the text of the answer n° 0, 1, 2, 3 If StringInStr($strAnswer, "CORRECT ANSWER") Then ; If I match the correct answer, then, I have to click on the input with answer id = answer_i $j = $i - 4 $objInput = _IEGetObjById($objIE, "answer_" & $j) If @error Then ConsoleWrite("Errore durante la lettura della risposta #" & $i & " - Errore: " & @error & @CRLF) Else _IEAction($objInput, "click") If @error Then ConsoleWrite("Errore durante il click sull'elemento. - Errore: " & @error) Else ConsoleWrite($j & ") Risposta: " & $strAnswer & @CRLF) ExitLoop EndIf EndIf EndIf $i+=1 Else $i+=1 EndIf Next $i = 0 $j = 0 This is what my script does.
      Questions are 20, in which I have 15 multiple choice question, 2 images ( on which I use _IEImgClick() ), and 3 text inputs ( a simple send with the text I have to enter ).
      How can I make all of these, faster?
      If I reach this goal, I've probably done one of my best goals in life, and I'd like to share my happiness with you, giving a huge donation to this fantastic community!
      Thanks for your help
    • Psyllex
      By Psyllex
      Is it possible for AutoIt to execute LALT+LSHIFT+NUMLOCK?  I've attempted the following:
      ;To show the desktop Send("{RWIN down}") Send("{d}") Send("{RWIN up}") WinWait(2) ;just to make sure the desktop is activated. ControlClick("", "",1[,button="left"[, clicks=2 [, x [, y]]]] ) Send("{LALT down}") Send("{CLT down}") Send("{DELETE down}") ;Send("{LSHIFT down}") ;Send("{NUMLOCK}") ;Send("{NUMLOCK}") ;Send ("{LALT up}") ;Send("{LSHIFT up}") This should bring up the MouseKeys dialog...but I'm getting nothing.  Thanks for any help!
    • WalkHappy
      By WalkHappy
      Hello, I'm in the process of recreation-ally expanding my Auto-it experience for some future awesome presentation-Al capabilities. So I have a user interface which loads a text document onto a GUI-object. The variables of this function are 1) The text document is dynamic in font size and face; giving the viewer an unpredictable horizontal/vertical scroll dimension. 2) The main objective is to animate the vertical scroll-bar so that it moves like the end of a movie with accreditation. 3) I'm trying to limit the accessibility in observation by excluding any mouse functions within the animation process. Any help would be greatly appreciated! Been trying to work on this for days now - from getting element properties, to embedding external code, and even exploiting ("$oIE.navigate") to loading several awkwardly formatted text documents in sequential order just to imitate a slightly offensive animation scheme. Anyways... Suggestions and comments welcomed please!!
      Script is from [Function GUICreate] - Doc:
      GUICreate("Embedded Web control Test", 640, 580, (@DesktopWidth - 640) / 2, (@DesktopHeight - 580) / 2, BitOR($WS_OVERLAPPEDWINDOW, $WS_CLIPSIBLINGS, $WS_CLIPCHILDREN)) GUICtrlCreateObj($oIE, 10, 40, 600, 360) $idButton_Back = GUICtrlCreateButton("Back", 10, 420, 100, 30) $idButton_Forward = GUICtrlCreateButton("Forward", 120, 420, 100, 30) $idButton_Home = GUICtrlCreateButton("Home", 230, 420, 100, 30) $idButton_Stop = GUICtrlCreateButton("Stop", 330, 420, 100, 30) GUISetState(@SW_SHOW) ;Show GUI After this.. This is the window that pop's-up:

    • amphoric
      By amphoric
      Good Morning,
      I hope someone can help, below is a stripped down version of my current .au3 file and there is a few bits I just cant for the life of me get to work;
      1. I would like the 'Allow Edit' checkbox once checked to allow the user to edit the contents of the 'AD Group Name' input box
      2. I would like the 'Other / Internal Use' radio button if checked to show the '$OtherInternalGroup' group of radio buttons
      My current code.....
      #include <ButtonConstants.au3> #include <EditConstants.au3> #include <GUIConstantsEx.au3> #include <StaticConstants.au3> #include <WindowsConstants.au3> #include <MsgBoxConstants.au3> #include <WinAPIShellEx.au3> #include <FileConstants.au3> #include <WinAPIFiles.au3> #include <WinAPI.au3> ; used for Lo/Hi word ; Declare Global variables for WM_COMMAND and '_GetAppName()' function. Global $__iComboWrite = 9999, $__iInputValue = 9999, $__iInputWrite = 9999 #Region ### START GUI section ### $Form1_1 = GUICreate("Packaging Wrapper", 435, 545, -1, -1) GUISetIcon("C.ico", -1) GUISetBkColor(0xFFFFFF) Global $ApplicationNameInput = GUICtrlCreateInput("", 24, 168, 385, 21) GUICtrlSetTip(-1, "EG: Corel_WinZip_21.0_0.1") $ADGroupNameInput = GUICtrlCreateInput("", 24, 224, 385, 21) $AllowEditChk = GUICtrlCreateCheckbox("Allow Edit", 344, 200, 65, 17, BitOR($GUI_SS_DEFAULT_CHECKBOX,$BS_RIGHT)) GUICtrlSetBkColor(-1, 0xE3E3E3) $ApplicationTypeGroup = GUICtrlCreateGroup("", 24, 272, 185, 113) $CatalogueAppChk = GUICtrlCreateRadio("Catalogue Application", 32, 288, 137, 17) GUICtrlSetBkColor(-1, 0xE3E3E3) $NonCatalogueAppChk = GUICtrlCreateRadio("Non-Catalogue Application", 32, 320, 161, 17) GUICtrlSetBkColor(-1, 0xE3E3E3) $OtherAppChk = GUICtrlCreateRadio("Other / Internal Use", 32, 352, 137, 17) GUICtrlSetBkColor(-1, 0xE3E3E3) GUICtrlCreateGroup("", -99, -99, 1, 1) ; $OtherInternalGroup = GUICtrlCreateGroup("", 216, 272, 193, 113) $OSCoreApplicationChk = GUICtrlCreateRadio("OS Core Application", 224, 288, 129, 17) GUICtrlSetState(-1, $GUI_HIDE) GUICtrlSetBkColor(-1, 0xE3E3E3) $InternalOnlyChk = GUICtrlCreateRadio("Internal ONLY Application", 224, 320, 161, 17) GUICtrlSetState(-1, $GUI_HIDE) GUICtrlSetBkColor(-1, 0xE3E3E3) GUICtrlCreateGroup("", -99, -99, 1, 1) $OtherInternalLabel = GUICtrlCreateLabel("Other / Internal Use", 216, 256, 118, 17, BitOR($SS_CENTER, $SS_CENTERIMAGE)) GUICtrlSetState(-1, $GUI_HIDE) GUICtrlSetFont(-1, 8, 800, 0, "MS Sans Serif") GUICtrlSetColor(-1, 0x000000) GUICtrlSetBkColor(-1, 0xE3E3E3) $ConfirmGroup = GUICtrlCreateGroup("", 24, 392, 385, 89) $Confirm = GUICtrlCreateCheckbox("I confirm all details are correct, and I wish to proceed", 48, 456, 329, 17, BitOR($GUI_SS_DEFAULT_CHECKBOX, $BS_CENTER)) GUICtrlSetFont(-1, 8, 800, 0, "MS Sans Serif") GUICtrlSetBkColor(-1, 0xE3E3E3) $ConfirmDesc = GUICtrlCreateEdit("", 32, 408, 361, 41, 0, 0) GUICtrlSetData(-1, StringFormat("Please confirm that all entered information is correct, and the you wish to \r\nproceed with the creation of the " & Chr(39) & "Application Toolkit Wrapper" & Chr(39) & ".")) GUICtrlSetBkColor(-1, 0xE3E3E3) GUICtrlCreateGroup("", -99, -99, 1, 1) $ProceedBtn = GUICtrlCreateButton("Proceed", 128, 488, 75, 25) GUICtrlSetFont(-1, 8, 800, 0, "MS Sans Serif") GUICtrlSetColor(-1, 0x008000) $CancelBtn = GUICtrlCreateButton("Cancel", 240, 488, 75, 25) GUICtrlSetFont(-1, 8, 800, 0, "MS Sans Serif") GUICtrlSetColor(-1, 0xFF0000) $CapitaLogo = GUICtrlCreatePic("Logo.jpg", 24, 8, 159, 33) $PackagingWrapperLabel = GUICtrlCreateLabel("Packaging Wrapper", 224, 8, 187, 28) GUICtrlSetFont(-1, 14, 800, 0, "MS Sans Serif") GUICtrlSetColor(-1, 0x000000) $CreatedByLabel = GUICtrlCreateLabel("...", 16, 528, 405, 15, $SS_CENTER) GUICtrlSetFont(-1, 7, 400, 0, "MS Serif") $VerLabel = GUICtrlCreateLabel("Ver. 1.0 ALPHA", 328, 40, 79, 17) $Step1Label = GUICtrlCreateLabel("Please Enter The Following Information", 24, 80, 274, 20, BitOR($SS_CENTER, $SS_CENTERIMAGE)) GUICtrlSetFont(-1, 10, 800, 0, "MS Sans Serif") GUICtrlSetColor(-1, 0x000080) GUICtrlSetBkColor(-1, 0xE3E3E3) $Graphic3 = GUICtrlCreateGraphic(16, 72, 404, 452, BitOR($GUI_SS_DEFAULT_GRAPHIC, $SS_SIMPLE)) GUICtrlSetColor(-1, 0xE3E3E3) GUICtrlSetBkColor(-1, 0xE3E3E3) $ApplicationNameDesc = GUICtrlCreateEdit("", 24, 128, 393, 33, 0, 0) GUICtrlSetData(-1, StringFormat("Must be in the following format: Manufacturer_AppName_AppVersion_Rev\r\nEG: Corel_WinZip_21.0_01")) GUICtrlSetBkColor(-1, 0xE3E3E3) $ADGroupNameLabel = GUICtrlCreateLabel("AD Group Name", 24, 200, 95, 17, BitOR($SS_CENTER, $SS_CENTERIMAGE)) GUICtrlSetFont(-1, 8, 800, 0, "MS Sans Serif") GUICtrlSetColor(-1, 0x000000) GUICtrlSetBkColor(-1, 0xE3E3E3) $ApplicationNameLabel = GUICtrlCreateLabel("Application Name", 24, 112, 103, 17, BitOR($SS_CENTER, $SS_CENTERIMAGE)) GUICtrlSetFont(-1, 8, 800, 0, "MS Sans Serif") GUICtrlSetColor(-1, 0x000000) GUICtrlSetBkColor(-1, 0xE3E3E3) $ApplicationTypeLabel = GUICtrlCreateLabel("Application Type", 24, 256, 99, 17, BitOR($SS_CENTER, $SS_CENTERIMAGE)) GUICtrlSetFont(-1, 8, 800, 0, "MS Sans Serif") GUICtrlSetColor(-1, 0x000000) GUICtrlSetBkColor(-1, 0xE3E3E3) GUISetState(@SW_SHOW) #EndRegion ### END GUI section ### ;GLOBAL VARIABLES Global $ApplicationName, $adstring1 Global $ADGroupName = $adstring1 Global $envSystemDrive = EnvGet("systemdrive") Global $PackagingDirectory = "Packaging\???_Apps" Global $FullDir = $envSystemDrive & "\" & $PackagingDirectory & "\" & $ApplicationName Global $currentDate = @MDAY &"-"& @MON &"-"& @YEAR Global $currentTime = @HOUR &":"& @MIN &":"& @SEC ; Register the WM_COMMAND to intercept when the value in the first input's value is changed. GUIRegisterMsg($WM_COMMAND, 'WM_COMMAND') While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE Exit Case $CancelBtn Exit Case $ProceedBtn ;Check Application Name Validity $ApplicationNameInputValid = StringSplit(GUICtrlRead($ApplicationNameInput), "_") If $ApplicationNameInputValid[0]<> 4 Then ConsoleWrite("Application Name - NOT Valid" & @CRLF) $ApplicationNameInputValidYes = 0 MsgBox(0,"Are The Details Correct?", "Application Name NOT Valid.") GUICtrlSetState($ApplicationNameInput,$GUI_FOCUS) Elseif $ApplicationNameInputValid[0]= 4 Then ConsoleWrite("Application Name - Valid" & @CRLF) $ApplicationNameInputValidYes = 1 EndIf ;Is Confirm Checkbox 'Checked' if GUICtrlRead($Confirm) = $GUI_CHECKED Then ConsoleWrite("$Confirm - ticked" & @CRLF) $ConfirmYes = 1 Else $ConfirmYes = 0 ConsoleWrite("$Confirm - NOT ticked" & @CRLF) MsgBox(0,"Are The Details Correct?", "Please complete confirmation.") EndIf ;Is $CatalogueAppChk 'Checked' if GUICtrlRead($CatalogueAppChk) = $GUI_CHECKED Then ConsoleWrite("$CatalogueAppChk - ticked" & @CRLF) $CatalogueAppChkYes = 1 Else ConsoleWrite("$CatalogueAppChk - NOT ticked" & @CRLF) $CatalogueAppChkYes = 0 EndIf ;Is $NonCatalogueAppChk 'Checked' if GUICtrlRead($NonCatalogueAppChk) = $GUI_CHECKED Then ConsoleWrite("$NonCatalogueAppChk - ticked" & @CRLF) $NonCatalogueAppChkYes = 1 Else ConsoleWrite("$NonCatalogueAppChk - NOT ticked" & @CRLF) $NonCatalogueAppChkYes = 0 EndIf ;Is $OtherAppChk 'Checked' if GUICtrlRead($OtherAppChk) = $GUI_CHECKED Then ConsoleWrite("$OtherAppChk - ticked" & @CRLF) $OtherAppChkYes = 1 Else ConsoleWrite("$OtherAppChk - NOT ticked" & @CRLF) $OtherAppChkYes = 0 EndIf ConsoleWrite(@CRLF & "##### COMPLETED #####" & @CRLF) EndSwitch WEnd ;Check for 'Application Name' update Func _GetAppName() $adString = StringReplace(GUICtrlRead($ApplicationNameInput), "_", " ",2) $adstrings = StringSplit($adString, "_") Global $adString1 = $adstrings[1] Global $ApplicationName = GUICtrlRead($ApplicationNameInput) GUICtrlSetData($ADGroupNameInput, $adstring1) EndFunc ;WM_COMMAND Function used for Auto Input of fields Func WM_COMMAND($hWnd, $iMsg, $wParam, $lParam) #forceref $hWnd, $iMsg, $lParam Local $iHiWord = _WinAPI_HiWord($wParam) ; Control identification code. Local $iLoWord = _WinAPI_LoWord($wParam) ; Control identifier. Switch $iHiWord Case $EN_CHANGE ; When a change event is intercepted. Switch $iLoWord ; Retrieve the controlid. Case _GetAppName() ; If it matches the first input then call the custom function _GetInput. ConsoleWrite("Debug: value was changed." & @CRLF) _GetAppName() EndSwitch EndSwitch Return $GUI_RUNDEFMSG EndFunc ;==>WM_COMMAND I hope all this makes sense.
      Thanks in advance