Jump to content
SorryButImaNewbie

ControlClick on Developed software for autotesting

Recommended Posts

SorryButImaNewbie

Hello,

I'm developing again :) (everybody ruuun! )

I would like to develope a script that goes through basicly every user control on a window, and log things that happens, and maybe do some screenshots. I did something like this before.

My problem, which I would like to avoid this time (to improve my understanding and skill) , was that when I was unable to get a ControlID or handler or anything, I simply did some math and clicked on the coordinates it should have been (for example, maxing the window, and knowing the initial set up I was ablo to calculate given control position). I know that this is a bad solution for a number of reasons. 

Now I got authorization to install autoIT here, and i started to the work, AU3Info was unable to find anything on the window (this could be a problem, since autoIT doesn't see anything on it then, if I understood the help file) So I got the SimpleSpy script (source:

)

I added a bit of code to the original to display ID as well, what I received is this:

Mouse position is retrieved 115-207
At least we have an element title: [ADD] class: [Button] ID: [50000] (<-- coded this to display ID here as well)

Having the following values for all properties: 
Title is: <ADD>    Class   := <Button>    controltype:= <UIA_ButtonControlTypeId>    ,<50000>    , (0000C350)    10;187;120;35
*** Parent Information top down ***
3: Title is: <Compass>    Class   := <Window>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    -8;-8;1936;1056
"Title:=Compass;controltype:=UIA_WindowControlTypeId;class:=Window""    
2: Title is: <>    Class   := <MainView>    controltype:= <UIA_CustomControlTypeId>    ,<50025>    , (0000C369)    0;23;1920;1017
"Title:=;controltype:=UIA_CustomControlTypeId;class:=MainView""    
1: Title is: <>    Class   := <TileNavigationView>    controltype:= <UIA_CustomControlTypeId>    ,<50025>    , (0000C369)    0;23;1920;967
"Title:=;controltype:=UIA_CustomControlTypeId;class:=TileNavigationView""    
0: Title is: <>    Class   := <AreasView>    controltype:= <UIA_CustomControlTypeId>    ,<50025>    , (0000C369)    0;132;1920;858
"Title:=;controltype:=UIA_CustomControlTypeId;class:=AreasView""

 

so far I wrote this script:

WinActivate('Test') ;It works!! :D first official interaction
Sleep(1000) ;1 sec sleep to be sure
ControlClick('Test', '', '50000')
If @error Then
    MsgBox($MB_SYSTEMMODAL, 'Error', 'ControlClick error')
EndIf
Sleep(1000)
MsgBox(1,"Tracer message", 'ControlClick has happened')
;MouseClick()
;ControlCommand()

AutoIt activates the window, but the click on the given button doesnt happen (I tried to write 50000 without ' ' on ID).

M'I doing the @error part correctly ? (no error Msg has been displayed), sorry I rarely use AutoIT and seems to forget less and less after each neglect, but still I'm far from a proffessional :)

 

Any help or suggestion is welcome, thank you for your time and insight!

Edited by SorryButImaNewbie

Share this post


Link to post
Share on other sites
SorryButImaNewbie

Hello, This could be a problem with the installation since MouseMove(60, 70, 0) or MouseMove(60, 70) doesn't do anything.

Should I just start with reinstalling AuotIT or SciTe?

Edit:

Could it be that since the development is going on, using MVVM method, the click is not registered somehow on the View (it turned out I didn't gave Sleep() and the script just run to fast to do the mousemovment i guess), the control I target changes color, but nothing happens. (a menu should be displayed). 

Edited by SorryButImaNewbie

Share this post


Link to post
Share on other sites
SorryButImaNewbie

I was able to move the coursor. I still have a problem with clicking. How can I test if the click has been sent?

I share the code Im working on:

;more includes then i use...
#include <EditConstants.au3>
#include <AutoItConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <MsgBoxConstants.au3>
#include <WinAPI.au3>
#include <Misc.au3>

AutoItSetOption("MustDeclareVars", 1)
Opt("TrayIconDebug", 1) ; Opt is equal to AutoItSetOption

Global $TestHandle

;[50006] ID of the picture from SimpleSpy
CompassHandelSearch()
;Try WinList to get compass win handel?

;WinSetState('Compass', '', @SW_MAXIMIZE) ; To Restore it if its minimized

;WinActivate('Compass')
;WinWaitActive('Compass') ;wait till its active Window
;WinSetOnTop('Compass', "", $WINDOWS_ONTOP)
WinActivate($TestHandle)
MsgBox($MB_SYSTEMMODAL, "Test Interaction Test", "WinActivate done", 2)
Sleep(1500) ;If there is no Sleep, mouseMove doesn't work
MouseMove(15, 40, 0)
Sleep(1500)
MouseClick($MOUSE_CLICK_LEFT) ;trying doubleclick, not working, Controls changes color, but menu that should apper doesn't appear
MouseClick($MOUSE_CLICK_LEFT)
Sleep(1500)
;MouseDown($MOUSE_CLICK_LEFT)
;Sleep(500)
;MouseUp($MOUSE_CLICK_LEFT)
;Sleep(1500)

;SendKeepActive -- Same as Send, attempts to Keep Window active after each SendkeepActive
MsgBox($MB_SYSTEMMODAL, "Tracer message", "MouseMove and MouseClickLeft done", 2)
MouseClick("left",60,70,1,10) ;On monitoring, doesn't work, mouse stay at original position
Sleep(1000)
MsgBox(1,"Tracer message", 'MouseClick has happened',2)
Sleep(1000)
ControlClick('Test', '', '[50006]')
Sleep(1000)
ControlClick($TestHandle, '', '[50006]')
Sleep(1000)
MsgBox(1,"Tracer message", 'ControlClick has happened',2)

If @error Then
    MsgBox($MB_SYSTEMMODAL, 'Error', 'ControlClick error')
EndIf

Sleep(1000)

;Functions-------------------------------------------------

Func CompassHandelSearch()
    ; Retrieve a list of active-window handles.
    Local $aList = WinList()
    ; Loop through the array until find one titeld Test, Save handel.
    For $i = 1 To $aList[0][0]
        If $aList[$i][0] <> "" And BitAND(WinGetState($aList[$i][1]), 2) Then
            ;MsgBox($MB_SYSTEMMODAL, "", "Title: " & $aList[$i][0] & @CRLF & "Handle: " & $aList[$i][1])
            If $aList[$i][0] = 'Test' Then
                $TestHandle = $aList[$i][1]
                MsgBox($MB_SYSTEMMODAL, "", "Title: " & $aList[$i][0] & @CRLF & "Handle: " & $aList[$i][1],2)
            EndIf
        EndIf
    Next
EndFunc

thanks for your help

Share this post


Link to post
Share on other sites
LarsJ

The control IDs you have found through SimpleSpy do not have anything to do with ordinary Windows control IDs. The numbers 50000, 50001, ... are global constants defined in CUIAutomation2.au3:

;module UIA_ControlTypeIds
Global Const $UIA_ButtonControlTypeId=50000
Global Const $UIA_CalendarControlTypeId=50001
Global Const $UIA_CheckBoxControlTypeId=50002
Global Const $UIA_ComboBoxControlTypeId=50003
Global Const $UIA_EditControlTypeId=50004
Global Const $UIA_HyperlinkControlTypeId=50005
Global Const $UIA_ImageControlTypeId=50006
Global Const $UIA_ListItemControlTypeId=50007
Global Const $UIA_ListControlTypeId=50008
Global Const $UIA_MenuControlTypeId=50009
Global Const $UIA_MenuBarControlTypeId=50010
Global Const $UIA_MenuItemControlTypeId=50011
...

If the AutoIt Window Info tool can't identify your controls, the best option is to try with the UI Automation framework.

Note that if you identify the controls with UI Automation code, you also have to automate the controls with UI Automation code. You cannot use classic automation code.

Take a look at this post.

  • Like 1

Share this post


Link to post
Share on other sites
junkew

If you are a starter first automate calc.exe to get used to AutoIt

Then automate www.google.com first with IE and then with chrome

Then you understand the concepts and are ready for your own application. See more links in frequently asked questions nr 31 or directives in uiautomation thread in examples section.

And please let know what is highlighted area by any of the spy tools

Edited by junkew

Share this post


Link to post
Share on other sites
jpm

Hi,

 be sure the AutoIt script run with the same Administrative right you will need to use #requireAdmin if the target program run in admin mode

  • Like 1

Share this post


Link to post
Share on other sites
SorryButImaNewbie

Thank you all Larsj, junkew, jpm!

I will start to look into UI Automation framework, I don't recall that i used it before.

 

This isn't really my first automatization project, I did the calculator for my own joy, and a few years ago another program automatization like this, now a days I have a few excel autoformating scripts that also get info from exchange rates given the dates from the line, and calculating a few stuffs and order them etc. I'm not a fresh starte (I wouldn't say I'm a good one neither :) )

Highlighted areas are usually good around the buttons, but it see the icons and textboxes on the buttons as stand alone components, I'm not sure how it should work, or how it is written, and I can't upload pictures, sorry :(

 

jpm, thanks, the moment I read your comment I knew this will be the problem, it seems like its working (I only tried to use it once, it worked and I wanted to get back to you guys as fast as possible)

 

Share this post


Link to post
Share on other sites
SorryButImaNewbie

I'm sorry, but I downloaded SDK 10, and run into a "little problem" with inspect.exe

After I was unable to find it manually, (was looking around in the wrong folder, Microsoft SDK I think) I used the good old, search the entire C drive for "ApplicationName" method. It was somewhat successful, I found 4 Inspect.exe from which 2 of them gave me an error message, and 2 others are working. I guess it has something to do about my windows 10 (x86) version.

My questions: why are there 4 of them? What do I need to keep to keep one of the working inspects running? Do I need to save any files other then the Inspect.exe?

Thank you for your insight 

Edit:

Also this is my first time (be gentel) when I would like to add additional UDFs to autoIT, I created the extra folder outside autoit3, and used sciteconfig to connect it to scite, but I cant find the CUIAutomation2.au3 UDF to download, where can I find it?

(Sorry for the basic questions, but maybe someone else will be happy to find the anserws too :) )

Edited by SorryButImaNewbie

Share this post


Link to post
Share on other sites
LarsJ

Four versions of Inspect.exe: A 32 and a 64 bit version for a standard CPU and an ARM CPU.

You need the two versions for a standard CPU in bin\x86 (32 bit) and bin\x64 (64 bit). They are standalone programs and are not depending on anything else. You can rename the files to Inspect_x86.exe and Inspect_x64.exe and copy the files to a folder of your own. If your application is running 64 bit you should use Inspect_x64.exe. If your application is running 32 bit you should use Inspect_x86.exe.

You can find CUIAutomation2.au3 in UIA_V0_63.zip in bottem of first post in the UI Automation framework.

Regards Lars.

  • Like 1

Share this post


Link to post
Share on other sites
SorryButImaNewbie

Thanks Lars!

I do what you told me regarding the Inspects.exe-s,  CUIAutomation2.au3 was already on my computer, I just didn't know... I placed it to the UDF folder

thank you directing me to find it (upsy)

Edit:

I have a little problem, I cant seem to add the UDF to the Scite CallTip manager (see picture), I choose the folder and the UDF, I think it should have the Header mode, but if I click Parse I get a Msg that reads: Parsing Complete (title), Please select another UDF or Exit(text) and an OK button. M'i done? :)  I thought (according to the help file) that I have to click Add and thats bothers me, giving me a feeling that something isn't as it should be.

pic:

 

ExtraUDF.PNG

Edited by SorryButImaNewbie

Share this post


Link to post
Share on other sites
LarsJ

Unfortunately, I cannot help with that. I'm using Notepad++ and not SciTE, so I don't know anything about SciTE. Maybe one of those who know something will help.

Share this post


Link to post
Share on other sites
AdamUL

CUIAutomation2.au3 does not have any functions defined in it, that is why call tips are not created.  User CallTips Manager only works with functions.

  

Adam

 

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • MaximusCZ
      By MaximusCZ
      Hello, and apologies if this have been discussed before, but I googled and couldnt find anything relevant.
      I have a problem where I try to send a control clink into to a window. 
      Part of window I try to send clicks to looks like this . The black border is showing when I mouse over it Au3Info, I get following information:

      I am testing its interaction with code

      The result I get is that it makes control clicks trough first line, opens and closes menus, as expected.
      When It gets to second line, it does not click a single icon. I can tell that coords are correct as the dropdown menus (No view active) gets focused(altough not rolled out). Same behaviour happens if roughly 40<y<60, so I know I am hitting good height. You can look at how it behaves here: output.webm 
      Dropdown 1 focuses at around 8 sec mark, and 13 sec mark it changes to second dropdown, defocusing first one, seemingly not registering any clinks inbetween.
       
      Have you ever encountered similar behaviour? I need to click the Key button, first on second line, with controlclick (or any other way not hijacking the mouse). I cant trigger that button by keyboard and There is no other control having control over it.
      Help
       
    • Skysnake
      By Skysnake
      I need to automate a specific GUI from a legacy system written in C.
      The spy tool gives me this

      I use this code
      Local $swintit = WinGetTitle("[active]") ConsoleWrite("active Window " & $swintit & @CRLF) If $swintit = "FILES NOTES" Then ; If $swintit = "FILES NOTES" Then ; got Window, now atomate ConsoleWrite("got Window " & @CRLF) ; now focus, then click :) ;EditPaste ;ControlCommand("FILES NOTES", "", 404, $acno) ; paste acno GUICtrlSetState(404, $GUI_Focus) ControlCommand("FILES NOTES", "", 404, $acno) ; paste acno GUICtrlSetState(401, $GUI_Focus) ControlClick("FILES NOTES", "", 401, $acno) ; click okay EndIf ; If $swintit = "FILES NOTES" Then The idea is to paste the acno value into the edit box, then click an Okay button.
      I get a console write that the active Window is as expected, but my loop completes without ever writing anything into the GUI.
      I also tried the control ID as text "404" - same result
      What am I missing?
      Skysnake
       
    • JNutt
      By JNutt
       

       
       
      I'm trying to use mouseclick in a panel above.  I want to click on a list item, such as 'Estimated'.  (I've given up on trying to use control click for this).  I'm using window info coordinates from 'Control' tab.

      ; click 'Columns' WinActivate("Innovaya Studio with Sage (Archtectural 2017_V1.invx") MouseClick("primary", 292, 88, 1, 1) WinActivate("Select Browsing Properties") Sleep(1000) MouseClick("primary", 27, 70, 0,0) Sleep(2000) Send("{TAB}{TAB}{TAB}{TAB}{TAB}") Sleep(1000) Send("{ENTER}") ControlClick( But when I used the code Autoit doesn't click where the coordinates are.  In the scipts it is the second MouseClick that won't work.  Any ideas?
    • Rammanan
      By Rammanan
      Hi all, Below is my script,
      Pls advise me if any wrong on that....File exit in location but the return -1.
      FileOpen($file, 0)
         $line = FileReadLine($file)
         MsgBox(0,'',$line)
      FileClose($file)
      Local $aArray = _FileListToArrayRec($serverpath, "*|*.txt", $FLTAR_FILES, $FLTAR_RECUR, $FLTAR_SORT, $FLTAR_NOPATH)
      If @error Then
              MsgBox($MB_SYSTEMMODAL, "Ooops!", "No files found")
          Else
          ;_ArrayDisplay($aArray, "Sorted tree")
                 Local $hSearch = FileFindFirstFile($line)
               MsgBox(0,'',$hSearch)
       
    • JNutt
      By JNutt
      I'm trying ot use controlClick to click on button. So I'm using the Window Info Finder tool. But I noticed that the control ID in windowInfo changes each time I restart the app.  
      Am I doing this wrong.
×