Jump to content
SorryButImaNewbie

ControlClick on Developed software for autotesting

Recommended Posts

SorryButImaNewbie

Hello,

I'm developing again :) (everybody ruuun! )

I would like to develope a script that goes through basicly every user control on a window, and log things that happens, and maybe do some screenshots. I did something like this before.

My problem, which I would like to avoid this time (to improve my understanding and skill) , was that when I was unable to get a ControlID or handler or anything, I simply did some math and clicked on the coordinates it should have been (for example, maxing the window, and knowing the initial set up I was ablo to calculate given control position). I know that this is a bad solution for a number of reasons. 

Now I got authorization to install autoIT here, and i started to the work, AU3Info was unable to find anything on the window (this could be a problem, since autoIT doesn't see anything on it then, if I understood the help file) So I got the SimpleSpy script (source:

)

I added a bit of code to the original to display ID as well, what I received is this:

Mouse position is retrieved 115-207
At least we have an element title: [ADD] class: [Button] ID: [50000] (<-- coded this to display ID here as well)

Having the following values for all properties: 
Title is: <ADD>    Class   := <Button>    controltype:= <UIA_ButtonControlTypeId>    ,<50000>    , (0000C350)    10;187;120;35
*** Parent Information top down ***
3: Title is: <Compass>    Class   := <Window>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    -8;-8;1936;1056
"Title:=Compass;controltype:=UIA_WindowControlTypeId;class:=Window""    
2: Title is: <>    Class   := <MainView>    controltype:= <UIA_CustomControlTypeId>    ,<50025>    , (0000C369)    0;23;1920;1017
"Title:=;controltype:=UIA_CustomControlTypeId;class:=MainView""    
1: Title is: <>    Class   := <TileNavigationView>    controltype:= <UIA_CustomControlTypeId>    ,<50025>    , (0000C369)    0;23;1920;967
"Title:=;controltype:=UIA_CustomControlTypeId;class:=TileNavigationView""    
0: Title is: <>    Class   := <AreasView>    controltype:= <UIA_CustomControlTypeId>    ,<50025>    , (0000C369)    0;132;1920;858
"Title:=;controltype:=UIA_CustomControlTypeId;class:=AreasView""

 

so far I wrote this script:

WinActivate('Test') ;It works!! :D first official interaction
Sleep(1000) ;1 sec sleep to be sure
ControlClick('Test', '', '50000')
If @error Then
    MsgBox($MB_SYSTEMMODAL, 'Error', 'ControlClick error')
EndIf
Sleep(1000)
MsgBox(1,"Tracer message", 'ControlClick has happened')
;MouseClick()
;ControlCommand()

AutoIt activates the window, but the click on the given button doesnt happen (I tried to write 50000 without ' ' on ID).

M'I doing the @error part correctly ? (no error Msg has been displayed), sorry I rarely use AutoIT and seems to forget less and less after each neglect, but still I'm far from a proffessional :)

 

Any help or suggestion is welcome, thank you for your time and insight!

Edited by SorryButImaNewbie

Share this post


Link to post
Share on other sites
SorryButImaNewbie

Hello, This could be a problem with the installation since MouseMove(60, 70, 0) or MouseMove(60, 70) doesn't do anything.

Should I just start with reinstalling AuotIT or SciTe?

Edit:

Could it be that since the development is going on, using MVVM method, the click is not registered somehow on the View (it turned out I didn't gave Sleep() and the script just run to fast to do the mousemovment i guess), the control I target changes color, but nothing happens. (a menu should be displayed). 

Edited by SorryButImaNewbie

Share this post


Link to post
Share on other sites
SorryButImaNewbie

I was able to move the coursor. I still have a problem with clicking. How can I test if the click has been sent?

I share the code Im working on:

;more includes then i use...
#include <EditConstants.au3>
#include <AutoItConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <MsgBoxConstants.au3>
#include <WinAPI.au3>
#include <Misc.au3>

AutoItSetOption("MustDeclareVars", 1)
Opt("TrayIconDebug", 1) ; Opt is equal to AutoItSetOption

Global $TestHandle

;[50006] ID of the picture from SimpleSpy
CompassHandelSearch()
;Try WinList to get compass win handel?

;WinSetState('Compass', '', @SW_MAXIMIZE) ; To Restore it if its minimized

;WinActivate('Compass')
;WinWaitActive('Compass') ;wait till its active Window
;WinSetOnTop('Compass', "", $WINDOWS_ONTOP)
WinActivate($TestHandle)
MsgBox($MB_SYSTEMMODAL, "Test Interaction Test", "WinActivate done", 2)
Sleep(1500) ;If there is no Sleep, mouseMove doesn't work
MouseMove(15, 40, 0)
Sleep(1500)
MouseClick($MOUSE_CLICK_LEFT) ;trying doubleclick, not working, Controls changes color, but menu that should apper doesn't appear
MouseClick($MOUSE_CLICK_LEFT)
Sleep(1500)
;MouseDown($MOUSE_CLICK_LEFT)
;Sleep(500)
;MouseUp($MOUSE_CLICK_LEFT)
;Sleep(1500)

;SendKeepActive -- Same as Send, attempts to Keep Window active after each SendkeepActive
MsgBox($MB_SYSTEMMODAL, "Tracer message", "MouseMove and MouseClickLeft done", 2)
MouseClick("left",60,70,1,10) ;On monitoring, doesn't work, mouse stay at original position
Sleep(1000)
MsgBox(1,"Tracer message", 'MouseClick has happened',2)
Sleep(1000)
ControlClick('Test', '', '[50006]')
Sleep(1000)
ControlClick($TestHandle, '', '[50006]')
Sleep(1000)
MsgBox(1,"Tracer message", 'ControlClick has happened',2)

If @error Then
    MsgBox($MB_SYSTEMMODAL, 'Error', 'ControlClick error')
EndIf

Sleep(1000)

;Functions-------------------------------------------------

Func CompassHandelSearch()
    ; Retrieve a list of active-window handles.
    Local $aList = WinList()
    ; Loop through the array until find one titeld Test, Save handel.
    For $i = 1 To $aList[0][0]
        If $aList[$i][0] <> "" And BitAND(WinGetState($aList[$i][1]), 2) Then
            ;MsgBox($MB_SYSTEMMODAL, "", "Title: " & $aList[$i][0] & @CRLF & "Handle: " & $aList[$i][1])
            If $aList[$i][0] = 'Test' Then
                $TestHandle = $aList[$i][1]
                MsgBox($MB_SYSTEMMODAL, "", "Title: " & $aList[$i][0] & @CRLF & "Handle: " & $aList[$i][1],2)
            EndIf
        EndIf
    Next
EndFunc

thanks for your help

Share this post


Link to post
Share on other sites
LarsJ

The control IDs you have found through SimpleSpy do not have anything to do with ordinary Windows control IDs. The numbers 50000, 50001, ... are global constants defined in CUIAutomation2.au3:

;module UIA_ControlTypeIds
Global Const $UIA_ButtonControlTypeId=50000
Global Const $UIA_CalendarControlTypeId=50001
Global Const $UIA_CheckBoxControlTypeId=50002
Global Const $UIA_ComboBoxControlTypeId=50003
Global Const $UIA_EditControlTypeId=50004
Global Const $UIA_HyperlinkControlTypeId=50005
Global Const $UIA_ImageControlTypeId=50006
Global Const $UIA_ListItemControlTypeId=50007
Global Const $UIA_ListControlTypeId=50008
Global Const $UIA_MenuControlTypeId=50009
Global Const $UIA_MenuBarControlTypeId=50010
Global Const $UIA_MenuItemControlTypeId=50011
...

If the AutoIt Window Info tool can't identify your controls, the best option is to try with the UI Automation framework.

Note that if you identify the controls with UI Automation code, you also have to automate the controls with UI Automation code. You cannot use classic automation code.

Take a look at this post.

  • Like 1

Share this post


Link to post
Share on other sites
junkew

If you are a starter first automate calc.exe to get used to AutoIt

Then automate www.google.com first with IE and then with chrome

Then you understand the concepts and are ready for your own application. See more links in frequently asked questions nr 31 or directives in uiautomation thread in examples section.

And please let know what is highlighted area by any of the spy tools

Edited by junkew

Share this post


Link to post
Share on other sites
jpm

Hi,

 be sure the AutoIt script run with the same Administrative right you will need to use #requireAdmin if the target program run in admin mode

  • Like 1

Share this post


Link to post
Share on other sites
SorryButImaNewbie

Thank you all Larsj, junkew, jpm!

I will start to look into UI Automation framework, I don't recall that i used it before.

 

This isn't really my first automatization project, I did the calculator for my own joy, and a few years ago another program automatization like this, now a days I have a few excel autoformating scripts that also get info from exchange rates given the dates from the line, and calculating a few stuffs and order them etc. I'm not a fresh starte (I wouldn't say I'm a good one neither :) )

Highlighted areas are usually good around the buttons, but it see the icons and textboxes on the buttons as stand alone components, I'm not sure how it should work, or how it is written, and I can't upload pictures, sorry :(

 

jpm, thanks, the moment I read your comment I knew this will be the problem, it seems like its working (I only tried to use it once, it worked and I wanted to get back to you guys as fast as possible)

 

Share this post


Link to post
Share on other sites
SorryButImaNewbie

I'm sorry, but I downloaded SDK 10, and run into a "little problem" with inspect.exe

After I was unable to find it manually, (was looking around in the wrong folder, Microsoft SDK I think) I used the good old, search the entire C drive for "ApplicationName" method. It was somewhat successful, I found 4 Inspect.exe from which 2 of them gave me an error message, and 2 others are working. I guess it has something to do about my windows 10 (x86) version.

My questions: why are there 4 of them? What do I need to keep to keep one of the working inspects running? Do I need to save any files other then the Inspect.exe?

Thank you for your insight 

Edit:

Also this is my first time (be gentel) when I would like to add additional UDFs to autoIT, I created the extra folder outside autoit3, and used sciteconfig to connect it to scite, but I cant find the CUIAutomation2.au3 UDF to download, where can I find it?

(Sorry for the basic questions, but maybe someone else will be happy to find the anserws too :) )

Edited by SorryButImaNewbie

Share this post


Link to post
Share on other sites
LarsJ

Four versions of Inspect.exe: A 32 and a 64 bit version for a standard CPU and an ARM CPU.

You need the two versions for a standard CPU in bin\x86 (32 bit) and bin\x64 (64 bit). They are standalone programs and are not depending on anything else. You can rename the files to Inspect_x86.exe and Inspect_x64.exe and copy the files to a folder of your own. If your application is running 64 bit you should use Inspect_x64.exe. If your application is running 32 bit you should use Inspect_x86.exe.

You can find CUIAutomation2.au3 in UIA_V0_63.zip in bottem of first post in the UI Automation framework.

Regards Lars.

  • Like 1

Share this post


Link to post
Share on other sites
SorryButImaNewbie

Thanks Lars!

I do what you told me regarding the Inspects.exe-s,  CUIAutomation2.au3 was already on my computer, I just didn't know... I placed it to the UDF folder

thank you directing me to find it (upsy)

Edit:

I have a little problem, I cant seem to add the UDF to the Scite CallTip manager (see picture), I choose the folder and the UDF, I think it should have the Header mode, but if I click Parse I get a Msg that reads: Parsing Complete (title), Please select another UDF or Exit(text) and an OK button. M'i done? :)  I thought (according to the help file) that I have to click Add and thats bothers me, giving me a feeling that something isn't as it should be.

pic:

 

ExtraUDF.PNG

Edited by SorryButImaNewbie

Share this post


Link to post
Share on other sites
LarsJ

Unfortunately, I cannot help with that. I'm using Notepad++ and not SciTE, so I don't know anything about SciTE. Maybe one of those who know something will help.

Share this post


Link to post
Share on other sites
AdamUL

CUIAutomation2.au3 does not have any functions defined in it, that is why call tips are not created.  User CallTips Manager only works with functions.

  

Adam

 

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Similar Content

    • tentacole
      By tentacole
      Afternoon!
      This is my first post, so I apologize if this is in the wrong place. 
      I've created a while loop to click in a certain area of an application, and have the y axis change at the end of each loop. My loop continues to click at x:27, x:10, even though the $y is adding 15 at the end of each loop. I did a Send($x) and Send($y) into a Notepad to see if the $y had changed after each loop, and the 15 was being added to $y each loop. (If this makes sense)
       
      I'm unsure where I'm going wrong, and would be extremely grateful if someone can point me in the correct direction to fix this. 
      Local $rDirectory = "H:\oDemandProject\fList.txt" Local $rLine = _FileCountLines($rDirectory) Local $x = 27 Local $y = 10 While $rline > 0 WinActivate("OnDemand", "-> 1") WinWaitActive("OnDemand", "-> 1") ControlClick("OnDemand", "", "[CLASS:AfxFrameOrView120u; INSTANCE:1]", "left", 4, $x, $y) $y += 15 $rLine -= 1 WEnd  
    • toto22
      By toto22
      Please help. How to enter text to my C# program textbox?
      window info
      " >>>> Window <<<< Title:    Form1 Class:    WindowsForms10.Window.8.app.0.141b42a_r10_ad1 Position:    1922, 615 Size:    553, 680 Style:    0x16CF0000 ExStyle:    0x00050100 Handle:    0x001E0812 >>>> Control <<<< Class:    WindowsForms10.EDIT.app.0.141b42a_r10_ad1 Instance:    4 ClassnameNN:    WindowsForms10.EDIT.app.0.141b42a_r10_ad14 Name:    tbSymbol Advanced (Class):    [NAME:tbSymbol] ID:    2492658 Text:    TSLA Position:    100, 16 Size:    63, 20 ControlClick Coords:    43, 10 Style:    0x560100C0 ExStyle:    0x00000200 Handle:    0x002608F2 >>>> Mouse <<<< Position:    2073, 672 Cursor ID:    0 Color:    0xFFFFFF >>>> StatusBar <<<< >>>> ToolsBar <<<< >>>> Visible Text <<<< Shars Request SELL BUY 4000 Price Update Money SYMBOL TSLA Disconnect Connect >>>> Hidden Text <<<< " tried everything, 
      $HWnd = WinActivate("Form1")             ControlSend($HWnd,"", "[CLASS:WindowsForms10.EDIT.app.0.141b42a_r10_ad1]", $Symbol)  
    • Yirrlaar
      By Yirrlaar
      #Include <WinAPI.au3> local $ClassName = _WinAPI_GetClassName(WinGetHandle(WinGetTitle("[ACTIVE]"))) ControlClick(WinGetTitle("[ACTIVE]"), "", "[CLASS:"&$ClassName&"; INSTANCE:1]", "primary") When I am running this script it just gives me a ConsoleWrite output of "0x001D04A8"
      When using "AutoIT v3 Window Info" all the info I need for the ControlClick is there and displayed perfectly as if I typed it in normally.
      Upon click the button it should be clicking it, I see the following inn the console window:
      If I replace the $ClassName with the actual ClassName of "WindowsForms10.BUTTON.app.0.3296db7_r14_ad1" it works just fine, any ideas???
    • caramen
      By caramen
      Is that correct?
      With SimpleSpy i can click on controls without any include/udf ON CHROME?
    • autotin
      By autotin
      Hello I am a first time user of AutoIT with little coding experience and I would REALLY really appreciate your help.
      I work in a warehouse where I am responsible for data entry along with a slew of other different tasks such as auditing and managing paperwork. Most of my time is tied up entering info to close receipts of product orders (8-12+ hours for thousands of products). The quantity actually produced by the warehouse is sometimes different from the quantity listed on the reciept (in this case 20). I need to change all the build qty for sublocation "1-WIP" from 20 to the actual quantity 22 and set all other numbers as 0. The form is an iframe coded with jQuery and freezes for 1-2 seconds after every entry and refreshes. As such, it doesn't allow me to use tab and I need to click and type every incorrect field.
      I have attached an image as an example from the webapp, FinaleInventory.
      I'm thinking that I probably need a macro that has a message box entry where I can type the corrected number entry I want. Then it uses image/character recognition (Tesseract?) to find "WIP" sublocation =,   Move the mouse over to the right to "Quantity" relative to location of WIP If quantity number = entry, then move to next "WIP" image (Not quite sure how to make Tesseract "find next") Elseif quantity number ≠ entry then click quantity number of "WIP" and type corrected entry Scroll down Loop until bottom of page I would greatly appreciate any help, ideas or pointers for this project. Thank you!
      -A newbie scripter

×