SorryButImaNewbie

ControlClick on Developed software for autotesting

12 posts in this topic

#1 ·  Posted (edited)

Hello,

I'm developing again :) (everybody ruuun! )

I would like to develope a script that goes through basicly every user control on a window, and log things that happens, and maybe do some screenshots. I did something like this before.

My problem, which I would like to avoid this time (to improve my understanding and skill) , was that when I was unable to get a ControlID or handler or anything, I simply did some math and clicked on the coordinates it should have been (for example, maxing the window, and knowing the initial set up I was ablo to calculate given control position). I know that this is a bad solution for a number of reasons. 

Now I got authorization to install autoIT here, and i started to the work, AU3Info was unable to find anything on the window (this could be a problem, since autoIT doesn't see anything on it then, if I understood the help file) So I got the SimpleSpy script (source:

)

I added a bit of code to the original to display ID as well, what I received is this:

Mouse position is retrieved 115-207
At least we have an element title: [ADD] class: [Button] ID: [50000] (<-- coded this to display ID here as well)

Having the following values for all properties: 
Title is: <ADD>    Class   := <Button>    controltype:= <UIA_ButtonControlTypeId>    ,<50000>    , (0000C350)    10;187;120;35
*** Parent Information top down ***
3: Title is: <Compass>    Class   := <Window>    controltype:= <UIA_WindowControlTypeId>    ,<50032>    , (0000C370)    -8;-8;1936;1056
"Title:=Compass;controltype:=UIA_WindowControlTypeId;class:=Window""    
2: Title is: <>    Class   := <MainView>    controltype:= <UIA_CustomControlTypeId>    ,<50025>    , (0000C369)    0;23;1920;1017
"Title:=;controltype:=UIA_CustomControlTypeId;class:=MainView""    
1: Title is: <>    Class   := <TileNavigationView>    controltype:= <UIA_CustomControlTypeId>    ,<50025>    , (0000C369)    0;23;1920;967
"Title:=;controltype:=UIA_CustomControlTypeId;class:=TileNavigationView""    
0: Title is: <>    Class   := <AreasView>    controltype:= <UIA_CustomControlTypeId>    ,<50025>    , (0000C369)    0;132;1920;858
"Title:=;controltype:=UIA_CustomControlTypeId;class:=AreasView""

 

so far I wrote this script:

WinActivate('Test') ;It works!! :D first official interaction
Sleep(1000) ;1 sec sleep to be sure
ControlClick('Test', '', '50000')
If @error Then
    MsgBox($MB_SYSTEMMODAL, 'Error', 'ControlClick error')
EndIf
Sleep(1000)
MsgBox(1,"Tracer message", 'ControlClick has happened')
;MouseClick()
;ControlCommand()

AutoIt activates the window, but the click on the given button doesnt happen (I tried to write 50000 without ' ' on ID).

M'I doing the @error part correctly ? (no error Msg has been displayed), sorry I rarely use AutoIT and seems to forget less and less after each neglect, but still I'm far from a proffessional :)

 

Any help or suggestion is welcome, thank you for your time and insight!

Edited by SorryButImaNewbie

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

Hello, This could be a problem with the installation since MouseMove(60, 70, 0) or MouseMove(60, 70) doesn't do anything.

Should I just start with reinstalling AuotIT or SciTe?

Edit:

Could it be that since the development is going on, using MVVM method, the click is not registered somehow on the View (it turned out I didn't gave Sleep() and the script just run to fast to do the mousemovment i guess), the control I target changes color, but nothing happens. (a menu should be displayed). 

Edited by SorryButImaNewbie

Share this post


Link to post
Share on other sites

#3 ·  Posted

I was able to move the coursor. I still have a problem with clicking. How can I test if the click has been sent?

I share the code Im working on:

;more includes then i use...
#include <EditConstants.au3>
#include <AutoItConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <MsgBoxConstants.au3>
#include <WinAPI.au3>
#include <Misc.au3>

AutoItSetOption("MustDeclareVars", 1)
Opt("TrayIconDebug", 1) ; Opt is equal to AutoItSetOption

Global $TestHandle

;[50006] ID of the picture from SimpleSpy
CompassHandelSearch()
;Try WinList to get compass win handel?

;WinSetState('Compass', '', @SW_MAXIMIZE) ; To Restore it if its minimized

;WinActivate('Compass')
;WinWaitActive('Compass') ;wait till its active Window
;WinSetOnTop('Compass', "", $WINDOWS_ONTOP)
WinActivate($TestHandle)
MsgBox($MB_SYSTEMMODAL, "Test Interaction Test", "WinActivate done", 2)
Sleep(1500) ;If there is no Sleep, mouseMove doesn't work
MouseMove(15, 40, 0)
Sleep(1500)
MouseClick($MOUSE_CLICK_LEFT) ;trying doubleclick, not working, Controls changes color, but menu that should apper doesn't appear
MouseClick($MOUSE_CLICK_LEFT)
Sleep(1500)
;MouseDown($MOUSE_CLICK_LEFT)
;Sleep(500)
;MouseUp($MOUSE_CLICK_LEFT)
;Sleep(1500)

;SendKeepActive -- Same as Send, attempts to Keep Window active after each SendkeepActive
MsgBox($MB_SYSTEMMODAL, "Tracer message", "MouseMove and MouseClickLeft done", 2)
MouseClick("left",60,70,1,10) ;On monitoring, doesn't work, mouse stay at original position
Sleep(1000)
MsgBox(1,"Tracer message", 'MouseClick has happened',2)
Sleep(1000)
ControlClick('Test', '', '[50006]')
Sleep(1000)
ControlClick($TestHandle, '', '[50006]')
Sleep(1000)
MsgBox(1,"Tracer message", 'ControlClick has happened',2)

If @error Then
    MsgBox($MB_SYSTEMMODAL, 'Error', 'ControlClick error')
EndIf

Sleep(1000)

;Functions-------------------------------------------------

Func CompassHandelSearch()
    ; Retrieve a list of active-window handles.
    Local $aList = WinList()
    ; Loop through the array until find one titeld Test, Save handel.
    For $i = 1 To $aList[0][0]
        If $aList[$i][0] <> "" And BitAND(WinGetState($aList[$i][1]), 2) Then
            ;MsgBox($MB_SYSTEMMODAL, "", "Title: " & $aList[$i][0] & @CRLF & "Handle: " & $aList[$i][1])
            If $aList[$i][0] = 'Test' Then
                $TestHandle = $aList[$i][1]
                MsgBox($MB_SYSTEMMODAL, "", "Title: " & $aList[$i][0] & @CRLF & "Handle: " & $aList[$i][1],2)
            EndIf
        EndIf
    Next
EndFunc

thanks for your help

Share this post


Link to post
Share on other sites

#4 ·  Posted

The control IDs you have found through SimpleSpy do not have anything to do with ordinary Windows control IDs. The numbers 50000, 50001, ... are global constants defined in CUIAutomation2.au3:

;module UIA_ControlTypeIds
Global Const $UIA_ButtonControlTypeId=50000
Global Const $UIA_CalendarControlTypeId=50001
Global Const $UIA_CheckBoxControlTypeId=50002
Global Const $UIA_ComboBoxControlTypeId=50003
Global Const $UIA_EditControlTypeId=50004
Global Const $UIA_HyperlinkControlTypeId=50005
Global Const $UIA_ImageControlTypeId=50006
Global Const $UIA_ListItemControlTypeId=50007
Global Const $UIA_ListControlTypeId=50008
Global Const $UIA_MenuControlTypeId=50009
Global Const $UIA_MenuBarControlTypeId=50010
Global Const $UIA_MenuItemControlTypeId=50011
...

If the AutoIt Window Info tool can't identify your controls, the best option is to try with the UI Automation framework.

Note that if you identify the controls with UI Automation code, you also have to automate the controls with UI Automation code. You cannot use classic automation code.

Take a look at this post.

1 person likes this

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

If you are a starter first automate calc.exe to get used to AutoIt

Then automate www.google.com first with IE and then with chrome

Then you understand the concepts and are ready for your own application. See more links in frequently asked questions nr 31 or directives in uiautomation thread in examples section.

And please let know what is highlighted area by any of the spy tools

Edited by junkew

Share this post


Link to post
Share on other sites

#6 ·  Posted

Hi,

 be sure the AutoIt script run with the same Administrative right you will need to use #requireAdmin if the target program run in admin mode

1 person likes this

Share this post


Link to post
Share on other sites

#7 ·  Posted

Thank you all Larsj, junkew, jpm!

I will start to look into UI Automation framework, I don't recall that i used it before.

 

This isn't really my first automatization project, I did the calculator for my own joy, and a few years ago another program automatization like this, now a days I have a few excel autoformating scripts that also get info from exchange rates given the dates from the line, and calculating a few stuffs and order them etc. I'm not a fresh starte (I wouldn't say I'm a good one neither :) )

Highlighted areas are usually good around the buttons, but it see the icons and textboxes on the buttons as stand alone components, I'm not sure how it should work, or how it is written, and I can't upload pictures, sorry :(

 

jpm, thanks, the moment I read your comment I knew this will be the problem, it seems like its working (I only tried to use it once, it worked and I wanted to get back to you guys as fast as possible)

 

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

I'm sorry, but I downloaded SDK 10, and run into a "little problem" with inspect.exe

After I was unable to find it manually, (was looking around in the wrong folder, Microsoft SDK I think) I used the good old, search the entire C drive for "ApplicationName" method. It was somewhat successful, I found 4 Inspect.exe from which 2 of them gave me an error message, and 2 others are working. I guess it has something to do about my windows 10 (x86) version.

My questions: why are there 4 of them? What do I need to keep to keep one of the working inspects running? Do I need to save any files other then the Inspect.exe?

Thank you for your insight 

Edit:

Also this is my first time (be gentel) when I would like to add additional UDFs to autoIT, I created the extra folder outside autoit3, and used sciteconfig to connect it to scite, but I cant find the CUIAutomation2.au3 UDF to download, where can I find it?

(Sorry for the basic questions, but maybe someone else will be happy to find the anserws too :) )

Edited by SorryButImaNewbie

Share this post


Link to post
Share on other sites

#9 ·  Posted

Four versions of Inspect.exe: A 32 and a 64 bit version for a standard CPU and an ARM CPU.

You need the two versions for a standard CPU in bin\x86 (32 bit) and bin\x64 (64 bit). They are standalone programs and are not depending on anything else. You can rename the files to Inspect_x86.exe and Inspect_x64.exe and copy the files to a folder of your own. If your application is running 64 bit you should use Inspect_x64.exe. If your application is running 32 bit you should use Inspect_x86.exe.

You can find CUIAutomation2.au3 in UIA_V0_63.zip in bottem of first post in the UI Automation framework.

Regards Lars.

1 person likes this

Share this post


Link to post
Share on other sites

#10 ·  Posted (edited)

Thanks Lars!

I do what you told me regarding the Inspects.exe-s,  CUIAutomation2.au3 was already on my computer, I just didn't know... I placed it to the UDF folder

thank you directing me to find it (upsy)

Edit:

I have a little problem, I cant seem to add the UDF to the Scite CallTip manager (see picture), I choose the folder and the UDF, I think it should have the Header mode, but if I click Parse I get a Msg that reads: Parsing Complete (title), Please select another UDF or Exit(text) and an OK button. M'i done? :)  I thought (according to the help file) that I have to click Add and thats bothers me, giving me a feeling that something isn't as it should be.

pic:

 

ExtraUDF.PNG

Edited by SorryButImaNewbie

Share this post


Link to post
Share on other sites

#11 ·  Posted

Unfortunately, I cannot help with that. I'm using Notepad++ and not SciTE, so I don't know anything about SciTE. Maybe one of those who know something will help.

Share this post


Link to post
Share on other sites

#12 ·  Posted

CUIAutomation2.au3 does not have any functions defined in it, that is why call tips are not created.  User CallTips Manager only works with functions.

  

Adam

 

1 person likes this

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • lattey
      By lattey
      I used the window info tool to get what I've got below, but it just doesn't click the button. I run the script, nothing happens. 
      WinActivate("SELECTION") Sleep(200) ControlClick("SELECTION", "", "[CLASS:Button;INSTANCE:2]") i think maybe the autoit cannot detect the windows. So what i do:
       
      MouseClick( "", 503, 500) local $blanktitle = WinGetTitle("[ACTIVE]") If StringInStr($blanktitle, "SELECTION") Then msgbox("","", "got") WinActivate($blanktitle) ControlClick($blanktitle, "", "[CLASS:Button;INSTANCE:2]") Endif The script can detect the windows as the it is prompt "got" message. But it just doesn't click the button.
      Please help.
      Thanks for any help you can offer.
    • breakbadsp
      By breakbadsp
      AutoIT AU3info doeas not detect all gui objects uniquely for .NET GUIs developed in C#.
      this is not working now i am using COM windows approach for this, But its very difficult.
      Please let me know if anyone has done it before.
    • KB505
      By KB505
      Hello,
       
      I'm starting a script in AutoIT to automize a task in a software. I need to click a several Controls. 
      I'm just starting to learn AutoIT, and discovered that we can use the Window Info tool (au3info) to identify controls precisely and targetting safely a specific button.
      However, when I try it, I realize that most of the controls in my toolbar have the same Control Info (No ID, same class, same instance, ...). The only thing that changes is the ControlClicks Coord but I don't think that it's interesting, I don't see the difference with the Mouse Coordinates. Besides, I was aiming to write a stable script that would work even if the button changes a little bit of place for example. 
      Should I use another tool that Window Info tool to detect this Controls ? Is there another way to identify a Control ? 
      Thank you in advance for your help
       
    • Ubermensch
      By Ubermensch
      Hello,
      I am currently trying to automatically click the "Yes" button in the ActiveX prompt/popup message after opening the IE (html).
      At 1st, I encounter the "Allow Blocked Content". I already resolve it just by changing settings in the IE Options. But after resolving the "Allow Blocked Content", there's a popup message appear.
      I have attached the ActiveX Prompt.
      Here is the 1st code that I try to use.
      #include <IE.au3> #include <MsgBoxConstants.au3> #include <WinAPI.au3> _IECreate("C:\Users\april\Documents\Logo\JRB\AutoIt\AutoBOT\AWD10\sampleAWD10.html",0,1,0) Local $oIE = _IEAttach("", "instance", 1) _IELoadWait($oIE) AdlibRegister("_ActiveXRun",250) Local $oLastName = _IEGetObjByName($oIE, "Text4") Local $oGetItem = _IEGetObjByName($oIE, "getitem") _IEAction($oGetItem, "click") MsgBox($MB_SYSTEMMODAL, "Form Element Value", _IEFormElementGetValue($oGetItem)) Func _ActiveXRun() $retWin = WinGetHandle("[Class:Button]","") $winTitle = "[HANDLE:" & $retWin &"]" $ctrlHandle = ControlGetHandle($winTitle,"", "[CLASS:Button; INSTANCE:2]") $ctrlTitle = "HANDLE:" & $ctrlHandle &"]" WinWaitActive($ctrlTitle,"[CLASS:Button; INSTANCE:2]",10) $k = ControlGetPos($winTitle, "","[CLASS:Button; INSTANCE:2]") $x = $k[0] $y = $k[1] WinActivate ($winTitle,"An ActiveX control on this page might be unsafe to interact with other parts of the page. Do you want to allow this interaction?") ControlFocus($winTitle,"An ActiveX control on this page might be unsafe to interact with other parts of the page. Do you want to allow this interaction?","[CLASS:Button; INSTANCE:2]") ControlClick($winTitle, "","[CLASS:Button; INSTANCE:2]","primary",1,$x,$y) ControlSend($winTitle, "", "[CLASS:Button; INSTANCE:2]", "{ENTER}", 0) EndFunc Here is the console output.
      >"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "C:\Users\april\Documents\Logo\JRB\AutoIt\AutoBOT\AWD10\AWD10.1.au3" /UserParams +>12:44:11 Starting AutoIt3Wrapper v.17.224.935.0 SciTE v.3.7.3.0 Keyboard:00000409 OS:WIN_10/ CPU:X64 OS:X64 Environment(Language:0409) CodePage:0 utf8.auto.check:4 +> SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE UserDir => C:\Users\april\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper SCITE_USERHOME => C:\Users\april\AppData\Local\AutoIt v3\SciTE >Running AU3Check (3.3.14.2) from:C:\Program Files (x86)\AutoIt3 input:C:\Users\april\Documents\Logo\JRB\AutoIt\AutoBOT\AWD10\AWD10.1.au3 +>12:44:11 AU3Check ended.rc:0 >Running:(3.3.14.2):C:\Program Files (x86)\AutoIt3\autoit3.exe "C:\Users\april\Documents\Logo\JRB\AutoIt\AutoBOT\AWD10\AWD10.1.au3" --> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop "C:\Users\april\Documents\Logo\JRB\AutoIt\AutoBOT\AWD10\AWD10.1.au3" (31) : ==> Subscript used on non-accessible variable.: $x = $k[0] $x = $k^ ERROR ->12:44:24 AutoIt3.exe ended.rc:1 +>12:44:24 AutoIt3Wrapper Finished. >Exit code: 1 Time: 13.76 I also tried a simpler code.
      include <IE.au3> #include <MsgBoxConstants.au3> #include <WinAPI.au3> _IECreate("C:\Users\april\Documents\Logo\JRB\AutoIt\AutoBOT\AWD10\sampleAWD10.html",0,1,0) Local $oIE = _IEAttach("", "instance", 1) _IELoadWait($oIE) $k = ControlGetPos("[CLASS:#32770]", "","[CLASS:Button; INSTANCE:2]") $x = $k[0] $y = $k[1] ControlClick("[CLASS:#32770]", "","[CLASS:Button; INSTANCE:2]","primary",1,$x,$y) Local $oLastName = _IEGetObjByName($oIE, "Text4") Local $oGetItem = _IEGetObjByName($oIE, "getitem") _IEAction($oGetItem, "click") MsgBox($MB_SYSTEMMODAL, "Form Element Value", _IEFormElementGetValue($oGetItem)) There's no error in the console output for the 2nd code I have tried.
      And here is the Window Info for the ActiveX prompt.
      >>>> Window <<<< Title: Internet Explorer Class: #32770 Position: 580, 338 Size: 376, 146 Style: 0x94C808C4 ExStyle: 0x00010101 Handle: 0x00000000001516FA >>>> Control <<<< Class: Button Instance: 2 ClassnameNN: Button2 Name: Advanced (Class): [CLASS:Button; INSTANCE:2] ID: 1 Text: &Yes Position: 184, 77 Size: 80, 22 ControlClick Coords: 37, 12 Style: 0x50010000 ExStyle: 0x00000004 Handle: 0x00000000001215DE >>>> Mouse <<<< Position: 228, 120 Cursor ID: 0 Color: 0xFFFFFF >>>> StatusBar <<<< >>>> ToolsBar <<<< >>>> Visible Text <<<< &No An ActiveX control on this page might be unsafe to interact with other parts of the page. Do you want to allow this interaction? &Yes >>>> Hidden Text <<<<  

    • VeryGut
      By VeryGut
      I am unable to access an annyoing pop-up window an press the "No" button. So far I have tried a combination (including using coordinates etc.) of the following:
      ControlClick("[TITLE: Worksite; CLASS: #32770]", "&No", "[CLASS: Button2]")
      Doesnt seem to be working :C
      >>>> Window <<<< Title: WorkSite Class: #32770 Position: 786, 471 Size: 356, 162 Style: 0x94C801C5 ExStyle: 0x00010101 Handle: 0x00000000000209E2 >>>> Control <<<< Class: Button Instance: 2 ClassnameNN: Button2 Name: Advanced (Class): [CLASS:Button; INSTANCE:2] ID: 7 Text: &No Position: 158, 96 Size: 88, 26 ControlClick Coords: 40, 11 Style: 0x50010000 ExStyle: 0x00000004 Handle: 0x000000000004075E >>>> Mouse <<<< Position: 987, 604 Cursor ID: 0 Color: 0x3687CE >>>> StatusBar <<<< >>>> ToolsBar <<<< >>>> Visible Text <<<< &Yes &No Cancel Do you want to save the changes you made to 'TEST.xls'? >>>> Hidden Text <<<<