Jump to content
ssheldon

Some basic questions to get me started

Recommended Posts

ssheldon

Hi all,

I am brand new to AutoIT.

I would like to try and automate testing a piece of software to verify that the UI matches a known good state.  What I mean by this is I would take a screen shot of the software being tested and save it.  Then I would use AutoIT to launch the software and take a screen shot of the software and compare it to the original saved screenshot.  The software I am testing has different licensed configurations, and I figure by taking different screenshots of the UI under different licenses I can automate the testing of the UI under different license configurations to verify that the correct commands are enabled (or not) for different licensed scenarios.

I have been reading about bitmap processing here:

To make my first attempt at this, I want to make a simple program that does this:

  1. Launch the target software.
  2. Take a screenshot.
  3. Compare to a previous saved screenshot.

I am currently working through the AutoIt_123 lessons.

Can anyone tell me how to launch an executable from AutoIT?

Thanks,

Steve

Share this post


Link to post
Share on other sites
Xandy
Posted (edited)
ShellExecute("path\filename.exe")

Run("path\filename.exe")

Either of these functions can launch an exe from AutoIt.

After typing the function, hover over function name and press F1 to bring up the help file containing additional parameter information.

Edited by Xandy
  • Like 1

Share this post


Link to post
Share on other sites
ssheldon

OK, found my answer in the help.  To launch the program I will use:

Run("myprogram.exe")

Next up, how to take a screenshot.

Steve

Share this post


Link to post
Share on other sites
water

Define "known good state".
Does this mean controls are in a defined position, state or have a defined content ...?
Do you want to test different use cases as well? Means: enter data and check that the program acts correctly?

Before starting to compare screenshots you should be sure that this is the best way to achieve your goal :)

An example of both screens would greatly help.


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2018-06-01 - Version 1.4.9.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (2018-01-27 - Version 1.3.3.1) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites
ssheldon
17 minutes ago, water said:

Define "known good state".
Does this mean controls are in a defined position, state or have a defined content ...?
Do you want to test different use cases as well? Means: enter data and check that the program acts correctly?

Before starting to compare screenshots you should be sure that this is the best way to achieve your goal :)

For example, if you launch Paint, the Crop, Outline, and Fill commands are all shown in grey, as they are disabled, while all of the commands in the Tools group are enabled.

If I took a screen shot of the Paint ribbon bar, and then checked it through automation, and time ribbon bar changed somehow (commands added or missing, or enabled/disabled state changed), then the bitmap comparison should detect that.

Sound good?

Steve

Share this post


Link to post
Share on other sites
JLogan3o13
52 minutes ago, ssheldon said:

For example, if you launch Paint, the Crop, Outline, and Fill commands are all shown in grey, as they are disabled, while all of the commands in the Tools group are enabled.

If I took a screen shot of the Paint ribbon bar, and then checked it through automation, and time ribbon bar changed somehow (commands added or missing, or enabled/disabled state changed), then the bitmap comparison should detect that.

Sound good?

Steve

Rube Goldberg would be proud...

You say you're checking license configuration, are you truly saying there is no other way (license file, registry entry, dll presence, etc. etc. etc.) that you can check for this? If the correct license is applied, one would think the commands would not be grayed out.


√-1 2^3 ∑ π, and it was delicious!

Share this post


Link to post
Share on other sites
ssheldon

 

1 minute ago, JLogan3o13 said:

Rube Goldberg would be proud...

You say you're checking license configuration, are you truly saying there is no other way (license file, registry entry, dll presence, etc. etc. etc.) that you can check for this? If the correct license is applied, one would think the commands would not be grayed out.

I have the license file, but that does not mean that the software is acting on it correctly.  The test we do now manually is fire up the software with a particular license file, and then go and verify that the parts of the software that should be active are, and that other parts are not.  We currently do this by manually exercising the UI.

I figured we could automate this by screen shot comparison of the various ribbon menus.

Another way would be to interrogate the state of all commands, and I think there are ways to do that also, but it seems to me a lot faster to just examine the entire ribbon at one go.  This will also catch any changes in the UI.

Steve

Share this post


Link to post
Share on other sites
JLogan3o13

And you don't think that, instead, working with the vendor to resolve the issue of why applying the license file isn't making the right features available would be a better idea? AutoIt is great at doing almost anything you want to automate; just seems like you're trying to automate a way around not fixing the underlying issue.

  • Like 1

√-1 2^3 ∑ π, and it was delicious!

Share this post


Link to post
Share on other sites
ssheldon

To be clear here, what I am trying to do is automate testing that traditionally was done by hand through the UI while still testing automatically through the UI.  We have APIs that I could exercise through Visual Studio to test functionality bypassing the UI, but this is not what I am trying to do.

Steve

Share this post


Link to post
Share on other sites
ssheldon
2 minutes ago, JLogan3o13 said:

And you don't think that, instead, working with the vendor to resolve the issue of why applying the license file isn't making the right features available would be a better idea? AutoIt is great at doing almost anything you want to automate; just seems like you're trying to automate a way around not fixing the underlying issue.

I am the vendor.  :)

I work in testing.  One of the things that we test is that various license configurations work.  This is done by installing the appropriate license file, then launching the software, and making sure that the correct set of commands is operational.  This testing is currently done manually. 

This is only one tiny thing that I hope to ultimately automate.  Ultimately I intend to test many aspects of our software via UI automation.  Initially I was looking at macro recording software (is it possible to record macros in AutoIT?) but I did not find tools that had enough programming oomph behind them to be able to analyze what happens when the macro does things in the software.  For example, I can easily record a macro that clicks File -> SaveAs.  But can I tell if the dialog that pops up looks right?  I haven't seen that kind functionality in a macro recording software.

Steve

 

Share this post


Link to post
Share on other sites
careca

Maybe different licenses could change the titlebar text, corresponding to specific things enabled/disabled, and it can be checked in autoit fast and easy.

Or am i looking at this totally backwards?

I know there are ways to compare screenshots, i've seen posts about it, one was about checking the surveilance camera for movement.

It would check for changes in every other pixel or so. I think you want something like that..


Spoiler

Paster - Main function is to paste text, but has more functions. (No longer mantained, switched to String Trigger)

Renamer - Rename files and folders, remove portions of text from the filename etc.

GPO Tool - Export/Import Group policy settings.

BeatsPlayer - Music player.

Params Tool - Right click an exe to see it's parameters or execute them.

String Trigger - Triggers pasting text or applications or internet links on specific strings.

Inconspicuous - Hide files in plain sight, not fully encrypted.

Regedit Control - Registry browsing history, quickly jump into any saved key.

Time4Shutdown - Write the time for shutdown in minutes.

Power Profiles Tool - Set a profile as active, delete, duplicate, export and import.

Firefox Profile Backup - Backup/restore previously saved profile.

Finished Task Shutdown - Shuts down pc when specified window/Wndl/process closes.

NetworkSpeedShutdown - Shuts down pc if download speed goes under "X" Kb/s.

IUIAutomation - Topic with framework and examples

Au3Record.exe

Share this post


Link to post
Share on other sites
Earthshine
Posted (edited)

What licensing software are you using? Some of them set up a port on the local host and you can view all the license information that you want there in a browser and capture the real info

Edited by Earthshine

My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
ssheldon

I cannot control what the license files do - that is already fixed.  So having them change the behavior of the software other than what the license files already do is not an option.

Imagine we launch paint.  The ribbon looks like this:

FI9dj0Sl.png

This is the "known good state".  In other words, this is what it should look like when we open Paint on the Home tab.

Now imagine I launch Paint and go to the home tab and it looks like this:

vbrMqs9l.png

Notice that some of the commands have changed state:

T7Pf1rIl.png

There is probably a way to programmatically interrogate each command and discover its state.  But this would require a check for every command.  In addition, it won't detect other changes in the UI, like commands moving around, or other anomolies. 

I figured an easy test here would be to create a "baseline" screen capture, and then when the automated test runs it could check a section of the screen against that baseline.

Not only will this detect changes in command states, but any other deviations in the UI presentation also.

Steve

Share this post


Link to post
Share on other sites
ssheldon
7 minutes ago, Earthshine said:

What licensing software are you using? Some of them set up a port on the local host and you can view all the license information that you want there in a browser and capture the real info

We use FlexLM.  It is possible that I could go "under the hood" to find the states, but I am trying to test it through the UI, as a user would. 

For example, our software has numerous APIs that can be called programmatically to do things that the software does.  Like draw a box, for example.  But this only tests the API, not the UI.  I want to test the software by simulating users actually poking buttons in the UI.

As an analogy, in the old days you could reach under the hood of your car and actuate the throttle linkage and rev the engine.  You can also, of course, sit in the driver's seat and push the gas pedal to rev the engine.  One is not an adequate test of the other.  Just because I can rev the engine under the hood does not mean that the gas pedal works like it is supposed to.  I'm trying to get through the actual user interface, not bypass it.  This is why I am looking to UI automation solutions rather than just writing programs that access our internal APIs.

Not to mention I think it will be easier.  :)

Steve

  • Like 1

Share this post


Link to post
Share on other sites
Earthshine

we use flexlm too. ok. As both systems and software engineer and avid tester I understand where you're coming from completely and I think a couple of post above somebody gave a link to some comparison stuff or we can help look it up


My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
Earthshine

 You have to understand that image comparison alone will only get you so far. You don't need to use the APIs and visual studio to mimic things but you could use their libraries to manipulate the actual controls that you care about in the UI to verify its behavior. Depending on the application you may be able to use auto IT alone to do the GUI manipulations and in doing so you could prove out the state of your license installation


My resources are limited. You must ask the right questions

 

Share this post


Link to post
Share on other sites
ssheldon

It's possible.  Image comparison just seemed easier to me.  I don't know how to manipulate the UI programmatically.  I just discovered that the buttons on the ribbon are not "controls", for example, so I don't know how to automate clicking on the Text button in Paint, for example, via AutoIT (or any other means).

Presumably one can click via x,y coordinates, but this might be a problem if the program is run minimized, or on a different resolution monitor, or whatnot (but I expect that will blow image comparison out of the water, also).  So my expectation is that this kind of program will need to be run on a specific machine.

Steve

Share this post


Link to post
Share on other sites
Xandy
Posted (edited)

The tools in this example will achieve BMP image comparison. 

Unless I am wrong, and I am never wrong.

Frog_Eyes_Fire2.gif

It's just an example of the tools however.

This is kind of what you want.

#include <GDIPlus.au3>

_GDIPlus_Startup()
$hBMP_somefile = _GDIPlus_BitmapCreateFromFile($filepath)
For $y To $height
  For $x To $width
    If _GDIPlus_BitmapGetPixel($hBMP_Screen, $x, $y) = _GDIPlus_BitmapGetPixel($hBMP_somefile, $x, $y) Then
        ; Maybe you want to say if Not Equal <>
        ; I do it with a counter checksum, but then you must set a counter
    EndIf
  Next; x
Next; y

_GDIPlus_Shutdown()

Forgive I didn't put much time into this example.  Also it may not be the 'best'.  It's not as fast as CPP (writing this in CPP is not that difficult and it's probably 100..1000 times faster).  There are lots of other examples that I have been hopeful of, but this is what works for me.  Perhaps a hash could be generated from screenshot area, and image loaded from file as hash.  Never done it that way, but this concept code can do what you want in a reasonable amount of time.  Remember to keep the compared area as small as you can.  Less pixels; done faster.  You can also use a mask to disregard pixels from a rectangular area or use a list.

Use the tools in my posts above to collect the image screen data.  ScreenCapture or whatever.

Edited by Xandy
  • Haha 1

Share this post


Link to post
Share on other sites
careca
3 hours ago, ssheldon said:

...one can click via x,y coordinates, but this might be a problem if the program is run minimized, or on a different resolution monitor, or whatnot

I guess you could force the application window to be enabled, active and have a specific resolution that would fit all possible screens, and then test it by pixels or that screenshot method.


Spoiler

Paster - Main function is to paste text, but has more functions. (No longer mantained, switched to String Trigger)

Renamer - Rename files and folders, remove portions of text from the filename etc.

GPO Tool - Export/Import Group policy settings.

BeatsPlayer - Music player.

Params Tool - Right click an exe to see it's parameters or execute them.

String Trigger - Triggers pasting text or applications or internet links on specific strings.

Inconspicuous - Hide files in plain sight, not fully encrypted.

Regedit Control - Registry browsing history, quickly jump into any saved key.

Time4Shutdown - Write the time for shutdown in minutes.

Power Profiles Tool - Set a profile as active, delete, duplicate, export and import.

Firefox Profile Backup - Backup/restore previously saved profile.

Finished Task Shutdown - Shuts down pc when specified window/Wndl/process closes.

NetworkSpeedShutdown - Shuts down pc if download speed goes under "X" Kb/s.

IUIAutomation - Topic with framework and examples

Au3Record.exe

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×