Some basic questions to get me started

[ssheldon]

Hi all,

I am brand new to AutoIT.

I would like to try and automate testing a piece of software to verify that the UI matches a known good state.  What I mean by this is I would take a screen shot of the software being tested and save it.  Then I would use AutoIT to launch the software and take a screen shot of the software and compare it to the original saved screenshot.  The software I am testing has different licensed configurations, and I figure by taking different screenshots of the UI under different licenses I can automate the testing of the UI under different license configurations to verify that the correct commands are enabled (or not) for different licensed scenarios.

I have been reading about bitmap processing here:

To make my first attempt at this, I want to make a simple program that does this:

1. Launch the target software.
2. Take a screenshot.
3. Compare to a previous saved screenshot.

I am currently working through the AutoIt_123 lessons.

Can anyone tell me how to launch an executable from AutoIT?

Thanks,

Steve

[Xandy]

ShellExecute("path\filename.exe")

Run("path\filename.exe")

Either of these functions can launch an exe from AutoIt.

After typing the function, hover over function name and press F1 to bring up the help file containing additional parameter information.

Edited June 5, 2018 by Xandy

[ssheldon]

OK, found my answer in the help.  To launch the program I will use:

Run("myprogram.exe")

Next up, how to take a screenshot.

Steve

[Xandy]

I use these:

Sample only specific window:

https://www.autoitscript.com/autoit3/docs/libfunctions/_ScreenCapture_CaptureWnd.htm

Sample Desktop:

https://www.autoitscript.com/autoit3/docs/libfunctions/_ScreenCapture_Capture.htm

Edited June 5, 2018 by Xandy

[water]

Define "known good state".
Does this mean controls are in a defined position, state or have a defined content ...?
Do you want to test different use cases as well? Means: enter data and check that the program acts correctly?

Before starting to compare screenshots you should be sure that this is the best way to achieve your goal

An example of both screens would greatly help.

[ssheldon]

17 minutes ago, water said:

Define "known good state".
Does this mean controls are in a defined position, state or have a defined content ...?
Do you want to test different use cases as well? Means: enter data and check that the program acts correctly?

Before starting to compare screenshots you should be sure that this is the best way to achieve your goal

For example, if you launch Paint, the Crop, Outline, and Fill commands are all shown in grey, as they are disabled, while all of the commands in the Tools group are enabled.

If I took a screen shot of the Paint ribbon bar, and then checked it through automation, and time ribbon bar changed somehow (commands added or missing, or enabled/disabled state changed), then the bitmap comparison should detect that.

Sound good?

Steve

[JLogan3o13]

52 minutes ago, ssheldon said:

For example, if you launch Paint, the Crop, Outline, and Fill commands are all shown in grey, as they are disabled, while all of the commands in the Tools group are enabled.

If I took a screen shot of the Paint ribbon bar, and then checked it through automation, and time ribbon bar changed somehow (commands added or missing, or enabled/disabled state changed), then the bitmap comparison should detect that.

Sound good?

Steve

Rube Goldberg would be proud...

You say you're checking license configuration, are you truly saying there is no other way (license file, registry entry, dll presence, etc. etc. etc.) that you can check for this? If the correct license is applied, one would think the commands would not be grayed out.

[ssheldon]

 

1 minute ago, JLogan3o13 said:

Rube Goldberg would be proud...

You say you're checking license configuration, are you truly saying there is no other way (license file, registry entry, dll presence, etc. etc. etc.) that you can check for this? If the correct license is applied, one would think the commands would not be grayed out.

I have the license file, but that does not mean that the software is acting on it correctly.  The test we do now manually is fire up the software with a particular license file, and then go and verify that the parts of the software that should be active are, and that other parts are not.  We currently do this by manually exercising the UI.

I figured we could automate this by screen shot comparison of the various ribbon menus.

Another way would be to interrogate the state of all commands, and I think there are ways to do that also, but it seems to me a lot faster to just examine the entire ribbon at one go.  This will also catch any changes in the UI.

Steve

[JLogan3o13]

And you don't think that, instead, working with the vendor to resolve the issue of why applying the license file isn't making the right features available would be a better idea? AutoIt is great at doing almost anything you want to automate; just seems like you're trying to automate a way around not fixing the underlying issue.

[ssheldon]

To be clear here, what I am trying to do is automate testing that traditionally was done by hand through the UI while still testing automatically through the UI.  We have APIs that I could exercise through Visual Studio to test functionality bypassing the UI, but this is not what I am trying to do.

Steve

[ssheldon]

2 minutes ago, JLogan3o13 said:

And you don't think that, instead, working with the vendor to resolve the issue of why applying the license file isn't making the right features available would be a better idea? AutoIt is great at doing almost anything you want to automate; just seems like you're trying to automate a way around not fixing the underlying issue.

I am the vendor. 

I work in testing.  One of the things that we test is that various license configurations work.  This is done by installing the appropriate license file, then launching the software, and making sure that the correct set of commands is operational.  This testing is currently done manually. 

This is only one tiny thing that I hope to ultimately automate.  Ultimately I intend to test many aspects of our software via UI automation.  Initially I was looking at macro recording software (is it possible to record macros in AutoIT?) but I did not find tools that had enough programming oomph behind them to be able to analyze what happens when the macro does things in the software.  For example, I can easily record a macro that clicks File -> SaveAs.  But can I tell if the dialog that pops up looks right?  I haven't seen that kind functionality in a macro recording software.

Steve

 

[careca]

Maybe different licenses could change the titlebar text, corresponding to specific things enabled/disabled, and it can be checked in autoit fast and easy.

Or am i looking at this totally backwards?

I know there are ways to compare screenshots, i've seen posts about it, one was about checking the surveilance camera for movement.

It would check for changes in every other pixel or so. I think you want something like that..

[Earthshine]

What licensing software are you using? Some of them set up a port on the local host and you can view all the license information that you want there in a browser and capture the real info

Edited June 5, 2018 by Earthshine

[ssheldon]

I cannot control what the license files do - that is already fixed.  So having them change the behavior of the software other than what the license files already do is not an option.

Imagine we launch paint.  The ribbon looks like this:

This is the "known good state".  In other words, this is what it should look like when we open Paint on the Home tab.

Now imagine I launch Paint and go to the home tab and it looks like this:

Notice that some of the commands have changed state:

There is probably a way to programmatically interrogate each command and discover its state.  But this would require a check for every command.  In addition, it won't detect other changes in the UI, like commands moving around, or other anomolies. 

I figured an easy test here would be to create a "baseline" screen capture, and then when the automated test runs it could check a section of the screen against that baseline.

Not only will this detect changes in command states, but any other deviations in the UI presentation also.

Steve

[ssheldon]

7 minutes ago, Earthshine said:

What licensing software are you using? Some of them set up a port on the local host and you can view all the license information that you want there in a browser and capture the real info

We use FlexLM.  It is possible that I could go "under the hood" to find the states, but I am trying to test it through the UI, as a user would. 

For example, our software has numerous APIs that can be called programmatically to do things that the software does.  Like draw a box, for example.  But this only tests the API, not the UI.  I want to test the software by simulating users actually poking buttons in the UI.

As an analogy, in the old days you could reach under the hood of your car and actuate the throttle linkage and rev the engine.  You can also, of course, sit in the driver's seat and push the gas pedal to rev the engine.  One is not an adequate test of the other.  Just because I can rev the engine under the hood does not mean that the gas pedal works like it is supposed to.  I'm trying to get through the actual user interface, not bypass it.  This is why I am looking to UI automation solutions rather than just writing programs that access our internal APIs.

Not to mention I think it will be easier. 

Steve

[Earthshine]

we use flexlm too. ok. As both systems and software engineer and avid tester I understand where you're coming from completely and I think a couple of post above somebody gave a link to some comparison stuff or we can help look it up

[Earthshine]

 You have to understand that image comparison alone will only get you so far. You don't need to use the APIs and visual studio to mimic things but you could use their libraries to manipulate the actual controls that you care about in the UI to verify its behavior. Depending on the application you may be able to use auto IT alone to do the GUI manipulations and in doing so you could prove out the state of your license installation

[ssheldon]

It's possible.  Image comparison just seemed easier to me.  I don't know how to manipulate the UI programmatically.  I just discovered that the buttons on the ribbon are not "controls", for example, so I don't know how to automate clicking on the Text button in Paint, for example, via AutoIT (or any other means).

Presumably one can click via x,y coordinates, but this might be a problem if the program is run minimized, or on a different resolution monitor, or whatnot (but I expect that will blow image comparison out of the water, also).  So my expectation is that this kind of program will need to be run on a specific machine.

Steve

[Xandy]

The tools in this example will achieve BMP image comparison. 

Unless I am wrong, and I am never wrong.

It's just an example of the tools however.

This is kind of what you want.

#include <GDIPlus.au3>

_GDIPlus_Startup()
$hBMP_somefile = _GDIPlus_BitmapCreateFromFile($filepath)
For $y To $height
  For $x To $width
    If _GDIPlus_BitmapGetPixel($hBMP_Screen, $x, $y) = _GDIPlus_BitmapGetPixel($hBMP_somefile, $x, $y) Then
        ; Maybe you want to say if Not Equal <>
        ; I do it with a counter checksum, but then you must set a counter
    EndIf
  Next; x
Next; y

_GDIPlus_Shutdown()

Forgive I didn't put much time into this example.  Also it may not be the 'best'.  It's not as fast as CPP (writing this in CPP is not that difficult and it's probably 100..1000 times faster).  There are lots of other examples that I have been hopeful of, but this is what works for me.  Perhaps a hash could be generated from screenshot area, and image loaded from file as hash.  Never done it that way, but this concept code can do what you want in a reasonable amount of time.  Remember to keep the compared area as small as you can.  Less pixels; done faster.  You can also use a mask to disregard pixels from a rectangular area or use a list.

Use the tools in my posts above to collect the image screen data.  ScreenCapture or whatever.

Edited June 6, 2018 by Xandy

[careca]

3 hours ago, ssheldon said:

...one can click via x,y coordinates, but this might be a problem if the program is run minimized, or on a different resolution monitor, or whatnot

I guess you could force the application window to be enabled, active and have a specific resolution that would fit all possible screens, and then test it by pixels or that screenshot method.