ImageFinder - DLL for custom objects

[gwizard]

Hi All,

This is my first post here (I think) but I wanted to make you all a Christmas present

First I'll describe the problem.

If you writing tests for application that includes some 3rd party controls like Stingray, Infragistics or such you probably know that AutoIt (or QTP for that matter) cannot see inside the object to click on some internal button or something. The only viable solution is to use coordinates but to use them hard coded is a maintenance nightmare.

This is where tools such as ImageSearch come by. You make a reference image that must be present at some point on the screen, say the button inside a Flex window. The DLL searches the screen for the reference image by using one of the available methods (pixel comparison, pattern match, NN, etc...) and locates the coordinates for the image returning them to calling app (AutoIt). Then you can click on it and move on.

Well, after using ImageSearch for couple of weeks I was really not happy with it's performance. Sometimes it took quite a while to find something, especially when my AUT has a lot of gray areas which are all alike. To this end, I decided to develop my own solution. After some research and a lot of trial and error I present ImageFinder.

ImageFinder is a DLL written in C# (2.0) and uses 2 step method to find it's prey. In my tests it showed 30%-40% faster then ImageSearch.

Second step is your basic pixel comparison, but I use specific pattern to make a match so it's not 100% accurate (in theory, never happened to me yet). I also employ multi threading to achieve better performance.

The first step looks for existing coordinates in file name and uses a fast verification to see if it's correct. If not, stage 2 will fire.

Since step 1 uses file name and it can be annoying to make by hand, I also made a small Windows app called Zoom. What it does is to allow you to create a lot of images at once based on the current screen. You just fire it up, hit Alt+C, select rectangles with your mouse and then hit Alt+C again to save them all.

Specs

ImageFinder have only 1 public method and this is how you use it:

CODE

$obj = ObjCreate("AutomationTools.ImageFinder")

$res = $obj.Find(path_to_ref_image, 1, 0)

Second param is mode - 1 to return center of image on screen, 0 for top left.

Third param is not in use at the moment.

Returns string in the following format:

xxx|yyy

IMPORTANT: You must register the DLL before use.

To do so, copy the DLL to C:\WINDOWS\System32 and execute the following

C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\regasm C:\windows\system32\AutomationUtils.dll /tlb:c:\windows\system32\AutomationUtils.tlb /codebase

Of course, you also must have .NET Framework 2 installed.

I'll be happy to hear comments, suggestions and also criticism.

As for licensing, you can use it for whatever you like as long as you don't make any profit from it.

If you would like the source code or to integrate it into your own product, please contact me so we can talk prices.

My email is: admin@mods4u.net

Enjoy guys/gals and Marry Christmas / Hanukkah / Yule to all of you :-)

AutomationUtils.dll

Zoom.exe

[solotio]

Very interesting project.

I have tried your dll with your example of code but I get an error with ObjCreate that return an error (error Nber is -2147221005 since I'm not a Windows developer I don't understand what it means).

I have register the dll and have .NET Framework 2.0 installed (the dll registration was successful).

Do you have any idea of what is going wrong ?

Here is my code

$obj = ObjCreate("AutomationTools.ImageFinder")
if($obj == 0) Then
    MsgBox(0,"Erreur","Object creation failed "&@error)
    exit
EndIf
$res = $obj.Find("G:\AutoIt3\work\test.img", 1, 0)
if $res == 0 Then
    MsgBox(0,"NOK","image not found")
Else
    $coords=StringSplit($res,"|")
    MsgBox(0,"OK","Found at "&$coords[1]&" - "&$coords[2])
EndIf

[gwizard]

I need a bit more details like, what kind of Windows do you use (I tested it only on XP SP2 & SP3) and what kind of .NET frameworks you have installed (I have them all).

Although I specifically compiled it for .NET 2, knowing M$ it could very well be that Framework 3 is required as well.

I think it is a problem with DLL registration, so...get this:

http://www.microsoft.com/downloads/details...;displaylang=en

It's a small program from MS site that you can view all the DLL's registered on your machine.

See if there is AutomationUtils in there.

[border]

same prob, with same error.

XP SP2 Microsoft.NET\Framework\v2.0.50727

Edited October 17, 2009 by border

[script4ever]

The exampe is wrong, it works with (tested with Vista):

$obj = ObjCreate("AutomationUtils.ImageFinder")

But it takes more than 20s to finde one image.

[dm83737]

Here is my code

$obj = ObjCreate("AutomationUtils.ImageFinder")

if($obj == 0) Then

MsgBox(0,"Erreur","Object creation failed "&@error)

exit

EndIf

$res = $obj.Find("G:\AutoIt3\work\test.img", 1, 0)

if $res == 0 Then

MsgBox(0,"NOK","image not found")

Else

$coords=StringSplit($res,"|")

MsgBox(0,"OK","Found at "&$coords[1]&" - "&$coords[2])

EndIf

But it takes more than 20s to finde one image.

This works for me as I am using software that has toolbars built using Stingray Objective Grids, but does take a while...

Is there a way to narrow down where this searches for the match (i.e. - Give it a rectangular area to search through)?

If I could tell it where to look, it would really cut down the time I wait.

Thanks,

dm83737

[Phenom]

This works for me as I am using software that has toolbars built using Stingray Objective Grids, but does take a while...

Is there a way to narrow down where this searches for the match (i.e. - Give it a rectangular area to search through)?

If I could tell it where to look, it would really cut down the time I wait.

Thanks,

dm83737

This isn't working on my machine. I'm using Windows7 64-bit. I've got Net Framework installed and have registered the dll.

[Chain]

I try to use it with VS2010 c# and dont work :/.

can you please send me the code?

i always got 0 as return .

thanks