Sign in to follow this  
Followers 0
gwizard

ImageFinder - DLL for custom objects

8 posts in this topic

Hi All,

This is my first post here (I think) but I wanted to make you all a Christmas present :)

First I'll describe the problem.

If you writing tests for application that includes some 3rd party controls like Stingray, Infragistics or such you probably know that AutoIt (or QTP for that matter) cannot see inside the object to click on some internal button or something. The only viable solution is to use coordinates but to use them hard coded is a maintenance nightmare.

This is where tools such as ImageSearch come by. You make a reference image that must be present at some point on the screen, say the button inside a Flex window. The DLL searches the screen for the reference image by using one of the available methods (pixel comparison, pattern match, NN, etc...) and locates the coordinates for the image returning them to calling app (AutoIt). Then you can click on it and move on.

Well, after using ImageSearch for couple of weeks I was really not happy with it's performance. Sometimes it took quite a while to find something, especially when my AUT has a lot of gray areas which are all alike. To this end, I decided to develop my own solution. After some research and a lot of trial and error I present ImageFinder.

ImageFinder is a DLL written in C# (2.0) and uses 2 step method to find it's prey. In my tests it showed 30%-40% faster then ImageSearch.

Second step is your basic pixel comparison, but I use specific pattern to make a match so it's not 100% accurate (in theory, never happened to me yet). I also employ multi threading to achieve better performance.

The first step looks for existing coordinates in file name and uses a fast verification to see if it's correct. If not, stage 2 will fire.

Since step 1 uses file name and it can be annoying to make by hand, I also made a small Windows app called Zoom. What it does is to allow you to create a lot of images at once based on the current screen. You just fire it up, hit Alt+C, select rectangles with your mouse and then hit Alt+C again to save them all.

Specs

ImageFinder have only 1 public method and this is how you use it:

CODE
$obj = ObjCreate("AutomationTools.ImageFinder")

$res = $obj.Find(path_to_ref_image, 1, 0)

Second param is mode - 1 to return center of image on screen, 0 for top left.

Third param is not in use at the moment.

Returns string in the following format:

xxx|yyy

IMPORTANT: You must register the DLL before use.

To do so, copy the DLL to C:\WINDOWS\System32 and execute the following

C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\regasm C:\windows\system32\AutomationUtils.dll /tlb:c:\windows\system32\AutomationUtils.tlb /codebase

Of course, you also must have .NET Framework 2 installed.

I'll be happy to hear comments, suggestions and also criticism.

As for licensing, you can use it for whatever you like as long as you don't make any profit from it.

If you would like the source code or to integrate it into your own product, please contact me so we can talk prices.

My email is: admin@mods4u.net

Enjoy guys/gals and Marry Christmas / Hanukkah / Yule to all of you :-)

AutomationUtils.dll

Zoom.exe

Share this post


Link to post
Share on other sites



Very interesting project.

I have tried your dll with your example of code but I get an error with ObjCreate that return an error (error Nber is -2147221005 since I'm not a Windows developer I don't understand what it means).

I have register the dll and have .NET Framework 2.0 installed (the dll registration was successful).

Do you have any idea of what is going wrong ?

Here is my code

$obj = ObjCreate("AutomationTools.ImageFinder")
if($obj == 0) Then
    MsgBox(0,"Erreur","Object creation failed "&@error)
    exit
EndIf
$res = $obj.Find("G:\AutoIt3\work\test.img", 1, 0)
if $res == 0 Then
    MsgBox(0,"NOK","image not found")
Else
    $coords=StringSplit($res,"|")
    MsgBox(0,"OK","Found at "&$coords[1]&" - "&$coords[2])
EndIf

Share this post


Link to post
Share on other sites

I need a bit more details like, what kind of Windows do you use (I tested it only on XP SP2 & SP3) and what kind of .NET frameworks you have installed (I have them all).

Although I specifically compiled it for .NET 2, knowing M$ it could very well be that Framework 3 is required as well.

I think it is a problem with DLL registration, so...get this:

http://www.microsoft.com/downloads/details...;displaylang=en

It's a small program from MS site that you can view all the DLL's registered on your machine.

See if there is AutomationUtils in there.

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

same prob, with same error.

XP SP2 Microsoft.NET\Framework\v2.0.50727

Edited by border

Share this post


Link to post
Share on other sites

The exampe is wrong, it works with (tested with Vista):

$obj = ObjCreate("AutomationUtils.ImageFinder")

But it takes more than 20s to finde one image.

Share this post


Link to post
Share on other sites

Here is my code

$obj = ObjCreate("AutomationUtils.ImageFinder")

if($obj == 0) Then

MsgBox(0,"Erreur","Object creation failed "&@error)

exit

EndIf

$res = $obj.Find("G:\AutoIt3\work\test.img", 1, 0)

if $res == 0 Then

MsgBox(0,"NOK","image not found")

Else

$coords=StringSplit($res,"|")

MsgBox(0,"OK","Found at "&$coords[1]&" - "&$coords[2])

EndIf

But it takes more than 20s to finde one image.

This works for me as I am using software that has toolbars built using Stingray Objective Grids, but does take a while...

Is there a way to narrow down where this searches for the match (i.e. - Give it a rectangular area to search through)?

If I could tell it where to look, it would really cut down the time I wait.

Thanks,

dm83737

Share this post


Link to post
Share on other sites

This works for me as I am using software that has toolbars built using Stingray Objective Grids, but does take a while...

Is there a way to narrow down where this searches for the match (i.e. - Give it a rectangular area to search through)?

If I could tell it where to look, it would really cut down the time I wait.

Thanks,

dm83737

This isn't working on my machine. I'm using Windows7 64-bit. I've got Net Framework installed and have registered the dll.

Share this post


Link to post
Share on other sites

I try to use it with VS2010 c# and dont work :/.

can you please send me the code?

i always got 0 as return :D.

thanks

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0