How To Detect Certain Images On Screen?

[2Kbushwick]

Hi, I am trying to write a scirpt that can detect an image (specifically a blue circle) and then click on it? The circle will appear in different locations at different times..any suggestions? By the way, the circle appears in an Internet Explorer window.

Thanks.

[CyberSlug]

See GetColor and PixelSearch in the help file. You can get the color of the image by using the AU3_Spy program included with AutoIt. (It's probably easiest to take a screen shot of the circle and use the spy program on that unchanging image.)

; use AU3_Spy to get the bounding coordinates of the region you need to check for the blue circle.  The smaller the region the better
$color = 11348009;adjust to your specific color of blue
$left = 0
$top = 0
$bottom = 640
$right = 480

$PAUSED = 0
HotKeySet("{Pause}", "Pause")
HotKeySet("{Esc}", "Terminate")

Func Pause()
   $PAUSED = NOT $PAUSED;toggle
   While $PAUSED
      sleep(100)
      ToolTip("Paused", 0, 0)
   WEnd
   ToolTip('')
EndFunc

Func Terminate()
   Exit
EndFunc


While 1
   For $x = $left to $right
      For $y = $top to $bottom
         If PixelGetColor ( $x, $y ) = $color Then
           MouseClick ( "left", $x, $y, 1, 0)
         EndIf
      Next
   Next
WEnd

Untested code, but should get you started. Press Escape to end the program; press the Pause/Break key to pause

[2Kbushwick]

WOW! Thanks a lot. I really appreciate your help. This is great.

[2Kbushwick]

This works great but I have one question regarding the coordinates. When I specify the search area coordinates how exactly do i determine the left, right, top, and bottom coordinates. I know how to use winspy included with autoit, but it only give x y coordinates. So what would be the left coordinate...x or y, and what would be the top coordinate...x or y. I'm confused.

Any help?

Thanks.

[CyberSlug]

you need to give the (x,y) coords for the upper-left corner and the (x,y) coords for the lower-right corner. You then search that rectangular region for the color you want.

left,top----------------------
|                            |
|                            |
|                            |
|                            |
------------------right,bottom

For example, to search the entire screen for a computer monitor set at 800x600 resolution, you could search (left,top) = (0,0) through (right,bottom) = (799,599)

See PixelCoordMode and PixelCoordMode for more info.

Edited May 17, 2004 by CyberSlug

[2Kbushwick]

I understand now. Thanks again - I appreciate it.

[trids]

Just out of interest:

• is it a circle (blue on the perimeter only) or a disc (filled in blue)?
• And how widely can the diameter range?
• And is the shade of blue unique to the circle?

... cos you might find the theorem useful that says only one circle can go through 3 points.

- OR -

• Is the circle displayed in the Web-page as a gif (or jpg etc) with a name that you know already?

... cos then you might be able to cheat by URLdownloadFile-ing the page and working out where it is by parsing the <DIV> tags (for example).

Sorry for all the lateral thoughts - Habanero chilli sauce always does that to me

[2Kbushwick]

Trids,

Thanks for the post. Here are the answers to your questions...

The circle is completely blue, and not just blue on the perimeter. The diameter of the circle is always the same, the problem is it appears in different locations on the web page. And yes, the shade of blue is unique to the circle.

I am interested in your thoughts on writing a theorem that says only one circle can go through 3 points at one time. Can you provide more information?

Thanks.

[CyberSlug]

I am interested in your thoughts on writing a theorem that says only one circle can go through 3 points at one time.  Can you provide more information?

this site might help For any three non-colinear points, there exists exactly one circle that goes through those three points. (Or something like that)

[trids]

Nice link, Sluggie

@ 2Kbushwick ..

If it's a solid circle, then you can use the one that says the perpendicular bisector of a chord passes through the centre of its circle:

• You could find the chord with a series of vertical scans throught the browser window, where distance between each vertical line is less than the diameter of the circle(*).
• As soon as you find the chord, stop the vertical scan.
• Because the circle has a fixed radius, you can then use pythagoras to find the centre of the circle: it lies on the horizontal bisector of the chord, where the hypotenuse is the fixed radius, and the other side of the triangle is half the length of the chord.

* Since the scans are just short of a diameter apart, the centre could lie either to the left or right of the chord. If you make the scans just short of a radius apart, you could always be sure that the centre would lie to the right of the chord, but then you would have to make twice as many scans just to find the chord!

Come to think of it, you can use the same method on an empty circle too. The only difference is that the chord would now only have the two "visible" end-points instead of a solid line

There are probably better search algorithms, this is just the first one that occurred to me. Hope it helps!

Edit: using pythagoras instead of trig

Edited May 18, 2004 by trids

[trids]

Hmm .. I just re-read your original post, which was that you simply want to click on the circle.

In that case, you don't really need to know exactly where the circle lies after all ... you just need to scan in a grid fashion: where each point is less than a diameter apart away from its closest neighbours. Then, as soon as you find a blue pixel, click on it

You might want to consider alternatives to a grid pattern: for speed, you want to find the pattern that has the fewest number of points. Possibly a honeycomb? Or maybe equilateral triangles?

Whatever pattern you choose, the distance between the points is crucial: they're not all like the grid, just short of a diameter