Jump to content

Image Search Library


kangkeng
 Share

Recommended Posts

  • 1 month later...

#include-once

; ------------------------------------------------------------------------------

;

; AutoIt Version: 3.0

; Language: English

; Description: Functions that assist with Image Search

; Require that the ImageSearchDLL.dll be loadable

;

; ------------------------------------------------------------------------------

;===============================================================================

;

; Description: Find the position of an image on the desktop

; Syntax: _ImageSearchArea, _ImageSearch

; Parameter(s):

; $findImage - the image to locate on the desktop

; $tolerance - 0 for no tolerance (0-255). Needed when colors of

; image differ from desktop. e.g GIF

; $resultPosition - Set where the returned x,y location of the image is.

; 1 for centre of image, 0 for top left of image

; $x $y - Return the x and y location of the image

; $transparency - TRANSBLACK, TRANSWHITE or hex value (e.g. 0xffffff) of

; the color to be used as transparency; can be omitted if

; not needed

;

; Return Value(s): On Success - Returns 1

; On Failure - Returns 0

;

; Note: Use _ImageSearch to search the entire desktop, _ImageSearchArea to specify

; a desktop region to search

;

;===============================================================================

Func _ImageSearch($findImage,$resultPosition,ByRef $x, ByRef $y,$tolerance,$transparency=0)

return _ImageSearchArea($findImage,$resultPosition,0,0,@DesktopWidth,@DesktopHeight,$x,$y,$tolerance,$transparency)

EndFunc

Func _ImageSearchArea($findImage,$resultPosition,$x1,$y1,$right,$bottom,ByRef $x, ByRef $y, $tolerance, $transparency=0)

;MsgBox(0,"asd","" & $x1 & " " & $y1 & " " & $right & " " & $bottom)

if not ($transparency = 0) then $findImage = "*" & $transparency & " " & $findImage

if $tolerance>0 then $findImage = "*" & $tolerance & " " & $findImage

$result = DllCall("ImageSearchDLL.dll","str","ImageSearch","int",$x1,"int",$y1,"int",$right,"int",$bottom,"str",$findImage)

; If error exit

if $result[0]="0" then return 0

; Otherwise get the x,y location of the match and the size of the image to

; compute the centre of search

$array = StringSplit($result[0],"|")

$x=Int(Number($array[2]))

$y=Int(Number($array[3]))

if $resultPosition=1 then

$x=$x + Int(Number($array[4])/2)

$y=$y + Int(Number($array[5])/2)

endif

return 1

EndFunc

;===============================================================================

;

; Description: Wait for a specified number of seconds for an image to appear

;

; Syntax: _WaitForImageSearch, _WaitForImagesSearch

; Parameter(s):

; $waitSecs - seconds to try and find the image

; $findImage - the image to locate on the desktop

; $tolerance - 0 for no tolerance (0-255). Needed when colors of

; image differ from desktop. e.g GIF

; $resultPosition - Set where the returned x,y location of the image is.

; 1 for centre of image, 0 for top left of image

; $x $y - Return the x and y location of the image

; $transparency - TRANSBLACK, TRANSWHITE or hex value (e.g. 0xffffff) of

; the color to be used as transparency can be omitted if

; not needed

;

; Return Value(s): On Success - Returns 1

; On Failure - Returns 0

;

;

;===============================================================================

Func _WaitForImageSearch($findImage,$waitSecs,$resultPosition,ByRef $x, ByRef $y,$tolerance,$transparency=0)

$waitSecs = $waitSecs * 1000

$startTime=TimerInit()

While TimerDiff($startTime) < $waitSecs

sleep(100)

$result=_ImageSearch($findImage,$resultPosition,$x, $y,$tolerance,$transparency)

if $result > 0 Then

return 1

EndIf

WEnd

return 0

EndFunc

;===============================================================================

;

; Description: Wait for a specified number of seconds for any of a set of

; images to appear

;

; Syntax: _WaitForImagesSearch

; Parameter(s):

; $waitSecs - seconds to try and find the image

; $findImage - the ARRAY of images to locate on the desktop

; - ARRAY[0] is set to the number of images to loop through

; ARRAY[1] is the first image

; $tolerance - 0 for no tolerance (0-255). Needed when colors of

; image differ from desktop. e.g GIF

; $resultPosition - Set where the returned x,y location of the image is.

; 1 for centre of image, 0 for top left of image

; $x $y - Return the x and y location of the image

; $transparent - TRANSBLACK, TRANSWHITE or hex value (e.g. 0xffffff) of

; the color to be used as transparent; can be omitted if

; not needed

;

; Return Value(s): On Success - Returns the index of the successful find

; On Failure - Returns 0

;

;

;===============================================================================

Func _WaitForImagesSearch($findImage,$waitSecs,$resultPosition,ByRef $x, ByRef $y,$tolerance,$transparency=0)

$waitSecs = $waitSecs * 1000

$startTime=TimerInit()

While TimerDiff($startTime) < $waitSecs

for $i = 1 to $findImage[0]

sleep(100)

$result=_ImageSearch($findImage[$i],$resultPosition,$x, $y,$tolerance,$transparency)

if $result > 0 Then

return $i

EndIf

Next

WEnd

return 0

EndFunc

; find recycle bin if it is in the top left corner of screen

; change 2nd argument to 0 to return the top left coord instead

$result = _ImageSearchArea("recycle.bmp",1,0,0,200,200,$x1,$y1,0,0x000000) ;perfect black used as transparency

if $result=1 Then

MouseMove($x1,$y1,3)

MsgBox(0,"Found","Found a recycle bin with stuff in top left corner")

EndIf

line:

$result = _ImageSearchArea("recycle.bmp",1,0,0,200,200,$x1,$y1,0,0x000000) ;perfect black used as transparency

why it not found "recycle.bmp" (this bmp file have some transparency 0x000000)

someone tell me about transparency parameter?

recycle.bmp

Link to comment
Share on other sites

  • 2 weeks later...
  • 4 weeks later...

Hi, nice project!

And my first question on this forum. There are some identical images on the screen. _ImageSearchArea returns the coordinates of 1st image it recognizes. Is it possible to get array with all the coordinates of identical images (without dll recompiling if possible ^) )?

Link to comment
Share on other sites

Hi, nice project!

And my first question on this forum. There are some identical images on the screen. _ImageSearchArea returns the coordinates of 1st image it recognizes. Is it possible to get array with all the coordinates of identical images (without dll recompiling if possible ^) )?

You can make UDF wrapper which will call this search function from DLL in a loop.

Everytime it find something you must add found coordinates to output array and shrink search area accordingly.

Link to comment
Share on other sites

  • 1 month later...

I've written the following to use it in my projects and figured you guys might find it useful, too:

; #FUNCTION# =======================================================================================================================================================================================================================================================
; Name...........:  _Wait_For()
;
; Description ...:  Waits for images, symbols or other graphics to be or to not be on the screen according to user definition
;                   This can be done by utilizing three different methods:
;
;   Method 1 ...:
;
;           Syntax ........:    _Wait_For($file, $left, $top , $right, $bottom, $duration, $not, $any_or_all)
;
;           Parameters ....:    $file       - name of the file holding the template
;                               $left       - left coordinate of search rectangle
;                               $top        - top coordinate of search rectangle
;                               $right      - right coordinate of search rectangle
;                               $bottom     - bottom coordinate of search rectangle
;                               $duration   - the duration for the timeout in milliseconds
;                               $not        - "not" or anything else; waits for an object to "not" be present in the search rectangle or to (anything else) be present in the search rectangle
;                               $any_or_all - can be omitted because it's irrelevant when searching for only one object
;
;           Return values .:    Success     - returns 1 if the object was found; this is what you'd want if you go with $not = 0
;                                           - returns -1 if the object was not found; this is what you'd want if you go with $not = "not"
;
;           Examples ......:    $result = _Wait_For("recycle_bin", 0, 0, 100, @DesktopHeight, 10000, 0); waits for the template in "recycle_bin" to appear within the specified rectangle during a 10 second timeout
;                               $result = _Wait_For("recycle_bin", 0, 0, 100, @DesktopHeight, 20000, 1); waits for the template in "recycle_bin" to NOT appear within the specified rectangle during a 20 second timeout
;
;                               Failure     - returns 0 if the function timed out
;
;
;
;   Method 2 ...:
;
;           Syntax.........:    _Wait_For("file 1, ..., file n", $top , $left, $bottom, $right, $duration, $not, $all)
;
;           Parameters ....:    $file       - name of the files holding the templates
;                               $left       - left coordinate of search rectangle
;                               $top        - top coordinate of search rectangle
;                               $right      - right coordinate of search rectangle
;                               $bottom     - bottom coordinate of search rectangle
;                               $duration   - the duration for the timeout in milliseconds
;                               $not        - "not" or anything else; waits for the objects to "not" be present in the search rectangle or to (anything else) be present in the search rectangle
;                               $any_or_all - "any" or "all"; specifies whether to wait for "all" the objects on the screen to be gone or present, or "any" of the objects to be gone or present
;
;           Return values .:    Success     - returns 1 if the object was found; this is what you'd want if you go with $not = 0
;                                           - returns -1 if the object was not found; this is what you'd want if you go with $not = "not"
;
;           Examples ......:    $result = _Wait_For("recycle_bin, control_panel", 0, 0, 100, @DesktopHeight, 10000, 0, "any"); waits for the templates in "recycle_bin" OR "control_panel" to appear within the specified rectangle during a 10 second timeout
;                               $result = _Wait_For("recycle_bin, control_panel", 0, 0, 100, @DesktopHeight, 20000, 0, "all"); waits for the templates in "recycle_bin" AND "control_panel" to appear within the specified rectangle during a 20 second timeout
;                               $result = _Wait_For("recycle_bin, control_panel", 0, 0, 100, @DesktopHeight, 30000, 1, "any"); waits for the templates in "recycle_bin" OR "control_panel" to NOT appear within the specified rectangle during a 30 second timeout
;                               $result = _Wait_For("recycle_bin, control_panel", 0, 0, 100, @DesktopHeight, 40000, 1, "all"); waits for the templates in "recycle_bin" AND "control_panel" to NOT appear within the specified rectangle during a 40 second timeout
;
;                               Failure     - returns 0 if the function timed out
;
;
;
;   Method 3 ...:
;
;           Syntax.........:    Dim $aFiles[n][10] = [[$file 1, $top, $left, $bottom, $right, $transparency, $shade_variation, $runs, $behavior, $not], _
;                                                     [  ...  , $top, $left, $bottom, $right, $transparency, $shade_variation, $runs, $behavior, $not], _
;                                                     [$file n, $top, $left, $bottom, $right, $transparency, $shade_variation, $runs, $behavior, $not]]
;                               $result = _Wait_For($aFiles, $duration, $any_or_all)
;
;           Parameters ....:    $aFiles - an array of file names of .BMP files containing the templates to look for
;                                   $file               = name of the file holding the template
;                                   $left               = left coordinate of search rectangle
;                                   $top                = top coordinate of search rectangle
;                                   $right              = right coordinate of search rectangle
;                                   $bottom             = bottom coordinate of search rectangle
;                                   $transparency       = "TRANSBLACK" or "TRANSWHITE", including the '"'; for hex values, use e.g. "TRANS00FF00"
;                                   $shade_variation    = a number between 0 and 255 to indicate the allowed number of shades of variation of the red, green, and blue components of the object
;                                   $runs               = how often the template should be looked for until a result is returned; depends on $behavior
;                                   $behavior           = 0: exit on any find
;                                                         1: exit on first positive find
;                                                         2: exit after $runs consecutive finds
;                                   $not                = "not" or anything else; waits for the objects to "not" be present in the search rectangle or to (anything else) be present in the search rectangle
;
;                               $duration   - the duration for the timeout in milliseconds
;                               $any_or_all - "any" or "all"; specifies whether to wait for "all" the objects on the screen to be gone or present, or "any" of the objects to be gone or present
;
;           Examples ......:    Dim $aFiles[2][10] = [["recycle_bin", 0, 0, 200, 200, "transblack", 10, 3, 1, "0", "all"], _            ; requires "recycle_bin" to be present for the function to report success
;                                                   ["control_panel", 800, 600, 1024, 768, "transblack", 10, 3, 1, "not", "all"]]       ; requires "control_panel" to not be present for the function to report success
;                               $result = _Wait_For($aFiles, 30000, "all")
;
;           Return values .:    Success     - returns the input array expanded by 5 result columns
;                                   $result[n][0] = name of the file holding the template
;                                   $result[n][1] = left coordinate of search rectangle
;                                   $result[n][2] = top coordinate of search rectangle
;                                   $result[n][3] = right coordinate of search rectangle
;                                   $result[n][4] = bottom coordinate of search rectangle
;                                   $result[n][5] = "TRANSBLACK" or "TRANSWHITE", including the '"'; for hex values, use e.g. "TRANS00FF00"
;                                   $result[n][6] = a number between 0 and 255 to indicate the allowed number of shades of variation of the red, green, and blue components of the object
;                                   $result[n][7] = how often the template should be looked for until a result is returned; depends on $behavior
;                                   $result[n][8] = behavior used
;                                   $result[n][9] = "not" or not ;)
;                                   $result[n][10] = 1 if template was found
;                                                    0 if template was not found
;                                   $result[n][11] = 0 if template was not found
;                                                    leftmost coordinate of the template if template was found
;                                   $result[n][12] = 0 if template was not found
;                                                    topmost coordinate of the template if template was found
;                                   $result[n][13] = 0 if template was not found
;                                                    x axis center of the template if template was found
;                                   $result[n][14] = 0 if template was not found
;                                                    y axis center of the template if template was found
;
;                               Failure     - returns 0 if the function timed out
;
; Author ........: Sven
; Dependencies...: _Imagesearch(); imagesearchdll.dll
; ==================================================================================================================================================================================================================================================================

This one makes use of my _Imagesearch() interpretation from one or two pages ago, so be sure to stick those two together. Also note that that _Imagesearch() function adds .bmp to the file name passed to it and looks for that in the \pics\ subdirectory of where it's saved to, so you might want to change that part of the _Imagesearch() function to fit your needs.

_Wait_For() worked with the examples shown in the description. I'll run it through more thorough tests today or tomorrow.

Any feedback is greatly appreciated.

History:

- added $any_or_all for array mode. Why I didn't put this in initially is beyond me... senility perhaps :)

_wait_for.zip

Edited by Sven
Link to comment
Share on other sites

I downloaded ImageSearch zip (which includes .au3 file, ImageSearchDLL.dll and demo.au3) and ran the demo and it gave me an error saying:

imagesearch\ImageSearch.au3 (40) : ==> Subscript used with non-Array variable.:

The code in question is:

if $result[0]="0" then return 0

So I replaced it with:

; If error exit
  If @error Then
    Return 0
  EndIf

This is the correct fix right? Well the problem is that I can't get anything to return anything but error, the DllCall never succeeds in finding anything. Is there anything I'm doing wrong?

I tried a few files with different file formats and different small portion of the screenshot and nothing seems to work. I even tried a file that's all white and ran it while displaying blank web browser that has the same hex color white, no dice.

Edited by Ryuho
Link to comment
Share on other sites

Thanks for the response but "Then...", then what?

I tried what you mentioned and the isarray method came back false. it's not an array. Either way if it throws an @error there is something wrong, don't you think?

I'm going to try calling the method directory:

$result = DllCall("ImageSearchDLL.dll","str","ImageSearch","int",$x1,"int",$y1,"int",$right,"int",$bottom,"str",$findImage)

See where that takes me...

Link to comment
Share on other sites

The point is you can only access a variable like an array if it really is an array. If it's just a string, you get the error you mentioned. To make sure the function proceeds only if the returned value is an array, you need to use "if isarray()". If It's an array, then start doing whatever you want. Without "if isarray()", if it's not an array, you get the error you mentioned, meaning that Imagesearch probably couldn't find what it was looking for.

#include <array.au3>

$result = DllCall("ImageSearchDLL.dll", "str", "ImageSearch", "int", $x1, "int", $y1, "int", $right, "int", $bottom, "str", $findImage)
If IsArray($result) Then
    _ArrayDisplay($result)
Else
    MsgBox(0, "imagesearch returned: " & $result)
EndIf
Edited by Sven
Link to comment
Share on other sites

Yeah, I gathered that... it's suppose to return the x,y of the found pattern.

My real question is, why isn't it working, is there a common reason why this module fails to work that I missed while googling. Sorry if I wasn't clear initially.

Link to comment
Share on other sites

I don't really have an idea, sorry. Works like a charm for me.

Made you an example. Open the picture file included in the archive with an image viewer that allows for 1:1 viewing (I was using IrfanView), and place it so it isn't covered by any other window. Then run test.au3.

You may want to replace the line

$findimage = "user_green.bmp"

with

$file = "user_green.bmp"
$findimage = "*" & "transblack" & " " & "*" & 30 & " " & $file

to get the transparency of the image recognised by the imagesearchdll.dll. This is to make it work with image displaying programs (like most webbrowsers) that handle transparency properly.

test imagesearch.zip

Edited by Sven
Link to comment
Share on other sites

  • 3 weeks later...

I have modified The image search wrapper so the error is shown.

Func _ImageSearchArea($findImage, $resultPosition, $x1, $y1, $right, $bottom, ByRef $x, ByRef $y, $tolerance, $transparency = 0)
    ;MsgBox(0,"asd","" & $x1 & " " & $y1 & " " & $right & " " & $bottom)
    If not ($transparency = 0) Then $findImage = "*" & $transparency & " " & $findImage
    If $tolerance > 0 Then $findImage = "*" & $tolerance & " " & $findImage
    MsgBox(262144,'Debug line ~' & @ScriptLineNumber,'Selection:' & @lf & 'Before Call' ) ;### Debug MSGBOX
    Local $result = DllCall("ImageSearchDLL.dll", "str", "ImageSearch", "int", $x1, "int", $y1, "int", $right, "int", $bottom, "str", $findImage)
    Local $err = @error
    MsgBox(262144,'Debug line ~' & @ScriptLineNumber,'Selection:' & @lf & 'After Call' ) ;### Debug MSGBOX
    if $err > 0 Then
     MsgBox(262144,'Debug line ~' & @ScriptLineNumber,'Selection:' & @lf & '$err' & @lf & @lf & 'Return:' & @lf & $err) ;### Debug MSGBOX
     return 0
    EndIf
    ; If error exit
    If $result[0] = "0" Then
        Return 0
    EndIf

Regardless of which DLL I use, I always get the @error=1 "unable to use the DLL file"

I am running under Windows 7. I am using Auto-IT with out the 64 bit features.

My goal is to find buttons to click on an any application. Especially those where control click will not work, since AutoIt Window Info shows nothing for the control name.

In this particular instance I what to control Fraps.

Thanks in advance

ImageSearch_hacked.au3

tbodine

Link to comment
Share on other sites

  • Jos locked this topic
Guest
This topic is now closed to further replies.
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...