Lasker

A new image search solution

18 posts in this topic

#1 ·  Posted (edited)

Hello,

Im a new AutoIt user, im from Spain and i would like to share here my own image search detection method.

What's about it?

It's not perfect, but with it you don't need to use any DLL file. You don't need to distribute the image file with your final AU3 Script, or Exe if compiled. The image size does not affect to the detection speed. It's bug free (i believe).

You can find here as an attach file, a distro package with all you need, and it includes a simple how-to guide.

Im going to paste here the how-to guide:

How to use this AutoIt image detector:

Point 1:

Install JRE (Java Runtime Environment), without this you can't execute Java Applications (.jar files). Open the Java tool named AISD with double click on it, or "java -jar aisd.jar" in the cmd. In AISD, open the image you want to detect in your script. Remember: the upper left pixel of the image, MUST be a not common color in the background if you want a fast detection. Then you will get a script file if you want, or only a clipboard copied version, full ready to paste into your main script because this one comes with extra help codes. Anyway, the generated code is the $sign definition of the image (calm down, point 3 explains what is the $sign and it's easy).

Point 2:

Paste the mentioned code in your main script, and try to follow the sample_usage script. If you know how to write au3 codes, it should be very easy. In general terms, all you need is to include image_search.au3, and then you can use the func image_search($sign, $width, $height). You need also the get_image_sign() function, that calculates the first parameter for image_search(). The get_image_sign() function is the one you have generated at point 1.

Point 3:

image_search() returns the center point of the target image. The func needs the $sign from the image, width and height. This $sign is the "footprint" of the image, in the form of an array of 9 integers. In most of the cases the $sign should be enough to identify the image, this is useful because using the $sign instead of the real image let you get a small script and a fast detection. You can also delete the image once you get the $sign. This $sign has always the same size (those 9 integers, it does not matter if the original image is small or big). How can you define the $sign for a certain image?, you don't need to generate the $sign, AISD does (point 1). Follow the sample_usage script and you will get it.

Files in this distro package:

* AISD.jar: The already explained Java app.

* image_search.au3: The library or module with the resources that lets you detect images.

* sample_usage.au3: The mentioned sample_usage script. It contains the smallest example, but i think it should be ok for learning purposes.

* sample_usage.exe: The sample_usage.au3 script compiled (au3 -> .exe). You can execute this to see the image_search() func working.

* sample_target.png: The image to be detected in the sample_usage. You can delete this file and the script will be fine.

* OffsetCalc.jar: It's only a Java tool to calculate the offset to transform an XY point into another one. It's useful if you want to use apply_offset function (from image_search.au3). Both things are only necessary when it's hard to find a version of the target image with upper left pixel in the way mentioned. As you can imagine, the idea is to detect another image and then apply the offset to get the real coordinates of the original target image.

Happy Detections

Lasker image_search.zip

Edited by Lasker
2 people like this

Share this post


Link to post
Share on other sites



Hi @Lasker Thank You very much for you sharing I tried it and found it very useful but why i cant detect another created "PNG" images I'm able to detect example image only !

 

Share this post


Link to post
Share on other sites

Thank you very much it's working now I forgot to paste copied code ..

It's very easy to use thank you again for making Image search Easier but is this working in other Pc resolution ?

Share this post


Link to post
Share on other sites

@Lasker - just popped in to thank for sharing your solution. Well documented, easy to use and lightning fast. Good job mate!

Share this post


Link to post
Share on other sites

@Lasker - thank you for this, I tested and it's clearly working.

Two questions please:

1. Does your image search support color variations? (e.g. imagesearch.au3 has smth like this - it can find image even if not perfectly same colors, maybe big image is from lower quality jpg etc).

2. Is it possible to find different sizes of the same image/pattern? This would be indeed great, imagesearch.au3 only solution is to have multiple target images.

I assume your 9 signs could be multiplied by the number of sizes to search, but not sure of speed then.

Thanks anyway

1 person likes this

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

Hola :) Nice, going to test, I would like to have image search which can scale images, so no worry about browser zoom  or different screen resolutions :)
Also like autog ask, tolerance would be great :)
What about images with transparent parts, where part of image can be a bit different as they have ex animated background ?
Thx anyway, great idea and good job man, best greetings from Poland :)

Edited by WiValdiBB

Share this post


Link to post
Share on other sites

Ok first of all it does not support dual screens. i used 2 screens with extrended view and it could not find the logo.I suspected it was because of 2 screens so i ddisconnected one and now it wiorks. But its kinda slow the image i used was a red button and it was a black background so I thought it would go faster if i had a red    pixel in the left uper corner but NO still slow..

 

 

Is it possible to scan an determined surface??

Share this post


Link to post
Share on other sites

Is this checked by anyone for effectivity (greater than imagesearch library?) Thanks

Share this post


Link to post
Share on other sites

#9 ·  Posted (edited)

On ‎18‎/‎05‎/‎2016 at 2:24 PM, Lasker said:

 

Can you direct me on where to do some research on methods to find "uncommon", or even better, "unique" pixel in an image sample? Thank you in advance.

Edited by Melba23
Huge quote removed

Share this post


Link to post
Share on other sites

#10 ·  Posted

r4sc4l,

Welcome to the AutoIt forums.

But when you post in future, please use the "Reply to this topic" button at the top of the thread or the "Reply to this topic" editor at the bottom rather than the "Quote" button - others know what they wrote and it just pads the thread unnecessarily.

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#11 ·  Posted

this doesn't work for me.

Share this post


Link to post
Share on other sites

#12 ·  Posted

@dwaynek "It's broke" doesn't help anyone help you with your issue. How about a detailed explanation as to what you tried, and what is or is not working, including screenshots, error codes, etc.?


√-1 2^3 ∑ π, and it was delicious!

Share this post


Link to post
Share on other sites

#13 ·  Posted (edited)

Hi,

I certainly can't get this working at all. I'm on Windows 10 and it throws up saying it failed to find the image.

I've tried it with the website open on every screen and I've also replaced the image with a fresh copy of the logo yet it still fails to find it... findBMP & ImageSearch both work just I want to try this one to see how much faster/slower it is and how resource intensive it is.

Edited by Hyflex

Share this post


Link to post
Share on other sites

#14 ·  Posted

2 hours ago, Hyflex said:

findBMP & ImageSearch both work just I want to try this one to see how much faster/slower it is and how resource intensive it is.

If it doesn't work, then I guess it won't work for you to compare against, try something else or figure out why this one doesn't work.

No one is going to be able to help you because you've given absolutely zero information. Imagine you were trying to figure out how to help you with the total lack of information received, then you will realize what your post actually tells us.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

#15 ·  Posted

2 minutes ago, BrewManNH said:

If it doesn't work, then I guess it won't work for you to compare against, try something else or figure out why this one doesn't work.

No one is going to be able to help you because you've given absolutely zero information. Imagine you were trying to figure out how to help you with the total lack of information received, then you will realize what your post actually tells us.

How can I possibly give more information... I've told you everything... what else would you require?

I gave you my OS, I told you that I tried the example script, I told you that I tried the example script with a fresh image. I told you what didn't work / what the result was.

Share this post


Link to post
Share on other sites

#16 ·  Posted

You told us, you're on Windows 10 and that it didn't find the image.

You said nothing about which script you used to find the image, or what error messages (if any) were returned. NOW you're telling us that you used the example script. Did you install the JRE?


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

#17 ·  Posted

12 hours ago, BrewManNH said:

You told us, you're on Windows 10 and that it didn't find the image.

You said nothing about which script you used to find the image, or what error messages (if any) were returned. NOW you're telling us that you used the example script. Did you install the JRE?

I told you there error it throws...

15 hours ago, Hyflex said:

and it throws up saying it failed to find the image.

From this you can see that it has failed an if/else argument and that also tells you that JRE is installed otherwise it would have thrown an exception which also tells you that JRE is installed otherwise it would have thrown an exception.

I also told you that I double checked the images were correct by replacing the image with a new copy just incase it was unable to find due to having slighty different desktop resolution/brightness... etc.

I also told you that I am capable of using both findBMP & ImageSearch therefore I probably checked most of the basic reasons that it might be failing...

Share this post


Link to post
Share on other sites

#18 ·  Posted

Great share. But this doesn't work for me when trying the other images

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • SimTheNo1
      By SimTheNo1
      I am busy with building a solution for change monitoring of VOIP call program and to be properly automated means among others need for some limited OCR functionality. Current works of others are way too much overkill for this case what makes the need to build it myself. But to do it properly I significantly have to increase my knowledge about digital graphics management.
      For now I already have discovered here and there some mind blowing  Autoit miracles what can be achieved with Windows own possibilities to manipulate that what is output to the monitor. And as far I can judge there are 2 options to process graphics without use of any external libraries like ImageMagick, FreeImage and so on. These are:
       WinAPI  GDIPlus It is for me quite obvious to have various holes in general understanding of graphics and it is once more very clear what advantages gives consistent general study in 1 or another official institutions like University.  Cause there you are introduced into certain domain of knowledge in a way which usually has been perfected over long period of time. So you are not overloaded by stuff which requires a certain amount of information to be initially clear for you. For example, before starting to solve physics you first learn to read, count and so on and then move to subjects like physics.
      Though in my case opportunity to study in such educational system I had only for 8 years, from my 7th to 15th year of age in the country that was falling apart now Ukraine but used to be USSR, was all what it was. After have immigrated to Netherlands possibilities to study further haven't occur. And this therefor causes often various implications when going deep in that or another field of practical knowledge acquiring for any needed physical result, like programming to perform enormous amount of tasks. In this particular case automating VOIP call program.
      Anyway, right now I think the best direction to move is to concentrate on as basic as possible image management and if someone would maybe explain in general what is a pixel will definitely help. Particularly I am very curious about how to do picture manipulations in Autoit. Especially would help a lot to produce eventually following functions:
      createImage($imageFileName, $width, $height, $color) readImagePixel($imageFileName, $x, $y)  writeImagePixel($imageFileName, $x, $y) I do not know exactly how image is handled in computer but preferably above mentioned functions should deal with so far possible origin of graphics creation on computer. But nevertheless I definitely would love to hear any proposition for solution.
      The problem with explaining screenshots:
       VOIPConnect full Window  Part of Window with control to monitor for changes  Exact location of area where actual changes occur and have to be processed  It comes down to a rectangle of approximately 51 pixel wide and 7 pixel high. In fact if I get to learn as far as to be able exactly read, write and compare 2 images consistently across different computers I could narrow down then even further the area to watch as little as a square of 2-5 pixels wide. 
      To finish here is last detail about particularly no need for ultra fast solution at all. This because it is needed only once when it is first run on a new computer and to have to wait few minutes while it is being set instead of just a few seconds make no sence.
      This is it and what I too think to do beside this very particular case is to purify out beautiful generic Autoit functions for core image manipulation by using WinAPI or/and GDIPlus. 
    • wakillon
      By wakillon
      Mp3SearchEngine v2.0.0.6

      May be some of you know Songr .
      This script do the same job, it can find more mp3 files but is not as fast as Songr.
       
       



      Sites used are music search engine Websites designed for LEGAL entertainment purposes only.
      Thanks to Brett Francis, Prog@ndy and Eukalyptus for >Bass Udf, trancex for >WinHttp Udf and the AutoIt Community for his help.

       
      Changes of v1.0.8.5
       
      Three websites replaced cause they are dead or use now js.
      All search engines updated ( not without difficulties for audiodump)
      I use RAGrid.dll for the first listview (more fast and stable, but with some inconvenients to manage the no-edit of cells)
      Input queries are saved ( the twenty latest)
      I use now an mp3 pre-Load management before playing and a double progressbar for visualize pre-load and play, where you can click for directly go play in the loaded part.
      Most includes needed are embedded and all external files are embedded in script with >BinaryToAu3Kompressor .
      Multi downloads available with embedded downloader.exe
       
      Changes of v1.0.8.8
      Search on audiodump and myfreemp3 fixed.
      New buttons.
      Added Gui Menu.
      Titles are no more editable.
      New "About" with >TaskDialog (Thanks Prog@andy)
      Query button permit now to check / uncheck all checkboxes
      And some few fixes and cleaning.
      Really more stable now.
      Changes of v1.0.9.2
      Dilandau is replaced by mp3chief and mp3ili by mp3clan 
      Search on mp3juices, baseofmp3 and soundcloud fixed.
      Soso now provide m4a (aac) instead of mp3 ( m4a can be played by MSE)
      Added possibility to encode automaticaly to mp3, aac or ogg ( at the end of download) using bassenc.dll and command line tools : lame, faac and oggenc.
       
      Changes of v1.0.9.3   mp3skull fixed mp3chief fixed myfreemp3 fixed mp3clan changed to tusmp3  mp3juices changed to emp3world baseofmp3 changed to imp3 and some minor improvements.  
      Version 2.0.0.6
      Most previous websites used are dead or have changed the way to get links, 
      so instead of try to repair the previous version, i have created a complete new version.
      The main tendency is the simplification :
      Only one website : audiodump (Up to 500 results by request)
      Script use now the little pearl created by Ward : curl.au3
      It permit to create tasks (get source and get multi mp3) in asynchronous mode.
      So now, no need to use several executables and no more gui who do not respond in case of connection problems. 
      Script use Bass.dll X86 loaded in memory for play songs.
      Result is light and fast, but don't abuse of audiodump servers who are not beasts of race.
      Warning : For avoid errors with curl.au3, you'll need to comment the line 63 : ;~ #Include <BinaryCall.au3>
      @AutoItX64 not supported and only tested on Win7X64 and Win8.1X64.
      As your browser, use Ctrl+w for remove the current Tab.(if there is no search or download running from it)
      And also Ctrl+q for set/remove Gridlines.
      Events are displayed to the bottom of the Gui.
       
      Version 2.0.1.1
      Added a Paste Button.
      Querry list is now correctly saved.
      Querry Combo is now sorted in alphabetical order
      After a 'No match', the next search will use the previous empty listview.
      Bug when removing tabs is corrected.
      Added string correction for the request that, in the previous version, was not always able to return a correct result.
       
      A big thanks to Ward for his great UDF, and Nina my favorite tester, (who between us is also my third daughter), for his precious advices .
      previous downloads : 1703
       
      As there is no more script downloads count, source and executable are available in the downloads section

      Enjoy ! 
      July 2017 Project Discontinued due to website changes
    • Ian_Mac
      By Ian_Mac
      Func dg_get_images() $x1=0 $y1=0 ;MsgBox(0,"start","start") do $result = _ImageSearch("C:\Users\Server\Desktop\my project\image.png",1,$x1,$y1,0) until $result = 1; if $result=1 Then MouseMove($x1,$y1,3) MouseClick("left", $x1,$y1, 1) WinWait("[Title:TightVNC File Transfer]", "", 10) EndIf  this code works when i search the image, but when i change the location of the desktop icon it can't find the desktop icon anymore.   i wanna try image search area. but i don't know how to use it because i cant understand the parameter for the image search area.  please help me how to solve it thank you in advance.
    • Mr_Was_geht_sie_das_an
      By Mr_Was_geht_sie_das_an
      Hi Autoit Community,
      do someone know, if is possible to creat an 3D-ModelSearch script ?
    • jonson1986
      By jonson1986
      Hello,
      I've thousands of URLs to check them these are safe, malware infected or any other type of error, that's why I searched and found Google Safe browsing API with this we can send HTTP GET request so different code will return to make us clear is it our sent URL is safe or not. Please guide me how can i make this possible I know basics of Auotit but don't know how to use this API to fulfill above mentioned purpose. Your help will be much appreciated. Thanks
      Here is API URL;
      https://developers.google.com/safe-browsing/v3/lookup-guide