Sign in to follow this  
Followers 0

Check image on screen and get closest one from library

3 posts in this topic

Hello all.

I'm trying to do something simple, but I can't manage to find where to start since I can have multiple possibilities and not sure what is the best one :

  1. Taking screen capture (I can do that)
  2. On this capture, I'll look for 10 particular regions that will give me 10 images of 40px * 40px (I can do that)
  3. Then what I want is for each of these regions, find on my library of ~140 images which one is the closest one to my region (= the match)


What do you think is the best approach ? Ican easily do step 1 and 2, however for the third step :

  • I can't use memcmp because they won't be exactly the same (original library can differ a very little bit from what I'll get from my screen capture)
  • There is no text, only drawing
  • Should I do a pixelSum of what I get on screen and do the same via GDIPlus for each of my images in the library?


Thanks a lot,



Share this post

Link to post
Share on other sites

#2 ·  Posted (edited)

Tried this ;

And this :

Without a lot of success, bot scripts fail for different reason and are quite old.

Edited by timmalos

Share this post

Link to post
Share on other sites
1 hour ago, timmalos said:

I'm trying to do something simple

Errm, no, not at all. Image recognition and interpretation is computationally very hard, otherwise companies wouldn't be spending hundreds of millions of dollars on developing better algorithms (self-learning, AI, PCA) to do something you take for granted such as distinguishing between the faces of your family members, for example.

That said, here are some simple suggestions, or you could use ImageMagick's compare functionality.

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  
Followers 0

  • Similar Content

    • corgano
      By corgano
      What I am trying to accomplish is to simplify a .PNG image and convert it to use only a specified pallet of colors. Eg if I have a pallet of 0xFF0000, 0x00FF00, 0x0000FF, it would change every pixel in the image to one of those three colors by whichever color is closest.
      The simplest method I have tried is simply converting the image to a 16 bit bitmap, and that does simplify the pallet of the image, but I want to be able to specify what colors it specifies to. The other problem is the resulting saved .BMP file is much larger (file size) than the original, and I don't know if it's possible to make a 16 bit jpg.
      It looks like _GDIPlus_BitmapConvertFormat is the solution but the help file doesn't go into how to use the pallet and I couldn't find an example that did what I needed. Can anyone help me?
    • timmalos
      By timmalos
      Hello all . i'm trying to build a tool to generate some pictures/graphs automatically based on some variables. I have completed the first part and I'm struggling now on what I thought would be the easiest part :
      In my tool there is a "preview" button, which helps to display on a new children GUI the generated picture. What I do in simplified :
      Global $g_hGraphics = _GDIPlus_GraphicsCreateFromHWND($g_hGUI) ;Then, I'll add multiple objects to this graphic, for example $Temp = _GDIPlus_ImageLoadFromFile($temp[1]) _GDIPlus_GraphicsDrawImage($g_hGraphics,$temp)  
      I'm then able to have a preview GUI showing all my elements perfectly. This works but now I want to be able to really generate it and have a ".png" file written to disk.
      What I tried without success:
      $g_hBitmap = _GDIPlus_BitmapCreateFromGraphics($previewWidth, $previewHeight, $g_hGraphics) _GDIPlus_ImageSaveToFile($g_hBitmap,$OUTPUTDIR & "\generated\test-"&@HOUR&@MIN&@SEC&".png") But what I write to file is a transparent PNG without anything.
      I'm pretty sure I'm missing something obvious there, I need your help
      NB : If this change your answer, my goal will be to generate transparent png, meaning I'll add multiple objects/pictures/texts on a transparent png background and if there are some "transparent holes" I need them to be transparent in the generated png.
      Thanks a lot for any help !
    • TheNorwegianUser
      By TheNorwegianUser
      I don't get why the images I take turns up like this, but it seems to me like the screenshot doesn't go from the actual positions specified. It's weird, cause what happends is that when I for example move the mouse over the number 25 on SciTE, the image shows that it's over 19... Anyone knows what I'm doing wrong?
      #include <screencapture.au3> #include <WindowsConstants.au3> Hotkeyset("{ESC}", "Terminate") $gui = GUICreate("", 100, 800, 100, 0, $WS_POPUP, $WS_EX_TOPMOST) _GDIPlus_Startup() $graphics = _GDIPlus_GraphicsCreateFromHWND($gui) GUISetState(@SW_SHOW, $gui) While 1 $scr = _ScreenCapture_Capture("", 0, 0, 100, 800) $bitmap = _GDIPlus_BitmapCreateFromHBITMAP($scr) _WinAPI_DeleteObject($scr) _GDIPlus_GraphicsDrawImage($graphics, $bitmap, 0, 0) WEnd Func Terminate() _GDIPlus_GraphicsDispose($graphics) _GDIPlus_BitmapDispose($bitmap) _GDIPlus_Shutdown() Exit EndFunc  
    • LegitStack
      By LegitStack
      Been struggling with this one for a while. 
      when I do a _screencapture_capture call on a high resolution monitor (like my surface book) it gives me an image that has 2 problems:
      1. its in the wrong location on the screen
      2. it gives me a picture that is larger than the area of the screen I selected, though it only has the content of what I selected. 
      I was able to easily fix problem #1 by manually adjusting the x y coordinates to compensate for the amount of DPI scale I have.
      for instance if I'm 200% zoomed in the code looks like this:
      Local $bmp = _ScreenCapture_Capture("", $iX1*2, $iY1*2, $iX2*2, $iY2*2, false)  it's problem #2 that is the big problem. I'd like to attach a screen shot of what I'm talking about (see capture.png)

      Now I basically understand why this is happening. ScreenCapture grabs each pixel of the screen. This screen, being a high resolution, when its zoomed in adds up several pixels to make one on the screen. 
      This is a problem for me because I'm taking images of the screen and later looking for those exact images on the screen. if everything is blown up by an indeterminate amount (in my case 2x) then those images can't be found later on. 
      Does anyone know what I can do? 
      I tried resizing the images back down to no avail. 
      _GDIPlus_ImageResize and _GDIPlus_ImageScale don't work because they don't compress the pixels correctly. quality is lost. and the exact image isn't preserved, so I can't search for it later. (see capture1.png)

      Anyway, I'm about to give up, been on this problem for too long! does anyone know what I can do? Seems to me that the ideal solution would be to eventually have autoit add an argument to _screencapture_capture that lets you specify a DPI scale amount or something.  that can be pulled from the registry at HKEY_CURRENT_USER\control panel\desktop\windowmetrics\appliedDPI.
      But in the meantime, does anyone have any suggestions for how I can make my program compatible with 4k resolution monitors? I either need to take the screen capture like normal, then scale it down appropriately without losing quality, or I need to capture the screen in the first place like the human sees it. But I don't know how to do that either. 
      I'll post my relevant code here: (in my project I call ScreenCapture_DPI_Aware)
      #include <Security.au3> Func _GetAppliedDPI()   Local $aArrayOfData = _Security__LookupAccountName(@UserName)   If IsArray($aArrayOfData) Then   ;msgbox(64, "SID String = ", $aArrayOfData[0] & @CRLF)   ;msgbox(64, "Domain name = ", $aArrayOfData[1] & @CRLF)   ;msgbox(64, "SID type = ", _Security__SidTypeStr($aArrayOfData[2]) & @CRLF)     ;Local $AppliedDPI = RegRead("HKEY_USERS\" & $aArrayOfData[0] & "\Control Panel\Desktop\WindowMetrics", "AppliedDPI") Local $AppliedDPI = RegRead("HKEY_CURRENT_USER\Control Panel\Desktop\WindowMetrics", "AppliedDPI")     return $AppliedDPI   EndIf EndFunc Func GetScale()   $applied = _GetAppliedDPI()   if $applied == "" then     return 1   else     return $applied / 96   EndIf EndFunc Func ScreenCapture_Capture_DPI_Aware($sBMP_Path, $iX1, $iY1, $iX2, $iY2, $bool)   $R = GetScale() ;Raito   Local $bmp = _ScreenCapture_Capture($sBMP_Path, $iX1*$R, $iY1*$R, $iX2*$R, $iY2*$R, $bool) ;Scaling didn't work: ;_ScaleImage($bmp, $sBMP_Path, abs($iX2 - $iX1), abs($iY2 - $iY1), $R)   ;return _ScreenCapture_Capture($sBMP_Path, $iX1*$R, $iY1*$R, $iX2*$R, $iY2*$R, $bool) EndFunc ;Func _ScaleImage($bmp, $outimage, $w, $h, $scale) ; _GDIPlus_Startup() ;Get the encoder of to save the resized image in the format you want. ; Local $Ext = StringUpper(StringMid($outimage, StringInStr($outimage, ".", 0, -1) + 1)) ; $CLSID = _GDIPlus_EncodersGetCLSID($Ext) ; code found here : ; Local $sImgCLSID = _GDIPlus_EncodersGetCLSID("png") ;create CLSID for a JPG image file type ; Local $tGUID = _WinAPI_GUIDFromString($sImgCLSID) ;convert CLSID GUID to binary form and returns $tagGUID structure ; Local $tParams = _GDIPlus_ParamInit(1) ;initialize an encoder parameter list and return $tagGDIPENCODERPARAMS structure ; Local $tData = DllStructCreate("int Quality") ;create struct to set JPG quality setting ; DllStructSetData($tData, "Quality", 100) ;quality 0-100 (0: lowest, 100: highest) ; Local $pData = DllStructGetPtr($tData) ;get pointer from quality struct ; _GDIPlus_ParamAdd($tParams, $GDIP_EPGQUALITY, 1, $GDIP_EPTLONG, $pData) ;add a value to an encoder parameter list ; Local $gbmp = _GDIPlus_BitmapCreateFromHBITMAP($bmp) ; _WinAPI_DeleteObject($bmp) ; Local $gsbmp = _GDIPlus_ImageResize($gbmp, $w * $scale, $h * $scale) ;Local $ext = _GDIPlus_EncodersGetCLSID("PNG") ; _GDIPlus_ImageSaveToFileEx($gsbmp, $outimage, $sImgCLSID) ; _GDIPlus_BitmapDispose($gbmp) ; _GDIPlus_BitmapDispose($gsbmp) ; _GDIPlus_Shutdown() ;EndFunc
      Thanks for any help you can give me!!!
    • GtaSpider
      By GtaSpider
      Hey everyone,
      When I try to add blur to an Image with _GDIPlus_EffectCreateBlur and _GDIPlus_BitmapApplyEffect the blur is only horizontal, not horizontal and vertical, if the blur radius is <= 20. 
      Easy to test with the Image I attached and the default example from the helpfile of 
      _GDIPlus_BitmapApplyEffect The second Image I attached is the result and as you can see the cross is only blurred in one direction.  Does anyone have a clue why this happens and furthermore a solution for this problem? (I could just rotate the picture but then it would be twice as time consuming...)
      Thanks in advance,
      p.s.: Running Win 10 Pro with all updates