Jump to content
Sign in to follow this  
InunoTaishou

PixelSearch in bitmap (handle to a bitmap, bitmap, or hdc)

Recommended Posts

Here's the goal:

  • Create a handle to a bitmap object using _WinApi_PrintWindow (this is done and working properly, I can take a screenshot of the window even when it's hidden or offscreen, does not work minimized)
  • Be able to search for a pixel color in memory on the handle. I can get it to work if I create a Bitmap from an HBITMAP (_GDIPlus_BitmapCreateFromHBITMAP) and then go through each pixel and check it using _GDIPlus_BitmapGetPixel but it's too slow. I've tried doing the _WinApi_GetPIxel using an $hDC but it's much slower than GDI+ (GD+ takes about 20seconds to search for almost 500,000 pixels, GetPixel takes almost 60seconds)

I found the FastFind library but, honestly, it's sloppy and I'm wanting to search a defined area, not set the starting position and then search the rest of the window. I also found a few examples on the forums but I could not get them to work.

Here's the CaptureWindow function. I'm only wanting to capture the client area (Not the border around the client, hence the -16 for the width and the -38 for the height, gets rid of the title bar and the resize bars on the sides. Also I'm wanting to keep the coordinates relative to the window, so if you only want to capture the screen from 480, 200 to 680, 400, then the rest of the area is black on purpose)

#include-once
Func CaptureWindow(Const $iLeft = 0, Const $iTop = 0, Const $iWidth = -1, Const $iHeight = -1, Const $hWindow = WinGetHandle("[Active]"))
    Local $rect_window = WinGetPos($hWindow)
    Local $hDC = _WinAPI_GetWindowDC($hWindow)
    Local $hDestDC = _WinAPI_CreateCompatibleDC($hDC)
    Local $hBitmap = _WinAPI_CreateCompatibleBitmap($hDC, $rect_window[2] - 16, $rect_window[3] - 38)
    Local $hDestSv = _WinAPI_SelectObject($hDestDC, $hBitmap)
    Local $hSrcDC = _WinAPI_CreateCompatibleDC($hDC)
    Local $hBmp = _WinAPI_CreateCompatibleBitmap($hDC, $rect_window[2] - 16, $rect_window[3] - 38)
    Local $hSrcSv = _WinAPI_SelectObject($hSrcDC, $hBmp)
    
    _WinAPI_PrintWindow($hWindow, $hSrcDC, True)
    
    If ($iWidth > 0 and $iHeight > 0) Then
        _WinAPI_BitBlt($hDestDC, $iLeft, $iTop, $iWidth, $iHeight, $hSrcDC, $iLeft, $iTop, $MERGECOPY)
    Else
        _WinAPI_BitBlt($hDestDC, $iLeft, $iTop, $rect_window[2] - 16 - $iLeft, $rect_window[3] - 38 - $iTop, $hSrcDC, $iLeft, $iTop, $MERGECOPY)
    EndIf
    
    _WinAPI_SelectObject($hDestDC, $hDestSv)
    _WinAPI_SelectObject($hSrcDC, $hSrcSv)
    _WinAPI_ReleaseDC($hWindow, $hDC)
    _WinAPI_DeleteDC($hDestDC)
    _WinAPI_DeleteDC($hSrcDC)
    _WinAPI_DeleteObject($hBmp)
    
    Return $hBitmap
EndFunc   ;==>CaptureWindow

And the first attempt for PixelSearch, using GDI+ (Fastest)

Func PixelSearchInhBitmap(Const ByRef $hHBmp, Const ByRef $color, Const $tolerance = 10, Const $iLeft = 0, Const $iTop = 0, $iWidth = -1, $iHeight = -1, Const $iStep = 1)
    Local $hBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hHBmp)
    Local $rbg_color = _ColorGetRGB("0x" & Hex($color, 6))
    Local $found_color = False
    Local $abscoord_color[2] = [0, 0]
    Local $red_low = 0
    Local $green_low = 0
    Local $blue_low = 0
    Local $red_high = 0
    Local $green_high = 0
    Local $blue_high = 0
    
    $red_low = ($tolerance > $rbg_color[0] ? 0 : $rbg_color[0] - $tolerance)
    $green_low = ($tolerance > $rbg_color[1] ? 0 : $rbg_color[1] - $tolerance)
    $blue_low = ($tolerance > $rbg_color[2] ? 0 : $rbg_color[2] - $tolerance)
    $red_high = ($tolerance > 255 - $rbg_color[0] ? 255 : $rbg_color[0] + $tolerance)
    $green_high = ($tolerance > 255 - $rbg_color[1] ? 255 : $rbg_color[1] + $tolerance)
    $blue_high = ($tolerance > 255 - $rbg_color[2] ? 255 : $rbg_color[2] + $tolerance)
    
    If ($iWidth = -1) Then $iWidth = _GDIPlus_ImageGetWidth($hBitmap)
    If ($iHeight = -1) Then $iHeight = _GDIPlus_ImageGetHeight($hBitmap)
    
    Local $start_time = TimerInit()
    For $iY = $iTop To $iHeight Step $iStep
        For $iX = $iLeft To $iWidth Step $iStep
            Local $get_pixel = _GDIPlus_BitmapGetPixel($hBitmap, $iX, $iY)
            If (@Error) Then ContinueLoop
            Local $pixel_color = _ColorGetRGB("0x" & Hex($get_pixel, 6))
            If (@Error) Then ContinueLoop
            
            If (($pixel_color[0] >= $red_low and $pixel_color[0] <= $red_high) and ($pixel_color[1] >= $green_low and $pixel_color[1] <= $green_high) and ($pixel_color[2] >= $blue_low and $pixel_color[2] <= $blue_high)) Then
                $found_color = True
                $abscoord_color[0] = $iX
                $abscoord_color[1] = $iY
                ExitLoop 2
            EndIf
        Next
    Next
    MsgBox("", "", TimerDiff($start_time) / 1000)
    _GDIPlus_BitmapDispose($hBitmap)
    
    If ($found_color) Then
        Return $abscoord_color
    Else
        Return SetError(-1, 0, 0)
    EndIf
EndFunc

Second attempt using _WInApi_GetPixel (Note. I replaced the Autoit function with my own where I replaced the string "gdi32.dll" with a handle to the opened DLL. In the hopes it would improve time. The time difference was not noticable)

Global $HWND_DLL_GDI32 = DLLopen("gdi32.dll")

Func PixelSearchInhDC(Const ByRef $color, Const $tolerance = 10, Const $iLeft = 0, Const $iTop = 0, $iWidth = -1, $iHeight = -1, Const $iStep = 1, Const $hWnd_window = WinGetHandle("[Active]"))
    Local $hDC = _WinAPI_GetWindowDC($hWnd_window)
    Local $rbg_color = _ColorGetRGB("0x" & Hex($color, 6))
    Local $found_color = False
    Local $abscoord_color[2] = [0, 0]
    Local $start_time, $end_time
    Local $red_low = 0
    Local $green_low = 0
    Local $blue_low = 0
    Local $red_high = 0
    Local $green_high = 0
    Local $blue_high = 0
    
    If (Not $hDC) Then
        Return SetError(1, 0, 0)
    EndIf
    
    If ($iWidth = -1 or $iHeight = -1) Then
        Local $rect_window = WinGetPos($hWnd_window)
        If ($iWidth = -1) Then $iWidth = $rect_window[2] - $iLeft
        If ($iHeight = -1) Then $iHeight = $rect_window[3] - $iTop
    EndIf
    
    $red_low = ($tolerance > $rbg_color[0] ? 0 : $rbg_color[0] - $tolerance)
    $green_low = ($tolerance > $rbg_color[1] ? 0 : $rbg_color[1] - $tolerance)
    $blue_low = ($tolerance > $rbg_color[2] ? 0 : $rbg_color[2] - $tolerance)
    $red_high = ($tolerance > 255 - $rbg_color[0] ? 255 : $rbg_color[0] + $tolerance)
    $green_high = ($tolerance > 255 - $rbg_color[1] ? 255 : $rbg_color[1] + $tolerance)
    $blue_high = ($tolerance > 255 - $rbg_color[2] ? 255 : $rbg_color[2] + $tolerance)
    
    $start_time = TimerInit()
    For $iY = $iTop To $iHeight Step $iStep
        For $iX = $iLeft To $iWidth Step $iStep
            Local $get_pixel = __WinAPI_GetPixel($hDC, $iX, $iY)
            If (@Error) Then ContinueLoop
            Local $pixel_color = _ColorGetRGB("0x" & Hex($get_pixel, 6))
            If (@Error) Then ContinueLoop
            
            If (($pixel_color[0] >= $red_low and $pixel_color[0] <= $red_high) and ($pixel_color[1] >= $green_low and $pixel_color[1] <= $green_high) and ($pixel_color[2] >= $blue_low and $pixel_color[2] <= $blue_high)) Then
                $found_color = True
                $abscoord_color[0] = $iX
                $abscoord_color[1] = $iY
                ExitLoop 2
            EndIf
        Next
    Next
    MsgBox("", "", TimerDiff($start_time) / 1000 & "s")
    
    If ($found_color) Then
        Return $abscoord_color
    Else
        Return SetError(1, 0, 0)
    EndIf
EndFunc

Func __WinAPI_GetPixel($hDC, $iX, $iY)
    Local $aRet = DllCall($HWND_DLL_GDI32, 'dword', 'GetPixel', 'handle', $hDC, 'int', $iX, 'int', $iY)
    If @error Or ($aRet[0] = 4294967295) Then Return SetError(@error, @extended, -1)
    ; If $aRet[0] = 4294967295 Then Return SetError(1000, 0, -1)

    Return __RGB($aRet[0])
EndFunc   ;==>__WinAPI_GetPixel

 

Edited by InunoTaishou

Share this post


Link to post
Share on other sites

Looked into _GDIPlus_BitmapLockBits, this seems to be a bit faster (about 8 seconds faster) but it still takes seconds to do the search.

#include <GDIPlus.au3>
#include <GUIConstantsEx.au3>
#include <MsgBoxConstants.au3>
#include <Color.au3>

_GDIPlus_Startup()

TestPixelSearch()

Func TestPixelSearch()
    Local $hImage = _GDIPlus_ImageLoadFromFile(@DesktopDir & "\CaptureWindow.jpg")
    Local $iW = _GDIPlus_ImageGetWidth($hImage), $iH = _GDIPlus_ImageGetHeight($hImage) ;get width and height of the image
    Local $hBitmap = _GDIPlus_BitmapCreateFromScan0($iW, $iH)
    Local $hContext = _GDIPlus_ImageGetGraphicsContext($hBitmap)
    _GDIPlus_GraphicsDrawImageRect($hContext, $hImage, 0, 0, $iW, $iH)

    Local $tBitmapData = _GDIPlus_BitmapLockBits($hBitmap, 0, 0, $iW, $iH, BitOR($GDIP_ILMWRITE, $GDIP_ILMREAD), $GDIP_PXF32RGB)
    Local $iScan0 = DllStructGetData($tBitmapData, "Scan0")
    Local $rbg_color = _ColorGetRGB(0xFFFC00)
    Local $tolerance = 10
    Local $red_low = ($tolerance > $rbg_color[0] ? 0 : $rbg_color[0] - $tolerance)
    Local $green_low = ($tolerance > $rbg_color[1] ? 0 : $rbg_color[1] - $tolerance)
    Local $blue_low = ($tolerance > $rbg_color[2] ? 0 : $rbg_color[2] - $tolerance)
    Local $red_high = ($tolerance > 255 - $rbg_color[0] ? 255 : $rbg_color[0] + $tolerance)
    Local $green_high = ($tolerance > 255 - $rbg_color[1] ? 255 : $rbg_color[1] + $tolerance)
    Local $blue_high = ($tolerance > 255 - $rbg_color[2] ? 255 : $rbg_color[2] + $tolerance)
    Local $tPixel = DllStructCreate("int[" & $iW * $iH & "];", $iScan0)
    Local $iPixel, $iRowOffset
    Local $found_pixel = False
    Local $abscoord_pixel[2] = [0, 0]

    Local $start_timer = TimerInit()
    For $iY = 0 To $iH - 1
        $iRowOffset = $iY * $iW + 1
        For $iX = 0 To $iW - 1 ;get each pixel in each line and row
            $iPixel = DllStructGetData($tPixel, 1, $iRowOffset + $iX) ;get pixel color
            Local $pixel_color = _ColorGetRGB("0x" & Hex($iPixel, 6))
            If (($pixel_color[0] >= $red_low and $pixel_color[0] <= $red_high) and ($pixel_color[1] >= $green_low and $pixel_color[1] <= $green_high) and ($pixel_color[2] >= $blue_low and $pixel_color[2] <= $blue_high)) Then
                $abscoord_pixel[0] = $iX
                $abscoord_pixel[1] = $iY
                $found_pixel = True
                ExitLoop
            EndIf
        Next
    Next
    Local $end_time = TimerDiff($start_timer) / 1000

    MsgBox("", "", $end_time & @CRLF & "Found pixel = " & $found_pixel)

    _GDIPlus_BitmapUnlockBits($hBitmap, $tBitmapData)

    ;cleanup resources
    _GDIPlus_ImageDispose($hImage)
    _GDIPlus_GraphicsDispose($hContext)
    _GDIPlus_BitmapDispose($hBitmap)
    _GDIPlus_Shutdown()
EndFunc

Will probably try to and rewrite this in C++ and give it a test in visual studio, perhaps it will be faster in a dll than in autoit. Also saw something about the WinApi GetDIBits but the apparently copies the whole bitmap into a buffer, not a pointer to a buffer so someone said it would be slow.

Does anyone know of any other way of checking pixels in memory or any way to make this faster? The Autoit Pixelsearch function takes miliseconds to complete. I did find the source code for another *ahem* scripting languages PixelSearch function and it looks like it uses GetDIBits.

Share this post


Link to post
Share on other sites

Why do you need to do pixel search?

Why not look for the control?

Why also do you need to do a snapshot then look at the picture?

Why not look at the screen directly? 

This seems a real difficult way to solve a problem when there are many simpler and more stable methods to do things. 

Share this post


Link to post
Share on other sites

Because it seems fun and an interesting way to learn more about bitmaps. The only problem with that is I don't know a whole lot about bitmaps and my knowledge of the WinApi is very limited. I was also working on a small image editor using GDI+ and being able to find colors in the bitmap will open up the possibility to search and replace colors in a bitmap. I just don't want it to take 20 seconds to go through and replace colors when programs like Photoshop and Gimp can do it in ms and Pixelsearch takes ms to find a pixel on the screen.

That snapshot allows to search in memory, so even if the window is hidden you can still search in it.

Edited by InunoTaishou

Share this post


Link to post
Share on other sites

So your saying you want to make a photo editor? I'm asking for none of your code shows anything to replace colors. It only shows how you want to search a client area. I'm assuming you have a GUI to run the controls for this editor?

Edited by Bert

Share this post


Link to post
Share on other sites

At this point all I'm trying to do is see if I can quickly, and efficiently, find a color in a bitmap (GDI+ bitmap or a WinAPI handle to a bitmap, since they can be converted easily to each other. I've been using the WinAPI one to capture the client area of a program). My GUI has... bugs in it still so I'm not even using it. Atm I've got 2 scripts on my desktop I've been running trying to test this.

Share this post


Link to post
Share on other sites

Hi,

writing a (32-Bit) bitmap into Memory takes a millisecond, writing a struct "over" the RGBA-information (some people say "Pixels" to that ;) ) takes a millisecond too, copying the Information from struct to a string takes another millisecond, searching/replacing "colors" via stringinstr/stringreplace/stringregex[replace] depends on string length, but never more than some milliseconds. 

If you use a slow idea, use slow functions and write slow scripts/programs, its not surprising that you have to wait a long time....

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By AnRios
      Greetings friends!
      I have been searching the help file and Google, with no success, to find a way to validate images from a folder and mark them somehow in a spreadsheet.
      My context is: I made a code with the help of the community that captures images from SAP and saves them in a folder.  Now I'd like to identify which ones are black, if it's even possible. I read about PixelSearch, but did not get it to work. If someone could point me in the right direction, I'd appreciate it.
      The code I'm using:
      #include <File.au3> #include <ScreenCapture.au3> $x = InputBox("Title", "Amount of Images To Capture", "", "", 320, 150) If @error Then Exit $x = Number($x) $y = InputBox("Title", "Batch Name", "", "", 320, 150) If @error Then Exit $y = String($y) HotKeySet("{HOME}", "printscreen") Func printscreen() $FilePath =("C:\Fiscalizacao\Fotos"&"/") $FileName = $y & " - " $FileList = _FileListToArray($FilePath, $FileName & '*.jpg', 1) If Not IsArray($FileList) Then $FileName&= '1.jpg' Else $FileName &= $FileList[0] + 1 & '.jpg' EndIf _ScreenCapture_Capture($FilePath & "\" & $FileName, 354, 196, 673, 436) EndFunc HotKeySet("{BS}", "Terminate") Func Terminate() Exit 0 EndFunc For $i = 1 to $x Opt("WinTitleMatchMode",2) If NOT WinExists("Relatorio") Then MsgBox(0, "Atenção!", "Relatório do MOM deve estar aberto!") Call("Terminate") EndIf Opt("WinTitleMatchMode",2) WinActivate("Relatorio") Sleep (250) Opt("WinTitleMatchMode",2) SendKeepActive("Relatorio") Send("{ENTER}") Sleep (1000) Send("{HOME}") Sleep (200) Opt("WinTitleMatchMode",2) WinActivate("Relatorio") Send("{DOWN}") Next  
    • By plankton
      While IsArray(PixelSearch(100, 100, 300, 300, 0xffffff))     Send("{ESC}")     Sleep(1000) WEnd  MsgBox(0,"Loop exited","") Hi, above is my function that will execute ESC button when white color is present in specific area which is color 0xffffff.
      But how do I do the opposite like when white color is not present in that specific area? Like this below code which gives me error
       
      While IsArray(PixelSearch(100, 100, 300, 300, <>0xffffff))     Send("{ESC}")     Sleep(1000) WEnd  MsgBox(0,"Loop exited","")  
       
    • By plankton
      Hi I want it to search for a specific color in certain area then excute the loop which presses ESC button and stop the loop when the color no longer present in that specific area.  Here is the example of my code. But it doesnt stop when color is disappeared.
       
      Pixelsearch(511, 455, 678, 501, 0xFFFFFF)
      If Not  @Error Then
      While 1
      Sleep(1000)
      Send("{ESC}")
      Pixelsearch(511, 455, 678, 501, 0xFFFFFF)
      If @Error Then Exitloop
       
    • By Beege
      Heres a function for searching for a bitmap within another bitmap. The heart of it is written assembly (source included) and working pretty quick I feel. I have included an example which is pretty basic and should be easily enough for anyone to get the concept. 
      You will be given a small blue window that will take a screencapture of that size:

       
      It will then take a full screenshot and highlight all locations that it found

      Please let me know if you have any issues or questions. Thanks!
       
      Update 8/5/2019:
      Rewrote for fasmg. Added full source with everything needed to modify
      BmpSearch_8-5-2019.7z
      BmpSearch.zip
       
      GAMERS - Asking for help with ANY kind of game automation is against the forum rules. DON'T DO IT.
    • By ManualIT
      Guys, i need help on creating a script that restarts a program once it starts using more than 1GB of memory.
      No idea how to start on the script, i don't know which functions i should use for process memory reading.
       
      My mind is in total blank at the moment, so i need a kick start
×
×
  • Create New...