Jump to content
Sign in to follow this  
InunoTaishou

PixelSearch in bitmap (handle to a bitmap, bitmap, or hdc)

Recommended Posts

InunoTaishou

Here's the goal:

  • Create a handle to a bitmap object using _WinApi_PrintWindow (this is done and working properly, I can take a screenshot of the window even when it's hidden or offscreen, does not work minimized)
  • Be able to search for a pixel color in memory on the handle. I can get it to work if I create a Bitmap from an HBITMAP (_GDIPlus_BitmapCreateFromHBITMAP) and then go through each pixel and check it using _GDIPlus_BitmapGetPixel but it's too slow. I've tried doing the _WinApi_GetPIxel using an $hDC but it's much slower than GDI+ (GD+ takes about 20seconds to search for almost 500,000 pixels, GetPixel takes almost 60seconds)

I found the FastFind library but, honestly, it's sloppy and I'm wanting to search a defined area, not set the starting position and then search the rest of the window. I also found a few examples on the forums but I could not get them to work.

Here's the CaptureWindow function. I'm only wanting to capture the client area (Not the border around the client, hence the -16 for the width and the -38 for the height, gets rid of the title bar and the resize bars on the sides. Also I'm wanting to keep the coordinates relative to the window, so if you only want to capture the screen from 480, 200 to 680, 400, then the rest of the area is black on purpose)

#include-once
Func CaptureWindow(Const $iLeft = 0, Const $iTop = 0, Const $iWidth = -1, Const $iHeight = -1, Const $hWindow = WinGetHandle("[Active]"))
    Local $rect_window = WinGetPos($hWindow)
    Local $hDC = _WinAPI_GetWindowDC($hWindow)
    Local $hDestDC = _WinAPI_CreateCompatibleDC($hDC)
    Local $hBitmap = _WinAPI_CreateCompatibleBitmap($hDC, $rect_window[2] - 16, $rect_window[3] - 38)
    Local $hDestSv = _WinAPI_SelectObject($hDestDC, $hBitmap)
    Local $hSrcDC = _WinAPI_CreateCompatibleDC($hDC)
    Local $hBmp = _WinAPI_CreateCompatibleBitmap($hDC, $rect_window[2] - 16, $rect_window[3] - 38)
    Local $hSrcSv = _WinAPI_SelectObject($hSrcDC, $hBmp)
    
    _WinAPI_PrintWindow($hWindow, $hSrcDC, True)
    
    If ($iWidth > 0 and $iHeight > 0) Then
        _WinAPI_BitBlt($hDestDC, $iLeft, $iTop, $iWidth, $iHeight, $hSrcDC, $iLeft, $iTop, $MERGECOPY)
    Else
        _WinAPI_BitBlt($hDestDC, $iLeft, $iTop, $rect_window[2] - 16 - $iLeft, $rect_window[3] - 38 - $iTop, $hSrcDC, $iLeft, $iTop, $MERGECOPY)
    EndIf
    
    _WinAPI_SelectObject($hDestDC, $hDestSv)
    _WinAPI_SelectObject($hSrcDC, $hSrcSv)
    _WinAPI_ReleaseDC($hWindow, $hDC)
    _WinAPI_DeleteDC($hDestDC)
    _WinAPI_DeleteDC($hSrcDC)
    _WinAPI_DeleteObject($hBmp)
    
    Return $hBitmap
EndFunc   ;==>CaptureWindow

And the first attempt for PixelSearch, using GDI+ (Fastest)

Func PixelSearchInhBitmap(Const ByRef $hHBmp, Const ByRef $color, Const $tolerance = 10, Const $iLeft = 0, Const $iTop = 0, $iWidth = -1, $iHeight = -1, Const $iStep = 1)
    Local $hBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hHBmp)
    Local $rbg_color = _ColorGetRGB("0x" & Hex($color, 6))
    Local $found_color = False
    Local $abscoord_color[2] = [0, 0]
    Local $red_low = 0
    Local $green_low = 0
    Local $blue_low = 0
    Local $red_high = 0
    Local $green_high = 0
    Local $blue_high = 0
    
    $red_low = ($tolerance > $rbg_color[0] ? 0 : $rbg_color[0] - $tolerance)
    $green_low = ($tolerance > $rbg_color[1] ? 0 : $rbg_color[1] - $tolerance)
    $blue_low = ($tolerance > $rbg_color[2] ? 0 : $rbg_color[2] - $tolerance)
    $red_high = ($tolerance > 255 - $rbg_color[0] ? 255 : $rbg_color[0] + $tolerance)
    $green_high = ($tolerance > 255 - $rbg_color[1] ? 255 : $rbg_color[1] + $tolerance)
    $blue_high = ($tolerance > 255 - $rbg_color[2] ? 255 : $rbg_color[2] + $tolerance)
    
    If ($iWidth = -1) Then $iWidth = _GDIPlus_ImageGetWidth($hBitmap)
    If ($iHeight = -1) Then $iHeight = _GDIPlus_ImageGetHeight($hBitmap)
    
    Local $start_time = TimerInit()
    For $iY = $iTop To $iHeight Step $iStep
        For $iX = $iLeft To $iWidth Step $iStep
            Local $get_pixel = _GDIPlus_BitmapGetPixel($hBitmap, $iX, $iY)
            If (@Error) Then ContinueLoop
            Local $pixel_color = _ColorGetRGB("0x" & Hex($get_pixel, 6))
            If (@Error) Then ContinueLoop
            
            If (($pixel_color[0] >= $red_low and $pixel_color[0] <= $red_high) and ($pixel_color[1] >= $green_low and $pixel_color[1] <= $green_high) and ($pixel_color[2] >= $blue_low and $pixel_color[2] <= $blue_high)) Then
                $found_color = True
                $abscoord_color[0] = $iX
                $abscoord_color[1] = $iY
                ExitLoop 2
            EndIf
        Next
    Next
    MsgBox("", "", TimerDiff($start_time) / 1000)
    _GDIPlus_BitmapDispose($hBitmap)
    
    If ($found_color) Then
        Return $abscoord_color
    Else
        Return SetError(-1, 0, 0)
    EndIf
EndFunc

Second attempt using _WInApi_GetPixel (Note. I replaced the Autoit function with my own where I replaced the string "gdi32.dll" with a handle to the opened DLL. In the hopes it would improve time. The time difference was not noticable)

Global $HWND_DLL_GDI32 = DLLopen("gdi32.dll")

Func PixelSearchInhDC(Const ByRef $color, Const $tolerance = 10, Const $iLeft = 0, Const $iTop = 0, $iWidth = -1, $iHeight = -1, Const $iStep = 1, Const $hWnd_window = WinGetHandle("[Active]"))
    Local $hDC = _WinAPI_GetWindowDC($hWnd_window)
    Local $rbg_color = _ColorGetRGB("0x" & Hex($color, 6))
    Local $found_color = False
    Local $abscoord_color[2] = [0, 0]
    Local $start_time, $end_time
    Local $red_low = 0
    Local $green_low = 0
    Local $blue_low = 0
    Local $red_high = 0
    Local $green_high = 0
    Local $blue_high = 0
    
    If (Not $hDC) Then
        Return SetError(1, 0, 0)
    EndIf
    
    If ($iWidth = -1 or $iHeight = -1) Then
        Local $rect_window = WinGetPos($hWnd_window)
        If ($iWidth = -1) Then $iWidth = $rect_window[2] - $iLeft
        If ($iHeight = -1) Then $iHeight = $rect_window[3] - $iTop
    EndIf
    
    $red_low = ($tolerance > $rbg_color[0] ? 0 : $rbg_color[0] - $tolerance)
    $green_low = ($tolerance > $rbg_color[1] ? 0 : $rbg_color[1] - $tolerance)
    $blue_low = ($tolerance > $rbg_color[2] ? 0 : $rbg_color[2] - $tolerance)
    $red_high = ($tolerance > 255 - $rbg_color[0] ? 255 : $rbg_color[0] + $tolerance)
    $green_high = ($tolerance > 255 - $rbg_color[1] ? 255 : $rbg_color[1] + $tolerance)
    $blue_high = ($tolerance > 255 - $rbg_color[2] ? 255 : $rbg_color[2] + $tolerance)
    
    $start_time = TimerInit()
    For $iY = $iTop To $iHeight Step $iStep
        For $iX = $iLeft To $iWidth Step $iStep
            Local $get_pixel = __WinAPI_GetPixel($hDC, $iX, $iY)
            If (@Error) Then ContinueLoop
            Local $pixel_color = _ColorGetRGB("0x" & Hex($get_pixel, 6))
            If (@Error) Then ContinueLoop
            
            If (($pixel_color[0] >= $red_low and $pixel_color[0] <= $red_high) and ($pixel_color[1] >= $green_low and $pixel_color[1] <= $green_high) and ($pixel_color[2] >= $blue_low and $pixel_color[2] <= $blue_high)) Then
                $found_color = True
                $abscoord_color[0] = $iX
                $abscoord_color[1] = $iY
                ExitLoop 2
            EndIf
        Next
    Next
    MsgBox("", "", TimerDiff($start_time) / 1000 & "s")
    
    If ($found_color) Then
        Return $abscoord_color
    Else
        Return SetError(1, 0, 0)
    EndIf
EndFunc

Func __WinAPI_GetPixel($hDC, $iX, $iY)
    Local $aRet = DllCall($HWND_DLL_GDI32, 'dword', 'GetPixel', 'handle', $hDC, 'int', $iX, 'int', $iY)
    If @error Or ($aRet[0] = 4294967295) Then Return SetError(@error, @extended, -1)
    ; If $aRet[0] = 4294967295 Then Return SetError(1000, 0, -1)

    Return __RGB($aRet[0])
EndFunc   ;==>__WinAPI_GetPixel

 

Edited by InunoTaishou

Share this post


Link to post
Share on other sites
InunoTaishou

Looked into _GDIPlus_BitmapLockBits, this seems to be a bit faster (about 8 seconds faster) but it still takes seconds to do the search.

#include <GDIPlus.au3>
#include <GUIConstantsEx.au3>
#include <MsgBoxConstants.au3>
#include <Color.au3>

_GDIPlus_Startup()

TestPixelSearch()

Func TestPixelSearch()
    Local $hImage = _GDIPlus_ImageLoadFromFile(@DesktopDir & "\CaptureWindow.jpg")
    Local $iW = _GDIPlus_ImageGetWidth($hImage), $iH = _GDIPlus_ImageGetHeight($hImage) ;get width and height of the image
    Local $hBitmap = _GDIPlus_BitmapCreateFromScan0($iW, $iH)
    Local $hContext = _GDIPlus_ImageGetGraphicsContext($hBitmap)
    _GDIPlus_GraphicsDrawImageRect($hContext, $hImage, 0, 0, $iW, $iH)

    Local $tBitmapData = _GDIPlus_BitmapLockBits($hBitmap, 0, 0, $iW, $iH, BitOR($GDIP_ILMWRITE, $GDIP_ILMREAD), $GDIP_PXF32RGB)
    Local $iScan0 = DllStructGetData($tBitmapData, "Scan0")
    Local $rbg_color = _ColorGetRGB(0xFFFC00)
    Local $tolerance = 10
    Local $red_low = ($tolerance > $rbg_color[0] ? 0 : $rbg_color[0] - $tolerance)
    Local $green_low = ($tolerance > $rbg_color[1] ? 0 : $rbg_color[1] - $tolerance)
    Local $blue_low = ($tolerance > $rbg_color[2] ? 0 : $rbg_color[2] - $tolerance)
    Local $red_high = ($tolerance > 255 - $rbg_color[0] ? 255 : $rbg_color[0] + $tolerance)
    Local $green_high = ($tolerance > 255 - $rbg_color[1] ? 255 : $rbg_color[1] + $tolerance)
    Local $blue_high = ($tolerance > 255 - $rbg_color[2] ? 255 : $rbg_color[2] + $tolerance)
    Local $tPixel = DllStructCreate("int[" & $iW * $iH & "];", $iScan0)
    Local $iPixel, $iRowOffset
    Local $found_pixel = False
    Local $abscoord_pixel[2] = [0, 0]

    Local $start_timer = TimerInit()
    For $iY = 0 To $iH - 1
        $iRowOffset = $iY * $iW + 1
        For $iX = 0 To $iW - 1 ;get each pixel in each line and row
            $iPixel = DllStructGetData($tPixel, 1, $iRowOffset + $iX) ;get pixel color
            Local $pixel_color = _ColorGetRGB("0x" & Hex($iPixel, 6))
            If (($pixel_color[0] >= $red_low and $pixel_color[0] <= $red_high) and ($pixel_color[1] >= $green_low and $pixel_color[1] <= $green_high) and ($pixel_color[2] >= $blue_low and $pixel_color[2] <= $blue_high)) Then
                $abscoord_pixel[0] = $iX
                $abscoord_pixel[1] = $iY
                $found_pixel = True
                ExitLoop
            EndIf
        Next
    Next
    Local $end_time = TimerDiff($start_timer) / 1000

    MsgBox("", "", $end_time & @CRLF & "Found pixel = " & $found_pixel)

    _GDIPlus_BitmapUnlockBits($hBitmap, $tBitmapData)

    ;cleanup resources
    _GDIPlus_ImageDispose($hImage)
    _GDIPlus_GraphicsDispose($hContext)
    _GDIPlus_BitmapDispose($hBitmap)
    _GDIPlus_Shutdown()
EndFunc

Will probably try to and rewrite this in C++ and give it a test in visual studio, perhaps it will be faster in a dll than in autoit. Also saw something about the WinApi GetDIBits but the apparently copies the whole bitmap into a buffer, not a pointer to a buffer so someone said it would be slow.

Does anyone know of any other way of checking pixels in memory or any way to make this faster? The Autoit Pixelsearch function takes miliseconds to complete. I did find the source code for another *ahem* scripting languages PixelSearch function and it looks like it uses GetDIBits.

Share this post


Link to post
Share on other sites
Bert

Why do you need to do pixel search?

Why not look for the control?

Why also do you need to do a snapshot then look at the picture?

Why not look at the screen directly? 

This seems a real difficult way to solve a problem when there are many simpler and more stable methods to do things. 

Share this post


Link to post
Share on other sites
InunoTaishou

Because it seems fun and an interesting way to learn more about bitmaps. The only problem with that is I don't know a whole lot about bitmaps and my knowledge of the WinApi is very limited. I was also working on a small image editor using GDI+ and being able to find colors in the bitmap will open up the possibility to search and replace colors in a bitmap. I just don't want it to take 20 seconds to go through and replace colors when programs like Photoshop and Gimp can do it in ms and Pixelsearch takes ms to find a pixel on the screen.

That snapshot allows to search in memory, so even if the window is hidden you can still search in it.

Edited by InunoTaishou

Share this post


Link to post
Share on other sites
Bert

So your saying you want to make a photo editor? I'm asking for none of your code shows anything to replace colors. It only shows how you want to search a client area. I'm assuming you have a GUI to run the controls for this editor?

Edited by Bert

Share this post


Link to post
Share on other sites
InunoTaishou

At this point all I'm trying to do is see if I can quickly, and efficiently, find a color in a bitmap (GDI+ bitmap or a WinAPI handle to a bitmap, since they can be converted easily to each other. I've been using the WinAPI one to capture the client area of a program). My GUI has... bugs in it still so I'm not even using it. Atm I've got 2 scripts on my desktop I've been running trying to test this.

Share this post


Link to post
Share on other sites
AndyG

Hi,

writing a (32-Bit) bitmap into Memory takes a millisecond, writing a struct "over" the RGBA-information (some people say "Pixels" to that ;) ) takes a millisecond too, copying the Information from struct to a string takes another millisecond, searching/replacing "colors" via stringinstr/stringreplace/stringregex[replace] depends on string length, but never more than some milliseconds. 

If you use a slow idea, use slow functions and write slow scripts/programs, its not surprising that you have to wait a long time....

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • Daka
      By Daka
      I find it very weird:
      if I run like this:
           Local $aCoord = PixelSearch($posX, $posY, $sizeX, $sizeY, '0xFF455E')
      works fine!
      If I run like this:
          Local $metaColor1 = '0xFF455E' or like this Local $metaColor1 = "0xFF455E" or Local $metaColor1 = "'0xFF455E'"
          Local $aCoord = PixelSearch($posX, $posY, $sizeX, $sizeY, $color)
      it doesn't want to work! So something with variable is not working and yes if I print it out like this:
      ConsoleWrite(@LF & $posX & ":"& $posY & ":"& $sizeX & ":" & $sizeY & ":" & $color&@LF)
      771:80:833:151:0xFF455E

      So I dont see the problem, maybe some of you people?
    • toto22
      By toto22
      I'm trying to get a "double" value from memory . However my code gives me error.
       
      Opt("WinTitleMatchMode", 4)     Global $ProcessID = WinGetProcess("TI Pro")     If $ProcessID = -1 Then         MsgBox(4096, "ERROR", "Failed to detect process.")         Exit     EndIf     Local $DllInformation = _MemoryOpen($ProcessID)     If @Error Then         MsgBox(4096, "ERROR", "Failed to open memory.")         Exit     EndIf   Local $dAddress = 0x1FECD474   Local $tNbSteps = DllStructCreate("double", $dAddress)   Local $value = DllStructSetData($tNbSteps, 1, (_MemoryRead($dAddress, $DllInformation)))      MsgBox($MB_SYSTEMMODAL, $value)
    • squidol
      By squidol
      I need help about pixel search. The problem with the script below is that PixelSearch does not continue on the coordinates where it has stopped.
      When the first pixelsearch finds the 1st pixel, it should move the mouse over it and new pixels would appear just like hovering over menus. Then upon hover, there would be another PixelSearch to see if the second red pixel is found. If not found, then it should resume the first PixelSearch instead of starting from left to right again. 
      For example we are doing a pixelsearch on two straight lines with coordinates  [x,y]  :
      [0,0] [1,0] [2,0]
      [1,0] [1,1] [1,2]
      Pixelsearch finds the coordinate [1,0] matching our color. So it checks the pixel just below it which is [1,1] to see if it is color red. If not red then it should continue searching starting on coordinates [2,0] instead of going back to [0,0]
      Local $bflag = False Do     ToolTip("finding..",0,0,"")     Sleep(500)     $var = PixelSearch(591, 169, 1365, 740, 0x464950,50) ; look for initial pixel     If Not @error Then ;         MouseMove($var[0],$var[1],0) ;move on the button to show new selections, new pixels         sleep(1000)         ;search for the red pixel on an area above the first pixel coords which was          ;generated when mouse cursor was hovered on the first pixel found.         $redpixel = PixelSearch($var[0]-50,$var[1]-50,$var[0]+50,$var[1]+50,0xFF0048)          If Not @error Then ; Found the 2nd pixel                 ToolTip("found...",0,0,"")             $bflag = True          EndIf     EndIf Until $bflag I can pay 50USD through Paypal for a working solution. thanks   
    • xEviiLx
      By xEviiLx
      I'm trying to read value of a base pointer + offset.
      With only address I can easily the value but with base addres (pointer) I really don't know how I can do that.
       
    • Xandy
      By Xandy
      I have a bunch of SDL_Surfaces loaded into memory.  I want to list them in a ComboBox.
      Using this code, I can load images to the combobox from file but not from existing SDL_Surface memory
      CODE ISN"T MEANT TO RUN
      ; Create combobox pic list example $aControl[$iControl_id][$eControl_data] = _GUICtrlComboBoxEx_Create($gui, $data_value, $data_x, $data_y, $data_w, $data_h, $CBS_DROPDOWNLIST) ; Image List for ComboboxEx Local $hImage = _GUIImageList_Create($gTile_w, $gTile_h, 6) ; Fill hImage with SDL_Surfaces stored in memory: aNPC_surf[scale][type][way][frame] For $i = 0 To 1 ; This works but only from file: ;_GUIImageList_AddBitmap($hImage, $gFolder_graphics & $gaWorld_info[$player.iWorld_cur][$eWi_filename] & "Tiles\" & $i & ".bmp") ; I've broken up a sprite sheet and want to insert into combobox from memory ;_GUIImageList_Add($hImage, $aNPC_surf[1][1][1][$i]) _GUIImageList_Add($hImage, _GDIPlus_BitmapCreateFromMemory($aNPC_surf[1][1][1][$i]), True) ;_GDIPlus_BitmapCreateFromMemory($aNPC_surf[1][1][1][$i]) ; Add the index number to combobox item _GUICtrlComboBoxEx_AddString($aControl[$eNPC_iPic_type][$eControl_data], $i, $i, $i) Next ; Set hImage list to combobox control _GUICtrlComboBoxEx_SetImageList($aControl[$eNPC_iPic_type][$eControl_data], $hImage) Anyone know if I can convert from SDL_Surface to hBitmap?  Maybe I'm doing something else wrong.
       
      I've seen hBitmap converted to SDL_Surface but I don't really understand it yet: 
       
      My full script can be found here:
       
×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.