Determine if screen is composed of 90% white pixels

[fusion400]

Yes just include

#include <GDIPlus.au3>

#Include <ScreenCapture.au3> in your code

That is shown in the sample code in the link I gave to FindBMP

I think both are included with autoit download

fixed is specific to another script.

Yes i have included them now and now i get the message box with the error.

"Error with the DLLcall"

[JohnOne]

What OS you using ?

Does it have gdiplus.dll

[fusion400]

What OS you using ?

Does it have gdiplus.dll

I am running XP and yes the GdiPlus.dll is available in

C:\WINDOWS\WinSxS\x86_Microsoft.Windows.GdiPlus_6595b64144ccf1df_1.0.0.0_x-ww_8d353f13 directory

[ShawnW]

I like ImageSearch.au3 better. I use it for bots all the time and its wonderful for finding images like this. I would recommend just cutting the image down to the rectangle and a little white around it. Then save that as a .bmp somewhere.

Then use code something like this...

#include <ImageSearch.au3>
Global $coords[2]
Global $pathToBMP = "" ; Path to the image you want to find

FindImage()

Func FindImage()
    ; The parameters for _ImageSearch are _ImageSearch($findImage, $resultPosition, ByRef $x, ByRef $y, $tolerance)
    ; Most are self explainitory but $resultPosition = 1 or 0.
    ; It determins what coords are returned, 1 for center of image and 0 for top left.
    $result = _ImageSearch($pathToBMP,1,$coords[0],$coords[1],20)
    If $result=1 Then
        MouseClick("left",$coords[0],$coords[1],2,0) ; Click twice in the center of the image that was found.
        MsgBox(0,"Image Found!!!","The location of the center of the image is "&$coords[0]&","&$coords[1])
    EndIf
EndFunc

You can put the function call in whatever kind of loop you want. I wouldn't recommend looping 100% of the time though. Also make sure the ImageSearch.au3 and .dll I'm attaching to this post are in the same directory as the script.

ImageSearch.zip

[junkew]

Based on the functions used in findbmp below the code to determine the percentage black

The main trick is getting a monochrome bitmap

$hIMG = _WinAPI_LoadImage(0, $Bitmap1Filename, $IMAGE_BITMAP, 0, 0, $LR_LOADFROMFILE + $LR_MONOCHROME )

and based on that getting the bits

and fastest counting is by replacing (stringregexp will be much slower for counting)

expandcollapse popup

Opt('MustDeclareVars', 1)
;#include <GUIConstants.au3>
#include <GDIPlus.au3>
#include <ScreenCapture.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <string.au3>

;~ Constants for type of picture matching
Const $c24RGBFullMatch = 1 ;Load as 24 bits and full match
Const $c24RGBPartialMatch = 2 ;Load as 24 bits and partial match
Const $c16RGBFullMatch = 3 ;Load as 16 bits and full match
Const $c16RGBPartialMatch = 4 ;Load as 16 bits and partial match

Dim $BMP1Data = "", $BMP1Width = 0, $BMP1Height = 0, $BMP1LineWidth = 0
Dim $imgBytes = 3
dim $hImg
dim $whiteHWND
dim $totalLength

Global $Bitmap1Filename = @tempdir & "\FULLSCREEN.BMP"  ;Full screen to search in

; Initialize GDI+ library
_GDIPlus_Startup()
; Capture full screen
;~ _ScreenCapture_Capture($Bitmap1Filename, 0, 0, -1, -1, False)

; Capture the whitescreen with the bar (in this example its within firefox)
$whiteHWND = WinGetHandle("Determine if screen is composed of 90% white pixels - AutoIt Forums - Mozilla Firefox")
_ScreenCapture_CaptureWnd($Bitmap1Filename, $whiteHWND, 0, 0, -1, -1, False)

; Load the bitmap to search in as a monochrome bitmap
$hIMG = _WinAPI_LoadImage(0, $Bitmap1Filename, $IMAGE_BITMAP, 0, 0, $LR_LOADFROMFILE + $LR_MONOCHROME )

;Get the image into a string
GetImage($hImg, $BMP1Data, $BMP1Width, $BMP1Height, $BMP1LineWidth, $imgBytes)
$BMP1Data = BinaryToString($BMP1Data)
$totalLength=stringlen($Bmp1Data) 
;Dump the string 000000 = 1 bit black FFFFFF = 1 bit white
;~ dumpHex("", $BMP1Data, $BMP1Height, $BMP1LineWidth)

;Now we can find the black and white pixels chr(255)=white
$bmp1data=stringreplace($bmp1data,chr(255),"")
consolewrite("Percentage black " & stringformat("%.2f",(stringlen($Bmp1Data)/$totallength)*100) & " %" &@crlf)
;~ dumpHex("", $BMP1Data, $BMP1Height, $BMP1LineWidth)
 
_GDIPlus_Shutdown()

Func GetImage($BMPFile, ByRef $BMPDataStart, ByRef $Width, ByRef $Height, ByRef $Stride, $imgBytes = 3)
    Local $Scan0, $pixelData, $hbScreen, $pBitmap, $pBitmapCap, $handle, $Bitmapdata, $pixelFormat

    ; Load the bitmap to search in
    If $BMPFile = "SCREEN" Then
        $hbScreen = _ScreenCapture_Capture("", 0, 0, -1, -1, False)
        $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen) ; returns memory bitmap
    Else
        ;Check if its already a handle else try to get one
        if stringleft($bmpFile,2)="0x" Then
            $handle=$bmpFile
            $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($handle) ; returns memory bitmap
        Else
            $handle = WinGetHandle($BMPFile)    
            If @error Then
                ;Assume its an unknown handle so correct filename should be given
                $pBitmap = _GDIPlus_BitmapCreateFromFile($BMPFile)      
            Else
                $hbScreen = _ScreenCapture_CaptureWnd("", $handle, 0, 0, -1, -1, False)
                $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen) ; returns memory bitmap
            EndIf
        EndIf
    EndIf
    
    ;Get $tagGDIPBITMAPDATA structure
;~  ConsoleWrite("Bitmap Width:       " & _GDIPlus_ImageGetWidth($pBitmap) & @CRLF )
;~  ConsoleWrite("Bitmap Height:      " & _GDIPlus_ImageGetHeight($pBitmap) & @CRLF)

;~  24 bits (3 bytes) or 16 bits (2 bytes) comparison or 1 bits comparison
     
    Select  
        case $imgBytes=1
            $BitmapData = _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_PXF01INDEXED)
        case $imgBytes=2
            $BitmapData = _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF16RGB555)
        case Else
        $BitmapData = _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF24RGB)
    endselect
    
    If @error Then MsgBox(0, "", "Error locking region " & @error)
    
    $Stride = DllStructGetData($BitmapData, "Stride") ;Stride - Offset, in bytes, between consecutive scan lines of the bitmap. If the stride is positive, the bitmap is top-down. If the stride is negative, the bitmap is bottom-up.
    $Width = DllStructGetData($BitmapData, "Width") ;Image width - Number of pixels in one scan line of the bitmap.
    $Height = DllStructGetData($BitmapData, "Height") ;Image height - Number of scan lines in the bitmap.
    $pixelFormat = DllStructGetData($BitmapData, "PixelFormat") ;Pixel format - Integer that specifies the pixel format of the bitmap
    $Scan0 = DllStructGetData($BitmapData, "Scan0") ;Scan0 - Pointer to the first (index 0) scan line of the bitmap.
    
    $pixelData = DllStructCreate("ubyte lData[" & (Abs($Stride) * $Height - 1) & "]", $Scan0)
    $BMPDataStart = $BMPDataStart & DllStructGetData($pixelData, "lData")
    
    _GDIPlus_BitmapUnlockBits($pBitmap, $BitmapData)
    _GDIPlus_ImageDispose($pBitmap)
    _WinAPI_DeleteObject($pBitmap)

EndFunc   ;==>GetImage

Func dumpHex($sFname, $bmData, $h, $w)
    Local $i, $tLine
    
    For $i = 0 To $h - 1
        $tLine = StringMid($bmData, 1 + ($i * $w), $w)
        ConsoleWrite(StringFormat("%06s;", 1 + ($i * $w)) & _StringToHex($tLine) & ";")
        ConsoleWrite(@CRLF)
    Next
EndFunc   ;==>dumpHex

Edited April 3, 2010 by junkew

[Malkey]

If you can get your head around the regular expression pattern, then this script will find the percentage of the colour/s indicated in the regular expression pattern.

To speed the process up, only every tenth pixel is checked.

expandcollapse popup

#include <GDIPlus.au3>

Local $filename, $begin, $aArr, $time
Local $filename = FileOpenDialog("Select image", @DesktopCommonDir, "All images (*.jpg;*.png;*.gif;*.bmp;)", 1)
If $filename = "" Then Exit

ShellExecute($filename)

$begin = TimerInit()

; =============== Colours are in 0xBBGGRRAA hex colour format ================
$iPercent = _PercentColorInImage($filename, "([E-F][0-9A-F]{3}FF)"); Almost white
;$iPercent = _PercentColorInImage($filename, "(FFFFFFFF)"); white
;$iPercent = _PercentColorInImage($filename, "(000000FF)"); Black
;$iPercent = _PercentColorInImage($filename, "(0000FFFF)"); Red
;$iPercent = _PercentColorInImage($filename, "(FF0000FF)"); Blue
;$iPercent = _PercentColorInImage($filename, "(00FF00FF)"); Green
;$iPercent = _PercentColorInImage($filename, "([0-9A-F]{6}[0-9A-E][0-9A-F])"); Any colours with any transparency.

$time = Round(TimerDiff($begin) / 1000, 3)
MsgBox(0, "", Round($iPercent, 1) & " % colour found" & @CRLF & "Time taken = " & $time & " secs")

; Checks every tenth pixel
Func _PercentColorInImage($sFileName, $sREPattern)
    Local $Reslt, $stride, $format, $Scan0, $iIW, $iIH, $hImage
    Local $v_Buffer, $width, $height, $Ret
    _GDIPlus_Startup()
    $hImage = _GDIPlus_ImageLoadFromFile($sFileName)
    $iIW = _GDIPlus_ImageGetWidth($hImage)
    $iIH = _GDIPlus_ImageGetHeight($hImage)

    $Reslt = _GDIPlus_BitmapLockBits($hImage, 0, 0, $iIW, $iIH, $GDIP_ILMREAD, $GDIP_PXF32ARGB)

    ;Get the returned values of _GDIPlus_BitmapLockBits ()
    $width = DllStructGetData($Reslt, "width")
    $height = DllStructGetData($Reslt, "height")
    $stride = DllStructGetData($Reslt, "stride")
    $format = DllStructGetData($Reslt, "format")
    $Scan0 = DllStructGetData($Reslt, "Scan0")

    $v_BufferA = DllStructCreate("byte[" & $height * $width * 4 & "]", $Scan0) ; Create DLL structure for all pixels
    $AllPixels = DllStructGetData($v_BufferA, 1)
    $sREResult1 = StringRegExpReplace(StringTrimLeft($AllPixels, 2), "(.{8}){10}", "\1 "); check every tenth pixel,
    StringRegExpReplace($sREResult1, $sREPattern, "\1")
    $Ret = 1000 * @extended / ($width * $height); )

    _GDIPlus_BitmapUnlockBits($hImage, $Reslt)
    _GDIPlus_ImageDispose($hImage)
    _GDIPlus_Shutdown()
    Return $Ret
EndFunc ;==>_PercentColorInImage

[fusion400]

I like ImageSearch.au3 better. I use it for bots all the time and its wonderful for finding images like this. I would recommend just cutting the image down to the rectangle and a little white around it. Then save that as a .bmp somewhere.

Then use code something like this...

#include <ImageSearch.au3>
Global $coords[2]
Global $pathToBMP = "" ; Path to the image you want to find

FindImage()

Func FindImage()
    ; The parameters for _ImageSearch are _ImageSearch($findImage, $resultPosition, ByRef $x, ByRef $y, $tolerance)
    ; Most are self explainitory but $resultPosition = 1 or 0.
    ; It determins what coords are returned, 1 for center of image and 0 for top left.
    $result = _ImageSearch($pathToBMP,1,$coords[0],$coords[1],20)
    If $result=1 Then
        MouseClick("left",$coords[0],$coords[1],2,0) ; Click twice in the center of the image that was found.
        MsgBox(0,"Image Found!!!","The location of the center of the image is "&$coords[0]&","&$coords[1])
    EndIf
EndFunc

You can put the function call in whatever kind of loop you want. I wouldn't recommend looping 100% of the time though. Also make sure the ImageSearch.au3 and .dll I'm attaching to this post are in the same directory as the script.

wow thanks this works really well.

Is it possible to make it work if the application is also running on a secondary screen?

Is it also possible to embed the rectangle.bmp image that will be the image it searches for directly in the executable file?

Edited April 5, 2010 by fusion400

[ShawnW]

wow thanks this works really well.

Is it possible to make it work if the application is also running on a secondary screen?

Is it also possible to embed the rectangle.bmp image that will be the image it searches for directly in the executable file?

Well the 2nd screen thing is a limitation of the .dll which I didn't make. Possible maybe but its beyond my current skill set.

On the embedding thing what you can do is...

FileInstall("rectangle.bmp",@TempDir,1)

The 1st parameter ("rectangle.bmp") is the path and must be literal not a variable path, but is only used when compiled. It stores the file in the .exe and extracts it to the temp directory when run. When you need to use it just use the path...

@TempDir & "\rectangle.bmp"

Don't forget the '\' before the image name because @TempDir will not return a trailing '\'. Also remember to delete the file afterwards with...

FileDelete(@TempDir & "\rectangle.bmp")

because it's good to tidy up.

[fusion400]

Well the 2nd screen thing is a limitation of the .dll which I didn't make. Possible maybe but its beyond my current skill set.

On the embedding thing what you can do is...

FileInstall("rectangle.bmp",@TempDir,1)

The 1st parameter ("rectangle.bmp") is the path and must be literal not a variable path, but is only used when compiled. It stores the file in the .exe and extracts it to the temp directory when run. When you need to use it just use the path...

@TempDir & "\rectangle.bmp"

Don't forget the '\' before the image name because @TempDir will not return a trailing '\'. Also remember to delete the file afterwards with...

FileDelete(@TempDir & "\rectangle.bmp")

because it's good to tidy up.

Sounds really good can the dll file also be embedded and if so can it be read directly from the temp dir so that autoit can use it directly or must i copy to system32 directory?

[ShawnW]

I don't believe temp will work on the dll you must use @SystemDir which is the system32 folder.

Edit: or the local folder the .exe is in of course, but I assume by the question you don't want that.

Edited April 5, 2010 by ShawnW