junkew Posted March 17, 2008 Share Posted March 17, 2008 (edited) Code below searches a certain bitmap (all supported formats of GDI+) picture in another picture or on screen (after saved as bitmap) not depending on 3rd party dll's Pentium 4 of 3 GHz takes * about 4.5 seconds on a 1280*1024 desktop to find a 260*260 bitmap. Partial match 0.5 seconds * about 1 seconds on a 1280*1024 desktop to find a 20*17 bitmap. Partial match 0.5 seconds Some references I used to come up with this solution * code to get a smaller bitmap: http://www.autoitscript.com/forum/index.ph...=pixelchecksum) * Pixelchecksum seems not to do the full job but in combination with below it can become very fast: http://www.autoitscript.com/forum/index.ph...l=pixelchecksum * A pixel by pixel searcher http://www.autoitscript.com/forum/index.ph...&hl=compare * UDF Get or Read Pixel from Memory * GetDIBits Not fully finished but all comments/enhancements are appreciated * Save to file or directly on special window description SCREEN, [ACTIVE] or name of picture file * Searching is done on a line by line basis instead of pixel by pixel * Bitmap to search in example program should be fully visible to work for first 3 examples as it are screenshots taken. Same applies for 3 in AUTOIT homepage * Speed depends on the size of the bitmap you are searching for but is about linear and < 5 seconds * Some doubts on left and top returned (sometimes seems to be off by 1 or several pixels) * Transparency (when search letters on a different background) http://www.winprog.org/tutorial/transparency.html * Match quicker by doing a bitblt with srcinvert when first line is found (instead of checking line by line) * $BMP1 and $BMP2 to be HBITMAP handle as input instead of filenames (will make searching within partial screen Sample to work * Install pics.zip into %temp% / Save BMP attachments for example to @tempdir (%temp%) for example to work Sample code expandcollapse popup#Region includes Opt('MustDeclareVars', 1) ;#include <GUIConstants.au3> #include <GDIPlus.au3> #Include <ScreenCapture.au3> #include <string.au3> #include <WinAPI.au3> #include <WindowsConstants.au3> #EndRegion ;~ If debugging output is wanted to understand and analyse the algorithm put this on true/false ;~ Const $bDebug=True Const $bDebug=False #region example ;~ Const $cMatchLinePercentage=0.97 ;Minimum matching, put on 1 if exact matching is needed on all lines Const $cMatchLinePercentage=0.99 ;Minimum matching, put on 1 if exact matching is needed on all lines ;~ Constants for type of picture matching const $c24RGBFullMatch=1 ;Load as 24 bits and full match const $c24RGBPartialMatch=2 ;Load as 24 bits and partial match const $c16RGBFullMatch=3 ;Load as 16 bits and full match const $c16RGBPartialMatch=4 ;Load as 16 bits and partial match ; ** Example start ** ; Screen samples Global Const $Bitmap1Filename = @scriptDir & "\FULLSCREEN.BMP" Global Const $Bitmap2Filename = @scriptDir & "\CALCULATOR.BMP" Global Const $Bitmap3Filename = @scriptDir & "\BACKSPACE.BMP" ;BMP samples Global Const $Bitmap4Filename = @scriptDir & "\7WDS_BW.BMP" Global Const $Bitmap5Filename = @scriptDir & "\SEVEN_BW.BMP" Global Const $Bitmap6Filename = @scriptDir & "\CAT_BW.BMP" Global Const $Bitmap7Filename = @scriptDir & "\AUTOIT3.bmp";Make sure the homepage of autoit with 3 is visible local $calcHWND, $begin, $pos, $aWinPos, $aWinCSize, $start local $pBitmap, $BitmapData, $pixelFormat Opt("WinTitleMatchMode", 4);Matching windows by advanced options ; Initialize GDI+ library _GDIPlus_Startup () ;Calculator, make sure its started $calcHWND = WinGetHandle("[REGEXPCLASS:.*Calc.*]") If $calcHWND = "" Then Run("calc.exe") Sleep(2000) $calcHWND = WinGetHandle("[REGEXPCLASS:.*Calc.*]") EndIf If $calcHWND = "" Then consolewrite("Using active window which is 99% sure equal to the calculator window" & @LF ) $calcHWND = WinGetHandle("[ACTIVE]") EndIf consolewrite($calcHWND) ;Capture the calculator screen $begin = TimerInit() winactivate($calcHWND) Sleep(1000) _ScreenCapture_CaptureWnd($Bitmap2Filename, $calcHWND,0,0,-1,-1,False) ConsoleWrite("Saving calculator window " & TimerDiff($begin) & " milliseconds " & @LF) winmove("[ACTIVE]","",223,341) ;~ Get the base information of the calc window $aWinPos = WinGetPos($calchWnd) $aWinCSize = WinGetClientSize($calchWnd) Sleep(500) ;Capture an area within the calculator area backspace $begin = TimerInit() $pos=controlgetpos("","","[CLASS:Button; TEXT:BACKSPACE]") ;~ $pos=controlgetpos("","","[CLASS:Button; INSTANCE:2]") if @error=1 Then consolewrite("Not recognizing so we are on a windows 10 environment most likely, fix by providing manual controls" & @CRLF) consolewrite("Read frequently asked questions in Wiki number 31 and use simplespy" & @CRLF) local $pos[4] ;~ Backspace button <394;693;75;74> ;~ Button 2 <315;927;75;74> $pos[0]=394 $pos[1]=693 $pos[2]=75 $pos[3]=74 ;~ Func _UIA_DrawRect($tLeft, $tRight, $tTop, $tBottom, $color = 0xFF, $PenWidth = 4) _UIA_DrawRect($pos[0],$pos[0]+$pos[2],$pos[1],$pos[1]+$pos[2]) ;~ Fix quick and dirty as simplespy gives screencoordinates not relative to window $pos[0]=394 - $aWinpos[0] $pos[1]=693 - $aWinpos[1] EndIf ;Q&D calculation of offsets to capture _ScreenCapture_Capture($Bitmap3Filename, ($aWinPos[0]+$pos[0]) + ($aWinPos[2] - $aWinCSize[0])-3, ($awinpos[1]+$pos[1])+ ($aWinPos[3]-$aWinCSize[1])-3, $aWinPos[0]+$pos[0]+$pos[2]+3, ($awinpos[1]+$pos[1])+ ($aWinPos[3]-$aWinCSize[1])+$pos[3]-3,false) ConsoleWrite("Saving backspacebutton " & TimerDiff($begin) & " milliseconds " & @LF) ; Capture full screen $begin = TimerInit() _ScreenCapture_Capture($Bitmap1Filename,0,0,-1,-1,False) ConsoleWrite("Saving full screen took " & TimerDiff($begin) & " milliseconds " & @LF) Sleep(500) func filegetname($f) local $i $i=stringinstr($f,"\",false,-1) if $i > 0 Then return stringmid($f,$i+1) Else return $f EndIf EndFunc ;Do the actual find Func FindTester($BMP1, $BMP2, $Bool) local $tResult $start = TimerInit() $tResult=findBMP($BMP1,$BMP2, $Bool) ConsoleWrite($tResult & " " & FileGetName($BMP2) & " in " & FileGetName($BMP1) & " ** matchtype " & $Bool & " time elapsed: " & TimerDiff($start) & " milliseconds" & @LF) EndFunc if not $bDebug=True Then ; Not very usefull to find full screen and tricky as screen is most likely changed (clock, output from application etc.) ; findTester("SCREEN",$Bitmap1Filename,TRUE);Find the full screen itself ; findTester($Bitmap1Filename,$Bitmap1Filename,$c24RGBFullMatch);Find the full screen itself 31 seconds findTester($Bitmap1Filename,$Bitmap1Filename,$c24RGBPartialMatch);Find the full screen itself 2.8529088255245 seconds ; findTester($Bitmap1Filename,$Bitmap1Filename,$c16RGBFullMatch);Find the full screen itself 21 seconds findTester($Bitmap1Filename,$Bitmap1Filename,$c16RGBPartialMatch);Find the full screen itself 1 seconds ; Be aware that overlapping windows, moving things on screen can make it difficult to find on full screen findTester("SCREEN",$Bitmap2Filename,$c24RGBFullMatch);Find the full calculatorscreen findTester("SCREEN",$Bitmap2Filename,$c24RGBPArtialMatch);Find the full calculatorscreen with partial match findTester("SCREEN",$Bitmap2Filename,$c16RGBFullMatch);Find the full calculatorscreen findTester("SCREEN",$Bitmap2Filename,$c16RGBPArtialMatch);Find the full calculatorscreen with partial match findTester($Bitmap1Filename,$Bitmap2Filename,$c16RGBFullMatch);Find the full calculatorscreen findTester($Bitmap1Filename,$Bitmap2Filename,$c16RGBPartialMatch);Find the full calculatorscreen with partial match findTester($Bitmap2Filename,$Bitmap3Filename,$c24RGBFullMatch);Find the backspace button findTester($Bitmap2Filename,$Bitmap3Filename,$c24RGBPartialMatch);Find the backspace button with partial match findTester($Bitmap2Filename,$Bitmap3Filename,$c16RGBFullMatch);Find the backspace button findTester($Bitmap2Filename,$Bitmap3Filename,$c16RGBPartialMatch);Find the backspace button with partial match winactivate($calcHWND);Make sure calculator is active on the screen findTester("[ACTIVE]",$Bitmap3Filename,$c24RGBFullMatch);Find the backspace button findTester("[ACTIVE]",$Bitmap3Filename,$c24RGBPartialMatch);Find the backspace button with partial match findTester("[ACTIVE]",$Bitmap3Filename,$c16RGBFullMatch);Find the backspace button on active screen findTester("[ACTIVE]",$Bitmap3Filename,$c16RGBPartialMatch);Find the backspace button with partial match findTester("SCREEN",$Bitmap3Filename,$c24RGBFullMatch);Find the backspace button findTester("SCREEN",$Bitmap3Filename,$c24RGBPartialMatch);Find the backspace button with partial match findTester("SCREEN",$Bitmap3Filename,$c16RGBFullMatch);Find the backspace button findTester("SCREEN",$Bitmap3Filename,$c16RGBPartialMatch);Find the backspace button with partial match findTester($Bitmap4Filename,$Bitmap5Filename,$c16RGBFullMatch);Find the seven findTester($Bitmap4Filename,$Bitmap5Filename,$c16RGBPartialMatch);Find the seven with partial match findTester($Bitmap4Filename,$Bitmap6Filename,$c16RGBFullMatch);Find the cat findTester($Bitmap4Filename,$Bitmap6Filename,$c16RGBPartialMatch);Find the cat with partial match findTester($Bitmap1Filename,$Bitmap7Filename,$c24RGBFullMatch);Find the 3 of Autoit Homepage findTester($Bitmap1Filename,$Bitmap7Filename,$c24RGBPartialMatch);Find the 3 of Autoit Homepage findTester($Bitmap1Filename,$Bitmap7Filename,$c16RGBFullMatch);Find the 3 of Autoit Homepage findTester($Bitmap1Filename,$Bitmap7Filename,$c16RGBPartialMatch);Find the 3 of Autoit Homepage Else ;~ findTester($Bitmap4Filename,$Bitmap6Filename,$c24RGBFullMatch);Find the cat ;~ findTester($Bitmap1Filename,$Bitmap2Filename,$c24RGBFullMatch);Find the full calculatorscreen findTester("SCREEN",$Bitmap2Filename,$c24RGBFullMatch);Find the full calculatorscreen EndIf _GDIPlus_Shutdown() ;** Example end ** #EndRegion #Region actual functions ;=============================================================================== ; Function Name: findBMP ; Description: Finds a bitmap (.BMP) in another BMP file (other formats PNG and TIFF work also other formats less usefull GIF, JPEG, Exif, WMF, and EMF should work) ; Syntax: findBMP($BMP1, $BMP2, $MatchType=TRUE) ; ; Parameter(s): $BMP1 = Filename of bitmap to search in ; $BMP2 = Filename of bitmap to search for ; $MatchType = c24RGBFullMatch, c24RGBPartialMatch, c16RGBFullMatch, c16RGBPartialMatch ; ; Return Value(s): On Success: = Returns Array List ; On Failure: = @error 1 (Control was found but there was an error with the DLLCall) ; ; Author(s): JunkEW ; ; Note(s): ; * Its never an exact match even with TRUE as last few bits are disposed in algorithm and lines below ; are not checked under assumption that those are 99.99% of the time correct ; * locking bits overview http://www.bobpowell.net/lockingbits.htm ; ToDo: ; * Transparency (when search letters on a different background) http://www.winprog.org/tutorial/transparency.html ; * Match quicker by doing a bitblt with srcinvert when first line is found (instead of checking line by line) ; * $BMP1 and $BMP2 to be HBITMAP handle as input instead of filenames (will make searching within partial screen easier) ; Example(s): ; ;=============================================================================== Func findBMP($BMP1, $BMP2, $MatchType=$c24RGBFullMatch) Dim $fLine[1];Line number of found line(s), redimmed when second picture size is known Dim $BMP1Data="", $BMP1Width=0, $BMP1Height=0, $BMP1LineWidth=0; Dim $BMP2Data="", $BMP2Width=0, $BMP2Height=0, $BMP2LineWidth=0 Dim $foundAt = "", $matchPossible=FALSE, $matchedLines=0, $foundAtLeft=-1, $foundAtTop=-1 Dim $bestMatchLine=-1, $HighestMatchingLines=-1; For knowing when no match is found where best area is Dim $iPos=1; dim $imgBytes, $avData; local $iFuzzyDist, $searchFor, $iAbove, $bMatchPossible, $aboveLine local $j, $imgBits local $tDebug if ($MatchType=$c24RGBFullMatch) or ($matchtype=$c24RGBPartialMatch) then $imgBytes=3 Else $imgBytes=2 endif ; Load the bitmap to search in getImage($BMP1, $BMP1Data, $BMP1Width, $BMP1Height, $BMP1LineWidth, $imgBytes) $BMP1Data = BinaryToString($BMP1Data) ; Load the bitmap to find getImage($BMP2, $BMP2Data, $BMP2Width, $BMP2Height, $BMP2LineWidth, $imgBytes) ;Make it strings to be able to use string functions for searching $BMP2Data = BinaryToString($BMP2Data) if $bDebug=True Then ;~ START debugging ;Get some debugging information FileDelete(@ScriptDir & "\BMP1DATA.TXT") FileDelete(@ScriptDir & "\BMP2DATA.TXT") FileDelete(@ScriptDir & "\COMPAREDLINES.TXT") consolewrite($BMP1Width & @crlf) consolewrite($BMP1Height & @crlf) consolewrite($BMP1LineWidth & @crlf) consolewrite(stringlen($BMP1Data) & @crlf) consolewrite(".{" & $BMP1LineWidth & "}"& @CRLF) consolewrite(".{" & $BMP2LineWidth & "}"& @CRLF) $tDebug=StringRegExpReplace(_stringtohex($BMP1Data),"(.{" & $BMP1LineWidth*2 & "})","$1" & @CRLF) consolewrite(@error & @extended & @CRLF) FileWrite (@ScriptDir & "\BMP1DATA.TXT" , $tdebug) $tDebug=StringRegExpReplace(_stringtohex($BMP2Data),"(.{" & $BMP2LineWidth*2 & "})","$1" & @CRLF) consolewrite(@error & @extended & @CRLF) FileWrite (@ScriptDir & "\BMP2DATA.TXT" , $tdebug) ;~ END debugging EndIf ;For reference of line where in BMP2FindIn a line of BMP2Find was found If $BMP2Height = 0 Then SetError(1,0,0) Return False EndIf ReDim $fline[$BMP2Height] ;If exact match check every 1 line else do it more fuzzy (as most likely other lines are unique) if ($MatchType=$c24RGBFullMatch) or ($matchtype=$c16RGBFullMatch) Then $iFuzzyDist = 1 Else ;Check fuzzy every 10% of lines $iFuzzyDist = ceiling(($bmp2height * 0.1)) endIf $begin = TimerInit() ;Look for each line of the bitmap if it exists in the bitmap to find in ;~ Split bitmap to search in lines ;~ $avData=stringregexp($BMP2Data, ".{1," & $BMP2lineWidth & "}+", 3) dim $searchForRegEx $searchForRegEx = StringMid($BMP2Data, 1 + (0 * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes)) $searchForRegEx = $searchForRegEx & ".*" & StringMid($BMP2Data, 1 + ($BMP2Height * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes)) Local $aArray=stringregexp($BMP1Data,$searchForRegEx,3) if @error=0 Then consolewrite("Regex found " & TimerDiff($begin) & " ms " & @crlf) ;~ For $i = 0 To UBound($aArray) - 1 ;~ consolewrite("RegExp Test with Option 3 - " & $i & $aArray[$i]) ;~ Next endif For $i = 0 To $BMP2Height - 1 ;Minus imgbytes as last bits are padded with unpredictable bytes (24 bits image assumption) or 2 when 16 bits $searchFor = StringMid($BMP2Data, 1 + ($i * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes)) ;~ $searchfor=stringleft($avData[$i],$BMP2lineWidth - $imgBytes) $iPos = StringInStr($BMP1Data, $searchFor, 2, 1, $iPos) ; $iPos = StringInStr($BMP1Data, $searchFor) ;Look for all lines above if there is also a match ;Not doing it for the lines below as speed is more important and risk of mismatch on lines below is small $iAbove=1 if $iPos > 0 then $bMatchPossible=True $matchedLines=1;As first found line is matched we start counting ;Location of the match $foundAtTop = Int($iPos / $BMP1lineWidth) -$i $foundAtLeft = int(mod($iPos,$bmp1linewidth) / $imgBytes) Else ;~ No 1st line found so far nothing to start matching $bMatchPossible=false exitloop endif while (($i+$iAbove) <= ($BMP2Height -1)) and ($bMatchPossible=True) $searchFor = StringMid($BMP2Data, 1 + (($i + $iAbove) * $BMP2lineWidth), ($BMP2lineWidth - $imgBytes)) ;~ $searchfor = stringleft($avData[$i+$iAbove],$BMP2lineWidth - $imgBytes) $aboveLine = stringmid($BMP1Data,$iPos + ($iAbove * $BMP1LineWidth), ($BMP2LineWidth - $imgBytes)) if $bDebug=True Then ;~ START debugging $tDebug=StringRegExpReplace(_stringtohex($searchfor),"(.{8})","$1_") FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , $tDebug & @crlf) $tDebug=StringRegExpReplace(_stringtohex($aboveline),"(.{8})","$1_") FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , $tDebug & @crlf) If $aboveLine <> $searchFor Then FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , "** MISMATCH ** of line " & (1+$iAbove) & " in 2nd bitmap at line " & ($foundAtTop + $i) & @crlf) endIf ;~ END debugging EndIf if comparePicLine($aboveline,$searchfor) = false then $bMatchPossible=False ;To remember the area with the best match if $matchedLines >= $HighestMatchingLines Then $HighestMatchingLines = $matchedLines ;Best guess of location ;~ $foundAtTop = $fline[$i] + $i - $BMP2Height $foundAtTop = Int($iPos / $BMP1lineWidth);+ $i - $BMP2Height $bestMatchLine = Int($iPos / $BMP1lineWidth) EndIf ExitLoop EndIf $matchedLines=$matchedLines + 1 $iAbove=$iAbove+$iFuzzyDist WEnd ;If bMatchPossible is still true most likely we have found the bitmap if $bmatchPossible = True then ;~ ConsoleWrite("Could match top: " & $foundAtTop & " left: " & $foundAtLeft & " in " & TimerDiff($begin) / 1000 & " seconds" & @LF) ; MouseMove($foundatleft,$foundatTop) exitloop else ;~ consolewrite("i not matched " & $ipos & " " & $matchedlines & @crlf ) EndIf Next ;For some debugging of time ; if $bMatchPossible = True Then ; ConsoleWrite("Searching took " & TimerDiff($begin) / 1000 & " seconds " & @LF) ; Else ; ConsoleWrite("NOT FOUND Searching took " & TimerDiff($begin) / 1000 & " seconds" & @LF) ; endif ;Return an error if not found else return an array with all information if $bMatchPossible = False Then SetError(1, 0, 0) endif ; return stringsplit($bMatchPossible & ";" & $matchedLines & ";" & $foundAtLeft & ";" & $foundAtTop & ";" & $bmp2width & ";" & $BMP2Height & ";" & $HighestMatchingLines & ";" & $bestMatchLine,";") return $bMatchPossible & ";" & $matchedLines & ";" & $foundAtLeft & ";" & $foundAtTop & ";" & $bmp2width & ";" & $BMP2Height & ";" & $HighestMatchingLines & ";" & $bestMatchLine EndFunc;==>findBMP func comparePicLine($s1,$s2) local $sLen, $iMatched if $s1=$s2 Then return True Else $iMatched=0 $sLen=stringlen($s1) for $tJ=1 to $slen if stringmid($s1,$tJ,1)=stringmid($s2,$tJ,1) Then $iMatched=$imatched+1 Else if $bDebug=True Then ;~ START debugging FileWrite (@ScriptDir & "\COMPAREDLINES.TXT" , "Debug mismatch pixel " & $tj & "=" & int($tj/4) & @crlf) endif endif Next if ($iMatched / $sLen ) > $cMatchLinePercentage then return true Else return false endif EndIf endfunc Func GetImage($BMPFile, byref $BMPDataStart, byref $Width, byRef $Height, byref $Stride, $imgBytes=3) local $Scan0, $pixelData, $hbScreen, $pBitmap, $pBitmapCap, $handle ; Load the bitmap to search in If $BMPFile="SCREEN" Then $hbScreen=_ScreenCapture_Capture("",0,0,-1,-1,False) $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap Else ;try to get a handle $handle = WinGetHandle($BMPFile) If @error Then ;Assume its an unknown handle so correct filename should be given $pBitmap = _GDIPlus_BitmapCreateFromFile($BMPFile) Else $hbScreen=_ScreenCapture_CaptureWnd("",$handle,0,0,-1,-1,False) $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap EndIf EndIf ;Get $tagGDIPBITMAPDATA structure ;~ ConsoleWrite("Bitmap Width: " & _GDIPlus_ImageGetWidth($pBitmap) & @CRLF ) ;~ ConsoleWrite("Bitmap Height: " & _GDIPlus_ImageGetHeight($pBitmap) & @CRLF) ;~ 24 bits (3 bytes) or 16 bits (2 bytes) comparison if ($imgBytes=2) then $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF16RGB555) ;~ $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF32ARGB) Else $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF24RGB) ;~ $BitmapData= _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF32ARGB) endIf If @ERROR Then MsgBox(0,"","Error locking region " & @error) $Stride = DllStructGetData($BitmapData, "Stride");Stride - Offset, in bytes, between consecutive scan lines of the bitmap. If the stride is positive, the bitmap is top-down. If the stride is negative, the bitmap is bottom-up. $Width = DllStructGetData($BitmapData, "Width");Image width - Number of pixels in one scan line of the bitmap. $Height = DllStructGetData($BitmapData, "Height");Image height - Number of scan lines in the bitmap. $PixelFormat = DllStructGetData($BitmapData, "PixelFormat");Pixel format - Integer that specifies the pixel format of the bitmap $Scan0 = DllStructGetData($BitmapData, "Scan0");Scan0 - Pointer to the first (index 0) scan line of the bitmap. $pixelData = DllStructCreate("ubyte lData[" & (abs($Stride) * $Height-1) & "]", $Scan0) $BMPDataStart = $BMPDataStart & DllStructGetData($pixeldata,"lData") _GDIPlus_BitmapUnlockBits($pBitmap, $BitmapData) _GDIPlus_ImageDispose ($pBitmap) _WinAPI_DeleteObject ($pBitmap) EndFunc;==>GetImage ; Draw rectangle on screen. Func _UIA_DrawRect($tLeft, $tRight, $tTop, $tBottom, $color = 0xFF, $PenWidth = 4) Local $hDC, $hPen, $obj_orig, $x1, $x2, $y1, $y2 $x1 = $tLeft $x2 = $tRight $y1 = $tTop $y2 = $tBottom $hDC = _WinAPI_GetWindowDC(0) ; DC of entire screen (desktop) $hPen = _WinAPI_CreatePen($PS_SOLID, $PenWidth, $color) $obj_orig = _WinAPI_SelectObject($hDC, $hPen) _WinAPI_DrawLine($hDC, $x1, $y1, $x2, $y1) ; horizontal to right _WinAPI_DrawLine($hDC, $x2, $y1, $x2, $y2) ; vertical down on right _WinAPI_DrawLine($hDC, $x2, $y2, $x1, $y2) ; horizontal to left right _WinAPI_DrawLine($hDC, $x1, $y2, $x1, $y1) ; vertical up on left ; clear resources _WinAPI_SelectObject($hDC, $obj_orig) _WinAPI_DeleteObject($hPen) _WinAPI_ReleaseDC(0, $hDC) EndFunc ;==>_UIA_DrawRect #EndRegion  Edit: Speed and partial match enhancements. Search is within a split second especially when partial match is turned on Edit2: Cleaned up code in a nicer way, returns best match results even if find is false Edit 24 march 2008: Becomes dependent on beta v3.2.11.5 (beta) ;$iPos = StringInStr($BMP1Data, $searchFor) $iPos = StringInStr($BMP1Data, $searchFor, 2, 1, $iPos+1) Edit 13th june 2008 updated for new autoit version Edit 31st august 2008 added some reference links Edit 04 sept 2008 completely rewritten using GDI plus functions and speedenhancements by using different 24 bits and 16 bits pictures Edit 23 nov 2008 small additional check on bmp2height beeing 0 Edit 1st dec 2008 fixed top/left, added [active] find on active screen, work with window special description Edit 20st feb 2009 fixed Stringinstr to use additional parameters, now matchin b/w pictures better $iPos = StringInStr($BMP1Data, $searchFor, 2, 1, $iPos) Edit jan 5th 2013 full updated all working under Windows 7 / 64 bits (not tested under windows XP but no reason to be broken) edit jan 20st 2018 checked with windows 10, quickfix as calculator is not recognized by AU3Inf, basic algorithm still works fine see also works with BMP, PNG and TIFF added threshold for small bits off on a single line (unclear why GDI writes those pixels/shade variation) speed I assume is no issue for nowadays CPU power pics.zip Edited January 20, 2018 by junkew check with W10 and 3.3.14.2 HansHenrik, ScrapeYourself and BatMan22 2 1 FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets Link to comment Share on other sites More sharing options...
sbgwb0769 Posted March 18, 2008 Share Posted March 18, 2008 very very thank you!hehe~~~~~~~~~~~ Link to comment Share on other sites More sharing options...
Armand Posted March 20, 2008 Share Posted March 20, 2008 ummzz.... not quit accurate working.... try finding pic A inside pic B in these:A.B.Could be very useful if it could work on such pictures... [u]My Au3 Scripts:[/u]____________(E)Lephant, A Share download manager (RS/MU etc)Http1.1 Console, The Ez Way!Internet Reconnection Automation Suite & A Macro Recording Tool.SK's Alarm Clock, Playing '.MP3 & .Wav' Files._________________Is GOD a mistake of the Humanity Or the Humanity is a mistake of GOD ?! Link to comment Share on other sites More sharing options...
junkew Posted March 20, 2008 Author Share Posted March 20, 2008 (edited) Searching the picture as suggested just works on my machine. Remove the 2 right vertical whitespaces of your picture to search. It seems that the 2 pictures are slightly different. Finding the other characters I would suggest to only match on the bottom 8 lines as it looks like the cats are not there and its unique enough to recognize what you want. Other suggestion I can give take a look at the hexadecimal values of the picture and make the line matching a little more advanced. As your pictures are black and white it should not be very hard. Study the code and you can make it work to not only match a rectangle but search for boundary. Offcourse its hard to get the cats out of it and if it becomes coloured it will become even harder to match. Edit: * Based on the seven sample I modified the UDF in such a way that if result is false you still get the best top/left location on a best guess basis. * Works only with BMP files not with PNG to see it working you should save the PNG as seven.BMP Something like Global Const $Bitmap4Filename = @TempDir & "\SEVEN.bmp" $start = TimerInit() $tResult=findBMP($Bitmap1Filename,$Bitmap4Filename,TRUE) ConsoleWrite($tResult & "** Full seven totalmatch time elapsed: " & TimerDiff($start) / 1000 & " seconds" & @LF) Edited March 21, 2008 by junkew FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets Link to comment Share on other sites More sharing options...
Armand Posted March 20, 2008 Share Posted March 20, 2008 @junkew how could there be any difference if picture A is in fact a slice of B ?! i am almost definite that there is a mistake in the comparing algorithm... anyhow, just trying to help you improving you UDF thanks in advance! [u]My Au3 Scripts:[/u]____________(E)Lephant, A Share download manager (RS/MU etc)Http1.1 Console, The Ez Way!Internet Reconnection Automation Suite & A Macro Recording Tool.SK's Alarm Clock, Playing '.MP3 & .Wav' Files._________________Is GOD a mistake of the Humanity Or the Humanity is a mistake of GOD ?! Link to comment Share on other sites More sharing options...
Impulse08 Posted March 20, 2008 Share Posted March 20, 2008 @junkewhow could there be any difference if picture A is in fact a slice of B ?!i am almost definite that there is a mistake in the comparing algorithm...anyhow, just trying to help you improving you UDF thanks in advance!I can see what he's saying. Just by do a copy image and pasting it into MSPaint, I see that your first 7 cuts off part of the cat on the right side. So the algorithm to find the whole image A is failing because there is some extra whitespace in image A on the right versus image B.Try it out for yourself from your post. Right-click on the image and copy image. Then paste into MSPaint. Then do the same for image A. You'll see that image A is slightly larger than the 7 in B -- If the apocalypse comes... beep me. Link to comment Share on other sites More sharing options...
Oldschool Posted March 21, 2008 Share Posted March 21, 2008 Well done!! This is awesome work!! I've been trying to come up with a way, but you beat me to it. Link to comment Share on other sites More sharing options...
Armand Posted March 24, 2008 Share Posted March 24, 2008 does anyone has anyone have any example with pre'made .bmp files ?! [u]My Au3 Scripts:[/u]____________(E)Lephant, A Share download manager (RS/MU etc)Http1.1 Console, The Ez Way!Internet Reconnection Automation Suite & A Macro Recording Tool.SK's Alarm Clock, Playing '.MP3 & .Wav' Files._________________Is GOD a mistake of the Humanity Or the Humanity is a mistake of GOD ?! Link to comment Share on other sites More sharing options...
junkew Posted March 24, 2008 Author Share Posted March 24, 2008 does anyone has anyone have any example with pre'made .bmp files ?!What do you mean? * http://www.autoitscript.com/forum/index.ph...=pixelchecksum) gives an idea how to store an BMP.If you remove all code between ; ** Example start ** and ; ** Example end ** and put in something like$tResult=findBMP("C:\TEMP\BITMAP1.BMP", "C:\TEMP\BITMAP2.BMP",TRUE)It should work although not thoroughly tested with BMP not created by AutoIt.Known Issues* Rounding rectangles on window can cause mismatch as backgroundcolor on corner can be different compared to moment when screenshot was taken. FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets Link to comment Share on other sites More sharing options...
Armand Posted March 24, 2008 Share Posted March 24, 2008 well.. i meant that you'll try and do something more like what i've done... post pic A. [PICTURE] post pic B. [PICTURE] and the script that handles both of these. Cause i'd really really really like to use this UDF for non-autoit pictures... [u]My Au3 Scripts:[/u]____________(E)Lephant, A Share download manager (RS/MU etc)Http1.1 Console, The Ez Way!Internet Reconnection Automation Suite & A Macro Recording Tool.SK's Alarm Clock, Playing '.MP3 & .Wav' Files._________________Is GOD a mistake of the Humanity Or the Humanity is a mistake of GOD ?! Link to comment Share on other sites More sharing options...
junkew Posted March 24, 2008 Author Share Posted March 24, 2008 (edited) I tried and actually the algorithm seems to be misbehaving on black/white pictures. (fixed)The less color in the picture(s) the more risk it will not properly recognize (Even if saved as 24 bits BMP)I will rewrite the algoritm as soon as I have new beta installed as to fix/enhance the algorithm I am in need of stringinstr function where I can give startposition or I have to put the bitmaps in an array for each line of data which will slow down performance (Not sure how long it takes to load a desktop of 1280*1024 into an array of 1280 lines and with an array I need to do more comparisons as done in current algorithm).* http://www.autoitscript.com/forum/index.ph...mp;#entry496554Edit 24 march 2008: Modified first posting and added attachments and samples on how to handle Edited March 24, 2008 by junkew FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets Link to comment Share on other sites More sharing options...
junkone Posted March 26, 2008 Share Posted March 26, 2008 i get error AVector: [] Out of bounds. in C:\Program Files\AutoIt3\Include\GDIPlus.au3 (2688) : ==> Subscript used with non-Array variable.: Return $aResult[0] <> 0 Return $aResult^ ERROR >Exit code: 1 Time: 275.393 I am not sure what this means. appreiate any help. I am running it on windows 2000 prof Link to comment Share on other sites More sharing options...
junkew Posted March 26, 2008 Author Share Posted March 26, 2008 First try if a script like below works #include <ScreenCapture.au3> _ScreenCapture_Capture(@tempdir & "\screen.bmp",0,0,-1,-1,false) If this gives the same problem you most likely have not GDIPLUS.DLL on the right path location FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets Link to comment Share on other sites More sharing options...
junkone Posted March 28, 2008 Share Posted March 28, 2008 bingo. gdiplus is not installed on win2k by default. i installed it and it works beautifully. however i havea smaill issue. i used the app to take a screenshot of my app. then i am using the same to take subsequent screenshots to compare and see if it is the same. however this is what i find. $tResult=findBMP($BitmapOriginalFile,$BitmapCurrentFile,False) it always returns "False;1;0;0;808;627;2;22" i would have imagined that it would return true as both the images are supposed to be taken the same way without any change. strange is the way it works. Link to comment Share on other sites More sharing options...
Toady Posted March 28, 2008 Share Posted March 28, 2008 this is neat, good work! www.itoady.com A* (A-star) Searching Algorithm - A.I. Artificial Intelligence bot path finding Link to comment Share on other sites More sharing options...
junkew Posted March 29, 2008 Author Share Posted March 29, 2008 bingo. gdiplus is not installed on win2k by default. i installed it and it works beautifully. however i havea smaill issue. i used the app to take a screenshot of my app. then i am using the same to take subsequent screenshots to compare and see if it is the same. however this is what i find. $tResult=findBMP($BitmapOriginalFile,$BitmapCurrentFile,False) it always returns "False;1;0;0;808;627;2;22" i would have imagined that it would return true as both the images are supposed to be taken the same way without any change. strange is the way it works. Try code like below. What happens if you compare the same screen bitmap to each other. On my system it works and returns True (takes about 30 seconds to match a full screen in all details). Remember that when anything changes on the screen you have a mismatch. Moving a mouse can already change highlighted pixels of a button, taskbar at the bottom, animated gifs, characters typed somewhere. Global Const $Bitmap1Filename = @TempDir & "\FULLSCREEN.bmp" ; Capture full screen $begin = TimerInit() _ScreenCapture_Capture($Bitmap1Filename,0,0,-1,-1,false) ConsoleWrite("Saving full screen took " & TimerDiff($begin) & " milliseconds " & @LF) ;Do the actual find ;Find the full screen itself $start = TimerInit() $tResult=findBMP($Bitmap1Filename,$Bitmap1Filename,TRUE) ConsoleWrite($tResult & "** Full screen totalmatch time elapsed: " & TimerDiff($start) / 1000 & " seconds" & @LF) FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets Link to comment Share on other sites More sharing options...
junkone Posted March 31, 2008 Share Posted March 31, 2008 i do not capture the full screen. i onlyh capture the specific window. obviously there maybe changes if i did fullscreen because of the clock at the bottom of my window. $targetHWND = WinGetHandle("Presentation Server") $begin = TimerInit() winactivate($targetHWND) _ScreenCapture_CaptureWnd($BitmapCurrentFile, $targetHWND) ConsoleWrite("Saving NR window " & TimerDiff($begin) & " milliseconds " & @LF) $WshShell.LogEvent ($INFORMATION, "Saved NR L window ") then i compare it with a copy of the window captured using teh same method previously $tResult=findBMP($BitmapOriginalFile,$BitmapCurrentFile,False) it alwys returns false with the same difference False;1;0;0;808;627;2;22" Link to comment Share on other sites More sharing options...
junkew Posted April 1, 2008 Author Share Posted April 1, 2008 i do not capture the full screen...it alwys returns false with the same difference False;1;0;0;808;627;2;22"1. Try to do the same for calculator screen take twice a calculator snapshot. On my system it returns correct result2. Attach your 2 bitmapfiles to the post I can check then if the algorithm has an issue.Actually the result you get back means that there actually is only 1 matching line. To me it suggest(s) that your picture is off by one line or one pixel column and the second screenshot is not fully equal to the previous one taken.Is the title on your window exactly the same (no spaces added etc.)? FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets Link to comment Share on other sites More sharing options...
Armand Posted April 1, 2008 Share Posted April 1, 2008 @junkew any luck there with the external monochrome bitmaps algorithm ? [u]My Au3 Scripts:[/u]____________(E)Lephant, A Share download manager (RS/MU etc)Http1.1 Console, The Ez Way!Internet Reconnection Automation Suite & A Macro Recording Tool.SK's Alarm Clock, Playing '.MP3 & .Wav' Files._________________Is GOD a mistake of the Humanity Or the Humanity is a mistake of GOD ?! Link to comment Share on other sites More sharing options...
junkew Posted April 1, 2008 Author Share Posted April 1, 2008 (edited) @junkewany luck there with the external monochrome bitmaps algorithm ?I modified the first post and included the samples you gave to work with. You can work with black and white pictures as long as you store them as 24 bits bitmap.With monochrome bitmaps the algorithm will only work if you align on 8 bits (which then becomes a byte and then it will work as algorithm is based on byte boundaries) but still its tricky as the picture you are searching in should also be on an 8 bits boundary.The "problem" with monochrome is that each pixel is bit encoded and as such 8 pixels are in a byte whereas in a 24 color bitmap you have 3 bytes for 1 pixel.If you want to find a monochrome bmp on a 24 bmp colored screen I would suggest to first convert it to a 24 bits color bitmap.To be able to do 2 color bitmap searching I have to shift each string each time 8 times and compare it 8 times with a shifted string of the picture to search in.Imagine that you want to find 00010001 on a screen of 32 pixels width which is 4 bytes like 00000001 00010000 00000000 10010000I then would have to shift like below 00010001shift 1: 00000010 00100000 00000001 00100000 would not match as the matching string is over the first 2 bytesshift 2: 00000100 01000000 00000010 01000000 would not match as the matching string is over the first 2 bytesshift 3: 00001000 10000000 00000100 10000000 would not match as the matching string is over the first 2 bytesshift 4: 00010001 00000000 00001001 00000000 would matchIncorporating that kind of logic is making it much more complicated then to switch to a getPixel algorithm Edited April 1, 2008 by junkew FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now