Google Earth + Osint = Pixel Search?

[Fractured]

Long time no see!! Changed jobs and moved on...

Quick question..if I want to search for a specific pattern in an image on google earth desk top...would I use pixel search? Any hints?

[mistersquirrle]

PixelSearch could work, but the main problem you're likely to encounter is what if the pattern is a different size, or rotation? PixelSearch is fairly fast, until you start searching for a lot of pixels at a time, especially if you're looking over a large area. You might have better luck looking into using something like OpenCV or some other established image search program.

If you want to use AutoIt to do it, I would recommend looking into taking your own screenshot and doing all of the "PixelSearch" manually, since doing it manually you can avoid the high cost of calling PixelSearch multiple times: taking the screenshot. Depending on the size of the area that you're searching/capturing, each PixelSearch could take 20-30ms, and if you're trying to do this 'quickly', that's IMO pretty expensive.

To do a 'manual' pixel search, check out:

• _ScreenCapture_Capture: https://www.autoitscript.com/autoit3/docs/libfunctions/_ScreenCapture_Capture.htm
• _GDIPlus_BitmapCreateFromHBITMAP: https://www.autoitscript.com/autoit3/docs/libfunctions/_GDIPlus_BitmapCreateFromHBITMAP.htm
• _GDIPlus_BitmapLockBits: https://www.autoitscript.com/autoit3/docs/libfunctions/_GDIPlus_BitmapLockBits.htm

There may be some other things needed, the _GDIPlus_BitmapLockBits has a good example for starting what you'd need to do with iterating over the bits. I haven't done this method personally in AutoIt, but that's the start of what you'd need to do.

[Fractured]

kool, thank you. I was hoping there was a way to do "live" but I will play with the screenshot idea!

 

[mistersquirrle]

You can definitely do it "live" using the 'screenshot'. You're not saving the screenshot to a file, or loading it from a file. The _ScreenCapture_Capture gives you an object that you can use/pass into those other functions. So it should be as "live" as PixelSearch, maybe a little slower since you're calling a few different functions versus PixelSearchs 'native' code calls. But you'll start to save a LOT of time starting from the 2nd search for a pixel, over PixelSearch (since you can search on the same screenshot, instead of PixelSearch taking a screenshot each time).

Also while you can do everything in AutoIt, there's going to be a lot of iterating over arrays, so if you know VB or C# at all, I strongly recommend that you check out: 

Using VB or C# to do array actions will give you a massive speed boost over doing it in AutoIt

Edit: And just FYI, I don't know how PixelSearch does its stuff, but almost guaranteed that it's using GDI or a similar API to take a screenshot, and doing a 1 time search on that screenshot. PixelSearch's biggest weakness is that you can't have it search over the screenshot that it just took more than once.

Edited February 22, 2023 by mistersquirrle

[Nine]

You may be tempted to use my Screen Scraping UDF (see my signature).

[mistersquirrle]

Hmm, I've been doing some testing (because this is an area I've been meaning to look into doing), and doing pixel searching in pure AutoIt is actually very slow because of the speed of dealing with arrays/DllStructs. Maybe I'm doing it wrong, but looping over large arrays/DllStruct/data just take a looong time. Anyway, here's my example that I did some testing with: 

expandcollapse popup

#include <ScreenCapture.au3>
#include <GDIPlus.au3>
#include <GDIPlusConstants.au3>
#include <WindowsConstants.au3>

Global $aiColors[] = [0xFFFF0001, 0xFFFF0001, 0xFFFF0001, 0xFFFF0001, 0xFFFF0001, 0xFFFF0001, 0xFFFF0001, 0xFFFFFFFF] ; Format is 0xAARRGGBB
Global $iW = Ceiling(@DesktopWidth/4), $iH = Ceiling(@DesktopHeight/2)
Global $hTimer = TimerInit()
For $iColor = 0 To UBound($aiColors) - 1
    PixelSearch(0, 0, $iW, $iH, $aiColors[$iColor], 0, 0)
Next
ConsoleWrite('PixelSearch multiple times: ' & Round(TimerDiff($hTimer), 2) & ' ms' & @CRLF)

$hTimer = TimerInit()
_GDIPlus_Startup()
Global $hHBitmap = _ScreenCapture_Capture('', 0, 0, $iW, $iH, False)
ConsoleWrite('_ScreenCapture_Capture: ' & Round(TimerDiff($hTimer), 2) & ' ms' & @CRLF)
Global $hBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hHBitmap)
Global $tBitmapData = _GDIPlus_BitmapLockBits($hBitmap, 0, 0, $iW, $iH, $GDIP_ILMREAD, $GDIP_PXF32PARGB)

Global $iBitScan0 = DllStructGetData($tBitmapData, "Scan0") ;get scan0 (pixel data) from locked bitmap
Global $tPixel = DllStructCreate("int[" & $iW * $iH & "];", $iBitScan0)
Global $iPixel, $iRowOffset

;~ #cs DllStructGetData
Global $iMatches = 0
Global $hLoopTimer = TimerInit()
For $iY = 0 To $iH - 1
    $iRowOffset = $iY * $iW + 1
    For $iX = 0 To $iW - 1 ;get each pixel in each line and row
        $iPixel = DllStructGetData($tPixel, 1, $iRowOffset + $iX) ;get pixel color
;~      If $iX = 100 And Mod($iY, 100) = 0 Then ConsoleWrite($iX & ', ' & $iY & ' (' & $iRowOffset + $iX & '): ' & Hex($iPixel, 6) & @CRLF)
        For $iColor = 0 To UBound($aiColors) - 1
            If $iPixel = $aiColors[$iColor] Then ;$iSearchPixel Then
                $iMatches += 1
;~              ConsoleWrite('Found pixel at: ' & $iX & ', ' & $iY & ', color: ' & Hex($aiColors[$iColor], 8) & ', $iPixel: ' & Hex($iPixel, 8) & @CRLF)
            EndIf
        Next
    Next
Next
ConsoleWrite('$iMatches: ' & $iMatches & @CRLF)
ConsoleWrite('$hLoopTimer: ' & Round(TimerDiff($hLoopTimer), 2) & ' ms' & @CRLF)
;~ #ce DllStructGetData

_GDIPlus_BitmapUnlockBits($hBitmap, $tBitmapData) ;unlocks a portion of a bitmap that was locked by _GDIPlus_BitmapLockBits
_GDIPlus_Shutdown()

The advantage that this gives is it's getting all occurances of the search color (white, the last value in the array), the disadvantages are speed (~24x longer for 1 run against 8 runs of PixelSearch) and in this version there's no way to do shade variation/tolerance. Here's the output:

PixelSearch multiple times: 133.54 ms
_ScreenCapture_Capture: 16.22 ms
$iMatches: 37465
$hLoopTimer: 3199.34 ms

So if you're looking for all occurances of a color, then great. Otherwise unless you use something that can process the data faster (possibly @Nines UDF, or some DotNet stuff as mentioned earlier), this method is probably pretty bad.

 

Edit: Also in addition to the DllStructGetData I also tried reading the binary data directly with StringMid (terrible performance), StringRegEx to array and parsing that (meh parsing speed, okay reading speed, combined slower than DllStructGetData), and DllStructGetData into a 2D array, then reading that (array reading was faster than 1D, generation was much slower)

Edited February 23, 2023 by mistersquirrle