Ballot Paper Analyzer

[tz45]

Hello,

Currently I'm trying to work on a project that should allow me to find the biggest matches quite easily by analyzing photos of some kind of "ballot paper".

Here is a picture of a possible ballot.

At the moment I'm looking for a way to get the position of the red dots and the black crosses. I'm trying to use PixelSearch, but would be very grateful for your tips.

The final goal would be to convert every single "ballot paper" (on the photo you can see 6 next to each other) into an array with the respective selections ($array = [1, 9, 89]).

Thank you for your tips

Edited October 18, 2019 by tz45

[Exit]

19 hours ago, tz45 said:

($array = [1, 9, 89]).    -->   ($array = [1, 9, 87]).   ???

 

[tz45]

As I'm new to this forum I can't edit my post yet

[tz45]

I have to admit that I'm quite astonished that I didn't even get a single hint or tip here

Meanwhile, I solved it myself by using FastFind. There are now a total of 12 red dots on the ballot paper helping the algorithm to find the corners and exact coordinates of the black dots using a little analytical geometry and arithmetic. 

 As o utput I get an array with all results, which I can then use further.

[dmob]

Would you care to show how you did it?

 

[tz45]

EDITED

 

Content of

ExtractDots.au3

expandcollapse popup

#include <Array.au3>
#include <CustomMsgBox.au3>
#include <FastFind.au3>
#include <File.au3>
#include <GuiConstants.au3>
#include <GUIConstantsEx.au3>
#Include <GDIPlus.au3>
#Include <WinAPI.au3>

Global $hGui, $hgraphic
Global $w = 800
Global $h = 600
Global $pos[0]
Global $grid[12][2]
Global $dotValues[0]
Global $colorThreshold = [20000, 25000]

Func ExtractDots($imgFile)
    Do
        Global $dotValues[0]

        $tmpFile = PrepareImg($imgFile)
        Gui($tmpFile)

        for $i = 0 to 11
            FindDot(1, $i)
        Next

        for $i = 1 to 6
            SetBallot($i)

            Local $dots[0][2]
            While 1
                $result = FindDot()

                if $result == false then _
                    ExitLoop

                _ArrayAdd($dots, $result[0] & "|" & $result[1])
            WEnd

            DotsToValue($i, $dots)
        Next

        $isDone = AdaptColors()
    Until $isDone

    _GDIPlus_Shutdown()
    GUIDelete()
    DirRemove("tmp", 1)

    return $dotValues
EndFunc

Func AdaptColors()
    $colorReturn = xMsgBox(16+0x200, _
        "Adapt Color?", "", _
        "Black <", "Done", "> Red", _
        Default, $pos[1] + $pos[3] + 5)

    if $colorReturn = 7 then _
        return true

    $colorIndex = ($colorReturn == 6) _
                    ? 0 _
                    : 1

    $fuzzReturn = xMsgBox(16+0x200, _
        "Adapt Fuzz?", "", _
        "Less <", "Done", "> More", _
        Default, $pos[1] + $pos[3] + 5)

    $factor = ($fuzzReturn == 6) _
                    ? -1 _
                    : 1
    $colorThreshold[$colorIndex] += $factor * 2500

    return false
EndFunc

Func PrepareImg($imgFile)
    Local $sDrive = "", $sDir = "", $sFileName = "", $sExtension = ""
    Local $aPathSplit = _PathSplit($imgFile, $sDrive, $sDir, $sFileName, $sExtension)
    $tmpFile = @ScriptDir & "\tmp\" & $sFileName & $sExtension

    Local $magickCmds = [ _
        "magick convert """ & $imgFile & """ -fuzz " & $colorThreshold[1] & " -fill red -opaque red """ & $tmpFile & """", _
        "magick convert """ & $tmpFile & """ -fuzz " & $colorThreshold[0] &" -fill black -opaque black """ & $tmpFile & """" ]

    DirCreate("tmp")
    for $cmd in $magickCmds
        RunWait($cmd, @ScriptDir, @SW_HIDE)
    Next

    return $tmpFile
EndFunc

Func Gui($tmpFile)
    _GDIPlus_Startup()
    Global $hGui
    $hBitmap = _GDIPlus_BitmapCreateFromFile($tmpFile)

    $hGui = GUICreate("ExtractDots", $w, $h, -1, -1, $WS_POPUP)
    GUISetState()

    $pos = WinGetPos("ExtractDots")

    $hgraphic = _GDIPlus_GraphicsCreateFromHWND($hGui)
    _GDIPlus_GraphicsDrawImageRect($hgraphic, $hBitmap, 0, 0, $w, $h)
    _GDIPlus_BitmapDispose($hBitmap)

    FFSetWnd($hGui)
    FFSnapShot()
EndFunc

Func DrawPoint($coords, $colorIndex = 0)
    Local $colors = [ _
        0xFF0ca2f0, _
        0xFFFFFF00, _
        0xFFFF0000, _
        0xFFFF00cc, _
        0xFF66FF00  _
    ]

    $colorIndex = Mod($colorIndex, UBound($colors))
    $color = $colors[$colorIndex]

    $hBrush = _GDIPlus_BrushCreateSolid($color)
    _GDIPlus_GraphicsFillEllipse($hgraphic, _
        $coords[0] - 5, _
        $coords[1] - 5, _
        10, 10, $hBrush)
EndFunc

Func DotsToValue($ballotIndex, $dots)
    $corners = GetCorners($ballotIndex)

    $first = SingleCorner($corners, 0)
    $second = SingleCorner($corners, 1)
    $third = SingleCorner($corners, 2)
    $fourth = SingleCorner($corners, 3)
    DrawPoint($first)
    DrawPoint($second)
    DrawPoint($third)
    DrawPoint($fourth)

    Local $values[0]
    for $i = 0 to UBound($dots) - 1
        Local $dot[0]
        _ArrayAdd($dot, $dots[$i][0] & "|" & $dots[$i][1])
        DrawPoint($dot, 1)

        $xVanishingPoint = GetPOI($first, $third, $second, $fourth)
        if Not $xVanishingPoint then
            $vector = VectorCalc($first, '-', $third)
            $xVanishingPoint = VectorCalc($dot, '+', $vector)
        EndIf

        $yVanishingPoint = GetPOI($first, $second, $third, $fourth)
        if Not $yVanishingPoint then
            $vector = VectorCalc($first, '-', $second)
            $yVanishingPoint = VectorCalc($dot, '+', $vector)
        EndIf

        $xPOI = GetPOI($first, $second, $dot, $xVanishingPoint)
        $yPOI = GetPOI($first, $third, $dot, $yVanishingPoint)

        ;DrawPoint($xPOI, 4)
        ;DrawPoint($yPOI, 4)

        $xVector = VectorCalc($xPOI, '-', $first)
        $yVector = VectorCalc($yPOI, '-', $first)

        $xLength = VectorCalc($xVector, '|')
        $top = VectorCalc($second, '-', $first)
        $topLength = VectorCalc($top, '|')
        $xPercentage = $xLength / $topLength
        $xDigit = Round($xPercentage * 11)

        $yLength = VectorCalc($yVector, '|')
        $left = VectorCalc($third, '-', $first)
        $leftLength = VectorCalc($left, '|')
        $yPercentage = $yLength / $leftLength
        $yDigit = Round($yPercentage * 11)

        $value = ($yDigit - 1)*10 + $xDigit

        _ArrayAdd($values, $value)
    Next

    _ArraySort($values)
    _ArrayAdd($dotValues, _ArrayToString($values, ","))
EndFunc

Func GetPOI($startPoint1, $endPoint1, $startPoint2, $endPoint2)
        $a = $startPoint1[0]
        $b = $startPoint1[1]
        $vector1 = VectorCalc($endPoint1, '-', $startPoint1)
        $c = $vector1[0]
        $d = $vector1[1]

        $e = $startPoint2[0]
        $f = $startPoint2[1]
        $vector2 = VectorCalc($endPoint2, '-', $startPoint2)
        $m = $vector2[0]
        $n = $vector2[1]

        if $d*$m - $c*$n == 0 then _
            return false

        $s = (-$n*($e - $a) + $m*($f - $b)) / ($d*$m - $c*$n)
        $x = $a + $s*$c
        $y = $b + $s*$d

        Local $result[2] = [$x, $y]
        return $result
EndFunc

Func SingleCorner($array, $index)
    Local $result[2]

    for $i = 0 to 1
        $result[$i] = $array[$index][$i]
    Next

    return $result
EndFunc

Func SetBallot($index)
    $corners = GetCorners($index)

    SetArea( _
        $corners[0][0], $corners[0][1], _
        $corners[3][0], $corners[3][1])
EndFunc

Func GetCorners($index)
    Local $corners[4]

    $missedSquare = Floor(($index - 1) / 3)
    $corners[0] = $index - 1 + $missedSquare
    $corners[1] = $corners[0] + 1
    $corners[2] = $index + 3 + $missedSquare
    $corners[3] = $corners[2] + 1

    Local $result[4][2]
    for $i = 0 to 3
        $entryIndex = $corners[$i]
        $cornerCoordinates = GetGridEntry($entryIndex)

        for $j = 0 to 1
            $result[$i][$j] = $cornerCoordinates[$j]
        Next
    Next

    return $result
EndFunc

Func SetGridEntry($array, $index)
    for $i = 0 to 1
        $grid[$index][$i] = $array[$i]
    Next
EndFunc

Func GetGridEntry($index)
    Local $array[2]

    For $i = 0 to 1
        $array[$i] = $grid[$index][$i]
    Next

    return $array
EndFunc

Func VectorCalc($par1, $operation, $par2 = 0)
    switch $operation
        case '+'
            Local $result[2]
            for $i = 0 to 1
                $result[$i] = $par1[$i] + $par2[$i]
            Next
        case '-'
            Local $result[2]
            for $i = 0 to 1
                $result[$i] = $par1[$i] - $par2[$i]
            Next
        case '*'
            Local $result[2]
            for $i = 0 to 1
                $result[$i] = $par1 * $par2[$i]
            Next
        case '|'
            Local $result = Sqrt($par1[0]^2 + $par1[1]^2)
    EndSwitch

    return $result
EndFunc

Func SetArea($a = -1, $b = -1, $c = -1, $d = -1)
    FFResetExcludedAreas()

    if $a + $b + $c + $c == -4 then _
        return

    ;top
    FFAddExcludedArea( _
        0, _
        0, _
        $w, _
        $b)

    ;left
    FFAddExcludedArea( _
        0, _
        $b, _
        $a, _
        $d)

    ;right
    FFAddExcludedArea( _
        $c, _
        $b, _
        $w, _
        $d)

    ;bottom
    FFAddExcludedArea( _
        0, _
        $d, _
        $w, _
        $h)
EndFunc

Func FindDot($isRed = 0, $gridIndex = -1)
    local $shadeVariation=0
    Local $shadeVariationMax = ($isRed) _
                                    ? 250 _
                                    : 60
    local $result
    Local $color = ($isRed) _
                        ? 0xFFFF0000 _
                        : 0xFF000000

    if $isRed then
        $horizontalFactor = Mod($gridIndex, 4)
        $firstX = $horizontalFactor * $w/4
        $secondX = $firstX + $w/4

        $verticalFactor = Floor($gridIndex/4)
        $firstY = $verticalFactor * $h/3
        $secondY = $firstY + $h/3

        SetArea($firstX, $firstY, $secondX, $secondY)
    EndIf

    do
        $result = FFNearestSpot(20, 50, 0, 0, $color, $shadeVariation, 0)

        if (Not IsArray($result)) Then _
            $shadeVariation += 5
    until (IsArray($result) OR $shadeVariation > $shadeVariationMax)

    if Not IsArray($result) then _
        return false

    if $isRed then
        SetGridEntry($result, $gridIndex)
    Else
        FFAddExcludedArea( _
        $result[0] - 20, _
        $result[1] - 20, _
        $result[0] + 20, _
        $result[1] + 20)
    EndIf

    return _ArrayExtract($result, 0, 1)
EndFunc

Can be called from another project like this:
 

#include <ExtractDots.au3>

Local $files = [ _
    "example3.jpg", _
    "example5.jpg", _
    "example9.jpg" ]

Local $votes[0]
for $file in $files
    $values = ExtractDots($file)
    _ArrayAdd($votes, $values)
Next

_ArrayDisplay($votes)

 

Depending on the brightness, saturation, and illumination of the image, individual points may not or too many may be detected.

Therefore, using ImageMagick (must be installed), each image is edited to highlight red and black.

Each image is followed by a query that allows you to adjust the two colors.

Here are some files to try out.

ExtractDots.zip

 

Edited October 19, 2019 by tz45