Sign in to follow this  
Followers 0
Mingre

Detect lines of black text on screen (not OCR)

2 posts in this topic

#1 ·  Posted (edited)

Hello forums,

I'd like to know if anyone has better idea than what I have. I'm creating a program that will detect (not OCR!) blocks of words (black text) on a white background displayed on the screen. (If anyone is curious about its purpose, this code will be part of a program that will randomly select a word on a PDF file and have the user guess that word. I'll be using it to quiz myself with my PDF notes. )

What I created so far: Using FastFind.au3, I was able to make it detect lines of black text (font Lucida Console) on a single column. It basically considers a line a separate one if bounded above and below by horizontal lines that do not contain black pixels. Next step, which I haven't done yet, will be to go through each line and detect the blocks of words.

The program works fine so far with Lucida Console font but it doesn't when tested on TNR font (see attached sample image). The former can be tested by setting $iOption to "text" while the latter by setting it to "select".

So does anyone have a better idea on going about this? Thank you!

Sorry if my code's a mess. I tried to make it cleaner before posting it but this is the best I can manage.

#include <FastFind.au3>
#include <array.au3>
HotKeySet('{ESC}', '__Exit')

Local $iImagePos[4], $iXa, $iXb, $iYa, $iYb
$iOption = 'text'
Switch $iOption
  Case 'text'   ; Creates sample text file for detection
    Local Const $sSampleTextFileName = 'sample.txt'
    FileWrite($sSampleTextFileName, '')
    FileOpen($sSampleTextFileName, 2)
    FileWrite($sSampleTextFileName, 'First line.' & @CRLF & 'Second line.' & @CRLF & @CRLF & 'Third line.')
    FileClose($sSampleTextFileName)
    ShellExecute($sSampleTextFileName)
    Local Const $sSampleTextFileTitle = 'sample - Notepad'
    WinWait($sSampleTextFileTitle)
    WinMove($sSampleTextFileTitle, '', 0, 0, @DesktopWidth / 2, @DesktopHeight / 2)
    Sleep(500)
    $iImagePos = __ControlGetPos($sSampleTextFileTitle, '[CLASS:Edit; INSTANCE:1]') ; Gets the absolute coordinates of the edit control.
    Local $iXa = $iImagePos[0], _
        $iXb = $iImagePos[0] + $iImagePos[2], _
        $iYa = $iImagePos[1], _
        $iYb = $iImagePos[1] + $iImagePos[3]
  Case 'select'
    MsgBox(0, 'Select', 'Select first')
    $iXa = MouseGetPos(0)
    $iYa = MouseGetPos(1)
    MsgBox(0, 'Select', 'Select second')
    $iXb = MouseGetPos(0)
    $iYb = MouseGetPos(1)
EndSwitch


Local $array[1][2] ; Will be used for saving coordinates of lines of text
Local $iBlackPixelCount
Local $bHasNoPreviousZero = True ; Will be used as a toggle for skipping succeeding lines that have no black pixels.
Local $iStep = 1
For $y = $iYa To $iYb Step $iStep
  FFSnapShot($iXa, $y, $iXb, $y)
  $iBlackPixelCount = FFColorCount(0x000000, 50, False)
  ; $iBlackPixelPercent = Round(100 * (FFColorCount(0x000000, 0, False) / $iImagePos[2]), 2) ; Computes for percentage of black pixels on horizontal line.
  ; If $iBlackPixelCount > 0 Or $bHasNoPreviousZero Then
  ; $bHasNoPreviousZero = True
  _ArrayAdd($array, $iBlackPixelCount)
  $array[UBound($array) - 1][1] = $y
  ; If $iBlackPixelCount = 0 Then $bHasNoPreviousZero = False
  ; EndIf
Next

; To check detected lines. This will move mouse to the lower-left corner of the text line.
For $i = 0 To UBound($array) - 1 Step +1
  If $array[$i][0] > 0 Then
    If $i + 1 <= UBound($array) - 1 Then
      If $array[$i + 1][0] = 0 Then
        MouseMove($iXa, $array[$i][1])
        Sleep(500)
      EndIf
    EndIf
  ;
  EndIf
Next

_ArrayDisplay($array)

Func __ControlGetPos($hWnd, $controlID, $bAbsolute = Default)
  ;; https://www.autoitscript.com/forum/topic/88345-absolute-position-of-guicontrol/
  If $bAbsolute = Default Then $bAbsolute = True
  Local $controlPos
  Switch $bAbsolute
    Case True
      Local Const $hWnd_Control = ControlGetHandle($hWnd, "", $controlID)
      $controlPos = WinGetPos($hWnd_Control)
    Case False
      $controlPos = ControlGetPos($hWnd, "", $controlID)
  EndSwitch
  If Not IsArray($controlPos) Then SetError(1)
  Return $controlPos
EndFunc   ;==>__ControlGetPos

Func __Exit()
  Exit
EndFunc   ;==>__Exit

 

FastFind.dll

FastFind.au3

FastFind64.dll

sample image.bmp

Edited by Mingre

Share this post


Link to post
Share on other sites



Hello instead of checking for black pixels, I changed it to check white pixels and it now works on TNR font. Thanks everyone! :) 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Similar Content

    • harvester2001
      By harvester2001
      Hi
      I need your help I need create broadcast text message/notification to multiple lan clients. I can install listener program on pc its not a problem.
      But I don`t know how to start, could somebody write me example script how broadcast script and client who listen non stop for message should look like.
      I try to find example on forum but I have problem to understand those scripts.
      Many thx for help.
    • ur
      By ur
      I have a text file whose data will be as below.
      win10x64 ~\erwin Notallowed1! "erwin Data Modeler r9.7 (32-bit)_2500.exe" SilentInstall.exe win10x64clone1 ~\erwin Notallowed1! "erwin Data Modeler r9.7 (64-bit)_2500.exe" DM64.exe win10x64clone2 ~\erwin Notallowed1! "erwin Mart Server r9.7 (32-bit).exe" SilentInstall.exe win10x64clone3 ~\erwin Notallowed1! "erwin License Server r9.7 (32-bit).exe" SilentInstall.exe Each line will have multiple values separated by space.
      If a value contains space in it, the value is surrounded by quotes.
      My task is to check how many values are there in each line.
      If the line contains 5 values, I need to replace the 4th value with the string contained in a variable.
      If it contains 4 values then also  I need to replace the 4th value followed by appending 5 th value to it as SilentInstall.exe
      If the value I am replacing contains spaces then I need to surround the new value with quotes.
       
      Any one can suggest how to do this,??
    • ronaldo97
      By ronaldo97
      Hello How can I make the GUI is compatible with all screen sizes ??  
      #include <GUIConstantsEx.au3> #include <StaticConstants.au3> #include <WindowsConstants.au3> #Region ### START Koda GUI section ### Form= $Form1 = GUICreate("Form1", 623, 445, 192, 124, BitOR($GUI_SS_DEFAULT_GUI,$WS_MAXIMIZE)) GUISetBkColor(0x1E1E1E) $Label1 = GUICtrlCreateLabel("Welcome Back !!", 424, 64, 161, 29) GUICtrlSetFont(-1, 16, 400, 0, "Tahoma") GUICtrlSetColor(-1, 0x800000) GUISetState(@SW_SHOW) #EndRegion ### END Koda GUI section ### While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE Exit EndSwitch WEnd How do I make Gui interface compatible with all screen sizes ?? 800*600
      1024*768
      1280*1024
    • marcoauto
      By marcoauto
      ciao,
      I need to scale an outlined GDI text in a GUI. The problem is that I can't do this. The outline is often out of the GUI
      I started from this script:
      https://www.autoitscript.com/autoit3/docs/libfunctions/_GDIPlus_PathTransform.htm
      that fit a string to windows and I changed
       $aBounds = _GDIPlus_PathGetWorldBounds($hPath) ;Get bounding rectangle of the path
      $aBounds = _GDIPlus_PathGetWorldBounds($hPath) ;Get bounding rectangle of the path with 
      $aBounds = _GDIPlus_PathGetWorldBounds($hPath,$hMatrix,$hPen) ;Get bounding rectangle of the path with pen size but doesn't work
      so, I tried to change other thins, but nothing... Sometimes works, but if I change the string often result change
      This is my script:
      #include <GUIConstantsEx.au3> #include <GDIPlus.au3> #include <array.au3> Example() Func Example() Local $iW, $iH, $hGUI, $hGraphic, $hBrush, $hPen, $hPath, $hFamily, $tLayout, $hMatrix, $aBounds ; Create GUI $iW = 650 $iH = 300 $hGUI = GUICreate("GDI+", $iW, $iH) GUISetState(@SW_SHOW) ; Draw a string using path _GDIPlus_Startup() $hGraphic = _GDIPlus_GraphicsCreateFromHWND($hGUI) ;Create a graphics object from a window handle _GDIPlus_GraphicsSetSmoothingMode($hGraphic, $GDIP_SMOOTHINGMODE_HIGHQUALITY) ;Sets the graphics object rendering quality (antialiasing) ;~ _GDIPlus_GraphicsClear($hGraphic, 0xFF000000) $MergedImageBackgroundColor = 0x00000000 _GDIPlus_GraphicsClear($hGraphic, $MergedImageBackgroundColor) $hBrush = _GDIPlus_BrushCreateSolid(0xFFDD2200) $stroke_size = 15 $hPen = _GDIPlus_PenCreate(0xFFFFBB00, $stroke_size) ;stroke size in pixel $hPen2 = _GDIPlus_PenCreate(0xFFFFff00, 2) ;stroke size in pixel _GDIPlus_PenSetLineJoin($hPen,$GDIP_PENSETLINEJOIN_ROUND) $hPath = _GDIPlus_PathCreate() ;Create new path object ;~ $fSize = 20 $hFamily = _GDIPlus_FontFamilyCreate("Myriad Pro") ;Create font family object $tLayout = _GDIPlus_RectFCreate() ;Create string bounding rectangle X=0, Y=0 Local $hFormat = _GDIPlus_StringFormatCreate() _GDIPlus_PathAddString($hPath, "autoitForeverLollipop", $tLayout, $hFamily);, 0, $fSize, $hFormat) ;Add the outline of the string to the path ;~ _GDIPlus_PathAddString($hPath, $sText, $tLayout, $hFamily, 0, $fSize, $hFormat) ; Tranform Path to fit to window $aBounds = _GDIPlus_PathGetWorldBounds($hPath) ;Get bounding rectangle of the path $aPoints = _GDIPlus_PathGetPoints($hPath) ConsoleWrite($aBounds[0] & @CRLF) ConsoleWrite($aBounds[1] & @CRLF) ConsoleWrite($aBounds[2] & @CRLF) ConsoleWrite($aBounds[3] & @CRLF) ConsoleWrite($aPoints[$aPoints[0][0]][0] & " - " & $aPoints[1][0] & " + 2*" & $stroke_size & " = " & $aPoints[$aPoints[0][0]][0]-$aPoints[1][0]+2*$stroke_size & @CRLF) ;~ $aBounds[0] = $aBounds[0] $aBoundsNp = _GDIPlus_PathGetWorldBounds($hPath,0,$hPen) ;Get bounding rectangle of the path with pen ConsoleWrite($aBoundsNp[0] & @CRLF) ConsoleWrite($aBoundsNp[1] & @CRLF) ConsoleWrite($aBoundsNp[2] & @CRLF) ConsoleWrite($aBoundsNp[3] & @CRLF) ConsoleWrite($iW & " / 2 = " & $iW / 2 & @CRLF) ConsoleWrite("$ih / 2 = " & $iH / 2 & @CRLF) ;~ ConsoleWrite($iW & " $aBounds[2] = " & $iW / $aBounds[2] & @CRLF) ConsoleWrite($iW & " $aBoundsNp[2] = " & $iW / $aBoundsNp[2] & @CRLF) ConsoleWrite($iH & " $aBoundsNp[3] = " & $iH / $aBoundsNp[3] & @CRLF) ;~ _GDIPlus_GraphicsDrawPath($hGraphic, $hPath, $hPen) ;Draw path to graphics handle (GUI) ;~ _GDIPlus_GraphicsFillPath($hGraphic, $hPath, $hBrush) ;Fill path to graphics handle (GUI) $hMatrix = _GDIPlus_MatrixCreate() $iW = $iW - $stroke_size ;+ $stroke_size $iH = $iH - $stroke_size ConsoleWrite(" -$aBounds[0]+1*$stroke_size/72 = " & -$aBounds[0]+1*$stroke_size/72 & @CRLF) ConsoleWrite(" -1.8+1*$stroke_size/72 = " & -1.8+1*$stroke_size/72 & @CRLF) _GDIPlus_MatrixTranslate($hMatrix, -$aBounds[0]+2*$stroke_size/72, -$aBounds[1]+1*$stroke_size/72) ;Translate Matrix to the offset of the bounding rectangle ;~ _GDIPlus_MatrixTranslate($hMatrix, -1.59166666666667+0*($stroke_size/72), -$aBounds[1]+1*$stroke_size/72) ;Translate Matrix to the offset of the bounding rectangle ;~ _GDIPlus_MatrixScale($hMatrix, $iW / $aBounds[2], $iH / $aBounds[3], True) ;Scale Matrix _GDIPlus_MatrixScale($hMatrix, $iW / $aBounds[2], $iH / $aBounds[3], True) ;Scale Matrix _GDIPlus_PathTransform($hPath, $hMatrix) ;Translate and Scale Path $aBoundsNp = _GDIPlus_PathGetWorldBounds($hPath,0,$hPen) ;Get bounding rectangle of the path with pen ConsoleWrite($aBoundsNp[0] & @CRLF) ConsoleWrite($aBoundsNp[1] & @CRLF) ConsoleWrite($aBoundsNp[2] & @CRLF) ConsoleWrite($aBoundsNp[3] & @CRLF) ;~ _GDIPlus_GraphicsFillPath($hGraphic, $hPath, $hBrush) ;Fill path to graphics handle (GUI) _GDIPlus_GraphicsDrawPath($hGraphic, $hPath, $hPen) ;Draw path to graphics handle (GUI) _GDIPlus_GraphicsFillPath($hGraphic, $hPath, $hBrush) ;Fill path to graphics handle (GUI) ; Loop until the user exits. Do Until GUIGetMsg() = $GUI_EVENT_CLOSE ; Clean up resources _GDIPlus_MatrixDispose($hMatrix) _GDIPlus_FontFamilyDispose($hFamily) _GDIPlus_PathDispose($hPath) _GDIPlus_PenDispose($hPen) _GDIPlus_BrushDispose($hBrush) _GDIPlus_GraphicsDispose($hGraphic) _GDIPlus_Shutdown() EndFunc ;==>Example if you change the line 
      _GDIPlus_PathAddString($hPath, "autoitForeverLollipop", $tLayout, $hFamily);, 0, $fSize, $hFormat) ;Add the outline of the string to the path with a smaller string
      _GDIPlus_PathAddString($hPath, "autoitFitted", $tLayout, $hFamily);, 0, $fSize, $hFormat) ;Add the outline of the string to the path the result is ok
      Can someone can help me?
      Thankyou
    • marcoauto
      By marcoauto
      Ciao,
      Is there a function to create a stroke text?
      I have found the solution to fill the text with a texture, to make a shadow and to make a text with glow , but I can't reproduce the stoke effect.)
      I have attached an example image of what I would like to reproduce
      I have tried to adapt this script of Malkey but with no result
      The Malkey script is this (I have edit some line) and the original post is here: 
       
      #include <GDIPlus.au3> #include <WinAPI.au3> #include <GuiConstants.au3> #include <WindowsConstants.au3> Global $nPI = 3.1415926535897932384626433832795 Global $ghGDIPDll $hWnd = GUICreate("GDI+ Example", 800, 150) $iGlow = 2; <== Amount of Glow min = 1 Max about 5 _GDIPlus_Startup() $hGraphicGUI = _GDIPlus_GraphicsCreateFromHWND($hWnd) $hBMPBuff = _GDIPlus_BitmapCreateFromGraphics(800, 150, $hGraphicGUI) $hGraphics = _GDIPlus_ImageGetGraphicsContext($hBMPBuff) _GDIPlus_GraphicsClear($hGraphics, 0xff003000) ; Black-green background ;~ _GDIPlus_GraphicsClear($hGraphics, 0xffffffff) ; White Background _AntiAlias($hGraphics, 4) $hFamily = _GDIPlus_FontFamilyCreate("Arial Black") $hFont = _GDIPlus_FontCreate($hFamily, 80) $hLayout = _GDIPlus_RectFCreate(0, 0, 800, 150) $hStringFormat = _GDIPlus_StringFormatCreate() _GDIPlus_StringFormatSetAlign($hStringFormat, 1) For $i = 1 To 15 Step 3 $hBrush3 = _GDIPlus_BrushCreateSolid("0x" & hex($iGlow,2) & "00ff00"); <====== Glow (Transparency) For $x = 0 To 360 Step 3 DllStructSetData($hLayout, "Y", ((18 - $i)) * Sin($x * $nPI / 180)) DllStructSetData($hLayout, "X", ((18 - $i)) * Cos($x * $nPI / 180)) $hStringFormat = _GDIPlus_StringFormatCreate() _GDIPlus_GraphicsDrawStringEx($hGraphics, "AutoIt Rocks", $hFont, $hLayout, $hStringFormat, $hBrush3) Next Next DllStructSetData($hLayout, "Y", 0) DllStructSetData($hLayout, "X", 0) $hStringFormat = _GDIPlus_StringFormatCreate() $hBrush3 = _GDIPlus_BrushCreateSolid("0xfF00FF00") _GDIPlus_GraphicsDrawStringEx($hGraphics, "AutoIt Rocks", $hFont, $hLayout, $hStringFormat, $hBrush3) GUISetState() GUIRegisterMsg(0xF, "MY_PAINT"); Register PAINT-Event 0x000F = $WM_PAINT (WindowsConstants.au3) GUIRegisterMsg(0x85, "MY_PAINT") ; $WM_NCPAINT = 0x0085 (WindowsConstants.au3)Restore after Minimize. _GDIPlus_GraphicsDrawImage($hGraphicGUI, $hBMPBuff, 0, 0) Do $msg = GUIGetMsg() Until $msg = $GUI_EVENT_CLOSE _GDIPlus_BrushDispose($hBrush3) _GDIPlus_FontFamilyDispose($hFamily) _GDIPlus_FontDispose($hFont) _GDIPlus_StringFormatDispose($hStringFormat) _GDIPlus_GraphicsDispose($hGraphics) _GDIPlus_GraphicsDispose($hGraphicGUI) _WinAPI_DeleteObject($hBMPBuff) _GDIPlus_Shutdown() Func _AntiAlias($hGraphics, $iMode) Local $aResult $aResult = DllCall($ghGDIPDll, "int", "GdipSetSmoothingMode", "hwnd", $hGraphics, "int", $iMode) If @error Then Return SetError(@error, @extended, False) Return SetError($aResult[0], 0, $aResult[0] = 0) EndFunc ;==>_AntiAlias ;Func to redraw on PAINT MSG Func MY_PAINT($hWnd, $msg, $wParam, $lParam) _GDIPlus_GraphicsDrawImage($hGraphicGUI, $hBMPBuff, 0, 0) _WinAPI_RedrawWindow($hWnd, "", "", BitOR($RDW_INVALIDATE, $RDW_UPDATENOW, $RDW_FRAME)) ; , $RDW_ALLCHILDREN Return $GUI_RUNDEFMSG EndFunc ;==>MY_PAINT Thank you very much,
      Marco