Sign in to follow this  
Followers 0
Mingre

Detect lines of black text on screen (not OCR)

2 posts in this topic

#1 ·  Posted (edited)

Hello forums,

I'd like to know if anyone has better idea than what I have. I'm creating a program that will detect (not OCR!) blocks of words (black text) on a white background displayed on the screen. (If anyone is curious about its purpose, this code will be part of a program that will randomly select a word on a PDF file and have the user guess that word. I'll be using it to quiz myself with my PDF notes. )

What I created so far: Using FastFind.au3, I was able to make it detect lines of black text (font Lucida Console) on a single column. It basically considers a line a separate one if bounded above and below by horizontal lines that do not contain black pixels. Next step, which I haven't done yet, will be to go through each line and detect the blocks of words.

The program works fine so far with Lucida Console font but it doesn't when tested on TNR font (see attached sample image). The former can be tested by setting $iOption to "text" while the latter by setting it to "select".

So does anyone have a better idea on going about this? Thank you!

Sorry if my code's a mess. I tried to make it cleaner before posting it but this is the best I can manage.

#include <FastFind.au3>
#include <array.au3>
HotKeySet('{ESC}', '__Exit')

Local $iImagePos[4], $iXa, $iXb, $iYa, $iYb
$iOption = 'text'
Switch $iOption
  Case 'text'   ; Creates sample text file for detection
    Local Const $sSampleTextFileName = 'sample.txt'
    FileWrite($sSampleTextFileName, '')
    FileOpen($sSampleTextFileName, 2)
    FileWrite($sSampleTextFileName, 'First line.' & @CRLF & 'Second line.' & @CRLF & @CRLF & 'Third line.')
    FileClose($sSampleTextFileName)
    ShellExecute($sSampleTextFileName)
    Local Const $sSampleTextFileTitle = 'sample - Notepad'
    WinWait($sSampleTextFileTitle)
    WinMove($sSampleTextFileTitle, '', 0, 0, @DesktopWidth / 2, @DesktopHeight / 2)
    Sleep(500)
    $iImagePos = __ControlGetPos($sSampleTextFileTitle, '[CLASS:Edit; INSTANCE:1]') ; Gets the absolute coordinates of the edit control.
    Local $iXa = $iImagePos[0], _
        $iXb = $iImagePos[0] + $iImagePos[2], _
        $iYa = $iImagePos[1], _
        $iYb = $iImagePos[1] + $iImagePos[3]
  Case 'select'
    MsgBox(0, 'Select', 'Select first')
    $iXa = MouseGetPos(0)
    $iYa = MouseGetPos(1)
    MsgBox(0, 'Select', 'Select second')
    $iXb = MouseGetPos(0)
    $iYb = MouseGetPos(1)
EndSwitch


Local $array[1][2] ; Will be used for saving coordinates of lines of text
Local $iBlackPixelCount
Local $bHasNoPreviousZero = True ; Will be used as a toggle for skipping succeeding lines that have no black pixels.
Local $iStep = 1
For $y = $iYa To $iYb Step $iStep
  FFSnapShot($iXa, $y, $iXb, $y)
  $iBlackPixelCount = FFColorCount(0x000000, 50, False)
  ; $iBlackPixelPercent = Round(100 * (FFColorCount(0x000000, 0, False) / $iImagePos[2]), 2) ; Computes for percentage of black pixels on horizontal line.
  ; If $iBlackPixelCount > 0 Or $bHasNoPreviousZero Then
  ; $bHasNoPreviousZero = True
  _ArrayAdd($array, $iBlackPixelCount)
  $array[UBound($array) - 1][1] = $y
  ; If $iBlackPixelCount = 0 Then $bHasNoPreviousZero = False
  ; EndIf
Next

; To check detected lines. This will move mouse to the lower-left corner of the text line.
For $i = 0 To UBound($array) - 1 Step +1
  If $array[$i][0] > 0 Then
    If $i + 1 <= UBound($array) - 1 Then
      If $array[$i + 1][0] = 0 Then
        MouseMove($iXa, $array[$i][1])
        Sleep(500)
      EndIf
    EndIf
  ;
  EndIf
Next

_ArrayDisplay($array)

Func __ControlGetPos($hWnd, $controlID, $bAbsolute = Default)
  ;; https://www.autoitscript.com/forum/topic/88345-absolute-position-of-guicontrol/
  If $bAbsolute = Default Then $bAbsolute = True
  Local $controlPos
  Switch $bAbsolute
    Case True
      Local Const $hWnd_Control = ControlGetHandle($hWnd, "", $controlID)
      $controlPos = WinGetPos($hWnd_Control)
    Case False
      $controlPos = ControlGetPos($hWnd, "", $controlID)
  EndSwitch
  If Not IsArray($controlPos) Then SetError(1)
  Return $controlPos
EndFunc   ;==>__ControlGetPos

Func __Exit()
  Exit
EndFunc   ;==>__Exit

 

FastFind.dll

FastFind.au3

FastFind64.dll

sample image.bmp

Edited by Mingre

Share this post


Link to post
Share on other sites



Hello instead of checking for black pixels, I changed it to check white pixels and it now works on TNR font. Thanks everyone! :) 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Similar Content

    • ur
      By ur
      I have a text file whose data will be as below.
      win10x64 ~\erwin Notallowed1! "erwin Data Modeler r9.7 (32-bit)_2500.exe" SilentInstall.exe win10x64clone1 ~\erwin Notallowed1! "erwin Data Modeler r9.7 (64-bit)_2500.exe" DM64.exe win10x64clone2 ~\erwin Notallowed1! "erwin Mart Server r9.7 (32-bit).exe" SilentInstall.exe win10x64clone3 ~\erwin Notallowed1! "erwin License Server r9.7 (32-bit).exe" SilentInstall.exe Each line will have multiple values separated by space.
      If a value contains space in it, the value is surrounded by quotes.
      My task is to check how many values are there in each line.
      If the line contains 5 values, I need to replace the 4th value with the string contained in a variable.
      If it contains 4 values then also  I need to replace the 4th value followed by appending 5 th value to it as SilentInstall.exe
      If the value I am replacing contains spaces then I need to surround the new value with quotes.
       
      Any one can suggest how to do this,??
    • ronaldo97
      By ronaldo97
      Hello How can I make the GUI is compatible with all screen sizes ??  
      #include <GUIConstantsEx.au3> #include <StaticConstants.au3> #include <WindowsConstants.au3> #Region ### START Koda GUI section ### Form= $Form1 = GUICreate("Form1", 623, 445, 192, 124, BitOR($GUI_SS_DEFAULT_GUI,$WS_MAXIMIZE)) GUISetBkColor(0x1E1E1E) $Label1 = GUICtrlCreateLabel("Welcome Back !!", 424, 64, 161, 29) GUICtrlSetFont(-1, 16, 400, 0, "Tahoma") GUICtrlSetColor(-1, 0x800000) GUISetState(@SW_SHOW) #EndRegion ### END Koda GUI section ### While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE Exit EndSwitch WEnd How do I make Gui interface compatible with all screen sizes ?? 800*600
      1024*768
      1280*1024
    • marcoauto
      By marcoauto
      ciao,
      I need to scale an outlined GDI text in a GUI. The problem is that I can't do this. The outline is often out of the GUI
      I started from this script:
      https://www.autoitscript.com/autoit3/docs/libfunctions/_GDIPlus_PathTransform.htm
      that fit a string to windows and I changed
       $aBounds = _GDIPlus_PathGetWorldBounds($hPath) ;Get bounding rectangle of the path
      $aBounds = _GDIPlus_PathGetWorldBounds($hPath) ;Get bounding rectangle of the path with 
      $aBounds = _GDIPlus_PathGetWorldBounds($hPath,$hMatrix,$hPen) ;Get bounding rectangle of the path with pen size but doesn't work
      so, I tried to change other thins, but nothing... Sometimes works, but if I change the string often result change
      This is my script:
      #include <GUIConstantsEx.au3> #include <GDIPlus.au3> #include <array.au3> Example() Func Example() Local $iW, $iH, $hGUI, $hGraphic, $hBrush, $hPen, $hPath, $hFamily, $tLayout, $hMatrix, $aBounds ; Create GUI $iW = 650 $iH = 300 $hGUI = GUICreate("GDI+", $iW, $iH) GUISetState(@SW_SHOW) ; Draw a string using path _GDIPlus_Startup() $hGraphic = _GDIPlus_GraphicsCreateFromHWND($hGUI) ;Create a graphics object from a window handle _GDIPlus_GraphicsSetSmoothingMode($hGraphic, $GDIP_SMOOTHINGMODE_HIGHQUALITY) ;Sets the graphics object rendering quality (antialiasing) ;~ _GDIPlus_GraphicsClear($hGraphic, 0xFF000000) $MergedImageBackgroundColor = 0x00000000 _GDIPlus_GraphicsClear($hGraphic, $MergedImageBackgroundColor) $hBrush = _GDIPlus_BrushCreateSolid(0xFFDD2200) $stroke_size = 15 $hPen = _GDIPlus_PenCreate(0xFFFFBB00, $stroke_size) ;stroke size in pixel $hPen2 = _GDIPlus_PenCreate(0xFFFFff00, 2) ;stroke size in pixel _GDIPlus_PenSetLineJoin($hPen,$GDIP_PENSETLINEJOIN_ROUND) $hPath = _GDIPlus_PathCreate() ;Create new path object ;~ $fSize = 20 $hFamily = _GDIPlus_FontFamilyCreate("Myriad Pro") ;Create font family object $tLayout = _GDIPlus_RectFCreate() ;Create string bounding rectangle X=0, Y=0 Local $hFormat = _GDIPlus_StringFormatCreate() _GDIPlus_PathAddString($hPath, "autoitForeverLollipop", $tLayout, $hFamily);, 0, $fSize, $hFormat) ;Add the outline of the string to the path ;~ _GDIPlus_PathAddString($hPath, $sText, $tLayout, $hFamily, 0, $fSize, $hFormat) ; Tranform Path to fit to window $aBounds = _GDIPlus_PathGetWorldBounds($hPath) ;Get bounding rectangle of the path $aPoints = _GDIPlus_PathGetPoints($hPath) ConsoleWrite($aBounds[0] & @CRLF) ConsoleWrite($aBounds[1] & @CRLF) ConsoleWrite($aBounds[2] & @CRLF) ConsoleWrite($aBounds[3] & @CRLF) ConsoleWrite($aPoints[$aPoints[0][0]][0] & " - " & $aPoints[1][0] & " + 2*" & $stroke_size & " = " & $aPoints[$aPoints[0][0]][0]-$aPoints[1][0]+2*$stroke_size & @CRLF) ;~ $aBounds[0] = $aBounds[0] $aBoundsNp = _GDIPlus_PathGetWorldBounds($hPath,0,$hPen) ;Get bounding rectangle of the path with pen ConsoleWrite($aBoundsNp[0] & @CRLF) ConsoleWrite($aBoundsNp[1] & @CRLF) ConsoleWrite($aBoundsNp[2] & @CRLF) ConsoleWrite($aBoundsNp[3] & @CRLF) ConsoleWrite($iW & " / 2 = " & $iW / 2 & @CRLF) ConsoleWrite("$ih / 2 = " & $iH / 2 & @CRLF) ;~ ConsoleWrite($iW & " $aBounds[2] = " & $iW / $aBounds[2] & @CRLF) ConsoleWrite($iW & " $aBoundsNp[2] = " & $iW / $aBoundsNp[2] & @CRLF) ConsoleWrite($iH & " $aBoundsNp[3] = " & $iH / $aBoundsNp[3] & @CRLF) ;~ _GDIPlus_GraphicsDrawPath($hGraphic, $hPath, $hPen) ;Draw path to graphics handle (GUI) ;~ _GDIPlus_GraphicsFillPath($hGraphic, $hPath, $hBrush) ;Fill path to graphics handle (GUI) $hMatrix = _GDIPlus_MatrixCreate() $iW = $iW - $stroke_size ;+ $stroke_size $iH = $iH - $stroke_size ConsoleWrite(" -$aBounds[0]+1*$stroke_size/72 = " & -$aBounds[0]+1*$stroke_size/72 & @CRLF) ConsoleWrite(" -1.8+1*$stroke_size/72 = " & -1.8+1*$stroke_size/72 & @CRLF) _GDIPlus_MatrixTranslate($hMatrix, -$aBounds[0]+2*$stroke_size/72, -$aBounds[1]+1*$stroke_size/72) ;Translate Matrix to the offset of the bounding rectangle ;~ _GDIPlus_MatrixTranslate($hMatrix, -1.59166666666667+0*($stroke_size/72), -$aBounds[1]+1*$stroke_size/72) ;Translate Matrix to the offset of the bounding rectangle ;~ _GDIPlus_MatrixScale($hMatrix, $iW / $aBounds[2], $iH / $aBounds[3], True) ;Scale Matrix _GDIPlus_MatrixScale($hMatrix, $iW / $aBounds[2], $iH / $aBounds[3], True) ;Scale Matrix _GDIPlus_PathTransform($hPath, $hMatrix) ;Translate and Scale Path $aBoundsNp = _GDIPlus_PathGetWorldBounds($hPath,0,$hPen) ;Get bounding rectangle of the path with pen ConsoleWrite($aBoundsNp[0] & @CRLF) ConsoleWrite($aBoundsNp[1] & @CRLF) ConsoleWrite($aBoundsNp[2] & @CRLF) ConsoleWrite($aBoundsNp[3] & @CRLF) ;~ _GDIPlus_GraphicsFillPath($hGraphic, $hPath, $hBrush) ;Fill path to graphics handle (GUI) _GDIPlus_GraphicsDrawPath($hGraphic, $hPath, $hPen) ;Draw path to graphics handle (GUI) _GDIPlus_GraphicsFillPath($hGraphic, $hPath, $hBrush) ;Fill path to graphics handle (GUI) ; Loop until the user exits. Do Until GUIGetMsg() = $GUI_EVENT_CLOSE ; Clean up resources _GDIPlus_MatrixDispose($hMatrix) _GDIPlus_FontFamilyDispose($hFamily) _GDIPlus_PathDispose($hPath) _GDIPlus_PenDispose($hPen) _GDIPlus_BrushDispose($hBrush) _GDIPlus_GraphicsDispose($hGraphic) _GDIPlus_Shutdown() EndFunc ;==>Example if you change the line 
      _GDIPlus_PathAddString($hPath, "autoitForeverLollipop", $tLayout, $hFamily);, 0, $fSize, $hFormat) ;Add the outline of the string to the path with a smaller string
      _GDIPlus_PathAddString($hPath, "autoitFitted", $tLayout, $hFamily);, 0, $fSize, $hFormat) ;Add the outline of the string to the path the result is ok
      Can someone can help me?
      Thankyou
    • marcoauto
      By marcoauto
      Ciao,
      Is there a function to create a stroke text?
      I have found the solution to fill the text with a texture, to make a shadow and to make a text with glow , but I can't reproduce the stoke effect.)
      I have attached an example image of what I would like to reproduce
      I have tried to adapt this script of Malkey but with no result
      The Malkey script is this (I have edit some line) and the original post is here: 
       
      #include <GDIPlus.au3> #include <WinAPI.au3> #include <GuiConstants.au3> #include <WindowsConstants.au3> Global $nPI = 3.1415926535897932384626433832795 Global $ghGDIPDll $hWnd = GUICreate("GDI+ Example", 800, 150) $iGlow = 2; <== Amount of Glow min = 1 Max about 5 _GDIPlus_Startup() $hGraphicGUI = _GDIPlus_GraphicsCreateFromHWND($hWnd) $hBMPBuff = _GDIPlus_BitmapCreateFromGraphics(800, 150, $hGraphicGUI) $hGraphics = _GDIPlus_ImageGetGraphicsContext($hBMPBuff) _GDIPlus_GraphicsClear($hGraphics, 0xff003000) ; Black-green background ;~ _GDIPlus_GraphicsClear($hGraphics, 0xffffffff) ; White Background _AntiAlias($hGraphics, 4) $hFamily = _GDIPlus_FontFamilyCreate("Arial Black") $hFont = _GDIPlus_FontCreate($hFamily, 80) $hLayout = _GDIPlus_RectFCreate(0, 0, 800, 150) $hStringFormat = _GDIPlus_StringFormatCreate() _GDIPlus_StringFormatSetAlign($hStringFormat, 1) For $i = 1 To 15 Step 3 $hBrush3 = _GDIPlus_BrushCreateSolid("0x" & hex($iGlow,2) & "00ff00"); <====== Glow (Transparency) For $x = 0 To 360 Step 3 DllStructSetData($hLayout, "Y", ((18 - $i)) * Sin($x * $nPI / 180)) DllStructSetData($hLayout, "X", ((18 - $i)) * Cos($x * $nPI / 180)) $hStringFormat = _GDIPlus_StringFormatCreate() _GDIPlus_GraphicsDrawStringEx($hGraphics, "AutoIt Rocks", $hFont, $hLayout, $hStringFormat, $hBrush3) Next Next DllStructSetData($hLayout, "Y", 0) DllStructSetData($hLayout, "X", 0) $hStringFormat = _GDIPlus_StringFormatCreate() $hBrush3 = _GDIPlus_BrushCreateSolid("0xfF00FF00") _GDIPlus_GraphicsDrawStringEx($hGraphics, "AutoIt Rocks", $hFont, $hLayout, $hStringFormat, $hBrush3) GUISetState() GUIRegisterMsg(0xF, "MY_PAINT"); Register PAINT-Event 0x000F = $WM_PAINT (WindowsConstants.au3) GUIRegisterMsg(0x85, "MY_PAINT") ; $WM_NCPAINT = 0x0085 (WindowsConstants.au3)Restore after Minimize. _GDIPlus_GraphicsDrawImage($hGraphicGUI, $hBMPBuff, 0, 0) Do $msg = GUIGetMsg() Until $msg = $GUI_EVENT_CLOSE _GDIPlus_BrushDispose($hBrush3) _GDIPlus_FontFamilyDispose($hFamily) _GDIPlus_FontDispose($hFont) _GDIPlus_StringFormatDispose($hStringFormat) _GDIPlus_GraphicsDispose($hGraphics) _GDIPlus_GraphicsDispose($hGraphicGUI) _WinAPI_DeleteObject($hBMPBuff) _GDIPlus_Shutdown() Func _AntiAlias($hGraphics, $iMode) Local $aResult $aResult = DllCall($ghGDIPDll, "int", "GdipSetSmoothingMode", "hwnd", $hGraphics, "int", $iMode) If @error Then Return SetError(@error, @extended, False) Return SetError($aResult[0], 0, $aResult[0] = 0) EndFunc ;==>_AntiAlias ;Func to redraw on PAINT MSG Func MY_PAINT($hWnd, $msg, $wParam, $lParam) _GDIPlus_GraphicsDrawImage($hGraphicGUI, $hBMPBuff, 0, 0) _WinAPI_RedrawWindow($hWnd, "", "", BitOR($RDW_INVALIDATE, $RDW_UPDATENOW, $RDW_FRAME)) ; , $RDW_ALLCHILDREN Return $GUI_RUNDEFMSG EndFunc ;==>MY_PAINT Thank you very much,
      Marco

    • Blueman
      By Blueman
      Hey Guys,
      I am Struggling with a issue that i cannot seem to fix.
      I hope that you guys can help me with this.
      I have a GUI with some text attributes, for all i use the following code;
      $Label_ts_Header = GUICtrlCreateLabel("Manual Adress", 60, 10, 500, 50) GUICtrlSetBkColor($Label_ts_Header, $COLOR_WHITE) GUICtrlSetFont ($Label_ts_Header, 26, 800, 0, "Calibri") This is all working like it should, but when i activate the GUI on another pc the text size is bigger and the GUI size is somewhat smaller.
      The problem is that it isn't on all computers, i can't seem to find the problem,..
      If i make it fit on one pc it is to small on another or to big,..
      What am i doing wrong?
       
      Is there some kind of attribute that i need to add to the top of my script to normalize text sizes in the script?
       
      Thanks!
       
      EDIT: Found the problem guys, some computers has changed the text size to 125% // Can i force 100% for my GUI somehow ?
       
      EDIT2: Fixed it guys! // I used the Function Below to get the DPI Setting and used that to determine the text size (example below)
       
      Func _GDIPlus_GraphicsGetDPIRatio($iDPIDef = 96) _GDIPlus_Startup() Local $hGfx = _GDIPlus_GraphicsCreateFromHWND(0) If @error Then Return SetError(1, @extended, 0) Local $aResult #forcedef $__g_hGDIPDll, $ghGDIPDll $aResult = DllCall($__g_hGDIPDll, "int", "GdipGetDpiX", "handle", $hGfx, "float*", 0) If @error Then Return SetError(2, @extended, 0) Local $iDPI = $aResult[2] Local $aresults[2] = [$iDPIDef / $iDPI, $iDPI / $iDPIDef] _GDIPlus_GraphicsDispose($hGfx) _GDIPlus_Shutdown() Return $aresults EndFunc ;==>_GDIPlus_GraphicsGetDPIRatio  
      $Label_S2 = GUICtrlCreateLabel("Data from The Base", 10, 300, 400, 20) GUICtrlSetBkColor($Label_S2, $COLOR_WHITE) GUICtrlSetFont ($Label_S2, 13 * _GDIPlus_GraphicsGetDPIRatio()[0])