Sign in to follow this  
Followers 0
Mingre

Detect lines of black text on screen (not OCR)

2 posts in this topic

#1 ·  Posted (edited)

Hello forums,

I'd like to know if anyone has better idea than what I have. I'm creating a program that will detect (not OCR!) blocks of words (black text) on a white background displayed on the screen. (If anyone is curious about its purpose, this code will be part of a program that will randomly select a word on a PDF file and have the user guess that word. I'll be using it to quiz myself with my PDF notes. )

What I created so far: Using FastFind.au3, I was able to make it detect lines of black text (font Lucida Console) on a single column. It basically considers a line a separate one if bounded above and below by horizontal lines that do not contain black pixels. Next step, which I haven't done yet, will be to go through each line and detect the blocks of words.

The program works fine so far with Lucida Console font but it doesn't when tested on TNR font (see attached sample image). The former can be tested by setting $iOption to "text" while the latter by setting it to "select".

So does anyone have a better idea on going about this? Thank you!

Sorry if my code's a mess. I tried to make it cleaner before posting it but this is the best I can manage.

#include <FastFind.au3>
#include <array.au3>
HotKeySet('{ESC}', '__Exit')

Local $iImagePos[4], $iXa, $iXb, $iYa, $iYb
$iOption = 'text'
Switch $iOption
  Case 'text'   ; Creates sample text file for detection
    Local Const $sSampleTextFileName = 'sample.txt'
    FileWrite($sSampleTextFileName, '')
    FileOpen($sSampleTextFileName, 2)
    FileWrite($sSampleTextFileName, 'First line.' & @CRLF & 'Second line.' & @CRLF & @CRLF & 'Third line.')
    FileClose($sSampleTextFileName)
    ShellExecute($sSampleTextFileName)
    Local Const $sSampleTextFileTitle = 'sample - Notepad'
    WinWait($sSampleTextFileTitle)
    WinMove($sSampleTextFileTitle, '', 0, 0, @DesktopWidth / 2, @DesktopHeight / 2)
    Sleep(500)
    $iImagePos = __ControlGetPos($sSampleTextFileTitle, '[CLASS:Edit; INSTANCE:1]') ; Gets the absolute coordinates of the edit control.
    Local $iXa = $iImagePos[0], _
        $iXb = $iImagePos[0] + $iImagePos[2], _
        $iYa = $iImagePos[1], _
        $iYb = $iImagePos[1] + $iImagePos[3]
  Case 'select'
    MsgBox(0, 'Select', 'Select first')
    $iXa = MouseGetPos(0)
    $iYa = MouseGetPos(1)
    MsgBox(0, 'Select', 'Select second')
    $iXb = MouseGetPos(0)
    $iYb = MouseGetPos(1)
EndSwitch


Local $array[1][2] ; Will be used for saving coordinates of lines of text
Local $iBlackPixelCount
Local $bHasNoPreviousZero = True ; Will be used as a toggle for skipping succeeding lines that have no black pixels.
Local $iStep = 1
For $y = $iYa To $iYb Step $iStep
  FFSnapShot($iXa, $y, $iXb, $y)
  $iBlackPixelCount = FFColorCount(0x000000, 50, False)
  ; $iBlackPixelPercent = Round(100 * (FFColorCount(0x000000, 0, False) / $iImagePos[2]), 2) ; Computes for percentage of black pixels on horizontal line.
  ; If $iBlackPixelCount > 0 Or $bHasNoPreviousZero Then
  ; $bHasNoPreviousZero = True
  _ArrayAdd($array, $iBlackPixelCount)
  $array[UBound($array) - 1][1] = $y
  ; If $iBlackPixelCount = 0 Then $bHasNoPreviousZero = False
  ; EndIf
Next

; To check detected lines. This will move mouse to the lower-left corner of the text line.
For $i = 0 To UBound($array) - 1 Step +1
  If $array[$i][0] > 0 Then
    If $i + 1 <= UBound($array) - 1 Then
      If $array[$i + 1][0] = 0 Then
        MouseMove($iXa, $array[$i][1])
        Sleep(500)
      EndIf
    EndIf
  ;
  EndIf
Next

_ArrayDisplay($array)

Func __ControlGetPos($hWnd, $controlID, $bAbsolute = Default)
  ;; https://www.autoitscript.com/forum/topic/88345-absolute-position-of-guicontrol/
  If $bAbsolute = Default Then $bAbsolute = True
  Local $controlPos
  Switch $bAbsolute
    Case True
      Local Const $hWnd_Control = ControlGetHandle($hWnd, "", $controlID)
      $controlPos = WinGetPos($hWnd_Control)
    Case False
      $controlPos = ControlGetPos($hWnd, "", $controlID)
  EndSwitch
  If Not IsArray($controlPos) Then SetError(1)
  Return $controlPos
EndFunc   ;==>__ControlGetPos

Func __Exit()
  Exit
EndFunc   ;==>__Exit

 

FastFind.dll

FastFind.au3

FastFind64.dll

sample image.bmp

Edited by Mingre

Share this post


Link to post
Share on other sites

Hello instead of checking for black pixels, I changed it to check white pixels and it now works on TNR font. Thanks everyone! :) 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Similar Content

    • Errious
      By Errious
      Hello,
      i am searching for a function that allows me some very specific functions for Screen Captures.
       
      First should be the possibillity to create a Screenshot from a specific window and this is possibly under another window but has to stay there and not getting forced to be in the first - layer position if this is clear enough :-) The other thing should be the ability to create a Screen Capture every couple minutes or a specific period.  
      I tried a lot of Screen Capture tools but neither can do both together.
      Also my try to create my own script with AutoIT failed, because of missing knoweledge about how to use it for this specific reasons, would be very kind if someone can tell me in the first place if this is possible at all and maybe someone has allready an idea about how to realize it?
       
      best regards
       
       
       
       
    • ur
      By ur
      I am trying to identify the window based on the window title and text.
      The title will be the "erwin DM - filename"

      It is working till date, but some operating systems our application is displaying window as "erwin DM - [filename]"
       
      I tried  "erwin DM - *filename*" But this regular expression is not working.
      Any suggestion?
       
      $sModelFile = "C:\Users\Administrator\Documents\My Models\eMovies.erwin" $wdModel = _WinWaitActivate1("erwin DM - "&FileNameOnly($sModelFile),"") Func _WinWaitActivate1($title,$text,$timeout=0);Will Return the window Handler Logging("Waiting for "&$title&":"&$text) $dHandle = WinWait($title,$text,$timeout) if not ($dHandle = 0) then If Not WinActive($title,$text) Then WinActivate($title,$text) return WinWaitActive($title,$text,$timeout) Else Logging("Timeout occured while waiting for the window...") Exit EndIf EndFunc Func FileNameOnly($sFilePath) Local $sDrive = "", $sDir = "", $sFileName = "", $sExtension = "" Local $aPathSplit = _PathSplit($sFilePath, $sDrive, $sDir, $sFileName, $sExtension) ;_ArrayDisplay($aPathSplit, "_PathSplit of " & @ScriptFullPath) return $sFileName EndFunc  
    • natedog102
      By natedog102
      This small UDF helps you save any text to speech to a .wav file. 
      UDF:
      #AutoIt3Wrapper_Au3Check_Parameters=-q -d -w 1 -w 2 -w 3 -w- 4 -w 5 -w 6 -w- 7 #include-once ; #INDEX# ======================================================================================================================= ; Title .........: _TTStoWav() ; AutoIt Version : 3.3.14.2 ; Language ......: English ; Author(s) .....: natedog102 ; Modifiers .....: ; Forum link ....: https://www.autoitscript.com/forum/topic/191573-tts-text-to-speech-to-wav-file-using-sapi/ ; Description ...: TTS to .wav file using SAPI ; =============================================================================================================================== ; #FUNCTION# ==================================================================================================================== ; Name...........: _TTStoWav ; Description ...: TTS to .wav file using SAPI ; Syntax.........: _TTStoWav($sText, $sLocation[, $iRate = 1[, $iVolume = 100]]) ; Parameters ....: $sText - String you want converted to a .wav ; $sLocation - Save location + file name, example: "C:\folder\sample.wav" ; $iRate - TTS speak rate between -10 and 10 ; $iVolume - Volume of the TTS between 0 and 100 ; Return values .: Success - Returns one ; Failure - Returns zero ; Author ........: natedog102 ; Modified.......: ; Remarks .......: ; Related .......: ; Link ..........: ; Example .......: No ; =============================================================================================================================== Func _TTStoWav($sText, $sLocation, $iRate = 1, $iVolume = 100) $oFileStream = ObjCreate("SAPI.SpFileStream") If @error Then Return 0 EndIf $oFileStream.Format.Type = 39 $oFileStream.Open($sLocation, 3) $oSpeech = ObjCreate('SAPI.SpVoice') If @error Then Return 0 EndIf $oSpeech.AudioOutputStream = $oFileStream $oSpeech.Rate = $iRate $oSpeech.Volume = $iVolume $oSpeech.Speak($sText, 3) $oSpeech.WaitUntilDone(10000) $oFileStream.Close() Return 1 EndFunc ;==>_TTStoWav And here's example usage:
      _TTStoWav("This is a test", "C:\folder-must-exist\sample2.wav") I have never made a UDF before so I don't know if I coded it correctly or if I followed all the recommended guidelines. Apologies if I haven't.
      DOWNLOAD:
       _TTStoWav.au3
    • Shirdish_chakravarthi
      By Shirdish_chakravarthi
      Hello Guys this is my gui and i need the numeric value typed in the highlighted  inputbox. if this value is greater than 1 i need to perform some operations
      \
       
      Here is the summary about that field which i got from wininfo tool

      i tried writing the code as shown below.. but the msgbox is popping up empty value..when the same code is pasted under some button action i am able to access the value present in the field..but when placed alone as written in the attached script i am unable to fetch it... 

      here i am using the valu present in the field

      can you guys tell me what's wrong??? or how can i get that value?
       
      My_GUI.au3

    • Shirdish_chakravarthi
      By Shirdish_chakravarthi
      in need the path/text present int above shown input box to be copied or retrieved into a variable .. how can i do that?
       
      Thanks