Sign in to follow this  
Followers 0
Mingre

Detect lines of black text on screen (not OCR)

2 posts in this topic

#1 ·  Posted (edited)

Hello forums,

I'd like to know if anyone has better idea than what I have. I'm creating a program that will detect (not OCR!) blocks of words (black text) on a white background displayed on the screen. (If anyone is curious about its purpose, this code will be part of a program that will randomly select a word on a PDF file and have the user guess that word. I'll be using it to quiz myself with my PDF notes. )

What I created so far: Using FastFind.au3, I was able to make it detect lines of black text (font Lucida Console) on a single column. It basically considers a line a separate one if bounded above and below by horizontal lines that do not contain black pixels. Next step, which I haven't done yet, will be to go through each line and detect the blocks of words.

The program works fine so far with Lucida Console font but it doesn't when tested on TNR font (see attached sample image). The former can be tested by setting $iOption to "text" while the latter by setting it to "select".

So does anyone have a better idea on going about this? Thank you!

Sorry if my code's a mess. I tried to make it cleaner before posting it but this is the best I can manage.

#include <FastFind.au3>
#include <array.au3>
HotKeySet('{ESC}', '__Exit')

Local $iImagePos[4], $iXa, $iXb, $iYa, $iYb
$iOption = 'text'
Switch $iOption
  Case 'text'   ; Creates sample text file for detection
    Local Const $sSampleTextFileName = 'sample.txt'
    FileWrite($sSampleTextFileName, '')
    FileOpen($sSampleTextFileName, 2)
    FileWrite($sSampleTextFileName, 'First line.' & @CRLF & 'Second line.' & @CRLF & @CRLF & 'Third line.')
    FileClose($sSampleTextFileName)
    ShellExecute($sSampleTextFileName)
    Local Const $sSampleTextFileTitle = 'sample - Notepad'
    WinWait($sSampleTextFileTitle)
    WinMove($sSampleTextFileTitle, '', 0, 0, @DesktopWidth / 2, @DesktopHeight / 2)
    Sleep(500)
    $iImagePos = __ControlGetPos($sSampleTextFileTitle, '[CLASS:Edit; INSTANCE:1]') ; Gets the absolute coordinates of the edit control.
    Local $iXa = $iImagePos[0], _
        $iXb = $iImagePos[0] + $iImagePos[2], _
        $iYa = $iImagePos[1], _
        $iYb = $iImagePos[1] + $iImagePos[3]
  Case 'select'
    MsgBox(0, 'Select', 'Select first')
    $iXa = MouseGetPos(0)
    $iYa = MouseGetPos(1)
    MsgBox(0, 'Select', 'Select second')
    $iXb = MouseGetPos(0)
    $iYb = MouseGetPos(1)
EndSwitch


Local $array[1][2] ; Will be used for saving coordinates of lines of text
Local $iBlackPixelCount
Local $bHasNoPreviousZero = True ; Will be used as a toggle for skipping succeeding lines that have no black pixels.
Local $iStep = 1
For $y = $iYa To $iYb Step $iStep
  FFSnapShot($iXa, $y, $iXb, $y)
  $iBlackPixelCount = FFColorCount(0x000000, 50, False)
  ; $iBlackPixelPercent = Round(100 * (FFColorCount(0x000000, 0, False) / $iImagePos[2]), 2) ; Computes for percentage of black pixels on horizontal line.
  ; If $iBlackPixelCount > 0 Or $bHasNoPreviousZero Then
  ; $bHasNoPreviousZero = True
  _ArrayAdd($array, $iBlackPixelCount)
  $array[UBound($array) - 1][1] = $y
  ; If $iBlackPixelCount = 0 Then $bHasNoPreviousZero = False
  ; EndIf
Next

; To check detected lines. This will move mouse to the lower-left corner of the text line.
For $i = 0 To UBound($array) - 1 Step +1
  If $array[$i][0] > 0 Then
    If $i + 1 <= UBound($array) - 1 Then
      If $array[$i + 1][0] = 0 Then
        MouseMove($iXa, $array[$i][1])
        Sleep(500)
      EndIf
    EndIf
  ;
  EndIf
Next

_ArrayDisplay($array)

Func __ControlGetPos($hWnd, $controlID, $bAbsolute = Default)
  ;; https://www.autoitscript.com/forum/topic/88345-absolute-position-of-guicontrol/
  If $bAbsolute = Default Then $bAbsolute = True
  Local $controlPos
  Switch $bAbsolute
    Case True
      Local Const $hWnd_Control = ControlGetHandle($hWnd, "", $controlID)
      $controlPos = WinGetPos($hWnd_Control)
    Case False
      $controlPos = ControlGetPos($hWnd, "", $controlID)
  EndSwitch
  If Not IsArray($controlPos) Then SetError(1)
  Return $controlPos
EndFunc   ;==>__ControlGetPos

Func __Exit()
  Exit
EndFunc   ;==>__Exit

 

FastFind.dll

FastFind.au3

FastFind64.dll

sample image.bmp

Edited by Mingre

Share this post


Link to post
Share on other sites



Hello instead of checking for black pixels, I changed it to check white pixels and it now works on TNR font. Thanks everyone! :) 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Similar Content

    • Shirdish_chakravarthi
      By Shirdish_chakravarthi
      Hello Guys this is my gui and i need the numeric value typed in the highlighted  inputbox. if this value is greater than 1 i need to perform some operations
      \
       
      Here is the summary about that field which i got from wininfo tool

      i tried writing the code as shown below.. but the msgbox is popping up empty value..when the same code is pasted under some button action i am able to access the value present in the field..but when placed alone as written in the attached script i am unable to fetch it... 

      here i am using the valu present in the field

      can you guys tell me what's wrong??? or how can i get that value?
       
      My_GUI.au3

    • Shirdish_chakravarthi
      By Shirdish_chakravarthi
      in need the path/text present int above shown input box to be copied or retrieved into a variable .. how can i do that?
       
      Thanks
       
    • Shirdish_chakravarthi
      By Shirdish_chakravarthi
      Hello Friends i am new to autoit and i am stuck in middle of automation of flashing tool..
      my requirement is i have a window and i have to get the value present in that window (below is the picture)

      in the above window i need to get the value of the highlighted field.the value of this filed keeps on changing...ii need to get the current value present in the field..below are the details of the window and field..

       
      CAN it be done?
       
      thanks

    • Skeletor
      By Skeletor
      MS Splash Screens
      The office 2016/17 office suite has made many people turn heads with its functions and especially their splash screen.
      I now present to you, the MS Style Splash Screen. For Rookies, Novices and anyone who wants a ready made splash screen for your gui application.

      You can now download it and also try it out yourself.
      You have three colors to choose from:
      Blue
      Red
      Green
      All three colours are from the famous programs.
      I also included the KODA form to you can have freedom with this splash screen.
      Note: the KODA form does not have the three colours as well as the correct blue. 

      Have fun and enjoy.
      Note:
      I will update the splash to add the automated 3 dots animation at a later stage. 
       
       
    • houser747
      By houser747
      I have previously used _IEFormElementGetObjByName and _IEFormElementSetValue to enter text into a search box on a form and then submit the form.
      I am now trying to enter text into a search box which is not part of a form. 
      Here is the HTML from the website that i'm trying to enter the data on and then submit the search.
      <div class="row">
          <div class="form-group col-xs-12">
              <span id="FullWidthWithSubmenuContent_FullWidthContent_MainContent_AircraftRegistry_lblSearchText" for="input-search">Registreringsbeteckning</span>
              <div class="input-group col-xs-12">
                  <span id="FullWidthWithSubmenuContent_FullWidthContent_MainContent_AircraftRegistry_preSearchText" class="input-group-addon">SE -</span>
                  <input name="ctl00$FullWidthWithSubmenuContent$FullWidthContent$MainContent$AircraftRegistry$txtSearchText" type="text" value="DTH" id="FullWidthWithSubmenuContent_FullWidthContent_MainContent_AircraftRegistry_txtSearchText" class="form-control" />
              </div>
          </div>
      </div>
      <div class="row">
          <div class="form-group col-xs-12">
              <label class="sr-only" for="">Sök</label>
              <input type="submit" name="ctl00$FullWidthWithSubmenuContent$FullWidthContent$MainContent$AircraftRegistry$btnSearch" value="Sök" id="FullWidthWithSubmenuContent_FullWidthContent_MainContent_AircraftRegistry_btnSearch" class="btn btn-primary ladda-button" data-style="expand-right" />
          </div>
      </div>
      Many thanks in advance
      cheers
      Roger