Jump to content
Sign in to follow this  
Mingre

Detect lines of black text on screen (not OCR)

Recommended Posts

Mingre

Hello forums,

I'd like to know if anyone has better idea than what I have. I'm creating a program that will detect (not OCR!) blocks of words (black text) on a white background displayed on the screen. (If anyone is curious about its purpose, this code will be part of a program that will randomly select a word on a PDF file and have the user guess that word. I'll be using it to quiz myself with my PDF notes. )

What I created so far: Using FastFind.au3, I was able to make it detect lines of black text (font Lucida Console) on a single column. It basically considers a line a separate one if bounded above and below by horizontal lines that do not contain black pixels. Next step, which I haven't done yet, will be to go through each line and detect the blocks of words.

The program works fine so far with Lucida Console font but it doesn't when tested on TNR font (see attached sample image). The former can be tested by setting $iOption to "text" while the latter by setting it to "select".

So does anyone have a better idea on going about this? Thank you!

Sorry if my code's a mess. I tried to make it cleaner before posting it but this is the best I can manage.

#include <FastFind.au3>
#include <array.au3>
HotKeySet('{ESC}', '__Exit')

Local $iImagePos[4], $iXa, $iXb, $iYa, $iYb
$iOption = 'text'
Switch $iOption
  Case 'text'   ; Creates sample text file for detection
    Local Const $sSampleTextFileName = 'sample.txt'
    FileWrite($sSampleTextFileName, '')
    FileOpen($sSampleTextFileName, 2)
    FileWrite($sSampleTextFileName, 'First line.' & @CRLF & 'Second line.' & @CRLF & @CRLF & 'Third line.')
    FileClose($sSampleTextFileName)
    ShellExecute($sSampleTextFileName)
    Local Const $sSampleTextFileTitle = 'sample - Notepad'
    WinWait($sSampleTextFileTitle)
    WinMove($sSampleTextFileTitle, '', 0, 0, @DesktopWidth / 2, @DesktopHeight / 2)
    Sleep(500)
    $iImagePos = __ControlGetPos($sSampleTextFileTitle, '[CLASS:Edit; INSTANCE:1]') ; Gets the absolute coordinates of the edit control.
    Local $iXa = $iImagePos[0], _
        $iXb = $iImagePos[0] + $iImagePos[2], _
        $iYa = $iImagePos[1], _
        $iYb = $iImagePos[1] + $iImagePos[3]
  Case 'select'
    MsgBox(0, 'Select', 'Select first')
    $iXa = MouseGetPos(0)
    $iYa = MouseGetPos(1)
    MsgBox(0, 'Select', 'Select second')
    $iXb = MouseGetPos(0)
    $iYb = MouseGetPos(1)
EndSwitch


Local $array[1][2] ; Will be used for saving coordinates of lines of text
Local $iBlackPixelCount
Local $bHasNoPreviousZero = True ; Will be used as a toggle for skipping succeeding lines that have no black pixels.
Local $iStep = 1
For $y = $iYa To $iYb Step $iStep
  FFSnapShot($iXa, $y, $iXb, $y)
  $iBlackPixelCount = FFColorCount(0x000000, 50, False)
  ; $iBlackPixelPercent = Round(100 * (FFColorCount(0x000000, 0, False) / $iImagePos[2]), 2) ; Computes for percentage of black pixels on horizontal line.
  ; If $iBlackPixelCount > 0 Or $bHasNoPreviousZero Then
  ; $bHasNoPreviousZero = True
  _ArrayAdd($array, $iBlackPixelCount)
  $array[UBound($array) - 1][1] = $y
  ; If $iBlackPixelCount = 0 Then $bHasNoPreviousZero = False
  ; EndIf
Next

; To check detected lines. This will move mouse to the lower-left corner of the text line.
For $i = 0 To UBound($array) - 1 Step +1
  If $array[$i][0] > 0 Then
    If $i + 1 <= UBound($array) - 1 Then
      If $array[$i + 1][0] = 0 Then
        MouseMove($iXa, $array[$i][1])
        Sleep(500)
      EndIf
    EndIf
  ;
  EndIf
Next

_ArrayDisplay($array)

Func __ControlGetPos($hWnd, $controlID, $bAbsolute = Default)
  ;; https://www.autoitscript.com/forum/topic/88345-absolute-position-of-guicontrol/
  If $bAbsolute = Default Then $bAbsolute = True
  Local $controlPos
  Switch $bAbsolute
    Case True
      Local Const $hWnd_Control = ControlGetHandle($hWnd, "", $controlID)
      $controlPos = WinGetPos($hWnd_Control)
    Case False
      $controlPos = ControlGetPos($hWnd, "", $controlID)
  EndSwitch
  If Not IsArray($controlPos) Then SetError(1)
  Return $controlPos
EndFunc   ;==>__ControlGetPos

Func __Exit()
  Exit
EndFunc   ;==>__Exit

 

FastFind.dll

FastFind.au3

FastFind64.dll

sample image.bmp

Edited by Mingre

Share this post


Link to post
Share on other sites
Mingre

Hello instead of checking for black pixels, I changed it to check white pixels and it now works on TNR font. Thanks everyone! :) 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • reida
      By reida
      Hello. On page there is mail address. How to keep in a text file this address? 
      It is page and email
      https://alpari.com/
      info@alpari.com
    • caramen
      By caramen
      Hey.  
       
      I requested help about how to get a value from a text in a variable. 
      Now i know how to do that. But i learned with the command FileRead. Now i whould like to know how to replace the command :
      FileRead('Dossier.txt') The purpose is to read a webpage text. To find some value inside. 
       
      Btw i tryed to play with WindowsInfo.au3 but i dont got much thing.  
    • nacerbaaziz
      By nacerbaaziz
      Hello, dear.
      I am a newbie in autoit language
      I don't master any other programming language.
      I also don't understand how to manage the DLL files
      In addition to all this I am blind, and as we all know the explanations provided for the blind in programming are very few.
      So I'm trying to rely on myself for learning.
      I work hard to design some simple software and tools that make it easier to use computers, as they perform some tasks that may be difficult for the blind.
      I am also looking for the programs that the blind need and cannot use it because it's haven't the compatibility with the screen reader, and i try to design a simple programs that do the same work.
      In these tools and programs I have to deal directly with screen readers, when i must to forcing these programs to read some processes and tasks that it cannot be read in normal mode.
      To force these programs to read, I have to deal with the API files.
      And as I said, I'm very weak in managing DLLs.
      So I hope you'll try to help me, please.
      I did a lot of research until I found the API file to manage the free screen reader (NVDA).
      It is open source
      It is based on CPP language and Python
      And I don't understand the both languages.
      For this I decided to put you an API file link here which is attached with it examples, source and DLL file
      I hope you can help me by convert these functions to AutoIt include file
      I offer you a very serious apology if the participation is a violation of the laws.
      I wish I could find here someone to help me.
      To download the file please click on this link
      http://www.nvda-project.org/nvdaControllerClient/nvdaControllerClient_20100219.7z
      and To download the free screen reader,  please following this link
      https://www.nvaccess.org/download/

      I repeat my apologies and thanks in advance.
       
    • therks
      By therks
      I'm trying to create a simple clock widget that automatically scales the text to the size of the window. I came up with the following method, but it doesn't work as well as I'd like. It especially has trouble scaling to the width of the window for some reason (in the example, try resizing the window to be narrow and tall).
      Does anyone have a better method?
      #include <Misc.au3> #include <WinAPIConv.au3> #include <GUIConstants.au3> #include <GDIPlus.au3> Opt('MustDeclareVars', 1) Global $_FONT_FAMILY = 'Arial', $_LB_TEXT Main() Func Main() _GDIPlus_Startup() Local $hGUI GUIRegisterMsg($WM_SIZE, WM_SIZE) $hGUI = GUICreate('', 300, 100, Default, Default, $WS_OVERLAPPEDWINDOW, BitOR($WS_EX_TOOLWINDOW, $WS_EX_TOPMOST)) $_LB_TEXT = GUICtrlCreateLabel('This is a string', 0, 0, 300, 100, BitOR($SS_CENTER, $SS_CENTERIMAGE)) GUICtrlSetFont($_LB_TEXT, _MeasureString($hGUI, GUICtrlRead($_LB_TEXT), $_FONT_FAMILY), 0, 0, $_FONT_FAMILY, 5) GUISetState() Local $iGM While 1 $iGM = GUIGetMsg() Switch $iGM Case $GUI_EVENT_CLOSE ExitLoop EndSwitch WEnd _GDIPlus_Shutdown() EndFunc Func WM_SIZE($hWnd, $iMsg, $wParam, $lParam) GUICtrlSetFont($_LB_TEXT, _MeasureString($hWnd, GUICtrlRead($_LB_TEXT), $_FONT_FAMILY), 0, 0, $_FONT_FAMILY, 5) EndFunc Func _MeasureString($hWnd, $sString, $sFont = 'Arial') Local $iError, $aSize, $hGraphic, $hFormat, $hFamily, $tLayout, $iFontSize, $hFont, $aInfo If Not IsHWnd($hWnd) Then $hWnd = GUICtrlGetHandle($hWnd) EndIf $aSize = WinGetClientSize($hWnd) $hGraphic = _GDIPlus_GraphicsCreateFromHWND($hWnd) $hFormat = _GDIPlus_StringFormatCreate() $hFamily = _GDIPlus_FontFamilyCreate($sFont) $tLayout = _GDIPlus_RectFCreate(0, 0, $aSize[0], $aSize[1]) $iFontSize = 0 Do If Not $hFamily Then $iError = 1 $iFontSize = 10 ExitLoop EndIf $iFontSize += 1 $hFont = _GDIPlus_FontCreate($hFamily, $iFontSize, 0) $aInfo = _GDIPlus_GraphicsMeasureString($hGraphic, $sString, $hFont, $tLayout, $hFormat) _GDIPlus_FontDispose($hFont) If $aInfo[1] = 0 Then ExitLoop Until DllStructGetData($aInfo[0], 3) >= $aSize[0] Or DllStructGetData($aInfo[0], 4) >= $aSize[1] $iFontSize -= 1 _GDIPlus_FontFamilyDispose($hFamily) _GDIPlus_StringFormatDispose($hFormat) _GDIPlus_GraphicsDispose($hGraphic) Return SetError($iError, 0, $iFontSize) EndFunc
    • dadalt95
      By dadalt95
      Hi!
      I want to get the signal in this website:
      https://binary-signal.com/pt/chart/eurusd
      I have tried using _IEBodyReadText  and some _StringBetween. What happens is that the text are being update every tick and _IEBodyReadText doesn't.
      To perform the update I used  _IEAction($oIE, "refresh") but it's not good because the website block me after some time due too many requests..
      Is there any other way to get this text every tick?
      PS: The text I want to get is WAIT, CALL or PUT.
       
      Here is the code:
      global $oIE = _IECreate ('https://binary-signal.com/pt/chart/eurusd', 0, 1 , 1 , 0) Local $sText = _IEBodyReadText($oIE) $result = _StringBetween ( $sText , 'PUTEUR/USD on Binary-signal.com', 'sinal está PRONTA') ;MsgBox ( 0, "asf", $result[0]) $espera=StringInStr($result[0], "WAIT") $compra=StringInStr($result[0], "CALL") $venda=StringInStr($result[0], "PUT") ;MsgBox($MB_SYSTEMMODAL, "", $espera) $n=0 $c=0 Captar() Func Captar() ;_IENavigate($oIE, "https://binary-signal.com/pt/chart/eurusd") ;MsgBox($MB_SYSTEMMODAL, "", $n) Local $sText = _IEBodyReadText($oIE) $result = _StringBetween ( $sText , 'PUTEUR/USD on Binary-signal.com', 'sinal está PRONTA') If (Not $compra=0) And $n=0 Then MsgBox($MB_SYSTEMMODAL, "", "COMPRE") $n=1 $c=$c+1 ;_IEAction($oIE, "refresh") Sleep(60000) Captar() ElseIf (Not $venda=0) And $n=0 Then MsgBox($MB_SYSTEMMODAL, "", "VENDA") $c=$c+1 $n=1 ;_IEAction($oIE, "refresh") ;MsgBox($MB_SYSTEMMODAL, "", $n) Sleep(60000) Captar() ElseIf (Not $venda=0) And $n=1 Then ;MsgBox($MB_SYSTEMMODAL, "", "Esperando próxima rodada") $n=1 ;MsgBox($MB_SYSTEMMODAL, "", $n) ;_IEAction($oIE, "refresh") Sleep(60000) Captar() ElseIf (Not $venda=0) And $n=1 Then ;MsgBox($MB_SYSTEMMODAL, "", "Esperando próxima rodada") $n=1 ;MsgBox($MB_SYSTEMMODAL, "", $n) ;_IEAction($oIE, "refresh") Sleep(60000) Captar() Else ;MsgBox("", "", "ESPERE") $n=0 Sleep(1000) ;_IEAction($oIE, "refresh") Local $sText = _IEBodyReadText($oIE) $result = _StringBetween ( $sText , 'PUTEUR/USD on Binary-signal.com', 'sinal está PRONTA') Captar() EndIf EndFunc  
×