Jump to content
Sign in to follow this  
Mingre

Detect lines of black text on screen (not OCR)

Recommended Posts

Mingre

Hello forums,

I'd like to know if anyone has better idea than what I have. I'm creating a program that will detect (not OCR!) blocks of words (black text) on a white background displayed on the screen. (If anyone is curious about its purpose, this code will be part of a program that will randomly select a word on a PDF file and have the user guess that word. I'll be using it to quiz myself with my PDF notes. )

What I created so far: Using FastFind.au3, I was able to make it detect lines of black text (font Lucida Console) on a single column. It basically considers a line a separate one if bounded above and below by horizontal lines that do not contain black pixels. Next step, which I haven't done yet, will be to go through each line and detect the blocks of words.

The program works fine so far with Lucida Console font but it doesn't when tested on TNR font (see attached sample image). The former can be tested by setting $iOption to "text" while the latter by setting it to "select".

So does anyone have a better idea on going about this? Thank you!

Sorry if my code's a mess. I tried to make it cleaner before posting it but this is the best I can manage.

#include <FastFind.au3>
#include <array.au3>
HotKeySet('{ESC}', '__Exit')

Local $iImagePos[4], $iXa, $iXb, $iYa, $iYb
$iOption = 'text'
Switch $iOption
  Case 'text'   ; Creates sample text file for detection
    Local Const $sSampleTextFileName = 'sample.txt'
    FileWrite($sSampleTextFileName, '')
    FileOpen($sSampleTextFileName, 2)
    FileWrite($sSampleTextFileName, 'First line.' & @CRLF & 'Second line.' & @CRLF & @CRLF & 'Third line.')
    FileClose($sSampleTextFileName)
    ShellExecute($sSampleTextFileName)
    Local Const $sSampleTextFileTitle = 'sample - Notepad'
    WinWait($sSampleTextFileTitle)
    WinMove($sSampleTextFileTitle, '', 0, 0, @DesktopWidth / 2, @DesktopHeight / 2)
    Sleep(500)
    $iImagePos = __ControlGetPos($sSampleTextFileTitle, '[CLASS:Edit; INSTANCE:1]') ; Gets the absolute coordinates of the edit control.
    Local $iXa = $iImagePos[0], _
        $iXb = $iImagePos[0] + $iImagePos[2], _
        $iYa = $iImagePos[1], _
        $iYb = $iImagePos[1] + $iImagePos[3]
  Case 'select'
    MsgBox(0, 'Select', 'Select first')
    $iXa = MouseGetPos(0)
    $iYa = MouseGetPos(1)
    MsgBox(0, 'Select', 'Select second')
    $iXb = MouseGetPos(0)
    $iYb = MouseGetPos(1)
EndSwitch


Local $array[1][2] ; Will be used for saving coordinates of lines of text
Local $iBlackPixelCount
Local $bHasNoPreviousZero = True ; Will be used as a toggle for skipping succeeding lines that have no black pixels.
Local $iStep = 1
For $y = $iYa To $iYb Step $iStep
  FFSnapShot($iXa, $y, $iXb, $y)
  $iBlackPixelCount = FFColorCount(0x000000, 50, False)
  ; $iBlackPixelPercent = Round(100 * (FFColorCount(0x000000, 0, False) / $iImagePos[2]), 2) ; Computes for percentage of black pixels on horizontal line.
  ; If $iBlackPixelCount > 0 Or $bHasNoPreviousZero Then
  ; $bHasNoPreviousZero = True
  _ArrayAdd($array, $iBlackPixelCount)
  $array[UBound($array) - 1][1] = $y
  ; If $iBlackPixelCount = 0 Then $bHasNoPreviousZero = False
  ; EndIf
Next

; To check detected lines. This will move mouse to the lower-left corner of the text line.
For $i = 0 To UBound($array) - 1 Step +1
  If $array[$i][0] > 0 Then
    If $i + 1 <= UBound($array) - 1 Then
      If $array[$i + 1][0] = 0 Then
        MouseMove($iXa, $array[$i][1])
        Sleep(500)
      EndIf
    EndIf
  ;
  EndIf
Next

_ArrayDisplay($array)

Func __ControlGetPos($hWnd, $controlID, $bAbsolute = Default)
  ;; https://www.autoitscript.com/forum/topic/88345-absolute-position-of-guicontrol/
  If $bAbsolute = Default Then $bAbsolute = True
  Local $controlPos
  Switch $bAbsolute
    Case True
      Local Const $hWnd_Control = ControlGetHandle($hWnd, "", $controlID)
      $controlPos = WinGetPos($hWnd_Control)
    Case False
      $controlPos = ControlGetPos($hWnd, "", $controlID)
  EndSwitch
  If Not IsArray($controlPos) Then SetError(1)
  Return $controlPos
EndFunc   ;==>__ControlGetPos

Func __Exit()
  Exit
EndFunc   ;==>__Exit

 

FastFind.dll

FastFind.au3

FastFind64.dll

sample image.bmp

Edited by Mingre

Share this post


Link to post
Share on other sites
Mingre

Hello instead of checking for black pixels, I changed it to check white pixels and it now works on TNR font. Thanks everyone! :) 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • dadalt95
      By dadalt95
      Hi!
      I want to get the signal in this website:
      https://binary-signal.com/pt/chart/eurusd
      I have tried using _IEBodyReadText  and some _StringBetween. What happens is that the text are being update every tick and _IEBodyReadText doesn't.
      To perform the update I used  _IEAction($oIE, "refresh") but it's not good because the website block me after some time due too many requests..
      Is there any other way to get this text every tick?
      PS: The text I want to get is WAIT, CALL or PUT.
       
      Here is the code:
      global $oIE = _IECreate ('https://binary-signal.com/pt/chart/eurusd', 0, 1 , 1 , 0) Local $sText = _IEBodyReadText($oIE) $result = _StringBetween ( $sText , 'PUTEUR/USD on Binary-signal.com', 'sinal está PRONTA') ;MsgBox ( 0, "asf", $result[0]) $espera=StringInStr($result[0], "WAIT") $compra=StringInStr($result[0], "CALL") $venda=StringInStr($result[0], "PUT") ;MsgBox($MB_SYSTEMMODAL, "", $espera) $n=0 $c=0 Captar() Func Captar() ;_IENavigate($oIE, "https://binary-signal.com/pt/chart/eurusd") ;MsgBox($MB_SYSTEMMODAL, "", $n) Local $sText = _IEBodyReadText($oIE) $result = _StringBetween ( $sText , 'PUTEUR/USD on Binary-signal.com', 'sinal está PRONTA') If (Not $compra=0) And $n=0 Then MsgBox($MB_SYSTEMMODAL, "", "COMPRE") $n=1 $c=$c+1 ;_IEAction($oIE, "refresh") Sleep(60000) Captar() ElseIf (Not $venda=0) And $n=0 Then MsgBox($MB_SYSTEMMODAL, "", "VENDA") $c=$c+1 $n=1 ;_IEAction($oIE, "refresh") ;MsgBox($MB_SYSTEMMODAL, "", $n) Sleep(60000) Captar() ElseIf (Not $venda=0) And $n=1 Then ;MsgBox($MB_SYSTEMMODAL, "", "Esperando próxima rodada") $n=1 ;MsgBox($MB_SYSTEMMODAL, "", $n) ;_IEAction($oIE, "refresh") Sleep(60000) Captar() ElseIf (Not $venda=0) And $n=1 Then ;MsgBox($MB_SYSTEMMODAL, "", "Esperando próxima rodada") $n=1 ;MsgBox($MB_SYSTEMMODAL, "", $n) ;_IEAction($oIE, "refresh") Sleep(60000) Captar() Else ;MsgBox("", "", "ESPERE") $n=0 Sleep(1000) ;_IEAction($oIE, "refresh") Local $sText = _IEBodyReadText($oIE) $result = _StringBetween ( $sText , 'PUTEUR/USD on Binary-signal.com', 'sinal está PRONTA') Captar() EndIf EndFunc  
    • vin1
      By vin1
      i need a script that can replace words in text with other words from a words list
      it has to have 2 edit boxes input/output and a button "run"
      from word_list.txt it choses which word to replace with. words are separated with commas for multiple choice
       

       
       

       
       
      ; AutoIt GUI Example ; Created: 17/01/2005 - CyberSlug ; Modifed: 05/12/2011 - guinness ; Modifed: 09/06/2014 - mLipok #Region INCLUDE #include <AVIConstants.au3> #include <GuiConstantsEx.au3> #include <TreeViewConstants.au3> #EndRegion INCLUDE #Region GUI GUICreate("Sample GUI", 400, 420) GUISetIcon(@SystemDir & "\mspaint.exe", 0) #EndRegion GUI #Region EDIT GUICtrlCreateEdit(@CRLF & " Sample Edit Control", 10, 10, 380, 175) GUICtrlSetTip(-1, '#Region EDIT') #EndRegion EDIT #Region EDIT2 GUICtrlCreateEdit(@CRLF & " Sample Edit Control", 10, 185, 380, 175) GUICtrlSetTip(-1, '#Region EDIT') #EndRegion EDIT2 #Region BUTTON GUICtrlCreateButton("Sample Button", 10, 360, 100, 30) GUICtrlSetTip(-1, '#Region BUTTON') #EndRegion BUTTON #Region GUI MESSAGE LOOP GUISetState(@SW_SHOW) While 1 Switch GUIGetMsg() Case $GUI_EVENT_CLOSE ExitLoop EndSwitch WEnd GUIDelete() #EndRegion GUI MESSAGE LOOP  
    • Skeletor
      By Skeletor
      MS Splash Screens
      The office 2016/17 office suite has made many people turn heads with its functions and especially their splash screen.
      I now present to you, the MS Style Splash Screen. For Rookies, Novices and anyone who wants a ready made splash screen for your gui application.

      You can now download it and also try it out yourself.
      You have three colors to choose from:
      Blue
      Red
      Green
      All three colours are from the famous programs.
      I also included the KODA form to you can have freedom with this splash screen.
      Note: the KODA form does not have the three colours as well as the correct blue. 

      Have fun and enjoy.
      Note:
      Thanks to kaz for updating the side scrolling dots.
       
       
    • Ambient
      By Ambient
      I have tried several way sto get this to work to no avail. Any help would be appreciated. #include <ListViewConstants.au3> #include <GuiListView.au3> #include "GuiListViewEx.au3" $idItem = GUICtrlCreateListViewItem($r & " | " & $c & $adoRs.Fields("TransactionAmount").value & " | " & $Timestamp & " | " & _StringProper($adoRs.Fields("Tillid").value) & @CRLF & @CRLF & "Card Number : " & $cardnum & " " , $idListView) ConsoleWrite( "IDITEM" & $idItem) If STRINGLEFT($adoRs.Fields("TransactionAmount").value,1)= "-" Then Consolewrite(" String is Negative" & @CRLF) ;Drops in Here as expected ;GUICtrlSetColor(-1, $COLOR_RED) ; Colour line if item is negative this didn' work $TxC= _GUICtrlListView_SetTextColor($hListView, $CLR_RED) ; I also tried $TxC= _GUICtrlListView_SetTextColor(-1, 0xFF0000) CONSOLEWRITE("TEXTCOLOUR RETURNED " & $TxC & @crlf) ; This is returning True ELSE Consolewrite(" String is NOT Negative" & @CRLF) EndIf  
    • AndyS19
      By AndyS19
      I want to copy just the selected text in an input control and put it to the clipboard, but GuiCtrlRead() always reads all of the text in the control.
      In this test code, when I highlight a  portion of the left control's text and press ^C, I want to copy just that text to the clipboard; but when I highlight a  portion of the right control's text and press ^C, I want to copy all of its text to the clipboard.
      #include <Debug.au3> _DebugSetup(@ScriptName & "_debug.txt", False, 2, "") _DebugOut("=============== " & @MON & "/" & @MDAY & "/" & @YEAR & " " & @HOUR & ":" & @MIN & ":" & @SEC & " ====================") #include <GUIConstants.au3> #include <WinAPI.au3> #include <Array.au3> Opt("GUICloseOnESC", 1) ; ESC closes GUI? (0 = no, 1 = yes) Opt("GUIOnEventMode", 1) ; Set OnEvent mode Opt('MustDeclareVars', 1) OnAutoItExitRegister("Event_GUIClose") Opt("GUIEventOptions", 1) ;0=default, 1=just notification, 2=GuiCtrlRead tab index Opt("WinTitleMatchMode", -2) ;1=start, 2=subStr, 3=exact, 4=advanced, -1 to -4=Nocase Global $hWnd_MainWin, $iID_Left, $iID_Right _Main() Func _Main() $hWnd_MainWin = GUICreate("test7", 200, 300) $iID_Left = GUICtrlCreateInput("AAA BBB CCC (get Selected text)", 10, 30, 200, 25) GUICtrlSetOnEvent($iID_Left, "handle_left_btn") $iID_Right = GUICtrlCreateInput("DDD EEE FFF (get All text)", 10, 60, 200, 25) GUICtrlSetOnEvent($iID_Right, "handle_right_btn") Local $aAccelKeys[1][2] = [["^c", $hWnd_MainWin]] GUISetAccelerators($aAccelKeys) GUISetOnEvent($GUI_EVENT_CLOSE, 'Event_GUIClose') GUISetState() ; Make everything visible While 1 Sleep(127) WEnd EndFunc ;==>_Main Func handle_CTRL_C_key() ConsoleWrite("+++: handle_CTRL_C_key() entered" & @CRLF) Local $hCtrl, $ctrlID, $control_classname, $text $control_classname = ControlGetFocus($hWnd_MainWin) $hCtrl = ControlGetHandle($hWnd_MainWin, "", $control_classname) $ctrlID = _WinAPI_GetDlgCtrlID($hCtrl) If ($ctrlID = $iID_Left) Then $text = GUICtrlRead($ctrlID) ClipPut($text) ConsoleWrite("+++: Copied selected text:" & @CRLF & "==>" & $text & "<==" & @CRLF) ElseIf ($ctrlID = $iID_Right) Then $text = GUICtrlRead($ctrlID) ClipPut($text) ConsoleWrite("+++: Copied all text:" & @CRLF & "==>" & $text & "<==" & @CRLF) EndIf EndFunc ;==>handle_CTRL_C_key Func Event_GUIClose() Exit (99) EndFunc ;==>Event_GUIClose Func handle_left_btn() ConsoleWrite("+++: handle_left_btn() entered" & @CRLF) EndFunc ;==>handle_left_btn Func handle_right_btn() ConsoleWrite("+++: handle_right_btn() entered" & @CRLF) EndFunc ;==>handle_right_btn
       
×