Jump to content

OCR and MODI MiRect property

Go to solution Solved by Nine,

Recommended Posts

I saw a forum question that asked how to OCR non selectable text from a screen and get its coordinates.

This intrigued me, as I would like to have this ability in my scripts.

I found the following VB code and I am having difficulty "Converting" it to Autoit code. It provides a pixel count to a rectangle around a specified word relative to the window it is in.

Sub TestRects()

  Dim miSelectRectDoc As MODI.Document
  Dim miSelectRectWord As MODI.Word
  Dim miSelectRectRects As MODI.MiRects
  Dim miSelectRectRect As MODI.MiRect
  Dim strRectInfo As String
  ' Load an existing TIFF file.
  Set miSelectRectDoc = New MODI.Document
  miSelectRectDoc.Create "C:\document1.tif"
  ' Perform OCR.
  ' Retrieve and display bounding rectangle information.
  Set miSelectRectRects = miSelectRectDoc.Images(0).Layout.Words(2).Rects
  strRectInfo = "Word falls within " & miSelectRectRects.Count & _
   " bounding rectangle(s)." & vbCrLf
    For Each miSelectRectRect In miSelectRectRects
    strRectInfo = strRectInfo & _
      " Rectangle coordinates: " & vbCrLf & _
      "  - Left: " & miSelectRectRect.Left & vbCrLf & _
      "  - Right: " & miSelectRectRect.Right & vbCrLf & _
      "  - Top: " & miSelectRectRect.Top & vbCrLf & _
      "  - Bottom: " & miSelectRectRect.Bottom
  MsgBox strRectInfo, vbInformation + vbOKOnly, _
    "Rectangle Information"
  Set miSelectRectRect = Nothing
  Set miSelectRectRects = Nothing
  Set miSelectRectWord = Nothing
  Set miSelectRectDoc = Nothing

End Sub


I already have an OCR function that closely resembles this code. I would like to insert just the MiRect function into it.

I get an error when attempting to create the MiRect and MiRects objects:

Func OCR_Region($x,$y,$width,$height)  ;note $height is NOT the height, it is the Y coordinate bottom right corner, width is bottom right corner X.
    Dim $miDoc, $Doc
    Dim $str
    Dim $oWord
    Dim $sArray[200]
    Dim $oMyError
    Dim $HexNumber
    Dim $msg
    Dim $i
        ;Const $miLANG_CZECH = 5
        ;Const $miLANG_DANISH = 6
        ;Const $miLANG_DUTCH = 19
        Const $miLANG_ENGLISH = 9
        ;Const $miLANG_FINNISH = 11
        ;Const $miLANG_FRENCH = 12
        ;Const $miLANG_GERMAN = 7
        ;Const $miLANG_GREEK = 8
        ;Const $miLANG_HUNGARIAN = 14
        ;Const $miLANG_ITALIAN = 16
        ;Const $miLANG_JAPANESE = 17
        ;Const $miLANG_KOREAN = 18
        ;Const $miLANG_NORWEGIAN = 20
        ;Const $miLANG_POLISH = 21
        ;Const $miLANG_PORTUGUESE = 22
        ;Const $miLANG_RUSSIAN = 25
        ;Const $miLANG_SPANISH = 10
        ;Const $miLANG_SWEDISH = 29
        ;Const $miLANG_TURKISH = 31
        ;Const $miLANG_SYSDEFAULT = 2048
        ;Const $miLANG_CHINESE_SIMPLIFIED = 2052
        ;Const $miLANG_CHINESE_TRADITIONAL = 1028

        Local $hBitmap1,$hImage1,$temp
        $temp=@ScriptDir & "\temp.bmp"
        _GDIPlus_Startup ()
        ; Capture screen region
        $hBitmap1 = _ScreenCapture_Capture ("",$x, $y, $width, $height, False)
        $hImage1 = _GDIPlus_BitmapCreateFromHBITMAP ($hBitmap1)
        _GDIPlus_ImageSaveToFile ($hImage1, $temp)
        _GDIPlus_ImageDispose ($hImage1)
        _WinAPI_DeleteObject ($hBitmap1)
        $miDoc = ObjCreate("MODI.Document")
        ;These two statements added for the MiRect function, they both produce an error
        $miSelectRectRects = ObjCreate("MODI.MiRects")
        $miSelectRectRect = ObjCreate("MODI.MiRect")

        ;MsgBox(0,"",$temp)         ;this is the bitmap image which is "temp.bmp"
        $miDoc.Ocr($miLANG_ENGLISH, True, False)

        ;These two statements added for the MiRect function, neither produce an error
         $miSelectRectRects = $miDoc.Images(0).Layout.Words(1).Rects
         $Left=$miSelectRectRects.count     ;I don need ".Count", but it does something

        $i = 0
            For $oWord in $miDoc.Images(0).Layout.Words
                $str = $str & $oWord.text & @CrLf
                ;ConsoleWrite($oWord.text & @CRLF)
                $sArray [$i] = $oWord.text
                $i += 1

            Return $sArray

Thanks for your help

Edited by AutoitMike
Link to post
Share on other sites
  • Solution

You may mean something like this :

Local $miRects = $miDoc.Images(0).Layout.Words(1).Rects
Local $strRectInfo = "Word falls within " & $miRects.Count & _
    " bounding rectangle(s)." & @CRLF
For $miRect In $miRects
  $strRectInfo &= _
      " Rectangle coordinates: " & @CRLF & _
      "  - Left: " & $miRect.Left & @CRLF & _
      "  - Right: " & $miRect.Right & @CRLF & _
      "  - Top: " & $miRect.Top & @CRLF & _
      "  - Bottom: " & $miRect.Bottom
MsgBox($MB_SYSTEMMODAL, "", $strRectInfo)


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Danyfirex
      Hello guys.  I recently saw some posts that Windows 10 provides OCR API. So I decided to create a UDF.
      What's UWPOCR?
      UWPOCR UDF is a simple library to use Universal Windows Platform Optical character recognition API.
      Get Text From Image File. Get Text From GDI+ Bitmap. Easy to use.  Usage:
      #include "..\UWPOCR.au3" _Example() Func _Example() Local $sOCRTextResult = _UWPOCR_GetText(FileOpenDialog("Select Image", @ScriptDir & "\", "Images (*.jpg;*.bmp;*.png;*.tif;*.gif)")) MsgBox(0,"",$sOCRTextResult) EndFunc Get Words Rect(Example):

      More examples here.
      Check UWPOCR UDF on GitHub.
    • By MrKm
      This tiny yet powerful UDF will help you to convert Images to text with the help of  OCRSpace API version 3.50 .
      Detect text from a local file.
      ; ========================================================= ; Example 2 : Gets text from an image from a local path reference ; : Searchable PDF is not requested by default. ; : Processes it using a basic OCR logic. ; ========================================================= $b_Create_Searchable_PDF = True ; Use a table logic for receipt OCR $b_Table = True ; Set your key here. $v_OCRSpaceAPIKey = "" $OCROptions = _OCRSpace_SetUpOCR($v_OCRSpaceAPIKey, 1, $b_Table, True, "eng", True, Default, Default, $b_Create_Searchable_PDF) $sText_Detected = _OCRSpace_ImageGetText($OCROptions, @scriptdir & "\receipt.jpg", 0, "SEARCHABLE_URL") ConsoleWrite( _ " Detected text : " & $sText_Detected & @CRLF & _ " Error Returned : " & @error & @CRLF & _ " PDF URL : " & Eval("SEARCHABLE_URL") & @CRLF)  
      Detect text from a URL reference.
      ; ========================================================= ; Example 1 : Gets text from an image using a url reference ; : Searchable PDF is not requested. ; : Processes it using a basic OCR logic. ; ========================================================= $v_OCRSpaceAPIKey = "" ; SetUp some preferences.. $OCROptions = _OCRSpace_SetUpOCR($v_OCRSpaceAPIKey, 1, False, True, "eng", True, Default, Default, False) ; Make the request.. $sText_Detected = _OCRSpace_ImageGetText($OCROptions, "https://i.imgur.com/vbYXwJm.png", 0) ConsoleWrite( _ " Detected text : " & $sText_Detected & @CRLF & _ " Error Returned : " & @error & @CRLF)    
      Detect text from a URL reference to an array
      #include "OCRSpaceUDF\_OCRSpace_UDF.au3" #include <array.au3> ; Set your key here. $v_OCRSpaceAPIKey = "" $OCROptions = _OCRSpace_SetUpOCR($v_OCRSpaceAPIKey, 1, $b_Table, True, "eng", True, Default, Default, False) ; Below, the return type is set to 1 to return an array containing the coordinates of the bounding boxes for each word detected, ; in the format : #WordDetected , #Left , #Top , 3Height, #Width $aText_Detected = _OCRSpace_ImageGetText($OCROptions, "https://i.imgur.com/Z1enogD.jpeg", 1) _ArrayDisplay($aText_Detected, "")  
      Download Latest Version : 
    • By Miliardsto
      Hello I wondering how to read more complicated text from image with Tesseract or other method.
      I used this script and it works with simple text on white background
      I need to read text which looks like this

      What I need to do?
      Download fonts or something from there https://github.com/tesseract-ocr/tesseract
      gimme some info please
    • By PuneetTewani
      #include <IE.au3>
      #include <Tesseract.au3>
      #include <MsgBoxConstants.au3>
      #include <Math.au3>
      #include <FileConstants.au3>
      #include <StringConstants.au3>
      #include <File.au3>
      #include <ScreenCapture.au3>
      #include <sound.au3>
      #Include <WinAPI.au3>
      #include <Date.au3>
      $OCR_Result = _TesseractScreenCapture(0,"",1,2,220,660,500,730,1)
      $OCR_Result1 = _TesseractScreenCapture(0,"",1,2,220,660,500,730,1)
      $OCR_Result2 = _TesseractScreenCapture(0,"",1,2,220,660,500,730,1)
      $OCR_Result3 = _TesseractScreenCapture(0,"",1,2,220,660,500,730,1)

      $sound = _SoundStatus("C:\ExpertAdvisorBuyAlert.wav")
      while _nowtime < 3.30 pm
          If $sound = True Then
             if $OCR_Result1 > $OCR_Result2
      Trying to ocr some values on chart in real time(once per minute) and buy/sell securities on basis of alert generated in my software.
      I am struck onto few steps.
      1. On Tesseract Screen Capture indentation parameters. How can we determine the exact parameters if I just want numeric values only.
      2. The Tesseract Screen Capture generates and error Obj1 on line 185 which needs to be resolved.
      3. Sometimes lines get overlapped with values. What to do in that case.
      3. Detecting the sound as and when it approaches and then comparing the ocr values to decide on either buy or sell.
      The values that needs to be fetched are encircled.

    • By mdepot
      I have a situation where I am repeatedly capturing a region of the screen and feeding it into Tesseract OCR.  Since the OCR is a relatively slow operation, I would like to create an in memory cache of the ocr results.  An ideal hash key for this cache would be a checksum of the captured image.  With this I could capture the region, checksum it, and then only if I don't get a cache hit I would write the image out to disk for external OCR.
      Now I know I can do this by saving the captured image out to disk, and then summing the disk file with _Crypt_HashFile().  But that's still slower than I would like, and it shouldn't be necessary.  Ideally, it should be possible to checksum the image data directly in memory so I don't have to go to disk at all.  In order to do that, I need a way to dump a representation of the image into a string  (or some equivalent).  Then I could use the _Crypt_HashData() function against that string to create my cache hash key.
      Googling around I found an article here that shows a way to convert an image object to a byte array using System.Drawing.  This was the closest thing I found to what I'm trying to do.  I don't know if that method could be used from within AutoIT, or if perhaps there may be a better way I don't know about.  If someone could give me a shove in the right direction it would be a big help.  Thanks!
  • Create New...