Jump to content

Tesseract (Screen OCR) UDF


Recommended Posts

  • 3 months later...
  • Replies 136
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

This UDF provides text capturing support for applications and controls using Tesseract - an OCR engine currently developed by Google. Tesseract was originally developed as proprietary software at H

Some example code to test if concept still works used version 3.05.1 output.txt will contain the result (on my system about 30 seconds to analyze full screen which you probably normally should not do)

Ok, so I'm smashing my head with autoit and tesseract and I have a problem. The code works and does what it needs to do on one environment but not on another. When running it on Windows Server 2012 (2

Posted Images

Hi,

The function "CaptureToTiff" in the tesseract udf fails under Windows 2008 R1.

At least temporarily, I'm stumped, and I'd really appreciate any hints, or even of course an outright fix to the tesseract udf ;) .

The failure happens with the line

$hGraphic = _GDIPlus_ImageGetGraphicsContext($hImage1)

it sets $hGraphic to 0. According to the doc, if it fails, _GDIPlus_ImageGetGraphicsContext should return -1. In this case, it returns no graphics context and also raises no error.

So I tried the same code under XP32, and it works fine.

I went to the help example for _GDIPlus_ImageGetGraphicsContext, tried it, and it also works fine under 2008R1. So _GDIPlus_ImageGetGraphicsContext probably works correctly, and Microsoft expects things to be done differently under win2008R1.

Any ideas on how to get around this?

Thanks!

Edited by rodent1
Link to post
Share on other sites
  • 4 months later...

Thanks to the author for his work! :)

As I understand it there is no other better and easier than Tesseract UDF means to get information about custom controls.

If so, look forward to the updated version.

Release date is not known to anyone? :huh:

Link to post
Share on other sites

Thanks to the author for his work! :)

As I understand it there is no other better and easier than Tesseract UDF means to get information about custom controls.

If so, look forward to the updated version.

Release date is not known to anyone? :huh:

What updated version are you referring to?

AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt   ipify 

Monkey's are, like, natures humans.

Link to post
Share on other sites
  • 2 weeks later...
  • 4 months later...

This is an awesome project! Thank you for sharing it. I would include some error catching around _WinAPI_CreateCompatibleBitmap($hDC...). I spent a day or two troubleshooting why my program wasn't capturing the required screen on Windows Server 2003. Turns out there are memory limitations with CreateCompatibleBitmap which may prevent you from processing the BitMap successfully if your "scale" or window size is too large. You can view my notes on this here:

I ended up increasing my bottom_indent value to reduce the amount of window I was capturing. The approach allowed me to limit the file size while increasing the scale (which seems to improve OCR reliability).

Thanks again!

Link to post
Share on other sites

Thanks to user "UEZ", I now have a function that doesn't seem to have the same memory limits as CreateCompatibleBitmap() does.

I'm now using this CaptureToTiff function instead of the one posted within this thread.

Func CaptureToTIFF($win_title = "", $win_text = "", $ctrl_id = "", $sOutImage = "", $scale = 1, $left_indent = 0, $top_indent = 0, $right_indent = 0, $bottom_indent = 0)
Local $hWnd, $hwnd2, $hDC, $hBMP, $hImage1, $hGraphic, $CLSID, $tParams, $pParams, $tData, $i = 0, $hImage2, $pos[4], $tar_leftx, $tar_lefty, $tar_rightx, $tar_righty, $winsize[4]
Local $Ext = StringUpper(StringMid($sOutImage, StringInStr($sOutImage, ".", 0, -1) + 1))
Local $giTIFColorDepth = 24
Local $giTIFCompression = $GDIP_EVTCOMPRESSIONNONE
; If capturing a control
if StringCompare($ctrl_id, "") <> 0 Then
$hwnd2 = ControlGetHandle($win_title, $win_text, $ctrl_id)
$pos = ControlGetPos($win_title, $win_text, $ctrl_id)
Else
; If capturing a window
if StringCompare($win_title, "") <> 0 Then
$hwnd2 = WinGetHandle($win_title, $win_text)
$pos = WinGetPos($win_title, $win_text)
Else
; If capturing the desktop
$hwnd2 = ""
$pos[0] = 0
$pos[1] = 0
$pos[2] = @DesktopWidth
$pos[3] = @DesktopHeight
EndIf
EndIf


; Capture an image of the window / control
if IsHWnd($hwnd2) Then
WinActivate($win_title, $win_text)
;added to calculate missing variables from function call needed to control the screen shot ProcessClose
$winsize = WinGetPos ( $win_title, $win_text )
$tar_leftx = $left_indent
$tar_lefty = $top_indent
$tar_rightx = $winsize[2] - $right_indent
$tar_righty = $winsize[3] - $bottom_indent
;msgbox(0,"Bottom_Indent",$bottom_indent)
$hBitmap2 = _ScreenCapture_CaptureWnd("", $hwnd2, $tar_leftx, $tar_lefty, $tar_rightx, $tar_righty, False)
Else
;added to calculate missing variables from function call needed to control the screen shot ProcessClose
$winsize = $pos
$tar_leftx = $left_indent
$tar_lefty = $top_indent
$tar_rightx = $winsize[2] - $right_indent
$tar_righty = $winsize[3] - $bottom_indent
$hBitmap2 = _ScreenCapture_Capture("", $tar_leftx, $tar_lefty, $tar_rightx, $tar_righty, False)
EndIf
;old version of if statement - correction to function
;if IsHWnd($hwnd2) Then
;
; WinActivate($win_title, $win_text)
; $hBitmap2 = _ScreenCapture_CaptureWnd("", $hwnd2, 0, 0, -1, -1, False)
;Else
;
; $hBitmap2 = _ScreenCapture_Capture("", 0, 0, -1, -1, False)
;EndIf
_GDIPlus_Startup ()
; Convert the image to a bitmap
$hImage2 = _GDIPlus_BitmapCreateFromHBITMAP ($hBitmap2)
;Commenting out what I had before
#cs
$hWnd = _WinAPI_GetDesktopWindow()
$hDC = _WinAPI_GetDC($hWnd)
;Old version of this function call
;$hBMP = _WinAPI_CreateCompatibleBitmap($hDC, ($pos[2] * $scale) - ($right_indent * $scale), ($pos[3] * $scale) - ($bottom_indent * $scale))
$hBMP = _WinAPI_CreateCompatibleBitmap($hDC, ($tar_rightx - $tar_leftx) * $scale, ($tar_righty - $tar_lefty) * $scale)
_WinAPI_ReleaseDC($hWnd, $hDC)
$hImage1 = _GDIPlus_BitmapCreateFromHBITMAP ($hBMP)
#ce
;Implementing UEZ's suggestion
$hImage1 = DllCall($ghGDIPDll, "uint", "GdipCreateBitmapFromScan0", _
"int", ($pos[2] * $scale) - ($right_indent * $scale), _
"int", ($pos[3] * $scale) - ($bottom_indent * $scale), _
"int", 0, "int", $GDIP_PXF24RGB, "ptr", 0, "int*", 0)
$hImage1 = $hImage1[6]
$hGraphic = _GDIPlus_ImageGetGraphicsContext($hImage1)
;Modified from orginal to support corrected screen captures
;_GDIPLus_GraphicsDrawImageRect($hGraphic, $hImage2, 0 - ($left_indent * $scale), 0 - ($top_indent * $scale), ($pos[2] * $scale) + $left_indent, ($pos[3] * $scale) + $top_indent)
_GDIPLus_GraphicsDrawImageRect($hGraphic, $hImage2, 0, 0, ($tar_rightx - $tar_leftx) * $scale, ($tar_righty - $tar_lefty) * $scale)
$CLSID = _GDIPlus_EncodersGetCLSID($Ext)
; Set TIFF parameters
$tParams = _GDIPlus_ParamInit(2)
$tData = DllStructCreate("int ColorDepth;int Compression")
DllStructSetData($tData, "ColorDepth", $giTIFColorDepth)
DllStructSetData($tData, "Compression", $giTIFCompression)
_GDIPlus_ParamAdd($tParams, $GDIP_EPGCOLORDEPTH, 1, $GDIP_EPTLONG, DllStructGetPtr($tData, "ColorDepth"))
_GDIPlus_ParamAdd($tParams, $GDIP_EPGCOMPRESSION, 1, $GDIP_EPTLONG, DllStructGetPtr($tData, "Compression"))
If IsDllStruct($tParams) Then $pParams = DllStructGetPtr($tParams)
; Save TIFF and cleanup
_GDIPlus_ImageSaveToFileEx($hImage1, $sOutImage, $CLSID, $pParams)
_GDIPlus_ImageDispose($hImage1)
_GDIPlus_ImageDispose($hImage2)
_GDIPlus_GraphicsDispose ($hGraphic)
_WinAPI_DeleteObject($hBMP)
_WinAPI_DeleteObject ($hBitmap2)
_GDIPlus_Shutdown()
EndFunc
Link to post
Share on other sites
  • 1 year later...
  • 1 month later...

Apparently I'm unable to get it to run.

I have this:

HotKeySet("{F2}", "Capture")

While 1
    Sleep(150)
WEnd

Func Capture()
    _TesseractScreenCapture(0, "", "", "", 10, 10, 10, 10, 1)
EndFunc

When I set it up with a 0 at the end, nothing happens. When there is 1 to actually show me the end result I receive following error:

"F:\Tesseract.au3" (185) : ==> Variable must be of type "Object".:
$Obj1.ShowFile ($capture_filename, 1)
$Obj1^ ERROR

Any ideas pls?

Link to post
Share on other sites
  • 1 month later...

Apparently I'm unable to get it to run.

I have this:

HotKeySet("{F2}", "Capture")

While 1
    Sleep(150)
WEnd

Func Capture()
    _TesseractScreenCapture(0, "", "", "", 10, 10, 10, 10, 1)
EndFunc

When I set it up with a 0 at the end, nothing happens. When there is 1 to actually show me the end result I receive following error:

"F:\Tesseract.au3" (185) : ==> Variable must be of type "Object".:
$Obj1.ShowFile ($capture_filename, 1)
$Obj1^ ERROR

Any ideas pls?

 

Change the path to tesseract like this:

 

;Used to be

;ShellExecuteWait(@ProgramFilesDir & "tesseracttesseract.exe", $capture_filename & " " & $ocr_filename)
;Set to this
ShellExecuteWait("C:Program Files (x86)Tesseract-OCRtesseract.exe", $capture_filename & " " & $ocr_filename, "", "", @SW_HIDE)
Link to post
Share on other sites
  • 3 months later...

Ok, so I'm smashing my head with autoit and tesseract and I have a problem. The code works and does what it needs to do on one environment but not on another. When running it on Windows Server 2012 (2 DELL monitors connected) it works, but when I used on my home environment, Windows 8 (2 TVs connected) it does not.

Here is the code:

#include <SimpleTesseract.au3>
#include <ScreenCapture.au3>
#include <Array.au3>
#include <String.au3>

GLOBAL $var1
GLOBAL $num1
GLOBAL $var2
GLOBAL $num2
GLOBAL $diff
GLOBAL $x
GLOBAL $z

;Tesseract Path for temp files (where you put the app)
_TesseractTempPathSet("D:\AUTOIT")

;Get First Val Screenshot
SplashImageOn ( "Test", "D:\AUTOIT\tst1.jpg" , 135 , 30 , 0 , 0, 1)
sleep(200)
_ScreenCapture_Capture("D:\AUTOIT\TEST1.jpg", 0, 0, 135, 30)
sleep(1000)

;Get First Val
SplashImageOn ( "Test", "D:\AUTOIT\TEST1.jpg" , 135 , 30 , 0 , 0, 1)
sleep(200)
$var1 =  _TesseractScreenCapture(0, "", 0, 3, 0, 0, 135, 30, 0)
sleep(200)

;Get Second Val Screenshot
SplashImageOn ( "Test", "D:\AUTOIT\tst2.jpg" , 135 , 30 , 0 , 0, 1)
sleep(200)
_ScreenCapture_Capture("D:\AUTOIT\TEST2.jpg", 0, 0, 135, 30)
sleep(1000)

;Get Second Val
SplashImageOn ( "Test", "D:\AUTOIT\tst2.jpg" , 135 , 30 , 0 , 0, 1)
sleep(200)
$var2 =  _TesseractScreenCapture(0, "", 0, 4, 0, 0, 135, 30, 0)
sleep(200)

;Parse First Val
$var1 = StringReplace($var1, ",", "")
$num1 = Number($var1)

;Parse Second Val
$var2 = StringReplace($var2, ",", "")
$num2 = Number($var2)

;Calculate
$diff = $num2 - $num1

;Parse XP Number ===========================
$x = StringLen($diff)
$z = Int($x/3)
if IsInt($x/$z) then
   if $z = 1 Then
      $diff = _StringInsert ($diff, ",", $x-3); under 1,000,000 XP
   else
      for $i = 1 to $z-1
      $diff = _StringInsert ($diff, ",", $x-3*$i); under 1,000,000,000 XP
      Next
   endif
   else
   for $i = 1 to $z
   $diff = _StringInsert ($diff, ",", $x-3*$i); under 1,000,000,000,000 XP
   Next
Endif
;End Of Parse XP Number ====================

;Display
Tooltip("First Image: " & $var1 & "   Second Image: " & $var2 & "    Difference: " & $diff, 500, 0)
sleep(10000) 

Here are the needed files:

http://www.filedropper.com/simpletesseract

http://www.filedropper.com/tst1

http://www.filedropper.com/tst2

Please help if you have any idea.

 

 

 

Apparently I'm unable to get it to run.

 

I have this:

HotKeySet("{F2}", "Capture")

While 1
    Sleep(150)
WEnd

Func Capture()
    _TesseractScreenCapture(0, "", "", "", 10, 10, 10, 10, 1)
EndFunc

When I set it up with a 0 at the end, nothing happens. When there is 1 to actually show me the end result I receive following error:

"F:Tesseract.au3" (185) : ==> Variable must be of type "Object".:
$Obj1.ShowFile ($capture_filename, 1)
$Obj1^ ERROR

Any ideas pls?

 

You should always set it up with 0 at the end. It always does something. If it "does" nothing when set with 0 at the end it means that the array is null. When you set it up with 1 at the end you will get that error which you stated. Also if you configured a path two files will be created.

Random name having the following:

1. file .tiff with image on which tesseract performs OCR

2. file .txt with result after ocr was performed on image

Here is an image if it is not clear:

http://www.filedropper.com/explanation

Link to post
Share on other sites
  • 3 months later...
  • 1 month later...

Doesn't work for me.. trying to do _TesseractScreenCapture over Google Spreadsheet on the title (just to test, it's big and standard).

#include <Tesseract.au3>

Global $Chrome = WinWait("[TITLE:Jobb - Google Kalkylark - Google Chrome]")
If $Chrome = 0 Then Exit 2

WinActivate($Chrome)

ConsoleWrite('"' & _TesseractScreenCapture(0, "", 1, 5, 781, 113, 832, 136, 1) & '"' & @LF)

It returns "" to console. And if I allow the debug thing at end of command, I get this:

>"C:\Program Files (x86)\AutoIt3\SciTE\..\autoit3.exe" /ErrorStdOut "\\valsrv02\regnrjobs\Misc\Lagerarbeten\asd.au3"    
"C:\Program Files (x86)\AutoIt3\Include\Tesseract.au3" (185) : ==> Variable must be of type "Object".:
$Obj1.ShowFile ($capture_filename, 1)
$Obj1^ ERROR
>Exit code: 1    Time: 1.151

Please help!

Link to post
Share on other sites
  • 3 months later...

Apparently I'm unable to get it to run.

I have this:

HotKeySet("{F2}", "Capture")

While 1
    Sleep(150)
WEnd

Func Capture()
    _TesseractScreenCapture(0, "", "", "", 10, 10, 10, 10, 1)
EndFunc

When I set it up with a 0 at the end, nothing happens. When there is 1 to actually show me the end result I receive following error:

"F:\Tesseract.au3" (185) : ==> Variable must be of type "Object".:
$Obj1.ShowFile ($capture_filename, 1)
$Obj1^ ERROR

Any ideas pls?

Doesn't work for me.. trying to do _TesseractScreenCapture over Google Spreadsheet on the title (just to test, it's big and standard).

#include <Tesseract.au3>

Global $Chrome = WinWait("[TITLE:Jobb - Google Kalkylark - Google Chrome]")
If $Chrome = 0 Then Exit 2

WinActivate($Chrome)

ConsoleWrite('"' & _TesseractScreenCapture(0, "", 1, 5, 781, 113, 832, 136, 1) & '"' & @LF)

It returns "" to console. And if I allow the debug thing at end of command, I get this:

>"C:\Program Files (x86)\AutoIt3\SciTE\..\autoit3.exe" /ErrorStdOut "\\valsrv02\regnrjobs\Misc\Lagerarbeten\asd.au3"    
"C:\Program Files (x86)\AutoIt3\Include\Tesseract.au3" (185) : ==> Variable must be of type "Object".:
$Obj1.ShowFile ($capture_filename, 1)
$Obj1^ ERROR
>Exit code: 1    Time: 1.151

Please help!

These are both the same issue and I believe that they both stem to the COM object being created for the "Preview.Preview.1" path in the Tesseract.au3 file. I don't believe this is a valid object to call anymore in Windows 8/8.1.

I don't know enough about using COM objects to figure out what the fix would be, but this is why it works when you don't display the results and leave the last parameter as 0.

 

Looking into this error I did find a few issues with the Tesseract.au3 file (I haven't dived in extensively, just issues I ran into).

1. The same thing that GregG mentioned. The path to "tesseract.exe" is no longer valid by default in the file. Instead of letting the file use @ProgramFilesDir I just created a variable and set it to "C:\Program Files (x86)\Tesseract-OCR" and then adjusted the string appended to it so it was just "tesseract.exe" after. (ex. $NewFilePath & "tesseract.exe") You could also probably just set the whole path to a variable then just call that, just as good. 

2. I noticed that the .tif file wasn't getting created in the default "C:/" location. So I just created a new folder in my documents and called _TesseractTempPathSet to set it to that folder. That took care of that. 

3. After the .tif file was getting created I noticed the image was blank. This one, I haven't completely figured out yet...but I at least got it to generate the .tif. There is an issue in the following portion of code and I don't know enough about it to track it down right now.

_GDIPlus_Startup ()

    ; Convert the image to a bitmap
    $hImage2 = _GDIPlus_BitmapCreateFromHBITMAP ($hBitmap2)

    $hWnd = _WinAPI_GetDesktopWindow()
    $hDC = _WinAPI_GetDC($hWnd)
    $hBMP = _WinAPI_CreateCompatibleBitmap($hDC, ($pos[2] * $scale) - ($right_indent * $scale), ($pos[3] * $scale) - ($bottom_indent * $scale))

    _WinAPI_ReleaseDC($hWnd, $hDC)
    $hImage1 = _GDIPlus_BitmapCreateFromHBITMAP ($hBMP)
    $hGraphic = _GDIPlus_ImageGetGraphicsContext($hImage1)
    _GDIPLus_GraphicsDrawImageRect($hGraphic, $hImage2, 0 - ($left_indent * $scale), 0 - ($top_indent * $scale), ($pos[2] * $scale) + $left_indent, ($pos[3] * $scale) + $top_indent)
    $CLSID = _GDIPlus_EncodersGetCLSID($Ext)

    ; Set TIFF parameters
    $tParams = _GDIPlus_ParamInit(2)
    $tData = DllStructCreate("int ColorDepth;int Compression")
    DllStructSetData($tData, "ColorDepth", $giTIFColorDepth)
    DllStructSetData($tData, "Compression", $giTIFCompression)
    _GDIPlus_ParamAdd($tParams, $GDIP_EPGCOLORDEPTH, 1, $GDIP_EPTLONG, DllStructGetPtr($tData, "ColorDepth"))
    _GDIPlus_ParamAdd($tParams, $GDIP_EPGCOMPRESSION, 1, $GDIP_EPTLONG, DllStructGetPtr($tData, "Compression"))
    If IsDllStruct($tParams) Then $pParams = DllStructGetPtr($tParams)

    ; Save TIFF and cleanup
    _GDIPlus_ImageSaveToFileEx($hImage1, $sOutImage, $CLSID, $pParams)
    _GDIPlus_ImageDispose($hImage1)
    _GDIPlus_ImageDispose($hImage2)
    _GDIPlus_GraphicsDispose ($hGraphic)
    _WinAPI_DeleteObject($hBMP)
    _GDIPlus_Shutdown()

This is from CaptureToTIFF in Tesseract.au3. I replaced all that with _ScreenCapture_SaveImage ( $sOutImage, $hBitmap2), which works, but you obviously can't doe the scaling or anything like that with it. So any help with that would be cool.

4. The last thing I found was that the Tesseract was not generating the .txt file. This ended up being because my folder path had spaces in it, so reducing that to no spaces worked. The other thing I did, just in case I wanted a space somewhere was change the ShellExecuteWait($ProgramFilesDir & "\tesseract.exe", $capture_filename & " " & $ocr_filename) lines to ShellExecuteWait($ProgramFilesDir & "\tesseract.exe", """" & $capture_filename & """ """ & $ocr_filename & """"). All that does is add quotes around the path names so they work with spaces. 

 

That's all I've been able to figure out, so if anyone else has any ideas or knows what's up let us know.

Link to post
Share on other sites

If it's just a matter of getting Tesseract to work then take a look at Getting Tesseract to Work.

As I said, Tesseract works fine...The issue the other two, and me, are having with this file is that it creates a COM object for "Preview.Preview.1" which doesn't seem to be valid for Windows 8 so that when it tries to call the ShowFile() function it can't because it doesn't existAnd the issues with Tesseract itself have already been worked out as I noted in the things that I had to correct.

Specifically, it's this code that is the issue.

$Obj1 = ObjCreate("Preview.Preview.1") ;This is the line creating the object that is incorrect for Windows 8
$Obj1_ctrl = GUICtrlCreateObj($Obj1, 0, 0, 640, 480)
$Obj1.ShowFile($capture_filename, 1) ;This is the line making the call to the function to an object that doesn't exist
Link to post
Share on other sites
  • 10 months later...

Hello,

I am trying to run tesseract, but I cant find any exe installator anywhere.

All the links seems to be broken/moved and on official repo I can only download source / binaries that doesnt install.

 

Can anyone upload theirs copy of working installator of tesseract for all the people that come looking for it? 

 

Thanks!

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...