Tesseract (Screen OCR) UDF

[madScientist]

Can you give some image examples? I played with it for a little while and got it working ok in the end.

I just finished a small event-driven AI to play a game (i had a choice between wow, one with planes, and fifa 2009) - you can see at

that i have managed to read the player's mana and hit points and other demo's coming soon (really want to get A+ on this.. it's my final project)

-

As a side-note, on a Core2Duo @ 2,3, running at 95% load, reading a few lines with Tesseract every second barely makes a dent in the cpu graph. Impressive!

Edited January 8, 2010 by madScientist

[Taerion]

Hi there

I just spent about 3 hours figuring out why the image seems to be moved somewhat randomly if the "indent"-parameters are changed.

I finally found the problem to be a math related problem in the CaptureToTIFF function.

Edited February 28, 2010 by Taerion

[Taerion]

Hi

I tried to edit my post but the button didn't show up...

I only tested it with the _TesseractScreenCapture function, because that's what I need for my project.

Please let me know if something is wrong with my correction.

I changed the accoording line in the code to:

...
    $hWnd = _WinAPI_GetDesktopWindow()
    $hDC = _WinAPI_GetDC($hWnd)

    ;this next line
    $hBMP = _WinAPI_CreateCompatibleBitmap($hDC, ($pos[2] - $right_indent - $left_indent) * $scale, ($pos[3] - $bottom_indent - $top_indent) * $scale)

    _WinAPI_ReleaseDC($hWnd, $hDC)
    $hImage1 = _GDIPlus_BitmapCreateFromHBITMAP ($hBMP)
    $hGraphic = _GDIPlus_ImageGetGraphicsContext($hImage1)

    _GDIPLus_GraphicsDrawImageRect($hGraphic, $hImage2, 0 - ($left_indent * $scale), 0 - ($top_indent * $scale), $pos[2] * $scale, $pos[3] * $scale)
    $CLSID = _GDIPlus_EncodersGetCLSID($Ext)
...

[Loshmi]

What fonts does this OCR support? Will i be able to read of small part of the screen? I Need only English but with custom made font..(thought it's pretty similar to standard Ariel).

I need to read off the screen a couple of words..have anybody tested this script in that manner??

Thanks in advance!

[1905russell]

I know this is a really dumb question because so many people are raving about this script and I'm dying to see the examples but I cannot load $oIE. It returns 403 Forbidden - You don't have permission to access /bgw/applets/1.02/ListDemo/ListDemo.html on this server.

How do I get permission?

[schilders]

If you are recognising text in a window, then I would be using _TesseractWinCapture() instead of _TesseractScreenCapture(). I can't work out the indent values for _TesseractScreenCapture() unless I know your screen width and height (ie. the resolution of your desktop, such as 1024 x 768).

If your desktop resolution was 1024 x 768 pixels, then the indent parameters for _TesseractScreenCapture() would be:

230, 360, 884, 438

because:

884 = (1024 - 230 + 90)

438 = (768 - 360 + 30)

If you use _TesseractWinCapture() then the indent parameters would be:

230, 360, 660, 243

because:

660 = (800 - 230 + 90)

243 = (573 - 360 + 30)

I'm finding that this formula is not working for me. My screen coordinates are 1024x768

say my real world coordinates are for a rectangle of text are: 429(leftx) 518(topy) 654(leftx) 532(bottomy)

then my indented coords should be: 429,518,(1024 - 654),(768 - 532)

right???

Everytime I do this I get a large box thats not even close to the right size or coordinates. I even set scaling to 1. Still too big. Am I doing something wrong??? I even tried turning my dual displays off.

[Realm]

I'm slightly new to AutoIt, so please excuse my noobishness. I had been using another ocr method using gocr.exe, but have many problems getting it to learn characters with different fonts, and a friend pointed this out as a possible solution. I made a small sample script, just to see if I understand it correctly, but unfortunately I get a error titled: OCR-Error, with a text box only returning '1' as the error description. Maybe some1 could take a moment to look at my test script, and tell me where I'm going wrong.

#include <Tesseract.au3>

Opt("WinTextMatchMode",2)

$texta = _TesseractScreenCapture(0,"",0,2,317,602,1243,800,1)
MsgBox(0,"Text is:","for ScreenCapture w/o cleanup:" & @CRLF & @CRLF & $texta)
$textb = _TesseractScreenCapture(0,"",1,2,317,602,1243,800,1)
MsgBox(0,"Text is:","for ScreenCapture with cleanup:" & @CRLF & @CRLF & $textb)
$textc = _TesseractWinCapture("Mozilla Firefox","",0,"",0,2,317,602,1243,800,1)
MsgBox(0,"Text is:","for WinCapture w/o cleanup:" & @CRLF & @CRLF & $textc)
$textd = _TesseractWinCapture("Mozilla Firefox","",0,"",1,2,317,602,1243,800,1)
MsgBox(0,"Text is:","for WinCapture with cleanup:" & @CRLF & @CRLF & $textd)

Edit: There capture parameters were wrong/fixed. Unfortunately that was not the problem, still have same error message

Edit2:kk first problem fixed, the Func commands on your include were lowercase, not sure if it saved that way, or if your copy that I downloaded is that way, but now its partially working once I corrected them. Problem is that the coords are way off, and while playing with the numbers to try and get them to read what I need, it creates more errors. It reads alot better than the previous OCR example I was using, now to attempt to understand how your coords work.

Edited May 27, 2010 by Realm

[bryhall2]

I have been working with these scripts for a few days now, and have been running into the same problem. I am hoping someone can assist.

1. The demo's are gone - unable to access those websites as another poster said.

My goal is to use the mouse clicks to determine the areas that I want the OCR to read. I created a script to determine the location of the mouse pointer, to provide me with the x/y coordinate when clicked and when released. The area that is then covered between the click and release needs to be passed on into the Tesseract for OCR.

When I set the top left coordinates and leave the bottom right coordinates blank, it seems to respond as I would expect. When I start changing the bottom right coordinates from 0, the top left coordinates seem to be changing and things are getting cut off. I can not figure out why indenting right and bottom, are modifying the indent on top and left.

In my example, I am using screen capture as I receive erorr messages when trying to use windowcapture. My envrionment is 1440x900.

expandcollapse popup

#Include <Tesseract.au3>
#include <Misc.au3> 
;~ Detect Mouse click and store position to variable $pos1. Once click detected, go to Release() to check for release of click
Func Press()
While 1
if _ispressed("01") Then
    $pos1 = ""
    $pos1 = MouseGetPos()
sleep (50)
Release()
else 
    sleep(1)
endif
WEnd
EndFunc

;~ Detect mouse release and store to variable $pos2.
Func Release()
while 1
    if not _ispressed("01") then 
        $pos2 = ""
        $pos2 = MouseGetPos()
        sleep (50)
        Output2()
        Else
        sleep(1)
    EndIf
WEnd
EndFunc



Func Output2()
;~ I was using this area to make modifications to my position, for testing. 
$pos1x=$pos1[0]
$pos1y=$pos1[1]
$pos2x=$pos2[0]
$pos2y=$pos2[1]

;~ msgbox(0,"test",$pos1x)

$textb = _TesseractScreenCapture(0,"",1,2,$pos1x,$pos1y,$pos2x,$pos2y,1)
MsgBox(0,"Text is:","for ScreenCapture with cleanup:" & $textb)
EndFunc

[bryhall2]

Also, I have tried modifying the scale back to 1. I have also tried taking my screen resolution and subtracting x/y coordinates for the bottom right coordinates. Neither options worked.

[DJKMan]

Bryhall, what if $textb is actually an array? Try that. Clearly I have no clue about this script but I'll try to help anyway.

Does anyone have a working example of Tesseract? The one on the first page denies access when I try the IE example.

[JohnOne]

Then you want to look a little further than the first page, because there are multiple examples in this thread.

[civilcalc]

Please help,

I am trying run the Tesseract.exe file but just get a black dos box pop up for like a nanosecond then vanish. no program at all.

I am running a vista machine, 32bit with runtime and visual express 2010.

Thanks in advance

[Phenom]

When I try to use _TesseractWinCapture() I get the following error:

C:\Program Files (x86)\AutoIt3\Include\Tesseract.au3 (968) : ==> Subscript used with non-Array variable.:

$hBMP = _WinAPI_CreateCompatibleBitmap($hDC, ($pos[2] * $scale) - ($right_indent * $scale), ($pos[3] * $scale) - ($bottom_indent * $scale))

$hBMP = _WinAPI_CreateCompatibleBitmap($hDC, ($pos^ ERROR

[Phenom]

When I use _TesseractScreenFind() it returns a number, but it doesn't return an array. Does it return the coordinates of where it finds the text? I'm trying to move the mouse to where the text is found, but I don't see how that's possible if only one number is returned. I need two numbers for the x and y coordinates.

[hornet]

I am looking for the same type of thing Phenom is...any help would be greatly appreciated.

-hornet

[PsaltyDS]

I am looking for the same type of thing Phenom is...any help would be greatly appreciated.

Asked and answered.

It just doesn't work that way.

[hornet]

Alright so if this function doesn't work that way...is there one that does. Phenom and myself have been asking a very basic question that I feel should have been answered quite a bit ago. All the answers we get seem to beat around the bush. We ask,"Can we do it with "X" function?" and the responder says,"Nope, don't work that way". We ask,"Can we do it with "Y" function?" and the responder says,"Nope, don't work that way". So is there anyone in this forum that can tell us,"Nope, don't work that way BUT you can use function "XYZ" to do what you're asking."????

-hornet

[JohnOne]

FindBmp.au3 and ImageSearch.au3 both work, and furthermore are both easily found with all kinds of search terms you must have used find what you are looking for.

But this really is not the thread to be outwardly venting your frustrations.

[Xibalba]

Does anyone have the ListDemo.html saved, or uploaded somewhere else?

The two example scripts in the first post doesn't work any longer because access is denied to url:

http://www73.pair.com/bgw/applets/1.02/ListDemo/ListDemo.html

So, anyone have them? Thx

[lightwave]

Firstly, thanks a lot to seangriffin for this!

Aww, man. I've been playing with this for a short while, and just came to implement it in one of my scripts and I get an error in Tesseract.au3 because $hwnd hasn't been declared. This means this UDF can't just be dropped in an used in scripts with

Opt("MustDeclareVars", 0)

Any chance of an update? I know I can declare these myself anywhere, but should that be necessary?

Cheers!