Jump to content

Tesseract (Screen OCR) UDF


Recommended Posts

Can you give some image examples? I played with it for a little while and got it working ok in the end.

I just finished a small event-driven AI to play a game (i had a choice between wow, one with planes, and fifa 2009) - you can see at

that i have managed to read the player's mana and hit points and other demo's coming soon (really want to get A+ on this.. it's my final project)


As a side-note, on a Core2Duo @ 2,3, running at 95% load, reading a few lines with Tesseract every second barely makes a dent in the cpu graph. Impressive!

Edited by madScientist
Link to comment
Share on other sites

  • 1 month later...

Hi there

I just spent about 3 hours figuring out why the image seems to be moved somewhat randomly if the "indent"-parameters are changed.

I finally found the problem to be a math related problem in the CaptureToTIFF function.

Edited by Taerion
Link to comment
Share on other sites


I tried to edit my post but the button didn't show up...

I only tested it with the _TesseractScreenCapture function, because that's what I need for my project.

Please let me know if something is wrong with my correction.

I changed the accoording line in the code to:

    $hWnd = _WinAPI_GetDesktopWindow()
    $hDC = _WinAPI_GetDC($hWnd)

    ;this next line
    $hBMP = _WinAPI_CreateCompatibleBitmap($hDC, ($pos[2] - $right_indent - $left_indent) * $scale, ($pos[3] - $bottom_indent - $top_indent) * $scale)

    _WinAPI_ReleaseDC($hWnd, $hDC)
    $hImage1 = _GDIPlus_BitmapCreateFromHBITMAP ($hBMP)
    $hGraphic = _GDIPlus_ImageGetGraphicsContext($hImage1)

    _GDIPLus_GraphicsDrawImageRect($hGraphic, $hImage2, 0 - ($left_indent * $scale), 0 - ($top_indent * $scale), $pos[2] * $scale, $pos[3] * $scale)
    $CLSID = _GDIPlus_EncodersGetCLSID($Ext)
Link to comment
Share on other sites

  • 2 weeks later...

What fonts does this OCR support? Will i be able to read of small part of the screen? I Need only English but with custom made font..(thought it's pretty similar to standard Ariel).

I need to read off the screen a couple of words..have anybody tested this script in that manner??

Thanks in advance!

Link to comment
Share on other sites

I know this is a really dumb question because so many people are raving about this script and I'm dying to see the examples but I cannot load $oIE. It returns 403 Forbidden - You don't have permission to access /bgw/applets/1.02/ListDemo/ListDemo.html on this server.

How do I get permission?

Link to comment
Share on other sites

  • 1 month later...

If you are recognising text in a window, then I would be using _TesseractWinCapture() instead of _TesseractScreenCapture(). I can't work out the indent values for _TesseractScreenCapture() unless I know your screen width and height (ie. the resolution of your desktop, such as 1024 x 768).

If your desktop resolution was 1024 x 768 pixels, then the indent parameters for _TesseractScreenCapture() would be:

230, 360, 884, 438


884 = (1024 - 230 + 90)

438 = (768 - 360 + 30)

If you use _TesseractWinCapture() then the indent parameters would be:

230, 360, 660, 243


660 = (800 - 230 + 90)

243 = (573 - 360 + 30)

I'm finding that this formula is not working for me. My screen coordinates are 1024x768

say my real world coordinates are for a rectangle of text are: 429(leftx) 518(topy) 654(leftx) 532(bottomy)

then my indented coords should be: 429,518,(1024 - 654),(768 - 532)


Everytime I do this I get a large box thats not even close to the right size or coordinates. I even set scaling to 1. Still too big. Am I doing something wrong??? I even tried turning my dual displays off.

Link to comment
Share on other sites

  • 4 weeks later...

I'm slightly new to AutoIt, so please excuse my noobishness. I had been using another ocr method using gocr.exe, but have many problems getting it to learn characters with different fonts, and a friend pointed this out as a possible solution. I made a small sample script, just to see if I understand it correctly, but unfortunately I get a error titled: OCR-Error, with a text box only returning '1' as the error description. Maybe some1 could take a moment to look at my test script, and tell me where I'm going wrong.

#include <Tesseract.au3>


$texta = _TesseractScreenCapture(0,"",0,2,317,602,1243,800,1)
MsgBox(0,"Text is:","for ScreenCapture w/o cleanup:" & @CRLF & @CRLF & $texta)
$textb = _TesseractScreenCapture(0,"",1,2,317,602,1243,800,1)
MsgBox(0,"Text is:","for ScreenCapture with cleanup:" & @CRLF & @CRLF & $textb)
$textc = _TesseractWinCapture("Mozilla Firefox","",0,"",0,2,317,602,1243,800,1)
MsgBox(0,"Text is:","for WinCapture w/o cleanup:" & @CRLF & @CRLF & $textc)
$textd = _TesseractWinCapture("Mozilla Firefox","",0,"",1,2,317,602,1243,800,1)
MsgBox(0,"Text is:","for WinCapture with cleanup:" & @CRLF & @CRLF & $textd)

Edit: There capture parameters were wrong/fixed. Unfortunately that was not the problem, still have same error message

Edit2:kk first problem fixed, the Func commands on your include were lowercase, not sure if it saved that way, or if your copy that I downloaded is that way, but now its partially working once I corrected them. Problem is that the coords are way off, and while playing with the numbers to try and get them to read what I need, it creates more errors. It reads alot better than the previous OCR example I was using, now to attempt to understand how your coords work.

Edited by Realm

My Contributions: Unix Timestamp: Calculate Unix time, or seconds since Epoch, accounting for your local timezone and daylight savings time. RegEdit Jumper: A Small & Simple interface based on Yashied's Reg Jumper Function, for searching Hives in your registry. 

Link to comment
Share on other sites

I have been working with these scripts for a few days now, and have been running into the same problem. I am hoping someone can assist.

1. The demo's are gone - unable to access those websites as another poster said.

My goal is to use the mouse clicks to determine the areas that I want the OCR to read. I created a script to determine the location of the mouse pointer, to provide me with the x/y coordinate when clicked and when released. The area that is then covered between the click and release needs to be passed on into the Tesseract for OCR.

When I set the top left coordinates and leave the bottom right coordinates blank, it seems to respond as I would expect. When I start changing the bottom right coordinates from 0, the top left coordinates seem to be changing and things are getting cut off. I can not figure out why indenting right and bottom, are modifying the indent on top and left.

In my example, I am using screen capture as I receive erorr messages when trying to use windowcapture. My envrionment is 1440x900.

#Include <Tesseract.au3>
#include <Misc.au3> 
;~ Detect Mouse click and store position to variable $pos1. Once click detected, go to Release() to check for release of click
Func Press()
While 1
if _ispressed("01") Then
    $pos1 = ""
    $pos1 = MouseGetPos()
sleep (50)

;~ Detect mouse release and store to variable $pos2.
Func Release()
while 1
    if not _ispressed("01") then 
        $pos2 = ""
        $pos2 = MouseGetPos()
        sleep (50)

Func Output2()
;~ I was using this area to make modifications to my position, for testing. 

;~ msgbox(0,"test",$pos1x)

$textb = _TesseractScreenCapture(0,"",1,2,$pos1x,$pos1y,$pos2x,$pos2y,1)
MsgBox(0,"Text is:","for ScreenCapture with cleanup:" & $textb)
Link to comment
Share on other sites

Bryhall, what if $textb is actually an array? Try that. Clearly I have no clue about this script but I'll try to help anyway.

Does anyone have a working example of Tesseract? The one on the first page denies access when I try the IE example.



Link to comment
Share on other sites

Please help,

I am trying run the Tesseract.exe file but just get a black dos box pop up for like a nanosecond then vanish. no program at all.

I am running a vista machine, 32bit with runtime and visual express 2010.

Thanks in advance

Link to comment
Share on other sites

When I try to use _TesseractWinCapture() I get the following error:

C:\Program Files (x86)\AutoIt3\Include\Tesseract.au3 (968) : ==> Subscript used with non-Array variable.:

$hBMP = _WinAPI_CreateCompatibleBitmap($hDC, ($pos[2] * $scale) - ($right_indent * $scale), ($pos[3] * $scale) - ($bottom_indent * $scale))

$hBMP = _WinAPI_CreateCompatibleBitmap($hDC, ($pos^ ERROR

Link to comment
Share on other sites

  • 3 weeks later...

When I use _TesseractScreenFind() it returns a number, but it doesn't return an array. Does it return the coordinates of where it finds the text? I'm trying to move the mouse to where the text is found, but I don't see how that's possible if only one number is returned. I need two numbers for the x and y coordinates.

Link to comment
Share on other sites

I am looking for the same type of thing Phenom is...any help would be greatly appreciated.

Asked and answered.

It just doesn't work that way.


Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law
Link to comment
Share on other sites

Alright so if this function doesn't work that way...is there one that does. Phenom and myself have been asking a very basic question that I feel should have been answered quite a bit ago. All the answers we get seem to beat around the bush. We ask,"Can we do it with "X" function?" and the responder says,"Nope, don't work that way". We ask,"Can we do it with "Y" function?" and the responder says,"Nope, don't work that way". So is there anyone in this forum that can tell us,"Nope, don't work that way BUT you can use function "XYZ" to do what you're asking."????


Link to comment
Share on other sites

FindBmp.au3 and ImageSearch.au3 both work, and furthermore are both easily found with all kinds of search terms you must have used find what you are looking for.

But this really is not the thread to be outwardly venting your frustrations.

AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt   ipify 

Monkey's are, like, natures humans.

Link to comment
Share on other sites

  • 4 weeks later...

Does anyone have the ListDemo.html saved, or uploaded somewhere else?

The two example scripts in the first post doesn't work any longer because access is denied to url:


So, anyone have them? Thx

Link to comment
Share on other sites

Firstly, thanks a lot to seangriffin for this!

Aww, man. I've been playing with this for a short while, and just came to implement it in one of my scripts and I get an error in Tesseract.au3 because $hwnd hasn't been declared. This means this UDF can't just be dropped in an used in scripts with

Opt("MustDeclareVars", 0)

Any chance of an update? I know I can declare these myself anywhere, but should that be necessary?


Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Create New...