holebergers

Position Recognition based on Text or image

6 posts in this topic

Hi guys, I am new to autoit and all and I am stuck in a problem. How do we recognize if the mouse is placed over the correct position or not. For example lets say we have a dialog box where we want to click on some button. I tried using the old coordinates of the dialog button (which I obtained by taking the snapshot of the entire window). Now the problem is the coordinates may change if we chane the resolution or someone decides to move the window so the operation may execute in some other position and not what I intend to do.

What I am asking for is an alternative to it. Like make the script smarter by image or text recognition and if so how.

Share this post


Link to post
Share on other sites



Depends on what the "some window" is. Look at the AutoIt Window Info Tool (in the same directory where you installed AutoIt, and hover over the button you want to press. If the tool returns information about the button, then you can use ControlClick to click on it without having to resort to the very unreliable MouseClick. In general, however, you need to provide more information about what you're trying to accomplish, in what application, and what you have tried on your own, in order to receive help on this forum.

Also, I have moved this question, along with your other one, to the proper forum. The Developers forum states (bolded and hyperlinked, no less):

Quote

Do not create AutoIt-related topics here, use AutoIt General Help and Support

Please pay attention to where you post in the future.


When you're dead, you don't know you're dead - it's only difficult for those that know you. It's the same way when you're stupid...

My Scripts: SCCM UDFInclude Source with Compiled Script, Windows Firewall UDF

Share this post


Link to post
Share on other sites

Sorry that I forgot to mention its actually a flash file in a web page.

Share this post


Link to post
Share on other sites

Flash is going to be very difficult with AutoIt. Your best bet will probably be the IUIAutomation thread in the Examples forum. But if you post a screenshot of the flash file you're dealing with, someone may have done something similar already.


When you're dead, you don't know you're dead - it's only difficult for those that know you. It's the same way when you're stupid...

My Scripts: SCCM UDFInclude Source with Compiled Script, Windows Firewall UDF

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

Okay I did some searching and here is the code I made. But unfortunately, its not working at all.

#include <MsgBoxConstants.au3>
#include <GDIPlus.au3>
#include <ScreenCapture.au3>
#include <GDIPlus.au3>

$x=20
$y=20
$k=0
$l=0

WinWaitActive("[CLASS:Chrome_WidgetWin_1]")
_GDIPlus_Startup()
While $l==0         ;>= search using screenshots
    _ScreenCapture_Capture("D:\shot.jpg", $x, $y, $x+23, $y+23)
    $x+=1;
    If $x>1000 Then
        $x=20;
        $y+=1;
    EndIf
    $fname1="D:\bttn.jpg"
    $fname2="D:\shot.jpg"
    $bm1 = _GDIPlus_ImageLoadFromFile($fname1)
    $bm2 = _GDIPlus_ImageLoadFromFile($fname2)
    If CompareBitmaps($bm1, $bm2) Then
        $k=1
        $l=1
    EndIf
    If $y>1000 Then
        $l=1
    EndIf
WEnd
_GDIPlus_ImageDispose($bm1)
_GDIPlus_ImageDispose($bm2)
_GDIPlus_Shutdown()

If $k==1 Then       ;click in the position found
    MouseMove($x, $y)
    MouseClick($MOUSE_CLICK_PRIMARY)
Else
MsgBox(0, "Unfortunate", "Button Not found", 5000)
EndIf


Func CompareBitmaps($bm1, $bm2)

    $Bm1W = _GDIPlus_ImageGetWidth($bm1)
    $Bm1H = _GDIPlus_ImageGetHeight($bm1)
    $BitmapData1 = _GDIPlus_BitmapLockBits($bm1, 0, 0, $Bm1W, $Bm1H, $GDIP_ILMREAD, $GDIP_PXF32RGB)
    $Stride = DllStructGetData($BitmapData1, "Stride")
    $Scan0 = DllStructGetData($BitmapData1, "Scan0")

    $ptr1 = $Scan0
    $size1 = ($Bm1H - 1) * $Stride + ($Bm1W - 1) * 4


    $Bm2W = _GDIPlus_ImageGetWidth($bm2)
    $Bm2H = _GDIPlus_ImageGetHeight($bm2)
    $BitmapData2 = _GDIPlus_BitmapLockBits($bm2, 0, 0, $Bm2W, $Bm2H, $GDIP_ILMREAD, $GDIP_PXF32RGB)
    $Stride = DllStructGetData($BitmapData2, "Stride")
    $Scan0 = DllStructGetData($BitmapData2, "Scan0")

    $ptr2 = $Scan0
    $size2 = ($Bm2H - 1) * $Stride + ($Bm2W - 1) * 4

    $smallest = $size1
    If $size2 < $smallest Then $smallest = $size2
    $call = DllCall("msvcrt.dll", "int:cdecl", "memcmp", "ptr", $ptr1, "ptr", $ptr2, "int", $smallest)

    _GDIPlus_BitmapUnlockBits($bm1, $BitmapData1)
    _GDIPlus_BitmapUnlockBits($bm2, $BitmapData2)

    Return ($call[0]=0)


EndFunc  ;==>CompareBitmaps

I am not getting whats wrong.

Edited by holebergers

Share this post


Link to post
Share on other sites

 

 

1 hour ago, holebergers said:

I am not getting whats wrong.

Maybe the resolution from your screenshot and the images are different? Attach the screenshot and the images and i will try to find what's wrong.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now