Position Recognition based on Text or image

[holebergers]

Hi guys, I am new to autoit and all and I am stuck in a problem. How do we recognize if the mouse is placed over the correct position or not. For example lets say we have a dialog box where we want to click on some button. I tried using the old coordinates of the dialog button (which I obtained by taking the snapshot of the entire window). Now the problem is the coordinates may change if we chane the resolution or someone decides to move the window so the operation may execute in some other position and not what I intend to do.

What I am asking for is an alternative to it. Like make the script smarter by image or text recognition and if so how.

[JLogan3o13]

Depends on what the "some window" is. Look at the AutoIt Window Info Tool (in the same directory where you installed AutoIt, and hover over the button you want to press. If the tool returns information about the button, then you can use ControlClick to click on it without having to resort to the very unreliable MouseClick. In general, however, you need to provide more information about what you're trying to accomplish, in what application, and what you have tried on your own, in order to receive help on this forum.

Also, I have moved this question, along with your other one, to the proper forum. The Developers forum states (bolded and hyperlinked, no less):

Quote

Do not create AutoIt-related topics here, use AutoIt General Help and Support

Please pay attention to where you post in the future.

[holebergers]

Sorry that I forgot to mention its actually a flash file in a web page.

[JLogan3o13]

Flash is going to be very difficult with AutoIt. Your best bet will probably be the IUIAutomation thread in the Examples forum. But if you post a screenshot of the flash file you're dealing with, someone may have done something similar already.

[holebergers]

Okay I did some searching and here is the code I made. But unfortunately, its not working at all.

expandcollapse popup

#include <MsgBoxConstants.au3>
#include <GDIPlus.au3>
#include <ScreenCapture.au3>
#include <GDIPlus.au3>

$x=20
$y=20
$k=0
$l=0

WinWaitActive("[CLASS:Chrome_WidgetWin_1]")
_GDIPlus_Startup()
While $l==0         ;>= search using screenshots
    _ScreenCapture_Capture("D:\shot.jpg", $x, $y, $x+23, $y+23)
    $x+=1;
    If $x>1000 Then
        $x=20;
        $y+=1;
    EndIf
    $fname1="D:\bttn.jpg"
    $fname2="D:\shot.jpg"
    $bm1 = _GDIPlus_ImageLoadFromFile($fname1)
    $bm2 = _GDIPlus_ImageLoadFromFile($fname2)
    If CompareBitmaps($bm1, $bm2) Then
        $k=1
        $l=1
    EndIf
    If $y>1000 Then
        $l=1
    EndIf
WEnd
_GDIPlus_ImageDispose($bm1)
_GDIPlus_ImageDispose($bm2)
_GDIPlus_Shutdown()

If $k==1 Then       ;click in the position found
    MouseMove($x, $y)
    MouseClick($MOUSE_CLICK_PRIMARY)
Else
MsgBox(0, "Unfortunate", "Button Not found", 5000)
EndIf


Func CompareBitmaps($bm1, $bm2)

    $Bm1W = _GDIPlus_ImageGetWidth($bm1)
    $Bm1H = _GDIPlus_ImageGetHeight($bm1)
    $BitmapData1 = _GDIPlus_BitmapLockBits($bm1, 0, 0, $Bm1W, $Bm1H, $GDIP_ILMREAD, $GDIP_PXF32RGB)
    $Stride = DllStructGetData($BitmapData1, "Stride")
    $Scan0 = DllStructGetData($BitmapData1, "Scan0")

    $ptr1 = $Scan0
    $size1 = ($Bm1H - 1) * $Stride + ($Bm1W - 1) * 4


    $Bm2W = _GDIPlus_ImageGetWidth($bm2)
    $Bm2H = _GDIPlus_ImageGetHeight($bm2)
    $BitmapData2 = _GDIPlus_BitmapLockBits($bm2, 0, 0, $Bm2W, $Bm2H, $GDIP_ILMREAD, $GDIP_PXF32RGB)
    $Stride = DllStructGetData($BitmapData2, "Stride")
    $Scan0 = DllStructGetData($BitmapData2, "Scan0")

    $ptr2 = $Scan0
    $size2 = ($Bm2H - 1) * $Stride + ($Bm2W - 1) * 4

    $smallest = $size1
    If $size2 < $smallest Then $smallest = $size2
    $call = DllCall("msvcrt.dll", "int:cdecl", "memcmp", "ptr", $ptr1, "ptr", $ptr2, "int", $smallest)

    _GDIPlus_BitmapUnlockBits($bm1, $BitmapData1)
    _GDIPlus_BitmapUnlockBits($bm2, $BitmapData2)

    Return ($call[0]=0)


EndFunc  ;==>CompareBitmaps

I am not getting whats wrong.

Edited September 13, 2016 by holebergers

[AutoBert]

 

 

1 hour ago, holebergers said:

I am not getting whats wrong.

Maybe the resolution from your screenshot and the images are different? Attach the screenshot and the images and i will try to find what's wrong.