Moving the mouse pointer to a dynamic area within an application

[emog]

I'd like to be able to automatically move a mouse pointer to the point shown in the small red circle in the following screengrab. (The application in this case is Jasc Paint Shop Pro, a graphics/paint program):-

http://img250.imageshack.us/img250/8868/screenshotfg1.png

Unfortunately I can't use pre-defined co-ordinates, because the code has to be relatively dynamic enough for other Paint Shop Pro owners to use it on different computers/setups and with the main application window semi-minimised in different places on the desktop, etc..

Using the AutoIt Window Info tool I've found that absolutely everything inside the green dashed area has the same Control ID ("59648" in this case), and this includes both the graphic in the middle and all the empty space around it. However, placing the mouse pointer on the graphic in the middle displays local co-ordinates in the "Status Bar Text (1):" entry in the AutoIt Window Info tool (starting from 0,0 at the top left pixel of the graphic).

So, how would I be able to get my mouse pointer to move to (0,0) of the graphic?

Thanks for any hints.

[Shevilie]

Get the size and posistion of the object.. then center the mouse ... then there should be some graphic ther (as i guess the picture is always centered) Then you should be able to read the status bar and then move tha mouse after that

[herewasplato]

...with the main application window semi-minimised...

I do not understand that requirement... but here is some code for a hint

For $y = 100 To 300 Step 100
    For $x = 100 To 300
        MouseMove($x, $y, 0)
        If StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), "(0,") > 0 Then ExitLoop 2
    Next
Next

Use MouseCoordMode 2 if you do not want to move the window of interest to 0,0 on the desktop. The "scrape" is very course so it should not take long. The loop exits when it finds the left edge (at least I'm assuming that the software shows coords anywhere on the object).

Once the course scrape has found the left edge, "scrape - up" to 0,0.

-MSP-

[emog]

Thank you. I've attempted to do what you suggested using a coarse scrape, but am a bit stuck on scraping it slowly back up to (0,0) of the graphic editing area (in the Status Bar Text) afterwards. Here's my code (which doesn't work, it stalls ad infinitum, requiring forceably shutting down the routine via Windows Task Manager), can anyone possibly tell me what I'm doing wrong, please?:

Winactivate("Jasc Paint Shop Pro")
WinWaitActive("Jasc Paint Shop Pro")
Opt("MouseCoordMode", 2) 
For $y = 100 To 600 Step 50
    For $x = 100 To 600 Step 50
        MouseMove($x, $y, 0)
            If StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), "(") > 0 Then Exitloop 2
    Next
Next
Do
    If StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), "(0,") <> 0 
    Then $x = $x -1
    MouseMove($x, $y, 0)
Until StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), "(0,")
Do
    If StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), ",0)") <> 0 Then $y = $y -1
    MouseMove($x, $y, 0)
Until StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), ",0)")

Thanks

Edited March 8, 2007 by emog

[lod3n]

Why are you doing this? Just wondering.

[emog]

Basically, I would like to use Paint Shop Pro as an advanced graphics program, and then export the final graphics to another, very simple "pixel-art" based paint program, effectively drawing the graphic in, pixel by pixel. The destination paint program doesn't currently support the usual copy/paste, or importing of image files, so it has to be drawn in.

The AutoIt routine would be used in this sense as an intelligent macro.I already have the colour pallette conversion and the exporting into the destination aspects roughly drafted out, so it's just the initial scanning from the source part (Paint Shop Pro) that I need to do. Once I find how to get to (0,0) in Paint Shop Pro I think i'm pretty much sorted. (This would be followed by getting the pixel colour, converting it to the destination pallette, selecting that colour from the destination pallette, then "painting" that colour into (0,0) of the destination... then looped for importing the rest of the pixels/colours. The full export process will probably take about 20-30minutes to do. I was hoping it would be a lot quicker, but that's obviously the time it needs to carry out these sorts of mouse commands).

[herewasplato]

...it stalls ad infinitum, requiring forceably shutting down the routine via Windows Task Manager...

Use the ESC key to terminate this code if it also "stalls"

Opt("TrayIconDebug", 1)  ;0-off
Opt("MouseCoordMode", 2)

WinActivate("Jasc Paint Shop Pro")
WinWaitActive("Jasc Paint Shop Pro")

HotKeySet("{ESC}", "Terminate")

For $y = 100 To 600 Step 50
    For $x = 100 To 600 Step 50
        MouseMove($x, $y, 0)
        If StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), "(") > 0 Then ExitLoop 2
    Next
Next

MsgBox(0, "", "MsgBox1")

While Not StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), "(0,")
    $x = $x - 1
    MouseMove($x, $y, 0)
WEnd

MsgBox(0, "", "MsgBox2")

While Not StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), ",0)")
    $y = $y - 1
    MouseMove($x, $y, 0)
WEnd

Func Terminate()
    Exit
EndFunc   ;==>Terminate

Edit: typo

Edited March 8, 2007 by herewasplato

[lod3n]

Why not export it as a BMP, then open the BMP in MSpaint. Mspaint puts 0,0 at the same point every time. Further, there are some scripts here in the forum that can tell you the width and height of an image file - try searching for DSOfile. Then you can scan for color values using those x and y values and PixelGetColor.

Also, on your target app, if the mouse seems slow, make sure you speed up the mouse movements with the mouse options in AutoSetOption.

[emog]

Many thanks indeed for your help, the mouse move to (0, 0) works perfectly!

However, I'm having a problem with the PixelGetColor command following this. It's picking up a different colour to the one shown in the AutoIt Window Info tool as well as on screen directly under the mouse pointer.

As an example, I captured a screengrab which should demonstrate what I mean. I painted a simple solid yellow box in Paint Shop Pro to test the PixelGetColor routine, at first. The small red circle on the top left of the yellow graphic is where the mouse pointer was. (Screengrabbing the image using the PrtScn button makes the mouse pointer invisible, so that's why it's not there.) The yellow graphic has a hex value of #FFFF00 (as confirmed in the AutoIt Window Info Tool on the right, but the pixel colour retrieved by the PixelGetColor command (the little popup in the middle of the screen) shows it as #000000 (black).

http://img147.imageshack.us/img147/5948/pixelcolourcf4.png

I've also tried different co-ordinates in case it was picking up the immediate edge of the graphic and not the graphic itself, but the pixel colour retrieved is still always wrong (giving varied hex values, mainly: 000000, ffffff, c0c0c0, eaead8, or other values - however I do know that c0c0c0 is the same colour as used in the diagonal grey grid in the space surrounding the graphic).

But the PixelGetColor wont pick up the yellow graphic at all.

Here's my code:

expandcollapse popup

Opt("TrayIconDebug", 1);0-off
Opt("MouseCoordMode", 2)

;~~~~~~~~~~ Paint Shop Pro calibration ~~~~~~~~

WinActivate("Jasc Paint Shop Pro")
WinWaitActive("Jasc Paint Shop Pro")

HotKeySet("{ESC}", "Terminate")

For $Y = 100 To 600 Step 100
    For $X = 100 To 600 Step 100
        MouseMove($X, $Y, 0)
        If StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), "(") > 0 Then ExitLoop 2
    Next
Next

While Not StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), "(0,")
    $X = $X - 1
    MouseMove($X, $Y, 0)
WEnd

While Not StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), "(0, 0)")
    $Y = $Y - 1
    MouseMove($X, $Y, 0)
WEnd

; The following mousemove test code was to determine if (0,0) was too 
; close to the edge of the graphic, giving an erroneous colour readout.
; For this test, co-ordinates (1, 1) are used instead for testing, to make sure.
;   $X = $X + 1
;   $Y = $Y + 1
;   MouseMove($X, $Y, 0)

$PSPMousePos = MouseGetPos()
MsgBox(0, "PSP Mouse X,Y:", "X = " & $PSPMousePos[0] & " , Y = " & $PSPMousePos[1])

$PixelColour = PixelGetColor($PSPMousePos[0], $PSPMousePos[1])
MsgBox(0,"The hex color is", Hex($PixelColour, 6))

;~~~~~~~ END Paint Shop Pro calibration ~~~~~~~~

Func Terminate()
    Exit
EndFunc;==>Terminate

Any hints?

Thanks indeed.

Edited March 9, 2007 by emog

[emog]

Simple answer, the mouse was being moved relative to a client, where the pixels were relative to the whole desktop.

Opt("PixelCoordMode", 2) in the global attributes sorted it.

Also I found the "Arrow" tool should be always selected in Paint Shop Pro before retrieving the colour, as some of the other paint tools seemingly override the graphic and it retrieves the colour of the paint tool mouse pointer instead.

Added a simple routine to select the arrow pointing tool prior to scraping to avoid this.

Thanks

[seandisanti]

hm... maybe I'm misunderstanding, but why couldn't you just....

$color = PixelSearch($left,$top,$right,$bottom);your color here
MouseMove($color[0],$color[1],0)

[emog]

Hi. The colour and position of the graphic would've been different every time I wanted to scan one. Also this way it wont matter what screen resolution people are using, or whether their window is semi-minimised (non full screen) or whatever, instead it just calibrates it accordingly prior to scanning. I would've used MSPaint but that app doesn't allow editing of layers/transparencies, and at least this way it saves you having to export from PSP to MSPaint beforehand every single time.

Cheers.

Must say I love this forum, though, so many helpful people!

[seandisanti]

Hi. The colour and position of the graphic would've been different every time I wanted to scan one. Also this way it wont matter what screen resolution people are using, or whether their window is semi-minimised (non full screen) or whatever, instead it just calibrates it accordingly prior to scanning. I would've used MSPaint but that app doesn't allow editing of layers/transparencies, and at least this way it saves you having to export from PSP to MSPaint beforehand every single time.

Cheers.

Must say I love this forum, though, so many helpful people!

it still seems like you could just do something like....

$pos = WinGetPos("The window title")
$search = PixelSearch($pos[0],$pos[1],$pos[0] + $pos[2],$pos[1] + $pos[3],0xFFFF00)
MouseMove($search[0],$search[1],0)

[herewasplato]

...It's picking up a different colour to the one shown in the AutoIt Window Info tool as well as on screen directly under the mouse pointer...

I have no idea why it would do that. Run this code:

Opt("TrayIconDebug", 1)  ;0-off
Opt("MouseCoordMode", 2)

WinActivate("Jasc Paint Shop Pro")
WinWaitActive("Jasc Paint Shop Pro")

HotKeySet("{ESC}", "Terminate")

For $y = 100 To 600 Step 50
    For $x = 100 To 600 Step 50
        MouseMove($x, $y, 0)
        If StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), "(") > 0 Then ExitLoop 2
    Next
Next

While Not StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), "(0,")
    $x = $x - 1
    MouseMove($x, $y, 0)
    ToolTip(PixelGetColor($x, $y), $x, $y)
    Sleep(100)
WEnd

While Not StringInStr(StatusbarGetText("Jasc Paint Shop Pro"), ",0)")
    $y = $y - 1
    MouseMove($x, $y, 0)
    ToolTip(PixelGetColor($x, $y), $x, $y)
    Sleep(100)
WEnd

Func Terminate()
    Exit
EndFunc   ;==>Terminate

and see it it shows you what is going on.

If not,

increase the Sleep time to about 500 in the While/WEnd loops

then open the "AutoIt Window Info" tool

select Options > Magnify

minimize the "AutoIt Window Info" tool

and run the slow version of the script again...

let us know your findings

[emog]

The way I worked it in the end was to physically change the pointer to one of the other paint tools that were utilitarian (one that didn't have the colour of "ink" at the pointer). Made a routine that would carry that out before it started scanning the image, and worked fine. Took me a while to find out what was happening.

Thanks for all your help.

Edited March 11, 2007 by emog

[herewasplato]

...Must say I love this forum, though, so many helpful people!..

I concur.

Hang around and you can be one of them :-)

Glad that you got it to work for you.