Pixel color to trigger a keystroke?

[Vicate]

This is my first post, so I'll take the opportunity to introduce myself.

Hello.

I am trying to create something and upon searching for programs, I came across this nifty application. It appears to be just what I am looking for, given proper instruction.

Basically, I'd like to create a script to where if a color other than a specified few appear within a box, type a keystroke according to that color's position on the x axis.

For example, in this box are colors #000000, #ffccff, and #999999.

These three would be the accepted colors.

In this box, however, there is another color, #cc0000 appears where x=56 and x=68.

This color exception would warrant two keystrokes, one for x=56 and one for x=68.

Let's have it enter, say, "h" when x=56 and "i" when x=68.

If that image flashed three times, ideally it would type "hihihi", "ihihih", or any other combination of the two keystrokes.

If that is not possible, I'll settle for help toward creating a script that enters a certain keystroke in the presence of a specific color at coordinate xy

Example:

If color #cccc33 appears at coordinates (487, 793), type keystroke "j"

If color #660000 appears at coordinates (563, 772), type keystroke "m"

etc.

It may sound like a dead-end script with no purpose, but it does serve a very practical use for me. If I have not made anything clear, feel free to ask.

I am not very knowledgeable in scripting, but I can learn fast.

I have looked at the tutorial and downloaded/read various example scripts to try and get a grip on it. I have a feeling it has something to do with commands "PixelGetColor" and "PixelSearch", but I cannot seem to compile a functional script.

Thanks in advance,

-Vicate

[stampy]

pixelgetcolor will allow you to check the pixel for color then the send() command would allow the keystroke. Do you have any code yet?

[Vicate]

Something like...

PixelGetColor, 0x6FBF00, 187, 137

Send {q}

...?

As I said, I'm not at all familiar with how to even start this, but I am more than willing to learn this; actually, I'd prefer to do this myself. I just cannot seem to find a tutorial that fits, and it doesn't seem to help by looking at some examples. Should I start with a GUI, etc? Any help would be appreciated.

That whole "AutoIt 1-2-3" seems to come up with an error...

"Line 18 (File "C:\....\Lesson-1.au3"):

If, And, Or, Then, Next, While, Wend.

If, and, Or, Then, Next, While, Wend^ ERROR

Error: Missing separator character after keyword"

Are there any resources or tutorials that would help me to learn where to start?

[Vicate]

Change of plans, folks. I've thought up an easier way to get my results and I just need to know how to do what I asked for second, which is:

"Creating a script that enters a certain keystroke in the presence of a specific color at coordinate xy

Example:

If color #cccc33 appears at coordinates (487, 793), type keystroke "j"

If color #660000 appears at coordinates (563, 772), type keystroke "m"

etc."

Also keep in mind that I want it to be an "if, then" script that looks for the color to appear at the coordinate and sends a single keystroke accordingly.

Thanks again.

[SaVoR]

Change of plans, folks. I've thought up an easier way to get my results and I just need to know how to do what I asked for second, which is:

"Creating a script that enters a certain keystroke in the presence of a specific color at coordinate xy

Example:

If color #cccc33 appears at coordinates (487, 793), type keystroke "j"

If color #660000 appears at coordinates (563, 772), type keystroke "m"

etc."

Also keep in mind that I want it to be an "if, then" script that looks for the color to appear at the coordinate and sends a single keystroke accordingly.

Thanks again.

while 1

$j = PixelSearch( 487, 487, 793, 793, 0xcccc33 )
$m = PixelSearch( 563, 563, 772, 772, 0x660000 )
    if $j = true then
        send("j")
    endif

       if $m = true then
                send("m")
       endif

wend

This will send keys to your screen if colors are at the coordinates specified. It will do this continuously since its in a loop.

PixelSearch has more functions. Take a look at the helpfile with more information on it.

Edited June 3, 2009 by SaVoR

[Vicate]

Thanks. It looks like you know what I'm going for, but that didn't seem to work.

While 1
    $z = PixelSearch( 1046, 1046, 59, 59, 0x000000 )
        if $z = true then
            send("z")
        endif
WEnd

That just seemed to continuously hold down the Z key.

Here's what I'm trying to do. I should've just included this in the first place

Here's a snapshot of a piano from a midi program that I'm using, and the keys become black whenever a key is pressed.

Ideally, I would like each piano key to correspond to a keystroke, and to send the keystroke whenever an xy contained within the piano key turns black(is pressed).

To use this image as an example, three notes are being hit currently.

Let's say one note here triggers the keystroke "e" to be hit, one triggers "f", and one triggers "g".

Because it is a chord, all three here are hit at once, and I would like it to hit E, F, and G at the same time, then release them when the keys are unpressed(turn white).

After some fiddling, I think I have just what I'm looking for:

While 1
    $n = PixelGetColor(386, 45)
        Switch $n
        Case 0x00FFFF
                send("{n down}")
        Case Else
                send("{n up}")
            EndSwitch
    $g = PixelGetColor(380, 43)
        Switch $g
        Case 0x00FFFF
                send("{g down}")
        Case Else
                send("{g up}")
        EndSwitch
WEnd

Two things, though.

1)Though this script works, it's VERY sluggish. Is there a more efficient code that produces similar results?

Edit: Actually no, it isn't sluggish at all, it rather seems to be TOO active. example: When a key is held down, the script sees the coordinate repetitively being the color specified and that warrants a flurry of the same command when it should only be executed once. Anybody know a way of how to "desensitize" this script?

Where if the key is pressed, it waits until the key is unpressed, then repressed before issuing another keystroke?

2)I am incorporating a GUI with it, and would like to have a button to toggle this on/off.

How would I go about doing that?

Thanks a bunch.

Edited June 4, 2009 by Vicate

[SaVoR]

Then what you are going to need to do is create functions and put them in a loop.

expandcollapse popup

Func _N()
    $n = PixelGetColor(386, 45)
        $nd = 0
        Select
        Case $n = 0x00FFFF and $nd = 0
                send("{n down}")
                               $nd = 1
        Case $n <> 0x00FFFF and $nd = 1
                send("{n up}")
        EndSwitch
endfunc

Func _G()
    $g = PixelGetColor(380, 43)
        $gd = 0
        Select
        Case $g = 0x00FFFF and $gd = 0
                send("{g down}")
                                $gd = 1
        Case $g <> 0x00FFFF and $gd = 1
                send("{g up}")
        EndSwitch
endfunc

; For both keys at once 
* If you want to do it this way. You seem to have it work the way you like with how fast it hits the keys. *

Func _NG
    $n = PixelGetColor(386, 45)
    $g = PixelGetColor(380, 43)
        $ngd = 0
        Select
        Case $n = 0x00FFFF and $g = 0x00FFFF and $ngd = 0
                send("{g down}" & "{n down}")
                                $ngd = 1
        Case $n <> 0x00FFFF and $g <> 0x00FFFF and $ngd = 1
                send("{g up}" & "{n up}")
        EndSwitch

endfunc

While 1

    _NG()
    _G()
    _N()

Wend

If you make chords, you want to put the chords first in the list, going from biggest chords to smallest chords. This will make them hit at the same time to make a chord sound. If you just make all single notes, then it will go through the loop in order of how you have it listed, and hit in that order, it also will run slower because you have them separated. That is quite a bit of coding, but if you want it to hit as a chord, its needed

Let me know how it goes!

EDIT: I didn't see you had a problem with that one. I will re-edit this to make it work so it only hits once.

EDIT2: There, fixed it. With the GUI, you are gonna have to read up on that one. It would be better if you understood the way a GUI works instead of me throwing code at you Gimme some examples of what you have come up with and I will let you know whats up.

Edited June 4, 2009 by SaVoR

[Vicate]

Thanks for your help.

Well, I'm scripting it to follow the keyboard as it hits the notes in realtime.

I have a feeling you're under the impression that I'm scripting it to play a song, with each note in the script.

Hope this explains it:

[SaVoR]

Thanks for your help.

Well, I'm scripting it to follow the keyboard as it hits the notes in realtime.

I have a feeling you're under the impression that I'm scripting it to play a song, with each note in the script.

Hope this explains it:

So you are having it so it saves the keys pressed in a file so you can look back at it later?

Or are you having it play as the keyboard plays?

The code above will play as the keyboard plays on the screen. If you want to log what keys are pressed while being played so you can read it for later use, it requires a little bit of change in the code.

[Vicate]

No, I just want it to play as the keyboard plays.

Thing is, I want it to be able to play pretty much any midi file.

I can follow the whole code, except for the d's that follow some of it

"$n = PixelGetColor(386, 45)

$nd = 0"

Wait, is that like a digit to where 0 is false and 1 is true?

If that's the case, I think I know how to get the GUI buttons working.

I have a GUI set up at my home computer, I'll post it when I'm home.

I'll fiddle with it when I get home tonight

Edited June 4, 2009 by Vicate

[SaVoR]

No, I just want it to play as the keyboard plays.

Thing is, I want it to be able to play pretty much any midi file.

I can follow the whole code, except for the d's that follow some of it

"$n = PixelGetColor(386, 45)

$nd = 0"

Wait, is that like a digit to where 0 is false and 1 is true?

If that's the case, I think I know how to get the GUI buttons working.

I have a GUI set up at my home computer, I'll post it when I'm home.

I'll fiddle with it when I get home tonight

$nd is the variable that keeps track whether that key is pressed down or not. when it =1, it wont keep pressing the key constantly, it will only keep it down until the color changes.

You dont need to use the chord portion then, just have it do each note individually and you should see the results you want

[Vicate]

That code you sent me has a Select command corresponding to a Switch command, or atleast I think; correct me if I'm wrong.

I tried both Switch...EndSwitch and Select...EndSelect; neither seems to work.

[SaVoR]

That code you sent me has a Select command corresponding to a Switch command, or atleast I think; correct me if I'm wrong.

I tried both Switch...EndSwitch and Select...EndSelect; neither seems to work.

Select

* Code *

EndSelect

That should be right.. sorry i must've missed that one.

What error does it give you?

[Vicate]

EDIT: WAIT, God, I'm so stupid.

I didn't even see that it was looking for 0x00FFFF and my color was 0x000000.

Sometimes a little sleep serves us well.

I'm so sorry, pal

It works fine.

Edit2:Actually wait, it doesn't seem to release the keys for some reason...

Running it in notepad, the following is the result of one chord being held down for a split second:

5y5y5y5y5y5y

It looks like it keeps hitting down the two notes until the key is released.

I used a keystroke recorder with it and here's what's happening on the same chord:

Keyboard : Y : KeyDown

Keyboard : D5 : KeyDown

DELAY : 12

Keyboard : Y : KeyDown

Keyboard : D5 : KeyDown

Keyboard : Y : KeyDown

DELAY : 12

Keyboard : D5 : KeyDown

Keyboard : Y : KeyDown

Keyboard : D5 : KeyDown

DELAY : 12

Keyboard : Y : KeyDown

Keyboard : D5 : KeyDown

DELAY : 12

Keyboard : Y : KeyDown

Keyboard : D5 : KeyDown

Keyboard : Y : KeyDown

DELAY : 12

Keyboard : D5 : KeyDown

Keyboard : Y : KeyDown

Keyboard : D5 : KeyDown

DELAY : 12

Keyboard : Y : KeyDown

Keyboard : D5 : KeyDown

DELAY : 12

Keyboard : Y : KeyDown

Keyboard : D5 : KeyDown

There doesn't seem to be a KeyUp initiated.

Upon messing around with it, If you remove the second case, and just have it like:

Func _5()
    $5 = PixelGetColor(363, 40)
        Select
        Case $5 = 0x000000
                send("{5 down}")
        EndSelect
    endfunc

it produces the same exact results.

I don't think that second case is even having an impact on the code.

I don't know, I'll keep messing with it.

On a good note, it's spot-on with the timing.

This here's the GUI script:

expandcollapse popup

#include <ButtonConstants.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>
#Region ### START Koda GUI section ### Form=C:\Documents and Settings\Owner\Desktop\GraalPlayer\GIUTest.kxf
$graalplay = GUICreate("Graal Player", 116, 200, 504, 238)
GUISetFont(8, 400, 0, "Arial")
GUISetBkColor(0xFFFFFF)
$Player = GUICtrlCreateGroup("Player", 8, 136, 97, 41)
$On = GUICtrlCreateButton("On", 12, 152, 41, 17, $WS_GROUP)
GUICtrlSetColor(-1, 0xC0C0C0)
GUICtrlSetBkColor(-1, 0x000000)
$Off = GUICtrlCreateButton("Off", 60, 152, 41, 17, $WS_GROUP)
GUICtrlSetColor(-1, 0xC0C0C0)
GUICtrlSetBkColor(-1, 0x000000)
GUICtrlCreateGroup("", -99, -99, 1, 1)
$Pic1 = GUICtrlCreatePic("C:\Documents and Settings\Owner\Desktop\GraalPlayer\image1.bmp", 8, 0, 98, 129, BitOR($SS_NOTIFY,$WS_GROUP,$WS_CLIPSIBLINGS))
$Info = GUICtrlCreateButton("Info", 36, 178, 41, 17, $WS_GROUP)
GUICtrlSetColor(-1, 0xC0C0C0)
GUICtrlSetBkColor(-1, 0x000000)
GUISetState(@SW_SHOW)
#EndRegion ### END Koda GUI section ###

While 1
    $nMsg = GUIGetMsg()
    Switch $nMsg
        Case $GUI_EVENT_CLOSE
            Exit
        Case $On
            I know the information would go here to turn them on, I'm guessing something like
            While 2
                _5()
                _6()
                _Y()
                _U()
                etc.
                if case $Off then
                    ExitLoop
                endif
            WEnd
        Case $Off
            ^^^ Up there, is that how to turn it off? ^^^
        Case $Info
            This'll pull up an "about" window including credits and instructions
    EndSwitch
WEnd

EDIT: Also, I see that animated gif images can't be used, so scratch the whole thing with getting it to show the image.

Here's what I'm trying to do, when the on button is pressed, turn it from black to orange to indicate that it's been pressed

When the off button is pressed, turn the off button orange and turn the on button black, to indicate that it is off.

I'm terribly sorry if this caused any confusion!

Thank you for your time.

Edited June 5, 2009 by Vicate

[Vicate]

Alright, it seems this here works fairly well:

Func _5()
    $5 = PixelGetColor(363, 40)
        Switch $5
            Case 0x000000;Color it's looking for when the key is pressed
                send("{5 down}")
            Case 0xADAAAD;Color it's looking for when the key is released
                send("{5 up}")
        EndSwitch
    endfunc

Let me assign all the keys and come back with some results.

Edit: Alright, I have a new problem:

Everything seems to run alright, except it only hits about 80% of the notes, and when the piano plays "fast runs", it doesn't play many of the notes.

It almost seems as though it isn't sensitive enough, now.

My theory:

...
            Case 0xADAAAD;Color it's looking for when the key is released
                send("{5 up}")
...

95% of the notes are constantly released, so it keeps sending the keyup commands nonstop.

Could this possibly be taking priority and "distracting" the script?

I mean, if it is constantly sending keyups for every piano key that isn't pressed(that's a lot of keys!), I'd say that most of the activity are these unnecessary keyup commands that really mean nothing until the key is pressed.

I just need a way to make it check to see if the key is black.

Going off from your script, SaVoR, with the $d command, would this work?

Func _s()
    $s = PixelGetColor(273, 40)
            $sd = 0
        Switch $s
        Case 0x000000 and $sd = 0;Color it's looking for when the key is pressed
                send("{s down}")
                    $sd = 1
        Case 0xAAAAAA and $sd = 1;Color it's looking for when the key is released
                send("{s up}")
        EndSwitch
endfunc

Edit: The above code works and hits every note spot-on!

Thing is, again, it doesn't send the keyups.

Could someone thoroughly explain the $sd = 0 and $sd = 1 for me? I cannot seem to find anything in regards to this, but apparently it sets conditions which I need to do.

It appears to be setting $sd = 0, as it hits the key, but because it isn't initiating a keydown, I have a feeling it isn't properly setting $sd = 1.

Do I have it written correctly?

I'm still not too sure about applying it to the GUI.

Some help with that too would be appreciated.