Monitoring colours - PixelSearch and PixelCheckSum - (Moved)

[Dave1]

Hello everyone,

I have several signals on the screen that need to be monitored while a program is running. These signals are scattered around an image which is maximized on 4  equal resolution screens - the AutoIT Window Info does not recognize any objects on the program meaning that I'm left with the PixelSearch() and PixelCheckSum() functions to monitor the signals:

The signals are as small as a 5x5 pixel area and their coordinates are known.

I've been digging around for a while now in this forum about the PixelSearch() and PixelCheckSum()  and found some interesting and useful ideas for the use of them. I also came across some other UDF functions like MultiMon(), FastFind(), TtColXY() and the ImageSearch2015 scripts that might be useful for the final output. I don't know if hovering the mouse by using TtColXY() and output its ToolTip() information onto the log file will be faster than using PixelSearch() and/or PixelCheckSum() in a loop for all signals' coordinates.

The colours of the signals are below:

red - 0x00FF00 (opaque red)

green - 0x00FF00 (opaque green)

yellow - 0xFFFF00 (opaque yellow)

black - 0x000000 (opaque black, default colour)

EDIT:

The desired output is to monitor and record/log the changes and status of each inside a .txt file or a .csv with the below format: 

Local Machine Time        Signal,      Change,               delta-t

14:32:07                  Signal1     Green - Yellow         DELTA-t1
14:34:02                  Signal1     Yellow - Red           DELTA-t1
14:35:14                  Signal2     Yellow - Red           DELTA-t2

...

Below is the code I came up with. 

expandcollapse popup

#include <AutoItConstants.au3>
#include <MsgBoxConstants.au3>
#include <misc.au3>
#include <Date.au3>
#include <Array.au3>
#include <File.au3>
#include <ScreenCapture.au3>
#include <WinAPI.au3>
#include <WinAPIHObj.au3>

Global $program_name, $program_open, $Wname, $tCurrent
Global $button_xy[2] = [150, 175]
Global $iniColour, $ColourCheck, $NewCheck

Global $Red, $Green, $Blue, $Nil
Global $sFilePath = @ScriptDir & "\Signals_status.txt"
Global $SignalID[10] = ["Signal_1", "Signal_2", "Signal_3", "Signal_4", "Signal_5", "Signal_6", "Signal_7", "Signal_8", "Signal_9", "Signal_10"]

$program_name = "Signals.exe"
$Wname = "Training Task 3"

$program_open = ShellExecute($program_name, @ScriptDir)

WinWait($Wname)


$iniColour = "0x" & Hex(PixelGetColor($button_xy[0], $button_xy[1]), 6)

;~ $iniChecksum = "0x" & Hex(PixelChecksum($button_xy[0], $button_xy[0] - 5, $button_xy[1], $button_xy[1] + 5), 6)

$ColourCheck = "0x" & Hex(PixelChecksum($button_xy[0], $button_xy[0] - 5, $button_xy[1], $button_xy[1] + 5, 1, $Wname), 6)
ConsoleWrite(_ColourID($iniColour) & " // " & _ColourID($ColourCheck) & @CRLF)

;~ $ColourCheck = "0x " & Hex(PixelGetColor($button_xy[0], $button_xy[1]), 6)

$iniColour = $ColourCheck
;$iniChecksum = $currentCheck

Do

;~ While 1

    _WindowOnTop()
    _Close_Notepad()

    $NewCheck = "0x" & Hex(PixelChecksum($button_xy[0], $button_xy[0] - 5, $button_xy[1], $button_xy[1] + 5, 1, $Wname), 6)
    $tCurrent = _NowCalc()

    $LogFile = FileOpen($sFilePath, 2)
    FileWriteLine($LogFile, "Local Machine Time        " & "Signal,      " & "Change,               " & "delta-t")


;~  $ColourCheck = "0x" & Hex(PixelChecksum($button_xy[0], $button_xy[0] - 5, $button_xy[1], $button_xy[1] + 5, 1, $Wname), 6)

    If $ColourCheck <> $NewCheck Then ;If there's a colour change from the current colour

        $ColourCheck = $NewCheck
        Local $tChange, $NewCheckID, $ColourCheckID, $sLogMsg

        $tChange = _NowCalc()
        ConsoleWrite("Colour changed!" & @CRLF)

        $ColourCheckID = _ColourID($ColourCheck)

        $NewCheckID = _ColourID($NewCheck)

        $iTimeDiffh = _DateDiff('h', $tChange, $tCurrent) ; time difference in hours
        $iTimeDiffm = _DateDiff('n', $tChange, $tCurrent) ; time difference in minutes
        $iTimeDiffs = _DateDiff('s', $tChange, $tCurrent) ; time difference in  seconds

        $durationCheckSum = $iTimeDiffh & ":" & $iTimeDiffm & ":" & $iTimeDiffs ; Timestamp of the signal until PixelCheckSum

        $sLogMsg = "    " & $SignalID[0] & "          " & $ColourCheckID & " - " & $NewCheckID & "          " & $durationCheckSum

        _FileWriteLog($LogFile, $sLogMsg)

    ElseIf $ColourCheck = $NewCheck Then

        Local $Colour_check = _ColourID("0x" & Hex(PixelChecksum($button_xy[0], $button_xy[0] - 5, $button_xy[1], $button_xy[1] + 5, 1, $Wname), 6))
        ConsoleWrite($tCurrent & " " & $Colour_check & @CRLF)

    EndIf

    FileClose($sFilePath)

Until Not ProcessExists($program_name) And Not WinExists($Wname)


_IsProgramOpen()

Func _Close_Notepad()
    $notepad_open = ProcessExists("notepad.exe") ? ProcessClose("notepad.exe") : ProcessClose("notepad.exe")
    $notepad_open = WinActive("[CLASS:Notepad]") ? WinClose("[CLASS:Notepad]") : ProcessClose("notepad.exe")
EndFunc   ;==>_Close_Notepad

Func _ColourID($sColour)

    $Red = Int("0x" & StringRegExpReplace($sColour, "(..)(..)(..)(..)", "\2"))
    $Green = Int("0x" & StringRegExpReplace($sColour, "(..)(..)(..)(..)", "\3"))
    $Blue = Int("0x" & StringRegExpReplace($sColour, "(..)(..)(..)(..)", "\4"))

    If $Green > $Blue And $Red > $Blue And $Green >= 0xB0 And $Red >= 0xB0 Then
        $sCol = "Yellow"
    ElseIf $Blue > 0xE0 And $Green > 0xE0 And $Red > 0xE0 Then
        $sCol = "White"
    ElseIf $Blue > 0x50 And $Blue = $Green And $Blue = $Red Then
        $sCol = "Grey"
    ElseIf $Red > $Green And $Red > $Blue And $Red > 0x70 Then
        $sCol = "Red"
    ElseIf $Green > $Red And $Green >= $Blue And $Green > 0x70 Then
        $sCol = "Green"
    ElseIf $Blue > $Red And $Blue > $Green And $Blue > 0x70 Then
        $sCol = "Blue"
    Else
        $sCol = "Nil"
    EndIf

    Return $sCol
EndFunc   ;==>_ColourID

Func _WindowOnTop()

    WinActivate($Wname)
    WinSetOnTop($Wname, "", $WINDOWS_ONTOP)
    Opt("MouseCoordMode", 0)

EndFunc   ;==>_WindowOnTop

Func _IsProgramOpen()
    If Not ProcessExists($program_name) And Not WinExists($Wname) Then
        Break(1)
    EndIf
EndFunc   ;==>_IsProgramOpen

When the program is running, the window opens in the centre of the screen however, the PixelChecksum function is not looking at the correct area. In addition:

1 - I'm not sure about how to put the message into the log file.

2 - I tested this script but it is not recording the message into the log file.

3 - Is there other way to calculate the duration in these lines?

$iTimeDiffh = _DateDiff('h', $tChange, $tCurrent) ; time difference in hours
        $iTimeDiffm = _DateDiff('n', $tChange, $tCurrent) ; time difference in minutes
        $iTimeDiffs = _DateDiff('s', $tChange, $tCurrent) ; time difference in  seconds

        $durationCheckSum = $iTimeDiffh & ":" & $iTimeDiffm & ":" & $iTimeDiffs ; Timestamp of the signal until PixelCheckSum

        $sLogMsg = "    " & $SignalID[0] & "          " & $ColourCheckID & " - " & $NewCheckID & "          " & $durationCheckSum

        _FileWriteLog($LogFile, $sLogMsg)

Thanks in advance!

Edited March 9, 2021 by Dave1
Came up with some ideas for the code

[Jos]

 

3 minutes ago, Dave1 said:

In addition, I'm not sure on which of the topics I would have to post this so I opened a new one - please advise if this goes against the rules!

* start of standard messages*
Moved to the appropriate forum, as the Developer General Discussion forum very clearly states:

Quote

General development and scripting discussions.

---

Do not create AutoIt-related topics here, use the AutoIt General Help and Support or AutoIt Technical Discussion forums.

Moderation Team

Edited February 25, 2021 by Jos

[Dave1]

Anyone?

[Nine]

That is the problem with PixelSearch running over an application that nobody knows about.  How do you think we can KNOW if the coordinates you have entered in your script are correct ? 

If you want help, you should consider using an application we actually can run.  Break you code to the simplest form and use Paint as the target application.  Provide a jpg image of the locations and colors you are looking for.  This is the best way to proceed if you want to get useful help.

[Dave1]

You're right, I forgot to attach it - apologies for that!

 

EDIT: The attachment is a test executable - I'm trying to make the code work with this sample .exe for now. The future aim is to loop the script to monitor the signals in several locations on the screen.

Edited March 9, 2021 by Jos
rar with exe removed.

[Nine]

Sorry I will not install a EXE that I do not know.  Like I said make your code work with Paint if you want help.

[Dave1]

you don't need to install it, however if you don't feel comfortable in opening the .exe I can understand - I wouldn't trust it too.

EDIT:

The window of the program, once it opens, it's always in the same position and it doesn't change - it's in the centre of the screen. The problem that I see is that the PixelChecksum function is not looking in the correct place (centre of the button - [150, 175]) even though I wrote the UDF _WindowOnTop() with the "PixelCoordMode" set to "0".

Below is a picture of the program when it is executed. Does this help? Let me know if there's more I can provide you to help understand the problem in the above code.

 

Edited March 9, 2021 by Dave1

[Jos]

When you want help you need to post your Source here in a code box!

Jos

Edited March 9, 2021 by Jos

[Nine]

Does your computer has set Windows scaling (125% - 150%) ?  That might explain why it does not search at the right area...

[Dave1]

Hi @Jos, I've posted a code already in the first post of this thread but please find below the most recent code I have for this.

expandcollapse popup

#include <AutoItConstants.au3>
#include <MsgBoxConstants.au3>
#include <misc.au3>
#include <Date.au3>
#include <Array.au3>
#include <File.au3>
#include <ScreenCapture.au3>
#include <WinAPI.au3>
#include <WinAPIHObj.au3>

Global $program_name, $program_open, $WButton, $Wname, $tCurrent
Global $button_xy[2] = [150, 175]
Global $iniColour, $ColourCheck, $ColourCheckID, $NewCheck

Global $Red, $Green, $Blue, $Nil
Global $sFilePath = @ScriptDir & "\Signals_status.txt"
Global $SignalID[10] = ["Signal_1", "Signal_2", "Signal_3", "Signal_4", "Signal_5", "Signal_6", "Signal_7", "Signal_8", "Signal_9", "Signal_10"]


$program_name = "Training Task 3.exe"
$Wname = "Training Task 3"
$program_open = ShellExecute($program_name, @ScriptDir)
WinWait($Wname)


Do
    _WindowOnTop()
    $ColourCheck = "0x" & Hex(PixelGetColor($button_xy[0], $button_xy[1], $Wname), 6)

    ;$ColourCheck = "0x" & Hex(PixelChecksum($button_xy[0], $button_xy[0] + 5, $button_xy[1], $button_xy[1] + 5, 1, $Wname), 6)
    ;   ConsoleWrite(_ColourID($iniColour) & " // " & _ColourID($ColourCheck) & @CRLF)
    ;ConsoleWrite("  " & _ColourID($ColourCheck) & @CRLF)
    $LogFile = FileOpen($sFilePath, 2)
    FileWriteLine($LogFile, "Local Machine Time        " & "Signal,      " & "Change,               " & "delta-t")

    ;While 1


    _Close_Notepad()

    ;$NewCheck = "0x" & Hex(PixelChecksum($button_xy[0], $button_xy[0] - 5, $button_xy[1], $button_xy[1] + 5, 1, $Wname), 6)
    $NewCheck = "0x" & Hex(PixelGetColor($button_xy[0], $button_xy[1], $Wname), 6)
    $tCurrent = _NowCalc()

    ;$ColourCheck = "0x" & Hex(PixelChecksum($button_xy[0], $button_xy[0] - 5, $button_xy[1], $button_xy[1] + 5, 1, $Wname), 6)

    If $NewCheck <> $ColourCheck Then

        $ColourCheck = $NewCheck

        Local $tChange, $NewCheckID, $sLogMsg
        $tChange = _NowCalc()

        $ColourCheckID = _ColourID($ColourCheck)

        $NewCheckID = _ColourID($NewCheck)
        ConsoleWrite($tCurrent & " Colour changed to " & $NewCheckID & @CRLF)

        $iTimeDiffh = _DateDiff('h', $tChange, $tCurrent) ; time difference in hours
        $iTimeDiffm = _DateDiff('n', $tChange, $tCurrent) ; time difference in minutes
        $iTimeDiffs = _DateDiff('s', $tChange, $tCurrent) ; time difference in  seconds

        $durationCheckSum = $iTimeDiffh & ":" & $iTimeDiffm & ":" & $iTimeDiffs ; Timestamp of the signal until PixelCheckSum

        $sLogMsg = "    " & $SignalID[0] & "          " & $ColourCheckID & " - " & $NewCheckID & "          " & $durationCheckSum

        _FileWriteLog($LogFile, $sLogMsg)

    ElseIf $NewCheck = $ColourCheck Then
        ; _WindowOnTop()
        Local $Colour_check = _ColourID("0x" & Hex(PixelChecksum($button_xy[0], $button_xy[0] - 5, $button_xy[1], $button_xy[1] + 5, 1, $Wname), 6))
        ConsoleWrite($tCurrent & " Same colour! " & $Colour_check & @CRLF)

    EndIf

    FileClose($sFilePath)
    ;WEnd

Until Not ProcessExists($program_name) And Not WinExists($Wname)


Func _Close_Notepad()
    $notepad_open = ProcessExists("notepad.exe") ? ProcessClose("notepad.exe") : ProcessClose("notepad.exe")
    $notepad_open = WinActive("[CLASS:Notepad]") ? WinClose("[CLASS:Notepad]") : ProcessClose("notepad.exe")
EndFunc   ;==>_Close_Notepad

Func _ColourID($sColour)

    $Red = Int("0x" & StringRegExpReplace($sColour, "(..)(..)(..)(..)", "\2"))
    $Green = Int("0x" & StringRegExpReplace($sColour, "(..)(..)(..)(..)", "\3"))
    $Blue = Int("0x" & StringRegExpReplace($sColour, "(..)(..)(..)(..)", "\4"))

    If $Green > $Blue And $Red > $Blue And $Green >= 0xB0 And $Red >= 0xB0 Then
        $sCol = "Yellow"
    ElseIf $Blue > 0xE0 And $Green > 0xE0 And $Red > 0xE0 Then
        $sCol = "White"
    ElseIf $Blue > 0x50 And $Blue = $Green And $Blue = $Red Then
        $sCol = "Grey"
    ElseIf $Red > $Green And $Red > $Blue And $Red > 0x70 Then
        $sCol = "Red"
    ElseIf $Green > $Red And $Green >= $Blue And $Green > 0x70 Then
        $sCol = "Green"
    ElseIf $Blue > $Red And $Blue > $Green And $Blue > 0x70 Then
        $sCol = "Blue"
    Else
        $sCol = "Nil"
    EndIf

    Return $sCol
EndFunc   ;==>_ColourID

Func _WindowOnTop()

    WinSetOnTop($Wname, "", $WINDOWS_ONTOP)
    WinActivate($Wname)
    AutoItSetOption("PixelCoordMode", 0)

EndFunc   ;==>_WindowOnTop

Func _IsProgramOpen()
    If Not ProcessExists($program_name) And Not WinExists($Wname) Then
        Break(1)
    EndIf
EndFunc   ;==>_IsProgramOpen

I wonder how to apply the Do... Until // While... WEnd in this code.

@Nine - good point, I'll check that once I run it on my other computer - I'll edit this reply asap.

 

[Dave1]

Quote

Does your computer has set Windows scaling (125% - 150%) ?  That might explain why it does not search at the right area...

There's no scaling on any of the screens.

[JockoDundee]

Is this thread cleared for responses by the AHJ?

I am tempted to make a material post, however, considering the dicey content and the lingering staff presence, I am fearful of violating the “posting while mods are working” edict.

[Somerset]

From what i seen ,your are clear to post what you want. No mod has declared any edicts .

They generally say something to the effect of asking others to stay out of it, (the topic or post at hand) after they have made a post of inquiry to the OP or other personages.

Edited March 12, 2021 by Somerset
clarification.

[JockoDundee]

2 hours ago, Somerset said:

They generally say something to the effect of asking others to stay out of it...

I’ve learned the hard way it’s best not to rely on an explicit statement

[Somerset]

He was doing his "voluntary work" he does on here, and was in the middle of an inquiry to someone.

It is assumed that others would know when to remain silent, until their inquiry is answered or action is taken.

Emphasis on voluntary!

[argumentum]

4 hours ago, JockoDundee said:

....and the lingering staff presence, I am fearful of violating the “posting while mods are working” edict.

lol, I agree that this pic. :

is nothing like anything that makes any sense for any purpose that I can think of.
Use common sense, and if you feel like working on the code, ...have fun

..unless a mod. starts asking questions about the post itself ( other than "wrong forum" or "learn to post" )  

[Dave1]

Given the posts above, I think I'm entitled to an explanation of some sort.

Given the forum rules and the "dicey" content, I realized now that posting the .exe is not (and never was!) a good practice and I can understand the questioning of its nature regarding rules number 3 and number 9 of the forum rules. This post and its code are not meant to be applied to games nor bots nor is, in any way, malware  The deletion of it makes it crystal clear the moderation team has acted upon it and their action is in favour of the Forum rules.

As human, I live and learn! 😅 

3 hours ago, argumentum said:

is nothing like anything that makes any sense for any purpose that I can think of.

I posted the above picture to give an idea on how the visual interface is. As the title of the window implies - it is a training task (I know, a bit dull 🙄). It is a simple one I was given since this is my first interaction with AutoIT.

The purpose of getting the code in post 10 to work for this single signal is to loop it in the future for different windows of the same nature (regardless of the layout).

 

[JockoDundee]

46 minutes ago, Dave1 said:

Given the posts above, I think I'm entitled to an explanation of some sort.

sure, although you should know that my original intention was not to explain, but only inquire about the status of the thread, with regard to posting substantive responses.
I do not myself make such decisions, but I have been asked to sense the possibility of an impending action and refrain if one seems imminent.

unfortunately, and after careful consideration, I was forced to evaluate your thread as CCC, using the Moody’s system of creditworthiness.

because of the perceived risk of default, and in search of a safe harbor, I asked for clarification from the powers that be.

that’s “sort” of an explanation, no?

[Dave1]

4 hours ago, JockoDundee said:

sure, although you should know that my original intention was not to explain, but only inquire about the status of the thread, with regard to posting substantive responses.
I do not myself make such decisions, but I have been asked to sense the possibility of an impending action and refrain if one seems imminent.

unfortunately, and after careful consideration, I was forced to evaluate your thread as CCC, using the Moody’s system of creditworthiness.

because of the perceived risk of default, and in search of a safe harbor, I asked for clarification from the powers that be.

that’s “sort” of an explanation, no?

Again - living and learning! Thanks for the explanation. Is there something I can do to get that rating to go up? Or is it a lost cause that I cannot help/control and depends on the actual subject of the post itself (being trendy somehow)?

Moving into the code matter - what's the impact on the possibility of having more people looking at this thread? Is the rating something that hinders people from actually looking at it and state their opinion on the code/post? Or is it something internal to the moderation team and is none of my business? (I'm just curious! 🙄)

Anyways, if someone cares to look at the code in post 10, please do! I'd be glad to exchange ideas on it! 

[Nine]

On 3/9/2021 at 11:34 AM, Nine said:

Like I said make your code work with Paint if you want help.

Give the true image, make sure anybody can run your script, and you will get some help.