Jump to content

Detect screen change (NOT FOR A GAME!)


Go to solution Solved by AlessandroAvolio,

Recommended Posts

I know this has been done before but a quick forum search didn't yield much in the way of results. Sorry for what must be an annoying repeat question!
132680409_contactswaiting.thumb.png.54af7b21d7692618896acc7e63a4e08f.png
Our company uses mitel phone system and has a web interface called "Ignite" that will let us know of the call %'s and most importantly if there is a call waiting in the queue. I have a screen refresh running on that tab in chrome to refreshes every 15seconds (dont know if it's relevant but I included this info just in case)
What I want to do is have Autoit watch a specific section of the screen and tell me if a "0" changes to any other number in that specific area. 
I've got a link to the screenshot I took of the page and it's source. I dont see much in the source that would help me to monitor. Anything that simple will watch the indicated area for a change from ")" to any other number and play a sound would be awesome. 

Link to post
Share on other sites

You can follow a simple approach, which alerts you to the generic change of a portion of the screen, using the PixelCheckSum function. If the program doesn't allow you to easily distinguish zero with PixelCheckSum, you can follow a more complicated approach: choose whether the program must distinguish zero from other numbers on the basis of graphic information or on the basis of textual information. In the first case you will have to collect graphic information, i.e. position and colors of the zero pixels in order to distinguish it from the other numbers and for this you'll have to use PixelSearch, PixelGetColor and/or ImageSearch UDF. In the second case, you can base your program on the text of the page that you can get from the source code or by "select all", "copy" and "paste" to the clipboard (Send, ClipGet, Stringinstr, StringMid etc).

Edited by AlessandroAvolio
Link to post
Share on other sites
  • Moderators

Atamisk,

Welcome to the AutoIt forum.

Does the AutoIt Window Info tool give any details of the GUI and the controls within it? I rather doubt it will, but you never know.

Is the GUI in which the number appears resizeable? If not, then you might be able to use PixelCheckSum on the specific area of the GUI and note when a change occurs - I have done this in the past on various "unhelpful" GUIs.

M23

Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Link to post
Share on other sites
48 minutes ago, AlessandroAvolio said:

You can follow a simple approach, which alerts you to the generic change of a portion of the screen, using the PixelCheckSum function. If the program doesn't allow you to easily distinguish zero with PixelCheckSum, you can follow a more complicated approach: choose whether the program must distinguish zero from other numbers on the basis of graphic information or on the basis of textual information. In the first case you will have to collect graphic information, i.e. position and colors of the zero pixels in order to distinguish it from the other numbers and for this you'll have to use PixelSearch, PixelGetColor and/or ImageSearch UDF. In the second case, you can base your program on the text of the page that you can get from the source code or by "select all", "copy" and "paste" to the clipboard (Send, ClipGet, Stringinstr, StringMid etc).

I'll begin working on these as options now. Per chance, is there a generic format I can see on building out something like this? I seem to remember a repository of auto-it scripts that others have done? I hate to be a script kiddy, but I'm not great at coding. :(

 

 

48 minutes ago, Melba23 said:

Atamisk,

Welcome to the AutoIt forum.

Does the AutoIt Window Info tool give any details of the GUI and the controls within it? I rather doubt it will, but you never know.

Is the GUI in which the number appears resizeable? If not, then you might be able to use PixelCheckSum on the specific area of the GUI and note when a change occurs - I have done this in the past on various "unhelpful" GUIs.

M23

Thank you for the welcome! 
Here is the window summary I pulled from AutoIt Windows Info tool. The GUI is just a simple chrome window and I can resize it as needed. 
 

window summary.png

Link to post
Share on other sites
  • Solution
Posted (edited)

Open Scite, type the function name, set the cursor on the word, press F1, you get definition and examples. 

An example of a program that might work  :)

#include <AutoItConstants.au3>

Main()

Func Main()

    Local Const $aScreenArea[4] = [39, 355, 123, 429]   ; Get coords with AutoIt v3 Window Info tool, Mouse tab
    Local $iCheckSumZero
    Local $iCheckSumNotZero

    Local $iCheckSum

    MsgBox(64, "Collecting graphic info", "Ensure number zero is on screen, inside $aScreenArea portion before continue")

    $iCheckSumZero = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])

    While 1

        $iCheckSum = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])

        While $iCheckSum == $iCheckSumZero
            Sleep(400)
            $iCheckSum = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])
        WEnd

        $iCheckSumNotZero = $iCheckSum

        SoundPlay(@WindowsDir & "\media\tada.wav", $SOUND_NOWAIT)

        $iCheckSum = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])

        While $iCheckSum == $iCheckSumNotZero
            Sleep(400)
            $iCheckSum = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])
        WEnd

    WEnd

EndFunc

 

Edited by AlessandroAvolio
Link to post
Share on other sites
22 minutes ago, AlessandroAvolio said:

Open Scite, type the function name, set the cursor on the word, press F1, you get definition and examples. 

An example of a program that might work  :)

#include <AutoItConstants.au3>

Main()

Func Main()

    Local Const $aScreenArea[4] = [39, 355, 123, 429]   ; Get coords with AutoIt v3 Window Info tool, Mouse tab
    Local $iCheckSumZero
    Local $iCheckSumNotZero

    Local $iCheckSum

    MsgBox(64, "Collecting graphic info", "Ensure number zero is on screen, inside $aScreenArea portion before continue")

    $iCheckSumZero = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])

    While 1

        $iCheckSum = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])

        While $iCheckSum == $iCheckSumZero
            Sleep(400)
            $iCheckSum = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])
        WEnd

        $iCheckSumNotZero = $iCheckSum

        SoundPlay(@WindowsDir & "\media\tada.wav", $SOUND_NOWAIT)

        $iCheckSum = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])

        While $iCheckSum == $iCheckSumNotZero
            Sleep(400)
            $iCheckSum = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])
        WEnd

    WEnd

EndFunc

 

OMG thank you so much! I'll use this as a template and work with it this evening. It'll help out the other T1 guys to get a notification that there is something waiting if I can append this to work on their computers as well. Thanks again!!!

Link to post
Share on other sites
3 minutes ago, Atamisk said:

OMG thank you so much! I'll use this as a template and work with it this evening. It'll help out the other T1 guys to get a notification that there is something waiting if I can append this to work on their computers as well. Thanks again!!!

Make sure it works first! 😄 If it goes well don't forget to mark the solution in this thread! :) 

Link to post
Share on other sites
3 minutes ago, Atamisk said:

I'm playing with it now. I will be back to let you know how it goes. 

You will need to check whether or not it was relevant to specify "refreshes every 15seconds (dont know if it's relevant but I included this info just in case)" because the script can only work if the page is static and doesn't move when reloading

Link to post
Share on other sites

Yep, noticed that was a problem so I disabled my tab refresher program. The portal should do the Contact waiting update on it's own but it was not 100% so I tossed a tab auto-refresher in there to help things along. For the purpose of this working I can disabled my addon and trust Mitel's software to update like it's supposed to (HA!)

Link to post
Share on other sites

So Changed a little bit here but I'm having some issues. I dont know if this is relevant or not but I have 3 monitors (laptop screen and 2 LCD's off the laptop's dock)

I made the following adjustments to the script
 

#include <AutoItConstants.au3>

Main()

Func Main()

    Local Const $aScreenArea[4] = [4160, 320, 4200, 350]   ; Get coords with AutoIt v3 Window Info tool, Mouse tab
    Local $iCheckSumZero
    Local $iCheckSumNotZero

    Local $iCheckSum

    MsgBox(64, "Collecting graphic info", "Ensure number zero is on screen, inside $aScreenArea portion before continue")

    $iCheckSumZero = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])

    While 1

        $iCheckSum = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])

        While $iCheckSum == $iCheckSumZero
            Sleep(1000)
            $iCheckSum = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])
        WEnd

        $iCheckSumNotZero = $iCheckSum

        SoundPlay(@WindowsDir & "\media\Alarm06.wav", $SOUND_NOWAIT)

        $iCheckSum = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])

        While $iCheckSum == $iCheckSumNotZero
            Sleep(1000)
            $iCheckSum = PixelChecksum($aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3])
        WEnd

    WEnd

EndFunc

Simple things like different sound and a bit longer sleep timer. 
Here is the screenshot of the window info on the watched area
It's not working though. I feel like I'm missing something basic here. 

call waiting window.png

Link to post
Share on other sites

so I changed it to the Beep(500,500) and it didn't work but I opened a program window and threw it over in that area and my computer beeped. I think it's a location thing. Maybe I should increase the size of the detection box and test that. I'm gonna play with a tiny window about the size of my detection box and see if it beeps while I moved it near that area.  

Link to post
Share on other sites

weird, it's above the detection area by about 1/4 an inch. Wonder if having the three screens is messing with the detection area..... or maybe I'm thinking about the X,Y positions wrong? 

Link to post
Share on other sites
1 minute ago, Atamisk said:

strano, è sopra l'area di rilevamento di circa 1/4 di pollice. Mi chiedo se avere i tre schermi stia rovinando l'area di rilevamento..... o forse sto pensando alle posizioni X,Y sbagliate? 

Find out with this

#include <ScreenCapture.au3>

Main()

Func Main()

    Local Const $aScreenArea[4] = [296, 0, 323, 31]   ; Get coords with AutoIt v3 Window Info tool, Mouse tab
    Local Const $sZeroFileName = "zero.bmp"
    Local Const $sZeroFilePath = @ScriptDir & "\" & $sZeroFileName

    _ScreenCapture_Capture($sZeroFilePath, $aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3], False)

    SplashImageOn("Zero in selected screen area", $sZeroFilePath, 400, 300, Default, @DesktopHeight * 0.10)
    Sleep(1000)
    MsgBox(64 + $MB_TOPMOST, "Confirm zero is in selected screen area", "Ensure zero is shown in the window above.")
    SplashOff()

EndFunc

 

Link to post
Share on other sites
23 minutes ago, AlessandroAvolio said:

Find out with this

#include <ScreenCapture.au3>

Main()

Func Main()

    Local Const $aScreenArea[4] = [296, 0, 323, 31]   ; Get coords with AutoIt v3 Window Info tool, Mouse tab
    Local Const $sZeroFileName = "zero.bmp"
    Local Const $sZeroFilePath = @ScriptDir & "\" & $sZeroFileName

    _ScreenCapture_Capture($sZeroFilePath, $aScreenArea[0], $aScreenArea[1], $aScreenArea[2], $aScreenArea[3], False)

    SplashImageOn("Zero in selected screen area", $sZeroFilePath, 400, 300, Default, @DesktopHeight * 0.10)
    Sleep(1000)
    MsgBox(64 + $MB_TOPMOST, "Confirm zero is in selected screen area", "Ensure zero is shown in the window above.")
    SplashOff()

EndFunc

 

Very odd. Using your program I was able to narrow down the cord's and the window info was way off
Blue lightning bolt is my Cursor position, red lines are detection area that makes noise when crossed
Right side has the cursor cord's while the bottom window shows what the program sees as the cursor cords. Wonder why the discrepancy. 

screen cap1.png

Link to post
Share on other sites
14 minutes ago, Atamisk said:

Very odd. Using your program I was able to narrow down the cord's and the window info was way off
Blue lightning bolt is my Cursor position, red lines are detection area that makes noise when crossed
Right side has the cursor cord's while the bottom window shows what the program sees as the cursor cords. Wonder why the discrepancy. 

screen cap1.png

Does it change something if you run it to work on main central display?

Link to post
Share on other sites

At the moment, it is running on the laptop screen which would be considered the main display but with the two externals still plugged in.  
I'll unplug the other screens here and test again. I'm betting it has to do with the multi-monitors messing with how things are detected. Gotta go take care of some stuff so I'll have to work more on this tomorrow. Thanks for the help, I'll let you know how it goes!

Link to post
Share on other sites
2 minutes ago, Atamisk said:

At the moment, it is running on the laptop screen which would be considered the main display but with the two externals still plugged in.  
I'll unplug the other screens here and test again. I'm betting it has to do with the multi-monitors messing with how things are detected. Gotta go take care of some stuff so I'll have to work more on this tomorrow. Thanks for the help, I'll let you know how it goes!

ok :) 

Link to post
Share on other sites

I think you can reach your goal with multiple solutions and imagesearch/pixelchecksum I would not start with (although it should be possible)

Alternatives

  • IUIAutomation search on the forum plenty of links
    With simplespy you should get a basic code example to start with how to inspect an element.
  • WebDriver
  • javascript (injected thru addressbar)
    javascript:if (document.getElementsByName("q")[0].value="abc") {alert(searchEl[0].value)}

     

  • pixelchecksum
  •  
Edited by junkew
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...