Jump to content

Improper Results for PixelSearch Right-to-Left


Recommended Posts

I am trying to automate a legacy application we never had the source code for...so I can only look at the output screen and work with that.

There are bars that indicate percent reagents.  There are minor variations in the color from full to weak concentration as the bar goes from full to weak.  The length of the bar gives an estimation of concentration remaining - I know where the LH edge of the bar is and I need to find the RH edge of the bar.  Once I have the RH edge of the bar I can then calculate the percent concentration using the current bar length divided by the total bar length when the concentration is 100%.  I need the percent concentration remaining to automate some clicks on another system control screen running in another window.  None of that is a problem (I'm a programmer and have used Autoit a lot).  The problem is getting inconsistent results from PixelSearch.

I have taken repeated samples of the bar and found all the various colors.  I've determined they vary by up to shade difference of 80 and found the average of all the colors in the bar.  The goal is to know what percentage of the bar - which has a known fixed maximum length - is currently colored.

Most of the time this works - here is a simplified version of the code to easily see the problem:
 

$pos=PixelSearch(1237,963,1143,963,4054566,80)   ; note: this is a RH to LH search since X1 > X2

If @error Then
  ; do error code
Else
  $percent = (($pos[0]-1143)/(1237-1143))*100
EndIf

; here is some useful debug code
$pos=PixelSearch(1237,963,1237,963,4054566,80)
If @error Then
  $full = False
Else
 $full = True
EndIf

ConsoleWrite("$full="&$full&"  $percent="&$percent&@CRLF)

 

Most of the time I'd get displayed something like 'False 89' however randomly I'd get 'False 100' - the bar is not 100% but somehow PixelSearch backwards across a single pixel high area thinks it is finding the RH edge whereas looking for the RH pixel does not (as is actually the case) find a pixel within the color variation.

The application moves constantly so I have been testing this with various screenshots of the application - so there is no change between the two PixelSearch calls.  I can't reliably reproduce this error, however, once the error occurs I can run the code on the same screenshot repeatedly the error goes away.  I can go to other screenshot then back to the 'error' screenshot and now the code works properly...or not...randomly.

I tried searching this forum but the only other forum post - describing the exact same thing I think - was shut down because he was trying to automate a game.  I am trying to automate an industrial process.

I don't know if this forum is active, but any help would be much appreciated.  I hope it is just a bit of bad coding on my part and fresh eyes...

 

A snippet from the real code I'm trying to implement:

For $i=0 To 4
    GetSelectedConcentration($i)
Next
Exit


Func GetSelectedConcentration($vatPos)
    ; Note: $vatPos = 0 = rightmost selected vat
    Local $xPos[5][2]
    $xPos[0][0] = 1237
    $xPos[0][1] = 1143
    $xPos[1][0] = 1114
    $xPos[1][1] = 1019
    $xPos[2][0] =  991
    $xPos[2][1] =  895
    $xPos[3][0] =  867
    $xPos[3][1] =  772
    $xPos[4][0] =  743
    $xPos[4][1] =  649
    Local $colorTolerance = 80
    Local $avgBarColor    = 4054566

    ; sometimes doing a nonsense search first will cause this to work
    $pos=PixelSearch(1,1,1,1,1,1)
    ; this is the actual search I want to do
    $pos=PixelSearch($xPos[$vatPos][0],963,$xpos[$vatPos][1],963,$avgBarColor,$colorTolerance)
    If @error Then
        ConsoleWrite("vat "&$vatPos&" 0 concentration"&@CRLF)
        Return 0
    Else
        $percent=Round((($pos[0]-$xPos[$vatPos][1])/($xPos[$vatPos][0]-$xPos[$vatPos][1]))*100)
        ConsoleWrite("$vatPos="&$vatPos&" = "&$percent&@CRLF)
        ConsoleWrite("$pos[0]="&$pos[0]&" $pos[1]="&$pos[1]&" IsColorAtPos="&IsColorAtPos($xpos[$vatPos][0],963,$avgBarColor,$colorTolerance,0,0)&@CRLF)
        ConsoleWrite("---"&@CRLF)
        Return $percent
    EndIf
EndFunc

Func IsColorAtPos($x,$y,$color,$colorTolerance=0,$deltaX=0,$deltay=0,$checkTimes=1,$minimumFoundCount=1,$sleepTime=10)
    If $colorTolerance = 0 AND $deltaX=0 AND $deltaY=0 Then
        $foundCount = 0
        For $i=1 To $checkTimes
            If PixelGetColor($x,$y)=$color Then
                $foundCount = $foundCount+1
            EndIf
        Next
        If $foundCount >= $minimumFoundCount Then
            Return True
        Else
            Return False
        EndIf
    Else
        $foundCount = 0
        For $i=1 To $checkTimes
            $pos=PixelSearch($x-$deltaX,$y-$deltaY,$x+$deltaX,$y+$deltaY,$color,$colorTolerance)
            If @error Then
            Else
                $foundCount=$foundCount+1
            EndIf
            Sleep($sleepTime)
        Next
        If $foundCount >= $minimumFoundCount Then
            Return True
        Else
            Return False
        EndIf
    EndIf
EndFunc

******** Example where the output is consistent with what is on the control screen ********
$vatPos=0 = 88
$pos[0]=1226 $pos[1]=963 IsColorAtPos=False
---
$vatPos=1 = 100
$pos[0]=1114 $pos[1]=963 IsColorAtPos=True
---
$vatPos=2 = 70
$pos[0]=962 $pos[1]=963 IsColorAtPos=False
---
$vatPos=3 = 88
$pos[0]=856 $pos[1]=963 IsColorAtPos=False
---
$vatPos=4 = 100
$pos[0]=743 $pos[1]=963 IsColorAtPos=True
---

********* Weird Output - see vat 1 and vat 2 **********
$vatPos=0 = 99
$pos[0]=1236 $pos[1]=963 IsColorAtPos=True
---
$vatPos=1 = 100
$pos[0]=1114 $pos[1]=963 IsColorAtPos=False
---
$vatPos=2 = 99
$pos[0]=990 $pos[1]=963 IsColorAtPos=True
---
$vatPos=3 = 88
$pos[0]=856 $pos[1]=963 IsColorAtPos=False
---
$vatPos=4 = 100
$pos[0]=743 $pos[1]=963 IsColorAtPos=True
---

Link to post
Share on other sites
  • Moderators

Zog,

Welcome to the AutoIt forums.

Quote

the only other forum post - describing the exact same thing I think - was shut down because he was trying to automate a game.  I am trying to automate an industrial process.

Not that I do not believe you , but please post a screenshot of this constantly moving application so we can see exactly what we are dealing with.

M23

P.S. And just to be absolutely clear - this is the Mod team determining the legality of the thread, so everyone else please keep out.

 

Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Link to post
Share on other sites

I'll ask management if I can post a screenshot tomorrow.  However, this seems to be a bug, or my code is somehow messed up.  I have used Autoit a lot for automating testing of our own gui-based applications, and I have successfully used PixelSearch on other projects (enough that I created the routine shown in the longer example above:

IsColorAtPos($x,$y,$color,$colorTolerance=0,$deltaX=0,$deltay=0,$checkTimes=1,$minimumFoundCount=1,$sleepTime=10)

This routine is useful when you want to get a true/false if a color (with/without tolerance) is at or around a spot on the screen. 

In chemical processing there are lots of display screens that work well for humans - we detect color changes sorta automatically - that I've found more difficult to automate.  In industry it is very common to buy control software that you have absolutely no control over and no source code and very often no API giving control information (such as the application I'm working on above.)  However for management and/or operator screens I will consolidate this information into a screen(s) so the operator/manager does not need to flip through and learn all the different component software we have bought or legacy software we have created.  Depending on the immediacy and/or the difficulty of the task I and/or other programmers have often used Autoit as the 'glue' because it runs with pretty darn good compatibility on both Windows and Unix applications - Autoit doesn't care if the screen I'm looking is running in Unix or Windows...I can look at the screen display either way.

I note all this because I want you to know I'm not a neophyte at Autoit and this problem has vexed me and I have read the forums here looking for a solution.  My reason for noting that reference to a gamer's question above was to demonstrate that I have read the forums.  Wish I hadn't done that now...

Link to post
Share on other sites
  • Moderators

Zog,

If it makes it easier to arrange then just send an image directly to me rather than post it publicly.

M23

Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Link to post
Share on other sites

In the staff meeting today the boss said it was not worth going to the lawyers to see if we could send a screenshot of the proprietary software outside the company.

We did brainstorm a bit and the guys gave me two good work-around ideas:

1 Look from the left to right as PixelSearch seems to be broken looking right to left.  I'll search for the background color instead of the bar color.  This was a bit of a <facepalm>Doh</facepalm> moment for me when two of my teammates immediately suggested this.

2. Upon startup find the bar edge location.  Then go pixel by pixel down/left (reagent concentrations always decrease after a recharge) until I find the bar.  This is slow but the concentrations do not change all that fast.

I remain intellectually curious about this.  When I get some time I'll try to make up a mock-up of the problem and get it to reproduce reliably then post it here.  To allow some discussion on this should I post it on this thread or create a new one?

Edited by Zog
Link to post
Share on other sites
  • Moderators

Zog,

I cannot imagine anyone going to such lengths to hide a game thread - you have convinced me of your case. Anyone who wants to step in and help - please do!

M23

Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...