Application Behavior Changed while scripting?

[blitzgeber]

Hello first time poster long time scripter here. I have ran into an interesting issue that I have not been able to wrap my head around and I was hoping you guys/gals might be able to shed some light into the solution.

Currently my company is using AutoIT in a production environment to script some third party software we are required to use. This software interacts with USB/Serial Ports and has a standard windows interface we are able to script without too much issue. The problem we run into, however, is that the software itself begins to act behave differently after a set period of time when scripted that doesn't normally occur when not scripted. Let me break things down so you can see this a bit better.

When software is run normally, users will press a 'Go' button and after a few moments they will receive a pass/fail result from the software.

When the script is running, it will start the software automatically but the users will still press 'Go' (Due to the physical process) and after a few moments they will receive a pass/fail result from the software. The script is monitoring the software and records the result.

This worked great in our production environment for about eight hours on three different machines, but suddenly as soon as the 'Go' button is pressed the process would automatically fail. This happened over and over again. They went back to processing without the script and by pressing the button manually and it worked without issue.

It appears to me that the only variable here is who/what pushes the button and who/what opens the software, because it is not to a point that would allow any other variables.

I have made sure that software wise everything is acting correctly, in that I am still pushing the correct button and that the physical process is done the same. I have ruled out everything other than the software we are scripting is breaking due to something I do not know about autoIT. I can provide examples of the script, but I cannot provide examples of the third party software unfortunately. I have tried contacting them but they of course do not support 'This level of troubleshooting'. Any help is greatly appreciated!

[jvanegmond]

Perhaps it's simply a timing issue? Make your script slower. You could try to make your script move the mouse to press the button, rather than a ControlClick.

[blitzgeber]

In either case, the user is actually the one moving the mouse and pushing the button. All the script does is 'watch' the application and log the results so in both cases the timing would be the same (it is a user pushing the button) and in both cases it is the user moving the mouse and clicking the 'Go' Button. It is very odd.

[JohnOne]

I think you should post the script, breaking after 8 hours stinks of memory leak.

[blitzgeber]

I thought the same thing, but after the computers were off all weekend and booted up this morning to the same issue it appears that a memory leak is *probably* not the case. Additionally, the script is closed out completely after it returns results back for each process, which is roughly every ten minutes or so, as well as the software we are automating. It seems that these actions would negate the possibility of a memory leak.

expandcollapse popup

#region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_Res_requestedExecutionLevel=asInvoker
#endregion ;**** Directives created by AutoIt3Wrapper_GUI ****
#cs ----------------------------------------------------------------------------

AutoIt Version: 3.3.8.1
Author:2354131
Script Function:
Template AutoIt script.

#ce ----------------------------------------------------------------------------
#RequireAdmin
#include <Array.au3>
#include <Date.au3>
#cs ------------------------------------------------------------------------------


#ce -------------------------------------------------------------------------------
;Port Control IDs
Global $port1 = 1072
Global $port2 = 1073
Global $port3 = 1074
Global $port4 = 1075
Global $port5 = 1076
Global $port6 = 1077
Global $port7 = 1078
Global $port8 = 1079
;Results Control IDs
Global $result1 = 1055
Global $result2 = 1056
Global $result3 = 1057
Global $result4 = 1058
Global $result5 = 1059
Global $result6 = 1060
Global $result7 = 1061
Global $result8 = 1062
;Results Messages
Global $failCode = "nFAIL"
Global $passCode = "nPASS"
Global $clearCode = "Completed"
Global $contentFailCode = "FAIL!!!"
;Program Iformation regarding ODIN and results
Global $filePath = "C:T679 MDL factory reset CMD serial" ;location of executable file
Global $executableName = "GT679_TEST_MDL_V4.49d.exe" ;full name of executable file
Global $ProgramName = "Odin1_MDL v4.49.d" ;Text the appears in the programs main window
Global $tDate = _Date_Time_GetLocalTime()
$tDate = _Date_Time_SystemTimeToDateTimeStr($tDate)
$tDate = StringRegExpReplace($tDate, "D", "")
Global $resultFileName = "T679_Results_" & $tDate & ".xml"
Global $resultFile = FileOpen($resultFileName, 9)
Global $hwnd = WinGetHandle($ProgramName)
; Accepting Parameters passed from ICE including # of phones and keys
Global $iceArgs = StringSplit($CmdLine[1], "|") ;"#ofPhones|Key|Key|Key|..."
Global $phoneKeys[$iceArgs[1]]
For $i = 2 To UBound($iceArgs) - 1
$phoneKeys[$i - 2] = $iceArgs[$i]
Next
Global $phoneCount = $iceArgs[1]
Global $stillFlashing = True ;The flashing process is still running
Global $clearing = True
Global $phoneCount = 0 ;Number of phones connected to the flashing machine
Global $phonePassCount = 0 ;Number of phones passed flashed to be counted for master clear
Global $portsNotRecorded = True ;Have we recorded the active ports
Global $phonesRecorded = 0 ;Number of phones a record has been found for
Global $returnArray[8][2] ;the 2D associative array reording the result with the phone keys
Global $returnString = "" ;String the will be built and returned with the result of flashing.

;Populate the Return array wiuth zero's then phone keys
For $i = 0 To UBound($returnArray, 1) - 1
For $j = 0 To UBound($returnArray, 2) - 1
$returnArray[$i][$j] = 0
Next
Next
For $i = 0 To UBound($phoneKeys) - 1
$returnArray[$i][0] = $phoneKeys[$i]
Next
;Run the script in the designated directory. Issue when run through an activeX web control
FileChangeDir($filePath)
Run($executableName, $filePath)
WinWaitActive("Odin1_MDL v4.49.d")

#CS --------------------------------------------------------------------------------

The Commented code below is designed for the full automation of the process.
Since the beginning of the program is unpredictable at the moment, this will
not be used until a better process has been determined for full automation
#CE ------------------------------------------------------------------------------------
Sleep(5000) ;allow for phones to power up and connect 10 seconds
;Checks for MD5 errors
$mdFiveErrors = True
While $mdFiveErrors = True
If (WinActive("GT679_TEST_MDL_V4.49d", "MD5 Checksum")) Then
ControlClick("GT679_TEST_MDL_V4.49d", "MD5 Checksum", 2)
Sleep(4000)
EndIf
If (WinActive("GT679_TEST_MDL_V4.49d", "MD5 Checksum")) Then
ContinueLoop
Else
$mdFiveErrors = False
ExitLoop
EndIf
WEnd
;$stillLooking = True
;While $stillLooking
;;Clicks the "Start" button to begin the flashing process change coordinates to live machine
;ControlClick($ProgramName, "", 1001)
;If (WinActive("GT679_TEST_MDL_V4.49d", "Please connect")) Then
;$stillLooking = True
;ControlClick("GT679_TEST_MDL_V4.49d", "Please connect", 2)
;Sleep(7000)
;EndIf
;If (ControlGetText($ProgramName, "Download Start...", 1054)) Then
;$stillLooking = False
;EndIf
;WEnd
#CS ---------------------------------------------------------------
We will wait to start the recording process of the script until
k the user has started the ODIN flashing process. The script will
look for the Download Started... message then begin its process
after the maching has started its process.
#CE ----------------------------------------------------------------
$stillLooking = True ;Determines if the script is still waiting for the user to start the program
While $stillLooking
If (ControlGetText($ProgramName, "Download Start...", 1054)) Then
$stillLooking = False
EndIf
WEnd
While $stillFlashing = True
#CS --------------------------------------------------------
Below the script looks at the coordinates given
for the Port status windows and determines if they
are the active color ($portCode) and records the number
of active ports
#CE --------------------------------------------------------
If $portsNotRecorded Then
Local $port[8]
WinActivate($ProgramName)
$port[0] = StringLeft(ControlGetText($ProgramName, "", $port1), 1)
$port[1] = StringLeft(ControlGetText($ProgramName, "", $port2), 1)
$port[2] = StringLeft(ControlGetText($ProgramName, "", $port3), 1)
$port[3] = StringLeft(ControlGetText($ProgramName, "", $port4), 1)
$port[4] = StringLeft(ControlGetText($ProgramName, "", $port5), 1)
$port[5] = StringLeft(ControlGetText($ProgramName, "", $port6), 1)
$port[6] = StringLeft(ControlGetText($ProgramName, "", $port7), 1)
$port[7] = StringLeft(ControlGetText($ProgramName, "", $port8), 1)
For $i = 0 To UBound($port) - 1
If (StringRegExp($port[$i], 'd')) Then ;If the color recorded for the port is showing an active phone
$phoneCount = $phoneCount + 1
EndIf
Next
$portsNotRecorded = False
;Debugging Message below, remove or comment out when in production
;MsgBox(0, "Active Ports Recorded", $phoneCount)
Else
$Flashing = True
Local $result[8]
Local $phoneFlashed[8]
For $i = 0 To UBound($phoneFlashed) - 1
$phoneFlashed[$i] = 0
Next
#CS --------------------------------------------------------
Below the script looks at the coordinates given
for the Status windows and determines if they are showing
a pass ($passCode), fail ($failCode), or still active ($whiteCode).
Once a record ($phonesRecorded for each phone on an active port ($phoneCount)
the flashing process is determined to be complete and the script
exits the while loop
#CE --------------------------------------------------------
FileWriteLine($resultFile, "<flashingResults>")
While $Flashing
$result[0] = ControlGetText($ProgramName, "", $result1)
$result[1] = ControlGetText($ProgramName, "", $result2)
$result[2] = ControlGetText($ProgramName, "", $result3)
$result[3] = ControlGetText($ProgramName, "", $result4)
$result[4] = ControlGetText($ProgramName, "", $result5)
$result[5] = ControlGetText($ProgramName, "", $result6)
$result[6] = ControlGetText($ProgramName, "", $result7)
$result[7] = ControlGetText($ProgramName, "", $result8)
For $i = 0 To UBound($result) - 1
If (StringRegExp($result[$i], $passCode) And $phoneFlashed[$i] = 0) Then
     $returnArray[$i][1] = 1
     $phoneFlashed[$i] = 1
     $phonesRecorded = $phonesRecorded + 1
     $phonePassCount = $phonePassCount + 1
     $result[$i] = StringRegExpReplace($result[$i], 'W', '')
     FileWriteLine($resultFile, @TAB & "<phonekey>" & $returnArray[$i][0] & "</phonekey>")
     FileWriteLine($resultFile, @TAB & "<port>" & $port[$i] & "</port>")
     FileWriteLine($resultFile, @TAB & "<result>" & $result[$i] & "</result>")
ElseIf (StringRegExp($result[$i], $failCode) And $phoneFlashed[$i] = 0) Then
     $phonesRecorded = $phonesRecorded + 1
     $phoneFlashed[$i] = 1
     $result[$i] = StringRegExpReplace($result[$i], 'W', '')
     FileWriteLine($resultFile, @TAB & "<phonekey>" & $returnArray[$i][0] & "</phonekey>")
     FileWriteLine($resultFile, @TAB & "<port>" & $port[$i] & "</port>")
     FileWriteLine($resultFile, @TAB & "<result>" & $result[$i] & "</result>")
EndIf
Next
;When we have recorded all the phones that are active exite the program
If ($phoneCount = $phonesRecorded) Then
$stillFlashing = False
$Flashing = False
FileWriteLine($resultFile, "</flashingResults>")
;Debugging Message below, remove or comment out when in production
;MsgBox(0, "Flashing Results", "Phones Recorded: " & $phonesRecorded & " Phones Passed: " & $phonePassCount)
EndIf
WEnd
EndIf
WEnd
;Select the button to start Master Clear
ControlClick($ProgramName, "", 1045)
Local $clearing = True
Local $phoneCleared[8]
Local $phoneClearCount = 0
For $i = 0 To UBound($phoneCleared) - 1
$phoneCleared[$i] = 0
Next
;If failed flash consider removing phones from phoneCount
;For $i = 0 to UBound($returnArray,1) - 1
;If($returnArray[$i][1] = 0) Then
;$phoneCleared[$i] = 1
;$phoneClearCount = $phoneClearCount + 1
;EndIf
;Next
;MsgBox(1, "ContentClear", "ContentClear")
FileWriteLine($resultFile, "<contentClear>")
While $clearing
$result[0] = ControlGetText($ProgramName, "", $result1)
$result[1] = ControlGetText($ProgramName, "", $result2)
$result[2] = ControlGetText($ProgramName, "", $result3)
$result[3] = ControlGetText($ProgramName, "", $result4)
$result[4] = ControlGetText($ProgramName, "", $result5)
$result[5] = ControlGetText($ProgramName, "", $result6)
$result[6] = ControlGetText($ProgramName, "", $result7)
$result[7] = ControlGetText($ProgramName, "", $result8)
For $i = 0 To UBound($result) - 1
If (StringReplace($result[$i], @CRLF, "") = $contentFailCode And $phoneCleared[$i] = 0) Then
$returnArray[$i][1] = 0
$phoneCleared[$i] = 1
$phoneClearCount = $phoneClearCount + 1
$result[$i] = StringRegExpReplace($result[$i], 'W', '')
FileWriteLine($resultFile, @TAB & "<phonekey>" & $returnArray[$i][0] & "</phonekey>")
FileWriteLine($resultFile, @TAB & "<port>" & $port[$i] & "</port>")
FileWriteLine($resultFile, @TAB & "<result>" & $result[$i] & "</result>")
EndIf
;MsgBox(0,"",$result[$i] & $clearCode & $result[$i] = $clearCode)
;$var = $result[$i]
;$clearCodeCheck = StringReplace($result[$i], @CRLF, "")
;If (StringStripCR($result[$i]) = $clearCode) Then
;MsgBox(0, "", "valid")
;EndIf
If (StringReplace($result[$i], @CRLF, "") = $clearCode And $phoneCleared[$i] = 0) Then
$phoneClearCount = $phoneClearCount + 1
$phoneCleared[$i] = 1
$result[$i] = StringRegExpReplace($result[$i], 'W', '')
FileWriteLine($resultFile, @TAB & "<phonekey>" & $returnArray[$i][0] & "</phonekey>")
FileWriteLine($resultFile, @TAB & "<port>" & $port[$i] & "</port>")
FileWriteLine($resultFile, @TAB & "<result>" & $result[$i] & "</result>")
EndIf
Next
;When we have recorded all the phones that are active exite the program
If ($phoneCount = $phoneClearCount) Then
$clearing = False
FileWriteLine($resultFile, "</contentClear>")
EndIf
WEnd

For $i = 0 To UBound($returnArray, 1) - 1
If ($returnArray[$i][0] > 0) Then
$returnString = $returnString & $returnArray[$i][0] & "|"
$returnString = $returnString & $returnArray[$i][1] & "|"
EndIf
Next
;Debugging Message below, remove or comment out when in production
;MsgBox(0, "Result String Output", $returnString)
ConsoleWrite($returnString)
FileWriteLine($resultFile, "<returnString>" & $returnString & "</returnString>")
If (ProcessExists($executableName)) Then
ProcessClose($executableName)
EndIf

Edited June 25, 2012 by blitzgeber

[Mechaflash]

talk about taking a top-down approach...

You may want to consider splitting up parts of the code into functions. It's easier to debug, read, and organize.

[rover]

This post should be moved to General Help and Support

blitzgeber

The problem is most likely due to the heavy continuous polling of your script on that program.

or maybe some Run environment or current directory issue as you mention with the activeX web control.

The monitored program has to process and respond to the messages sent by WinActivate and ControlGetText,

perhaps at some point something locks up in the app from missed/blocked internal messages from being polled so heavily, combined with the scripts high cpu usage.

Monitor/wait for window existance with WinWaitActive before calling ControlGetText in while loop

and use Sleep in the tight while/wend loops (high cpu usage by script) you are calling WinActive, WinActivate and ControlGetText in.

Update the log file at a regular Sleep interval instead of continuously.

Add error handling - function errors and return values/arrays-cmdline array/windows/controls/verify sent command returns, etc.

No handle/window verification

WinGetHandle/ControlGetHandle - IsHwnd/WinExists/_WinApi_GetClassName/WinGetTitle

It is best to get a valid/verified window or control handle and use that with the Winxxx Controlxxx functions.

When title/class/text is used there is an internal call to the FindWindow apis in those functions to locate the window.

When a handle is used that additional step is bypassed in the internal code of the Winxxx Controlxxx functions.

Edit: typos

Edited June 26, 2012 by rover

[jaberwacky]

For what it's worth I noticed that you have the same variable $phonecount declared and assigned twice with different values.

Global $phonecount = $iceargs[1]
Global $stillflashing = True ;The flashing process is still running
Global $clearing = True
Global $phonecount = 0 ;Number of phones connected to the flashing machine

Edited June 26, 2012 by LaCastiglione

[rover]

You state in post #5 you are closing the monitored program after each run,

maybe there is some issue with temp files or memory because the process does not get the chance to cleanup on exit.

Try using WinClose($ProgramName) instead of ProcessClose.

[blitzgeber]

This post should be moved to General Help and Support

blitzgeber

The problem is most likely due to the heavy continuous polling of your script on that program.

or maybe some Run environment or current directory issue as you mention with the activeX web control.

The monitored program has to process and respond to the messages sent by WinActivate and ControlGetText,

perhaps at some point something locks up in the app from missed/blocked internal messages from being polled so heavily, combined with the scripts high cpu usage.

Monitor/wait for window existance with WinWaitActive before calling ControlGetText in while loop

and use Sleep in the tight while/wend loops (high cpu usage by script) you are calling WinActive, WinActivate and ControlGetText in.

Update the log file at a regular Sleep interval instead of continuously.

Add error handling - function errors and return values/arrays-cmdline array/windows/controls/verify sent command returns, etc.

No handle/window verification

WinGetHandle/ControlGetHandle - IsHwnd/WinExists/_WinApi_GetClassName/WinGetTitle

It is best to get a valid/verified window or control handle and use that with the Winxxx Controlxxx functions.

When title/class/text is used there is an internal call to the FindWindow apis in those functions to locate the window.

When a handle is used that additional step is bypassed in the internal code of the Winxxx Controlxxx functions.

Edit: typos

These are all very good pionts. I could see any of these issues causing problems long term with a running process. I beleive long term stability is something we definiately have to address, but I can't figure out why the machine is still acting the same way after five or six reboots. I can't imagine one of these issues would cause it to work for a while and then suddenly break everything. Even having a blank script running now seems to cause the software to fail instead of pass.

For what it's worth I noticed that you have the same variable $phonecount declared and assigned twice with different values.

Global $phonecount = $iceargs[1]
Global $stillflashing = True ;The flashing process is still running
Global $clearing = True
Global $phonecount = 0 ;Number of phones connected to the flashing machine

Change has been made to the script =) I beleive the script itself is functioning correctly, so it didn't really remove a bug, but it certainly made it easier to read.

You state in post #5 you are closing the monitored program after each run,

maybe there is some issue with temp files or memory because the process does not get the chance to cleanup on exit.

Try using WinClose($ProgramName) instead of ProcessClose.

I thought about this too, as well as the memory leak issue, but with the restart of the computer *should* have fixed the issue but it didn't. This issue also appeared on three different computers at the same time. The timing is just a little 'too good' for a memory leak or temporary file from my experience.

[rover]

These are all very good pionts. I could see any of these issues causing problems long term with a running process. I beleive long term stability is something we definiately have to address, but I can't figure out why the machine is still acting the same way after five or six reboots. I can't imagine one of these issues would cause it to work for a while and then suddenly break everything. Even having a blank script running now seems to cause the software to fail instead of pass.

I thought about this too, as well as the memory leak issue, but with the restart of the computer *should* have fixed the issue but it didn't. This issue also appeared on three different computers at the same time. The timing is just a little 'too good' for a memory leak or temporary file from my experience.

Even having a blank script running now seems to cause the software to fail instead of pass.

AutoIt does not rely on or install runtime dependencies, so I fail to see how running a compiled blank script

(actually, an interpreted script with stub) on these machines would interfere with the program.

When used normally, is the program running continuously?

i.e. phones are connected, flash is run, then phones are disconnected, repeat?

Your script runs/closes the program for each batch, perhaps the issue lies there.

Is the AutoIt script run directly by the user on the machine or by some other process?

Is it run remotely over a network?

What does this line refer to?

;Run the script in the designated directory. Issue when run through an activeX web control

This issue also appeared on three different computers at the same time. The timing is just a little 'too good' for a memory leak or temporary file from my experience.

Add code to check when the program becomes unresponsive/crashes and add the time/date and cumulative iteration count of the script run and While/WEnd loop iteration count to your logfile

so you can get better data to evaluate the pattern of failure.

There was an unresolved issue on the forum some months back with RunAs consistently failing after being called exactly 33 times in a session (required a reboot to clear), the script was run from a task (NT AUTHORITYSYSTEM).

It is unknown if that is an AutoIt or Windows imposed security limitation, perhaps something similar to that is happening when this program is run/closed repeatedly an unknown number of times per session?

How is this program failing?

Does it crash or become unresponsive?

Does the 'not responding' text appear in the programs caption title bar?

Any error message or log file from the program?

Check with taskmanager or better yet ProcessExplorer from MicroSoft/SysInternals, and see

if there is more than one instance of the monitored program process running when this crash/freeze happens.

Or add a ProcessList monitor to flag if more than one instance of the monitored program is running.

(if an unclosed process from previous script run still exists without any visible windows)

If this phone program does not use a mutex/singleton to enforce a single instance, you can add code

to your script to check for that.

It could be this program is buggy or it is something with these machines.

Try running something like Prime95 or some other stability testing software to see if the machine/os is stable.

A few random questions:

What AutoIt version?

How many machines does this run on? (you mention three)

I fails on all machines it is run on?

New or older hardware?

What OS/Version/SP on the production machines?

Is this run in a Virtual environment?

Are these production machines re-imaged regularly?

(I assume they are dedicated production machines, loaded only with the software for the phone flashing, not network connected, etc.)

Is this software being run on an OS it wasn't developed for?

Does the developer of this program regularly update it, or is this one of those 'it was written a few years back,

but there are no updates or patches, just new versions and new licences to pay for kind of programs?

I can't see the programs developers targeting third party automation software by causing random lockups.

Do they sell an automation option?

Does this program have a com interface?

Edit: changes, additions

Edited June 27, 2012 by rover

[rover]

One other thing.

Add FileClose($resultFile) at the end of the script

Edit: added example

An example using handles with Winxx/Controlxx functions

and ControlSend instead of ControlClick.

You could use "[REGEXPTITLE:GT679_TEST_MDL_(?i)]" or any regex variation so the script is not fixed to a version number

Check for window class or title stringinstr for handle verification

$hWin = WinGetHandle("[GT679_TEST_MDL_V4.49d]", "")
ConsoleWrite('+WinGetText($hWin) = ' & WinGetTitle($hWin) & @crlf)
If WinGetTitle($hWin) == "GT679_TEST_MDL_V4.49d" Then
    $hBtnMD5 = ControlGetHandle($hWin, "MD5 Checksum", 2)
    ConsoleWrite('+ControlGetText($hWin, "", $hBtnMD5) = ' & ControlGetText($hWin, "", $hBtnMD5) & @crlf)
    ;Verify the button exists and is enabled before sending clicks
    If ControlGetText($hWin, "", $hBtnMD5) == "MD5 Checksum" Then
        ControlFocus($hWin, "", $hBtnMD5)
        If ControlCommand($hWin, "", $hBtnMD5, "IsEnabled", "") Then
            ;ControlClick($hWin, "", $hBtnMD5)
            ControlSend($hWin, "", $hBtnMD5, "{SPACE}")
        EndIf
    EndIf
EndIf

Edited June 26, 2012 by rover

[rover]

How could I forget...

Is the script compiled for x64?

Is the OS x64?

Run from admin account?

Try ShellExecute() or some of the posted advanced examples of the CreateProcess api instead of the version coded in run()

Edited June 27, 2012 by rover

[blitzgeber]

Responses inside the quote =D

AutoIt does not rely on or install runtime dependencies, so I fail to see how running a compiled blank script

(actually, an interpreted script with stub) on these machines would interfere with the program. I am in the same boat =)

When used normally, is the program running continuously? Yes it is, for hours and hours

i.e. phones are connected, flash is run, then phones are disconnected, repeat? Correct

Your script runs/closes the program for each batch, perhaps the issue lies there. I thought this might be the case. It is a bit different in terms of the process, but the software itself isn't very stable or

Is the AutoIt script run directly by the user on the machine or by some other process? An activeX control will start the script

Is it run remotely over a network? Remotely

What does this line refer to? We had to specify the working directory when ran from the activeX control. It was not in the correct working directory at first and thus it lost access to the ini files.

Add code to check when the program becomes unresponsive/crashes and add the time/date and cumulative iteration count of the script run and While/WEnd loop iteration count to your logfile

so you can get better data to evaluate the pattern of failure. The software remains responsive and we actually get the results as a fail and return them back with the script.

There was an unresolved issue on the forum some months back with RunAs consistently failing after being called exactly 33 times in a session (required a reboot to clear), the script was run from a task (NT AUTHORITYSYSTEM).

It is unknown if that is an AutoIt or Windows imposed security limitation, perhaps something similar to that is happening when this program is run/closed repeatedly an unknown number of times per session? Perhaps, however a reboot did not clear the issue for us so I am unsure if that could be the cause.

How is this program failing? Normally the flashing process takes 6-7 minutes and shows a 'Pass' result on the screen for each handset. When it fails the software remains responsive but it displays 'FAIL' in the result area after a few seconds after the start of the process.

Does it crash or become unresponsive? Not at all

Does the 'not responding' text appear in the programs caption title bar? Never

Any error message or log file from the program? Unfortunately no, the logging is very minimal

Check with taskmanager or better yet ProcessExplorer from MicroSoft/SysInternals, and see

if there is more than one instance of the monitored program process running when this crash/freeze happens. I remotely monitored the computer and checked this at first. Only one version of the script was running at the time as well as the flashing software. I beleive the below would be helpful though.

Or add a ProcessList monitor to flag if more than one instance of the monitored program is running.

(if an unclosed process from previous script run still exists without any visible windows) Adding this to the next version.

If this phone program does not use a mutex/singleton to enforce a single instance, you can add code

to your script to check for that. It does not, but we will add it.

It could be this program is buggy or it is something with these machines. This runs very stable for the most part due to it's simplicity.

Try running something like Prime95 or some other stability testing software to see if the machine/os is stable. They have been in operation for months now without issue but I can run this if you feel it would still have a benefit.

A few random questions:

What AutoIt version? 3.3.8.1

How many machines does this run on? (you mention three) Currently three in production. Has been tested/ran on over 15 in total

I fails on all machines it is run on? Only the three machines in production see this non stop at the moment. In development we would see this periodically but we assumed it was due to us having development machines and not the exact physical setup that our engineering team has for this process.

New or older hardware? ~2008 machines

What OS/Version/SP on the production machines? XP SP 2

Is this run in a Virtual environment? No

Are these production machines re-imaged regularly? About once a month

(I assume they are dedicated production machines, loaded only with the software for the phone flashing, not network connected, etc.)

Is this software being run on an OS it wasn't developed for? Developed on Windows 7 box, tested/flushed out with Windows XP machine. They then copy the script to the flashing computer and compile the script there.

Does the developer of this program regularly update it, or is this one of those 'it was written a few years back, It is regularly updated, however generally only to add additional model flashing capability.

but there are no updates or patches, just new versions and new licences to pay for kind of programs?

I can't see the programs developers targeting third party automation software by causing random lockups.I would hope not but they aren't exactly helpful in this case. It is a major cell phone OEM, so they really just do what they want in regards to this software.

Do they sell an automation option? No, and we are contractually required to use this software.

Does this program have a com interface? I wish =)

Edit: changes, additions

Thanks for the help!

[blitzgeber]

In response to this I think we have found the issue. It seems like this software just will not function part of the time when it is opened, but closing it and reopening it and trying again until it works was what the users were doing. They were then leaving the application up for weeks on end because it was still functioning. It looks like if we let the user hit the sweet spot with the application being open and then just work off of a functioning instance of the application then we are golden. Shew.

[rover]

In response to this I think we have found the issue. It seems like this software just will not function part of the time when it is opened, but closing it and reopening it and trying again until it works was what the users were doing. They were then leaving the application up for weeks on end because it was still functioning. It looks like if we let the user hit the sweet spot with the application being open and then just work off of a functioning instance of the application then we are golden. Shew.

Glad to hear you sorted it out.

It seems like this software just will not function part of the time when it is opened,

Ah, the diplomacy required when dealing with issues in an enterprise environment,

and the software in question is from a major OEM and there are contractual obligations...

So, a user workaround kept it going.

If their comments on this problem were passed up the chain you would have been informed of this valuable data earlier.

It's possible the software is failing because of drivers it installs or a dependency issue.

At any rate, the ball's back in your vendors court now.

Cheers

Edits: the usual misspelling, typos, rewording and additions...

Edited July 4, 2012 by rover