InetGet() fails every 253rd request

[suthers]

Hi All,

I am using InetGet() to retreive the Google homepage every 30 seconds in a simple internet connectivity monitoring application.

Every 253rd request fails. Requests 1 to 252 are fine, 253 fails, 254 is fine and so the pattern continues with a failure every 253rd time.

Here's the code for the loop that does the retrieval

expandcollapse popup

While 1
    ;download the test file
    $DownloadStatus = "bad"
    $DownloadStatusReason = "Outgoing Connection Failure or Timeout"
    GUICtrlSetData($StatusLableID, "Downloading " & $URLToDownload & " ....")
    $TimerStart = TimerInit()
    $Success = InetGet($URLToDownload, @ScriptDir & "/LastDownload.dat", 1) ; waits for download
    $DownLoadTimeMilliSeconds = Round(TimerDiff($TimerStart))
    GUICtrlSetData($StatusLableID, "Pausing for " & $TestIntervalMinutes & " minutes")
    If $Success = 1 Then ; got file down - check content
        $Content = FileRead(@ScriptDir & "/LastDownload.dat")
        If StringInStr($Content, $TextToCheckInDownloadedFile) <> 0 Then ; file contained right content
            $DownloadStatus = "good"
            $StatsGood = $StatsGood + 1
            FileWriteLine($LogFileNameAndPath, @MDAY & "/" & @MON & "/" & @YEAR & " " & @HOUR & ":" & @MIN & ":" & @SEC & ",Up," & $DownLoadTimeMilliSeconds)
            $StatsTotal = $StatsGood + $StatsBad
            $PercentGood = Round(($StatsGood / $StatsTotal) * 100)
            if $PercentGood = 100 And $StatsBad <> 0 then $PercentGood = 99
            GUICtrlSetData($LastResultLable, "Download Successful at " & @HOUR & ":" & @MIN & ":" & @SEC & @CRLF & _
                    "Time = " & $DownLoadTimeMilliSeconds & "MilliSeconds" & @CRLF & @CRLF & _
                    $StatsGood & " good, " & $StatsBad &" bad samples since " & $StartTimeAndDate & " - " & $PercentGood & "% Successful")
        Else
            $StatsBadFileContentsBad = $StatsBadFileContentsBad + 1
            $DownloadStatusReason = "Downloaded Contents do not contain Text To Check string"
        EndIf
    Else
        $StatsBadConFail = $StatsBadConFail + 1
    EndIf
    If $DownloadStatus = "bad" Then ; download attempt failed
        $StatsBad = $StatsBad + 1
        $StatsTotal = $StatsGood + $StatsBad
        If $StatsGood = 0 Then
            $PercentGood = 0
        Else
            $PercentGood = Round(($StatsGood / $StatsTotal) * 100)
            if $PercentGood = 100 And $StatsBad <> 0 then $PercentGood = 99
        EndIf
        FileWriteLine($LogFileNameAndPath, @MDAY & "/" & @MON & "/" & @YEAR & " " & @HOUR & ":" & @MIN & ":" & @SEC & ",Down,"&$DownloadStatusReason)
        GUICtrlSetData($LastResultLable, "Download Failed at " & @HOUR & ":" & @MIN & ":" & @SEC & @CRLF & _
                $DownloadStatusReason & @CRLF & @CRLF & _
                $StatsGood & " good, " & $StatsBad &" bad samples since " & $StartTimeAndDate & " - " & $PercentGood & "% Successful")
    EndIf
    
    ;pause time
    For $Counter = 1 To $TestIntervalMinutes * 60
        Sleep(900) ; slightly less than 1000 to allow for display writing
        GUICtrlSetData($StatusLableID, "Pausing for " & $TestIntervalMinutes & " minutes" & @CRLF & ($TestIntervalMinutes * 60) - $Counter & " seconds to go...")
    Next
    
WEnd

[Scriptonize]

I don't see any fileopen() and fileclose() statements.

Could it be that this is causing your problem?

When using FileWriteLine you should use it like this:

$file = FileOpen("test.txt", 1)

; Check if file opened for writing OK
If $file = -1 Then
    MsgBox(0, "Error", "Unable to open file.")
    Exit
EndIf

FileWriteLine($file, "Line1")
FileWriteLine($file, "Line2" & @CRLF)
FileWriteLine($file, "Line3")

FileClose($file)

[WolfWorld]

I don't see any fileopen() and fileclose() statements.

Could it be that this is causing your problem?

When using FileWriteLine you should use it like this:

$file = FileOpen("test.txt", 1)

; Check if file opened for writing OK
If $file = -1 Then
    MsgBox(0, "Error", "Unable to open file.")
    Exit
EndIf

FileWriteLine($file, "Line1")
FileWriteLine($file, "Line2" & @CRLF)
FileWriteLine($file, "Line3")

FileClose($file)

It's the same thing.

FileWriteLine(FName, Content);

Is equals to

$hFile = FileOpen(FName);
FileWriteLine($hFile, Content);
FileClose($hFile)

And yes if you want to use

FileWriteLine(FName, Content);
FileWriteLine(FName, Content);
FileWriteLine(FName, Content);

You should use the FileOpen

[Scriptonize]

@athiwatc

You are right, I missed the line in the help: >_<

If a filename is given rather than a file handle, the file will beopened and closed during the function call. For parsing large textfiles this will be much slower than using filehandles. However,filename will be created if it does not already exist.

[[EDIT]]

Failing at 254 (0 to 253). That number keeps telling me there is a limit being crossed.

I just cant remember which.... aaargh

Edited August 24, 2009 by Scriptonize

[WolfWorld]

@athiwatc

You are right, I missed the line in the help: >_<

If a filename is given rather than a file handle, the file will beopened and closed during the function call. For parsing large textfiles this will be much slower than using filehandles. However,filename will be created if it does not already exist.

[[EDIT]]

Failing at 254 (0 to 253). That number keeps telling me there is a limit being crossed.

I just cant remember which.... aaargh

I can't seem the find a reason also but it happen at the 254, always? Is google doing something?

[BrettF]

Why are we loading the page? Why not ping or something?

[Scriptonize]

A successful ping doesn't mean that a website is up and running.

[herewasplato]

A successful ping doesn't mean that a website is up and running.

While true, the OP has the stated goal of "a simple internet connectivity monitoring application".

[herewasplato]

Hi All,

I am using InetGet() to retreive the Google homepage every 30 seconds in a simple internet connectivity monitoring application.

Every 253rd request fails. Requests 1 to 252 are fine, 253 fails, 254 is fine and so the pattern continues with a failure every 253rd time.

Here's the code for the loop that does the retrieval ...

I don't see why it fails as you described - but I can only glance at the code. You might want to post code that we can copy/paste/run without having to fill in the missing variables/includes... then we can quickly edit parts of the code to remove certain possibilities.

While Ping is probably a better option for your script, it might not give you the info that you are currently reporting via the timer function.

[suthers]

While Ping is probably a better option for your script, it might not give you the info that you are currently reporting via the timer function.

Hi All,

Much thanks for all contributions. Ping is not an option from our site as we have an extremely restrictive firewall in place that will not allow ping requests (or much else for that matter). Also, this is intended to be a 'total system' monitor, to emulate as closely as possible what a user experiences what they click the link to pull up Google. So we want to include performances (ie losses) through the proxys, firewalls, content filters etc.

Here's the full operational code for those who want to play :-) It is intended to operate from an INI file with the same name as the script, in the same folder, with contents like this

;ConMon INI File
[Global]
$TestIntervalMinutes=0.5
$URLToDownload=http://www.google.com.au
$TextToCheckInDownloadedFile=<title>Google</title>
$UseProxy=no
$ProxySpec =10.33.0.7:8080

expandcollapse popup

#NoTrayIcon
#include <GUIConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <StaticConstants.au3>


AutoItSetOption("GUIOnEventMode", 1)

;get parameters
$ScriptVersion = "1.4"
$AppName = "ConMon"
$GUITitle = $AppName & " " & $ScriptVersion
$InifilenameAndPath = StringTrimRight(@ScriptFullPath, 3) & "ini"
$TestIntervalMinutes = IniRead($InifilenameAndPath, "Global", "$TestIntervalMinutes", 0.5)
If $TestIntervalMinutes < 0.1 Then $TestIntervalMinutes = 0.1
$URLToDownload = IniRead($InifilenameAndPath, "Global", "$URLToDownload", "http://www.google.com")
$TextToCheckInDownloadedFile = IniRead($InifilenameAndPath, "Global", "$TextToCheckInDownloadedFile", "<title>Google</title>")
$UseProxy = IniRead($InifilenameAndPath, "Global", "$UseProxy", "no")
$ProxySpec = IniRead($InifilenameAndPath, "Global", "$ProxySpec", "10.33.0.7:8080")
$CloseOutLog = "no"

$LogFileName = @MDAY & "_" & @MON & "_" & @YEAR & " " & @HOUR & "_" & @MIN & ".log"
$LogPath = @ScriptDir & "\logs"
DirCreate($LogPath)
$LogFileNameAndPath = $LogPath & "\" & $LogFileName

Dim $StatsGood=0, $StatsBad=0, $StatsTotal=0, $PercentGood=0, $StatsBadConFail=0, $StatsBadFileContentsBad=0

$ConfirmationText = "Test Interval (Minutes) = " & $TestIntervalMinutes & @CRLF & _
        "URL To Download = " & $URLToDownload & @CRLF & _
        "Text to Confirm in Downloaded File = " & $TextToCheckInDownloadedFile & @CRLF & _
        "Use Proxy = " & $UseProxy & @CRLF
If $UseProxy = "yes" Then $ConfirmationText = $ConfirmationText & "Proxy Specification = " & $ProxySpec & @CRLF
$ConfirmationTextMBV = $ConfirmationText & @CRLF & "If these settings need changing, Select 'Cancel' and the INI file will be opened for editing."

$Proceed = MsgBox(3, "Proceed With These Settings?", $ConfirmationTextMBV)
If $Proceed = 7 Then Exit ; no
If $Proceed = 2 Then ; cancel
    Run('notepad.exe "' & $InifilenameAndPath & '"')
    Exit
EndIf
$CloseOutLog = "yes"

;set proxy if required
$ProxyErrorText = "Proxy Set Successfully"
If $UseProxy = "yes" Then
    $check = HttpSetProxy(2, $ProxySpec)
    If $check = 0 Then
        $ProxyErrorText = "Error - Could not set Proxy to " & $ProxySpec & @CRLF & @CRLF & "Using default IE Proxy settings instead"
        MsgBox(0, "ConMon Proxy Set Error", $ProxyErrorText)
    EndIf
Else
    $check = HttpSetProxy(1, $ProxySpec)
    If $check = 0 Then
        $ProxyErrorText = "Error - Could not set direct internet access" & @CRLF & @CRLF & "Using default IE Proxy settings instead"
        MsgBox(0, "ConMon Proxy Set Error", $ProxyErrorText)
    Else
        $ProxyErrorText = "Direct Internet Access Set Successfully"
    EndIf
EndIf
$ConfirmationTextGUI = $ConfirmationText & @CRLF & $ProxyErrorText

;initialise log
$LogHeaderText = $GUITitle & " Log File " & @MDAY & "/" & @MON & "/" & @YEAR & " " & @HOUR & ":" & @MIN & @CRLF & @CRLF & _
        "Using These Settings:" & @CRLF & _
        $ConfirmationText & @CRLF & _
        $ProxyErrorText & @CRLF & @CRLF & _
        "DateTime,Link Up/Down,Response Time (Milliseconds)" & @CRLF & _
        "------------------------------------------------------------"
FileWriteLine($LogFileNameAndPath, $LogHeaderText)

;set up gui
$MainGUIHandle = GUICreate($GUITitle, 400, 360)
GUICtrlCreateLabel("Current Settings", 10, 10, 380, 20, $SS_CENTER)
$ConfLableID = GUICtrlCreateLabel($ConfirmationTextGUI, 10, 30, 380, 100, $SS_CENTER)
GUICtrlSetBkColor(-1, 0xBBBBBB)
GUICtrlCreateLabel("Current Status", 10, 140, 380, 20, $SS_CENTER)
$StatusLableID = GUICtrlCreateLabel("Current status...waiting", 10, 160, 380, 30, $SS_CENTER)
GUICtrlSetBkColor(-1, 0xBBBBBB)
GUICtrlCreateLabel("Last Result", 10, 200, 380, 20, $SS_CENTER)
$LastResultLable = GUICtrlCreateLabel("Last Result...waiting", 10, 220, 380, 60, $SS_CENTER)
GUICtrlSetBkColor(-1, 0xBBBBBB)
$OpenLogButton = GUICtrlCreateButton("Open Log", 100, 300, 200, 30)
GUICtrlSetOnEvent($OpenLogButton, "GUIEvents")

GUISetOnEvent($GUI_EVENT_CLOSE, "GUIEvents")

GUISetState()

$StartTimeAndDate = @MDAY & "/" & @MON & "/" & @YEAR & " " & @HOUR & ":" & @MIN & ":" & @SEC

While 1
    ;download the test file
    $DownloadStatus = "bad"
    $DownloadStatusReason = "Outgoing Connection Failure or Timeout"
    GUICtrlSetData($StatusLableID, "Downloading " & $URLToDownload & " ....")
    $TimerStart = TimerInit()
    $Success = InetGet($URLToDownload, @ScriptDir & "/LastDownload.dat", 1) ; waits for download
    $DownLoadTimeMilliSeconds = Round(TimerDiff($TimerStart))
    GUICtrlSetData($StatusLableID, "Pausing for " & $TestIntervalMinutes & " minutes")
    If $Success = 1 Then ; got file down - check content
        $Content = FileRead(@ScriptDir & "/LastDownload.dat")
        If StringInStr($Content, $TextToCheckInDownloadedFile) <> 0 Then ; file contained right content
            $DownloadStatus = "good"
            $StatsGood = $StatsGood + 1
            FileWriteLine($LogFileNameAndPath, @MDAY & "/" & @MON & "/" & @YEAR & " " & @HOUR & ":" & @MIN & ":" & @SEC & ",Up," & $DownLoadTimeMilliSeconds)
            $StatsTotal = $StatsGood + $StatsBad
            $PercentGood = Round(($StatsGood / $StatsTotal) * 100)
            if $PercentGood = 100 And $StatsBad <> 0 then $PercentGood = 99
            GUICtrlSetData($LastResultLable, "Download Successful at " & @HOUR & ":" & @MIN & ":" & @SEC & @CRLF & _
                    "Time = " & $DownLoadTimeMilliSeconds & "MilliSeconds" & @CRLF & @CRLF & _
                    $StatsGood & " good, " & $StatsBad &" bad samples since " & $StartTimeAndDate & " - " & $PercentGood & "% Successful")
        Else
            $StatsBadFileContentsBad = $StatsBadFileContentsBad + 1
            $DownloadStatusReason = "Downloaded Contents do not contain Text To Check string"
        EndIf
    Else
        $StatsBadConFail = $StatsBadConFail + 1
    EndIf
    If $DownloadStatus = "bad" Then ; download attempt failed
        $StatsBad = $StatsBad + 1
        $StatsTotal = $StatsGood + $StatsBad
        If $StatsGood = 0 Then
            $PercentGood = 0
        Else
            $PercentGood = Round(($StatsGood / $StatsTotal) * 100)
            if $PercentGood = 100 And $StatsBad <> 0 then $PercentGood = 99
        EndIf
        FileWriteLine($LogFileNameAndPath, @MDAY & "/" & @MON & "/" & @YEAR & " " & @HOUR & ":" & @MIN & ":" & @SEC & ",Down,"&$DownloadStatusReason)
        GUICtrlSetData($LastResultLable, "Download Failed at " & @HOUR & ":" & @MIN & ":" & @SEC & @CRLF & _
                $DownloadStatusReason & @CRLF & @CRLF & _
                $StatsGood & " good, " & $StatsBad &" bad samples since " & $StartTimeAndDate & " - " & $PercentGood & "% Successful")
    EndIf
    
    ;pause time
    For $Counter = 1 To $TestIntervalMinutes * 60
        Sleep(900) ; slightly less than 1000 to allow for display writing
        GUICtrlSetData($StatusLableID, "Pausing for " & $TestIntervalMinutes & " minutes" & @CRLF & ($TestIntervalMinutes * 60) - $Counter & " seconds to go...")
    Next
    
WEnd

Exit

;FUNCTIONS
Func GUIEvents()
    Select
        Case @GUI_CtrlId = $GUI_EVENT_CLOSE
            Exit
            
        Case @GUI_CtrlId =  $GUI_EVENT_RESTORE
            GUISetState (@SW_RESTORE,$MainGUIHandle)
            
        Case @GUI_CtrlId = $OpenLogButton
            Run('notepad.exe "' & $LogFileNameAndPath & '"')
            
    EndSelect
EndFunc   ;==>GUIEvents

Func GUIControls()
    Select
        Case @GUI_CtrlId = $OpenLogButton
            Exit
    EndSelect
EndFunc   ;==>GUIControls

Func OnAutoITExit()
    if $CloseOutLog = "yes" then
        FileWriteLine($LogFileNameAndPath, "------------------------------------------------------------")
        FileWriteLine($LogFileNameAndPath, "Total Samples = " & $StatsTotal & @CRLF & "Successes = " & $StatsGood & "(" & $PercentGood & "%)")
        FileWriteLine($LogFileNameAndPath, $StatsBadConFail & " failures due to inability to establish connection")
        FileWriteLine($LogFileNameAndPath, $StatsBadFileContentsBad & " failures due to downloaded content not containing Text To Match string")
        if MsgBox (4,"Results View","Open the Session Log File in Notepad?") = 6 Then Run ('notepad.exe "' & $LogFileNameAndPath & '"')
    Else
        FileDelete ($LogFileNameAndPath)
    EndIf
EndFunc   ;==>OnAutoITExit

[BrettF]

I chose to do it a different method, just by seeing if it could establish a connection on TCP Port 80.

TCPStartup()

$igPort = 80
$sgIp = TCPNameToIP("www.google.com")

If @error Then
    MsgBox(0, "", "Unable to resolve google's IP.")
    Exit
EndIf

$timer = TimerInit()

While 1
    If TimerDiff($timer) > 3000 Then
        $timer = TimerInit()
        If _CheckCon($sgIp, $igPort) Then
            TrayTip("Connection Status", "Connected", 10)
        Else
            TrayTip("Connection Status", "Alert! Is not connected", 10)
        EndIf
    EndIf
WEnd

Func _CheckCon($sIP, $iPort)
    $main = TCPConnect($sIP, $iPort)
    If @error Then Return 0
    TCPCloseSocket($main)
    Return 1
EndFunc   ;==>_CheckCon

See if it works better... Im not sure if its better for bandwidth or not, but meh.

[Scriptonize]

@suthers

So far does your code work as expected.

294 checks, not one of them has failed.

Maybe your problem lies outside the script.

[suthers]

I chose to do it a different method, just by seeing if it could establish a connection on TCP Port 80.

I like that idea for getting around ping blocking :-)

But for our situation where we're wanting to emulate the users online experience (as described above) we do want to actually download some content.

[BrettF]

>_<

Okay. No idea why it would do this, it seems weird. And I have no idea why.

Maybe try the HTTP UDFs if we can't work it out...

[Richard Robertson]

I've been reading this topic for a couple days and I was wondering, perhaps if your script is failing in the tight environment, you have instead hit some kind of barrier at work? Have you tested the same script at home?

[suthers]

I've been reading this topic for a couple days and I was wondering, perhaps if your script is failing in the tight environment, you have instead hit some kind of barrier at work? Have you tested the same script at home?

You have a really good point, especially in light of Scriptonizer being unable to replicate the fault. I have been running it from compiled exe compiled a long time ago, so today re-ran it from script under beta v3.2.13.9 which is the latest I have here. The first failure point was on the 256th try, up from 253. ?????? I don't get it. So I think the things to do are (1) test @ home, and (2) try under the latest beta (which I can't download here. Have to do that @ home too.)

[Richard Robertson]

Let us know how that goes.

[suthers]

Let us know how that goes.

At home the script runs faultlessly. 1000 checks without one failure. I'd say Richard is right and it's something in the network connectivity environment here at work that is causing the issue.

Thanks for everyone who posted & helped get to the bottom of this one. It's outside AutoIT.

Shalom

Bill