Problem writing to text file

[czhe]

Hi all,

I have some code that writes to a file using:

_FileCreate($fileName)
_FileWriteToLine($fileName, 1, $timeCount, 1)

The second command gets called periodically to write the current value to the same file and it just keeps replacing the first line of text.

While the code is running, I can open the text file and it will have the latest value at all times. However, if my system unexpectedly crashes, the text file will have nothing in it upon system recovery.

Is there any way to write the file so that its contents are always saved and will retain even after a crash?

Thanks for any input,

czhe

[Malkey]

You want the current contents to always be saved, 100% of the time. Well this method would save the contents 99.9% of the time. You would have to be unlucky to have your system crash during the FileOpen command and the CloseFile command which takes between 1 and 5 millisecond.

HotKeySet("{ESC}", "Terminate")

Global $sFileName = "TestFile.txt"
Local $sContents

While 1
    $sContents = @HOUR & ":" & @MIN & ":" & @SEC & @CRLF
    _SaveContents($sContents)
    Sleep(100)
WEnd

Func _SaveContents($Data)
    ;$begin = TimerInit()
    Local $file = FileOpen($sFileName, 10) ; 2 + 8 = Write mode (erase previous contents) + Create if it doesn't exist.
    FileWrite($file, $Data)
    FileClose($file)
    ;ConsoleWrite( TimerDiff($begin) & @CRLF)
    Return
EndFunc ;==>_SaveContents

Func Terminate()
    ShellExecute($sFileName)
    Exit 0
EndFunc ;==>Terminate

[dani]

You could also use FileCopy to backup the old file first, then writing the new data. If your machine crashes during copy then you still have the original, if it crashes during writing you still have the backup.

[jchd]

However, if my system unexpectedly crashes, the text file will have nothing in it upon system recovery.

Did this happen once or is it always the case? Sorry but I find it hard to believe that you are unlucky enough to have your system crash everytime exactly <after the file is re-created and committed to directory> and <before your string makes it to the disk>. Repeatidly hitting such a small window has a really very low odd on a standard Win system using typical disks and controllers.

You may want to turn off the write cache for this disk, even if write cacheing doesn't explain 100% of what you experience.

Anyway, Malkey solution is way better than what you use.

[czhe]

Thanks guys for the help. Unfortunately my file is still empty when my system restarts. I've also disabled write caching. If it helps, I've posted my entire body of code below. I'm a beginner, so my code may be very inefficient . Any pointers are appreciated.

The most relevant piece of code is in the _SaveTimeToLog function. I basically used Malkey's solution there.

expandcollapse popup

#include <GUIConstantsEx.au3>
#include <GuiConstantsEx.au3>
#include <WindowsConstants.au3>
#include <Date.au3>
#include <File.au3>

Opt('MustDeclareVars', 1)
Opt("GUIOnEventMode", 1)

Global $startCount
Global $timeStamp, $timeCount
Global $file, $fileName, $search, $latestFile, $getFileTime, $line
Global $startID, $stopID, $timeLabel
Global $fileDay, $fileMonth, $fileYear, $fileHour, $fileMin, $fileSec

_Main()

Func _Main()
    GUICreate("Ageless Time", 220, 100) ;title of GUI
    $timeLabel = GUICtrlCreateLabel("0 HR 0 MIN 0 SEC", 10, 60, 400, 200) ;label to display current count
    GUICtrlSetFont ($timeLabel, 14, 400, 2, "Verdana")
    $startID = GUICtrlCreateButton("Start", 10, 15, 60, 30) ;start button
    GUICtrlSetOnEvent($startID, "onstart")
    $stopID = GUICtrlCreateButton("Stop", 80, 15, 60, 30) ;stop button
    GUICtrlSetOnEvent($stopID, "onstop")
    GuiCtrlSetState($stopID, $GUI_DISABLE)

    GUISetOnEvent($GUI_EVENT_CLOSE, "OnExit") ;exit routine when window is closed

    GUISetState()  ; display the GUI

    $fileDay=0
    $fileMonth=0
    $fileYear=0
    $fileHour=0
    $fileMin=0
    $fileSec=0

    $search = FileFindFirstFile("*.txt")

    ; if statement below runs through algorithm to find the latest modified file
    If $search = 1 Then ; Check if the search was successful
        While 1
            $fileName = FileFindNextFile($search)
            If @error Then ExitLoop

            $getFileTime = FileGetTime($fileName, 0, 0)

            If $getFileTime[0] > $fileYear Then
                UpdateTime()
            ElseIf $getFileTime[0] = $fileYear Then
                If $getFileTime[1] > $fileMonth Then
                    UpdateTime()
                ElseIf $getFileTime[1] = $fileMonth Then
                    If $getFileTime[2] > $fileDay Then
                        UpdateTime()
                    ElseIf $getFileTime[2] = $fileDay Then
                        If $getFileTime[3] > $fileHour Then
                            UpdateTime()
                        ElseIf $getFileTime[3] = $fileHour Then
                            If $getFileTime[4] > $fileMin Then
                                UpdateTime()
                            ElseIf $getFileTime[4] = $fileMin Then
                                If $getFileTime[5] > $fileSec Then
                                    UpdateTime()
                                EndIf
                            EndIf
                        EndIf
                    EndIf
                EndIf
            EndIf
        WEnd
    EndIf

    $file = FileOpen($latestFile, 0)

    ; if statement below displays the latest count
    If $file <> -1 Then; Check if file opened for reading OK
        $line = FileReadLine($file, 1)
        If @error <> -1 Then
            GUICtrlSetData($timeLabel, $line)
            FileClose($file)
        EndIf
    EndIf

    ; while loop below updates count every second and write this new count to log file
    While 1
        Sleep(1000)
        If $startCount=1 Then
            $timeCount=Int(TimerDiff($timeStamp)/1000/60/60) & " HR " & Int(Mod(TimerDiff($timeStamp)/1000/60, 60)) & " MIN " & Int(Mod(TimerDiff($timeStamp)/1000, 60)) & " SEC"
            GUICtrlSetData($timeLabel, $timeCount)

            ;_FileWriteToLine($fileName, 1, $timeCount, 1)
            _SaveTimeToLog($timeCount)
        EndIf
    WEnd
EndFunc   ;==>_Main

;--------------- Functions ---------------

Func _SaveTimeToLog($time)
    $file = FileOpen($fileName, 10)
    FileWrite($file, $time)
    FileClose($file)
    Return
EndFunc ;==>_SaveTimeToLog

;function used to determine the latest log file
Func UpdateTime()
    $fileDay=$getFileTime[2]
    $fileMonth=$getFileTime[1]
    $fileYear=$getFileTime[0]
    $fileHour=$getFileTime[3]
    $fileMin=$getFileTime[4]
    $fileSec=$getFileTime[5]
    $latestFile = $fileName
EndFunc ;==>UpdateTime

;if start button is clicked, then start counting and logging
Func onstart()
    if $startCount=0 Then
        GuiCtrlSetState($startID, $GUI_DISABLE)
        GuiCtrlSetState($stopID, $GUI_ENABLE)
        $startCount=1
        $timeStamp=TimerInit()

        $fileName = @MON & "_" & @MDAY & "_" & @YEAR & "_" & @HOUR & "_" & @MIN & "_" & @SEC & ".txt"

        ;_FileCreate($fileName)
    EndIf
EndFunc ;==>onstart

;if stop button is clicked, stop counting
Func onstop()
    GuiCtrlSetState($startID, $GUI_ENABLE)
    GuiCtrlSetState($stopID, $GUI_DISABLE)
    $startCount=0
EndFunc ;==>onstop

Func OnExit()
    Exit
EndFunc ;==>OnExit

I can't really use FileCopy as that would create too many backup files since I'm writing every second.

Also, to the question of how often this happens: it happens all the time as I'm running stress tests on my systems and they hang very often (mostly due to overclocking). I want this timer to log how long my systems have been running before hangs happen.

What am I doing wrong?

Much thanks,

czhe

[omikron48]

Have you tried looking at eventvwr.msc to see if there's an event timestamp you can use to determine when your system crashed?

[jchd]

Here's a corrected version. I've changed a few things

expandcollapse popup

#include <GUIConstantsEx.au3>
#include <GuiConstantsEx.au3>
#include <WindowsConstants.au3>
#include <Date.au3>
#include <File.au3>
#include <Array.au3>

Opt('MustDeclareVars', 1)
Opt("GUIOnEventMode", 1)

Global $startCount
Global $timeStamp
Global $LogName
Global $startID, $stopID, $timeLabel

_Main()

Func _Main()
    GUICreate("Ageless Time", 240, 100) ;title of GUI
    $timeLabel = GUICtrlCreateLabel("---- HR -- MIN -- SEC", 10, 60, 400, 200) ;label to display current count
    GUICtrlSetFont ($timeLabel, 14, 400, 2, "Verdana")
    $startID = GUICtrlCreateButton("Start", 10, 15, 60, 30) ;start button
    GUICtrlSetOnEvent($startID, "onstart")
    $stopID = GUICtrlCreateButton("Stop", 80, 15, 60, 30) ;stop button
    GUICtrlSetOnEvent($stopID, "onstop")
    GuiCtrlSetState($stopID, $GUI_DISABLE)
    GUISetOnEvent($GUI_EVENT_CLOSE, "OnExit") ;exit routine when window is closed
    GUISetState() ; display the GUI

    Local $timediff
    ; we use ISO date time formatas file format as it sorts lexicographically
    Local $files = _FileListToArray(@ScriptDir, '*.log', 1)
    If Not @error Then
        _ArrayDelete($files, 0)
        _ArraySort($files)
        $LogName = $files[UBound($files) - 1]   ; latest available log
    GUICtrlSetData($timeLabel, FileReadLine($LogName))
    EndIf
    ; while loop below updates count every second and write this new count to log file
    Local $hh, $mm, $ss
    Local $hFile
    While 1
    Sleep(1000)
    If $startCount Then
            _TicksToTime(TimerDiff($timeStamp), $hh, $mm, $ss)
            $timediff = StringFormat("%04i HR %02i MIN %02i SEC", $hh, $mm, $ss)
    GUICtrlSetData($timeLabel, $timediff)
            $hFile = FileOpen($LogName, 2)
            FileWriteline($hFile, $timediff)
            FileClose($hFile)
    EndIf
    WEnd
EndFunc ;==>_Main

;--------------- Functions ---------------
;if start button is clicked, then start counting and logging
Func onstart()
    If $startCount = 0 Then
    GuiCtrlSetState($startID, $GUI_DISABLE)
    GuiCtrlSetState($stopID, $GUI_ENABLE)
    $startCount = 1
        $timeStamp = TimerInit()
        $LogName = @ScriptDir & '\' & StringRegExpReplace(_NowCalc(), "/|:", "-") & '.log'
        FileWriteline($LogName, '')     ; create the file
    EndIf
EndFunc ;==>onstart

;if stop button is clicked, stop counting
Func onstop()
    GuiCtrlSetState($startID, $GUI_ENABLE)
    GuiCtrlSetState($stopID, $GUI_DISABLE)
    $startCount = 0
EndFunc ;==>onstop

Func OnExit()
    Exit
EndFunc ;==>OnExit

[czhe]

Thanks again for the help, and I appreciate the new code: it looks much more cleaner and efficient than mine . Hope you don't mind me using it.

Unfortunately, the log file is still empty upon a sudden restart. The eventvwr.msc suggestion helps, but it only covers cases where the system detects the problem. I need to cover all cases, even in the case that somehow someone pressed the restart button by accident. This is why I'm trying to create my own timer. It seems even with write caching disabled, the data is still not being saved permanently to the file.

Any more ideas?

Regards,

czhe

[jchd]

Try adding FileFlush($hFile) before the FileClose($hFile).

Beside pathological hardware (or overclocked to death ) or hacked software (e.g. not WGA ) there is one thing that comes to mind. It's also possible that your disk controller (the one in the HDD) is caching writes for more than one second. If this is the case, it will receive fresh data to write at the same place (same sectors) before it has the chance to write the previous buffer. It doesn't explain why data doesn't get writen eventually, but you never know what happens with heavily overclocked systems.

BTW which type of drive do you have? Is it a basic SATA or a RAID with a software driver...?

Try extending the wait time from 1000ms to, say, 20000ms: it's rather unlikely that a disk cache would hold data that long.

Try logging to another drive.

Try prayers.

Underclock your system.

[99ojo]

Hi,

I think it's your FileOpen:

$hFile = FileOpen($LogName, 2) !This opens your logfile in write mode erasing all data!

So if your computer crashes between FileOpen and FileWrite your logfile is empty.

;-))

Stefan

Edited February 24, 2010 by 99ojo

[omikron48]

From what I recall, the content is only saved after FileClose. So if your system crashes before you are able to execute FileClose, you end up with an empty file.

Best method is as suggested above, backup the current log file, before attempting to overwrite it.

expandcollapse popup

Opt("MustDeclareVars", 1)
Global $logfile = "timestamp.txt"
Global $backup = "timestamp.bak.txt"
Global $start = TimerInit()

FileWrite($logfile, _ConvertTime(TimerDiff($start)))
While 1
    FileCopy($logfile, $backup, 1)
    FileWrite($logfile, _ConvertTime(TimerDiff($start)))
    Sleep(1000)
WEnd

Func _ConvertTime($millis)
    Local $days = Int($millis / 86400000)
    $millis -= $days * 86400000
    Local $hours = Int($millis / 3600000)
    $millis -= $hours * 3600000
    Local $minutes = Int($millis / 60000)
    $millis -= $minutes * 60000
    Local $seconds = Int($millis / 1000)
    $millis -= $seconds * 1000
    Local $result = ""
    If $days > 0 Then
        $result &= $days & "d:"
    EndIf
    If $hours > 0 Then
        $result &= StringFormat("%.2d", $hours) & "h:"
    EndIf
    If $minutes > 0 Then
        $result &= StringFormat("%.2d", $minutes) & "m:"
    EndIf
    If $seconds > 0 Then
        $result &= StringFormat("%.2d", $seconds) & "s"
    Else
        $result &= "00s"
    EndIf
    Return $result
EndFunc

Edited February 24, 2010 by omikron48

[jchd]

That's exactly what the OP wants. What I'm saying is that the time window between the commitment to disk of the file [re]create in the NTFS disk structure and the time where FileClose gets enforced (on the disk itself as well) is very small. Except on a heavily loaded system, you can expect the two events occur within few disk rotations and two avg seeks. Even with stock SATA disks @7200rpm (then 5 rotations are 500µs, avg seek is 8ms), that leaves fairly little chances to occur _once_. But it seems that the OP is experiencing this everytime its system crashes.

I agree that pushing hardware to the limits like this makes no sense, IMO, but that doesn't explain the situation.

[czhe]

I just tried the FileFlush suggestion and it worked! I guess it forced a write to the hard disk while before the cache was holding the value for more than a second each time the file was written to.

In any case, thanks to ALL who made suggestions!

Cheers,

czhe

Edited February 25, 2010 by czhe

[jchd]

It would be interesting to know the type/brand/model of your hard drive, knowing now that a "sleepy cache" is most probably the right explanation. There are many tools around able to give you the information without need to remove screws, in case you don't know what you have in the box. But since you overclock likepublic/style_emoticons/autoit/blink.gif , you likely know what you buy!

[czhe]

Hi jchd,

My hard drive is a mobile 2.5" Hitachi 5K320-160 @ 5400rpm.

Also, my system is not overclocked at all. I was only doing some preliminary testing by forcing restarts to see how reliable my timer is.

Thanks again,

czhe

[jchd]

Good to know.

Last point: could you take the time (on occasion) to remove the FileFlush statement and decrease the time interval gradually from say 10 seconds downwards, to determine how long this kind of slow device is holding data in cache before deciding to write to disk?