Sign in to follow this  
Followers 0
czhe

Problem writing to text file

16 posts in this topic

Hi all,

I have some code that writes to a file using:

_FileCreate($fileName)
_FileWriteToLine($fileName, 1, $timeCount, 1)

The second command gets called periodically to write the current value to the same file and it just keeps replacing the first line of text.

While the code is running, I can open the text file and it will have the latest value at all times. However, if my system unexpectedly crashes, the text file will have nothing in it upon system recovery.

Is there any way to write the file so that its contents are always saved and will retain even after a crash?

Thanks for any input,

czhe

Share this post


Link to post
Share on other sites



You want the current contents to always be saved, 100% of the time. Well this method would save the contents 99.9% of the time. You would have to be unlucky to have your system crash during the FileOpen command and the CloseFile command which takes between 1 and 5 millisecond.

HotKeySet("{ESC}", "Terminate")

Global $sFileName = "TestFile.txt"
Local $sContents

While 1
    $sContents = @HOUR & ":" & @MIN & ":" & @SEC & @CRLF
    _SaveContents($sContents)
    Sleep(100)
WEnd

Func _SaveContents($Data)
    ;$begin = TimerInit()
    Local $file = FileOpen($sFileName, 10) ; 2 + 8 = Write mode (erase previous contents) + Create if it doesn't exist.
    FileWrite($file, $Data)
    FileClose($file)
    ;ConsoleWrite( TimerDiff($begin) & @CRLF)
    Return
EndFunc ;==>_SaveContents

Func Terminate()
    ShellExecute($sFileName)
    Exit 0
EndFunc ;==>Terminate

Share this post


Link to post
Share on other sites

You could also use FileCopy to backup the old file first, then writing the new data. If your machine crashes during copy then you still have the original, if it crashes during writing you still have the backup.

Share this post


Link to post
Share on other sites

However, if my system unexpectedly crashes, the text file will have nothing in it upon system recovery.

Did this happen once or is it always the case? Sorry but I find it hard to believe that you are unlucky enough to have your system crash everytime exactly <after the file is re-created and committed to directory> and <before your string makes it to the disk>. Repeatidly hitting such a small window has a really very low odd on a standard Win system using typical disks and controllers.

You may want to turn off the write cache for this disk, even if write cacheing doesn't explain 100% of what you experience.

Anyway, Malkey solution is way better than what you use.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Thanks guys for the help. Unfortunately my file is still empty when my system restarts. I've also disabled write caching. If it helps, I've posted my entire body of code below. I'm a beginner, so my code may be very inefficient :mellow:. Any pointers are appreciated.

The most relevant piece of code is in the _SaveTimeToLog function. I basically used Malkey's solution there.

#include <GUIConstantsEx.au3>
#include <GuiConstantsEx.au3>
#include <WindowsConstants.au3>
#include <Date.au3>
#include <File.au3>

Opt('MustDeclareVars', 1)
Opt("GUIOnEventMode", 1)

Global $startCount
Global $timeStamp, $timeCount
Global $file, $fileName, $search, $latestFile, $getFileTime, $line
Global $startID, $stopID, $timeLabel
Global $fileDay, $fileMonth, $fileYear, $fileHour, $fileMin, $fileSec

_Main()

Func _Main()
    GUICreate("Ageless Time", 220, 100) ;title of GUI
    $timeLabel = GUICtrlCreateLabel("0 HR 0 MIN 0 SEC", 10, 60, 400, 200) ;label to display current count
    GUICtrlSetFont ($timeLabel, 14, 400, 2, "Verdana")
    $startID = GUICtrlCreateButton("Start", 10, 15, 60, 30) ;start button
    GUICtrlSetOnEvent($startID, "onstart")
    $stopID = GUICtrlCreateButton("Stop", 80, 15, 60, 30) ;stop button
    GUICtrlSetOnEvent($stopID, "onstop")
    GuiCtrlSetState($stopID, $GUI_DISABLE)

    GUISetOnEvent($GUI_EVENT_CLOSE, "OnExit") ;exit routine when window is closed

    GUISetState()  ; display the GUI

    $fileDay=0
    $fileMonth=0
    $fileYear=0
    $fileHour=0
    $fileMin=0
    $fileSec=0

    $search = FileFindFirstFile("*.txt")

    ; if statement below runs through algorithm to find the latest modified file
    If $search = 1 Then ; Check if the search was successful
        While 1
            $fileName = FileFindNextFile($search)
            If @error Then ExitLoop

            $getFileTime = FileGetTime($fileName, 0, 0)

            If $getFileTime[0] > $fileYear Then
                UpdateTime()
            ElseIf $getFileTime[0] = $fileYear Then
                If $getFileTime[1] > $fileMonth Then
                    UpdateTime()
                ElseIf $getFileTime[1] = $fileMonth Then
                    If $getFileTime[2] > $fileDay Then
                        UpdateTime()
                    ElseIf $getFileTime[2] = $fileDay Then
                        If $getFileTime[3] > $fileHour Then
                            UpdateTime()
                        ElseIf $getFileTime[3] = $fileHour Then
                            If $getFileTime[4] > $fileMin Then
                                UpdateTime()
                            ElseIf $getFileTime[4] = $fileMin Then
                                If $getFileTime[5] > $fileSec Then
                                    UpdateTime()
                                EndIf
                            EndIf
                        EndIf
                    EndIf
                EndIf
            EndIf
        WEnd
    EndIf

    $file = FileOpen($latestFile, 0)

    ; if statement below displays the latest count
    If $file <> -1 Then; Check if file opened for reading OK
        $line = FileReadLine($file, 1)
        If @error <> -1 Then
            GUICtrlSetData($timeLabel, $line)
            FileClose($file)
        EndIf
    EndIf

    ; while loop below updates count every second and write this new count to log file
    While 1
        Sleep(1000)
        If $startCount=1 Then
            $timeCount=Int(TimerDiff($timeStamp)/1000/60/60) & " HR " & Int(Mod(TimerDiff($timeStamp)/1000/60, 60)) & " MIN " & Int(Mod(TimerDiff($timeStamp)/1000, 60)) & " SEC"
            GUICtrlSetData($timeLabel, $timeCount)

            ;_FileWriteToLine($fileName, 1, $timeCount, 1)
            _SaveTimeToLog($timeCount)
        EndIf
    WEnd
EndFunc   ;==>_Main

;--------------- Functions ---------------

Func _SaveTimeToLog($time)
    $file = FileOpen($fileName, 10)
    FileWrite($file, $time)
    FileClose($file)
    Return
EndFunc ;==>_SaveTimeToLog

;function used to determine the latest log file
Func UpdateTime()
    $fileDay=$getFileTime[2]
    $fileMonth=$getFileTime[1]
    $fileYear=$getFileTime[0]
    $fileHour=$getFileTime[3]
    $fileMin=$getFileTime[4]
    $fileSec=$getFileTime[5]
    $latestFile = $fileName
EndFunc ;==>UpdateTime

;if start button is clicked, then start counting and logging
Func onstart()
    if $startCount=0 Then
        GuiCtrlSetState($startID, $GUI_DISABLE)
        GuiCtrlSetState($stopID, $GUI_ENABLE)
        $startCount=1
        $timeStamp=TimerInit()

        $fileName = @MON & "_" & @MDAY & "_" & @YEAR & "_" & @HOUR & "_" & @MIN & "_" & @SEC & ".txt"

        ;_FileCreate($fileName)
    EndIf
EndFunc ;==>onstart

;if stop button is clicked, stop counting
Func onstop()
    GuiCtrlSetState($startID, $GUI_ENABLE)
    GuiCtrlSetState($stopID, $GUI_DISABLE)
    $startCount=0
EndFunc ;==>onstop

Func OnExit()
    Exit
EndFunc ;==>OnExit

I can't really use FileCopy as that would create too many backup files since I'm writing every second.

Also, to the question of how often this happens: it happens all the time as I'm running stress tests on my systems and they hang very often (mostly due to overclocking). I want this timer to log how long my systems have been running before hangs happen.

What am I doing wrong?

Much thanks,

czhe

Share this post


Link to post
Share on other sites

Have you tried looking at eventvwr.msc to see if there's an event timestamp you can use to determine when your system crashed?

Share this post


Link to post
Share on other sites

Here's a corrected version. I've changed a few things :mellow:

#include <GUIConstantsEx.au3>
#include <GuiConstantsEx.au3>
#include <WindowsConstants.au3>
#include <Date.au3>
#include <File.au3>
#include <Array.au3>

Opt('MustDeclareVars', 1)
Opt("GUIOnEventMode", 1)

Global $startCount
Global $timeStamp
Global $LogName
Global $startID, $stopID, $timeLabel

_Main()

Func _Main()
    GUICreate("Ageless Time", 240, 100) ;title of GUI
    $timeLabel = GUICtrlCreateLabel("---- HR -- MIN -- SEC", 10, 60, 400, 200) ;label to display current count
    GUICtrlSetFont ($timeLabel, 14, 400, 2, "Verdana")
    $startID = GUICtrlCreateButton("Start", 10, 15, 60, 30) ;start button
    GUICtrlSetOnEvent($startID, "onstart")
    $stopID = GUICtrlCreateButton("Stop", 80, 15, 60, 30) ;stop button
    GUICtrlSetOnEvent($stopID, "onstop")
    GuiCtrlSetState($stopID, $GUI_DISABLE)
    GUISetOnEvent($GUI_EVENT_CLOSE, "OnExit") ;exit routine when window is closed
    GUISetState() ; display the GUI

    Local $timediff
    ; we use ISO date time formatas file format as it sorts lexicographically
    Local $files = _FileListToArray(@ScriptDir, '*.log', 1)
    If Not @error Then
        _ArrayDelete($files, 0)
        _ArraySort($files)
        $LogName = $files[UBound($files) - 1]   ; latest available log
    GUICtrlSetData($timeLabel, FileReadLine($LogName))
    EndIf
    ; while loop below updates count every second and write this new count to log file
    Local $hh, $mm, $ss
    Local $hFile
    While 1
    Sleep(1000)
    If $startCount Then
            _TicksToTime(TimerDiff($timeStamp), $hh, $mm, $ss)
            $timediff = StringFormat("%04i HR %02i MIN %02i SEC", $hh, $mm, $ss)
    GUICtrlSetData($timeLabel, $timediff)
            $hFile = FileOpen($LogName, 2)
            FileWriteline($hFile, $timediff)
            FileClose($hFile)
    EndIf
    WEnd
EndFunc ;==>_Main

;--------------- Functions ---------------
;if start button is clicked, then start counting and logging
Func onstart()
    If $startCount = 0 Then
    GuiCtrlSetState($startID, $GUI_DISABLE)
    GuiCtrlSetState($stopID, $GUI_ENABLE)
    $startCount = 1
        $timeStamp = TimerInit()
        $LogName = @ScriptDir & '\' & StringRegExpReplace(_NowCalc(), "/|:", "-") & '.log'
        FileWriteline($LogName, '')     ; create the file
    EndIf
EndFunc ;==>onstart

;if stop button is clicked, stop counting
Func onstop()
    GuiCtrlSetState($startID, $GUI_ENABLE)
    GuiCtrlSetState($stopID, $GUI_DISABLE)
    $startCount = 0
EndFunc ;==>onstop

Func OnExit()
    Exit
EndFunc ;==>OnExit

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Thanks again for the help, and I appreciate the new code: it looks much more cleaner and efficient than mine :(. Hope you don't mind me using it.

Unfortunately, the log file is still empty upon a sudden restart. The eventvwr.msc suggestion helps, but it only covers cases where the system detects the problem. I need to cover all cases, even in the case that somehow someone pressed the restart button by accident. This is why I'm trying to create my own timer. It seems even with write caching disabled, the data is still not being saved permanently to the file.

Any more ideas? :mellow:

Regards,

czhe

Share this post


Link to post
Share on other sites

Try adding FileFlush($hFile) before the FileClose($hFile).

Beside pathological hardware (or overclocked to death :mellow:) or hacked software (e.g. not WGA :() there is one thing that comes to mind. It's also possible that your disk controller (the one in the HDD) is caching writes for more than one second. If this is the case, it will receive fresh data to write at the same place (same sectors) before it has the chance to write the previous buffer. It doesn't explain why data doesn't get writen eventually, but you never know what happens with heavily overclocked systems.

BTW which type of drive do you have? Is it a basic SATA or a RAID with a software driver...?

Try extending the wait time from 1000ms to, say, 20000ms: it's rather unlikely that a disk cache would hold data that long.

Try logging to another drive.

Try prayers.

Underclock your system.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

#10 ·  Posted (edited)

Hi,

I think it's your FileOpen:

$hFile = FileOpen($LogName, 2) !This opens your logfile in write mode erasing all data!

So if your computer crashes between FileOpen and FileWrite your logfile is empty.

;-))

Stefan

Edited by 99ojo

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

From what I recall, the content is only saved after FileClose. So if your system crashes before you are able to execute FileClose, you end up with an empty file.

Best method is as suggested above, backup the current log file, before attempting to overwrite it.

Opt("MustDeclareVars", 1)
Global $logfile = "timestamp.txt"
Global $backup = "timestamp.bak.txt"
Global $start = TimerInit()

FileWrite($logfile, _ConvertTime(TimerDiff($start)))
While 1
    FileCopy($logfile, $backup, 1)
    FileWrite($logfile, _ConvertTime(TimerDiff($start)))
    Sleep(1000)
WEnd

Func _ConvertTime($millis)
    Local $days = Int($millis / 86400000)
    $millis -= $days * 86400000
    Local $hours = Int($millis / 3600000)
    $millis -= $hours * 3600000
    Local $minutes = Int($millis / 60000)
    $millis -= $minutes * 60000
    Local $seconds = Int($millis / 1000)
    $millis -= $seconds * 1000
    Local $result = ""
    If $days > 0 Then
        $result &= $days & "d:"
    EndIf
    If $hours > 0 Then
        $result &= StringFormat("%.2d", $hours) & "h:"
    EndIf
    If $minutes > 0 Then
        $result &= StringFormat("%.2d", $minutes) & "m:"
    EndIf
    If $seconds > 0 Then
        $result &= StringFormat("%.2d", $seconds) & "s"
    Else
        $result &= "00s"
    EndIf
    Return $result
EndFunc
Edited by omikron48

Share this post


Link to post
Share on other sites

That's exactly what the OP wants. What I'm saying is that the time window between the commitment to disk of the file [re]create in the NTFS disk structure and the time where FileClose gets enforced (on the disk itself as well) is very small. Except on a heavily loaded system, you can expect the two events occur within few disk rotations and two avg seeks. Even with stock SATA disks @7200rpm (then 5 rotations are 500µs, avg seek is 8ms), that leaves fairly little chances to occur _once_. But it seems that the OP is experiencing this everytime its system crashes.

I agree that pushing hardware to the limits like this makes no sense, IMO, but that doesn't explain the situation.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

#13 ·  Posted (edited)

I just tried the FileFlush suggestion and it worked! I guess it forced a write to the hard disk while before the cache was holding the value for more than a second each time the file was written to.

In any case, thanks to ALL who made suggestions! :mellow:

Cheers,

czhe

Edited by czhe

Share this post


Link to post
Share on other sites

It would be interesting to know the type/brand/model of your hard drive, knowing now that a "sleepy cache" is most probably the right explanation. There are many tools around able to give you the information without need to remove screws, in case you don't know what you have in the box. But since you overclock likepublic/style_emoticons/autoit/blink.gif , you likely know what you buy!


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Hi jchd,

My hard drive is a mobile 2.5" Hitachi 5K320-160 @ 5400rpm.

Also, my system is not overclocked at all. I was only doing some preliminary testing by forcing restarts to see how reliable my timer is.

Thanks again,

czhe

Share this post


Link to post
Share on other sites

Good to know.

Last point: could you take the time (on occasion) to remove the FileFlush statement and decrease the time interval gradually from say 10 seconds downwards, to determine how long this kind of slow device is holding data in cache before deciding to write to disk?


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0