Problem with _FileWriteToLine ? help needed

[SpinningCone]

FOr a while now i've been running a script that uses a flat text file to keep track of users to perform actions on.

 

I have some logic in the block that adds new users with some comma delimited fields with FileWriteLine and then updates that line with _FileWriteToLine

this has been running fine for years but suddenly i'm having problems with the script crashing because when the very last line of the file is losing it's CRLF so the next appended line concatenates with the last line and the parser fails to parse the last line.

this has been driving me nuts for a while so i finally took apart all my code and have discovered that for some reason _FileWriteToLine deletes CRLF off the bottom of the file after use. i don't know why this started suddenly, i even copied an old file.au3 library and it still does it (maybe a change to shell32.dll? )

I was able to replicate the bug by making a new file with each line having CRLF at the end then  with the following code:
 

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_UseX64=y
#AutoIt3Wrapper_Res_requestedExecutionLevel=asInvoker
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****

_FileWriteToLine($FilePath,12,"Test rewrite",1)

After running the code and refreshing the file i can see that the last blank line disappears (no final CRLF)

Is there a way to prevent this behavior? (without a huge logic rewrite to use a DB or another write method)

EDIT: actually it seems that replacing the old au3 is working, or at least working some of the time. not sure if it's a *sometimes* bug with the older library.

Edited November 12, 2015 by Melba23
Amended title

[Melba23]

SpinningCone,

I can confirm that the current function does indeed strip a trailing @‌‌CRLF from the final line - as does the new version that I have recently written to allow you to add lines to make up the count if required. The only earlier version I have to hand (3.3.8.1) also strips a trailing EOL, so this is not something that has happened recently - checking the SVN logs shows no obvious change to alter the function to do this.

However, as we now know this happens, it is not difficult to code a workaround when there is a need to replace the EOL:

#include <MsgBoxConstants.au3>
#include <File.au3>

$sFile = "Test.txt"

$hFile = FileOpen($sFile, $FO_OVERWRITE)
ConsoleWrite(@error & @CRLF)

$sText = "Line 1" & @CRLF & "Line 2" & @CRLF & "Line 3" & @CRLF & "Line 4" & @CRLF & "Line 5" & @CRLF

$iRet = FileWrite($hFile, $sText)
ConsoleWrite($iRet & @CRLF)

FileClose($hFile)

$sContent = FileRead($sFile)
MsgBox($MB_SYSTEMMODAL, "Original", "-" & @CRLF & $sContent & "-")

_FileWriteToLine($sFile, 3, "Test rewrite", True)
FileWrite($sFile, @CRLF) ; Re-add the EOL <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

$sContent = FileRead($sFile)
MsgBox($MB_SYSTEMMODAL, "Overwritten", "-" & @CRLF & $sContent & "-")

FileDelete($sFile)

That works fine for me, how about you?

We will discuss internally whether the function needs to be amended to check whether there is a trailing EOL and replace it if it exists - but I make no promises.

M23

Edit: I have amended the thread title - I do not consider this a "bug", more a "feature" that you do not like!

Edited November 12, 2015 by Melba23

[guinness]

Good find actually!

[SpinningCone]

Thanks , i'll take a closer look at it when i get the chance (last minute changes before higher priority items)  , for the moment I have rolled to 3.2 since that's what i was using (I was slow to do updates, if it ain't broke don't fix it right?).

this is the old code from 3.2:

 

Func _FileWriteToLine($sFile, $iLine, $sText, $fOverWrite = 0)
    If $iLine <= 0 Then Return SetError(4, 0, 0)
    If Not IsString($sText) Then
        $sText = String($sText)
        If $sText = "" Then Return SetError(6, 0, 0)
    EndIf
    If $fOverWrite <> 0 And $fOverWrite <> 1 Then Return SetError(5, 0, 0)
    If Not FileExists($sFile) Then Return SetError(2, 0, 0)

    Local $sRead_File = FileRead($sFile)
    Local $aSplit_File = StringSplit(StringStripCR($sRead_File), @LF)
    If UBound($aSplit_File) < $iLine Then Return SetError(1, 0, 0)
    Local $hFile = FileOpen($sFile, $FO_OVERWRITE)
    If $hFile = -1 Then Return SetError(3, 0, 0)

    $sRead_File = ""

    For $i = 1 To $aSplit_File[0]
        If $i = $iLine Then
            If $fOverWrite = 1 Then
                If $sText <> '' Then $sRead_File &= $sText & @CRLF
            Else
                $sRead_File &= $sText & @CRLF & $aSplit_File[$i] & @CRLF
            EndIf
        ElseIf $i < $aSplit_File[0] Then
            $sRead_File &= $aSplit_File[$i] & @CRLF
        ElseIf $i = $aSplit_File[0] Then
            $sRead_File &= $aSplit_File[$i]
        EndIf
    Next

    FileWrite($hFile, $sRead_File)
    FileClose($hFile)

    Return 1
EndFunc   ;==>_FileWriteToLine

 

[Melba23]

SpinningCone,

I have the answer. The function in 3.2 that you were using splits the file into an array of lines using StringSplit - this gives an empty element in the returned array if there is a trailing EOL. The current function uses FileReadToArray which does not return an empty element at the end of the array. Hence the difference in behaviour.

As I said, we will discuss internally whether we need to modify the function.

M23

[SpinningCone]

Thanks Melba23 , for now i'll just stick with the old library instead of manually inserting EOL after each call. I'll just mark it as dependent on that version (this way i don't end up with doubles if the library changes again).  This will also help fix some errors I've been getting in another process that uses a flat file for storage.

Long term I might switch out to a sqlite db though that will take quite a bit of re-writing. or i'll write a function smart enough to correct errors in the document format to accommodate changes in library functions.

[Melba23]

SpinningCone,

I have written a new version of the function which maintains the original EOLs - we are currently discussing the finer points of implementation. Would you be interested in seeing a beta version?

M23

[SpinningCone]

Cool, yeah I can take a look at the beta and see if it exhibits the same problems.


Might even be better, part of the reason it took so long for me to suspect a real bug is that i would occasionally see EOL errors with autoit (emails not formatted right and other storage files with missing EOLs) but it was pretty rare and only really happened with a script that uses a file that's now larger than it should be.