SpinningCone

Problem with _FileWriteToLine ? help needed

8 posts in this topic

#1 ·  Posted (edited)

FOr a while now i've been running a script that uses a flat text file to keep track of users to perform actions on.

 

I have some logic in the block that adds new users with some comma delimited fields with FileWriteLine and then updates that line with _FileWriteToLine

this has been running fine for years but suddenly i'm having problems with the script crashing because when the very last line of the file is losing it's CRLF so the next appended line concatenates with the last line and the parser fails to parse the last line.

this has been driving me nuts for a while so i finally took apart all my code and have discovered that for some reason _FileWriteToLine deletes CRLF off the bottom of the file after use. i don't know why this started suddenly, i even copied an old file.au3 library and it still does it (maybe a change to shell32.dll? )

I was able to replicate the bug by making a new file with each line having CRLF at the end then  with the following code:
 

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_UseX64=y
#AutoIt3Wrapper_Res_requestedExecutionLevel=asInvoker
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****

_FileWriteToLine($FilePath,12,"Test rewrite",1)

After running the code and refreshing the file i can see that the last blank line disappears (no final CRLF)

Is there a way to prevent this behavior? (without a huge logic rewrite to use a DB or another write method)

EDIT: actually it seems that replacing the old au3 is working, or at least working some of the time. not sure if it's a *sometimes* bug with the older library.

Edited by Melba23
Amended title

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

SpinningCone,

I can confirm that the current function does indeed strip a trailing @CRLF from the final line - as does the new version that I have recently written to allow you to add lines to make up the count if required. The only earlier version I have to hand (3.3.8.1) also strips a trailing EOL, so this is not something that has happened recently - checking the SVN logs shows no obvious change to alter the function to do this.

However, as we now know this happens, it is not difficult to code a workaround when there is a need to replace the EOL:

#include <MsgBoxConstants.au3>
#include <File.au3>

$sFile = "Test.txt"

$hFile = FileOpen($sFile, $FO_OVERWRITE)
ConsoleWrite(@error & @CRLF)

$sText = "Line 1" & @CRLF & "Line 2" & @CRLF & "Line 3" & @CRLF & "Line 4" & @CRLF & "Line 5" & @CRLF

$iRet = FileWrite($hFile, $sText)
ConsoleWrite($iRet & @CRLF)

FileClose($hFile)

$sContent = FileRead($sFile)
MsgBox($MB_SYSTEMMODAL, "Original", "-" & @CRLF & $sContent & "-")

_FileWriteToLine($sFile, 3, "Test rewrite", True)
FileWrite($sFile, @CRLF) ; Re-add the EOL <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

$sContent = FileRead($sFile)
MsgBox($MB_SYSTEMMODAL, "Overwritten", "-" & @CRLF & $sContent & "-")

FileDelete($sFile)

That works fine for me, how about you?

We will discuss internally whether the function needs to be amended to check whether there is a trailing EOL and replace it if it exists - but I make no promises.

M23

Edit: I have amended the thread title - I do not consider this a "bug", more a "feature" that you do not like!

Edited by Melba23

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Good find actually!


_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 04/09/2015

Share this post


Link to post
Share on other sites

Thanks , i'll take a closer look at it when i get the chance (last minute changes before higher priority items)  , for the moment I have rolled to 3.2 since that's what i was using (I was slow to do updates, if it ain't broke don't fix it right?).

this is the old code from 3.2:

 

Func _FileWriteToLine($sFile, $iLine, $sText, $fOverWrite = 0)
    If $iLine <= 0 Then Return SetError(4, 0, 0)
    If Not IsString($sText) Then
        $sText = String($sText)
        If $sText = "" Then Return SetError(6, 0, 0)
    EndIf
    If $fOverWrite <> 0 And $fOverWrite <> 1 Then Return SetError(5, 0, 0)
    If Not FileExists($sFile) Then Return SetError(2, 0, 0)

    Local $sRead_File = FileRead($sFile)
    Local $aSplit_File = StringSplit(StringStripCR($sRead_File), @LF)
    If UBound($aSplit_File) < $iLine Then Return SetError(1, 0, 0)
    Local $hFile = FileOpen($sFile, $FO_OVERWRITE)
    If $hFile = -1 Then Return SetError(3, 0, 0)

    $sRead_File = ""

    For $i = 1 To $aSplit_File[0]
        If $i = $iLine Then
            If $fOverWrite = 1 Then
                If $sText <> '' Then $sRead_File &= $sText & @CRLF
            Else
                $sRead_File &= $sText & @CRLF & $aSplit_File[$i] & @CRLF
            EndIf
        ElseIf $i < $aSplit_File[0] Then
            $sRead_File &= $aSplit_File[$i] & @CRLF
        ElseIf $i = $aSplit_File[0] Then
            $sRead_File &= $aSplit_File[$i]
        EndIf
    Next

    FileWrite($hFile, $sRead_File)
    FileClose($hFile)

    Return 1
EndFunc   ;==>_FileWriteToLine

 

Share this post


Link to post
Share on other sites

SpinningCone,

I have the answer. The function in 3.2 that you were using splits the file into an array of lines using StringSplit - this gives an empty element in the returned array if there is a trailing EOL. The current function uses FileReadToArray which does not return an empty element at the end of the array. Hence the difference in behaviour.

As I said, we will discuss internally whether we need to modify the function.

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Thanks Melba23 , for now i'll just stick with the old library instead of manually inserting EOL after each call. I'll just mark it as dependent on that version (this way i don't end up with doubles if the library changes again).  This will also help fix some errors I've been getting in another process that uses a flat file for storage.

Long term I might switch out to a sqlite db though that will take quite a bit of re-writing. or i'll write a function smart enough to correct errors in the document format to accommodate changes in library functions.

Share this post


Link to post
Share on other sites

SpinningCone,

I have written a new version of the function which maintains the original EOLs - we are currently discussing the finer points of implementation. Would you be interested in seeing a beta version?

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Cool, yeah I can take a look at the beta and see if it exhibits the same problems.


Might even be better, part of the reason it took so long for me to suspect a real bug is that i would occasionally see EOL errors with autoit (emails not formatted right and other storage files with missing EOLs) but it was pretty rare and only really happened with a script that uses a file that's now larger than it should be.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • SenseEye
      By SenseEye
      Hello All, 
      I am having some issues with the below script. I want to take a value from an input box and write it to a text file (stored in the same location named Config.txt).
       
      Results: script runs however the text file does not get populated with the result from the input. Any help would be greatly appreciated. 
      #include <File.au3> Global $sIniFile = @ScriptDir & "\" & "Config.txt" MsgBox(1,"",$sIniFile) ;Test to see if $sIniFile is working - Comes back as expected  Global $script1 = InputBox("Window Info", "Enter the name of your program identified using Auto Window Info") _FileWriteToLine($sIniFile, 1, $script1, 1)  
    • monter
      By monter
      Hello, I just discovered bad thing. If I used _FileWriteToLine function in AutoIt3 beta since 3.3.9.20 and newer, after launching the file it's partially damaged itself. Older versions work correctly and there is no file damaging.
      Look at the example and test on different versions - older and newer.
       
      (Added): I think the problem is in IncludeFile.au3
      v.3.3.9.20, line 677:
      Local $aArray = StringRegExp(@CRLF & $sFileRead & @CRLF, "(*BSR_ANYCRLF)([^\R]*)(?:\R)(?:\R$)?", 3) v.3.3.9.19, line 683:
      Local $aArray = StringRegExp(@CRLF & $sFileRead & @CRLF, "([^\r\n]*)(?:\r\n|\n|\r)(?:[\r\n]$)?", 3) Maybe the pipe "|" I used is the reason?
      #AutoIt3Wrapper_Res_Field=Release date|18.10.2013 partypooper, MrCreatoR?
       
      P.S. Excuse me for strange characters in line number 5.
      It's an online artifact, there is no such characters in my code.
       
      #region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_Compression=4 #AutoIt3Wrapper_Res_Comment=http://monter.FM #AutoIt3Wrapper_Res_Description=Example file showing beta bug since AutoIt3.3.9.20. #AutoIt3Wrapper_Res_Field=Release date|18.10.2013 #AutoIt3Wrapper_Res_LegalCopyright=monter.FM #endregion ;**** Directives created by AutoIt3Wrapper_GUI **** #include <Date.au3> #include <File.au3> Global $msg, $flg, $sec $title = 'AutoIt3 - version ' & @AutoItVersion FileChangeDir(@ScriptDir) If StringReplace(@AutoItVersion, '.', '') > 33919 Then VerBug() Else VerOK() EndIf FileInfo() MsgBox($flg, $title, $msg, $sec) If $flg = 48 Then Run('notepad.exe ' & @ScriptName) Func VerBug() $msg = 'This version has just damaged this file.' & @CRLF & 'Fortunately I made the *.bak copy :)' & @CRLF & 'Look at this damaged script now.' $flg = 48 $sec = 6 FileCopy(@ScriptName, @ScriptName & '.bak') EndFunc ;==>VerBug Func VerOK() $msg = "This version works fine and shouldn't damage this file." $flg = 64 $sec = 3 EndFunc ;==>VerOK Func FileInfo() Opt('TrayIconDebug', 1) $strRes = '#AutoIt3Wrapper_' For $ln = 1 To 10 $srchRes = StringInStr(FileReadLine(@ScriptFullPath, $ln), $strRes) If $srchRes > 0 Then $lr = $ln ExitLoop EndIf Next $strRes = '#AutoIt3Wrapper_Res_Field=Release date|' For $ln = $lr To $lr + 16 $srchRes = StringInStr(FileReadLine(@ScriptFullPath, $ln), $strRes) If $srchRes > 0 Then $lnRes = $ln ExitLoop EndIf Next $rd = FileGetTime(@ScriptFullPath, 0, 0) _FileWriteToLine(@ScriptFullPath, $lnRes, '#AutoIt3Wrapper_Res_Field=Release date|' & $rd[2] & '.' & $rd[1] & '.' & $rd[0], 1) Global $dateRlse = $rd[2] & '.' & $rd[1] & '.' & $rd[0] FileSetTime(@ScriptFullPath, $rd[0] & $rd[1] & $rd[2] & $rd[3] & $rd[4] & $rd[5], 0) EndFunc ;==>FileInfo FileWriteToLine_bug_Au3.3.9.20+.au3
    • guinness
      By guinness
      I'm trying to create/see if it's possible for an alternative to _FileWriteToLine with no memory limitation.
      For example: >jchd's -_FileCountLines