Sign in to follow this  
Followers 0
BrianH2

FileReadLine returns partial line?

3 posts in this topic

I have a script which reads a log file that as it is being written to by another program.

It seems that it is possible to read an incomplete record from the end of the file.

If I pause the program writing to the file, I always read complete records.

If I run both together, the reading script sometimes gets an incomplete last record.

If FileReadLine left the LF on the record I could use that to tell, but it strips these off for you.

Has anyone got a suggestion to avoid receiving a partial record or be able to tell that it was incomplete

I would rather receive an 'end-of-file' condition than the partial data.

The code logic:

$hand=FileOpen("myfile",0)  
Do
 Do
    $nline=FileReadLine($hand)
    if @error=-1 Then ExitLoop
             ..
    do some stuff with the record
             ..
 Until 0
 sleep(5000)
 if $fedup = 1 then exitloop
Until 0
fileclose($hand)

Share this post


Link to post
Share on other sites



use file count lines

#include <file.au3>
$CountLines = _FileCountLines("error.log")
MsgBox(64, "Error log  recordcount", "There are " & $CountLines & " in the  error.log.")
Exit

then if you are at the last line just exit the loop

8)


NEWHeader1.png

Share this post


Link to post
Share on other sites

use file count lines

#include <file.au3>
$CountLines = _FileCountLines("error.log")
MsgBox(64, "Error log  recordcount", "There are " & $CountLines & " in the  error.log.")
Exit

then if you are at the last line just exit the loop

8)

Thanks Valuater, I am sure that would work but it is also a potential overhead as the file grows very large.

I came up with another solution which avoids the need to repeatedly count the file.

I set up these demo scripts to prove the problem (and solution) to myself...

This script writes the file to be processed.

; This script writes a block of 10 numbered lines to a file, pauses for 5 seconds and repeats until you quit it
Global Const $xontop = 0x40000                      ; MsgBox flag top-most attribute 262144
Global Const $xsystem =  0x1000                     ; MsgBox flag System modal (dialog has an icon) 4096
Global $terminate=0
HotKeySet("{Esc}", "Terminate"); Allow ESC key to terminate the script
$outf=FileOpen("C:\myfile.txt",1)  
If $outf=-1 Then 
    MsgBox($xontop + $xsystem,"A","output file open failed")
    Exit
EndIf
$line=0
Do
    for $i = 1 to 10
        $line=$line+1
        $data=$line&@TAB&" abcdefghijklm    nop qrst    uvwxyz"&@TAB&"1234567890"
        FileWriteLine($outf,$data)
    Next
    if $terminate = 1 then exitloop
    sleep(5000)
    if $terminate = 1 then exitloop
Until 0
fileclose($outf)
Exit
Func Terminate()
; *********   Function to allow the script to be manually terminated if necessary  ****
 MsgBox($xontop + $xsystem, "Request Accepted", @scriptname&" Terminating")
 $terminate=1
EndFunc  ;==>Terminate

This next script shows the problem of reading a file being written.

; This script reads the file being written by testwc until an EOF condition, pauses 5 seconds and then continues
; It will write a partial 10th record every now and then as it reads the record currently being written.
Global Const $xontop = 0x40000                      ; MsgBox flag top-most attribute 262144
Global Const $xsystem =  0x1000                     ; MsgBox flag System modal (dialog has an icon) 4096
Global $terminate=0
HotKeySet("{Esc}", "Terminate"); Allow ESC key to terminate the script
$hand=FileOpen("C:\myfile.txt",0)  
If $hand=-1 Then 
    MsgBox($xontop + $xsystem,"A","input file open failed")
    Exit
EndIf
$outf=FileOpen("C:\mynewfile.txt",1)
If $hand=-1 Then 
    MsgBox($xontop + $xsystem,"B", "output file open failed")
    Exit
EndIf
Do
    Do
        $nline=FileReadLine($hand)
        if @error=-1 Then ExitLoop
        if @error=1 Then ExitLoop
        FileWriteLine($outf,$nline)
        ToolTip("out "&$nline&@error,500,0)
    Until 0
    if $terminate = 1 then exitloop
    sleep(5000)
    if $terminate = 1 then exitloop
Until 0
fileclose($hand)
fileclose($outf)
Exit
Func Terminate()
; *********   Function to allow the script to be manually terminated if necessary  ****
 MsgBox($xontop + $xsystem, "Request Accepted", @scriptname&" Terminating")
 $terminate=1
EndFunc  ;==>Terminate

This script is modified to avoid processing the last (potentially partial) record

; This script reads the file being written by testwc until an EOF condition, pauses 5 seconds and then continues
; It does not suffer the problem of partial reads as it avoids processing the last record of the file.
Global Const $xontop = 0x40000                      ; MsgBox flag top-most attribute 262144
Global Const $xsystem =  0x1000                     ; MsgBox flag System modal (dialog has an icon) 4096
Global $terminate=0
HotKeySet("{Esc}", "Terminate"); Allow ESC key to terminate the script
$hand=FileOpen("C:\myfile.txt",0)  
If $hand=-1 Then 
    MsgBox($xontop + $xsystem,"A","input file open failed")
    Exit
EndIf
$outf=FileOpen("C:\mynewfile2.txt",1)
If $hand=-1 Then 
    MsgBox($xontop + $xsystem,"B", "output file open failed")
    Exit
EndIf
$lineprocessed=0
Do
    Do
        $nline=FileReadLine($hand,$lineprocessed+1)
        if @error=-1 Then ExitLoop
        if @error=1 Then ExitLoop
        Do
            $pline=FileReadLine($hand)
            if @error=-1 Then ExitLoop 2
            if @error=1 Then ExitLoop 2
            ; nline was not the last line so OK to process
            FileWriteLine($outf,$nline)
            ToolTip("out "&$nline&@error,500,20)
            $lineprocessed=$lineprocessed+1
            $nline=$pline
        Until 0
    Until 0
    if $terminate = 1 then exitloop
    sleep(5000)
    if $terminate = 1 then exitloop
Until 0
fileclose($hand)
fileclose($outf)
Exit
Func Terminate()
; *********   Function to allow the script to be manually terminated if necessary  ****
 MsgBox($xontop + $xsystem, "Request Accepted", @scriptname&" Terminating")
 $terminate=1
EndFunc  ;==>Terminate

regards

Brian.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0