Sign in to follow this  
Followers 0
Automan Empire

Can I Resume Filereadline After Source File Is Appended?

7 posts in this topic

I am using FileReadLine to read text from a file, line by line. I know how to read it until the end of the file. Some time later, the file gets appended several times. I would like to be able to resume reading the text, line by line, at the point where I last read it. I know that FileReadLine will accept a line number argument, but I am not sure how I would know the line number where I previously stopped (or some other method to resume using FileReadLine where I left off).

$fileid = FileOpen($file, 0)
While 1
   $readline = FileReadLine($fileid)
   If @error = -1 Then ExitLoop
  ;code to process the $readline goes here
  ;how can I keep track of the last line that was read, so that I can resume FileReadLine at the same place after the $file has been appended?
WEnd
FileClose $fileid
;later, after the $file has been appended by another process, how can I resume FileReadLine where I stopped?

Thanks for any help.


"I've seen your work in the past, and it's novice at best..." SmOke_N

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

Well the example you've provided is a bit off... But this may be what your looking for

Dim $KeepTrack = 1
While 1
    If "Condition I Create Is True" Then
        $FileOpen = FileOpen('FileLocation\Name', 0)
        While 1
            $FileReadLine = FileReadLine($FileOpen, $KeepTrack)
            If @error = -1 Then ExitLoop
    ;Do something if line is what you want
            $KeepTrack = $KeepTrack + 1
        WEnd
        FileClose($FileOpen)
    EndIf
WEnd

Edit:

Another alternative if you are using beta could be:

#include <File.Au3>
Dim $KeepTrack = 1
Dim $FilePath = 'LocationOfFile\FileName', $nArray = ''

While 1
    If "Condition I Create Is True" Then
        If _FileReadToArray($FilePath, $nArray) Then
            For $iCount = $KeepTrack To UBound($nArray) - 1
            ;Do Something If Line Is What You Want
            Next
            $KeepTrack = UBound($nArray) - 1
        EndIf
    EndIf
    Sleep(10)
WEnd
Edited by SmOke_N

[center]Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.[/center]

Share this post


Link to post
Share on other sites

Thank you very much!

I can certainly understand the counter method.


"I've seen your work in the past, and it's novice at best..." SmOke_N

Share this post


Link to post
Share on other sites

The first example you gave me that uses a counter

$KeepTrack = $KeepTrack + 1

to keep track of the number of lines read, and to use this to specifiy which line to read

$FileReadLine = FileReadLine($FileOpen, $KeepTrack)

works, but I found out that it is VERY SLOW when reading large text files.

I ran some tests and found that the optional use of the line number in

FileReadLine ( filehandle or "filename" [, line] )

is MUCH SLOWER than just using

FileReadLine ( filehandle or "filename") in a loop.

So, I am still looking for a solution that allows me to resume a FileReadLine on a text file after it has been appended, at the point where I read it last before it was appended, without being too slow.

While researching all this, I found the _FileCountLines function. So I conclude that it might be possible to obtain the number of lines in the data file, read it to the end of file, close the file, wait until another process appends the file, then open it and resume reading from the point of the last FileReadLine. At least the original file will be read using the faster method.

Any suggestions?


"I've seen your work in the past, and it's novice at best..." SmOke_N

Share this post


Link to post
Share on other sites

The first example you gave me that uses a counter

$KeepTrack = $KeepTrack + 1

to keep track of the number of lines read, and to use this to specifiy which line to read

$FileReadLine = FileReadLine($FileOpen, $KeepTrack)

works, but I found out that it is VERY SLOW when reading large text files.

I ran some tests and found that the optional use of the line number in

FileReadLine ( filehandle or "filename" [, line] )

is MUCH SLOWER than just using

FileReadLine ( filehandle or "filename") in a loop.

So, I am still looking for a solution that allows me to resume a FileReadLine on a text file after it has been appended, at the point where I read it last before it was appended, without being too slow.

While researching all this, I found the _FileCountLines function. So I conclude that it might be possible to obtain the number of lines in the data file, read it to the end of file, close the file, wait until another process appends the file, then open it and resume reading from the point of the last FileReadLine. At least the original file will be read using the faster method.

Any suggestions?

That's not going to speed anything up, FileRead() may in fact be a quicker option, or maybe you want to just FileRead() then Delete the old contents (FileOpen(FilePath, 2)) and do FileRead() again, I don't know. But for large text files, it's not going to get any faster using Native AutoIt options IMHO.

[center]Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.[/center]

Share this post


Link to post
Share on other sites

Thank you. That makes sense to me.

Your idea of reading the file using the faster

FileReadLine ( filehandle or "filename") syntax to the EOF,

then deleting the source file, and then reading it again after it is appended (in this case, created again)

will probably be faster.

Good thinking.


"I've seen your work in the past, and it's novice at best..." SmOke_N

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

Another option (I don't know your programming background so I don't know if your ready for something like that) is maybe to split the file up and multithread reading using neogia's multithread options in scripts and scraps.

Edit:

Or if your looking for a specific string to maybe even use StringRegExp() (That would be tons faster if you know what you want to find).

Edited by SmOke_N

[center]Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.[/center]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0