Read the last line in a text file without reading the entire file first - is it possible?

[MadMaxToronto]

Hello Everybody,

Is it possible to read the last line in a text file (or the last few lines) without reading the entire text file line by line first?

I am using AutoIt to monitor the output of a command line utility (robocopy.exe) and I find that after a while, as the robocopy.exe output log file grows in size, it takes longer to get an update on the current progress status of robocopy.exe. The system resources also seem to get more "stressed" as the log file grows.

I'm hoping that maybe there is a way to start reading text from a file at a specific "offset"

Thank you,

Max

[Valuater]

; Read the last line of a text file
; author - SmOke_N

$aSplit = StringSplit(StringStripCR(FileRead('FileLocation.Name')), @LF)
MsgBox(64, 'Last Line', $aSplit[$aSplit[0]])oÝ÷ Ù©Ýjëh×6$File = @ScriptDir & "\Text.txt"
StringReplace(FileRead($File), @LF, "")
FileReadLine($file, @extended)

8)

[youknowwho4eva]

FileReadLine ( filehandle or "filename" [, line] )

Parameters

filehandle The handle of a file, as returned by a previous call to FileOpen. Alternatively you may use a string filename as the first parameter.

line [optional] The line number to read. The first line of a text file is line 1 (not zero), last line is -1.

^ Doesn't that do the trick?

[Manko]

Hi!

After opening file with:

_WinAPI_CreateFile

I'd go with these windows APIs:

SetFilePointer

ReadFile

What I'd do is record size of logfile, if it gets larger, i would set filepointer to old size and read the "line".

You ofcourse have to strip off any @CR/@LFs...

Read new size and wait to repeat...

DAMN!!! There was a simple native autoit solution... Well if it is fast, go with it!

[EDIT: Better answers? ]

/Manko

Edited April 6, 2009 by Manko

[Valuater]

FileReadLine ( filehandle or "filename" [, line] )

Parameters

filehandle The handle of a file, as returned by a previous call to FileOpen. Alternatively you may use a string filename as the first parameter.

line [optional] The line number to read. The first line of a text file is line 1 (not zero), last line is -1.

^ Doesn't that do the trick?

Yes, the -1 will read the last line... ( I forgot that )

The stuff I found is older approaches to the same result!!

8)

[GEOSoft]

There is no slower method to read a file than FileReadLine.

$sFile = "WordList2.txt"
$sHold = ""
;;  Figure out how to put the following lines in a loop -- While ProcessExists("process.exe") maybe?
$sStr = FileRead($sFile)
$aStr = StringRegExp($sStr, "(?i)(?s).+(\n.+?)(?:\v|$)", 1)
$sRtn = $aStr[0]
If $sRtn <> $sHold Then MsgBox(0, "Result", $sRtn)
$sHold = $sRtn

Edited April 6, 2009 by GEOSoft

[MadMaxToronto]

You guys are great!!!

I should have read the FileReadLine function reference more carefully. I will implement the -1 option and see how it goes.

Thanks everybody!

[trancexx]

Wouldn't any of those methods read the whole file before anything?

If would, try this:

#include <WinAPI.au3>

Local $sFile = @ScriptFullPath ;"ThatFile.txt" ; whatever
Local $hFile = _WinAPI_CreateFile($sFile, 2, 2) ; open for reading

Local $iDesiredSize = 188 ; read 188 characters for example

_WinAPI_SetFilePointer($hFile, FileGetSize($sFile) - $iDesiredSize) ; move the file pointer to 188 chars before end

Local $tBuffer = DllStructCreate("char[" & $iDesiredSize & "]") ; will read to this buffer
Local $iRead ; use it to verify read amount
;!->
_WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $iDesiredSize, $iRead)
_WinAPI_CloseHandle($hFile)

Local $sRead = DllStructGetData($tBuffer, 1)

ConsoleWrite($sRead & @CRLF)

[Manko]

Wouldn't any of those methods read the whole file before anything?

If would, try this:

#include <WinAPI.au3>

Local $sFile = @ScriptFullPath ;"ThatFile.txt" ; whatever
Local $hFile = _WinAPI_CreateFile($sFile, 2, 2) ; open for reading

Local $iDesiredSize = 188 ; read 188 characters for example

_WinAPI_SetFilePointer($hFile, FileGetSize($sFile) - $iDesiredSize) ; move the file pointer to 188 chars before end

Local $tBuffer = DllStructCreate("char[" & $iDesiredSize & "]") ; will read to this buffer
Local $iRead ; use it to verify read amount
;!->
_WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $iDesiredSize, $iRead)
_WinAPI_CloseHandle($hFile)

Local $sRead = DllStructGetData($tBuffer, 1)

ConsoleWrite($sRead & @CRLF)

Good of you to post example code of my proposal!

/Manko

[trancexx]

Good of you to post example code of my proposal!

/Manko

Yeah, I wouldn' want you to wear out your keyboard.

Btw, FileReadLine($hFile, -1) method is not increasing RAM usage but if the file is some big one CPU will go wild for a moment. (... maybe it happens too fast to detect RAM part)

edit:

It would be smart to make that string null-terminated so just increase the size of $tBuffer by 1.

Edited April 6, 2009 by trancexx

[GEOSoft]

For future reference here are the test results I just got between Using FileReadLine with the -1 param and the regexp that I posted, This is on a text file with 88,711 one word lines (wordlist2.txt)

FileReadLine -1 = 167.133908449251

RegExp = 58.9924365377471

Edited April 6, 2009 by GEOSoft

[Authenticity]

$sStr = FileRead($sFile)
$iFirst = StringInStr($sStr, @LF, 0, -2)+1
$iCount = StringInStr($sStr, @LF, 0, -1) - $iFirst
$sLine = StringMid($sStr, $iFirst, $iCount)

[rajeshontheweb]

I am using AutoIt to monitor the output of a command line utility (robocopy.exe)

Why not use STDERR if u can get a console output? just a small doubt.

Also, thought this might be useful for you Robocopy ErrorCodes & Robocopy GUI from Microsoft Technet

Edited April 7, 2009 by rajeshontheweb