Jump to content
Sign in to follow this  
MadMaxToronto

Read the last line in a text file without reading the entire file first - is it possible?

Recommended Posts

MadMaxToronto

Hello Everybody,

Is it possible to read the last line in a text file (or the last few lines) without reading the entire text file line by line first?

I am using AutoIt to monitor the output of a command line utility (robocopy.exe) and I find that after a while, as the robocopy.exe output log file grows in size, it takes longer to get an update on the current progress status of robocopy.exe. The system resources also seem to get more "stressed" as the log file grows.

I'm hoping that maybe there is a way to start reading text from a file at a specific "offset"

Thank you,

Max

Share this post


Link to post
Share on other sites
youknowwho4eva

FileReadLine ( filehandle or "filename" [, line] )

Parameters

filehandle The handle of a file, as returned by a previous call to FileOpen. Alternatively you may use a string filename as the first parameter.

line [optional] The line number to read. The first line of a text file is line 1 (not zero), last line is -1.

^ Doesn't that do the trick?


Giggity

Share this post


Link to post
Share on other sites
Manko

Hi!

After opening file with:

_WinAPI_CreateFile

I'd go with these windows APIs:

SetFilePointer

ReadFile

What I'd do is record size of logfile, if it gets larger, i would set filepointer to old size and read the "line".

You ofcourse have to strip off any @CR/@LFs...

Read new size and wait to repeat...

DAMN!!! There was a simple native autoit solution... Well if it is fast, go with it! :D

[EDIT: Better answers? :) ]

/Manko

Edited by Manko

Yes i rush things! (I sorta do small bursts inbetween doing nothing.) Things I have rushed and reRushed:* ProDLLer - Process manager - Unload viri modules (dll) and moore...* _WinAPI_ProcessListOWNER_WTS() - Get Processes owner list...* _WinAPI_GetCommandLineFromPID() - Get commandline of target process...* _WinAPI_ThreadsnProcesses() Much info if expanded - optional Indented "Parent/Child"-style Processlist. Moore to come... eventually...

Share this post


Link to post
Share on other sites
Valuater

FileReadLine ( filehandle or "filename" [, line] )

Parameters

filehandle The handle of a file, as returned by a previous call to FileOpen. Alternatively you may use a string filename as the first parameter.

line [optional] The line number to read. The first line of a text file is line 1 (not zero), last line is -1.

^ Doesn't that do the trick?

Yes, the -1 will read the last line... ( I forgot that )

The stuff I found is older approaches to the same result!!

8)


NEWHeader1.png

Share this post


Link to post
Share on other sites
GEOSoft

There is no slower method to read a file than FileReadLine.

$sFile = "WordList2.txt"
$sHold = ""
;;  Figure out how to put the following lines in a loop -- While ProcessExists("process.exe") maybe?
$sStr = FileRead($sFile)
$aStr = StringRegExp($sStr, "(?i)(?s).+(\n.+?)(?:\v|$)", 1)
$sRtn = $aStr[0]
If $sRtn <> $sHold Then MsgBox(0, "Result", $sRtn)
$sHold = $sRtn
Edited by GEOSoft

George

Question about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.

Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.***

The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number.

Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else.

"Old age and treachery will always overcome youth and skill!"

Share this post


Link to post
Share on other sites
MadMaxToronto

You guys are great!!!

I should have read the FileReadLine function reference more carefully. I will implement the -1 option and see how it goes.

Thanks everybody!

Share this post


Link to post
Share on other sites
trancexx

Wouldn't any of those methods read the whole file before anything?

If would, try this:

#include <WinAPI.au3>

Local $sFile = @ScriptFullPath ;"ThatFile.txt" ; whatever
Local $hFile = _WinAPI_CreateFile($sFile, 2, 2) ; open for reading

Local $iDesiredSize = 188 ; read 188 characters for example

_WinAPI_SetFilePointer($hFile, FileGetSize($sFile) - $iDesiredSize) ; move the file pointer to 188 chars before end

Local $tBuffer = DllStructCreate("char[" & $iDesiredSize & "]") ; will read to this buffer
Local $iRead ; use it to verify read amount
;!->
_WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $iDesiredSize, $iRead)
_WinAPI_CloseHandle($hFile)

Local $sRead = DllStructGetData($tBuffer, 1)

ConsoleWrite($sRead & @CRLF)

♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites
Manko

Wouldn't any of those methods read the whole file before anything?

If would, try this:

#include <WinAPI.au3>

Local $sFile = @ScriptFullPath ;"ThatFile.txt" ; whatever
Local $hFile = _WinAPI_CreateFile($sFile, 2, 2) ; open for reading

Local $iDesiredSize = 188 ; read 188 characters for example

_WinAPI_SetFilePointer($hFile, FileGetSize($sFile) - $iDesiredSize) ; move the file pointer to 188 chars before end

Local $tBuffer = DllStructCreate("char[" & $iDesiredSize & "]") ; will read to this buffer
Local $iRead ; use it to verify read amount
;!->
_WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $iDesiredSize, $iRead)
_WinAPI_CloseHandle($hFile)

Local $sRead = DllStructGetData($tBuffer, 1)

ConsoleWrite($sRead & @CRLF)
Good of you to post example code of my proposal! :D

/Manko


Yes i rush things! (I sorta do small bursts inbetween doing nothing.) Things I have rushed and reRushed:* ProDLLer - Process manager - Unload viri modules (dll) and moore...* _WinAPI_ProcessListOWNER_WTS() - Get Processes owner list...* _WinAPI_GetCommandLineFromPID() - Get commandline of target process...* _WinAPI_ThreadsnProcesses() Much info if expanded - optional Indented "Parent/Child"-style Processlist. Moore to come... eventually...

Share this post


Link to post
Share on other sites
trancexx

Good of you to post example code of my proposal! :D

/Manko

Yeah, I wouldn' want you to wear out your keyboard.

Btw, FileReadLine($hFile, -1) method is not increasing RAM usage but if the file is some big one CPU will go wild for a moment. (... maybe it happens too fast to detect RAM part)

edit:

It would be smart to make that string null-terminated so just increase the size of $tBuffer by 1.

Edited by trancexx

♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites
GEOSoft

For future reference here are the test results I just got between Using FileReadLine with the -1 param and the regexp that I posted, This is on a text file with 88,711 one word lines (wordlist2.txt)

FileReadLine -1 = 167.133908449251

RegExp = 58.9924365377471

Edited by GEOSoft

George

Question about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.

Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.***

The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number.

Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else.

"Old age and treachery will always overcome youth and skill!"

Share this post


Link to post
Share on other sites
rajeshontheweb

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×