Work with new lines as they are added to a log file

[qaguy]

Hi,

Is there any way to monitor a file and keep sending the last line to a variable?

I can't afford reading the entire file each time, I need to match some log lines and if two events happen at the same time, launch a ControlClick and so on.

Thanks in advance.

Edited March 19, 2010 by qaguy

[JohnOne]

I'm sure there is yes ?

But you're going to need to provide some code you have tried.

[Richard Robertson]

In a loop you could check the size of the file. If the size has changed, open the file for reading and then FileSetPos($old_length) and then read lines until it ends. Close the file when you're done and save the new size.

[qaguy]

Thanks!

So far I have this.

ParseFile("mylog.log")

Func ParseFile($file_to_parse)

    $file_size = FileGetSize($file_to_parse)
    $file_handle = FileOpen($file_to_parse, 0)
    FileSetPos($file_handle, 0, 2)
    
    While 1 
        $file_pos = FileGetPos($file_handle)
        $line = FileReadLine($file_handle)
        If $line <> -1 Then         
            Sleep(5)
            FileSetPos($file_handle, 0, $file_pos)
        Else
            ConsoleWrite($line)
        EndIf
    WEnd
EndFunc

I think it's related to what you are telling me to do Richard.

I just translated pseudocode to autoit but it doesn't work.

Need more coffee...

Edited March 19, 2010 by qaguy

[qaguy]

Ok, the problem is that I can't read a file that is being in use by another app.

FileOpen() always returns -1

I can't find a workaround with FileOpen(), is this possible with autoit?

[Richard Robertson]

I'm glad you're trying, but not quite what I said. Something like this.

expandcollapse popup

Global $my_log_file = "mylog.log"
Global $my_old_size = 0

Global $filecontents = GetNewLines($my_log_file, $my_old_size)
MsgBox(0, "", $filecontents)

HotKeySet("{ESC}", "_Exit")

While 1
    Sleep(100)
    $filecontents = GetNewLines($my_log_file, $my_old_size)
    If $filecontents <> "" Then MsgBox(0, "", $filecontents)
WEnd

Func GetNewLines($filename, ByRef $oldsize)
    If FileGetSize($filename) == $oldsize Then Return ""
    Local $handle = FileOpen($filename, 0)
    Local $return = ""
    
    FileSetPos($handle, $oldsize, 0)
    
    Local $line = FileReadLine($handle)
    If $line == -1 Then
        FileClose($handle)
        Return ""
    EndIf
    $return = $line
    
    $line = FileReadLine($handle)
    
    While $line <> -1
        $return &= @CRLF & $line
        $line = FileReadLine($handle)
    WEnd
    
    FileClose($handle)
    $oldsize = FileGetSize($filename)
    Return $return
EndFunc

Func _Exit()
    Exit
EndFunc

This code was not tested, but I believe it should be a good start for you.

Edit: Ok I see your problem. Yeah, some programs don't let you open files they are using. Some let you in read only mode but I guess not this one.

Edited March 19, 2010 by Richard Robertson

[Spiff59]

Eww, you may have to do some sort of shadow copy to a temp file and process that...

Technically, no matter what, you're processing the entire file. It may be faster to simply count bytes to get to the area of interest, instead of reading every line into memory, but you're still processing the entire file.

If it's a huge log file, I'd be looking for ways to put it on a diet, to trim it down periodically. If it's constantly accessed by multiple users, maybe you could schedule a clean-up job during the night. If the log file is opened by a single service, maybe you could briefly stop the service, process the file, then restart the service.

Edited March 19, 2010 by Spiff59

[l3ill]

Obviously I'm a noob but what about "FileReadLine" and then use the "last line" option. Haven't tried it yet...but I might later

Edit:

Nevermind I see it now in RR's post...

Edited March 19, 2010 by billo

[qaguy]

Edit: Are we allowed to talk about "assing" in the forums here?

LOL, sorry, no doubt that the title doesn't reflect what I had in mind, who should I ask to change it?

As you may have already noticed, English is not my native language

Richard, thanks a lot, I'm going to try it and I'll be posting the results here.

[...]but you're still processing the entire file.

Yes, you're right, fortunately the size is not a problem, the log doesn't get bigger than 3-5MB

Eww, you may have to do some sort of shadow copy to a temp file and process that...

That's what worries me most, also I can't stop the service, I'm working with a SIPPhone that receives calls from an Asterisk server and killing the phone on every call would be painful.

Ok, I'm going back to work, thanks all for your time and help.

Lets see what comes out of this.

[Spiff59]

LOL, sorry, no doubt that the title doesn't reflect what I had in mind, who should I ask to change it?

As you may have already noticed, English is not my native language

If you edit your original post, then choose "Use full editor" you can change the topic and sub-topic information.

And... I think your English is quite good, I wish I had as decent a command of a "foreign" language.

[Richard Robertson]

There's nothing you can do about it. The program that owns the file opened it and did not set the read-able sharing privilege. I guess you could either hack the program (hack as a generic term) or not read the file...

[qaguy]

If you edit your original post, then choose "Use full editor" you can change the topic and sub-topic information.

Done, thanks.

I ended up using Richard script, I just changed one while loop condition:

Instead of

While $line <> -1

..

While $line = -1

did the trick

There's nothing you can do about it. The program that owns the file opened it and did not set the read-able sharing privilege. I guess you could either hack the program (hack as a generic term) or not read the file...

But wordpad ant notepad can work on it, that's what keeps me with hope.

It's possible to give FileOpen the ability to open the file and allow others to write on it at the same time?

Something like:

FileOpen(filename, FileAccess.Read, FileShare.ReadWrite)

I know that FileOpen only takes 2 arguments, filename and mode, but can I specify the fileshare? as a power of two on the read argument?

Edited March 19, 2010 by qaguy

[Richard Robertson]

Why did you change that while loop condition? Your new condition should actually cause problems. o.o

[Spiff59]

This thread may also be of interest to the OP:

Problem with func. "FileReadLine" in AutoIt Version 3.3.4.0