Sign in to follow this  
Followers 0
Hypertrophy

best way to parse through or datamine files

25 posts in this topic

Ok, so I've been having no luck with this and been working on it for hours. >_< What I'm trying to do is read a history log file (that is being updated every so many seconds) and then compare the data to my own .ini file. If it matches take a certain action Let's say my .ini file looks like this:

[Section]
joe=tight
bob=shark

What I am trying to do is check the log file to see if joe is in it, then find what seat he is sitting in. Now the problem is since it's constantly being updated there may be 50 entries of his Seat #. This could obviously change and my program would go a function numerous times because it sees 50 entries of the Seat # joe is sitting at. So in the end it ended up creating 50 different labels in several different spots because joe moved Seat #'s at the table. I'm very frustrated and don't know the logic on how to go about doing this. Maybe start reading from the end of the file, like read the last occurrence of where he was sitting? Please someone help I'm very lost and would be extremely grateful. Please tell me anything you don't understand.

Share this post


Link to post
Share on other sites



Can you post a few lines of the log content?

Share this post


Link to post
Share on other sites

You can start by reading the log file first time entirely. Once it reaches the last line, keep that line number in a variable.

Next time when reading the log, read it from the line number you kept in that variable to the end, once the end is reached, update your variable with the new number ... rinse and repeat ...

Good luck,


SNMP_UDF ... for SNMPv1 and v2c so far, GetBulk and a new example script

wannabe "Unbeatable" Tic-Tac-Toe

Paper-Scissor-Rock ... try to beat it anyway :)

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

Can you post a few lines of the log content?

edit: removed

Edited by Hypertrophy

Share this post


Link to post
Share on other sites

You can start by reading the log file first time entirely. Once it reaches the last line, keep that line number in a variable.

Next time when reading the log, read it from the line number you kept in that variable to the end, once the end is reached, update your variable with the new number ... rinse and repeat ...

Good luck,

thats sounds like a good idea. what command do i use to know when it reaches the last line in the file?

Share this post


Link to post
Share on other sites

thats sounds like a good idea. what command do i use to know when it reaches the last line in the file?

Look at _FileCountLines($sFilePath)

Share this post


Link to post
Share on other sites

I'm having trouble understanding the logic. Would I first read the file then set $counted = True and then from that point on it would only count the lines in the file?

Share this post


Link to post
Share on other sites

It can be slow if the file gets constantly updated (and is also a big file). Try to open the file in share read mode, look at the _WinAPI_CreateFile() function in the help file for the necessary details. If the file was opened using share read mode, you can only read and must specify this share access mode when you open the file, otherwise it'll fail. Try a few combinations of share mode like read-write, either or all (i.e. read-write-execute) if you haven't managed to get a valid handle to the file, drop this approach and try the line counter approach.

Share this post


Link to post
Share on other sites

Authenticity: I have no problem getting a valid handle to the file and reading data in. It's the process of doing it that I'm still working on.

Share this post


Link to post
Share on other sites

This piece of code will give you the idea:

#Include <File.au3>

$file = "your file here"
$LastLine_nbr = _FileCountLines($file)      ;use this first time to get the last line number

;when you read it after that to check for changes

For $i = $LastLine_nbr To _FileCountLines($file)
    ;check for changes ... your own code here
Next
$LastLine_nbr = _FileCountLines($file)      ;update the value of the last line

Authenticity is right, when yoy have to deal with a big file which is updated often the above method will be slow, but if it's not the case then you can use my code to do it.


SNMP_UDF ... for SNMPv1 and v2c so far, GetBulk and a new example script

wannabe "Unbeatable" Tic-Tac-Toe

Paper-Scissor-Rock ... try to beat it anyway :)

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

do you have any idea on where i should put that in my loop? and the file is updated about every minute.

Edited by Hypertrophy

Share this post


Link to post
Share on other sites

#12 ·  Posted (edited)

YOU should have the idea not me >_<

I don't usually debug other's code (especially when the code hasn't been checked by the owner ... I had to spend some time to close your For, If's and Func) but today I've made an exception.

EDIT: code removes as OP requested

Edited by enaiman

SNMP_UDF ... for SNMPv1 and v2c so far, GetBulk and a new example script

wannabe "Unbeatable" Tic-Tac-Toe

Paper-Scissor-Rock ... try to beat it anyway :)

Share this post


Link to post
Share on other sites

I really appreciate your help. It does not go unnoticed. BTW, did you use a function in Scite to close the Fors,If,and Func?

Share this post


Link to post
Share on other sites

yep ... the function is called "myGoodWill()"

Just kidding - I'll quote myself:

I had to spend some time to close your ...

I had to do it "by hand"


SNMP_UDF ... for SNMPv1 and v2c so far, GetBulk and a new example script

wannabe "Unbeatable" Tic-Tac-Toe

Paper-Scissor-Rock ... try to beat it anyway :)

Share this post


Link to post
Share on other sites

I'm looking at your code you fixed for me and noticed there is no _FileCountLines(). did you decide to use the _FileReadToArray()? thats what it looks like...sorry for newb question.

Share this post


Link to post
Share on other sites

You were using it so I've decided to go your way.


SNMP_UDF ... for SNMPv1 and v2c so far, GetBulk and a new example script

wannabe "Unbeatable" Tic-Tac-Toe

Paper-Scissor-Rock ... try to beat it anyway :)

Share this post


Link to post
Share on other sites

But thats what's been causing all my problems? >_< I thought the reason to use _FileCountLines() was to fix it...

Share this post


Link to post
Share on other sites

When I implemented your code it went to the else statement for the If IsArray($aSection)?

Share this post


Link to post
Share on other sites

#19 ·  Posted (edited)

Both ways are working the same, there is no way that _FileReadToArray was causing your problems.

Other than that - do yourself some debugging. The function I've modified for you is supposed to read everytime the new added part to your text file (that's what $begin variable is used for; to store the last line number at the end of function call and to make it available at the next function call) and to do whatever you wanted (it is your code). If it doesn't then you have to work on it. It is you having the files and the whole code.

IMO I've spent enough time today working on this.

Edited by enaiman

SNMP_UDF ... for SNMPv1 and v2c so far, GetBulk and a new example script

wannabe "Unbeatable" Tic-Tac-Toe

Paper-Scissor-Rock ... try to beat it anyway :)

Share this post


Link to post
Share on other sites

#20 ·  Posted (edited)

i've posted all my code related to this issue. but here's an example of the file its reading from:

thanks enaiman for the help you've given me.

Edit: text file removed

Edited by Hypertrophy

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0