Sign in to follow this  
Followers 0
Grantson

Stripping text from a string

9 posts in this topic

Hi Guys

Im working on a script to take plain text from a log file and convert it to a csv file

the logs look like this

TEST NUMBER   0001
DATE     11-OCT-10
APP NO        0001
TEST MODE     TEST
VISUAL CHECK     P
EARTH CURRENT   4A
EARTH        OHM S
INS  >99.99  MEG P
USER         GRANT
TEXT SHREDDER
TEXT 
TEXT 
TEXT 

TEST NUMBER   0002
DATE     20-OCT-10
APP NO     9334005
TEST MODE     TEST
VISUAL CHECK     P
EARTH CURRENT 100~
EARTH 00.04  OHM P
INS  >99.99  MEG P
USER         GRANT
TEXT PROJECTOR
TEXT FUSE OK
TEXT CLASSRM 1
TEXT 1 FLOOR

Currently I use filereadline to read the log a line at a time

what i then wanted to do was strip out the field names leaving the relevent info from each line and pass it into an array.

What Im wondering is rather than using stringtrimleft and having to know the precice number of chars each time is there a way i can tell it to trim off everything before the double space as multiple spaces seem to be the only token

Thanks

Grant

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

Hi Guys

Im working on a script to take plain text from a log file and convert it to a csv file

the logs look like this

TEST NUMBER   0001
DATE     11-OCT-10
APP NO        0001
TEST MODE     TEST
VISUAL CHECK     P
EARTH CURRENT   4A
EARTH        OHM S
INS  >99.99  MEG P
USER         GRANT
TEXT SHREDDER
TEXT 
TEXT 
TEXT 

TEST NUMBER   0002
DATE     20-OCT-10
APP NO     9334005
TEST MODE     TEST
VISUAL CHECK     P
EARTH CURRENT 100~
EARTH 00.04  OHM P
INS  >99.99  MEG P
USER         GRANT
TEXT PROJECTOR
TEXT FUSE OK
TEXT CLASSRM 1
TEXT 1 FLOOR

Currently I use filereadline to read the log a line at a time

what i then wanted to do was strip out the field names leaving the relevent info from each line and pass it into an array.

What Im wondering is rather than using stringtrimleft and having to know the precice number of chars each time is there a way i can tell it to trim off everything before the double space as multiple spaces seem to be the only token

Thanks

Grant

Hello !

If the log file is using some kind of static structure, you may exploit it.

1. Put each line title (TEST NUMBER, DATE etc.) into an array

2. Read one block of the log at a time(FileReadLine, 13 times)

3. Strip out the title (StringReplace) depending on line number read, use the line number to determine which title to strip from the 'title array'.

4. Strip out beginning and ending spaces of the remaining string (StringStripWS)

There's probably more clever way existing but that's one solution !!

Good luck,

hench

Edit : Clarification

Edited by hench

Share this post


Link to post
Share on other sites

TEST NUMBER 0001
DATE 11-OCT-10
APP NO 0001
TEST MODE TEST
VISUAL CHECK P
EARTH CURRENT 4A
EARTH OHM S
INS >99.99 MEG P
USER GRANT
TEXT SHREDDER
TEXT 
TEXT 
TEXT 

TEST NUMBER 0002
DATE 20-OCT-10
APP NO 9334005
TEST MODE TEST
VISUAL CHECK P
EARTH CURRENT 100~
EARTH 00.04 OHM P
INS >99.99 MEG P
USER GRANT
TEXT PROJECTOR
TEXT FUSE OK
TEXT CLASSRM 1
TEXT 1 FLOOR

what i then wanted to do was strip out the field names leaving the relevent info from each line and pass it into an array.

...

Can you give an example what do you mean exactly on the basis of your text file in the code box? Which information are relevant?

Br,

UEZ


Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

Can you give an example what do you mean exactly on the basis of your text file in the code box? Which information are relevant?

Br,

UEZ

Hi UEZ

the bits im looking to strip off are the field names (underlined below)

TEST NUMBER 0002

DATE 20-OCT-10

APP NO 9334005

TEST MODE TEST

VISUAL CHECK P

EARTH CURRENT 100~

EARTH 00.04 OHM P

INS >99.99 MEG P

USER GRANT

TEXT PROJECTOR

TEXT FUSE OK

TEXT CLASSRM 1

TEXT 1 FLOOR

thanks hench ill give that a try

I think that is probably a good idea as i hope to eventually integrate downloading the logs directly from the tester over serial port

;)

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

There is no consistancy that I can see to the multiple spaces so I'm not sure exacly what you want returned from your example. It can easily be handled with a regular expression if I know what you actually expect to be returned.

Edited by GEOSoft

George

Question about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.

Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.***

The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number.

Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else.

"Old age and treachery will always overcome youth and skill!"

Share this post


Link to post
Share on other sites

Hmm, I cannot see the logic behind your underlined field names! Sometimes only one is underlined, then 2 or 3! ;)

Maybe you can explain...

Br,

UEZ


Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

 

$content = FileRead("logfile.log")
$line_array = StringSplit($content, @CRLF, 3) ;0-based, CRLF is delimiter
Dim $csv_array[13] ;if you need the data in an array
$csv_line = "" ;complete line csv, CRLF terminated
For $i = 0 To UBound($line_array) - 14 Step 14 ;13 lines are 1 "block" od data
    For $x = 0 To 5 ;first 6 lines in block
        if $x=4 then continueloop
        $csv_array[$x] = StringTrimLeft($line_array[$i + $x], StringInStr($line_array[$i + $x], " ", 1, -1)) ;trimm all up to the last blank
        $csv_line &= $csv_array[$x] & "," ;csvdata
    Next
    For $x = 6 To 7 ;earth and ins
        $csv_array[$x] = StringTrimLeft($line_array[$i + $x], StringInStr($line_array[$i + $x], " ")) ;trimm all up to the first blank
        $csv_line &= $csv_array[$x] & "," ;csvdata
    Next
    For $x = 9 To 12 ;text
        $csv_array[$x] = StringTrimLeft($line_array[$i + $x], StringInStr($line_array[$i + $x], " ")) ;trimm all up to the first blank
        $csv_line &= $csv_array[$x] & "," ;csvdata
    Next
    $csv_line &= @CRLF   ;delete line if necessary
Next
MsgBox(0, "CSV-Data", $csv_line)
$csv_file = FileOpen("csvfile.csv", 1);append csv-data
FileWrite($csv_file, $csv_line) ;appends new data to existing csv (delete log recommended!)
/edit/ i think he wants to have every data excluded  the underlined separated by comma 

Edited by AndyG

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

Here my version to convert the log file to a 2D array:

#include <Array.au3>
$hFile = FileOpen("Test.log", 0)
Local $aSplit, $aLog[1][4]
While 1
    $line = FileReadLine($hFile)
    If @error = -1 Then ExitLoop
    If $line <> "" Then $aSplit = StringSplit(StringStripWS($line, 7), " ", 2)
    Fill_Array($aSplit, $aLog)
Wend
FileClose($hFile)
ReDim $aLog[UBound($aLog) -1][4]
_ArrayDisplay($aLog)
Exit

Func Fill_Array($sArray, ByRef $dArray)
    Local $i, $u = UBound($dArray)
    For $i = 0 To UBound($sArray) - 1
        $dArray[$u -1][$i] = $sArray[$i]
    Next
    ReDim $aLog[$u + 1][4]
EndFunc

You can traverse the array, extract the relevant information and save it finally to a csv.

Br,

UEZ

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

Here is another interpretation of what's required.

Local $sStr = "TEST NUMBER 0001" & @CRLF & _
        "DATE   11-OCT-10" & @CRLF & _
        "APP NO     0001" & @CRLF & _
        "TEST MODE  TEST" & @CRLF & _
        "VISUAL CHECK   P" & @CRLF & _
        "EARTH CURRENT 4A" & @CRLF & _
        "EARTH  OHM S" & @CRLF & _
        "INS >99.99 MEG P" & @CRLF & _
        "USER   GRANT" & @CRLF & _
        "TEXT SHREDDER" & @CRLF & _
        "TEXT " & @CRLF & _
        "TEXT " & @CRLF & _
        "TEXT " & @CRLF & @CRLF & _
        "TEST NUMBER 0002" & @CRLF & _
        "DATE   20-OCT-10" & @CRLF & _
        "APP NO     9334005" & @CRLF & _
        "TEST MODE  TEST" & @CRLF & _
        "VISUAL CHECK   P" & @CRLF & _
        "EARTH CURRENT 100~" & @CRLF & _
        "EARTH 00.04 OHM P" & @CRLF & _
        "INS >99.99 MEG P" & @CRLF & _
        "USER   GRANT" & @CRLF & _
        "TEXT PROJECTOR" & @CRLF & _
        "TEXT FUSE OK" & @CRLF & _
        "TEXT CLASSRM 1" & @CRLF & _
        "TEXT 1 FLOOR" & @CRLF

; or
;Local $sStr = FileRead("logfile.log")

Local $sCSV = StringRegExpReplace($sStr & @CRLF, _
        "TEST NUMBER\h*(\d+)\v+" & _
        "DATE\h*([0-9A-Z\-]*)\v+" & _
        "APP NO\h*(\d*)\v+" & _
        "TEST MODE\h*(\w*)\v+" & _
        "VISUAL CHECK\h*(\w*)\v+" & _
        "EARTH CURRENT\h*([\w~]*)\v+" & _
        "EARTH\h*([ \.\w]*)\v+" & _
        "INS\h*([ \.\w>]*)\v+" & _
        "USER\h*([ \w]*)\v+" & _
        "TEXT\h*([ \w]*)\v+" & _
        "TEXT\h*([ \w]*)\v+" & _
        "TEXT\h*([ \w]*)\v+" & _
        "TEXT\h*([ \w]*)\v+" & _
        "", "\1,\2,\3,\4,\5,\6,\7,\8,\9,\10,\11,\12,\13" & @CRLF)

; Add column header labels
$sCSV = "TEST NUMBER,DATE,APP NO,TEST MODE,VISUAL CHECK,EARTH CURRENT,EARTH,INS,USER,TEXT,TEXT,TEXT,TEXT" & @CRLF & StringTrimRight($sCSV, 2) ; Remove trailing @CRLF
ConsoleWrite($sCSV & @CRLF)

;== Save to csv file ==
Local $csv_file = FileOpen("csvfile.csv", 10) ; 2+8=10 [2 = Write mode (erase previous contents) & 8 = Create directory structure if it doesn't exist.]
FileWrite($csv_file, $sCSV)
FileClose($csv_file)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0