Grantson Posted October 20, 2010 Share Posted October 20, 2010 Hi Guys Im working on a script to take plain text from a log file and convert it to a csv file the logs look like this TEST NUMBER 0001 DATE 11-OCT-10 APP NO 0001 TEST MODE TEST VISUAL CHECK P EARTH CURRENT 4A EARTH OHM S INS >99.99 MEG P USER GRANT TEXT SHREDDER TEXT TEXT TEXT TEST NUMBER 0002 DATE 20-OCT-10 APP NO 9334005 TEST MODE TEST VISUAL CHECK P EARTH CURRENT 100~ EARTH 00.04 OHM P INS >99.99 MEG P USER GRANT TEXT PROJECTOR TEXT FUSE OK TEXT CLASSRM 1 TEXT 1 FLOOR Currently I use filereadline to read the log a line at a time what i then wanted to do was strip out the field names leaving the relevent info from each line and pass it into an array. What Im wondering is rather than using stringtrimleft and having to know the precice number of chars each time is there a way i can tell it to trim off everything before the double space as multiple spaces seem to be the only token Thanks Grant Link to comment Share on other sites More sharing options...
hench Posted October 20, 2010 Share Posted October 20, 2010 (edited) Hi Guys Im working on a script to take plain text from a log file and convert it to a csv file the logs look like this TEST NUMBER 0001 DATE 11-OCT-10 APP NO 0001 TEST MODE TEST VISUAL CHECK P EARTH CURRENT 4A EARTH OHM S INS >99.99 MEG P USER GRANT TEXT SHREDDER TEXT TEXT TEXT TEST NUMBER 0002 DATE 20-OCT-10 APP NO 9334005 TEST MODE TEST VISUAL CHECK P EARTH CURRENT 100~ EARTH 00.04 OHM P INS >99.99 MEG P USER GRANT TEXT PROJECTOR TEXT FUSE OK TEXT CLASSRM 1 TEXT 1 FLOOR Currently I use filereadline to read the log a line at a time what i then wanted to do was strip out the field names leaving the relevent info from each line and pass it into an array. What Im wondering is rather than using stringtrimleft and having to know the precice number of chars each time is there a way i can tell it to trim off everything before the double space as multiple spaces seem to be the only token Thanks Grant Hello ! If the log file is using some kind of static structure, you may exploit it. 1. Put each line title (TEST NUMBER, DATE etc.) into an array 2. Read one block of the log at a time(FileReadLine, 13 times) 3. Strip out the title (StringReplace) depending on line number read, use the line number to determine which title to strip from the 'title array'. 4. Strip out beginning and ending spaces of the remaining string (StringStripWS) There's probably more clever way existing but that's one solution !! Good luck, hench Edit : Clarification Edited October 20, 2010 by hench Link to comment Share on other sites More sharing options...
UEZ Posted October 20, 2010 Share Posted October 20, 2010 TEST NUMBER 0001 DATE 11-OCT-10 APP NO 0001 TEST MODE TEST VISUAL CHECK P EARTH CURRENT 4A EARTH OHM S INS >99.99 MEG P USER GRANT TEXT SHREDDER TEXT TEXT TEXT TEST NUMBER 0002 DATE 20-OCT-10 APP NO 9334005 TEST MODE TEST VISUAL CHECK P EARTH CURRENT 100~ EARTH 00.04 OHM P INS >99.99 MEG P USER GRANT TEXT PROJECTOR TEXT FUSE OK TEXT CLASSRM 1 TEXT 1 FLOOR what i then wanted to do was strip out the field names leaving the relevent info from each line and pass it into an array. ... Can you give an example what do you mean exactly on the basis of your text file in the code box? Which information are relevant? Br, UEZ Please don't send me any personal message and ask for support! I will not reply! Selection of finest graphical examples at Codepen.io The own fart smells best! ✌Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!¯\_(ツ)_/¯ ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ Link to comment Share on other sites More sharing options...
Grantson Posted October 20, 2010 Author Share Posted October 20, 2010 Can you give an example what do you mean exactly on the basis of your text file in the code box? Which information are relevant? Br, UEZ Hi UEZ the bits im looking to strip off are the field names (underlined below) TEST NUMBER 0002 DATE 20-OCT-10 APP NO 9334005 TEST MODE TEST VISUAL CHECK P EARTH CURRENT 100~ EARTH 00.04 OHM P INS >99.99 MEG P USER GRANT TEXT PROJECTOR TEXT FUSE OK TEXT CLASSRM 1 TEXT 1 FLOOR thanks hench ill give that a try I think that is probably a good idea as i hope to eventually integrate downloading the logs directly from the tester over serial port Link to comment Share on other sites More sharing options...
GEOSoft Posted October 20, 2010 Share Posted October 20, 2010 (edited) There is no consistancy that I can see to the multiple spaces so I'm not sure exacly what you want returned from your example. It can easily be handled with a regular expression if I know what you actually expect to be returned. Edited October 20, 2010 by GEOSoft George Question about decompiling code? Read the decompiling FAQ and don't bother posting the question in the forums.Be sure to read and follow the forum rules. -AKA the AutoIt Reading and Comprehension Skills test.*** The PCRE (Regular Expression) ToolKit for AutoIT - (Updated Oct 20, 2011 ver:3.0.1.13) - Please update your current version before filing any bug reports. The installer now includes both 32 and 64 bit versions. No change in version number. Visit my Blog .. currently not active but it will soon be resplendent with news and views. Also please remove any links you may have to my website. it is soon to be closed and replaced with something else. "Old age and treachery will always overcome youth and skill!" Link to comment Share on other sites More sharing options...
UEZ Posted October 20, 2010 Share Posted October 20, 2010 Hmm, I cannot see the logic behind your underlined field names! Sometimes only one is underlined, then 2 or 3! Maybe you can explain... Br, UEZ Please don't send me any personal message and ask for support! I will not reply! Selection of finest graphical examples at Codepen.io The own fart smells best! ✌Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!¯\_(ツ)_/¯ ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ Link to comment Share on other sites More sharing options...
AndyG Posted October 20, 2010 Share Posted October 20, 2010 (edited) $content = FileRead("logfile.log") $line_array = StringSplit($content, @CRLF, 3) ;0-based, CRLF is delimiter Dim $csv_array[13] ;if you need the data in an array $csv_line = "" ;complete line csv, CRLF terminated For $i = 0 To UBound($line_array) - 14 Step 14 ;13 lines are 1 "block" od data For $x = 0 To 5 ;first 6 lines in block if $x=4 then continueloop $csv_array[$x] = StringTrimLeft($line_array[$i + $x], StringInStr($line_array[$i + $x], " ", 1, -1)) ;trimm all up to the last blank $csv_line &= $csv_array[$x] & "," ;csvdata Next For $x = 6 To 7 ;earth and ins $csv_array[$x] = StringTrimLeft($line_array[$i + $x], StringInStr($line_array[$i + $x], " ")) ;trimm all up to the first blank $csv_line &= $csv_array[$x] & "," ;csvdata Next For $x = 9 To 12 ;text $csv_array[$x] = StringTrimLeft($line_array[$i + $x], StringInStr($line_array[$i + $x], " ")) ;trimm all up to the first blank $csv_line &= $csv_array[$x] & "," ;csvdata Next $csv_line &= @CRLF ;delete line if necessary Next MsgBox(0, "CSV-Data", $csv_line) $csv_file = FileOpen("csvfile.csv", 1);append csv-data FileWrite($csv_file, $csv_line) ;appends new data to existing csv (delete log recommended!)/edit/ i think he wants to have every data excluded the underlined separated by comma Edited October 20, 2010 by AndyG Link to comment Share on other sites More sharing options...
UEZ Posted October 20, 2010 Share Posted October 20, 2010 (edited) Here my version to convert the log file to a 2D array: #include <Array.au3> $hFile = FileOpen("Test.log", 0) Local $aSplit, $aLog[1][4] While 1 $line = FileReadLine($hFile) If @error = -1 Then ExitLoop If $line <> "" Then $aSplit = StringSplit(StringStripWS($line, 7), " ", 2) Fill_Array($aSplit, $aLog) Wend FileClose($hFile) ReDim $aLog[UBound($aLog) -1][4] _ArrayDisplay($aLog) Exit Func Fill_Array($sArray, ByRef $dArray) Local $i, $u = UBound($dArray) For $i = 0 To UBound($sArray) - 1 $dArray[$u -1][$i] = $sArray[$i] Next ReDim $aLog[$u + 1][4] EndFunc You can traverse the array, extract the relevant information and save it finally to a csv. Br, UEZ Edited October 20, 2010 by UEZ Please don't send me any personal message and ask for support! I will not reply! Selection of finest graphical examples at Codepen.io The own fart smells best! ✌Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!¯\_(ツ)_/¯ ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ Link to comment Share on other sites More sharing options...
Malkey Posted October 21, 2010 Share Posted October 21, 2010 Here is another interpretation of what's required. expandcollapse popupLocal $sStr = "TEST NUMBER 0001" & @CRLF & _ "DATE 11-OCT-10" & @CRLF & _ "APP NO 0001" & @CRLF & _ "TEST MODE TEST" & @CRLF & _ "VISUAL CHECK P" & @CRLF & _ "EARTH CURRENT 4A" & @CRLF & _ "EARTH OHM S" & @CRLF & _ "INS >99.99 MEG P" & @CRLF & _ "USER GRANT" & @CRLF & _ "TEXT SHREDDER" & @CRLF & _ "TEXT " & @CRLF & _ "TEXT " & @CRLF & _ "TEXT " & @CRLF & @CRLF & _ "TEST NUMBER 0002" & @CRLF & _ "DATE 20-OCT-10" & @CRLF & _ "APP NO 9334005" & @CRLF & _ "TEST MODE TEST" & @CRLF & _ "VISUAL CHECK P" & @CRLF & _ "EARTH CURRENT 100~" & @CRLF & _ "EARTH 00.04 OHM P" & @CRLF & _ "INS >99.99 MEG P" & @CRLF & _ "USER GRANT" & @CRLF & _ "TEXT PROJECTOR" & @CRLF & _ "TEXT FUSE OK" & @CRLF & _ "TEXT CLASSRM 1" & @CRLF & _ "TEXT 1 FLOOR" & @CRLF ; or ;Local $sStr = FileRead("logfile.log") Local $sCSV = StringRegExpReplace($sStr & @CRLF, _ "TEST NUMBER\h*(\d+)\v+" & _ "DATE\h*([0-9A-Z\-]*)\v+" & _ "APP NO\h*(\d*)\v+" & _ "TEST MODE\h*(\w*)\v+" & _ "VISUAL CHECK\h*(\w*)\v+" & _ "EARTH CURRENT\h*([\w~]*)\v+" & _ "EARTH\h*([ \.\w]*)\v+" & _ "INS\h*([ \.\w>]*)\v+" & _ "USER\h*([ \w]*)\v+" & _ "TEXT\h*([ \w]*)\v+" & _ "TEXT\h*([ \w]*)\v+" & _ "TEXT\h*([ \w]*)\v+" & _ "TEXT\h*([ \w]*)\v+" & _ "", "\1,\2,\3,\4,\5,\6,\7,\8,\9,\10,\11,\12,\13" & @CRLF) ; Add column header labels $sCSV = "TEST NUMBER,DATE,APP NO,TEST MODE,VISUAL CHECK,EARTH CURRENT,EARTH,INS,USER,TEXT,TEXT,TEXT,TEXT" & @CRLF & StringTrimRight($sCSV, 2) ; Remove trailing @CRLF ConsoleWrite($sCSV & @CRLF) ;== Save to csv file == Local $csv_file = FileOpen("csvfile.csv", 10) ; 2+8=10 [2 = Write mode (erase previous contents) & 8 = Create directory structure if it doesn't exist.] FileWrite($csv_file, $sCSV) FileClose($csv_file) Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now