jackylee0908 Posted June 26, 2019 Share Posted June 26, 2019 (edited) Hi sir, I am developing a test script and I need a feature that to list and count every different string from test logs, say that, a test log(BMC.log) will be generated automatically by test script on each system power on, and then, I will need the script to check non-repeated string from the log and if find any non-repeated string then list it into another log file(filtered.log), and count the appearance number if repeated string is detected by the test script. For example: 1st time system boot up and get below BMC.log file by test script: 1 | 05/27/2019 | 10:13:18 | Voltage S0 OK | State Asserted | Deasserted 2 | 05/27/2019 | 10:14:05 | Drive Slot / Bay SXM0 Status | Drive Present | Deasserted 3 | 05/27/2019 | 10:17:36 | Voltage 12V Good | State Asserted | Asserted 4 | 05/27/2019 | 10:17:36 | Voltage AC Good | State Asserted | Asserted 5 | 05/27/2019 | 10:17:36 | Voltage S0 OK | State Asserted | Asserted 6 | 05/27/2019 | 10:17:37 | Processor CPU0 Status | Presence detected | Asserted 7 | 05/27/2019 | 10:17:37 | Processor CPU1 Status | Presence detected | Asserted 8 | 05/27/2019 | 10:18:02 | Power Supply PSU0 Status | Presence detected | Asserted 9 | 05/27/2019 | 10:18:02 | Power Supply PSU1 Status | Presence detected | Asserted 10 | 05/27/2019 | 10:18:21 | Drive Slot / Bay SXM0 Status | Drive Present | Asserted Then, because it is first cycle and there is no record in filtered test log, so, every string is new and non-repeated, and I need test script to list it into filtered.log, below is an example format(text file). ------------------------------------------------------------------------------------------------- Counter | events ------------------------------------------------------------------------------------------------- 1 Voltage S0 OK | State Asserted | Deasserted 1 Drive Slot / Bay SXM0 Status | Drive Present | Deasserted 1 Voltage 12V Good | State Asserted | Asserted 1 Voltage AC Good | State Asserted | Asserted 1 Voltage S0 OK | State Asserted | Asserted 1 Processor CPU0 Status | Presence detected | Asserted 1 Processor CPU1 Status | Presence detected | Asserted 1 Power Supply PSU0 Status | Presence detected | Asserted 1 Power Supply PSU1 Status | Presence detected | Asserted 1 Drive Slot / Bay SXM0 Status | Drive Present | Asserted I will need the filtered string to ignore string number, string date, and string time, only list the string name and string status. Then, when system do the 2nd time boot up and get below new BMC.log file by test script: 1 | 05/27/2019 | 10:25:09 | Microcontroller #0x16 | Transition to Running | Asserted 2 | 05/27/2019 | 10:25:09 | Microcontroller #0x16 | Transition to Running | Asserted 3 | 05/27/2019 | 10:25:15 | Voltage 12V Good | State Asserted | Asserted 4 | 05/27/2019 | 10:25:15 | Voltage AC Good | State Asserted | Asserted 5 | 05/27/2019 | 10:25:15 | Voltage S0 OK | State Asserted | Asserted 6 | 05/27/2019 | 10:25:16 | Processor CPU0 Status | Presence detected | Asserted 7 | 05/27/2019 | 10:25:16 | Processor CPU1 Status | Presence detected | Asserted 8 | 05/27/2019 | 10:25:39 | Power Supply PSU1 Status | Presence detected | Asserted 9 | 05/27/2019 | 10:25:41 | Power Supply PSU0 Status | Presence detected | Asserted a | 05/27/2019 | 10:26:01 | Drive Slot / Bay SXM0 Status | Drive Present | Asserted And test script detects that the first 2 strings are new and repeated for filtered.log, and others are repeated with filtered.log, so, the test script add first two items into filtered.log and update appearance count for repeated strings. ------------------------------------------------------------------------------------------------- Counter | events ------------------------------------------------------------------------------------------------- 1 Voltage S0 OK | State Asserted | Deasserted 1 Drive Slot / Bay SXM0 Status | Drive Present | Deasserted 2 Voltage 12V Good | State Asserted | Asserted 2 Voltage AC Good | State Asserted | Asserted 2 Voltage S0 OK | State Asserted | Asserted 2 Processor CPU0 Status | Presence detected | Asserted 2 Processor CPU1 Status | Presence detected | Asserted 2 Power Supply PSU0 Status | Presence detected | Asserted 2 Power Supply PSU1 Status | Presence detected | Asserted 2 Drive Slot / Bay SXM0 Status | Drive Present | Asserted 2 Microcontroller #0x16 | Transition to Running | Asserted It is hard for me to develop the test script to do above things because I am new, could you please help to provide me the code to achieve that, your help is highly appreciated. Thanks, Jacky Edited July 3, 2019 by jackylee0908 Link to comment Share on other sites More sharing options...
jackylee0908 Posted June 26, 2019 Author Share Posted June 26, 2019 (edited) Hi sir, I am developing a test script and I need a feature that to list and count every different string from test logs, say that, a test log(BMC.log) will be generated automatically by test script on each system power on, and then, I will need the script to check non-repeated string from the log and if find any non-repeated string then list it into another log file(filtered.log), and count the appearance number if repeated string is detected by the test script. For example: 1st time system boot up and get below BMC.log file by test script: 1 | 05/27/2019 | 10:13:18 | Voltage S0 OK | State Asserted | Deasserted 2 | 05/27/2019 | 10:14:05 | Drive Slot / Bay SXM0 Status | Drive Present | Deasserted 3 | 05/27/2019 | 10:17:36 | Voltage 12V Good | State Asserted | Asserted 4 | 05/27/2019 | 10:17:36 | Voltage AC Good | State Asserted | Asserted 5 | 05/27/2019 | 10:17:36 | Voltage S0 OK | State Asserted | Asserted 6 | 05/27/2019 | 10:17:37 | Processor CPU0 Status | Presence detected | Asserted 7 | 05/27/2019 | 10:17:37 | Processor CPU1 Status | Presence detected | Asserted 8 | 05/27/2019 | 10:18:02 | Power Supply PSU0 Status | Presence detected | Asserted 9 | 05/27/2019 | 10:18:02 | Power Supply PSU1 Status | Presence detected | Asserted 10 | 05/27/2019 | 10:18:21 | Drive Slot / Bay SXM0 Status | Drive Present | Asserted Then, because it is first cycle and there is no record in filtered test log, so, every string is new and non-repeated, and I need test script to list it into filtered.log, below is an example format(text file). ------------------------------------------------------------------------------------------------- Counter | events ------------------------------------------------------------------------------------------------- 1 Voltage S0 OK | State Asserted | Deasserted 1 Drive Slot / Bay SXM0 Status | Drive Present | Deasserted 1 Voltage 12V Good | State Asserted | Asserted 1 Voltage AC Good | State Asserted | Asserted 1 Voltage S0 OK | State Asserted | Asserted 1 Processor CPU0 Status | Presence detected | Asserted 1 Processor CPU1 Status | Presence detected | Asserted 1 Power Supply PSU0 Status | Presence detected | Asserted 1 Power Supply PSU1 Status | Presence detected | Asserted 1 Drive Slot / Bay SXM0 Status | Drive Present | Asserted I will need the filtered string to ignore string number, string date, and string time, only list the string name and string status. Then, when system do the 2nd time boot up and get below new BMC.log file by test script: 1 | 05/27/2019 | 10:25:09 | Microcontroller #0x16 | Transition to Running | Asserted 2 | 05/27/2019 | 10:25:09 | Microcontroller #0x16 | Transition to Running | Asserted 3 | 05/27/2019 | 10:25:15 | Voltage 12V Good | State Asserted | Asserted 4 | 05/27/2019 | 10:25:15 | Voltage AC Good | State Asserted | Asserted 5 | 05/27/2019 | 10:25:15 | Voltage S0 OK | State Asserted | Asserted 6 | 05/27/2019 | 10:25:16 | Processor CPU0 Status | Presence detected | Asserted 7 | 05/27/2019 | 10:25:16 | Processor CPU1 Status | Presence detected | Asserted 8 | 05/27/2019 | 10:25:39 | Power Supply PSU1 Status | Presence detected | Asserted 9 | 05/27/2019 | 10:25:41 | Power Supply PSU0 Status | Presence detected | Asserted a | 05/27/2019 | 10:26:01 | Drive Slot / Bay SXM0 Status | Drive Present | Asserted And test script detects that the first 2 strings are new and repeated for filtered.log, and others are repeated with filtered.log, so, the test script add first two items into filtered.log and update appearance count for repeated strings. ------------------------------------------------------------------------------------------------- Counter | events ------------------------------------------------------------------------------------------------- 1 Voltage S0 OK | State Asserted | Deasserted 1 Drive Slot / Bay SXM0 Status | Drive Present | Deasserted 2 Voltage 12V Good | State Asserted | Asserted 2 Voltage AC Good | State Asserted | Asserted 2 Voltage S0 OK | State Asserted | Asserted 2 Processor CPU0 Status | Presence detected | Asserted 2 Processor CPU1 Status | Presence detected | Asserted 2 Power Supply PSU0 Status | Presence detected | Asserted 2 Power Supply PSU1 Status | Presence detected | Asserted 2 Drive Slot / Bay SXM0 Status | Drive Present | Asserted 2 Microcontroller #0x16 | Transition to Running | Asserted It is hard for me to develop the test script to do above things because I am new, could you please help to provide me the code to achieve that, your help is highly appreciated. Thanks, Jacky Edited June 26, 2019 by jackylee0908 Link to comment Share on other sites More sharing options...
Subz Posted June 26, 2019 Share Posted June 26, 2019 (edited) Here is a basic example to get you started, it's up to you to figure out how it works, although most of it is self explanatory. Everything was based upon the example logs you provided above and I can confirm it works, so if it doesn't work then you need to figure out what you left out of the example logs above and fix accordingly. #include <Array.au3> #include <File.au3> Global $g_aUnfiltered_LogFile, $g_sUnfiltered_LogData, $g_sUnfiltered_LogPath = @ScriptDir & "\Test.log" _FileReadToArray($g_sUnfiltered_LogPath, $g_aUnfiltered_LogFile, 4, " | ") If @error Then Exit MsgBox(4096, "Error", 'Failure reading "' & $g_sUnfiltered_LogPath & '"') _ArrayColDelete($g_aUnfiltered_LogFile, 0) _ArrayColDelete($g_aUnfiltered_LogFile, 0) _ArrayColDelete($g_aUnfiltered_LogFile, 0) $g_sUnfiltered_LogData = _ArrayToString($g_aUnfiltered_LogFile, " | ", -1, -1, @LF) $g_aUnfiltered_LogFile = StringSplit($g_sUnfiltered_LogData, @LF, 3) Global $g_aFiltered_LogFile, $g_iFiltered_Search, $g_sFiltered_LogData, $g_sFiltered_LogPath = @ScriptDir & "\Filtered.log" If FileExists($g_sFiltered_LogPath) Then _FileReadToArray($g_sFiltered_LogPath, $g_aFiltered_LogFile, 4, @TAB) If @error Then Exit MsgBox(4096, "Error", 'Failure reading "' & $g_sFiltered_LogPath & '"') For $i = 0 To UBound($g_aUnfiltered_LogFile) - 1 $g_iFiltered_Search = _ArraySearch($g_aFiltered_LogFile, $g_aUnfiltered_LogFile[$i], 0, 0, 1, 0, 1, 1) If $g_iFiltered_Search > -1 Then $g_aFiltered_LogFile[$g_iFiltered_Search][0] = Number($g_aFiltered_LogFile[$i][0]) + 1 Else _ArrayAdd($g_aFiltered_LogFile, "1" & @TAB & $g_aUnfiltered_LogFile[$i], 0, @TAB) EndIf Next $g_sFiltered_LogData = _ArrayToString($g_aFiltered_LogFile, @TAB) Else $g_sFiltered_LogData = "1" & @TAB & _ArrayToString($g_aUnfiltered_LogFile, @CRLF & "1" & @TAB) EndIf Global $g_hFiltered_LogPath = FileOpen($g_sFiltered_LogPath, 10) FileWrite($g_hFiltered_LogPath, $g_sFiltered_LogData & @CRLF) FileClose($g_hFiltered_LogPath) Edited June 26, 2019 by Subz Link to comment Share on other sites More sharing options...
jackylee0908 Posted June 26, 2019 Author Share Posted June 26, 2019 (edited) Hi @Subz, Thanks for your code, I will try it. BR, Jacky Edited June 26, 2019 by jackylee0908 Link to comment Share on other sites More sharing options...
Subz Posted June 26, 2019 Share Posted June 26, 2019 For some reason since an Autoit forum update, I've found that "sometimes" copying data from the forum, it will include hidden characters, so use something like NotePad++ to show hidden characters and remove any that you find, I'm unsure if you put the ? after the <File.au3> but that usually indicates a hidden character. Secondly I have updated my code above, I forgot to to remove MsgBox within the loop. Link to comment Share on other sites More sharing options...
jackylee0908 Posted June 26, 2019 Author Share Posted June 26, 2019 (edited) Hi @Subz, Thanks, I have tried but found the counter for specific string is reversed. I just tried to filter BMC_event_1.log firstly and got correct output Filtered.log, and then I tried to filter BMC_event_2.log, the drive slot count with "Asserted" should be counted but it counted with "Deasserted" in Filtered.log. Also, when I tried to filter BMC_event_2.log then BMC_event_1.log, the counter output to Filtered.log also is incorrect, especially for BMC_event_2.log, the first two strings are the same but both strings are also be listed in Filtered.log. I have attached two example log files, please help to check it, thanks a lot! BR, Jacky BMC_event_2.logBMC_event_1.log Edited June 26, 2019 by jackylee0908 Link to comment Share on other sites More sharing options...
jackylee0908 Posted June 26, 2019 Author Share Posted June 26, 2019 Moved this topic to general help, as post to wrong category. Link to comment Share on other sites More sharing options...
Subz Posted June 26, 2019 Share Posted June 26, 2019 Sorry see where I messed up, have updated the code above. Link to comment Share on other sites More sharing options...
jackylee0908 Posted June 26, 2019 Author Share Posted June 26, 2019 7 minutes ago, Subz said: Sorry see where I messed up, have updated the code above. Hi @Subz, That's fine, but, I did not see the code update by you, please help confirm that, thanks again! BR, Jacky Link to comment Share on other sites More sharing options...
Subz Posted June 26, 2019 Share Posted June 26, 2019 Don't know why, I've updated the code above again, but will post here as well: #include <Array.au3> #include <File.au3> Global $g_aUnfiltered_LogFile, $g_sUnfiltered_LogData, $g_sUnfiltered_LogPath = @ScriptDir & "\Test.log" _FileReadToArray($g_sUnfiltered_LogPath, $g_aUnfiltered_LogFile, 4, " | ") If @error Then Exit MsgBox(4096, "Error", 'Failure reading "' & $g_sUnfiltered_LogPath & '"') _ArrayColDelete($g_aUnfiltered_LogFile, 0) _ArrayColDelete($g_aUnfiltered_LogFile, 0) _ArrayColDelete($g_aUnfiltered_LogFile, 0) $g_sUnfiltered_LogData = _ArrayToString($g_aUnfiltered_LogFile, " | ", -1, -1, @LF) $g_aUnfiltered_LogFile = StringSplit($g_sUnfiltered_LogData, @LF, 3) Global $g_aFiltered_LogFile, $g_iFiltered_Search, $g_sFiltered_LogData, $g_sFiltered_LogPath = @ScriptDir & "\Filtered.log" If FileExists($g_sFiltered_LogPath) Then _FileReadToArray($g_sFiltered_LogPath, $g_aFiltered_LogFile, 4, @TAB) If @error Then Exit MsgBox(4096, "Error", 'Failure reading "' & $g_sFiltered_LogPath & '"') For $i = 0 To UBound($g_aUnfiltered_LogFile) - 1 $g_iFiltered_Search = _ArraySearch($g_aFiltered_LogFile, $g_aUnfiltered_LogFile[$i], 0, 0, 1, 0, 1, 1) If $g_iFiltered_Search > -1 Then $g_aFiltered_LogFile[$g_iFiltered_Search][0] = Number($g_aFiltered_LogFile[$i][0]) + 1 Else _ArrayAdd($g_aFiltered_LogFile, "1" & @TAB & $g_aUnfiltered_LogFile[$i], 0, @TAB) EndIf Next $g_sFiltered_LogData = _ArrayToString($g_aFiltered_LogFile, @TAB) Else $g_sFiltered_LogData = "1" & @TAB & _ArrayToString($g_aUnfiltered_LogFile, @CRLF & "1" & @TAB) EndIf Global $g_hFiltered_LogPath = FileOpen($g_sFiltered_LogPath, 10) FileWrite($g_hFiltered_LogPath, $g_sFiltered_LogData & @CRLF) FileClose($g_hFiltered_LogPath) Link to comment Share on other sites More sharing options...
jackylee0908 Posted June 26, 2019 Author Share Posted June 26, 2019 Hi @Subz, Thanks for your update, but still got problem when filtering BMC_event_1.log then BMC_event_1.log, some strings have been counted as 3 but actually should be 2. And, if I filter BMC_event_2.log then BMC_event_1.log, the first two strings are duplicated and should be listed as only one with count 2. Please help to update the code, thanks! BR, Jacky Link to comment Share on other sites More sharing options...
Subz Posted June 26, 2019 Share Posted June 26, 2019 What have you tried? I mean It works fine for me, if its not working for you, you may need to play around with the _ArraySearch function, also use some debugging using functions like _ArrayDisplay and ConsoleWrite to see what the values you're searching for etc.. Please show some effort in debugging the code, otherwise you'll never understand the code for future debugging, Link to comment Share on other sites More sharing options...
jackylee0908 Posted June 27, 2019 Author Share Posted June 27, 2019 Thanks, @Subz, I will try it. Link to comment Share on other sites More sharing options...
jackylee0908 Posted June 29, 2019 Author Share Posted June 29, 2019 Hi @Subz, Sorry, I've tried but I am really new, I understood the functions mean after I check the guide, but I still could not able to complete it, could you help? Thanks, Jacky Link to comment Share on other sites More sharing options...
FrancescoDiMuro Posted June 29, 2019 Share Posted June 29, 2019 (edited) @jackylee0908 and @Subz That is the problem when you give a man a fish instead of teaching him to fish. Edited June 29, 2019 by FrancescoDiMuro Click here to see my signature: Spoiler ALWAYS GOOD TO READ: Forum Rules Forum Etiquette Link to comment Share on other sites More sharing options...
mikell Posted June 29, 2019 Share Posted June 29, 2019 1 hour ago, jackylee0908 said: Sorry, I've tried but I am really new OK, here is the code from Subz extremely simplified with comments, as a proof of concept. The rest (read files, write files, convert strings to arrays) is trivial so you could do it yourself expandcollapse popup#Include <Array.au3> ; initialize the filter (this array should come from the log) Global $res[0][2] ; have a string (should come from reading the file to be checked) $s1 = "a,b,c,d,e,f" ; check it against $res $res = _Check($s1) ; display _ArrayDisplay($res) $s2 = "c,d,e,f,g,h" $res = _Check($s2) _ArrayDisplay($res) $s3 = "e,f,g,h,i,j" $res = _Check($s3) _ArrayDisplay($res) Func _Check($string) ; build array from $string (should be modified depending on the text format) Local $tmp = StringSplit($string, ",", 2) ; check if elements in $tmp exist in $res For $i = 0 to UBound($tmp)-1 $n = _ArraySearch($res, $tmp[$i]) ; yes : increase the number by 1 If $n > -1 Then $res[$n][0] += 1 ; no : add the element with number 1 Else _ArrayAdd($res, "1"&"|"& $tmp[$i]) EndIf Next ; return result Return $res EndFunc Link to comment Share on other sites More sharing options...
jackylee0908 Posted July 1, 2019 Author Share Posted July 1, 2019 (edited) Hi @mikell, Thanks for the hint, I have tried but seems not work for me, could you help to check my code please? expandcollapse popup#include <Array.au3> #include <File.au3> #include <MsgBoxConstants.au3> Global $g_sTemp_LogPath = @ScriptDir & "\Temp.log", $g_sFiltered_LogData, $res[0][2] $sFile1 = "BMC_event_1.log" FilterLog($sFile1) Global $aTemp _FileReadToArray($g_sTemp_LogPath, $aTemp, $FRTA_NOCOUNT) _ArrayDisplay($aTemp, "Load temp to array") $str1 = _ArrayToString($aTemp, @CRLF, -1, -1, @LF) MsgBox($MB_SYSTEMMODAL, "show string", $str1) $res = _Check($str1) _ArrayDisplay($res) Func FilterLog($v1) Global $g_aUnfiltered_LogFile, $g_sUnfiltered_LogData, $g_sUnfiltered_LogPath = @ScriptDir & "\" & $v1, $g_sTemp_LogPath = @ScriptDir & "\Temp.log" _FileReadToArray($g_sUnfiltered_LogPath, $g_aUnfiltered_LogFile, 4, " | ") _ArrayDisplay($g_aUnfiltered_LogFile, "Original") If @error Then Exit MsgBox(4096, "Error", 'Failure reading "' & $g_sUnfiltered_LogPath & '"') _ArrayColDelete($g_aUnfiltered_LogFile, 0) _ArrayColDelete($g_aUnfiltered_LogFile, 0) _ArrayColDelete($g_aUnfiltered_LogFile, 0) _ArrayDisplay($g_aUnfiltered_LogFile, "Delete first 3 columns") $g_sUnfiltered_LogData = _ArrayToString($g_aUnfiltered_LogFile, " | ", -1, -1, @LF) $g_aUnfiltered_LogFile = StringSplit($g_sUnfiltered_LogData, @LF, 3) _ArrayDisplay($g_aUnfiltered_LogFile, "Filtered") FileOpen($g_sTemp_LogPath) _FileWriteFromArray($g_sTemp_LogPath, $g_aUnfiltered_LogFile, 1) FileClose($g_sTemp_LogPath) EndFunc Func _Check($string) Local $tmp = StringSplit($string, "|", 2) For $i = 0 to UBound($string) - 1 $n = _ArraySearch($res, $tmp[$i]) If $n > -1 Then $res[$n][0] += 1 Else _ArrayAdd($res, "1"&"|"& $tmp[$i]) EndIf Next ; return result Return $res EndFunc Sample log file was attached. BMC_event_1.log Thanks in advanced. Edited July 1, 2019 by jackylee0908 Link to comment Share on other sites More sharing options...
jackylee0908 Posted July 1, 2019 Author Share Posted July 1, 2019 (edited) Tried on this but still doesn't work for me..... expandcollapse popup#include <Array.au3> #include <File.au3> #include <MsgBoxConstants.au3> Global $g_sTemp_LogPath = @ScriptDir & "\Temp.log", $g_sFiltered_LogData, $res[55][2], $aTemp[55] $sFile1 = "BMC_event_1.log" FilterLog($sFile1) _ArrayDisplay($res, "final") Func FilterLog($v1) Global $g_aUnfiltered_LogFile, $g_sUnfiltered_LogData, $g_sUnfiltered_LogPath = @ScriptDir & "\" & $v1, $g_sTemp_LogPath = @ScriptDir & "\Temp.log" _FileReadToArray($g_sUnfiltered_LogPath, $g_aUnfiltered_LogFile, 4, " | ") _ArrayDisplay($g_aUnfiltered_LogFile, "Original") If @error Then Exit MsgBox(4096, "Error", 'Failure reading "' & $g_sUnfiltered_LogPath & '"') _ArrayColDelete($g_aUnfiltered_LogFile, 0) _ArrayColDelete($g_aUnfiltered_LogFile, 0) _ArrayColDelete($g_aUnfiltered_LogFile, 0) _ArrayDisplay($g_aUnfiltered_LogFile, "Delete first 3 columns") $g_sUnfiltered_LogData = _ArrayToString($g_aUnfiltered_LogFile, " | ", -1, -1, @LF) $g_aUnfiltered_LogFile = StringSplit($g_sUnfiltered_LogData, @LF, 3) _ArrayDisplay($g_aUnfiltered_LogFile, "Filtered") FileOpen($g_sTemp_LogPath) _FileWriteFromArray($g_sTemp_LogPath, $g_aUnfiltered_LogFile, 1) FileClose($g_sTemp_LogPath) _FileReadToArray($g_sTemp_LogPath, $aTemp, $FRTA_NOCOUNT) _ArrayDisplay($aTemp, "Load temp to array") $v = UBound($aTemp, $UBOUND_ROWS) MsgBox($MB_SYSTEMMODAL, "array length by raw", $v) _ArrayDisplay($aTemp, "display raw content") For $i = 0 to $v - 1 $n = _ArraySearch($res, $aTemp[$i]) If $n > -1 Then $res[$n][0] += 1 Else _ArrayAdd($res, "1"&"|"& $aTemp[$i]) EndIf Next Return($res) EndFunc Edited July 1, 2019 by jackylee0908 Link to comment Share on other sites More sharing options...
mikell Posted July 1, 2019 Share Posted July 1, 2019 OK here is my own version. I didn't use _Array* funcs because I prefer regular expressions, and their usage can be tricky when the strings/elements contain the delimiter "|" So the script is a bit longer - and because of the comments too I used the files provided in your 4th post expandcollapse popup#Include <Array.au3> ; define the files $log = @scriptdir & "\" & "filter.log" $file1 = @scriptdir & "\" & "BMC_event_1.log" $file2 = @scriptdir & "\" & "BMC_event_2.log" ; build / initialize the filter (this array should come from the log) If FileExists($log) Then $tmp = StringRegExp(FileRead($log), '(?m)^(\d+)\s*\|\s*(\N+)', 3) ; _ArrayDisplay($tmp, "$tmp") $u = UBound($tmp) Global $filter[$u/2][2] For $i = 0 to $u/2 - 1 $filter[$i][0] = $tmp[$i*2] $filter[$i][1] = $tmp[$i*2+1] Next ; _ArrayDisplay($filter, "filter") Else Global $filter[0][2] EndIf ; check files against $filter _Check($file1) _ArrayDisplay($filter, "after file1") _Check($file2) _ArrayDisplay($filter, "after file2") ; save new filter text $txt = "" For $i = 0 to UBound($filter)-1 $txt &= $filter[$i][0] & " | " & $filter[$i][1] & @crlf Next $txt = StringTrimRight($txt, 2) Msgbox(0,"to be written in filter.log (overwrite mode)", $txt) ;====================================== Func _Check($file) ; read file $tmp = FileRead($file) ; Msgbox(0,"", $tmp) ; clean file $string = StringRegExpReplace($tmp, '(?m)^.*?\|.*?\|.*?\|\s*', "") ; Msgbox(0,"", $string) ; build array from $string (should be modified depending on the text format) Local $tmp = StringRegExp($string, '(?m)^\N+', 3) _ArrayDisplay($tmp, "checking string") ; check if elements in $tmp exist in $filter For $i = 0 to UBound($tmp)-1 $n = _ArraySearch($filter, $tmp[$i]) ; yes : increase the number by 1 If $n > -1 Then $filter[$n][0] += 1 ; no : add the element with number 1 Else $u = UBound($filter) Redim $filter[$u+1][2] $filter[$u][0] = "1" $filter[$u][1] = $tmp[$i] ; _ArrayDisplay($filter, "adding") EndIf Next EndFunc Link to comment Share on other sites More sharing options...
jackylee0908 Posted July 2, 2019 Author Share Posted July 2, 2019 (edited) Hi @mikell, That's great for me, but, while I tried the other test logs as below attached, some errors for the counting by below code which modified from yours. expandcollapse popup#Include <Array.au3> #include <File.au3> ; define the files $log = @scriptdir & "\" & "filter.log" ; build / initialize the filter (this array should come from the log) If FileExists($log) Then $tmp = StringRegExp(FileRead($log), '(?m)^(\d+)\s*\|\s*(\N+)', 3) _ArrayDisplay($tmp, "$tmp") $u = UBound($tmp) Global $filter[$u/2][2] For $i = 0 to $u/2 - 1 $filter[$i][0] = $tmp[$i*2] $filter[$i][1] = $tmp[$i*2+1] Next _ArrayDisplay($filter, "filter") Else Global $filter[0][2] EndIf For $c = 1 To 10 $file1 = @scriptdir & "\" & "cycle_log" & "\" & $c & "\" & "bmc_sel_1.log" ; check files against $filter _Check($file1) _ArrayDisplay($filter, "after file1") ;_Check($file2) ;_ArrayDisplay($filter, "after file2") Next ; save new filter text $txt = "" For $i = 0 to UBound($filter)-1 $txt &= $filter[$i][0] & " | " & $filter[$i][1] & @crlf Next $txt = StringTrimRight($txt, 2) Msgbox(0,"to be written in filter.log (overwrite mode)", $txt) Global $g_hFilter_LogPath = FileOpen($log, 10) FileWrite($g_hFilter_LogPath, $txt) FileClose($g_hFilter_LogPath) ;====================================== Func _Check($file) ; read file $tmp = FileRead($file) ; Msgbox(0,"", $tmp) ; clean file $string = StringRegExpReplace($tmp, '(?m)^.*?\|.*?\|.*?\|\s*', "") ; Msgbox(0,"", $string) ; build array from $string (should be modified depending on the text format) Local $tmp = StringRegExp($string, '(?m)^\N+', 3) _ArrayDisplay($tmp, "checking string") ; check if elements in $tmp exist in $filter For $i = 0 to UBound($tmp)-1 $n = _ArraySearch($filter, $tmp[$i]) ; yes : increase the number by 1 If $n > -1 Then $filter[$n][0] += 1 ; no : add the element with number 1 Else $u = UBound($filter) Redim $filter[$u+1][2] $filter[$u][0] = "1" $filter[$u][1] = $tmp[$i] ; _ArrayDisplay($filter, "adding") EndIf Next EndFunc The filtered log I got is as below: Quote 11 | Event Logging Disabled Event log Dis | Log area reset/cleared | Asserted 7 | 2 | OEM record e0 | cc30312e30394a4d4233000000 11 | Chassis Chassis control | State Asserted 10 | OS Stop/Shutdown | OS graceful shutdown | Asserted 10 | OEM record dd | 000137 | 00ff00050000 10 | OS Stop/Shutdown | Run-time critical stop | Asserted 10 | OEM record dd | 000157 | 00ff00050000 10 | System ACPI Power State Sys ACPI Pow | S5/G2: soft-off | Asserted 10 | System ACPI Power State Sys ACPI Pow | G3: mechanical off | Asserted 10 | Device Installed | Asserted 8 | S0/G0: working | Asserted 10 | System Event System Event | OEM System boot event | Asserted 20 | System Firmware Error POST Error | Unrecoverable hard-disk controller failure | Asserted 10 | OS Boot OS Boot | 😄 boot completed | Asserted 1 | OEM record dc | 000137 | 006e43e05c00 1 | OEM record dc | 000137 | 004545e05c00 1 | OEM record dc | 000137 | 00b946e05c00 1 | OEM record dc | 000137 | 002048e05c00 3 | System ACPI Power State Sys ACPI Pow | S0/G0: working | Asserted 1 | OEM record dc | 000137 | 008949e05c00 1 | OEM record dc | 000137 | 00fb4ae05c00 1 | OEM record dc | 000137 | 00644ce05c00 1 | OEM record dc | 000137 | 00d44de05c00 1 | OEM record dc | 000137 | 003f4fe05c00 1 | OEM record dc | 000137 | 00ac50e05c00 But, actually the counter should be as below, which I filtered it by grep/cut/sort/uniq tools for Windows: Quote ------------------------------------------------------------------------------------------------- Counter | Events ------------------------------------------------------------------------------------------------- 10 Chassis Chassis control | State Asserted 10 Device Installed | Asserted 10 Event Logging Disabled Event log Dis | Log area reset/cleared | Asserted 10 OS Boot OS Boot | 😄 boot completed | Asserted 10 OS Stop/Shutdown | OS graceful shutdown | Asserted 10 OS Stop/Shutdown | Run-time critical stop | Asserted 8 S0/G0: working | Asserted 10 System ACPI Power State Sys ACPI Pow | G3: mechanical off | Asserted 2 System ACPI Power State Sys ACPI Pow | S0/G0: working | Asserted 10 System ACPI Power State Sys ACPI Pow | S5/G2: soft-off | Asserted 10 System Event System Event | OEM System boot event | Asserted 20 System Firmware Error POST Error | Unrecoverable hard-disk controller failure | Asserted 10 1 OEM record dc | 000137 | 002048e05c00 1 OEM record dc | 000137 | 003f4fe05c00 1 OEM record dc | 000137 | 004545e05c00 1 OEM record dc | 000137 | 00644ce05c00 1 OEM record dc | 000137 | 006e43e05c00 1 OEM record dc | 000137 | 008949e05c00 1 OEM record dc | 000137 | 00ac50e05c00 1 OEM record dc | 000137 | 00b946e05c00 1 OEM record dc | 000137 | 00d44de05c00 1 OEM record dc | 000137 | 00fb4ae05c00 10 OEM record dd | 000137 | 00ff00050000 10 OEM record dd | 000157 | 00ff00050000 The Windows batch script I used for filter logs: setlocal enabledelayedexpansion for /l %%i in (1 1 %1) do ( type C:\cycle_log\%%i\bmc_sel_1.log >> C:\cycle_log\bmc_sel_temp.log ) grep -v "OEM record" C:\cycle_log\BMC_sel_temp.log | cut -d "|" -f 4- | sort | uniq -c >> C:\cycle_log\bmc_sel_counter_1.log grep "OEM record" C:\cycle_log\BMC_sel_temp.log | cut -d "|" -f 4- | sort | uniq -c >> C:\cycle_log\bmc_sel_counter_1.log del C:\cycle_log\bmc_sel_temp.log Compared from above two filtered logs you can see that there are some difference, seems your code still got bugs on counting. Your help is highly appreciated, thanks so much for your help, so, could you please help to update your code for solving the bugs, also, could you please also add string soring by A-Z when saving it to filter.log file? Again, your help is highly appreciated! Thanks. cycle_log.zip bmc_sel_counter_1.log Edited July 2, 2019 by jackylee0908 Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now