[Solved] How to record different string and count repeated string in a loop test?

jackylee0908 · June 26, 2019

Hi sir,

I am developing a test script and I need a feature that to list and count every different string from test logs, say that, a test log(BMC.log) will be generated automatically by test script on each system power on, and then, I will need the script to check non-repeated string from the log and if find any non-repeated string then list it into another log file(filtered.log), and count the appearance number if repeated string is detected by the test script.

For example:

1st time system boot up and get below BMC.log file by test script:

1 | 05/27/2019 | 10:13:18 | Voltage S0 OK | State Asserted | Deasserted
   2 | 05/27/2019 | 10:14:05 | Drive Slot / Bay SXM0 Status | Drive Present | Deasserted
   3 | 05/27/2019 | 10:17:36 | Voltage 12V Good | State Asserted | Asserted
   4 | 05/27/2019 | 10:17:36 | Voltage AC Good | State Asserted | Asserted
   5 | 05/27/2019 | 10:17:36 | Voltage S0 OK | State Asserted | Asserted
   6 | 05/27/2019 | 10:17:37 | Processor CPU0 Status | Presence detected | Asserted
   7 | 05/27/2019 | 10:17:37 | Processor CPU1 Status | Presence detected | Asserted
   8 | 05/27/2019 | 10:18:02 | Power Supply PSU0 Status | Presence detected | Asserted
   9 | 05/27/2019 | 10:18:02 | Power Supply PSU1 Status | Presence detected | Asserted
  10 | 05/27/2019 | 10:18:21 | Drive Slot / Bay SXM0 Status | Drive Present | Asserted

Then, because it is first cycle and there is no record in filtered test log, so, every string is new and non-repeated, and I need test script to list it into filtered.log, below is an example format(text file).

-------------------------------------------------------------------------------------------------
Counter |                                 events 
-------------------------------------------------------------------------------------------------
      1  Voltage S0 OK | State Asserted | Deasserted
      1  Drive Slot / Bay SXM0 Status | Drive Present | Deasserted
      1  Voltage 12V Good | State Asserted | Asserted
      1  Voltage AC Good | State Asserted | Asserted
      1  Voltage S0 OK | State Asserted | Asserted
      1  Processor CPU0 Status | Presence detected | Asserted
      1  Processor CPU1 Status | Presence detected | Asserted
      1  Power Supply PSU0 Status | Presence detected | Asserted
      1  Power Supply PSU1 Status | Presence detected | Asserted
      1  Drive Slot / Bay SXM0 Status | Drive Present | Asserted

I will need the filtered string to ignore string number, string date, and string time, only list the string name and string status.

Then, when system do the 2nd time boot up and get below new BMC.log file by test script:

1 | 05/27/2019 | 10:25:09 | Microcontroller #0x16 | Transition to Running | Asserted
   2 | 05/27/2019 | 10:25:09 | Microcontroller #0x16 | Transition to Running | Asserted
   3 | 05/27/2019 | 10:25:15 | Voltage 12V Good | State Asserted | Asserted
   4 | 05/27/2019 | 10:25:15 | Voltage AC Good | State Asserted | Asserted
   5 | 05/27/2019 | 10:25:15 | Voltage S0 OK | State Asserted | Asserted
   6 | 05/27/2019 | 10:25:16 | Processor CPU0 Status | Presence detected | Asserted
   7 | 05/27/2019 | 10:25:16 | Processor CPU1 Status | Presence detected | Asserted
   8 | 05/27/2019 | 10:25:39 | Power Supply PSU1 Status | Presence detected | Asserted
   9 | 05/27/2019 | 10:25:41 | Power Supply PSU0 Status | Presence detected | Asserted
   a | 05/27/2019 | 10:26:01 | Drive Slot / Bay SXM0 Status | Drive Present | Asserted

And test script detects that the first 2 strings are new and repeated for filtered.log, and others are repeated with filtered.log, so, the test script add first two items into filtered.log and update appearance count for repeated strings.

-------------------------------------------------------------------------------------------------
Counter |                                 events 
-------------------------------------------------------------------------------------------------
      1  Voltage S0 OK | State Asserted | Deasserted
      1  Drive Slot / Bay SXM0 Status | Drive Present | Deasserted
      2  Voltage 12V Good | State Asserted | Asserted
      2  Voltage AC Good | State Asserted | Asserted
      2  Voltage S0 OK | State Asserted | Asserted
      2  Processor CPU0 Status | Presence detected | Asserted
      2  Processor CPU1 Status | Presence detected | Asserted
      2  Power Supply PSU0 Status | Presence detected | Asserted
      2  Power Supply PSU1 Status | Presence detected | Asserted
      2  Drive Slot / Bay SXM0 Status | Drive Present | Asserted
      2  Microcontroller #0x16 | Transition to Running | Asserted

It is hard for me to develop the test script to do above things because I am new, could you please help to provide me the code to achieve that, your help is highly appreciated.

Thanks,

Jacky

Edited July 3, 2019 by jackylee0908

jackylee0908 · June 26, 2019

Hi sir,

I am developing a test script and I need a feature that to list and count every different string from test logs, say that, a test log(BMC.log) will be generated automatically by test script on each system power on, and then, I will need the script to check non-repeated string from the log and if find any non-repeated string then list it into another log file(filtered.log), and count the appearance number if repeated string is detected by the test script.

For example:

1st time system boot up and get below BMC.log file by test script:

1 | 05/27/2019 | 10:13:18 | Voltage S0 OK | State Asserted | Deasserted
   2 | 05/27/2019 | 10:14:05 | Drive Slot / Bay SXM0 Status | Drive Present | Deasserted
   3 | 05/27/2019 | 10:17:36 | Voltage 12V Good | State Asserted | Asserted
   4 | 05/27/2019 | 10:17:36 | Voltage AC Good | State Asserted | Asserted
   5 | 05/27/2019 | 10:17:36 | Voltage S0 OK | State Asserted | Asserted
   6 | 05/27/2019 | 10:17:37 | Processor CPU0 Status | Presence detected | Asserted
   7 | 05/27/2019 | 10:17:37 | Processor CPU1 Status | Presence detected | Asserted
   8 | 05/27/2019 | 10:18:02 | Power Supply PSU0 Status | Presence detected | Asserted
   9 | 05/27/2019 | 10:18:02 | Power Supply PSU1 Status | Presence detected | Asserted
  10 | 05/27/2019 | 10:18:21 | Drive Slot / Bay SXM0 Status | Drive Present | Asserted

Then, because it is first cycle and there is no record in filtered test log, so, every string is new and non-repeated, and I need test script to list it into filtered.log, below is an example format(text file).

-------------------------------------------------------------------------------------------------
Counter |                                 events 
-------------------------------------------------------------------------------------------------
      1  Voltage S0 OK | State Asserted | Deasserted
      1  Drive Slot / Bay SXM0 Status | Drive Present | Deasserted
      1  Voltage 12V Good | State Asserted | Asserted
      1  Voltage AC Good | State Asserted | Asserted
      1  Voltage S0 OK | State Asserted | Asserted
      1  Processor CPU0 Status | Presence detected | Asserted
      1  Processor CPU1 Status | Presence detected | Asserted
      1  Power Supply PSU0 Status | Presence detected | Asserted
      1  Power Supply PSU1 Status | Presence detected | Asserted
      1  Drive Slot / Bay SXM0 Status | Drive Present | Asserted

I will need the filtered string to ignore string number, string date, and string time, only list the string name and string status.

Then, when system do the 2nd time boot up and get below new BMC.log file by test script:

1 | 05/27/2019 | 10:25:09 | Microcontroller #0x16 | Transition to Running | Asserted
   2 | 05/27/2019 | 10:25:09 | Microcontroller #0x16 | Transition to Running | Asserted
   3 | 05/27/2019 | 10:25:15 | Voltage 12V Good | State Asserted | Asserted
   4 | 05/27/2019 | 10:25:15 | Voltage AC Good | State Asserted | Asserted
   5 | 05/27/2019 | 10:25:15 | Voltage S0 OK | State Asserted | Asserted
   6 | 05/27/2019 | 10:25:16 | Processor CPU0 Status | Presence detected | Asserted
   7 | 05/27/2019 | 10:25:16 | Processor CPU1 Status | Presence detected | Asserted
   8 | 05/27/2019 | 10:25:39 | Power Supply PSU1 Status | Presence detected | Asserted
   9 | 05/27/2019 | 10:25:41 | Power Supply PSU0 Status | Presence detected | Asserted
   a | 05/27/2019 | 10:26:01 | Drive Slot / Bay SXM0 Status | Drive Present | Asserted

And test script detects that the first 2 strings are new and repeated for filtered.log, and others are repeated with filtered.log, so, the test script add first two items into filtered.log and update appearance count for repeated strings.

-------------------------------------------------------------------------------------------------
Counter |                                 events 
-------------------------------------------------------------------------------------------------
      1  Voltage S0 OK | State Asserted | Deasserted
      1  Drive Slot / Bay SXM0 Status | Drive Present | Deasserted
      2  Voltage 12V Good | State Asserted | Asserted
      2  Voltage AC Good | State Asserted | Asserted
      2  Voltage S0 OK | State Asserted | Asserted
      2  Processor CPU0 Status | Presence detected | Asserted
      2  Processor CPU1 Status | Presence detected | Asserted
      2  Power Supply PSU0 Status | Presence detected | Asserted
      2  Power Supply PSU1 Status | Presence detected | Asserted
      2  Drive Slot / Bay SXM0 Status | Drive Present | Asserted
      2  Microcontroller #0x16 | Transition to Running | Asserted

It is hard for me to develop the test script to do above things because I am new, could you please help to provide me the code to achieve that, your help is highly appreciated.

Thanks,

Jacky

Edited June 26, 2019 by jackylee0908

Subz · June 26, 2019

Here is a basic example to get you started, it's up to you to figure out how it works, although most of it is self explanatory. Everything was based upon the example logs you provided above and I can confirm it works, so if it doesn't work then you need to figure out what you left out of the example logs above and fix accordingly.

#include <Array.au3>
#include <File.au3>

Global $g_aUnfiltered_LogFile, $g_sUnfiltered_LogData, $g_sUnfiltered_LogPath = @ScriptDir & "\Test.log"
_FileReadToArray($g_sUnfiltered_LogPath, $g_aUnfiltered_LogFile, 4, " | ")
    If @error Then Exit MsgBox(4096, "Error", 'Failure reading "' & $g_sUnfiltered_LogPath & '"')
    _ArrayColDelete($g_aUnfiltered_LogFile, 0)
    _ArrayColDelete($g_aUnfiltered_LogFile, 0)
    _ArrayColDelete($g_aUnfiltered_LogFile, 0)
    $g_sUnfiltered_LogData = _ArrayToString($g_aUnfiltered_LogFile, " | ", -1, -1, @LF)
    $g_aUnfiltered_LogFile = StringSplit($g_sUnfiltered_LogData, @LF, 3)

Global $g_aFiltered_LogFile, $g_iFiltered_Search, $g_sFiltered_LogData, $g_sFiltered_LogPath = @ScriptDir & "\Filtered.log"
If FileExists($g_sFiltered_LogPath) Then
    _FileReadToArray($g_sFiltered_LogPath, $g_aFiltered_LogFile, 4, @TAB)
        If @error Then Exit MsgBox(4096, "Error", 'Failure reading "' & $g_sFiltered_LogPath & '"')
    For $i = 0 To UBound($g_aUnfiltered_LogFile) - 1
        $g_iFiltered_Search = _ArraySearch($g_aFiltered_LogFile, $g_aUnfiltered_LogFile[$i], 0, 0, 1, 0, 1, 1)
        If $g_iFiltered_Search > -1 Then
            $g_aFiltered_LogFile[$g_iFiltered_Search][0] = Number($g_aFiltered_LogFile[$i][0]) + 1
        Else
            _ArrayAdd($g_aFiltered_LogFile, "1" & @TAB & $g_aUnfiltered_LogFile[$i], 0, @TAB)
        EndIf
    Next
    $g_sFiltered_LogData = _ArrayToString($g_aFiltered_LogFile, @TAB)
Else
    $g_sFiltered_LogData = "1" & @TAB & _ArrayToString($g_aUnfiltered_LogFile, @CRLF & "1" & @TAB)
EndIf

Global $g_hFiltered_LogPath = FileOpen($g_sFiltered_LogPath, 10)
    FileWrite($g_hFiltered_LogPath, $g_sFiltered_LogData & @CRLF)
    FileClose($g_hFiltered_LogPath)

Edited June 26, 2019 by Subz

jackylee0908 · June 26, 2019

Hi @Subz,

Thanks for your code, I will try it.

BR,

Jacky

Edited June 26, 2019 by jackylee0908

Subz · June 26, 2019

For some reason since an Autoit forum update, I've found that "sometimes" copying data from the forum, it will include hidden characters, so use something like NotePad++ to show hidden characters and remove any that you find, I'm unsure if you put the ? after the <File.au3> but that usually indicates a hidden character.

Secondly I have updated my code above, I forgot to to remove MsgBox within the loop.

jackylee0908 · June 26, 2019

Hi @Subz,

Thanks, I have tried but found the counter for specific string is reversed.

I just tried to filter BMC_event_1.log firstly and got correct output Filtered.log, and then I tried to filter BMC_event_2.log, the drive slot count with "Asserted" should be counted but it counted with "Deasserted" in Filtered.log.

Also, when I tried to filter BMC_event_2.log then BMC_event_1.log, the counter output to Filtered.log also is incorrect, especially for BMC_event_2.log, the first two strings are the same but both strings are also be listed in Filtered.log.

I have attached two example log files, please help to check it, thanks a lot!

BR,

Jacky

BMC_event_2.log BMC_event_1.log

Edited June 26, 2019 by jackylee0908

jackylee0908 · June 26, 2019

Moved this topic to general help, as post to wrong category.

Subz · June 26, 2019

Sorry see where I messed up, have updated the code above.

jackylee0908 · June 26, 2019

7 minutes ago, Subz said:

Sorry see where I messed up, have updated the code above.

Hi @Subz,

That's fine, but, I did not see the code update by you, please help confirm that, thanks again!

BR,

Jacky

Subz · June 26, 2019

Don't know why, I've updated the code above again, but will post here as well:

#include <Array.au3>
#include <File.au3>

Global $g_aUnfiltered_LogFile, $g_sUnfiltered_LogData, $g_sUnfiltered_LogPath = @ScriptDir & "\Test.log"
_FileReadToArray($g_sUnfiltered_LogPath, $g_aUnfiltered_LogFile, 4, " | ")
    If @error Then Exit MsgBox(4096, "Error", 'Failure reading "' & $g_sUnfiltered_LogPath & '"')
    _ArrayColDelete($g_aUnfiltered_LogFile, 0)
    _ArrayColDelete($g_aUnfiltered_LogFile, 0)
    _ArrayColDelete($g_aUnfiltered_LogFile, 0)
    $g_sUnfiltered_LogData = _ArrayToString($g_aUnfiltered_LogFile, " | ", -1, -1, @LF)
    $g_aUnfiltered_LogFile = StringSplit($g_sUnfiltered_LogData, @LF, 3)

Global $g_aFiltered_LogFile, $g_iFiltered_Search, $g_sFiltered_LogData, $g_sFiltered_LogPath = @ScriptDir & "\Filtered.log"
If FileExists($g_sFiltered_LogPath) Then
    _FileReadToArray($g_sFiltered_LogPath, $g_aFiltered_LogFile, 4, @TAB)
        If @error Then Exit MsgBox(4096, "Error", 'Failure reading "' & $g_sFiltered_LogPath & '"')
    For $i = 0 To UBound($g_aUnfiltered_LogFile) - 1
        $g_iFiltered_Search = _ArraySearch($g_aFiltered_LogFile, $g_aUnfiltered_LogFile[$i], 0, 0, 1, 0, 1, 1)
        If $g_iFiltered_Search > -1 Then
            $g_aFiltered_LogFile[$g_iFiltered_Search][0] = Number($g_aFiltered_LogFile[$i][0]) + 1
        Else
            _ArrayAdd($g_aFiltered_LogFile, "1" & @TAB & $g_aUnfiltered_LogFile[$i], 0, @TAB)
        EndIf
    Next
    $g_sFiltered_LogData = _ArrayToString($g_aFiltered_LogFile, @TAB)
Else
    $g_sFiltered_LogData = "1" & @TAB & _ArrayToString($g_aUnfiltered_LogFile, @CRLF & "1" & @TAB)
EndIf

Global $g_hFiltered_LogPath = FileOpen($g_sFiltered_LogPath, 10)
    FileWrite($g_hFiltered_LogPath, $g_sFiltered_LogData & @CRLF)
    FileClose($g_hFiltered_LogPath)

jackylee0908 · June 26, 2019

Hi @Subz,

Thanks for your update, but still got problem when filtering BMC_event_1.log then BMC_event_1.log, some strings have been counted as 3 but actually should be 2.

And, if I filter BMC_event_2.log then BMC_event_1.log, the first two strings are duplicated and should be listed as only one with count 2.

Please help to update the code, thanks!

BR,

Jacky

Subz · June 26, 2019

What have you tried? I mean It works fine for me, if its not working for you, you may need to play around with the _ArraySearch function, also use some debugging using functions like _ArrayDisplay and ConsoleWrite to see what the values you're searching for etc.. Please show some effort in debugging the code, otherwise you'll never understand the code for future debugging,

jackylee0908 · June 27, 2019

Thanks, @Subz, I will try it.

jackylee0908 · June 29, 2019

Hi @Subz,

Sorry, I've tried but I am really new, I understood the functions mean after I check the guide, but I still could not able to complete it, could you help?

Thanks,

Jacky

FrancescoDiMuro · June 29, 2019

@jackylee0908 and @Subz

That is the problem when you give a man a fish instead of teaching him to fish.

Edited June 29, 2019 by FrancescoDiMuro

mikell · June 29, 2019

1 hour ago, jackylee0908 said:

Sorry, I've tried but I am really new

OK, here is the code from Subz extremely simplified with comments, as a proof of concept. The rest (read files, write files, convert strings to arrays) is trivial so you could do it yourself

#Include <Array.au3>

; initialize the filter (this array should come from the log)
Global $res[0][2]

; have a string (should come from reading the file to be checked)
$s1 = "a,b,c,d,e,f"
; check it against $res
$res = _Check($s1)
; display
_ArrayDisplay($res)

$s2 = "c,d,e,f,g,h"
$res = _Check($s2)
_ArrayDisplay($res)

$s3 = "e,f,g,h,i,j"
$res = _Check($s3)
_ArrayDisplay($res)


Func _Check($string)
   ; build array from $string (should be modified depending on the text format)
   Local $tmp = StringSplit($string, ",", 2)
   ; check if elements in $tmp exist in $res
   For $i = 0 to UBound($tmp)-1
      $n = _ArraySearch($res, $tmp[$i])
      ; yes : increase the number by 1
      If $n > -1 Then
            $res[$n][0] += 1
      ; no : add the element with number 1
      Else
            _ArrayAdd($res, "1"&"|"& $tmp[$i])
      EndIf  
   Next
   ; return result
   Return $res
EndFunc

jackylee0908 · July 1, 2019

Hi @mikell,

Thanks for the hint, I have tried but seems not work for me, could you help to check my code please?

#include <Array.au3>
#include <File.au3>
#include <MsgBoxConstants.au3>

Global $g_sTemp_LogPath = @ScriptDir & "\Temp.log", $g_sFiltered_LogData, $res[0][2]

$sFile1 = "BMC_event_1.log"
FilterLog($sFile1)

Global $aTemp
_FileReadToArray($g_sTemp_LogPath, $aTemp, $FRTA_NOCOUNT)
_ArrayDisplay($aTemp, "Load temp to array")

$str1 = _ArrayToString($aTemp, @CRLF, -1, -1, @LF)
MsgBox($MB_SYSTEMMODAL, "show string", $str1)

$res = _Check($str1)
_ArrayDisplay($res)

Func FilterLog($v1)
    Global $g_aUnfiltered_LogFile, $g_sUnfiltered_LogData, $g_sUnfiltered_LogPath = @ScriptDir & "\" & $v1, $g_sTemp_LogPath = @ScriptDir & "\Temp.log"
    _FileReadToArray($g_sUnfiltered_LogPath, $g_aUnfiltered_LogFile, 4, " | ")
    _ArrayDisplay($g_aUnfiltered_LogFile, "Original")
        If @error Then Exit MsgBox(4096, "Error", 'Failure reading "' & $g_sUnfiltered_LogPath & '"')
        _ArrayColDelete($g_aUnfiltered_LogFile, 0)
        _ArrayColDelete($g_aUnfiltered_LogFile, 0)
        _ArrayColDelete($g_aUnfiltered_LogFile, 0)
        _ArrayDisplay($g_aUnfiltered_LogFile, "Delete first 3 columns")
        $g_sUnfiltered_LogData = _ArrayToString($g_aUnfiltered_LogFile, " | ", -1, -1, @LF)
        $g_aUnfiltered_LogFile = StringSplit($g_sUnfiltered_LogData, @LF, 3)
        _ArrayDisplay($g_aUnfiltered_LogFile, "Filtered")
        FileOpen($g_sTemp_LogPath)
        _FileWriteFromArray($g_sTemp_LogPath, $g_aUnfiltered_LogFile, 1)
        FileClose($g_sTemp_LogPath)
EndFunc

Func _Check($string)
   Local $tmp = StringSplit($string, "|", 2)
   For $i = 0 to UBound($string) - 1
      $n = _ArraySearch($res, $tmp[$i])
      If $n > -1 Then
            $res[$n][0] += 1
      Else
            _ArrayAdd($res, "1"&"|"& $tmp[$i])
      EndIf
   Next
   ; return result
   Return $res
EndFunc

Sample log file was attached.

BMC_event_1.log

Thanks in advanced.

Edited July 1, 2019 by jackylee0908

jackylee0908 · July 1, 2019

Tried on this but still doesn't work for me.....

#include <Array.au3>
#include <File.au3>
#include <MsgBoxConstants.au3>

Global $g_sTemp_LogPath = @ScriptDir & "\Temp.log", $g_sFiltered_LogData, $res[55][2], $aTemp[55]

$sFile1 = "BMC_event_1.log"
FilterLog($sFile1)
_ArrayDisplay($res, "final")

Func FilterLog($v1)
    Global $g_aUnfiltered_LogFile, $g_sUnfiltered_LogData, $g_sUnfiltered_LogPath = @ScriptDir & "\" & $v1, $g_sTemp_LogPath = @ScriptDir & "\Temp.log"
    _FileReadToArray($g_sUnfiltered_LogPath, $g_aUnfiltered_LogFile, 4, " | ")
    _ArrayDisplay($g_aUnfiltered_LogFile, "Original")
        If @error Then Exit MsgBox(4096, "Error", 'Failure reading "' & $g_sUnfiltered_LogPath & '"')
        _ArrayColDelete($g_aUnfiltered_LogFile, 0)
        _ArrayColDelete($g_aUnfiltered_LogFile, 0)
        _ArrayColDelete($g_aUnfiltered_LogFile, 0)
        _ArrayDisplay($g_aUnfiltered_LogFile, "Delete first 3 columns")
        $g_sUnfiltered_LogData = _ArrayToString($g_aUnfiltered_LogFile, " | ", -1, -1, @LF)
        $g_aUnfiltered_LogFile = StringSplit($g_sUnfiltered_LogData, @LF, 3)
        _ArrayDisplay($g_aUnfiltered_LogFile, "Filtered")
        FileOpen($g_sTemp_LogPath)
        _FileWriteFromArray($g_sTemp_LogPath, $g_aUnfiltered_LogFile, 1)
        FileClose($g_sTemp_LogPath)

        _FileReadToArray($g_sTemp_LogPath, $aTemp, $FRTA_NOCOUNT)
        _ArrayDisplay($aTemp, "Load temp to array")
        $v = UBound($aTemp, $UBOUND_ROWS)
        MsgBox($MB_SYSTEMMODAL, "array length by raw", $v)
        _ArrayDisplay($aTemp, "display raw content")

        For $i = 0 to $v - 1
            $n = _ArraySearch($res, $aTemp[$i])
            If $n > -1 Then
                $res[$n][0] += 1
            Else
                _ArrayAdd($res, "1"&"|"& $aTemp[$i])
            EndIf
        Next
        Return($res)
EndFunc

Edited July 1, 2019 by jackylee0908

mikell · July 1, 2019

OK here is my own version. I didn't use _Array* funcs because I prefer regular expressions, and their usage can be tricky when the strings/elements contain the delimiter "|"
So the script is a bit longer - and because of the comments too :idiot:
I used the files provided in your 4th post

#Include <Array.au3>


; define the files
$log = @scriptdir & "\" & "filter.log"
$file1 = @scriptdir & "\" & "BMC_event_1.log"
$file2 = @scriptdir & "\" & "BMC_event_2.log"


; build / initialize the filter (this array should come from the log)
If FileExists($log) Then 
    $tmp = StringRegExp(FileRead($log), '(?m)^(\d+)\s*\|\s*(\N+)', 3)
  ;  _ArrayDisplay($tmp, "$tmp")
    $u = UBound($tmp)
    Global $filter[$u/2][2]
    For $i = 0 to $u/2 - 1 
         $filter[$i][0] = $tmp[$i*2]
         $filter[$i][1] = $tmp[$i*2+1]
    Next
   ; _ArrayDisplay($filter, "filter")
Else
   Global $filter[0][2]
EndIf


; check files against $filter
_Check($file1)
_ArrayDisplay($filter, "after file1")

_Check($file2)
_ArrayDisplay($filter, "after file2")


; save new filter text
$txt = ""
For $i = 0 to UBound($filter)-1
  $txt &= $filter[$i][0] & " | " & $filter[$i][1] & @crlf
Next
$txt = StringTrimRight($txt, 2)
Msgbox(0,"to be written in filter.log (overwrite mode)", $txt)

;======================================

Func _Check($file)
   ; read file
    $tmp = FileRead($file)
; Msgbox(0,"", $tmp)
   ; clean file
    $string = StringRegExpReplace($tmp, '(?m)^.*?\|.*?\|.*?\|\s*', "")
; Msgbox(0,"", $string)
   ; build array from $string (should be modified depending on the text format)
   Local $tmp = StringRegExp($string, '(?m)^\N+', 3)
_ArrayDisplay($tmp, "checking string")
   ; check if elements in $tmp exist in $filter
   For $i = 0 to UBound($tmp)-1
      $n = _ArraySearch($filter, $tmp[$i])
      ; yes : increase the number by 1
      If $n > -1 Then
            $filter[$n][0] += 1
      ; no : add the element with number 1
      Else
           $u = UBound($filter)
           Redim $filter[$u+1][2]     
           $filter[$u][0] = "1"
           $filter[$u][1] = $tmp[$i]
        ; _ArrayDisplay($filter, "adding")
      EndIf  
   Next
EndFunc

jackylee0908 · July 2, 2019

Hi @mikell,

That's great for me, but, while I tried the other test logs as below attached, some errors for the counting by below code which modified from yours.

#Include <Array.au3>
#include <File.au3>


; define the files
$log = @scriptdir & "\" & "filter.log"

; build / initialize the filter (this array should come from the log)
If FileExists($log) Then
    $tmp = StringRegExp(FileRead($log), '(?m)^(\d+)\s*\|\s*(\N+)', 3)
    _ArrayDisplay($tmp, "$tmp")
    $u = UBound($tmp)
    Global $filter[$u/2][2]
    For $i = 0 to $u/2 - 1
         $filter[$i][0] = $tmp[$i*2]
         $filter[$i][1] = $tmp[$i*2+1]
    Next
    _ArrayDisplay($filter, "filter")
Else
   Global $filter[0][2]
EndIf


For $c = 1 To 10
    $file1 = @scriptdir & "\" & "cycle_log" & "\" & $c & "\" & "bmc_sel_1.log"

    ; check files against $filter
    _Check($file1)
    _ArrayDisplay($filter, "after file1")

    ;_Check($file2)
    ;_ArrayDisplay($filter, "after file2")
Next

; save new filter text
$txt = ""
For $i = 0 to UBound($filter)-1
  $txt &= $filter[$i][0] & " | " & $filter[$i][1] & @crlf
Next
$txt = StringTrimRight($txt, 2)
Msgbox(0,"to be written in filter.log (overwrite mode)", $txt)
Global $g_hFilter_LogPath = FileOpen($log, 10)
FileWrite($g_hFilter_LogPath, $txt)
FileClose($g_hFilter_LogPath)

;======================================

Func _Check($file)
   ; read file
    $tmp = FileRead($file)
; Msgbox(0,"", $tmp)
   ; clean file
    $string = StringRegExpReplace($tmp, '(?m)^.*?\|.*?\|.*?\|\s*', "")
; Msgbox(0,"", $string)
   ; build array from $string (should be modified depending on the text format)
   Local $tmp = StringRegExp($string, '(?m)^\N+', 3)
_ArrayDisplay($tmp, "checking string")
   ; check if elements in $tmp exist in $filter
   For $i = 0 to UBound($tmp)-1
      $n = _ArraySearch($filter, $tmp[$i])
      ; yes : increase the number by 1
      If $n > -1 Then
            $filter[$n][0] += 1
      ; no : add the element with number 1
      Else
           $u = UBound($filter)
           Redim $filter[$u+1][2]
           $filter[$u][0] = "1"
           $filter[$u][1] = $tmp[$i]
        ; _ArrayDisplay($filter, "adding")
      EndIf
   Next
EndFunc

The filtered log I got is as below:

Quote

11 | Event Logging Disabled Event log Dis | Log area reset/cleared | Asserted
7 | 2 | OEM record e0 | cc30312e30394a4d4233000000
11 | Chassis Chassis control | State Asserted
10 | OS Stop/Shutdown | OS graceful shutdown | Asserted
10 | OEM record dd | 000137 | 00ff00050000
10 | OS Stop/Shutdown | Run-time critical stop | Asserted
10 | OEM record dd | 000157 | 00ff00050000
10 | System ACPI Power State Sys ACPI Pow | S5/G2: soft-off | Asserted
10 | System ACPI Power State Sys ACPI Pow | G3: mechanical off | Asserted
10 | Device Installed | Asserted
8 | S0/G0: working | Asserted
10 | System Event System Event | OEM System boot event | Asserted
20 | System Firmware Error POST Error | Unrecoverable hard-disk controller failure | Asserted
10 | OS Boot OS Boot | 😄 boot completed | Asserted
1 | OEM record dc | 000137 | 006e43e05c00
1 | OEM record dc | 000137 | 004545e05c00
1 | OEM record dc | 000137 | 00b946e05c00
1 | OEM record dc | 000137 | 002048e05c00
3 | System ACPI Power State Sys ACPI Pow | S0/G0: working | Asserted
1 | OEM record dc | 000137 | 008949e05c00
1 | OEM record dc | 000137 | 00fb4ae05c00
1 | OEM record dc | 000137 | 00644ce05c00
1 | OEM record dc | 000137 | 00d44de05c00
1 | OEM record dc | 000137 | 003f4fe05c00
1 | OEM record dc | 000137 | 00ac50e05c00

But, actually the counter should be as below, which I filtered it by grep/cut/sort/uniq tools for Windows:

Quote

-------------------------------------------------------------------------------------------------
Counter | Events
-------------------------------------------------------------------------------------------------
10 Chassis Chassis control | State Asserted
10 Device Installed | Asserted
10 Event Logging Disabled Event log Dis | Log area reset/cleared | Asserted
10 OS Boot OS Boot | 😄 boot completed | Asserted
10 OS Stop/Shutdown | OS graceful shutdown | Asserted
10 OS Stop/Shutdown | Run-time critical stop | Asserted
8 S0/G0: working | Asserted
10 System ACPI Power State Sys ACPI Pow | G3: mechanical off | Asserted
2 System ACPI Power State Sys ACPI Pow | S0/G0: working | Asserted
10 System ACPI Power State Sys ACPI Pow | S5/G2: soft-off | Asserted
10 System Event System Event | OEM System boot event | Asserted
20 System Firmware Error POST Error | Unrecoverable hard-disk controller failure | Asserted
10
1 OEM record dc | 000137 | 002048e05c00
1 OEM record dc | 000137 | 003f4fe05c00
1 OEM record dc | 000137 | 004545e05c00
1 OEM record dc | 000137 | 00644ce05c00
1 OEM record dc | 000137 | 006e43e05c00
1 OEM record dc | 000137 | 008949e05c00
1 OEM record dc | 000137 | 00ac50e05c00
1 OEM record dc | 000137 | 00b946e05c00
1 OEM record dc | 000137 | 00d44de05c00
1 OEM record dc | 000137 | 00fb4ae05c00
10 OEM record dd | 000137 | 00ff00050000
10 OEM record dd | 000157 | 00ff00050000

The Windows batch script I used for filter logs:

setlocal enabledelayedexpansion

for /l %%i in (1 1 %1) do (
    type C:\cycle_log\%%i\bmc_sel_1.log >> C:\cycle_log\bmc_sel_temp.log
)

grep -v "OEM record" C:\cycle_log\BMC_sel_temp.log | cut -d "|" -f 4- | sort | uniq -c >> C:\cycle_log\bmc_sel_counter_1.log
grep "OEM record" C:\cycle_log\BMC_sel_temp.log | cut -d "|" -f 4- | sort | uniq -c >> C:\cycle_log\bmc_sel_counter_1.log

del C:\cycle_log\bmc_sel_temp.log

Compared from above two filtered logs you can see that there are some difference, seems your code still got bugs on counting.

Your help is highly appreciated, thanks so much for your help, so, could you please help to update your code for solving the bugs, also, could you please also add string soring by A-Z when saving it to filter.log file?

Again, your help is highly appreciated!

Thanks.

cycle_log.zip

bmc_sel_counter_1.log

Edited July 2, 2019 by jackylee0908

[Solved] How to record different string and count repeated string in a loop test?

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members