Sign in to follow this  
Followers 0
steelsking

Detect almost duplicate log entries

3 posts in this topic

Hi,

I have the following log file.

2011-10-18 07:38:39 Local7.Notice 172.17.164.134 1671: Oct 18 07:45:40 SIN: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up

2011-10-18 07:38:40 Local7.Error 172.17.164.134 1672: Oct 18 07:45:40 SIN: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to down

2011-10-18 07:38:41 Local7.Notice 172.17.164.134 1673: Oct 18 07:45:41 SIN: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down

2011-10-18 07:38:41 Local7.Error 172.17.164.134 1674: Oct 18 07:45:42 SIN: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up

2011-10-18 07:38:43 Local7.Notice 172.17.164.134 1675: Oct 18 07:45:44 SIN: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up

2011-10-18 07:38:47 Local7.Error 172.17.164.134 1676: Oct 18 07:45:48 SIN: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to down

2011-10-18 07:38:48 Local7.Notice 172.17.164.134 1677: Oct 18 07:45:49 SIN: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down

2011-10-18 07:38:49 Local7.Error 172.17.164.134 1678: Oct 18 07:45:49 SIN: %LINK-3-UPDOWN: Interface FastEthernet0/1, changed state to up

2011-10-18 07:38:49 Local7.Notice 172.17.164.134 1679: Oct 18 07:45:50 SIN: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to up

What I need to do is to detect if any particular IP address is having an up and down status based on the logs files, within a stipulated time.

meaning to detect if within a space of 5 mins, if there are X numbers of ups and down, to sent an alert.

I have tried the following. Not sure if it is the tried easy logical method. I couldnt proceed further as I got stuck. Hope someone can guide me along.

Thanks.

#include <file.au3>
#include <Array.au3>
#include <Date.au3>
Global $YYMMDD = StringMid(@YEAR & @MON & @MDAY, 3)
Global $inifile = @ScriptDir & "\settings.ini"
Global $Search = IniReadSection($inifile, "Search")
Global $Temp = @ScriptDir & "\tempdb" & "_" & $YYMMDD & ".txt"
Dim $aRecords, $aTemp
If Not _FileReadToArray("error.log", $aRecords) Then
MsgBox(4096, "Error", "Error reading log to Array error: " & @error)
Exit
EndIf
If Not FileExists(@ScriptDir & "\tempDB" & "_" & $YYMMDD & ".txt") Then
FileInstall("temp.txt", @ScriptDir & "\tempDB" & "_" & $YYMMDD & ".txt", 1)
IniWrite($inifile, "config", "LastEOF", "0")
;Exit
EndIf
For $j = 1 To $Search[0][0]
For $x = 1 To $aRecords[0]
  $aParts = StringSplit($aRecords[$x], @TAB)
  $result = StringInStr($aParts[4], $Search[$j][1])
  If $result = 0 Then
   $DT = StringReplace($aParts[1], "-", "/")
   $Combine = $DT & @TAB & $aParts[3]
   If Not _FileReadToArray($Temp, $aTemp) Then
    MsgBox(4096, "Error", "Error reading tempDB to Array error: " & @error)
    Exit
   EndIf
   If (_ArraySearch($aTemp, $Combine) = -1) Then
    FileWriteLine($Temp, $Combine)
   Else
    For $i = 1 To $aTemp[0] Step 1
    $aParts2 = StringSplit($aTemp[$i], @TAB)
    If IsArray($aParts2) And ($aParts2[2] = $aParts[3]) Then
     MsgBox(0, "", "Found.")
     FileWriteLine($Temp, $Combine)
    EndIf
    Next
   EndIf
  EndIf
Next
Next
Exit

ini file

[Config]

FileLocName="error.log"

[search]

1=changed state to up

2=changed state to down

[settings]

Frequency=10

Threshold=5

Share this post


Link to post
Share on other sites



steelsking,

I have made the assumption that the state changes are always the reverse of the previous - i.e the order is always up-down-up-down-up-down.... If that is the case then this seems to work: ;)

#include <File.au3>
#Include <Date.au3>
#include <Array.au3>
 
Global $sIni = "settings.ini"
Global $aLines
 
; Read log file
$sLog = IniRead($sIni, "Config", "FileLocName", "")
_FileReadToArray($sLog, $aLines)
 
; Create a suitably sized 2D array
Global $aSelection[$aLines[0] + 1][2] = [[$aLines[0], 0]]
$sStartDTG = ""
 
; Fill 2D array with time difference and actual DTG
For $i = 1 To $aSelection[0][0]
    If StringInStr($aLines[$i], "%LINEPROTO-5-UPDOWN") Then ; I have assumed that we ned only look for lines containing this text <<<<<<
        If $sStartDTG = "" Then
            $sStartDTG = StringLeft($aLines[$i], 19)
            $aSelection[$i][0] = StringFormat("%08i", 0)
            $aSelection[$i][1] = $sStartDTG
        Else
            $sDTG = StringLeft($aLines[$i], 19)
            $aSelection[$i][0] = StringFormat("%08i", _DateDiff("s", $sStartDTG, $sDTG))
            $aSelection[$i][1] = $sDTG
        EndIf
    EndIf
Next
 
; Remove all blank lines from the 2D array
For $i = $aSelection[0][0] To 1 Step -1
    If $aSelection[$i][0] = "" Then
        _ArrayDelete($aSelection, $i)
        $aSelection[0][0] -= 1
    EndIf
Next
 
; Read the required values from the ini file
$iThreshold = Number(IniRead($sIni, "Settings", "Threshold", 0)) ; assumed seconds for checking <<<<<<<<<<<<
$iEventFreq = Number(IniRead($sIni, "Settings", "Frequency", 0)) ; set to 2 for checking <<<<<<<<<<<<<<<<<<<< You need to alter your ini file
 
; Now run through the array checking the Frequency of events in every Threshold period
For $i = 1 To $aSelection[0][0]
    $iStartValue = $aSelection[$i][0]
    For $j = $i + 1 To $aSelection[0][0]
        ; If we reach the Threshold period
        If $aSelection[$j][0] - $aSelection[$i][0] > $iThreshold Then
            ; Count the events and see if they meet/exceed the required frequency
            $iFreq = $j - $i
            If $iFreq >= $iEventFreq Then
                ; If so then announce it
                ConsoleWrite($iFreq & " events between " & $aSelection[$i][1] & " and " & $aSelection[$j][1] & @CRLF)
            EndIf
            ExitLoop
        EndIf
    Next
Next

Note I have changed the Frequency value in the ini file to get sensible returns from the short log file section you posted. You would have to either multiply $iThreshold by 60 or change the _DateDiff Type parameter to get minutes. :)

Does it help? :graduated:

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

hi melba23,

thanks alot!!! it works!!!

sorry for the delay, was testing it out.

Thanks!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0