IIS FTP log parser

[james3mg]

Hello again AutoIt world.

Here's a script I wrote that serves me very well, but your mileage may vary. Consider this more of a starting point unless you have your FTP server configured just like mine. It prompts you for a log file *.log such as those created when logging is turned on in IIS for FTP access. This script expects headers in the following order:

  Quote

#Fields: date time c-ip cs-username s-port cs-method cs-uri-stem cs-uri-query sc-status sc-win32-status sc-bytes cs-bytes time-taken cs-version

You may need to change a few lines in the script if your logging is different (look for the lines with $CurrLine[x] and replace x with the number of that field.

My typical log files are around 100MB per month, and it takes about 2 hours to parse that size file. It's very beneficial for me however: I can view:

-successful transfer attempts by username and date to get the full log entry,

-unsuccessful transfer attempts by username and date (no full entry - they failed, so who cares about the details...you can uncomment a few noted lines to get the details here too, but remember you're limited to 4096 controls in the window - each entry is a control), or

-attempts by IP address, separated by successful and unsuccessful, then username - successful attempts have full log entry details also.

Also, the "By IP Address" tree is color-coded based on number of successful to failed attempts on a single IP address within a single log file:

-Green means likely legit user (successful login and acceptable number of failed transfers [200 by default, though you can change the noted line in the script to raise/lower this threshold],

-Yellow means likely hack attempt, but it was unsuccessful (zero successful transfers and more than [200 by default] failed transfers)

-White means possible hack attempt, but it was unsuccessful (zero successful transfers and less than [200 by default] failed transfers - possibly just mistakes, but probably just someone at a keyboard trying to gain entry)

-Red indicates a likely successful hack (more than [200 by default] failed transfers, but one or more successful transfers as well, indicating that they probably tried and were eventually successful in guessing a password, then used that from there on). If this condition is found, an audible alarm will also beep out of your computer every 2 seconds, making this script potentially useful as a scheduled task on a server (you'd have to change the file open dialog routine to accept a commandline parameter).

As I said, this works for me (which is why I wrote it, isn't it?), but it may not work with your log files. A bit of tweaking should be expected if you want to use it. It's fairly self-explanatory when you get into it, however. So this is a jumping-off point for admins who want it.

Here you go; hope it's useful to you.

expandcollapse popup

#include <String.au3>
#include <GUIConstants.au3>
Global $LogFile=FileOpenDialog("Open log file",@ScriptDir,"log files (*.log)")
$FileSize=FileGetSize($LogFile)
If Not FileExists($LogFile) Then
    Exit
EndIf
$LogFile=FileOpen($LogFile,0)
If @error Then Exit

Global $OKFailNumber=200;change this as necessary
Global $Alarm=0
AdlibEnable("AlarmAdlib",2000)

Global $TreeGUI=GUICreate("Log file examiner (0%)",640,480,-1,-1,$GUI_SS_DEFAULT_GUI+$WS_MAXIMIZEBOX+$WS_SIZEBOX)
Global $TreeView=GUICtrlCreateTreeView(0,0,640,480)
Global $SuccessfulTree=GUICtrlCreateTreeViewItem("Successful transactions (0)",$TreeView)
Global $UnsuccessfulTree=GUICtrlCreateTreeViewItem("Unsuccessful transactions (0)",$TreeView)
Global $ByIpTree=GUICtrlCreateTreeViewItem("By IP address",$TreeView)

GUISetState()
$BytesRead=0
$TitleTimer=TimerInit()

While 1
    Global $CurrLineStr=FileReadLine($LogFile)
    If @error Then ExitLoop;end of log file reached
    $BytesRead+=StringLen($CurrLineStr);bytes into the file I've read: for comparison to $filesize to get percentage read (displayed in title bar)
    If TimerDiff($TitleTimer) > 10000 Then;update the percentage every 10 seconds
        WinSetTitle(WinGetTitle($TreeGUI),"",StringLeft(WinGetTitle($TreeGUI),StringInStr(WinGetTitle($TreeGUI),"("))&Round(($BytesRead/$FileSize)*100,2)&"%)")
        $TitleTimer=TimerInit()
    EndIf
    Global $CurrLine=StringSplit($CurrLineStr," ");break the current line in the log into its columns
    If @error OR StringLeft($CurrLine[1],1) = "#" OR $CurrLine[4] = "-" Then ContinueLoop;invalid log entry, comment, or password prompt: ignore this line
    
    $CurrDate=$CurrLine[1];change these lines if your log is different
    $CurrUser=$CurrLine[4]
    
    If StringLeft($CurrLine[9],1)=2 Then;file it in the successful transactions tree
        If NOT IsDeclared("Success"&$CurrUser) Then
            Assign("Success"&$CurrUser,GUICtrlCreateTreeViewItem($CurrUser&" (0)",$SuccessfulTree),2)
        EndIf
        If NOT IsDeclared("Success"&$CurrUser&$CurrDate) Then
            Assign("Success"&$CurrUser&$CurrDate,GUICtrlCreateTreeViewItem($CurrDate&" (0)",Eval("Success"&$CurrUser)),2)
        EndIf
        GUICtrlCreateTreeViewItem($CurrLineStr,Eval("Success"&$CurrUser&$CurrDate))     ;comment for VERY large or very active legit user activity: Autoit limits number of controls to 4096
        DoCountIncrease($SuccessfulTree)
        DoCountIncrease(Eval("Success"&$CurrUser))
        DoCountIncrease(Eval("Success"&$CurrUser&$CurrDate))

    Else;file it in the unsuccessful transactions tree
        If NOT IsDeclared("Failed"&$CurrUser) Then
            Assign("Failed"&$CurrUser,GUICtrlCreateTreeViewItem($CurrUser&" (0)",$UnsuccessfulTree),2)
        EndIf
        If NOT IsDeclared("Failed"&$CurrUser&$CurrDate) Then
            Assign("Failed"&$CurrUser&$CurrDate,GUICtrlCreateTreeViewItem($CurrDate&" (0)",Eval("Failed"&$CurrUser)),2)
        EndIf
        ;GUICtrlCreateTreeViewItem($CurrLineStr,Eval("Failed"&$CurrUser&$CurrDate))     ;uncomment for very small log file: otherwise too many controls are created (AutoIt limit of 4096)
        DoCountIncrease($UnsuccessfulTree)
        DoCountIncrease(Eval("Failed"&$CurrUser))
        DoCountIncrease(Eval("Failed"&$CurrUser&$CurrDate))
    EndIf
    
    If NOT IsDeclared("IP"&$CurrLine[3]) Then
        Assign("IP"&$CurrLine[3],GUICtrlCreateTreeViewItem($CurrLine[3]&" (0)",$ByIpTree),2)
        Assign("IP"&$CurrLine[3]&"Success",GUICtrlCreateTreeViewItem("Successful transactions (0)",Eval("IP"&$CurrLine[3])),2)
        Assign("IP"&$CurrLine[3]&"Failed",GUICtrlCreateTreeViewItem("Unsuccessful transactions (0)",Eval("IP"&$CurrLine[3])),2)
    EndIf
    If StringLeft($CurrLine[9],1)=2 Then;file it in the successful transaction branch of the ip address' tree
        If NOT IsDeclared("IP"&$CurrLine[3]&"Success"&$CurrUser) Then
            Assign("IP"&$CurrLine[3]&"Success"&$CurrUser,GUICtrlCreateTreeViewItem($CurrUser&" (0)",Eval("IP"&$CurrLine[3]&"Success")),2)
        EndIf
        GUICtrlCreateTreeViewItem($CurrLineStr,Eval("IP"&$CurrLine[3]&"Success"&$CurrUser))     ;comment for VERY large or very active legit user activity: Autoit limits number of controls to 4096
        DoCountIncrease(Eval("IP"&$CurrLine[3]))
        DoCountIncrease(Eval("IP"&$CurrLine[3]&"Success"))
        DoCountIncrease(Eval("IP"&$CurrLine[3]&"Success"&$CurrUser))
        $CurrIPFailedCount=_StringBetween(GUICtrlRead(Eval("IP"&$CurrLine[3]&"Failed"),1),"(",")")
        $CurrIPFailedCount=$CurrIPFailedCount[0]
        $CurrIPSuccessCount=_StringBetween(GUICtrlRead(Eval("IP"&$CurrLine[3]&"Success"),1),"(",")")
        $CurrIPSuccessCount=$CurrIPSuccessCount[0]
        If $CurrIPFailedCount > $OKFailNumber Then
            $Alarm=1
            GUICtrlSetBkColor(Eval("IP"&$CurrLine[3]),0xFF0000)
        Else
            GUICtrlSetBkColor(Eval("IP"&$CurrLine[3]),0x00FF00)
        EndIf
    Else;file it in the unsuccessful transaction branch of the ip address' tree
        If NOT IsDeclared("IP"&$CurrLine[3]&"Failed"&$CurrUser) Then
            Assign("IP"&$CurrLine[3]&"Failed"&$CurrUser,GUICtrlCreateTreeViewItem($CurrUser&" (0)",Eval("IP"&$CurrLine[3]&"Failed")),2)
        EndIf
        ;GUICtrlCreateTreeViewItem($CurrLineStr,Eval("IP"&$CurrLine[3]&"Failed"&$CurrUser))     ;uncomment for very small log file: otherwise too many controls are created (AutoIt limit of 4096)
        DoCountIncrease(Eval("IP"&$CurrLine[3]))
        DoCountIncrease(Eval("IP"&$CurrLine[3]&"Failed"))
        DoCountIncrease(Eval("IP"&$CurrLine[3]&"Failed"&$CurrUser))
        $CurrIPFailedCount=_StringBetween(GUICtrlRead(Eval("IP"&$CurrLine[3]&"Failed"),1),"(",")")
        $CurrIPFailedCount=$CurrIPFailedCount[0]
        $CurrIPSuccessCount=_StringBetween(GUICtrlRead(Eval("IP"&$CurrLine[3]&"Success"),1),"(",")")
        $CurrIPSuccessCount=$CurrIPSuccessCount[0]
        If $CurrIPFailedCount = $OKFailNumber Then GUICtrlSetBkColor(Eval("IP"&$CurrLine[3]),0xFFFF00)
        If $CurrIPFailedCount > $OKFailNumber AND $CurrIPSuccessCount > 0 Then
            $Alarm=1
            GUICtrlSetBkColor(Eval("IP"&$CurrLine[3]),0xFF0000)
        EndIf
    EndIf
    If GUIGetMsg()=-3 Then Exit
WEnd
    
WinSetTitle(WinGetTitle($TreeGUI),"","Log file examiner")

Do
    sleep(15)
Until GUIGetMsg()=-3

Func DoCountIncrease($whichControl)
    $CurrCount=_StringBetween(GUICtrlRead($whichControl,1),"(",")")
    If @error Then Return SetError(1)
    $CurrCount=$CurrCount[0]+1
    GUICtrlSetData($whichControl,StringLeft(GUICtrlRead($whichControl,1),StringInStr(GUICtrlRead($whichControl,1),"("))&$CurrCount&")")
EndFunc

Func AlarmAdlib()
    If $Alarm Then Beep()
EndFunc

Edited January 8, 2008 by james3mg

[ptrex]

@James3mg

Good job. But maybe you could have use a better tool to create the IIS Log Parser.

MS Log Parser COM gives you all the flexibity including graph output etc ...

MS LogParser

Regards

ptrex

[james3mg]

  ptrex said:

@James3mg

Good job. But maybe you could have use a better tool to create the IIS Log Parser.

MS Log Parser COM gives you all the flexibity including graph output etc ...

MS LogParser

Regards

ptrex

Good to know...thanks