Jump to content
Sign in to follow this  
james3mg

IIS FTP log parser

Recommended Posts

james3mg

Hello again AutoIt world.

Here's a script I wrote that serves me very well, but your mileage may vary. Consider this more of a starting point unless you have your FTP server configured just like mine. It prompts you for a log file *.log such as those created when logging is turned on in IIS for FTP access. This script expects headers in the following order:

#Fields: date time c-ip cs-username s-port cs-method cs-uri-stem cs-uri-query sc-status sc-win32-status sc-bytes cs-bytes time-taken cs-version

You may need to change a few lines in the script if your logging is different (look for the lines with $CurrLine[x] and replace x with the number of that field.

My typical log files are around 100MB per month, and it takes about 2 hours to parse that size file. It's very beneficial for me however: I can view:

-successful transfer attempts by username and date to get the full log entry,

-unsuccessful transfer attempts by username and date (no full entry - they failed, so who cares about the details...you can uncomment a few noted lines to get the details here too, but remember you're limited to 4096 controls in the window - each entry is a control), or

-attempts by IP address, separated by successful and unsuccessful, then username - successful attempts have full log entry details also.

Also, the "By IP Address" tree is color-coded based on number of successful to failed attempts on a single IP address within a single log file:

-Green means likely legit user (successful login and acceptable number of failed transfers [200 by default, though you can change the noted line in the script to raise/lower this threshold],

-Yellow means likely hack attempt, but it was unsuccessful (zero successful transfers and more than [200 by default] failed transfers)

-White means possible hack attempt, but it was unsuccessful (zero successful transfers and less than [200 by default] failed transfers - possibly just mistakes, but probably just someone at a keyboard trying to gain entry)

-Red indicates a likely successful hack (more than [200 by default] failed transfers, but one or more successful transfers as well, indicating that they probably tried and were eventually successful in guessing a password, then used that from there on). If this condition is found, an audible alarm will also beep out of your computer every 2 seconds, making this script potentially useful as a scheduled task on a server (you'd have to change the file open dialog routine to accept a commandline parameter).

post-5585-1195676819_thumb.jpg

As I said, this works for me (which is why I wrote it, isn't it?), but it may not work with your log files. A bit of tweaking should be expected if you want to use it. It's fairly self-explanatory when you get into it, however. So this is a jumping-off point for admins who want it.

Here you go; hope it's useful to you.

#include <String.au3>
#include <GUIConstants.au3>
Global $LogFile=FileOpenDialog("Open log file",@ScriptDir,"log files (*.log)")
$FileSize=FileGetSize($LogFile)
If Not FileExists($LogFile) Then
    Exit
EndIf
$LogFile=FileOpen($LogFile,0)
If @error Then Exit

Global $OKFailNumber=200;change this as necessary
Global $Alarm=0
AdlibEnable("AlarmAdlib",2000)

Global $TreeGUI=GUICreate("Log file examiner (0%)",640,480,-1,-1,$GUI_SS_DEFAULT_GUI+$WS_MAXIMIZEBOX+$WS_SIZEBOX)
Global $TreeView=GUICtrlCreateTreeView(0,0,640,480)
Global $SuccessfulTree=GUICtrlCreateTreeViewItem("Successful transactions (0)",$TreeView)
Global $UnsuccessfulTree=GUICtrlCreateTreeViewItem("Unsuccessful transactions (0)",$TreeView)
Global $ByIpTree=GUICtrlCreateTreeViewItem("By IP address",$TreeView)

GUISetState()
$BytesRead=0
$TitleTimer=TimerInit()

While 1
    Global $CurrLineStr=FileReadLine($LogFile)
    If @error Then ExitLoop;end of log file reached
    $BytesRead+=StringLen($CurrLineStr);bytes into the file I've read: for comparison to $filesize to get percentage read (displayed in title bar)
    If TimerDiff($TitleTimer) > 10000 Then;update the percentage every 10 seconds
        WinSetTitle(WinGetTitle($TreeGUI),"",StringLeft(WinGetTitle($TreeGUI),StringInStr(WinGetTitle($TreeGUI),"("))&Round(($BytesRead/$FileSize)*100,2)&"%)")
        $TitleTimer=TimerInit()
    EndIf
    Global $CurrLine=StringSplit($CurrLineStr," ");break the current line in the log into its columns
    If @error OR StringLeft($CurrLine[1],1) = "#" OR $CurrLine[4] = "-" Then ContinueLoop;invalid log entry, comment, or password prompt: ignore this line
    
    $CurrDate=$CurrLine[1];change these lines if your log is different
    $CurrUser=$CurrLine[4]
    
    If StringLeft($CurrLine[9],1)=2 Then;file it in the successful transactions tree
        If NOT IsDeclared("Success"&$CurrUser) Then
            Assign("Success"&$CurrUser,GUICtrlCreateTreeViewItem($CurrUser&" (0)",$SuccessfulTree),2)
        EndIf
        If NOT IsDeclared("Success"&$CurrUser&$CurrDate) Then
            Assign("Success"&$CurrUser&$CurrDate,GUICtrlCreateTreeViewItem($CurrDate&" (0)",Eval("Success"&$CurrUser)),2)
        EndIf
        GUICtrlCreateTreeViewItem($CurrLineStr,Eval("Success"&$CurrUser&$CurrDate))     ;comment for VERY large or very active legit user activity: Autoit limits number of controls to 4096
        DoCountIncrease($SuccessfulTree)
        DoCountIncrease(Eval("Success"&$CurrUser))
        DoCountIncrease(Eval("Success"&$CurrUser&$CurrDate))

    Else;file it in the unsuccessful transactions tree
        If NOT IsDeclared("Failed"&$CurrUser) Then
            Assign("Failed"&$CurrUser,GUICtrlCreateTreeViewItem($CurrUser&" (0)",$UnsuccessfulTree),2)
        EndIf
        If NOT IsDeclared("Failed"&$CurrUser&$CurrDate) Then
            Assign("Failed"&$CurrUser&$CurrDate,GUICtrlCreateTreeViewItem($CurrDate&" (0)",Eval("Failed"&$CurrUser)),2)
        EndIf
        ;GUICtrlCreateTreeViewItem($CurrLineStr,Eval("Failed"&$CurrUser&$CurrDate))     ;uncomment for very small log file: otherwise too many controls are created (AutoIt limit of 4096)
        DoCountIncrease($UnsuccessfulTree)
        DoCountIncrease(Eval("Failed"&$CurrUser))
        DoCountIncrease(Eval("Failed"&$CurrUser&$CurrDate))
    EndIf
    
    If NOT IsDeclared("IP"&$CurrLine[3]) Then
        Assign("IP"&$CurrLine[3],GUICtrlCreateTreeViewItem($CurrLine[3]&" (0)",$ByIpTree),2)
        Assign("IP"&$CurrLine[3]&"Success",GUICtrlCreateTreeViewItem("Successful transactions (0)",Eval("IP"&$CurrLine[3])),2)
        Assign("IP"&$CurrLine[3]&"Failed",GUICtrlCreateTreeViewItem("Unsuccessful transactions (0)",Eval("IP"&$CurrLine[3])),2)
    EndIf
    If StringLeft($CurrLine[9],1)=2 Then;file it in the successful transaction branch of the ip address' tree
        If NOT IsDeclared("IP"&$CurrLine[3]&"Success"&$CurrUser) Then
            Assign("IP"&$CurrLine[3]&"Success"&$CurrUser,GUICtrlCreateTreeViewItem($CurrUser&" (0)",Eval("IP"&$CurrLine[3]&"Success")),2)
        EndIf
        GUICtrlCreateTreeViewItem($CurrLineStr,Eval("IP"&$CurrLine[3]&"Success"&$CurrUser))     ;comment for VERY large or very active legit user activity: Autoit limits number of controls to 4096
        DoCountIncrease(Eval("IP"&$CurrLine[3]))
        DoCountIncrease(Eval("IP"&$CurrLine[3]&"Success"))
        DoCountIncrease(Eval("IP"&$CurrLine[3]&"Success"&$CurrUser))
        $CurrIPFailedCount=_StringBetween(GUICtrlRead(Eval("IP"&$CurrLine[3]&"Failed"),1),"(",")")
        $CurrIPFailedCount=$CurrIPFailedCount[0]
        $CurrIPSuccessCount=_StringBetween(GUICtrlRead(Eval("IP"&$CurrLine[3]&"Success"),1),"(",")")
        $CurrIPSuccessCount=$CurrIPSuccessCount[0]
        If $CurrIPFailedCount > $OKFailNumber Then
            $Alarm=1
            GUICtrlSetBkColor(Eval("IP"&$CurrLine[3]),0xFF0000)
        Else
            GUICtrlSetBkColor(Eval("IP"&$CurrLine[3]),0x00FF00)
        EndIf
    Else;file it in the unsuccessful transaction branch of the ip address' tree
        If NOT IsDeclared("IP"&$CurrLine[3]&"Failed"&$CurrUser) Then
            Assign("IP"&$CurrLine[3]&"Failed"&$CurrUser,GUICtrlCreateTreeViewItem($CurrUser&" (0)",Eval("IP"&$CurrLine[3]&"Failed")),2)
        EndIf
        ;GUICtrlCreateTreeViewItem($CurrLineStr,Eval("IP"&$CurrLine[3]&"Failed"&$CurrUser))     ;uncomment for very small log file: otherwise too many controls are created (AutoIt limit of 4096)
        DoCountIncrease(Eval("IP"&$CurrLine[3]))
        DoCountIncrease(Eval("IP"&$CurrLine[3]&"Failed"))
        DoCountIncrease(Eval("IP"&$CurrLine[3]&"Failed"&$CurrUser))
        $CurrIPFailedCount=_StringBetween(GUICtrlRead(Eval("IP"&$CurrLine[3]&"Failed"),1),"(",")")
        $CurrIPFailedCount=$CurrIPFailedCount[0]
        $CurrIPSuccessCount=_StringBetween(GUICtrlRead(Eval("IP"&$CurrLine[3]&"Success"),1),"(",")")
        $CurrIPSuccessCount=$CurrIPSuccessCount[0]
        If $CurrIPFailedCount = $OKFailNumber Then GUICtrlSetBkColor(Eval("IP"&$CurrLine[3]),0xFFFF00)
        If $CurrIPFailedCount > $OKFailNumber AND $CurrIPSuccessCount > 0 Then
            $Alarm=1
            GUICtrlSetBkColor(Eval("IP"&$CurrLine[3]),0xFF0000)
        EndIf
    EndIf
    If GUIGetMsg()=-3 Then Exit
WEnd
    
WinSetTitle(WinGetTitle($TreeGUI),"","Log file examiner")

Do
    sleep(15)
Until GUIGetMsg()=-3

Func DoCountIncrease($whichControl)
    $CurrCount=_StringBetween(GUICtrlRead($whichControl,1),"(",")")
    If @error Then Return SetError(1)
    $CurrCount=$CurrCount[0]+1
    GUICtrlSetData($whichControl,StringLeft(GUICtrlRead($whichControl,1),StringInStr(GUICtrlRead($whichControl,1),"("))&$CurrCount&")")
EndFunc

Func AlarmAdlib()
    If $Alarm Then Beep()
EndFunc
Edited by james3mg

"There are 10 types of people in this world - those who can read binary, and those who can't.""We've heard that a million monkeys at a million keyboards could produce the complete works of Shakespeare; now, thanks to the Internet, we know that is not true." ~Robert Wilensky0101101 1001010 1100001 1101101 1100101 1110011 0110011 1001101 10001110000101 0000111 0001000 0001110 0001101 0010010 1010110 0100001 1101110

Share this post


Link to post
Share on other sites
ptrex

Share this post


Link to post
Share on other sites
james3mg

@James3mg

Good job. But maybe you could have use a better tool to create the IIS Log Parser.

MS Log Parser COM gives you all the flexibity including graph output etc ...

MS LogParser

Regards

ptrex

Good to know...thanks :P

"There are 10 types of people in this world - those who can read binary, and those who can't.""We've heard that a million monkeys at a million keyboards could produce the complete works of Shakespeare; now, thanks to the Internet, we know that is not true." ~Robert Wilensky0101101 1001010 1100001 1101101 1100101 1110011 0110011 1001101 10001110000101 0000111 0001000 0001110 0001101 0010010 1010110 0100001 1101110

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×