sublimnl Posted March 25, 2005 Share Posted March 25, 2005 I'm writing a script to parse a directory of IIS log files to find every time a user accesses an Exchange 2000 mailbox other than their own. Right now the script is hardcoded for a specific file and the log file that I have been testing with is ~10600 lines and about 1.9MB. This script takes over 10 minutes to parse the file line by line. It seems to start out fast and then get slower as I continue to read each consecutive line. Any ideas why? Thanks! expandcollapse popup#include <file.au3> $file = "C:\ex041230.log" $logfile = FileOpen ("C:\ex041230.log",0) $resultfile = FileOpen ( "C:\eResult.log", 2) $exchip = "172.16.4.3" ; ; This loop traverses files in the directory ; (not written yet) ; Do ; ; This loop reads the file ; $linecount=1 ProgressOn("Parsing...", "","Current File: " & $file,-1,-1,18) $totlines = _FileCountLines($file) Do If StringRight($linecount,2) = "00" Then $fileprog = Round(($linecount/$totlines)*100) ProgressSet($fileprog) EndIf $logline = FileReadLine($logfile, $linecount) If @error = -1 Then ExitLoop ; ;Check current line and make sure it accesses a mailbox ; ex. "/exchange/username" instead of "/exchange/ - 200 Exchange-Server-Frontend-Proxy" ; If StringInStr($logline, $exchip & " 80 GET /exchange/") > 0 Then If StringInStr($logline, " 80 GET /exchange/ ") = 0 Then $charcount=0 $charcount2=0 ; ;Get location of first character of Login ID ; Do $charcount=$charcount+1 $loginid=StringMid($logline,$charcount,1) Until StringIsAlpha($loginid)=1 ; ;Get last letter of Login ID, trim trailing whitespace ; Do $charcount2=$charcount2+1 $loginid=StringMid($logline,$charcount,$charcount2) Until StringRight($loginid,1) = " " $loginid=StringTrimRight($loginid,1) ; ;Write to results file is user is not accessing their own mailbox ; If StringInStr($logline,"GET /exchange/" & $loginid) = 0 Then FileWriteLine($resultfile, $logline) EndIf EndIf EndIf $linecount = $linecount + 1 Until $linecount = $totlines+1 ProgressOff() MsgBox(4096,"Done!","Done!") FileClose($logfile) FileClose($resultfile) Exit Here is a sample of the logfile. The first line is a match. 2004-12-30 00:23:36 172.16.4.46 carlos.aya 172.16.4.3 80 GET /exchange/Jim.Chase/Inbox/Ocean+Service+SMDL+survey+update+(0600+hours,+December+29,+2004).EML Cmd=open 200 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0) 2004-12-30 00:00:10 172.16.4.46 jim.chase 172.16.4.3 80 POLL /exchange/Jim.Chase/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0) 2004-12-30 00:00:31 172.16.4.46 jan.coyle 172.16.4.3 80 POLL /exchange/Jan.Coyle/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+YPC+3.0.1;+.NET+CLR+1.1.4322) 2004-12-30 00:00:33 172.16.4.43 - 172.16.4.3 80 HEAD / - 401 WhatsUp_Gold/7.0 2004-12-30 00:00:51 172.16.4.46 jason.taylor 172.16.4.3 80 POLL /exchange/Jason.Taylor/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0) 2004-12-30 00:01:08 172.16.4.46 armin.tavassoli 172.16.4.3 80 POLL /exchange/Armin.Tavassoli/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+Q312461;+GIL+2;+.NET+CLR+1.0.3705;+.NET+CLR+1.1.4322) 2004-12-30 00:01:33 172.16.4.43 - 172.16.4.3 80 HEAD / - 401 WhatsUp_Gold/7.0 2004-12-30 00:02:01 172.16.4.46 ben.wheeler 172.16.4.3 80 POLL /exchange/Ben.Wheeler/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322) 2004-12-30 00:02:10 172.16.4.46 jim.chase 172.16.4.3 80 POLL /exchange/Jim.Chase/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0) 2004-12-30 00:02:31 172.16.4.46 jan.coyle 172.16.4.3 80 POLL /exchange/Jan.Coyle/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+YPC+3.0.1;+.NET+CLR+1.1.4322) 2004-12-30 00:02:33 172.16.4.43 - 172.16.4.3 80 HEAD / - 401 WhatsUp_Gold/7.0 2004-12-30 00:02:51 172.16.4.46 jason.taylor 172.16.4.3 80 POLL /exchange/Jason.Taylor/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0) 2004-12-30 00:03:08 172.16.4.46 armin.tavassoli 172.16.4.3 80 POLL /exchange/Armin.Tavassoli/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+Q312461;+GIL+2;+.NET+CLR+1.0.3705;+.NET+CLR+1.1.4322) 2004-12-30 00:03:33 172.16.4.43 - 172.16.4.3 80 HEAD / - 401 WhatsUp_Gold/7.0 2004-12-30 00:04:02 172.16.4.46 ben.wheeler 172.16.4.3 80 POLL /exchange/Ben.Wheeler/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322) 2004-12-30 00:04:10 172.16.4.46 jim.chase 172.16.4.3 80 POLL /exchange/Jim.Chase/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0) 2004-12-30 00:04:31 172.16.4.46 jan.coyle 172.16.4.3 80 POLL /exchange/Jan.Coyle/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+YPC+3.0.1;+.NET+CLR+1.1.4322) 2004-12-30 00:04:33 172.16.4.43 - 172.16.4.3 80 HEAD / - 401 WhatsUp_Gold/7.0 2004-12-30 00:04:51 172.16.4.46 jason.taylor 172.16.4.3 80 POLL /exchange/Jason.Taylor/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0) 2004-12-30 00:05:08 172.16.4.46 armin.tavassoli 172.16.4.3 80 POLL /exchange/Armin.Tavassoli/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+Q312461;+GIL+2;+.NET+CLR+1.0.3705;+.NET+CLR+1.1.4322) 2004-12-30 00:05:33 172.16.4.43 - 172.16.4.3 80 HEAD / - 401 WhatsUp_Gold/7.0 2004-12-30 00:05:51 172.16.4.46 ben.wheeler 172.16.4.3 80 POLL /exchange/Ben.Wheeler/Calendar - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322) 2004-12-30 00:05:59 172.16.4.46 ben.wheeler 172.16.4.3 80 POLL /exchange/Ben.Wheeler/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1;+.NET+CLR+1.1.4322) 2004-12-30 00:06:11 172.16.4.46 jim.chase 172.16.4.3 80 POLL /exchange/Jim.Chase/Inbox - 207 Exchange-Server-Frontend-Proxy/6.0+Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0) Link to comment Share on other sites More sharing options...
Insolence Posted March 25, 2005 Share Posted March 25, 2005 Well before you process everything, maybe you should use _FileReadToArray and work through that. "I thoroughly disapprove of duels. If a man should challenge me, I would take him kindly and forgivingly by the hand and lead him to a quiet place and kill him." - Mark TwainPatient: "It hurts when I do $var_"Doctor: "Don't do $var_" - Lar. Link to comment Share on other sites More sharing options...
sublimnl Posted March 25, 2005 Author Share Posted March 25, 2005 Thanks for the tips! I'll give it a go. Link to comment Share on other sites More sharing options...
SlimShady Posted March 25, 2005 Share Posted March 25, 2005 Now I'm positive I won't use FileReadLine ever again. _FileReadToArray works much better, in every situation. Link to comment Share on other sites More sharing options...
sublimnl Posted March 25, 2005 Author Share Posted March 25, 2005 Now I'm positive I won't use FileReadLine ever again._FileReadToArray works much better, in every situation.<{POST_SNAPBACK}>AMEN TO THAT! What once took over 10 minutes now takes less than 10 seconds. Thanks! Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now