Sign in to follow this  
Followers 0
mburris

Removing Lines from text file that don't start with

6 posts in this topic

#1 ·  Posted (edited)

I have some log files that I need to remove all lines from, except for lines that start with a particular IP address (mine). I have a feeling there is a very simple way to do this, but I'm not really sure where to start.

Here is a sample of the log file I need to parse:

(if it wasn't for word wrap on this forum, all the lines would start with an IP address, as they do in the actual files.)

71.120.69.92 - - [22/Dec/2008:16:06:43 -0600] "GET /favicon.ico HTTP/1.1" 404 1544 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5"
71.120.69.92 - - [22/Dec/2008:16:06:46 -0600] "GET /favicon.ico HTTP/1.1" 404 1544 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5"
98.215.25.169 - - [22/Dec/2008:16:39:16 -0600] "GET /components/com_marketplace/slimbox/js/slimbox.js HTTP/1.1" 304 - "http://websitenameiremoved.com/index.php?option=com_marketplace&page=show_ad&catid=9&adid=33&Itemid=68" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9"
71.120.69.92 - - [22/Dec/2008:16:06:51 -0600] "GET /images/stories/magazines/current.pdf HTTP/1.1" 206 32984 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5"
71.120.69.92 - - [22/Dec/2008:16:06:51 -0600] "GET /images/stories/magazines/current.pdf HTTP/1.1" 206 32984 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5"
71.120.69.92 - - [22/Dec/2008:16:06:54 -0600] "GET /images/stories/magazines/current.pdf HTTP/1.1" 206 744078 "-" "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5"
12.43.88.121 - - [22/Dec/2008:16:07:05 -0600] "GET /index.php?option=com_weblinks&catid=17&Itemid=23 HTTP/1.1" 200 11036 "http://www.google.com/search?q=wheels+and+deals+-+decatur%2C+IL&rls=com.microsoft:en-us&ie=UTF-8&oe=UTF-8&startIndex=&startPage=1" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; InfoPath.2)"
12.43.88.121 - - [22/Dec/2008:16:07:05 -0600] "GET /templates/my_photo/css/ HTTP/1.1" 200 44 "http://websitenameiremoved.com/index.php?option=com_weblinks&catid=17&Itemid=23" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; InfoPath.2)"
98.215.25.169 - - [22/Dec/2008:16:39:16 -0600] "GET /components/com_marketplace/slimbox/js/slimbox.js HTTP/1.1" 304 - "http://websitenameiremoved.com/index.php?option=com_marketplace&page=show_ad&catid=9&adid=33&Itemid=68" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9"
12.43.88.121 - - [22/Dec/2008:16:07:05 -0600] "GET /templates/my_photo/src/mootools.v1.1.js HTTP/1.1" 200 42950 "http://websitenameiremoved.com/index.php?option=com_weblinks&catid=17&Itemid=23" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; InfoPath.2)"
12.43.88.121 - - [22/Dec/2008:16:07:05 -0600] "GET /templates/my_photo/src/sve.js HTTP/1.1" 200 3619 "http://websitenameiremoved.com/index.php?option=com_weblinks&catid=17&Itemid=23" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; InfoPath.2)"
98.215.25.169 - - [22/Dec/2008:16:39:16 -0600] "GET /index.php?option=com_marketplace&page=show_ad&catid=9&adid=33&Itemid=68 HTTP/1.1" 200 16891 "http://websitenameiremoved.com/index.php?option=com_marketplace&page=list&Itemid=68" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9"
98.215.25.169 - - [22/Dec/2008:16:39:16 -0600] "GET /components/com_marketplace/slimbox/js/slimbox.js HTTP/1.1" 304 - "http://websitenameiremoved.com/index.php?option=com_marketplace&page=show_ad&catid=9&adid=33&Itemid=68" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.9) Gecko/20071025 Firefox/2.0.0.9"

Thanks,

Matt

Edited by mburris

Share this post


Link to post
Share on other sites



Try this:

$OPEN = FileOpenDialog("OPEN",@ScriptDir,"All (*.*)",1)
$SAVE = FileSaveDialog("SAVE","","Text (*.txt)")
$IP = InputBox("IP","Type here IP address")
$FILE_OPEN = FileOpen($OPEN,0)
$FILE_SAVE = FileOpen($SAVE & ".TXT",2)
$INDEX = 1
While 1
    $LINE = FileReadLine($FILE_OPEN,$INDEX)
    $INDEX += 1
    If @error = -1 Then ExitLoop
    If StringInStr($LINE,$IP) <> 0 Then FileWriteLine($FILE_SAVE,$LINE)
WEnd
FileClose($FILE_SAVE)
FileClose($FILE_OPEN)

When the words fail... music speaks

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

Thats in credible... thank you so much for the fast reply. it works like a charm!!!

Would a progress bar be doable?

Edited by mburris

Share this post


Link to post
Share on other sites

Thats in credible... thank you so much for the fast reply. it works like a charm!!!

Would a progress bar be doable?

The fast method + progress:

$sFile = @ScriptDir & "\Test.txt"
$sRead = FileRead($sFile)

$IP = InputBox("IP","Type here IP address")
If @error Then Exit

$aString = StringSplit($sRead, @CRLF, 1)
If Not IsArray($aString) Then Exit

$sResult = ""

For $i = 1 To $aString[0]
    If StringInStr($aString[$i], $IP, 2) Then $sResult &= $aString[$i] & @CRLF
    ConsoleWrite("!> " & Int($i / $aString[0] * 100) & "%" & @LF)
Next

If $sResult Then
    $hFile = FileOpen($sFile, 2)
    FileWrite($hFile, $sResult)
    FileClose($hFile)
EndIf

:)

Share this post


Link to post
Share on other sites

Hi.

This is a typical task to be solved in *ONE* line, with OS included capabilities:

; -------- start of e.g. "getmyip.cmd" -----------
find "71.120.69.92 " <original.log >myiponly.txt
; -------- end of e.g. "getmyip.cmd" -----------

Drop it beside your original log file and run it from there :)

Regards, Rudi.


Earth is flat, pigs can fly, and Nuclear Power is SAFE!

Share this post


Link to post
Share on other sites

Thats in credible... thank you so much for the fast reply. it works like a charm!!!

Would a progress bar be doable?

Try this example with progress bar:

#include <File.au3>
$OPEN = FileOpenDialog("OPEN",@ScriptDir,"All (*.*)",1)
$SAVE = FileSaveDialog("SAVE","","Text (*.txt)")
$IP = InputBox("IP","Type here IP address")
$LINES = _FileCountLines($OPEN)
$FILE_OPEN = FileOpen($OPEN,0)
$FILE_SAVE = FileOpen($SAVE & ".TXT",2)
$INDEX = 1
ProgressOn("Searching " & $IP,"Search...")
While 1
    $LINE = FileReadLine($FILE_OPEN,$INDEX)
    $PERCENT = (100*$INDEX)/$LINES
    ProgressSet($PERCENT)
    $INDEX += 1
    If @error = -1 Then ExitLoop
    If StringInStr($LINE,$IP) <> 0 Then FileWriteLine($FILE_SAVE,$LINE)
    Sleep(80)
WEnd
FileClose($FILE_SAVE)
FileClose($FILE_OPEN)
ProgressOff()

When the words fail... music speaks

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0