Jump to content

Read Binary index.dat file


Recommended Posts

The goal here is to get the Internet History for endpoints and users in the enterprise to find what business use websites use Java.

When opening index.dat in Notepad, I can see all the data I need. Unfortunatly, trying to read that data in AutoIT is proving diffucult.

I've tried :

Func _ParseIndexdat($indexdatpath)
; Parse index.dat file for useable info
; The tools I've seen don't grab all the info I want :(
$Bindexdat = FileOpen($indexdatpath,16)
$indexdat = FileRead($Bindexdat)
$strIndexdat = BinaryToString($indexdat,2)
MsgBox(0,"String from Binary",$strIndexdat)
$FileArray = StringSplit($strIndexdat,@CRLF)
;This may get complex...
Dim $r = 1 ; to count the records
Dim $e = 0; to count the entries

; Start reading from line 1
For $line In $FileArray
$content = StringStripWS($line,7)
  Select
   Case StringInStr($line,"REDR")
    ; this is the start of a record
    MsgBox(0,"REDR",$content)
   Case StringInStr($line,"URL")
    ; this may be the start of a record
    MsgBox(0,"URL",$content)
   Case StringInStr($line,"LEAK")
    ; this is the start of an error record
    ; I will likely ignore it.
    MsgBox(0,"LEAK",$content)
   Case StringInStr($line,"http://")
    ; this is the line with a couple entries
    MsgBox(0,"http",$content)
   Case StringInStr($line,"Content-Type:")
    ; this is an entry I want
    MsgBox(0,"Content-Type",$content)
   Case StringInStr($line,"X-Powered-By:")
    ; this is an entry I want
    MsgBox(0,"Powered-By",$content)
   Case StringInStr($line,"~U:")
    ; this is an entry I want and it marks the end of a record
    MsgBox(0,"~U",$content)
   Case Else
    ; do nothing with the line
  EndSelect
Next
EndFunc

That doesn't get the info... That code doesn't return anything.

Using

$Bindexdat = FileOpen($indexdatpath,16)
$indexdat = FileRead($Bindexdat)
$strIndexdat = BinaryToString($indexdat,1)
MsgBox(0,"String from Binary",$strIndexdat)

the "string from binary" msg box shows a LOT of data... but URL and http case are both blank or gibberish. BinaryToString($indexdat,2) and ,3 return Nothing in the cases - 4 does the same as 1.

Using :

$Bindexdat = FileOpen($indexdatpath, 16)
$indexdat = FileRead($Bindexdat);$indexdatpath)
MsgBox(0,"Index dat",$indexdat)
$strIndexdat = BinaryToString($indexdat,1)
MsgBox(0,"String from Binary",$indexdat);$strIndexdat & @CRLF & @error)
$FileArray = StringSplit($strIndexdat,@CRLF)

Shows that $indexdat and $strIndexdat are effectively the same.

Using :

; Parse index.dat file for useable info
; The tools I've seen don't grab all the info I want :(
;$Bindexdat = FileOpen($indexdatpath,16)
$indexdat = FileRead($indexdatpath);$Bindexdat)
;$strIndexdat = BinaryToString($indexdat,4)
MsgBox(0,"String from Binary",$indexdat);$strIndexdat & @CRLF & @error)
$FileArray = StringSplit($indexdat,@CRLF)

It reads a lot more info... Content-Type is fine. ~U: is fine. URL is mostly blank.

Case http:// it will pop up the msg box, but $content is blank. So the data is there... I just can't figure out how to get it into a string.

Using :

$Bindexdat = FileOpen($indexdatpath,256)
$indexdat = FileRead($Bindexdat);$indexdatpath)
MsgBox(0,"Index dat",$indexdat)
$strIndexdat = BinaryToString($indexdat,1)
MsgBox(0,"String from Binary",$indexdat);$strIndexdat & @CRLF & @error)
$FileArray = StringSplit($strIndexdat,@CRLF)

Returns data, but URL and Http:// pop up as blank...

$Bindexdat = FileOpen($indexdatpath,48) - URL and Http are blank.

I've found a VBScript that is supposed to read the files... so far no luck on Win7 - it can't find index.dat file... lol! On remote XP machines, it can't find a history folder. So I don't know if it DOES read index.dat files.

I've attached it.

Any ideas or code help is greatly appreciated!!

IE_Network - Copy.txt

Edited by Graywalker
Link to comment
Share on other sites

I altered the vbscript to point directly to the index.dat file and it gets info, but returns all kinds of gibberish.

... that totally messes up the reply even when pasted as code!

So, I've attached it as a screen cap.

post-38206-0-27179900-1327335952_thumb.p

Edited by Graywalker
Link to comment
Share on other sites

Okay, I've found out that the data IS there. I've just got to find a way to get TO it.

Func _ParseIndexdat($indexdatpath)
; Parse index.dat file for useable info
; The tools I've seen don't grab all the info I want :(
$Bindexdat = FileOpen($indexdatpath, 16)
$indexdat = FileRead($Bindexdat);$indexdatpath)
;MsgBox(0, "Index dat", $indexdat)
$strIndexdat = BinaryToString($indexdat, 1)
$strIndexdat = StringStripWS($strIndexdat, 7)
;MsgBox(0, "String from Binary", $strIndexdat);$strIndexdat & @CRLF & @error)
$FileArray = StringSplit($strIndexdat, @CRLF)
;This may get complex...
Dim $r = 1 ; to count the records
Dim $e = 0; to count the entries
; Start reading from line 1
For $line In $FileArray
  $line = StringReplace($line, @CRLF, "")
  $line = StringReplace($line, @CR, "")
  $line = StringReplace($line, @LF, "")
  Select
   Case StringInStr($line, "http://")
    ; this is the line with a couple entries
    ;MsgBox(0, "http", $line)
    ; Check to see if URL is in the line
    If StringInStr($line, "URL") Then
     ; it is a REDR or LEAK, trim to URL
     $urlpos = StringInStr($line, "URL")
     $line = StringTrimLeft($line, $urlpos)
    EndIf
    $httppos = StringInStr($line, "http")
    $line = StringTrimLeft($line, $httppos - 1)
    $dotpos = StringInStr($line, ".", "", 3)
    If $dotpos > 10 Then
     $content = $line
     $linelen = StringLen($line)
     $trimfromright = $linelen - ($dotpos + 3)
     $line = StringTrimRight($line, $trimfromright)
     ;MsgBox(0,"Trim", "string lenght : " & $linelen & @CRLF & "dot position: " & $dotpos & @CRLF & "Trim from Right : " & _
     ;$trimfromright & @CRLF & $line)
    Else
     ; nothing
    EndIf
    $record = $line & ","
   Case StringInStr($line, "Content-Type:")
    ; this is an entry I want
    $line = StringStripWS($line, 7)
    $record = $record & $line & ","
   Case StringInStr($line, "X-Powered-By:")
    ; this is an entry I want
    $line = StringStripWS($line, 7)
    $record = $record & $line & ","
   Case StringInStr($line, "~U:")
    ; this is an entry I want and it marks the end of a record
    $line = StringReplace($line, "~U:", "")
    $line = StringStripWS($line, 7)
    $record = $record & $line
    FileWriteLine($logfile, $record)
    $record = ""
   Case Else
    ; do nothing with the line
  EndSelect
Next
EndFunc   ;==>_ParseIndexdat

I am getting fairly spotty results - leaving in lots of random characters before and after the URLs.

Could anyone come up with a way to use StringRegExp to pull the URLs - all of them - from a line?

I've tried several from : http://regexlib.com/Search.aspx?k=URL&AspxAutoDetectCookieSupport=1

with no consistent luck.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...