Jump to content

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Find out more here. X
X


Photo

MFT-Access - Reading & parsing the Master File Table on NTFS Filesystems


  • Please log in to reply
46 replies to this topic

#1 KaFu

KaFu

    Hey, it's just me, KhaFoo...

  • MVPs
  • 3,327 posts

Posted 02 May 2009 - 12:11 PM

HiHo Community,

for some time now I'm following on the topic of reading & parsing the Master File Table on NTFS Filesystems.

Though I believe I'm on the right track, it's quiet complex and time consuming. So I wanted to share what I have and invite those who are interested to contribute in creating an UDF in the long-run.

Besides the code I also attached an excellent reading I found on the topic containing all relevant information.

AutoIt         
;=============================================================================== ; ; Description:      MFT-Access, UDF under development ;                   Reading & parsing the Master File Table on NTFS Filesystems ; Version:          v0.0.0.3 ; Last modified:    2009-May-02 ; URL:              http://www.autoitscript.com/forum/index.php?showtopic=94269 ; Author(s):        KaFu (http://www.funk.eu), trancexx ; ;=============================================================================== ; #RequireAdmin #include <winapi.au3> Global  $sDriveLetter = "c", $nBytes, $MFT_Offset, $bytes_to_read, $tBuffer, $hdd_handle, $hFile ConsoleWrite("Analyzing Drive " & $sDriveLetter & ":, Filesystem detected: " & DriveGetFileSystem($sDriveLetter & ":") & @crlf & @crlf) if DriveGetFileSystem($sDriveLetter & ":") <> "NTFS" then exit ; Read Boot-Sector ; http://www.autoitscript.com/forum/index.php?showtopic=92529&view=findpost&p=665010 ; trancexx Global $hComIn = FileOpen("\\.\" & $sDriveLetter & ":", 16) Global $bRaw = FileRead($hComIn, 84); 84 bytes is enough ConsoleWrite(_HexEncode($bRaw) & @CRLF) FileClose($hComIn) Global $tRaw = DllStructCreate("byte [" & BinaryLen($bRaw) & "]") DllStructSetData($tRaw, 1, $bRaw) Global $tBootSectorSections = DllStructCreate("align 1;byte Jump[3];" & _         "char SystemName[8];" & _         "ushort BytesPerSector;" & _         "ubyte SectorsPerCluster;" & _         "ushort ReservedSectors;" & _         "ubyte[3];" & _         "ushort;" & _         "ubyte MediaDescriptor;" & _         "ushort;" & _         "ushort SectorsPerTrack;" & _         "ushort NumberOfHeads;" & _         "dword HiddenSectors;" & _         "dword;" & _         "dword;" & _         "int64 TotalSectors;" & _         "int64 LogicalClusterNumberforthefileMFT;" & _         "int64 LogicalClusterNumberforthefileMFTMirr;" & _         "dword ClustersPerFileRecordSegment;" & _         "dword ClustersPerIndexBlock;" & _         "int64 NTFSVolumeSerialNumber;" & _         "dword Checksum", _         DllStructGetPtr($tRaw)) ConsoleWrite("Jump:  " & DllStructGetData($tBootSectorSections, "Jump") & @CRLF) ConsoleWrite("SystemName:  " & DllStructGetData($tBootSectorSections, "SystemName") & @CRLF) ConsoleWrite("BytesPerSector:  " & DllStructGetData($tBootSectorSections, "BytesPerSector") & @CRLF) ConsoleWrite("SectorsPerCluster:  " & DllStructGetData($tBootSectorSections, "SectorsPerCluster") & @CRLF) ConsoleWrite("ReservedSectors:  " & DllStructGetData($tBootSectorSections, "ReservedSectors") & @CRLF) ;ConsoleWrite("6:  " & DllStructGetData($tBootSectorSections, 6) & @CRLF) ;ConsoleWrite("7:  " & DllStructGetData($tBootSectorSections, 7) & @CRLF) ConsoleWrite("MediaDescriptor:  " & DllStructGetData($tBootSectorSections, "MediaDescriptor") & @CRLF) ;ConsoleWrite("9:  " & DllStructGetData($tBootSectorSections, 9) & @CRLF) ConsoleWrite("SectorsPerTrack:  " & DllStructGetData($tBootSectorSections, "SectorsPerTrack") & @CRLF) ConsoleWrite("NumberOfHeads:  " & DllStructGetData($tBootSectorSections, "NumberOfHeads") & @CRLF) ConsoleWrite("HiddenSectors:  " & DllStructGetData($tBootSectorSections, "HiddenSectors") & @CRLF) ;ConsoleWrite("13:  " & DllStructGetData($tBootSectorSections, 13) & @CRLF) ;ConsoleWrite("14:  " & DllStructGetData($tBootSectorSections, 14) & @CRLF) ConsoleWrite("TotalSectors:  " & DllStructGetData($tBootSectorSections, "TotalSectors") & @CRLF) ConsoleWrite("LogicalClusterNumberforthefileMFT:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT") & @CRLF) ConsoleWrite("LogicalClusterNumberforthefileMFTMirr:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFTMirr") & @CRLF) ConsoleWrite("ClustersPerFileRecordSegment:  " & DllStructGetData($tBootSectorSections, "ClustersPerFileRecordSegment") & @CRLF) ConsoleWrite("ClustersPerIndexBlock:  " & DllStructGetData($tBootSectorSections, "ClustersPerIndexBlock") & @CRLF) ConsoleWrite("VolumeSerialNumber:  " & Ptr(DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber")) & @CRLF) ConsoleWrite("NTFSVolumeSerialNumber:  " & DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber") & @CRLF) ConsoleWrite("Checksum:  " & DllStructGetData($tBootSectorSections, "Checksum") & @CRLF) ConsoleWrite(@CRLF) ; ============================================= ; Seek & Read MFT ; ============================================= $MFT_Offset = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") * DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT") ConsoleWrite("$MFT_Offset: " & $MFT_Offset & @CRLF & @CRLF) $MFT_Record_Size = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") / 4 ; ClustersPerFileRecordSegment = 246 ; ConsoleWrite("0x" & hex(246,2)) = 0xF6 >>> 0xF6 = 1/4 Cluster $tBuffer = DllStructCreate("byte[" & $MFT_Record_Size & "]") $hdd_handle = "\\.\" & $sDriveLetter & ":" ConsoleWrite($hdd_handle & @crlf) $hFile = _WinAPI_CreateFile($hdd_handle, 2, 6, 6) ;_stprintf(szDriveCreate, _T(\\\\.\\%c), cDrive); ; handle = CreateFile("\\.\PhysicalDrive" & "0", GENERIC_READ + GENERIC_WRITE, FILE_SHARE_READ + FILE_SHARE_WRITE, 0, OPEN_EXISTING, 0, 0) ConsoleWrite("Harddisk Handle: " & $hFile & @crlf) ; ============================================= ; first 16 records of MFT contain special information to describe the master file table itself ; http://www.ntfs.com/ntfs-mft.htm ; ============================================= for $i = 0 to 15     ; first 16 records of MFT contain special information to describe the master file table itself     ; http://www.ntfs.com/ntfs-mft.htm     _WinAPI_SetFilePointerEx($hFile, $MFT_Offset + $MFT_Record_Size * $i)     ;ConsoleWrite("_WinAPI_GetLastError(): " & _WinAPI_GetLastError() & @crlf)     ;ConsoleWrite("_WinAPI_SetFilePointer: " & @error & @crlf & @crlf)     _WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $MFT_Record_Size, $nBytes)     ConsoleWrite("MFT Table Record "& $i & @crlf)     ConsoleWrite(_HexEncode(DllStructGetData($tBuffer, 1)) & @CRLF) next ; ============================================= ; File & Dir information starts at record 23++; A_Dir or A_File ; http://www.scribd.com/doc/2187280/NTFS-Documentation ; ============================================= for $i = 24 to 34     _WinAPI_SetFilePointerEx($hFile, $MFT_Offset + $MFT_Record_Size * $i)     ;ConsoleWrite("_WinAPI_GetLastError(): " & _WinAPI_GetLastError() & @crlf)     ;ConsoleWrite("_WinAPI_SetFilePointer: " & @error & @crlf & @crlf)     _WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $MFT_Record_Size, $nBytes)     ConsoleWrite("MFT Table Record "& $i & @crlf)     ConsoleWrite(_HexEncode(DllStructGetData($tBuffer, 1)) & @CRLF) next _WinAPI_CloseHandle($hFile) ;$sText = BinaryToString(DllStructGetData($tBuffer, 1)) ;ConsoleWrite('2) ' & $sText & @LF) Func _WinAPI_SetFilePointerEx($hFile, $iPos, $iMethod = 0)     Local $aResult     $aResult = DllCall("kernel32.dll", "dword", "SetFilePointerEx", "hwnd", $hFile, "uint64", $iPos, "uint64*", 0, "dword", $iMethod)     If @error Then Return SetError(1, 0, -1)     If $aResult[0] = $__WINAPCONSTANT_INVALID_SET_FILE_POINTER Then Return SetError(2, 0, -1)     Return $aResult[0] EndFunc   ;==>_WinAPI_SetFilePointer Func _HexEncode($bInput)     Local $tInput = DllStructCreate("byte[" & BinaryLen($bInput) & "]")     DllStructSetData($tInput, 1, $bInput)     Local $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _             "ptr", DllStructGetPtr($tInput), _             "dword", DllStructGetSize($tInput), _             "dword", 11, _             "ptr", 0, _             "dword*", 0)     If @error Or Not $a_iCall[0] Then         Return SetError(1, 0, "")     EndIf     Local $iSize = $a_iCall[5]     Local $tOut = DllStructCreate("char[" & $iSize & "]")     $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _             "ptr", DllStructGetPtr($tInput), _             "dword", DllStructGetSize($tInput), _             "dword", 11, _             "ptr", DllStructGetPtr($tOut), _             "dword*", $iSize)     If @error Or Not $a_iCall[0] Then         Return SetError(2, 0, "")     EndIf     Return SetError(0, 0, DllStructGetData($tOut, 1)) EndFunc  ;==>_HexEncode


Happy Coding and with best regards ^_^...

Edit: Added #RequireAdmin on trancexx remark.

Edit: Replaced attachement with link to the NTFS Documentation

Edited by KaFu, 31 January 2010 - 02:44 PM.








#2 ptrex

ptrex

    Universalist

  • MVPs
  • 2,408 posts

Posted 03 May 2009 - 08:03 PM

@Kafu

Not that I can use this right now.

Could be a start of a MFT defrag analyser.

Great script !! ^_^

Regards

ptrex

#3 Zedna

Zedna

    AutoIt rulez!

  • MVPs
  • 8,557 posts

Posted 07 June 2009 - 01:30 PM

@KaFu



Nice script. Great example of raw disk/file reading!

#4 trancexx

trancexx

    Queen F. Elizabeth MCXI

  • Active Members
  • PipPipPipPipPipPip
  • 5,975 posts

Posted 16 June 2009 - 05:41 PM

I didn't mention (when writing my part) that #RequireAdmin is... required - for Vista and similar.

Maybe I'm in love (Sshh, it's a secret)

.

 

eMyvnE


#5 JFX

JFX

    Polymath

  • Active Members
  • PipPipPipPip
  • 220 posts

Posted 14 March 2010 - 02:42 PM

Nice Scipt.

Anyone get it working with Autoit 3.3.6.0?

Guess this line no longer works correctly

Global $hComIn = FileOpen("\\.\" & $sDriveLetter & ":", 16)


#6 ProgAndy

ProgAndy

    You need AutoItObject

  • MVPs
  • 2,508 posts

Posted 14 March 2010 - 02:58 PM

Raw-Reading mode was removed in 3.3.4.0 or so. You have to use the _WinAPI_... functions now. (_WinAPI_CreateFle, _WinAPI_ReadFile, _WinAPI_CloseHandle).
*GERMAN* Posted Image [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

#7 Digisoul

Digisoul

    Universalist

  • Active Members
  • PipPipPipPipPipPip
  • 328 posts

Posted 17 March 2010 - 10:37 PM

Nice Scipt.

Anyone get it working with Autoit 3.3.6.0?

Guess this line no longer works correctly

Global $hComIn = FileOpen("\\.\" & $sDriveLetter & ":", 16)

This will work for you :(
AutoIt         
Global $tBuffer=DllStructCreate("byte[512]"),$nBytes Global $hComIn = _WinAPI_CreateFile("\\.\" & $sDriveLetter & ":",2,2,7) If $hComIn = 0 then exit $read = _WinAPI_ReadFile($hComIn, DllStructGetPtr($tBuffer), 512, $nBytes) If $read = 0 then exit Global $bRaw = DllStructGetData($tBuffer,1) ConsoleWrite(_HexEncode($bRaw)& @CRLF) _WinAPI_CloseHandle($hComIn) Global $tBootSectorSections = DllStructCreate("align 1;byte Jump[3];" & _         "char SystemName[8];" & _         "ushort BytesPerSector;" & _         "ubyte SectorsPerCluster;" & _         "ushort ReservedSectors;" & _         "ubyte[3];" & _         "ushort;" & _         "ubyte MediaDescriptor;" & _         "ushort;" & _         "ushort SectorsPerTrack;" & _         "ushort NumberOfHeads;" & _         "dword HiddenSectors;" & _         "dword;" & _         "dword;" & _         "int64 TotalSectors;" & _         "int64 LogicalClusterNumberforthefileMFT;" & _         "int64 LogicalClusterNumberforthefileMFTMirr;" & _         "dword ClustersPerFileRecordSegment;" & _         "dword ClustersPerIndexBlock;" & _         "int64 NTFSVolumeSerialNumber;" & _         "dword Checksum", _     DllStructGetPtr($tBuffer))

73 108 111 118 101 65 117 116 111 105 116 Posted Image

#8 asdf8

asdf8

    Wayfarer

  • Active Members
  • Pip
  • 81 posts

Posted 18 March 2010 - 10:41 AM

Hi!
Maybe anyone else knows how to read from the MFT list of all files on disk?

#9 trancexx

trancexx

    Queen F. Elizabeth MCXI

  • Active Members
  • PipPipPipPipPipPip
  • 5,975 posts

Posted 19 March 2010 - 07:36 AM

Hi!
Maybe anyone else knows how to read from the MFT list of all files on disk?

You missed Digisoul's post or what?

Maybe I'm in love (Sshh, it's a secret)

.

 

eMyvnE


#10 asdf8

asdf8

    Wayfarer

  • Active Members
  • Pip
  • 81 posts

Posted 19 March 2010 - 10:53 AM

You missed Digisoul's post or what?


I do not understand how to get out of this list of files and folders available on the volume

#11 KaFu

KaFu

    Hey, it's just me, KhaFoo...

  • MVPs
  • 3,327 posts

Posted 19 March 2010 - 01:45 PM

I do not understand how to get out of this list of files and folders available on the volume

That's the part still missing :(... the script can point you to the location on an NTFS volume where the MFT is located, but I didn't succeeded in writing a proper parser for that. Look at the documentation mentioned above, it's quiet complex and what discouraged me most time consuming :).

#12 joakim

joakim

    Adventurer

  • Active Members
  • PipPip
  • 109 posts

Posted 31 July 2011 - 08:36 PM

I expanded on the code and can now reassemble the complete MFT, regardless of runs (fragments) in either direction. The below code will exit after the MFT is exported. However, it should not be a problem to use it on any of the files in the MFT and use the real name when exporting the file.

AutoIt         
#RequireAdmin #include <winapi.au3> #include <Array.au3> $outputMFT = @ScriptDir & '\MFTTest1.bin' $testfile = _WinAPI_CreateFile($outputMFT,3,6,7) Global Const $HX_REF="0123456789ABCDEF" Global $RUN_Cluster[1], $RUN_Sectors[1] Global  $sDriveLetter = "c", $nBytes, $MFT_Offset, $bytes_to_read, $tBuffer, $hdd_handle, $hFile, $MFTFull ConsoleWrite("Analyzing Drive " & $sDriveLetter & ":, Filesystem detected: " & DriveGetFileSystem($sDriveLetter & ":") & @crlf & @crlf) if DriveGetFileSystem($sDriveLetter & ":") <> "NTFS" then exit Global $tBuffer=DllStructCreate("byte[512]"),$nBytes Global $hComIn = _WinAPI_CreateFile("\\.\" & $sDriveLetter & ":",2,2,7) If $hComIn = 0 then exit $read = _WinAPI_ReadFile($hComIn, DllStructGetPtr($tBuffer), 512, $nBytes) If $read = 0 then exit Global $bRaw = DllStructGetData($tBuffer,1) ;ConsoleWrite(_HexEncode($bRaw)& @CRLF) _WinAPI_CloseHandle($hComIn) Global $tBootSectorSections = DllStructCreate("align 1;byte Jump[3];" & _         "char SystemName[8];" & _         "ushort BytesPerSector;" & _         "ubyte SectorsPerCluster;" & _         "ushort ReservedSectors;" & _         "ubyte[3];" & _         "ushort;" & _         "ubyte MediaDescriptor;" & _         "ushort;" & _         "ushort SectorsPerTrack;" & _         "ushort NumberOfHeads;" & _         "dword HiddenSectors;" & _         "dword;" & _         "dword;" & _         "int64 TotalSectors;" & _         "int64 LogicalClusterNumberforthefileMFT;" & _         "int64 LogicalClusterNumberforthefileMFTMirr;" & _         "dword ClustersPerFileRecordSegment;" & _         "dword ClustersPerIndexBlock;" & _         "int64 NTFSVolumeSerialNumber;" & _         "dword Checksum", _     DllStructGetPtr($tBuffer)) ConsoleWrite("Jump:  " & DllStructGetData($tBootSectorSections, "Jump") & @CRLF) ConsoleWrite("SystemName:  " & DllStructGetData($tBootSectorSections, "SystemName") & @CRLF) ConsoleWrite("BytesPerSector:  " & DllStructGetData($tBootSectorSections, "BytesPerSector") & @CRLF) ConsoleWrite("SectorsPerCluster:  " & DllStructGetData($tBootSectorSections, "SectorsPerCluster") & @CRLF) ConsoleWrite("ReservedSectors:  " & DllStructGetData($tBootSectorSections, "ReservedSectors") & @CRLF) ;ConsoleWrite("6:  " & DllStructGetData($tBootSectorSections, 6) & @CRLF) ;ConsoleWrite("7:  " & DllStructGetData($tBootSectorSections, 7) & @CRLF) ConsoleWrite("MediaDescriptor:  " & DllStructGetData($tBootSectorSections, "MediaDescriptor") & @CRLF) ;ConsoleWrite("9:  " & DllStructGetData($tBootSectorSections, 9) & @CRLF) ConsoleWrite("SectorsPerTrack:  " & DllStructGetData($tBootSectorSections, "SectorsPerTrack") & @CRLF) ConsoleWrite("NumberOfHeads:  " & DllStructGetData($tBootSectorSections, "NumberOfHeads") & @CRLF) ConsoleWrite("HiddenSectors:  " & DllStructGetData($tBootSectorSections, "HiddenSectors") & @CRLF) ;ConsoleWrite("13:  " & DllStructGetData($tBootSectorSections, 13) & @CRLF) ;ConsoleWrite("14:  " & DllStructGetData($tBootSectorSections, 14) & @CRLF) ConsoleWrite("TotalSectors:  " & DllStructGetData($tBootSectorSections, "TotalSectors") & @CRLF) ConsoleWrite("LogicalClusterNumberforthefileMFT:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT") & @CRLF) ConsoleWrite("LogicalClusterNumberforthefileMFTMirr:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFTMirr") & @CRLF) ConsoleWrite("ClustersPerFileRecordSegment:  " & DllStructGetData($tBootSectorSections, "ClustersPerFileRecordSegment") & @CRLF) ConsoleWrite("ClustersPerIndexBlock:  " & DllStructGetData($tBootSectorSections, "ClustersPerIndexBlock") & @CRLF) ConsoleWrite("VolumeSerialNumber:  " & Ptr(DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber")) & @CRLF) ConsoleWrite("NTFSVolumeSerialNumber:  " & DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber") & @CRLF) ConsoleWrite("Checksum:  " & DllStructGetData($tBootSectorSections, "Checksum") & @CRLF) ConsoleWrite(@CRLF) Global $SectorsPerCluster = DllStructGetData($tBootSectorSections, "SectorsPerCluster") ; ============================================= ; Seek & Read MFT ; ============================================= $MFT_Offset = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") * DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT") ConsoleWrite("$MFT_Offset: " & $MFT_Offset & @CRLF & @CRLF) $MFT_Record_Size = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") / 4 ; ClustersPerFileRecordSegment = 246 ; ConsoleWrite("0x" & hex(246,2)) = 0xF6 >>> 0xF6 = 1/4 Cluster $tBuffer = DllStructCreate("byte[" & $MFT_Record_Size & "]") $hdd_handle = "\\.\" & $sDriveLetter & ":" ConsoleWrite($hdd_handle & @crlf) $hFile = _WinAPI_CreateFile($hdd_handle, 2, 6, 6) ;_stprintf(szDriveCreate, _T(\\\\.\\%c), cDrive); ; handle = CreateFile("\\.\PhysicalDrive" & "0", GENERIC_READ + GENERIC_WRITE, FILE_SHARE_READ + FILE_SHARE_WRITE, 0, OPEN_EXISTING, 0, 0) ConsoleWrite("Harddisk Handle: " & $hFile & @crlf) ;------------- 2*.... $SI_Sig = '10'; hex $FILE_NAME = '30' $DATA = '80' $BITMAP = 'B0' $MFTHeaderSize = 56*2 ;Hardcoded for XP and later, but will fail on 2k ;$SI_Offset = $MFTHeaderSize+(1*2)+1 ;$Resident = 1 for $i = 0 to 1     If $i = 1 Then ExitLoop     _WinAPI_SetFilePointerEx($hFile, $MFT_Offset + $MFT_Record_Size * $i)     _WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $MFT_Record_Size, $nBytes)     ConsoleWrite("MFT Table Record "& $i & @crlf) ;    ConsoleWrite(_HexEncode(DllStructGetData($tBuffer, 1)) & @CRLF)     $MFTEntry = DllStructGetData($tBuffer, 1)     $SI_Offset = (Dec(StringMid($MFTEntry,43,2))*2)+3     ConsoleWrite("$SI_Offset = " & $SI_Offset & @crlf)     ConsoleWrite("(Dec(StringMid($MFTEntry,43,2))*2)+3 = " & (Dec(StringMid($MFTEntry,43,2))*2)+3 & @crlf)     $SI_Size = StringMid($MFTEntry,$SI_Offset+8,8)     ConsoleWrite("$SI_Size = " & $SI_Size & @crlf)     $SI_Size = StringMid($SI_Size,7,2) & StringMid($SI_Size,5,2) & StringMid($SI_Size,3,2) & StringMid($SI_Size,1,2)     $SI_Size = Dec($SI_Size)     ConsoleWrite("$SI_Size = " & $SI_Size & @crlf)     $FN_Offset = $SI_Offset + ($SI_Size*2)     ConsoleWrite("$FN_Offset = " & $FN_Offset & @crlf)     $FN_Size = StringMid($MFTEntry,$FN_Offset+8,8)     ConsoleWrite("$FN_Size = " & $FN_Size & @crlf)     $FN_Size = StringMid($FN_Size,7,2) & StringMid($FN_Size,5,2) & StringMid($FN_Size,3,2) & StringMid($FN_Size,1,2)     $FN_Size = Dec($FN_Size)     ConsoleWrite("$FN_Size = " & $FN_Size & @crlf)     $DATA_Offset = $FN_Offset + ($FN_Size*2)     ConsoleWrite("$DATA_Offset = " & $DATA_Offset & @crlf)     $DATA_Size = StringMid($MFTEntry,$DATA_Offset+8,8)     ConsoleWrite("$DATA_Size = " & $DATA_Size & @crlf)     $DATA_Size = StringMid($DATA_Size,7,2) & StringMid($DATA_Size,5,2) & StringMid($DATA_Size,3,2) & StringMid($DATA_Size,1,2)     $DATA_Size = Dec($DATA_Size)     ConsoleWrite("$DATA_Size = " & $DATA_Size & @crlf)     $DATA_VCNs = StringMid($MFTEntry,$DATA_Offset+48,16) ;   $DATA_VCNs = StringMid($MFTEntry,$DATA_Offset+48,8)     ConsoleWrite("$DATA_VCNs = " & $DATA_VCNs & @crlf)     $DATA_VCNs = StringMid($DATA_VCNs,15,2) & StringMid($DATA_VCNs,13,2) & StringMid($DATA_VCNs,11,2) & StringMid($DATA_VCNs,9,2) & StringMid($DATA_VCNs,7,2) & StringMid($DATA_VCNs,5,2) & StringMid($DATA_VCNs,3,2) & StringMid($DATA_VCNs,1,2) ;   $DATA_VCNs = StringMid($DATA_VCNs,7,2) & StringMid($DATA_VCNs,5,2) & StringMid($DATA_VCNs,3,2) & StringMid($DATA_VCNs,1,2)     ConsoleWrite("$DATA_VCNs = " & $DATA_VCNs & @crlf)     $DATA_VCNs = _HexToDec($DATA_VCNs) ;   $DATA_VCNs = Dec($DATA_VCNs)     ConsoleWrite("$DATA_VCNs = " & $DATA_VCNs & @crlf)     $NonResidentFlag = StringMid($MFTEntry,$DATA_Offset+16,2)     ConsoleWrite("$NonResidentFlag = " & $NonResidentFlag & @crlf)     If $NonResidentFlag = '01' Then         $RunListOffset = StringMid($MFTEntry,$DATA_Offset+20,4)         ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf)         $RunListOffset = StringMid($RunListOffset,3,2) & StringMid($RunListOffset,1,2)         $RunListOffset = Dec($RunListOffset)         ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf)         $RunListID = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2),2)         ConsoleWrite("$RunListID = " & $RunListID & @crlf)         $RunListSectorsLenght = Dec(StringMid($RunListID,2,1))         ConsoleWrite("$RunListSectorsLenght = " & $RunListSectorsLenght & @crlf)         $RunListClusterLenght = Dec(StringMid($RunListID,1,1))         ConsoleWrite("$RunListClusterLenght = " & $RunListClusterLenght & @crlf)         $RunListSectors = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2,$RunListSectorsLenght*2)         ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)         $r = 1         $entry = ''         For $u = 0 To $RunListSectorsLenght-1             $mod = StringMid($RunListSectors,($RunListSectorsLenght*2)-(($u*2)+1),2)             $entry &= $mod         Next         $RunListSectors = Dec($entry)         _ArrayInsert($RUN_Sectors,1,$RunListSectors)         ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)         $RunListCluster = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght*2),$RunListClusterLenght*2)         ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)         $entry = ''         For $u = 0 To $RunListClusterLenght-1             $mod = StringMid($RunListCluster,($RunListClusterLenght*2)-(($u*2)+1),2)             $entry &= $mod         Next         $RunListCluster = Dec($entry)         _ArrayInsert($RUN_Cluster,1,$RunListCluster)         ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)         If $RunListSectors = $DATA_VCNs+1 Then             ConsoleWrite("No more data runs." & @crlf)         Else             ConsoleWrite("More data runs exist." & @crlf)             $NewRunOffsetBase = $DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght+$RunListClusterLenght)*2             _GetAllRuns($MFTEntry,$DATA_Offset,$DATA_Size,$DATA_VCNs,$NewRunOffsetBase)         EndIf         _ReassembleDataRuns($DATA_VCNs)     EndIf     $BITMAP_Offset = $DATA_Offset + ($DATA_Size*2)     ConsoleWrite("$BITMAP_Offset = " & $BITMAP_Offset & @crlf)     $BITMAP_Size = StringMid($MFTEntry,$BITMAP_Offset+8,8)     ConsoleWrite("$BITMAP_Size = " & $BITMAP_Size & @crlf)     $BITMAP_Size = StringMid($BITMAP_Size,7,2) & StringMid($BITMAP_Size,5,2) & StringMid($BITMAP_Size,3,2) & StringMid($BITMAP_Size,1,2)     $BITMAP_Size = Dec($BITMAP_Size)     ConsoleWrite("$BITMAP_Size = " & $BITMAP_Size & @crlf) Next ;_ArrayDisplay($RUN_Sectors,"Sectors per RUN") ;_ArrayDisplay($RUN_Cluster,"Cluster number for RUN") _WinAPI_CloseHandle($hFile) Exit Func _WinAPI_SetFilePointerEx($hFile, $iPos, $iMethod = 0)     Local $aResult     $aResult = DllCall("kernel32.dll", "dword", "SetFilePointerEx", "hwnd", $hFile, "uint64", $iPos, "uint64*", 0, "dword", $iMethod)     If @error Then Return SetError(1, 0, -1)     If $aResult[0] = $__WINAPCONSTANT_INVALID_SET_FILE_POINTER Then Return SetError(2, 0, -1)     Return $aResult[0] EndFunc   ;==>_WinAPI_SetFilePointer Func _HexEncode($bInput)     Local $tInput = DllStructCreate("byte[" & BinaryLen($bInput) & "]")     DllStructSetData($tInput, 1, $bInput)     Local $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _             "ptr", DllStructGetPtr($tInput), _             "dword", DllStructGetSize($tInput), _             "dword", 11, _             "ptr", 0, _             "dword*", 0)     If @error Or Not $a_iCall[0] Then         Return SetError(1, 0, "")     EndIf     Local $iSize = $a_iCall[5]     Local $tOut = DllStructCreate("char[" & $iSize & "]")     $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _             "ptr", DllStructGetPtr($tInput), _             "dword", DllStructGetSize($tInput), _             "dword", 11, _             "ptr", DllStructGetPtr($tOut), _             "dword*", $iSize)     If @error Or Not $a_iCall[0] Then         Return SetError(2, 0, "")     EndIf     Return SetError(0, 0, DllStructGetData($tOut, 1)) EndFunc  ;==>_HexEncode Func _GetAllRuns($MFTEntry,$DATA_Offset,$DATA_Size,$DATA_VCNs,$RunListOffset) Local $RunListClusterNext _ArrayDelete($RUN_Cluster,1) _ArrayDelete($RUN_Sectors,1) $RunListOffset = StringMid($MFTEntry,$DATA_Offset+20,4) ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf) $RunListOffset = StringMid($RunListOffset,3,2) & StringMid($RunListOffset,1,2) $RunListOffset = Dec($RunListOffset) ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf) $RunListID = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2),2) ConsoleWrite("$RunListID = " & $RunListID & @crlf) $RunListSectorsLenght = Dec(StringMid($RunListID,2,1)) ConsoleWrite("$RunListSectorsLenght = " & $RunListSectorsLenght & @crlf) $RunListClusterLenght = Dec(StringMid($RunListID,1,1)) ConsoleWrite("$RunListClusterLenght = " & $RunListClusterLenght & @crlf) $RunListSectors = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2,$RunListSectorsLenght*2) ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf) $entry = '' For $u = 0 To $RunListSectorsLenght-1     $mod = StringMid($RunListSectors,($RunListSectorsLenght*2)-(($u*2)+1),2)     $entry &= $mod Next $RunListSectors = Dec($entry) ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf) _ArrayInsert($RUN_Sectors,1,$RunListSectors) $RunListCluster = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght*2),$RunListClusterLenght*2) ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf) $entry = '' For $u = 0 To $RunListClusterLenght-1     $mod = StringMid($RunListCluster,($RunListClusterLenght*2)-(($u*2)+1),2)     $entry &= $mod Next $RunListCluster = Dec($entry) ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf) _ArrayInsert($RUN_Cluster,1,$RunListCluster) If $RunListSectors = $DATA_VCNs+1 Then     ConsoleWrite("No more data runs." & @crlf)     Return EndIf $NewRunOffsetBase = $DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght+$RunListClusterLenght)*2 ConsoleWrite("$NewRunOffsetBase = " & $NewRunOffsetBase & @crlf) $r = 1 $RunListSectorsTotal = $RunListSectors While $RunListSectors < $DATA_VCNs+1     $r += 1     ConsoleWrite("$r = " & $r & @crlf)     $RunListID = StringMid($MFTEntry,$NewRunOffsetBase,2)     ConsoleWrite("$RunListID = " & $RunListID & @crlf)     $RunListSectorsLenght = Dec(StringMid($RunListID,2,1))     ConsoleWrite("$RunListSectorsLenght = " & $RunListSectorsLenght & @crlf)     $RunListClusterLenght = Dec(StringMid($RunListID,1,1))     ConsoleWrite("$RunListClusterLenght = " & $RunListClusterLenght & @crlf)     $RunListSectors = StringMid($MFTEntry,$NewRunOffsetBase+2,$RunListSectorsLenght*2)     ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)     $entry = ''     For $u = 0 To $RunListSectorsLenght-1         $mod = StringMid($RunListSectors,($RunListSectorsLenght*2)-(($u*2)+1),2)         $entry &= $mod     Next     $RunListSectors = Dec($entry)     ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)     _ArrayInsert($RUN_Sectors,$r,$RunListSectors)     $RunListCluster = StringMid($MFTEntry,$NewRunOffsetBase+2+($RunListSectorsLenght*2),$RunListClusterLenght*2)     ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)     $NegativeMove = 0 ;Here we must read to check positive/negative relative move     If Dec(StringMid($RunListCluster,5,1)) > 7 Then         $NegativeMove = 1     EndIf     ConsoleWrite("$NegativeMove = " & $NegativeMove & @crlf)     $entry = ''     For $u = 0 To $RunListClusterLenght-1         $mod = StringMid($RunListCluster,($RunListClusterLenght*2)-(($u*2)+1),2)     $entry &= $mod     Next     $RunListCluster = Dec($entry)     ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)     If $NegativeMove = 1 Then         $p2 = ''         For $hexnum = 1 To $RunListClusterLenght*2 ;My stupid xor substitute             $p1 = 'F'             $p2 &= $p1         Next         $FFXor = _HexToDec($p2) - $RunListCluster         $RunListClusterTmp = $FFXor + 1         $RunListClusterNext = $RUN_Cluster[$r-1] - $RunListClusterTmp     ElseIf $NegativeMove = 0 Then         $RunListClusterNext = $RUN_Cluster[$r-1] + $RunListCluster     EndIf     ConsoleWrite("$RunListClusterNext = " & $RunListClusterNext & @crlf)     _ArrayInsert($RUN_Cluster,$r,$RunListClusterNext)     $RunListSectorsTotal += $RunListSectors     If $RunListSectorsTotal = $DATA_VCNs+1 Then         ConsoleWrite("No more data runs." & @crlf)         Return     EndIf     $NewRunOffsetBase = $NewRunOffsetBase+2+($RunListSectorsLenght+$RunListClusterLenght)*2     ConsoleWrite("$NewRunOffsetBase = " & $NewRunOffsetBase & @crlf) WEnd ConsoleWrite("DataRuns: Something must have gone wrong. Should not be here..." & @crlf) Return EndFunc Func _ReassembleDataRuns($DATA_VCNs) _WinAPI_SetFilePointer($testfile, ($DATA_VCNs+1)*$SectorsPerCluster*512) ;$DATA_VCNs _WinAPI_SetEndOfFile($testfile) _WinAPI_FlushFileBuffers($testfile) _WinAPI_SetFilePointer($testfile, 0,$FILE_BEGIN) $TargetOff = 0 For $runs = 1 To Ubound($RUN_Cluster)-1     ConsoleWrite("Run " & $runs & " reassembling.." & @crlf)     $tBuffer = DllStructCreate("byte[" & ($RUN_Sectors[$runs]*$SectorsPerCluster*512) & "]")     _WinAPI_SetFilePointerEx($hFile, $RUN_Cluster[$runs]*$SectorsPerCluster*512,$FILE_BEGIN)     _WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), ($RUN_Sectors[$runs]*$SectorsPerCluster*512), $nBytes)     $MFTFragment1 = DllStructGetData($tBuffer, 1)     ConsoleWrite("Size of buffer = " & DllStructGetSize($tBuffer) & @crlf)     _WinAPI_WriteFile($testfile, DllStructGetPtr($tBuffer), DllStructGetSize($tBuffer), $nBytes)     _WinAPI_FlushFileBuffers($testfile)     $TargetOff += $RUN_Sectors[$runs]*$SectorsPerCluster*512     _WinAPI_SetFilePointer($testfile, $TargetOff,$FILE_BEGIN) Next _WinAPI_CloseHandle($testfile) Return EndFunc ; #INDEX# ================================================================ ; Title .........: Hex ; Description ...: This module contains numeric conversions between hexadecimal and ;                  decimal numbers overriding the AutoIt limits / capabilities ; Author ........: jennico (jennicoattminusonlinedotde) ; ======================================================================= ; #FUNCTION# ============================================================== ; Function Name..: _HexToDec ( "expression" ) ; Description ...: Returns decimal expression of a hexadecimal string. ; Parameters ....: expression   - String representation of a hexadecimal expression to be converted to decimal. ; Return values .: Success      - Returns decimal expression of a hexadecimal string. ;                  Failure      - Returns "" (blank string) and sets @error to 1 if string is not hexadecimal type. ; Author ........: jennico (jennicoattminusonlinedotde) ; Remarks .......: working input format: "FFFF" or "0xFFFF" (string format), do NOT pass 0xFFFF without quotation marks (number format). ;                  current AutoIt Dec() limitation: 0x7FFFFFFF (2147483647). ; Related .......: Hex(), Dec(), _DecToHex() ; ======================================================================= Func _HexToDec($hx_hex)     If StringLeft($hx_hex, 2) = "0x" Then $hx_hex = StringMid($hx_hex, 3)     If StringIsXDigit($hx_hex) = 0 Then         SetError(1)         Return ""     EndIf     Local $ret="", $hx_count=0, $hx_array = StringSplit($hx_hex, ""), $Ii, $hx_tmp     For $Ii = $hx_array[0] To 1 Step -1         $hx_tmp = StringInStr($HX_REF, $hx_array[$Ii]) - 1         $ret += $hx_tmp * 16 ^ $hx_count         $hx_count += 1     Next     Return $ret EndFunc  ;==>_HexToDec() ; #FUNCTION# ============================================================== ; Function Name..: _DecToHex ( expression [, length] ) ; Description ...: Returns a string representation of an integer converted to hexadecimal. ; Parameters ....: expression   - The integer to be converted to hexadecimal. ;                  length       - [optional] Number of characters to be returned (no limit). ;                                 If no length specified, leading zeros will be stripped from result. ; Return values .: Success      - Returns a string of length characters representing a hexadecimal expression, zero-padded if necessary. ;                  Failure      - Returns "" (blank string) and sets @error to 1 if expression is not an integer. ; Author ........: jennico (jennicoattminusonlinedotde) ; Remarks .......: Output format "FFFF". ;                  The function will also set @error to 1 if requested length is not sufficient - the returned string will be left truncated. ;                  Be free to modify the function to be working with binary type input - I did not try it though. ;                  current AutoIt Hex() limitation: 0xFFFFFFFF (4294967295). ; Related .......: Hex(), Dec(), _HexToDec() ; ======================================================================= Func _DecToHex($hx_dec, $hx_length = 21)     If IsInt($hx_dec) = 0 Then         SetError(1)         Return ""     EndIf     Local $ret = "", $Ii, $hx_tmp, $hx_max     If $hx_dec < 4294967296 Then         If $hx_length < 9 Then Return Hex($hx_dec, $hx_length)         If $hx_length = 21 Then             $ret = Hex($hx_dec)             While StringLeft($ret, 1) = "0"                 $ret = StringMid($ret, 2)             WEnd             Return $ret         EndIf     EndIf     For $Ii = $hx_length - 1 To 0 Step -1         $hx_max = 16 ^ $Ii - 1         If $ret = "" And $hx_length = 21 And $hx_max > $hx_dec Then ContinueLoop         $hx_tmp = Int($hx_dec/($hx_max+1))         If $ret = "" And $hx_length = 21 And $Ii > 0 And $hx_tmp = 0 Then ContinueLoop         $ret &= StringMid($HX_REF, $hx_tmp+1, 1)         $hx_dec -= $hx_tmp * ($hx_max + 1)     Next     $ret=String($ret)     If $hx_length < 21 And StringLen($ret) < $hx_length Then SetError(1)     Return $ret EndFunc  ;==>_DecToHex()


I'm working on a MFT parser as well, a sort of a "decode and log to csv" sort of stuff.

Joakim

#13 KaFu

KaFu

    Hey, it's just me, KhaFoo...

  • MVPs
  • 3,327 posts

Posted 02 August 2011 - 10:50 AM

Joakim, the result looks most excellent :) (on my system I had to add Global Const $__WINAPCONSTANT_INVALID_SET_FILE_POINTER = -1 to the top). Looking forward to a working parser ;).

#14 joakim

joakim

    Adventurer

  • Active Members
  • PipPip
  • 109 posts

Posted 02 August 2011 - 11:34 AM

Contrary to what I rpeviously insinuated, the code will currently only work for exporting $MFT (and maybe a few others) as the "sequence" of the attributes are hardcoded. The parser in its alpha state is almost done and will be uploaded in short time. With that code I will guess it is also possible to recover deleted files (unless their sectors are overwritten).

#15 joakim

joakim

    Adventurer

  • Active Members
  • PipPip
  • 109 posts

Posted 20 December 2011 - 01:07 PM

I completely forgot to repost here that I created the project at; http://code.google.com/p/mft2csv/downloads/list

One of the important things missing is the filename to mftnumber resolving. Apart from that it is possible to recover deleted files, as long as the sectors are not overwritten and the MFT entry (with the runs information) is also there.
  • KaFu likes this

#16 KaFu

KaFu

    Hey, it's just me, KhaFoo...

  • MVPs
  • 3,327 posts

Posted 20 December 2011 - 02:51 PM

Now, this is not a starting point anymore but a full framework of utilities for full-blown NTFS data extraction!

Great work Joakim, posting your update is really appreciated :)!

#17 joakim

joakim

    Adventurer

  • Active Members
  • PipPip
  • 109 posts

Posted 21 December 2011 - 12:11 PM

The newest sources attached.

Attached Files



#18 llewxam

llewxam

    Universalist

  • Active Members
  • PipPipPipPipPipPip
  • 376 posts

Posted 21 December 2011 - 01:27 PM

Love it, thanks!

Ian
  • IP Scanner - Multi-threaded ping tool to scan your available networks for used and available IP addresses, shows ping times, resolves IPs in to host names, and allows individual IPs to be pinged.
  • INFSniff - Great technicians tool - a tool which scans DriverPacks archives for INF files and parses out the HWIDs to a database file, and rapidly scans the local machine's HWIDs, searches the database for matches, and installs them.
  • PPK3 (Persistent Process Killer V3) - Another for the techs - suppress running processes that you need to keep away, helpful when fighting spyware/viruses.
  • Sync Tool - Folder sync tool with lots of real time information and several checking methods.
  • USMT Front End - Front End for Microsoft's User State Migration Tool, including all files needed for USMT 3.01 and 4.01, 32 bit and 64 bit versions.
  • Audit Tool - Computer audit tool to gather vital hardware, Windows, and Office information for IT managers and field techs. Capabilities include creating a customized siteagent.
  • CSV Viewer - Displays CSV files with automatic column sizing and font selection. Lines can also be copied to the clipboard for data extraction.
  • MyDirStat - Lists number and size of files on a drive or specified path, allows for deletion within the app.

#19 joakim

joakim

    Adventurer

  • Active Members
  • PipPip
  • 109 posts

Posted 22 December 2011 - 12:17 PM

One of the important things missing is the filename to mftnumber resolving.

A slightly incorrect statement. It should say that the filename path to mtf record number is not yet resolved. Filenames are of course solved, but doing a a raw extraction based on that, may lead to several files with the same name without knowing the path to each of them. Also just updated mft2csv with some fixes regarding integrity checks of records..

Attached Files



#20 joakim

joakim

    Adventurer

  • Active Members
  • PipPip
  • 109 posts

Posted 28 December 2011 - 12:55 AM

The calculation of the runs are redone with several fixes and I'm confident that badly fragmented files are now also correctly extracted. My worst $MFT with 121 runs (including many negative moves) are now correct. However compressed and/or sparse files are not yet supported, but should not be too hard to solve I think.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users