Sign in to follow this  
Followers 0

MFT-Access - Reading & parsing the Master File Table on NTFS Filesystems

47 posts in this topic

Posted (edited)

HiHo Community,

for some time now I'm following on the topic of reading & parsing the Master File Table on NTFS Filesystems.

Though I believe I'm on the right track, it's quiet complex and time consuming. So I wanted to share what I have and invite those who are interested to contribute in creating an UDF in the long-run.

Besides the code I also attached an excellent reading I found on the topic containing all relevant information.

;===============================================================================
;
; Description:      MFT-Access, UDF under development
;                   Reading & parsing the Master File Table on NTFS Filesystems
; Version:          v0.0.0.3
; Last modified:    2009-May-02
; URL:              http://www.autoitscript.com/forum/index.php?showtopic=94269
; Author(s):        KaFu (http://www.funk.eu), trancexx
;
;===============================================================================
;
#RequireAdmin
#include <winapi.au3>

Global  $sDriveLetter = "c", $nBytes, $MFT_Offset, $bytes_to_read, $tBuffer, $hdd_handle, $hFile

ConsoleWrite("Analyzing Drive " & $sDriveLetter & ":, Filesystem detected: " & DriveGetFileSystem($sDriveLetter & ":") & @crlf & @crlf)

if DriveGetFileSystem($sDriveLetter & ":") <> "NTFS" then exit

; Read Boot-Sector
; http://www.autoitscript.com/forum/index.php?showtopic=92529&view=findpost&p=665010
; trancexx
Global $hComIn = FileOpen("\\.\" & $sDriveLetter & ":", 16)
Global $bRaw = FileRead($hComIn, 84); 84 bytes is enough
ConsoleWrite(_HexEncode($bRaw) & @CRLF)
FileClose($hComIn)
Global $tRaw = DllStructCreate("byte [" & BinaryLen($bRaw) & "]")
DllStructSetData($tRaw, 1, $bRaw)
Global $tBootSectorSections = DllStructCreate("align 1;byte Jump[3];" & _
        "char SystemName[8];" & _
        "ushort BytesPerSector;" & _
        "ubyte SectorsPerCluster;" & _
        "ushort ReservedSectors;" & _
        "ubyte[3];" & _
        "ushort;" & _
        "ubyte MediaDescriptor;" & _
        "ushort;" & _
        "ushort SectorsPerTrack;" & _
        "ushort NumberOfHeads;" & _
        "dword HiddenSectors;" & _
        "dword;" & _
        "dword;" & _
        "int64 TotalSectors;" & _
        "int64 LogicalClusterNumberforthefileMFT;" & _
        "int64 LogicalClusterNumberforthefileMFTMirr;" & _
        "dword ClustersPerFileRecordSegment;" & _
        "dword ClustersPerIndexBlock;" & _
        "int64 NTFSVolumeSerialNumber;" & _
        "dword Checksum", _
        DllStructGetPtr($tRaw))


ConsoleWrite("Jump:  " & DllStructGetData($tBootSectorSections, "Jump") & @CRLF)
ConsoleWrite("SystemName:  " & DllStructGetData($tBootSectorSections, "SystemName") & @CRLF)
ConsoleWrite("BytesPerSector:  " & DllStructGetData($tBootSectorSections, "BytesPerSector") & @CRLF)
ConsoleWrite("SectorsPerCluster:  " & DllStructGetData($tBootSectorSections, "SectorsPerCluster") & @CRLF)
ConsoleWrite("ReservedSectors:  " & DllStructGetData($tBootSectorSections, "ReservedSectors") & @CRLF)
;ConsoleWrite("6:  " & DllStructGetData($tBootSectorSections, 6) & @CRLF)
;ConsoleWrite("7:  " & DllStructGetData($tBootSectorSections, 7) & @CRLF)
ConsoleWrite("MediaDescriptor:  " & DllStructGetData($tBootSectorSections, "MediaDescriptor") & @CRLF)
;ConsoleWrite("9:  " & DllStructGetData($tBootSectorSections, 9) & @CRLF)
ConsoleWrite("SectorsPerTrack:  " & DllStructGetData($tBootSectorSections, "SectorsPerTrack") & @CRLF)
ConsoleWrite("NumberOfHeads:  " & DllStructGetData($tBootSectorSections, "NumberOfHeads") & @CRLF)
ConsoleWrite("HiddenSectors:  " & DllStructGetData($tBootSectorSections, "HiddenSectors") & @CRLF)
;ConsoleWrite("13:  " & DllStructGetData($tBootSectorSections, 13) & @CRLF)
;ConsoleWrite("14:  " & DllStructGetData($tBootSectorSections, 14) & @CRLF)
ConsoleWrite("TotalSectors:  " & DllStructGetData($tBootSectorSections, "TotalSectors") & @CRLF)
ConsoleWrite("LogicalClusterNumberforthefileMFT:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT") & @CRLF)
ConsoleWrite("LogicalClusterNumberforthefileMFTMirr:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFTMirr") & @CRLF)
ConsoleWrite("ClustersPerFileRecordSegment:  " & DllStructGetData($tBootSectorSections, "ClustersPerFileRecordSegment") & @CRLF)
ConsoleWrite("ClustersPerIndexBlock:  " & DllStructGetData($tBootSectorSections, "ClustersPerIndexBlock") & @CRLF)
ConsoleWrite("VolumeSerialNumber:  " & Ptr(DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber")) & @CRLF)
ConsoleWrite("NTFSVolumeSerialNumber:  " & DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber") & @CRLF)
ConsoleWrite("Checksum:  " & DllStructGetData($tBootSectorSections, "Checksum") & @CRLF)
ConsoleWrite(@CRLF)



; =============================================
; Seek & Read MFT
; =============================================

$MFT_Offset = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") * DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT")
ConsoleWrite("$MFT_Offset: " & $MFT_Offset & @CRLF & @CRLF)

$MFT_Record_Size = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") / 4
; ClustersPerFileRecordSegment = 246
; ConsoleWrite("0x" & hex(246,2)) = 0xF6 >>> 0xF6 = 1/4 Cluster

$tBuffer = DllStructCreate("byte[" & $MFT_Record_Size & "]")
$hdd_handle = "\\.\" & $sDriveLetter & ":"
ConsoleWrite($hdd_handle & @crlf)
$hFile = _WinAPI_CreateFile($hdd_handle, 2, 6, 6)
;_stprintf(szDriveCreate, _T(\\\\.\\%c), cDrive);
; handle = CreateFile("\\.\PhysicalDrive" & "0", GENERIC_READ + GENERIC_WRITE, FILE_SHARE_READ + FILE_SHARE_WRITE, 0, OPEN_EXISTING, 0, 0)

ConsoleWrite("Harddisk Handle: " & $hFile & @crlf)

; =============================================
; first 16 records of MFT contain special information to describe the master file table itself
; http://www.ntfs.com/ntfs-mft.htm
; =============================================
for $i = 0 to 15
	; first 16 records of MFT contain special information to describe the master file table itself
	; http://www.ntfs.com/ntfs-mft.htm
	_WinAPI_SetFilePointerEx($hFile, $MFT_Offset + $MFT_Record_Size * $i)
	;ConsoleWrite("_WinAPI_GetLastError(): " & _WinAPI_GetLastError() & @crlf)
	;ConsoleWrite("_WinAPI_SetFilePointer: " & @error & @crlf & @crlf)
	_WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $MFT_Record_Size, $nBytes)
	ConsoleWrite("MFT Table Record "& $i & @crlf)
	ConsoleWrite(_HexEncode(DllStructGetData($tBuffer, 1)) & @CRLF)
next

; =============================================
; File & Dir information starts at record 23++; A_Dir or A_File
; http://www.scribd.com/doc/2187280/NTFS-Documentation
; =============================================
for $i = 24 to 34
	_WinAPI_SetFilePointerEx($hFile, $MFT_Offset + $MFT_Record_Size * $i)
	;ConsoleWrite("_WinAPI_GetLastError(): " & _WinAPI_GetLastError() & @crlf)
	;ConsoleWrite("_WinAPI_SetFilePointer: " & @error & @crlf & @crlf)
	_WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $MFT_Record_Size, $nBytes)
	ConsoleWrite("MFT Table Record "& $i & @crlf)
	ConsoleWrite(_HexEncode(DllStructGetData($tBuffer, 1)) & @CRLF)
next

_WinAPI_CloseHandle($hFile)

;$sText = BinaryToString(DllStructGetData($tBuffer, 1))
;ConsoleWrite('2) ' & $sText & @LF)





Func _WinAPI_SetFilePointerEx($hFile, $iPos, $iMethod = 0)
	Local $aResult

	$aResult = DllCall("kernel32.dll", "dword", "SetFilePointerEx", "hwnd", $hFile, "uint64", $iPos, "uint64*", 0, "dword", $iMethod)
	If @error Then Return SetError(1, 0, -1)
	If $aResult[0] = $__WINAPCONSTANT_INVALID_SET_FILE_POINTER Then Return SetError(2, 0, -1)
	Return $aResult[0]
EndFunc   ;==>_WinAPI_SetFilePointer





Func _HexEncode($bInput)
    Local $tInput = DllStructCreate("byte[" & BinaryLen($bInput) & "]")
    DllStructSetData($tInput, 1, $bInput)
    Local $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _
            "ptr", DllStructGetPtr($tInput), _
            "dword", DllStructGetSize($tInput), _
            "dword", 11, _
            "ptr", 0, _
            "dword*", 0)

    If @error Or Not $a_iCall[0] Then
        Return SetError(1, 0, "")
    EndIf

    Local $iSize = $a_iCall[5]
    Local $tOut = DllStructCreate("char[" & $iSize & "]")

    $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _
            "ptr", DllStructGetPtr($tInput), _
            "dword", DllStructGetSize($tInput), _
            "dword", 11, _
            "ptr", DllStructGetPtr($tOut), _
            "dword*", $iSize)

    If @error Or Not $a_iCall[0] Then
        Return SetError(2, 0, "")
    EndIf

    Return SetError(0, 0, DllStructGetData($tOut, 1))

EndFunc  ;==>_HexEncode

Happy Coding and with best regards ^_^...

Edit: Added #RequireAdmin on trancexx remark.

Edit: Replaced attachement with link to the NTFS Documentation

Edited by KaFu

Share this post


Link to post
Share on other sites



Posted

@Kafu

Not that I can use this right now.

Could be a start of a MFT defrag analyser.

Great script !! ^_^

Regards

ptrex

Share this post


Link to post
Share on other sites

Posted

@KaFu

Nice script. Great example of raw disk/file reading!

Share this post


Link to post
Share on other sites

Posted

I didn't mention (when writing my part) that #RequireAdmin is... required - for Vista and similar.

Share this post


Link to post
Share on other sites

Posted

Nice Scipt.

Anyone get it working with Autoit 3.3.6.0?

Guess this line no longer works correctly

Global $hComIn = FileOpen("\\.\" & $sDriveLetter & ":", 16)

Share this post


Link to post
Share on other sites

Posted

Raw-Reading mode was removed in 3.3.4.0 or so. You have to use the _WinAPI_... functions now. (_WinAPI_CreateFle, _WinAPI_ReadFile, _WinAPI_CloseHandle).

Share this post


Link to post
Share on other sites

Posted

Nice Scipt.

Anyone get it working with Autoit 3.3.6.0?

Guess this line no longer works correctly

Global $hComIn = FileOpen("\\.\" & $sDriveLetter & ":", 16)

This will work for you :(

Global $tBuffer=DllStructCreate("byte[512]"),$nBytes
Global $hComIn = _WinAPI_CreateFile("\\.\" & $sDriveLetter & ":",2,2,7)
If $hComIn = 0 then exit
$read = _WinAPI_ReadFile($hComIn, DllStructGetPtr($tBuffer), 512, $nBytes)
If $read = 0 then exit
Global $bRaw = DllStructGetData($tBuffer,1)
ConsoleWrite(_HexEncode($bRaw)& @CRLF)
_WinAPI_CloseHandle($hComIn)


Global $tBootSectorSections = DllStructCreate("align 1;byte Jump[3];" & _
        "char SystemName[8];" & _
        "ushort BytesPerSector;" & _
        "ubyte SectorsPerCluster;" & _
        "ushort ReservedSectors;" & _
        "ubyte[3];" & _
        "ushort;" & _
        "ubyte MediaDescriptor;" & _
        "ushort;" & _
        "ushort SectorsPerTrack;" & _
        "ushort NumberOfHeads;" & _
        "dword HiddenSectors;" & _
        "dword;" & _
        "dword;" & _
        "int64 TotalSectors;" & _
        "int64 LogicalClusterNumberforthefileMFT;" & _
        "int64 LogicalClusterNumberforthefileMFTMirr;" & _
        "dword ClustersPerFileRecordSegment;" & _
        "dword ClustersPerIndexBlock;" & _
        "int64 NTFSVolumeSerialNumber;" & _
        "dword Checksum", _
   	DllStructGetPtr($tBuffer))

Share this post


Link to post
Share on other sites

Posted

Hi!

Maybe anyone else knows how to read from the MFT list of all files on disk?

Share this post


Link to post
Share on other sites

Posted

Hi!

Maybe anyone else knows how to read from the MFT list of all files on disk?

You missed Digisoul's post or what?

Share this post


Link to post
Share on other sites

Posted

You missed Digisoul's post or what?

I do not understand how to get out of this list of files and folders available on the volume

Share this post


Link to post
Share on other sites

Posted

I do not understand how to get out of this list of files and folders available on the volume

That's the part still missing :(... the script can point you to the location on an NTFS volume where the MFT is located, but I didn't succeeded in writing a proper parser for that. Look at the documentation mentioned above, it's quiet complex and what discouraged me most time consuming :).

Share this post


Link to post
Share on other sites

Posted

I expanded on the code and can now reassemble the complete MFT, regardless of runs (fragments) in either direction. The below code will exit after the MFT is exported. However, it should not be a problem to use it on any of the files in the MFT and use the real name when exporting the file.

#RequireAdmin
#include <winapi.au3>
#include <Array.au3>
$outputMFT = @ScriptDir & '\MFTTest1.bin'
$testfile = _WinAPI_CreateFile($outputMFT,3,6,7)
Global Const $HX_REF="0123456789ABCDEF"
Global $RUN_Cluster[1], $RUN_Sectors[1]
Global  $sDriveLetter = "c", $nBytes, $MFT_Offset, $bytes_to_read, $tBuffer, $hdd_handle, $hFile, $MFTFull
ConsoleWrite("Analyzing Drive " & $sDriveLetter & ":, Filesystem detected: " & DriveGetFileSystem($sDriveLetter & ":") & @crlf & @crlf)
if DriveGetFileSystem($sDriveLetter & ":") <> "NTFS" then exit

Global $tBuffer=DllStructCreate("byte[512]"),$nBytes
Global $hComIn = _WinAPI_CreateFile("\\.\" & $sDriveLetter & ":",2,2,7)
If $hComIn = 0 then exit
$read = _WinAPI_ReadFile($hComIn, DllStructGetPtr($tBuffer), 512, $nBytes)
If $read = 0 then exit
Global $bRaw = DllStructGetData($tBuffer,1)
;ConsoleWrite(_HexEncode($bRaw)& @CRLF)
_WinAPI_CloseHandle($hComIn)

Global $tBootSectorSections = DllStructCreate("align 1;byte Jump[3];" & _
        "char SystemName[8];" & _
        "ushort BytesPerSector;" & _
        "ubyte SectorsPerCluster;" & _
        "ushort ReservedSectors;" & _
        "ubyte[3];" & _
        "ushort;" & _
        "ubyte MediaDescriptor;" & _
        "ushort;" & _
        "ushort SectorsPerTrack;" & _
        "ushort NumberOfHeads;" & _
        "dword HiddenSectors;" & _
        "dword;" & _
        "dword;" & _
        "int64 TotalSectors;" & _
        "int64 LogicalClusterNumberforthefileMFT;" & _
        "int64 LogicalClusterNumberforthefileMFTMirr;" & _
        "dword ClustersPerFileRecordSegment;" & _
        "dword ClustersPerIndexBlock;" & _
        "int64 NTFSVolumeSerialNumber;" & _
        "dword Checksum", _
    DllStructGetPtr($tBuffer))

ConsoleWrite("Jump:  " & DllStructGetData($tBootSectorSections, "Jump") & @CRLF)
ConsoleWrite("SystemName:  " & DllStructGetData($tBootSectorSections, "SystemName") & @CRLF)
ConsoleWrite("BytesPerSector:  " & DllStructGetData($tBootSectorSections, "BytesPerSector") & @CRLF)
ConsoleWrite("SectorsPerCluster:  " & DllStructGetData($tBootSectorSections, "SectorsPerCluster") & @CRLF)
ConsoleWrite("ReservedSectors:  " & DllStructGetData($tBootSectorSections, "ReservedSectors") & @CRLF)
;ConsoleWrite("6:  " & DllStructGetData($tBootSectorSections, 6) & @CRLF)
;ConsoleWrite("7:  " & DllStructGetData($tBootSectorSections, 7) & @CRLF)
ConsoleWrite("MediaDescriptor:  " & DllStructGetData($tBootSectorSections, "MediaDescriptor") & @CRLF)
;ConsoleWrite("9:  " & DllStructGetData($tBootSectorSections, 9) & @CRLF)
ConsoleWrite("SectorsPerTrack:  " & DllStructGetData($tBootSectorSections, "SectorsPerTrack") & @CRLF)
ConsoleWrite("NumberOfHeads:  " & DllStructGetData($tBootSectorSections, "NumberOfHeads") & @CRLF)
ConsoleWrite("HiddenSectors:  " & DllStructGetData($tBootSectorSections, "HiddenSectors") & @CRLF)
;ConsoleWrite("13:  " & DllStructGetData($tBootSectorSections, 13) & @CRLF)
;ConsoleWrite("14:  " & DllStructGetData($tBootSectorSections, 14) & @CRLF)
ConsoleWrite("TotalSectors:  " & DllStructGetData($tBootSectorSections, "TotalSectors") & @CRLF)
ConsoleWrite("LogicalClusterNumberforthefileMFT:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT") & @CRLF)
ConsoleWrite("LogicalClusterNumberforthefileMFTMirr:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFTMirr") & @CRLF)
ConsoleWrite("ClustersPerFileRecordSegment:  " & DllStructGetData($tBootSectorSections, "ClustersPerFileRecordSegment") & @CRLF)
ConsoleWrite("ClustersPerIndexBlock:  " & DllStructGetData($tBootSectorSections, "ClustersPerIndexBlock") & @CRLF)
ConsoleWrite("VolumeSerialNumber:  " & Ptr(DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber")) & @CRLF)
ConsoleWrite("NTFSVolumeSerialNumber:  " & DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber") & @CRLF)
ConsoleWrite("Checksum:  " & DllStructGetData($tBootSectorSections, "Checksum") & @CRLF)
ConsoleWrite(@CRLF)
Global $SectorsPerCluster = DllStructGetData($tBootSectorSections, "SectorsPerCluster")
; =============================================
; Seek & Read MFT
; =============================================
$MFT_Offset = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") * DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT")
ConsoleWrite("$MFT_Offset: " & $MFT_Offset & @CRLF & @CRLF)
$MFT_Record_Size = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") / 4
; ClustersPerFileRecordSegment = 246
; ConsoleWrite("0x" & hex(246,2)) = 0xF6 >>> 0xF6 = 1/4 Cluster
$tBuffer = DllStructCreate("byte[" & $MFT_Record_Size & "]")
$hdd_handle = "\\.\" & $sDriveLetter & ":"
ConsoleWrite($hdd_handle & @crlf)
$hFile = _WinAPI_CreateFile($hdd_handle, 2, 6, 6)
;_stprintf(szDriveCreate, _T(\\\\.\\%c), cDrive);
; handle = CreateFile("\\.\PhysicalDrive" & "0", GENERIC_READ + GENERIC_WRITE, FILE_SHARE_READ + FILE_SHARE_WRITE, 0, OPEN_EXISTING, 0, 0)
ConsoleWrite("Harddisk Handle: " & $hFile & @crlf)
;------------- 2*....
$SI_Sig = '10'; hex
$FILE_NAME = '30'
$DATA = '80'
$BITMAP = 'B0'
$MFTHeaderSize = 56*2 ;Hardcoded for XP and later, but will fail on 2k
;$SI_Offset = $MFTHeaderSize+(1*2)+1
;$Resident = 1
for $i = 0 to 1
	If $i = 1 Then ExitLoop
	_WinAPI_SetFilePointerEx($hFile, $MFT_Offset + $MFT_Record_Size * $i)
	_WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $MFT_Record_Size, $nBytes)
	ConsoleWrite("MFT Table Record "& $i & @crlf)
;    ConsoleWrite(_HexEncode(DllStructGetData($tBuffer, 1)) & @CRLF)
	$MFTEntry = DllStructGetData($tBuffer, 1)
	$SI_Offset = (Dec(StringMid($MFTEntry,43,2))*2)+3
	ConsoleWrite("$SI_Offset = " & $SI_Offset & @crlf)
	ConsoleWrite("(Dec(StringMid($MFTEntry,43,2))*2)+3 = " & (Dec(StringMid($MFTEntry,43,2))*2)+3 & @crlf)
	$SI_Size = StringMid($MFTEntry,$SI_Offset+8,8)
	ConsoleWrite("$SI_Size = " & $SI_Size & @crlf)
	$SI_Size = StringMid($SI_Size,7,2) & StringMid($SI_Size,5,2) & StringMid($SI_Size,3,2) & StringMid($SI_Size,1,2)
	$SI_Size = Dec($SI_Size)
	ConsoleWrite("$SI_Size = " & $SI_Size & @crlf)
	$FN_Offset = $SI_Offset + ($SI_Size*2)
	ConsoleWrite("$FN_Offset = " & $FN_Offset & @crlf)
	$FN_Size = StringMid($MFTEntry,$FN_Offset+8,8)
	ConsoleWrite("$FN_Size = " & $FN_Size & @crlf)
	$FN_Size = StringMid($FN_Size,7,2) & StringMid($FN_Size,5,2) & StringMid($FN_Size,3,2) & StringMid($FN_Size,1,2)
	$FN_Size = Dec($FN_Size)
	ConsoleWrite("$FN_Size = " & $FN_Size & @crlf)
	$DATA_Offset = $FN_Offset + ($FN_Size*2)
	ConsoleWrite("$DATA_Offset = " & $DATA_Offset & @crlf)
	$DATA_Size = StringMid($MFTEntry,$DATA_Offset+8,8)
	ConsoleWrite("$DATA_Size = " & $DATA_Size & @crlf)
	$DATA_Size = StringMid($DATA_Size,7,2) & StringMid($DATA_Size,5,2) & StringMid($DATA_Size,3,2) & StringMid($DATA_Size,1,2)
	$DATA_Size = Dec($DATA_Size)
	ConsoleWrite("$DATA_Size = " & $DATA_Size & @crlf)
	$DATA_VCNs = StringMid($MFTEntry,$DATA_Offset+48,16)
;	$DATA_VCNs = StringMid($MFTEntry,$DATA_Offset+48,8)
	ConsoleWrite("$DATA_VCNs = " & $DATA_VCNs & @crlf)
	$DATA_VCNs = StringMid($DATA_VCNs,15,2) & StringMid($DATA_VCNs,13,2) & StringMid($DATA_VCNs,11,2) & StringMid($DATA_VCNs,9,2) & StringMid($DATA_VCNs,7,2) & StringMid($DATA_VCNs,5,2) & StringMid($DATA_VCNs,3,2) & StringMid($DATA_VCNs,1,2)
;	$DATA_VCNs = StringMid($DATA_VCNs,7,2) & StringMid($DATA_VCNs,5,2) & StringMid($DATA_VCNs,3,2) & StringMid($DATA_VCNs,1,2)
	ConsoleWrite("$DATA_VCNs = " & $DATA_VCNs & @crlf)
	$DATA_VCNs = _HexToDec($DATA_VCNs)
;	$DATA_VCNs = Dec($DATA_VCNs)
	ConsoleWrite("$DATA_VCNs = " & $DATA_VCNs & @crlf)
	$NonResidentFlag = StringMid($MFTEntry,$DATA_Offset+16,2)
	ConsoleWrite("$NonResidentFlag = " & $NonResidentFlag & @crlf)
	If $NonResidentFlag = '01' Then
		$RunListOffset = StringMid($MFTEntry,$DATA_Offset+20,4)
		ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf)
		$RunListOffset = StringMid($RunListOffset,3,2) & StringMid($RunListOffset,1,2)
		$RunListOffset = Dec($RunListOffset)
		ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf)
		$RunListID = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2),2)
		ConsoleWrite("$RunListID = " & $RunListID & @crlf)
		$RunListSectorsLenght = Dec(StringMid($RunListID,2,1))
		ConsoleWrite("$RunListSectorsLenght = " & $RunListSectorsLenght & @crlf)
		$RunListClusterLenght = Dec(StringMid($RunListID,1,1))
		ConsoleWrite("$RunListClusterLenght = " & $RunListClusterLenght & @crlf)
		$RunListSectors = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2,$RunListSectorsLenght*2)
		ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
		$r = 1
		$entry = ''
		For $u = 0 To $RunListSectorsLenght-1
			$mod = StringMid($RunListSectors,($RunListSectorsLenght*2)-(($u*2)+1),2)
			$entry &= $mod
		Next
		$RunListSectors = Dec($entry)
		_ArrayInsert($RUN_Sectors,1,$RunListSectors)
		ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
		$RunListCluster = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght*2),$RunListClusterLenght*2)
		ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
		$entry = ''
		For $u = 0 To $RunListClusterLenght-1
			$mod = StringMid($RunListCluster,($RunListClusterLenght*2)-(($u*2)+1),2)
			$entry &= $mod
		Next
		$RunListCluster = Dec($entry)
		_ArrayInsert($RUN_Cluster,1,$RunListCluster)
		ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
		If $RunListSectors = $DATA_VCNs+1 Then
			ConsoleWrite("No more data runs." & @crlf)
		Else
			ConsoleWrite("More data runs exist." & @crlf)
			$NewRunOffsetBase = $DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght+$RunListClusterLenght)*2
			_GetAllRuns($MFTEntry,$DATA_Offset,$DATA_Size,$DATA_VCNs,$NewRunOffsetBase)
		EndIf
		_ReassembleDataRuns($DATA_VCNs)
	EndIf
	$BITMAP_Offset = $DATA_Offset + ($DATA_Size*2)
	ConsoleWrite("$BITMAP_Offset = " & $BITMAP_Offset & @crlf)
	$BITMAP_Size = StringMid($MFTEntry,$BITMAP_Offset+8,8)
	ConsoleWrite("$BITMAP_Size = " & $BITMAP_Size & @crlf)
	$BITMAP_Size = StringMid($BITMAP_Size,7,2) & StringMid($BITMAP_Size,5,2) & StringMid($BITMAP_Size,3,2) & StringMid($BITMAP_Size,1,2)
	$BITMAP_Size = Dec($BITMAP_Size)
	ConsoleWrite("$BITMAP_Size = " & $BITMAP_Size & @crlf)
Next
;_ArrayDisplay($RUN_Sectors,"Sectors per RUN")
;_ArrayDisplay($RUN_Cluster,"Cluster number for RUN")
_WinAPI_CloseHandle($hFile)
Exit

Func _WinAPI_SetFilePointerEx($hFile, $iPos, $iMethod = 0)
    Local $aResult

    $aResult = DllCall("kernel32.dll", "dword", "SetFilePointerEx", "hwnd", $hFile, "uint64", $iPos, "uint64*", 0, "dword", $iMethod)
    If @error Then Return SetError(1, 0, -1)
    If $aResult[0] = $__WINAPCONSTANT_INVALID_SET_FILE_POINTER Then Return SetError(2, 0, -1)
    Return $aResult[0]
EndFunc   ;==>_WinAPI_SetFilePointer

Func _HexEncode($bInput)
    Local $tInput = DllStructCreate("byte[" & BinaryLen($bInput) & "]")
    DllStructSetData($tInput, 1, $bInput)
    Local $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _
            "ptr", DllStructGetPtr($tInput), _
            "dword", DllStructGetSize($tInput), _
            "dword", 11, _
            "ptr", 0, _
            "dword*", 0)

    If @error Or Not $a_iCall[0] Then
        Return SetError(1, 0, "")
    EndIf

    Local $iSize = $a_iCall[5]
    Local $tOut = DllStructCreate("char[" & $iSize & "]")

    $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _
            "ptr", DllStructGetPtr($tInput), _
            "dword", DllStructGetSize($tInput), _
            "dword", 11, _
            "ptr", DllStructGetPtr($tOut), _
            "dword*", $iSize)

    If @error Or Not $a_iCall[0] Then
        Return SetError(2, 0, "")
    EndIf

    Return SetError(0, 0, DllStructGetData($tOut, 1))

EndFunc  ;==>_HexEncode

Func _GetAllRuns($MFTEntry,$DATA_Offset,$DATA_Size,$DATA_VCNs,$RunListOffset)
Local $RunListClusterNext
_ArrayDelete($RUN_Cluster,1)
_ArrayDelete($RUN_Sectors,1)
$RunListOffset = StringMid($MFTEntry,$DATA_Offset+20,4)
ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf)
$RunListOffset = StringMid($RunListOffset,3,2) & StringMid($RunListOffset,1,2)
$RunListOffset = Dec($RunListOffset)
ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf)
$RunListID = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2),2)
ConsoleWrite("$RunListID = " & $RunListID & @crlf)
$RunListSectorsLenght = Dec(StringMid($RunListID,2,1))
ConsoleWrite("$RunListSectorsLenght = " & $RunListSectorsLenght & @crlf)
$RunListClusterLenght = Dec(StringMid($RunListID,1,1))
ConsoleWrite("$RunListClusterLenght = " & $RunListClusterLenght & @crlf)
$RunListSectors = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2,$RunListSectorsLenght*2)
ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
$entry = ''
For $u = 0 To $RunListSectorsLenght-1
	$mod = StringMid($RunListSectors,($RunListSectorsLenght*2)-(($u*2)+1),2)
	$entry &= $mod
Next
$RunListSectors = Dec($entry)
ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
_ArrayInsert($RUN_Sectors,1,$RunListSectors)
$RunListCluster = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght*2),$RunListClusterLenght*2)
ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
$entry = ''
For $u = 0 To $RunListClusterLenght-1
	$mod = StringMid($RunListCluster,($RunListClusterLenght*2)-(($u*2)+1),2)
	$entry &= $mod
Next
$RunListCluster = Dec($entry)
ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
_ArrayInsert($RUN_Cluster,1,$RunListCluster)
If $RunListSectors = $DATA_VCNs+1 Then
	ConsoleWrite("No more data runs." & @crlf)
	Return
EndIf
$NewRunOffsetBase = $DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght+$RunListClusterLenght)*2
ConsoleWrite("$NewRunOffsetBase = " & $NewRunOffsetBase & @crlf)
$r = 1
$RunListSectorsTotal = $RunListSectors
While $RunListSectors < $DATA_VCNs+1
	$r += 1
	ConsoleWrite("$r = " & $r & @crlf)
	$RunListID = StringMid($MFTEntry,$NewRunOffsetBase,2)
	ConsoleWrite("$RunListID = " & $RunListID & @crlf)
	$RunListSectorsLenght = Dec(StringMid($RunListID,2,1))
	ConsoleWrite("$RunListSectorsLenght = " & $RunListSectorsLenght & @crlf)
	$RunListClusterLenght = Dec(StringMid($RunListID,1,1))
	ConsoleWrite("$RunListClusterLenght = " & $RunListClusterLenght & @crlf)
	$RunListSectors = StringMid($MFTEntry,$NewRunOffsetBase+2,$RunListSectorsLenght*2)
	ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
	$entry = ''
	For $u = 0 To $RunListSectorsLenght-1
		$mod = StringMid($RunListSectors,($RunListSectorsLenght*2)-(($u*2)+1),2)
		$entry &= $mod
	Next
	$RunListSectors = Dec($entry)
	ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
	_ArrayInsert($RUN_Sectors,$r,$RunListSectors)
	$RunListCluster = StringMid($MFTEntry,$NewRunOffsetBase+2+($RunListSectorsLenght*2),$RunListClusterLenght*2)
	ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
	$NegativeMove = 0
;Here we must read to check positive/negative relative move
	If Dec(StringMid($RunListCluster,5,1)) > 7 Then
		$NegativeMove = 1
	EndIf
	ConsoleWrite("$NegativeMove = " & $NegativeMove & @crlf)
	$entry = ''
	For $u = 0 To $RunListClusterLenght-1
		$mod = StringMid($RunListCluster,($RunListClusterLenght*2)-(($u*2)+1),2)
	$entry &= $mod
	Next
	$RunListCluster = Dec($entry)
	ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
	If $NegativeMove = 1 Then
		$p2 = ''
		For $hexnum = 1 To $RunListClusterLenght*2 ;My stupid xor substitute
			$p1 = 'F'
			$p2 &= $p1
		Next
		$FFXor = _HexToDec($p2) - $RunListCluster
		$RunListClusterTmp = $FFXor + 1
		$RunListClusterNext = $RUN_Cluster[$r-1] - $RunListClusterTmp
	ElseIf $NegativeMove = 0 Then
		$RunListClusterNext = $RUN_Cluster[$r-1] + $RunListCluster
	EndIf
	ConsoleWrite("$RunListClusterNext = " & $RunListClusterNext & @crlf)
	_ArrayInsert($RUN_Cluster,$r,$RunListClusterNext)
	$RunListSectorsTotal += $RunListSectors
	If $RunListSectorsTotal = $DATA_VCNs+1 Then
		ConsoleWrite("No more data runs." & @crlf)
		Return
	EndIf
	$NewRunOffsetBase = $NewRunOffsetBase+2+($RunListSectorsLenght+$RunListClusterLenght)*2
	ConsoleWrite("$NewRunOffsetBase = " & $NewRunOffsetBase & @crlf)
WEnd
ConsoleWrite("DataRuns: Something must have gone wrong. Should not be here..." & @crlf)
Return
EndFunc

Func _ReassembleDataRuns($DATA_VCNs)
_WinAPI_SetFilePointer($testfile, ($DATA_VCNs+1)*$SectorsPerCluster*512) ;$DATA_VCNs
_WinAPI_SetEndOfFile($testfile)
_WinAPI_FlushFileBuffers($testfile)
_WinAPI_SetFilePointer($testfile, 0,$FILE_BEGIN)
$TargetOff = 0
For $runs = 1 To Ubound($RUN_Cluster)-1
	ConsoleWrite("Run " & $runs & " reassembling.." & @crlf)
	$tBuffer = DllStructCreate("byte[" & ($RUN_Sectors[$runs]*$SectorsPerCluster*512) & "]")
	_WinAPI_SetFilePointerEx($hFile, $RUN_Cluster[$runs]*$SectorsPerCluster*512,$FILE_BEGIN)
	_WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), ($RUN_Sectors[$runs]*$SectorsPerCluster*512), $nBytes)
	$MFTFragment1 = DllStructGetData($tBuffer, 1)
	ConsoleWrite("Size of buffer = " & DllStructGetSize($tBuffer) & @crlf)
	_WinAPI_WriteFile($testfile, DllStructGetPtr($tBuffer), DllStructGetSize($tBuffer), $nBytes)
	_WinAPI_FlushFileBuffers($testfile)
	$TargetOff += $RUN_Sectors[$runs]*$SectorsPerCluster*512
	_WinAPI_SetFilePointer($testfile, $TargetOff,$FILE_BEGIN)
Next
_WinAPI_CloseHandle($testfile)
Return

EndFunc
; #INDEX# ================================================================
; Title .........: Hex
; Description ...: This module contains numeric conversions between hexadecimal and
;                  decimal numbers overriding the AutoIt limits / capabilities
; Author ........: jennico (jennicoattminusonlinedotde)
; =======================================================================

; #FUNCTION# ==============================================================
; Function Name..: _HexToDec ( "expression" )
; Description ...: Returns decimal expression of a hexadecimal string.
; Parameters ....: expression   - String representation of a hexadecimal expression to be converted to decimal.
; Return values .: Success      - Returns decimal expression of a hexadecimal string.
;                  Failure      - Returns "" (blank string) and sets @error to 1 if string is not hexadecimal type.
; Author ........: jennico (jennicoattminusonlinedotde)
; Remarks .......: working input format: "FFFF" or "0xFFFF" (string format), do NOT pass 0xFFFF without quotation marks (number format).
;                  current AutoIt Dec() limitation: 0x7FFFFFFF (2147483647).
; Related .......: Hex(), Dec(), _DecToHex()
; =======================================================================
Func _HexToDec($hx_hex)
    If StringLeft($hx_hex, 2) = "0x" Then $hx_hex = StringMid($hx_hex, 3)
    If StringIsXDigit($hx_hex) = 0 Then
        SetError(1)
        Return ""
    EndIf
    Local $ret="", $hx_count=0, $hx_array = StringSplit($hx_hex, ""), $Ii, $hx_tmp
    For $Ii = $hx_array[0] To 1 Step -1
        $hx_tmp = StringInStr($HX_REF, $hx_array[$Ii]) - 1
        $ret += $hx_tmp * 16 ^ $hx_count
        $hx_count += 1
    Next
    Return $ret
EndFunc  ;==>_HexToDec()

; #FUNCTION# ==============================================================
; Function Name..: _DecToHex ( expression [, length] )
; Description ...: Returns a string representation of an integer converted to hexadecimal.
; Parameters ....: expression   - The integer to be converted to hexadecimal.
;                  length       - [optional] Number of characters to be returned (no limit).
;                                 If no length specified, leading zeros will be stripped from result.
; Return values .: Success      - Returns a string of length characters representing a hexadecimal expression, zero-padded if necessary.
;                  Failure      - Returns "" (blank string) and sets @error to 1 if expression is not an integer.
; Author ........: jennico (jennicoattminusonlinedotde)
; Remarks .......: Output format "FFFF".
;                  The function will also set @error to 1 if requested length is not sufficient - the returned string will be left truncated.
;                  Be free to modify the function to be working with binary type input - I did not try it though.
;                  current AutoIt Hex() limitation: 0xFFFFFFFF (4294967295).
; Related .......: Hex(), Dec(), _HexToDec()
; =======================================================================
Func _DecToHex($hx_dec, $hx_length = 21)
    If IsInt($hx_dec) = 0 Then
        SetError(1)
        Return ""
    EndIf
    Local $ret = "", $Ii, $hx_tmp, $hx_max
    If $hx_dec < 4294967296 Then
        If $hx_length < 9 Then Return Hex($hx_dec, $hx_length)
        If $hx_length = 21 Then
            $ret = Hex($hx_dec)
            While StringLeft($ret, 1) = "0"
                $ret = StringMid($ret, 2)
            WEnd
            Return $ret
        EndIf
    EndIf
    For $Ii = $hx_length - 1 To 0 Step -1
        $hx_max = 16 ^ $Ii - 1
        If $ret = "" And $hx_length = 21 And $hx_max > $hx_dec Then ContinueLoop
        $hx_tmp = Int($hx_dec/($hx_max+1))
        If $ret = "" And $hx_length = 21 And $Ii > 0 And $hx_tmp = 0 Then ContinueLoop
        $ret &= StringMid($HX_REF, $hx_tmp+1, 1)
        $hx_dec -= $hx_tmp * ($hx_max + 1)
    Next
    $ret=String($ret)
    If $hx_length < 21 And StringLen($ret) < $hx_length Then SetError(1)
    Return $ret
EndFunc  ;==>_DecToHex()

I'm working on a MFT parser as well, a sort of a "decode and log to csv" sort of stuff.

Joakim

Share this post


Link to post
Share on other sites

Posted

Joakim, the result looks most excellent :) (on my system I had to add Global Const $__WINAPCONSTANT_INVALID_SET_FILE_POINTER = -1 to the top). Looking forward to a working parser ;).

Share this post


Link to post
Share on other sites

Posted

Contrary to what I rpeviously insinuated, the code will currently only work for exporting $MFT (and maybe a few others) as the "sequence" of the attributes are hardcoded. The parser in its alpha state is almost done and will be uploaded in short time. With that code I will guess it is also possible to recover deleted files (unless their sectors are overwritten).

Share this post


Link to post
Share on other sites

Posted

I completely forgot to repost here that I created the project at; http://code.google.com/p/mft2csv/downloads/list

One of the important things missing is the filename to mftnumber resolving. Apart from that it is possible to recover deleted files, as long as the sectors are not overwritten and the MFT entry (with the runs information) is also there.

KaFu likes this

Share this post


Link to post
Share on other sites

Posted

Now, this is not a starting point anymore but a full framework of utilities for full-blown NTFS data extraction!

Great work Joakim, posting your update is really appreciated :)!

Share this post


Link to post
Share on other sites

Posted

The newest sources attached.

NTFS_211211.zip

Share this post


Link to post
Share on other sites

Posted

Love it, thanks!

Ian

Share this post


Link to post
Share on other sites

Posted

One of the important things missing is the filename to mftnumber resolving.

A slightly incorrect statement. It should say that the filename path to mtf record number is not yet resolved. Filenames are of course solved, but doing a a raw extraction based on that, may lead to several files with the same name without knowing the path to each of them. Also just updated mft2csv with some fixes regarding integrity checks of records..

mft2csv_v1.4.zip

Share this post


Link to post
Share on other sites

Posted

The calculation of the runs are redone with several fixes and I'm confident that badly fragmented files are now also correctly extracted. My worst $MFT with 121 runs (including many negative moves) are now correct. However compressed and/or sparse files are not yet supported, but should not be too hard to solve I think.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0