Sign in to follow this  
Followers 0

MFT-Access - Reading & parsing the Master File Table on NTFS Filesystems

47 posts in this topic

#1 ·  Posted (edited)

HiHo Community,

for some time now I'm following on the topic of reading & parsing the Master File Table on NTFS Filesystems.

Though I believe I'm on the right track, it's quiet complex and time consuming. So I wanted to share what I have and invite those who are interested to contribute in creating an UDF in the long-run.

Besides the code I also attached an excellent reading I found on the topic containing all relevant information.

;===============================================================================
;
; Description:      MFT-Access, UDF under development
;                   Reading & parsing the Master File Table on NTFS Filesystems
; Version:          v0.0.0.3
; Last modified:    2009-May-02
; URL:              http://www.autoitscript.com/forum/index.php?showtopic=94269
; Author(s):        KaFu (http://www.funk.eu), trancexx
;
;===============================================================================
;
#RequireAdmin
#include <winapi.au3>

Global  $sDriveLetter = "c", $nBytes, $MFT_Offset, $bytes_to_read, $tBuffer, $hdd_handle, $hFile

ConsoleWrite("Analyzing Drive " & $sDriveLetter & ":, Filesystem detected: " & DriveGetFileSystem($sDriveLetter & ":") & @crlf & @crlf)

if DriveGetFileSystem($sDriveLetter & ":") <> "NTFS" then exit

; Read Boot-Sector
; http://www.autoitscript.com/forum/index.php?showtopic=92529&view=findpost&p=665010
; trancexx
Global $hComIn = FileOpen("\\.\" & $sDriveLetter & ":", 16)
Global $bRaw = FileRead($hComIn, 84); 84 bytes is enough
ConsoleWrite(_HexEncode($bRaw) & @CRLF)
FileClose($hComIn)
Global $tRaw = DllStructCreate("byte [" & BinaryLen($bRaw) & "]")
DllStructSetData($tRaw, 1, $bRaw)
Global $tBootSectorSections = DllStructCreate("align 1;byte Jump[3];" & _
        "char SystemName[8];" & _
        "ushort BytesPerSector;" & _
        "ubyte SectorsPerCluster;" & _
        "ushort ReservedSectors;" & _
        "ubyte[3];" & _
        "ushort;" & _
        "ubyte MediaDescriptor;" & _
        "ushort;" & _
        "ushort SectorsPerTrack;" & _
        "ushort NumberOfHeads;" & _
        "dword HiddenSectors;" & _
        "dword;" & _
        "dword;" & _
        "int64 TotalSectors;" & _
        "int64 LogicalClusterNumberforthefileMFT;" & _
        "int64 LogicalClusterNumberforthefileMFTMirr;" & _
        "dword ClustersPerFileRecordSegment;" & _
        "dword ClustersPerIndexBlock;" & _
        "int64 NTFSVolumeSerialNumber;" & _
        "dword Checksum", _
        DllStructGetPtr($tRaw))


ConsoleWrite("Jump:  " & DllStructGetData($tBootSectorSections, "Jump") & @CRLF)
ConsoleWrite("SystemName:  " & DllStructGetData($tBootSectorSections, "SystemName") & @CRLF)
ConsoleWrite("BytesPerSector:  " & DllStructGetData($tBootSectorSections, "BytesPerSector") & @CRLF)
ConsoleWrite("SectorsPerCluster:  " & DllStructGetData($tBootSectorSections, "SectorsPerCluster") & @CRLF)
ConsoleWrite("ReservedSectors:  " & DllStructGetData($tBootSectorSections, "ReservedSectors") & @CRLF)
;ConsoleWrite("6:  " & DllStructGetData($tBootSectorSections, 6) & @CRLF)
;ConsoleWrite("7:  " & DllStructGetData($tBootSectorSections, 7) & @CRLF)
ConsoleWrite("MediaDescriptor:  " & DllStructGetData($tBootSectorSections, "MediaDescriptor") & @CRLF)
;ConsoleWrite("9:  " & DllStructGetData($tBootSectorSections, 9) & @CRLF)
ConsoleWrite("SectorsPerTrack:  " & DllStructGetData($tBootSectorSections, "SectorsPerTrack") & @CRLF)
ConsoleWrite("NumberOfHeads:  " & DllStructGetData($tBootSectorSections, "NumberOfHeads") & @CRLF)
ConsoleWrite("HiddenSectors:  " & DllStructGetData($tBootSectorSections, "HiddenSectors") & @CRLF)
;ConsoleWrite("13:  " & DllStructGetData($tBootSectorSections, 13) & @CRLF)
;ConsoleWrite("14:  " & DllStructGetData($tBootSectorSections, 14) & @CRLF)
ConsoleWrite("TotalSectors:  " & DllStructGetData($tBootSectorSections, "TotalSectors") & @CRLF)
ConsoleWrite("LogicalClusterNumberforthefileMFT:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT") & @CRLF)
ConsoleWrite("LogicalClusterNumberforthefileMFTMirr:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFTMirr") & @CRLF)
ConsoleWrite("ClustersPerFileRecordSegment:  " & DllStructGetData($tBootSectorSections, "ClustersPerFileRecordSegment") & @CRLF)
ConsoleWrite("ClustersPerIndexBlock:  " & DllStructGetData($tBootSectorSections, "ClustersPerIndexBlock") & @CRLF)
ConsoleWrite("VolumeSerialNumber:  " & Ptr(DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber")) & @CRLF)
ConsoleWrite("NTFSVolumeSerialNumber:  " & DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber") & @CRLF)
ConsoleWrite("Checksum:  " & DllStructGetData($tBootSectorSections, "Checksum") & @CRLF)
ConsoleWrite(@CRLF)



; =============================================
; Seek & Read MFT
; =============================================

$MFT_Offset = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") * DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT")
ConsoleWrite("$MFT_Offset: " & $MFT_Offset & @CRLF & @CRLF)

$MFT_Record_Size = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") / 4
; ClustersPerFileRecordSegment = 246
; ConsoleWrite("0x" & hex(246,2)) = 0xF6 >>> 0xF6 = 1/4 Cluster

$tBuffer = DllStructCreate("byte[" & $MFT_Record_Size & "]")
$hdd_handle = "\\.\" & $sDriveLetter & ":"
ConsoleWrite($hdd_handle & @crlf)
$hFile = _WinAPI_CreateFile($hdd_handle, 2, 6, 6)
;_stprintf(szDriveCreate, _T(\\\\.\\%c), cDrive);
; handle = CreateFile("\\.\PhysicalDrive" & "0", GENERIC_READ + GENERIC_WRITE, FILE_SHARE_READ + FILE_SHARE_WRITE, 0, OPEN_EXISTING, 0, 0)

ConsoleWrite("Harddisk Handle: " & $hFile & @crlf)

; =============================================
; first 16 records of MFT contain special information to describe the master file table itself
; http://www.ntfs.com/ntfs-mft.htm
; =============================================
for $i = 0 to 15
    ; first 16 records of MFT contain special information to describe the master file table itself
    ; http://www.ntfs.com/ntfs-mft.htm
    _WinAPI_SetFilePointerEx($hFile, $MFT_Offset + $MFT_Record_Size * $i)
    ;ConsoleWrite("_WinAPI_GetLastError(): " & _WinAPI_GetLastError() & @crlf)
    ;ConsoleWrite("_WinAPI_SetFilePointer: " & @error & @crlf & @crlf)
    _WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $MFT_Record_Size, $nBytes)
    ConsoleWrite("MFT Table Record "& $i & @crlf)
    ConsoleWrite(_HexEncode(DllStructGetData($tBuffer, 1)) & @CRLF)
next

; =============================================
; File & Dir information starts at record 23++; A_Dir or A_File
; http://www.scribd.com/doc/2187280/NTFS-Documentation
; =============================================
for $i = 24 to 34
    _WinAPI_SetFilePointerEx($hFile, $MFT_Offset + $MFT_Record_Size * $i)
    ;ConsoleWrite("_WinAPI_GetLastError(): " & _WinAPI_GetLastError() & @crlf)
    ;ConsoleWrite("_WinAPI_SetFilePointer: " & @error & @crlf & @crlf)
    _WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $MFT_Record_Size, $nBytes)
    ConsoleWrite("MFT Table Record "& $i & @crlf)
    ConsoleWrite(_HexEncode(DllStructGetData($tBuffer, 1)) & @CRLF)
next

_WinAPI_CloseHandle($hFile)

;$sText = BinaryToString(DllStructGetData($tBuffer, 1))
;ConsoleWrite('2) ' & $sText & @LF)





Func _WinAPI_SetFilePointerEx($hFile, $iPos, $iMethod = 0)
    Local $aResult

    $aResult = DllCall("kernel32.dll", "dword", "SetFilePointerEx", "hwnd", $hFile, "uint64", $iPos, "uint64*", 0, "dword", $iMethod)
    If @error Then Return SetError(1, 0, -1)
    If $aResult[0] = $__WINAPCONSTANT_INVALID_SET_FILE_POINTER Then Return SetError(2, 0, -1)
    Return $aResult[0]
EndFunc   ;==>_WinAPI_SetFilePointer





Func _HexEncode($bInput)
    Local $tInput = DllStructCreate("byte[" & BinaryLen($bInput) & "]")
    DllStructSetData($tInput, 1, $bInput)
    Local $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _
            "ptr", DllStructGetPtr($tInput), _
            "dword", DllStructGetSize($tInput), _
            "dword", 11, _
            "ptr", 0, _
            "dword*", 0)

    If @error Or Not $a_iCall[0] Then
        Return SetError(1, 0, "")
    EndIf

    Local $iSize = $a_iCall[5]
    Local $tOut = DllStructCreate("char[" & $iSize & "]")

    $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _
            "ptr", DllStructGetPtr($tInput), _
            "dword", DllStructGetSize($tInput), _
            "dword", 11, _
            "ptr", DllStructGetPtr($tOut), _
            "dword*", $iSize)

    If @error Or Not $a_iCall[0] Then
        Return SetError(2, 0, "")
    EndIf

    Return SetError(0, 0, DllStructGetData($tOut, 1))

EndFunc  ;==>_HexEncode

Happy Coding and with best regards ^_^...

Edit: Added #RequireAdmin on trancexx remark.

Edit: Replaced attachement with link to the NTFS Documentation

Edited by KaFu

Share this post


Link to post
Share on other sites



#4 ·  Posted

I didn't mention (when writing my part) that #RequireAdmin is... required - for Vista and similar.


 

.

eMyvnE

Share this post


Link to post
Share on other sites

#5 ·  Posted

Nice Scipt.

Anyone get it working with Autoit 3.3.6.0?

Guess this line no longer works correctly

Global $hComIn = FileOpen("\\.\" & $sDriveLetter & ":", 16)

Share this post


Link to post
Share on other sites

#6 ·  Posted

Raw-Reading mode was removed in 3.3.4.0 or so. You have to use the _WinAPI_... functions now. (_WinAPI_CreateFle, _WinAPI_ReadFile, _WinAPI_CloseHandle).


*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites

#7 ·  Posted

Nice Scipt.

Anyone get it working with Autoit 3.3.6.0?

Guess this line no longer works correctly

Global $hComIn = FileOpen("\\.\" & $sDriveLetter & ":", 16)

This will work for you :(

Global $tBuffer=DllStructCreate("byte[512]"),$nBytes
Global $hComIn = _WinAPI_CreateFile("\\.\" & $sDriveLetter & ":",2,2,7)
If $hComIn = 0 then exit
$read = _WinAPI_ReadFile($hComIn, DllStructGetPtr($tBuffer), 512, $nBytes)
If $read = 0 then exit
Global $bRaw = DllStructGetData($tBuffer,1)
ConsoleWrite(_HexEncode($bRaw)& @CRLF)
_WinAPI_CloseHandle($hComIn)


Global $tBootSectorSections = DllStructCreate("align 1;byte Jump[3];" & _
        "char SystemName[8];" & _
        "ushort BytesPerSector;" & _
        "ubyte SectorsPerCluster;" & _
        "ushort ReservedSectors;" & _
        "ubyte[3];" & _
        "ushort;" & _
        "ubyte MediaDescriptor;" & _
        "ushort;" & _
        "ushort SectorsPerTrack;" & _
        "ushort NumberOfHeads;" & _
        "dword HiddenSectors;" & _
        "dword;" & _
        "dword;" & _
        "int64 TotalSectors;" & _
        "int64 LogicalClusterNumberforthefileMFT;" & _
        "int64 LogicalClusterNumberforthefileMFTMirr;" & _
        "dword ClustersPerFileRecordSegment;" & _
        "dword ClustersPerIndexBlock;" & _
        "int64 NTFSVolumeSerialNumber;" & _
        "dword Checksum", _
    DllStructGetPtr($tBuffer))

73 108 111 118 101 65 117 116 111 105 116

Share this post


Link to post
Share on other sites

#8 ·  Posted

Hi!

Maybe anyone else knows how to read from the MFT list of all files on disk?

Share this post


Link to post
Share on other sites

#9 ·  Posted

Hi!

Maybe anyone else knows how to read from the MFT list of all files on disk?

You missed Digisoul's post or what?

 

.

eMyvnE

Share this post


Link to post
Share on other sites

#10 ·  Posted

You missed Digisoul's post or what?

I do not understand how to get out of this list of files and folders available on the volume

Share this post


Link to post
Share on other sites

#11 ·  Posted

I do not understand how to get out of this list of files and folders available on the volume

That's the part still missing :(... the script can point you to the location on an NTFS volume where the MFT is located, but I didn't succeeded in writing a proper parser for that. Look at the documentation mentioned above, it's quiet complex and what discouraged me most time consuming :).

Share this post


Link to post
Share on other sites

#12 ·  Posted

I expanded on the code and can now reassemble the complete MFT, regardless of runs (fragments) in either direction. The below code will exit after the MFT is exported. However, it should not be a problem to use it on any of the files in the MFT and use the real name when exporting the file.

#RequireAdmin
#include <winapi.au3>
#include <Array.au3>
$outputMFT = @ScriptDir & '\MFTTest1.bin'
$testfile = _WinAPI_CreateFile($outputMFT,3,6,7)
Global Const $HX_REF="0123456789ABCDEF"
Global $RUN_Cluster[1], $RUN_Sectors[1]
Global  $sDriveLetter = "c", $nBytes, $MFT_Offset, $bytes_to_read, $tBuffer, $hdd_handle, $hFile, $MFTFull
ConsoleWrite("Analyzing Drive " & $sDriveLetter & ":, Filesystem detected: " & DriveGetFileSystem($sDriveLetter & ":") & @crlf & @crlf)
if DriveGetFileSystem($sDriveLetter & ":") <> "NTFS" then exit

Global $tBuffer=DllStructCreate("byte[512]"),$nBytes
Global $hComIn = _WinAPI_CreateFile("\\.\" & $sDriveLetter & ":",2,2,7)
If $hComIn = 0 then exit
$read = _WinAPI_ReadFile($hComIn, DllStructGetPtr($tBuffer), 512, $nBytes)
If $read = 0 then exit
Global $bRaw = DllStructGetData($tBuffer,1)
;ConsoleWrite(_HexEncode($bRaw)& @CRLF)
_WinAPI_CloseHandle($hComIn)

Global $tBootSectorSections = DllStructCreate("align 1;byte Jump[3];" & _
        "char SystemName[8];" & _
        "ushort BytesPerSector;" & _
        "ubyte SectorsPerCluster;" & _
        "ushort ReservedSectors;" & _
        "ubyte[3];" & _
        "ushort;" & _
        "ubyte MediaDescriptor;" & _
        "ushort;" & _
        "ushort SectorsPerTrack;" & _
        "ushort NumberOfHeads;" & _
        "dword HiddenSectors;" & _
        "dword;" & _
        "dword;" & _
        "int64 TotalSectors;" & _
        "int64 LogicalClusterNumberforthefileMFT;" & _
        "int64 LogicalClusterNumberforthefileMFTMirr;" & _
        "dword ClustersPerFileRecordSegment;" & _
        "dword ClustersPerIndexBlock;" & _
        "int64 NTFSVolumeSerialNumber;" & _
        "dword Checksum", _
    DllStructGetPtr($tBuffer))

ConsoleWrite("Jump:  " & DllStructGetData($tBootSectorSections, "Jump") & @CRLF)
ConsoleWrite("SystemName:  " & DllStructGetData($tBootSectorSections, "SystemName") & @CRLF)
ConsoleWrite("BytesPerSector:  " & DllStructGetData($tBootSectorSections, "BytesPerSector") & @CRLF)
ConsoleWrite("SectorsPerCluster:  " & DllStructGetData($tBootSectorSections, "SectorsPerCluster") & @CRLF)
ConsoleWrite("ReservedSectors:  " & DllStructGetData($tBootSectorSections, "ReservedSectors") & @CRLF)
;ConsoleWrite("6:  " & DllStructGetData($tBootSectorSections, 6) & @CRLF)
;ConsoleWrite("7:  " & DllStructGetData($tBootSectorSections, 7) & @CRLF)
ConsoleWrite("MediaDescriptor:  " & DllStructGetData($tBootSectorSections, "MediaDescriptor") & @CRLF)
;ConsoleWrite("9:  " & DllStructGetData($tBootSectorSections, 9) & @CRLF)
ConsoleWrite("SectorsPerTrack:  " & DllStructGetData($tBootSectorSections, "SectorsPerTrack") & @CRLF)
ConsoleWrite("NumberOfHeads:  " & DllStructGetData($tBootSectorSections, "NumberOfHeads") & @CRLF)
ConsoleWrite("HiddenSectors:  " & DllStructGetData($tBootSectorSections, "HiddenSectors") & @CRLF)
;ConsoleWrite("13:  " & DllStructGetData($tBootSectorSections, 13) & @CRLF)
;ConsoleWrite("14:  " & DllStructGetData($tBootSectorSections, 14) & @CRLF)
ConsoleWrite("TotalSectors:  " & DllStructGetData($tBootSectorSections, "TotalSectors") & @CRLF)
ConsoleWrite("LogicalClusterNumberforthefileMFT:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT") & @CRLF)
ConsoleWrite("LogicalClusterNumberforthefileMFTMirr:  " & DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFTMirr") & @CRLF)
ConsoleWrite("ClustersPerFileRecordSegment:  " & DllStructGetData($tBootSectorSections, "ClustersPerFileRecordSegment") & @CRLF)
ConsoleWrite("ClustersPerIndexBlock:  " & DllStructGetData($tBootSectorSections, "ClustersPerIndexBlock") & @CRLF)
ConsoleWrite("VolumeSerialNumber:  " & Ptr(DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber")) & @CRLF)
ConsoleWrite("NTFSVolumeSerialNumber:  " & DllStructGetData($tBootSectorSections, "NTFSVolumeSerialNumber") & @CRLF)
ConsoleWrite("Checksum:  " & DllStructGetData($tBootSectorSections, "Checksum") & @CRLF)
ConsoleWrite(@CRLF)
Global $SectorsPerCluster = DllStructGetData($tBootSectorSections, "SectorsPerCluster")
; =============================================
; Seek & Read MFT
; =============================================
$MFT_Offset = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") * DllStructGetData($tBootSectorSections, "LogicalClusterNumberforthefileMFT")
ConsoleWrite("$MFT_Offset: " & $MFT_Offset & @CRLF & @CRLF)
$MFT_Record_Size = DllStructGetData($tBootSectorSections, "BytesPerSector") * DllStructGetData($tBootSectorSections, "SectorsPerCluster") / 4
; ClustersPerFileRecordSegment = 246
; ConsoleWrite("0x" & hex(246,2)) = 0xF6 >>> 0xF6 = 1/4 Cluster
$tBuffer = DllStructCreate("byte[" & $MFT_Record_Size & "]")
$hdd_handle = "\\.\" & $sDriveLetter & ":"
ConsoleWrite($hdd_handle & @crlf)
$hFile = _WinAPI_CreateFile($hdd_handle, 2, 6, 6)
;_stprintf(szDriveCreate, _T(\\\\.\\%c), cDrive);
; handle = CreateFile("\\.\PhysicalDrive" & "0", GENERIC_READ + GENERIC_WRITE, FILE_SHARE_READ + FILE_SHARE_WRITE, 0, OPEN_EXISTING, 0, 0)
ConsoleWrite("Harddisk Handle: " & $hFile & @crlf)
;------------- 2*....
$SI_Sig = '10'; hex
$FILE_NAME = '30'
$DATA = '80'
$BITMAP = 'B0'
$MFTHeaderSize = 56*2 ;Hardcoded for XP and later, but will fail on 2k
;$SI_Offset = $MFTHeaderSize+(1*2)+1
;$Resident = 1
for $i = 0 to 1
    If $i = 1 Then ExitLoop
    _WinAPI_SetFilePointerEx($hFile, $MFT_Offset + $MFT_Record_Size * $i)
    _WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $MFT_Record_Size, $nBytes)
    ConsoleWrite("MFT Table Record "& $i & @crlf)
;    ConsoleWrite(_HexEncode(DllStructGetData($tBuffer, 1)) & @CRLF)
    $MFTEntry = DllStructGetData($tBuffer, 1)
    $SI_Offset = (Dec(StringMid($MFTEntry,43,2))*2)+3
    ConsoleWrite("$SI_Offset = " & $SI_Offset & @crlf)
    ConsoleWrite("(Dec(StringMid($MFTEntry,43,2))*2)+3 = " & (Dec(StringMid($MFTEntry,43,2))*2)+3 & @crlf)
    $SI_Size = StringMid($MFTEntry,$SI_Offset+8,8)
    ConsoleWrite("$SI_Size = " & $SI_Size & @crlf)
    $SI_Size = StringMid($SI_Size,7,2) & StringMid($SI_Size,5,2) & StringMid($SI_Size,3,2) & StringMid($SI_Size,1,2)
    $SI_Size = Dec($SI_Size)
    ConsoleWrite("$SI_Size = " & $SI_Size & @crlf)
    $FN_Offset = $SI_Offset + ($SI_Size*2)
    ConsoleWrite("$FN_Offset = " & $FN_Offset & @crlf)
    $FN_Size = StringMid($MFTEntry,$FN_Offset+8,8)
    ConsoleWrite("$FN_Size = " & $FN_Size & @crlf)
    $FN_Size = StringMid($FN_Size,7,2) & StringMid($FN_Size,5,2) & StringMid($FN_Size,3,2) & StringMid($FN_Size,1,2)
    $FN_Size = Dec($FN_Size)
    ConsoleWrite("$FN_Size = " & $FN_Size & @crlf)
    $DATA_Offset = $FN_Offset + ($FN_Size*2)
    ConsoleWrite("$DATA_Offset = " & $DATA_Offset & @crlf)
    $DATA_Size = StringMid($MFTEntry,$DATA_Offset+8,8)
    ConsoleWrite("$DATA_Size = " & $DATA_Size & @crlf)
    $DATA_Size = StringMid($DATA_Size,7,2) & StringMid($DATA_Size,5,2) & StringMid($DATA_Size,3,2) & StringMid($DATA_Size,1,2)
    $DATA_Size = Dec($DATA_Size)
    ConsoleWrite("$DATA_Size = " & $DATA_Size & @crlf)
    $DATA_VCNs = StringMid($MFTEntry,$DATA_Offset+48,16)
;   $DATA_VCNs = StringMid($MFTEntry,$DATA_Offset+48,8)
    ConsoleWrite("$DATA_VCNs = " & $DATA_VCNs & @crlf)
    $DATA_VCNs = StringMid($DATA_VCNs,15,2) & StringMid($DATA_VCNs,13,2) & StringMid($DATA_VCNs,11,2) & StringMid($DATA_VCNs,9,2) & StringMid($DATA_VCNs,7,2) & StringMid($DATA_VCNs,5,2) & StringMid($DATA_VCNs,3,2) & StringMid($DATA_VCNs,1,2)
;   $DATA_VCNs = StringMid($DATA_VCNs,7,2) & StringMid($DATA_VCNs,5,2) & StringMid($DATA_VCNs,3,2) & StringMid($DATA_VCNs,1,2)
    ConsoleWrite("$DATA_VCNs = " & $DATA_VCNs & @crlf)
    $DATA_VCNs = _HexToDec($DATA_VCNs)
;   $DATA_VCNs = Dec($DATA_VCNs)
    ConsoleWrite("$DATA_VCNs = " & $DATA_VCNs & @crlf)
    $NonResidentFlag = StringMid($MFTEntry,$DATA_Offset+16,2)
    ConsoleWrite("$NonResidentFlag = " & $NonResidentFlag & @crlf)
    If $NonResidentFlag = '01' Then
        $RunListOffset = StringMid($MFTEntry,$DATA_Offset+20,4)
        ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf)
        $RunListOffset = StringMid($RunListOffset,3,2) & StringMid($RunListOffset,1,2)
        $RunListOffset = Dec($RunListOffset)
        ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf)
        $RunListID = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2),2)
        ConsoleWrite("$RunListID = " & $RunListID & @crlf)
        $RunListSectorsLenght = Dec(StringMid($RunListID,2,1))
        ConsoleWrite("$RunListSectorsLenght = " & $RunListSectorsLenght & @crlf)
        $RunListClusterLenght = Dec(StringMid($RunListID,1,1))
        ConsoleWrite("$RunListClusterLenght = " & $RunListClusterLenght & @crlf)
        $RunListSectors = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2,$RunListSectorsLenght*2)
        ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
        $r = 1
        $entry = ''
        For $u = 0 To $RunListSectorsLenght-1
            $mod = StringMid($RunListSectors,($RunListSectorsLenght*2)-(($u*2)+1),2)
            $entry &= $mod
        Next
        $RunListSectors = Dec($entry)
        _ArrayInsert($RUN_Sectors,1,$RunListSectors)
        ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
        $RunListCluster = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght*2),$RunListClusterLenght*2)
        ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
        $entry = ''
        For $u = 0 To $RunListClusterLenght-1
            $mod = StringMid($RunListCluster,($RunListClusterLenght*2)-(($u*2)+1),2)
            $entry &= $mod
        Next
        $RunListCluster = Dec($entry)
        _ArrayInsert($RUN_Cluster,1,$RunListCluster)
        ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
        If $RunListSectors = $DATA_VCNs+1 Then
            ConsoleWrite("No more data runs." & @crlf)
        Else
            ConsoleWrite("More data runs exist." & @crlf)
            $NewRunOffsetBase = $DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght+$RunListClusterLenght)*2
            _GetAllRuns($MFTEntry,$DATA_Offset,$DATA_Size,$DATA_VCNs,$NewRunOffsetBase)
        EndIf
        _ReassembleDataRuns($DATA_VCNs)
    EndIf
    $BITMAP_Offset = $DATA_Offset + ($DATA_Size*2)
    ConsoleWrite("$BITMAP_Offset = " & $BITMAP_Offset & @crlf)
    $BITMAP_Size = StringMid($MFTEntry,$BITMAP_Offset+8,8)
    ConsoleWrite("$BITMAP_Size = " & $BITMAP_Size & @crlf)
    $BITMAP_Size = StringMid($BITMAP_Size,7,2) & StringMid($BITMAP_Size,5,2) & StringMid($BITMAP_Size,3,2) & StringMid($BITMAP_Size,1,2)
    $BITMAP_Size = Dec($BITMAP_Size)
    ConsoleWrite("$BITMAP_Size = " & $BITMAP_Size & @crlf)
Next
;_ArrayDisplay($RUN_Sectors,"Sectors per RUN")
;_ArrayDisplay($RUN_Cluster,"Cluster number for RUN")
_WinAPI_CloseHandle($hFile)
Exit

Func _WinAPI_SetFilePointerEx($hFile, $iPos, $iMethod = 0)
    Local $aResult

    $aResult = DllCall("kernel32.dll", "dword", "SetFilePointerEx", "hwnd", $hFile, "uint64", $iPos, "uint64*", 0, "dword", $iMethod)
    If @error Then Return SetError(1, 0, -1)
    If $aResult[0] = $__WINAPCONSTANT_INVALID_SET_FILE_POINTER Then Return SetError(2, 0, -1)
    Return $aResult[0]
EndFunc   ;==>_WinAPI_SetFilePointer

Func _HexEncode($bInput)
    Local $tInput = DllStructCreate("byte[" & BinaryLen($bInput) & "]")
    DllStructSetData($tInput, 1, $bInput)
    Local $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _
            "ptr", DllStructGetPtr($tInput), _
            "dword", DllStructGetSize($tInput), _
            "dword", 11, _
            "ptr", 0, _
            "dword*", 0)

    If @error Or Not $a_iCall[0] Then
        Return SetError(1, 0, "")
    EndIf

    Local $iSize = $a_iCall[5]
    Local $tOut = DllStructCreate("char[" & $iSize & "]")

    $a_iCall = DllCall("crypt32.dll", "int", "CryptBinaryToString", _
            "ptr", DllStructGetPtr($tInput), _
            "dword", DllStructGetSize($tInput), _
            "dword", 11, _
            "ptr", DllStructGetPtr($tOut), _
            "dword*", $iSize)

    If @error Or Not $a_iCall[0] Then
        Return SetError(2, 0, "")
    EndIf

    Return SetError(0, 0, DllStructGetData($tOut, 1))

EndFunc  ;==>_HexEncode

Func _GetAllRuns($MFTEntry,$DATA_Offset,$DATA_Size,$DATA_VCNs,$RunListOffset)
Local $RunListClusterNext
_ArrayDelete($RUN_Cluster,1)
_ArrayDelete($RUN_Sectors,1)
$RunListOffset = StringMid($MFTEntry,$DATA_Offset+20,4)
ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf)
$RunListOffset = StringMid($RunListOffset,3,2) & StringMid($RunListOffset,1,2)
$RunListOffset = Dec($RunListOffset)
ConsoleWrite("$RunListOffset = " & $RunListOffset & @crlf)
$RunListID = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2),2)
ConsoleWrite("$RunListID = " & $RunListID & @crlf)
$RunListSectorsLenght = Dec(StringMid($RunListID,2,1))
ConsoleWrite("$RunListSectorsLenght = " & $RunListSectorsLenght & @crlf)
$RunListClusterLenght = Dec(StringMid($RunListID,1,1))
ConsoleWrite("$RunListClusterLenght = " & $RunListClusterLenght & @crlf)
$RunListSectors = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2,$RunListSectorsLenght*2)
ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
$entry = ''
For $u = 0 To $RunListSectorsLenght-1
    $mod = StringMid($RunListSectors,($RunListSectorsLenght*2)-(($u*2)+1),2)
    $entry &= $mod
Next
$RunListSectors = Dec($entry)
ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
_ArrayInsert($RUN_Sectors,1,$RunListSectors)
$RunListCluster = StringMid($MFTEntry,$DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght*2),$RunListClusterLenght*2)
ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
$entry = ''
For $u = 0 To $RunListClusterLenght-1
    $mod = StringMid($RunListCluster,($RunListClusterLenght*2)-(($u*2)+1),2)
    $entry &= $mod
Next
$RunListCluster = Dec($entry)
ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
_ArrayInsert($RUN_Cluster,1,$RunListCluster)
If $RunListSectors = $DATA_VCNs+1 Then
    ConsoleWrite("No more data runs." & @crlf)
    Return
EndIf
$NewRunOffsetBase = $DATA_Offset+($RunListOffset*2)+2+($RunListSectorsLenght+$RunListClusterLenght)*2
ConsoleWrite("$NewRunOffsetBase = " & $NewRunOffsetBase & @crlf)
$r = 1
$RunListSectorsTotal = $RunListSectors
While $RunListSectors < $DATA_VCNs+1
    $r += 1
    ConsoleWrite("$r = " & $r & @crlf)
    $RunListID = StringMid($MFTEntry,$NewRunOffsetBase,2)
    ConsoleWrite("$RunListID = " & $RunListID & @crlf)
    $RunListSectorsLenght = Dec(StringMid($RunListID,2,1))
    ConsoleWrite("$RunListSectorsLenght = " & $RunListSectorsLenght & @crlf)
    $RunListClusterLenght = Dec(StringMid($RunListID,1,1))
    ConsoleWrite("$RunListClusterLenght = " & $RunListClusterLenght & @crlf)
    $RunListSectors = StringMid($MFTEntry,$NewRunOffsetBase+2,$RunListSectorsLenght*2)
    ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
    $entry = ''
    For $u = 0 To $RunListSectorsLenght-1
        $mod = StringMid($RunListSectors,($RunListSectorsLenght*2)-(($u*2)+1),2)
        $entry &= $mod
    Next
    $RunListSectors = Dec($entry)
    ConsoleWrite("$RunListSectors = " & $RunListSectors & @crlf)
    _ArrayInsert($RUN_Sectors,$r,$RunListSectors)
    $RunListCluster = StringMid($MFTEntry,$NewRunOffsetBase+2+($RunListSectorsLenght*2),$RunListClusterLenght*2)
    ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
    $NegativeMove = 0
;Here we must read to check positive/negative relative move
    If Dec(StringMid($RunListCluster,5,1)) > 7 Then
        $NegativeMove = 1
    EndIf
    ConsoleWrite("$NegativeMove = " & $NegativeMove & @crlf)
    $entry = ''
    For $u = 0 To $RunListClusterLenght-1
        $mod = StringMid($RunListCluster,($RunListClusterLenght*2)-(($u*2)+1),2)
    $entry &= $mod
    Next
    $RunListCluster = Dec($entry)
    ConsoleWrite("$RunListCluster = " & $RunListCluster & @crlf)
    If $NegativeMove = 1 Then
        $p2 = ''
        For $hexnum = 1 To $RunListClusterLenght*2 ;My stupid xor substitute
            $p1 = 'F'
            $p2 &= $p1
        Next
        $FFXor = _HexToDec($p2) - $RunListCluster
        $RunListClusterTmp = $FFXor + 1
        $RunListClusterNext = $RUN_Cluster[$r-1] - $RunListClusterTmp
    ElseIf $NegativeMove = 0 Then
        $RunListClusterNext = $RUN_Cluster[$r-1] + $RunListCluster
    EndIf
    ConsoleWrite("$RunListClusterNext = " & $RunListClusterNext & @crlf)
    _ArrayInsert($RUN_Cluster,$r,$RunListClusterNext)
    $RunListSectorsTotal += $RunListSectors
    If $RunListSectorsTotal = $DATA_VCNs+1 Then
        ConsoleWrite("No more data runs." & @crlf)
        Return
    EndIf
    $NewRunOffsetBase = $NewRunOffsetBase+2+($RunListSectorsLenght+$RunListClusterLenght)*2
    ConsoleWrite("$NewRunOffsetBase = " & $NewRunOffsetBase & @crlf)
WEnd
ConsoleWrite("DataRuns: Something must have gone wrong. Should not be here..." & @crlf)
Return
EndFunc

Func _ReassembleDataRuns($DATA_VCNs)
_WinAPI_SetFilePointer($testfile, ($DATA_VCNs+1)*$SectorsPerCluster*512) ;$DATA_VCNs
_WinAPI_SetEndOfFile($testfile)
_WinAPI_FlushFileBuffers($testfile)
_WinAPI_SetFilePointer($testfile, 0,$FILE_BEGIN)
$TargetOff = 0
For $runs = 1 To Ubound($RUN_Cluster)-1
    ConsoleWrite("Run " & $runs & " reassembling.." & @crlf)
    $tBuffer = DllStructCreate("byte[" & ($RUN_Sectors[$runs]*$SectorsPerCluster*512) & "]")
    _WinAPI_SetFilePointerEx($hFile, $RUN_Cluster[$runs]*$SectorsPerCluster*512,$FILE_BEGIN)
    _WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), ($RUN_Sectors[$runs]*$SectorsPerCluster*512), $nBytes)
    $MFTFragment1 = DllStructGetData($tBuffer, 1)
    ConsoleWrite("Size of buffer = " & DllStructGetSize($tBuffer) & @crlf)
    _WinAPI_WriteFile($testfile, DllStructGetPtr($tBuffer), DllStructGetSize($tBuffer), $nBytes)
    _WinAPI_FlushFileBuffers($testfile)
    $TargetOff += $RUN_Sectors[$runs]*$SectorsPerCluster*512
    _WinAPI_SetFilePointer($testfile, $TargetOff,$FILE_BEGIN)
Next
_WinAPI_CloseHandle($testfile)
Return

EndFunc
; #INDEX# ================================================================
; Title .........: Hex
; Description ...: This module contains numeric conversions between hexadecimal and
;                  decimal numbers overriding the AutoIt limits / capabilities
; Author ........: jennico (jennicoattminusonlinedotde)
; =======================================================================

; #FUNCTION# ==============================================================
; Function Name..: _HexToDec ( "expression" )
; Description ...: Returns decimal expression of a hexadecimal string.
; Parameters ....: expression   - String representation of a hexadecimal expression to be converted to decimal.
; Return values .: Success      - Returns decimal expression of a hexadecimal string.
;                  Failure      - Returns "" (blank string) and sets @error to 1 if string is not hexadecimal type.
; Author ........: jennico (jennicoattminusonlinedotde)
; Remarks .......: working input format: "FFFF" or "0xFFFF" (string format), do NOT pass 0xFFFF without quotation marks (number format).
;                  current AutoIt Dec() limitation: 0x7FFFFFFF (2147483647).
; Related .......: Hex(), Dec(), _DecToHex()
; =======================================================================
Func _HexToDec($hx_hex)
    If StringLeft($hx_hex, 2) = "0x" Then $hx_hex = StringMid($hx_hex, 3)
    If StringIsXDigit($hx_hex) = 0 Then
        SetError(1)
        Return ""
    EndIf
    Local $ret="", $hx_count=0, $hx_array = StringSplit($hx_hex, ""), $Ii, $hx_tmp
    For $Ii = $hx_array[0] To 1 Step -1
        $hx_tmp = StringInStr($HX_REF, $hx_array[$Ii]) - 1
        $ret += $hx_tmp * 16 ^ $hx_count
        $hx_count += 1
    Next
    Return $ret
EndFunc  ;==>_HexToDec()

; #FUNCTION# ==============================================================
; Function Name..: _DecToHex ( expression [, length] )
; Description ...: Returns a string representation of an integer converted to hexadecimal.
; Parameters ....: expression   - The integer to be converted to hexadecimal.
;                  length       - [optional] Number of characters to be returned (no limit).
;                                 If no length specified, leading zeros will be stripped from result.
; Return values .: Success      - Returns a string of length characters representing a hexadecimal expression, zero-padded if necessary.
;                  Failure      - Returns "" (blank string) and sets @error to 1 if expression is not an integer.
; Author ........: jennico (jennicoattminusonlinedotde)
; Remarks .......: Output format "FFFF".
;                  The function will also set @error to 1 if requested length is not sufficient - the returned string will be left truncated.
;                  Be free to modify the function to be working with binary type input - I did not try it though.
;                  current AutoIt Hex() limitation: 0xFFFFFFFF (4294967295).
; Related .......: Hex(), Dec(), _HexToDec()
; =======================================================================
Func _DecToHex($hx_dec, $hx_length = 21)
    If IsInt($hx_dec) = 0 Then
        SetError(1)
        Return ""
    EndIf
    Local $ret = "", $Ii, $hx_tmp, $hx_max
    If $hx_dec < 4294967296 Then
        If $hx_length < 9 Then Return Hex($hx_dec, $hx_length)
        If $hx_length = 21 Then
            $ret = Hex($hx_dec)
            While StringLeft($ret, 1) = "0"
                $ret = StringMid($ret, 2)
            WEnd
            Return $ret
        EndIf
    EndIf
    For $Ii = $hx_length - 1 To 0 Step -1
        $hx_max = 16 ^ $Ii - 1
        If $ret = "" And $hx_length = 21 And $hx_max > $hx_dec Then ContinueLoop
        $hx_tmp = Int($hx_dec/($hx_max+1))
        If $ret = "" And $hx_length = 21 And $Ii > 0 And $hx_tmp = 0 Then ContinueLoop
        $ret &= StringMid($HX_REF, $hx_tmp+1, 1)
        $hx_dec -= $hx_tmp * ($hx_max + 1)
    Next
    $ret=String($ret)
    If $hx_length < 21 And StringLen($ret) < $hx_length Then SetError(1)
    Return $ret
EndFunc  ;==>_DecToHex()

I'm working on a MFT parser as well, a sort of a "decode and log to csv" sort of stuff.

Joakim

Share this post


Link to post
Share on other sites

#13 ·  Posted

Joakim, the result looks most excellent :) (on my system I had to add Global Const $__WINAPCONSTANT_INVALID_SET_FILE_POINTER = -1 to the top). Looking forward to a working parser ;).

Share this post


Link to post
Share on other sites

#14 ·  Posted

Contrary to what I rpeviously insinuated, the code will currently only work for exporting $MFT (and maybe a few others) as the "sequence" of the attributes are hardcoded. The parser in its alpha state is almost done and will be uploaded in short time. With that code I will guess it is also possible to recover deleted files (unless their sectors are overwritten).

Share this post


Link to post
Share on other sites

#15 ·  Posted

I completely forgot to repost here that I created the project at; http://code.google.com/p/mft2csv/downloads/list

One of the important things missing is the filename to mftnumber resolving. Apart from that it is possible to recover deleted files, as long as the sectors are not overwritten and the MFT entry (with the runs information) is also there.

1 person likes this

Share this post


Link to post
Share on other sites

#16 ·  Posted

Now, this is not a starting point anymore but a full framework of utilities for full-blown NTFS data extraction!

Great work Joakim, posting your update is really appreciated :)!

Share this post


Link to post
Share on other sites

#17 ·  Posted

The newest sources attached.

NTFS_211211.zip

Share this post


Link to post
Share on other sites

#18 ·  Posted

Love it, thanks!

Ian


My projects:

  • IP Scanner - Multi-threaded ping tool to scan your available networks for used and available IP addresses, shows ping times, resolves IPs in to host names, and allows individual IPs to be pinged.
  • INFSniff - Great technicians tool - a tool which scans DriverPacks archives for INF files and parses out the HWIDs to a database file, and rapidly scans the local machine's HWIDs, searches the database for matches, and installs them.
  • PPK3 (Persistent Process Killer V3) - Another for the techs - suppress running processes that you need to keep away, helpful when fighting spyware/viruses.
  • Sync Tool - Folder sync tool with lots of real time information and several checking methods.
  • USMT Front End - Front End for Microsoft's User State Migration Tool, including all files needed for USMT 3.01 and 4.01, 32 bit and 64 bit versions.
  • Audit Tool - Computer audit tool to gather vital hardware, Windows, and Office information for IT managers and field techs. Capabilities include creating a customized site agent.
  • CSV Viewer - Displays CSV files with automatic column sizing and font selection. Lines can also be copied to the clipboard for data extraction.
  • MyDirStat - Lists number and size of files on a drive or specified path, allows for deletion within the app.
  • 2048 Game - My version of 2048, fun tile game.
  • Juice Lab - Ecigarette liquid making calculator.
  • Data Protector - Secure notes to save sensitive information.
  • VHD Footer - Add a footer to a forensic hard drive image to allow it to be mounted or used as a virtual machine hard drive.
  • Find in File - Searches files containing a specified phrase.

Share this post


Link to post
Share on other sites

#19 ·  Posted

One of the important things missing is the filename to mftnumber resolving.

A slightly incorrect statement. It should say that the filename path to mtf record number is not yet resolved. Filenames are of course solved, but doing a a raw extraction based on that, may lead to several files with the same name without knowing the path to each of them. Also just updated mft2csv with some fixes regarding integrity checks of records..

mft2csv_v1.4.zip

Share this post


Link to post
Share on other sites

#20 ·  Posted

The calculation of the runs are redone with several fixes and I'm confident that badly fragmented files are now also correctly extracted. My worst $MFT with 121 runs (including many negative moves) are now correct. However compressed and/or sparse files are not yet supported, but should not be too hard to solve I think.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0