FileWipe - very fast (using memory mapped files + memset)

[Zedna]

This function rewrite whole content of given file by defined (implicitly 0x0) character and finally delete it, it's called as secure deleting.

This should be VERY fast!!

Just note that there is no error checking inside FileWipe()

Maybe I will post later also version with error checking.

You can comment line FileDelete($sFileName) to see how it's rewritten before deletion.

EDIT: There can be problems with too big files (>2GB or > amount of RAM) see post #8 by Kafu

expandcollapse popup

#include <WinAPI.au3>

; prepare testing file
FileDelete('1.txt')
FileWrite('1.txt','123abc')

FileWipe('1.txt')

Func FileWipe($sFileName, $nByte = 0x0)
    If Not FileExists($sFileName) Then Return
    $iSize = FileGetSize($sFileName)
    $hFile = _WinAPI_CreateFile($sFileName, 2, 6)
    $hMapping = _WinAPI_CreateFileMapping($hFile)
    $pAddress = _WinAPI_MapViewOfFile($hMapping)
    MemSet($pAddress, $nByte, $iSize)
    _WinAPI_UnmapViewOfFile($pAddress)
    _WinAPI_CloseHandle($hMapping)
    _WinAPI_CloseHandle($hFile)
     FileDelete($sFileName)
EndFunc

Func MemSet($pDest, $nChar, $nCount)
    DllCall("msvcrt.dll", "ptr:cdecl", "memset", "ptr", $pDest, "int", $nChar, "int", $nCount)
    If @error Then Return SetError(1,0,False)
    Return True
EndFunc

; from WinAPIEx - just simplified for this purpose

Func _WinAPI_CreateFileMapping($hFile)
    Local $Ret = DllCall('kernel32.dll', 'ptr', 'CreateFileMappingW', 'ptr', $hFile, 'ptr', 0, 'dword', 0x4, 'dword', 0, 'dword', 0, 'ptr', 0)
    If (@error) Or (Not $Ret[0]) Then Return SetError(1, 0, 0)
    Return $Ret[0]
EndFunc

Func _WinAPI_MapViewOfFile($hMapping)
    Local $Ret = DllCall('kernel32.dll', 'ptr', 'MapViewOfFile', 'ptr', $hMapping, 'dword', 0x6, 'dword', 0, 'dword', 0, 'dword', 0)
    If (@error) Or (Not $Ret[0]) Then Return SetError(1, 0, 0)
    Return $Ret[0]
EndFunc

Func _WinAPI_UnmapViewOfFile($pAddress)
    DllCall('kernel32.dll', 'int', 'UnmapViewOfFile', 'ptr', $pAddress)
    If @error Then Return SetError(1, 0, 0)
    Return 1
EndFunc

Here is my first test version without memory mapped files:

Func FileWipe($sFileName, $nByte = 0x0)
    If Not FileExists($sFileName) Then Return
    $iSize = FileGetSize($sFileName)
    $tBuffer = DLLStructCreate("byte[" & $iSize & "]")
    MemSet(DLLStructGetPtr($tBuffer), $nByte, $iSize)
    $hFile = _WinAPI_CreateFile($sFileName, 2, 6)
    _WinAPI_WriteFile($hFile, DLLStructGetPtr($tBuffer), $iSize, $iSize)
    _WinAPI_CloseHandle($hFile)
    FileDelete($sFileName)
EndFunc

Edited May 7, 2012 by Zedna

[Zedna]

I could swear I read somewhere that overwriting a file with Null characters does absolutely nothing, but I can't find anything to support that statement. I did find this though. http://www.planetsourcecode.com/vb/scripts/ShowCode.asp?txtCodeId=29245

1) absolut nonsense :-)

There are several wipe methods, some of them rewrite file several times with different and/or random data, this my method is basic one.

You can easily accomodate it to your needs.

2) your link is obsolete, http://www.planetsourcecode.com/vb/scrip...ts/ShowCode.asp?txtCodeId=2924 --> anyway this code does nothing as I see in first quick look

Please post only if you know something about it. Thanks.

EDIT:

some links

http://en.wikipedia.org/wiki/Data_remanence

http://en.wikipedia.org/wiki/Data_erasure

Edited May 3, 2012 by Zedna

[knightz93]

nice!! i search for secure deleting method but i didn't find it in Autoit.. Thanks for Zedna . so, file will be null before deleting and we can't recover this file again, right??

[MvGulik]

? ... Skeptic here! ... Its this code writing faster to disk than AutoIt itself will do?

(That is at least what it sounds like to me.)

[Yashied]

@Zedna

You can use _WinAPI_FillMemory() or _WinAPI_ZeroMemory() instead of MemSet().

[ricky]

Hello,

thanks for you code, very useful.

In the code without mapped files, there is a little mistake in the code :

$hFile = _WinAPI_CreateFile($sFileName, 2, 6)

What do you think about my code :

Func _FileWipeSecure($sFileName, $times = 4)
  Local $iSize, $tBuffer, $hFile   
  Local $nByte[3] = [1,0x0,0x1]
  if $times < 1 then $times = 1
  If Not FileExists($sFileName) Then SetError(1,0,0)
    $iSize = FileGetSize($sFileName)
    $tBuffer = DLLStructCreate("byte[" & $iSize & "]")
For $i = 1 to $times
  MemSet(DLLStructGetPtr($tBuffer), $nByte[$nByte[0]], $iSize)
  $hFile = _WinAPI_CreateFile($sFileName, 2, 6)
  _WinAPI_WriteFile($hFile, DLLStructGetPtr($tBuffer), $iSize, $iSize)
  _WinAPI_CloseHandle($hFile)
  if $nByte[0] = 1 Then
   $nByte[0] += 1
  Else
   $nByte[0] = 1
  Endif
Next
    Return FileDelete($sFileName)
EndFunc

Edited May 7, 2012 by ricky03

[Zedna]

@ricky03

Thanks for correction in variable name, fixed.

I don't like multiple overwriting, so I didn't do that. Anyway feel free to make it in your modifications.

I think memory mapped files will be much faster, maybe I will do some test scripts later.

Edited May 7, 2012 by Zedna

[KaFu]

Hi Zedna,

I really like this one , but some points came to my mind. Better use _Winapi_FileGetSizeEx to pass the 2 GB limit. Also I would recommend to process larger files in a loop, as the files are mapped to the process address space and thus large files are likly to consume too much RAM (up to crashing the script). My post on " should contain all required details for that.

Regards

Edited May 7, 2012 by KaFu

[MvGulik]

Other version. (although for all I know it might actually NOT overwrite the original file data. ... Disk cluster and such.)

Func WipeFile_Zeros($sFilespec)
    If FileExists($sFilespec) Then
        Local Const $FILE_EREASE = 2
        Local Const $FILE_BEGIN = 0 ;; <constants.au3>
        Local $iLenght = FileGetSize($sFilespec)
        Local $hFile = FileOpen($sFilespec, $FILE_EREASE) ;; Binary(+16) don't seems to matter.)
        FileSetPos($hFile, $iLenght - 1, $FILE_BEGIN)
        FileWrite($hFile, Chr(0))
        FileClose($hFile)
    EndIf
EndFunc

Err ... think I just got scooped.

Whats wrong with this code. ? (... fixed)

---

Note to self: forum (update) message might be a little older than you think.</constants.au3>

Edited May 7, 2012 by MvGulik

[MvGulik]

(although for all I know it might actually NOT overwrite the original file data. ... Disk cluster and such.)

After a little testing. This seems to be exactly the case. ('seems': as in, although the test I did is leaving little room for any other conclusion, its also not 100% foolproof. And to boot its was only done on one windows version.)

... Bummer! ...

[Zedna]

@Yashied,Kafu

Thanks for the tips.

I will play with it maybe later.

Edited May 7, 2012 by Zedna

[Zedna]

nice!! i search for secure deleting method but i didn't find it in Autoit.. Thanks for Zedna . so, file will be null before deleting and we can't recover this file again, right??

Yes.

[wraithdu]

I wasn't going to pipe in, but here's my older implementation:

I have a bunch of unpublished updates that use the work I did with the windows defrag api that can be used to really test this. That code can pull raw data directly from the disk sectors to verify overwriting of the file. I also use some neat tricks with the defrag api to overwrite EFS encrypted and OS compressed and sparse files. The thing with those is using CreateFile or mapped files does not overwrite the physical disk where the files actually reside. The new data is simply written somewhere else and the old data marked deallocated and lost to posterity.

At some point soon I hope to finalize what I've done and post an update. I really had meant to go through the SDelete source code to compare to my implementation... maybe I'll get to that as well.

[Zedna]

@wraithdu

Great!! I'm amazed.

[wraithdu]

I'll pop these in here for now for some preliminary review. There's still some debugging code in them, and the free space wipe is mostly untested - it's the main reason I wanted to review the SDelete source code. But it works, so any feedback is appreciated. If you have questions on usage, let me know. Check the utility functions in the _FileMapping UDF for functions to print the file map and read it back to a file - used to extract files directly from disk both before and after wiping for verification purposes (you'll need a hex editor for after ).

Using the _FileMapping functions on compressed and EFS encrypted files is actually pretty neat, as you get to see the actual stored compressed/encrypted content which is usually transparent to the user.

_SecureFileDelete.au3

_FileMapping.au3

_FileEx.au3

Edited May 8, 2012 by wraithdu

[michaelslamet]

Hi Zedna,

Is this FileWipe function accept wildcard like *.txt for a paramter?

If no, is there any similiar function that accept wildcard as a paramter?

Thanks :-)

[guinness]

Hi Zedna,

Is this FileWipe function accept wildcard like *.txt for a paramter?

If no, is there any similiar function that accept wildcard as a paramter?

Thanks :-)

Looking at the code I would say no, but why not try it first before asking.

[michaelslamet]

Looking at the code I would say no, but why not try it first before asking.

You're right, sorry, my bad thanks for remind

[nullschritt]

Is there a way to get this to write files over 2gb or ram?

My understanding is that any time you "map" the file, the whole file goes into memory, so any file over the maximum ram size would cause an error?

Is there a way to add data without loading the already existing content into memory?

I think this would be a great utility to use as a disk cleaner (map over all "free" space with empty bytes to perm. erase data)

If there is not a way, I suppose I could use a loop to make multiple 2gb files and delete them, but I think that may take longer.

[BrewManNH]

I think this would be a great utility to use as a disk cleaner (map over all "free" space with empty bytes to perm. erase data)

http://www.dban.org/