Jump to content

FileWipe - very fast (using memory mapped files + memset)

Recommended Posts

This function rewrite whole content of given file by defined (implicitly 0x0) character and finally delete it, it's called as secure deleting.

This should be VERY fast!!

Just note that there is no error checking inside FileWipe()

Maybe I will post later also version with error checking.

You can comment line FileDelete($sFileName) to see how it's rewritten before deletion.

EDIT: There can be problems with too big files (>2GB or > amount of RAM) see post #8 by Kafu

#include <WinAPI.au3>

; prepare testing file


Func FileWipe($sFileName, $nByte = 0x0)
    If Not FileExists($sFileName) Then Return
    $iSize = FileGetSize($sFileName)
    $hFile = _WinAPI_CreateFile($sFileName, 2, 6)
    $hMapping = _WinAPI_CreateFileMapping($hFile)
    $pAddress = _WinAPI_MapViewOfFile($hMapping)
    MemSet($pAddress, $nByte, $iSize)

Func MemSet($pDest, $nChar, $nCount)
    DllCall("msvcrt.dll", "ptr:cdecl", "memset", "ptr", $pDest, "int", $nChar, "int", $nCount)
    If @error Then Return SetError(1,0,False)
    Return True

; from WinAPIEx - just simplified for this purpose

Func _WinAPI_CreateFileMapping($hFile)
    Local $Ret = DllCall('kernel32.dll', 'ptr', 'CreateFileMappingW', 'ptr', $hFile, 'ptr', 0, 'dword', 0x4, 'dword', 0, 'dword', 0, 'ptr', 0)
    If (@error) Or (Not $Ret[0]) Then Return SetError(1, 0, 0)
    Return $Ret[0]

Func _WinAPI_MapViewOfFile($hMapping)
    Local $Ret = DllCall('kernel32.dll', 'ptr', 'MapViewOfFile', 'ptr', $hMapping, 'dword', 0x6, 'dword', 0, 'dword', 0, 'dword', 0)
    If (@error) Or (Not $Ret[0]) Then Return SetError(1, 0, 0)
    Return $Ret[0]

Func _WinAPI_UnmapViewOfFile($pAddress)
    DllCall('kernel32.dll', 'int', 'UnmapViewOfFile', 'ptr', $pAddress)
    If @error Then Return SetError(1, 0, 0)
    Return 1

Here is my first test version without memory mapped files:

Func FileWipe($sFileName, $nByte = 0x0)
    If Not FileExists($sFileName) Then Return
    $iSize = FileGetSize($sFileName)
    $tBuffer = DLLStructCreate("byte[" & $iSize & "]")
    MemSet(DLLStructGetPtr($tBuffer), $nByte, $iSize)
    $hFile = _WinAPI_CreateFile($sFileName, 2, 6)
    _WinAPI_WriteFile($hFile, DLLStructGetPtr($tBuffer), $iSize, $iSize)
Edited by Zedna
Link to post
Share on other sites

I could swear I read somewhere that overwriting a file with Null characters does absolutely nothing, but I can't find anything to support that statement. I did find this though. http://www.planetsourcecode.com/vb/scripts/ShowCode.asp?txtCodeId=29245

1) absolut nonsense :-)

There are several wipe methods, some of them rewrite file several times with different and/or random data, this my method is basic one.

You can easily accomodate it to your needs.

2) your link is obsolete, http://www.planetsourcecode.com/vb/scrip...ts/ShowCode.asp?txtCodeId=2924 --> anyway this code does nothing as I see in first quick look

Please post only if you know something about it. Thanks.


some links



Edited by Zedna
Link to post
Share on other sites

? ... Skeptic here! ... Its this code writing faster to disk than AutoIt itself will do?

(That is at least what it sounds like to me.)

"Straight_and_Crooked_Thinking" : A "classic guide to ferreting out untruths, half-truths, and other distortions of facts in political and social discussions."
"The Secrets of Quantum Physics" : New and excellent 2 part documentary on Quantum Physics by Jim Al-Khalili. (Dec 2014)

"Believing what you know ain't so" ...

Knock Knock ...

Link to post
Share on other sites


thanks for you code, very useful.

In the code without mapped files, there is a little mistake in the code :

$hFile = _WinAPI_CreateFile($sFileName, 2, 6)

What do you think about my code :

Func _FileWipeSecure($sFileName, $times = 4)
  Local $iSize, $tBuffer, $hFile   
  Local $nByte[3] = [1,0x0,0x1]
  if $times < 1 then $times = 1
  If Not FileExists($sFileName) Then SetError(1,0,0)
    $iSize = FileGetSize($sFileName)
    $tBuffer = DLLStructCreate("byte[" & $iSize & "]")
For $i = 1 to $times
  MemSet(DLLStructGetPtr($tBuffer), $nByte[$nByte[0]], $iSize)
  $hFile = _WinAPI_CreateFile($sFileName, 2, 6)
  _WinAPI_WriteFile($hFile, DLLStructGetPtr($tBuffer), $iSize, $iSize)
  if $nByte[0] = 1 Then
   $nByte[0] += 1
   $nByte[0] = 1
    Return FileDelete($sFileName)
Edited by ricky03
Link to post
Share on other sites

Hi Zedna,

I really like this one ;), but some points came to my mind. Better use _Winapi_FileGetSizeEx to pass the 2 GB limit. Also I would recommend to process larger files in a loop, as the files are mapped to the process address space and thus large files are likly to consume too much RAM (up to crashing the script). My post on " should contain all required details for that.


Edited by KaFu
Link to post
Share on other sites

Other version. (although for all I know it might actually NOT overwrite the original file data. ... Disk cluster and such.)

Func WipeFile_Zeros($sFilespec)
    If FileExists($sFilespec) Then
        Local Const $FILE_EREASE = 2
        Local Const $FILE_BEGIN = 0 ;; <constants.au3>
        Local $iLenght = FileGetSize($sFilespec)
        Local $hFile = FileOpen($sFilespec, $FILE_EREASE) ;; Binary(+16) don't seems to matter.)
        FileSetPos($hFile, $iLenght - 1, $FILE_BEGIN)
        FileWrite($hFile, Chr(0))

Err ... think I just got scooped.

Whats wrong with this code. ? (... fixed)


Note to self: forum (update) message might be a little older than you think.</constants.au3>

Edited by MvGulik

"Straight_and_Crooked_Thinking" : A "classic guide to ferreting out untruths, half-truths, and other distortions of facts in political and social discussions."
"The Secrets of Quantum Physics" : New and excellent 2 part documentary on Quantum Physics by Jim Al-Khalili. (Dec 2014)

"Believing what you know ain't so" ...

Knock Knock ...

Link to post
Share on other sites

(although for all I know it might actually NOT overwrite the original file data. ... Disk cluster and such.)

After a little testing. This seems to be exactly the case. ('seems': as in, although the test I did is leaving little room for any other conclusion, its also not 100% foolproof. And to boot its was only done on one windows version.)

... Bummer! ...

"Straight_and_Crooked_Thinking" : A "classic guide to ferreting out untruths, half-truths, and other distortions of facts in political and social discussions."
"The Secrets of Quantum Physics" : New and excellent 2 part documentary on Quantum Physics by Jim Al-Khalili. (Dec 2014)

"Believing what you know ain't so" ...

Knock Knock ...

Link to post
Share on other sites

I wasn't going to pipe in, but here's my older implementation:

I have a bunch of unpublished updates that use the work I did with the windows defrag api that can be used to really test this. That code can pull raw data directly from the disk sectors to verify overwriting of the file. I also use some neat tricks with the defrag api to overwrite EFS encrypted and OS compressed and sparse files. The thing with those is using CreateFile or mapped files does not overwrite the physical disk where the files actually reside. The new data is simply written somewhere else and the old data marked deallocated and lost to posterity.

At some point soon I hope to finalize what I've done and post an update. I really had meant to go through the SDelete source code to compare to my implementation... maybe I'll get to that as well.

Link to post
Share on other sites

I'll pop these in here for now for some preliminary review. There's still some debugging code in them, and the free space wipe is mostly untested - it's the main reason I wanted to review the SDelete source code. But it works, so any feedback is appreciated. If you have questions on usage, let me know. Check the utility functions in the _FileMapping UDF for functions to print the file map and read it back to a file - used to extract files directly from disk both before and after wiping for verification purposes (you'll need a hex editor for after ;) ).

Using the _FileMapping functions on compressed and EFS encrypted files is actually pretty neat, as you get to see the actual stored compressed/encrypted content which is usually transparent to the user.




Edited by wraithdu
Link to post
Share on other sites
  • 4 months later...

Hi Zedna,

Is this FileWipe function accept wildcard like *.txt for a paramter?

If no, is there any similiar function that accept wildcard as a paramter?

Thanks :-)

Looking at the code I would say no, but why not try it first before asking.

UDF List:

_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Link to post
Share on other sites
  • 1 month later...

Is there a way to get this to write files over 2gb or ram?

My understanding is that any time you "map" the file, the whole file goes into memory, so any file over the maximum ram size would cause an error?

Is there a way to add data without loading the already existing content into memory?

I think this would be a great utility to use as a disk cleaner (map over all "free" space with empty bytes to perm. erase data)

If there is not a way, I suppose I could use a loop to make multiple 2gb files and delete them, but I think that may take longer.

Link to post
Share on other sites

I think this would be a great utility to use as a disk cleaner (map over all "free" space with empty bytes to perm. erase data)


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By AutoitMike
      A process of mine opens a HML file, modifies it and then saves, closes it.
      I then need to run an.exe that opens this file right after this.. However, nothing that I find on this forum works that supposedly tests to see if the fie is still open, ready to be used by another process.
      Right now, I just Sleep(2000) .
      I would like something cleaner than this.
      This function, for example, returns False when nothing has it opened or if I open the file with notepad or any other editor.
      I am running on a win7 laptop, if that has any bearing.
      $File="C:\Utilities\test\report.hr5" MsgBox(0,"", _FileInUse($File)) Exit Func _FileInUse($sFilePath) ; By Nessie. Modified by guinness. Local Const $hFileOpen = _WinAPI_CreateFile($sFilePath, $CREATE_ALWAYS, $FILE_SHARE_WRITE) If $hFileOpen Then _WinAPI_CloseHandle($hFileOpen) Return False EndIf Local $fReturn = False If _WinAPI_GetLastError() = 32 Then $fReturn = True Return $fReturn EndFunc  
    • By lIlIIlIllIIIIlI
      $input = $CmdLine[1] $bytes = 1000000 ; 1000000 for 1 MB, 1000 for 1 KB $size = FileGetSize($input) $file = fileopen($input, 16) $max = ceiling($size / $bytes) for $i = 1 to $max $data = fileread($file, $bytes) $output = $input & '_' & $i & 'of' & $max filewrite($output, $data) next ^file split
      file path for a 20MB "video.mp4" is input, it will be output as "video.mp4_1of20", "video.mp4_2of20", etc. change $bytes to affect the split files size, this example made them 1MB (10^6 bytes)
      $input = $CmdLine[1] $name = stringtrimright($input, 1 + stringlen($input) - stringinstr($input, '_', 0, -1)) $split = stringsplit($input, 'of', 3) $max = $split[ubound($split) - 1] for $i = 1 to $max $in = $name & '_' & $i & 'of' & $max $file = fileopen($in, 16) $data = fileread($file) fileclose($file) filedelete($in) filewrite($name, $data) next ^ file join
      file path for any of the "video.mp4_Xof20" segments is input. the "video.mp4_Xof20" segments are read and written to "video.mp4" and then deleted, leaving the newly joined "video.mp4" in the end
      i made this today because i had a big file that i couldn't really open. i split it into 1000 chunks to make processing and stuff easier. i've also used it to split files to just under 25MB segments so i can attach and mail big files over gmail which is silly but it worked
    • By TheSaint
      I like and have been using TeraCopy, a third party program, for many years. Mostly it is a great program, but it does have some issues. On Windows XP for instance, Thumbs.db files could often hold up a copy or move process until the user manually responded to the error prompt. Some other issues I discuss below.
      PLEASE NOTE - I am not related to or affiliated in any way with the 3rd party TeraCopy program developers.
      ALSO NOTE - I myself have only tested TeraCopy Cure at this point, on Windows 7 (32 bit), and only with the free version of TeraCopy 2.27.
      This program, TeraCopy Cure, is related to another one of my TeraCopy assistant programs, TeraCopy Timer, but aims at being simpler and quicker to use ... if lacking its more advanced features.
      TeraCopy Cure is a frontend for TeraCopy and sets out to make up for its flaws and limitations.
      One of those flaws is queuing order, and the limitation relates to a same destination issue.
      You would think that queue order would be the same as add order, but that is not the case, and if you are doing a mix of COPY and MOVE then the COPY process could easily fail. It could fail with some source items, even if the  COPY process has already started before the same source MOVE process begins ... especially if the destination folder is on the source drive ... only the currently copying file is locked to that process.
      If you drag and drop another source for a same destination as an existing or impending COPY or MOVE process, then usually TeraCopy nicely adds it for you to that existing job. However, you might wish to avoid that, or it might occur during the testing phase of that active process, and then not be properly processed etc. But you are not given a choice and it just gets added.
      HOW TO USE
      See the right-click menu of the 'Batch List' (lowest field) and the right-click menu of the Tree field, for some useful options.
      (1) If desired, enable 'Auto Start'. NOTE - Even if enabled, this can be bypassed.
      (2) Set the destination folder, either by browsing on the tree or by dragging a folder to the Destination input field or label. Right-clicking on a folder in Explorer will also work, if enabled. The destination folder path will also show in the Tree if that right-click option is enabled.
      (3) Then drag & drop source file or folder onto one of three five areas - Folder (Drive) Tree field, or COPY or MOVE buttons ... this now also includes the source input and label.
      NOTE - If the Folder (Drive) Tree field is used, then you will additionally need to click either the COPY or MOVE button, to have that job added to the Batch List ... but this avoids 'Auto Start' if it is enabled. Drag & dropping to the buttons instead, saves on clicking, but starts the first job etc immediately if 'Auto Start' is enabled.
      (4) If needed and ready, click the START button to run the first job and those that follow.
      More information is included in program and in the NOTES etc sections below.
      WARNING - Depending on the amount of content on your destination drive, and folder level depth, and the speed of your PC, display of the full path in the Folder (Drive) Tree can take a while to show ... if you have that (right-click) option enabled ... it isn't by default. I found this feature quite tricky to get right, and I'm still not 100% sure it is now full-proof.
      TeraCopy Cure v1.5.zip  (source is included)
      TeraCopy Cure v1.6.zip  (source is included)
    • By Zedna
      #include <WinAPI.au3> $text = FileReadLastChars("C:\Program Files\AutoIt3\Include\Array.au3", 1024) MsgBox(0, 'FileReadLastChars', $text) Func FileReadLastChars($sFile, $nChars)     Local $nBytes     $tBuffer = DLLStructCreate("char["&$nChars&"]")     $hFile = _WinAPI_CreateFile($sFile, 2, 2) ; open for read     _WinAPI_SetFilePointer($hFile, -1 * $nChars, 2) ; from end     _WinAPI_ReadFile($hFile, DLLStructGetPtr($tBuffer), $nChars, $nBytes)     _WinAPI_CloseHandle($hFile)     Return DLLStructGetData($tBuffer, 1) EndFunc ; included as standard UDF since AutoIt version Func _WinAPI_SetFilePointer($hFile, $iPos, $iMethod = 0)     $aResult = DllCall( "kernel32.dll", "long", "SetFilePointer", "hwnd", $hFile, "long", $iPos, "long_ptr", 0, "long", $iMethod)     If @error Then Return SetError(1, 0, -1)     If $aResult[0] = -1 Then Return SetError(2, 0, -1) ; $INVALID_SET_FILE_POINTER = -1     Return $aResult[0] EndFunc ;==>_WinAPI_SetFilePointer Here is my topic about _WinAPI_SetFilePointer()
      EDIT: simpler version compatible with latest AutoIt
      $text = FileReadLastChars("C:\Program Files\AutoIt3\Include\Array.au3", 1024) MsgBox(0, 'FileReadLastChars', $text) Func FileReadLastChars($sFile , $nChars) $hFile = FileOpen($sFile, 0) ; open for read FileSetPos($hFile, -1 * $nChars, 2) ; from end $sRet = FileRead($hFile) FileClose($hFile) Return $sRet EndFunc  
    • By nacerbaaziz
      goodmorning autoit team
      today am comming with some winhttp problems, i hope that you can help me to solve them.
      the first problem
      is when opening a request
      my forums api allow me to delete any post using the api key
      all functions work, i mean post / get
      but when i tried to use the delete verb it's gave me an html 404 error
      here is what am tried
      #include "WinHttp.au3" ; Open needed handles Global $hOpen = _WinHttpOpen() Global $hConnect = _WinHttpConnect($hOpen, "xxxxxxxx.com") ; Specify the reguest: Global $hRequest = _WinHttpOpenRequest($hConnect, "Delete", "/vb/Api/posts/10447/?hard_delete=true", default, default) _WinHttpAddRequestHeaders($hRequest, "XF-Api-Key:xxxxx") _WinHttpAddRequestHeaders($hRequest, "XF-Api-User:xxxxx") ; Send request _WinHttpSendRequest($hRequest) ; Wait for the response _WinHttpReceiveResponse($hRequest) Global $sHeader = 0, $sReturned = 0 ; If there is data available... If _WinHttpQueryDataAvailable($hRequest) Then $sHeader = _WinHttpQueryHeaders($hRequest, $WINHTTP_QUERY_CONTENT_DISPOSITION) ;Or maybe: ; $sHeader = _WinHttpQueryHeaders($hRequest, BitOR($WINHTTP_QUERY_RAW_HEADERS_CRLF, $WINHTTP_QUERY_CUSTOM), "Content-Disposition") Do $sReturned &= _WinHttpReadData($hRequest) Until @error msgBox(64, "", $sReturned) endIf ; Close handles _WinHttpCloseHandle($hRequest) _WinHttpCloseHandle($hConnect) _WinHttpCloseHandle($hOpen)  
      and here is the error message
      <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>403 Forbidden</title> </head><body> <h1>Forbidden</h1> <p>You don't have permission to access /vb/Api/posts/10447/ on this server.<br /> </p> </body></html>  
      i hope you can help me 
      thanks in advance
  • Create New...