Sign in to follow this  
Followers 0
NDog

solved - Hash of an Entire Folder Structure

11 posts in this topic

#1 ·  Posted (edited)

I want to point to a folder and generate hash of all contents.

I'm familiar with using which works very well with single files... a fallback option I have is to take a zip archive the aforementioned folder and then hash the archive... but I'm hoping to not have to do that.

Kind regards

Edited by NDog

Share this post


Link to post
Share on other sites



I think you can use FileFindFirstFile & FileFindFirstFile to walk across your folder and going using MD5 from

Share this post


Link to post
Share on other sites

Thanks for the suggestion, however I want to get a single hash of the folder and its contents, not individual file hashes .

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

Then you'll have to do something like this:

#include <Crypt.au3>
_Crypt_Startup()
; Assuming $aAListOfFilesInAFolder is an array of file names.
For $i = 1 To $aAListOfFilesInAFolder[0]
    $sFileContent = FileRead( $aAListOfFilesInAFolder[$i])
    ; Third parameter should be True for the last iteration.
    $sHash = _Crypt_HashData($sFileContent, $CALG_MD5, False, $sHash)
Next
_Crypt_Shutdown()
Edited by dany

[center]Spiderskank Spiderskank[/center]GetOpt Parse command line options UDF | AU3Text Program internationalization UDF | Identicon visual hash UDF

Share this post


Link to post
Share on other sites

Solved. Thanks for your suggestions danny and Danyfirex

- md5 enumerator

- _RecFileListToArray

#include <RecFileListToArray.au3>
#include <md5.au3>

$dir = FileSelectFolder("Select Folder to Traverse","",4)
$FilesList = _RecFileListToArray($dir, "*", 1, 1, 1, 2)

Dim $BufferSize = 0x20000, $Timer = TimerInit()

$MD5CTX = _MD5Init() ; start reading input into the buffer
For $i = 1 To $FilesList[0]
    $Filename = $FilesList[$i]
    $FileHandle = FileOpen($FilesList[$i], 16)

    For $j = 1 To Ceiling(FileGetSize($Filename) / $BufferSize)
        _MD5Input($MD5CTX, FileRead($FileHandle, $BufferSize)) ; add input into the buffer
    Next

    FileClose($FileHandle)
Next
$Hash = _MD5Result($MD5CTX) ; finish reading input into the buffer

ConsoleWrite("MD5 Result: "& $Hash & " in " & Round(TimerDiff($Timer)) & " ms"&@CRLF)

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

Ndog,

To save on time with large files maybe only hash a small percentage of the file. Here is an idea >>

#include <File.au3>
#include <Crypt.au3>

_Crypt_Startup()

Example()

_Crypt_Shutdown()

Func Example()
    Local $aArray = _FileListToArray(@ScriptDir, '*', 1), $sHash = ''
    For $i = 1 To $aArray[0]
        $sHash = _MD5Hash($aArray[$i], 25) ; Read only 25% of the file.
        ConsoleWrite($aArray[$i] & ' :: ' & $sHash & @CRLF)
    Next
EndFunc   ;==>Example

; By guinness - 2012.
Func _MD5Hash($sFilePath, $iRead = Default) ; Default is 100%
    If ($iRead > 100) Or ($iRead < 0) Or $iRead = Default Then
        $iRead = 100
    EndIf
    $iRead = ($iRead / 100) * FileGetSize($sFilePath)
    Return _Crypt_HashData(FileRead($sFilePath, $iRead), $CALG_MD5)
EndFunc   ;==>_MD5Hash

OR look at trancexx's MD5 UDF too, in particular this entry >>

Edited by guinness

_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 04/09/2015

Share this post


Link to post
Share on other sites

Thanks for the example Guinness.

Sorry if I sound daft but how could I concatenate multiple md5 hash into one md5 hash? With my example I am feeding multiple values into the buffer, but if there is a way to have an array of multiple md5 hashes, how could I take that array and make it into a single md5 hash?

Share this post


Link to post
Share on other sites

Using my example or yours? As I can only offer advice with mine at this time.


_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 04/09/2015

Share this post


Link to post
Share on other sites

Perhaps you can do something like this ---> dir c:windows /s > c:list.txt

Or use a _FileListToArray function like the one guinness posted.

And then hash the list.txt file?


"The mediocre teacher tells. The Good teacher explains. The superior teacher demonstrates. The great teacher inspires." -William Arthur Ward

Share this post


Link to post
Share on other sites

Perhaps you can do something like this ---> dir c:windows /s > c:list.txt

Or use a _FileListToArray function like the one guinness posted.

And then hash the list.txt file?

I can only assume logic here from the OP.

Knowing if a file was added or removed is not necessarily what they're looking for.

Knowing if there has been "any change" to the structure ( file added, removed, or data inside the file changed in anyway ( not just byte size ) ) would be what they're after.

Hashing File names, Directory names, and File contents, on a concatenation process is the only thing that makes sense to me for this to work.


Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

SmOke_N,

True, but... what if list.txt contained all the file hashes?

A single hash can then be made on list.txt.

If there is a difference between checks, then finding the difference between the text files with a compare function should produce the desired results on which file changed.

I haven't played with it enough to know what would be the most efficient in this regard, but it's a place to start, thats all I was trying to convey.

-edited forum formatting-

Edited by ripdad

"The mediocre teacher tells. The Good teacher explains. The superior teacher demonstrates. The great teacher inspires." -William Arthur Ward

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0