Jump to content

Duplicate Files Finder - MD5 Hash CheckSum


ptrex
 Share

Recommended Posts

Duplicate Files Finder - MD5 Hash CheckSum

Someone in the support forum was pointing to the Xstandard MD5 Hash COM object.

For those of you who don't know what a MD5 Hash CheckSum is :

In cryptography, MD5 (Message-Digest algorithm 5) is a widely used cryptographic hash function with a 128-bit hash value. As an Internet standard (RFC 1321), MD5 has been employed in a wide variety of security applications, and is also commonly used to check the integrity of files. An MD5 hash is typically expressed as a 32-character hexadecimal number.

MD5 was designed by Ron Rivest in 1991 to replace an earlier hash function, MD4. In 1996, a flaw was found with the design of MD5[citation needed]; while it was not a clearly fatal weakness, cryptographers began recommending the use of other algorithms, such as SHA-1 (which has since been found vulnerable itself). In 2004, more serious flaws were discovered making further use of the algorithm for security purposes questionable.

From Wikipedia, the free encyclopedia

Well this would be an example on how to use the CheckSum to find duplicated files or Scripts on your system :

Download here XStandard - MD5 Com Object

This small application is my first using the new syntax of 3.2.9.10.

It contains most of the extensive functionality that AU3 has to offer :

- DLLCalls

- SQL

- In Memory Database

- COM object

- RegisterMsg

- Native AU3 functions

- ...

post-6305-1198791790_thumb.jpg post-6305-1198791800_thumb.jpg

DuplicateFiles_MD5_SQL_New.au3

Au3 - 3.2.13.13 version and higher

DuplicateFiles_MD5_SQL_New.au3

The speed of the application is more than reasonable.

It even scans your server if needed.

Enjoy finding duplicate scripts, audio, zip, exe, ... files

Enjoy !!

regards,

ptrex

Edited by ptrex
Link to comment
Share on other sites

@speedy6

This function seems to be removed in the newer versions of AU3.

You can just comment out these lines.

I will see of there is function that replaces this.

This is the one according to the new naming conventions.

_GUICtrlListView_SetColumnWidth

Thanks

regards

ptrex

Edited by ptrex
Link to comment
Share on other sites

  • 4 weeks later...
  • 2 weeks later...

ptrex... I just found this, Very NICE!! I downloaded the XMD5.dll and added these lines to your IsObj($oMD5) to autosetup the dll.. Thanks so much for sharing..

If Not IsObj($oMD5) Then
FileInstall(@ScriptDir&"\x-md5\XMD5.dll", @SystemDir & "\XMD5.dll", 0)
RunWait(@ComSpec & " /c regsvr32 /s XMD5.dll", @SystemDir, @SW_HIDE)
MsgBox(48,"Error","You don't have the MD5 COM object installed on !! We are installing it now.")
 ;Exit
EndIf
Link to comment
Share on other sites

@gesller

Thanks for the compliment and additions to the script.

In the meantime I have complete a version, that has no dependency on an EXTERNAL COM DLL.

It has all the hash functionaluty included in AU3.

Runs really as fast, if not faster than the previous version.

I run it to scan my networks servers (remotely). And I am amazed to see how many GB on duplicate data is around.

But unfortunately it is not open source. :D

Regards,

ptrex

Link to comment
Share on other sites

Hi,

I get this:

C:\Downloads\AutoIt-Skripte\Entwicklung\ForumTests\MD5Duplicates.au3 (244) : ==> Error in expression.:

$oDict1_key = $oMD5.GetCheckSumFromFile($aFilesList[$n])

$oDict1_key = ^ ERROR

You also have to check your includes. My script says you needn't Array.au3 and String.au3.

So long,

Mega

Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Link to comment
Share on other sites

@Xenobiologist

I don't know what kind of errors you get ?

On my machine Running:(3.2.9.14) (and others on which I tested it) it runs like a charm.

2. conditions

- You should use the version Running:(3.2.9.10) and higher.

- You should have the MD5 Object registered on your machine to run the Application.

Regards

ptrex

Link to comment
Share on other sites

  • 3 months later...

@Polyphem

You need to have the SQLite DLL avaiable on the machine that runs the code.

Best way is to have the DLL somewhere in the Windows root folder.

This will solve your problem.

@JamesB

I guess ptrex here likes SQL

Indead !!

Since almost anything is accessible through SQL these day.

And it won't be long before the OS will support native SQL.

So you better get used to it. :D

regards,

ptrex

Link to comment
Share on other sites

You need to have the SQLite DLL avaiable on the machine that runs the code.

Hmmmm, thinking about all the SQLite function calls in the script... I just can't explain why I have though about that :)... guess I assumed that SQLite is already part of AutoIt's pre-installed UDF's :D.... thanks for your quick response :D ...

Cheers

Poly

This post will be edited again by Polyphem: Tomorrow, 11:55 AM
Link to comment
Share on other sites

Hmmmm, sorry to bother again :D.... installed md5 and sqllite... now the program works fine for me... except that there seems to be a problem with the passing of the values from one listview to the other (see screen attached)... I'll try to dig into the code myself...

post-11346-1209403596_thumb.jpg

Cheers

UPDATE:

Ups, just took me 20 seconds :D.... replace all occurrences of

_GUICtrlListView_DeleteAllItems ($ListView1)

with

_GUICtrlListView_DeleteAllItems (GUICtrlGetHandle($ListView1))

and

_GUICtrlListView_DeleteAllItems ($ListView2)

with

_GUICtrlListView_DeleteAllItems (GUICtrlGetHandle($ListView2))

and

_GUICtrlListView_DeleteItem($ListView2,$Index)

with

_GUICtrlListView_DeleteItem(GUICtrlGetHandle($ListView2),$Index)

Thats it

Btw, did I already say thank you for this great piece of code? Thank you :)...

Cheers

Edited by Polyphem
This post will be edited again by Polyphem: Tomorrow, 11:55 AM
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...