SMF - The fastest duplicate files finder... [Updated 2023-Jun-03]

[KaFu]

Updated first Post...

[KaFu]

Change log v0.4.8.5.0 > v0.4.8.6.0

General

[*]Updated sqlitedll 3.6.5 to sqlitedll 3.6.7

[*]Changed filename to FileGetShortName() in many instances (better compatibility)

[*]GUI embedded pictures (CC, PayPal) set Size to fixed values (hopefully prevents resizing of pics after minimize-maximize)

[*]md5 calc did throw a fatal error during scan of open files => added _FileInUse()

[*]File erase function did throw a fatal error during scan of open files => added _FileInUse()

Report

[*]Added functions to re-evaluate number of duplicates (Duplicate Occurrence) on deletion / move of duplicate files for Duplicate Report

[*]Report selection produced errors for deletion/copy/move, if Filepath_Long and Filename_Long were NOT the first to fields (doooh)

DupSlider

[*]Minor Design changes

[*]Added functions to re-evaluate number of duplicates (Duplicate Occurrence) on deletion / move of duplicate files for DupSlider

[*]Dupslider did not work correctly, if Filepath_Long and Filename_Long were NOT the first to fields

[*]DubSlider lagging on large gif's => Work-Around implemented, animation of gif's now stopped on display

[*]Added Checkbox to skip none-associated File Extensions from display

[-]Initially also all none-associated File Extensions are displayed (empty Info-Screen)

[-]If they are hidden (by checking the box) AND the the first dup OR the last dup is hit, DupSlider exits (behavior is set to prevent endless-loops)

ToDo's

[*]Update Change log in Help-File

Lines of Source-Code (Main-App + "own" ShellTristateTreeView include, including comments):

5.974 + 571 = 6.545

For the Program Description see

http://www.autoitscript.com/forum/index.php?showtopic=80747

Source and Executable are available at

http://www.funk.eu

Comments and suggestions are warmly welcome...

Wish you all a merry Christmas and a happy new year...

Have Fun and Best Regards

Edited December 22, 2008 by KaFu

[KaFu]

Change log v0.4.8.6.0 > v0.4.8.6.1

General

[*]Bugfix: FileFindFirst() search handle was not closed properly, leaving lots of open file handles

Source and Executable are available at

http://www.funk.eu

Comments and suggestions are warmly welcome...

Best Regards

[ptrex]

@KaFu

I just looked at your last update.

And this is a real nice piece of work !!

I can't add much more to this.

It has lot's of features.

Nice GUI, not too overwhelming.

Good coding style.

Good use of good existing UDF's

...

(Didn't look through all the 6000 lines of source code yet)

The only comment so far is when running to scan for duplicate files.

When the result page is op the CPU stays at 100 % ?

An other enhancement I would make is to embed the Ico, BMP and JPG in the source code.

In order to minimize the files that need to be shipped separately.

But that's a personal preference of course, I don't like other depending files other than the compiled EXE.

[KaFu]

And this is a real nice piece of work !!

I can't add much more to this.

Thanks for this nice feedback m8 ...

The only comment so far is when running to scan for duplicate files.

When the result page is op the CPU stays at 100 % ?

And even more thanks for this observation , didn't notice it until now, but you're right... hmmm, maybe it has something to do with the adlib function i use to readjust the listview column width, will definitely take a deeper look into this.

An other enhancement I would make is to embed the Ico, BMP and JPG in the source code.

In order to minimize the files that need to be shipped separately.

But that's a personal preference of course, I don't like other depending files other than the compiled EXE.

The additional files are embedded in the EXE and shipped with it. I use fileinstall() to @tempdir & "\smf" to extract them, i only provide them in the source for completeness. Or did I get you wrong here and you had something else in mind?

Best Regards

[ptrex]

@Kafu

Or did I get you wrong here and you had something else in mind?

Yes, I think so.

Let me explain. When you use fileinstall in the source. The files included will be extracted to the USER TEMP FOLDER in your case.

The will be put in the -> C:\Documents and Settings\USERNAME\Local Settings\Temp\smf folder to be exact.

When the appl. is shutdown than you can still the leftovers of the files in the folder, which is +3 Mb.

You can imagine that if all AU3 developers start doing this, the disks will be digitally polluted very soon.

In order to avoid that. You can use the method to include the graphics in the GUI without having to use fileinstall.

Look for BinaryImage.au3 from Zedna / Prog@ndy,

on how to use this.

BTW how did you produce the nice helptext file in .MHT format?

Regards,

ptrex

[trancexx]

There is absolutely nothing wrong with FileInstall() and temp folder is created almost just for that purpose and should be used for that and similar stuff.

Just clean after yourself. Like this maybe:

FileInstall()
GUICtrlCreatePic()
FileDelete()

FileInstall() gives a lots of advantages.

Sometimes when you think that you don't use the disk you are using it indirectly thru called functions.

[KaFu]

When the appl. is shutdown than you can still the leftovers of the files in the folder, which is +3 Mb.

You can imagine that if all AU3 developers start doing this, the disks will be digitally polluted very soon.

Look for BinaryImage.au3 from Zedna / Prog@ndy,

Just clean after yourself. Like this maybe:

See your point m8s, will add a DirRemove (@tempdir & "\smf",1) on AutoitExit in the next release ... Had a look at the example you provided, but I don't just fileinstall() images but also the help-file and the latest dll release of sqlite, so i'll stick to fileinstall this time.

BTW how did you produce the nice helptext file in .MHT format?

It's just a plain MS Powerpoint presentation saved as WebPage .mht file displayed in an embedded instance of IE. Had to tweak the file a little with notepad after generation to behave it like i wanted it (e.g. TOC initially hidden).

Best Regards,

[KaFu]

Changelog v0.4.8.6.1 > v0.4.8.6.2

General

• Solved 100% CPU usage during report display; added sleep(20) to while loops in __func_Report_PreSelectColumns(), _func_report() and _func_report_dupslider()
• DirRemove (@tempdir & "\smf\",1) on Exit to clean up installed files
• Ported script from AutoIt version 3.2.12.1 to AutoIt version 3.3.0.0

  [-]Removed custom function _WinAPI_SetFilePointer => from 3.3.0.0 on standard UDF

  [-]UDF mciSendString() renamed to __mciSendString()

  [-]UDF RandomStr() renamed to __RandomStr()

Source and Executable are available at

http://www.funk.eu

Best Regards

[KaFu]

Changelog v0.4.8.6.2 > v0.4.8.6.4

General

• Increased default amount of data analyzed for md5 short from 3*2KB=6KB to 3*16KB=48KB (more reliable, still fast)
• Fixed: Deletion of temp data on exit also deleted the saved Settings. Fixed, settings.ini now excluded from deletion
• Added options to report right click menu

  [-]Added "Open with" dialog box option for selected file (only valid for first file if multiple files are selected)

  [-]Added "Show file properties" dialog selected file (only valid for first file if multiple files are selected)

• Added options for the "Erase files" function under the "Settings" tab

  [-]Option 1: Erase files using 0x00 1-pass method (just read an article that this is totally sufficient on modern HDDs)

  [-]Option 2: Erase files using DoD 3-pass method (random character decimal 0 to 255, complement of first pass, random character)

• Fixed release date from 2008 to 2009

Source and Executable are available at

http://www.funk.eu

Best Regards

Edited January 19, 2009 by KaFu

[iHonda]

mm just relized that SMF is a copyright or trademark or w/e its called of Simple Machines LLC soooo.. yeah.. just lettin u know

[KaFu]

mm just relized that SMF is a copyright or trademark or w/e its called of Simple Machines LLC soooo.. yeah.. just lettin u know

Lol, yeah, searched google and saw that... hmmm, so I clearly point out, that this program this is not "SMF" but "SMF - Search my Files", will search and replace all sole occurences of SMF...

Looked up the trademark database at

http://tess2.uspto.gov/bin/gate.exe?f=tess...tate=kuir3b.1.1

and saw 26 occurences of SMF! Insane...

Best Regards

[KaFu]

Changelog v0.4.8.6.4 > v0.4.8.9.2

General

• Added: A nice Startup Splash screen to show progress of program initialization

• Added: Replaced DirGetSize() with UDF _DirGetSizeEx() to display progress during initial scan of amount of files and folders to analyze and to make this scan interruptible (though a little bit slower then DirGetSize())
• Added: 7-Zip archive creation (based on rasim's 7-Zip UDF)

• Fixed (minor): Display bug in progress window, some field values were reset to 0 instead of NULL
• Fixed (minor): If search is canceled and there are no records in the database, all 'Current Result in Memory Database' related buttons were enabled => now with no records found buttons stay disabled
• Fixed (minor): If filesize filter was checked initially, corresponding input fields were disabled => if checkbox checked initially, input fields are now enabled
• Fixed (medium): Path+Filename length of files analyzed limited to 259 characters. For files violating this rule an '\\?\' is prepended to the path info (let's see if this approach works out as intended).
• Fixed (behavior): Folder-Tree sort order. In last version sort oder was [a-z][special characters], now it is like the normal windows behavior [special characters][a-z]
• Fixed (behavior): File Dates to search for are now only save to ini if <> today (old behavior was to always save first date of ini creation)
• Fixed (behavior): Canceling a duplicate search while calculating md5's prevented an analysis of already obtained md5 results. Now after the cancellation during md5 calculation the already availalbe results are parsed for duplicates. If that takes too long for you too (should be very fast), press cancel again

Report

• Added: Context Menu now colorized with icons (based on Holger's _ModernMenuLib_with_Tray())
• Added: Plugin to IrfanView 4.23+

  [-]Since IrfanView 4.23 the program supports the command-line switch /filelist (by my request ). What SMF does is export the selected filenames to a temp text-file and open up that text-file in IrfanViews thumbnail mode.

  [-]Support custom defined thumbnail views accross different directories. You're able to pre-define the results, e.g. filter by picture dimension (by acquiring Extended file infos first)

  [-]You have to download and install IrfanView 4.23+ first from http://www.irfanview.com, then go to 'Settings' and point SMF to the executable

• Added: HotKey 'CTRL+a' to selecte all entries in Report
• Added: HotKey 'CTRL+t' to toggle selection
• Added: Hyperlink to SQLite tutorial (to research how to tweak your own SQL queries)
• Added: Context Menu Option - Option to export selected file path+name to text-file (Long format)
• Added: Context Menu Option - Option to export selected file path+name to text-file (Short format 8.3)
• Added: Context Menu Option - ZIP Option to create archive from selected files
• Added: Context Menu Option - Open first file Folder
• Added: Context Menu Option - Open Command Prompt in first file's Folder
• Added: Context Menu Option - Selected executable RunAs() (under the context of a different user)
• Added: Button to change displayed columns

• Fixed (minor): Added _GUICtrlListView_SetItemCount() to speed up ListView creation peformance

FileSlider

Fixed (style): Tweaked layout and style a little...

Source and Executable are available at

http://www.funk.eu

Eager for feedback, especially for behavior on Vista installs...

Best Regards

[trancexx]

It's loading too long, dammit! Why is that?

Can you add this somewhere for that spinning gif:

$yourIEobject.document.write('<body onselectstart="return false" oncontextmenu="return false" onclick="return false" ondragstart="return false" ondragover="return false">')

It should stop that ugly current behaviour to navigate to what is dropped on it (thus lose gif) and there would no longer be default IE context menu for that gif.

[KaFu]

It's loading too long, dammit! Why is that?

You mean initially? It's installing (copying to @tempdir & 'smf') all needed help-files on start (sqlite.dll v 3.6.7, 7-zip.dl, help-file) and deleting them on shutdown.

$yourIEobject.document.write('<body onselectstart="return false" oncontextmenu="return false" onclick="return false" ondragstart="return false" ondragover="return false">')

It should stop that ugly current behaviour to navigate to what is dropped on it (thus lose gif) and there would no longer be default IE context menu for that gif.

, didn't even tried this ... will do, thanks for the feedback.

[trancexx]

You mean initially? It's installing (copying to @tempdir & 'smf') all needed help-files on start (sqlite.dll v 3.6.7, 7-zip.dl, help-file) and deleting them on shutdown.

How many files? Like zillion?

It's like starting nero. Don't have nero installed any more. Theorize why.

[KaFu]

It's like starting nero. Don't have nero installed any more. Theorize why.

Really? Startup takes 7 seconds here, and this included start of IE in the background ...

[trancexx]

Really? Startup takes 7 seconds here, and this included start of IE in the background ...

Never under 20 sec for me

[Guest KeeForm]

Thanks for sharing this app and source. I downloaded

http://www.funk.eu/downloads/smf/SMF_v0_4_8_9_2.exe.zip

But when run I the app (on Vista) I get the following error message.

Error:
_GUIImageList_AddIcon: -1

Any idea what could be wrong here?

Thanks,

Dave

[KaFu]

Any idea what could be wrong here?

Thanks for the feedback . Yeah, trancexx already pointed that out to me. I think it's an Icon definition (Desktop Icon) in "SMF_ShellTristateTreeView_v15.au3".

Replace

If FileExists(@SystemDir & "\cdfview.dll") Then
    _GUIImageList_AddIcon($hImage, @SystemDir & "\cdfview.dll", 7); 9 = Desktop
Else
    _GUIImageList_AddIcon($hImage, @SystemDir & "\CRPE32.dll", 1); 9 = Desktop
EndIf

with

_GUIImageList_AddIcon($hImage, @SystemDir & "\shell32.dll", 34)

, that should do the trick (assuming that's the error that pops up ). Stupid me , did this only because I overlooked the Desktop icon in shell32.dll.

Last release was into functionallity (too much?), next release will be more into speed (working on it). I'll skip the installation of all files on every start-up (tmp-files will go into the app-folder), and will try to get rid of IE objects as much as possible.

Cheers

Edited February 16, 2009 by KaFu