Jump to content

Duplicate file script

MattHiggs

By MattHiggs,
in AutoIt Example Scripts

 Share

Recommended Posts

MattHiggs Universalist

MattHiggs

Posted
Posted (edited)

Hello auto it community.  I wrote a script in order assist me with cleaning up some of my duplicate files.  I host a file server in my home environment, on which...lets just say I'm a "hoarder" of software I think might be useful, and I sometimes mistakenly download the same software I have already downloaded.  So, I wrote this little utility to accept two directories as input, destination (folder you want to merge into) and source (folder you ultimately want removed after all relevant files are moved to destination).  It will then scan both directories, then loop through the source directory to see if a particular file is present in the destination directory (the relevant paths in each directory will have to match as well, which is done on purpose for scenarios like a directory with extracted ISO contents).  If the file is not present, it, along with its relative directory structure, is moved to the destination directory.  If it is present, the tool uses (script downloads and extracts to app data directory at beginning) Microsoft File Checksum Integrity Verifier tool to check and see if the two files are the same (have the same hash value).  If they are the same, the tool will automatically determine which file is the most recent, and will perform whatever action is required to make it so that the destination folder has the most recent version of the file.  If the hash values are not the same, the tool will then prompt user on what they would like done, user inputs answer into console, and action is performed.  Had a lot of fun with this one.  Enjoy!  **Edit  Sorry.   Forgot to mention you would need the attached UDF to compile.  My bad. 

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_UseX64=y
#AutoIt3Wrapper_Res_SaveSource=y
#AutoIt3Wrapper_Res_Language=1033
#AutoIt3Wrapper_Res_requestedExecutionLevel=highestAvailable
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****
; *** Start added by AutoIt3Wrapper ***
#include <FileConstants.au3>
; *** End added by AutoIt3Wrapper ***
#cs ----------------------------------------------------------------------------

 AutoIt Version: 3.3.15.0 (Beta)
 Author:         myName

 Script Function:
    Template AutoIt script.

#ce ----------------------------------------------------------------------------

; Script Start - Add your code below here
#include <File.au3>
#include <Array.au3>
#include <Date.au3>
#include <Console.au3>
#include <Constants.au3>
$checksumpath = @AppDataDir & "\checksumverify"
If Not FileExists ( $checksumpath & "\fciv.exe" ) Then
    If Not FileExists ( $checksumpath ) Then
        DirCreate ( $checksumpath )
    EndIf
    $URL = InetGet ( "https://download.microsoft.com/download/c/f/4/cf454ae0-a4bb-4123-8333-a1b6737712f7/Windows-KB841290-x86-ENU.exe", @AppDataDir & "\checksumpackage.exe", 3 )
    RunWait ( @ComSpec & ' /c checksumpackage.exe /Q /T:"' & $checksumpath & '" /C', @AppDataDir, @SW_HIDE )
    InetClose ( $URL )
EndIf

Cout ("")
$a = FileSelectFolder ( "Select destination (the folder you want to merge the files into)", "" )
$b = FileSelectFolder ( "Select source (the folder you want to remove files from and delete)", "" )
$dest = _FileListToArrayRec ( $a, "*", $FLTA_FILESFOLDERS, $FLTAR_RECUR )
$source = _FileListToArrayRec ( $b, "*", $FLTA_FILESFOLDERS, $FLTAR_RECUR )
For $i = 1 To $source[0] Step 1
    $index = _ArraySearch ( $dest, $source[$i] )
    If @error Then
        SetError (0)
        Cout ( "Copying " & $source[$i] & @CRLF )
        If IsFile ( $b & "\" & $source[$i] ) = 0 Then
            DirMove ( $b & "\" & $source[$i], $a & "\" & $source[$i], 1 )
        Else
            FileMove ( $b & "\" & $source[$i], $a & "\" & $source[$i], 9 )
        EndIf
    Else
        If IsFile ( $b & "\" & $source[$i] ) = 0 Then
            ContinueLoop
        Else
            $hash = Run ( @ComSpec & " /c fciv -add " & '"' & $a & "\" & $dest[$index] & '"' & " -sha1", $checksumpath, @SW_HIDE, $STDOUT_CHILD )
            ProcessWaitClose ( $hash )
            $hash2 = Run ( @ComSpec & " /c fciv -add " & '"' & $b & "\" & $source[$i] & '"' & " -sha1", $checksumpath, @SW_HIDE, $STDOUT_CHILD )
            ProcessWaitClose ( $hash2 )
            $hashstring = StringTrimLeft ( StringTrimRight ( StdoutRead ( $hash ), 2 ), 58 )
            $hashstring2 = StringTrimLeft ( StringTrimRight ( StdoutRead ( $hash2 ), 2 ), 58 )
            $firstsplit = StringSplit ( $hashstring, " " )
            $secondsplit = StringSplit ( $hashstring2, " " )
            $num = 0

            If StringCompare ( $firstsplit[1], $secondsplit[1] ) <> 0 Then
                Do
                    Cout ( "The file to be copied from the source (" & $source[$i] & ") has the same file name as a file in the destination, but the hash values are different.  What do you want to do?" & @CRLF & "[R]ename   [D]elete   [M]ove(Replace)" & @CRLF )
                    $user = Getch ()
                Until $user == "r" Or $user == "R" Or $user == "d" Or $user == "D" Or $user == "m" Or $user == "M"
                Cout (@CRLF)
                Select
                    Case $user == "r" Or $user == "R"
                        $holdstring = ""
                        $filepathstring = StringSplit ( $source[$i], "\" )
                        If @error Then
                            SetError (0)
                            $justfile = $source[$i]
                        Else
                            $justfile = $filepathstring[$filepathstring[0]]
                            If $filepathstring[0] = 2 Then
                                $holdstring = $filepathstring[1]
                            ElseIf $filepathstring[0] = 3 Then
                                $holdstring = $filepathstring[1] & "\" & $filepathstring[2]
                            Else
                                For $hh = 1 To $filepathstring[0] - 1 Step 1
                                    $holdstring = $holdstring & $filepathstring[$hh] & "\"
                                Next
                                $holdstring = StringTrimRight ( $holdstring, 1 )
                            EndIf
                        EndIf


                        $rename = StringSplit ( $justfile, "." )
                        $newname = ""
                        $num = 0
                            Do
                                $num += 1
                                If $rename[0] = 2 Then
                                    $newname = $rename[1] & "(" & $num & ")." & $rename[2]
                                ElseIf $rename[0] = 3 Then
                                    $newname = $rename[1] & "." & $rename[2] & "(" & $num & ")." & $rename[3]
                                Else
                                    For $f = 1 To $rename[0] Step 1
                                        If $f = $rename[0] - 1 Then
                                            $newname = $newname & $rename[$f] & "(" & $num & ")."
                                        Else
                                            $newname = $newname & $rename[$f] & "."
                                        EndIf
                                    Next
                                    $newname = StringTrimRight ( $newname, 1 )
                                EndIf
                                $foldhold = ""
                                If $holdstring == "" Then
                                    $foldhold = $newname
                                Else
                                    $foldhold = $holdstring & "\" & $newname
                                EndIf


                            Until Not FileExists ( $a & "\" & $foldhold )
                            Cout ( "Copying " & $foldhold & @CRLF )
                            Do
                                FileMove ( $b & "\" & $source[$i], $a & "\" & $foldhold, 9 )
                            Until Not FileExists ( $b & "\" & $source[$i] ) And FileExists ( $a & "\" & $foldhold )


                    Case $user == "d" Or $user == "D"
                        Cout ( "Deleting " & $source[$i] & @CRLF )
                        Do
                            FileDelete ( $b & "\" & $source[$i] )
                        Until Not FileExists ( $b & "\" & $source[$i] )
                    Case $user == "m" Or $user == "M"
                        Cout ( "Copying " & $source[$i] & @CRLF )
                        Do
                            FileMove ( $b & "\" & $source[$i], $a & "\" & $source[$i], 9 )
                        Until Not FileExists ( $b & "\" & $source[$i] )
                EndSelect
            Else
                $desttime = _Date_Time_GetFileTime ( $a & "\" & $dest[$index] )
                $sourcetime = _Date_Time_GetFileTime ( $b & "\" & $source[$i] )
                $compare = _Date_Time_CompareFileTime ( $desttime[2], $sourcetime[2] )
                If $compare = 0 Or $compare = 1 Then
                    Cout ( "Deleting " & $source[$i] & @CRLF )
                    Do
                        FileDelete ( $b & "\" & $source[$i] )
                    Until Not FileExists ( $b & "\" & $source[$i] )
                ElseIf $compare = -1 Then
                    Cout ( "Copying " & $source[$i] & @CRLF )
                    Do
                        FileMove ( $b & "\" & $source[$i], $a & "\" & $source[$i], 9 )
                    Until Not FileExists ( $b & "\" & $source[$i] )
                Else
                    Cout ( "????????????????????" & @CRLF )
                EndIf
            EndIf

        EndIf
    EndIf
Next
DirRemove ( $b, $DIR_REMOVE )


Func IsFile($sFilePath)
    Return Number(FileExists($sFilePath) And StringInStr(FileGetAttrib($sFilePath), "D", 2, 1) = 0)
EndFunc   ;==>IsFile

 

Console.au3

Edited by MattHiggs
Link to comment
Share on other sites
  • 4 weeks later...
spudw2k Universalist

spudw2k

Posted
Posted

Looks pretty good.  Just curious, is there a reason you opted for the MS tool instead of the hashing functions in the Crypt UDF?  
 

Spoiler

Things I've Made: Always On Top Tool ◊ AU History ◊ Deck of Cards ◊ HideIt ◊ ICU ◊ Icon Freezer ◊ Ipod Ejector ◊ Junos Configuration Explorer ◊ Link Downloader ◊ MD5 Folder Enumerator ◊ PassGen ◊ Ping Tool ◊ Quick NIC ◊ Read OCR ◊ RemoteIT ◊ SchTasksGui ◊ SpyCam ◊ System Scan Report Tool ◊ System UpTime ◊ Transparency Machine ◊ VMWare ESX Builder
Misc Code Snippets: ADODB Example ◊ CheckHover ◊ Detect SafeMode ◊ DynEnumArray ◊ GetNetStatData ◊ HashArray ◊ IsBetweenDates ◊ Local Admins ◊ Make Choice ◊ Recursive File List ◊ Remove Sizebox Style ◊ Retrieve PNPDeviceID ◊ Retreive SysListView32 Contents ◊ Set IE Homepage ◊ Tickle Expired Password ◊ Transpose Array
Projects: Drive Space Usage GUI ◊ LEDkIT ◊ Plasma_kIt ◊ Scan Engine Builder ◊ SpeeDBurner ◊ SubnetCalc
Cool Stuff: AutoItObject UDF ◊ Extract Icon From Proc ◊ GuiCtrlFontRotate ◊ Hex Edit Funcs ◊ Run binary ◊ Service_UDF

 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...