File Purger

[romulous]

Hi,

I've been looking for a file purging utility, but haven't been able to find one either in the examples forum, or otherwise existing, so I've decided to try writing my own. Basically, what I want is a utility to run via Windows task scheduler (so no GUI required) that compares a source folder (and sub-folders) to a destination folder (and sub-folders) and deletes any files+folders in the destination that aren't in the source - and which are at least 10 days old. The FileGetTime function looks to be what I need, followed by (probably) FileDelete. However, I can't think of how to get a complete file listing into FileGetTime (it only works on one file at a time and I want it to work on C:\Test\*.*), and can't work out the best way to then pass that whole listing to FileDelete (which also looks at one file at a time). Any help would be appreciated.

Thanks,

CM

[rasim]

Hi! Easy way

[martin]

Hi,

I've been looking for a file purging utility, but haven't been able to find one either in the examples forum, or otherwise existing, so I've decided to try writing my own. Basically, what I want is a utility to run via Windows task scheduler (so no GUI required) that compares a source folder (and sub-folders) to a destination folder (and sub-folders) and deletes any files+folders in the destination that aren't in the source - and which are at least 10 days old. The FileGetTime function looks to be what I need, followed by (probably) FileDelete. However, I can't think of how to get a complete file listing into FileGetTime (it only works on one file at a time and I want it to work on C:\Test\*.*), and can't work out the best way to then pass that whole listing to FileDelete (which also looks at one file at a time). Any help would be appreciated.

Thanks,

CM

For what you are doing I wouldn't worry that the functions only deal with one file at a time, since you only want to delete from one folder files which aren't in another or which are a certain age. The functions you need is FileFindFirstFile and FileFindNextFile.

; Shows the filenames of all files in the current directory.
FileChangeDir($folderB);chnage to the folder
$search = FileFindFirstFile("*.*")  

; Check if the search was successful
If $search = -1 Then
    MsgBox(0, "Error", "No files/directories matched the search pattern")
    Exit
EndIf

While 1
    $file = FileFindNextFile($search) 
    If @error Then ExitLoop
    If Not FileExists($folderA & '\' & $file) Then 
      FileDelete($file)
  Else
    If FileAgeMore($file,10) then FileDelete($file);FileAgeMore to be written
    EndIf
WEnd

; Close the search handle
FileClose($search)

The code above only works for one folder and needs work for subfolders and then you have to be carefule about recursion linits.

[Danny35d]

Also you can use command-line Robocopy "Robust File Copy" which is a folder replication tool.

#include <Date.au3>

$Source = 'C:\SourceFolder'
$Target = 'C:\TargetFolder'

; Julian date of today.
$sJulDate = _DateToDayValue (@YEAR, @MON, @MDAY)

; 10 days ago calculation.
Dim $Y, $M, $D
$sJulDate = _DayValueToDate ($sJulDate - 10, $Y, $M, $D)

RunWait(@ComSpec & ' /c Robocopy.exe "' & $Source & '" "' & $Target & '" *.* /R:3 /W:3 /S /E /ZB /PURGE /MINAGE:' & $Y & $M & $D & ' /TEE /FP /NDL /NS /NP /LOG:C:\PurgeLog.TXT', @ScriptDir)

[romulous]

Hi! Easy way

Hi rasim,

That looks interesting - is it available for download do you know? There doesn't seem to be a download link on that page, and I only run Windows XP, which it doesn't seem to be available natively on.

Regards,

CM

[romulous]

Also you can use command-line Robocopy "Robust File Copy" which is a folder replication tool.

#include <Date.au3>

$Source = 'C:\SourceFolder'
$Target = 'C:\TargetFolder'

; Julian date of today.
$sJulDate = _DateToDayValue (@YEAR, @MON, @MDAY)

; 10 days ago calculation.
Dim $Y, $M, $D
$sJulDate = _DayValueToDate ($sJulDate - 10, $Y, $M, $D)

RunWait(@ComSpec & ' /c Robocopy.exe "' & $Source & '" "' & $Target & '" *.* /R:3 /W:3 /S /E /ZB /PURGE /MINAGE:' & $Y & $M & $D & ' /TEE /FP /NDL /NS /NP /LOG:C:\PurgeLog.TXT', @ScriptDir)

Hi Danny35d,

Actually, I use Robocopy to make the destination folder to begin with. And it does have a /purge switch - but the problem is, the /purge switch does not accept any other modifiers, like the date switch (/minage). So, Robocopy /purge will delete all files in the destination not existing in the source. The Robocopy help does not mention that the date switches do not modify purge, but I don't believe it to be a bug - I think it is a deliberate design decision by Microsoft. I also tried XXCOPY - it's purge switch can be date modified, and thus it can do what I want, but the freeware version of XXCOPY is actually better termed nagware, and the constant prompts to upgrade to the pro version ruin it as an unattended batch tool, which is basically what I need in this case.

Regards,

CM

[romulous]

For what you are doing I wouldn't worry that the functions only deal with one file at a time, since you only want to delete from one folder files which aren't in another or which are a certain age. The functions you need is FileFindFirstFile and FileFindNextFile.

; Shows the filenames of all files in the current directory.
FileChangeDir($folderB);chnage to the folder
$search = FileFindFirstFile("*.*")  

; Check if the search was successful
If $search = -1 Then
    MsgBox(0, "Error", "No files/directories matched the search pattern")
    Exit
EndIf

While 1
    $file = FileFindNextFile($search) 
    If @error Then ExitLoop
    If Not FileExists($folderA & '\' & $file) Then 
      FileDelete($file)
  Else
    If FileAgeMore($file,10) then FileDelete($file);FileAgeMore to be written
    EndIf
WEnd

; Close the search handle
FileClose($search)

The code above only works for one folder and needs work for subfolders and then you have to be carefule about recursion linits.

Thanks martin - I'll use that as a base to start with, and see how I go from there.

Regards,

CM

[Danny35d]

Hi Danny35d,

Actually, I use Robocopy to make the destination folder to begin with. And it does have a /purge switch - but the problem is, the /purge switch does not accept any other modifiers, like the date switch (/minage). So, Robocopy /purge will delete all files in the destination not existing in the source.

Regards,

CM

That is true, English been my second language I miss understood what you really want. You can still use robocopy by adding /L to create a list of all the file needed to delete then use Martin idea FileAgeMore($file,10) by reading the list and deleting any file/folder older than 10 days.

expandcollapse popup

#NoTrayIcon
#include <Date.au3>

Dim $CommnadLine[2]
Global $DeleteFolder
Local $Source = 'SourceFolder'
Local $Target = 'TargetFolder'
Local $CopyLog = EnvGet('SystemDrive') & '\CopyLog.TXT'
Local $PurgeLog = EnvGet('SystemDrive') & '\PurgeLog.TXT'
Global $PurgeSkipLog = EnvGet('SystemDrive') & '\PurgeSkipLog.TXT'

$CommnadLine[0] = '/R:3 /W:3 /S /E /ZB /TEE /FP /NDL /NS /NP /XX /LOG:' & $CopyLog
$CommnadLine[1] = '/R:3 /W:3 /S /E /ZB /PURGE /TEE /NDL /FP /NS /NP /XC /XN /XO /XL /L /NJH /NJS /NC /LOG:' & $PurgeLog

If FileExists($PurgeSkipLog) Then FileDelete($PurgeSkipLog)
For $x = 0 To UBound($CommnadLine) - 1
    RunWait(@ComSpec & ' /c Robocopy.exe "' & $Source & '" "' & $Target & '" *.* ' & $CommnadLine[$x], @ScriptDir, @SW_HIDE)
Next

$PurgeFile = FileOpen($PurgeLog, 0)
If $PurgeFile <> -1 Then
    While 1
        $ReadLine = FileReadLine($PurgeFile)
        If @error = -1 Then ExitLoop
        _FileAgeMore($ReadLine, 10)
    Wend
    $DeleteFolder = StringSplit(StringTrimRight($DeleteFolder, 1), '|')
    For $x = 1 To $DeleteFolder[0]
        DirRemove($DeleteFolder[$x])
    Next
EndIf
FileClose($PurgeFile)
Exit

Func _FileAgeMore($sFullPath, $iAge)
    Dim $Y, $M, $D  
    
    $sFullPath = StringStripWS($sFullPath, 3)       
    If StringInStr(FileGetAttrib($sFullPath), 'D') Then
        $DeleteFolder &= StringTrimRight($sFullPath , 1) & '|'
    Else
        $sSymbol = '/'
        $sTime = FileGetTime($sFullPath, 0, 1)
        $sLastModified = StringMid($sTime,1, 4) & $sSymbol
        $sTime = StringTrimLeft($sTime, 4)
        For $x = 1 To 5
            If $x = 2 Then $sSymbol = ' '
            If $x > 2 Then $sSymbol = ':'
            $sLastModified &= StringMid($sTime, 1, 2) & $sSymbol
            $sTime = StringTrimLeft($sTime, 2)          
        Next
        $sLastModified = StringTrimRight($sLastModified, 1)
        If _DateDiff('d', $sLastModified, _NowCalc()) >= $iAge Then
            FileSetAttrib ($sFullPath, "-RASHNOT")
            FileDelete($sFullPath)
        ElseIf $sFullPath <> '' Then
            FileWriteLine($PurgeSkipLog, $sFullPath)
        EndIf
    EndIf
EndFunc

Edit: Added FileSetAttrib ($sFullPath, "-RASHNOT") after posting the script found out it won't delete files with read only and system attributes.

Edited March 1, 2008 by Danny35d

[randallc]

Hi,

For Files, you can get your list in AutoIt with this script; then loop through the array to check the age and delete if necessary;

you could then run again with attributes for folders if you want (or change the flag to "0" if you are confident!);

Best, Randall

; FileListCompare.au3
#include-once
#include<ArrayDupes8.au3>
#include<Array.au3>
#include<FileListToArray3.au3>
Local $sPath1 = "c:\programs", $sPath2 = "e:\programs", $filter = "*", $Flag = 1, $Recurse = 1, $BasDir = 0
Local $ar_Array1 = _FileListToArray3($sPath1, $filter, $Flag, $Recurse, $BasDir)
Local $ar_Array2 = _FileListToArray3($sPath2, $filter, $Flag, $Recurse, $BasDir)
; _ArrayCompare(ByRef $arrItems, ByRef $arrItems2, $iDetails = 0, $iType = 0, $istarti = 0, $istartj = 0) ;$iType=1 returns the Dupes instead of non-matching lines
Local $ar_UnMatchedItems = _ArrayCompare($ar_Array1, $ar_Array2, 1, 0, 1, 1), $ar_UnMatchedItems2[UBound($ar_UnMatchedItems)], $iCount=1,$iItem
; retrieve only those additional in $Array2;
If $ar_UnMatchedItems[UBound($ar_UnMatchedItems) - 1] <> "" Then
    For $i = 1 To UBound($ar_UnMatchedItems) - 1
        If StringInStr($ar_UnMatchedItems[$i], "Items2") Then
            $iItem=StringInStr($ar_UnMatchedItems[$i], "|") 
            $ar_UnMatchedItems2[$iCount] = $sPath2&"\"&StringLeft($ar_UnMatchedItems[$i], $iItem - 1)
            $iCount += 1
        EndIf
    Next
    $ar_UnMatchedItems2[0] = $iCount-1
    ReDim $ar_UnMatchedItems2[$iCount]
EndIf
If $ar_UnMatchedItems2[UBound($ar_UnMatchedItems2) - 1] <> "" Then _ArrayDisplay($ar_UnMatchedItems2, "Extra in $ar_Array2")

ArrayDupes8.au3

Edited March 1, 2008 by randallc

[FreeRider]

Hi Romulous,

I know you already had some answers...but may I ask you to try this function to search files (including recursive search, et...).

I found this function on one of the autoit forums (don't remember which one), I use it very often and it works very well.

Remarks :

- The $szRoot (this is the path) musts end with a "\"

- $Szmask is the file(s) (wild card are accepted)

- $nFlag is important :

$iRec set at "1 will perform recursive search (this is what you need)

$iPath set at "1" will retrun the full path and filename (this is what you need)

$iSort performs data sorting (I never use it)

$nOcc ... I assume it's an occurance related parameter but as I neved used it...

Now you're ready to try this script (copy it into an au3 file).

Have fun,

FreeRider

PS : By the way... I had the need to make a save and restore script in the past and I developed something similar with what you want to.

The only difference is I compared the source and the destination in both ways (A file canceled in the source was not canceled immediately in the destination, etc..)

If you're interested in I'm ready to give you the full code but I have to translate all help comments from french to english... so it could take a bit time...

;==================

#include <Array.au3>

$Path="C:\Windows\"

$Files="*.txt"

$Flag=3 ; Result will contain full path and file name for each record

$SearchRes=_FileSearch($Path,$Files,$Flag)

_ArrayDisplay($SearchRes,"Search Result with full path")

$Flag=1 ; Only file names will be returned (no path)

$SearchRes=_FileSearch($Path,$Files,$Flag)

_ArrayDisplay($SearchRes,"Search Result with only file name")

Func _FileSearch($szRoot, $Szmask = "*.*", $nFlag = 7, $nOcc = 0)

Local $hArray = $Szmask, $iRec, $iPath, $iSort

Switch $nFlag

Case 1

$iRec = 1

$iPath = 0

$iSort = 0

Case 2

$iRec = 0

$iPath = 1

$iSort = 0

Case 3

$iRec = 1

$iPath = 1

$iSort = 0

Case 4

$iRec = 0

$iPath = 0

$iSort = 1

Case 5

$iRec = 1

$iPath = 0

$iSort = 1

Case 6

$iRec = 0

$iPath = 1

$iSort = 1

Case Else

$iRec = 1

$iPath = 1

$iSort = 1

EndSwitch

If NOT IsArray($hArray) Then $hArray = StringSplit($hArray, '|')

Local $Hfile = 0, $F_List = ''

$szBuffer = ""

$szReturn = ""

$szPathlist = "*"

For $I = 1 To Ubound($hArray)-1

$szMask = $hArray[$I]

If NOT StringInStr ($Szmask, "\") Then

$szRoot &= ''

Else

$iTrim = StringInStr ($Szmask, "\",0,-1)

$szRoot &= StringLeft ($Szmask, $iTrim)

$Szmask = StringTrimLeft ($Szmask, $iTrim)

EndIf

If $iRec = 0 Then

$Hfile = FileFindFirstFile ($szRoot & $szMask)

If $Hfile >= 0 Then

$szBuffer = FileFindNextFile ($Hfile)

While NOT @Error

If $iPath = 1 Then $szReturn &= $szRoot

If $szBuffer <> "." AND $szBuffer <> ".." Then $szReturn &= $szBuffer & "*"

$szBuffer = FileFindNextFile ($Hfile)

Wend

FileClose ($Hfile)

EndIf

Else

While 1

$Hfile = FileFindFirstFile ($szRoot & "*.*")

If $Hfile >= 0 Then

$szBuffer = FileFindNextFile ($Hfile)

While NOT @Error

If $szBuffer <> "." AND $szBuffer <> ".." AND StringInStr (FileGetAttrib ($szRoot & $szBuffer), "D") Then _

$szPathlist &= $szRoot & $szBuffer & "*"

$szBuffer = FileFindNextFile ($Hfile)

Wend

FileClose ($Hfile)

EndIf

If StringInStr ($szReturn, $Szmask) > 0 AND $nOcc = 1 Then

$szRoot = ''

ExitLoop

EndIf

$Hfile = FileFindFirstFile ($szRoot & $szMask)

If $Hfile >= 0 Then

$szBuffer = FileFindNextFile ($Hfile)

While NOT @Error

If $iPath = 1 Then $szReturn &= $szRoot

If $szBuffer <> "." AND $szBuffer <> ".." Then $szReturn &= $szBuffer & "*"

$szBuffer = FileFindNextFile ($Hfile)

Wend

FileClose ($Hfile)

EndIf

If $szPathlist == "*" Then ExitLoop

$szPathlist = StringTrimLeft ($szPathlist, 1)

$szRoot = StringLeft ($szPathlist, StringInStr ($szPathlist, "*") - 1) & "\"

$szPathlist = StringTrimLeft ($szPathlist, StringInStr ($szPathlist, "*") - 1)

Wend

EndIf

Next

If $szReturn = "" Then

Return 0

Else

;$szReturn = StringReplace($szReturn,'\', '')

$F_List = StringSplit (StringTrimRight ($szReturn, 1), "*")

;If $iSort = 1 Then _ArraySort($F_List)

Return $F_List

EndIf

EndFunc ;<===> _FileSearch()