Compare Files (Filesize and Filetime)

[banged]

Hi. I wanna compare (Filesize OR/AND FileTime(modified) ) for a few thousand files (40.000-50.000).I know a few ways for that but my problem is the time for that ...For 40.000 files, robocopy make 4-5 sec (ONLY for compare) when destination files not exist and little more time when destination files exist.With Standar ways ( Filegetsize($file) and FileGetTime($file) on loop) i need 15+15sec for compare.Anyone knows the really fastway (like robocopy) for my problem ?

Edited February 25, 2013 by banged

[PhoenixXL]

what do you mean by

Compare Files

Does it mean whether their content is the same ?

[Melba23]

banged,

Welcome to the AutoIt forum.

The simple answer to your question is: use another language! AutoIt is interpreted (even when it is "compiled") and will therefore always be slower than a true compiled executable such as robocopy. If you want the simplicity of coding in AutoIt you also have to have to accept the longer execution time.

But if you post the code you are using we can see if we can offer some tips to speed up the process for you - just put [autoit] before and [/autoit] after the code when you post.

M23

[banged]

Phoenix i mean differences on datemodified and filesize ....

melba23 i think your answer is correct.

Thank you guys ..

[kylomas]

banged,

You might want to do as M23 suggested and show your code. I just ran through a data directory (91,515 files) looking for anything created this month in 5.4 seconds, on average.

kylomas

[banged]

Kylomas,

What is M23 ? ( my english is bad)

You say "looking for anything created"

I wanna just compile files between server and local machine serverdataMyApplicationFirstfile.exe <----> "C:Program FilesFirstfile.exe"

serverdataMyApplicationSecfile.dat <-----> "C:Program filesSecFile.dat" etc ....

and i need to know if firstfile on server it's same with firstfile on local machine, but this is next step from my script.

The first step is collect results from server and this is my problem.

In my example code i collected the results in 12.5 sec for 62000 files but i was check files on local machine from my hard disk , not on server

if i try to check 62000 files on server , i collect the results at 40sec. the point is the speed diff, between my script(autoit) and robocopy.

My Example Code:

expandcollapse popup

#include
#include

$Location = (@WindowsDir)

$FileListToArrayPlus_List = _FileListToArrayPlus($Location,"*",0,0,"",2,True)

_ArraySort($FileListToArrayPlus_List)

Dim $filelist[UBound($FileListToArrayPlus_List)][3]

$Time = _Timer_Init()
For $i = 1 To UBound($FileListToArrayPlus_List) - 1
$filelist[$i][0] = $FileListToArrayPlus_List[$i] ; ~151 ms
$filelist[$i][1] = FileGetSize($filelist[$i][0]) ; ~6635 ms
$filelist[$i][2] = FileGetTime($filelist[$i][0],0,1) ; and all togather ~12100 ms
Next

ConsoleWrite(Round(_Timer_Diff($Time)) & " ms")

_ArrayDisplay($filelist)


; #FUNCTION# =====================================================================================================================
; _FileListToArrayPlus($sPath, $sInclude = "*", $iFlag = 0, $sExcludeFolder = "", $sExclude = "", $iPathType = 0, $bRecursive = False)
; Name...........: _FileListToArray_Rec
; Parameters ....: $sPath: Folder to search
; $sInclude: String to match on (wildcards allowed, multiples delimited by ;)
; $iFlag: Returned data type. 0 = Files and folders (default), 1 = Files only, 2 = Folders only
; $sExcludeFolder: List of folders to exclude from search (wildcards allowed, multiples delimited by ;)
; $sExclude: List of filenames to exclude from search (wildcards allowed, multiples delimited by ;)
; $iPathType: Returned data format. 0 = Filename only (default), 1 = Path relative to $sPath, 2 = Full path/filename
; $bRecursive: False = Search $sPath folder only (default), True = Search $sPath and all subfolders
; Author ........: Half the Autoit Community (Forum thread #96952)
;===================================================================================================================================



Func _FileListToArrayPlus($sPath, $sInclude = "", $iFlag = 0, $sExcludeFolder = "", $sExclude = "", $iPathType = 0, $bRecursive = False)
Local $sRet = "", $sReturnFormat = ""
$sPath = StringRegExpReplace($sPath, "[\\/]+\z", "") & "\" ; ensure single trailing slash
If Not FileExists($sPath) Then Return SetError(1, 1, "")
; Edit include files list
If $sInclude = "*" Then $sInclude = ""
If $sInclude Then
If StringRegExp($sInclude, "[\\/ :><\|]|(?s)\A\s*\z") Then Return SetError(2, 2, "") ; invalid characters test
$sInclude = StringRegExpReplace(StringRegExpReplace($sInclude, "(\s*;\s*)+", ";"), "\A;|;\z", "") ; Strip unwanted whitespace
$sInclude = StringRegExpReplace($sInclude, "[][$.+^{}()]", "\\$0"); Ignore special characters
$sInclude = StringReplace(StringReplace(StringReplace($sInclude, "?", "."), "*", ".*?"), ";", "$|") ; Convert ? to ., * to .*?, and ; to |
$sInclude = "(?i)\A(" & $sInclude & "$)"; case-insensitive, match from first char, terminate strings
EndIf
; Edit exclude folders list
If $sExcludeFolder Then
$sExcludeFolder = StringRegExpReplace(StringRegExpReplace($sExcludeFolder, "(\s*;\s*)+", ";"), "\A;|;\z", "") ; Strip unwanted whitespace
$sExcludeFolder = StringRegExpReplace($sExcludeFolder, "[][$.+^{}()]", "\\$0"); Ignore special characters
$sExcludeFolder = StringReplace(StringReplace(StringReplace($sExcludeFolder, "?", "."), "*", ".*?"), ";", "$|") ; Convert ?=. *=.*? ;=|
$sExcludeFolder = "(?i)\A(?!" & $sExcludeFolder & "$)"; case-insensitive, match from first char, terminate strings
EndIf
; Edit exclude files list
If $sExclude Then
$sExclude = StringRegExpReplace(StringRegExpReplace($sExclude, "(\s*;\s*)+", ";"), "\A;|;\z", "") ; Strip unwanted whitespace
$sExclude = StringRegExpReplace($sExclude, "[][$.+^{}()]", "\\$0"); Ignore special characters
$sExclude = StringReplace(StringReplace(StringReplace($sExclude, "?", "."), "*", ".*?"), ";", "$|") ; Convert ?=. *=.*? ;=|
$sExclude = "(?i)\A(?!" & $sExclude & "$)"; case-insensitive, match from first char, terminate strings
EndIf
; MsgBox(1,"Masks","File include: " & $sInclude & @CRLF & "File exclude: " & $sExclude & @CRLF & "Dir exclude : " & $sExcludeFolder)

If Not ($iFlag = 0 Or $iFlag = 1 Or $iFlag = 2) Then Return SetError(3, 3, "")

Local $sOrigPathLen = StringLen($sPath), $aQueue[64] = [1,$sPath], $iQMax = 63
While $aQueue[0]
$WorkFolder = $aQueue[$aQueue[0]]
$aQueue[0] -= 1
$search = FileFindFirstFile($WorkFolder & "*")
If @error Then ContinueLoop
Switch $iPathType
Case 1 ; relative path
$sReturnFormat = StringTrimLeft($WorkFolder, $sOrigPathLen)
Case 2 ; full path
$sReturnFormat = $WorkFolder
EndSwitch
; ConsoleWrite($sReturnFormat & @CRLF )
While 1
$file = FileFindNextFile($search)
If @error Then ExitLoop
If @extended Then ; Folder

If $sExcludeFolder And Not StringRegExp($file, $sExcludeFolder) Then ContinueLoop
If $bRecursive Then
If $aQueue[0] = $iQMax Then
$iQMax += 128
ReDim $aQueue[$iQMax + 1]
EndIf
$aQueue[0] += 1
$aQueue[$aQueue[0]] = $WorkFolder & $file & "\"
EndIf
If $iFlag = 1 Then ContinueLoop
$sRet &= $sReturnFormat & $file & "|"
Else ; File
If $iFlag = 2 Then ContinueLoop
If $sInclude And Not StringRegExp($file, $sInclude) Then ContinueLoop
If $sExclude And Not StringRegExp($file, $sExclude) Then ContinueLoop
$sRet &= $sReturnFormat & $file & "|"

EndIf
WEnd
FileClose($search)
WEnd
If Not $sRet Then Return SetError(4, 4, "")

Return StringSplit(StringTrimRight($sRet, 1), "|")
EndFunc

Edited February 25, 2013 by banged

[kylomas]

What is M23 ?

Melba23, god of gui's and resident policeman.

You say "looking for anything created"

I wanna just compile files between server and local machine serverdataMyApplicationFirstfile.exe <----> "C:Program FilesFirstfile.exe"

serverdataMyApplicationSecfile.dat <-----> "C:Program filesSecFile.dat" etc ....

and i need to know if firstfile on server it's same with firstfile on local machine, but this is next step from my script.

The first step is collect results from server and this is my problem.

Excellent, now that you've made it clear what you are asking, M23's origional advice stands.

Good Luck,

kylomas

[Jos]

Melba23, god of gui's and resident policeman.

Not sure you just made him happy or not

[kylomas]

Yes, awaiting his wrath....

[Melba23]

Jos & kylomas,

The jury is still out!

But I think I will take as a compliment.

M23

Edited February 25, 2013 by Melba23

[guinness]

kylomas is full of compliments today!