Jump to content
Sign in to follow this  
banged

Compare Files (Filesize and Filetime)

Recommended Posts

banged

Hi. I wanna compare (Filesize OR/AND FileTime(modified) ) for a few thousand files (40.000-50.000).I know a few ways for that but my problem is the time for that ...For 40.000 files, robocopy make 4-5 sec (ONLY for compare) when destination files not exist and little more time when destination files exist.With Standar ways ( Filegetsize($file) and FileGetTime($file) on loop) i need 15+15sec for compare.Anyone knows the really fastway (like robocopy) for my problem ?

Edited by banged

Share this post


Link to post
Share on other sites
PhoenixXL

what do you mean by

Compare Files

Does it mean whether their content is the same ?


My code:

PredictText: Predict Text of an Edit Control Like Scite. Remote Gmail: Execute your Scripts through Gmail. StringRegExp:Share and learn RegExp.

Run As System: A command line wrapper around PSEXEC.exe to execute your apps scripts as System (LSA). Database: An easier approach for _SQ_LITE beginners.

MathsEx: A UDF for Fractions and LCM, GCF/HCF. FloatingText: An UDF for make your text floating. Clipboard Extendor: A clipboard monitoring tool. 

Custom ScrollBar: Scroll Bar made with GDI+, user can use bitmaps instead. RestrictEdit_SRE: Restrict text in an Edit Control through a Regular Expression.

Share this post


Link to post
Share on other sites
Melba23

banged,

Welcome to the AutoIt forum. :)

The simple answer to your question is: use another language! AutoIt is interpreted (even when it is "compiled") and will therefore always be slower than a true compiled executable such as robocopy. If you want the simplicity of coding in AutoIt you also have to have to accept the longer execution time. ;)

But if you post the code you are using we can see if we can offer some tips to speed up the process for you - just put [autoit] before and [/autoit] after the code when you post. :)

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites
banged

Phoenix i mean differences on datemodified and filesize ....

melba23 i think your answer is correct.

Thank you guys ..

Share this post


Link to post
Share on other sites
kylomas

banged,

You might want to do as M23 suggested and show your code. I just ran through a data directory (91,515 files) looking for anything created this month in 5.4 seconds, on average.

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites
banged

Kylomas,

What is M23 ? ( my english is bad)

You say "looking for anything created"

I wanna just compile files between server and local machine serverdataMyApplicationFirstfile.exe <----> "C:Program FilesFirstfile.exe"

serverdataMyApplicationSecfile.dat <-----> "C:Program filesSecFile.dat" etc ....

and i need to know if firstfile on server it's same with firstfile on local machine, but this is next step from my script.

The first step is collect results from server and this is my problem.

In my example code i collected the results in 12.5 sec for 62000 files but i was check files on local machine from my hard disk , not on server

if i try to check 62000 files on server , i collect the results at 40sec. the point is the speed diff, between my script(autoit) and robocopy.

My Example Code:

#include
#include

$Location = (@WindowsDir)

$FileListToArrayPlus_List = _FileListToArrayPlus($Location,"*",0,0,"",2,True)

_ArraySort($FileListToArrayPlus_List)

Dim $filelist[UBound($FileListToArrayPlus_List)][3]

$Time = _Timer_Init()
For $i = 1 To UBound($FileListToArrayPlus_List) - 1
$filelist[$i][0] = $FileListToArrayPlus_List[$i] ; ~151 ms
$filelist[$i][1] = FileGetSize($filelist[$i][0]) ; ~6635 ms
$filelist[$i][2] = FileGetTime($filelist[$i][0],0,1) ; and all togather ~12100 ms
Next

ConsoleWrite(Round(_Timer_Diff($Time)) & " ms")

_ArrayDisplay($filelist)


; #FUNCTION# =====================================================================================================================
; _FileListToArrayPlus($sPath, $sInclude = "*", $iFlag = 0, $sExcludeFolder = "", $sExclude = "", $iPathType = 0, $bRecursive = False)
; Name...........: _FileListToArray_Rec
; Parameters ....: $sPath: Folder to search
; $sInclude: String to match on (wildcards allowed, multiples delimited by ;)
; $iFlag: Returned data type. 0 = Files and folders (default), 1 = Files only, 2 = Folders only
; $sExcludeFolder: List of folders to exclude from search (wildcards allowed, multiples delimited by ;)
; $sExclude: List of filenames to exclude from search (wildcards allowed, multiples delimited by ;)
; $iPathType: Returned data format. 0 = Filename only (default), 1 = Path relative to $sPath, 2 = Full path/filename
; $bRecursive: False = Search $sPath folder only (default), True = Search $sPath and all subfolders
; Author ........: Half the Autoit Community (Forum thread #96952)
;===================================================================================================================================



Func _FileListToArrayPlus($sPath, $sInclude = "", $iFlag = 0, $sExcludeFolder = "", $sExclude = "", $iPathType = 0, $bRecursive = False)
Local $sRet = "", $sReturnFormat = ""
$sPath = StringRegExpReplace($sPath, "[\\/]+\z", "") & "\" ; ensure single trailing slash
If Not FileExists($sPath) Then Return SetError(1, 1, "")
; Edit include files list
If $sInclude = "*" Then $sInclude = ""
If $sInclude Then
If StringRegExp($sInclude, "[\\/ :><\|]|(?s)\A\s*\z") Then Return SetError(2, 2, "") ; invalid characters test
$sInclude = StringRegExpReplace(StringRegExpReplace($sInclude, "(\s*;\s*)+", ";"), "\A;|;\z", "") ; Strip unwanted whitespace
$sInclude = StringRegExpReplace($sInclude, "[][$.+^{}()]", "\\$0"); Ignore special characters
$sInclude = StringReplace(StringReplace(StringReplace($sInclude, "?", "."), "*", ".*?"), ";", "$|") ; Convert ? to ., * to .*?, and ; to |
$sInclude = "(?i)\A(" & $sInclude & "$)"; case-insensitive, match from first char, terminate strings
EndIf
; Edit exclude folders list
If $sExcludeFolder Then
$sExcludeFolder = StringRegExpReplace(StringRegExpReplace($sExcludeFolder, "(\s*;\s*)+", ";"), "\A;|;\z", "") ; Strip unwanted whitespace
$sExcludeFolder = StringRegExpReplace($sExcludeFolder, "[][$.+^{}()]", "\\$0"); Ignore special characters
$sExcludeFolder = StringReplace(StringReplace(StringReplace($sExcludeFolder, "?", "."), "*", ".*?"), ";", "$|") ; Convert ?=. *=.*? ;=|
$sExcludeFolder = "(?i)\A(?!" & $sExcludeFolder & "$)"; case-insensitive, match from first char, terminate strings
EndIf
; Edit exclude files list
If $sExclude Then
$sExclude = StringRegExpReplace(StringRegExpReplace($sExclude, "(\s*;\s*)+", ";"), "\A;|;\z", "") ; Strip unwanted whitespace
$sExclude = StringRegExpReplace($sExclude, "[][$.+^{}()]", "\\$0"); Ignore special characters
$sExclude = StringReplace(StringReplace(StringReplace($sExclude, "?", "."), "*", ".*?"), ";", "$|") ; Convert ?=. *=.*? ;=|
$sExclude = "(?i)\A(?!" & $sExclude & "$)"; case-insensitive, match from first char, terminate strings
EndIf
; MsgBox(1,"Masks","File include: " & $sInclude & @CRLF & "File exclude: " & $sExclude & @CRLF & "Dir exclude : " & $sExcludeFolder)

If Not ($iFlag = 0 Or $iFlag = 1 Or $iFlag = 2) Then Return SetError(3, 3, "")

Local $sOrigPathLen = StringLen($sPath), $aQueue[64] = [1,$sPath], $iQMax = 63
While $aQueue[0]
$WorkFolder = $aQueue[$aQueue[0]]
$aQueue[0] -= 1
$search = FileFindFirstFile($WorkFolder & "*")
If @error Then ContinueLoop
Switch $iPathType
Case 1 ; relative path
$sReturnFormat = StringTrimLeft($WorkFolder, $sOrigPathLen)
Case 2 ; full path
$sReturnFormat = $WorkFolder
EndSwitch
; ConsoleWrite($sReturnFormat & @CRLF )
While 1
$file = FileFindNextFile($search)
If @error Then ExitLoop
If @extended Then ; Folder

If $sExcludeFolder And Not StringRegExp($file, $sExcludeFolder) Then ContinueLoop
If $bRecursive Then
If $aQueue[0] = $iQMax Then
$iQMax += 128
ReDim $aQueue[$iQMax + 1]
EndIf
$aQueue[0] += 1
$aQueue[$aQueue[0]] = $WorkFolder & $file & "\"
EndIf
If $iFlag = 1 Then ContinueLoop
$sRet &= $sReturnFormat & $file & "|"
Else ; File
If $iFlag = 2 Then ContinueLoop
If $sInclude And Not StringRegExp($file, $sInclude) Then ContinueLoop
If $sExclude And Not StringRegExp($file, $sExclude) Then ContinueLoop
$sRet &= $sReturnFormat & $file & "|"

EndIf
WEnd
FileClose($search)
WEnd
If Not $sRet Then Return SetError(4, 4, "")

Return StringSplit(StringTrimRight($sRet, 1), "|")
EndFunc
Edited by banged

Share this post


Link to post
Share on other sites
kylomas

What is M23 ?

Melba23, god of gui's and resident policeman.

You say "looking for anything created"

I wanna just compile files between server and local machine serverdataMyApplicationFirstfile.exe <----> "C:Program FilesFirstfile.exe"

serverdataMyApplicationSecfile.dat <-----> "C:Program filesSecFile.dat" etc ....

and i need to know if firstfile on server it's same with firstfile on local machine, but this is next step from my script.

The first step is collect results from server and this is my problem.

Excellent, now that you've made it clear what you are asking, M23's origional advice stands.

Good Luck,

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites
kylomas

Yes, awaiting his wrath....


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites
Melba23

Jos & kylomas,

The jury is still out! :huh2:

But I think I will take as a compliment.

;)

M23

Edited by Melba23

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites
guinness

kylomas is full of compliments today!


UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×