Sign in to follow this  
Followers 0
/dev/null

_FileChangeRecursive()

19 posts in this topic

#1 ·  Posted (edited)

Hi,

here is my contribution to a reccuring problem. Ever wanted to

change something in, or do something with a lot of files? Here

is my solution.

_FileChangeRecursive()

This functions finds files based on a given pattern (like FileFindFirstFile),

while it walks recursively through a given directory tree (up to a given

recursion level). Then it takes the files that match the pattern, and

pass them to a "worker function", one by one. That worker function actually

does the job of "changing" the files.

Sample #1

Task: Rename all files *.au3 -> *.au4

Function Call:

$retval = _FileChangeRecursive("C:\temp\autoit","*.au3",-1,"_RenameFile",".au4")oÝ÷ Ùµ¨®G«~éܶ*'ý¶®¶­s`¦gVæ2õ&VæÖTfÆRb33c¶fÆWFÂb33c¶WFVç7Föâ Æö6Âb33c·'G2Ò7G&æu7ÆBb33c¶fÆWFÂgV÷C²âgV÷C² Æö6Âb33c¶æWwFÒgV÷C²gV÷C° f÷"b33c¶âÒFòb33c·'G5³ÒÓ b33c¶æWwFf׳Òb33c·'G5²b33c¶åÐ æW@  fÆTÖ÷fRb33c¶fÆWFÂb33c¶æWwFfײb33c¶WFVç7Föâ¦VæFgVæ0 oÝ÷ Ù´^ÛöÛM«$ý³æiÉ.¦ÚWâë-¡Ø}êÞ×b­ç-¢¼©­ë®÷­çè׫²ÚîrÛ«y±nËb¢p_Ûjëh×6$retval = _FileChangeRecursive("C:\temp\autoit","*.au3",-1,"_BackupFile","C:\temp\autoit","c:\temp\backup")oÝ÷ Ùµ¨®G«~éܶ*'ý¶®¶­s`¦gVæ2ô&6·WfÆRb33c¶fÆWFÂb33c¶&6WFÂb33c¶&6·WföÆFW" Æö6Âb33c·&VÇFÒ7G&æu&WÆ6Rb33c¶fÆWFÂb33c¶&6WFÂgV÷C²gV÷C² fÆT6÷b33c¶fÆWFÂb33c¶&6·WföÆFW"fײb33c·&VÇFó¦VæFgVæ0

As you can see, you'll have to pass the name of the worker function and

all function parameters for the worker (up to 5 parameters are

possible).

The benefit of _FileChangeRecursive()is that you have one function that

searches the files recursively and you can write several "worker" functions

without the need to change a single line of _FileChangeRecursive().

For more samples see the attached files. Documentation is in the

function code itself!

Have fun!

Cheers

Kurt

_FileChangeRecursive.au3

_FileChangeRecursive_Samples.au3

Edited by /dev/null

__________________________________________________________(l)user: Hey admin slave, how can I recover my deleted files?admin: No problem, there is a nice tool. It's called rm, like recovery method. Make sure to call it with the "recover fast" option like this: rm -rf *

Share this post


Link to post
Share on other sites



Brilliant. Somebody actually using Call() for the purpose I re-designed it for.

Share this post


Link to post
Share on other sites

Brilliant. Somebody actually using Call() for the purpose I re-designed it for.

Thanks!


__________________________________________________________(l)user: Hey admin slave, how can I recover my deleted files?admin: No problem, there is a nice tool. It's called rm, like recovery method. Make sure to call it with the "recover fast" option like this: rm -rf *

Share this post


Link to post
Share on other sites

This would be perfect for my recently posted Ipod Music Transfer. Too bad I'm kind of lazy, I might add it in sooner or later. Good Job!

Kurt


Awaiting Diablo III..

Share this post


Link to post
Share on other sites

Hi everybody!

@ /dev/null:

I like this UDF a lot, thanks for sharing it. I'm PMing you some comments

alc

Share this post


Link to post
Share on other sites

Hi everybody!

@ /dev/null:

I like this UDF a lot, thanks for sharing it. I'm PMing you some comments

alc

The MVPs can only receive PMs from someone that has over 250 posts or has been on the board for at least 6 months. You're best bet is to PM Valik, he will give you the proper reply based on the relevance and intelligence of your PM.

[center]Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.[/center]

Share this post


Link to post
Share on other sites

Don't encourage people to PM me. I get enough crap as is. Encourage people to post in public threads so we can all have a go at them.

Share this post


Link to post
Share on other sites

The MVPs can only receive PMs from someone that has over 250 posts or has been on the board for at least 6 months. You're best bet is to PM Valik, he will give you the proper reply based on the relevance and intelligence of your PM.

:whistle::)

Anyway, too late....


__________________________________________________________(l)user: Hey admin slave, how can I recover my deleted files?admin: No problem, there is a nice tool. It's called rm, like recovery method. Make sure to call it with the "recover fast" option like this: rm -rf *

Share this post


Link to post
Share on other sites

Don't encourage people to PM me. I get enough crap as is. Encourage people to post in public threads so we can all have a go at them.

I would have thought that to be discouragement to PM anyone... :whistle: ... but I agree whole heartedly!!

[center]Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.[/center]

Share this post


Link to post
Share on other sites

@ SmOke_N: sorry for my incorrect PM, I'll try to behave myself in the future

@ Valik: ok, I don't want to crap you neither anybody. I sent it by PM to /dev/null because it is a bit long

@ /dev/null: What means too late?

Share this post


Link to post
Share on other sites

@ /dev/null: What means too late?

never mind, all O.K. It will take some time to answer your PM, as it's a bit "lengthy". BTW can you post or PM the modifications you have made? Thanks!

Cheers

Kurt


__________________________________________________________(l)user: Hey admin slave, how can I recover my deleted files?admin: No problem, there is a nice tool. It's called rm, like recovery method. Make sure to call it with the "recover fast" option like this: rm -rf *

Share this post


Link to post
Share on other sites

@ /dev/null

Ok, but I need to finished it and polish the look. Give me some time, cause right now my boss is standing behind me and asking for some results he is expecting. I'll try to post it today, but don't sure I'll have time, may be I'll have to post on Monday, I don't have Internet at home. Ok?

Just in case, happy weekend for everybody

Share this post


Link to post
Share on other sites

Ok, but I need to finished it and polish the look. Give me some time, cause right now my boss is standing behind me and asking for some results he is expecting. I'll try to post it today, but don't sure I'll have time, may be I'll have to post on Monday, I don't have Internet at home. Ok?

Just in case, happy weekend for everybody

O.K. Have a nice weekend!


__________________________________________________________(l)user: Hey admin slave, how can I recover my deleted files?admin: No problem, there is a nice tool. It's called rm, like recovery method. Make sure to call it with the "recover fast" option like this: rm -rf *

Share this post


Link to post
Share on other sites

Hi AutoItians!

OK, here is my proposition. In short:

- The use of another worker function, this time for folders

Both functions should be now optional

- Split the function in two, one master and one "executive"

This is in order to build $call_array only once, which will reduce significantly the excution time if parameters are used, especially if they are large (give it a try)

- Not to build the full path of the file when we call the worker function

Now, if I need the relative path, then I shoult strip it out, so I will be wasting time in two operations (perhaps in thousands of iterations). If I need the full path, then I always have the value of $sSearchPath

- Some rearrangements, minor optimizations and dropping of unused or unnecessary variables to make the code as fast as possible

- Instead of a single value, return an array with the number of found files and folders

- I think I updated comments correctly

I suggest /dev/null to rename the function, since it is far more than only for "change", and, if folder business is accepted, not only for "files"

And ask you to excuse my English

CODE

#include "File.au3"

#include "Array.au3"

Local $debug_prefix = "_FileChangeRecursive: "

;#============================================================================

;#== Author: Kurt (aka /dev/null)

;#==

;#== _FileChangeRecursive($sSearchPath, $sSearchPattern, $iMaxRecurseLevel, $sFileWorkerFunction = "", $sFolderWorkerFunction = "", $p1 = "__NP__", $p2 = "__NP__", $p3 = "__NP__", $p4 = "__NP__", $p5 = "__NP__")

;#==

;#==

;#== Search a file or folder pattern recursive and calls a worker function for each file and or folder

;#==

;#== Operation:

;#==

;#== The function walks recursively, starting at the search path $sSearchPath, and

;#== searches the files that match the search pattern $sSearchPattern. If the

;#== pattern is found, the worker function is called with the filepath (relative to $sSearchPath) as first

;#== parameter and probably more parameters, depending if those are given to

;#== this function ($p1, $p2, ..., $p5). If the

;#==

;#==

;#== Parameters:

;#==

;#== $sSearchPath the directory where to start the recursive search

;#== sholud not end in backslash

;#== $sSearchPattern the file search pattern

;#== $iMaxRecurseLevel the maximum recursion level.

;#== 0 = don't walk into sub dirs

;#== 1 = walk into 1 level of sub dirs

;#== -1 = walk into all sub dirs

;#== $sFileWorkerFunction the name of the worker function for files. The function is optional, but if

;#== declared, then MUST EXIST, OTHERWISE this function will throw an error

;#== $sFolderWorkerFunction the name of the worker function for folders. The function is optional, but if

;#== declared, then MUST EXIST, OTHERWISE this function will throw an error

;#==

;#== it is posible to use the same function for files and folders

;#== the functions should return 0 to indicate normal termination. Any other value forces the UDF

;#== to abort

;#==

;#== $p1,$p2,...,$p5 the optional parameters for the worker functions

;#== they will be passed to both worker functions

;#==

;#== Debugging:

;#==

;#== If the the global variable $__glb_FileChangeRecursive_debug_file is set

;#== the

;#== The value Recurse shows not the numbers of levels traversed, but the number of levels that should be

;#== still traversed

;#==

;#== Return Value(s): Returns an array

;#== On Success [0] - the number of found files

;#== [1] - the number of found folders

;#==

;#== On Failure [0] - the return error code

;#== -1 if _FileListToArray() throws an error

;#==

;#== @error @error of _FileListToArray

;#==

;#== -2 if the file worker function does not exist or

;#== the number of parameters for the worker is

;#== wrong

;#==

;#== @error 1

;#==

;#== -3 if the file worker function returns a value OTHER

;#== than 0 (the default), we will stop the operation

;#== and pass the error state of the worker up

;#==

;#== @error @error of the worker function

;#== @extended return code of the worker function

;#==

;#== -4 if the folder worker function does not exist or

;#== the number of parameters for the worker is

;#== wrong

;#==

;#== @error 1

;#==

;#== -5 if the folder worker function returns a value OTHER

;#== than 0 (the default), we will stop the operation

;#== and pass the error state of the worker up

;#==

;#== @error @error of the worker function

;#== @extended return code of the worker function

;#==

;#== [1] - has no meaning

;#==

;#============================================================================

Func _FileChangeRecursive($sSearchPath, $sSearchPattern, $iMaxRecurseLevel, $sFileWorkerFunction="", $sFolderWorkerFunction="", $p1="__NP__", $p2="__NP__", $p3="__NP__", $p4="__NP__", $p5="__NP__" )

;#=========================================================================

;#== If the max recursion level < 0 (e.g. -1) then, we set an insane high

;#== value for $iMaxRecurseLevel.

;#=========================================================================

If $iMaxRecurseLevel < 0 Then

$iMaxRecurseLevel = 0xFFFFFFF

EndIf

;#=========================================================================

;#== If the global debug variable is set and contains a string, the

;#== file name of a debug file, then print debug messages.

;#== ATTENTION: Please no "comments" about the global debug variable

;#== i designed it loke this, as don't want a debug function parameter!!!

;#=========================================================================

If IsDeclared("__glb_FileChangeRecursive_debug_file") then

Local $debug_file = Eval("__glb_FileChangeRecursive_debug_file")

Local $debug_msg = $debug_prefix & " Function: " & $sFileWorkerFunction & _

" Search Pattern: " & $sSearchPattern & _

" Parameter: " & $p1 & "," & $p2 & "," & $p3 & "," & $p4 & "," & $p5 & _

" Search Path: " & $sSearchPath & @CRLF

Local $retcode = _FileWriteLog($debug_file,$debug_msg)

If $retcode = 0 Then MsgBox(0,"DEBUGGING FATAL ERROR","_FileWriteLog Error code: " & @error)

EndIf

;#===========================================================================

;#== The worker functions will be executed through the Call() function, using

;#== the special array variant for passing parameters. We'll declare and build it here

;#== The first "working" parameter will be the path of the found file, now

;#== we just reserv the space for it

;#== Next, we add the received parameters, if any

;#===========================================================================

Local $call_array[2] = ["CallArgArray",""]

If $p1 <> "__NP__" then

_ArrayAdd($call_array,$p1)

If $p2 <> "__NP__" then

_ArrayAdd($call_array,$p2)

If $p3 <> "__NP__" then

_ArrayAdd($call_array,$p3)

If $p4 <> "__NP__" then

_ArrayAdd($call_array,$p4)

If $p5 <> "__NP__" then

_ArrayAdd($call_array,$p5)

EndIf

EndIf

EndIf

EndIf

EndIf

;#=================================================================

;#== Call the executive function

;#=================================================================

return _FileChangeRecursive_TheExecutiveOne($sSearchPath, $sSearchPattern, $iMaxRecurseLevel, $sFileWorkerFunction, $sFolderWorkerFunction, $call_array)

EndFunc

;-------------------------

func _FileChangeRecursive_TheExecutiveOne($sSearchPath, $sSearchPattern, $iMaxRecurseLevel, $sFileWorkerFunction="", $sFolderWorkerFunction="", $call_array=0)

;#=========================================================================

;#== Declare and initialize the return array

;#== Elements correspond to found files, found folders

;#=========================================================================

Local $ReturnArray[2] = [0,0]

;#=========================================================================

;#== This array will receive the result array when calling the function recursively

;#=========================================================================

local $retval[2] = [0,0]

;#=========================================================================

;#== get all FILES in the given Path $sSearchPath. If there is an error with

;#== _FileListToArray, pass @error and @extended up and return -1

;#=========================================================================

Local $files = _FileListToArray($sSearchPath,$sSearchPattern,1)

;#=========================================================================

;#== If there were FILES found, process them, by calling the worker function

;#== with the appropriate number of parameters.

;#=========================================================================

if IsArray($files) then

if @error > 0 then

SetError(@error,@extended)

$ReturnArray[0] = -1

return $ReturnArray

EndIf

If $sFileWorkerFunction <> "" Then

For $n = 1 To $files[0]

;#=================================================================

;#== If global DEBUG variable is set

;#=================================================================

If IsDeclared("__glb_FileChangeRecursive_debug_file") then

Local $debug_file = Eval("__glb_FileChangeRecursive_debug_file")

Local $debug_msg = $debug_prefix & " File: " & _

$sSearchPath & "\" & $files[$n] & @CRLF

Local $retcode = _FileWriteLog($debug_file,$debug_msg)

If $retcode = 0 Then MsgBox(0,"DEBUGGING FATAL ERROR","_FileWriteLog Error code: " & @error)

EndIf

;#=================================================================

;#== Load the file path and call the worker function

;#=================================================================

$call_array[1] = $files[$n]

$retval = Call($sFileWorkerFunction,$call_array)

;_ArrayDisplay($call_array,"")

;#=============================================================

;#== If @error is set, then there is a problem with call()

;#== Either the worker function name is wrong, or the number

;#== of parameters is wrong. We cannot tell which one, as Call()

;#== sets @error to 1 in both cases !!

;#=============================================================

If @error = 1 Then

SetError(1,@extended)

$ReturnArray[0] = -2

return $ReturnArray

EndIf

;#=============================================================

;#== If the return value is different from 0 (the default for

;#== any function in AutoIT !) then we assume the worker function

;#== returned some error code which we have to pass up!

;#== If so, we preserve @error (from the worker function) and

;#== set @extended to the return value of the worker function,

;#== then we return -3 to flag the worker function error

;#=============================================================

If $retval <> 0 Then

SetError(@error,$retval)

$ReturnArray[0] = -3

return $ReturnArray

EndIf

next

EndIf

;#=============================================================

;#== The number of files at this level

;#=============================================================

$ReturnArray[0] = $files[0]

endif

;#=========================================================================

;#== get all DIRECTORIES in the given Path $sSearchPath. If there is an error with

;#== _FileListToArray, pass @error and @extended up and return -1

;#=========================================================================

Local $dirs = _FileListToArray($sSearchPath,"*",2)

if IsArray($dirs) then

if @error > 0 then

SetError(@error,@extended)

$ReturnArray[0] = -1

return $ReturnArray

EndIf

;#=========================================================================

;#== Now walk through all directories

;#=========================================================================

for $n = 1 to $dirs[0]

;#=================================================================

;#== If global DEBUG variable is set

;#=================================================================

If IsDeclared("__glb_FileChangeRecursive_debug_file") then

Local $debug_file = Eval("__glb_FileChangeRecursive_debug_file")

Local $debug_msg = $debug_prefix & " Dir: " & _

$sSearchPath & "\" & $dirs[$n] & _

" Recurse: " & $iMaxRecurseLevel & @CRLF

Local $retcode = _FileWriteLog($debug_file,$debug_msg)

If $retcode = 0 Then MsgBox(0,"DEBUGGING FATAL ERROR","_FileWriteLog Error code: " & @error)

EndIf

;#=================================================================

;#== Load the folder path and call the worker function

;#=================================================================

If $sFolderWorkerFunction <> "" Then

$call_array[1] = $dirs[$n]

$retval = Call($sFolderWorkerFunction,$call_array)

;_ArrayDisplay($call_array,"")

;#=============================================================

;#== If @error is set, then there is a problem with call()

;#== Either the worker function name is wrong, or the number

;#== of parameters is wrong. We cannot tell which one, as Call()

;#== sets @error to 1 in both cases !!

;#=============================================================

If @error = 1 Then

SetError(1,@extended)

$ReturnArray[0] = -4

return $ReturnArray

EndIf

;#=============================================================

;#== If the return value is different from 0 (the default for

;#== any function in AutoIT !) then we assume the worker function

;#== returned some error code which we have to pass up!

;#== If so, we preserve @error (from the worker function) and

;#== set @extended to the return value of the worker function,

;#== then we return -5 to flag the worker function error

;#=============================================================

If $retval <> 0 Then

SetError(@error,$retval)

$ReturnArray[0] = -5

return $ReturnArray

EndIf

EndIf

;#=========================================================================

;#== Now walk through all directories

;#=========================================================================

If $iMaxRecurseLevel > 0 Then

$retval = _FileChangeRecursive_TheExecutiveOne($sSearchPath & "\" & $dirs[$n], $sSearchPattern,$iMaxRecurseLevel-1,$sFileWorkerFunction,$sFolderWorkerFunction,$call_array)

;#=================================================================

;#== If there was no error, just add the found number of files

;#=================================================================

If $retval[0] >= 0 Then

$ReturnArray[0] += $retval[0]

$ReturnArray[1] += $retval[1]

;#=================================================================

;#== If there was ANY error, pass it up, icluding @error and

;#== @extended

;#=================================================================

Else

SetError(@error,@extended)

return $retval

EndIf

EndIf

Next

$ReturnArray[1] += $dirs[0]

EndIf

;#=========================================================================

;#== That's it. Return the number of files and folders found.

;#=========================================================================

return $ReturnArray

endfunc

Thats all

alc

Share this post


Link to post
Share on other sites

Hi alc,

as I posted the code here, your are free to use and modify based on your needs. So, go ahead and change whatever you like...

Cheers

Kurt


__________________________________________________________(l)user: Hey admin slave, how can I recover my deleted files?admin: No problem, there is a nice tool. It's called rm, like recovery method. Make sure to call it with the "recover fast" option like this: rm -rf *

Share this post


Link to post
Share on other sites

Hi /dev/null

Thanks again for your code. I'm using it already

bye

alc

Share this post


Link to post
Share on other sites

Hi everybody!

I swallow back part of my words

It's too hard to figure out the path when you go into subfolders. So, when you call the worker function(s), add the $sSearchPath to the file or folder name, or pass it as a parameter

Let The Force be with you

alc

Share this post


Link to post
Share on other sites

#18 ·  Posted (edited)

Hi and thanks a lot for your script.

I'm using it and I really like it. that's a great job you have done.

I wish to have other functionnalities :

=> choose folder / file or Folder+file

=> an array of file exceptions to work on *.* but not *.zip nor *.rar etc.

I made a little function to be used with your script.

please feel free to used it.

any comment/suggestion are welcome.

CODE

; Compact an access database

; required At least MDAC 2.1

; No access installation is needed

;

; $Destination must be different from $Source

;

; $Replace

; 0 => keep $Source and $Destination file

; 1 => replace $Source with $Destination ; keep $Destination file

; 2 => replace $Source with $Destination ; remove $Destination file

;

func _CompactMDB($Source,$Destination, $Replace=0)

If FileExists($Source) Then

if FileExists($Destination)=0 Then

$oMDB = ObjCreate("JRO.JetEngine")

If IsObj($oMDB) Then

$oMDB.CompactDatabase( "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & $Source, "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & $Destination)

EndIf

Switch $Replace

case 1

filecopy( $Destination, $Source, 1+8)

case 2

filemove( $Destination, $Source, 1+8)

EndSwitch

EndIf

EndIf

endFunc

Edited by Krem

Share this post


Link to post
Share on other sites

@ Krem

Hi

Thank you for your code

- your suggestion about a filter array: FileChangeRecursive is just a framework for the execution of the worker function, which is where you do the real job, and I think it should be kept as fast as possible for everything. I can't figure out a fast way of implementing the filter you mention (an interesting one, BTW), so if we include it in the UDF, we will be wasting time when we will not need it (Is this in English?); in many cases the UDF will process thousands of files. Nevertheless, if I find a smart solution, I'll tell you

- I don't fully understand what you want about files/folders. The variant I suggested can solve this, since you can have one function for files, another one for folders, or just one for both. If your question is about the possibility of changing things at runtime, I'd do it this way: have only one worker function for files and folders, and use a parameter to inform the function what I want to do now. Again, to keep the UDF fast, I think it's important

Anyway, let's hear what /dev/null or anybody else has to say

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0