Jump to content

File Search and Display script?


Recommended Posts

Hi,

I'm quite new to AutoIT and I want to try my hands out on a little coding. I have a few machines that have their documents synced together. Sometimes I go computer by computer to check if the files and syncing properly and to check if certain files are there. I currently am using a batch file (.bat) file to find the files and display them in a text file.

I would like to encrypt my script and to put in a password to make sure that only I can use the script that I created. I would also want the script to be able to find the files then tell me the located files and what they are. Could someone please help me out in this? I hope that its not too much asking somebody to code a script that I need, but I'm hoping that someone could at least guide me in the way that I should go.

Thank you very much!

Link to comment
Share on other sites

Most of the code for this is already written, at least the tricky parts, you'd just need to look around, pick up the "parts", and glue them altogether into a GUI or something. You say, "I want to try my hands out on a little coding", so this glue-ing process sounds like an ideal project.

Here's something you might find useful; I use it a *lot* in applications. That old favourite, the recursive search..

;
; RecurseDir()  v2.2
;
; Recursively search a directory structure for files matching a pattern
; and return its files as an array of file paths, the first value being the
; total number of file paths in the array (a-la "AutoIt Array").
;
; I spent some time testing many different routines. _GetFileList, by Jos van
; der Zande, always gave me the best results, and nicely coded, but around ten 
; times slower (52s) than Larry (and Beerman's) recursion routines (4.9s) when 
; recursing my home folder.**
;
; ** 1000 folders, 4773 files, 702MB. [cpu:1.3GHz, 640MB RAM] at time of writing
;
; This function is based on _GetFileList, but after a few hacks and tweaks is now
; more than fifteen times faster than the original.** The results are still as good, 
; but instead of 50+ seconds to recurse my home folder, it now takes 3.3 seconds, 
; making it fastest and bestest of all the AutoIt recursion routines! *tic*
;
; Note: you can now supply multiple file masks (needed for backup). It makes a lot 
; of sense to grab any other extensions while we are in a directory. 
; The delimiter is a comma..
;
;       RecurseDir("C:\Program Files", "*.reg,*.ini,*.cfg")
;
; When searching for multiple masks the speed improvements are *staggering*, and 
; logarithmic; basically multiply the number of masks. For instance, a backup of
; all the pref files in my Program Files folder is scanned, copied, zipped and 
; completed in around a minute. pretty good, considering it's over 12GB; all
; tools, no big game installs, and ten file masks to search.
;
; The optional third parameter "$dont_recurse" tells RecurseDir() to only return
; matches for files in the top level directory, no deeper. (true/false)
; see also ReadDir() above.
;
; You can also supply an optional fourth parameter which is a string; the path to
; a "dump file" to dump (log) the paths of all matched files; for debugging only,
; because it will slow things down some.
;
; In v2.2 you can also supply an optional fifth parameter, "$return_dirs" which
; will do exactly that, returning an AutoIt array of all the directories in the
; path, and *only* the directories. (true/false) 
;
; This function gets used a lot in my apps.
;
;
; **    A lot to do with using the "&=" operator. Those wee differences mount up.
;       Probably that operator wasn't available when the code was first written.
;
func RecurseDir($dir, $mask, $dont_recurse=false, $dump="", $return_dirs=false)
;debug(@ScriptLineNumber & ": " & "func: RecurseDir " & $dir)

   dim $n_dirnames[333333]  ; maximum number of directories which can be scanned
   local $n_dircount = 0    ; ^ could be set much higher, if required
   local $n_file
   local $n_search
   local $n_tfile
   local $file_array
   local $filenames
   local $filecount
   local $dircount = 1
   local $msg = "error"

   ; if there was an "\" on the end of the given directory, remove that..
   if StringRight($dir, 1) = "\" then $dir = StringTrimRight($dir, 1)

   $n_dirnames[$dircount] = $dir

   if not FileExists($dir) then return 0

    while $dircount > $n_dircount ; keep on looping until all directories are scanned..

        $n_dircount += 1
        $n_search = FileFindFirstFile($n_dirnames[$n_dircount] & "\*.*")

        while 1  ; find all subdirs in this directory and store them in a array..
            $n_file = FileFindNextFile($n_search) 
            if @error then exitloop
            ; skip directory references..
            if $n_file = "." or $n_file = ".." then continueloop

            $n_tfile = $n_dirnames[$n_dircount] & "\" & $n_file

            ; if it's a directory, add it to the list of directories to be processed..
            if StringInStr(FileGetAttrib($n_tfile ), "D") and not $dont_recurse then                                
                $dircount += 1
                $n_dirnames[$dircount] = $n_tfile
            endif
        wend
        FileClose($n_search)

        ; multiple masks..
        if StringInStr($mask, ",", 2) then
            $mask_array = StringSplit($mask, ",")
        else ; or else create a dummy array..
            dim $mask_array[2] = [1, $mask]
        endif

        ; loop through the array of masks..
        for $mask_c = 1 to $mask_array[0]
            ; find all files that match this mask..
            $n_search = FileFindFirstFile($n_dirnames[$n_dircount] & "\" & $mask_array[$mask_c] )  
            if $n_search = -1 then continueloop

            while 1
                $n_file = FileFindNextFile($n_search) 
                if @error then exitloop ; end of dir
                if $n_file = "." or $n_file = ".." then continueloop

                $n_tfile = $n_dirnames[$n_dircount] & "\" & $n_file
                if not StringInStr(FileGetAttrib( $n_tfile ), "D") then
                    $filecount += 1
                    $filenames &= $n_tfile & @LF
                endif
            wend
            FileClose($n_search)
        next
    wend

    ; flip to a string and back to remove extraneous entries
    ; this is quicker than redimming on every loop
    if $return_dirs then 
        $tmp_str = ""
        $i = 1
        while $n_dirnames[$i] <> ""
            $tmp_str &= $n_dirnames[$i] & "|"
            $i += 1
        wend
        $tmp_str = StringTrimRight($tmp_str, 1)
        $n_dirnames = StringSplit($tmp_str, "|")
        return $n_dirnames
    endif

    $filenames = StringTrimRight($filenames, 1)
    if $filenames = "" then return 0
    $file_array = StringSplit($filenames, @LF)

    ; dump results to a file..
    if $dump then 
        $dump_file = FileOpen($dump, 2)
        FileWrite($dump_file, $filenames)
        FileClose($dump_file)
    endif
    return($file_array)
endfunc

I'm determined to post every one of my "essential" functions, somewhere on this forum!

Joking aside, that function could easily form the basis of you application. You run it once on the source folder, again on the destination, check through the two lists, comparing, and then <do stuff> with the results. There's your app in a nutshell.

Of course, you may find more suitable search functions kicking around the forum, and other useful code you can glue into place. The forum search is definitely your friend. Remember to add + before words in multiple searches to get better results (I made a Firefox search plug for this - very handy!)

Have fun!

;o)

(or

nothing is foolproof to the sufficiently talented fool..

Link to comment
Share on other sites

That's too easy! You need to RTFM!

At the top of the notes it tells you that the (rather useful, what!) function returns an array...

$path_list_array = RecurseDir($path, "*.*")

If you don't know how to iterate through an array, go find out! I wouldn't be helping you much by just spilling code. It's something you'll need to do a lot in coding life, so if you have some mental block about arrays (this is common) just do it, and you'll see how easy it is; they are just lists.

And near the bottom of RecurseDir's notes (TFM), it states..

; In v2.2 you can also supply an optional fifth parameter, "$return_dirs" which
; will do exactly that, returning an AutoIt array of all the directories in the
; path, and *only* the directories. (true/false)

TADA!

And all without messing with the time-line! *phew*

;o)

(or

ps. it's "cor", though I use parenthesis for the "(", because I'm "special" :) "corz" is a valid TLD way to say "cor's", you see, as in "cor's org"... corz.org (TADA2!)

nothing is foolproof to the sufficiently talented fool..

Link to comment
Share on other sites

The helpfile... Also, Do a search for AutoIt 1-2-3. There is no real way but to try, and if you run into problems, then post a question, search the forums. :)

Link to comment
Share on other sites

The AutoIt help comes in .chm format. Most IDE's can plugin .chm help files so you can just click on a function name (or whatever) and go straight to the help section for that function. It's all free energy from using the .chm format.

If you use EditPlus, I can help you out, here. If you use another IDE, it'll probably be something similar.

That page isn't finished by any means (just when I'm in the mood, and have time), but the .chm help tool part is done.

Have fun!

;o)

(or

ps. the AutoIt help file is excellent (though it could do with more interesting examples), and along with the forum search, you have all you need to learn AutoIt.

nothing is foolproof to the sufficiently talented fool..

Link to comment
Share on other sites

; In v2.2 you can also supply an optional fifth parameter, "$return_dirs" which

; will do exactly that, returning an AutoIt array of all the directories in the

; path, and *only* the directories. (true/false)

Hi,

I think the number of folders is returned -only- on recursion, in "DirRecurse" script ;not if current directory folder names requested?

Is that intended?

thanks, Randall

[PS Also my proposed UDF from signature is a little faster; interested in your thoughts on accuracy?]

So, if you use the zipped files, and the filelistToArray UDF from my sig,

>Running:(3.2.9.3):c:\Program Files\AutoIt3\beta\autoit3.exe "C:\Programs\SearchEngine\_FileListToArraytest5.au3"

all files and folders / Not Recurse / Not Full Directory Path / Not DisplayArray =4615 _FileBigListToArray4 : 103 mseconds

all files and folders / Not Recurse / Not Full Directory Path / Not DisplayArray =4615 _FileListToArrayExFixed : 573 mseconds

all files and folders / Not Recurse / Not Full Directory Path / Not DisplayArray =4615 _FileSearchGEO : 157 mseconds

all files and folders / Not Recurse / Not Full Directory Path / Not DisplayArray =4615 _FileListToArray3 : 77 mseconds to make

;===============================================================================

all files and folders / Not Recurse / Full Directory Path / Not DisplayArray =4559 RecurseDir : 2311 mseconds

all files and folders / Not Recurse / Full Directory Path / Not DisplayArray =4615 _FileBigListToArray4 : 107 mseconds

all files and folders / Not Recurse / Full Directory Path / Not DisplayArray =4615 _FileListToArrayExFixed : 533 mseconds

all files and folders / Not Recurse / Full Directory Path / Not DisplayArray =4615 _FileSearchGEO : 202 mseconds

all files and folders / Not Recurse / Full Directory Path / Not DisplayArray =4615 _FileListToArray3 : 123 mseconds to make

;===============================================================================

files only / Not Recurse / Not Full Directory Path / Not DisplayArray =4559 _FileBigListToArray4 : 548 mseconds

files only / Not Recurse / Not Full Directory Path / Not DisplayArray =4561 _FileListToArrayExFixed : 1310 mseconds

files only / Not Recurse / Not Full Directory Path / Not DisplayArray =4559 _FileListToArray3 : 601 mseconds to make

;===============================================================================

files only / Not Recurse / Full Directory Path / Not DisplayArray =4559 _RecurseDir : 1167 mseconds

files only / Not Recurse / Full Directory Path / Not DisplayArray =4559 _FileBigListToArray4 : 534 mseconds

files only / Not Recurse / Full Directory Path / Not DisplayArray =4561 _FileListToArrayExFixed : 1302 mseconds

files only / Not Recurse / Full Directory Path / Not DisplayArray =4559 _FileListToArray3 : 540 mseconds to make

;===============================================================================

folders only / Not Recurse / Not Full Directory Path / Not DisplayArray =56 _FileBigListToArray4 : 506 mseconds

folders only / Not Recurse / Not Full Directory Path / Not DisplayArray =54 _FileListToArrayExFixed : 1095 mseconds

folders only / Not Recurse / Not Full Directory Path / Not DisplayArray =56 _FileListToArray3 : 484 mseconds to make

;===============================================================================

folders only / Not Recurse / Full Directory Path / yyy DisplayArray =1 _RecurseDir : 1099 mseconds

folders only / Not Recurse / Full Directory Path / yyy DisplayArray =56 _FileBigListToArray4 : 506 mseconds

folders only / Not Recurse / Full Directory Path / yyy DisplayArray =54 _FileListToArrayExFixed : 1091 mseconds

folders only / Not Recurse / Full Directory Path / yyy DisplayArray =56 _FileListToArray3 : 486 mseconds to make

;===============================================================================

all files and folders / Recurse / Not Full Directory Path / Not DisplayArray =12900 _FileListToArrayExFixed : 1234 mseconds

all files and folders / Recurse / Not Full Directory Path / Not DisplayArray =12900 _FileSearchGEO : 2104 mseconds

all files and folders / Recurse / Not Full Directory Path / Not DisplayArray =12900 _FileListToArray3 : 1752 mseconds to make

;===============================================================================

all files and folders / Recurse / Full Directory Path / Not DisplayArray =12900 RecurseDir : 6244 mseconds

all files and folders / Recurse / Full Directory Path / Not DisplayArray =12900 _FileListToArrayExFixed : 1243 mseconds

all files and folders / Recurse / Full Directory Path / Not DisplayArray =12900 _FileSearchGEO : 2584 mseconds

all files and folders / Recurse / Full Directory Path / Not DisplayArray =12900 _FileListToArray3 : 1750 mseconds to make

;===============================================================================

files only / Recurse / Not Full Directory Path / Not DisplayArray =12591 _FileListToArrayExFixed : 3459 mseconds

files only / Recurse / Not Full Directory Path / Not DisplayArray =12588 _FileListToArray3 : 2966 mseconds to make

;===============================================================================

files only / Recurse / Full Directory Path / Not DisplayArray =12588 _RecurseDir : 3066 mseconds

files only / Recurse / Full Directory Path / Not DisplayArray =12591 _FileListToArrayExFixed : 3527 mseconds

files only / Recurse / Full Directory Path / Not DisplayArray =12588 _FileListToArray3 : 2993 mseconds to make

;===============================================================================

folders only / Recurse / Not Full Directory Path / Not DisplayArray =309 _FileListToArrayExFixed : 3039 mseconds

folders only / Recurse / Not Full Directory Path / Not DisplayArray =312 _FileListToArray3 : 1405 mseconds to make

;===============================================================================

folders only / Recurse / Full Directory Path / Not DisplayArray =313 _RecurseDir : 3004 mseconds

folders only / Recurse / Full Directory Path / Not DisplayArray =309 _FileListToArrayExFixed : 3056 mseconds

folders only / Recurse / Full Directory Path / Not DisplayArray =311 _FileListToArray3 : 1411 mseconds to make

Best, randall Edited by randallc
Link to comment
Share on other sites

Thanks for the msg randallc, I'd skipped over this not realizing it was aimed at me; big lists put me out of phase. :/

Hi,

I think the number of folders is returned -only- on recursion, in "DirRecurse" script ;not if current directory folder names requested?

Is that intended?

if StringInStr(FileGetAttrib($n_tfile ), "D") and not $dont_recurse then                                
                $dircount += 1 ...

yup. I have no need to return a list of folders for a single folder; I have a smaller function for that.

[PS Also my proposed UDF from signature is a little faster; interested in your thoughts on accuracy?]

I ran a few tests of my own (I have a widescreen monitor, but not wide enough to view that output!) It seems accurate enough, though not as fast as mine. I ran yours and mine on my Imaging program files and got..

---------- RUN ----------
UBound($aSplit) =3
time: 17078.1375845254 <- _FileListToArray3()
time: 13066.138852843 <- RecurseDir

Thinking my memory cache might be affecting the reults, I ran them the other way around (mine first) The results were practically identical.

When I told them to return only *.ini files, I got..

---------- RUN ----------
UBound($aSplit) =3
time: 15294.4765326319 <- _FileListToArray3()
time: 11491.9918593006 <- RecurseDir

Then I attempted to return multiple masks. I tried to get yours to search for "*.ini,*.exe,*.cfg,*.pal", but it wouldn't do it. How do you do multiple masks with FileListToArray3?

As for accuracy, when I did that search for ini file, yours returned more results! On further investigation, it's returning files like "C:\Program Files\audio\cool\_not\cool.ini [old]". But I probably misunderstand how your masks (filters) are working.

When I asked them to return folders only, yours was quicker..

---------- RUN ----------
UBound($aSplit) =3
time: 9546.83514245526 <- _FileListToArray3()
time: 11706.625283381 <- RecurseDir

Hmm. I'll have to look into that!

Bottom line; they are different, with different uses. But mine is still faster! :)

Anyways, I didn't originate the code, just took something good, and made it better.

;o)

(or

nothing is foolproof to the sufficiently talented fool..

Link to comment
Share on other sites

time: 17078.1375845254 <- _FileListToArray3()

time: 13066.138852843 <- RecurseDir

Hi,

1. Masks In my "_FileListToArraytest5.au3" script, you will see "$filter = "*.?x?|*.exe""

2. "*.ini" as mask you might be right that I would return ".ini[..]" with "*.ini" as mask; is that a problem?; I could look at it..

3. speedI can't reproduce your speed; can you give me a script which tests from a given directory? ;maybe I'm running it incorrectly.

Best, randall

[PS was speed script for "files only / Recurse / Full Directory Path "?]

Edited by randallc
Link to comment
Share on other sites

Ahh, gotcha! Pipe it is. I ran the multi-maks search and got this..

---------- RUN ----------
<your debug output snipped>
items: 445
time: 16193.0273260987  <- _FileListToArray3()
items: 445
time: 11650.0328952423  <- RecurseDir

The code I'm using is simply (it varies, but this is the one I just used)..

$begin = TimerInit()
$filelist1 = _FileListToArray3("C:\Program Files\Imaging", "*.ini|*.exe|*.cfg|*.pal", 1, 1)
$dif = TimerDiff($begin)
ConsoleWrite("items: " & $filelist1[0] & @LF)
ConsoleWrite("time: " & $dif & @LF)
DumpArray($filelist1, "multi 1")

$begin = TimerInit()
$filelist2 = RecurseDir("C:\Program Files\Imaging", "*.ini,*.exe,*.cfg,*.pal")
$dif = TimerDiff($begin)
ConsoleWrite("items: " & $filelist2[0] & @LF)
ConsoleWrite("time: " & $dif & @LF)
DumpArray($filelist2, "multi 2")

Hey, maybe I'm running yours wrong! :)

;o)

(or

ps. DumpArray() is in my "cor's essentials", along with RecurseDir() and others.

nothing is foolproof to the sufficiently talented fool..

Link to comment
Share on other sites

Hi,

oK, thanks; my conclusion is that yours may be faster (borderline) for

1. "files only / Recurse" when multiple masks [and maybe unfiltered?]

[Although I think "DirRecurse" may be faster partly because it does not check for duplicate returns when there are multiple masks? ]

If i am looking at trying to make a universal UDF, there are still reasons to go my path, though your routine is "cleaner".

1. "files only / Recurse" faster if just one mask? (borderline)

2. "files only / Not Recurse" much faster?

3. "folders only / Not Recurse" possible

4. "all files and folders / Recurse" much faster?

5. "folders / Recurse" much faster?

6. I have "exclude" option.

If i could modify yours suitably, I would prefer it!

Best, Randall

Link to comment
Share on other sites

I think it's great that classics like this (recursive search) get a re-work every now and again; for one, new functions and improved operators mean greater efficiency and speed, and that's always a good thing. I also think your idea of a "Universal Recurse UDF" is a sound one; it's one of a few "essential" (at least for me) functions that should be in the factory-supplied UDF's, and they, of course, should always be the very best examples of current code.

Getting the perfect mix; a bit from here, a bit from there; of all the best recursion routines and mixing them up into one super-recurse function will require some toil, no doubt, and you're definitely in line for a gold star for taking on the task. The extra options you are looking for would be fairly easy to implement into RecurseDir(), in some cases, you would be removing code, in fact. Everything is by design, so its easy enough to redesign, too. I'm not entirely sure what your numbered list represents, so I'm not going to comment on individual items.

As for cleanness of code; my feeling is that the UDF's are more than just functions that folk blindly include; they are a reference for AutoIt coders everywhere, examples of how to write good code with AutoIt. For this reason alone, the code inside them should be clean and as readable as possible. As you might have guessed, it's something I'm fairly passionate about.

I don't mind if you base your super-recurse function on mine, but if you Proper Case all the keywords and change the indentation, man! you gotta promise me you don't give me any credit in the UDF; I want nothing to do with that $hit! :)

Here's where the credit begins (I just now tracked this down on the forum), RecurseDir(), before I got my grubby mits on it..

; AutoIt Version: 3.0
; Language:       English
; Platform:       Win9x/NT/XP
; Author:         jos van der Zande
;
; Find files in directory and subdirectories and return it in an Array, all coded in Autoit3
;
Dim $FILES
Dim $DIR="c:\temp"     ; specify the directory to search
Dim $FILEMASK="*.*"    ; specify the filemask

$FILES = _GetFileList($DIR, $FILEMASK)
For $X = 0 To UBound($FILES)-1
   MsgBox(0,'test:' & $X,$FILES[$X])
Next
Exit
;********************************************************************************
;Recursive search for filemask
;********************************************************************************

Func _GetFileList($T_DIR,$T_MASK)
   Dim $N_DIRNAMES[200000]; max number of directories that can be scanned
   Local $N_DIRCOUNT = 0
   Local $N_FILE
   Local $N_SEARCH
   Local $N_TFILE
   Local $N_OFILE
   Local $T_FILENAMES
   Local $T_FILECOUNT
   Local $T_DIRCOUNT = 1
  ; remove the end \ If specified
   If StringRight($T_DIR,1) = "\" Then $T_DIR = StringTrimRight($T_DIR,1)
   $N_DIRNAMES[$T_DIRCOUNT] = $T_DIR
  ; Exit if base dir doesn't exists
   If Not FileExists($T_DIR) Then Return 0
  ; keep on looping until all directories are scanned
   While $T_DIRCOUNT > $N_DIRCOUNT
      $N_DIRCOUNT = $N_DIRCOUNT + 1
     ; find all subdirs in this directory and save them in a array
      $N_SEARCH = FileFindFirstFile($N_DIRNAMES[$N_DIRCOUNT] & "\*.*")  
      While 1
         $N_FILE = FileFindNextFile($N_SEARCH)
         If @error Then ExitLoop
        ; skip these references
         If $N_FILE = "." Or $N_FILE = ".." Then ContinueLoop
         $N_TFILE = $N_DIRNAMES[$N_DIRCOUNT] & "\" & $N_FILE
        ; if Directory than add to the list of directories to be processed
         If StringInStr(FileGetAttrib( $N_TFILE ),"D") > 0 Then
            $T_DIRCOUNT = $T_DIRCOUNT + 1
            $N_DIRNAMES[$T_DIRCOUNT] = $N_TFILE
         EndIf
      Wend
      FileClose($N_SEARCH)
     ; find all Files that mtach the MASK
      $N_SEARCH = FileFindFirstFile($N_DIRNAMES[$N_DIRCOUNT] & "\" & $T_MASK )  
      If $N_SEARCH = -1 Then ContinueLoop
      While 1
         $N_FILE = FileFindNextFile($N_SEARCH)
         If @error Then ExitLoop
        ; skip these references
         If $N_FILE = "." Or $N_FILE = ".." Then ContinueLoop
         $N_TFILE = $N_DIRNAMES[$N_DIRCOUNT] & "\" & $N_FILE
        ; if Directory than add to the list of directories to be processed
         If StringInStr(FileGetAttrib( $N_TFILE ),"D") = 0 Then
            $T_FILENAMES  = $T_FILENAMES & $N_TFILE & @CR
            $T_FILECOUNT = $T_FILECOUNT + 1
           ;MsgBox(0,'filecount ' & $T_FILECOUNT ,$N_TFILE)
         EndIf
      Wend
      FileClose($N_SEARCH)
   Wend
   $T_FILENAMES  = StringTrimRight($T_FILENAMES,1)
   $N_OFILE = StringSplit($T_FILENAMES,@CR)
   Return( $N_OFILE )
EndFunc  ;==>_GetFileList

I didn't realize I'd changed it so much! Anyways, it might be worth getting back to the original (+ my speed improvements), and then, for each extra feature, finding the best, most efficient method from all the different routines, and inserting that. Or perhaps another approach might be better.

I'm always going back to my own code, adding and improving things; so there will probably be a v2.3 at some point. I hadn't considered an exclude option; I can see where that might be handy. But I have no use for it (yet), and it probably won't find its way in until I do, such is life. But you go ahead and use whatever code you like!

Have fun!

;o)

(or

nothing is foolproof to the sufficiently talented fool..

Link to comment
Share on other sites

  • 1 year later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...