Jump to content
Sign in to follow this  
SpookMeister

Help me with my logic here...

Recommended Posts

SpookMeister

I am using Larry's _FileSearch function in a little script a wrote, and I am having some concerns about the limitations it has. I tested it in a smaller subset of data, but now that I have it running on my real data, I'm having doubts about it working.

Essentially, the part I am concerned about is that I am searching a directory tree that contains a whole LOT of files... like... just under 80 million total, broken up into about 8000 subdirectories.

The path of each file looks like:

X:\XX\XXXX\xxxxxxxxxxxxx.xxx

I make that out to be 28 characters per file (full path)

Here's where I need your help...

First, do I correctly understand how Larry's little function works? I believe he basically makes a long string out of all the full paths of the files, separated by an *, then uses the StringSplit function to turn that string into an array of all the files found.

Second, if the above is correct, and the FAQ is accurate in that a "Maximum string length: 2,147,483,647 characters", won't I run into an error of some kind before it completes the search?

If (File length + Seperator) * number of files = total length of string

(28 + 1) * 80,000,000 = 2,320,000,000

Here's the version of Larry's function that I am using:

CODE
; _FileSearch( <Path and Mask>, <Option (0 - normal, 1- recursive)>)
; Returns array. Either Array of files (full path) where...
; Array[0] is number of files.
; Array[0] = 0 if nothing found.
;
; EXAMPLE USAGE
;--------------------------------------------
$a = _FileSearch("C:\surf\*.exe",0)
If $a[0] > 0 Then
    For $i = 1 to $a[0]
         MsgBox(4096,"",$a[$i])
    Next
EndIf
;--------------------------------------------

Func _FileSearch($szMask,$nOption)
    $szRoot = ""
    $hFile = 0
    $szBuffer = ""
    $szReturn = ""
    $szPathList = "*"
    Dim $aNULL[1]

    If Not StringInStr($szMask,"\") Then
         $szRoot = @SCRIPTDIR & "\"
    Else
         While StringInStr($szMask,"\")
              $szRoot = $szRoot & StringLeft($szMask,StringInStr($szMask,"\"))
              $szMask = StringTrimLeft($szMask,StringInStr($szMask,"\"))
         Wend
    EndIf
    If $nOption = 0 Then
         _FileSearchUtil($szRoot, $szMask, $szReturn)
    Else
         While 1
              $hFile = FileFindFirstFile($szRoot & "*.*")
              If $hFile >= 0 Then
                   $szBuffer = FileFindNextFile($hFile)
                   While Not @ERROR
                        If $szBuffer <> "." And $szBuffer <> ".." And _
                             StringInStr(FileGetAttrib($szRoot & $szBuffer),"D") Then _
                             $szPathList = $szPathList & $szRoot & $szBuffer & "*"
                        $szBuffer = FileFindNextFile($hFile)
                   Wend
                   FileClose($hFile)
              EndIf
              _FileSearchUtil($szRoot, $szMask, $szReturn)
              If $szPathList == "*" Then ExitLoop
              $szPathList = StringTrimLeft($szPathList,1)
              $szRoot = StringLeft($szPathList,StringInStr($szPathList,"*")-1) & "\"
              $szPathList = StringTrimLeft($szPathList,StringInStr($szPathList,"*")-1)
         Wend
    EndIf
    If $szReturn = "" Then
         $aNULL[0] = 0
         Return $aNULL
    Else
         Return StringSplit(StringTrimRight($szReturn,1),"*")
    EndIf
EndFunc

Func _FileSearchUtil(ByRef $ROOT, ByRef $MASK, ByRef $RETURN)
    $hFile = FileFindFirstFile($ROOT & $MASK)
    If $hFile >= 0 Then
         $szBuffer = FileFindNextFile($hFile)
         While Not @ERROR
              If $szBuffer <> "." And $szBuffer <> ".." Then _
                   $RETURN = $RETURN & $ROOT & $szBuffer & "*"
              $szBuffer = FileFindNextFile($hFile)
         Wend
         FileClose($hFile)
    EndIf
EndFunc

So... if I am right about my concerns here.. how else might I go about my goal, which by the way is to perform a file by file size comparison to verify that I have made a good copy of the original files into another location (the files have been copied already, I'm just validating that the copy went well)

TIA

-Spook

Edited by SpookMeister

[u]Helpful tips:[/u]If you want better answers to your questions, take the time to reproduce your issue in a small "stand alone" example script whenever possible. Also, make sure you tell us 1) what you tried, 2) what you expected to happen, and 3) what happened instead.[u]Useful links:[/u]BrettF's update to LxP's "How to AutoIt" pdfValuater's Autoit 1-2-3 Download page for the latest versions of Autoit and SciTE[quote]<glyph> For example - if you came in here asking "how do I use a jackhammer" we might ask "why do you need to use a jackhammer"<glyph> If the answer to the latter question is "to knock my grandmother's head off to let out the evil spirits that gave her cancer", then maybe the problem is actually unrelated to jackhammers[/quote]

Share this post


Link to post
Share on other sites
Zedna

Place your code directly into copy of _FileSearch().

So you will make your desired action with each file inside this function and needn't whole buffer/StringSplit/array of all files.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×