Sign in to follow this  
Followers 0
sshrum

Polling a dir with 30,000+ files

12 posts in this topic

#1 ·  Posted (edited)

I'm using the _FileSearch function(and t works great for sub 1,000 file searches) I got from someone here and find that pulling this list takes FOREVER to poll a folder of 30,000+ files!

A DOS dir /b /o command takes like 20-30 seconds.

Is there a way to make the DOS dir call and harvest the output?

TIA

BTW: AutoIT really should have some sort of built-in higher level FileDir function (in addition to the filefind options) (IMHO) :P

Edited by sshrum

Sean Shrum :: http://www.shrum.net

All my published AU3-based apps and utilities

'Make it idiot-proof, and someone will make a better idiot'

 

Share this post


Link to post
Share on other sites



I haven't tried StdOut yet .. but you can also re-route the output of the DIR command to a file which you can then interrogate ..

DIR /B/O>c:\Files.txt

:P

Share this post


Link to post
Share on other sites

Are you using this one.. I use several different versions like this for different things because of the speed?

http://www.autoitscript.com/forum/index.ph...c=16421&hl=


[center]Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.[/center]

Share this post


Link to post
Share on other sites

That kind of thing, yes :P

The DOS DIR command is very useful (and fast) .. here are some more links..

:nuke:

Share this post


Link to post
Share on other sites

Those are great (and old :P )!!!


[center]Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.[/center]

Share this post


Link to post
Share on other sites

LOL .. there are quite a few old things that are great :P

The best part is that AU3 allows you to leverage all sorts of great things from outside too.

Share this post


Link to post
Share on other sites

This is what I came up with on my own last night. It's a ton faster then the old _FileSearch()!

Uses STDOUT (no file writting/parsing like the other one you mentioned), took 14 seconds to return a listing of ~19,000 files versus ~3+ minutes the old way.

This can be used to replace your _FileSearch() and _FileSearchUtil(). Uses the same function syntax.

Func _FileSearch($sQuery, $bSubdir=0)
    $iLine = 0
    $sLine = ""
    $aLine = ""
    Dim $aFiles[100000];just a big number, we'll redim later
    $sArguments = "/a-d /b /on";filenames, not folders
    If $bSubdir = 1 Then $sArguments = $sArguments & " /s"
    $aRaw = Run(@ComSpec & " /c dir " & $sQuery & " " & $sArguments, @SystemDir, @SW_HIDE, 2)
    While 1
        $sLine = StdoutRead($aRaw)
        If @error Then ExitLoop
        $aLine = StringSplit($SLine, @CRLF)
        If $aLine[0] > 1 Then 
            For $i = 1 To $aLine[0]
                If $aLine[$i] = "" Then Continueloop
                $iLine = $iLine + 1
                $aFiles[$iLine] = $aLine[$i]
            Next
        Else
            $iLine = $iLine + 1
            $aFiles[$iLine] = $sLine
        EndIf
    Wend
    $aFiles[0] = $iLine
    ReDim $aFiles[$iLine + 1]
    Return $aFiles
EndFunc

Work's for me! Enjoy!


Sean Shrum :: http://www.shrum.net

All my published AU3-based apps and utilities

'Make it idiot-proof, and someone will make a better idiot'

 

Share this post


Link to post
Share on other sites

#12 ·  Posted (edited)

Hey valuater...I noticed that the STDOUT in AutoIT munges accented characters (I have another thread on this). I'll assume the FileListToArray will do the same thing as it uses the same sort of call. Any way to correct this?

Edited by sshrum

Sean Shrum :: http://www.shrum.net

All my published AU3-based apps and utilities

'Make it idiot-proof, and someone will make a better idiot'

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0