Sign in to follow this  
Followers 0
photonbuddy

_FileBigListToArray()

20 posts in this topic

Hi All,

I've been using _FileListToArray in one of my programs, and noted it was getting a bit slow as the number of files grew in the folder.

I decided to have a look at the source for this function, and thought I had spotted the problem. After knocking-up a little test script, it became obvious. The ReDim'ing of the main array for each new file was really taking it's toll on things, so I changed that part of the code to make _FileBigListToArray():

Func _FileBigListToArray($sPath, $sFilter = "*", $iFlag = 0, $sSize=1000)
    Local $hSearch, $sFile, $asFileList[$sSize], $rDimCnt=1, $rDimChk
    If Not FileExists($sPath) Then Return SetError(1, 1, "")
    If (StringInStr($sFilter, "\")) Or (StringInStr($sFilter, "/")) Or (StringInStr($sFilter, ":")) Or (StringInStr($sFilter, ">")) Or (StringInStr($sFilter, "<")) Or (StringInStr($sFilter, "|")) Or (StringStripWS($sFilter, 8) = "") Then Return SetError(2, 2, "")
    If Not ($iFlag = 0 Or $iFlag = 1 Or $iFlag = 2) Then Return SetError(3, 3, "")
    $hSearch = FileFindFirstFile($sPath & "\" & $sFilter)
    If $hSearch = -1 Then Return SetError(4, 4, "")
    While 1
        $sFile = FileFindNextFile($hSearch)
        If @error Then
            SetError(0)
            ExitLoop
        EndIf
        If $iFlag = 1 And StringInStr(FileGetAttrib($sPath & "\" & $sFile), "D") <> 0 Then ContinueLoop
        If $iFlag = 2 And StringInStr(FileGetAttrib($sPath & "\" & $sFile), "D") = 0 Then ContinueLoop
        ReDim $asFileList[UBound($asFileList) + 1]
        $rDimChk=Int($asFileList[0]/$rDimCnt)
        If $rDimChk>$sSize-1 Then
            $rDimCnt=$rDimCnt+1
            ReDim $asFileList[$rDimCnt*1000]
        EndIf
        $asFileList[0] = $asFileList[0] + 1
        $asFileList[$asFileList[0]] = $sFile
    WEnd
    FileClose($hSearch)
    ReDim $asFileList[$asFileList[0]+1]
    Return $asFileList
EndFunc  ;==>_FileListToArray

The only change the programmer has to be aware of is the addition of the $sSize parameter. This is the number the array is ReDim'ed by. This is a trade-off in memory versus number of files. If you've got loads of memory, you could set this to something silly, and never have to ReDim the array.

Just wondering if anyone can see any glaring oversights that might cause problems later?

Any other general thoughts or comments?

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

Hi,

Not sure it's any faster despite the theory;

Time _FileBigListToArray :6121 mseconds to make

Time _FileListToArray :6087 mseconds to make

Time __FileListToArray :1184 mseconds to make

;_FileBigListToArray.au3
#include<array.au3>
#include<file.au3>
#include<process.au3>
$sPath=@ScriptDir
;~ #include "_GUILVSort4.au3"
$timerstamp1 = TimerInit()
$ar_Array=_FileBigListToArray($sPath)
ConsoleWrite("Time _FileBigListToArray :" & Round(TimerDiff($timerstamp1)) & " mseconds to make" & @LF)
_ArrayDisplay($ar_Array)
$timerstamp1 = TimerInit()
$ar_Files = _FileListToArray ($sPath)
ConsoleWrite("Time _FileListToArray :" & Round(TimerDiff($timerstamp1)) & " mseconds to make" & @LF)
_ArrayDisplay($ar_Files)
$timerstamp1 = TimerInit()
$ar_Files2 = __FileListToArray ($sPath)
ConsoleWrite("Time __FileListToArray :" & Round(TimerDiff($timerstamp1)) & " mseconds to make" & @LF)
_ArrayDisplay($ar_Files2)
Func _FileBigListToArray($sPath, $sFilter = "*", $iFlag = 0, $sSize=1000)
    Local $hSearch, $sFile, $asFileList[$sSize], $rDimCnt=1, $rDimChk
    If Not FileExists($sPath) Then Return SetError(1, 1, "")
    If (StringInStr($sFilter, "\")) Or (StringInStr($sFilter, "/")) Or (StringInStr($sFilter, ":")) Or (StringInStr($sFilter, ">")) Or (StringInStr($sFilter, "<")) Or (StringInStr($sFilter, "|")) Or (StringStripWS($sFilter, 8) = "") Then Return SetError(2, 2, "")
    If Not ($iFlag = 0 Or $iFlag = 1 Or $iFlag = 2) Then Return SetError(3, 3, "")
    $hSearch = FileFindFirstFile($sPath & "\" & $sFilter)
    If $hSearch = -1 Then Return SetError(4, 4, "")
    While 1
        $sFile = FileFindNextFile($hSearch)
        If @error Then
            SetError(0)
            ExitLoop
        EndIf
        If $iFlag = 1 And StringInStr(FileGetAttrib($sPath & "\" & $sFile), "D") <> 0 Then ContinueLoop
        If $iFlag = 2 And StringInStr(FileGetAttrib($sPath & "\" & $sFile), "D") = 0 Then ContinueLoop
        ReDim $asFileList[UBound($asFileList) + 1]
        $rDimChk=Int($asFileList[0]/$rDimCnt)
        If $rDimChk>$sSize-1 Then
            $rDimCnt=$rDimCnt+1
            ReDim $asFileList[$rDimCnt*1000]
        EndIf
        $asFileList[0] = $asFileList[0] + 1
        $asFileList[$asFileList[0]] = $sFile
    WEnd
    FileClose($hSearch)
    ReDim $asFileList[$asFileList[0]+1]
    Return $asFileList
EndFunc  ;==>_FileListToArray
Func __FileListToArray($sPath, $sFilter = "*", $iFlag = 0)
    Local $i_PM, $s_AnswerFile = @ScriptDir & "\FindFiles.txt", $i_AMPM, $aArray[1], $sPathShort = $sPath
    If Not (StringRight($sPathShort, 1) == "\") Then $sPathShort &= "\"
    _RunDOS(' dir "' & $sPathShort & '*.*"   /TW/O-d/4 /-C /a-d-h-s | FIND ' & '"/"' & '> "' & $s_AnswerFile & '"')
    $s_Line = FileReadLine($s_AnswerFile)
    If StringInStr($s_Line, " AM ") Or StringInStr($s_Line, " PM ") Then $i_PM = 1
    ;=============================================================
    $code = "Function FileListToArray()"
    $code &= @LF & '    Dim objFileSystem: Set objFileSystem = CreateObject( "Scripting.FileSystemObject") '
    $code &= @LF & "    Dim fso, fso1,f1, InFile, ReadString,FileArray()"
    $code &= @LF & '    Set InFile = objFileSystem.OpenTextFile("' & $s_AnswerFile & '", 1)'
    $code &= @LF & "    Dim intPointer, intUpper, intLower, varHolder: intPointer=0"
    $code &= @LF & "    ReDim Preserve FileArray(0)"
    $code &= @LF & "    Do While Not InFile.AtEndOfStream"
    $code &= @LF & "        ReDim Preserve FileArray(ubound(FileArray)+1)"
    $code &= @LF & "        ReadString = InFile.ReadLine"
    If $i_PM Then $code &= @LF & 'FileArray(intPointer) =left(ReadString,10)&"|"&mid(ReadString,40)&"|"&mid(ReadString,13,8)&"|"&round(CSng(mid(ReadString,21,18))/1024)&"|"&right(ReadString,4) '
    If Not $i_PM Then $code &= @LF & 'FileArray(intPointer) =left(ReadString,10)&"|"&mid(ReadString,37)&"|"&mid(ReadString,13,5)&"|"&round(CSng(mid(ReadString,18,18))/1024)&"|"&right(ReadString,4) '
    $code &= @LF & "        intPointer=intPointer+1"
    $code &= @LF & "    Loop"
    $code &= @LF & "    InFile.Close"
    $code &= @LF & "    ReDim Preserve FileArray(ubound(FileArray)-1)"
    $code &= @LF & "    FileListToArray=FileArray"
    $code &= @LF & "End function   "
    $vbs = ObjCreate("ScriptControl")
    $vbs.language = "vbscript"
    $vbs.addcode ($code)
    $aArray = $vbs.Run ("FileListToArray")
    $vbs = ""
    Return $aArray
EndFunc   ;==>__FileListToArray

what do you think?

Randall

Edited by randallc

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

Hi,

Not sure it's any faster despite the theory;

;_FileBigListToArray.au3
#include<array.au3>
#include<file.au3>
#include<process.au3>
$sPath=@ScriptDir
;~ #include "_GUILVSort4.au3"
$timerstamp1 = TimerInit()
$ar_Array=_FileBigListToArray($sPath)
ConsoleWrite("Time _FileBigListToArray :" & Round(TimerDiff($timerstamp1)) & " mseconds to make" & @LF)
_ArrayDisplay($ar_Array)
$timerstamp1 = TimerInit()
$ar_Files = _FileListToArray ($sPath)
ConsoleWrite("Time _FileListToArray :" & Round(TimerDiff($timerstamp1)) & " mseconds to make" & @LF)
_ArrayDisplay($ar_Files)
$timerstamp1 = TimerInit()
$ar_Files2 = __FileListToArray ($sPath)
ConsoleWrite("Time __FileListToArray :" & Round(TimerDiff($timerstamp1)) & " mseconds to make" & @LF)
_ArrayDisplay($ar_Files2)
Func _FileBigListToArray($sPath, $sFilter = "*", $iFlag = 0, $sSize=1000)
    Local $hSearch, $sFile, $asFileList[$sSize], $rDimCnt=1, $rDimChk
    If Not FileExists($sPath) Then Return SetError(1, 1, "")
    If (StringInStr($sFilter, "\")) Or (StringInStr($sFilter, "/")) Or (StringInStr($sFilter, ":")) Or (StringInStr($sFilter, ">")) Or (StringInStr($sFilter, "<")) Or (StringInStr($sFilter, "|")) Or (StringStripWS($sFilter, 8) = "") Then Return SetError(2, 2, "")
    If Not ($iFlag = 0 Or $iFlag = 1 Or $iFlag = 2) Then Return SetError(3, 3, "")
    $hSearch = FileFindFirstFile($sPath & "\" & $sFilter)
    If $hSearch = -1 Then Return SetError(4, 4, "")
    While 1
        $sFile = FileFindNextFile($hSearch)
        If @error Then
            SetError(0)
            ExitLoop
        EndIf
        If $iFlag = 1 And StringInStr(FileGetAttrib($sPath & "\" & $sFile), "D") <> 0 Then ContinueLoop
        If $iFlag = 2 And StringInStr(FileGetAttrib($sPath & "\" & $sFile), "D") = 0 Then ContinueLoop
        ReDim $asFileList[UBound($asFileList) + 1]
        $rDimChk=Int($asFileList[0]/$rDimCnt)
        If $rDimChk>$sSize-1 Then
            $rDimCnt=$rDimCnt+1
            ReDim $asFileList[$rDimCnt*1000]
        EndIf
        $asFileList[0] = $asFileList[0] + 1
        $asFileList[$asFileList[0]] = $sFile
    WEnd
    FileClose($hSearch)
    ReDim $asFileList[$asFileList[0]+1]
    Return $asFileList
EndFunc  ;==>_FileListToArray
Func __FileListToArray($sPath, $sFilter = "*", $iFlag = 0)
    Local $i_PM, $s_AnswerFile = @ScriptDir & "\FindFiles.txt", $i_AMPM, $aArray[1], $sPathShort = $sPath
    If Not (StringRight($sPathShort, 1) == "\") Then $sPathShort &= "\"
    _RunDOS(' dir "' & $sPathShort & '*.*"   /TW/O-d/4 /-C /a-d-h-s | FIND ' & '"/"' & '> "' & $s_AnswerFile & '"')
    $s_Line = FileReadLine($s_AnswerFile)
    If StringInStr($s_Line, " AM ") Or StringInStr($s_Line, " PM ") Then $i_PM = 1
    ;=============================================================
    $code = "Function FileListToArray()"
    $code &= @LF & '    Dim objFileSystem: Set objFileSystem = CreateObject( "Scripting.FileSystemObject") '
    $code &= @LF & "    Dim fso, fso1,f1, InFile, ReadString,FileArray()"
    $code &= @LF & '    Set InFile = objFileSystem.OpenTextFile("' & $s_AnswerFile & '", 1)'
    $code &= @LF & "    Dim intPointer, intUpper, intLower, varHolder: intPointer=0"
    $code &= @LF & "    ReDim Preserve FileArray(0)"
    $code &= @LF & "    Do While Not InFile.AtEndOfStream"
    $code &= @LF & "        ReDim Preserve FileArray(ubound(FileArray)+1)"
    $code &= @LF & "        ReadString = InFile.ReadLine"
    If $i_PM Then $code &= @LF & 'FileArray(intPointer) =left(ReadString,10)&"|"&mid(ReadString,40)&"|"&mid(ReadString,13,8)&"|"&round(CSng(mid(ReadString,21,18))/1024)&"|"&right(ReadString,4) '
    If Not $i_PM Then $code &= @LF & 'FileArray(intPointer) =left(ReadString,10)&"|"&mid(ReadString,37)&"|"&mid(ReadString,13,5)&"|"&round(CSng(mid(ReadString,18,18))/1024)&"|"&right(ReadString,4) '
    $code &= @LF & "        intPointer=intPointer+1"
    $code &= @LF & "    Loop"
    $code &= @LF & "    InFile.Close"
    $code &= @LF & "    ReDim Preserve FileArray(ubound(FileArray)-1)"
    $code &= @LF & "    FileListToArray=FileArray"
    $code &= @LF & "End function   "
    $vbs = ObjCreate("ScriptControl")
    $vbs.language = "vbscript"
    $vbs.addcode ($code)
    $aArray = $vbs.Run ("FileListToArray")
    $vbs = ""
    Return $aArray
EndFunc   ;==>__FileListToArray

what do you think?

Randall

Damn, Thats fast. :) nice use of VBscript. Edited by SolidSnake

HKTunes:Softpedia | GoogleCodeLyricToy:Softpedia | GoogleCodeRCTunes:Softpedia | GoogleCodeMichtaToolsProgrammer n. - An ingenious device that turns caffeine into code.

Share this post


Link to post
Share on other sites

Hi,

Not sure it's any faster despite the theory;

$sPath=@ScriptDir

what do you think?

Just wondering, how many files in @ScriptDir?

The folder that started this has close on 3000 files in it. How about trying it on a folder with loads of files to really impress me :)

Looks like some funky VB coding ... way beyond my humble skills :)

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

Again, though your idea can be blistering fast for the detail it requires, with further mods...?

Time _FileBigListToArray2 :122 mseconds to make

Time _FileListToArray :7314 mseconds to make

Time __FileListToArray :811 mseconds to make

;_FileBigListToArray.au3
#include<array.au3>
#include<file.au3>
#include<process.au3>
$sPath=@ScriptDir
;~ #include "_GUILVSort4.au3"
$timerstamp1 = TimerInit()
$ar_Array=_FileBigListToArray2($sPath)
ConsoleWrite("Time _FileBigListToArray 2:" & Round(TimerDiff($timerstamp1)) & " mseconds to make" & @LF)
_ArrayDisplay($ar_Array)
$timerstamp1 = TimerInit()
$ar_Files = _FileListToArray ($sPath)
ConsoleWrite("Time _FileListToArray :" & Round(TimerDiff($timerstamp1)) & " mseconds to make" & @LF)
_ArrayDisplay($ar_Files)
$timerstamp1 = TimerInit()
$ar_Files2 = __FileListToArray ($sPath)
ConsoleWrite("Time __FileListToArray :" & Round(TimerDiff($timerstamp1)) & " mseconds to make" & @LF)
_ArrayDisplay($ar_Files2)
Func _FileBigListToArray2($sPath, $sFilter = "*", $iFlag = 0, $sSize=1000)
    Local $hSearch, $sFile, $asFileList[$sSize], $rDimCnt=1, $rDimChk
    If Not FileExists($sPath) Then Return SetError(1, 1, "")
    If (StringInStr($sFilter, "\")) Or (StringInStr($sFilter, "/")) Or (StringInStr($sFilter, ":")) Or (StringInStr($sFilter, ">")) Or (StringInStr($sFilter, "<")) Or (StringInStr($sFilter, "|")) Or (StringStripWS($sFilter, 8) = "") Then Return SetError(2, 2, "")
    If Not ($iFlag = 0 Or $iFlag = 1 Or $iFlag = 2) Then Return SetError(3, 3, "")
    $hSearch = FileFindFirstFile($sPath & "\" & $sFilter)
    If $hSearch = -1 Then Return SetError(4, 4, "")
    While 1
        $sFile = FileFindNextFile($hSearch)
        If @error Then
            SetError(0)
            ExitLoop
        EndIf
        If $iFlag = 1 And StringInStr(FileGetAttrib($sPath & "\" & $sFile), "D") <> 0 Then ContinueLoop
        If $iFlag = 2 And StringInStr(FileGetAttrib($sPath & "\" & $sFile), "D") = 0 Then ContinueLoop
        If $rDimCnt+1>UBound($asFileList)-1 Then ReDim $asFileList[$rDimCnt+1000]
        $asFileList[$rDimCnt] = $sFile
        $rDimCnt+=1
    WEnd
    FileClose($hSearch)
    $asFileList[0]=$rDimCnt
    ReDim $asFileList[$asFileList[0]]
    Return $asFileList
EndFunc  ;==>_FileListToArray
Func __FileListToArray($sPath, $sFilter = "*", $iFlag = 0)
    Local $i_PM, $s_AnswerFile = @ScriptDir & "\FindFiles.txt", $i_AMPM, $aArray[1], $sPathShort = $sPath
    If Not (StringRight($sPathShort, 1) == "\") Then $sPathShort &= "\"
    _RunDOS(' dir "' & $sPathShort & '*.*"   /TW/O-d/4 /-C /a-d-h-s | FIND ' & '"/"' & '> "' & $s_AnswerFile & '"')
    $s_Line = FileReadLine($s_AnswerFile)
    If StringInStr($s_Line, " AM ") Or StringInStr($s_Line, " PM ") Then $i_PM = 1
    ;=============================================================
    $code = "Function FileListToArray()"
    $code &= @LF & '    Dim objFileSystem: Set objFileSystem = CreateObject( "Scripting.FileSystemObject") '
    $code &= @LF & "    Dim fso, fso1,f1, InFile, ReadString,FileArray()"
    $code &= @LF & '    Set InFile = objFileSystem.OpenTextFile("' & $s_AnswerFile & '", 1)'
    $code &= @LF & "    Dim intPointer, intUpper, intLower, varHolder: intPointer=0"
    $code &= @LF & "    ReDim Preserve FileArray(0)"
    $code &= @LF & "    Do While Not InFile.AtEndOfStream"
    $code &= @LF & "        ReDim Preserve FileArray(ubound(FileArray)+1)"
    $code &= @LF & "        ReadString = InFile.ReadLine"
    If $i_PM Then $code &= @LF & 'FileArray(intPointer) =left(ReadString,10)&"|"&mid(ReadString,40)&"|"&mid(ReadString,13,8)&"|"&round(CSng(mid(ReadString,21,18))/1024)&"|"&right(ReadString,4) '
    If Not $i_PM Then $code &= @LF & 'FileArray(intPointer) =left(ReadString,10)&"|"&mid(ReadString,37)&"|"&mid(ReadString,13,5)&"|"&round(CSng(mid(ReadString,18,18))/1024)&"|"&right(ReadString,4) '
    $code &= @LF & "        intPointer=intPointer+1"
    $code &= @LF & "    Loop"
    $code &= @LF & "    InFile.Close"
    $code &= @LF & "    ReDim Preserve FileArray(ubound(FileArray)-1)"
    $code &= @LF & "    FileListToArray=FileArray"
    $code &= @LF & "End function   "
    $vbs = ObjCreate("ScriptControl")
    $vbs.language = "vbscript"
    $vbs.addcode ($code)
    $aArray = $vbs.Run ("FileListToArray")
    $vbs = ""
    Return $aArray
EndFunc   ;==>__FileListToArray
Randall! Edited by randallc

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

ok, so who's gonna get it to recurse too? :)

Edited by Rick

Who needs puzzles when we have AutoIt!!

Share this post


Link to post
Share on other sites

Here, I'll jump in the mix since you mentioned recurse: _FileListToArrayEx


Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Share this post


Link to post
Share on other sites

Here, I'll jump in the mix since you mentioned recurse: _FileListToArrayEx

nice, but i'm after one that doesnt go to dos, as if someone has ZoneAlarm a warning comes up as to system resources trying to be used.


Who needs puzzles when we have AutoIt!!

Share this post


Link to post
Share on other sites

nice, but i'm after one that doesnt go to dos, as if someone has ZoneAlarm a warning comes up as to system resources trying to be used.

I have zonealarm suite... it does in fact do that, but you are going to find VERY quickly, this is an automation language, and alot of the functions you use are going to bring that up. You'll never be able to use @Comspec as an example.

Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Share this post


Link to post
Share on other sites

ok, t'was just a thought


Who needs puzzles when we have AutoIt!!

Share this post


Link to post
Share on other sites

#12 ·  Posted (edited)

Here, I'll jump in the mix since you mentioned recurse: _FileListToArrayEx

Hi,

Time _FileBigListToArray :124 mseconds to make

Time __FileListToArrayEX :351 mseconds to make

Time $iFlag=0 _FileListToArrayExRecursive, defaults :168 mseconds to make

Time $iFlag=0 _FileListToArrayExRecursive , Recursive :194 mseconds to make

1. Thanks for that; I tried testing, and got your first version going slowly? Was that expected? OK!

2. Your first version returns folders, but not recursive?; is that expected?

3. The recursive version submitted by "Waffle" in post 13 does not return files in current folder, only folder names , then folder names and files in subfolders; looks good and fast, but maybe that's why?; it should be fast if fixed, though! [?works in the example on @homeDir, but not in my example of @scriptDir]

4. Do you think the mods I made to "_BigFileListToArray" are faulty in any way [i just modified quickly and have not looked at the original function yet], as it is 70x faster?; if not, we could submit for standard as it might have been mis-written to start?

@Rick; I think there are recursive versions out there in AutoIt, and wouldn't be hard [for you?] to write [and might be quite quick and possible in "pure" AutoIt?].

Best, randall

Edited by randallc

Share this post


Link to post
Share on other sites

4. Do you think the mods I made to "_BigFileListToArray" are faulty in any way [i just modified quickly and have not looked at the original function yet], as it is 70x faster?; if not, we could submit for standard as it might have been mis-written to start?

Looks ok after a quick inspection. I think I might just have to implement these mods into my code :) Only thing I noticed (and this was in my code as well ... oops) is that we are ReDim'ing and extra 1000 each time, rather than $sSize

Now ... I really need to find a blistering fast array sort function. Maybe I'll do a search, because it's been a while since I looked on here.

Share this post


Link to post
Share on other sites

@Rick; I think there are recursive versions out there in AutoIt, and wouldn't be hard [for you?] to write [and might be quite quick and possible in "pure" AutoIt?].

Best, randall

Well true, i must admit my prefered script is _GetFileList.au3, cant remember who wrote it but with a bit of tinkering it works for me, i was just being lazy lol


Who needs puzzles when we have AutoIt!!

Share this post


Link to post
Share on other sites

Looks ok after a quick inspection. I think I might just have to implement these mods into my code :) Only thing I noticed (and this was in my code as well ... oops) is that we are ReDim'ing and extra 1000 each time, rather than $sSize

Now ... I really need to find a blistering fast array sort function. Maybe I'll do a search, because it's been a while since I looked on here.

Hi,

Here's a vbs sort too;

don't forget you can get them back sorted by name, size, ext etc from DOS as in Rick's ref to _FileGetList, so you don't need to sort.

Randall

Time _FileBigListToArray :125 mseconds to make

Time _FileBigListSortArray :sorted _ArraySort1026 mseconds to make

Time _FileBigListSort Array _Quick_SortColumn:125 mseconds to make

;_FileBigListSortArray.au3
#include<array.au3>
#include<file.au3>
#include<process.au3>
$sPath=@ScriptDir
;~ #include "_GUILVSort4.au3"
$timerstamp1 = TimerInit()
$ar_Array=_FileBigListToArray2($sPath)
ConsoleWrite("Time _FileBigListToArray :" & Round(TimerDiff($timerstamp1)) & " mseconds to make" & @LF)
_ArrayDisplay($ar_Array,"_FileBigListToArray :")
_ArrayReverse($ar_Array,1)
_ArrayDisplay($ar_Array,"_FileBigListToArray :Reversed")
$ar_Array1=$ar_Array
$timerstamp1 = TimerInit()
_ArraySort($ar_Array1,0,1)
ConsoleWrite("Time _FileBigListSortArray :sorted _ArraySort" & Round(TimerDiff($timerstamp1)) & " mseconds to make" & @LF)
_ArrayDisplay($ar_Array1,"_ArraySort($ar_Array1,1)")
$ar_Array2=$ar_Array
$timerstamp1 = TimerInit()
_Quick_SortColumn($ar_Array2, 1, UBound($ar_Array2)-1);_Quick_SortColumn(ByRef $ar_Array, $First, $Last, $i_ViewColNum = 1)
ConsoleWrite("Time _FileBigListSort Array _Quick_SortColumn:" & Round(TimerDiff($timerstamp1)) & " mseconds to make" & @LF)
_ArrayDisplay($ar_Array2,"Time _FileBigListSortArray _Quick_SortColumn:")
Func _FileBigListToArray2($sPath, $sFilter = "*", $iFlag = 0, $sSize=1000)
    Local $hSearch, $sFile, $asFileList[$sSize], $rDimCnt=1, $rDimChk
    If Not FileExists($sPath) Then Return SetError(1, 1, "")
    If (StringInStr($sFilter, "\")) Or (StringInStr($sFilter, "/")) Or (StringInStr($sFilter, ":")) Or (StringInStr($sFilter, ">")) Or (StringInStr($sFilter, "<")) Or (StringInStr($sFilter, "|")) Or (StringStripWS($sFilter, 8) = "") Then Return SetError(2, 2, "")
    If Not ($iFlag = 0 Or $iFlag = 1 Or $iFlag = 2) Then Return SetError(3, 3, "")
    $hSearch = FileFindFirstFile($sPath & "\" & $sFilter)
    If $hSearch = -1 Then Return SetError(4, 4, "")
    While 1
        $sFile = FileFindNextFile($hSearch)
        If @error Then
            SetError(0)
            ExitLoop
        EndIf
        If $iFlag = 1 And StringInStr(FileGetAttrib($sPath & "\" & $sFile), "D") <> 0 Then ContinueLoop
        If $iFlag = 2 And StringInStr(FileGetAttrib($sPath & "\" & $sFile), "D") = 0 Then ContinueLoop
        If $rDimCnt+1>UBound($asFileList)-1 Then ReDim $asFileList[$rDimCnt+$sSize]
        $asFileList[$rDimCnt] = $sFile
        $rDimCnt+=1
    WEnd
    FileClose($hSearch)
    $asFileList[0]=$rDimCnt
    ReDim $asFileList[$asFileList[0]]
    Return $asFileList
EndFunc  ;==>_FileListToArray
Func _Quick_SortColumn(ByRef $ar_Array, $First, $Last, $i_ViewColNum = 1)
    If (Not IsArray($ar_Array)) Or $i_ViewColNum > UBound($ar_Array) Or $i_ViewColNum < 1 Then
        SetError(1)
        Return ""
    EndIf
    If UBound($ar_Array) = 1 Then; If the array is only 1 element in size then we can't sort the 1 element.
        SetError(2)
        Return ""
    EndIf
    Local $vbs, $code = "function Quick_SortColumn1(ByRef SortArray, First, Last,i_ViewColNum)"
    $code &= @LF & "    dim strWrite,ar_Col,ar_Column()"
    $code &= @LF & "    ReDim Preserve ar_Column(ubound(SortArray))"
    $code &= @LF & "    Dim intPointer, booIsNumeric: booIsNumeric = True"
    $code &= @LF & "    For intPointer = First To Last"
    $code &= @LF & '        ar_Col = Split( SortArray(intPointer), "|", -1,0 )'
    $code &= @LF & "        If Not IsNumeric( ar_Col(i_ViewColNum) ) Then"
    $code &= @LF & "            booIsNumeric = False"
    $code &= @LF & "            Exit For"
    $code &= @LF & "        End If"
    $code &= @LF & "    Next"
    $code &= @LF & "    For intPointer = First To Last"
    $code &= @LF & '        ar_Col = Split( SortArray(intPointer), "|", -1,0 )'
    $code &= @LF & '            If booIsNumeric Then'
    $code &= @LF & "            ar_Column(intPointer)  = CSng( ar_Col(i_ViewColNum) )"
    $code &= @LF & "        else"
    $code &= @LF & "            ar_Column(intPointer)  =  ar_Col(i_ViewColNum)"
    $code &= @LF & "        End If"
    $code &= @LF & "    Next"
    $code &= @LF & "    Quick_SortColumn  SortArray, ar_Column,First, Last,i_ViewColNum "
    $code &= @LF & "    Quick_SortColumn1= SortArray"
    $code &= @LF & "End function   "
    $code &= @LF & "function Quick_SortColumn(ByRef SortArray, ByRef ar_Column, First, Last,i_ViewColNum)"
    $code &= @LF & "    dim Low,High,collitem,ar_Col"
    $code &= @LF & "    dim Temp,List_Separator"
    $code &= @LF & "    Low = First"
    $code &= @LF & "    High = Last"
    $code &= @LF & "    List_Separator=ar_Column((First + Last) / 2)"
    $code &= @LF & "    Do"
    $code &= @LF & "        While (ar_Column(Low) < List_Separator)"
    $code &= @LF & "            Low = Low + 1"
    $code &= @LF & "        WEnd"
    $code &= @LF & "        While (ar_Column(High) > List_Separator)"
    $code &= @LF & "            High = High - 1"
    $code &= @LF & "        WEnd"
    $code &= @LF & "        If (Low <= High) Then"
    $code &= @LF & "            Temp = SortArray(Low)"
    $code &= @LF & "            SortArray(Low) = SortArray(High)"
    $code &= @LF & "            SortArray(High) = Temp"
    $code &= @LF & "            Temp = ar_Column(Low)"
    $code &= @LF & "            ar_Column(Low) = ar_Column(High)"
    $code &= @LF & "            ar_Column(High) = Temp"
    $code &= @LF & "            Low = Low + 1"
    $code &= @LF & "            High = High - 1"
    $code &= @LF & "        End If"
    $code &= @LF & "    Loop While (Low <= High)"
    $code &= @LF & "    If (First < High) Then Quick_SortColumn  SortArray, ar_Column,First, High,i_ViewColNum "
    $code &= @LF & "    If (Low < Last) Then Quick_SortColumn  SortArray, ar_Column,Low, Last,i_ViewColNum "
    $code &= @LF & "End function   "
    $vbs = ObjCreate("ScriptControl")
    If @error Then ;Return 0; SetError(Hex(@error, 8));EndIf
        SetError(3)
        Return ""
    EndIf
    $vbs.language = "vbscript"
    $vbs.addcode ($code)
    $ar_Array = $vbs.Run ("Quick_SortColumn1", $ar_Array, $First, $Last, $i_ViewColNum - 1)
    If @error Then ;Return 0; SetError(Hex(@error, 8));EndIf
        SetError(4)
        Return ""
    EndIf
    $vbs = ""
    SetError(0)
    Return 1
EndFunc   ;==>_Quick_SortColumn

Share this post


Link to post
Share on other sites

#16 ·  Posted (edited)

Hi, actually I agree with the original poster, I tested 5 times and got the results.

6 ms _FileBigListToArray

8 ms _FileListToArray

555 ms __FileListToArray

windows xp pro sp2

Edited by mrRevoked

Don't bother, It's inside your monitor!------GUISetOnEvent should behave more like HotKeySet()

Share this post


Link to post
Share on other sites

Hi, actually I agree with the original poster, I tested 5 times and got the results.

6 ms _FileBigListToArray

8 ms _FileListToArray

555 ms __FileListToArray

windows xp pro sp2

A great script to use and faster ! thanks !

My question is how can i return the file names with full path listing including subdirectories :

ex ..

c:\mymapp1\first.doc

c:\mymapp1\undermapp\some.doc

c:\othermapp\last.doc

Share this post


Link to post
Share on other sites

#18 ·  Posted (edited)

Time _FileFixToArray :63 mseconds to make

Time _FileBigListToArray :4387 mseconds to make

Time _FileListToArray :4265 mseconds to make

Time __FileListToArray :335 mseconds to make

Func _FileFixToArray($sPath, $sFilter = "*", $iFlag = 0)
    Local $hSearch, $sFile, $asFileList[3000], $k = 0
    If Not FileExists($sPath) Then Return SetError(1, 1, "")
    If (StringInStr($sFilter, "\")) Or (StringInStr($sFilter, "/")) Or (StringInStr($sFilter, ":")) Or (StringInStr($sFilter, ">")) Or (StringInStr($sFilter, "<")) Or (StringInStr($sFilter, "|")) Or (StringStripWS($sFilter, 8) = "") Then Return SetError(2, 2, "")
    If Not ($iFlag = 0 Or $iFlag = 1 Or $iFlag = 2) Then Return SetError(3, 3, "")
    $hSearch = FileFindFirstFile($sPath & "\" & $sFilter)
    If $hSearch = -1 Then Return SetError(4, 4, "")
    While 1
        $sFile = FileFindNextFile($hSearch)
        If @error Then
            SetError(0)
            ExitLoop
        EndIf
        If $iFlag = 1 And StringInStr(FileGetAttrib($sPath & "\" & $sFile), "D") <> 0 Then ContinueLoop
        If $iFlag = 2 And StringInStr(FileGetAttrib($sPath & "\" & $sFile), "D") = 0 Then ContinueLoop
        $k += 1
        $asFileList[$k] = $sFile
    WEnd
    FileClose($hSearch)
    $asFileList[0] = $k
    ReDim $asFileList[$k]
    Return $asFileList
EndFunc   ;==>_FileFixToArray

Or basic one:

Time F2A :49 mseconds to make

Time _FileBigListToArray :4393 mseconds to make

Time _FileListToArray :4259 mseconds to make

Time __FileListToArray :337 mseconds to make

Func F2A($sPath)
    Local $hSearch, $sFile, $asFileList[3000], $k = 1
    If Not FileExists($sPath) Then Return
    $hSearch = FileFindFirstFile($sPath & "\*")
    If $hSearch = -1 Then Return
    While 1
        $sFile = FileFindNextFile($hSearch)
        If @error Then ExitLoop
        $k += 1
        $asFileList[$k - 1] = $sFile
    WEnd
    FileClose($hSearch)
    $asFileList[0] = UBound($asFileList) - 1
    ReDim $asFileList[$k]
    Return $asFileList
EndFunc  ;==>F2A

Ovoid

Edited by ovoid

Ovoid

Share this post


Link to post
Share on other sites

What happens when you go over 3000 files?

Share this post


Link to post
Share on other sites

#20 ·  Posted (edited)

Hi @photonbuddy - I agree

Did you see my comparison tester of these in ;

New File Search function, with recursive support?

What do you think?

Test your own inside the tester func.

Best, randall

Edited by randallc

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0