Improvement of included _FileListToArray function.

[GEOSoft]

In the case of anything that is using a disk read, consistant timing can be a bit difficult due to disk caching on the system. You will probably notice that the first test after system startup takes longer than subsequent tests.

[Beege]

@spiff59

You are right about the filters not workings. There has to be a way to arrange this.. Im going to think about for a while.

[Spiff59]

@spiff59

You are right about the filters not workings. There has to be a way to arrange this.. Im going to think about for a while.

A second loop is how post 175 ended up, something like:

expandcollapse popup

#include<array.au3>
#include<file.au3>

$Path = @SystemDir
;$Path = "C:\Program Files\Autoit3\AutoitX"
$Filter = "*"
$RetItemType = 0; 0 = files/folder, 1 = files only, 2 = folders only
$RetPathType = 0; 0 = files/folder name only , 1 = relative path, 2 = full path
$Recursive = 0

$repeat = 100

$timer = TimerInit()
For $j = 1 to $repeat
    $x = _FLTA_NonRecursiveRecursion($Path, $Filter, $RetItemType, $RetPathType, $Recursive)
Next
$t1 = TimerDiff ($timer)
_ArrayDisplay($x)

$timer = TimerInit()
For $j = 1 to $repeat
    $x = _FileListToArray($Path, $Filter, $RetItemType)
Next
$t2 = TimerDiff ($timer)
_ArrayDisplay($x)

MsgBox (0, "", "$iRetItemType = " & $RetItemType & @CRLF &"Ver1:  " & Int($t1/10)/100 & @CRLF & "Ver2:  " & Int($t2/10)/100)


;===============================================================================
Func _FLTA_NonRecursiveRecursion($sPath, $sFilter= "*", $iRetItemType = 0, $iRetPathType = 0, $bRecursive = False)
    Local $sRet = "", $sRetPath = ""
    $sPath = StringRegExpReplace($sPath, "[\\/]+\z", "")
    If Not FileExists($sPath) Then Return SetError(1, 1, "")
    If StringRegExp($sFilter, "[\\/ :> <\|]|(?s)\A\s*\z") Then Return SetError(2, 2, "")
    $sPath &= "\|"
    $sOrigPathLen = StringLen($sPath) - 1
    While $sPath
        $sCurrPathLen = StringInStr($sPath, "|") - 1
        $sCurrPath = StringLeft($sPath, $sCurrPathLen)
        $search = FileFindFirstFile($sCurrPath & $sFilter)
        If @error Then
            $sPath = StringTrimLeft($sPath, $sCurrPathLen + 1)
            ContinueLoop
        EndIf
        Switch $iRetPathType
            Case 1 ; relative path
                $sRetPath = StringTrimLeft($sCurrPath, $sOrigPathLen)
            Case 2 ; full path
                $sRetPath = $sCurrPath
        EndSwitch
        While 1
            $file = FileFindNextFile($search)
            If @error Then ExitLoop
            If ($iRetItemType + @extended = 2) Then ContinueLoop
            $sRet &= $sRetPath & $file & "|"
        WEnd
        FileClose($search)
        If $bRecursive Then
            $hSearch = FileFindFirstFile($sCurrPath & "*")
            While 1
                $file = FileFindNextFile($hSearch)
                If @error Then ExitLoop
                If @extended Then $sPath &= $sCurrPath & $file & "\|"
            WEnd
            FileClose($hSearch)
        EndIf
        $sPath = StringTrimLeft($sPath, $sCurrPathLen + 1)
    WEnd
    If Not $sRet Then Return SetError(4, 4, "")
    Return StringSplit(StringTrimRight($sRet, 1), "|")
EndFunc  ;==>_FLTA_NonRecursiveRecursion

42 lines

Edited July 8, 2009 by Spiff59

[Valik]

Spiff59, I can tell you right now, it's unlikely I will ever include any version of this function that contains multiple virtually identical while loops.

[MrCreatoR]

Looking at the first time in those functions i also did not understood, why to use many loops on one parameter?

I would do it like this (also added FileGetAttrib method to check if the current path is dir, so the function will work on latest final release):

expandcollapse popup

#include <Array.au3>
#include <File.au3>
;

;$Path = @SystemDir
$Path = @ProgramFilesDir & "\AutoIt3\AutoItX"
$Filter = "*"
$RetItemType = 1 ;0 = files/folder, 1 = files only, 2 = folders only
$RetPathType = 1 ;0 = files/folder name only , 1 = relative path, 2 = full path
$Recursive = 1

$x = _FLTA_NonRecursiveRecursion($Path, $Filter, $RetItemType, $RetPathType, $Recursive)

_ArrayDisplay($x)

;===============================================================================
Func _FLTA_NonRecursiveRecursion($sPath, $sFilter, $iRetItemType, $iRetPathType, $bRecursive)
    $sPath = StringRegExpReplace($sPath, "[\\/]+\z", "")
    
    If Not FileExists($sPath) Then Return SetError(1, 1, "")
    If StringRegExp($sFilter, "[\\/ :> <\|]|(?s)\A\s*\z") Then Return SetError(2, 2, "")
    
    $sPath &= "\|"
    Local $sRet = "", $sRetPath = "", $sOrigPathLen = StringLen($sPath) - 1
    
    While $sPath
        $sCurrPathLen = StringInStr($sPath, "|") - 1
        $sCurrPath = StringLeft($sPath, $sCurrPathLen)
        $hSearch = FileFindFirstFile($sCurrPath & $sFilter)
        
        If @error Then
            $sPath = StringTrimLeft($sPath, $sCurrPathLen + 1)
            ContinueLoop
        EndIf
        
        Switch $iRetPathType
            Case 1 ;Relative path
                $sRetPath = StringTrimLeft($sCurrPath, $sOrigPathLen)
            Case 2 ;Full path
                $sRetPath = $sCurrPath
        EndSwitch
        
        While 1
            $sNextFile = FileFindNextFile($hSearch)
            If @error Then ExitLoop
            
            Switch $iRetItemType
                Case 1 ;Files only
                    If @extended = 0 And Not StringInStr(FileGetAttrib($sCurrPath & $sNextFile), "D") Then _
                        $sRet &= $sRetPath & $sNextFile & "|"
                Case 2 ;Folders only
                    If @extended Or StringInStr(FileGetAttrib($sCurrPath & $sNextFile), "D") Then _
                        $sRet &= $sRetPath & $sNextFile & "|"
                Case Else ;Files and Folders
                    $sRet &= $sRetPath & $sNextFile & "|"
            EndSwitch
        WEnd
        
        FileClose($hSearch)
        
        If $bRecursive Then
            $hSearch = FileFindFirstFile($sCurrPath & "*")
            
            While 1
                $sNextFile = FileFindNextFile($hSearch)
                If @error Then ExitLoop
                
                If @extended Or StringInStr(FileGetAttrib($sCurrPath & $sNextFile), "D") Then _
                    $sPath &= $sCurrPath & $sNextFile & "\|"
            WEnd
            
            FileClose($hSearch)
        EndIf
        
        $sPath = StringTrimLeft($sPath, $sCurrPathLen + 1)
    WEnd
    
    Return StringSplit(StringTrimRight($sRet, 1), "|")
EndFunc ;==>_FLTA_NonRecursiveRecursion

[Spiff59]

Spiff59, I can tell you right now, it's unlikely I will ever include any version of this function that contains multiple virtually identical while loops.

Otay. Prior post updated. It would have been 56 lines with the one-liner to correct the production bug. It's a bit slower, but is down to 43 lines. It has optional folder recursion, a choice of what type of pathnames are returned (Zedna's BugTrac), the fix for the production version, is backwards compatible, and still much faster than the 3.3.1.1 version.

It is also, of course untested, and I've chosen to not edit the $iRetItemType and $iRetPathType variables and instead default invalid values. I'm not sure how that practice will be received.

[Danny35d]

Need more testing, but this one is a combination of Wus information and bchris01 script.

expandcollapse popup

; #FUNCTION# ================================================================================
; Name...........: _FileListToArrayEx
; Description ...: Lists files and\or folders in a specified path (Similar to using Dir with the /B Switch)
; Syntax.........: _FileListToArrayEx($sPath[, $sFilter = '*'[, $iFlag = 0, $iPathType = 1, $iRecursive = 0, $sExclude = '', $iRetFormat = 1]])
; Parameters ....: $sPath   - Path to generate filelist for.
;                  $sFilter - Optional: the filter to use, default is *, semicolon delimited. Search the Autoit3 helpfile for the word 'WildCards' For details.
;                             (Example: '*.exe;*.txt')
;                  $iFlag   - Optional: specifies whether to return files folders or both
;                    0 = (Default) Return both files and folders
;                    1 = Return files only
;                    2 = Return Folders only
;                  $iPathType
;                    0 = relative path
;                    1 = (Default) full path
;                  $iRecursive - Search files in specified directory and all subdirectories.
;                    0 = (Default) Search in specified directory.
;                    1 = Search in specified and all subdirectories.
;                  $sExclude = optional: Exclude filter(s), semicolon delimited. Wildcards allowed.
;                    (Example: 'Unins*' will remove all files/folders that begin with 'Unins')
;                  $iRetFormat =  optional: return format
;                    0 = String ( '|' delimited)
;                    1 = (Default) one-dimensional array, 1-based
;                    2 = one-dimensional array, 0-based
;                  $sDelim - Delimiter for combined string.
; Return values .: @Error -
;                    1 = Path not found or invalid
;                    2 = Invalid $sFilter
;                    3 = Invalid $iFlag
;                    4 = No File(s) Found
;                    5 = Invalid $iPathType
;                    6 = Invalid $iRecursive
;                    7 = Invalid $iRetFormat
; Author ........:
; Modified.......:
; Remarks .......: The array returned is one-dimensional and is made up as follows:
;                    $array[0] = Number of Files\Folders returned
;                    $array[1] = 1st File\Folder
;                    $array[2] = 2nd File\Folder
;                    $array[3] = 3rd File\Folder
;                    $array[n] = nth File\Folder
; Related .......:
; Link ..........;
; Example .......; Yes
; ===========================================================================================

Func _FileListToArrayEx($sPath, $sFilter = '*', $iFlag = 0, $iPathType = 1, $bRecursive = 0, $sExclude = '', $iRetFormat = 1, $sDelim = '|')
    Local $sFile, $hSearch, $sWorkingdir, $asFilelist, $aFolderStack[2]

    $sPath = StringRegExpReplace($sPath, '[\\/]+\z', '')
    If Not FileExists($sPath) Then Return SetError(1, 1, '')
    If $sFilter = '' Or $sFilter = '-1' Or $sFilter = Default Then $sFilter = '*'
    If StringRegExp($sFilter, '[:\\</>\|]|(?s)\A\s*\z') Then Return SetError(2, 2, '')
    If Not StringRegExp($iFlag, '[012]') Then Return SetError(3, 3, '')
    If Not StringRegExp($iPathType, '[01]') Then Return SetError(5, 5, '')
    If Not StringRegExp($bRecursive, '[01]') Then Return SetError(6, 6, '')
    If Not StringRegExp($iRetFormat, '[012]') Then Return SetError(7, 7, '')
    $sFilter = StringReplace(StringStripWS(StringRegExpReplace($sFilter, '\s*;\s*', ';'), 3), ';', '|')
    $sMask = '(?i)^' & StringReplace(StringReplace(StringReplace($sFilter, '.', '\.'), '*', '.*'), '?', '.')
    $sExclude = StringStripWS(StringRegExpReplace($sExclude, '\s*;\s*', ';'), 3)
    $sExclude = StringRegExpReplace($sExclude, '([\Q\.+[^]$(){}=!\E])', '\\$1')
    $sExclude = '(?i)\A' & StringReplace(StringReplace(StringReplace($sExclude, '?', '.'), '*', '.*?'), ';', '|') & '\z'

    $aFolderStack[0] = 1
    $aFolderStack[1] = $sPath
    While $aFolderStack[0] > 0
        $sWorkingdir = $aFolderStack[$aFolderStack[0]]
        $aFolderStack[0] -= 1
        $hSearch = FileFindFirstFile($sWorkingdir & '\*')
        While 1
            $sFile = FileFindNextFile($hSearch)
            If @error Then ExitLoop
            If @extended Or StringInStr(FileGetAttrib($sWorkingdir & '\' & $sFile), 'D') Then
                If $bRecursive Then ; Increse the FolderStack
                    If $sExclude <> '' And StringRegExp(StringRegExpReplace($sWorkingdir & '\' & $sFile, '(.*?[\\/]+)*(.*?\z)', '\2'), $sExclude) Then ContinueLoop
                    $aFolderStack[0] += 1
                    If UBound($aFolderStack) <= $aFolderStack[0] Then ReDim $aFolderStack[UBound($aFolderStack) * 2]
                    $aFolderStack[$aFolderStack[0]] = $sWorkingdir & '\' & $sFile
                EndIf
                If $iFlag <> 1 Then ; Add Folders
                    If Not $bRecursive And $sExclude <> '' And StringRegExp(StringRegExpReplace($sWorkingdir & '\' & $sFile, '(.*?[\\/]+)*(.*?\z)', '\2'), $sExclude) Then ContinueLoop
                    If Not StringRegExp($sFile, $sMask) Then ContinueLoop
                    $asFilelist &= $sWorkingdir & '\' & $sFile & $sDelim
                EndIf
            Else ; Add Files
                If $iFlag <> 2 Then
                    If Not StringRegExp($sFile, $sMask) Then ContinueLoop
                    If Not StringRegExp(StringRegExpReplace($sFilter, '^.*\.', ''), '[?*]') Then
                        If StringInStr($sFilter, StringRegExpReplace($sFile, '^.*\.', '')) = 0 Then ContinueLoop
                    EndIf
                    If $sExclude <> '' And StringRegExp(StringRegExpReplace($sWorkingdir & '\' & $sFile, '(.*?[\\/]+)*(.*?\z)', '\2'), $sExclude) Then ContinueLoop
                    $asFilelist &= $sWorkingdir & '\' & $sFile & $sDelim
                EndIf
            EndIf
        WEnd
        FileClose($hSearch)
    WEnd

    $asFilelist = StringTrimRight($asFilelist, 1)
    If StringLen($asFilelist) = 0 Then Return SetError(4, 4, '')
    If $iPathType = 0 Then $asFilelist = StringReplace($asFilelist, $sPath & '\', '')
    If $iRetFormat <> 0 Then $asFilelist = StringSplit($asFilelist, $sDelim, $iRetFormat)
    Return $asFilelist
EndFunc   ;==>_FileListToArrayEx

Note: Fixed error if try to do a nonrecursive or recursive search for only folders using some kind of filter.

Edited July 28, 2009 by Danny35d

[Spiff59]

why to use many loops on one parameter?

Well, my reasoning was that when a large directory, or multiple large directories, are searched that the function spends the majority of it's time in the innermost while loop. Splitting the while loop into 3 parts eliminated from 2 or 3 compares per file/folder read. It made, if I recall, up to an additional 13% difference in performance depending on the value of $iRetItemType ($iFlag) and $sPath.

Edit: But I've decided to be happy without that particular tweak

Edited July 8, 2009 by Spiff59

[Beege]

Looking at the first time in those functions i also did not understood, why to use many loops on one parameter?

I would do it like this (also added FileGetAttrib method to check if the current path is dir, so the function will work on latest final release):

expandcollapse popup

#include <Array.au3>
#include <File.au3>
;

;$Path = @SystemDir
$Path = @ProgramFilesDir & "\AutoIt3\AutoItX"
$Filter = "*"
$RetItemType = 1 ;0 = files/folder, 1 = files only, 2 = folders only
$RetPathType = 1 ;0 = files/folder name only , 1 = relative path, 2 = full path
$Recursive = 1

$x = _FLTA_NonRecursiveRecursion($Path, $Filter, $RetItemType, $RetPathType, $Recursive)

_ArrayDisplay($x)

;===============================================================================
Func _FLTA_NonRecursiveRecursion($sPath, $sFilter, $iRetItemType, $iRetPathType, $bRecursive)
    $sPath = StringRegExpReplace($sPath, "[\\/]+\z", "")
    
    If Not FileExists($sPath) Then Return SetError(1, 1, "")
    If StringRegExp($sFilter, "[\\/ :> <\|]|(?s)\A\s*\z") Then Return SetError(2, 2, "")
    
    $sPath &= "\|"
    Local $sRet = "", $sRetPath = "", $sOrigPathLen = StringLen($sPath) - 1
    
    While $sPath
        $sCurrPathLen = StringInStr($sPath, "|") - 1
        $sCurrPath = StringLeft($sPath, $sCurrPathLen)
        $hSearch = FileFindFirstFile($sCurrPath & $sFilter)
        
        If @error Then
            $sPath = StringTrimLeft($sPath, $sCurrPathLen + 1)
            ContinueLoop
        EndIf
        
        Switch $iRetPathType
            Case 1 ;Relative path
                $sRetPath = StringTrimLeft($sCurrPath, $sOrigPathLen)
            Case 2 ;Full path
                $sRetPath = $sCurrPath
        EndSwitch
        
        While 1
            $sNextFile = FileFindNextFile($hSearch)
            If @error Then ExitLoop
            
            Switch $iRetItemType
                Case 1 ;Files only
                    If @extended = 0 And Not StringInStr(FileGetAttrib($sCurrPath & $sNextFile), "D") Then _
                        $sRet &= $sRetPath & $sNextFile & "|"
                Case 2 ;Folders only
                    If @extended Or StringInStr(FileGetAttrib($sCurrPath & $sNextFile), "D") Then _
                        $sRet &= $sRetPath & $sNextFile & "|"
                Case Else ;Files and Folders
                    $sRet &= $sRetPath & $sNextFile & "|"
            EndSwitch
        WEnd
        
        FileClose($hSearch)
        
        If $bRecursive Then
            $hSearch = FileFindFirstFile($sCurrPath & "*")
            
            While 1
                $sNextFile = FileFindNextFile($hSearch)
                If @error Then ExitLoop
                
                If @extended Or StringInStr(FileGetAttrib($sCurrPath & $sNextFile), "D") Then _
                    $sPath &= $sCurrPath & $sNextFile & "\|"
            WEnd
            
            FileClose($hSearch)
        EndIf
        
        $sPath = StringTrimLeft($sPath, $sCurrPathLen + 1)
    WEnd
    
    Return StringSplit(StringTrimRight($sRet, 1), "|")
EndFunc ;==>_FLTA_NonRecursiveRecursion

I don't think the filters are working on this script either..

[MrCreatoR]

I don't think the filters are working on this script either..

I didn't trying to fix anything related to filters, i just showed how i would do that instead of using many loops.

[Beege]

Okay this one should not give any major problems (I hope). All filters should be working for recursive and non-recursive. I did quite a bit or testing. This one again is only for beta. (31 lines)

Func _FileListToArrayEx100000($sPath, $sFilter = "*", $iFlag = 0, $bRecursive = False)
    Local $sFile, $hSearch, $sWorkingdir, $asFilelist
    If Not FileExists($sPath) Then Return SetError(1, 1, "")
    If StringRegExp($sFilter, "[\\/ :> <\|]|(?s)\A\s*\z") Then Return SetError(2, 2, "")
    If Not StringRegExp($iFlag, '[012]') Then Return SetError(3, 3, "")
    If $sFilter <> "*" Then $sFilter = "(?i)\A(" & StringReplace(StringReplace(StringReplace(StringRegExpReplace(StringRegExpReplace(StringRegExpReplace($sFilter, "(\s*;\s*)+", ";"), "\A;|;\z", ""), '[.$]', '\[\0\]'), "?", "."), "*", ".*?"), ";", "$|") & "$)"
    Local $aFolderStack[2] = [1, StringRegExpReplace($sPath, "[\\]+\z", "")]
    While $aFolderStack[0] > 0
        $sWorkingdir = $aFolderStack[$aFolderStack[0]]
        $aFolderStack[0] -= 1
        $hSearch = FileFindFirstFile($sWorkingdir & '\*')
        While 1
            $sFile = FileFindNextFile($hSearch)
            If @error Then ExitLoop
            If @extended Then
                If $bRecursive Then
                    $aFolderStack[0] += 1
                    If UBound($aFolderStack) <= $aFolderStack[0] Then ReDim $aFolderStack[UBound($aFolderStack) * 2]
                    $aFolderStack[$aFolderStack[0]] = $sWorkingdir & "\" & $sFile
                EndIf
                If $iFlag = 1 Or ($sFilter <> "*" And Not StringRegExp($sFile, $sFilter)) Then ContinueLoop
            Else
                If $iFlag = 2 Or ($sFilter <> "*" And Not StringRegExp($sFile, $sFilter)) Then ContinueLoop
            EndIf
            $asFilelist &= $sWorkingdir & "\" & $sFile & '|'
        WEnd
        FileClose($hSearch)
    WEnd
    If StringRight($asFilelist, 1) = '|' Then $asFilelist = StringTrimRight($asFilelist, 1)
    Return StringSplit($asFilelist, '|')
EndFunc   ;==>_FileListToArrayEx100000

Edited July 11, 2009 by bchris01

[Spiff59]

I thought I'd try whipping the problem of having both an include parameter, and an exclude parameter. It really causes a lot of grief when those are implemented as the combinations between them (and also the "files and/or folders" flag, and the manner in which FileFindFirstFile behaves) make for a lot of confusion. Just one type of example would be, if you want to list all the '*.exe' files on your drive, but exclude the 'Windows' directory, a file named 'WindowsPlus.exe' (for example) would be excluded, even though it resided in a folder other than 'Windows'.

Separate include and exclude parameters for files, and for folders, is the route I've tried. I think the following works pretty well. It accepts semi-colon delimited lists for both the includelist, the excludelist, and the EXCLUDEfolderlist. It scans recursively in a single pass, never rereading the same directory. The Regular Expression seems to function better than others I've seen so far. Try out '*.exe', '*.exe*', '*.exe.*' or '*.*.*' on your @systemdir.

expandcollapse popup

Func _FileListToArrayZ($sPath, $sExcludeFolderList = "", $sIncludeList = "*", $sExcludeList = "", $iReturnType = 0, $iReturnFormat = 0, $bRecursive = False)
    Local $sRet = "", $sReturnFormat = ""

    ; Edit include path (strip trailing slashes, and append single slash)
    $sPath = StringRegExpReplace($sPath, "[\\/]+\z", "") & "\"
    If Not FileExists($sPath) Then Return SetError(1, 1, "")

    ; Edit exclude folders list
    If $sExcludeFolderList Then
        ; Strip leading/trailing spaces and semi-colons, any adjacent semi-colons, and spaces surrounding semi-colons
        $sExcludeFolderList = StringRegExpReplace(StringRegExpReplace($sExcludeFolderList, "(\s*;\s*)+", ";"), "\A;|;\z", "")
        ; Convert to Regular Expression, step 1: Wrap brackets around . and $ (what other characters needed?)
        $sExcludeFolderList = StringRegExpReplace($sExcludeFolderList, '[.$]', '\[\0\]')
        ; Convert to Regular Expression, step 2: Convert '?' to '.', and '*' to '.*?'
        $sExcludeFolderList = StringReplace(StringReplace($sExcludeFolderList, "?", "."), "*", ".*?")
        ; Convert to Regular Expression, step 3; case-insensitive, convert ';' to '|', match from first char, terminate strings
        $sExcludeFolderList = "(?i)\A(?!" & StringReplace($sExcludeFolderList, ";", "$|")  & "$)"
    EndIf

    ; Edit include files list
    If $sIncludeList ="*" Then
        $sIncludeList = ""
    Else
        If StringRegExp($sIncludeList, "[\\/ :> <\|]|(?s)\A\s*\z") Then Return SetError(2, 2, "")
        ; Strip leading/trailing spaces and semi-colons, any adjacent semi-colons, and spaces surrounding semi-colons
        $sIncludeList = StringRegExpReplace(StringRegExpReplace($sIncludeList, "(\s*;\s*)+", ";"), "\A;|;\z", "")
        ; Convert to Regular Expression, step 1: Wrap brackets around . and $ (what other characters needed?)
        $sIncludeList = StringRegExpReplace($sIncludeList, '[.$]', '\[\0\]')
        ; Convert to Regular Expression, step 2: Convert '?' to '.', and '*' to '.*?'
        $sIncludeList = StringReplace(StringReplace($sIncludeList, "?", "."), "*", ".*?")
        ; Convert to Regular Expression, step 3; case-insensitive, convert ';' to '|', match from first char, terminate strings
        $sIncludeList = "(?i)\A(" & StringReplace($sIncludeList, ";", "$|")  & "$)"
    EndIf

    ; Edit exclude files list
    If $sExcludeList Then
        ; Strip leading/trailing spaces and semi-colons, any adjacent semi-colons, and spaces surrounding semi-colons
        $sExcludeList = StringRegExpReplace(StringRegExpReplace($sExcludeList, "(\s*;\s*)+", ";"), "\A;|;\z", "")
        ; Convert to Regular Expression, step 1: Wrap brackets around . and $ (what other characters needed?)
        $sExcludeList = StringRegExpReplace($sExcludeList, '[.$]', '\[\0\]')
        ; Convert to Regular Expression, step 2: Convert '?' to '.', and '*' to '.*?'
        $sExcludeList = StringReplace(StringReplace($sExcludeList, "?", "."), "*", ".*?")
        ; Convert to Regular Expression, step 3; case-insensitive, convert ';' to '|', match from first char, terminate strings
        $sExcludeList = "(?i)\A(?!" & StringReplace($sExcludeList, ";", "$|")  & "$)"
    EndIf

;   MsgBox(1,"Masks","File include: " & $sIncludeList & @CRLF & "File exclude: " & $ExcludeList & @CRLF _
;           & "Dir include : " & $FolderInclude & @CRLF & "Dir exclude : " & $ExcludeFolderList)

    If Not ($iReturnType = 0 Or $iReturnType = 1 Or $iReturnType = 2) Then Return SetError(3, 3, "")

    Local $sOrigPathLen = StringLen($sPath), $aQueue[64] = [1,$sPath], $iQMax = 63
    While $aQueue[0]
        $WorkFolder = $aQueue[$aQueue[0]]
        $aQueue[0] -= 1
        $search = FileFindFirstFile($WorkFolder & "*")
        If @error Then ContinueLoop
        Switch $iReturnFormat
            Case 1 ; relative path
                $sReturnFormat = StringTrimLeft($WorkFolder, $sOrigPathLen)
            Case 2 ; full path
                $sReturnFormat = $WorkFolder
        EndSwitch
        While 1
            $file = FileFindNextFile($search)
            If @error Then ExitLoop
            If @extended Then ; Folder
                If $sExcludeFolderList And Not StringRegExp($file, $sExcludeFolderList) Then ContinueLoop
                If $bRecursive Then
                    If $aQueue[0] = $iQMax Then
                        $iQMax += 128
                        ReDim $aQueue[$iQMax + 1]
                    EndIf
                    $aQueue[0] += 1
                    $aQueue[$aQueue[0]] = $WorkFolder & $file & "\"
                EndIf
                If $iReturnType = 1 Then ContinueLoop
            Else ; File
                If $iReturnType = 2 Then ContinueLoop
            EndIf
            If $sIncludeList And Not StringRegExp($file, $sIncludeList) Then ContinueLoop
            If $sExcludeList And Not StringRegExp($file, $sExcludeList) Then ContinueLoop
            $sRet &= $sReturnFormat & $file & "|"
        WEnd
        FileClose($search)
    WEnd
    If Not $sRet Then Return SetError(4, 4, "")
    Return StringSplit(StringTrimRight($sRet, 1), "|")
EndFunc

Edit: changed the name of the routine

Edited August 17, 2010 by Spiff59

[BaKaMu]

Based on some new suggestions, I have developed a following version of _FileListToArrayNT7 --> _FileListToArrayXT

Some preliminary remarks:

This version is designed for AutoIt Version 3.3.1.1 or newer.

Otherwise you have to change all --> @extended <-- back to --> StringInStr(FileGetAttrib($sPath & $sFile), "D") <> 0 <-- (not tested)

In no circumstances a recommendation will be made to include it in a general UDF. ;-)

So use it or not , it is your decision.

According to a suggestion from Spiff59 I have changed the search strategy in some cases:

old _FileListToArrayNT7 (post #175): filtered search for each filter (strategy 1).

If recursion (search in subdirs) is necessary then additional unfiltered searches for subdirs are done.

new _FileListToArrayXT:

if no recursion: search like _FileListToArrayNT7 (strategy 1).

if recursion: do an unfiltered search for all items and filter out unwanted items (strategy 2).

To my surprise, this combination of search strategies gives an impressive speed improvement (up to 100%),

especially if more than on filter is used (ex: "*.avi;*,mpg;*.flv;*.mov")

There are also two branches of code in each case of recursion (if excluding is set or not) for speed improvement.

(and due to the new pseudo-recursion the unpractical "do not use" parameter $sWorkPath could be removed)

This results in noticeable more code lines (please don't flame therefore), but the operating profit is again a really good speed improvement

(compared to the already fast _FileListToArrayNT7) in most of the cases.

The functionality is still the same as in _FileListToArrayNT7 (multi-path, multi-filter, recursion, exclude and so on)

Two small changes:

Excluding now depends on $iRetItemType: If files only, excluding is only done for file names, otherwise also for folder names.

Search for "*.dat" returns only *.dat files (so no *.data will be returned as in the case of dir command)

Example:

Search for all video files on disk C: and D: , but no videos with "demo" or "test" in filename:

$aRetArray = _FileListToArrayXT("C:\;D:\", "*.avi;*.mpg;*.mpeg;*.mp4;*.mkv;*.flv;*.ogg;*.ogm;*.mov", 1, 2, True, "*demo*;*test*")

The first run is as usual somewhat slower (filling cache), but the next runs will be very fast (reading from cache).

Some test results (all done out of cache):

Test1: path: System32-Dir, filter="*.dll", return files/folders, full path, no recursion, no exclusion, repeats 300

_FileListToArray: Sec = 7.70 , 1832 items

_FileListToArrayNT7: Sec = 4.61 , 1832 items

_FileListToArrayXT: Sec = 4.62 , 1832 items

Test2: path: System32-Dir, filter="*", return files/folders, full path, no recursion, no exclusion, repeats 300

_FileListToArray: Sec = 12.80 , 3114 items

_FileListToArrayNT7: Sec = 7.30 , 3114 items

_FileListToArrayXT: Sec = 7.31 , 3114 items

Test3: path: System32-Dir, filter="*nothing*.xyz", return files/folders, full path, no recursion, no exclusion, repeats 300

_FileListToArray: Sec = 0.53 , 0 items

_FileListToArrayNT7: Sec = 0.56 , 0 items

_FileListToArrayXT: Sec = 0.56 , 0 items

Test4: path: System32-Dir, filter="*.dll", return files/folders, full path, recursion, no exclusion, repeats 50

_FileListToArrayNT7: Sec = 7.52 , 2895 items

_FileListToArrayXT: Sec = 6.44 , 2895 items

Test5: path: System32-Dir, filter="*.dll;*.exe", return files/folders, full path, recursion, no exclusion, repeats 50

_FileListToArrayNT7: Sec = 9.14 , 3555 items

_FileListToArrayXT: Sec = 7.03 , 3555 items

Test6: path: System32-Dir, filter="*.dll;*.exe;*.sys;*.log", return files/folders, full path, recursion, no exclusion, repeats 50

_FileListToArrayNT7: Sec = 12.06 , 4180 items

_FileListToArrayXT: Sec = 7.88 , 4180 items

Test7: path: System32-Dir, filter="*", return files/folders, full path, recursion, exclusion: "*.dll;*.exe", repeats 50

_FileListToArrayNT7: Sec = 12.08 , 3669 items

_FileListToArrayXT: Sec = 8.16 , 3669 items

Test8: path: HDD C:, filter="*.avi;*.mpg;*.mpeg;*.mp4;*.mkv;*.flv;*.ogg;*.ogm;*.mov", return files, full path, recursion, exclusion: "*Demo*.exe", repeats 1

_FileListToArrayNT7: Sec = 16.16 , 116 items

_FileListToArrayXT: Sec = 8.72 , 116 items

Test9: path: Test-Dir (13608 files, 1092 folders), 6 filters, return files, full path, recursion, 1 exclusion, repeats 30

_FileListToArrayNT7: Sec = 36.94 , 10750 items

_FileListToArrayXT: Sec = 17.05 , 10750 items

As you can see, the performance isn't bad at all.

Please make your own tests to check stability and functionality and report if you find a bug.

_FileListToArrayXT (For Spiff59: _FileListToArraySuperTurbo5000ExtraPlusProWithMaximumUltraPowerboost)

expandcollapse popup

; #FUNCTION# ===========================================================================================
; Name:             _FileListToArrayXT
; Description:      Lists files and\or folders in specified path(s) (Similar to using Dir with the /B Switch)
;                   additional features: multi-path, multi-filter, multi-exclude-filter, path format options, recursive search
; Syntax:           _FileListToArrayXT([$sPath = @ScriptDir, [$sFilter = "*", [$iRetItemType, [$bRecursive = False, [$sExclude = "", [$iRetFormat = 1]]]]]])
; Parameter(s):     $sPath = optional: Search path(s), semicolon delimited (default: @ScriptDir)
;                            (Example: "C:\Tmp;D:\Temp")
;                   $sFilter = optional: Search filter(s), semicolon delimited . Wildcards allowed. (default: "*")
;                              (Example: "*.exe;*.txt")
;                   $iRetItemType = Include in search: 0 = Files and Folder, 1 = Files Only, 2 = Folders Only
;                   $iRetPathType = Returned element format: 0 = file/folder name only, 1 = relative path, 2 = full path
;                   $bRecursive = optional: True: recursive search including all subdirectories
;                                           False (default): search only in specified folder
;                   $sExclude = optional: Exclude filter(s), semicolon delimited. Wildcards allowed.
;                               (Example: "Unins*" will remove all files/folders that begin with "Unins")
;                   $iRetFormat =  optional: return format
;                                  0 = one-dimensional array, 0-based
;                                  1 = one-dimensional array, 1-based (default)
;                                  2 = String ( "|" delimited)
; Requirement(s):   AutoIt Version 3.3.1.1 or newer
; Return Value(s):  on success: 1-based or 0-based array or string (dependent on $iRetFormat)
;                   If no path is found, @error and @extended are set to 1, returns empty string
;                   If no filter is found, @error and @extended are set to 2, returns empty string
;                   If $iRetFormat is invalid, @error and @extended are set to 3, returns empty string
;                   If no data is found, @error and @extended are set to 4, returns empty string
; Author(s):        Half the AutoIt Community
; ====================================================================================================
Func _FileListToArrayXT($sPath = @ScriptDir, $sFilter = "*", $iRetItemType = 0, $iRetPathType = 0, $bRecursive = False, $sExclude = "", $iRetFormat = 1)
  Local $hSearchFile, $sFile, $sFileList, $sWorkPath, $sRetPath, $iRootPathLen, $iPCount, $iFCount, $fDirFlag

  ;[check and prepare parameters]
  ;---------------
  If $sPath = -1 Or $sPath = Default Then $sPath = @ScriptDir
  ;strip leading/trailing spaces and semi-colons, all adjacent semi-colons, and spaces surrounding semi-colons
  $sPath = StringRegExpReplace(StringRegExpReplace($sPath, "(\s*;\s*)+", ";"), "\A;|;\z", "")
  ;check that at least one path is set
  If $sPath = "" Then Return SetError(1, 1, "")
  ;-----
  If $sFilter = -1 Or $sFilter = Default Then $sFilter = "*"
  ;prepare filter
  ;strip leading/trailing spaces and semi-colons, all adjacent semi-colons, and spaces surrounding semi-colons
  $sFilter = StringRegExpReplace(StringRegExpReplace($sFilter, "(\s*;\s*)+", ";"), "\A;|;\z", "")
  ;check for invalid chars or that at least one filter is set
  If StringRegExp($sFilter, "[\\/><:\|]|(?s)\A\s*\z") Then Return SetError(2, 2, "")
  If $bRecursive Then
    ;Convert $sFilter for Regular Expression
    $sFilter = StringRegExpReplace($sFilter, '([\Q\.+[^]$(){}=!\E])', '\\$1')
    $sFilter = StringReplace($sFilter, "?", ".")
    $sFilter = StringReplace($sFilter, "*", ".*?")
    $sFilter = "(?i)\A(" & StringReplace($sFilter, ";", "$|") & "$)" ;case-insensitive, convert ';' to '|', match from first char, terminate strings
    ;$sFilter = "(?i)\A" & StringReplace($sFilter, ";", "|") & "\z"
  EndIf
  ;-----
  If $iRetItemType <> "1" And $iRetItemType <> "2" Then $iRetItemType = "0"
  ;-----
  If $iRetPathType <> "1" And $iRetPathType <> "2" Then $iRetPathType = "0"
  ;-----
  $bRecursive = ($bRecursive = "1")
  ;-----
  If $sExclude = -1 Or $sExclude = Default Then $sExclude = ""
  If $sExclude Then
    ;prepare $sExclude
    ;strip leading/trailing spaces and semi-colons, all adjacent semi-colons, and spaces surrounding semi-colons
    $sExclude = StringRegExpReplace(StringRegExpReplace($sExclude, "(\s*;\s*)+", ";"), "\A;|;\z", "")
    ;Convert $sExclude for Regular Expression
    $sExclude = StringRegExpReplace($sExclude, '([\Q\.+[^]$(){}=!\E])', '\\$1')
    $sExclude = StringReplace($sExclude, "?", ".")
    $sExclude = StringReplace($sExclude, "*", ".*?")
    $sExclude = "(?i)\A(" & StringReplace($sExclude, ";", "$|") & "$)" ;case-insensitive, convert ';' to '|', match from first char, terminate strings
    ;$sExclude = "(?i)\A" & StringReplace($sExclude, ";", "|") & "\z"
  EndIf
  ;-----
  ;If $iRetFormat <> "0" And $iRetFormat <> "2" Then $iRetFormat = "1"
  If Not ($iRetItemType = 0 Or $iRetItemType = 1 Or $iRetItemType = 2) Then Return SetError(3, 3, "")
  ;---------------
  ;[/check and prepare parameters]

  ;---------------

  Local $aPath = StringSplit($sPath, ';', 1) ;paths array
  Local $aFilter = StringSplit($sFilter, ';', 1) ;filters array

  ;---------------

  If $bRecursive Then ;different handling for recursion (strategy: unfiltered search for all items and filter unwanted)

    If $sExclude Then ;different handling dependent on $sExclude parameter is set or not

      For $iPCount = 1 To $aPath[0] ;Path loop
        $sPath = StringRegExpReplace($aPath[$iPCount], "[\\/]+\z", "") & "\" ;ensure exact one trailing slash
        If Not FileExists($sPath) Then ContinueLoop
        $iRootPathLen = StringLen($sPath) - 1

        Local $aPathStack[1024] = [1, $sPath]

        While $aPathStack[0] > 0
          $sWorkPath = $aPathStack[$aPathStack[0]]
          $aPathStack[0] -= 1
          ;-----
          $hSearchFile = FileFindFirstFile($sWorkPath & '*')
          If @error Then ContinueLoop
          ;-----
          Switch $iRetPathType
            Case 2 ;full path
              $sRetPath = $sWorkPath
            Case 1 ;relative path
              $sRetPath = StringTrimLeft($sWorkPath, $iRootPathLen + 1)
          EndSwitch
          ;-----
          Switch $iRetItemType
            Case 1
              While True ;Files only
                $sFile = FileFindNextFile($hSearchFile)
                If @error Then ExitLoop
                $fDirFlag = @extended
                If $fDirFlag Then
                  $aPathStack[0] += 1
                  If UBound($aPathStack) <= $aPathStack[0] Then ReDim $aPathStack[UBound($aPathStack) * 2]
                  $aPathStack[$aPathStack[0]] = $sWorkPath & $sFile & "\"
                  ContinueLoop
                EndIf
                If StringRegExp($sFile, $sExclude) Then ContinueLoop
                If StringRegExp($sFile, $sFilter) Then
                  $sFileList &= $sRetPath & $sFile & "|"
                EndIf
              WEnd
            Case 2
              While True ;Folders only
                $sFile = FileFindNextFile($hSearchFile)
                If @error Then ExitLoop
                $fDirFlag = @extended
                If StringRegExp($sFile, $sExclude) Then ContinueLoop
                If $fDirFlag Then
                  $aPathStack[0] += 1
                  If UBound($aPathStack) <= $aPathStack[0] Then ReDim $aPathStack[UBound($aPathStack) * 2]
                  $aPathStack[$aPathStack[0]] = $sWorkPath & $sFile & "\"
                  If StringRegExp($sFile, $sFilter) Then
                    $sFileList &= $sRetPath & $sFile & "|"
                  EndIf
                EndIf
              WEnd
            Case Else
              While True ;Files and Folders
                $sFile = FileFindNextFile($hSearchFile)
                If @error Then ExitLoop
                $fDirFlag = @extended
                If StringRegExp($sFile, $sExclude) Then ContinueLoop
                If $fDirFlag Then
                  $aPathStack[0] += 1
                  If UBound($aPathStack) <= $aPathStack[0] Then ReDim $aPathStack[UBound($aPathStack) * 2]
                  $aPathStack[$aPathStack[0]] = $sWorkPath & $sFile & "\"
                EndIf
                If StringRegExp($sFile, $sFilter) Then
                  $sFileList &= $sRetPath & $sFile & "|"
                EndIf
              WEnd
          EndSwitch
          ;-----
        WEnd

        FileClose($hSearchFile)

      Next ;$iPCount - next path

    Else ;If Not $sExclude

      For $iPCount = 1 To $aPath[0] ;Path loop
        $sPath = StringRegExpReplace($aPath[$iPCount], "[\\/]+\z", "") & "\" ;ensure exact one trailing slash
        If Not FileExists($sPath) Then ContinueLoop
        $iRootPathLen = StringLen($sPath) - 1

        Local $aPathStack[1024] = [1, $sPath]

        While $aPathStack[0] > 0
          $sWorkPath = $aPathStack[$aPathStack[0]]
          $aPathStack[0] -= 1
          ;-----
          $hSearchFile = FileFindFirstFile($sWorkPath & '*')
          If @error Then ContinueLoop
          ;-----
          Switch $iRetPathType
            Case 2 ;full path
              $sRetPath = $sWorkPath
            Case 1 ;relative path
              $sRetPath = StringTrimLeft($sWorkPath, $iRootPathLen + 1)
          EndSwitch
          ;-----
          Switch $iRetItemType
            Case 1
              While True ;Files only
                $sFile = FileFindNextFile($hSearchFile)
                If @error Then ExitLoop
                If @extended Then
                  $aPathStack[0] += 1
                  If UBound($aPathStack) <= $aPathStack[0] Then ReDim $aPathStack[UBound($aPathStack) * 2]
                  $aPathStack[$aPathStack[0]] = $sWorkPath & $sFile & "\"
                  ContinueLoop
                EndIf
                If StringRegExp($sFile, $sFilter) Then
                  $sFileList &= $sRetPath & $sFile & "|"
                EndIf
              WEnd
            Case 2
              While True ;Folders only
                $sFile = FileFindNextFile($hSearchFile)
                If @error Then ExitLoop
                If @extended Then
                  $aPathStack[0] += 1
                  If UBound($aPathStack) <= $aPathStack[0] Then ReDim $aPathStack[UBound($aPathStack) * 2]
                  $aPathStack[$aPathStack[0]] = $sWorkPath & $sFile & "\"
                  If StringRegExp($sFile, $sFilter) Then
                    $sFileList &= $sRetPath & $sFile & "|"
                  EndIf
                EndIf
              WEnd
            Case Else
              While True ;Files and Folders
                $sFile = FileFindNextFile($hSearchFile)
                If @error Then ExitLoop
                If @extended Then
                  $aPathStack[0] += 1
                  If UBound($aPathStack) <= $aPathStack[0] Then ReDim $aPathStack[UBound($aPathStack) * 2]
                  $aPathStack[$aPathStack[0]] = $sWorkPath & $sFile & "\"
                EndIf
                If StringRegExp($sFile, $sFilter) Then
                  $sFileList &= $sRetPath & $sFile & "|"
                EndIf
              WEnd
          EndSwitch
          ;-----
        WEnd

        FileClose($hSearchFile)

      Next ;$iPCount - next path

    EndIf ;If $sExclude

  Else ;If Not $bRecursive (strategy: filtered search for items)

    If $sExclude Then ;different handling dependent on $sExclude parameter is set or not

      For $iPCount = 1 To $aPath[0] ;Path loop

        $sPath = StringRegExpReplace($aPath[$iPCount], "[\\/]+\z", "") & "\" ;ensure exact one trailing slash
        If Not FileExists($sPath) Then ContinueLoop
        ;-----
        Switch $iRetPathType
          Case 2 ;full path
            $sRetPath = $sPath
          Case 1 ;relative path
            $sRetPath = ""
        EndSwitch

        For $iFCount = 1 To $aFilter[0] ;filter loop
          ;-----
          $hSearchFile = FileFindFirstFile($sPath & $aFilter[$iFCount])
          If @error Then ContinueLoop
          ;-----
          Switch $iRetItemType
            Case 1 ;files Only
              While True
                $sFile = FileFindNextFile($hSearchFile)
                If @error Then ExitLoop
                If @extended Then ContinueLoop ;bypass folder
                ;check for exclude files
                If StringRegExp($sFile, $sExclude) Then ContinueLoop
                $sFileList &= $sRetPath & $sFile & "|"
              WEnd
            Case 2 ;folders Only
              While True
                $sFile = FileFindNextFile($hSearchFile)
                If @error Then ExitLoop
                If @extended Then ;bypass file
                  ;check for exclude folder
                  If StringRegExp($sFile, $sExclude) Then ContinueLoop
                  $sFileList &= $sRetPath & $sFile & "|"
                EndIf
              WEnd
            Case Else ;files and folders
              While True
                $sFile = FileFindNextFile($hSearchFile)
                If @error Then ExitLoop
                ;check for exclude files/folder
                If StringRegExp($sFile, $sExclude) Then ContinueLoop
                $sFileList &= $sRetPath & $sFile & "|"
              WEnd
          EndSwitch
          FileClose($hSearchFile)
        Next ;$iFCount - next filter

      Next ;$iPCount - next path

    Else ;If Not $sExclude

      For $iPCount = 1 To $aPath[0] ;Path loop

        $sPath = StringRegExpReplace($aPath[$iPCount], "[\\/]+\z", "") & "\" ;ensure exact one trailing slash
        If Not FileExists($sPath) Then ContinueLoop
        ;-----
        Switch $iRetPathType
          Case 2 ;full path
            $sRetPath = $sPath
          Case 1 ;relative path
            $sRetPath = ""
                EndSwitch

        For $iFCount = 1 To $aFilter[0] ;filter loop
          ;-----
          $hSearchFile = FileFindFirstFile($sPath & $aFilter[$iFCount])
          If @error Then ContinueLoop
          ;-----
          Switch $iRetItemType
            Case 1 ;files Only
              While True
                $sFile = FileFindNextFile($hSearchFile)
                If @error Then ExitLoop
                If @extended Then ContinueLoop ;bypass folder
                $sFileList &= $sRetPath & $sFile & "|"
              WEnd
            Case 2 ;folders Only
              While True
                $sFile = FileFindNextFile($hSearchFile)
                If @error Then ExitLoop
                If @extended Then ;bypass file
                  $sFileList &= $sRetPath & $sFile & "|"
                EndIf
              WEnd
            Case Else ;files and folders
              While True
                $sFile = FileFindNextFile($hSearchFile)
                If @error Then ExitLoop
                $sFileList &= $sRetPath & $sFile & "|"
              WEnd
          EndSwitch
          FileClose($hSearchFile)
        Next ;$iFCount - next filter

      Next ;$iPCount - next path

    EndIf ;If $sExclude

  EndIf ;If $bRecursive

  ;---------------

  ;set according return value
  If $sFileList Then
    Switch $iRetFormat
      Case 2 ;return a delimited string
        Return StringTrimRight($sFileList, 1)
      Case 0 ;return a 0-based array
        Return StringSplit(StringTrimRight($sFileList, 1), "|", 2)
      Case Else ;return a 1-based array
        Return StringSplit(StringTrimRight($sFileList, 1), "|", 1)
    EndSwitch
  Else
    Return SetError(4, 4, "")
  EndIf

EndFunc   ;==>_FileListToArrayXT

[Wus]

Some test results (all done out of cache):

Test1: path: System32-Dir, filter="*.dll", return files/folders, full path, no recursion, no exclusion, repeats 300

_FileListToArray: Sec = 7.70 , 1832 items

_FileListToArrayNT7: Sec = 4.61 , 1832 items

_FileListToArrayXT: Sec = 4.62 , 1832 items

Test2: path: System32-Dir, filter="*", return files/folders, full path, no recursion, no exclusion, repeats 300

_FileListToArray: Sec = 12.80 , 3114 items

_FileListToArrayNT7: Sec = 7.30 , 3114 items

_FileListToArrayXT: Sec = 7.31 , 3114 items

Test3: path: System32-Dir, filter="*nothing*.xyz", return files/folders, full path, no recursion, no exclusion, repeats 300

_FileListToArray: Sec = 0.53 , 0 items

_FileListToArrayNT7: Sec = 0.56 , 0 items

_FileListToArrayXT: Sec = 0.56 , 0 items

...

First, the more interesting statistic (in my opinion) is the reciprocal of the time, which is directly proportional with the speed. This can then be used to find a good metric of your % increase in speed.

Test1:

_FileListToArray: Sec = 7.70 , 1832 items, 0.130 1/s

_FileListToArrayXT: Sec = 4.62 , 1832 items, 0.216 1/s

* XT version is 66.2% faster

Test2:

_FileListToArray: Sec = 12.80 , 3114 items, 0.0781 1/s

_FileListToArrayXT: Sec = 7.31 , 3114 items, 0.137 1/s

* XT version is 75.1% faster

Thus you have made the function 60-80% faster. My question to you is, under what circumstances is a 80% increase in speed acceptable when a ~300% increase has been made to the source code? There are definitely situations where this would be warranted, but what is it about this situation that makes it worthwhile. I will admit that a significant amount of that bloat is probably from the added features, but still even an increase in source code commensurate with the increase in speed seems like a very poor choice in most cases.

I would say that the increase in function speed should be orders of magnitude larger than the source code bloat in order to make it worthwhile in most cases.

Edited July 14, 2009 by Wus

[KaFu]

I would say that the increase in function speed should be orders of magnitude larger than the source code bloat in order to make it worthwhile in most cases.

I would be curious what this means in KB for the compiled code... TB HDDs are standard and every simple games consumes 100's of MB to GB... do you really complain about 20kb?

Edit:

Compiled _FileListToArrayXT() is 3KB smaller than compiled _FileListToArray()... unstripped, talking of source-code bloating...

Edited July 14, 2009 by KaFu

[Wus]

My use of the phrase 'source code bloat' references the number of lines in the source code, *not* the compiled size; I apologize if this is a non-standard use of the phrase. You are absolutely correct in that a few extra kilobytes of code isn't an issue. What *is* a problem is trying to debug, update, and manage an extra few hundred lines of code.

At this juncture I will stop posting in this thread. The important points have been made and at this point I feel that I am beating a dead horse. We clearly and irrevocably disagree. End of story.

Cheers.

Edited July 14, 2009 by Wus

[Spiff59]

The extra options don't necessarily require much more code, Post 252 competes pretty well, and ended up at 63 lines. Twenty-four of those are one-time edits/formats of the input parameters.

[Spiff59]

I haven't even begun to look into all the available options with this, but I wonder how this MS Word filesearch stacks up, performance-wise, versus the other recursive versions?

#include<array.au3>

$x = FLTA_MSWord("C:\", "*.txt")
_ArrayDisplay($x)

;-------------------------------------------------------------------------------
Func FLTA_MSWord($sPath, $sFilter)
    $objWord = ObjCreate("Word.Application")
    With $objWord.FileSearch
        .LookIn = $sPath
        .FileName = $sFilter
        .SearchSubfolders = True
        .Execute
    EndWith
    $str = ""
    For $objFile in $objWord.FileSearch.FoundFiles
        $str &= $objFile & "|"
    Next
    $objWord.Quit
    Return StringSplit($str, "|")
EndFunc

;   .NewSearch
;   .SearchSubFolders = True
;   .TextOrProperty = "San*"
;   .MatchTextExactly = True
;   .MatchAllWordForms = True
;   .FileType = msoFileTypeAllFiles

;   $objWord.FileSearch.FoundFiles.Count

Edit: And yes, I know this is not a valid FLTA replacement, as it relies upon the user having MS Office (or Word) installed.

Edited July 29, 2009 by Spiff59

[Tlem]

@BaKaMu

I use your function _FileListToArrayXT from a long time, and I never have any problem since today.

One of member of the French forum notice me on a strange behavior.

Create a directory structure with some directory and subdirectory.

Create directory and subdirectory with specific name.

Use _FileListToArrayXT to find these specifics directory and try to delete them.

The first time you lunch the script, directory can't be deleted, but you lunch the script a second time, it work !!!

#include <Array.au3>

$SrcDir = "D:\test"
$KillDir = "TOTO"

$ar_Array = _FileListToArrayXT($SrcDir, $KillDir, 2, 2, 1)
If Not IsArray($ar_Array) Then Exit

For $i = $ar_Array[0] To 1 Step -1
    DirRemove($ar_Array[$i], 1)
Next

$ar_Array = _FileListToArrayXT($SrcDir, $KillDir, 2, 2, 1)
If Not IsArray($ar_Array) Then
    Exit
Else
    MsgBox(16, 'Error', 'Some folder can''t be deleted' & @CRLF & _
    'Push enter to see the folder list')
    _ArrayDisplay($ar_Array, 'Liste des dossiers')
EndIf

I try to add a FileClose after each While on each case, but doesn't work.

So perhaps have you an idea.

Just one additional thing: The _FileListToArrayNT7 work find on this case.

[BaKaMu]

@BaKaMu

...

@Tiem

unfortunately my tests have confirmed this behavior, but at the moment i have no idea why.

The result of _FileListToArrayXT is correct, but it seems, that the dir is locked.

_FileListToArrayXT only uses three file-operations: FileExists, FileFindFirstFile and FileFindNextFile.

All of these are normal internal autoit functions and besides there is nothing special in _FileListToArrayXT.

Maybe one of this functions is problematical (does not release file-handle or something else).

Maybe DirRemove is problematical.

Because these are internal AutoIt functions, i can not go deeper in source.

But it is good to know, that there is a problem.

Thank you for reporting this behavior, i will observe this further on.

bakamu