Jump to content



Photo

Improvement of included _FileListToArray function.


  • Please log in to reply
262 replies to this topic

#41 Tlem

Tlem

    Universalist

  • Active Members
  • PipPipPipPipPip
  • 279 posts

Posted 21 June 2009 - 08:35 PM

In fact, it depends if result is true or false. :D
Try your test in this way :
$fred = 0 ;xxxxxxxxxxxxxxxxxxxxxxxxx If... Then... EndIf xxxxxxxxxxxxxxxxxxxxxxxx $x = 0 $begin = TimerInit() For $i = 1 To 500000     If $fred <> 0 Then         $x += 1     EndIf Next ConsoleWrite("If... Then... EndIf: " & Timerdiff($begin) & "  Result to verify: " & $x & @CRLF) ;xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx ;xxxxxxxxxxxxxxxxxxxxxxxxxxx If...Then xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx $x = 0 $begin = TimerInit() For $i = 1 To 500000     If $fred <> 0 Then $x += 1 Next ConsoleWrite("If... Then.........: " & Timerdiff($begin) & "  Result to verify: " & $x & @CRLF) ;xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

In that case, If ... Then ... is faster. :D
So what is the good choice?
It would be necessary to know if the function will be true by default...

Edited by Tlem, 21 June 2009 - 08:52 PM.

Best Regards.Thierry







#42 trancexx

trancexx

    Hm, I really shouldn't.

  • Active Members
  • PipPipPipPipPipPip
  • 5,242 posts

Posted 21 June 2009 - 09:05 PM

In fact, it depends if result is true or false. :D
Try your test in this way :

$fred = 0 ;xxxxxxxxxxxxxxxxxxxxxxxxx If... Then... EndIf xxxxxxxxxxxxxxxxxxxxxxxx $x = 0 $begin = TimerInit() For $i = 1 To 500000     If $fred <> 0 Then         $x += 1     EndIf Next ConsoleWrite("If... Then... EndIf: " & Timerdiff($begin) & "  Result to verify: " & $x & @CRLF) ;xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx ;xxxxxxxxxxxxxxxxxxxxxxxxxxx If...Then xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx $x = 0 $begin = TimerInit() For $i = 1 To 500000     If $fred <> 0 Then $x += 1 Next ConsoleWrite("If... Then.........: " & Timerdiff($begin) & "  Result to verify: " & $x & @CRLF) ;xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

In that case, If ... Then ... is faster. :D
So what is the good choice?
It would be necessary to know if the function will be true by default...

I remember one president-to-be winning the presidency by saying: "Read my lips: no new taxes!"
With the same importance I say: ""Read my post again: ..."
You are just repeating what I said.



Btw, he raised taxes when elected.

eMyvnE


#43 Tlem

Tlem

    Universalist

  • Active Members
  • PipPipPipPipPip
  • 279 posts

Posted 21 June 2009 - 09:18 PM

Sorry.
I do not really master the English langage, thus I often pass in quoted by certain things.
I use online translator for certain sentence, but it doesn't really help me sometimes.

Be indulgent to me and still forgive I.
I promiss you that I never raise your taxes because I will never be president. :D

Edited by Tlem, 21 June 2009 - 09:21 PM.

Best Regards.Thierry

#44 Spiff59

Spiff59

    Universalist

  • Active Members
  • PipPipPipPipPipPip
  • 1,313 posts

Posted 24 June 2009 - 03:41 AM

Glancing at this again, it seemed odd that first there was a line of code to strip a trailing slash off of $sPath, but after that all four times that $sPath is referenced, a trailing slash is being appended.
This ought to be a fraction cleaner/faster than the prior fastest example...
[codebox]
Plain Text         
Func _FileListToArray($sPath, $sFilter = "*", $iFlag = 0) Local $hSearch, $sFile, $sFileList, $sDelim = "|" If Not FileExists($sPath) Then Return SetError(1, 1, "") If StringRegExp($sFilter, "[\\/:<>|]") Or (Not StringStripWS($sFilter, 8)) Then Return SetError(2, 2, "") If StringRight($sPath, 1) <> "\" Then $sPath &= "\" If $iFlag > 3 Then     $sDelim &= $sPath     $iFlag -= 4 EndIf $hSearch = FileFindFirstFile($sPath & $sFilter) If $hSearch = -1 Then Return SetError(4, 4, "") Switch $iFlag     Case 0; Files and Folders         While 1             $sFile = FileFindNextFile($hSearch)             If @error Then ExitLoop             $sFileList &= $sDelim & $sFile         WEnd     Case 1; Files Only         While 1             $sFile = FileFindNextFile($hSearch)             If @error Then ExitLoop ;           If StringInStr(FileGetAttrib($sPath & $sFile), "D") <> 0 Then ContinueLoop; bypass folder (ver 3.3.0.0)             If @extended Then ContinueLoop; bypass folder (for Autoit versions > 3.3.0.0)             $sFileList &= $sDelim & $sFile         WEnd     Case 2; Folders Only         While 1             $sFile = FileFindNextFile($hSearch)             If @error Then ExitLoop ;           If StringInStr(FileGetAttrib($sPath & $sFile), "D") = 0 Then ContinueLoop; bypass file (ver 3.3.0.0)             If @extended = 0 Then ContinueLoop; bypass file (for Autoit versions > 3.3.0.0)             $sFileList &= $sDelim & $sFile         WEnd     Case Else         Return SetError(3, 3, "") EndSwitch FileClose($hSearch) Return StringSplit(StringTrimLeft($sFileList, 1), "|") EndFunc;==>_FileListToArray


typo

Edited by Spiff59, 24 June 2009 - 04:49 AM.


#45 Tlem

Tlem

    Universalist

  • Active Members
  • PipPipPipPipPip
  • 279 posts

Posted 25 June 2009 - 07:38 PM

I can't edit my first post. If I do it, this is what I see inside :

EndFunc ;==>_FileListToArrayƒo݊÷ Ù8^¨§¡ûb™è¬Ÿm­æ¥—+"‚xŸ‰Æ§µ¦èºÝv±ç(Û¢½ºÓM4~)^±»­‰ü¨ºë¶¬~éܶ*'jZ-¥êáj›"µÆ§…éiʋ­¢¢÷°±ç(Û.‰±ŠßÛþ/îêފË(™ë^²Û+zË¥¶ËajÒ!j÷¢w¨žl¡j·pj·„iÈmzËpjÇhŠx9¶)ž²)ږŠ)jwm…êÞ²émŠËay«Þ­¨¡÷â½ëږç¬êފËazÈh®×^±Êâ¦Ø¨ž‡í…éšµêâj[ajا

±êïy×è®Ø^µë-°ð

:D

Also, I do not have a response on why check @extented is not faster than check StringInStr ?
Best Regards.Thierry

#46 Zedna

Zedna

    AutoIt rulez!

  • MVPs
  • 8,409 posts

Posted 25 June 2009 - 08:11 PM

I can't edit my first post. If I do it, this is what I see inside :

:D

Also, I do not have a response on why check @extented is not faster than check StringInStr ?


Look here


http://www.autoitscript.com/forum/index.ph...st&p=699744

#47 Tlem

Tlem

    Universalist

  • Active Members
  • PipPipPipPipPip
  • 279 posts

Posted 25 June 2009 - 08:31 PM

Thanks.
But I had already tried in full edit mode and quick edit with Firefox and IE without success. :D
Best Regards.Thierry

#48 jpm

jpm

    a Real GUI/debug lover

  • Developers
  • 8,925 posts

Posted 26 June 2009 - 08:04 AM

I can't edit my first post. If I do it, this is what I see inside :

:D

Also, I do not have a response on why check @extented is not faster than check StringInStr ?

@extended is return by the FileFindNextFile , so it is faster than calling a new function.

#49 Tlem

Tlem

    Universalist

  • Active Members
  • PipPipPipPipPip
  • 279 posts

Posted 26 June 2009 - 03:17 PM

Normally yes, but in my tests, it is not the case between 3.3.0.0 and beta version 3.3.1.1...
Take a look on times here : http://www.autoitscript.com/forum/index.ph...st&p=698072
Best Regards.Thierry

#50 Spiff59

Spiff59

    Universalist

  • Active Members
  • PipPipPipPipPipPip
  • 1,313 posts

Posted 26 June 2009 - 06:20 PM

Normally yes, but in my tests, it is not the case between 3.3.0.0 and beta version 3.3.1.1...

I show huge speed improvements with @extended, when $iFlag is 1,2,5 or 6. Values of 0 and 4 don't employ @extended or StringInStr, so only then do I show the two routines comparable in performance.

[codebox]
Plain Text         
#include<file.au3> #include<array.au3> $iflag = 1 ; Prod Version ------------------------------- $timer = TimerInit() For $j = 1 to 100     $x = _FileListToArray(@SystemDir,"*",$iflag) Next $t1 = TimerDiff ($timer) ; Beta w/ StrInStr --------------------------- $timer = TimerInit() For $j = 1 to 100     $x = _FileListToArray2(@SystemDir,"*",$iflag) Next $t2 = TimerDiff ($timer) ; Beta w/ @extended -------------------------- $timer = TimerInit() For $j = 1 to 100     $x = _FileListToArray3(@SystemDir,"*",$iflag) Next $t3 = TimerDiff ($timer) MsgBox (0, "", "$iFlag = " & $iflag & @CRLF &"Prod:     " & $t1 & @CRLF & "StrInStr:   "& $t2 & @CRLF & "Extended: " & $t3) ;_ArrayDisplay($x) ; ------------------------------------------------ Func _FileListToArray2($sPath, $sFilter = "*", $iFlag = 0) Local $hSearch, $sFile, $sFileList, $sDelim = "|" If Not FileExists($sPath) Then Return SetError(1, 1, "") If StringRegExp($sFilter, "[\\/:<>|]") Or (Not StringStripWS($sFilter, 8)) Then Return SetError(2, 2, "") If StringRight($sPath, 1) <> "\" Then $sPath &= "\" If $iFlag > 3 Then     $sDelim &= $sPath     $iFlag -= 4 EndIf $hSearch = FileFindFirstFile($sPath & $sFilter) If $hSearch = -1 Then Return SetError(4, 4, "") Switch $iFlag     Case 0; Files and Folders         While 1             $sFile = FileFindNextFile($hSearch)             If @error Then ExitLoop             $sFileList &= $sDelim & $sFile         WEnd     Case 1; Files Only         While 1             $sFile = FileFindNextFile($hSearch)             If @error Then ExitLoop             If StringInStr(FileGetAttrib($sPath & $sFile), "D") <> 0 Then ContinueLoop; bypass folder (ver 3.3.0.0) ;           If @extended Then ContinueLoop; bypass folder (for Autoit versions > 3.3.0.0)             $sFileList &= $sDelim & $sFile         WEnd     Case 2; Folders Only         While 1             $sFile = FileFindNextFile($hSearch)             If @error Then ExitLoop             If StringInStr(FileGetAttrib($sPath & $sFile), "D") = 0 Then ContinueLoop; bypass file (ver 3.3.0.0) ;           If @extended = 0 Then ContinueLoop; bypass file (for Autoit versions > 3.3.0.0)             $sFileList &= $sDelim & $sFile         WEnd     Case Else         Return SetError(3, 3, "") EndSwitch FileClose($hSearch) Return StringSplit(StringTrimLeft($sFileList, 1), "|") EndFunc;==>_FileListToArray ; ------------------------------------------------ Func _FileListToArray3($sPath, $sFilter = "*", $iFlag = 0) Local $hSearch, $sFile, $sFileList, $sDelim = "|" If Not FileExists($sPath) Then Return SetError(1, 1, "") If StringRegExp($sFilter, "[\\/:<>|]") Or (Not StringStripWS($sFilter, 8)) Then Return SetError(2, 2, "") If StringRight($sPath, 1) <> "\" Then $sPath &= "\" If $iFlag > 3 Then     $sDelim &= $sPath     $iFlag -= 4 EndIf $hSearch = FileFindFirstFile($sPath & $sFilter) If $hSearch = -1 Then Return SetError(4, 4, "") Switch $iFlag     Case 0; Files and Folders         While 1             $sFile = FileFindNextFile($hSearch)             If @error Then ExitLoop             $sFileList &= $sDelim & $sFile         WEnd     Case 1; Files Only         While 1             $sFile = FileFindNextFile($hSearch)             If @error Then ExitLoop ;           If StringInStr(FileGetAttrib($sPath & $sFile), "D") <> 0 Then ContinueLoop; bypass folder (ver 3.3.0.0)             If @extended Then ContinueLoop; bypass folder (for Autoit versions > 3.3.0.0)             $sFileList &= $sDelim & $sFile         WEnd     Case 2; Folders Only         While 1             $sFile = FileFindNextFile($hSearch)             If @error Then ExitLoop ;           If StringInStr(FileGetAttrib($sPath & $sFile), "D") = 0 Then ContinueLoop; bypass file (ver 3.3.0.0)             If @extended = 0 Then ContinueLoop; bypass file (for Autoit versions > 3.3.0.0)             $sFileList &= $sDelim & $sFile         WEnd     Case Else         Return SetError(3, 3, "") EndSwitch FileClose($hSearch) Return StringSplit(StringTrimLeft($sFileList, 1), "|") EndFunc;==>_FileListToArray
edit: pasted wrong code

Edited by Spiff59, 26 June 2009 - 06:22 PM.


#51 Tlem

Tlem

    Universalist

  • Active Members
  • PipPipPipPipPip
  • 279 posts

Posted 26 June 2009 - 09:21 PM

Your probably right. It seems that I forgot to change the $iFlag (I use 4 in my tests).
I will remade the tests later. Thank for your response.

Edit : Whaouuu, that's an improvement ...
I just do the test whit $Flag = 1 and I was pleasantly surprised with the speed of execution (more than 4 times faster between the too versions).

So I think that this version is the better choice to use if future version _FileListToArray() :
Plain Text         
Func _FileListToArray($sPath, $sFilter = "*", $iFlag = 0)     Local $hSearch, $sFile, $sFileList, $sDelim = "|"     If Not FileExists($sPath) Then Return SetError(1, 1, "")     If StringRegExp($sFilter, "[\\/:<>|]") Or (Not StringStripWS($sFilter, 8)) Then Return SetError(2, 2, "")     If StringRight($sPath, 1) <> "\" Then $sPath &= "\"     If $iFlag > 3 Then         $sDelim &= $sPath         $iFlag -= 4     EndIf     $hSearch = FileFindFirstFile($sPath & $sFilter)     If $hSearch = -1 Then         Return SetError(4, 4, "")     EndIf     Switch $iFlag         Case 0; Files and Folders             While 1                 $sFile = FileFindNextFile($hSearch)                 If @error Then                     ExitLoop                 EndIf                 $sFileList &= $sDelim & $sFile             WEnd         Case 1; Files Only             While 1                 $sFile = FileFindNextFile($hSearch)                 If @error Then                     ExitLoop                 EndIf     ; bypass folder                 If @extended Then ContinueLoop; This version for new beta     ; Replace by this if AutoIt < 3.3.1.1     ;If StringInStr(FileGetAttrib($sPath & $sFile), "D") <> 0 Then     ;   ContinueLoop     ;EndIf                 $sFileList &= $sDelim & $sFile             WEnd         Case 2; Folders Only             While 1                 $sFile = FileFindNextFile($hSearch)                 If @error Then                     ExitLoop                 EndIf     ; bypass file                 If @extended = 0 Then ContinueLoop; This version for new beta     ; Replace by this if AutoIt < 3.3.1.1     ;If StringInStr(FileGetAttrib($sPath & $sFile), "D") = 0 Then     ;   ContinueLoop     ;EndIf                 $sFileList &= $sDelim & $sFile             WEnd         Case Else             Return SetError(3, 3, "")     EndSwitch     FileClose($hSearch)     Return StringSplit(StringTrimLeft($sFileList, 1), "|") EndFunc;==>_FileListToArray

Edited by Tlem, 27 June 2009 - 07:26 AM.

Best Regards.Thierry

#52 Spiff59

Spiff59

    Universalist

  • Active Members
  • PipPipPipPipPipPip
  • 1,313 posts

Posted 27 June 2009 - 03:01 AM

The FileFindFirstFile (and the two commented-out StringInStr) don't require the "& "\"", as we've ensured that $sPath has a trailing slash at the top of the function.

Edit: And I agree, it is vastly faster than the production version, and has added functionality. I can't see room for any more improvement.

Edited by Spiff59, 27 June 2009 - 03:03 AM.


#53 Tlem

Tlem

    Universalist

  • Active Members
  • PipPipPipPipPip
  • 279 posts

Posted 27 June 2009 - 07:21 AM

The FileFindFirstFile (and the two commented-out StringInStr) don't require the "& "\"

I edited my post to do the correction.

And I agree, it is vastly faster than the production version

Hoooo yes.

I can't see room for any more improvement.

Perhaps a recursive function. :D
Best Regards.Thierry

#54 BaKaMu

BaKaMu

    Wayfarer

  • Active Members
  • Pip
  • 76 posts

Posted 27 June 2009 - 08:36 AM

Perhaps a recursive function. :D

Here is it :-)

I have made a combination from different solutions here in this forum (thanks to all inspirations)

_FileListToArrayEx is a full compatible _FileListToArray replacement with more speed and additional features
additional: multi-path, multi-filter, multi exclude-filter, recursiv search
optional returns full pathname or filename only

This function needs at least AutoIt V3.3.1.0 Beta
For AutoIt V3.3.0.0 replace @extended with StringInStr(FileGetAttrib($sPath & $sFile), "D")
(or comment lines with @extended and uncomment lines with StringInStr(FileGetAttrib($sPath & $sFile), "D"))

The speed is better then _FileListToArray (non recursiv) and mostly better than other recursiv FileListToArray solutions
(for example no OEM to Ansi conversion needed like dir-cmd solutions etc)

Please check it out, i hope there is more room for improvements

PS: I have added a function for creating a files and folders test structure, so everyone can test with the same basics.

edit: wrong location of $sDelim definition ( thanks Tlem)


AutoIt         
#include <Array.au3> #include <File.au3> ;Here you can create a file and folder structure for testing (9072 files, 780 folders) ;CreateTestFileFolders("D:\Temp\Test") ;------------------- ;Testing parameters Dim $Repeat = 1 ;Dim $Repeat = 100 Dim $SearchPath = "D:\Temp\Test" ;Dim $SearchPath = "C:\Windows\System32" Dim $Pattern = "*" ;Dim $Pattern = "*.dll" Dim $Flag = 0 ;Dim $Flag = 0 + 4 Dim $Exclude = "" ;Dim $Exclude = "*.txt;*.sys;*.dat;*äöü*" Dim $Recursiv = False ;Dim $Recursiv = True ;Test-Start########################################## ;_FileListToArray $TimeBegin = TimerInit() For $TCount = 1 To $Repeat   $aRet = _FileListToArray($SearchPath, $Pattern, 0) Next $TimeDiff = StringFormat("Sec = %.2f", TimerDiff($TimeBegin) / 1000) ConsoleWrite(@HOUR & ":" & @MIN & ":" & @SEC & " - _FileListToArray: " & $TimeDiff & " , " & $aRet[0] & @LF) ;#KBM ;_ArrayDisplay($aRet, '') ;_FileListToArrayEx $TimeBegin = TimerInit() For $TCount = 1 To $Repeat   $aRet = _FileListToArrayEx($SearchPath, $Pattern, $Flag, $Recursiv, $Exclude, 1) Next $TimeDiff = StringFormat("Sec = %.2f", TimerDiff($TimeBegin) / 1000) ConsoleWrite(@HOUR & ":" & @MIN & ":" & @SEC & " - _FileListToArrayEx: " & $TimeDiff & " , " & $aRet[0] & @LF) ;#KBM ;_ArrayDisplay($aRet, '') ;******************************************************************** ; #FUNCTION# ===================================================== ; Name:             _FileListToArrayEx ; Description:      full compatible _FileListToArray replacement  (with more speed and additional features) ;                   additional: multi-path, multi-filter, multi exclude-filter, recursiv search ;                   optional full pathname ; Syntax:           _FileListToArrayEx([$sPath = @ScriptDir, [$sFilter = "*", [$iFlag = 0, [$bRecursiv = False, [$sExclude = "", [$iFormat = 1]]]]]]) ; Parameter(s):     $sPath = optional: path to generate filelist for, multi paths separated with semicolon (ex: "C:\";D:\Temp") ;                            if no path is given then @ScriptDir is used ;                   $sFilter = optional: The filter to use. (default: "*") ;                              multi filters separated with semicolon (ex: *.exe; *.txt will find all .exe and .txt files) ;                              (Search the Autoit3 manual for the word "WildCards" for details) ;                   $iFlag = Optional: specifies whether to return files, folders or both ;                            0 = (Default) Return both files and folders ;                            1 = Return files only ;                            2 = Return folders only ;                            $iFlag + 4 = Return Filenames and/or Folders incl full Path ;                   $bRecursiv = optional: true: recursive search in rootdir and subdirs ;                                          False (default): search only in rootdir ;                   $sExclude = optional: exclude a file from the list by all or part of its name, various statements delimited with semicolon ;                               (ex: Unins* will remove all files/folders that start with Unins) ;                   $iFormat =  optional: return format ;                               0 = String ( "|" delimited) ;                               1 = (default) one-dimensional array, array[0] = number of files\folders returned ;                               2 = one-dimensional array, 0-based ; Requirement(s):   none ; Return Value(s):  on success: string or array (dependent on $iFormat) ; Author(s):        bernd670, Tlem, Spiff59, Zedna, KaFu, SmOke_N, GEOSoft, BaKaMu ; =================================================================== Func _FileListToArrayEx($sPath = @ScriptDir, $sFilter = "*", $iFlag = 0, $bRecursiv = False, $sExclude = "", $iFormat = 1)   Local $hSearch, $iPCount, $iFCount, $sFile, $TFlag = 0, $sFileList = ""   If $sPath = -1 Or $sPath = Default Then $sPath = @ScriptDir   If $sFilter = -1 Or $sFilter = Default Then $sFilter = "*"   If $iFlag = -1 Or $iFlag = Default Then $iFlag = 0   If $bRecursiv = Default Then $bRecursiv = False   If $sExclude = -1 Or $sExclude = Default Then $sExclude = ""   If $iFormat = -1 Or $iFormat = Default Then $iFormat = 1   ;separate multi path   Local $aPath = StringSplit($sPath, ';')   ;separate multi filter   Local $aFilter = StringSplit($sFilter, ';')   For $iPCount = 1 To $aPath[0]     Local $sPathItem = StringStripWS($aPath[$iPCount], 3) ;Strip leading and trailing spaces     Local $sDelim = "|"     If StringRight($sPathItem, 1) <> "\" Then $sPathItem = $sPathItem & "\" ;check for trailing "\"     ;return full-path or filename only ($iFlag 0,1,2 = Filename, $iFlag 4,5,6 = full path)     If $iFlag > 3 Then       $sDelim &= $sPathItem       $TFlag = $iFlag - 4     Else       $TFlag = $iFlag     EndIf     ;perform the search     For $iFCount = 1 To $aFilter[0]       Local $FilterItem = StringStripWS($aFilter[$iFCount], 3) ;Strip leading and trailing spaces       If StringRegExp($FilterItem, "[\\/:<>|]") Then ContinueLoop ;Look for bad chars       $hSearch = FileFindFirstFile($sPathItem & $FilterItem)       If @error Then ContinueLoop       Switch $TFlag         Case 0 ;Files and Folders           While True             $sFile = FileFindNextFile($hSearch)             If @error Then ExitLoop             $sFileList &= $sDelim & $sFile           WEnd         Case 1 ;Files Only           While True             $sFile = FileFindNextFile($hSearch)             If @error Then ExitLoop             ;If StringInStr(FileGetAttrib($sPath & $sFile), "D") <> 0 Then ContinueLoop; bypass folder (ver 3.3.0.0)             If @extended Then ContinueLoop ;bypass folder (for Autoit versions > 3.3.0.0)             $sFileList &= $sDelim & $sFile           WEnd         Case 2 ;Folders Only           While True             $sFile = FileFindNextFile($hSearch)             If @error Then ExitLoop             ;If StringInStr(FileGetAttrib($sPath & $sFile), "D") = 0 Then ContinueLoop; bypass file (ver 3.3.0.0)             If @extended = 0 Then ContinueLoop ;bypass file (for Autoit versions > 3.3.0.0)             $sFileList &= $sDelim & $sFile           WEnd         Case Else           Return SetError(3, 3, "")       EndSwitch       FileClose($hSearch)     Next     ;---------------     ;optional do a recursive search     If $bRecursiv Then       $hSearch = FileFindFirstFile($sPathItem & "*.*")       If Not @error Then         While True           $sFile = FileFindNextFile($hSearch)           If @error Then ExitLoop           ;If StringInStr(FileGetAttrib($sPath & $sFile), "D") = 0 Then ContinueLoop; bypass file (ver 3.3.0.0)           If @extended = 0 Then ContinueLoop ;bypass file (for Autoit versions > 3.3.0.0)           $sFileList &= _FileListToArrayEx($sPathItem & $sFile, $sFilter, $iFlag, $bRecursiv, "", 0)         WEnd         FileClose($hSearch)       EndIf     EndIf   Next ;$iPCount   ;---------------   ;optional exclude filename/folder   If $sExclude Then ;will be executed only once (at the end)     Local $aFilesFound = StringSplit(StringTrimLeft($sFileList, 1), "|", 0)     ;Strip leading and trailing spaces and spaces between semi colon delimiter     $sExclude = StringStripWS(StringRegExpReplace($sExclude, "\s*;\s*", ";"), 3)     ;convert $sExclude to RegExp formated string     $sExclude = StringReplace($sExclude, ".", "\.")     $sExclude = StringReplace($sExclude, "?", ".")     $sExclude = StringReplace($sExclude, "*", ".*?")     $sExclude = StringReplace($sExclude, ";", "|")     $sFileList = ""     ;Compare Filenames with $sExclude     For $iFCount = 1 To $aFilesFound[0]       If StringRegExp(StringRegExpReplace($aFilesFound[$iFCount], "(.*?[\\/]+)*(.*?\z)", "\2"), "(?i)\A" & $sExclude & "\z") Then ContinueLoop       $sFileList &= "|" & $aFilesFound[$iFCount]     Next   EndIf   ;---------------   ;Set according return value   Switch $iFormat     Case 0       Return $sFileList     Case 1       If $sFileList = "" Then         Local $aRet[1] = [0]         Return $aRet       Else         Return StringSplit(StringTrimLeft($sFileList, 1), "|", $iFormat)       EndIf     Case 2       If $sFileList = "" Then         Return ""       Else         Return StringSplit(StringTrimLeft($sFileList, 1), "|", $iFormat)       EndIf   EndSwitch EndFunc   ;==>_FileListToArrayEx ;Create test files and folders (9072 files, 780 folders) --------------------------------- Func CreateTestFileFolders($RootPath)   Local $Dir0 = $RootPath   DirCreate($Dir0)   Local $File0 = "File-Root"   For $FCount = 1 To 2     $hFile = FileOpen($Dir0 & "\" & $File0 & "(" & $FCount & ").txt", 1)     FileClose($hFile)     $hFile = FileOpen($Dir0 & "\" & $File0 & "(" & $FCount & ").dat", 1)     FileClose($hFile)     $hFile = FileOpen($Dir0 & "\" & $File0 & "äöü(" & $FCount & ").ß~°²³", 1)     FileClose($hFile)     $hFile = FileOpen($Dir0 & "\" & $File0 & "(" & $FCount & ").hid", 1)     FileSetAttrib($Dir0 & "\" & $File0 & "(" & $FCount & ").hid", "+H", 0)     FileClose($hFile)     $hFile = FileOpen($Dir0 & "\" & $File0 & "(" & $FCount & ").sys", 1)     FileSetAttrib($Dir0 & "\" & $File0 & "(" & $FCount & ").sys", "+S", 0)     FileClose($hFile)     $hFile = FileOpen($Dir0 & "\" & $File0 & "(" & $FCount & ").rsh", 1)     FileSetAttrib($Dir0 & "\" & $File0 & "(" & $FCount & ").rsh", "+R+S+H", 0)     FileClose($hFile)   Next   For $Count1 = 1 To 5     Local $Dir1 = $Dir0 & "\" & "Dir-" & $Count1     DirCreate($Dir1)     Local $File1 = "File-" & $Count1     For $FCount = 1 To 2       $hFile = FileOpen($Dir1 & "\" & $File1 & "(" & $FCount & ").txt", 1)       FileClose($hFile)       $hFile = FileOpen($Dir1 & "\" & $File1 & "(" & $FCount & ").dat", 1)       FileClose($hFile)       $hFile = FileOpen($Dir1 & "\" & $File1 & "äöü(" & $FCount & ").ß~°²³", 1)       FileClose($hFile)       $hFile = FileOpen($Dir1 & "\" & $File1 & "(" & $FCount & ").hid", 1)       FileSetAttrib($Dir1 & "\" & $File1 & "(" & $FCount & ").hid", "+H", 0)       FileClose($hFile)       $hFile = FileOpen($Dir1 & "\" & $File1 & "(" & $FCount & ").sys", 1)       FileSetAttrib($Dir1 & "\" & $File1 & "(" & $FCount & ").sys", "+S", 0)       FileClose($hFile)       $hFile = FileOpen($Dir1 & "\" & $File1 & "(" & $FCount & ").rsh", 1)       FileSetAttrib($Dir1 & "\" & $File1 & "(" & $FCount & ").rsh", "+R+S+H", 0)       FileClose($hFile)     Next     For $Count2 = 1 To 5       Local $Dir2 = $Dir1 & "\" & "Dir-" & $Count1 & $Count2       DirCreate($Dir2)       For $Count3 = 1 To 5         Local $Dir3 = $Dir2 & "\" & "Dir-" & $Count1 & $Count2 & $Count3         DirCreate($Dir3)         Local $File3 = "File-" & $Count1 & $Count2 & $Count3         For $FCount = 1 To 2           $hFile = FileOpen($Dir3 & "\" & $File3 & "(" & $FCount & ").txt", 1)           FileClose($hFile)           $hFile = FileOpen($Dir3 & "\" & $File3 & "(" & $FCount & ").dat", 1)           FileClose($hFile)           $hFile = FileOpen($Dir3 & "\" & $File3 & "äöü(" & $FCount & ").ß~°²³", 1)           FileClose($hFile)           $hFile = FileOpen($Dir3 & "\" & $File3 & "(" & $FCount & ").hid", 1)           FileSetAttrib($Dir3 & "\" & $File3 & "(" & $FCount & ").hid", "+H", 0)           FileClose($hFile)           $hFile = FileOpen($Dir3 & "\" & $File3 & "(" & $FCount & ").sys", 1)           FileSetAttrib($Dir3 & "\" & $File3 & "(" & $FCount & ").sys", "+S", 0)           FileClose($hFile)           $hFile = FileOpen($Dir3 & "\" & $File3 & "(" & $FCount & ").rsh", 1)           FileSetAttrib($Dir3 & "\" & $File3 & "(" & $FCount & ").rsh", "+R+S+H", 0)           FileClose($hFile)         Next         For $Count4 = 1 To 5           Local $Dir4 = $Dir3 & "\" & "Dir-" & $Count1 & $Count2 & $Count3 & $Count4           DirCreate($Dir4)           Local $File4 = "File-" & $Count1 & $Count2 & $Count3 & $Count4           For $FCount = 1 To 2             $hFile = FileOpen($Dir4 & "\" & $File4 & "(" & $FCount & ").txt", 1)             FileClose($hFile)             $hFile = FileOpen($Dir4 & "\" & $File4 & "(" & $FCount & ").dat", 1)             FileClose($hFile)             $hFile = FileOpen($Dir4 & "\" & $File4 & "äöü(" & $FCount & ").ß~°²³", 1)             FileClose($hFile)             $hFile = FileOpen($Dir4 & "\" & $File4 & "(" & $FCount & ").hid", 1)             FileSetAttrib($Dir4 & "\" & $File4 & "(" & $FCount & ").hid", "+H", 0)             FileClose($hFile)             $hFile = FileOpen($Dir4 & "\" & $File4 & "(" & $FCount & ").sys", 1)             FileSetAttrib($Dir4 & "\" & $File4 & "(" & $FCount & ").sys", "+S", 0)             FileClose($hFile)             $hFile = FileOpen($Dir4 & "\" & $File4 & "(" & $FCount & ").rsh", 1)             FileSetAttrib($Dir4 & "\" & $File4 & "(" & $FCount & ").rsh", "+R+S+H", 0)             FileClose($hFile)           Next         Next       Next     Next   Next EndFunc   ;==>CreateTestFileFolders

Edited by BaKaMu, 27 June 2009 - 01:29 PM.


#55 Tlem

Tlem

    Universalist

  • Active Members
  • PipPipPipPipPip
  • 279 posts

Posted 27 June 2009 - 10:08 AM

Really great for me.
I do a comparative test on AutoIt Beta 3.3.1.1 between _FileListToArrayEx (With Dir cmd), _FileListToArray3 and your new version, and the winner is ...

Your new version. :D

Test with $iFlag = 0 and loop of 5 tests :
_FileListToArrayEx (With Dir cmd) = 2389 ms for 19800 files/folders
_FileListToArray3 = 1316 ms for 19800 files/folders
_FileListToArrayEx (BaKaMu) = 964 ms for 19800 files/folders

Test with $iFlag = 1 and loop of 5 tests :
_FileListToArrayEx (With Dir cmd) = 4274 ms for 18080 files
_FileListToArray3 = 1302 ms for 18080 files
_FileListToArrayEx (BaKaMu) = 970 ms for 18080 files

Really great. :D

And I find your addition to use multiple search really great too.

Edited by Tlem, 27 June 2009 - 10:10 AM.

Best Regards.Thierry

#56 BaKaMu

BaKaMu

    Wayfarer

  • Active Members
  • Pip
  • 76 posts

Posted 27 June 2009 - 10:49 AM

Really great for me.
And I find your addition to use multiple search really great too.

Thanks :-)

But i'm sure (better: i hope so) this is not the end of the road.

#57 Tlem

Tlem

    Universalist

  • Active Members
  • PipPipPipPipPip
  • 279 posts

Posted 27 June 2009 - 01:03 PM

I think you have a bug in your function when use multi directory search and $iFlag=4 (Full path).
The first search is ok, but for the second search the path of the first search is added on the left. :D
Best Regards.Thierry

#58 BaKaMu

BaKaMu

    Wayfarer

  • Active Members
  • Pip
  • 76 posts

Posted 27 June 2009 - 01:33 PM

I think you have a bug in your function when use multi directory search and $iFlag=4 (Full path).
The first search is ok, but for the second search the path of the first search is added on the left. :D

Oh yes, sorry for that.
The definition of $sDelim was on wrong place.
I have forgotten to reset $sDelim in the loop
Corrected in Post #54

Thanks for the tip

Edited by BaKaMu, 27 June 2009 - 01:35 PM.


#59 Zedna

Zedna

    AutoIt rulez!

  • MVPs
  • 8,409 posts

Posted 27 June 2009 - 03:41 PM

BaKaMu I like your version very much.

Good work!

#60 Spiff59

Spiff59

    Universalist

  • Active Members
  • PipPipPipPipPipPip
  • 1,313 posts

Posted 27 June 2009 - 07:13 PM

Wow! Now this is huge change.
It looks like a filter could exclude a folder name from being returned by the function, but if $bRecursive is set, the files within that folder are still returned?
If exclude removes a folder name from the list, the files names within that folder are still returned. Maybe that's cool.
If you're retuning files only, should filters apply to folder names?
I'm just trying to figure out filters, excludes and the $iFlag all interact, or how they should interact?




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users