Another _FileListToArray()

[mfecteau]

I've modified a bit the code to add support for Unicode. So character like éèàêç work. Instead of grabbing the result for the dir command from the console (autoit doesn't support Unicode from the console), it writes the result to a file, then reads the result of the file (Unicode supported by Autoit when reading a file).

There's a new parameter ($unicode_support) that needs to be set to true, otherwise, the function will work as before (no Unicode support). Some characters seems not to be recognized correctly like the œ (like in the french word cœur). Maybe other characters like those are not processed correctly. So beware it's not perfect.

expandcollapse popup

Func _FileListToArrayEx($s_path, $s_mask = "*.*", $i_flag = 0, $s_exclude = -1, $f_recurse = False, $f_full_path = False, $unicode_support = false)

    If FileExists($s_path) = 0 Then Return SetError(1, 1, 0)

    ; Strip trailing backslash, and add one after to make sure there's only one
    $s_path = StringRegExpReplace($s_path, "[\\/]+\z", "") & "\"

    ; Set all defaults
    If $s_mask = -1 Or $s_mask = Default Then $s_mask = "*.*"
    If $i_flag = -1 Or $i_flag = Default Then $i_flag = 0
    If $s_exclude = -1 Or $s_exclude = Default Then $s_exclude = ""

    ; Look for bad chars
    If StringRegExp($s_mask, "[/:><\|]") Or StringRegExp($s_exclude, "[/:><\|]") Then
        Return SetError(2, 2, 0)
    EndIf

    ; Strip leading spaces between semi colon delimiter
    $s_mask = StringRegExpReplace($s_mask, "\s*;\s*", ";")
    If $s_exclude Then $s_exclude = StringRegExpReplace($s_exclude, "\s*;\s*", ";")

    ; Confirm mask has something in it
    If StringStripWS($s_mask, 8) = "" Then Return SetError(2, 2, 0)
    If $i_flag < 0 Or $i_flag > 2 Then Return SetError(3, 3, 0)

    ; Validate and create path + mask params
    Local $a_split = StringSplit($s_mask, ";"), $s_hold_split = ""
    For $i = 1 To $a_split[0]
        If StringStripWS($a_split[$i], 8) = "" Then ContinueLoop
        If StringRegExp($a_split[$i], "^\..*?\..*?\z") Then
            $a_split[$i] &= "*" & $a_split[$i]
        EndIf
        $s_hold_split &= '"' & $s_path & $a_split[$i] & '" '
    Next
    $s_hold_split = StringTrimRight($s_hold_split, 1)
    If $s_hold_split = "" Then $s_hold_split = '"' & $s_path & '*.*"'

    Local $i_pid, $s_stdout, $s_hold_out, $s_dir_file_only = "", $s_recurse = "/s "
    If $i_flag = 1 Then $s_dir_file_only = ":-d"
    If $i_flag = 2 Then $s_dir_file_only = ":D"
    If Not $f_recurse Then $s_recurse = ""

    if $unicode_support then
        dim $command =  @ComSpec & " /u /c dir /b " & $s_recurse & "/a" & $s_dir_file_only & " " & $s_hold_split & " > " & @TempDir & "\autoit_mf.tmp"
        $i_pid = RunWait($command, "", @SW_HIDE, 4 + 2)
        $s_hold_out = FileRead(@TempDir & "\autoit_mf.tmp")
        $s_hold_out = StringReplace($s_hold_out,Chr(0),"") ;plein de character NULL à enlever à cause de la conversion Unicode ..
        FileDelete(@TempDir & "\autoit_mf.tmp")
    Else
        $i_pid = Run(@ComSpec & " /c dir /b " & $s_recurse & "/a" & $s_dir_file_only & " " & $s_hold_split, "", @SW_HIDE, 4 + 2)
        While 1
            $s_stdout = StdoutRead($i_pid)
            If @error Then ExitLoop
            $s_hold_out &= $s_stdout
        WEnd
    endif



    $s_hold_out = StringRegExpReplace($s_hold_out, "\v+\z", "")
    If Not $s_hold_out Then Return SetError(4, 4, 0)

    ; Parse data and find matches based on flags
    Local $a_fsplit = StringSplit(StringStripCR($s_hold_out), @LF), $s_hold_ret
    $s_hold_out = ""

    If $s_exclude Then $s_exclude = StringReplace(StringReplace($s_exclude, "*", ".*?"), ";", "|")

    For $i = 1 To $a_fsplit[0]
        If $s_exclude And StringRegExp(StringRegExpReplace( _
            $a_fsplit[$i], "(.*?[\\/]+)*(.*?\z)", "\2"), "(?i)\Q" & $s_exclude & "\E") Then ContinueLoop
        If StringRegExp($a_fsplit[$i], "^\w:[\\/]+") = 0 Then $a_fsplit[$i] = $s_path & $a_fsplit[$i]
        If $f_full_path Then
            $s_hold_ret &= $a_fsplit[$i] & Chr(1)
        Else
            $s_hold_ret &= StringRegExpReplace($a_fsplit[$i], "((?:.*?[\\/]+)*)(.*?\z)", "$2") & Chr(1)
        EndIf
    Next

    $s_hold_ret = StringTrimRight($s_hold_ret, 1)
    If $s_hold_ret = "" Then Return SetError(5, 5, 0)

    Return StringSplit($s_hold_ret, Chr(1))
EndFunc

[Tlem]

Did you take a look on this one? : http://www.autoitscript.com/forum/index.php?showtopic=96952&view=findpost&p=706937

If you test it, you will see a really difference of time execution.

Edited May 13, 2010 by Tlem

[Ascend4nt]

I've modified a bit the code to add support for Unicode. So character like éèàêç work. Instead of grabbing the result for the dir command from the console (autoit doesn't support Unicode from the console), it writes the result to a file, then reads the result of the file (Unicode supported by Autoit when reading a file).

mfecteau, AWESOME find! I wasn't even aware of the Unicode @comspec '/u' switch (available even on Windows 2000 - who knew?!). Great great find. By the way, all you need to do with your code to get the Unicode data properly (including characters like œ) is one of the following. (Change is for the 'If $unicode_support then' section):

Option 1 - Force a read of the file as Unicode text by opening it in 'UTF16 Little Endian' mode:

If $unicode_support then
    dim $command = @ComSpec & " /u /c dir /b " & $s_recurse & "/a" & $s_dir_file_only & " " & $s_hold_split & " > " & @TempDir & "\autoit_mf.tmp"
    $i_pid = RunWait($command, "", @SW_HIDE, 4 + 2)
    $hFile=FileOpen(@TempDir & "\autoit_mf.tmp",32)
    $s_hold_out=FileRead($hFile)
    FileClose($hFile)
    FileDelete(@TempDir & "\autoit_mf.tmp")
Else

Option 2 - Create the file first using 'UTF16 Little Endian' mode, and change the '>' pipe to '>>' (append):

If $unicode_support then
    ; create (and clear contents of) tmp file, with Unicode prefix added at start (0xFF+0xFE)
    FileClose(FileOpen(@TempDir & "\autoit_mf.tmp",32+2))
    ; Change pipe from > to >> (append). This will keep Unicode marker (OXFF+0xFE) at start of file intact
    dim $command = @ComSpec & " /u /c dir /b " & $s_recurse & "/a" & $s_dir_file_only & " " & $s_hold_split & " >> " & @TempDir & "\autoit_mf.tmp"
    $i_pid = RunWait($command, "", @SW_HIDE, 4 + 2)
    $s_hold_out=FileRead(@TempDir & "\autoit_mf.tmp") ; Now the contents will be read as Unicode
    FileDelete(@TempDir & "\autoit_mf.tmp")
Else

Option 3 - Just do a 'BinaryToString' conversion on the text that was read. (May be slower than the above (?), but it just requires one line to be changed:

; Read file and convert the 'UTF16 Little Endian' data to a Unicode string
    $s_hold_out=BinaryToString(FileRead(@TempDir & "\autoit_mf.tmp"),2)

Your modification (with one of the above alterations) now makes the DOS Dir command a viable alternative. Thanks so much for this!

*edit: added option 3

Edited May 13, 2010 by Ascend4nt

[DXRW4E]

Hi All, Sorry for my English, first Thanks for all the work you do, Thank you Very much

check this too in case you find it useful (looks like the original way to get directory/file subdirectory, takes about 9 seconds to make the list in @windowsdir, cmd.exe Dir it should be of about 12 seconds)

Ciao.

Edited July 28, 2011 by DXRW4E