Looking for a Faster Folder Sorting Solution

[Trong]

Below is the code that gets the directory listing and sorts the directories by order of magnitude from smallest to largest.
With small amount folder it seems to work fine, but with large amount it works a lot of hours:

expandcollapse popup

#RequireAdmin
#include <File.au3>
#include <Array.au3>

Global $PathIN = "C:\"

Global $aFolderList = _FolderListToArraySort($PathIN)
For $f = 1 To $aFolderList[0]
    _ConsoleWrite(">[" & $f & "] File: " & $aFolderList[$f] & @CRLF)
Next

Func _FolderListToArraySort($PathIN)
    Local $aFolderPath, $x = 0
    $PathIN = _PathAddSlash($PathIN)
    _ConsoleWrite("> Get Dir List: " & $PathIN)
    Local $A_List_Dir_IN = _FileListToArrayRec($PathIN, "*", 2, 1, 1, 2)
    _ConsoleWrite("  Done !" & @CRLF)
    ;For $d = 1 To $A_List_Dir_IN[0]
    ;   _ConsoleWrite(">["&$d&"] File: "&$A_List_Dir_IN[$d]&@CRLF)
    ;Next
    If IsArray($A_List_Dir_IN) Then
        _ConsoleWrite("- Number of Dir: " & $A_List_Dir_IN[0] & @CRLF)
        _ConsoleWrite("> Sort Dir ..." & @CRLF)
        _ArrayColInsert($A_List_Dir_IN, 1)
        For $i = 1 To $A_List_Dir_IN[0][0]
            $aFolderPath = StringSplit($A_List_Dir_IN[$i][0], "\")
            ;_ConsoleWrite("-: "&$aFolderPath[0]&@CRLF)
            If _ArraySearch($A_List_Dir_IN, Number($aFolderPath[0] & "." & $x), 1, 0, 0, 0, 1, 1) > -1 Then
                ConsoleWrite("+ " & $i & " / " & $A_List_Dir_IN[0][0] & " : " & $A_List_Dir_IN[$i][0] & @CRLF)
                $x += 1
                $A_List_Dir_IN[$i][1] = Number($aFolderPath[0] & "." & $x)
            Else
                ConsoleWrite("- " & $i & " / " & $A_List_Dir_IN[0][0] & " : " & $A_List_Dir_IN[$i][0] & @CRLF)
                $x = 0
                While _ArraySearch($A_List_Dir_IN, Number($aFolderPath[0] & "." & $x), 1, 0, 0, 0, 1, 1) > -1
                    $x += 1
                WEnd
                $A_List_Dir_IN[$i][1] = Number($aFolderPath[0] & "." & $x)
            EndIf
            $aFolderPath = 0
        Next
        _ArraySort($A_List_Dir_IN, 1, 1, 0, 1)
        ;For $d = 1 To $A_List_Dir_IN[0][0]
        ;   _ConsoleWrite("! [" & $d & "] : " & $A_List_Dir_IN[$d][0] & @CRLF)
        ;Next
        _ArrayColDelete($A_List_Dir_IN, 1, True)
        _ConsoleWrite(@CRLF & "- Sort Dir - DONE !" & @CRLF)
        Return $A_List_Dir_IN
    EndIf
EndFunc   ;==>_FolderListToArraySort

Func _ConsoleWrite($Text)
    ConsoleWrite($Text)
    ;FileWriteLine(@ScriptFullPath & ".log", $Text)
EndFunc   ;==>_ConsoleWrite

Func _PathRemoveSlash($sPath)
    ;_ConsoleWrite(StringRegExpReplace($PathIN, "\\$", "")&@CRLF)
    ;_ConsoleWrite(StringRegExpReplace($PathIN,"(.*)(\\)\z","$1")&@CRLF)
    While (StringRight($sPath, 1) == '\')
        $sPath = StringTrimRight($sPath, 1)
    WEnd
    Return $sPath
EndFunc   ;==>_PathRemoveSlash

Func _PathAddSlash($sPath)
    $sPath = _PathRemoveSlash($sPath)
    If StringRight($sPath, 1) == '\' Then Return $sPath
    Return $sPath & '\'
EndFunc   ;==>_PathAddSlash

;Global $uStringIN = ";!#$%&'()+-,.0123456789=@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{}~"

 

Very happy if you have a solution for this problem, thanks a lot.

[rudi]

You could check, just to get a comparation time value, how long powershell takes to do that sort for you:

 

Get-ChildItem -file -Recurse -ErrorAction SilentlyContinue | sort length | Format-Table length,fullname -AutoSize

 

[Trong]

5 hours ago, rudi said:

You could check, just to get a comparation time value, how long powershell takes to do that sort for you:

 

Get-ChildItem -file -Recurse -ErrorAction SilentlyContinue | sort length | Format-Table length,fullname -AutoSize

 

Your command executes and prints the results in ten minutes.

But it's not my desired sort result.

[rudi]

Try this one, derived from a similar task I coded already:

 

expandcollapse popup

#include <Array.au3>
#include <Debug.au3>



$BaseDir=@ProgramFilesDir & "\" ; Trailing BACKSLASH! - another " just to fix up the forum code formatting
$deepth=0
$Str4ArrAdd=""
$CurDir=""
$CountDir=0
$CountFile=0

$Start=TimerInit()
dim $A2SizeFPFN[0][2]

_DebugArrayDisplay($A2SizeFPFN)

AdlibRegister("ToolUpdate",2000)

Collector($BaseDir)

AdlibUnRegister("ToolUpdate")
ToolTip("Building up with ArrayAdd")
$Str4ArrAdd=StringTrimRight($Str4ArrAdd,2) ; cut off final @CRLF

_ArrayAdd($A2SizeFPFN,$Str4ArrAdd)
ToolTip("Converting Col-0 back to numbers")
for $i = 0 to UBound($A2SizeFPFN) - 1
    $A2SizeFPFN[$i][0]=Number($A2SizeFPFN[$i][0]) ; convert string back to number
Next

ToolTip("Sorting result array")
_ArraySort($A2SizeFPFN)
ToolTip("Displaying result array")
_DebugArrayDisplay($A2SizeFPFN)
ToolTip("")
MsgBox(0,"Done","Time total: " & Round(TimerDiff($Start)/1000) & "sec." & @CRLF & @CRLF & "The END")

Func Collector($_Dir)
    $CurDir=$_dir ; for ToolTipUpdate
    $CountDir+=1
    $deepth+=1
    Local $s = FileFindFirstFile($_Dir & "*.*")

    if $s = -1 Then
        FileClose($s)
        $deepth-=1
        Return
    Else
        while 1
            $NxFile=FileFindNextFile($s)
            if @error Then ; end reached
                FileClose($s)
                $deepth-=1
                Return
            Else
                if @extended then ; Directory
                    if $NxFile="." or $NxFile=".." Then ContinueLoop ; skip evetually showing up "CurrentDirectory" and "ParentDirectory"
                    Collector($_dir & $NxFile & "\") ; trailing BACKSLASH  - another " just to fix up the forum code formatting
                Else ; file
                    $CountFile+=1
                    $Str4ArrAdd&= FileGetSize($_Dir & $NxFile) & "|" & $_Dir & $NxFile &  @CRLF
                EndIf
            EndIf
        WEnd
    EndIf
EndFunc

Func ToolUpdate()
    ToolTip("Depth: " & $deepth & @CRLF & "Dirs so far: " & $CountDir & @CRLF & "Files so far: " & $CountFile & @CRLF & "Current Dir: " & $CurDir)
EndFunc

 

what time does that one take?

Edited January 12, 2023 by rudi
added " to fixup forum code formatting

[Trong]

Your script doesn't sort as I want:

Sort Output:
|   |   |   |   |   |   |   \---Level-n-
|   |   |   |   |   |   \---Level-8-
|   |   |   |   |   \---Level-7-
|   |   |   |   \---Level-6-
|   |   |   \---Level-5-
|   |   \---Level-4-
|   \---Level-3-
\---Level-2-
Level-1

EG:

C:\Level-1-\Level-2-\Level-3-\Level-4-\Level-5-\Level-6-\Level-7-\Level-8-\Level-n
C:\Level-1-\Level-2-\Level-3-\Level-4-\Level-5-\Level-6-\Level-7-\Level-8-\
C:\Level-1-\Level-2-\Level-3-\Level-4-\Level-5-\Level-6-\Level-7-\
C:\Level-1-\Level-2-\Level-3-\Level-4-\Level-5-\Level-6-\
C:\Level-1-\Level-2-\Level-3-\Level-4-\Level-5-\
C:\Level-1-\Level-2-\Level-3-\Level-4-\
C:\Level-1-\Level-2-\Level-3-\
C:\Level-1-\Level-2-\
C:\Level-1-\

 

[rudi]

First of all, could you kindly tell the time this script takes?

If I got correctly your request in the the original post you wanted a "sort by size ascending order", isn't it?
Honestly, I miss to get what you want exactly...

 

#2: You have made over 1000 postings since 5½ years now:

I think you can sort out how to modify the output to meet your special needs, at least you might want to post some code trying to do so?

 

 

[Nine]

My take is that you want to "sort by size (number of subfolders) in descending order".  For the same number of subfolders, the order of the directories seems random.

Func _FolderListToArraySort2($sPath)
  Local $aFolder = _FileListToArrayRec($sPath, "*", $FLTAR_FOLDERS, $FLTAR_RECUR, $FLTAR_NOSORT, $FLTAR_FULLPATH)
  ;_ArrayDisplay($aFolder)
  Local $aSize[UBound($aFolder - 1)][2]
  For $i = 1 to $aFolder[0]
    $aSize[$i - 1][0] = $aFolder[$i]
    StringReplace($aFolder[$i], "\", "\")
    $aSize[$i - 1][1] = @extended
  Next
  _ArraySort($aSize, 1, Default, Default, 1)
  _ArrayColDelete($aSize, 1, True)
  ;_ArrayDisplay($aSize)
  Return $aSize
EndFunc

About 3 times faster than OP

Edited January 13, 2023 by Nine

[SOLVE-SMART]

Little typo @Nine, line 2 s$Path instead of $sPath.

Best regards
Sven

[pixelsearch]

1 hour ago, Nine said:

For the same number of subfolders, the order of the directories seems random.

Great job, Nine.
Could this fix it, so it's not random anymore ?

; $aSize[$i - 1][1] = @extended
$aSize[$i - 1][1] = StringFormat("%3s", @extended) & $aFolder[$i]

 

[Nine]

What I meant is that OP does not seem to want any kind of sort for a specific number of subfolders.  A question more than a statement in fact...

[Trong]

Thanks for everyone's help.
The purpose of getting the directory tree from the top level is that synology's synchronization software does not work with unicode files and directories (English). So I need to get the folder name and convert unicode characters to ASCII (Vietnamese can do that)

I'm pretty bad at logic up it doesn't matter I have 10 or 1000 posts. I am still the same, of course my coding skills have improved a lot and are getting better day by day. But writing and programming is not my main job. I use AutoIt to make my life easier, I haven't made any money from coding.
With difficult problems, I am often lazy when I am powerless, I come here and ask my seniors for help, in my free time I also look over and help other friends in my ability.

[rudi]

Do I get your last posting correctly, that you basically want to rename folder names (what about the files, that contain unicode in there names as well) to a particular encoding?

What about the initial request "sort by size ascending" ?

 

If your task basically is about Synology-to-Synology replication, I would expect, that same firmware versions and adding the same NLS (national language support) packages to both boxes should end up in a smooth replication without any constraints?

 

And one more try: How long does the sample code from above run?

I think the main difference is, that the reduced number and complexity of array operations I used should consume far less time ❓

Edited January 17, 2023 by rudi
typo