Jump to content

Looking for a Faster Folder Sorting Solution


Recommended Posts

Below is the code that gets the directory listing and sorts the directories by order of magnitude from smallest to largest.
With small amount folder it seems to work fine, but with large amount it works a lot of hours:

#RequireAdmin
#include <File.au3>
#include <Array.au3>

Global $PathIN = "C:\"

Global $aFolderList = _FolderListToArraySort($PathIN)
For $f = 1 To $aFolderList[0]
    _ConsoleWrite(">[" & $f & "] File: " & $aFolderList[$f] & @CRLF)
Next

Func _FolderListToArraySort($PathIN)
    Local $aFolderPath, $x = 0
    $PathIN = _PathAddSlash($PathIN)
    _ConsoleWrite("> Get Dir List: " & $PathIN)
    Local $A_List_Dir_IN = _FileListToArrayRec($PathIN, "*", 2, 1, 1, 2)
    _ConsoleWrite("  Done !" & @CRLF)
    ;For $d = 1 To $A_List_Dir_IN[0]
    ;   _ConsoleWrite(">["&$d&"] File: "&$A_List_Dir_IN[$d]&@CRLF)
    ;Next
    If IsArray($A_List_Dir_IN) Then
        _ConsoleWrite("- Number of Dir: " & $A_List_Dir_IN[0] & @CRLF)
        _ConsoleWrite("> Sort Dir ..." & @CRLF)
        _ArrayColInsert($A_List_Dir_IN, 1)
        For $i = 1 To $A_List_Dir_IN[0][0]
            $aFolderPath = StringSplit($A_List_Dir_IN[$i][0], "\")
            ;_ConsoleWrite("-: "&$aFolderPath[0]&@CRLF)
            If _ArraySearch($A_List_Dir_IN, Number($aFolderPath[0] & "." & $x), 1, 0, 0, 0, 1, 1) > -1 Then
                ConsoleWrite("+ " & $i & " / " & $A_List_Dir_IN[0][0] & " : " & $A_List_Dir_IN[$i][0] & @CRLF)
                $x += 1
                $A_List_Dir_IN[$i][1] = Number($aFolderPath[0] & "." & $x)
            Else
                ConsoleWrite("- " & $i & " / " & $A_List_Dir_IN[0][0] & " : " & $A_List_Dir_IN[$i][0] & @CRLF)
                $x = 0
                While _ArraySearch($A_List_Dir_IN, Number($aFolderPath[0] & "." & $x), 1, 0, 0, 0, 1, 1) > -1
                    $x += 1
                WEnd
                $A_List_Dir_IN[$i][1] = Number($aFolderPath[0] & "." & $x)
            EndIf
            $aFolderPath = 0
        Next
        _ArraySort($A_List_Dir_IN, 1, 1, 0, 1)
        ;For $d = 1 To $A_List_Dir_IN[0][0]
        ;   _ConsoleWrite("! [" & $d & "] : " & $A_List_Dir_IN[$d][0] & @CRLF)
        ;Next
        _ArrayColDelete($A_List_Dir_IN, 1, True)
        _ConsoleWrite(@CRLF & "- Sort Dir - DONE !" & @CRLF)
        Return $A_List_Dir_IN
    EndIf
EndFunc   ;==>_FolderListToArraySort

Func _ConsoleWrite($Text)
    ConsoleWrite($Text)
    ;FileWriteLine(@ScriptFullPath & ".log", $Text)
EndFunc   ;==>_ConsoleWrite

Func _PathRemoveSlash($sPath)
    ;_ConsoleWrite(StringRegExpReplace($PathIN, "\\$", "")&@CRLF)
    ;_ConsoleWrite(StringRegExpReplace($PathIN,"(.*)(\\)\z","$1")&@CRLF)
    While (StringRight($sPath, 1) == '\')
        $sPath = StringTrimRight($sPath, 1)
    WEnd
    Return $sPath
EndFunc   ;==>_PathRemoveSlash

Func _PathAddSlash($sPath)
    $sPath = _PathRemoveSlash($sPath)
    If StringRight($sPath, 1) == '\' Then Return $sPath
    Return $sPath & '\'
EndFunc   ;==>_PathAddSlash

;Global $uStringIN = ";!#$%&'()+-,.0123456789=@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{}~"

 

Very happy if you have a solution for this problem, thanks a lot.

Regards,
 

Link to post
Share on other sites

You could check, just to get a comparation time value, how long powershell takes to do that sort for you:

 

Get-ChildItem -file -Recurse -ErrorAction SilentlyContinue | sort length | Format-Table length,fullname -AutoSize

 

Earth is flat, pigs can fly, and Nuclear Power is SAFE!

Link to post
Share on other sites
5 hours ago, rudi said:

You could check, just to get a comparation time value, how long powershell takes to do that sort for you:

 


Get-ChildItem -file -Recurse -ErrorAction SilentlyContinue | sort length | Format-Table length,fullname -AutoSize

 

Your command executes and prints the results in ten minutes.

But it's not my desired sort result.

Regards,
 

Link to post
Share on other sites

Try this one, derived from a similar task I coded already:

 

#include <Array.au3>
#include <Debug.au3>



$BaseDir=@ProgramFilesDir & "\" ; Trailing BACKSLASH! - another " just to fix up the forum code formatting
$deepth=0
$Str4ArrAdd=""
$CurDir=""
$CountDir=0
$CountFile=0

$Start=TimerInit()
dim $A2SizeFPFN[0][2]

_DebugArrayDisplay($A2SizeFPFN)

AdlibRegister("ToolUpdate",2000)

Collector($BaseDir)

AdlibUnRegister("ToolUpdate")
ToolTip("Building up with ArrayAdd")
$Str4ArrAdd=StringTrimRight($Str4ArrAdd,2) ; cut off final @CRLF

_ArrayAdd($A2SizeFPFN,$Str4ArrAdd)
ToolTip("Converting Col-0 back to numbers")
for $i = 0 to UBound($A2SizeFPFN) - 1
    $A2SizeFPFN[$i][0]=Number($A2SizeFPFN[$i][0]) ; convert string back to number
Next

ToolTip("Sorting result array")
_ArraySort($A2SizeFPFN)
ToolTip("Displaying result array")
_DebugArrayDisplay($A2SizeFPFN)
ToolTip("")
MsgBox(0,"Done","Time total: " & Round(TimerDiff($Start)/1000) & "sec." & @CRLF & @CRLF & "The END")

Func Collector($_Dir)
    $CurDir=$_dir ; for ToolTipUpdate
    $CountDir+=1
    $deepth+=1
    Local $s = FileFindFirstFile($_Dir & "*.*")

    if $s = -1 Then
        FileClose($s)
        $deepth-=1
        Return
    Else
        while 1
            $NxFile=FileFindNextFile($s)
            if @error Then ; end reached
                FileClose($s)
                $deepth-=1
                Return
            Else
                if @extended then ; Directory
                    if $NxFile="." or $NxFile=".." Then ContinueLoop ; skip evetually showing up "CurrentDirectory" and "ParentDirectory"
                    Collector($_dir & $NxFile & "\") ; trailing BACKSLASH  - another " just to fix up the forum code formatting
                Else ; file
                    $CountFile+=1
                    $Str4ArrAdd&= FileGetSize($_Dir & $NxFile) & "|" & $_Dir & $NxFile &  @CRLF
                EndIf
            EndIf
        WEnd
    EndIf
EndFunc

Func ToolUpdate()
    ToolTip("Depth: " & $deepth & @CRLF & "Dirs so far: " & $CountDir & @CRLF & "Files so far: " & $CountFile & @CRLF & "Current Dir: " & $CurDir)
EndFunc

 

what time does that one take?

Edited by rudi
added " to fixup forum code formatting

Earth is flat, pigs can fly, and Nuclear Power is SAFE!

Link to post
Share on other sites

Your script doesn't sort as I want:

Sort Output:
|   |   |   |   |   |   |   \---Level-n-
|   |   |   |   |   |   \---Level-8-
|   |   |   |   |   \---Level-7-
|   |   |   |   \---Level-6-
|   |   |   \---Level-5-
|   |   \---Level-4-
|   \---Level-3-
\---Level-2-
Level-1

EG:

C:\Level-1-\Level-2-\Level-3-\Level-4-\Level-5-\Level-6-\Level-7-\Level-8-\Level-n
C:\Level-1-\Level-2-\Level-3-\Level-4-\Level-5-\Level-6-\Level-7-\Level-8-\
C:\Level-1-\Level-2-\Level-3-\Level-4-\Level-5-\Level-6-\Level-7-\
C:\Level-1-\Level-2-\Level-3-\Level-4-\Level-5-\Level-6-\
C:\Level-1-\Level-2-\Level-3-\Level-4-\Level-5-\
C:\Level-1-\Level-2-\Level-3-\Level-4-\
C:\Level-1-\Level-2-\Level-3-\
C:\Level-1-\Level-2-\
C:\Level-1-\

 

Regards,
 

Link to post
Share on other sites

First of all, could you kindly tell the time this script takes?

If I got correctly your request in the the original post you wanted a "sort by size ascending order", isn't it?
Honestly, I miss to get what you want exactly...

 

#2: You have made over 1000 postings since 5½ years now:

I think you can sort out how to modify the output to meet your special needs, at least you might want to post some code trying to do so?

 

 

Earth is flat, pigs can fly, and Nuclear Power is SAFE!

Link to post
Share on other sites

My take is that you want to "sort by size (number of subfolders) in descending order".  For the same number of subfolders, the order of the directories seems random.

Func _FolderListToArraySort2($sPath)
  Local $aFolder = _FileListToArrayRec($sPath, "*", $FLTAR_FOLDERS, $FLTAR_RECUR, $FLTAR_NOSORT, $FLTAR_FULLPATH)
  ;_ArrayDisplay($aFolder)
  Local $aSize[UBound($aFolder - 1)][2]
  For $i = 1 to $aFolder[0]
    $aSize[$i - 1][0] = $aFolder[$i]
    StringReplace($aFolder[$i], "\", "\")
    $aSize[$i - 1][1] = @extended
  Next
  _ArraySort($aSize, 1, Default, Default, 1)
  _ArrayColDelete($aSize, 1, True)
  ;_ArrayDisplay($aSize)
  Return $aSize
EndFunc

About 3 times faster than OP

Edited by Nine
Link to post
Share on other sites

Little typo @Nine, line 2 s$Path instead of $sPath.

Best regards
Sven

Stay innovative!

Spoiler

🌍 Au3Forums

📊 AutoIt limits/defaults

💎 Code Katas: [...] (comming soon)

🎭 Collection of GitHub users with AutoIt projects

🐞 False-Positives

🔍 Forum search

🔮 Me on GitHub

💬 Opinion about new forum sub category

 📑 UDF wiki list

✂ VSCode-AutoItSnippets

📑 WebDriver FAQs

👨‍🏫 WebDriver Tutorial (coming soon)

Link to post
Share on other sites
Link to post
Share on other sites

Thanks for everyone's help.
The purpose of getting the directory tree from the top level is that synology's synchronization software does not work with unicode files and directories (English). So I need to get the folder name and convert unicode characters to ASCII (Vietnamese can do that)

I'm pretty bad at logic up it doesn't matter I have 10 or 1000 posts. I am still the same, of course my coding skills have improved a lot and are getting better day by day. But writing and programming is not my main job. I use AutoIt to make my life easier, I haven't made any money from coding.
With difficult problems, I am often lazy when I am powerless, I come here and ask my seniors for help, in my free time I also look over and help other friends in my ability.

Regards,
 

Link to post
Share on other sites

Do I get your last posting correctly, that you basically want to rename folder names (what about the files, that contain unicode in there names as well) to a particular encoding?

What about the initial request "sort by size ascending" ?

 

If your task basically is about Synology-to-Synology replication, I would expect, that same firmware versions and adding the same NLS (national language support) packages to both boxes should end up in a smooth replication without any constraints?

 

And one more try: How long does the sample code from above run?

I think the main difference is, that the reduced number and complexity of array operations I used should consume far less time 

Edited by rudi
typo

Earth is flat, pigs can fly, and Nuclear Power is SAFE!

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...