Sign in to follow this  
Followers 0

RecFileListToArray - Deprecated

329 posts in this topic




Posted

Whoa, a really useful script! Maybe for a explorer look-a-like with grouped extensions?

Share this post


Link to post
Share on other sites

Posted

Well done, i impressed by the speed of that UDF!

But here is a few problems and remarks...

1. When i run this example:

#include "RecFileListToArray.au3"

; A sorted list of all files and folders in the Program files dir
$aArray = _RecFileListToArray(@ProgramFilesDir, "*", 0, 1, 1, 2)

i got this error:

RecFileListToArray.au3 (494) : ==> Array variable has incorrect number of subscripts or subscript dimension range exceeded.:
$avArray[$iElement] = $vValue
^ ERROR

2. StringRegExpReplace($sInitialPath, "\\", "") better replace with regular StringReplace function, i.e:

StringReplace($sInitialPath, "\", "")
...
StringReplace($sCurrentPath, "\", "")

3. You can increase the speed even more, if you use this construction:

If $hSearch = -1 Then
    ContinueLoop
EndIf

instead of this:

If $hSearch = -1 Then ContinueLoop

Especialy in loops.

Share this post


Link to post
Share on other sites

Posted

MrCreatoR,

Thanks for the comments. I worked very hard on the speed - you would not have been impressed by the early versions. :D

- 1. Array variable has incorrect number of subscripts or subscript dimension range exceeded. I do not get that error when I run exactly the same code as you - so I cannot help there unless I can have some more clues. :)

- 2. StringRegExpReplace vs StringReplace. Why do you suggest changing? :)

- 3. Single line If vs If...EndIf. I did some timing tests on the various forms of If compared to Switch and Select some time ago and (on my machine at least) the single line If was faster then the If...End structure - although not by a lot. :P

M23

Share this post


Link to post
Share on other sites

Posted (edited)

I would even suggest to replace it with StringReplace($sInitialPath, "\", "", 0, 2).

$sInitialPath = @ScriptFullPath

$timer = TimerInit()
For $i = 0 To 100000
    StringRegExpReplace($sInitialPath, "\\", "")
Next
ConsoleWrite(TimerDiff($timer) & @CRLF)

$timer = TimerInit()
For $i = 0 To 100000
    StringReplace($sInitialPath, "\", "")
Next
ConsoleWrite(TimerDiff($timer) & @CRLF)

$timer = TimerInit()
For $i = 0 To 100000
    StringReplace($sInitialPath, "\", "", 0, 2)
Next
ConsoleWrite(TimerDiff($timer) & @CRLF)

Edit: Maybe you want to check every standard function call supporting a "casesense" flag, as imho if you know the case (e.g. like "\" has no case) setting the flag to "2 = not case sensitive" is always much faster.

Edited by KaFu

Share this post


Link to post
Share on other sites

Posted

KaFu,

Interesting timings -

SRER      = 1310
SR        = 1248
SR flag 2 = 557

I will keep that in mind the next time I need to count character occurences in a string! :)

But as the line is used just the one time in this UDF, it would only save around 0.05 ms - I am not sure that even you would notice the difference. :)

M23

Share this post


Link to post
Share on other sites

Posted (edited)

I think there are more lines which could be replaced as imho they do not need to be checked for case sensitivity...

StringRegExpReplace($sInitialPath, "\\", "")

$sRetPath = StringReplace($sCurrentPath, $sInitialPath, "")

StringRegExpReplace($sCurrentPath, "\\", "")

$sList = StringReplace(StringStripWS(StringRegExpReplace($sList, "\s*;\s*", ";"), 3), ";", "|")

$sList = StringReplace(StringReplace(StringRegExpReplace($sList, "(\^|\$|\.)", "\\$1"), "?", "."), "*", ".*?") <<< double replacement possible

Maybe also

If StringInStr($avArray[$i], $vValue) > 0 Then Return $i

StringCompare() ?

And as you're defining array[0] = count, you could also drop some ubound() calls :).

Edited by KaFu

Share this post


Link to post
Share on other sites

Posted

KaFu,

The third one of those is probably worth doing as it is a loop - I will keep it in mind for the next update. Danke sehr. :)

M23

Share this post


Link to post
Share on other sites

Posted

...

- 1. Array variable has incorrect number of subscripts or subscript dimension range exceeded. I do not get that error when I run exactly the same code as you - so I cannot help there unless I can have some more clues. :)

...

It runs properly on my system, too! Win7 SP1 x64.

Br,

UEZ

Share this post


Link to post
Share on other sites

Posted

I do not get that error when I run exactly the same code as you - so I cannot help there unless I can have some more clues

It's probably related to mount of files in the folder (i got 16519 files in my program files folder), you should never write code in the way that will trigger this kind of errors, use something like this:

Func _RFLTA_ArrayInsert(ByRef $avArray, $iElement, $vValue = "")

    Local $iUBound = UBound($avArray) + 1
    ReDim $avArray[$iUBound]
    For $i = $iUBound - 1 To $iElement + 1 Step -1
        $avArray[$i] = $avArray[$i - 1]
    Next
    
    If $iElement >= $iUBound Then
        ReDim $avArray[$iElement+1]
    EndIf
    
    $avArray[$iElement] = $vValue

EndFunc

Btw, it's related to sort parameter, if it's 0 then no errors.

StringRegExpReplace vs StringReplace. Why do you suggest changing?

Why to use RegExp when it's not needed?

I did some timing tests on the various forms of If compared to Switch and Select some time ago and (on my machine at least) the single line If was faster then the If...End structure

A while ago i thought that way too, but i did some new tests and they shows other results.

Share this post


Link to post
Share on other sites

Posted (edited)

I thought the same as Melba23 too, because I asked this question (about If Statements) a while back. It also it seems logical, less lines = more performance. But testing this I get a difference result to what I would have expected :)

Global $Variable = 1, $Timer
$Timer = TimerInit()
If $Variable = 1 Then $Variable = 1
ConsoleWrite(TimerDiff($Timer) & @CRLF) ; Average = 0.00830911064656123

$Timer = TimerInit()
If $Variable = 1 Then
    $Variable = 1
EndIf
ConsoleWrite(TimerDiff($Timer) & @CRLF) ; Average = 0.00586525457404322

Note: Of course its not humanly noticeable!

Edited by guinness

Share this post


Link to post
Share on other sites

Posted

@guiness

I modified the tests a bit, running each one 100 times and recording the results and running both test segments 100 times and comparing the results. So after running the tests 10,000 each the results are pretty consistent that the second test is a tiny bit faster. I don't think changing one for the other will affect much of anything in the long run unless you have 1000's of comparisons of this sort to run, but it's fun to find out the facts behind the conjecture.

Here's the code I used

Global $Variable = 1, $Timer
Global $File = FileOpen("C:\Test.txt", 2)
For $X = 1 To 100
    $Timer = TimerInit()
    For $I = 1 To 100
        If $Variable = 1 Then $Variable = 1
    Next
    FileWrite($File, TimerDiff($Timer) & " -------- ")
    $Timer = TimerInit()
    For $I = 1 To 100
        If $Variable = 1 Then
            $Variable = 1
        EndIf
    Next
    FileWriteLine($File, TimerDiff($Timer) & @CRLF)
Next

And here's the results from one test run. Some of the results are very different than the rest probably due to the slow computer I'm running it on and hard drive access.

0.556495308761309 -------- 1.66697164025037

3.43647027764702 -------- 0.253384159159893

0.258971461456694 -------- 0.171809545626609

0.258971461456694 -------- 0.171809545626609

0.258412731227013 -------- 0.172088910741449

0.258692096341853 -------- 0.171809545626609

0.258692096341853 -------- 0.171530180511769

0.258133366112173 -------- 0.171530180511769

0.258692096341853 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.258133366112173 -------- 0.171809545626609

0.258412731227013 -------- 0.171530180511769

0.258412731227013 -------- 0.171809545626609

0.258133366112173 -------- 0.172368275856289

0.258971461456694 -------- 0.171809545626609

0.258971461456694 -------- 0.171530180511769

0.258412731227013 -------- 0.171809545626609

0.258692096341853 -------- 0.172088910741449

0.258692096341853 -------- 0.171809545626609

0.258133366112173 -------- 0.171809545626609

0.258692096341853 -------- 0.171809545626609

0.258692096341853 -------- 0.171530180511769

0.258133366112173 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.258692096341853 -------- 0.171809545626609

0.257854000997333 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.258412731227013 -------- 0.171250815396929

0.258971461456694 -------- 0.171809545626609

0.258692096341853 -------- 0.171809545626609

0.310374642587256 -------- 0.171809545626609

0.258971461456694 -------- 0.171809545626609

0.258692096341853 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.258692096341853 -------- 0.171530180511769

0.258692096341853 -------- 0.171809545626609

0.258133366112173 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.257854000997333 -------- 0.171809545626609

0.258412731227013 -------- 0.171530180511769

0.259530191686374 -------- 0.171809545626609

0.260088921916054 -------- 0.171809545626609

0.259250826571534 -------- 0.171530180511769

0.258692096341853 -------- 0.171809545626609

0.258133366112173 -------- 0.171809545626609

0.258971461456694 -------- 0.171809545626609

0.258971461456694 -------- 0.171530180511769

0.258133366112173 -------- 0.171809545626609

0.258692096341853 -------- 0.171530180511769

0.258692096341853 -------- 0.171809545626609

0.258133366112173 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.257854000997333 -------- 0.171809545626609

0.258133366112173 -------- 0.172368275856289

0.258971461456694 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.258692096341853 -------- 0.171530180511769

0.258971461456694 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.258692096341853 -------- 0.171809545626609

0.352838140042938 -------- 0.172088910741449

0.258412731227013 -------- 0.171809545626609

0.258692096341853 -------- 0.171809545626609

0.258692096341853 -------- 0.171809545626609

0.258133366112173 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.258692096341853 -------- 0.171530180511769

0.257854000997333 -------- 0.171530180511769

0.258133366112173 -------- 0.171809545626609

0.258971461456694 -------- 0.171809545626609

0.258412731227013 -------- 0.171530180511769

0.258692096341853 -------- 0.171530180511769

0.258971461456694 -------- 0.171809545626609

0.258133366112173 -------- 0.171809545626609

0.258692096341853 -------- 0.171809545626609

0.258692096341853 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.258412731227013 -------- 0.171809545626609

0.257854000997333 -------- 0.171809545626609

0.258692096341853 -------- 0.171809545626609

0.258133366112173 -------- 0.172088910741449

0.258971461456694 -------- 0.171809545626609

0.258971461456694 -------- 0.171809545626609

0.258971461456694 -------- 0.171809545626609

0.258692096341853 -------- 0.171809545626609

0.258692096341853 -------- 0.172088910741449

0.258971461456694 -------- 0.171809545626609

0.258412731227013 -------- 0.172088910741449

0.258692096341853 -------- 0.172088910741449

0.258692096341853 -------- 0.171809545626609

0.258133366112173 -------- 0.171809545626609

0.372393698081739 -------- 0.171809545626609

0.258692096341853 -------- 0.171809545626609

0.258133366112173 -------- 0.172088910741449

0.294730196156215 -------- 0.171809545626609

0.259809556801214 -------- 0.171530180511769

Share this post


Link to post
Share on other sites

Posted

MrCreatoR,

- 1. _ArrayInsert function.

I do not think it is the number of files - I have nearly 70000 in my Programs folder and, although it takes some time, the UDF sorts them all perfectly well.

Thanks for the coding suggestion, but it is not a valid one in this case - and I do try to put errorchecking in my code when it is needed. :P

The value of $iElement passed to the function is the index of an existing array counting up from the max to 1, so it must be less that the current size of the array. And you can seee that the lines prior to the one which threw the error actually run from the new increased max to the value of $iElement. As I check in the main code to prevent -1 being passed, I am at a loss to explain to what might be happening. As it seems to be a problem unique to your machine and folder structure, would you mind running the code again with an added line at the very beginning of the function:

ConsoleWrite($iElement & @CRLF)

and seeing what value is passed when it crashes? Thanks in advance. :)

- 2. SRER vs StringReplace. Good point - when I was writing this I was deep in my continuing struggle to learn about the dratted things and therefore an SRER jumped to mind. Having seen KaFu's results above I will change my coding strategy from now on. :D

- 3. If structure differences. It seems to be very machine/code dependant. I remember going through all of my scripts checking that I had removed all the single item If..EndIf statements - looks like I might have to go back and do it again, but the other way round! ;)

; -------

guinness and BrewManNH,

Go start your own topic if you want to discuss If speeds! :)

M23

Share this post


Link to post
Share on other sites

Posted

Go start your own topic if you want to discuss If speeds!

Sorry :) I use this UDF quite a bit :) so Thanks!

Share this post


Link to post
Share on other sites

Posted (edited)

would you mind running the code again with an added line at the very beginning of the function:

I added theese lines:

ConsoleWrite("Index:" & $iElement & @LF)
    ConsoleWrite("Ubound: " & UBound($avArray) & @LF)

Here is what i got:

Index: 16569
Ubound: 13205

as you can see, the index is grater than ubound.

You can reproduce the problem like this:

#include <Array.au3>
#include "RecFileListToArray.au3"

DirCreate("C:\Test\Folder\SubFolder")

$aArray = _RecFileListToArray("C:\Test", "*", 0, 1, 1, 1)
_ArrayDisplay($aArray)
Edited by MrCreatoR

Share this post


Link to post
Share on other sites

Posted

MrCreatoR,

Thank you so much for that - I have identified the problem. It was entirely of my own making - I was using the [0] element as a count and increased it too early. :)

I now use UBound and the problem is solved - for the test code you posted, at least. This error only seemed to occur if there were empty folders with subfolders at the end of the tree, which was not a scenario I had tested - I love how people manage to find dusty corners of the envelope to explore as soon as you release something! :P

New version coming as soon as I can post it - and I promise to change some SRERs as well to keep you and KaFu happy! :)

M23

Share this post


Link to post
Share on other sites

Posted

New Version - 6 Mar 2011

Fixed - crash when sorting empty folders with subfolders at the end of the tree (thanks MrCreatoR)

Changed - Should be even faster with some StringReplace and If structure changes (thanks KaFu and guinness)

New UDF and zip in first post. :)

M23

Share this post


Link to post
Share on other sites

Posted (edited)

Ooops!

If you downloaded the new version before 1220 UTC Sun 6 Mar, please do it again - the wrong file was in the zip! :)

M23

Edit: Added time stamp as the initial panic is over. :)

Edited by Melba23

Share this post


Link to post
Share on other sites

Posted

Melba,

greate UDF - one of my favs! :)

You changed "Local $iLastIndex = $asReturnList[0]" (old version)

to "Local $iLastIndex = UBound($asReturnList)" (new version).

Is "$asReturnList" a one-based array, isn't it?

Shouldn't it be "Local $iLastIndex = UBound($asReturnList) - 1"?

I' am I wrong?

Greets,

-supersonic.

Share this post


Link to post
Share on other sites

Posted

supersonic,

Shouldn't it be "Local $iLastIndex = UBound($asReturnList) - 1"?

No! :P

Explanation:

The reason it is called $iLastIndex is not because it is the final index in the array, but the most recent index used to insert an element into the array.

I needed to change its initial setting value to prevent the crash which MrCreatoR found. As you point out the old code looked at the count (and I had added the count too early in the process :) ) so in some very special circumstances I was trying to insert the new element beyond the end of the array. By using UBound($asReturnList) I know that the new element will always fit as the returned value will always match the final element index of the array once ReDim has added an additional element to hold the new value - which might well be at the end of the array.

I hope that is clear enough. :)

M23

Share this post


Link to post
Share on other sites
This topic is now closed to further replies.
Sign in to follow this  
Followers 0

  • Recently Browsing   0 members

    No registered users viewing this page.