ReDim - When?

[Valik]

Jon, when are we going to get a built-in ReDim keyword that handles multi-dimensional arrays? It's no big deal to write a single dimension ReDim function (I've had one for months and David posted one recently) or even a couple dimensions, but it gets very tedious after 2 or 3 dimensions. Can we get ReDim as the last new big feature in this current unstable?

[Nutster]

I agree. I see a few options in order of desirablity for me:

• Jon finds the ReDim I submitted a few months ago, and integrates into the program.
• I find the version I submitted a few months ago and test to make sure it still works with all the changes we have done to the variable system and then submit that.
• I recreate the keyword in the current unstable, if I can not find the submittion with it. After testing, I submit it to Jon.
• Jon thinks that ReDim is too complicated to integrate into the current version.

[Jon]

I agree.  I see a few options in order of desirablity for me:

• Jon finds the ReDim I submitted a few months ago, and integrates into the program.

• I find the version I submitted a few months ago and test to make sure it still works with all the changes we have done to the variable system and then submit that.

• I recreate the keyword in the current unstable, if I can not find the submittion with it.  After testing, I submit it to Jon.

• Jon thinks that ReDim is too complicated to integrate into the current version.

I can't find it - I only seem to have stuff from March and I think it was before that. JP might have a copy as it was in his unstable for a while.

[Nutster]

I will look for it. At worst I will re-write it on the weekend.

[Nutster]

Could not find it. Nuts! I will re-write on the weekend.

[Jon]

Could not find it.  Nuts!  I will re-write on the weekend. 

Wait until JP says if he has it or not first.

I still hate the idea of redim anyway.

[Nutster]

What do you not like about it? It allows the user to expand or contract an array and preserve the data in the array.

[Jon]

I don't know, just one of those things.

The way arrays in AutoIt work is pretty funky too, the elements aren't actually allocated until they are referenced (either read or write), so if you read each element while doing the "preserve" thing then you could end up using way more memory than if you just started with a much larger array in the first place. I never have chance to study the last code to see if this was how it was implemented so this may have been taken into account.

I think it's just all the C programming and not being able to resize stuff easily that has made me work around it to the point I don't have any use for it - but I'm in the minority on this one so whatever

[jpm]

Could not find it.  Nuts!  I will re-write on the weekend. 

Do you want I look in the dbg-bastard code?

[Nutster]

That could save some work. Yes please. Hmm, do I still the dbg-bastard source lying around here someplace?

[jpm]

That could save some work.  Yes please.  Hmm, do I still the dbg-bastard source lying around here someplace?

I put it at the usual place in dbg-bastard

all extension are proteted if a "#ifdef AUT_REDIM_1" so just scan the sequence related with redim

now is time to your homework

[jlandes]

I put it at the usual place in dbg-bastard

all extension are proteted if a "#ifdef AUT_REDIM_1" so just scan the sequence related with redim

now is time to your homework

I think it would also be cool if the GetLastError() function could be added from the Dbg-Bastard version.

[Nutster]

I have submitted the ReDim functionality to Jon earlier this week (June 10). It should appear in the unstable build fairly soon.

JP: There were enough changes (and I did not download the debug version soon enough) that I rewrote the affected parts from scratch.

Edited June 12, 2004 by Nutster

[pekster]

I've got a question about this built-in function. I had created a UDF to redim, and/or insert elements into an array by creating a new array, moving over the old values, and replacing the original array. As soon as the function exited, the memory was freed up since the old array was only stored in the function locally.

Will the new ReDim function be faster or save memory in any way? If it does, I'd like to speed up my scripts that could be modified. If not, I won't waste my time replacing the code.

[Nutster]

Ok, now that ReDim is in the unstable (Thanks Jon ), we need people to test that it works. I have done my tests and it seems to work. What do others find?

As for speed, it should be a lot faster as it is written entirely in C++, not AutoIt. Make some benchmark tests and tell us (me) what you find.

As for memory used, it is better at managing the memory because I am actually looking at the guts of AutoIt's memory management while doing my copy so that I do not copy elements that I do not have to. Also, I am generally using smaller variables than AutoIt's Variant variable type for most operations.

Also, ReDim keyword handles multiple dimensions in one routine, not one UDF for each size of dimension.

As far as converting your scripts, I would wait until some stability tests have been passed. But that is up to you.

[pekster]

As for speed, it should be a lot faster as it is written entirely in C++, not AutoIt.  Make some benchmark tests and tell us (me) what you find.

Using a stripped down version of my UDF and the ReDim function, I created a benchmark script to test the speed difference between the two. The results varried a bit from run to run, but here are some fairly average numbers I got back. The number represents how many times longer the UDF took to complete the listed number of element additions.

• 10 elements: 4.24 times as slow
• 50 elements: 4.88 times as slow
• 100 elements 26.93 times as slow
• 500 elements 36.09 times as slow
• 1000 elements 29.99 times as slow

I would be interested if anyone on a faster computer could give me a similar list after running my script. These results were generated on a 400 MHz celeron.

[Nutster]

Good! Now try something with multiple dimensions.

Also, when expanding an array, it is better to enlarge it using a few large steps instead of many small steps.

Dim $a[99]
...
Redim $a[UBound($a)+99]

Edited June 14, 2004 by Nutster

[Valik]

These were done on an Athlon XP 3200+ (2.2 ghz). I did have SpeedDisk running in the background so results may be a bit funky. I'll test later after SpeedDisk finishes.

TRIAL 1 with 10 elements

  redim method clocked 0.87

  older method clocked 8.61

  old method was 9.9 times as slow

TRIAL 2 with 50 elements

  redim method clocked 7.24

  older method clocked 52.47

  old method was 7.25 times as slow

TRIAL 3 with 100 elements

  redim method clocked 6.61

  older method clocked 184.62

  old method was 27.93 times as slow

TRIAL 4 with 500 elements

  redim method clocked 102.93

  older method clocked 4629.92

  old method was 44.98 times as slow

TRIAL 5 with 1000 elements

  redim method clocked 488.73

  older method clocked 19823.26

  old method was 40.56 times as slow

[pekster]

Also, when expanding an array, it is better to enlarge it using a few large steps instead of many small steps.

If I was designing it to work quickly, yes. However, this is designed to test how the ReDim function holds up against my UDF, and so I actually wanted to slow it down as much as I could. This way, the more times each method has to expand the array, the more it became clear how slow each one was relative to the other for that number of dimention changes of the test lengths.

[pekster]

Good!  Now try something with multiple dimensions.

Compleated, and avaiable for download. See also my benchmark results below. The max element number per dimension has been limited to 100 because my computer took too long generating the 500 benchmark results. I'm not sure if it's because I have a getto computer, or if 250,000 elements is just a bit too high for my poor UDF to compute on a timely basis. At any rate, the default option (that is without modification) is to only run the 10, 50, and 100 element tests.

TRIAL 1 with 10 elements

  redim method clocked 39.35

  older method clocked 167.73

  old method was 4.26 times as slow

TRIAL 2 with 50 elements

  redim method clocked 551.06

  older method clocked 12917.31

  old method was 23.44 times as slow

TRIAL 3 with 100 elements

  redim method clocked 3789.61

  older method clocked 110642.05

  old method was 29.2 times as slow