Why are arrays in AutoIt so screwed up?

[Telamon]

Totally new to AutoIt. Lots of experience with other languages. I'm finding the array implementation in AutoIt to be a source of constant frustration.

I'm maintaining a list of items. I would like to write this:

dim $oResults[0]

_ArrayAdd($oResults, $oItem)

ConsoleWrite(UBound($oResults)) ; prints 1, because I put one item in the array

That's not legal, though. I have to do this:

dim $oResults[1]

_ArrayAdd($oResults, $oItem)

ConsoleWrite(UBound($oResults)) ; prints 2, because I sent a 1-element array to a function that appends an element. Ok.

But now if I want to pass this array to another function that does something to each element in the array, I have to remember that the first element is bullshit and should be ignored.

I also tried the following, which is legal code, but barfs since IsArray($oResults) = false.

dim $oResults = ""

_ArrayAdd($oResults, $oItem)

So it appears the dummy element is required. But this sucks, because I now have to have some dummy element in every array I want to use with _ArrayAdd. I can't ever user _ArrayAdd to add the first item to an array.

Someone justify this design choice. I don't see how someone could be simultaneously smart enough to create AutoIt while overlooking a something so basic, so I assume there is a good reason for why things are the way they are.

[Jos]

good first post .... (for making friend here )

_ArrayAdd() is a UDF (User Defined Function) that should/could make live easier. Now since you seem to know about programming, why don't you just Dim or ReDim the Array to its proper dimensions a fill the array with your own code ?

Jos

Edited January 30, 2008 by Jos

[PsaltyDS]

Totally new to AutoIt. Lots of experience with other languages. I'm finding the array implementation in AutoIt to be a source of constant frustration.

I'm maintaining a list of items. I would like to write this:

dim $oResults[0]

_ArrayAdd($oResults, $oItem)

ConsoleWrite(UBound($oResults)) ; prints 1, because I put one item in the array

That's not legal, though. I have to do this:

dim $oResults[1]

_ArrayAdd($oResults, $oItem)

ConsoleWrite(UBound($oResults)) ; prints 2, because I sent a 1-element array to a function that appends an element. Ok.

But now if I want to pass this array to another function that does something to each element in the array, I have to remember that the first element is bullshit and should be ignored.

I also tried the following, which is legal code, but barfs since IsArray($oResults) = false.

dim $oResults = ""

_ArrayAdd($oResults, $oItem)

So it appears the dummy element is required. But this sucks, because I now have to have some dummy element in every array I want to use with _ArrayAdd. I can't ever user _ArrayAdd to add the first item to an array.

Someone justify this design choice. I don't see how someone could be simultaneously smart enough to create AutoIt while overlooking a something so basic, so I assume there is a good reason for why things are the way they are.

All those other languages and you never learned to use a syntax checker? You can't declare an empty array in AutoIt, as you should have seen if you tried it:

>Running:(3.2.10.0):C:\Program Files\AutoIt3\autoit3.exe "C:\Temp\Test\Test1.au3"   
C:\Temp\Test\Test1.au3 (3) : ==> Array variable subscript badly formatted.: 
Global $oResults[0] 
Global $oResults[^ ERROR

What's wrong with:

Global $oResults[1] = [$oItem]
ConsoleWrite(UBound($oResults)) ; prints 1, because I put one item in the array

[Telamon]

good first post .... (for making friend here )

_ArrayAdd() is a UDF (User Defined Function) that should/could make live easier. Now since you seem to know about programming, why don;t you just Dim or ReDim the Array to its proper dimensions a fill the array with your own code ?

Jos

I'm extracting all links from a page and I'd like to iterate through them once, adding them to the list.

Otherwise I have to loop twice, the first time to find all the links so that I know how big the array needs to be.

[Telamon]

Yes I pointed out in my original post that you can't declare empty arrays. My point is that

Dim $arr[0] SHOULD be legal for very good reasons.

[Jos]

I'm extracting all links from a page and I'd like to iterate through them once, adding them to the list.

Otherwise I have to loop twice, the first time to find all the links so that I know how big the array needs to be.

No need to worry about the size, just define enough "buckets" and keep count on how many you used. you can then either redim the array at the end of you routine or use the counter to check them.

_ArrayAdd() just redim(s) the array everytime, which is an option too if speed isn't an issue.

Func _ArrayAdd(ByRef $avArray, $sValue)
    If IsArray($avArray) Then
        ReDim $avArray[UBound($avArray) + 1]
        $avArray[UBound($avArray) - 1] = $sValue
        SetError(0)
        Return 1
    Else
        SetError(1)
        Return 0
    EndIf
EndFunc ;==>_ArrayAdd

Edited January 30, 2008 by Jos

[PsaltyDS]

Yes I pointed out in my original post that you can't declare empty arrays. My point is that

Dim $arr[0] SHOULD be legal for very good reasons.

I'm not a programmer, so I'll assume Jon, and Jos, and Valik, etc. would know if that was true. Since they didn't, I'll assume it isn't.

As for your actual problem, it has been found that _ArrayAdd() is awful slow. Every function that values speed avoids that UDF. The trick is to assemble your collection as a string (up to a couple hundred MB) with a good delimiter, like Chr(1). Then you just split it once: $avResults = StringSplit($sItems, Chr(1))

The count will be in $avResults[0], the first element of data will be $avResults[1], the last element's data will be $avResults[$avResults[0]], and the speed difference from _ArrayAdd() will be amazing.

[Telamon]

No need to worry about the size, just define enough "buckets" and keep count on how many you used. you can then either redim the array at the end of you routine or use the counter to check them.

_ArrayAdd() just redim(s) the array everytime, which is an option too if speed isn't an issue.

Func _ArrayAdd(ByRef $avArray, $sValue)
    If IsArray($avArray) Then
        ReDim $avArray[UBound($avArray) + 1]
        $avArray[UBound($avArray) - 1] = $sValue
        SetError(0)
        Return 1
    Else
        SetError(1)
        Return 0
    EndIf
EndFunc;==>_ArrayAdd

That's not a bad solution and is what I ultimately settled on.

The pages I'm parsing have multiple links to the same page that I'm pruning out, which is really tough if you have mixed types in your results array (which you would have with a dummy element). I've been working with Lua and C# a lot recently and I'm totally spoiled as far as having flexible, easy-to-use collection generics.

[PsaltyDS]

That's not a bad solution and is what I ultimately settled on.

The pages I'm parsing have multiple links to the same page that I'm pruning out, which is really tough if you have mixed types in your results array (which you would have with a dummy element). I've been working with Lua and C# a lot recently and I'm totally spoiled as far as having flexible, easy-to-use collection generics.

Assemble the list in a string, as I mentioned earlier, and you can check it quickly with StringInStr() or a RegExp before adding it:

Global $sList = Chr(1), $avList[1]

; Some process that adds items...
For $n = 0 To $iSomeUnknown
    ; ...some unknown parse process...
    _AddItemToList($sSomeParsedString)
Next

; Split string into array
$avList = StringSplit(StringTrimRight(StringTrimLeft($sList, 1), 1), Chr(1))


Func _AddItemToList($sItem)
    If Not StringInStr($sList, Chr(1) & $sItem & Chr(1)) Then
        $sList &= $sItem & Chr(1)
    EndIf
EndFunc   ;==>_AddItemToList

If you need to refer to $avList during the parsing process, then the line that splits the string into an array just gets moved inside the _AddItemToList() function.

This will still be much faster that walking the array every time to look for duplicates.

[HyperVista]

However, AutoIT should supply an API to get the length of array without looking at array[0], which I customarily delete because it interferes with constructs like FOR $sItem In $aList.

[weaponx]

However, AutoIT should supply an API to get the length of array without looking at array[0], which I customarily delete because it interferes with constructs like FOR $sItem In $aList.

You mean Ubound() ?

[PsaltyDS]

You mean Ubound() ?

And I believe we found in another topic that in a tight loop repeated use of Ubound() was slower that the direct reference to [0].

[weaponx]

And I believe we found in another topic that in a tight loop repeated use of Ubound() was slower that the direct reference to [0].

Ubound: 2.58844034285285 seconds

$array[0]: 2.19632394832145 seconds

$timestamp1 = TimerInit()

Dim $array1[500000]

For $X = 0 to 500000-1
    $array1[$X] = $X
    Ubound($array1)
Next

$diff1 = TimerDiff($timestamp1) / 1000

$timestamp2 = TimerInit()

Dim $array2[500001]

For $X = 1 to 500000
    $array2[$X] = $X
    $array2[0]+=1
Next

$diff2 = TimerDiff($timestamp2) / 1000

MsgBox(0, "", "Ubound: " & $diff1 & " seconds" & @CRLF & "$array[0]: " & $diff2 & " seconds")

[PsaltyDS]

Ubound: 2.58844034285285 seconds

$array[0]: 2.19632394832145 seconds

I get similar. I'll have to back off on that until I can find that topic or remember what the circumstances were.

[Bert]

I'm extracting all links from a page and I'd like to iterate through them once, adding them to the list.

Otherwise I have to loop twice, the first time to find all the links so that I know how big the array needs to be.

_IELinkGetCollection will do what you need, and put them in a array for you. No need to beat yourself up about it. Fast and neat.

Global $Array[1], $links[1]
$oLinks = _IELinkGetCollection($oIE)
$iNumLinks = @extended
For $oLink In $oLinks
    _ArrayAdd($Array, _IEPropertyGet($oLink, "innertext"))
    _ArrayAdd($links, $oLink.href)
Next
_arraydisplay($array)
_arraydisplay($links)

[weaponx]

I get similar. I'll have to back off on that until I can find that topic or remember what the circumstances were.

Redim has much more damaging effects. In this example we will start with single element arrays and fill them with only 5,000 items (as opposed to 500,000 in my previous example).

;Redim
$timestamp1 = TimerInit()

Dim $array1[1]

For $X = 0 to 5000-1
    If $X > 0 Then Redim $array1[$X+1]
    $array1[$X] = $X
Next

$diff1 = TimerDiff($timestamp1) / 1000

;Stringsplit
$timestamp2 = TimerInit()

Dim $array2[1]

$string = ""

For $X = 0 to 5000-1
    $string &= $X & Chr(1)
Next

$array2 = StringSplit(StringTrimRight($string, 1), Chr(1))

$diff2 = TimerDiff($timestamp2) / 1000

ConsoleWrite( "Redim: " & $diff1 & " seconds" & @CRLF & "StringSplit: " & $diff2 & " seconds")

Redim: 14.4562593526397 seconds

StringSplit: 0.0321554767484117 seconds

[randallc]

OK!

Take it further to get the array without the [0] counter;

Twice as fast without using arraydel;

expandcollapse popup

; arsplit.au3
#include<array.au3>
#include<stringsplit.au3>
Local $iNum = 1000;00
;Redim
;~ $timestamp1 = TimerInit()

;~ Dim $array1[1]

;~ For $X = 0 To 5000 - 1
;~  If $X > 0 Then ReDim $array1[$X + 1]
;~  $array1[$X] = $X
;~ Next

;~ $diff1 = TimerDiff($timestamp1) / 1000

;Stringsplit
$timestamp2 = TimerInit()

Dim $array2[1]

$string = ""

For $X = 0 To $iNum - 1
    $string &= $X & Chr(1)
Next

$array2 = _StringSplit(StringTrimRight($string, 1), Chr(1))
;~ _ArrayDelete($array2,0)
$diff2 = TimerDiff($timestamp2)
;_Stringsplit
$timestamp3 = TimerInit()

Dim $array3[1]

$string = ""

For $X = 0 To $iNum - 1
    $string &= $X & Chr(1)
Next

$array3 = StringSplit(StringTrimRight($string, 1), Chr(1))
_ArrayDelete($array2, 0)

$diff3 = TimerDiff($timestamp3)

ConsoleWrite("_StringSplit: " & $diff2 & " msecs" & @CRLF & "StringSplit: " & $diff3 & " msecs" & @CRLF)

;~ ConsoleWrite("Redim: " & $diff1 & " seconds" & @CRLF & "StringSplit: " & $diff2 & " msecs"& @CRLF & "_StringSplit: " & $diff3 & " msecs"& @CRLF )oÝ÷ Ù«­¢+ØìMÑÉ¥¹MÁ±¥Ð¹ÔÌ(Õѽ%ÐÍ]ÉÁÁÉ}TÍ
¡­}AɵÑÉÌôµµÜĵÜȵÜ̵ÜеÜÔµÜØ(ìU9
Q%=8ìôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôô(ì(ì9µ¸¸¸¸¸¸¸¸¸¸¸è}MÑÉ¥¹MÁ±¥Ð(ìÍÉ¥ÁÑ¥½¸¸¸¸è¨¨MµÌ¥¹Õ¥±ÐÅÕ½ÐíMÑÉ¥¹MÁ±¥ÐÅÕ½ÐìÕй¼ÅÕ½ÐíÕ½Õ¹ÅÕ½ÐìÌÉÉå±Ñͱ½ÜìMÁ±¥ÑÌÕÀÍÑÉ¥¹¥¹Ñ¼ÍÕÍÑÉ¥¹ÌÁ¹¥¹½¸Ñ¡¥Ù¸±¥µ¥ÑÉÌ(ìMå¹Ñุ¸¸¸¸¸¸¸è}MÑÉ¥¹MÁ±¥Ð ÅÕ½ÐíÍÑÉ¥¹ÅÕ½Ðì°ÅÕ½Ðí±¥µ¥ÑÉÌÅÕ½Ðìl°±t¤(ìAɵÑÉ̸¸¸¸èÍÑÉ¥¹Q¡ÍÑÉ¥¹Ñ¼Ù±ÕѸ(ì±¥µ¥ÑÉÌ=¹½Èµ½É¡ÉÑÉÌѼÕÍ̱¥µ¥ÑÉ̸(ì±m½ÁÑ¥½¹±t%±¥ÌÀ¡Ñ¡Õ±Ð¤°Ñ¡¸ ¡ÉÑÈ¥¸Ñ¡±¥µ¥ÑÈÍÑÉ¥¹Ý¥±°µÉ¬Ý¡ÉѼÍÁ±¥ÐÑ¡ÍÑÉ¥¹(ìIÑÕɸٱÕ̸èMÕÍÌ´IÑÕɹ̸ÉÉä°Ñ¡¥ÉÍбµ¹Ð ÀÌØíÉÉålÁt¤½¹Ñ¥¹ÌÑ¡¹ÕµÈ½ÍÑÉ¥¹ÌÉÑÕɹ°(ì$$$$$$$µÑ¡Éµ¥¹¥¹±µ¹ÑÌ ÀÌØíÉÉålÅt°ÀÌØíÉÉålÉt°Ñ¸¤½¹Ñ¥¸Ñ¡±¥µ¥ÑÍÑÉ¥¹Ì¸(쥱ÕÉ´(ìðÀ´9¼Éɽȸ(ìðÄ´%¹Ù±¥´%¹¼±¥µ¥ÑÉÌÝɽչÉɽȥÌÍÐѼݹѡձ°ÍÑÉ¥¹¥ÌÉÑÕɹ ÀÌØíÉÉålÁt¤¸(ìÕÑ¡½È¸¸¸¸¸¸¸¸èI¹±±(ì5½¥¥¸¸¸¸¸¸¸è(ìIµÉ­Ì¸¸¸¸¸¸¸è%å½ÔÕͱ¹¬ÍÑÉ¥¹ÅÕ½ÐìÅÕ½Ðì½ÈÑ¡±¥µ¥ÑÉÌ° ¡ÉÑÈÝ¥±°ÉÑÕɹ̸±µ¹Ð¸(ìIµÉ­Ì¸¸¸¸¸¸¸è%Ñ¡±¥µ¥ÑÈå½ÔÝ¥Í Ñ¼ÕÍ¥ÌÍÕÍÑÉ¥¹¥¹Íѽ¥¹¥Ù¥Õ°Í¥¹±¡ÉÑÉÌ°ÍÑ¡áµÁ±±½Ü¸(ìIµÉ­Ì¸¸¸¸¸¸¸èMÑÉ¥¹MÁ±¥Ð¥ÌÙÉäÕÍհ̸±ÑɹѥÙѼMÑÉ¥¹%¹MÑȹ̵¹ÌѼÁ½ÁձѸÉÉä(ìIµÉ­Ì¸¸¸¸¸¸¸èݥѡ½ÕÐÅÕ½ÐíÕ½Õ¹ÅÕ½Ð쥹½Ð¹ìÌÅÕ½Ðí}ÉÉå±Ñ ÀÌØíÉÉÉä°À¤Í¼Í±½Ü(ìIµÉ­Ì¸¸¸¸¸¸¸è
ÕÑ¥½¸¥å½ÔÕÍÑ¡µÉ¼
I1å½ÔÉÉÉÉ¥¹Ñ¼È¡ÉÑÈÍÑÉ¥¹Í¼å½ÔÝ¥±°¹ÉÑáÑɱ¹­Ì±¥¹Ì¸(ìIµÉ­Ì¸¸¸¸¸¸¸è
ÕÑ¥½¸¥å½ÔÕÍÑ¡µÉ¼
I1å½ÔÉÉÉÉ¥¹Ñ¼È¡ÉÑÈÍÑÉ¥¹Í¼å½ÔÝ¥±°¹ÉÑáÑɱ¹­Ì±¥¹Ì¸(ìI±Ñ¸¸¸¸¸¸¸èMÑÉ¥¹MÁ±¥Ð±MÑÉ¥¹°MÑÉ¥¹%¹MÑÈ°MÑÉ¥¹1аMÑÉ¥¹1¸°MÑÉ¥¹1½ÝÈ°MÑÉ¥¹5¥°MÑÉ¥¹IÁ±°MÑÉ¥¹I¥¡Ð°MÑÉ¥¹QÉ¥µ1аMÑÉ¥¹QÉ¥µI¥¡Ð°MÑÉ¥¹UÁÁÈ(ì1¥¹¬¸¸¸¸¸¸¸¸¸¸ì(ìáµÁ±¸¸¸¸¸¸¸ìeÌ(ì(ììôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôôô()Õ¹}MÑÉ¥¹MÁ±¥Ð ÀÌØíÍMÑÈ°ÀÌØíÍMÁ°ÀÌØí¥±ôÀ¤(%1½°ÀÌØí¥A½ÌôMÑÉ¥¹%¹MÑÈ ÀÌØíÍMÑÈ°ÀÌØíÍMÁ¤°ÀÌØíÍQµÀôMÑÉ¥¹1Ð ÀÌØíÍMÑÈ°ÀÌØí¥A½Ì´Ä¤($ÀÌØíÍMÑÈôMÑÉ¥¹5¥ ÀÌØíÍMÑÈ°ÀÌØí¥A½Ì¬MÑÉ¥¹1¸ ÀÌØíÍMÁ¤¤(%1½°ÀÌØíÉ¥±ôMÑÉ¥¹MÁ±¥Ð ÀÌØíÍMÑÈ°ÀÌØíÍMÁ°ÀÌØí¥±¤(%¥ÉɽÈÑ¡¸($$ÀÌØíÉ¥±lÁtôÀÌØíÍMÑÈ($%É¥´ÀÌØíÉ¥±lÅt($%ÉÑÕɸMÑÉɽȡÉɽȰÀ°ÀÌØíÉ¥±¤(%¹%($ÀÌØíÉ¥±lÁtôÀÌØíÍQµÀ(%IÑÕɸMÑÉɽȡÉɽȰÀ°ÀÌØíÉ¥±¤ìÀÌØíÉ¥±)¹Õ¹ìôôÐí}MÑÉ¥¹MÁ±¥Ð)Õ¹}±µ¹Ð ÀÌØíÍ5åÉÉä°ÀÌØí¥¹Õ´ôÀ°ÀÌØíÍMÁôÅÕ½ÐíðÅÕ½Ðì°ÀÌØí¥±ôĤ(%1½°ÀÌØí¥A½ÌÈôMÑÉ¥¹%¹MÑÈ ÀÌØíÍ5åÉÉä°ÀÌØíÍMÁ°À°ÀÌØí¥¹Õ´¬Ä¤ì°ÀÌØíÍQµÀôMÑÉ¥¹1Ð ÀÌØíÍ5åÉÉä°ÀÌØí¥A½ÌȴĤ(%¥ÀÌØí¥¹Õ´ôÀÑ¡¸ÉÑÕɸMÑÉ¥¹1Ð ÀÌØíÍ5åÉÉä°ÀÌØí¥A½ÌȴĤ(%1½°ÀÌØí¥A½ÌÄôMÑÉ¥¹%¹MÑÈ ÀÌØíÍ5åÉÉä°ÀÌØíÍMÁ°À°ÀÌØí¥¹Õ´¤(%¥ÀÌØí¥A½ÌÄôÀÑ¡¸ÉÑÕɸ(%¥ÀÌØí¥A½ÌÈôÀÑ¡¸ÉÑÕɸMÑÉ¥¹5¥ ÀÌØíÍ5åÉÉä°ÀÌØí¥A½ÌĬMÑÉ¥¹1¸ ÀÌØíÍMÁ¤¤(%ÉÑÕɸMÑÉ¥¹5¥ ÀÌØíÍ5åÉÉä°ÀÌØí¥A½ÌĬMÑÉ¥¹1¸ ÀÌØíÍMÁ¤°ÀÌØí¥A½ÌÈ´ÀÌØí¥A½ÌĴĤ)¹Õ¹