Array generation

[SmartiePants]

I have an idea, but do not know how to put it to work. I wanted to write a function that would be used by the script to store data in arrays. The conundrum is that I want to create a pretty much unlimited number of arrays with a limit at say... 1000 entries. It seemed simpler in my mind until I threw together this preliminary code:

Func WriteArray($FileName)
        If IsDeclared( String(String($array) & Number($aNum))) and String(String($array) & Number($aNum))[0] < 1000 then; if array is declared and is less than 1000 elements
            String(String($array) & Number($aNum))[$count] = $FileName
            $count += 1
            String(String($array) & Number($aNum))[0] = UBound(String(String($array) & Number($aNum))[0]) - 1
        ElseIf not IsDeclared( String(String($array) & Number($aNum))) then; if array is not declared
            Dim String(String($array) & Number($aNum))[2]
            String(String($array) & Number($aNum))[$count] = $FileName
            $count += 1
            String(String($array) & Number($aNum))[0] = UBound(String(String($array) & Number($aNum))[0]) - 1
        ElseIf IsDeclared( String(String($array) & Number($aNum))) and String(String($array) & Number($aNum))[0] >= 1000 then; if array is declared and has greater than or equal to 1000 elements
            $aNum += 1; increase array variable title. Ex: $array1
            $count = 1
            Dim String(String($array) & Number($aNum))[2]
            String(String($array) & Number($aNum))[$count] = $FileName
            $count += 1
            String(String($array) & Number($aNum))[0] = UBound(String(String($array) & Number($aNum))[0]) - 1           
        EndIf
    ToolTip($FileName,0,0)
EndFunc

I know the code has many errors and warnings. But the idea is valid. I just do not know how to write a code that would perform what I described above. I have asked a question before on the similar subject, but I was vague and didn't have code. I only came to the functions Eval, IsDeclared and Assign. Hope someone could help me figure this out.. Thanks in advance

[martin]

I have an idea, but do not know how to put it to work. I wanted to write a function that would be used by the script to store data in arrays. The conundrum is that I want to create a pretty much unlimited number of arrays with a limit at say... 1000 entries. It seemed simpler in my mind until I threw together this preliminary code:

Func WriteArray($FileName)
        If IsDeclared( String(String($array) & Number($aNum))) and String(String($array) & Number($aNum))[0] < 1000 then; if array is declared and is less than 1000 elements
            String(String($array) & Number($aNum))[$count] = $FileName
            $count += 1
            String(String($array) & Number($aNum))[0] = UBound(String(String($array) & Number($aNum))[0]) - 1
        ElseIf not IsDeclared( String(String($array) & Number($aNum))) then; if array is not declared
            Dim String(String($array) & Number($aNum))[2]
            String(String($array) & Number($aNum))[$count] = $FileName
            $count += 1
            String(String($array) & Number($aNum))[0] = UBound(String(String($array) & Number($aNum))[0]) - 1
        ElseIf IsDeclared( String(String($array) & Number($aNum))) and String(String($array) & Number($aNum))[0] >= 1000 then; if array is declared and has greater than or equal to 1000 elements
            $aNum += 1; increase array variable title. Ex: $array1
            $count = 1
            Dim String(String($array) & Number($aNum))[2]
            String(String($array) & Number($aNum))[$count] = $FileName
            $count += 1
            String(String($array) & Number($aNum))[0] = UBound(String(String($array) & Number($aNum))[0]) - 1           
        EndIf
    ToolTip($FileName,0,0)
EndFunc

I know the code has many errors and warnings. But the idea is valid. I just do not know how to write a code that would perform what I described above. I have asked a question before on the similar subject, but I was vague and didn't have code. I only came to the functions Eval, IsDeclared and Assign. Hope someone could help me figure this out.. Thanks in advance

That's a bit of a mess

I assume that $array is the string name of the array variable so you can use it in IsDeclared. I don't think that will work because you would need to use Assign which can't be used with arrays

Maybe explain in simple steps what you want. Do you want up to 1000 pieces of information stored or are there going to be many sets of data each with up to 1000 pieces of data?

[Paulie]

I would use a multi-dim array.

Edited June 29, 2008 by Paulie

[martin]

Well, it's "possible" to use a multi-dim array, but it has a limitation of 64 dimensions or w/e So if my script would have 65 sets of say... 1000, then the idea fails.

That's not correct, I think you have misunderstood.

From the help

Arrays: A maximum of 64 dimensions and/or a total of 16 million elements

A 2-dimensional array is all you need. If you had

Dim $array1[4000][4000]

you would have the maximum of 16 million elements, and 4000 sets of 4000 pieces of data.

[martin]

Then maybe I am just totally confused with arrays I am starting to see what is meant by the dimensions, but it's still a bit confusing. then again, from the dimensions, is there an efficiency loss after a while? I will try to get my code to work with a multi-dim array...

Ummmm, so every time I want to update the 0,0 element to contain the total # of elements in the array, I "must" use UBound? I guess am confused about keeping track of how much data is stored within the array.

Maybe you could correct me if I get this wrong:

Dim $array[5][5] - 25 elements

$array[0][0] - this is the total elements in the array

$array[0][1] - element

$array[0][2] - element

$array[0][3] - element

$array[0][4] - element

$array[1][0] - element

$array[1][1] - element

$array[1][2] - element

$array[1][3] - element

$array[1][4] - element

$array[2][0] - element

$array[2][1] - element

$array[2][2] - element

$array[2][3] - element

$array[2][4] - element

$array[3][0] - element

$array[3][1] - element

$array[3][2] - element

$array[3][3] - element

$array[3][4] - element

$array[4][0] - element

$array[4][1] - element

$array[4][2] - element

$array[4][3] - element

$array[4][4] - element

Correct except that $array[0][0] is only the number of elements if you store that information there. You camn put whatever you like in [0][0], though many functions do use it for the UBound value.

[PsaltyDS]

Would it be safe to keep a track of the elements manually? I mean without the use of UBound. Say I had 25 possible elements and 0,0 element would contain the total # of elements (24 in this case), is it normal to make the array larger and keep track of all the elements without UBound, since I heard it takes more time?

Yes, you can create the array large enough to hold the expected data and keep track of how many elements are actually valid in any variable you want, including one of the elements in the array itself, like [0][0].

If you need to resize the array without losing data, you can use ReDim. Write some short demo scripts for yourself to try it out.

[PsaltyDS]

Here is something more juicy than the original code This is the whole script. And the reason I am posting it here is I cannot figure out what I am doing wrong with the 2-dimensional array I thought it looks about right, but I can't get rid of the errors no matter what I try. The problem is in assigning data to array elements and ReDimming the array. Please look and see if you see what I am doing wrong. Maybe you could try to run the code and figure out the problem Also, with some of my tries to fix this, I got the script to run, but with the counter either at 1 or 4001 constantly. PLEASE HELP

; ...

Func WriteArray($FileName)
    If not $aCount = $limit Then
        
    ; ...
        
    Else

    ; ...

        $bCount += 1
        $aCount = 1
        $array[$aCount][$bCount] = $FileName
        $aCount += 1
        ReDim $array[$aCount][$bCount]
    EndIf
EndFunc ;==>WriteArray

So... when you reach [4000][1], you reset $aCount to 1 and ReDim to [1][2]... which deletes all the entries from [2][1] thru [4000][1].

What was the point of this?

[nikink]

Looks to me (and I'm no expert) like you've got your dimensions the wrong way around.

I think, if I've understood you correctly, you want to get to [1][4000] then redim to [2][4000], then [3][4000]. Right? So in that case $aCount should be incremented before the redim and you can leave $bCount at 4000, then after the redim, set it back to one to begin counting again.

[PsaltyDS]

What is the point of this whole exercise? Why are you using this array structure? How is it indexed for referencing individual items after it is filled? It is so unlikely that this array will be useful once it's done.

If you really have millions of items to track, use SQLite with a memory resident database.

If it's just a list of every file on a hard disk, why not just a text file, which can be parsed later with just FileReadLine()?

DIR C:\*.* /s/b > C_FileList.txt

A [4000][4000] array is so NOT the answer here...

[MagnumXL]

I agree with PsaltyDS. What are we actually trying to do here?

If you want an example of how a large array looks on the inside, try this...

#include <array.au3>
Dim $k = 0
Dim $a_array[4000][4000]
For $i = 0 to 3999
    For $j = 0 to 3999
        $k += 1
        $a_array[$i][$j] = $k
    Next
Next
_ArrayDisplay($a_array)

Warning, that takes awhile to run.

I use arrays with MSAccess all the time. I think a database is what your probably needing here.

Edited July 1, 2008 by MagnumXL

[SmOke_N]

So, I figured out my problem after all. The script works with a 2D array and runs in half time of what another script did. This is grand and all, but I still want to figure out the original problem, because a single array isn't going to cut it. I want to limit arrays to about 60,000 elements. That seems to be the noticeable slow down point. So I would want to use $array[15][4000], then $array1[15][4000], then $array2[15][4000], then $array3[15][4000]. I do not want to declare huge arrays from get-go. I don't want to say declare $array[15][4000] from the start, but just $array[1][4000] as it is now. But then I somehow want to Dim more arrays if I need to with a variable number in the array name like $array1, $array2, $array3... Isn't there a way to perform this in AutoIt?

Well, you want to work with arrays, but do an eval type of an array... could just make them 3 dimensional, and the first dimension is to handle "Count"

$array[1][15][4000], $array[2][15][4000] etc...

To "Dim More" look at "ReDim" you can increase/decrease the size of your arrays this way.

[YellowLab]

There are different purposes for declaration and assign. First, declaration reserves the variable name and if MustDeclareVariables is true, then a check is made that can prevent errors due to misspellings of variable names. The one great use for Assign, that I have found, is it allows you to pass a variable name as a string. Thus if you have a family of variables, $MyVar1, $MyVar2, $MyVar3, you can pass the name "MyVar"&$iCount as the argument to assign. It has limited uses, but uses none-the-less.

I think, with your problem, you are running into a limitation on how arrays are stored and accessed. Once they reach a certain size, they become inefficient. I think, realistically, in dealing with such large array sizes, I will reiterate what PsaltyDS has stated: use an SQLite memory resident database. This type of structure is much more suited for large sets of data. You should find that the access times are much improved.

Bob

[SmOke_N]

I would say that more than likely, the bad times you are receiving isn't because of the arrays, but what you are doing with the files you are manipulating and or how you coded them. If all you did was enum through those files with the array, and did nothing else, you may find that they are much faster than you think. If you're attempting any kind of string functions, this would definately slow them down.

As others have said here, the database option might be a good idea if you need to manipulate them quickly.

[SmartiePants]

I am sorry, I did not have a clear understanding of arrays at the time... Thanks to Paulie, I believe I can understand how arrays work muttley

Edited July 16, 2008 by SmartiePants

[PsaltyDS]

I have yet to use SQlite, so I am trying to get my 3D array write and read to work properly.. I have a problem and I can't figure it out I "supposedly" store 5499 values in the array with total elements $array[5][5][42] and then want to read them back, but script fails @ $array[4][9][99]. So, $array[5][0][0] through $array[5][5][42] is good, and rest I can't figure out what is wrong with I tried to read $array[4][9][99], $array[4][8][98], $array[4][9][50], and they all seem to fail. Please help me... (Original limits were $aLimit = 1600, $bLimit = 10, $cLimit = 1000, but I changed them to test and fix some problems with my script)

This still makes no sense whatsoever. Why is this a 3D array? Why not 1D, or 2D? Is there any logical relationship between the dimensions at all? What does your script DO? And no, I will not study it for hours to figure that out for myself.

If you can't clearly explain what you want to accomplish in this array, there is no hope of achieving it. To paraphrase Anna Leonowens to the King of Siam: "If you can't code what you mean, you will never mean what you code."

muttley