[Solved]_ArrayConcatenate

[goldenix]

Hi

I made an array $main_array[1][2], then I use _FileReadToArray, next I try to _ArrayConcatenate both arrays

It does not work. I assume it only works with 1 dimensional arrays ?

I tried to do this manually by creating 1 empty array, then use _FileReadToArray to make second array, Then I loop thru second array & add stuff into my first array column 1 & 2.

But this literally takes forever to add even 7k entries. the time goes into minutes.

Is there any other way to join arrays in autoit or should I look in some other language?

Any tips are appreciated.

Edited June 27, 2010 by goldenix

[Tvern]

But this literally takes forever to add even 7k entries. the time goes into minutes.

I've got a script thatjoins arrays with 1000+collumns and 10 rows in no time at all*

Can We see (a part of) your code to see what is slowing it down? Are you using a redim in a loop?

*Length of no time at all might in fact be a meassurable amount of time.

Edit:

The functional part of this example runs in about 60-70 ms for me, It adds 34k values to an array and redims 4 times.

expandcollapse popup

#include <array.au3> ;just for the arraydisplays

#region create and fill arrays for testing
    Local $2DArray[200][1]
    Local $AddArray1[700]
    Local $AddArray2[800]
    Local $AddArray3[900]
    Local $AddArray4[1000]

    For $i = 0 To UBound($AddArray1) -1
        $AddArray1[$i] = "Add1 " & $i
    Next
    For $i = 0 To UBound($AddArray2) -1
        $AddArray2[$i] = "Add2 " & $i
    Next
    For $i = 0 To UBound($AddArray3) -1
        $AddArray3[$i] = "Add3 " & $i
    Next
    For $i = 0 To UBound($AddArray4) -1
        $AddArray4[$i] = "Add4 " & $i
    Next
    For $i = 0 To UBound($2DArray) -1
        $2DArray[$i][0] = "Original" & $i
    Next
#endregion

#region show array contents
    _ArrayDisplay($AddArray1,"$AddArray1")
    _ArrayDisplay($AddArray2,"$AddArray2")
    _ArrayDisplay($AddArray3,"$AddArray3")
    _ArrayDisplay($AddArray4,"$AddArray4")
    _ArrayDisplay($2DArray,"$2DArray")
#endregion

Local $Timer = TimerInit()
_ArrayJoin($2DArray, $AddArray1)
_ArrayJoin($2DArray, $AddArray2)
_ArrayJoin($2DArray, $AddArray3)
_ArrayJoin($2DArray, $AddArray4)
ConsoleWrite("Joining took: " & Round(TimerDiff($Timer),0) & "ms." & @CRLF)

_ArrayDisplay($2DArray,"Result")

Func _ArrayJoin(ByRef $aTarget, $aSource)
    If UBound($aSource) > UBound($aTarget) Then
        $nRows = UBound($aSource)
    Else
        $nRows = UBound($aTarget)
    EndIf
    $nCollumns = UBound($aTarget,2)
    ReDim $aTarget[$nRows][$nCollumns+1]
    For $i0 = 0 To UBound($aSource) -1
        $aTarget[$i0][$nCollumns] = $aSource[$i0]
    Next
EndFunc

Edited June 27, 2010 by Tvern

[goldenix]

I've got a script thatjoins arrays with 1000+collumns and 10 rows in no time at all*

Can We see (a part of) your code to see what is slowing it down? Are you using a redim in a loop?

Yes Im using Redim in a loop.

Hire it the loop:

PS: Im adding 100 - 200 chars long lines into the array & it will eventually be 500k entries or even more

Edit, umm ur script is adding colums, good idea, but let me check how it will work

If Not _FileReadToArray($FullFilePath, $temp_fileread_to_array) Then
       MsgBox(4096,"Error", " Error reading file to Array     error:" & @error & @CRLF & $FullFilePath)
       Exit
    EndIf

    ConsoleWrite('Joining arrays ' & $filename & @CRLF)

    For $x = 1 to $temp_fileread_to_array[0] ; we want to add to existing array so we must start from next row

        $main_array_last_entry_pos = UBound($main_array) ; last entry of the mail array index

        $iRow = UBound($main_array) ;add Rows
        $iCol = UBound($main_array, 2)  ;Add Columns

        ReDim $main_array[$iRow + 1][$iCol] ; add more rows +1 ; ReDim $main_array[$iRow + 1][$iCol + 1] - Adds both

        $main_array[$main_array_last_entry_pos][0] = $filename
        $main_array[$main_array_last_entry_pos][1] = $temp_fileread_to_array[$x]

    Next

Edited June 27, 2010 by goldenix

[sahsanu]

I made an array $main_array[1][2], then I use _FileReadToArray, next I try to _ArrayConcatenate both arrays

It does not work. I assume it only works with 1 dimensional arrays ?

Yes, _ArrayConcatenate only works with 1D but take a look here. PsaltyDS made a function to concatenate 1D and 2D arrays.

I hope this helps.

Cheers,

sahsanu

[Tvern]

I modified your script to keep the loop light. If I didn't make any cockups it should be much faster now, but I didn't test it.

#include <file.au3>
If Not _FileReadToArray($FullFilePath, $temp_fileread_to_array) Then
    MsgBox(4096,"Error", " Error reading file to Array  error:" & @error & @CRLF & $FullFilePath)
    Exit
EndIf

ConsoleWrite('Joining arrays ' & $filename & @CRLF)

$iRow = UBound($main_array)
$iCol = UBound($main_array, 2) ;seems to always be 2?
;$iRow+$temp_fileread_to_array[0]-1 should be the final size right?
ReDim $main_array[$iRow+$temp_fileread_to_array[0]-1][$iCol] ;only 1 redim means speed!

;Now we have a nice and light loop
For $x = 1 to $temp_fileread_to_array[0]
    $main_array[$iRow+$x-1][0] = $filename ;$iRow+$x-1 should always be the first empty cell
    $main_array[$iRow+$x-1][1] = $temp_fileread_to_array[$x]
Next

[goldenix]

I modified your script to keep the loop light. If I didn't make any cockups it should be much faster now, but I didn't test it.

Nop this sample didnt work some error with range exceeded & I did not look into it too deep cuz I understood the principle [Redim once = speed like you said] & mod it myself.

Result 142 623 entries in less than 1 second. (4,5MB Data)

But for some reason it Freezed when I asked it to show me the array once it finished adding values. Maybe ther eis some limit or a bug that refuses to display you the array if it gets too big.

k thanx, next ill try to search the array.

Edited June 27, 2010 by goldenix