ReDim error code

[argumentum]

Global $aArray[100000][10]
ReDim  $aArray[100000][10000000000000000]
ConsoleWrite(UBound($aArray) & ',' & UBound($aArray, 2) & @CRLF)

This will crash with "Error allocating memory."
Would there be a way to return an error when the script tries to ReDim more than it can chew ?
Or should be making a calculation of ( X_Columns + Y_Rows > Z_Number ) = oops ? ( and do share that, if you know the formula )

I know that the same error would occur if just declaring the array to start with, so maybe a Func DoArraySafely($Cols,$Rows) ???, or a Trac entry to make it safer ?. The limitations are in the help file, so I can not say that "OMG!, why ?", but that's why I'm posting. Thanks for looking at this.

[pixelsearch]

Hi argumentum
The help file indicates this, in its topic "AutoIt3 Limits/defaults"

VAR_SUBSCRIPT_ELEMENTS
Value : 16,777,216
Description : Maximum number of elements for an array. 

So I just tested this value with the following examples, hope it will help :

; 1D
; Local $aArray[16777216] ; ok
; Local $aArray[16777216 + 1] ; crash

; 2D's
; Local $aArray[16777216][1] ; ok
; Local $aArray[16777216 + 1][1] ; crash

; Local $aArray[16777216 / 2][2] ; ok
; Local $aArray[(16777216 / 2) + 1][2] ; crash

; Local $aArray[16777216 / 4][4] ; ok
; Local $aArray[(16777216 / 4) + 1][4] ; crash

; ...

; Local $aArray[4][16777216 / 4] ; ok
; Local $aArray[4][(16777216 / 4) +1] ; crash

; Local $aArray[2][16777216 / 2] ; ok
; Local $aArray[2][(16777216 / 2) +1] ; crash

; Local $aArray[1][16777216] ; ok
; Local $aArray[1][16777216 + 1] ; crash

 

[argumentum]

so (X * Y  > 16777216) = oops is the formula ?

[argumentum]

Func __ArrayFromString($sArrayStr, $sDelim_Col = "|", $sDelim_Row = @CRLF, $bOpt = Default, $iStripWS = $STR_STRIPLEADING + $STR_STRIPTRAILING)
    If $sDelim_Col = Default Then $sDelim_Col = "|"
    If $sDelim_Row = Default Then $sDelim_Row = @CRLF
    If $bOpt = Default Then $bOpt = 0 ; $bForce2D = 1, redim array = 2
    If $iStripWS = Default Then $iStripWS = $STR_STRIPLEADING + $STR_STRIPTRAILING
    Local $aRow, $aCol = StringSplit($sArrayStr, $sDelim_Row, $STR_ENTIRESPLIT + $STR_NOCOUNT)
;~  ConsoleWriteArray($aCol, @ScriptLineNumber)
    $aRow = StringSplit($aCol[0], $sDelim_Col, $STR_ENTIRESPLIT + $STR_NOCOUNT)
    If UBound($aCol) = 1 And Not BitAND(1, Int($bOpt)) Then
        For $m = 0 To UBound($aRow) - 1
            $aRow[$m] = ($iStripWS ? StringStripWS($aRow[$m], $iStripWS) : $aRow[$m])
        Next
        Return $aRow
    EndIf
;~  ConsoleWriteArray($aRow, @ScriptLineNumber)

    Local $aRet[UBound($aCol)][UBound($aRow)]
    For $n = 0 To UBound($aCol) - 1
        $aRow = StringSplit($aCol[$n], $sDelim_Col, $STR_ENTIRESPLIT + $STR_NOCOUNT)
        If UBound($aRow) > UBound($aRet, 2) Then
            If Not BitAND(2, $bOpt) Then Return SetError(1)
            ReDim $aRet[UBound($aRet)][UBound($aRow)] ; <<<<< here's where I'd have to do something in case that expanding the array it would not crash the script
            ;                                           and that is the change/feature I wanted to add. That if you do that mods. I'll be super thankful =)
        EndIf
        For $m = 0 To UBound($aRow) - 1
            $aRet[$n][$m] = ($iStripWS ? StringStripWS($aRow[$m], $iStripWS) : $aRow[$m])
        Next
    Next
    Return $aRet
EndFunc   ;==>__ArrayFromString

@pixelsearch, if you were to modify it to make it safer to use and/or any other ideas, I'd be super thankful
The ReDim question was about the new option to expand the array instead of returning an error. But it should be implemented from the start, as to avoid loading a file that was too big to load anyways. Would you go at it ? Thanks. ( otherwise it'd fall on my and I'm a bit out, mind wise )
Tho you can say, "no thanks" too, I'm not trying to impose.

[pixelsearch]

It seems correct, X * Y calculates the total number of elements, then compares it to AutoIt limit value

Edit: we posted at same time. My sentence "It seems correct..." was an answer to your penultimate post.

Edited February 10, 2023 by pixelsearch

[pixelsearch]

@argumentum first of all I'd modify 2 variables names so the script becomes much easier to read.
These 2 variables are $aRow and $aCol, their names should really be swapped because $aRow should refer to rows and $aCol to columns... unless you like headaches 

This would lead to this basic script, where some original lines are commented out, so everybody can see easily what has been modified :

expandcollapse popup

Func _ArrayFromString($sArrayStr, $sDelim_Col = "|", $sDelim_Row = @CRLF, $bOpt = Default, $iStripWS = $STR_STRIPLEADING + $STR_STRIPTRAILING)
    If $sDelim_Col = Default Then $sDelim_Col = "|"
    If $sDelim_Row = Default Then $sDelim_Row = @CRLF
    If $bOpt = Default Then $bOpt = 0 ; Force 2D = 1, redim array = 2
    If $iStripWS = Default Then $iStripWS = $STR_STRIPLEADING + $STR_STRIPTRAILING

    ; Local $aRow, $aCol = StringSplit($sArrayStr, $sDelim_Row, $STR_ENTIRESPLIT + $STR_NOCOUNT)
    Local $aCol, $aRow = StringSplit($sArrayStr, $sDelim_Row, $STR_ENTIRESPLIT + $STR_NOCOUNT)

    ; $aRow = StringSplit($aCol[0], $sDelim_Col, $STR_ENTIRESPLIT + $STR_NOCOUNT)
    $aCol = StringSplit($aRow[0], $sDelim_Col, $STR_ENTIRESPLIT + $STR_NOCOUNT)

    ; If UBound($aCol) = 1 And Not BitAND(1, Int($bOpt)) Then
    If UBound($aRow) = 1 And Not BitAND(1, Int($bOpt)) Then

        ; For $m = 0 To UBound($aRow) - 1
        For $m = 0 To UBound($aCol) - 1

            ; $aRow[$m] = ($iStripWS ? StringStripWS($aRow[$m], $iStripWS) : $aRow[$m])
            $aCol[$m] = ($iStripWS ? StringStripWS($aCol[$m], $iStripWS) : $aCol[$m])

        Next

        ; Return $aRow
        Return $aCol

    EndIf

    ; Local $aRet[UBound($aCol)][UBound($aRow)]
    Local $aRet[UBound($aRow)][UBound($aCol)]

    ; For $n = 0 To UBound($aCol) - 1
    For $n = 0 To UBound($aRow) - 1

        ; $aRow = StringSplit($aCol[$n], $sDelim_Col, $STR_ENTIRESPLIT + $STR_NOCOUNT)
        $aCol = StringSplit($aRow[$n], $sDelim_Col, $STR_ENTIRESPLIT + $STR_NOCOUNT)

        ; If UBound($aRow) > UBound($aRet, 2) Then
        If UBound($aCol) > UBound($aRet, 2) Then

            If Not BitAND(2, $bOpt) Then Return SetError(1)

            ; ReDim $aRet[UBound($aRet)][UBound($aRow)]
            ReDim $aRet[UBound($aRet)][UBound($aCol)]

        EndIf

        ; For $m = 0 To UBound($aRow) - 1
        For $m = 0 To UBound($aCol) - 1

            ; $aRet[$n][$m] = ($iStripWS ? StringStripWS($aRow[$m], $iStripWS) : $aRow[$m])
            $aRet[$n][$m] = ($iStripWS ? StringStripWS($aCol[$m], $iStripWS) : $aCol[$m])

        Next
    Next
    Return $aRet
EndFunc   ;==>_ArrayFromString

I just checked that you're working on this function for a while, in these links :
https://www.autoitscript.com/forum/topic/197277-_arrayfromstring/
https://www.autoitscript.com/trac/autoit/ticket/3696

On June 12 2021, Jon added your function _ArrayFromString in AutoIt v3.3.15.4 Beta and it's here now in actual release 3.3.16.1 . Maybe most users still don't know this function exists, which could explain why it's not more used, because it's an interesting function. For the record, I discovered your function when @OJBakker mentioned it (and used it) in this post.

Apparently, _FileReadToArray() is an alternative to your function (though _FileReadToArray requires a file to be read, when your function requires a string, no big deal) . One major difference between _FileReadToArray and _ArrayFromString is that _FileReadToArray generates an @error 3 ("3 - File lines have different numbers of fields, only if $FRTA_INTARRAYS flag not set") but you want _ArrayFromString to be able to ReDim "on the fly" while the function is executing, in case a row being processed got more column delimiters than the number of column delimiters found initially in Row 0

21 hours ago, argumentum said:

The ReDim question was about the new option to expand the array instead of returning an error. But it should be implemented from the start, as to avoid loading a file that was too big to load anyways.

An idea would be to check the number of elements immediately after these 2 lines :

Local $aCol, $aRow = StringSplit($sArrayStr, $sDelim_Row, $STR_ENTIRESPLIT + $STR_NOCOUNT)
$aCol = StringSplit($aRow[0], $sDelim_Col, $STR_ENTIRESPLIT + $STR_NOCOUNT)

Local $iNbElements = Ubound($aRow) * Ubound($aCol) ; check if not > 16.777.216

Of course this will not work in case $aRow (mostly) got more than 16.777.216 elements (rows) because StringSplit() would already have generated a fatal error.

Let's go on. Later in the script, just before you use the eventual ReDim function, just check again what will become the expanded number of elements (compared to 16.777.216) to decide if an error will be generated or not.

I'm not forgetting what I asked you a couple of months ago : why this kind of string outputs as a 2D array ? Shouldn't it output as a 1D array ?

Local $aArray = _ArrayFromString("1" & @CRLF & "2")
ConsoleWrite(Ubound($aArray, $UBOUND_DIMENSIONS) & @crlf) ; 2

For the record, there were 7 persons working on _FileReadToArray, maybe we could hire a couple of them to make yours 100% safe too 

; _FileReadToArray
; Author ........: Jonathan Bennett <jon at autoitscript dot com>, Valik - Support Windows Unix and Mac line separator
; Modified ......: Jpm - fixed empty line at the end, Gary Fixed file contains only 1 line, guinness - Optional flag to return the array count.
;                : Melba23 - Read to 1D/2D arrays, guinness & jchd - Removed looping through 1D array with $FRTA_COUNT flag.

 

[argumentum]

7 hours ago, pixelsearch said:

For the record, there were 7 persons working on _FileReadToArray, maybe we could hire a couple of them to make yours 100% safe too

hmm, most of 'em are gone. The only great difference is that in this function you pass a string, and on the other a filename. They could have been merged, I guess.

This function, _ArrayFromString() is the supplementary to the priors discussed in the thread where I proposed the inclusion in the standard UDF shipping with AutoIt3
And the parameter calling is so similar, it makes it easy to remember and use.

7 hours ago, pixelsearch said:

One major difference between _FileReadToArray and _ArrayFromString is that _FileReadToArray generates an @error 3 ("3 - File lines have different numbers of fields

By default _ArrayFromString() will always generate the error, as it expects the given string to be a dump of the array by one of the complementary functions ( _ArrayToString() and  _SQLite_Display2DResult() ).

What I was thinking on doing is to relax the stern limitation and make it more flexible. Providing then a "dump.txt" as a string and adjust the array's 2nd dimension as needed.
And since we are in the revision mode, add code to the error check so that we don't go out of bounds in the initial 2nd dimension determination or 2nd dimension expansion (ReDim).

All these changes would not break the code from prior versions and make it better.

My question remains: would you pleeeeeassse take it over and handle the trac ticket ? ( after all you cached the code nonsense that even tho functional, still wrong. Adding the extra code and attending to it, admittedly, you have shown to better at it than me. But again, you can say, IDK, "I'll help with this, but this' still your baby" )