Hard crash when parsing larger strings using StringSplit()

[Bowmore]

Environment = 3.3.1.1 under WIN_XP/Service Pack 3 X86
Environment = 3.3.0.0 under WIN_XP/Service Pack 3 X86

When loading a large file to an array using string split Autoit crashes
i.e. The error message 'Error allocating memory' is displayed and AutoIt exits
I've checked that string length and array limits are not been exceeded
Perhaps I am expecting too much but I would have hoped for a more controlled way of handling a memory error, so that if my data is too large for StringSplit() to split, I could branch to a disk storage based method of processing the data I have

$sTemp = ''
$aTemp = 0
;Increase the size of the string until problem occurs
For $j = 1 To 5
  ; Create string to replicate a medium sized file having being read.
  For $i = 1 To 1000000  * $j Step 1
    $sTemp &=  String ($i) & " This is the average length of line that is in my data file" & @CRLF
  Next
  MsgBox(0, "String Split Test", "Length of string = " & StringLen($sTemp))
  $aTemp = StringSplit($sTemp, @LF ) ; Memory error occurs at this point.
  MsgBox(0, "String Split Test", "Array Ubound = " & UBound($aTemp))
  $sTemp = ''
  $aTemp = 0
Next

[Valik]

The practical limits to the amount of characters is much less than the theoretical limit. Right off the bat concatenation means you only get 2³⁰ characters instead of the listed 2³¹ characters since there must be room for both the old string and the new string (with data concatenated) to exist. Then you need to understand that every character is actually 2 bytes because AutoIt uses UNICODE. So now you're down to 2²⁹. Then you factor in some memory is reserved and some is used for the program itself. So the total number of character available is less than 2²⁹.

Now, as for the error message: It's working as designed. Running out of memory is not an easy error to recover from. For one thing, AutoIt itself doesn't know why it ran out of memory. It would be a tremendous amount of effort, bloat the code and slow the language down if we tried to handle memory exhaustion in a recoverable manner. And even then, how would AutoIt even know if it could recover from the error? What would you expect to happen in your example? How would you even test for out-of-memory in AutoIt?

So there's no bug to see here. Everything is working as expected and designed.

[Bowmore]

Replying to Valik:
Thanks for taking the time to explain the reasons this happens I'd forgotten that characters are now 2 bytes, I will now start to add checks on the size of my data and estimate if I am likely to encounter memory problems and act appropriately.