Modify

Opened 10 years ago

Closed 10 years ago

Last modified 10 years ago

#1046 closed Bug (No Bug)

Hard crash when parsing larger strings using StringSplit()

Reported by: Bowmore Owned by:
Milestone: Component: AutoIt
Version: 3.3.1.1 Severity: None
Keywords: StringSplit Cc:

Description

Environment = 3.3.1.1 under WIN_XP/Service Pack 3 X86
Environment = 3.3.0.0 under WIN_XP/Service Pack 3 X86

When loading a large file to an array using string split Autoit crashes
i.e. The error message 'Error allocating memory' is displayed and AutoIt exits
I've checked that string length and array limits are not been exceeded
Perhaps I am expecting too much but I would have hoped for a more controlled way of handling a memory error, so that if my data is too large for StringSplit() to split, I could branch to a disk storage based method of processing the data I have

$sTemp = ''
$aTemp = 0
;Increase the size of the string until problem occurs
For $j = 1 To 5
  ; Create string to replicate a medium sized file having being read.
  For $i = 1 To 1000000  * $j Step 1
    $sTemp &=  String ($i) & " This is the average length of line that is in my data file" & @CRLF
  Next
  MsgBox(0, "String Split Test", "Length of string = " & StringLen($sTemp))
  $aTemp = StringSplit($sTemp, @LF ) ; Memory error occurs at this point.
  MsgBox(0, "String Split Test", "Array Ubound = " & UBound($aTemp))
  $sTemp = ''
  $aTemp = 0
Next

Attachments (0)

Change History (2)

comment:1 follow-up: Changed 10 years ago by Valik

  • Resolution set to No Bug
  • Status changed from new to closed

The practical limits to the amount of characters is much less than the theoretical limit. Right off the bat concatenation means you only get 230 characters instead of the listed 231 characters since there must be room for both the old string and the new string (with data concatenated) to exist. Then you need to understand that every character is actually 2 bytes because AutoIt uses UNICODE. So now you're down to 229. Then you factor in some memory is reserved and some is used for the program itself. So the total number of character available is less than 229.

Now, as for the error message: It's working as designed. Running out of memory is not an easy error to recover from. For one thing, AutoIt itself doesn't know why it ran out of memory. It would be a tremendous amount of effort, bloat the code and slow the language down if we tried to handle memory exhaustion in a recoverable manner. And even then, how would AutoIt even know if it could recover from the error? What would you expect to happen in your example? How would you even test for out-of-memory in AutoIt?

So there's no bug to see here. Everything is working as expected and designed.

comment:2 in reply to: ↑ 1 Changed 10 years ago by Bowmore

Replying to Valik:
Thanks for taking the time to explain the reasons this happens I'd forgotten that characters are now 2 bytes, I will now start to add checks on the size of my data and estimate if I am likely to encounter memory problems and act appropriately.

Guidelines for posting comments:

  • You cannot re-open a ticket but you may still leave a comment if you have additional information to add.
  • In-depth discussions should take place on the forum.

For more information see the full version of the ticket guidelines here.

Add Comment

Modify Ticket

Action
as closed The ticket will remain with no owner.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.