Sign in to follow this  
Followers 0
VelvetElvis

_FileReadToArray limits

6 posts in this topic

Can someone tell me what the maximum file size is that can be read from an array with _FileReadToArray ?

I've checked the "AutoIt 3 limits/defaults" in the help, but no joy there.

Share this post


Link to post
Share on other sites



Maximum string length is 2147483647 characters

You can use up to 64 dimensions in an Array. The total number of entries cannot be greater than 2^24 (16 777 216).


Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Share this post


Link to post
Share on other sites

I would guess there is 3 limits you can bump into:

  • Array limit: 16 million elements
  • Maximum string length: 2,147,483,647 characters
  • AutoIt reaches the maximum amount of memory it can allocate
Since all files and computers are different it is impossible to say in advance which of them you reach first, and if it is the last one it is impossible to know when it will be reached.

Why do you ask? The proper way to read (very) large files would be to read parts of it, not all at once. You can do that with the _WinAPI_*() funcs.

Share this post


Link to post
Share on other sites

I would guess there is 3 limits you can bump into:

  • Array limit: 16 million elements
  • Maximum string length: 2,147,483,647 characters
  • AutoIt reaches the maximum amount of memory it can allocate
Since all files and computers are different it is impossible to say in advance which of them you reach first, and if it is the last one it is impossible to know when it will be reached.

Why do you ask? The proper way to read (very) large files would be to read parts of it, not all at once. You can do that with the _WinAPI_*() funcs.

I'm writing a small app (replacing an older Delphi-based one) to automate adding records to a mainframe database. Often this can involve several thousand records that would otherwise be keyed by hand. The average record is < 100 characters, and the average number of fields is 5.

Each record takes about 1 second to enter using the old application. Speed is important, so I'd prefer to read the entire CSV text file into memory at one time.

Share this post


Link to post
Share on other sites

I'm writing a small app (replacing an older Delphi-based one) to automate adding records to a mainframe database. Often this can involve several thousand records that would otherwise be keyed by hand. The average record is < 100 characters, and the average number of fields is 5.

Each record takes about 1 second to enter using the old application. Speed is important, so I'd prefer to read the entire CSV text file into memory at one time.

That is well within operational limits for any XP-class machine or better.

It sounds like your file is only 100KB or so. The string function limits with 32-bit windows are functionally more like 128MB due to how process memory is allocated in Windows, not an inherent limitation of AutoIt.

I've heard that limit is much higher with 64-bit windows, but don't have it to test.

:P


Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites

That is well within operational limits for any XP-class machine or better.

It sounds like your file is only 100KB or so. The string function limits with 32-bit windows are functionally more like 128MB due to how process memory is allocated in Windows, not an inherent limitation of AutoIt.

I've heard that limit is much higher with 64-bit windows, but don't have it to test.

:P

Thanks for the info.

Some of our files can be as large as 100,000 records or more. In preliminary testing, I read 200,000 records of 8 fields to an array, using _FileReadToArray, in under 4 seconds. That's great.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0