_FileReadToArray limits

[VelvetElvis]

Can someone tell me what the maximum file size is that can be read from an array with _FileReadToArray ?

I've checked the "AutoIt 3 limits/defaults" in the help, but no joy there.

[SmOke_N]

Maximum string length is 2147483647 characters

You can use up to 64 dimensions in an Array. The total number of entries cannot be greater than 2^24 (16 777 216).

[AdmiralAlkex]

I would guess there is 3 limits you can bump into:

• Array limit: 16 million elements
• Maximum string length: 2,147,483,647 characters
• AutoIt reaches the maximum amount of memory it can allocate

Since all files and computers are different it is impossible to say in advance which of them you reach first, and if it is the last one it is impossible to know when it will be reached.

Why do you ask? The proper way to read (very) large files would be to read parts of it, not all at once. You can do that with the _WinAPI_*() funcs.

[VelvetElvis]

I would guess there is 3 limits you can bump into:

• Array limit: 16 million elements
• Maximum string length: 2,147,483,647 characters
• AutoIt reaches the maximum amount of memory it can allocate

Since all files and computers are different it is impossible to say in advance which of them you reach first, and if it is the last one it is impossible to know when it will be reached.

Why do you ask? The proper way to read (very) large files would be to read parts of it, not all at once. You can do that with the _WinAPI_*() funcs.

I'm writing a small app (replacing an older Delphi-based one) to automate adding records to a mainframe database. Often this can involve several thousand records that would otherwise be keyed by hand. The average record is < 100 characters, and the average number of fields is 5.

Each record takes about 1 second to enter using the old application. Speed is important, so I'd prefer to read the entire CSV text file into memory at one time.

[PsaltyDS]

I'm writing a small app (replacing an older Delphi-based one) to automate adding records to a mainframe database. Often this can involve several thousand records that would otherwise be keyed by hand. The average record is < 100 characters, and the average number of fields is 5.

Each record takes about 1 second to enter using the old application. Speed is important, so I'd prefer to read the entire CSV text file into memory at one time.

That is well within operational limits for any XP-class machine or better.

It sounds like your file is only 100KB or so. The string function limits with 32-bit windows are functionally more like 128MB due to how process memory is allocated in Windows, not an inherent limitation of AutoIt.

I've heard that limit is much higher with 64-bit windows, but don't have it to test.

[VelvetElvis]

That is well within operational limits for any XP-class machine or better.

It sounds like your file is only 100KB or so. The string function limits with 32-bit windows are functionally more like 128MB due to how process memory is allocated in Windows, not an inherent limitation of AutoIt.

I've heard that limit is much higher with 64-bit windows, but don't have it to test.

Thanks for the info.

Some of our files can be as large as 100,000 records or more. In preliminary testing, I read 200,000 records of 8 fields to an array, using _FileReadToArray, in under 4 seconds. That's great.