any FileWrite limitations ?

[kiboost]

Hi,

I'm doing some stringreplace in a file, and having no problem with file less than 500Ko, when files reach 900Ko/1Mo, my result file get errors. It cut some portions in some lines randomely. On a 50 000 line xml, hard to exactly see what happend, but it happens.

Here is the code I use (part of)

$compfile = FileOpen($rw_comp, 0)
$compfile = FileRead($compfile)

; some StringRegExpReplace there !

$hFile = FileOpen($rw_comp, 2)
FileWrite($hFile, $compfile)
FileClose($hFile)

So, did anyone heard of some size limitation for these functions ? Or anything could cause that ?

Thanks

Kib

[Melba23]

kiboost,

The problem is that the file is too large to be read into memory in one piece and have a second amended copy as well.

I would suggest looking at _WinAPI_SetFilePointer in conjunction with _WinAPI_ReadFile - that way you can get manageable sized chunks into memory to amend. Write the amended text to a temporary file as you progress and then rewrite the main file once everything is done.

M23

[kiboost]

Thks Melba !

I changed my code to this : only one fileopen

$compfileopen = FileOpen($rw_comp, 2)
$compfile = FileRead($compfileopen )

; some StringRegExpReplace there !

FileWrite($compfileopen, $compfile)
FileClose($compfileopen)

Now it works.

I'm on x64 windows7 with 4Go of ram, so is it possible to reach such limit ? Or did it was just the two fileopen that caused this ? In case it happens again later with even larger files (10Mo should be the maximum).

[Melba23]

kiboost,

Sooner or later you will run into the memory problem - it all depends on what else is trying to use it at the time. Reading seriously large files into memory is always fraught with danger - that is why "proper" applications use the "chunk" technique I mentioned above and fool the user into thinking everything is in memory at once.

M23

[kiboost]

thanks Melba, you are totally right pointe me in good method direction, and thanks for that. I will definitely have a look at this method to implement it soon.

Thanks for your time on this board, allways very pertinent comment.

Kib