File IO and poor performance

[Tutech]

Hi there.

I need a small script that reads a file and writes the changed output to another file.

The script works fine and the result is what i need.

The problem that I have to solve is the poor performance of the file io.

Here is my script.

<---- Snip ----->

$file = FileOpen("test.txt", 0)

$filew = FileOpen("test-umgewandelt.txt", 1)

If $file = -1 Then

MsgBox(0, "Error", "Unable to open file.")

Exit

EndIf

$lastchar = ""

While 1

$chars = FileRead($file, 1)

If @error = -1 Then ExitLoop

if $lastchar <> chr(13) then $chars=StringAddCR($chars);

FileWrite($filew, $chars)

$lastchar = $chars

Wend

FileClose($file)

FileClose($filew)

<---- Snap ---->

When the script is run it takes up to 2 minutes to read and write a 1k file!

I`ve read that some coders read the file into an array. That is not possible in this case beacause there are some very large files that have to be read.

Is there another way to speed up the whole thing?

Thanks

Greetings

Micha

Edited June 23, 2005 by Tutech

[Lazycat]

Reading by one char is already slow process... And probably mistake in the algorithm, that I completely fail fo understand

Can you explain what this script should do?

[Tutech]

Reading by one char is already slow process... And probably mistake in the algorithm, that I completely fail fo understand

Can you explain what this script should do?

<{POST_SNAPBACK}>

Sure :">

The problem I`d like to solve is to change all LineFeeds in a textfile into CRLF.

This is needed by our cyrus-imap server.

I read all files char by char because I have to check if the char is a LF.

Did you understand me????

I hope so.

Greetings

Micha

[jpm]

can you use the Filereadline in your environment to replace the fileread and the comparison to chr(13). look at the doc example.

[Tutech]

can you use the Filereadline in your environment to replace the fileread and the comparison to chr(13). look at the doc example.

<{POST_SNAPBACK}>

Hi. I think that this won´t work in my case.

I use StringAddCR to prefix all LF with a CR.

The problem of StringAddCR is that it prefixes all LF's.

This means that a CRLF will be changed into a CRCRLF .

So I have to check the every char. This is the reason why I read the file char by char.

Or is there a possibility to change the behaviour of StringAddCR?

Thanks.

Micha

Edited June 23, 2005 by Tutech

[Lazycat]

FileReadLine read lines, both ends with LF or CRLF. If this ends with CRLF then CR remain in the string. So you can strip this CR and write new line with FileWriteLine that automatically appends CRLF if no CR or LF at the end of string.

While 1
    $line = FileReadLine($file)
    If @error = -1 Then ExitLoop
    $line = StringStripCR($line)
    FileWriteLine($filew, $line)
Wend

[Tutech]

FileReadLine read lines, both ends with LF or CRLF. If this ends with CRLF then CR remain in the string. So you can strip this CR and write new line with FileWriteLine that automatically appends CRLF if no CR or LF at the end of string.

While 1
    $line = FileReadLine($file)
    If @error = -1 Then ExitLoop
    $line = StringStripCR($line)
    FileWriteLine($filew, $line)
Wend

<{POST_SNAPBACK}>

Wow! Lazycat, you are my hero!

This is what I needed. Thanks!

Now the script works perfect.

Thank you!

Greetings

Micha

[Tutech]

Ok. Here is the whole script.

I think that this small script is very usefull to avoid the "bare newlines" messages. :-)

You just have to convert the Thunderbox mbox-files.

$dir = @AppDataDir & "\Thunderbird\Profiles"

while 1
    $myfile = FileOpenDialog("Choose mbox-file for conversion",$dir,"All (*.*)")
    if $myfile= 1 then ExitLoop
    $file = FileOpen($myfile, 0)
    $filew = FileOpen($myfile & "-crlf-add", 1)

    If $file = -1 Then
        MsgBox(0, "Error", "Unable to open mbox-file.")
        Exit
    EndIf

    While 1
        $line = FileReadLine($file)
        If @error = -1 Then ExitLoop
        $line = StringStripCR($line)
        FileWriteLine($filew, $line)
    Wend

    FileClose($file)
    FileClose($filew)
    msgbox(0, "Meldung","Konvertierung von " & $myfile & " beendet")
    $dir = $myfile
wend

Ok.

Greetings

Micha

Edited June 23, 2005 by Tutech