Sign in to follow this  
Followers 0
Tutech

File IO and poor performance

8 posts in this topic

#1 ·  Posted (edited)

Hi there.

I need a small script that reads a file and writes the changed output to another file.

The script works fine and the result is what i need.

The problem that I have to solve is the poor performance of the file io.

Here is my script.

<---- Snip ----->

$file = FileOpen("test.txt", 0)

$filew = FileOpen("test-umgewandelt.txt", 1)

If $file = -1 Then

MsgBox(0, "Error", "Unable to open file.")

Exit

EndIf

$lastchar = ""

While 1

$chars = FileRead($file, 1)

If @error = -1 Then ExitLoop

if $lastchar <> chr(13) then $chars=StringAddCR($chars);

FileWrite($filew, $chars)

$lastchar = $chars

Wend

FileClose($file)

FileClose($filew)

<---- Snap ---->

When the script is run it takes up to 2 minutes to read and write a 1k file! :)

I`ve read that some coders read the file into an array. That is not possible in this case beacause there are some very large files that have to be read.

Is there another way to speed up the whole thing?

Thanks :evil:

Greetings

Micha

Edited by Tutech

Share this post


Link to post
Share on other sites



Reading by one char is already slow process... And probably mistake in the algorithm, that I completely fail fo understand :)

Can you explain what this script should do?

<{POST_SNAPBACK}>

Sure :">

The problem I`d like to solve is to change all LineFeeds in a textfile into CRLF.

This is needed by our cyrus-imap server.

I read all files char by char because I have to check if the char is a LF.

Did you understand me???? :evil:

I hope so.

Greetings

Micha :D

Share this post


Link to post
Share on other sites

can you use the Filereadline in your environment to replace the fileread and the comparison to chr(13). look at the doc example.

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

can you use the Filereadline in your environment to replace the fileread and the comparison to chr(13). look at the doc example.

<{POST_SNAPBACK}>

Hi. I think that this won´t work in my case.

I use StringAddCR to prefix all LF with a CR.

The problem of StringAddCR is that it prefixes all LF's.

This means that a CRLF will be changed into a CRCRLF .

So I have to check the every char. This is the reason why I read the file char by char.

Or is there a possibility to change the behaviour of StringAddCR?

Thanks.

Micha :)

Edited by Tutech

Share this post


Link to post
Share on other sites

FileReadLine read lines, both ends with LF or CRLF. If this ends with CRLF then CR remain in the string. So you can strip this CR and write new line with FileWriteLine that automatically appends CRLF if no CR or LF at the end of string.

While 1
    $line = FileReadLine($file)
    If @error = -1 Then ExitLoop
    $line = StringStripCR($line)
    FileWriteLine($filew, $line)
Wend

Share this post


Link to post
Share on other sites

FileReadLine read lines, both ends with LF or CRLF. If this ends with CRLF then CR remain in the string. So you can strip this CR and write new line with FileWriteLine that automatically appends CRLF if no CR or LF at the end of string.

While 1
    $line = FileReadLine($file)
    If @error = -1 Then ExitLoop
    $line = StringStripCR($line)
    FileWriteLine($filew, $line)
Wend

<{POST_SNAPBACK}>

Wow! :) Lazycat, you are my hero!

This is what I needed. Thanks! :evil:

Now the script works perfect.

Thank you!

Greetings

Micha

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

Ok. Here is the whole script.

I think that this small script is very usefull to avoid the "bare newlines" messages. :-)

You just have to convert the Thunderbox mbox-files.

$dir = @AppDataDir & "\Thunderbird\Profiles"

while 1
    $myfile = FileOpenDialog("Choose mbox-file for conversion",$dir,"All (*.*)")
    if $myfile= 1 then ExitLoop
    $file = FileOpen($myfile, 0)
    $filew = FileOpen($myfile & "-crlf-add", 1)

    If $file = -1 Then
        MsgBox(0, "Error", "Unable to open mbox-file.")
        Exit
    EndIf

    While 1
        $line = FileReadLine($file)
        If @error = -1 Then ExitLoop
        $line = StringStripCR($line)
        FileWriteLine($filew, $line)
    Wend

    FileClose($file)
    FileClose($filew)
    msgbox(0, "Meldung","Konvertierung von " & $myfile & " beendet")
    $dir = $myfile
wend

Ok.

Greetings

Micha

Edited by Tutech

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0