Replace all instances in a file

[Tsukihime]

I have a file with thousands of lines and want to replace specific patterns of strings.

So I will use regexpreplace for that. I have already come up with the pattern, and everything works.

Except.....I think it could be faster.

I'm currently just reading lines from the source, doing the replace, and then writing it out to a file, but is there a way to just grab all lines at once as one huge string, run the regex (which I expect is much faster than file I/O), then write everything out at once?

[Tsukihime]

Found a solution using FileRead.

It seems to read all bytes by default if I don't provide a count. Not sure how it will handle large files though.

Edited April 27, 2011 by Tsukihime

[Melba23]

Tsukihime,

I fear you might run into RAM problems doing that as you will have at least 2 copies (original and amended) of the file in memory.

You might try using FileRead with a fairly large buffer size along these lines:

expandcollapse popup

$sFileIn = "Fred.txt"
$sFileOut = "Harry.txt"

; Create a large file
$sString = "My Name is Fred Bloggs "

For $i = 1 To 20
    $sString &= $sString
Next

ConsoleWrite(StringLen($sString) & @CRLF)

FileWrite($sFileIn, $sString)

; Open input and output files
$hFileInHandle = FileOpen(@ScriptDir & "\" & $sFileIn, 0)
$hFileOutHandle = FileOpen(@ScriptDir & "\" & $sFileOut, 2)

$iRemaining = FileGetSize($sFileIn)
$iTotal = 0
$iBufferSize = 20000

; Read in by chunks
For $i = 1 To Ceiling($iRemaining / $iBufferSize)
    If $iRemaining > $iBufferSize Then
        ConsoleWrite($i & @CRLF)
        $sSection = FileRead($hFileInHandle, $iBufferSize)
        ; Change what you need
        $sNewSection = StringReplace($sSection, "Fred", "Harry")
        ; Write to output
        FileWrite($hFileOutHandle, $sNewSection)
        $iRemaining -= $iBufferSize
        $iTotal += $iBufferSize
    Else
        ConsoleWrite("Final" & @CRLF)
        $sSection = FileRead($hFileInHandle, $iRemaining)
        $sNewSection = StringReplace($sSection, "Fred", "Harry")
        FileWrite($hFileOutHandle, $sNewSection)
        $iTotal += $iRemaining
    EndIf
Next

; Close files
FileClose($hFileInHandle)
FileClose($hFileOutHandle)

You can obviously adjust the buffer size to match your available RAM.

Any help?

M23

Edit: Our posts crossed.

Not sure how it will handle large files though

You have your answer! ;D Edited April 27, 2011 by Melba23