Quicker Way To Write Sorted Data to a File?

[buymeapc]

Hi all,

I'm trying to pull some data out of a large text file (about 35MB or more), sort the found data and write it to a file. I have a function to do this, but it's incredibly slow. Is there a faster way to do this?

Thanks for your help!

#include <Array.au3>

WConfigKey("C:\TestIn.txt", "C:\TestOut.txt")

Func WConfigKey($sInFile, $sOutFile)
    Local $aTemp[1], $iTemp = 0
    Local $hInFile  = FileOpen($sInFile)
    Local $hOutFile = FileOpen($sOutFile, 1)

    ; Read the file, line by line and take only specified data and add it to a temporary array
    While 1
        $sLine = FileReadLine($hInFile)
        If @error = -1 Then ExitLoop
        ReDim $aTemp[$iTemp+1]
        $aTemp[$iTemp] = StringMid($sLine, 1, 29) & StringMid($sLine, 35, 8)
        $iTemp += 1
    WEnd

    ; Sort the array
    _ArraySort($aTemp)

    ; Then write the data to the out file
    For $i = 0 To UBound($aTemp)-1
        FileWrite($hOutFile, $aTemp[$i])
    Next

    FileClose($hInFile)
    FileClose($hOutFile)
EndFunc

[JohnOne]

; Read the file, line by line and take only specified data and add it to a temporary array

Here, get shut of the array you are placing sorted data in, and add it to a string.

Write whole string to file.

Or write to file in chunks.

[Bowmore]

This should work with files up to about 200Mb

#include <Array.au3>

WConfigKey("C:\TestIn.txt", "C:\TestOut.txt")

Func WConfigKey($sInFile, $sOutFile)
    Local $aTemp[1], $iTemp = 0
    Local $hInFile  = FileOpen($sInFile)
    Local $hOutFile = FileOpen($sOutFile, 1)

    ; Read the file into an array, and the process array line by line and take only specified data and add it back to array
    $sLine = FileReadToArray($hInFile,$aTemp)
    If @error Then Return
    for $i = 1 To UBound($aTemp) - 1
        $aTemp[$iTemp] = StringMid($aTemp[$iTemp], 1, 29) & StringMid($aTemp[$iTemp], 35, 8)
    WEnd

    ; Sort the array
    _ArraySort($aTemp)

    ; Then write the data to the out file
    For $i = 1 To UBound($aTemp)-1
        FileWrite($hOutFile, $aTemp[$i])
    Next

    FileClose($hInFile)
    FileClose($hOutFile)
EndFunc

Edit: Just tidied up a few bits and pieces Edited September 13, 2013 by Bowmore

[buymeapc]

Thanks Bowmore! That worked great.