Jump to content
Sign in to follow this  
buymeapc

Quicker Way To Write Sorted Data to a File?

Recommended Posts

buymeapc

Hi all,

I'm trying to pull some data out of a large text file (about 35MB or more), sort the found data and write it to a file. I have a function to do this, but it's incredibly slow. Is there a faster way to do this?

Thanks for your help!

#include <Array.au3>

WConfigKey("C:\TestIn.txt", "C:\TestOut.txt")

Func WConfigKey($sInFile, $sOutFile)
    Local $aTemp[1], $iTemp = 0
    Local $hInFile  = FileOpen($sInFile)
    Local $hOutFile = FileOpen($sOutFile, 1)

    ; Read the file, line by line and take only specified data and add it to a temporary array
    While 1
        $sLine = FileReadLine($hInFile)
        If @error = -1 Then ExitLoop
        ReDim $aTemp[$iTemp+1]
        $aTemp[$iTemp] = StringMid($sLine, 1, 29) & StringMid($sLine, 35, 8)
        $iTemp += 1
    WEnd

    ; Sort the array
    _ArraySort($aTemp)

    ; Then write the data to the out file
    For $i = 0 To UBound($aTemp)-1
        FileWrite($hOutFile, $aTemp[$i])
    Next

    FileClose($hInFile)
    FileClose($hOutFile)
EndFunc

Share this post


Link to post
Share on other sites
JohnOne

; Read the file, line by line and take only specified data and add it to a temporary array

Here, get shut of the array you are placing sorted data in, and add it to a string.

Write whole string to file.

Or write to file in chunks.


AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt   ipify 

Monkey's are, like, natures humans.

Share this post


Link to post
Share on other sites
Bowmore

This should work with files up to about 200Mb

#include <Array.au3>

WConfigKey("C:\TestIn.txt", "C:\TestOut.txt")

Func WConfigKey($sInFile, $sOutFile)
    Local $aTemp[1], $iTemp = 0
    Local $hInFile  = FileOpen($sInFile)
    Local $hOutFile = FileOpen($sOutFile, 1)

    ; Read the file into an array, and the process array line by line and take only specified data and add it back to array
    $sLine = FileReadToArray($hInFile,$aTemp)
    If @error Then Return
    for $i = 1 To UBound($aTemp) - 1
        $aTemp[$iTemp] = StringMid($aTemp[$iTemp], 1, 29) & StringMid($aTemp[$iTemp], 35, 8)
    WEnd

    ; Sort the array
    _ArraySort($aTemp)

    ; Then write the data to the out file
    For $i = 1 To UBound($aTemp)-1
        FileWrite($hOutFile, $aTemp[$i])
    Next

    FileClose($hInFile)
    FileClose($hOutFile)
EndFunc
Edit: Just tidied up a few bits and pieces Edited by Bowmore

"Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to build bigger and better idiots. So far, the universe is winning."- Rick Cook

Share this post


Link to post
Share on other sites
buymeapc

Thanks Bowmore! That worked great. :thumbsup:

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×