Sign in to follow this  
Followers 0
mrbig1479

best way to read a large txt file

4 posts in this topic

#1 ·  Posted (edited)

Hi

i have a few text files i want to read , i tried using a simple script like

func Rip_file ($FILE_TO_RIP)    
    $FILE=FILEOPEN ($FILE_TO_RIP,0)
        While 1
        $LINE = FileReadLine($FILE)
        If (@error = -1) Then ExitLoop
        $size=STRINGLEN ($LINE)
        ConsoleWrite ($LINE&" size is "&$size)
        $LINE=STRINGTRIMLEFT ($LINE,$size)
        ConsoleWrite ($LINE)
    WEnd
    FileClose ($FILE_TO_RIP)
EndFunc

problem is , every time i run it it takes about 120 sec to finish (on a 250KB file where each line is about 80 characters

number of lines is about 3800

i can't understand why its so slow, also i can't run it again since autoit become very slow , i have to shut it down and open again.

what im doing wrong here ?

Edited by mrbig1479

Share this post


Link to post
Share on other sites



Your function finishes in about 11.5 seconds for me which could just be better hardware, but without the Console Write it drops it down to .33 seconds so that might be the problem?

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

Your function finishes in about 11.5 seconds for me which could just be better hardware, but without the Console Write it drops it down to .33 seconds so that might be the problem?

thanks for your quick response , i tried using this simple script

func ripfile ()
Dim $FileLines

    $FILE=FILEOPEN ("File to rip.txt",0)
_FileReadToArray ($FILE,$FileLines)
For $x = 1 to $FileLines[0]
        ConsoleWrite ($FileLines[$x])
Next
    FileClose ($FILE_TO_RIP)
EndFunc
is did drop the time to 44 sec but the pc is very slow after , i feel like there is some memory leak or something , i have to close Autoit and reopen for it to work well again...

EDIT: i think i found out whats the problem.... its the ConsoleWrite command that makes it soooooo slow

i still need to manipulate the file......

EDIT#2 : without the for - next loop it takes 1.766sec , and with it it takes 44sec..... thats a huge difference

Edited by mrbig1479

Share this post


Link to post
Share on other sites

Hi,

logging is always expensive. Try to store the info and write it at ones.

Mega


Scripts & functions Organize Includes Let Scite organize the include files

Yahtzee The game "Yahtzee" (Kniffel, DiceLion)

LoginWrapper Secure scripts by adding a query (authentication)

_RunOnlyOnThis UDF Make sure that a script can only be executed on ... (Windows / HD / ...)

Internet-Café Server/Client Application Open CD, Start Browser, Lock remote client, etc.

MultipleFuncsWithOneHotkey Start different funcs by hitting one hotkey different times

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0