KiwiCoder Posted August 3, 2011 Share Posted August 3, 2011 (edited) Hello again helpful autoit community I would appreciate it if someone can help me optimize this and make it run faster. this turns my non-comma separated text files into comma separated ones and adds a comma after each two chars. this code must be ugly because it takes 40 mins to process 500k file! and im looking forward to process 100mb files!! also what would be the best way to deal with large files? put them in array's? thank you #include <File.au3> #include<string.au3> HotKeySet("{ESC}", "Terminate") _LoadMainInput("Numerical-4.txt") Func _LoadMainInput($File) $MList = FileRead($File) $len = StringLen($MList) For $j = 0 To $len * 2 $j += 2 ConsoleWrite($j & "," & $len * 2 & @CRLF) $MList = _StringInsert($MList, ",", $j) Next FileWrite("Num2Comma.txt", $MList) Return $MList EndFunc ;==>_LoadMainInput Func Terminate() Exit 0 EndFunc ;==>Terminate the consolewrite() is important to make sure its not going to take double the age of the universe to process the file. Edited August 3, 2011 by KiwiCoder Link to comment Share on other sites More sharing options...
hannes08 Posted August 3, 2011 Share Posted August 3, 2011 Func _LoadMainInput($File) $fh_in = FileOpen($File) $fh_out = FileOpen("Output.txt", 2) While 1 $twochars = FileRead($fh_in, 2) If @error = -1 Then ExitLoop FileWrite($fh_out, $twochars & ",") WEnd FileClose($fh_in) FileClose($fh_out) EndFunc ;==>_LoadMainInput ... Takes about 3 seconds on my PC for a 500kb file. Regards,Hannes[spoiler]If you can't convince them, confuse them![/spoiler] Link to comment Share on other sites More sharing options...
KiwiCoder Posted August 3, 2011 Author Share Posted August 3, 2011 (edited) Func _LoadMainInput($File) $fh_in = FileOpen($File) $fh_out = FileOpen("Output.txt", 2) While 1 $twochars = FileRead($fh_in, 2) If @error = -1 Then ExitLoop FileWrite($fh_out, $twochars & ",") WEnd FileClose($fh_in) FileClose($fh_out) EndFunc ;==>_LoadMainInput ... Takes about 3 seconds on my PC for a 500kb file. haha awesome m8 this is magic! you saved my time!! EDIT: nvm dont care how it works as long as it works... Edited August 3, 2011 by KiwiCoder Link to comment Share on other sites More sharing options...
hannes08 Posted August 3, 2011 Share Posted August 3, 2011 I'm sure there's even a faster way than that. Regards,Hannes[spoiler]If you can't convince them, confuse them![/spoiler] Link to comment Share on other sites More sharing options...
hannes08 Posted August 3, 2011 Share Posted August 3, 2011 Here it is: Func _LoadMainInput($File) $s_t = "" Local $a_content _FileReadToArray($File, $a_content) For $i = 1 To $a_content[0] For $j = 1 To StringLen($a_content[$i]) Step 2 $s_t &= StringMid($a_content[$i], $j, 2) & "," Next $s_t &= @CRLF Next $fh_out = FileOpen("Output.txt", 2) FileWrite($fh_out, $s_t) FileClose($fh_out) EndFunc ;==>_LoadMainInput Same file 590ms vs 3000ms of prior version. Regards,Hannes[spoiler]If you can't convince them, confuse them![/spoiler] Link to comment Share on other sites More sharing options...
PowerCat Posted August 3, 2011 Share Posted August 3, 2011 Hannes, are you a wizard? Link to comment Share on other sites More sharing options...
Blue_Drache Posted August 3, 2011 Share Posted August 3, 2011 Nah, just bored. Lofting the cyberwinds on teknoleather wings, I am...The Blue Drache Link to comment Share on other sites More sharing options...
Moderators SmOke_N Posted August 4, 2011 Moderators Share Posted August 4, 2011 (edited) Curious, how does this compare? Global $gs_FileToConvert = "PutFileHere" Global $gs_FileToConvertTo = "PutFileHere" _AddDelimToFile($gs_FileToConvert, $gs_FileToConvertTo) Func _AddDelimToFile($s_infile, $s_outfile, $s_delim = ",", $n_everxchars = 2) Local $s_fread = FileRead($s_infile) Local $h_outfile = FileOpen($s_outfile, 2) FileWrite($h_outfile, StringRegExpReplace($s_fread, "(.{" & $n_everxchars & "}(?!(?:\z|\v+)))", "$1,")) FileClose($h_outfile) Return EndFunc Edited August 4, 2011 by SmOke_N Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer. Link to comment Share on other sites More sharing options...
hannes08 Posted August 4, 2011 Share Posted August 4, 2011 Nah, just bored. Regards,Hannes[spoiler]If you can't convince them, confuse them![/spoiler] Link to comment Share on other sites More sharing options...
hannes08 Posted August 4, 2011 Share Posted August 4, 2011 Curious, how does this compare? Global $gs_FileToConvert = "PutFileHere" Global $gs_FileToConvertTo = "PutFileHere" _AddDelimToFile($gs_FileToConvert, $gs_FileToConvertTo) Func _AddDelimToFile($s_infile, $s_outfile, $s_delim = ",", $n_everxchars = 2) Local $s_fread = FileRead($s_infile) Local $h_outfile = FileOpen($s_outfile, 2) FileWrite($h_outfile, StringRegExpReplace($s_fread, "(.{" & $n_everxchars & "}(?!(?:\z|\v+)))", "$1,")) FileClose($h_outfile) Return EndFunc 400ms RegEx is still a big mystery for me! Regards,Hannes[spoiler]If you can't convince them, confuse them![/spoiler] Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now