steve8tch Posted September 29, 2005 Share Posted September 29, 2005 I was writing some code to show a problem I have with a script that uses 4 large arays - and the fact that each set of files that it was processing seem to take longer and longer. It turns out that the first part of this illustrative code exhibits the same issue.If you run this code - (it take 10 seconds to complete) you will see that the first set of loops take approx 100ms to execute - but the 20th set take 600ms - 6x the time for the identical bit of code. I don't think memory use is an issue (autoit.exe memory use in Task manager hardly moves) and the resultant 40k text file is hardly huge. The CPU will run at 100% throughout.I ran it again so that it executes 60 sets of loops - and the last loops are taking 1.8 seconds - this is 20x as long to exectute the same bit of code.Here are the results of a couple of runs20 loops - times in millisecond for each batch of 10 loops.71 , 83 , 105 , 127 , 156 , 179 , 206 , 226 , 303 , 319 , 350 , 331 , 357 , 381 , 405 , 430 , 497 , 559 , 583 , 62060 loops65 , 82 , 105 , 126 , 155 , 178 , 206 , 231 , ,,,,,,,, 1636 , 1676 , 1674 , 1702 , 1749 , 1818 , 1860 , 1879 , 1887 The code - I have also attached the test script.DirCreate(@Scriptdir & "\test files")$x = FileOpen(@ScriptDir & "\test files\log.txt",1)dim $str = ""dim $time = ""dim $counter = 0$secs = TimerInit()For $i = 1 to 200For $j = 1 to 200$str = $str & Chr(Random(97,122,1)) ; this bit just generates random lowercase charactors Next$str = $str & @CRLF$counter = $counter + 1If $counter = 10 Then ; lets get the time for each group of 10 loops$time = $time & StringFormat("%.0f",TimerDiff($secs)) & " , "$counter = 0$secs = TimerInit()EndIfNext$x = FileOpen(@ScriptDir & "\test files\log.txt",1)FileWriteLine($x,$str & @CRLF)FileClose($x)MsgBox(0,"Time in milliseconds for each group of 10 loops",$time)Has anyone any thoughts. Thanks for your help. Link to comment Share on other sites More sharing options...
Developers Jos Posted September 29, 2005 Developers Share Posted September 29, 2005 I was writing some code to show a problem I have with a script that uses 4 large arays - and the fact that each set of files that it was processing seem to take longer and longer. It turns out that the first part of this illustrative code exhibits the same issue.It is this line causing the code to start running slower after each loop because the string is growing and takes longer to copy:$str = $str & Chr(Random(97,122,1)) ; this bit just generates random lowercase charactors Try replacing it with :FileWrite($x,Chr(Random(97,122,1)))and see the result... SciTE4AutoIt3 Full installer Download page - Beta files Read before posting How to post scriptsource Forum etiquette Forum Rules Live for the present, Dream of the future, Learn from the past. Link to comment Share on other sites More sharing options...
steve8tch Posted September 29, 2005 Author Share Posted September 29, 2005 Thankyou for your reply. As you suggest - in this situation it is best to write to data direct to file. I thought that RAM was much faster than hard drive - and that constructing strings in "memory" would be much faster than continually writing out to hard drive. Would you suggest that if we (AutoIt users) are writing scripts that "loop" round collecting information - we should avoid collecting data in $str and $array and write to temp files - and then read these files in when required ? I'm slightly confused Link to comment Share on other sites More sharing options...
w0uter Posted September 29, 2005 Share Posted September 29, 2005 it wont probly help that much but "bla &= new" is faster then "bla = bla & new" My UDF's:;mem stuff_Mem;ftp stuff_FTP ( OLD );inet stuff_INetGetSource ( OLD )_INetGetImage _INetBrowse ( Collection )_EncodeUrl_NetStat_Google;random stuff_iPixelSearch_DiceRoll Link to comment Share on other sites More sharing options...
Somerset Posted September 29, 2005 Share Posted September 29, 2005 here i just passed off the $str to another varible $grand if the length of $str gets beyond 5000 pass it onto another varible and then make $str equal nothing. i even raised the number of loops the times are pretty much the same. a simple ditch and switch. the times are all faster than what you were getting. expandcollapse popupDirCreate(@Scriptdir & "\test files") $x = FileOpen(@ScriptDir & "\test files\log.txt",1) dim $str = "" dim $time = "" dim $counter = 0 dim $grand = "" $secs = TimerInit() For $i = 1 to 400 For $j = 1 to 200 $str = $str & Chr(Random(97,122,1)); this bit just generates random lowercase charactors $len = StringLen($str) If $len > 5000 Then $grand = $grand & $str $str = "" endif Next $str = $str & @CRLF $counter = $counter + 1 If $counter = 10 Then; lets get the time for each group of 10 loops $time = $time & StringFormat("%.0f",TimerDiff($secs)) & " , " $counter = 0 $secs = TimerInit() EndIf Next $grand = $grand & $str $str = $grand $x = FileOpen(@ScriptDir & "\test files\log.txt",1) FileWriteLine($x,$str & @CRLF) FileClose($x) MsgBox(0,"Time in milliseconds for each group of 10 loops",$time) Link to comment Share on other sites More sharing options...
steve8tch Posted September 30, 2005 Author Share Posted September 30, 2005 Thankyou for your replies. As beerman suggested - if I alter the code to this ... ... If $counter = 10 Then; lets get the time for each group of 10 loops $strX = $strX & $str $str = "" $time = $time & StringFormat("%.0f",TimerDiff($secs)) & " , " $counter = 0 $secs = TimerInit() EndIf ... ... I get the following timevalues (milliseconds) 149 , 149 , 156 , 146 , 149 , 152 , 149 , 150 , 148 , 149 , 151 , 151 , 150 , 150 , 151 , 149 , 150 , 151 , 150 , 151 What I am still slightly confused about is this: I thought that writing to RAM was quick - I understand that AutoIt needs to manage its bit of memory - but are these times taken comparable with other script laguages ? I am not questioning the ability of the devs here - I am jst trying to understand what is happening in order to optimise some scripts. Thanks for your help. Link to comment Share on other sites More sharing options...
Developers Jos Posted September 30, 2005 Developers Share Posted September 30, 2005 Try this script and see the difference in speed. my test shows that example 2 is 8 times faster. I haven't looked in the source yet but assume that the first example will move the $var in temp mem space, then add $x at the end of it in temp mem space and then move the whole thing to $var. This is needed or else statements like :$var = $x & $var would fail...... The second example seems to find the "end" of the $var memspace and adds $x to it. I am sure dev's will chime in when i am incorrect... $Var = "" $t = TimerInit() For $x = 1 to 10000 $var = $var & $x Next ConsoleWrite('@@ Debug(6) : TimerDiff($t) = ' & TimerDiff($t) & @lf & '>Error code: ' & @error & @lf) ;### Debug Console $Var = "" $t = TimerInit() For $x = 1 to 10000 $var &= $x Next ConsoleWrite('@@ Debug(6) : TimerDiff($t) = ' & TimerDiff($t) & @lf & '>Error code: ' & @error & @lf) ;### Debug Console SciTE4AutoIt3 Full installer Download page - Beta files Read before posting How to post scriptsource Forum etiquette Forum Rules Live for the present, Dream of the future, Learn from the past. Link to comment Share on other sites More sharing options...
steve8tch Posted September 30, 2005 Author Share Posted September 30, 2005 @w0uter and @JdeB what a difference.. replacing $str = $str & Chr(Random(97,122,1)) with $str &= Chr(Random(97,122,1)) results in 208 , 113 , 113 , 123 , 114 , 113 , 113 , 118 , 113 , 117 , 112 , 114 , 116 , 112 , 116 , 114 , 113 , 115 , 116 , 115 , This must be the way to go. Gents - thankyou Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now