Is writing to Hard drive faster than writing to RAM

[steve8tch]

This is a follow up to this topic Script getting slower and slower, ..should it be like this ??

One of the suggestions from JdeB was

Try replacing it with :

FileWrite($x,Chr(Random(97,122,1)))

and then later on one of the suggestions from w0uter and JdeB was to use

$str &= Chr(Random(97,122,1))

In the speed test script below - I set out to try and record the time it takes to complete a "FileWrite" function and a "$str &=" function and a "$str = $str &" - 3 ways of achieving the same reult. As shown below on vitually every occasion writing to hard drive is faster than writing to RAM.

311 , 804 , 1302 , 1712 , 2286 , 3077 , 3847 , 5092 , 8803 , 15019 ,

Total time using RAM and $str = $str & $blah to store information : 42313

96 , 137 , 190 , 240 , 286 , 330 , 379 , 428 , 470 , 516 ,

Total time using RAM and $str &= $blah to store information : 3130

97 , 83 , 96 , 85 , 82 , 84 , 82 , 83 , 85 , 81 ,

Total time using hard drive to store information : 915

Hard drive 5x faster than RAM - Does anyone else think this is odd ?

This test script takes about 45 seconds to run SpeedTest1.au3

[w0uter]

in my case RAM &= gets like 2xxx and HD gets 3xxx

maby your ram is slow ?

Edited October 2, 2005 by w0uter

[steve8tch]

I ran the same test script on 2 PCs and on one PC whilst running BartPE( ~no optimised system drivers). Whilst the numbers are slightly different - my slow PC is 2/3 speed of the faster one - the relationship between the numbers is the same. One has SD100 RAM - the other has DDR3200 RAM - both have identical 7200 hard drives. - so I was thinking that the speed of the functions was more CPU dependant.

I was just expecting the write to RAM options to be much quicker than the write to HardDrive - I realise that with DMA set the OS itself will handle the writes to the hard drive with little management required by the calling programme - but I thought that similar functionality existed for writing to RAM - so I am confused as to why the Hard drive writes are quicker.

[Valik]

You're confused because you're trying to think of this as writing conventional code which will be compiled into machine code. AutoIt is interpreted. Things like variable lookups and concatenation take a long time; string operations in general are tremendously slow. Arrays should be significantly faster than strings; it may even be faster than writing to disk. Array's are pretty fast in AutoIt.

[Jos]

This is a follow up to this topic Script getting slower and slower, ..should it be like this ??

One of the suggestions from JdeB was

and then later on one of the suggestions from w0uter and JdeB was to use

In the speed test script below - I set out to try and record the time it takes to complete a "FileWrite" function and a "$str &=" function and a "$str = $str &" - 3 ways of achieving the same reult. As shown below on vitually every occasion writing to hard drive is faster than writing to RAM.

Hard drive 5x faster than RAM - Does anyone else think this is odd ?

This test script takes about 45 seconds to run SpeedTest1.au3

Don't think I said that File IO is faster than RAM (at least I hope I didn't )

The reason I suggested to do a FileWrite was simply because your script was writing the total $str to that file anyway, so why would you first keep on copying the $str back and forth in memory while adding a new random string to before writing it to the file?

I've also tried to explain why $str &= $a is much faster then $str = $str & $a....

[steve8tch]

Thanks again for your comments.

JdeB - your explainations were fine !!

Valik - you mention that "string operations in general are tremendously slow." - is that script type languages in general or AutoIt in particular.

I think I owe you a bit of explaination. Most of the scripts I write are for GUI front end for users to automation repetative or quite complex things - to make sure they happen right first time every time (and frankly -speed is not an issue) Also other scripts that go out onto the network checking amd monitoring PCs. Just recently I wrote this script whick looks for 'events' in 100GB of data sitting on 70 PCs. The stupid thing is - analysing the data with StringRegExp is really very quick - but some of the other 'string' work was quite slow - I have now used more arrays and minimizing some work with 'pre-checks'. But it was through this exercise in trying to speed the whole thing up the some of the 'observations' and questions appeared.

[randallc]

Hi,

This is similar to the issue I had with AutoIt vs Vbscript with sorting; but this was comparing a vbscript with file write sorting, so not sure of the outcome.

I might go back and try and convert it after all, just to check...

Best, Randall Soting different times

[steve8tch]

Hi randallc

If you were able to convert the SpeedTest1.au3 or something like into vbs - it would be good to do a comparison between the two - and see if it is an AutoIt issue or not. - (it would be good if you could post the results !!)

I am not one of the devs - I don't like questioning what they do - because between them they have created a brilliant tool in AutoIt and I don't know enough to contribute in any way to the source. If it proves to be an AutoIt issue - we'll just have to work with it.

[Valik]

Strings are usually slow in general unless a high emphasis has been placed on fast string operations. Using strings for storage is particularly slow because each time you concatenate new data, the string has to be iterated over to find the end and then the new text is copied into the buffer (This assumes the buffer is the correct size, if it's too small, the buffer will have to be expanded which involves allocation, copy, deallocation and then the new characters can be copied). Compare that to an array which would get an index to the correct item and so no iteration is required. Also, depending on what or how things are stored in the array, copying may be cheap so there will be less of a performance hit from having to grow the size of storage.

This is generally true for any string implementation, however, in languages like C++ which allow the user to write their own types, significant optimizations can be made. For example, concatenation can become a simple operation if a write pointer is used which stores the position of where to write to next.