Jump to content
Sign in to follow this  
Mechaflash

FileOpen() and FileWrite() performance question

Recommended Posts

Mechaflash

As long as I want to write with ASCII chracters, is there a benefit to FileOpen() the file first before writing to it as opposed to just letting FileWrite() do all the work? Most instances, I would only write to a file a handful of times throughout a program for logging, reporting etc... which in my mind there wouldn't be any performance issues just using FileWrite(). But if I had a program which did huge amounts of logging and reporting, would it be recommended to FileOpen() it first... write to it throughout the program, then FileClose() it after all is said and done?


Spoiler

“Hello, ladies, look at your man, now back to me, now back at your man, now back to me. Sadly, he isn’t me, but if he stopped using ladies scented body wash and switched to Old Spice, he could smell like he’s me. Look down, back up, where are you? You’re on a boat with the man your man could smell like. What’s in your hand, back at me. I have it, it’s an oyster with two tickets to that thing you love. Look again, the tickets are now diamonds. Anything is possible when your man smells like Old Spice and not a lady. I’m on a horse.”

 

Share this post


Link to post
Share on other sites
Blue_Drache

I don't really know much about the performance issues, but I just FileOpen(), do stuff, FileClose() if I need to do onesie-twosie stuff, and then FileOpen(), loop, FileClose() if I'm writing a bunch of stuff.

The benefits to using FileOpen() are helping to ensure that it's available when you need to write to it, instead of having to wait for another process to release the file, or risk unintentionally overwriting data or having your data unintentionally overwritten.


Lofting the cyberwinds on teknoleather wings, I am...The Blue Drache

Share this post


Link to post
Share on other sites
Andreik

FileWrite() will perform for you open/close operations. So should't be any any performance problem with just using FileWrite().


When the words fail... music speaks

Share this post


Link to post
Share on other sites
JLogan3o13

So, this interests me, as I have always used FileOpen first. I tested this out on an XP box with this code for FileOpen:

$var = FileOpen("C:Test.txt", 1)
For $i = 1 To 5000
  FileWrite($var, "This is a test of the Emergency Broadcast Network." & @CRLF)
Next

And this code for FileWrite:

For $i = 1 To 5000
  FileWrite("C:Test.txt", "This is a test of the Emergency Broadcast Network." & @CRLF)
Next

These are the times that I saw. I am using a virtualized instance of AutoIt, but don't believe that is a factor, as I get nearly the same times using a full install.

>Exit code: 0    Time: 1.851 - FileOpen   
                                                                         Creating File
>Exit code: 0    Time: 3.440 - FileWrite Only

>Exit code: 0    Time: 1.829 - FileOpen   
                                                                         File Exists
>Exit code: 0    Time: 3.433

√-1 2^3 ∑ π, and it was delicious!

How to get your question answered on this forum!

Share this post


Link to post
Share on other sites
kylomas

mechaflash213,

Try this and see what you think...

;
;
;

#include <file.au3>
#include <date.au3>

Local $file1  = "c:tmpwrite_noopen_test", $block,$st, $incr = 200
Local $file2  = "c:tmpwrite_open_test", $block,$st

For $i = 0 To 2000
    $block &= chr(Random(32,127,1))
Next

$block &= @crlf

ConsoleWrite(' Block Used for filewrite test = ' & $block & @lf)
ConsoleWrite(' Start test w/o explicit open at ' & _now() & @lf)

For $i = 1 To 10
    $st = TimerInit()
    For $j = 1 To $i * $incr
        FileWrite($file1,$block)
    next
    ConsoleWrite('Iteration # ' & $i & ' = ' & Round(TimerDiff($st)/1000,4) & @lf)
next

ConsoleWrite(' Start test w   explicit open at ' & _now() & @lf)
Local $hfl = FileOpen($file2,1)

For $i = 1 To 10
    $st = TimerInit()
    For $j = 1 To $i * $incr
        FileWrite($hfl,$block)
    next
    ConsoleWrite('Iteration # ' & $i & ' = ' & Round(TimerDiff($st)/1000,4) & @lf)
next

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites
BrewManNH

If your file is very large, using FileOpen before FileWrite is a huge savings in time over just using FileWrite. Using parts of kylomas' script above I changed it so that it wrote 5000 lines of 2000 characters each, and this is what I get after running it the first time:

Start test w/o explicit open at 3/23/2012 2:04:11 PM

Iteration # 5001 = 24.8967

Start test w explicit open at 3/23/2012 2:04:36 PM

Iteration # 5001 = 0.0746

And this is what I get after running it twice where the file exists and is about 9.5MB in size after the first run.

Start test w/o explicit open at 3/23/2012 2:01:31 PM

Iteration # 5001 = 25.0751

Start test w explicit open at 3/23/2012 2:01:56 PM

Iteration # 5001 = 0.183


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
Mechaflash

are my eyes deceiving me... or am I reading it as if you don't call FileOpen() first, it takes 25 seconds... as opposed to 0.183 seconds??? (referencing your second run).


Spoiler

“Hello, ladies, look at your man, now back to me, now back at your man, now back to me. Sadly, he isn’t me, but if he stopped using ladies scented body wash and switched to Old Spice, he could smell like he’s me. Look down, back up, where are you? You’re on a boat with the man your man could smell like. What’s in your hand, back at me. I have it, it’s an oyster with two tickets to that thing you love. Look again, the tickets are now diamonds. Anything is possible when your man smells like Old Spice and not a lady. I’m on a horse.”

 

Share this post


Link to post
Share on other sites
BrewManNH

Correct, it's writing 5000 lines of 2000 characters per line using FileWrite for each line. When you don't use FileOpen it is opening and closing the file everytime you use FileWrite, plus it has to start from the beginning of the file until it finds the end of the file so that it can append the new data to it every time, as the file gets bigger, the time gets longer because it's running through, eventually, 4999 lines of text just to add one more line. Whereas FileOpen/FileWrite keeps track of where the next line is supposed to go so it just puts it there right away.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
Mechaflash

Okay that's understandable. Thanks for taking some time to perform a demonstration. I'm floating between like 3 projects at work now and didn't get a chance to plug in his code to run my own test.


Spoiler

“Hello, ladies, look at your man, now back to me, now back at your man, now back to me. Sadly, he isn’t me, but if he stopped using ladies scented body wash and switched to Old Spice, he could smell like he’s me. Look down, back up, where are you? You’re on a boat with the man your man could smell like. What’s in your hand, back at me. I have it, it’s an oyster with two tickets to that thing you love. Look again, the tickets are now diamonds. Anything is possible when your man smells like Old Spice and not a lady. I’m on a horse.”

 

Share this post


Link to post
Share on other sites
guinness

I've personally use FileOpen, FileWrite & FileClose for the reasons mentioned above, it's the same for _FileWriteLog which I pass the handle to instead of the filename.


UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×