File update/overwrite

[jugador]

Need suggestion 😅
I have to run the code for almost 18 hours
so what the most logical / memory friendly way to update the file

Method 1

;~ method 1
While sleep(250)
    ;code logic....
    ;........
    
    $hFileOpen = FileOpen($sFilePath, 2)
    FileWrite($hFileOpen, $abc)
    FileClose($hFileOpen)
WEnd

Method 2

;~ method 2
$hFileOpen = FileOpen($sFilePath, 2)
While sleep(250)
    ;code logic....
    ;........
    
    FileSetPos($hFileOpen, 0, 0)
    FileWrite($hFileOpen, $abc)
    FileSetEnd($hFileOpen)
WEnd
FileClose($hFileOpen)

example Method 1

Local Const $sFilePath = @ScriptDir & "\abcd.txt"
Local $hFileOpen
Local $i = 0
Local $abc = ""

While $i <= 50
    If Mod($i, 2) = 0 Then
        $abc = $i & @CRLF 
    Else
        $abc = ($i * 100) & @CRLF & ($i * 200) & @CRLF & ($i * 300) & @CRLF
    EndIf
    
    $hFileOpen = FileOpen($sFilePath, 2)
    FileWrite($hFileOpen, $abc)
    FileClose($hFileOpen)

    $i = $i + 1
WEnd
MsgBox(0, "", "Done")

example Method 2

Local Const $sFilePath = @ScriptDir & "\abcd.txt"
Local $hFileOpen = FileOpen($sFilePath, 2)
Local $i = 0
Local $abc = ""

While $i <= 50
    If Mod($i, 2) = 0 Then
        $abc = $i & @CRLF 
    Else
        $abc = ($i * 100) & @CRLF & ($i * 200) & @CRLF & ($i * 300) & @CRLF
    EndIf
    
    FileSetPos($hFileOpen, 0, 0)
    FileWrite($hFileOpen, $abc)
    FileSetEnd($hFileOpen)

    $i = $i + 1
WEnd
FileClose($hFileOpen)
MsgBox(0, "", "Done")

 

Edited March 20, 2021 by jugador

[Nine]

Not sure I understand the purpose of all your examples.  Could you show an example of the input and the output. I must admit I am kind tired.  Maybe tomorrow will make it clearer 

[pseakins]

I don't think there would be any memory implications, these are standard IO operations. However, Method 1 achieves nothing because your target file is erased on each iteration of the loop, because you are opening the file with option $FO_OVERWRITE (2) = Write mode (erase previous contents).

Method 2 also achieves nothing because you are resetting the file position pointer to zero during each iteration of the loop.

In my opinion neither of these methods will achieve anything except waste 18 hours of computing time. However, I have been proven wrong many times in the past, this could be another.

[JockoDundee]

5 hours ago, jugador said:

so what the most logical / memory friendly way to update the file

impossible to answer without at least two additional data points (maybe more)

1. when you say update, do you mean overwrite completely with new data or append new data?
2. could any process be trying to access (read) this file at the same time?

btw, whatever you are trying to do with filesetend/filesetpos is probably not necessary - just use filewrite.

[pixelsearch]

I'd go with @JockoDundee on that one, though we may be both wrong !
OP certainly checked the file content after he ran his 2 examples, so Overwrite seems to be what he needs.
Maybe another software will continuously read the content of this "bridge" file to get real-time data and process it ?

@jugador : I'd go with Method2 (which opens the file only once) and tested two scripts below :
* 1st script (writing) to be run from Explorer

; 1) Writing script : run from Explorer

Sleep(3000) ; delay to enter Scite and Run the other script (reading script)
Local Const $sFilePath = @ScriptDir & "\abcd.txt"
Local $hFileOpen = FileOpen($sFilePath, 2) ; Write mode (erase previous contents)
Local $i = 0
Local $hTimer = TimerInit(), $fDiff
While 1
    $i += 1
    FileSetPos($hFileOpen, 0, 0)
    FileWrite($hFileOpen, $i)
    FileSetEnd($hFileOpen) ; mandatory, so next Write will erase totally a preceding value

    $fDiff = TimerDiff($hTimer)
    If $fDiff > 10000 Then ExitLoop ; 10s
    Sleep(10)
WEnd
FileClose($hFileOpen)
ConsoleWrite(TimerDiff($hTimer) / 1000 & @lf)

* 2nd script (reading) to be run from Scite during the 3 seconds delay (Sleep 3000)

; 2) Reading script : run from Scite (just after Writing script was run from Explorer)

Local Const $sFilePath = @ScriptDir & "\abcd.txt"
Local $hFileOpen = FileOpen($sFilePath) ; Read mode (default)
Local $hTimer = TimerInit(), $fDiff
While 1
    FileSetPos($hFileOpen, 0, 0)
    $sFileRead = FileRead($hFileOpen)
    If @error Then Exit MsgBox(0, "Read Error", "")

    $fDiff = TimerDiff($hTimer)
    If $fDiff > 10000 Then ExitLoop ; 10s
    ConsoleWrite($sFileRead & "   " & $fDiff / 1000 & @lf)
    Sleep(10)
WEnd
FileClose($hFileOpen)
ConsoleWrite(TimerDiff($hTimer) / 1000 & @lf)

Console excerpt, choosing a part where a value is doubled (211) and another one is missing (214)

...
209   1.68841933821198
210   1.70512732763522
211   1.7157971194663
211   1.72742513364129
212   1.73909896369511
213   1.75097673028276
215   1.76267095398996
216   1.77430427610213
...

I'm not sure if FileFlush() is needed after each write operation (it could slow down the whole process) but maybe other readers will give their opinion about that (to read the most accurate value from the file in real time)

Anyway, the task manager didn't show anything special while both scripts were running, minimal CPU charge and memory usage.

Edit: FileSetEnd() seems important when writing, this is what happens with it or without it :

With FileSetEnd, correct result if we write "a" when $i = 501, then "a" will be read
...
500   7.30425794339783
a   7.31545936704246
a   7.32574000326857
a   7.34107211950122

Without FileSetEnd, incorrect result if we write "a" when $i = 501 ("a" overwrites the "5", reading will return "a00")
...
500   5.33945850659791
a00   5.35076692708151
a00   5.3613953728756
a00   5.37214562185976

 

Edited March 21, 2021 by pixelsearch

[jugador]

1 hour ago, pixelsearch said:

OP certainly checked the file content after he ran his 2 examples, so Overwrite seems to be what he needs.
Maybe another software will continuously read the content of this "bridge" file to get real-time data and process it ?

Yes @pixelsearch  is correct.

It’s another software that will continuously read the content of the file in real-time.

So overwrite is the need otherwise file size will increase and make whole process slow plus if not overwrite then duplicate data problem and have to write more code to exclude those data.

I have tested both the method for almost 4hour not seen any difference.

1 hour ago, pixelsearch said:

I'd go with Method2 (which opens the file only once)

as mention by @pixelsearch  this is what I want to ask.

It is logical to open/close file handle every second as describe in method 1 or use method 2 which open file handle only once.
 

[JockoDundee]

36 minutes ago, jugador said:

It is logical to open/close file handle every second as describe in method 1 or use method 2 which open file handle only once.

It’s logical to do either.  Personally, when I’m continuously writing a semaphore file, I just overwrite it, it’s just simpler.

However, if you do seek back to the beginning of the file and rewrite, I do believe you will have to call fileflush, or risk getting stale data.

You may get away with it, depending on the data size and other OS dependent things, but you might not.

But fileflush is going to cost you a lot of the savings you get from not closing and reopening the file.

[seadoggie01]

If another process is reading/writing the file, don't you want to have it open for as little time as possible? Maybe I'm not understanding correctly 😐