Sign in to follow this  
Followers 0
GaryC

Write binary data to a file?

22 posts in this topic

I want to store data in a file to allow the program to recover if it crashes. I want this to happen fast so that it doesn't interrupt the user. I have been able to write integers to a binary file one at a time but haven't found a way to write an array at one time-- actually I think I read that you can't. I looked at the binary type but can't see a way to concatenate data to it to hold more than one number. I tried a DllStruct but it wouldn't write. The only other way I can think of is to "manyually" convert the bytes to a string. Otherwise I'll have to write the stuff in ASCII numbers as in a print-out, which seems like a lot of wasted conversion just so that it can read it back in and convert it back to binary. My data is primarily a list of up to about 3000 2-byte integers indicating selected items.

Any ideas?

Thanks.

Gary

Share this post


Link to post
Share on other sites



Look at this my example

Instead of "byte" you can use "int","float",... structure ;-)

Thanks for your prompt response.

In some places the docs seem to indicate that you can assign a whole array at a time, in others they seem to say you can assign only a number or a string (or maybe a binary type). Can I assign a whole array at once, or do I have to set each element of the struct?

This program tries to assign an array to a struct array of ushorts and then read it back. The data returned by DllStructGetData is a number rather than an array, and the contents of the struct is all 0.

#include <array.au3>

Const $gciCheckinsSize = 3000
Const $gcsLogFile = @ScriptDir & "\tststruct.log"
Global $gaCheckins[$gciCheckinsSize]

Global $gsLog = "" ; holds output messages.

Main()

Func Main()
    MakeData()
    ;_ArrayDisplay($gaCheckins, "test data")

    Local $tagBuf = "USHORT[" & UBound($gaCheckins) & "]"
    MsgOut(0, "Info", "Creating struct " & $tagBuf)
    Local $stBuf = DllStructCreate($tagBuf)
    If @error Then
        MsgOut(0, "Info", "After DllCreateStruct($tagBuf) @error was " & @error)
    EndIf
    DllStructSetData($stBuf, 1, $gaCheckins)
    If @error Then
        MsgOut(0, "Info", "After DllStructSetData($stBuf) @error was " & @error)
    EndIf
    Local $sMsg = "First elements of $stBuf: "
    For $i = 1 To 5
        $sMsg &= DllStructGetData($stBuf, 1, $i) & " "
        If @error Then
            $sMsg &= @CRLF & "After GetData for element " & $i & " @error = " & @error & @CRLF
        EndIf
    Next ; $i
    $sMsg &= @CRLF
    MsgOut(0, "Info", $sMsg)
    Local $gaCheckins2 = DllStructGetData($stBuf, 1)
    If @error Then
        MsgOut(0, "Info", "After DllStructGetData($stBuf) @error was " & @error)
    EndIf
    If Not IsArray($gaCheckins2) Then
        MsgOut(0, "Error", "DllStructGetData for $gaCheckins2 did not return array, type is " & VarGetType($gaCheckins2) & ", value = " & $gaCheckins2)
    EndIf
    ;_ArrayDisplay($gaCheckins2, "$gaCheckins2")
    CheckArrays($gaCheckins, $gaCheckins2)
    If @error Then
        MsgOut(0, "Error", "CheckArrays returned @error = " & @error)
    EndIf

    ; Write log.
    FileDelete($gcsLogFile)
    FileWrite($gcsLogFile, $gsLog)
    MsgBox(0, "Done", "Log written to " & $gcsLogFile)
EndFunc   ;==>Main

Func MakeData()
    Local $iIdx = 1
    For $i = 0 To UBound($gaCheckins) - 1
        $gaCheckins[$i] = ($iIdx + 1) * 256 + $iIdx
        If $iIdx >= 253 Then
            $iIdx = 1
        Else
            $iIdx += 2
        EndIf
    Next ; $i
EndFunc   ;==>MakeData

; Compares 2 arrays.  If they match returns 0, if different sizes sets @error=2, if contents differ sets @error=1 and @extended=first differing element.
; If either is not an array sets @error = 3.
Func CheckArrays(Const ByRef $a1, Const ByRef $a2)
    If Not IsArray($a1) Or Not IsArray($a2) Then Return SetError(3, 0, 0)
    If UBound($a1) <> UBound($a2) Then Return SetError(2, 0, 0)
    Local $fDiff = False
    For $i = 0 To UBound($a1) - 1
        If $a1[$i] <> $a2[$i] Then
            $fDiff = True
            ExitLoop
        EndIf
    Next ; $i
    If $fDiff Then
        Return SetError(1, $i)
    EndIf
    Return 0
EndFunc   ;==>CheckArrays

Func MsgOut($iFlag, $sTitle, $sMsg)
    $gsLog &= $sMsg & @CRLF
EndFunc   ;==>MsgOut

tststruct.log contents:

Creating struct USHORT[3000]

First elements of $stBuf: 0 0 0 0 0

DllStructGetData for $gaCheckins2 did not return array, type is Int32, value = 0

CheckArrays returned @error = 3

Thanks.

Gary

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

Gary,

Try the following for writing bin values (I'm sure that they are string representations of bin values).

I'm sure that this can be made to run faster if it satisfies your requirements.

;
; The following write 300 8 byte bin values to a file
;

#include <array.au3>
#include <file.au3>

global $bNums[300]

for $i = 0 to 299
    $bNums[$i] = binary(random(1,10000,1))
Next

_arraydisplay($bnums)

local $bFl = fileopen("c:\bin_test",17)
if $bFl = -1 then msgbox(0,'','Unable to open file')

; each element followed by a crlf
;_FileWriteFromArray($bFl,$bNums)

; comma delimited string
FileWrite($bFl,_ArrayToString($bNums,','))


fileclose($bFl)

kylomas

Edited by kylomas

Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

Hello Structure Experts (especially you, Zedna),

I have been trying to get my arms around dll structures. I'm not a "C" coder, however, my language of choice is 386 assembler and in many ways these structures are like "control blocks".

I cannot figure out why Gary's code does not work. I've tried changing the structure element type to int64 thinking that ushort may not be big enough. I tries changing the way the array is generated by limiting array element values to random integers between 1 and 256. None of this worked.

I compared Zedna's example (in the thread) to what Gary is doing and the only difference I see is that Gary's array contains elements that are more than 1 byte long.

Lastly,

IS the winapi for file reading/writing being used because non string data is being written?

Thanks,

kylomas

P.S. Gary please excuse me if it seems like I am hijacking this thread but it seemed like the place to ask!


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

Hello Structure Experts (especially you, Zedna),

:-)

IS the winapi for file reading/writing being used because non string data is being written?

My example was written especially as example for

_WinAPI_SetFilePointer()

because in that time Autoit's native file functions didn't support it.

Latest version can do SetFilePointer already.

But with this API and structures you can also easily read/write all data types directly to files.

Share this post


Link to post
Share on other sites

Structure Experts,

I tried to simplify the code as follows. I think I am missing the point entirely cause it looks like this should work but does'nt.

;
;

#include <array.au3>

local $struct,$a10[3000],$i,$s1_var

for $i = 1 to 3000 - 1
    $a10[$i] = hex(random(1,32767,1))
Next

;_arraydisplay($a10)

$struct = dllstructcreate("int64 s1_var[50000]")
if @error then msgbox(0,'','struct create error = ' & @error)

dllstructsetdata($struct,$s1_var,$a10)                          ; <----------    this returns 2, element out of range
if @error then msgbox(0,'','struct set data error = ' & @error) 

for $i = 1 to 100
    consolewrite(dllstructgetdata($struct,$s1_var,$i) & ' ')
next

consolewrite(@crlf)

Thanks,

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

Yes,

I finally figured it out. This works. Gary this may or may not be applicable to your needs. Chalk it up to "too stuborn to let it go without understanding it".

;~ ;
;~ ;

#include <array.au3>

local $struct,$i

$struct = dllstructcreate("int[4000]")
if @error then msgbox(0,'','struct create error = ' & @error)

for $i = 1 to 1000
    dllstructsetdata($struct,1,binary(random(1,32767,1)),$i)
    if @error then msgbox(0,'','struct set data error = ' & @error)
Next

for $i = 1 to 1000
    consolewrite(binary(dllstructgetdata($struct,1,$i)) & ' ')
next

consolewrite(@crlf)

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

;
...
local $struct,$a10[3000],$i,$s1_var
...
$struct = dllstructcreate("int64 s1_var[50000]")
...
dllstructsetdata($struct,$s1_var,$a10)                          ; <----------    this returns 2, element out of range
...

It gets an error because $s1_var is not set. You would need either

$s1_var = "s1_var"

dllstructsetdata($struct,$s1_var,$a10)

or

dllstructsetdata($struct,"s1_var",$a10)

What I am trying to do is save some data to implement a recovery file. I want to save it as fast as possible so as to be as transparent as possible. If I need to recover I basically want to restore the array as it was before the crash, so I don't want to have to convert to binary or hex. I am using ushort because I know my numbers will fit in two bytes and don't want to waste disk space if I don't have to. That's why I am using the WinAPI functions. The documentation says if you set a struct element[...] to an array it takes as much of the array as there is room for, but it also talks about only being able to store strings or binary types, otherwise it has to be numbers. I guess what it means is that you can set any kind of array you want as long as it's a string or binary type. I have it working using the for loop to load the struct and I guess that's what I'll have to use. I was just hoping not to have to copy the data in a loop. Oh well, I guess most of the time there won't be much data.

About hijacking the thread: that was exactly what I wanted to know, so it's fine with me :-).

Thanks.

Gary

Share this post


Link to post
Share on other sites

Gary,

The one that I finally got working did NOT use the s1_var variable...my confusion is that the doc seems to indicate that you can load an array in dllstructsetdata but I could not make it work til I loaded it in a loop.

Anyway, good luck,

kylomas


Forum Rules         Procedure for posting code

"I like pigs.  Dogs look up to us.  Cats look down on us.  Pigs treat us as equals."

- Sir Winston Churchill

Share this post


Link to post
Share on other sites

Hi folks,

i got one more question. I try to write a byte value in an array. How to get this going?

$structByteStructure = DllStructCreate("byte[1024]")

For $iCounter = 1 To 1024

$iResult = DllStructSetData($structByteStructure , 1, random(1,15,1), $iCounter)

ConsoleWrite(DllStructGetData($sStructure, 1,$iCounter) & @CRLF)

Next

for example won't work. I know that the random value is a int but how to create a byte value between 0 and F ?

Has someone a hint for me?

Share this post


Link to post
Share on other sites

What won't work?


♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites

#14 ·  Posted (edited)

Hi trancexx,

the variable $iResult gets 0 and the dllstructure stays at 0 also.

I want to write 1024 randomly choosen bytes into the structure. Do you think i'am missing something?

Mmmhh. Can be. Sometimes i won't see the forest because of to many trees :)

PS: Now i'am trying it with an int array.

Edited by Sundance

Share this post


Link to post
Share on other sites

try this:

$structByteStructure = DllStructCreate("byte[1024]")

For $iCounter = 1 To 1024
    $iResult = DllStructSetData($structByteStructure, 1, Random(1, 15, 1), $iCounter)
Next
ConsoleWrite(DllStructGetData($structByteStructure, 1) & @CRLF)

Visit the SciTE4AutoIt3 Download page for the latest versions  - Beta files                                How to post scriptsource        Forum Rules
 
Live for the present,
Dream of the future,
Learn from the past.
  :)

Share this post


Link to post
Share on other sites

Ups. Sorry. It's a copy paste error.

I did a function with 'byref' the structure. In the function the structure is called $sStructure and in the main its called $structByteStructure . Sorry. But it won't run anyway. I'll think about it...

try this:

$structByteStructure = DllStructCreate("byte[1024]")

For $iCounter = 1 To 1024
    $iResult = DllStructSetData($structByteStructure, 1, Random(1, 15, 1), $iCounter)
Next
ConsoleWrite(DllStructGetData($structByteStructure, 1) & @CRLF)

Share this post


Link to post
Share on other sites

Maybe something is broken with my machine but that works just fine here.


♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites

Maybe something is broken with my machine but that works just fine here.

Your machine is as fine and mine when it comes to this script so guess the OP has an issue somewhere.... :)

Visit the SciTE4AutoIt3 Download page for the latest versions  - Beta files                                How to post scriptsource        Forum Rules
 
Live for the present,
Dream of the future,
Learn from the past.
  :)

Share this post


Link to post
Share on other sites

Your machine is as fine and mine when it comes to this script so guess the OP has an issue somewhere.... :)

As else maybe.

Btw, you never know, I've been watching some porn yesterday. ;)


♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0