Parsing Binary file.

[shadowfiend]

Hi AutoIT gurus I could do with some help and will try to keep this brief and concise.

I would like to gather data from a proprietary database (golf irrigation control system) which is in binary. Each record in the file is 59 characters long and each field is at a fixed position relative to the start of each record.

My first ever AutoIT script exported these records by taking the data directly from the control software. It works ok but is very slow and has other inherent problems.

The file in question is only 23KB in size and contains up to 396 (4 lots of 99) records.

How can I read each character into an array so that I can read them back based on the number of characters from the start of the file? (Is this efficient? Would it be better to store them as hex?)

I would like to do things like export a table to excel or CSV for a start and later manipulate the data.

As a secondary concern, how do I convert these into decimal?

I maintain a growing number of these databases and need to unify the data and build tools to answer questions like How many 720G sprinklers do we have... ect

Any help would be greatly appreciated.

Cheers.

[Siao]

Open file in binary mode, read whole of it into variable, then BinaryMid the bytes you need and do whatever.

[shadowfiend]

Open file in binary mode, read whole of it into variable, then BinaryMid the bytes you need and do whatever.

Ahh I see! This seems to be exactly what I've been looking for...

and I have been looking quite a bit

Thanks Siao, I will give this a try.

Shadowfiend.

[shadowfiend]

Ok so I tried this

$path = "J:\Database-test\Copy stations.sc3"
   
  ;Open file in binary mode
   $file = FileOpen($path, 16)
   
  ; Check if file opened for reading OK
   If $file = -1 Then
       MsgBox(0, "Error", "Unable to open file.")
       Exit
   EndIf
   
   $extract = BinaryMid($file, 1,8)
   MsgBox(0, "Chars read:", $extract)
   
   FileClose($file)

And I get a result of '0x01000000' from the $extract variable. This is supposed to be the first 8 bytes in hex if I have understood right, yes?

But if I change the first character in the file using XVI32 hex editor I get the same result. What is going on?

[Siao]

Um, obviously, after you open the file, you have to read it too...

[randallc]

Ok so I tried this

$path = "J:\Database-test\Copy stations.sc3"
   
 ;Open file in binary mode
   $file = FileOpen($path, 16)
   
 ; Check if file opened for reading OK
   If $file = -1 Then
       MsgBox(0, "Error", "Unable to open file.")
       Exit
   EndIf
   
   $extract = BinaryMid($file, 1,8)
   MsgBox(0, "Chars read:", $extract)
   
   FileClose($file)

And I get a result of '0x01000000' from the $extract variable. This is supposed to be the first 8 bytes in hex if I have understood right, yes?

But if I change the first character in the file using XVI32 hex editor I get the same result. What is going on?

Hi,

I can't remember the details; hex editor might only be changing the 3rd or 4th char in the file; maybe read the first 8 chars into array and see if any of them changed;;

1. What sort of binary file is it?

2. Waht is your purpose and required outcome?

Best, randall

[shadowfiend]

I get you Siao dude,

$path = "J:\Database-test\Copy stations.sc3"
    
  ;Open file in binary mode
    $file = FileOpen($path, 16)
    
  ; Check if file opened for reading OK
    If $file = -1 Then
        MsgBox(0, "Error", "Unable to open file.")
        Exit
    EndIf
    
    While 1
        $chars = FileRead($file)
        If @error = -1 Then ExitLoop
      ;MsgBox(0, "Char read $chars:", $chars)
    
        $extract = BinaryMid($chars, 1, 59)
    MsgBox(0, "Chars read $extract:", $extract)
    Wend
    FileClose($file)

This is working now. This is now reading 1 whole record into the $extract variable. I think I can do something with this now.

Thank you very much, it might have been obvious to you but was of great value to me

Sorted!

Edited February 1, 2008 by shadowfiend

[shadowfiend]

Randallc, sorry I did'nt see your post earlier,

1. What sort of binary file is it?

2. Waht is your purpose and required outcome?

Q1. As this is the first time I have ever dealt with binary files I'm not sure but I think that it is 8 bit per character (if that makes sense). If I open it with notepad I can read the text fields e.g. "Front Left" or "Putting Green". I am working out what the fields do by changing one parameter at a time and comparing the old and new file in notepad and then in XVI32 hex editor to see what the values are. Numbers seem to be mostly stored like this:- "12.5" will be stored as "7D" hex witch is 125 in decimal. I don't think any number gets bigger than 255 in this part of the database, so each number can be stored in one byte.

Q2. The primary goal of my script is first of all to get the data out of the binary file and into excel in a readable form. As I get time I will add functions like import from Excel or process all the data within the script. Each golf club has a computer controlled sprinkler system where each sprinkler is one record storing information like flow and spray radius. At the moment each database must be read in via floppy disk (no I'm not joking) in order to be loaded and backed up. It's a real headache managing the data with 40 or so clubs changing all the time. I want to build tools to manage this major problem. Be able to modify the databases without needing to use the irrigation control software.

Sorry I got a bit carried away there.

Cheers.

Shadowfiend.