Parsing binary data frames from an asynchronous source
I was just working on a project that involved decoding a stream of binary data from a serial port in AutoIt. It took me a few hours to figure out how to process the data efficiently in AutoIt and I did not find any helpful examples on how to do so, so I thought I would share my core example and maybe save someone else some time. There may be a more efficient way to do this, but this works well for me.
This is an example of a way to parse streaming binary data that follows a strict format with a header and footer.
In this example, each frame is 5 bytes with a 2-byte header of 0xD5AA and a 1-byte footer of 0xAD.
The _BinaryParse() function accumulates incoming data in a buffer. Once a footer is found, it searches backward
for the header, and if it is in the right position, it extracts the remaining 2 bytes in the middle,
then moves on to looking for the next frame.
; The data source might be something asynchronous like serial or TCP, but since this is just an example, I'm just putting the data in a variable.
$fSomeData = Binary("0xD5AA24B1") ; Binary data constituting almost a complete frame.
_BinaryParse($fSomeData) ; Call the function with the received data. It isn't a complete frame, so it is just stored in the buffer until more data is received.
$fSomeData = Binary("0xAD62D5AA92E7AD") ; Remainder of the previous frame, one garbage byte (0x62) which should be skipped, and a complete additional frame.
_BinaryParse($fSomeData) ; The function should be able to parse both frames now.
Local Static $fBinaryReceived = Binary("") ; Buffer for received data
ConsoleWrite("Hey, the function is called!" & @CRLF)
; Add new data to the buffer.
; This ridiculous monstrosity is the only way I could find to append binary data to binary data in AutoIt. It must be converted to strings first.
; Both, one, or no substrings will begin with "0x" depending on whether they contained binary data. To be converted back to binary properly, only one instance
; of "0x" must exist at the beginning of the string.
$fBinaryReceived = Binary("0x" & StringReplace(String($fBinaryReceived) & String($fNewData),"0x",""))
ConsoleWrite("Data in the buffer: " & String($fBinaryReceived) & @CRLF)
Local $iLength = BinaryLen($fBinaryReceived) ; Count the bytes in the data
If $iLength > 0 Then
Local $fBinaryReceivedTemp = $fBinaryReceived ; Create temporary copy to work on
For $i = 1 To $iLength
If BinaryMid($fBinaryReceivedTemp,$i,1) = 0xAD Then ; If the 1-byte footer found
ConsoleWrite("Footer found at end of " & $i & " of " & $iLength & " bytes!" & @CRLF)
If BinaryMid($fBinaryReceivedTemp,$i - 4,1) = 0xD5 And BinaryMid($fBinaryReceivedTemp,$i - 3,1) = 0xAA Then ; and the 2-byte header is found 4 bytes before that
ConsoleWrite("Header found before the footer!" & @CRLF)
$fByte1 = BinaryMid($fBinaryReceivedTemp,$i - 2,1) ; Get 1st byte in the body (between header and footer)
$fByte2 = BinaryMid($fBinaryReceivedTemp,$i - 1,1) ; Get 2nd byte in the body (between header and footer)
ConsoleWrite("Here is the critical data: " & String($fByte1) & " " & String($fByte2) & @CRLF) ; Just display the 2 bytes for demonstration purposes. Normally, you'd do something more useful with it here.
$fBinaryReceived = BinaryMid($fBinaryReceivedTemp,$i + 1) ; Truncate the original data to remove all of the bytes just processed, then continue processing $fBinaryReceivedTemp