Jump to content
Sign in to follow this  
toasterking

Parsing binary data frames from an asynchronous source

Recommended Posts

I was just working on a project that involved decoding a stream of binary data from a serial port in AutoIt.  It took me a few hours to figure out how to process the data efficiently in AutoIt and I did not find any helpful examples on how to do so, so I thought I would share my core example and maybe save someone else some time.  There may be a more efficient way to do this, but this works well for me.

 

#cs
Author: ToasterKing
This is an example of a way to parse streaming binary data that follows a strict format with a header and footer.
In this example, each frame is 5 bytes with a 2-byte header of 0xD5AA and a 1-byte footer of 0xAD.
The _BinaryParse() function accumulates incoming data in a buffer.  Once a footer is found, it searches backward
for the header, and if it is in the right position, it extracts the remaining 2 bytes in the middle,
then moves on to looking for the next frame.
#ce

; The data source might be something asynchronous like serial or TCP, but since this is just an example, I'm just putting the data in a variable.
Local $fSomeData
$fSomeData = Binary("0xD5AA24B1") ; Binary data constituting almost a complete frame.
_BinaryParse($fSomeData) ; Call the function with the received data.  It isn't a complete frame, so it is just stored in the buffer until more data is received.
$fSomeData = Binary("0xAD62D5AA92E7AD") ; Remainder of the previous frame, one garbage byte (0x62) which should be skipped, and a complete additional frame.
_BinaryParse($fSomeData) ; The function should be able to parse both frames now.


Func _BinaryParse($fNewData)
    Local Static $fBinaryReceived = Binary("") ; Buffer for received data
    ConsoleWrite("Hey, the function is called!" & @CRLF)
    ; Add new data to the buffer.
    ; This ridiculous monstrosity is the only way I could find to append binary data to binary data in AutoIt.  It must be converted to strings first.
    ; Both, one, or no substrings will begin with "0x" depending on whether they contained binary data.  To be converted back to binary properly, only one instance
    ; of "0x" must exist at the beginning of the string.
    $fBinaryReceived = Binary("0x" & StringReplace(String($fBinaryReceived) & String($fNewData),"0x",""))
    ConsoleWrite("Data in the buffer: " & String($fBinaryReceived) & @CRLF)
    Local $iLength = BinaryLen($fBinaryReceived) ; Count the bytes in the data
    If $iLength > 0 Then
        Local $fBinaryReceivedTemp = $fBinaryReceived ; Create temporary copy to work on
        Local $fByte1,$fByte2
        For $i = 1 To $iLength
            If BinaryMid($fBinaryReceivedTemp,$i,1) = 0xAD Then ; If the 1-byte footer found
                ConsoleWrite("Footer found at end of " & $i & " of " & $iLength & " bytes!" & @CRLF)
                If BinaryMid($fBinaryReceivedTemp,$i - 4,1) = 0xD5 And BinaryMid($fBinaryReceivedTemp,$i - 3,1) = 0xAA Then ; and the 2-byte header is found 4 bytes before that
                ConsoleWrite("Header found before the footer!" & @CRLF)
                    $fByte1 = BinaryMid($fBinaryReceivedTemp,$i - 2,1) ; Get 1st byte in the body (between header and footer)
                    $fByte2 = BinaryMid($fBinaryReceivedTemp,$i - 1,1) ; Get 2nd byte in the body (between header and footer)
                    ConsoleWrite("Here is the critical data: " & String($fByte1) & " " & String($fByte2) & @CRLF) ; Just display the 2 bytes for demonstration purposes.  Normally, you'd do something more useful with it here.
                EndIf
                $fBinaryReceived = BinaryMid($fBinaryReceivedTemp,$i + 1) ; Truncate the original data to remove all of the bytes just processed, then continue processing $fBinaryReceivedTemp
            EndIf
        Next
    EndIf
EndFunc

 

Edited by toasterking

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By cruisepandey
      Hi, 
      I have a string like this : 
      Global $Msga = "urrent directory is /send.  (Submission of file with log number 29381077284 is confirmed)";
      I want to extract the number 29381077284  from the string. I did StringSplit to split based on "(" and then use space to reach there, But it's not a good choice. 
      Can anyone help me with regular expression to find the number from String using AutoIT. TIA
    • By DannyJ
      I have this example string:
       
      2020-06-08 09:23:33 : abcdefghifjklm How to trim just the right part? Or how to trim this text right to left?
      I want to trim this string right to left until ' : '.
      In ohter words: I need the 'abcdefghifklm' text.
      I have tried StringTrimRight, but that requires 
      The number of characters to trim. Unfortunately StringSplit trims the left part of the strings.
       
       
    • By careca
      This is my take on string triggers, triggers on specific strings.
      Able to simple text pasting,
      opening links (as long as there's a www. http:\\ or https:\\ at the beggining)
      and is able to open applications, if there is a parameter in the parameter field, it uses it.
      Shows your external, lan, and gateway ip's.
      Able to change system volume by a set percentage, reading from the inputbox the number the user sets, if 0 or empty uses system default.
      I made this because the existing string trigger applications didn't do it for me.
      I did this for me, but if someone finds it useful, all the better.
    • By TheXman
      Purpose (from Microsoft's website)
      The HTTP Server API enables applications to communicate over HTTP without using Microsoft Internet Information Server (IIS). Applications can register to receive HTTP requests for particular URLs, receive HTTP requests, and send HTTP responses. The HTTP Server API includes SSL support so that applications can exchange data over secure HTTP connections without IIS.
      Description
      There have been several times in the past that I wanted to either retrieve information from one of my PCs or execute commands on one of my PCs, whether it was from across the world or sitting on my couch.  Since AutoIt is one of my favorite tools for automating just about anything on my PC, I looked for ways to make to make it happen.  Setting up a full blown IIS server seemed like overkill so I looked for lighter weight solutions.  I though about creating my own AutoIt UDP or TCP server but that just wasn't robust enough,  Then I found Microsoft's HTTP Server API and it looked very promising.  After doing a little research into the APIs, I found that it was flexible & robust enough to handle just about any of the tasks that I required now and in the future.  So a while back I decided to wrap the API functionality that I needed into an AutoIt UDF file to allow me to easily create the functionality I needed at the time.  It has come in very handy over the years.  Of course it wasn't all wrapped up with a nice little bow like it is now.  That only happened when I decided to share it with anyone else who could use it or learn from it.
      The example file that I included is a very granular example of the steps required to get a lightweight HTTP Server up and listening for GET requests.  The UDF is a wrapper for the Microsoft APIs.  That means to do anything over and above what I show in the example, one would probably have to have at least a general knowledge of APIs or the ability to figure out which APIs/functions to use, what structures and data is needed to be passed to them, and in what order.  However, the UDF gives a very solid foundation on which to build upon.  Of course, if anyone has questions about the UDF or how to implement any particular functionality, I would probably help to the extent that I could or point you in the right direction so that you can figure out how to implement your own solution.
      The APIs included in the UDF are the ones that I needed in the past to do what I needed to do.  If any additional APIs need to be added to the UDF file, please make those suggestions in the related forum topic.
      Being that this is basically an AutoIt wrapper for the Microsoft API functions, there's no need to create AutoIt-specific documentation.  All of the UDF functions, structures, constants, and enumerations are named after their Microsoft API counterparts.  Therefore, you can refer to Microsoft's extensive documentation of their HTTP Server API.  As stated earlier, if there is one or more APIs that you find yourself needing for your particular solution, please suggest it in the related Example Scripts forum topic.
      Related Links
      Microsoft HTTP Server API - Start Page
      Microsoft HTTP Server API - API v2 Reference
      Microsoft HTTP Server API - Programming Model
    • By Colduction
      Hi guys!, i have a problem to convert Python code to AutoIt code, in fact i had not coded with Python yet!, this code is about permutation a string's case, i will be happy with your comments :)❤;

      Python code:
       
      # Python code to print all permutations # with respect to cases # Function to generate permutations def permute(inp): n = len(inp) # Number of permutations is 2^n mx = 1 << n # Converting string to lower case inp = inp.lower() # Using all subsequences and permuting them for i in range(mx): # If j-th bit is set, we convert it to upper case combination = [k for k in inp] for j in range(n): if (((i >> j) & 1) == 1): combination[j] = inp[j].upper() temp = "" # Printing current combination for i in combination: temp += i print(temp), # Driver code permute("Hello") # This code is contributed by Sachin Bisht
      My code in AutoIt:
      ; https://www.geeksforgeeks.org/permute-string-changing-case/ _PermuteCase("ABC") Func _PermuteCase($sText) If StringRegExp($sText, "^[A-Za-z]{1,}$") Then Local $iLength = StringLen($sText) ; Get length of the text. Local $iMaxPerm = 2 ^ $iLength ; Number of permutations is 2^n Local $sLow_Text = StringLower($sText) ; Converting string to lower case Local $asChrs = StringToASCIIArray($sLow_Text) ; Split the text into array of chars. For $i = 1 To $iMaxPerm Step 1 For $j = 0 To $asChrs[0] ;................................................... Next Next Else Return SetError(-1, 0, "Error: Input is incorrect!") EndIf EndFunc ;==>_PermuteCase  
       
       
       
       
       
      ====================== SOLUTION by @TheXman ======================
       
×
×
  • Create New...