AnyOne2

How can I directly access / manipulate bytes in binary data?

10 posts in this topic

#1 ·  Posted (edited)

I extracted some binary data using the function BinaryMid.
Now I want to modify some of those byte before interpreting the whole binary sequence as UTF-16LE string.

How to do this?

Note: searching for UDFs I found one called Binary.UDF (by Ward), but I seems not to be available anymore; the function "_BinaryPoke" would have been very helpful I think...

Edited by AnyOne2

Share this post


Link to post
Share on other sites



Thanks for the quick response!
I'll have a look if this really fits my needs

Share this post


Link to post
Share on other sites

OK, here's my conclusion after some tries with this Binary.au3: the manipulations of Binaries using DLLStructs are incredible slow.
The compiled AutoIt script needs about 100 seconds, using PHP the same algorithm finishes after just 2 seconds.

So why using AutoIt? Because I need to be able to compile the algorithm and distribute it as self-contained executable...

Any hints?

Share this post


Link to post
Share on other sites
9 hours ago, AnyOne2 said:

the manipulations of Binaries using DLLStructs are incredible slow.
The compiled AutoIt script needs about 100 seconds, using PHP the same algorithm finishes after just 2 seconds.

Manipulating of Binaries using DLLStructs is incredibly FAST (if you know what to do!)

Without a Script to verify your statement, any other discussion is useless...please show a little example script. 

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

19 hours ago, AndyG said:

Manipulating of Binaries using DLLStructs is incredibly FAST (if you know what to do!)

You're right, this was probably a hasty conclusion, I'm sorry.

So here's a simplified version of my code to show you what I'm doing in my script.
I would really appreciate it if someone could tell me why it runs so slow.

; open input file
$hFileIn = FileOpen($sFilePath, $FO_READ + $FO_BINARY)
$dBinary = FileRead($hFileIn)
FileClose($hFileIn)
$iBinLen = BinaryLen($dBinary)

; open output file
$sFileOut = $sFilePath & '.txt'
$hFileOut = FileOpen($sFileOut, $FO_OVERWRITE)

$iOffset = 4

Do
  ; extracting length
  $iLenStr = Int(BinaryMid($dBinary, $iOffset+8, 1)) + (256 * Int(BinaryMid($dBinary, $iOffset+9, 1)))

  ; get raw text
  $dRawStr = BinaryMid($dBinary, $iOffset, $iLenStr)
  $iOffset += $iLenStr + 10

  ; adapt and convert text
  $iVal = Mod(iLenStr, 32) ; added/updated
  adapt($dRawStr, $iVal);
    $sText = BinaryToString($dRawStr, 2) ; 2=UTF16LE

  ; write output
  FileWriteLine($hFileOut, $sText)

    $iCnt += 1
    If $iCnt >= 500 Then
        ConsoleWrite('*')
        $iCnt = 0
  EndIf

Until $iOffset >= $iBinLen

FileClose($hFileOut)

;------- INTERNAL FUNCTION ------------------------

Func adapt(ByRef $dBinary, $iVal)

    ; create C struct from binary variable
    Local $dBuffer = DllStructCreate("byte[" & (BinaryLen($dBinary)) & "]")
    DllStructSetData($dBuffer, 1, $dBinary)

    ; loop through structure and add val to all bytes
    For $i=1 To BinaryLen($dBinary) Step 1
        $iNewValue = DllStructGetData($dBuffer, 1, $i) + $iVal
        DllStructSetData($dBuffer, 1, $iNewValue, $i);
    Next

    ; write result back to binary variable
    $dBinary = DllStructGetData($dBuffer, 1)

EndFunc

 

Edited by AnyOne2

Share this post


Link to post
Share on other sites

Hi,

do you have an example file to convert and a value for $iVal?

I think you have been caught by the trap called XY-Problem:)

The goal is to add a (always the same?) value to some bytes within a file?!

The job should be done within some milliseconds...

Share this post


Link to post
Share on other sites

AndyG,

Love the link!

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

#9 ·  Posted (edited)

You're right again, I forgot to include the following line (I now updated my initial post):

$iVal = Mod($iStrLen,32)

it shows that the value of $iVal changes for each loop iteration (between 0 and 31).

Another info: the input files have sizes between 5 and 10 MB, each loop iteration handles about 250 bytes (to average).
My PHP script takes 2260 ms for a file of about 6.5 MB (Win10, Intel i7-4900MQ 2.8 GHz, 16GB RAM, 500 GB SSD).

Thank you for bearing with me!

Edited by AnyOne2

Share this post


Link to post
Share on other sites

#10 ·  Posted (edited)

I solved my speed problems, the script now finishes within 6-8 seconds, that's acceptable for me.

What did I do?
Instead of reading the file as binary, I read it as ANSI string and used StringToASCIIArray to get an array of integers in the range of 0-255.
Like this I could do all my manipulations on the array and I made the byte-swap (to read UTF16LE) myself before calling StringFromASCIIArray.

The conversion to an array takes about 2 seconds, 4-6 seconds are needed to process the file (7MB).
I'm happy :)

Thanks for your replies anyway!

Edited by AnyOne2

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now