Jump to content

How can I directly access / manipulate bytes in binary data?


AnyOne2
 Share

Recommended Posts

I extracted some binary data using the function BinaryMid.
Now I want to modify some of those byte before interpreting the whole binary sequence as UTF-16LE string.

How to do this?

Note: searching for UDFs I found one called Binary.UDF (by Ward), but I seems not to be available anymore; the function "_BinaryPoke" would have been very helpful I think...

Edited by AnyOne2
Link to comment
Share on other sites

OK, here's my conclusion after some tries with this Binary.au3: the manipulations of Binaries using DLLStructs are incredible slow.
The compiled AutoIt script needs about 100 seconds, using PHP the same algorithm finishes after just 2 seconds.

So why using AutoIt? Because I need to be able to compile the algorithm and distribute it as self-contained executable...

Any hints?

Link to comment
Share on other sites

9 hours ago, AnyOne2 said:

the manipulations of Binaries using DLLStructs are incredible slow.
The compiled AutoIt script needs about 100 seconds, using PHP the same algorithm finishes after just 2 seconds.

Manipulating of Binaries using DLLStructs is incredibly FAST (if you know what to do!)

Without a Script to verify your statement, any other discussion is useless...please show a little example script. 

Link to comment
Share on other sites

19 hours ago, AndyG said:

Manipulating of Binaries using DLLStructs is incredibly FAST (if you know what to do!)

You're right, this was probably a hasty conclusion, I'm sorry.

So here's a simplified version of my code to show you what I'm doing in my script.
I would really appreciate it if someone could tell me why it runs so slow.

; open input file
$hFileIn = FileOpen($sFilePath, $FO_READ + $FO_BINARY)
$dBinary = FileRead($hFileIn)
FileClose($hFileIn)
$iBinLen = BinaryLen($dBinary)

; open output file
$sFileOut = $sFilePath & '.txt'
$hFileOut = FileOpen($sFileOut, $FO_OVERWRITE)

$iOffset = 4

Do
  ; extracting length
  $iLenStr = Int(BinaryMid($dBinary, $iOffset+8, 1)) + (256 * Int(BinaryMid($dBinary, $iOffset+9, 1)))

  ; get raw text
  $dRawStr = BinaryMid($dBinary, $iOffset, $iLenStr)
  $iOffset += $iLenStr + 10

  ; adapt and convert text
  $iVal = Mod(iLenStr, 32) ; added/updated
  adapt($dRawStr, $iVal);
    $sText = BinaryToString($dRawStr, 2) ; 2=UTF16LE

  ; write output
  FileWriteLine($hFileOut, $sText)

    $iCnt += 1
    If $iCnt >= 500 Then
        ConsoleWrite('*')
        $iCnt = 0
  EndIf

Until $iOffset >= $iBinLen

FileClose($hFileOut)

;------- INTERNAL FUNCTION ------------------------

Func adapt(ByRef $dBinary, $iVal)

    ; create C struct from binary variable
    Local $dBuffer = DllStructCreate("byte[" & (BinaryLen($dBinary)) & "]")
    DllStructSetData($dBuffer, 1, $dBinary)

    ; loop through structure and add val to all bytes
    For $i=1 To BinaryLen($dBinary) Step 1
        $iNewValue = DllStructGetData($dBuffer, 1, $i) + $iVal
        DllStructSetData($dBuffer, 1, $iNewValue, $i);
    Next

    ; write result back to binary variable
    $dBinary = DllStructGetData($dBuffer, 1)

EndFunc

 

Edited by AnyOne2
Link to comment
Share on other sites

  • Moderators

AndyG,

Love the link!

M23

Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Link to comment
Share on other sites

You're right again, I forgot to include the following line (I now updated my initial post):

$iVal = Mod($iStrLen,32)

it shows that the value of $iVal changes for each loop iteration (between 0 and 31).

Another info: the input files have sizes between 5 and 10 MB, each loop iteration handles about 250 bytes (to average).
My PHP script takes 2260 ms for a file of about 6.5 MB (Win10, Intel i7-4900MQ 2.8 GHz, 16GB RAM, 500 GB SSD).

Thank you for bearing with me!

Edited by AnyOne2
Link to comment
Share on other sites

I solved my speed problems, the script now finishes within 6-8 seconds, that's acceptable for me.

What did I do?
Instead of reading the file as binary, I read it as ANSI string and used StringToASCIIArray to get an array of integers in the range of 0-255.
Like this I could do all my manipulations on the array and I made the byte-swap (to read UTF16LE) myself before calling StringFromASCIIArray.

The conversion to an array takes about 2 seconds, 4-6 seconds are needed to process the file (7MB).
I'm happy :)

Thanks for your replies anyway!

Edited by AnyOne2
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...