13 replies to this topic

[Ward]

"Binary" is a built-in variable type in AutoIt. But functions that deal with binary variable are inadequate. Someone says string functions can also deal with binary variable, but actually, it may cause some problem. For example, using StringInStr to search "0x11" in "0x0110" will return a false result.

This UDF solves the problem. It provide a lot of binary related functions. Furthermore, most functions in the UDF are optimized by machine code, so the speed is incredible.

Here is the function list:

_BinaryAnd($Binary1, $Binary2)
_BinaryOR($Binary1, $Binary2)
_BinaryXOR($Binary1, $Binary2)
_BinaryNot($Binary)
_BinaryShift($Binary, $Shift)
_BinaryRotate($Binary, $Shift)
_BinaryReverse($Binary)
_BinaryInBin($Binary, $Search, $Occur = 1, $Start = 1)
_BinaryReplace($Binary, $Search, $Replace, $Occur = 0)
_BinarySplit($Binary, $Search)
_BinaryRight($Binary, $Count)
_BinaryLeft($Binary, $Count)
_BinaryTrimLeft($Binary, $Count)
_BinaryTrimRight($Binary, $Count)

The usage of these functions are just like the built-in StringXXXX or BitXXXX functions.

Attentions:
BinarySplit has a little bit different, it don't support flag parameter in StringSplit.
BinaryInBin and BinaryReplace also don't need casesense parameter.

_BinaryPeek($Binary, $Start, $Type = "byte")
_BinaryPoke($Binary, $Start, $Value, $Type = "byte")
_BinaryToInt32($Binary)
_BinaryFromInt32($Value)
_BinaryToInt64($Binary)
_BinaryFromInt64($Value)
_BinaryRandom($Length, $Start = 0, $To = 255)

These functions are some helper function.

BinaryPeek and BinaryPoke work like stardand BASIC keyword peek and poke. They can handle types defined in DllStructCreate and DllCall (include str and wstr). The $Start are also like other AutoIt functions, count from 1, not from 0.

BinaryRandom generate a random binary data for test or something. It has built-in Mersenne Twister PRNG.

BinaryToXXX and BinaryFromXXX convert variable type between binary and int32/int64.

_BitShift64($Value, $Shift)
_BitRotate64($Value, $Shift)
_BitNOT64($Value)
_BitOR64($Value1, $Value2, [$Value3, ...])
_BitAND64($Value1, $Value2, [$Value3, ...])
_BitXOR64($Value1, $Value2, [$Value3, ...])
_Hex64($Value, $Length = 16)
_Dec64($HexString)

These functions are just the same as built-in functin in the same name, but these can handle 64 bit signed integer.
(Built-in function only for 32 bit integer)

 Binary UDF.zip   6.86K   253 downloads


2011/09/15 Update Note:

• Add following functions:
  

  _BinaryToInt16($Binary)
  _BinaryFromInt16($Value)
  _BinaryToDouble($Binary)
  _BinaryFromDouble($Value)
  _BinaryToMemory($Binary, $Ptr)
  _BinaryFromMemory($Ptr, $Size)
  _BinaryToDLLStruct($Binary)
  _BinaryFromDLLStruct(ByRef $DLLStruct)
  _BinaryFillDLLStruct($Binary, ByRef $DLLStruct)

• _BinaryRandom can specify the random seed now. $Seed = 0 mean no seed is specified.
  

  _BinaryRandom($Length, $Start = 0, $To = 255, $Seed = 0)

• Use static variable to store the binary code to avoid repeat memory allocation.

 Binary UDF.zip   7.19K   801 downloads

Edited by Ward, 15 September 2011 - 06:35 AM.

[wraithdu]

Excellent! The 64-bit binary functions will really be useful.

[ynbIpb]

Hi Ward!
Thank you for this UDF
I have a problem: I need to replace a few bytes of binary data. But replace the data is longer than the original data. In this situation it would fit function _BinaryReplace, but it changes, not where I want. I know the exact start Offset from which data and their length.

example:
Input data: 00 00 11 00 11 00 11 00 00 00 ; I need to replace the red
Output data: 00 00 11 00 FF FF 00 11 00 00 00 ; like this
or
Input data: 00 00 11 00 11 00 11 00 00 00
Output data: 00 00 11 00 FF 11 00 00 00

I do not know exactly what it is first or last, but I know the offset and size

It is possible to refine the function _BinaryReplace, so she was not looking for a string and tell it the exact offset

sorry for bad english

[Ward]

Hi Ward!
Thank you for this UDF
I have a problem: I need to replace a few bytes of binary data. But replace the data is longer than the original data. In this situation it would fit function _BinaryReplace, but it changes, not where I want. I know the exact start Offset from which data and their length.

Since you know the offset to start replace, maybe you can use BinaryMid() to get part of binary data before _BinaryReplace().

[MvGulik]

Interesting ... Bummer I don't have 64bit capabilities yet.

---

Collection some stuff while playing around with it ...

Spoiler

• Could use some '#forceref's (5x) to make it not trip on main scripts that use '#AutoIt3Wrapper_AU3Check_Parameters=-q -d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6 -w 7'
• Some way to control random with some seed feature. (if possible)
• _BinaryRandom() ... speed is, well

  [1] |St| A | 1.74 mSec.   | 256 * Random()
  [2] |St| B | 145.95 mSec. | 65536 * Random()
  [3] |St| A | 105.52 mSec. | 256 * _BinaryRandom(8)
  [4] |St| B | 26.31 Sec.   | 65536 * _BinaryRandom(8)

  Probably because function startup penalty. ... (taking a other approche to using it) ...

  [4] |St| B | 31.68 mSec.  | _BinaryRandom(8*65536)

  That's more like it. But Random() not leaving much slack to play around with.
• ...

Edited by iEvKI3gv9Wrkd41u, 28 August 2011 - 11:44 PM.

[Ward]

@iEvKI3gv9Wrkd41u
Thanks for your testing.

_BinaryRandom (or other functions use machine code) convert HEX to binary and call it every time.
A way to avoid it is use static variable, for example:

AutoIt        expandcollapse  popup

Func _BinaryRandom2($Length, $Start = 0, $To = 255)
    If $Length = 0 Then Return Binary("")
    Static $CodeBufferPtr

    If Not $CodeBufferPtr Then
        Local $Code
        If @AutoItX64 Then
            $Code = Binary("0x4157415641554154555756534881ECF80900004538C876034587C185D20F847F0100004C8D5C2410450FB6C9410FB6C04183C10183EA0148C78424D0090000000000004129C1498D8338060000488D5C1101498D730431D2498DBB3406000048894424084D8D93C0090000EB4789C2C1EA0B31C289D0C1E0072580562C9D31D031D289C5C1E50F81E50000C6EF31C589E8C1E81231E841F7F1418D141088114883C1014839D90F84F6000000488B9424D00900004885D20F84850000004C8BBC24E00900004C8BA424D80900004D8D6F044C89E0448B308B2A4C89A424D00900004983C4044C89AC24E00900004C89A424D80900004489F081E5000000804183E60125FFFFFF7F41F7DE09E84181E6DFB00899D1E84431F04133074D39D5890274164D39D40F8542FFFFFF4C899C24D8090000E935FFFFFF4C899C24E0090000EBE00F3131D04989F4BD010000008944241089C2C1EA1E31C269C26589076C01E883C501418904244983C40481FD7002000075DE4C899C24D00900004889B424D80900004889F04889BC24E00900004C8B6C24084C89DA4989FF4989F4E932FFFFFF4881C4F80900005B5E5F5D415C415D415E415FC3")
        Else
            $Code = Binary("0x5557565381ECE40900000FB68424000A0000884424130FB68424040A000038442413760A0FB654241392885424138B9424FC09000085D20F84740100000FB65424130FB6C031DB83C0018DB424D8090000C78424D8090000000000008D4C241829D031D28944240C893424EB5289C2C1EA0B31C289D0C1E0072580562C9D31D031D289C6C1E60F81E60000C6EF31C689F0C1E81231F0F774240C8B8424F80900000254241388141883C301399C24FC0900000F86F90000008B9424D809000085D20F84850000008BAC24E00900008BBC24DC0900008D750489F8897424048B008B3289BC24D809000083C70489BC24DC0900008944240825FFFFFF7F81E60000008009F08B742408D1E883E601F7DE81E6DFB0089931F08B342433450089028B54240439F2899424E009000074153B3C240F8536FFFFFF898C24DC090000E92AFFFFFF898C24E0090000EBE20F3131D0BE010000008944241889C2C1EA1E31C269C26589076C01F08904B183C60181FE7002000075E38D44241C8D94244C06000089C78DB42450060000899424E00900008DAC244C06000089CA898C24D8090000898424DC09000089742404E92DFFFFFF81C4E40900005B5E5F5DC21000")
        EndIf
        $CodeBufferPtr = __BinaryCodeBufferAlloc($Code)
    EndIf

    Local $Buffer = DllStructCreate("byte[" & $Length & "]")
    Local $Ret = DllCall($__Binary_User32Dll, "uint", "CallWindowProc", "ptr", $CodeBufferPtr, _
                                                                        "ptr", DllStructGetPtr($Buffer), _
                                                                        "uint", $Length, _
                                                                        "byte", $Start, _
                                                                        "byte", $To)
    Return DllStructGetData($Buffer, 1)
EndFunc

This function is faster than the old one.
But the memory space can't be free (it seems not important).
Is convert all functions to this format necessary?

[wraithdu]

I think the Static method is good practice to avoid repeated memory allocations. You could always build in a memory clearing feature to the function if it is important to free that buffer at some point.

[MvGulik]

This function is faster than the old one.

Definitely puts a nice dent in the time needed.

[5] |St| B    | 7.51 Sec.    | 65536 * _BinaryRandom_Static(8)

But the memory space can't be free (it seems not important).

Yep. Speed versus Memory use. Don't think where talking that much memory here to. (The Asm code only by the look of it.)

Is convert all functions to this format necessary?

I personally would use Static ... Without it the general function call(s) seem way to slow for any serious(high call rate) use. Even with Static use the function startup penalty's are not to be ignored. (for those functions that are available in AutoIt itself -> 32-bit one's)

e: forum troubles ...

---

Alternative (older) way/topic for some 64-bit support:Mod() and Bitwise operations on 64-bit integers, by jchd.

Edited by iEvKI3gv9Wrkd41u, 30 August 2011 - 11:20 AM.

[Ward]

@iEvKI3gv9Wrkd41u
What you suggested is added now.

[czardas]

Interesting thread. I occasionally use bitwise functions, but I mostly use string functions with binary strings. Why? Perhaps, as you say,. because the available bitwise functions seem inflexible and extremely limited. The motivation for creating my modal UDF was aimed at modular arithmatic, binary being the first target base. I don't know all the methods and I still believe that there must be other solutions (perhaps using Struct which I haven't tried yet). My recent three dimensional bit spin topic uses strings of ones and zeros, because the bitwise functions don't hack it.

[MvGulik]

@iEvKI3gv9Wrkd41u
What you suggested is added now.

Cool.

[MvGulik]

Just to see if _BinaryRandom() is compatible in some way with AutoIt's Random().

AutoIt        expandcollapse  popup

#include "Binary.au3"
Global Const $INT32u = Int('0x0100000000') ;; (2^32), int64. (4294967296)

_Test5(8, 2, 1000) ;; Count, Seed, LoopIt.

Func _Test5(Const $iCount = 1, Const $iSeed=0, Const $iLoopIt = 1) ;; $iSeed=0 -> random seed.
    Local Const $iBytes = 1, $iMax = Int(2^(8*$iBytes))-1 ;0, 255, 65535, ...
    Local $R = ''

    For $i = 1 to $iLoopIt

    ;; AutoIt random() output.
    SRandom($iSeed)
    Local $bFirstBytes = Binary('')
    For $1 = 1 To $iCount
        $R = Floor(Random() * ($INT32u + 1))
        $bFirstBytes &= BinaryMid($R,1,1)
    Next

    ;; _BinaryRandom() output.
    $R =  _BinaryRandom($iCount, 0, $iMax, $iSeed)

    If $R <> $bFirstBytes Then
        MsgBox(0,'MsgBox','Oops: $R <> $bFirtsBytes ?')
        Exit 911
    EndIf

    Next
    MsgBox(0,'MsgBox',' :) ')
EndFunc

Although I do wonder a bit what happened to the other bytes.

E: Oops, forgot some things.
+ _BinaryRandom() probably using one Byte where Random it using 4 bytes.

Edited by iEvKI3gv9Wrkd41u, 16 September 2011 - 12:08 PM.

[wraithdu]

Just thought I'd toss out there that the UDF works great with the new DllCallAddress() function in 3.3.7.15 beta (ignore the Au3Check warnings about wrong number of parameters, that will be fixed in the next beta). Anyone feel like running a speed test versus CallWindowProc? This would also allow for an eventual rewrite with more sane parameters.

Edited by wraithdu, 16 September 2011 - 03:52 PM.

[Ward]

Finally we have standard support for DllCallAddress. Not need my MemoryFuncCall anymore. . I can develop my machine code library more easily.
I thank DllCallAddress should much faster than MemoryFuncCall, but have no much difference to CAllWindowProc. (just guess, not yes test.)