CSL

How to get arbitrary section of bits from a binary

4 posts in this topic

I want to take a binary data from any source (string,number,files,etc..) and iterate over each X bits of it in a loop, say take bits 1-5, then 6-10, etc.. Then I want to convert these bits to their corresponding decimal value.

but all the binary functions I found in autoit only work with full bytes, and do not let me get smaller sections of bits, like "BinaryMid()" that "Extracts a number of bytes from a binary variant"

Can anyone tell me if this is possible to do and how? and also if there is a function to convert those bits to/from decimals?

I'm not that familiar with dealing directly with binary, so any help or tips will be very appreciated

Context:

I'm trying to make a function to encode/decode any given binary data into/from a given array of characters. just like Base64 but using different bases then [a-z A-Z 0-9 +/= ]. My approach currently is to figure out how many bits of binary I can encode with each character, read those bits and convert them to a decimal number, then I will use that number as an index and take the character at that index from the character array and add it to the result string.

I'm aware that there may be some padding needed.

 

 

Share this post


Link to post
Share on other sites



This is possible to do with the DllStruct- and the Bit-commands. If you want 5 bit blocks you can store 5 bytes at a time from your binary data into a 5 byte structure (40 bits). Then you can handle each of the 5 bit blocks one by one. 8 blocks in a total.

The CPU only handles 1, 2, 4 or 8 (64 bit CPU) bytes at a time. It does not handle 5 or 7 bytes. To handle 5 bytes you can for example split them up in 2 bytes, 2 bytes and 1 byte. If some of the 5 bit blocks have bits spread over byte 2 and 3 or byte 4 and 5 then you must deal with it.

To do all this you will need to be able to handle binary data. And you must be able to convert from binary to decimal numbers in your head. At least for the first 8 bits.

If you have large amounts of binary data AutoIt is not the fastest programming language. You'll need a compiled language.

What's the purpose for this? I do not think it's possible to make better algorithms for simple encoding/decoding, than those that already exists.

Share this post


Link to post
Share on other sites
14 minutes ago, LarsJ said:

This is possible to do with the DllStruct- and the Bit-commands. If you want 5 bit blocks you can store 5 bytes at a time from your binary data into a 5 byte structure (40 bits). Then you can handle each of the 5 bit blocks one by one. 8 blocks in a total.

The CPU only handles 1, 2, 4 or 8 (64 bit CPU) bytes at a time. It does not handle 5 or 7 bytes. To handle 5 bytes you can for example split them up in 2 bytes, 2 bytes and 1 byte. If some of the 5 bit blocks have bits spread over byte 2 and 3 or byte 4 and 5 then you must deal with it.

To do all this you will need to be able to handle binary data. And you must be able to convert from binary to decimal numbers in your head. At least for the first 8 bits.

If you have large amounts of binary data AutoIt is not the fastest programming language. You'll need a compiled language.

What's the purpose for this? I do not think it's possible to make better algorithms for simple encoding/decoding, than those that already exists.

Thanks for the reply LarsJ,  I did some searching before and tried the bit shifting method but it got complicated fast, I will only deal with small binary data so I instead chose to convert the binary to 0/1 strings and handling that instead, pretty dirty and slow, I Know.

Are there any existing encoding functions/libraries that can encode to an arbitrary list of characters? this is exactly what I need.

This function is for a little encryption based game I'm doing for fun, I know autoit may not be the best fit but it's fast to write & easy to use, and it's just a small project.

Share this post


Link to post
Share on other sites

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • wakillon
      By wakillon
      BinaryToAu3Kompressor v1.0.5.4
       

       
      It's now possible to see the best compression ratio using LZMA, LZNT and Base64 compressions with differents combinations.
      Nothing too complicate, you drag'n drop a file on the picture and script Test all compression types and return the ratios.
      ( Test duration depends of file size, slowest compression is LZNT, but all decompressions are fast  )
      Free to you after, to choose the compression(s) you want...
      Yes, LZMA needs a dll ( embedded & compressed in script ) but brings a powerfull compression. 
      It opens scite with your file compressed to an au3 script with or without decompression function as you want.
      Hold Left Shift key when clicking button for just copy script to clipboard.
      Use the 3 compressions at a time works but doesn't give a good ratio, that's why i don't display it.
      Usefull for little files you want include in your scripts !
      No externals files needed, they are already in script.
      Previous downloads : 1103
      Source and Executable
      BinaryToAu3Kompressor will be added to the next version of >SciTEHopper
      Thanks to Ward for his >Base64.au3 and LZMA.au3, and trancexx for his >LZNT functions and his >Base64Decode function.
    • virhonestum
      By virhonestum
      Hey,
      I've coma accross a very odd problem. I want to download a CSV-File, and process the contents.
      This is the extremely simplyfied AutoIT-Code, given the file is already downloaded:
      $f= @ScriptDir & "\TestFile.csv" $file = FileOpen($f,0) Local $line = FileReadLine($file) MsgBox(0,"",$line) The downloaded CSV file I want to process contains something like this:
      Artikelnummer;EAN-Code;Artikelname;Artikelgewicht;Beschreibung;Kurzbeschreibung;Eigenschaften;Technische-Daten;Bild1;Bild2;Bild3;Bild4;Bild5;Bild6;Bild7;Bild8;Lieferbar;"Lieferbar Ab";Versandzeit;"UVP-Preis inkl. MwSt.";"Preis1";"Preis2";Hersteller L7335272;5420025602129; Mini Light XLR ;0.1000; JB Systems Schwanenhalsleuchte mit XLR Anschluss. ;;;;http://www.example.com/media/images/org/pic20070114153500a.jpg;;;;;;;;JA;; 1-3 Tage ;12,90;12,90;6,57; JB Systems L3320502;540207025601636; Mini Light LED BNC ;0.1000; JB Systems LED Schwanenhalsleuchte mit BNC Anschluss. ;;;;http://www.example.com/media/images/org/pic20061231171705a.jpg;;;;;;;;JA;; 1-3 Tage ;29,90;25,89;15,26; JB Systems L1332254;542002556023143; Mini Light LED XLR ;0.1000; JB Systems LED Schwanenhalsleuchte mit XLR Anschluss. ;;;;http://www.example.com/media/images/org/pic20061231171728a.jpg;;;;;;;;JA;; 1-3 Tage ;29,90;25,89;15,26; JB Systems L8302591;504200256280277; Spiegelkugel 10cm ;0.5000; JB Systems Spiegelkugel 10cm Durchmesser mit einer hohen Dichte durch 10 x 10 mm Echtglasspiegel. ;;;;http://www.example.com/media/images/org/pic20060324214825a.jpg;;;;;;;;JA;; 1-3 Tage ;5,50;4,90;2,81; JB Systems L7302932;542000256510222; Spiegelkugel 20cm ;0.8400; JB Systems Spiegelkugel 20cm Durchmesser mit einer hohen Dichte durch 10 x 10 mm Echtglasspiegel. ;;;;http://www.example.com/media/images/org/pic20060324214907a.jpg;;;;;;;;JA;; 1-3 Tage ;12,90;11,50;6,58; JB Systems L2350293;534200562064239; Spiegelkugel 30cm ;2.1300; JB Systems Spiegelkugel 30 cm Durchmesser mit einer hohen Dichte durch 10 x 10 mm Echtglasspiegel. ;;;;http://www.example.com/media/images/org/pic20060324214956a.jpg;;;;;;;;JA;; 1-3 Tage ;26,90;23,00;13,72; JB Systems L3302984;545200252024246; Spiegelkugel 40cm ;3.5000; JB Systems Spiegelkugel 40cm Durchmesser mit Sicherungsring und einer hohen Dichte durch 10 x 10 mm Echtglasspiegel. ;;;;http://www.example.com/media/images/org/pic20060324215050a.jpg;;;;;;;;JA;; 1-3 Tage ;54,90;49,00;28,00; JB Systems L9302495;542205056225600; Spiegelkugel 50cm ;5.3900; JB Systems Spiegelkugel 50cm Durchmesser mit Sicherungsring und einer hohen Dichte durch 10 x 10 mm Echtglasspiegel. ;;;;http://www.example.com/media/images/org/pic20060324215122a.jpg;;;;;;;;JA;; 1-3 Tage ;89,00;79,00;45,39; JB Systems But the message box that pops up after FileReadLine contains this:
      䅲瑩步汮畭浥爻䕁中䍯摥㭁牴楫敬湡浥㭁牴楫敬来睩捨琻䉥獣桲敩扵湧㭋畲穢敳捨牥楢畮朻䕩来湳捨慦瑥渻呥捨湩獣桥ⵄ慴敮㭂楬搱㭂楬搲㭂楬搳㭂楬搴㭂楬搵㭂楬搶㭂楬搷㭂楬搸㭌楥晥牢慲㬢䱩敦敲扡爠䅢∻噥牳慮摺敩琻≕噐ⵐ牥楳⁩湫氮⁍睓琮∻≐牥楳ㄢ㬢偲敩猲∻䡥牳瑥汬敲ੌ㜳㌵㈷㈻㔴㈰〲㔶〲ㄲ㤻M楮椠䱩杨琠塌刀㬰⸱〰〻J䈠卹獴敭猠卣桷慮敮桡汳汥畣桴攠浩琠塌删䅮獣桬畳献;㬻㭨瑴瀺⼯睷眮數慭灬攮捯洯浥摩愯業慧敳⽯牧⽰楣㈰〷〱ㄴㄵ㌵〰愮橰朻㬻㬻㬻㭊䄻㬀ㄭ㌠呡来;ㄲⰹ〻ㄲⰹ〻㘬㔷㬀䩂⁓祳瑥浳 I've attached both files I use. 
      My guess is, that there's something wrong with the encoding, but I'm not sure how to fix it.
       
      Thank you very much for your help
      - virhonestum
      Encodingtester.au3
      TestFile.csv
    • 4bst1n3nz
      By 4bst1n3nz
      Hello,
      i need to save files with ANSI-Encoding. Since 3.3.14.2 Auto-It it doesn't work in any direction.
      I tried the following:
      #include <FileConstants.au3> FileDelete(@ScriptDir&"\Test.txt") $o = FileOpen(@ScriptDir&"\Test.txt", BitOR($FO_BINARY,$FO_ANSI,$FO_OVERWRITE)) FileWrite($o, "Test") FileClose($o) Or
      #include <FileConstants.au3> FileDelete(@ScriptDir&"\Test.txt") $o = FileOpen(@ScriptDir&"\Test.txt", 514) FileWrite($o, "Test") FileClose($o) Both create UTF-8 encoded files.
      What am i doing wrong?
      Thank you!
    • WoodGrain
      By WoodGrain
      Hi All,
      Trying to convert a number to binary zeros and ones but I'm getting a result I don't understand and looks more like hex than binary.
      Here's my basic code:
      $myNum = 11 $myNumBin = Binary($myNum) MsgBox(0, "Binary result", $myNumBin) What I want is "1011", what I get is 0x0B000000.
      Thanks!