Sign in to follow this  
Followers 0
CarlD

Use text editor to view/edit/save binary files

1 post in this topic

#1 ·  Posted (edited)

This code is a subset of an AutoIt tool I wrote for the classic DOS word-processor XyWrite. The tool allows 7-bit Ascii encoding of XyWrite Programming Language programs, with "readability aids" that mimic the way this code looks natively on the DOS screen. This is a special 7-bit encoding that we've used for many years in the XyWrite community to discuss XPL code on the XyWrite Mailing List. But that's neither here nor there.

The present subset consists of three utilities, which I offer here for what they're worth. The first, DVIEW.AU3, takes a binary file and displays it in the default Windows text editor, displaying only Ascii chars 32-127, the other chars being represented by ".". The command-line usage is:
DVIEW.AU3 <file_in><Enter>

The second, DREAD.AU3 (that's "Dee-Read", not "dread" ;) ), provides a similar display, except that characters outside the Ascii 32-127 range are represented by "{nnn}", where "nnn" is the 3-digit decimal Ascii number. (The initial "D" in these utilities' names stands for "decimal".) The output from DREAD.AU3 can be edited to make simple patches to binary files. The output file is named DREAD.TXT. The usage is:
DREAD.AU3 <file_in><Enter>

The third utility, DWRITE.AU3, takes DREAD output and writes it back to disk as a binary file. So, once you edit the output from DREAD, you write it to disk with:
DWRITE.AU3 <file_in><Enter>
The default file_in is DREAD.TXT -- i.e., the output of DREAD.AU3. The output file is named DWRITE.BIN, which can be renamed as desired.

You'll see that each of these scripts processes the input file character by character. If there's a faster way of doing this, for example by manipulating bit patterns, I'd be pleased to hear about it.

Here are the three scripts. Enjoy.

; DVIEW.AU3 -- AutoIt v3 [CarlD rev.9/27/15]
; Display a decimal view of a binary file
;
; Usage:
; DREAD.AU3 file_in

ProgressOn(@ScriptName,"Working")

Global $iLnLen = 0; Line length meter
Local $sTmp = "";   Temp string var

Local $sInFile = @ScriptDir & "\DVIEW.IN"
If $CmdLine[0] > 0 Then $sInFile = $CmdLine[1]
If Not FileExists($sInFile) Then
    ProgressOff()
    MsgBox(16, @Scriptname, $sInFile & " does not exist!", 3)
    Exit
EndIf

Local $sTmpFile = @ScriptDir & "\DVIEW.TMP"
Local $sOutFile = @ScriptDir & "\DVIEW.TXT"

If FileExists($sTmpFile) Then FileDelete($sTmpFile)
If FileExists($sOutFile) Then FileDelete($sOutFile)

Local $hWrIn = FileOpen($sInFile, 16);  Handle for source file
Local $sToEncode = FileRead($hWrIn);    Binary (hex) string to encode
FileClose($hWrIn)

Global $sEncoded = "";  Encoded output (string)
Local $aEncoded = HexToDec($sToEncode); Binary (hex) to decimal array
Local $iAsc = "";   Decimal Ascii number of current char

; Loop through each byte of input string
For $i = 1 To UBound($aEncoded) - 1
    $iAsc = StringFormat("%03u", $aEncoded[$i])
    $sTmp = ""

    If $iAsc > 31 And $iAsc < 128 Then
        $sTmp = Chr($aEncoded[$i])
    Else
        $sTmp = "."
    EndIf

    $sTmp = AddCrLf($sTmp)
    If $iLnLen = 0 And $sTmp = "." Then $sTmp = "{046}"
    If $iLnLen = 0 And $sTmp = ">" Then $sTmp = "{062}"
    $sEncoded &= $sTmp  
Next    

; Trim double CrLf to one; change trailing space to "{032}"
If StringRight($sEncoded, 2) = @CRLF Then _
    $sEncoded = StringTrimRight($sEncoded, 2)
If StringRight($sEncoded, 1) = " " Then _
    $sEncoded = StringTrimRight($sEncoded, 1) & "{032}"

; Add header and footer
Local $sHeader = "DVIEW v1.0" & @CRLF
$sEncoded = $sHeader & "b-gin [" & $sInFile & "]" & @CRLF & _
    $sEncoded & @CRLF & "-nd DVIEW" & @CRLF

; Write output file
Local $hWrOut = FileOpen($sTmpFile, 2)
FileWrite($sTmpFile, $sEncoded)
FileClose($hWrOut)
FileMove($sTmpFile, $sOutFile, 1)

ProgressSet(100, "Done")
Sleep(2000)
ProgressOff()
ShellExecute($sOutFile)

; --------- Function DeFinitions ---------

Func HexToDec($sHexIn); Convert hex string to decimal array
    $aHexChars = StringSplit($sHexIn, "")
    Local $aHexIn[UBound($aHexChars) / 2]
    Local $j = 0
    For $i = 1 To UBound($aHexChars) Step 2
        If $i + 1 <= UBound($aHexChars) Then
            $aHexIn[$j] = $aHexChars[$i] & $aHexChars[$i + 1]
            $j += 1
        Else
            ExitLoop
        EndIf
    Next
    Local $aDecOut[UBound($aHexIn)]
    For $i = 0 To UBound($aHexIn) - 1
        $aDecOut[$i] = Dec($aHexIn[$i])
    Next
    Return $aDecOut
EndFunc   ;==>HexToDec

Func AddCrLf($sIn); Add line breaks to output
    $iLnLen += StringLen($sIn)
    If $iLnLen > 74 Then
        $sIn &= @CRLF
        $iLnLen = 0
    EndIf
    Return $sIn
EndFunc   ;==>AddCrLf
; DREAD.AU3 -- AutoIt v3 [CarlD rev.9/27/15]
; Display a decimal view of a binary file
;
; Usage:
; DREAD.AU3 file_in

ProgressOn(@ScriptName,"Working")

Global $iLnLen = 0; Line length meter
Local $sTmp = "";   Temp string var

Local $sInFile = @ScriptDir & "\DREAD.IN"
If $CmdLine[0] > 0 Then $sInFile = $CmdLine[1]
If Not FileExists($sInFile) Then
    ProgressOff()
    MsgBox(16, @Scriptname, $sInFile & " does not exist!", 3)
    Exit
EndIf

Local $sTmpFile = @ScriptDir & "\DREAD.TMP"
Local $sOutFile = @ScriptDir & "\DREAD.TXT"

If FileExists($sTmpFile) Then FileDelete($sTmpFile)
If FileExists($sOutFile) Then FileDelete($sOutFile)

Local $hWrIn = FileOpen($sInFile, 16);  Handle for source file
Local $sToEncode = FileRead($hWrIn);    Binary (hex) string to encode
FileClose($hWrIn)

Global $sEncoded = "";  Encoded output (string)
Local $aEncoded = HexToDec($sToEncode); Binary (hex) to decimal array
Local $iAsc = "";   Decimal Ascii number of current char

; Loop through each byte of input string
For $i = 1 To UBound($aEncoded) - 1
    $iAsc = StringFormat("%03u", $aEncoded[$i])
    $sTmp = ""

    If $iAsc > 31 And $iAsc < 128 Then
        $sTmp = Chr($aEncoded[$i])
    Else
        $sTmp = "{" & $iAsc & "}"
    EndIf

    $sTmp = AddCrLf($sTmp)
    If $iLnLen = 0 And $sTmp = "." Then $sTmp = "{046}"
    If $iLnLen = 0 And $sTmp = ">" Then $sTmp = "{062}"
    $sEncoded &= $sTmp  
Next    

; Trim double CrLf to one; change trailing space to "{032}"
If StringRight($sEncoded, 2) = @CRLF Then _
    $sEncoded = StringTrimRight($sEncoded, 2)
If StringRight($sEncoded, 1) = " " Then _
    $sEncoded = StringTrimRight($sEncoded, 1) & "{032}"

; Add header and footer
Local $sHeader = "DeeREAD v1.0" & @CRLF
$sEncoded = $sHeader & "b-gin [" & $sInFile & "]" & @CRLF & _
    $sEncoded & @CRLF & "-nd DeeREAD" & @CRLF

; Write output file
Local $hWrOut = FileOpen($sTmpFile, 2)
FileWrite($sTmpFile, $sEncoded)
FileClose($hWrOut)
FileMove($sTmpFile, $sOutFile, 1)

ProgressSet(100, "Done")
Sleep(2000)
ProgressOff()
ShellExecute($sOutFile)

; --------- Function DeFinitions ---------

Func HexToDec($sHexIn); Convert hex string to decimal array
    $aHexChars = StringSplit($sHexIn, "")
    Local $aHexIn[UBound($aHexChars) / 2]
    Local $j = 0
    For $i = 1 To UBound($aHexChars) Step 2
        If $i + 1 <= UBound($aHexChars) Then
            $aHexIn[$j] = $aHexChars[$i] & $aHexChars[$i + 1]
            $j += 1
        Else
            ExitLoop
        EndIf
    Next
    Local $aDecOut[UBound($aHexIn)]
    For $i = 0 To UBound($aHexIn) - 1
        $aDecOut[$i] = Dec($aHexIn[$i])
    Next
    Return $aDecOut
EndFunc   ;==>HexToDec

Func AddCrLf($sIn); Add line breaks to output
    $iLnLen += StringLen($sIn)
    If $iLnLen > 74 Then
        If $sIn = " " Then $sIn = "{032}"
        $sIn &= @CRLF
        $iLnLen = 0
    EndIf
    Return $sIn
EndFunc   ;==>AddCrLf
; DWRITE.AU3 -- AutoIt v3 [CarlD rev.9/27/15]
; Write DVIEW encoding as binary file
;
;   Usage:
; DWRITE.AU3 file_in
; Output is sent to @ScriptDir & "DWRITE.BIN"

ProgressOn(@ScriptName,"Working")

Local $sInFile = @ScriptDir & "\DREAD.TXT"
If $CmdLine[0] > 0 Then $sInFile = $CmdLine[1]
If Not FileExists($sInFile) Then
    ProgressOff()
    MsgBox(16, @Scriptname, $sInFile & " does not exist!", 3)
    Exit
EndIf

Local $sTmpFile = @ScriptDir & "\DWRITE.TMP"
Local $sOutFile = @ScriptDir & "\DWRITE.BIN"

Local $hWrIn = FileOpen($sInFile);  Handle for source file
Local $sMaster = FileRead($hWrIn);  Master string to decode
FileClose($hWrIn)
Local $sToDecode = ""
Local $aTmp = ""

; Remove header|footer
If StringLeft($sMaster, 9) = "DeeREAD v" Then _
        $sMaster = StringTrimLeft($sMaster, StringInStr($sMaster, "]"))
If StringRight($sMaster, 13) = "-nd DeeREAD" & @CRLF Then _
        $sMaster = StringTrimRight($sMaster, 13)

Local $sFinished = ""
Local $iChunkSz = 512
Local $iAdd = 0

; - - - - - - Main Loop - - - - - -
While $sMaster
    If StringLen($sMaster) > $iChunkSz Then
        $sToDecode = StringLeft($sMaster, $iChunkSz)
        $sMaster = StringTrimLeft($sMaster, $iChunkSz)
        If StringRight($sToDecode, 2) <> @CRLF Then
            $iAdd = 1 + StringInStr($sMaster, @CRLF)
            $sToDecode &= StringLeft($sMaster, $iAdd)
            $sMaster = StringTrimLeft($sMaster, $iAdd)
        EndIf
    Else
        $sToDecode = $sMaster
        $sMaster = ""
    EndIf

    ;   Strip CrLfs
    $sToDecode = StringReplace($sToDecode, @CRLF, "")

    ;       "{nnn}" ==> 1-byte Ascii char;
    Local $aTmp = StringSplit($sToDecode, "{")
    Local $iAsc = -1
    For $i = 1 To UBound($aTmp) - 1
        $iAsc = StringLeft($aTmp[$i], 3)
        If StringInStr($aTmp[$i], "}") = 4 And _
                StringIsDigit($iAsc) Then
            If $iAsc > -1 And $iAsc < 256 Then
                $sToDecode = StringReplace($sToDecode, "{" & _
                        StringLeft($aTmp[$i], 4), Chr($iAsc))
            EndIf
        EndIf
    Next

    $sFinished &= $sToDecode
    $sToDecode = ""
WEnd
; - - - - - End Main Loop - - - - -

; Write output file
Local $hWrOut = FileOpen($sTmpFile, 2)
FileWrite($sTmpFile, $sFinished)
FileClose($hWrOut)
FileMove($sTmpFile, $sOutFile, 1)
ProgressOff()
MsgBox(0, @ScriptName, "Output in " & $sOutFile, 5)
; Done
Edited by CarlD

Share this post


Link to post
Share on other sites



Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Similar Content

    • BigDaddyO
      By BigDaddyO
      After much searching I finally found a method to get eMails from an Exchange eMail account inbox.  I was hoping for AutoIT or vbscript but I couldn't find any that would read the inbox messages without using Outlook.
      I needed this because I'm testing a web-form that generates an eMail sent to a shared mailbox "not what my current outlook is configured for".  so, I needed to connect to a different account, then get the inbox messages, and see if the auto-generated eMail message body contains what I submitted in the form.
       
      I found a PowerShell script that was close and modified it to do just what I want, but I'd still like it to run in AutoIT but I'm not sure how to use the Microsoft.Exchange.WebServices.dll
      Anybody have some ideas? 
      #To Launch! # C:\Windows\System32> powershell -ExecutionPolicy ByPass #This launches PowerShell and allows execution of .ps1 files # PS C:\Windows\System32> . "C:\Temp\eMail\getInbox.ps1" #The period . in front of the .ps1 file forces PS to display results on-screen # Where is the EWS .DLL file that you are using # Get the installer from https://www.microsoft.com/en-us/download/details.aspx?id=42022 # We only need 2 dll's from the install and they can be stored anywhere: "Microsoft.Exchange.WebServices.Auth.dll" & "Microsoft.Exchange.WebServices.dll" $EWSdll = "C:\Temp\eMail\Microsoft.Exchange.WebServices.dll" # Where do you want the output text file to be saved $Output = "C:\Temp\eMails.txt" # replace with your email address $email = "MyemailAddress@work.net" # only need to populate these if you're impersonating... $username = "myemail" $password = "Sup3rS3cre+" $domain = "ad.work.net" # load the assembly : point to the dll in the location you have the .dll file [void] [Reflection.Assembly]::LoadFile($EWSdll) # set ref to exchange, first references 2007, 2nd is 2010 (default) #$s = New-Object Microsoft.Exchange.WebServices.Data.ExchangeService([Microsoft.Exchange.WebServices.Data.ExchangeVersion]::Exchange2007_SP1) $s = New-Object Microsoft.Exchange.WebServices.Data.ExchangeService # use first option if you want to impersonate, otherwise, grab your own credentials with the 3rd one. not sure what the 2nd one is for $s.Credentials = New-Object Net.NetworkCredential($username, $password, $domain) ##$s.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials #$s.UseDefaultCredentials = $true # discover the url from your email address $s.AutodiscoverUrl($email) # get a handle to the inbox $inbox = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($s,[Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Inbox) #create a property set (to let us access the body & other details not available from the FindItems call) $psPropertySet = new-object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.BasePropertySet]::FirstClassProperties) $psPropertySet.RequestedBodyType = [Microsoft.Exchange.WebServices.Data.BodyType]::Text; # If you have a set number of items you want to get, use this and insert the # in the () # $items = $inbox.FindItems(5) # If you want to retrieve all items (Server limit is usually at 1000) then use this line # Details on the max returned by server: https://blogs.msdn.microsoft.com/exchangedev/2010/03/12/throttling-policies-and-the-ewsfindcountlimit/ $items = $inbox.FindItems($inbox.TotalCount) # Put some counts at the top of the output Write-host "Total Inbox count: $($inbox.TotalCount)" Write-host "Unread count: $($inbox.UnreadCount)" #These two lines, write the output to the specified text file Add-Content $Output "Total Inbox count: $($inbox.TotalCount)" Add-Content $Output "Unread count: $($inbox.UnreadCount)" foreach ($item in $items.Items) { # load the property set to allow us to get to the body $item.load($psPropertySet) # Get the Body text as-is $bod = $item.Body.Text #if you only want a short summary of the Body, then comment the above line and un-comment these 4 lines # $bod = $item.Body.Text -replace '\s+', ' ' # $bodCutOff = (100,$bod.Length | Measure-Object -Minimum).Minimum # $bod = $bod.Substring(0,$bodCutOff) # $bod = "$bod..." # output the results - first of all the From, Subject, References and Message ID write-host "====================================================================" Write-host "From: $($item.From.Name)" Write-host "Subject: $($item.Subject)" Write-host "Body: $($bod)" write-host "====================================================================" "" # Output the results to the specified Text file Add-Content $Output "" Add-Content $Output "====================================================================" Add-Content $Output "From: $($item.From.Name)" Add-Content $Output "Subject: $($item.Subject)" Add-Content $Output "Body:",$($bod) Add-Content $Output "====================================================================" Add-Content $Output "" } #see these URLs for more info # EWS Stuff # folder members: https://msdn.microsoft.com/en-us/library/microsoft.exchange.webservices.data.folder_members%28v=exchg.80%29.aspx # exporting headers: https://www.allabout365.com/2010/10/export-email-headers-exchange-powershell/ # read emails with EWS: https://social.technet.microsoft.com/Forums/en-US/3fbf8348-2945-43aa-a0bc-f3b1d34da27c/read-emails-with-ews?forum=exchangesvrdevelopment  
      Thanks,
      Mike
    • Dragonfighter
      By Dragonfighter
      I'm searching a way to do xor and shift and if possible also other operations. Thanks in advance for the replies.
    • rudi
      By rudi
      Hello.
      I'm too stupid to see my mistake:
      To investigate the internal "dictionary" of TIFF files I'd like to read in the files in binary mode and to check, if there are more than one pages "in" this TIFF.
      Notepad++, "View as Hex" is presenting the first bytes as "49 49 2a 20 08 20 20 20 12" for the TIF attached to this posting
      The "TIFF Header Format" is easy:
      Offset 00h, 2 Byte = Byte Order, "II"=intel, "MM"=motorola. (I = 0x49)
      --> II
      Offset 02h, 2 Byte = Version Nr.
      Offset 04h, 4 Byte = pointer to first IFD entry
      Description of TIFF header: https://www.awaresystems.be/imaging/tiff/faq.html#q3
       

      Howto read and analyse the binary content correctly? This is my messy, not operational code:
       
      $sampleTiff="H:\daten\tif\11\11\111111.TIF" $h=FileOpen($sampleTiff,16) $content=FileRead($h) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $content = ' & $content & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console FileClose($h) $type=VarGetType($content) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $type = ' & $type & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console $ToString=BinaryToString($content) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $ToString = ' & $ToString & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console ConsoleWrite(@CRLF & @CRLF) $content=StringTrimLeft($content,2) ; cut off the leading "0x" ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $content = ' & $content & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console for $i = 1 to 8 step 8 $next=StringMid($content,$i,2) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $next = ' & $next & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console $Chr=BinaryToString($next) ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $Chr = ' & $Chr & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console ConsoleWrite(@CRLF & "---" & @CRLF) Next Regards, Rudi.
      111111.TIF
    • ur
      By ur
      When I am trying to compile the autoit files with aut2exe.
      I am getting below error.
      There is no issue in code as the same code is getting compiled on different machine.
      I tried reinstalling the AUtoIT, but the issue replicates.

      Any suggestions?
    • wakillon
      By wakillon
      BinaryToAu3Kompressor v1.0.5.4
       

       
      It's now possible to see the best compression ratio using LZMA, LZNT and Base64 compressions with differents combinations.
      Nothing too complicate, you drag'n drop a file on the picture and script Test all compression types and return the ratios.
      ( Test duration depends of file size, slowest compression is LZNT, but all decompressions are fast  )
      Free to you after, to choose the compression(s) you want...
      Yes, LZMA needs a dll ( embedded & compressed in script ) but brings a powerfull compression. 
      It opens scite with your file compressed to an au3 script with or without decompression function as you want.
      Hold Left Shift key when clicking button for just copy script to clipboard.
      Use the 3 compressions at a time works but doesn't give a good ratio, that's why i don't display it.
      Usefull for little files you want include in your scripts !
      No externals files needed, they are already in script.
      Previous downloads : 1103
      Source and Executable
      BinaryToAu3Kompressor will be added to the next version of >SciTEHopper
      Thanks to Ward for his >Base64.au3 and LZMA.au3, and trancexx for his >LZNT functions and his >Base64Decode function.