Jump to content
Sign in to follow this  
AndyS01

StringReplace() does not work on binary data

Recommended Posts

I'm trying to replace all occurrences of one string in a binary file with another string (e.g. all "C:\" with "D:\"), but StringReplace() does nothing.

I opened my files in Binary mode and set the data read in to binary(), but nothing works.

My test script reads a compiled version of the script, does the replacements, then writes the results to an output file.

Prior to calling StringReplace(), I call StringInStr() to see if my 'from' string is found. The StringInStr() reports that it found the string, but the StringReplace() reports 0 replacements. Just to be sure, I did a binary compare (BeyondCompare) and the input and output files were exactly the same.

Here is my test code:

#AutoIt3Wrapper_UseUpx=n
#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6
Opt('MustDeclareVars', 1)

_Main()

Func _Main()
Local $infn, $infh, $outfn, $outfh, $cnt, $str_from, $str_to
Local $buffer, $offset = 0, $bufsize, $mcnt, $matches = 0, $ret

$infn = StringTrimRight(@ScriptName, 4) & ".exe"
$outfn = "test2_out.exe"

$infh = FileOpen($infn, 16 + 0)
If ($infh <= 0) Then Exit (1)

$outfh = FileOpen($outfn, 16 + 2) ; 2 = Write mode (erase previous contents)
If ($infh <= 0) Then Exit (1)

$str_from = "D:\"
$str_to = "C:\"
$bufsize = 2000

While (1)
FileSetPos($infh, $offset, 0)

$buffer = FileRead($infh, $bufsize)
If (@error == -1) Then ExitLoop
$cnt = @extended

$buffer = Binary($buffer)

$ret = StringInStr($buffer, $str_from)
If ($ret >= 0) Then ConsoleWrite("+++: 1 or more strings are in the buffer" & @CRLF)

$buffer = StringReplace( _
$buffer, _
$str_from, _
$str_to _
)

$mcnt = @extended
$matches += $mcnt
ConsoleWrite("+++: " & $mcnt & " replacement(s) made" & @CRLF)

FileWrite($outfh, $buffer)

$offset += $cnt
WEnd

FileClose($infh)
FileClose($outfh)

ConsoleWrite("+++: matches: " & $matches & @CRLF)
ConsoleWrite("+++: processed " & $offset & " bytes." & @CRLF)
EndFunc ;==>_Main

Share this post


Link to post
Share on other sites

You're opening the file in binary mode, then converting what you read from the file to binary, then you're looking for an ASCII string in it, you might want to rething what you're looking for and what the contents of $buffer actually looks like.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

AndyS01,

As I read that code, you trying to use this to rewrite a compiled AutoIt file:

$infn = StringTrimRight(@ScriptName, 4) & ".exe" ; Remove .au3/.exe and replace it with .exe
$outfn = "test2_out.exe"

$infh = FileOpen($infn, 16 + 0) ; Open the .exe file

If that is indeed the case then you need to know that the script part of the compiled .exe is compressed before being added to the interpreter stub and so is very unlikely to hold the literal strings you seem to want to replace:

$str_from = "D:"
$str_to = "C:"

even if you use the correct format for the strings as BrewManNH has suggested. So I am not at all sure that you will ever succeed. ;)

What I have done in the past as a coding exercise was to store the basic text script in the resources of the compiled exe (using #AutoIt3Wrapper_Res_SaveSource) and then when needed extract it, modify it and then recompile the whole thing. But you are now into FileInstalling Aut2Exe and AutoItSC.bin which seriously increases the size of the compiled file. Are you sure that you want to go to so much trouble? :huh:

M23


Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Just a note, if you do decide to use #AutoIt3Wrapper_Res_SaveSource (as Melba23 suggest,) then look at _GetSavedSource in my signature.


UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Share this post


Link to post
Share on other sites

Thank you for your suggestions. You're right, a compiled AutoIT file will not have what I want. However, making the compiled script an input file was just a test.

What I'm going to use my script for, is to replace "C:" with "D:" in a very large binary file. I will actually be editing it 'in place'.

I used a 'strings' utility on several .exe files until I found relatively small one with an instance of "D:". I used a copy of "C:WindowsSystem32poqexec.exe" as a test file. This file has 1 occurrence of "D:" (at offset 0x0950).

StringReplace() still didn't do any substitutions.

I'n not sure what 'binary()' buys me. Perhaps I should to a StringReplace() using patterns enclosed with 'binary()'.

Share this post


Link to post
Share on other sites

Well, after much experientation, I got a lot farther. I found that mixing Binary...() functions and String...() functions doesn't really work.

My new code, below, successfully finds all occurrences of '$str_from'. However, I cannot figure out how to replace binary($buffer) data with new data ($str_to).

I would want to say "BinaryMid($buffer, $x1, 3) = StringToBinary($str_to)", but apparently "BinaryMid($buffer, $x1, 3)" cannot be on the left side of an assignment (I get syntax errors when I try to run with this line in).

How can I replace data in a binary($buffer)?

#AutoIt3Wrapper_UseUpx=n
#AutoIt3Wrapper_Au3Check_Parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6
Opt('MustDeclareVars', 1)

_Main()

Func _Main()
   Local $infn, $infh, $outfn, $outfh, $cnt, $str_from, $str_to, $x1
   Local $buffer, $offset = 0, $bufsize, $matches = 0

   $infn = "poqexec.exe"
   $outfn = "test2_out.exe"

   $infh = FileOpen($infn, 16 + 0)
   If ($infh <= 0) Then Exit (1)

   $outfh = FileOpen($outfn, 16 + 2) ; 2 = Write mode (erase previous contents)
   If ($infh <= 0) Then Exit (1)

   $str_from = "D:"
   $str_to = "C:"
   $bufsize = 0x1000

   While (1)
      FileSetPos($infh, $offset, 0)

      $buffer = FileRead($infh, $bufsize)
      If (@error == -1) Then ExitLoop
      $cnt = @extended
      $buffer = Binary($buffer)

      For $x1 = 0 To $cnt - 3
         If (BinaryToString(BinaryMid($buffer, $x1, 3), 1) = $str_from) Then
            ConsoleWrite("+++: found it at 0x" & Hex($offset + $x1) & @CRLF)
            $matches += 1
            ; The next line replaces the 'found' data with the 'repace with' data
            ;BinaryMid($buffer, $x1, 1) = StringToBinary($str_to)
         EndIf
      Next

      FileWrite($outfh, $buffer)
      $offset += $cnt
   WEnd

   FileClose($infh)
   FileClose($outfh)

   ConsoleWrite("+++: matches:  " & $matches & @CRLF)
   ConsoleWrite("+++: processed " & $offset & " bytes." & @CRLF)
EndFunc   ;==>_Main

Share this post


Link to post
Share on other sites

Look

If the binary data, the lines must also be hexadecimal numbers. You can replace the text in the EXE-file, as if it were a string. I replaced and comparing the result was correct.

screenshot (Change in binary data)

Edited by AZJIO

Share this post


Link to post
Share on other sites

Look

If the binary data, the lines must also be hexadecimal numbers. You can replace the text in the EXE-file, as if it were a string. I replaced and comparing the result was correct.

screenshot (Change in binary data)

AZJIO - your utility works great, but I need to be able to modify a binary file from within my script. I dug through your code, but it's obviously beyond me :wacko: . I can easily read in a binary file to a variable and dump it back out to another file:

$hFile = FileOpen(@ScriptDir & "binaryfile1.tmp", 16)
$hFileContents = FileRead($hFile)
FileClose($hFile)

$nFile = FileOpen(@ScriptDir & "binaryfile2.tmp", 17)
FileWrite($nFile, $hFileContents)
FileClose($nFile)

... but how to replace/insert data within $hFileContents?! Could one split the binary data at the position to edit (into "before" & "after" variables), convert the replace/insert data (with _StringToHex) and then:

FileWrite($nFile, $hFileContentsBefore & $EditData & $hFileContentsAfter)

?? :think: I apologize, I'm clueless on dealing with binary data - can anyone provide an example? :wacko:

Share this post


Link to post
Share on other sites

Never mind, I think I figured it out:

#include 
#include 
$Insert = "Blahblahblah"
$InsertHex = _StringToHex($Insert)

$nFile = FileOpen(@ScriptDir & "test.bin", 18)
FileWrite($nFile, "0x0B00000000" & $InsertHex & "000D0A000000")
FileClose($nFile)

I didn't understand the hex format and how AutoIt handles binary data. Now I know a little bit more via trial and error. :unsure:

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By cdeb
      As for the object, in this case the StringReplace() function does not work for me.
      These are my steps:
      1) Extract the subject from an EML file.
       (the first character is an emoji )
      source from EML file: 
      $sSubject = "?utf-8?B?8J+TiCA1IG9mIHRoZSBCZXN0IFN0b2NrcyB0byBCdXkgZm9yIERlY2VtYmVy?=" 2) in this case I perform a decoding with _QuotedPrintable_DecodeEncodedWord()
      output is: 
      $sSubject = "?? 5 of the Best Stocks to Buy for December"  
      3) I perform StringReplace():
      $sSubject = StringReplace($sSubject, "??", "") or
      $sSubject = StringReplace($sSubject, Chr(63)&Chr(63), "")  
      But the characters ?? they are not replaced.
       
      Yet if I make an Asc of every character of the string with a for loop, are the first two characters really? that is 63
      For cycle
              Local $aArray = StringSplit($sSubject, "", $STR_NOCOUNT)         For $i = 0 To UBound($aArray)-1             ConsoleWrite($aArray[$i] & " - " & Asc($aArray[$i])& @CR)         Next Output:        
      ? - 63
      ? - 63
        - 32
      5 - 53
        - 32
      o - 111
      f - 102
        - 32
      t - 116
      h - 104
      e - 101
        - 32
      Does anyone have an explanation of why it doesn't work?
      Thank you all
       
    • By jmp
      I am adding labour charge to total paid amount using : 
      #include <IE.au3> #include <Array.au3> $oIE = _IEAttach ("Shop") $oTable = _IETableGetCollection ($oIE, 1) $aTableData3 = _IETableWriteToArray ($oTable) Local $sitem1 = $aTableData3[5][1] Local $sitem2 = $aTableData3[5][2] Local $lcharge = "10" ;add manualy using inputbox, becuase not generating online Local $atotPric = "Payable Total Price " Local $oTds = _IETagNameGetCollection($oIE, "td") For $oTd In $oTds If $oTd.Innertext = $atotPric Then $iatotPric = $oTd.NextElementSibling.innertext MsgBox (0, "2", $iatotPric) EndIf Next $irCtotal = StringFormat("%.2f", $sitem1 + $sitem2 + $lcharge) $crTotp = StringReplace(_IEBodyReadHTML($oIE), $iatotPric, $irCtotal) _IEBodyWriteHTML ($oIE, $crTotp) But, It was also changing Total price, I want to change only Payable Total Price.

    • By Stew
      (Edited from original.  Please note that I AM NOT AN AUTOIT EXPERT.  I write code using Autoit frequently but I am no expert, especially when it comes to I/O.  So any remarks that start with "Why did you..." can be answered by referring to the first sentence.  This project was done in Autoit because of an interface I built to display the data.)
      Attached is a program and ascii input file I wrote to read stock price data, convert it to binary and then read it back into the program in binary.  The goal was to show increased performance for reading the files in binary and provide a demo on how to read/write binary for int32, int64, double and strings for anyone who might find it helpful.  The results on my PC show the following:
      Time to read ascii file only: 456.981951167202
      Ascii read & process time: 6061.83075631701
      Binary write file time: 14787.9184635239
      Time just to read binary file: 42.418867292311
      Binary read and process time: 4515.16129830537
      A couple things to note:
      1) The 32 MB ascii file took 10x longer to read than the 15 MB binary file.  Not entirely sure why.  Both were read into a buffer.
      2) The Binary write takes a long time but I made no effort to optimize this because the plan was to write this file one time only so I don't mind if it takes longer to write this file.  I care much more about how long it takes to read the file because I will be reading it many times.
      3) There was a modest gain in converting the ascii file to binary in terms of file size and reading speed.
      So big picture... not sure it's worth the effort to convert the files to binary even though most of the data is numerical data in the binary file.  That was actually surprising as I expected there would be more of a difference.  Any ideas on how to get the binary data to read at a faster rate would be great.
       
      binary.au3
      2019_02_08.zip
    • By milos83
      Default keyword for optional parameter is interpreted wrongly.
      ConsoleWrite(StringReplace("aa", "a", "b", Default, 1) & @CRLF) StringReplace ( "string", "searchstring/start", "replacestring" [, occurrence = 0 [, casesense = 0]] ) The code above will output ab even thou the default value for the occurrence is 0 (replace all).
      Of course using zero instead of Default will work fine.
    • By smartkey
      Hi All,
           I have written a UDF for one of my requirement which replaces a single character in string with a sub string/another character. 
           I am using this for my requirement by calling below function as StrReplace("C:\Software\Autoit\Substr","\","\\") and gives result as C:\\Software\\Autoit\\Substr
           Please let me know if this can be improvised or any mistakes to correct.   
      ;===============================================================================
      ;
      ; Function Name:    StrReplace($INPUT_STRING)
      ; Description:      This function is to replace a character with another in a string.
      ; Parameter(s):     $INPUT_STRING     - Original String Value
      ;                    $STR_2_FIND     - Single Character to find the $INPUT_STRING
      ;                    $STR_2_REPLACE     - Substring/Multiple Characters to replace in place of     $STR_2_FIND value
      ; Requirement(s):   Replacing one single Character in a string with multiple Characters
      ; Return Value(s):  success - Output string after replacing a character with required character
      ;                    failure - 0
      ; Author(s):        smartkey
      ;
      ;===============================================================================
      Func StrReplace($INPUT_STRING, $STR_2_FIND, $STR_2_REPLACE)
          Local $OUTPUT_STRING = ""
          If StringLen($INPUT_STRING) > 0 Then
              If StringMid($INPUT_STRING,1,1) = $STR_2_FIND Then
                  $OUTPUT_STRING = $OUTPUT_STRING & $STR_2_REPLACE
              Else
                  $OUTPUT_STRING = StringMid($INPUT_STRING,1,1)
              EndIf
              For $i=2 to StringLen($INPUT_STRING)
                  If StringMid($INPUT_STRING,$i,1) = $STR_2_FIND Then
                      $OUTPUT_STRING= $OUTPUT_STRING & $STR_2_REPLACE
                  Else
                      $OUTPUT_STRING= $OUTPUT_STRING & StringMid($INPUT_STRING,$i,1)
                  EndIf
              Next
              Return $OUTPUT_STRING
          Else
              Return 0
          EndIf
      EndFunc
×
×
  • Create New...