HEX edit on a dll file

[Irongeek]

I need to scan a dll and patch it by replacing an 8 byte string with another one, but I can't seem to figure out how. I tried with this sample code:

#include <File.au3>

$find = "MSFT 5.0"
$replace = "12345678"

$filename = "C:\Documents and Settings\me\Desktop\dhcpcsvc.dll"


$retval = _ReplaceStringInFile($filename,$find,$replace)
if $retval = -1 then
    msgbox(0, "ERROR", "The pattern could not be replaced in file: " & $filename & " Error: " & @error)
    exit
else
    msgbox(0, "INFO", "Found " & $retval & " occurances of the pattern: " & $find & " in the file: " & $filename)
endif

$msg = FileRead($filename, 1000)
msgbox(0,"AFTER",$msg)

but _ReplaceStringInFile does not seem to work on a binary blog. Any ideas?

Edited October 1, 2008 by Irongeek

[Pain]

FileOpen in binary mode.

[Irongeek]

Thanks, but I'm still working on figuring out how to search it and save it out without screwing it up. If anyone has done something like this before, it would help if I could see the code.

[Pain]

$find = "4d53465420352e30"

$replace = "12345678"

$filename = "C:\Documents and Settings\me\Desktop\dhcpcsvc.dll"

$new = "dhcpcsvc.dll"

$file = FileOpen(filename , 16)

$chars = FileRead($file , FileGetSize($file))

$replaced = StringReplace($chars, $find, $replace)

FileWrite($new, $replaced) ;create new updated file

Edited October 1, 2008 by Pain

[Irongeek]

$find = "4d53465420352e30"

$replace = "12345678"

$filename = "C:\Documents and Settings\me\Desktop\dhcpcsvc.dll"

$new = "dhcpcsvc.dll"

$file = FileOpen(filename , 16)

$chars = FileRead($file , FileGetSize($file))

$replaced = StringReplace($chars, $find, $replace)

FileWrite($new, $replaced) ;create new updated file

Thanks, but that seems to just make a blank output file. This comes closer:

#include <string.au3>
;Base on work from piccaso

$line = "4d53465420352e30"
$lrepl= "4675636b20204954"

$file = FileOpenDialog("", @WorkingDir, "All (*.dll)", 1)
If @error Then Exit -1
$filesize = FileGetSize($file)
$data = FileRead($file)
If Not IsBinary ($data) Then $data = Binary ($data)
FileMove($file, "*.bak")
$hex = Hex($data)
;If StringInStr($hex,"55505830") And StringInStr($hex,"55505831") And StringInStr($hex,"55505821") Then
;    ConsoleWrite("Would you de-upx it for me?" & @LF)
;    Exit -2
;EndIf
$hex = StringReplace($hex,$line,$lrepl)
If @extended = 1 Then
    ConsoleWrite("Done" & @LF)
Else
    ConsoleWrite("Something bad happend with hex replace!" & @LF)
EndIf
FileWrite($file,Binary("0x" & $hex))
If FileGetSize($file) <> $filesize Then ConsoleWrite("Bad Filesize")

But the filesize comes back wrong for some reason.

[Irongeek]

Ok, I figured out that the above code double wrote to the file, this one works:

#include <string.au3>
;Base on work from piccaso

$line = "4d53465420352e30"
$lrepl= "4675636b20204954"

$file = FileOpenDialog("", @WorkingDir, "All (*.dll)", 1)
If @error Then Exit -1
$filesize = FileGetSize($file)
$data = FileRead($file)
If Not IsBinary ($data) Then $data = Binary ($data)
FileMove($file, "*.bak")
$hex = Hex($data)
;If StringInStr($hex,"55505830") And StringInStr($hex,"55505831") And StringInStr($hex,"55505821") Then
;    ConsoleWrite("Would you de-upx it for me?" & @LF)
;    Exit -2
;EndIf
$hex = StringReplace($hex,$line,$lrepl)
If @extended = 1 Then
    ConsoleWrite("Done" & @LF)
Else
    ConsoleWrite("Something bad happend with hex replace!" & @LF)
EndIf
FileDelete("patched.dll")
FileWrite("patched.dll",Binary("0x" & $hex))
If FileGetSize($file) <> $filesize Then ConsoleWrite("Bad Filesize")

Thanks for the help.