Hex | per byte binary file read & write | search & replace function via Autoit?

[Voodooman]

Hi, Im making a patcher for games with Autoit and i need to implement per byte patching, can you please provide me any suggestions and script examples?

To be more specific, i need to do this:

1)  universal function that will search all instances of specific HEX values in binary files and will return array that contains
[1] - number of instances found in binary files [2] - instance number [3] - starting offset of  N instance [4]  ending offset of N instance

array should better be mulidimensional

2) search should support wildcards  ?? 

values like this [?0]  which will search for 00 10 20 ... E0 F0  and any byte thats ends by 0 
in case of [0?] stars by 0 and ends by anything from 0-F

double [??] should match entire 00-FF range 
 

3) universal function that will accept at least 3 input modes to edit binary file: [1] cut number of specified bytes before or after specified offset [2] add extra bytes after or before specified offset [3] replace number of bytes with specified bytes - this mode should be smart enough to detect number of bytes to be replaced without input of this number, it also should repspect wildcard values and leave wildcard values unpatches

lets say we have this search pattern,

AA BB CC DD EE FF

from function 1 we already know where it starts and ends,

so we use starting offset value to start replacement, but we need to get number of replace bytes 1st and compare every byte value delimited to left and right parts before applying patch

this is our replace pattern:

BB ?C ?? AA A? 00

So in result we should have this

BB C [CC] AA A[E] FF

[ ] - bytes preserved by wildcard

If search pattern included wildcards as well, that we can have something like this

?? BB ?C DD ?E FF

replaced by this 

BB ?C ?? AA A? 00

and result will be this

BB [?]C [?C] AA A[E] FF

[?] - same as above but in this case its not specified during seach and comparision and could be anything

----

Search and replace values could be of diffent lengh, whic means if we only search for 5 specific bytes, we can enter replace value that will change 10 bytes or only 3

for example

Search for  

?? BB ?C DD ?E FF

Replace by 

BB ?C ?? AA A? 00 BB ?C ?? AA A? 00

Means that next 5 bytes after original search pattern will be replaced as well and anything at replative position of ? or ?? wil be preserved as is

And we would have in result

BB [?]C [?C] AA A[E] FF BB [?]C [??] AA A[?] FF - note the difference of second part, instead of specific bytes we have ? which means whatever value was in this place originally

4) File should have optionally has creation and modification date preserved as originally was, or replaced with specified, or use default automatic date handling, yet again this should be separate function

5) Search and replace patterns should support ANSI UTF-8 chars as well, mixed syntax would be great. For example we can ditinguish HEX from ansi by counting number of symbols in row and case.

obviously anything like AA should be treated as hex, while A or a as ansi and case insensetive or sensation could be forced with extra parameter. 

This is especially helpfull to search and replace values like 

S.E.A.R.C.H. where . = 00 in hex

so we can do the search with string like this

s 00 e 00 a 00 r 00 c 00 h 00

or this

s ?? e ?? a ?? r ?? c ?? h ?

Note - if we need to search for real ?? we can still use HEX or use [??] as indication of actual ansi string, actually brakets could be use optionally for any ansi chars to write them without spaces, real space should be found by hex 20 like this s ?? 20 ?? a ?? r ?? c ?? h, if ansi string is no not mixed with HEX in could be written without extra spaces between characters and with brakets at start and end like this [search]

6) Any large files should be supported be it 1 gb or 10 gb

7) read of file should be segmented into small streamed chunks if possible, we dont want to load entire 2 or 6 gb file in ram, expeciialy when we are limited to 2-3gb per process on 32 bit machines without usage of AWE

8) WinApi Dllcalls should be used as less as possible or not used at all, i prefer native Autoit implementation than dealing with memory leaks Autoit tends to have with Dllcalls.
Undocumented hacks and tricks via WinApi should not be used as well, i dont want patcher to need a patch when MS will brake another api compatibility with new windows or update, also i want it work on any windows since XP and up to 8.1, also i want it to work under wine without extra headaches to end users.

9) If there is some well coded free to use dll available that has this hex search functionality to make things faster and easier, its ok to use it as sort of patching engine.

10) Oh almost forgot this - i dont want patch to copy original file and patch copy and then replace original like some byte patchers does, temp files especially if original is few GB in size on little SSD (like i currently have, because my 2 TB HDD broken atm) unacceptable and should be avoided at any cost, original files must be patched without taking any extra space.

11) Function to read files SHA-1 MD5 checksum and file creation date without external tools to get idea that we dealing with correct file before we start to search for pattern

12) ability to specify number of instances to replace, something like 1st one, last one, 1st two, 1st and third. last 2, number 8, all of them etc. you got the idea now

13) Function to analyze file header and compare it with template to get idea about file type we dealing with. For example i want to tell tell the difference between Win 32 PE and DOS 16 bit MZ (in case if dos exe or com detected im going to use patcher for some DOS games and installers as well to autowrap them into 32 or 64 bit emulator to run natively in 64 bit windows to compensate total lack of already crappy NTVDM, ), however this can be implemented once i learn how to read binary files per byte and do it fast.

14) And last thing im thinking about now - everything should work fast.

 

You may ask, why im not using any available patch engines? they all have flaws and does not have all features i need in one single solution, most of them dont allow me to use custom made UI, or if they does (dup), i cant add any custom functonality such as control of file version and checksum verification (xdelta corrupts files 1st, ask second).

I probably that should have started this patcher with ASM or C for better results, but i have no space and time to deal with MS studio and all of these monstrous dev envs loaded with crapware i dont need, and learning again long forgotten syntax and specs for making quite simple tools is kinda overkill of time, autoit easier and faster to learn despite some flaws, also i already started patcher with Autoit a while ago, and did almost anything i need except for what i described above.

So please, feel free to contribute ideas and portions of code to implement at least part of this. I believe such AutoIt patching functions would be usefull not only for me, but for many people as well.

[Melba23]

Voodooman,

Welcome to the AutoIt forum . I bet that post took some time to write - a pity it was completely wasted.

 

Im making a patcher for games with Autoit

Unfortunately you appear to have missed the Forum rules on your way in. Please read them now (there is also a link at bottom right of each page) - particularly the bits about not discussing game interaction and running or injecting any code (in any form) intended to alter the original functionality of another process - and then you will understand why you will get no help and this thread will now be locked.

See you soon with a legitimate question I hope.

M23