VMWare, Dll profiling, and an idea

[JRowe]

So, I know that getting the functions in a dll, then iterating through the parameters until I get a successful DllCall would be a dumb thing to do, because there's no telling what a function will do (like format your hard drive, or something similiarly horrible.)

I was thinking, though, that inside a virtual box, you could automate combinations of parameters, returns, and so on, until you got a working DllCall (@error = 0,) and have it output a list of properly formatted DllCalls, ready for you to turn into a usable UDF.

You could up a restricted directory beforehand, then specify expected file extensions for reads and writes, maybe set up a valid file in the directory for the dll to use. I'd block internet access, but otherwise only have windows and the appropriate dll installed to the VM.

I'm basing the whole scenario on the idea that you'd be using a dll from a known source that's verified to work in the way it's intended, not something randomly downloaded. I think that, ultimately, this would make using dllcall a bit easier, but I wanted to get an opinion before I try it. Is there anything dangerous that could happen, from inside the VM?

[Nutster]

Is there a DLL you can not find docs for? There are a couple of tools available to identify the calling interface of a DLL. Use one of them, or find the documentation for the DLL.

I recommend against just poking around with DLL's, especially system DLL's, just willy-nilly. Poking around at random settings for DLL's could cause major problems, even within a virtual environment. You do have a way of regenerating your inner OS, right? Just in case you break things badly? "Oh, so that is what happens when you wipe out an entire hive in the registry." Do not let this happen to you.

[Authenticity]

Well, REing the dll with an offline internet connectivity inside VM can be better. But I'll stick with Nutster. If you find a function that should not be called by the outside world other than the dll's own code then you shouldn't call it. Maybe it's a function the get called only by the dll's public interface, or maybe the function bases it's job heavily on the value of the return type of some other private functions with it's internal structure, then you should RE them as well. Just me thinking/

Edited October 20, 2009 by Authenticity

[monoceres]

JRowe, in all respect, the idea is really stupid. It will never work. Lets take an example. I'll give you this dll:

external "C" __declspec(dllexport) void* get_something(HMODULE module, const char* name)
{
 return GetProcAddress(module,name);
}

There's no way for you to determine how the parameters are made up by "brute-forcing" parameters. @error won't give you shit. It only tells you if somethings wrong with the dll, the export isn't found or stuff like that. Secondly, stack corruption (sign for wrong size of parameters) will not take place in several situations.

• DllCall("dll.dll","none:cdecl","get_something","ushort","","ushort","","ushort","","ushort")
• DllCall("dll.dll","int:cdecl","get_something","dword*","","dword","","dword","","dword","","dword","")
• DllCall("dll.dll","int","get_something")
• And the list goes on.

How would you determine which is correct? Also, what input are you going to supply to the dll? Most functions (just like my example) will give access violations and more funky exceptions when giving bad input.

If you wanna know how to call unknown dll's disassembly them or the programs that uses them. It's the only way.

Edited October 20, 2009 by monoceres

[JRowe]

There wasn't a particular dll, it's just that I was trying to automate the process I went through with some of the dlls lately. This was just me trying to think of a way to be lazy next time I find a dll I want to work with.

Maybe a doc-parsing tool would be better, to generate function frameworks from regular expressions customized to a particular set of docs. It's not so much that dllCall is hard, it's the time it takes to get it working right, and I thought that if I could generate a bunch of ready to work with DllCalls straight from a dll, it would speed up the process. Back to the drawing board!

I'm probably bouncing around ideas like a rabbit on crack, but that's nothing new, I suppose. Thanks for the input!

[Richard Robertson]

It's best to only use well documented dlls. Safer that way too.