Jump to content

Implementing an equivalent of DllCall() in C++


Recommended Posts

Hi all,

One of my projects requires the use of dynamically calling a Dll where the name, or parameters of the Dll are not known at compile time. In other words, I'm trying to implement the dynamic functionality of DllCall().

This is a challenge I'm not quite sure how to approach.

Loading up the the Dll and getting the address of the function is trivial, it's just pushing the values to the stack, and getting the return values that would be the problem.

I have a feeling I will need assembly.

Does anyone know how I can do this? I simply have no idea where to start. If anyone has any example code on this it would be most appreciated :oops:

Thank you very much in advance,

Hyperzap

ongoing projects:-firestorm: Largescale P2P Social NetworkCompleted Autoit Programs/Scripts: Variable Pickler | Networked Streaming Audio (in pure autoIT) | firenet p2p web messenger | Proxy Checker | Dynamic Execute() Code Generator | P2P UDF | Graph Theory Proof of Concept - Breadth First search

Link to comment
Share on other sites

Both cdecl and stdcall. I need support for both.

ongoing projects:-firestorm: Largescale P2P Social NetworkCompleted Autoit Programs/Scripts: Variable Pickler | Networked Streaming Audio (in pure autoIT) | firenet p2p web messenger | Proxy Checker | Dynamic Execute() Code Generator | P2P UDF | Graph Theory Proof of Concept - Breadth First search

Link to comment
Share on other sites

You shouldn't need assembly, provided you are using a language where you can call function pointers. It may take a bit of work to write something the complier allows though. Simple summary of function pointers in C:

http://publications.gbdirect.co.uk/c_book/chapter5/function_pointers.html

For a quick guide to stack and major calling conventions I'd go here:

http://www.unixwiz.net/techtips/win32-callconv-asm.html

Or use AutoIt.

Edit: For using calling conventions with function pointers, there is no standard way to do it, but see here: http://www.newty.de/fpt/fpt.html

Edited by Mat
Link to comment
Share on other sites

You shouldn't need assembly, provided you are using a language where you can call function pointers. It may take a bit of work to write something the complier allows though. Simple summary of function pointers in C:

http://publications.gbdirect.co.uk/c_book/chapter5/function_pointers.html

For a quick guide to stack and major calling conventions I'd go here:

http://www.unixwiz.net/techtips/win32-callconv-asm.html

Or use AutoIt.

Edit: For using calling conventions with function pointers, there is no standard way to do it, but see here: http://www.newty.de/fpt/fpt.html

The problem is not calling the function. The problem is calling the function dynamically, and passing the arguments at runtime. The arguments are not known at compile time, so I cannot simply create a function pointer with the relevant arguments.

ongoing projects:-firestorm: Largescale P2P Social NetworkCompleted Autoit Programs/Scripts: Variable Pickler | Networked Streaming Audio (in pure autoIT) | firenet p2p web messenger | Proxy Checker | Dynamic Execute() Code Generator | P2P UDF | Graph Theory Proof of Concept - Breadth First search

Link to comment
Share on other sites

I'm working on something you maybe could use. Sec.

Thank you :oops:

Edited by twitchyliquid64

ongoing projects:-firestorm: Largescale P2P Social NetworkCompleted Autoit Programs/Scripts: Variable Pickler | Networked Streaming Audio (in pure autoIT) | firenet p2p web messenger | Proxy Checker | Dynamic Execute() Code Generator | P2P UDF | Graph Theory Proof of Concept - Breadth First search

Link to comment
Share on other sites

Okay, here goes. First off, this is nontested, and just so you can get an idea of how to implement it. I think it works though lol. There are some comments around, but basically, you need to assemble asm in your function, you need to know convention, return type, etc. etc. It's not that hard to do with standard types, but floats and doubles use other registers and stacks, so you might wanna read up on that. functions, that returns structs, is usually very implementation defined and often optimized away so you probably wont have luck with that.

All in all, if you are not 100% about what you do, this is not the road you wanna go down if you need a stable system. Nevertheless, this is quite fun :doh:

//by shaggi: quick dirty nontested example of dllcall
//x86 ofc
//only works with very primitive types and stdcall atm
//should be used like:
#include <windows.h>
#include <iostream>
/*
    int args [] = {1, 2};
    int result;
    result_t res = DllCall<rsint>(ret_t::rint, conv_t::cstdcall, (void*)0xDEADBEEF, ARRAYSIZE(args), args, & result);
*/

enum conv_t
{
    ccdecl,
    cstdcall,
    cfastcall,
    cthiscall
};

enum ret_t
{
    rint,
    rsint,
    rfloat,
    rdouble,
    rptr
};

enum result_t
{
    success,
    bad_address,
    bad_arguments,
    bad_return,
    nothing_happened
};
#define OPCODE_PUSH_SIZE 0x1
#define OPCODE_CALL_SIZE 0x1
#define CALL_ASM_SIZE 0x5
#define MOV_PTR_EAX_SIZE 0x5
#define RETN_SIZE 0x1
#define OPCODE_RETN 0xC3
#define POS_BUF 0x10
#define OPCODE_PUSH 0x68
#define OPCODE_CALL 0xE8
#define OPCODE_MOV_DWORD_PTR_FROM_EAX 0xA3
#define OPCODE_MOV_DWORD_PTR_FROM_EAX_SIZE 1
#define OPCODE_MOV_EAX 0xB8
#define MOV_EAX_SIZE 0x1
#define OPCODE_CALL_EAX 0xD0FF
#define CALL_EAX_SIZE 0x2
#define MOV_EAX_TOTAL_SIZE 0x5
//this might be wrong can't remember lol.
#define addressOf(abs, eip) 
    ((abs - eip) - CALL_ASM_SIZE)

typedef void (_stdcall * empty_func)();

template<typename T> //retain SOME type security :p

result_t DllCall(ret_t return_type, conv_t convention, void * address, int argc, int args [], T * _return)
{
    /* some checks */
    if(!address)
        return bad_address;
    if(argc && !args)
        return bad_arguments;
    if(!_return)
        return bad_return;
    /* now we need to compile our own assembly, and floating point types differs a lot from standard dwords
        so we split here */


    switch (return_type)
    {
        case ret_t::rint:
        case ret_t::rsint:
        case ret_t::rptr:
            /* these look quite similar for a start */
        {
            /* next up, conventions differ. */
            switch(convention)
            {
                case conv_t::cstdcall:
                /* in stdcall, callee cleans up so we dont have to worry about that.
                    arguments pushed right to left (?) */
                {
                    /* so now we need a buffer. it needs to be:
                        (sizeof(native_int) + push_size) * argc +
                        mov eax, call_address (5)
call eax(2)
                        mov [void_buf], eax; // should be 5, cant remember
                        return (always just ret, we define the asm as taking no arguments and stdcall
                    */

volatile DWORD buf = 0; // <- this will hold the return type
volatile DWORD * void_buf = &buf;

                    int iSize = (sizeof(int) + OPCODE_PUSH_SIZE) * argc + MOV_EAX_TOTAL_SIZE + CALL_EAX_SIZE + MOV_PTR_EAX_SIZE + RETN_SIZE + POS_BUF;

                    BYTE * asm_buf = (BYTE*)VirtualAlloc(NULL,iSize,MEM_RESERVE | MEM_COMMIT, PAGE_EXECUTE_READWRITE);
                    //now, asm_buf points to out little function
int temp = 0;
                    for(int i = argc - 1; i >= 0; i--) {
                        //Here we fill in push, and the param for each argc
                        *(BYTE*) & asm_buf[i * 5] = OPCODE_PUSH;
                        *(DWORD*) & asm_buf[i * 5 + OPCODE_PUSH_SIZE] = args[temp++];
                    }
                    int args_offset = argc ? argc * 5 : 0 ;
                    // assemble mov eax, address
                    *(BYTE*) & asm_buf[args_offset] = OPCODE_MOV_EAX;
                    *(DWORD*) & asm_buf[args_offset + MOV_EAX_SIZE] = (DWORD)address;

//assemble call eax
*(WORD*) & asm_buf[args_offset + MOV_EAX_TOTAL_SIZE] = (WORD)OPCODE_CALL_EAX;

int current_offset = args_offset + MOV_EAX_TOTAL_SIZE + CALL_EAX_SIZE;
                    // assemble mov [void_buf], eax
                    *(BYTE*) & asm_buf[current_offset] = OPCODE_MOV_DWORD_PTR_FROM_EAX;

                    *(DWORD*) & asm_buf[current_offset + OPCODE_MOV_DWORD_PTR_FROM_EAX_SIZE] = reinterpret_cast<DWORD>(void_buf);

                    *(BYTE*) & asm_buf[current_offset + MOV_PTR_EAX_SIZE] = OPCODE_RETN;
                    /* now we have hopefully assembled a small function that looks like this:
                        for each arg in args
                            push arg
mov eax, address
                        call eax
                        mov [void_buf], eax
                        retn 0

                        lets call it! lol
                    */

                    empty_func  asm_func = (empty_func)asm_buf;

                    (*asm_func)(); //omg

                    // now buf should hold the return value...

                    *_return = (T) buf;

                    //all done! now clean up
VirtualFree(asm_buf,iSize,MEM_DECOMMIT);

                    //and return
                    return result_t::success;
                }




            }


            break;
        }

        case ret_t::rfloat:
        {

            break;
        }
        case ret_t::rdouble:
        {


            break;
        }

    }



    return result_t::nothing_happened;
}

int __stdcall func_one(int a, int b, int c) {
    std::cout << "a: " << a  << ", b: " << b << ", c: " << c << std::endl;
    return a + b + c;
}

int main() {
    int args[] = {1,2,3};
    int result;
    result_t res = DllCall<signed int>(ret_t::rsint, conv_t::cstdcall, reinterpret_cast<void*>(func_one), ARRAYSIZE(args), args, & result);

    std::cout << "result of DllCall: " << res << ", returned " << (int)result << std::endl;
    system("pause");

return 0;
}

E: if you're having a hard time reading this, i suggest you look at my processcall udf, which does the same thing, only in autoit :bye:

E2: small error calculating the relative address, fixed

E3: It actually works now :oops:

Edited by Shaggi

Ever wanted to call functions in another process? ProcessCall UDFConsole stuff: Console UDFC Preprocessor for AutoIt OMG

Link to comment
Share on other sites

Okay, here goes. First off, this is nontested, and just so you can get an idea of how to implement it. I think it works though lol. There are some comments around, but basically, you need to assemble asm in your function, you need to know convention, return type, etc. etc. It's not that hard to do with standard types, but floats and doubles use other registers and stacks, so you might wanna read up on that. functions, that returns structs, is usually very implementation defined and often optimized away so you probably wont have luck with that.

All in all, if you are not 100% about what you do, this is not the road you wanna go down if you need a stable system. Nevertheless, this is quite fun :bye:

//by shaggi: quick dirty nontested example of dllcall

//x86 ofc
//only works with very primitive types and stdcall atm
//should be used like:

/*
    int args [] = {1, 2};
    int result;
    result_t res = DllCall<rsint>(ret_t::rint, conv_t::cstdcall, (void*)0xDEADBEEF, ARRAYSIZE(args), args, &amp; result);
*/

enum conv_t
{
    ccdecl,
    cstdcall,
    cfastcall,
    cthiscall
};

enum ret_t
{
    rint,
    rsint
    rfloat,
    rdouble,
    rptr
};

enum result_t
{
    success,
    bad_address,
    bad_arguments,
    bad_return,
    nothing_happened
};
#define OPCODE_PUSH_SIZE 0x1
#define OPCODE_CALL_SIZE 0x1
#define CALL_ASM_SIZE 0x5
#define MOV_PTR_EAX_SIZE 0x5
#define RETN_SIZE 0x1
#define OPCODE_RETN 0xC3
#define POS_BUF 0x10
#define OPCODE_PUSH 0x68
#define OPCODE_CALL 0xE8
#define OPCODE_MOV_DWORD_PTR_FROM_EAX 0xA3
#define OPCODE_MOV_DWORD_PTR_FROM_EAX_SIZE 0xA3

//this might be wrong can't remember lol.
#define addressOf(abs, eip) 
    ((abs - eip) - CALL_ASM_SIZE)

typedef void (_stdcall * empty_func)();

template<typename T> //retain SOME type security :p

result_t DllCall(ret_t return_type, conv_t convention, void * address, int argc, int args [], T * _return)
{
    /* some checks */
    if(!address)
        return bad_address;
    if(argc &amp;&amp; !args)
        return bad_arguments;
    if(!_return)
        return bad_return;
    /* now we need to compile our own assembly, and floating point types differs a lot from standard dwords
        so we split here */

    DWORD buf; // <- this will hold the return type
    DWORD * void_buf = &amp; buf;
    switch (return_type)
    {
        case ret_t::rint:
        case ret_t::sint:
        case ret_t::rptr:
            /* these look quite similar for a start */
        {
            /* next up, conventions differ. */
            switch(convention)
            {
                case conv_t::cstdcall:
                /* in stdcall, callee cleans up so we dont have to worry about that.
                    arguments pushed right to left (?) */
                {
                    /* so now we need a buffer. it needs to be:
                        (sizeof(native_int) + push_size) * argc +
                        call address (constant, 5 bytes) +
                        mov [void_buf], eax; // should be 5, cant remember
                        return (always just ret, we define the asm as taking no arguments and stdcall
                    */
                    int iSize = (sizeof(int) + OPCODE_PUSH_SIZE) * argc + CALL_ASM_SIZE + MOV_PTR_EAX_SIZE + RETN_SIZE + POS_BUF;

                    BYTE * asm_buf = new BYTE[iSize];

                    //now, asm_buf points to out little function

                    for(int i = argc; i ; i--) {
                        //Here we fill in push, and the param for each argc
                        *(BYTE*) &amp; asm_buf[i * 5] = OPCODE_PUSH;
                        *(DWORD*) &amp; asm_buf[i * 5 + OPCODE_PUSH_SIZE] = args[i];
                    }
                    int args_offset = argc ? 0 : argc * 5 + 1;
                    // assemble CALL address
                    *(BYTE*) &amp; asm_buf[args_offset] = OPCODE_CALL;
                    *(DWORD*) &amp; asm_buf[args_offset + OPCODE_CALL_SIZE] = (DWORD*)address;

                    // assemble mov [void_buf], eax
                    *(BYTE*) &amp; asm_buf[args_offset + CALL_ASM_SIZE] = OPCODE_MOV_DWORD_PTR_FROM_EAX;

                    *(DWORD*) &amp; asm_buf[args_offset
                                        + CALL_ASM_SIZE
                    + OPCODE_MOV_DWORD_PTR_FROM_EAX_SIZE] = void_buf;

                    *(BYTE*) &amp; asm_buf[args_offset + CALL_ASM_SIZE + MOV_PTR_EAX_SIZE] = OPCODE_RETN;
                    /* now we have hopefully assembled a small function that looks like this:
                        for each arg in args
                            push arg
                        call address
                        mov [void_buf], eax
                        retn 0

                        lets call it! lol
                    */

                    empty_func * asm_func = (empty_func*)asm_buf;

                    asm_func(); //omg

                    // now buf should hold the return value...

                    *_return = (T) buf;

                    //all done! now clean up

                    delete[] asm_func;

                    //and return
                    return result_t::success;
                }




            }


            break;
        }

        case ret_t::rfloat:
        {

            break;
        }
        case ret_t::fdouble:
        {


            break;
        }

    }



    return result_t::nothing_happened;
}

E: if you're having a hard time reading this, i suggest you look at my processcall udf, which does the same thing, only in autoit :oops:

Hang on a sec ... Your generating an asm stub ... ON THE FLY to deal with dynamic calls. THAT'S FUCKING AWESOME! thanks, i hope autoit does it in just as cool a way :doh:

ongoing projects:-firestorm: Largescale P2P Social NetworkCompleted Autoit Programs/Scripts: Variable Pickler | Networked Streaming Audio (in pure autoIT) | firenet p2p web messenger | Proxy Checker | Dynamic Execute() Code Generator | P2P UDF | Graph Theory Proof of Concept - Breadth First search

Link to comment
Share on other sites

Hang on a sec ... Your generating an asm stub ... ON THE FLY to deal with dynamic calls. THAT'S FUCKING AWESOME! thanks, i hope autoit does it in just as cool a way :oops:

I think they did at some point, but hey... who knows ^^ maybe devs could give some insight/help?

btw beware, it seems quation of the code returns invalid code...

// assemble mov [void_buf], eax                  *(BYTE*) &amp; asm_buf[args_offset + CALL_ASM_SIZE] = OPCODE_MOV_DWORD_PTR_FROM_EAX;

compared to:

// assemble mov [void_buf], eax                  *(BYTE*) & asm_buf[args_offset + CALL_ASM_SIZE] = OPCODE_MOV_DWORD_PTR_FROM_EAX;
Edited by Shaggi

Ever wanted to call functions in another process? ProcessCall UDFConsole stuff: Console UDFC Preprocessor for AutoIt OMG

Link to comment
Share on other sites

By the way, C/invoke is a quite complete implementation for runtime binding: https://bitbucket.org/bogen/cinvoke/ It even has a mechanism similar to DLLStructCreate.

Edited by ProgAndy

*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Link to comment
Share on other sites

I'm curious as to how you could possibly deal with arbitrary function prototypes and still know what to pass it and what it gives back.

You have to know the calling convention, the size and the type of the parameters. Then you use asm to push the data in the correct order on the stack and fill the necessary registers. Now call the function pointer and finally do the cleanup required by the calling convention.

Edit: libffi is another alternative to C/invoke: http://sourceware.org/libffi/

Edited by ProgAndy

*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Link to comment
Share on other sites

You have to know the calling convention, the size and the type of the parameters. Then you use asm to push the data in the correct order on the stack and fill the necessary registers. Now call the function pointer and finally do the cleanup required by the calling convention.

Edit: libffi is another alternative to C/invoke: http://sourceware.org/libffi/

I understand how to call arbitrary things. My question is how is his compiled program going to know what to execute in the future? That's my question.
Link to comment
Share on other sites

The OP is the one I intended to answer the question.

Its pretty much for a scripting language interpeter. Thats the easiest way of thinking of it.

ongoing projects:-firestorm: Largescale P2P Social NetworkCompleted Autoit Programs/Scripts: Variable Pickler | Networked Streaming Audio (in pure autoIT) | firenet p2p web messenger | Proxy Checker | Dynamic Execute() Code Generator | P2P UDF | Graph Theory Proof of Concept - Breadth First search

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...