DllCall with C++ not returning correct values &/| not working

[DiDawson]

I have been scanning the forum threads with a little success, but haven't found the answer to the issue I have. I'm sure this has been asked & answered before, but here goes.

I am the developer of a DLL and I want to use AutoIT to automate some unit testing.

by directly calling some of the methods I have written. I am new to AutoIT but have been a developer for a long time.

The issue I have is that when I use DllCall() in my script

- It works fine for very basic C++ methods I have written (specifically for use by my AutoIT script)

For example, my DLL will expose a method called 'GetSomeIDThatINeed()'.

Now GetSomeIDThatINeed() is a completely trivial method that returns the value of a #define from the .h file.

The script code looks similar to

$res = DllCall($MyDll, "ULONG:cdecl", '?GetButtonID@CMyClass@@QAEHK@Z')

$buttonID = $res[0]

This works fine - everytime (You can assume $MyDll is valid & I am in the correct directory...etc)

The problem is now I want to call a method where the return value is dynamic.

e.g. it may be true or false at various times while the program is running.

GetTheCurrentState(ULONG id)

The script code looks similar to

$res = DllCall($MyDll, "BOOL:cdecl", '?GetCurrentState@CMyClass@@QAEHK@Z', "ULONG", $buttonID)

$bCurrentStat = $res[0]

The call 'succeeds' in that there are no errors returned, but the returned value is always the same

i.e. the return value is bogus.

GetTheCurrentState(ULONG id) is a more complex C++ method that that returns true or false based on a set of 3 variables. For example, I know the method should be returning false, but it still returns true.

The method runs correctly if I just run the application without AutoIT - the problem only occurs when I run the few lines of simple script.

Obviously, I'm using the undecorated C++ methods exposed by my DLL, which I think is legitimate.

I'll keep searching the forum, but thanks in advance.

Di.

[funkey]

Never had such a problem. And I do a lot of this things.

But try to use the ordinal value instead of the function name to call the function, just to be sure this isn't the problem.

[DiDawson]

ThHi, Thanks. It's not the Ordinal vs. undecorated function.

I have made some progress though.

Basically here's what works and what doesn't

1. If I use DllCall() and the method doesn't make any Afx***/MFC calls, it works

2. If I use DllCall() and the method makes Afx*** calls, it will often fail

- Eventually (after ASSERTS) AfxGetResourceHandle() will fail

- So basically, the act of Loading the DLL from the Script does not

give the same result as just running the EXE

3. Also, member variables that are changed/initialized before the DLL is loaded

do not pick up those (new) values.

I'm guessing this is an artifact of the way DllCall loads the DLL into memory

or a memory mapping issue or something similar.

I'm going to try various combinations of things I've read about, but if anyone has a suggestion,

thanks in advance.

[trancexx]

What exactly is your $MyDll? How is that declared and initialized?

[DiDawson]

$MyDLL is one DLL (of quite a few) in a large program.

It is using MFC as a Shared DLL.

It is not delay loaded or anything unusual. It's loaded at run-time, similar to any other DLL.

Does this answer the question?

[trancexx]

No it doesn't.

Use Dllopen() to load the dll and pass returned handle to DllCall's, just in case you pass dll name.

[DiDawson]

I've tried it both ways - using DllOpen() and just passing in the $DllName.

Either way - the proper C++ function is called - I can step into it in a debugger.

The problem is that it doesn't do the right thing when called from a script, even though it does the right thing if you just run the EXE (and call that same method in the normal course of running the problem)

When the method in question is called from the script, I get Afx and/or MFC errors.

[trancexx]

You are missing something, probably something obvious. Maybe you forgot to call some initialization function or something. As far as AutoIt is concerned you shouldn't have problems with DllCall() function because there isn't any magic involved with it. It simply calls GetProcAddress WinAPI function to get function pointer and then it calls it passing arguments you specified during the call.

If you want more precise help provide more info (hopefully in form of your code).

[DiDawson]

I'm uploading the script and the C++ code snippets. Hope it helps.

I'm also pasting them into this post.

I've tried all combos of str/wstr and with/without :cdecl

;; -------------------------- Script Start --------------------------

;;

$App = 'TestApp'

$Title = 'AppTitle'

$MyDLL = 'C:projectsMyAppMyApp.dll'

FileChangeDir("C:projectsMyAppMyApp.dll") ;; Required to find other dependent DLL's

Local $dllHnd = DllOpen($MyDLL)

Run($App)

WinWaitActive($Title)

$MsgStr = ""

;; these are the correct values

Local $SimpleTest = 0x10D2 ;

Local $ComplexTest = 0x10D8 ;

;; --------------------------

;; This works fine

;; --------------------------

$res = DllCall($dllHnd, "str", $SimpleTest)

if @error Then

$MsgStr = StringFormat("String test error %d", @error)

MsgBox(0,"Get Button Char Error", $MsgStr)

Else

$rv = $res[0]

$st = StringFormat("String test was %s", $rv)

MsgBox(0,"First test Results", $st)

EndIf

;; --------------------------

;; This doesn't work (no errors but wrong result)

;; --------------------------

$res = DllCall($dllHnd, "str", $ComplexTest)

if @error Then

$MsgStr = StringFormat("Get Button Strings test error %d", @error)

MsgBox(0,"Get Button Char Error", $MsgStr)

Else

$rv = $res[0]

$st = StringFormat("String test was %s", $rv)

MsgBox(0,"Second test Results", $st)

EndIf

DllClose($dllHnd)

;; -------------------------- end of script --------------------------

// ------------------------------- C / C++ code --------------------------------

#define MAX_AUTO_IT_CHARS (64*1024)

...

static char g_chAutoITbuf[MAX_AUTO_IT_CHARS]; // 64k buffer

// this works fine & returns the expected result - namely

// "Testing ability to return a string to AutoIT from C++"

CHAR *OMyClass::SimpleCharTest()

{

_snprintf(g_chAutoITbuf,

MAX_AUTO_IT_CHARS,

"Testing ability to return a string to AutoIT from C++");

return g_chAutoITbuf;

}

// This does not generate any errors in the Auto-IT Script.

// However it does not return the proper result.

// And, if you run the Debug Version, you will get ASSERTS in

// AfxTempl.h and AfxWin1.inl

//

CHAR *OMyClass::ComplexCharTest()

{

// This is where I need the code to go out and do a bunch of stuff

// and place the results into a string

// If I call this method from the EXE directly - it works as expected

// If I call it from DllCall() via AutoIT, it doesn't work

CString str = DoMFCandATLStuffHere();

_snprintf(g_chAutoITbuf, AUTO_IT_MAX_CHAR_RETURN_SZ, "%s", str);

return g_chAutoITbuf;

}

// ------------------------------- End C / C++ code --------------------------------

Script.txt

C_Code.txt

Edited March 28, 2013 by DiDawson

[trancexx]

I'm telling you, you are missing some initialization routines inside the dll.

Besides, what EXE are you talking about?