C++ DLL and AutoIt causing crash

[Chance]

I'm trying to play around with making dlls, but apparently things are crashing, I suspect it's due to the way I'm passing it information.

$Struc = DllStructCreate("byte[255]")
DllStructSetData($Struc, 1, Binary("AutoIt"))
$Test = DllCall("test.dll", "none", "CrashAutoIt", "ptr", DllStructGetPtr($Struc, 1))

Compilable example below

#include <windows.h>

#ifndef _DLLTUT_DLL_H_
#define _DLLTUT_DLL_H_

#define DLL_EXPORT __declspec(dllexport)

extern "C"
{
    DLL_EXPORT void CrashAutoIt(LPVOID pBinary);
}

#endif


void Crash(LPVOID pFile);

extern "C"
{
    DLL_EXPORT void CrashAutoIt(LPVOID pBinary)
    {
        if (pBinary) {
            Crash(pBinary);
        }
    }
}

void Crash(LPVOID pFile)
{
    // ( >._.)> just here to show the call worked
    // in actuall application, I do some stuff here with binary data <(._.< )
    MessageBox(NULL, "( >._.)> Call successful      ", "Success!", MB_ICONINFORMATION | MB_OK);
    MessageBox(NULL, "But we're going to crash now... (._.  )     ", "But...", MB_ICONINFORMATION | MB_OK);
}

What's the correct method I should be using when passing a pointer to binary data?

[Andreik]

Try

$Test = DllCall("test.dll", "none:cdecl", "CrashAutoIt", "ptr", DllStructGetPtr($Struc, 1))

[Chance]

Try

$Test = DllCall("test.dll", "none:cdecl", "CrashAutoIt", "ptr", DllStructGetPtr($Struc, 1))

Thanks, that worked!

I also have another question now...

$File = FileOpenDialog("Select a file please...", "", "Executable (*.exe)")
$Test = DllCall("test.dll", "none:cdecl", "CrashAutoIt")

If I click cancel in the dialog, the call to "CrashAutoIt" works nicely.

If I select a file and press "Ok", the DLL will not be executed ( .__.)

This is strange and confusing because I don't see the interaction causing this..

Edit: people were misunderstanding the question...

Edited November 15, 2012 by FlutterShy

[JohnOne]

Because Filopendialog is waiting for input.

[bogQ]

If I select a file and press "Ok", the DLL will not be executed ( .__.)

just tested it, works (dll call that is) with "Ok" button, double click on exe or "cancel" button

edit: guess jon is playing with the forum at this moment

Edited November 15, 2012 by bogQ

[Chance]

That's really weird, it's not working at all on my system if I press ok or double click an exe unless I click cancel or close the window.

[Andreik]

Can you attach the dll and your test code to make a try?

[bogQ]

dont FileOpenDialog win close when you doubleclick or click 'OK' button?

test.dll

Edited November 15, 2012 by bogQ

[Chance]

Can you attach the dll and your test code to make a try?

Of course, I just tried BogQ's dll and it's not working either.

Let me explain more about the problem.

I'm using AutoIt v3.3.8.1.

DllCall("test.dll", "none:cdecl", "CrashAutoIt"); gets called correctly and two message boxes appear
DllCall("test.dll", "none:cdecl", "CrashAutoIt"); gets called correctly and two message boxes appear
; depending on whether I select a file, the two following calls will not work like they did before...
FileOpenDialog("Select a file please...", "", "Executable (*.exe)"); however, if I didn't select a file, the two calls to the dll do work
ConsoleWrite("test" & @CR)
DllCall("test.dll", "none:cdecl", "CrashAutoIt"); from hear on now, these calls may or may not work and messageboxes may or may not show up depending on what actions were taken with AutoIt function FileOpenDialog()
DllCall("test.dll", "none:cdecl", "CrashAutoIt")

here is my DLL.

test.dll

Also, the code is exactly the same as above.

#include <windows.h>
#ifndef _DLLTUT_DLL_H_
#define _DLLTUT_DLL_H_
#define DLL_EXPORT __declspec(dllexport)
extern "C"
{
    DLL_EXPORT void CrashAutoIt(LPVOID pBinary);
}
#endif
void Crash(LPVOID pFile);
extern "C"
{
    DLL_EXPORT void CrashAutoIt(LPVOID pBinary)
    {
        if (pBinary) {
            Crash(pBinary);
        }
    }
}
void Crash(LPVOID pFile)
{
    // ( >._.)> just here to show the call worked
    // in actuall application, I do some stuff here with binary data <(._.< )
    MessageBox(NULL, "( >._.)> Call successful!      ", "Success!", MB_ICONINFORMATION | MB_OK);
    MessageBox(NULL, "But we're going to crash now... (._.  )     ", "But...", MB_ICONINFORMATION | MB_OK);
}

Edited November 15, 2012 by FlutterShy

[Andreik]

Hmm I tried your code and works great as it suppose to work, even if I select or not a file. What OS you have?

[Chance]

Hmm I tried your code and works great as it suppose to work, even if I select or not a file. What OS you have?

So then why doesn't it work with me?

XP X86 SP3 Intel Atom N280

[Chance]

HA!

I get it now, I checked the working dorectry in my script and it was being changed by the fileopen dialog to the place where the file selected was located...

this works.

DllCall("test.dll", "none:cdecl", "CrashAutoIt")

FileOpenDialog("Select a file please...", "", "Executable (*.exe)")

FileChangeDir(@ScriptDir)

DllCall("test.dll", "none:cdecl", "CrashAutoIt")

[Chance]

I have one more question...

Why does autoit crash if I spawn a thread?

#include <windows.h>
#include <process.h>
#ifndef _DLLTUT_DLL_H_
#define _DLLTUT_DLL_H_
#define DLL_EXPORT __declspec(dllexport)
extern "C"
{
    DLL_EXPORT int SpawnThread();
}
#endif
void AutoThread(void* Argument);
extern "C"
{
    DLL_EXPORT int SpawnThread() {
        _beginthread( AutoThread, 0, NULL );
        Sleep(10);
        return 1;
    }
}
void AutoThread(void* Argument) {
    // ( >._.)> just here to show the call worked <(._.< )
    MessageBox(
        NULL,
        "( >._.)> Call successful!nBut we're going to crash now... (._.  )", "Success! But...",
        MB_ICONINFORMATION | MB_OK
    );
    _endthread();
}
[/PHP]

Windows tells me that AutoIt has crashed if I close anyone of the message boxes, but apparantly AutoIt continues to run and do things, but windows insists that something happened and forces its termination after I click the bug report thing..

MsgBox(0,"","This is from autoit.")

DllCall("test.dll", "none:cdecl", "SpawnThread")
DllCall("test.dll", "none:cdecl", "SpawnThread")

MsgBox(0,"","Now going to exit after crash reported.")
MsgBox(0,"","But here is another messagebox to show that we're still working.")

Oddly enough, AutoIt doesn't crash if I use an external application to kill the threads..

Edited November 15, 2012 by FlutterShy

[AdmiralAlkex]

Wouldn't it be simpler to specify the path and forget about the working dir?

DllCall(@ScriptDir & "test.dll"...

Also look up DllOpen() if you haven't.