Jump to content
Sign in to follow this  
twitchyliquid64

[SOLVED] passing 'pointer to pointer' in DLLCall problem

Recommended Posts

twitchyliquid64

Hi all,

I have a DLL Function which requires the type BIGNUM**. I have tried multiple ways, but I just cant get it to work.

Currently I am:

1. Making a DLLCall() to get a pointer to a new BigNum Structure.

2. Making another DLL call to assign a number to that structure <--- AutoIt hard crash at this point.

So does anyone know how to pass a pointer to a pointer via DLLCall? This is what I have so far (dw, only 33 lines)

$_BigNum_DLL = DllOpen("libeay32.dll")

; BIGNUM *BN_new(void);
Func BigNum_New()
$val = DllCall($_BigNum_DLL, "ptr", "BN_new")
return SetError( @error, @extended, $val)
EndFunc

; int BN_dec2bn(BIGNUM **a, const char *str);
Func BigNum_Assign($BNNum, $sNum)
;$struct = DllStructCreate("ptr num;")
;DllStructSetData($struct, "num", $BNNum) ;Passing the pointer from this didnt work either.

$retval = DllCall($_BigNum_DLL, "int", "BN_dec2bn", "ptr*", $BNNum, "str", $sNum)
ConsoleWrite("RETVAL: " & $retval & @CRLF)
EndFunc



$test = BigNum_New()
BigNum_Assign($test, "1000")

Thanks in advance for all the help you can give me. Having not used DLLCall much, I am stumped.

Edited by twitchyliquid64

ongoing projects:-firestorm: Largescale P2P Social NetworkCompleted Autoit Programs/Scripts: Variable Pickler | Networked Streaming Audio (in pure autoIT) | firenet p2p web messenger | Proxy Checker | Dynamic Execute() Code Generator | P2P UDF | Graph Theory Proof of Concept - Breadth First search

Share this post


Link to post
Share on other sites
Shaggi

1. dllcall returns the result in retval[0], so this should be fixed in both functions.

2.

;$struct = DllStructCreate("ptr num;")
;DllStructSetData($struct, "num", $BNNum) ;Passing the pointer from this didnt work either.

this is correct too, you just need to pass

DllStructGetPtr($struct,1)

instead of $BNNUm in second call


Ever wanted to call functions in another process? ProcessCall UDFConsole stuff: Console UDFC Preprocessor for AutoIt OMG

Share this post


Link to post
Share on other sites
twitchyliquid64

Thankyou Shaggi!

I made the changes you recommended, however I am still getting hard crashes.

<snip>

Func BigNum_New()
$val = DllCall($_BigNum_DLL, "ptr", "BN_new")
return $val[0]
EndFunc

; int BN_dec2bn(BIGNUM **a, const char *str);
Func BigNum_Assign($BNNum, $sNum)
$struct = DllStructCreate("ptr num;")
DllStructSetData($struct, "num", $BNNum) ;Passing the pointer from this didnt work either.

$retval = DllCall($_BigNum_DLL, "int", "BN_dec2bn", "ptr*", DllStructGetPtr($struct), "str", $sNum)
ConsoleWrite("RETVAL: " & $retval[0] & @CRLF)
EndFunc



$test = BigNum_New()
BigNum_Assign($test, "1000")

1. Using DllStructGetPtr($struct, 1) instead of DllStructGetPtr($struct) makes no difference

2. Using "ptr" instead of "ptr*" makes no difference.

Edited by twitchyliquid64

ongoing projects:-firestorm: Largescale P2P Social NetworkCompleted Autoit Programs/Scripts: Variable Pickler | Networked Streaming Audio (in pure autoIT) | firenet p2p web messenger | Proxy Checker | Dynamic Execute() Code Generator | P2P UDF | Graph Theory Proof of Concept - Breadth First search

Share this post


Link to post
Share on other sites
trancexx

Probably should be "ptr*" (asterisk). Try changing calling convention to cdecl.

Edited by trancexx

♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites
twitchyliquid64

Probably should be "ptr*" (asterisk). Try changing calling convention to cdecl.

I realised how much of a retard I was as soon as you said it. AutoIt uses STDCALL by default doesnt it?

Thanks trancexx and Shaggi for your help!

For completeness, here is the final solution: To pointer* pass you setup a struct and pass the pointer to the struct.

; int BN_dec2bn(BIGNUM **a, const char *str);
Func BigNum_Assign($BNNum, $sNum)
    $struct = DllStructCreate("ptr num;")
    DllStructSetData($struct, "num", $BNNum) ;Passing the pointer from this didnt work either.

    $retval = DllCall($_BigNum_DLL, "int:cdecl", "BN_dec2bn", "ptr", DllStructGetPtr($struct), "str", $sNum)
    ConsoleWrite("RETVAL: " & $retval[0] & @CRLF)
EndFunc

ongoing projects:-firestorm: Largescale P2P Social NetworkCompleted Autoit Programs/Scripts: Variable Pickler | Networked Streaming Audio (in pure autoIT) | firenet p2p web messenger | Proxy Checker | Dynamic Execute() Code Generator | P2P UDF | Graph Theory Proof of Concept - Breadth First search

Share this post


Link to post
Share on other sites
Shaggi

Yes it does. If im not mistaken, it should possible to pass the struct as the variable, if you define the type in the dllcall as struct*:

Dllcall( .... , "struct*", $struct)


Ever wanted to call functions in another process? ProcessCall UDFConsole stuff: Console UDFC Preprocessor for AutoIt OMG

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×