Jump to content
Sign in to follow this  
wraithdu

Internal handling 64-bit integers (uint64)

Recommended Posts

wraithdu

I'm having trouble using the VerSetConditionMask() function to prepare the bitmask for the VerifyVersionInfo() function. The mask input and the return value of this function are both 64-bit integers. I think the returned 64-bit integer is being internally mishandled, the high and low parts of the integer look like they're being reversed. If this is really a bug in AutoIt, I'll post a Trac ticket, but I wanted to confirm it first.

Global Const $VER_PLATFORMID = 0x8
Global Const $VER_EQUAL = 1

$dwlConditionalMask = 0
$ret = DllCall("kernel32.dll", "uint64", "VerSetConditionMask", "uint64", $dwlConditionalMask, "dword", $VER_PLATFORMID, "byte", $VER_EQUAL)
$dwlConditionalMask = $ret[0]
;; the following value should be 9223372036854776320 (0x8000000000000200), but returns 2201170739200 (0x20080000000) instead
;; looks like the high and low parts of the 64-bit integer are reversed
ConsoleWrite($dwlConditionalMask & @CRLF)
Edited by wraithdu

Share this post


Link to post
Share on other sites
wraithdu

I threw together a full script with a hack I just came up with to correct the return value. Seems that ConsoleWrite() doesn't print uint64's correctly either (it will print a uint correctly though).

Global Const $OSVERSIONINFOEXW = "dword dwOSVersionInfoSize;dword dwMajorVersion;dword dwMinorVersion;dword dwBuildNumber;dword dwPlatformId;" & _
                                "wchar szCSDVersion[128];ushort wServicePackMajor;ushort wServicePackMinor;ushort wSuiteMask;byte wProductType;byte wReserved"
Global Const $VER_PLATFORMID = 0x8
Global Const $VER_EQUAL = 1

; initialize structure
$OSVI = DllStructCreate($OSVERSIONINFOEXW)
DllStructSetData($OSVI, "dwOSVersionInfoSize", DllStructGetSize($OSVI))
; set data we want to compare
DllStructSetData($OSVI, "dwPlatformId", 2)

; structs for return value
$longlong1 = DllStructCreate("byte[8]")
$longlong2 = DllStructCreate("uint64", DllStructGetPtr($longlong1))
$longlong3 = DllStructCreate("int;int", DllStructGetPtr($longlong1))

; initialize and set the mask
$dwlConditionalMask = 0
$ret = DllCall("kernel32.dll", "uint64", "VerSetConditionMask", "uint64", $dwlConditionalMask, "dword", $VER_PLATFORMID, "byte", $VER_EQUAL)
$dwlConditionalMask = $ret[0]
;; the following value should be 9223372036854776320 (0x8000000000000200), but returns 2201170739200 (0x20080000000) instead
;; looks like the high and low parts of the 64-bit integer are reversed
ConsoleWrite("wrong value:  " & $dwlConditionalMask & @CRLF)
;; function fails
$ret = DllCall("kernel32.dll", "int", "VerifyVersionInfoW", "ptr", DllStructGetPtr($OSVI), "dword", $VER_PLATFORMID, "uint64", $dwlConditionalMask)
ConsoleWrite("Success?  " & $ret[0] & @CRLF)

ConsoleWrite("==============" & @CRLF)
ConsoleWrite("Conversion??" & @CRLF)
DllStructSetData($longlong1, 1, $dwlConditionalMask)
ConsoleWrite("Binary Data (Little Endian):  " & DllStructGetData($longlong1, 1) & @CRLF)
ConsoleWrite("As uint64:  " & DllStructGetData($longlong2, 1) & @CRLF)
ConsoleWrite("Low Part:  " & Hex(DllStructGetData($longlong3, 1)) & @CRLF)
ConsoleWrite("High Part:  " & Hex(DllStructGetData($longlong3, 2)) & @CRLF)
;; conversion??
$temp = DllStructGetData($longlong3, 1)
DllStructSetData($longlong3, 1, DllStructGetData($longlong3, 2))
DllStructSetData($longlong3, 2, $temp)
;; correct value
$dwlConditionalMask = DllStructGetData($longlong2, 1)
ConsoleWrite("Corrected Binary (Little Endian):  " & DllStructGetData($longlong1, 1) & @CRLF)
ConsoleWrite("Not sure why this is printing as a signed int64..." & @CRLF)
ConsoleWrite("As uint64:  " & $dwlConditionalMask & @CRLF)
ConsoleWrite("==============" & @CRLF)

;; function works now
$ret = DllCall("kernel32.dll", "int", "VerifyVersionInfoW", "ptr", DllStructGetPtr($OSVI), "dword", $VER_PLATFORMID, "uint64", $dwlConditionalMask)
ConsoleWrite("Success?  " & $ret[0] & @CRLF)
Edited by wraithdu

Share this post


Link to post
Share on other sites
ProgAndy

In my UDF for libmysql i use this func to fix the return values:

;===============================================================================
;
; Function Name:   __MySQL_ReOrderULONGLONG
; Description::    INTERNAL USE
; Parameter(s):
; Requirement(s):  libmysql.dll
; Return Value(s):
; Author(s):       Prog@ndy
;
;===============================================================================
;
Func __MySQL_ReOrderULONGLONG($UINT64)
    Local $int = DllStructCreate("uint64")
    Local $longlong = DllStructCreate("ulong;ulong", DllStructGetPtr($int))
    DllStructSetData($int, 1, $UINT64)
    Return 4294967296 * DllStructGetData($longlong, 1) + DllStructGetData($longlong, 2)
EndFunc   ;==>__MySQL_ReOrderULONGLONG

//Edit: in the forum i saw some more problems with this. I think it's time for a ticket ^_^

Edited by ProgAndy

*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites
wraithdu

I wonder though, does this happen only on return values from functions, or also when giving a parameter when calling a function? During other internal calculations regarding (u)int64's? There's no way for users to know for sure in the last two scenarios I don't think.

Edited by wraithdu

Share this post


Link to post
Share on other sites
ProgAndy

Well, you use the uint64 as param in the next func do you?

1) You get the uint64 in wrong order

2) Then You use it as param

.... -> if it was given in wrong order, it should be right again ( change order 2 times -> original order)

.... -> since this is not the case and we have to reorder it manually, so uint64 as param works right.

It is the return value wich has wrong order and not the param: I found in the BASS.dll thread, that for smaller values "int" instead of "int64" returns the correct value.

Edited by ProgAndy

*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites
wraithdu

Good point...

Share this post


Link to post
Share on other sites
ProgAndy

I have written some code to prove the mishandling:

freebasic-DLL:

#include "windows.bi"
declare Function      test lib "test" alias "test"(zahl as ULongInt) as LongInt
 function test(zahl as ULongInt) as LongInt Export
    MessageBox(0, "Expected: 99123456789" & chr(13, 10) & "Recieved: " & Str(zahl), "INT64 test: param", 0)
    Return -999888777666
 end function
AutoIt:
$ret = DllCall("C:\Programme\FreeBASIC\FbEdit\Projects\test\test.dll","int64", "test@8", "uint64", 99123456789)

MsgBox(0, 'REturn', "Expected: -999888777666" & @CRLF & "Received: " & $ret[0] & @CRLF & "Reordered: " & __MySQL_ReOrderULONGLONG($ret[0]))

Func __MySQL_ReOrderULONGLONG($UINT64)
    ; Prog@ndy
    Local $int = DllStructCreate("uint64")
    Local $longlong = DllStructCreate("ulong;ulong", DllStructGetPtr($int))
    DllStructSetData($int, 1, $UINT64)
    Return 4294967296 * DllStructGetData($longlong, 1) + DllStructGetData($longlong, 2)
EndFunc   ;==>__MySQL_ReOrderULONGLONG

//Edit: ULongInt in freebasic is uint64 in AutoIt, LongInt is int64

Edited by ProgAndy

*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites
SkinnyWhiteGuy

To quote the help file:

Language Reference - Datatypes

Datatypes and Ranges

The following table shows the internal variant datatypes and their ranges.

Data Sub-type -- Range and Notes

Int32 -- A 32bit signed integer number.

Int64 -- A 64bit signed integer number

Double -- A double-precision floating point number.

String -- Can contain strings of up to 2147483647 characters.

Binary -- Binary data, can contain up to 2147483647 bytes.

Pointer -- A memory address pointer. 32bit or 64bit depending on the version of AutoIt used.

Some functions in AutoIt only work with 32 bit numbers (e.g. BitAND) and are converted automatically - these functions are documented where required.

The highest integer value an AutoIt variable can hold is 9,223,372,036,854,775,807 (or 0x7FFFFFFFFFFFFFFF in Hex). After that, it wraps to negative numbers. I tried to use unsigned 64-bit integers myself at one point, and the AutoIt variant could hold the value, but not display it correctly for me. If you use that returned value in other Dll functions that expect it (without modifying it in AutoIt), all will work.

If you need to do actual calculations on unsigned 64-bit integers, you either have to do it in your own Dll, or work around it. If you want to work around it, I have a few ways I discovered to do it, but they are all ugly to have to use.

Share this post


Link to post
Share on other sites
wraithdu

If you use that returned value in other Dll functions that expect it (without modifying it in AutoIt), all will work.

Hmm, not true though. Try my second example after removing the conversion code. Passing the return from VerSetConditionMask() directly to VerifyVersionInfo() fails because the returned value is wrong. The high and low parts are reversed. Edited by wraithdu

Share this post


Link to post
Share on other sites
Ascend4nt

I've noticed the same problems with 64-bit numbers as well. AutoIT provides the 'uint64' datatype but only supports signed 64-bit numbers, which might seem to function okay - until the uppermost bit is set, that is. In my sig. there are special 64-bit base-2/base-16 conversions UDF's that 'pop' the top bit off in order to work correctly, and the same basic idea can be used to 'set' the top bit - but that's only useful when you have numbers that don't exceed 9,223,372,036,854,775,807 (max unsigned value as SkinnyWhiteGuy pointed out).

It would be nice to have the 64-bit integer issue either documented or fixed though.

Share this post


Link to post
Share on other sites
wraithdu

@ascendant

Is that related to the problem with returning a (u)int64 from a DllCall (reversing the high and low parts)?

Either way, if a dev doesn't chime in with any objections, I'll start a Trac ticket for at least my issue mentioned above, since I'm not sure exactly what you mean when you say they "function okay - until the uppermost bit is set" (I'm unclear on the 'function okay' part). Perhaps you should clarify the problem and submit a ticket as well?

Share this post


Link to post
Share on other sites
Valik

Personally I'm not happy with the way numbers are handled. 64-bit is sort of a tacked on extra that feels exactly that. Unsigned versus signed is sometimes an important distinction that needs made but AutoIt stores everything internally as signed and doesn't provide an easy way to work with unsigned numbers.

With that being said, is this an issue with AutoIt's less than stellar implementation of numbers or is this a genuine bug? If it's the former, don't waste time creating a ticket because I'll just close it and/or re-write it completely (and then I will ignore it for years). However, if you can demonstrate there is an issue where the data is being mangled, well, that's something that can probably be fixed.

Share this post


Link to post
Share on other sites
Ascend4nt

@wraithdu:

I was commenting to the point of 64-bit 'unsigned' integers not being a possibility, not the reversal of low/high-order parts - which I hadn't encountered myself. I actually went back through code where I've used 'uint64' and realize now that I've never used them on 'ULONGLONG' returns, only for 'ULONGLONG's inside DLLStructs and also in replacing the Windows 'FILETIME' structures (since the structure is arranged in little-endian order). So, while only the latter is accessed through the DLL call's return array, its not *technically* a ULONGLONG, and is passed as a pointer anyway ('uint64*').

Now that I've checked your code though, I do get the same result as you. If this can be verified through another language then I'd definitely submit a ticket for it - but only for the DLLCall()'s returns.

64-bit integers in AutoIT are otherwise working as expected - for signed numbers, anyway. When used in a way where the data isn't modified, it doesn't make a difference if 0x8000000000000000 (or 9223372036854775807+1) displays as -9223372036854775808. Its only when math needs to be done that things can get messy.

@Valik:

What about modifying the documentation for DLLCall and DLLStructCreate to indicate that 'uin64' is treated internally as signed integers? That might help avoid confusion in the future.

As for the "less than stellar implementation of numbers", you've already created ticket 767 to cover 64-bit integer support, so I wouldn't add anything further to that.

Share this post


Link to post
Share on other sites
jpm

Unless I am wrong,

Uint64 are always displayed as int64 as explain by Valik that the reason why you get a negative value.

Definetly something wrong with the return of an uint64 by dllcall.

The return value of an updated parameter is well handled.

Share this post


Link to post
Share on other sites
ProgAndy

As you see in my example above, even a signed int64 as return value returns the wrong number and has to be reordered. In all other places, (u)int64 works fine, just as return value in DLLCall it does not.

If you need to display an unsigned Integer or use it in calculations, you have to use the BigNum.au3 ^_^

This is a func to convert the signed int64 to unsigned int64:

#include<BigNum.au3>
Func _Int64TounsignedString($Int64)
    Local Const $UINT64_OFFSET = "18446744073709551616"
    Local $Res = _BigNum_Add(String($Int64), $UINT64_OFFSET)
    If @error Then Return SetError(1,0,"")
    Return $Res
EndFunc

MsgBox(0, '', _Int64TounsignedString("-9223372036854775808"))
Edited by ProgAndy

*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Share this post


Link to post
Share on other sites
SkinnyWhiteGuy

Here's some more weirdness...

WinAPI documentation lists the type for the dwlConditionMask as ULONGLONG, which in our documentation translates to a uint64. But, according to another section of WinAPI:

ULONGLONG 64-bit unsigned integer.

This type is declared in Winnt.h as follows:

typedef unsigned __int64 ULONGLONG;

#else

typedef double ULONGLONG

Would the values we are seeing be explained by it returning as a double, and not the unsigned __int64 we think we should be getting?

This seems to show the same thing...

Global Const $VER_PLATFORMID = 0x8
Global Const $VER_EQUAL = 1

$dwlConditionalMask = 0
$ret = DllCall("kernel32.dll", "uint64", "VerSetConditionMask", "uint64", $dwlConditionalMask, "dword", $VER_PLATFORMID, "byte", $VER_EQUAL)
$dwlConditionalMask = $ret[0]
;; the following value should be 9223372036854776320 (0x8000000000000200), but returns 2201170739200 (0x20080000000) instead
;; looks like the high and low parts of the 64-bit integer are reversed
ConsoleWrite($dwlConditionalMask & @CRLF)
$dwlConditionalMask = 0
$ret = DllCall("kernel32.dll", "uint64", "VerSetConditionMask", "double", $dwlConditionalMask, "dword", $VER_PLATFORMID, "byte", $VER_EQUAL)
$dwlConditionalMask = $ret[0]
;; the following value should be 9223372036854776320 (0x8000000000000200), but returns 2201170739200 (0x20080000000) instead
;; looks like the high and low parts of the 64-bit integer are reversed
ConsoleWrite($dwlConditionalMask & @CRLF)

It seems to treat the input the same (if you change either of the output uint64's to a double, you get a wrong answer).

Edit: And even more curiosities from the Documentation:

Some Windows functions use the LARGE_INTEGER structure to represent 64-bit signed integer values, and the ULARGE_INTEGER structure to specify 64-bit unsigned integer values.

This function doesn't seem to, but we are getting two dword's back in a different order, which this structure would seem to be using (LowPart, then HighPart).

Edited by SkinnyWhiteGuy

Share this post


Link to post
Share on other sites
jpm

In fact 3.3.0 does not return properly the uint64 (high/low swapping). That will be fix in next Beta/release.

Internally Autoit handle as good as Valik likes the int64.

double/float is not a good solution as it loosing precision against int64.

Uint64 cannot be handle properly inside autoit code so uint64 with 0x8.... will be displayed with a negative value.

Share this post


Link to post
Share on other sites
wraithdu

Good to hear, and I'm clear now on the display of uint64's as signed. I won't clutter Trac with another ticket since you are aware of the DllCall return problem. Thanks for looking into it!

Share this post


Link to post
Share on other sites
jpm

Good to hear, and I'm clear now on the display of uint64's as signed. I won't clutter Trac with another ticket since you are aware of the DllCall return problem. Thanks for looking into it!

no problem the pb is fix for the next delivery

Share this post


Link to post
Share on other sites
geww

It seems this bug isn't fixed yet. All 20 digits numbers what I am using always messing up. Do have anyone fix for that?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×