Int, Dec and Number return incorrect results

[timsky]

Seems like Int function works incorrect sometimes.
 
Int of binary data works fine on 1-byte data only. 2+ byte data always returns incorrect results.
 
Int of 4-byte binary data and it's string representation:
  Int of 4-byte binary data returns bswap as result.
  Int64 of string representation of 4-byte binary data returns 32-bit signed integer.

$bin = Binary('0xff'); 1 BYTES
MsgBox(524288, BinaryLen($bin) & ' ' & VarGetType($bin) & ' bytes', 'HEX: ' & $bin & @LF & 'Int: ' & Int($bin) & @LF & 'Int32 <- String: ' & Int(String($bin), 1) & @LF & 'Int (auto) <- String: ' & Int(String($bin)) & @LF & 'Int64 <- String: ' & Int(String($bin), 2))

$bin = Binary('0xffd8'); 2 BYTES
MsgBox(524288, BinaryLen($bin) & ' ' & VarGetType($bin) & ' bytes', 'HEX: ' & $bin & @LF & 'Int: ' & Int($bin) & @LF & 'Int32 <- String: ' & Int(String($bin), 1) & @LF & 'Int (auto) <- String: ' & Int(String($bin)) & @LF & 'Int64 <- String: ' & Int(String($bin), 2))

$bin = Binary('0xffd88d'); 3 BYTES
MsgBox(524288, BinaryLen($bin) & ' ' & VarGetType($bin) & ' bytes', 'HEX: ' & $bin & @LF & 'Int: ' & Int($bin) & @LF & 'Int32 <- String: ' & Int(String($bin), 1) & @LF & 'Int (auto) <- String: ' & Int(String($bin)) & @LF & 'Int64 <- String: ' & Int(String($bin), 2))

$bin = Binary('0xffd88d8a'); 4 BYTES
MsgBox(524288, BinaryLen($bin) & ' ' & VarGetType($bin) & ' bytes', 'HEX: ' & $bin & @LF & 'Int: ' & Int($bin) & @LF & 'Int32 <- String: ' & Int(String($bin), 1) & @LF & 'Int (auto) <- String: ' & Int(String($bin)) & @LF & 'Int64 <- String: ' & Int(String($bin), 2))

$bin = Binary('0xffd88d8abc'); 5 BYTES
MsgBox(524288, BinaryLen($bin) & ' ' & VarGetType($bin) & ' bytes', 'HEX: ' & $bin & @LF & 'Int: ' & Int($bin) & @LF & 'Int32 <- String: ' & Int(String($bin), 1) & @LF & 'Int (auto) <- String: ' & Int(String($bin)) & @LF & 'Int64 <- String: ' & Int(String($bin), 2))

$bin = Binary('0xffd88d8abcd1'); 6 BYTES
MsgBox(524288, BinaryLen($bin) & ' ' & VarGetType($bin) & ' bytes', 'HEX: ' & $bin & @LF & 'Int: ' & Int($bin) & @LF & 'Int32 <- String: ' & Int(String($bin), 1) & @LF & 'Int (auto) <- String: ' & Int(String($bin)) & @LF & 'Int64 <- String: ' & Int(String($bin), 2))

$bin = Binary('0xffd88d8abcd1e5'); 7 BYTES
MsgBox(524288, BinaryLen($bin) & ' ' & VarGetType($bin) & ' bytes', 'HEX: ' & $bin & @LF & 'Int: ' & Int($bin) & @LF & 'Int32 <- String: ' & Int(String($bin), 1) & @LF & 'Int (auto) <- String: ' & Int(String($bin)) & @LF & 'Int64 <- String: ' & Int(String($bin), 2))

$bin = Binary('0xffd88d8abcd1e5ff'); 8 BYTES
MsgBox(524288, BinaryLen($bin) & ' ' & VarGetType($bin) & ' bytes', 'HEX: ' & $bin & @LF & 'Int: ' & Int($bin) & @LF & 'Int32 <- String: ' & Int(String($bin), 1) & @LF & 'Int (auto) <- String: ' & Int(String($bin)) & @LF & 'Int64 <- String: ' & Int(String($bin), 2))

AutoIt 3.3.12.0 x86 and x64
OS: Win 7 x64 SP1

Edited February 23, 2015 by timsky
Amended title

[Melba23]

timsky,

Why are you doing the double Binary-String conversion?

If you use sensible datatypes then you get the correct results:

#include <MsgBoxConstants.au3>

Global $aBin[] = ["0xff", '0xffd8', '0xffd88d', '0xffd88d8a', '0xffd88d8abc', '0xffd88d8abcd1', '0xffd88d8abcd1e5', '0xffd88d8abcd1e5ff']

For $i = 0 To UBound($aBin) - 1

    $bin = $aBin[$i]

    $Bin_bin = Binary($bin)

    $sData = 'Hex: ' & @TAB & @TAB & $bin & @LF & _
            'Binary: ' & @TAB & @TAB & StringLower($Bin_bin) & @LF & _
            'Dec: ' & @TAB & @TAB & Dec(StringTrimLeft($bin, 2)) & @LF & _
            'Int32: ' & @TAB & @TAB & Int($bin, 1) & @LF & _
            'Int (auto): ' & @TAB & @TAB & Int($bin) & @LF & _
            'Int64: ' & @TAB & @TAB & Int($bin, 2) & @LF & _
            'Int32 <- String: ' & @TAB & Int(String($bin), 1) & @LF & _
            'Int (auto) <- String: ' & @TAB & Int(String($bin)) & @LF & _
            'Int64 <- String: ' & @TAB & Int(String($bin), 2) & @LF & _
            'Int32 <- Binary: ' & @TAB & Int($Bin_bin, 1) & @LF & _
            'Int (auto) <- Binary: ' & @TAB & Int($Bin_bin) & @LF & _
            'Int64 <- Binary: ' & @TAB & Int($Bin_bin, 2)

    MsgBox($MB_SYSTEMMODAL, $bin & " - " & BinaryLen($bin) & ' bytes', $sData)

Next

M23

[timsky]

Melba23,

 

Why are you doing the double Binary-String conversion?

Because I read file in binary mode and got strange results with Int(). My code was small demonstration of issues.

Int64 of of 4-byte binary or string data anyway does not return correct data:

 

4 bytes (32 bits)

Hex:   0xffd88d8a
Binary:   0xffd88d8a
Dec:   -2585206
Int32:   -2585206
Int (auto):   -2585206
Int64:   -2585206
Int32 <- String:  -2585206
Int (auto) <- String:  -2585206
Int64 <- String:  -2585206
Int32 <- Binary:  -1970415361
Int (auto) <- Binary:  -1970415361
Int64 <- Binary:  -1970415361

There should be 4292382090 instead of -2585206 and -1970415361 for Int64.

You can see that Binary data is interpreted by Int() wrong way always if data is 2+ bytes. String representation gives better results but still don't work with 4-byte data.

Aren't these issues strange?

Thank you for Dec() Dec64 works perfect and returns correct result for 4-byte binary or string data: 4292382090

Here is a bit modified version of your example:

Global $aBin[] = ["0xff", '0xffd8', '0xffd88d', '0xffd88d8a', '0xffd88d8abc', '0xffd88d8abcd1', '0xffd88d8abcd1e5', '0xffd88d8abcd1e5ff']

For $i = 0 To UBound($aBin) - 1
    $real_str = $aBin[$i]
    $real_bin = Binary($real_str)
    ; MsgBox(0, VarGetType($real_str), VarGetType($real_bin))

    $sData = 'Hex: ' & @TAB & @TAB & $real_str & @LF & _
            'Binary: ' & @TAB & @TAB & StringLower($real_bin) & @LF & _
            'Dec32: ' & @TAB & @TAB & Dec(StringTrimLeft($real_str, 2), 1) & @LF & _    ; added
            'Dec:  (auto)' & @TAB & @TAB & Dec(StringTrimLeft($real_str, 2)) & @LF & _  ; Renamed
            'Dec64: ' & @TAB & @TAB & Dec(StringTrimLeft($real_str, 2), 2) & @LF & _    ; added
            'Dec (double): ' & @TAB & @TAB & Dec(StringTrimLeft($real_str, 2), 3) & @LF & _ ; added
            'Int32: ' & @TAB & @TAB & Int($real_str, 1) & @LF & _
            'Int (auto): ' & @TAB & @TAB & Int($real_str) & @LF & _
            'Int64: ' & @TAB & @TAB & Int($real_str, 2) & @LF & _
            'Int32 <- String: ' & @TAB & Int(String($real_str), 1) & @LF & _
            'Int (auto) <- String: ' & @TAB & Int(String($real_str)) & @LF & _
            'Int64 <- String: ' & @TAB & Int(String($real_str), 2) & @LF & _
            'Int32 <- Binary: ' & @TAB & Int($real_bin, 1) & @LF & _
            'Int (auto) <- Binary: ' & @TAB & Int($real_bin) & @LF & _
            'Int64 <- Binary: ' & @TAB & Int($real_bin, 2)

    MsgBox(524288, $real_str & " - " & BinaryLen($real_str) & ' bytes', $sData)

Next

Double means scientific notation?

Dec (double) return strange results too  starting from 1-byte data:

1 byte:

Hex:   0xff
Binary:   0xff
Dec32:   255
Dec:  (auto)  255
Dec64:   255
Dec (double):   1.25986739689518e-321
Int32:   255
Int (auto):   255
Int64:   255
Int32 <- String:  255
Int (auto) <- String:  255
Int64 <- String:  255
Int32 <- Binary:  255
Int (auto) <- Binary:  255
Int64 <- Binary:  255

 

Edited February 21, 2015 by timsky

[timsky]

And finally I tried Number(). It has issues too and works similar to Int() function:

1) Binary data is interpreted by Number() wrong way always if data is 2+ bytes. "Double" flag returns 0

2) String representation gives better results but still don't work with 4-byte data.  "Double" flag returns correct number: 4292382090 just like Dec64

3) But starting 5 bytes of data String representationwith "Double" flag returns 0

Demo:

Global $aBin[] = ["0xff", '0xffd8', '0xffd88d', '0xffd88d8a', '0xffd88d8abc', '0xffd88d8abcd1', '0xffd88d8abcd1e5', '0xffd88d8abcd1e5ff']

For $i = 0 To UBound($aBin) - 1
    $real_str = $aBin[$i]
    $real_bin = Binary($real_str)
    ; MsgBox(0, VarGetType($real_str), VarGetType($real_bin))

    $sData = 'Hex: ' & @TAB & @TAB & $real_str & @LF & _
            'Binary: ' & @TAB & @TAB & StringLower($real_bin) & @LF & _
            'Dec32: ' & @TAB & @TAB & Dec(StringTrimLeft($real_str, 2), 1) & @LF & _    ; added
            'Dec:  (auto)' & @TAB & @TAB & Dec(StringTrimLeft($real_str, 2)) & @LF & _  ; Renamed
            'Dec64: ' & @TAB & @TAB & Dec(StringTrimLeft($real_str, 2), 2) & @LF & _    ; added
            'Dec (double): ' & @TAB & @TAB & Dec(StringTrimLeft($real_str, 2), 3) & @LF & _ ; added
            'Number32 <- String: ' & @TAB & @TAB & Number($real_str, 1) & @LF & _   ; added
            'Number <- String:  (auto)' & @TAB & @TAB & Number($real_str) & @LF & _ ; added
            'Number64 <- String: ' & @TAB & @TAB & Number($real_str, 2) & @LF & _   ; added
            'Number (double) <- String: ' & @TAB & @TAB & Number($real_str, 3) & @LF & _    ; added
            'Number32 <- Binary ' & @TAB & @TAB & Number($real_bin, 1) & @LF & _    ; added
            'Number <- Binary  (auto)' & @TAB & @TAB & Number($real_bin) & @LF & _  ; added
            'Number64 <- Binary ' & @TAB & @TAB & Number($real_bin, 2) & @LF & _    ; added
            'Number (double) <- Binary ' & @TAB & @TAB & Number($real_bin, 3) & @LF & _ ; added
            'Int32: ' & @TAB & @TAB & Int($real_str, 1) & @LF & _
            'Int (auto): ' & @TAB & @TAB & Int($real_str) & @LF & _
            'Int64: ' & @TAB & @TAB & Int($real_str, 2) & @LF & _
            'Int32 <- String: ' & @TAB & Int(String($real_str), 1) & @LF & _
            'Int (auto) <- String: ' & @TAB & Int(String($real_str)) & @LF & _
            'Int64 <- String: ' & @TAB & Int(String($real_str), 2) & @LF & _
            'Int32 <- Binary: ' & @TAB & Int($real_bin, 1) & @LF & _
            'Int (auto) <- Binary: ' & @TAB & Int($real_bin) & @LF & _
            'Int64 <- Binary: ' & @TAB & Int($real_bin, 2)

    MsgBox(524288, $real_str & " - " & BinaryLen($real_str) & ' bytes', $sData)

Next

[timsky]

Opened Ticket #2992 https://www.autoitscript.com/trac/autoit/ticket/2992#ticket

[Melba23]

timsky,

As I have already pointed out, these functions are, quite reasonably, not designed to be used on binary data. As I showed, if you use sensible datatypes, you get the correct return each time.

Why do you have to read your file in binary? Why do you need to check if come elements of the file content are integers? Why can you not convert the binary data back into a string so it does work correctly?

M23

[timsky]

Melba23,

 

As I have already pointed out, these functions are, quite reasonably, not designed to be used on binary data.

I do not understand what means "quite reasonably, not designed to be used on binary data".  If so then why it is documented as feature? What functions are designed to do that?

As I showed, if you use sensible datatypes, you get the correct return each time.

All mentioned functions do not return correct results each time. Seems like you did not read carefully my messages and ticket. Copy-paste for you:

1) Int and Number of binary data works fine on 1-byte data only. 2+ byte binary data always returns incorrect results.
2) Case of 4-byte (32 bit) binary data and it's string representation:

   a. Int and Number of 4-byte binary data returns wrong result.
   b.  Int (64 bit flag) of string representation of 4-byte binary data returns 32-bit signed integer. Example string "0xffd88d8a" return -2585206 instead of 4292382090.
   c. Only Dec (64 bit flag) and Number (double flag) return correct results.

3) Number (double flag) does not work on binary data always. It returns 0.
4) Number (double flag) stop working on string data starting 5+ byte data. It returns 0.
5) Dec (double flag) doesn't work always. It returns strange numbers like 1.25986739689518e-321 for "0xff".

Just try Demo code from ticket to see all problems.

Why do you have to read your file in binary? Why do you need to check if come elements of the file content are integers?

Very strange question Because I need that for my program I am working on.

Why can you not convert the binary data back into a string so it does work correctly?

It does not work correctly too. Take a closer look at sections 4), 5) and section 2), subsection b. in copy-paste above.

Edited February 23, 2015 by timsky

[Melba23]

timsky,

 

I do not understand what means "quite reasonably, not designed to be used on binary data".

Int is designed to return the integer part of a numeric value, or an expression which resolves to a numeric value. A string of binary data does not necessarily resolve to a numeric value, so why would you expect the function to work with binary data?

If so then why it is documented as feature?

Where do you see this? All I see is "An expression to convert into an integer" - no mention of binary data, and that does not surprise me at all given that I would not expect to work.

Very strange question

Why? I am trying to find out what you are trying to do when you use Int on your binary data to see if we can come up with some code that will work for you - because what you are trying to do will not.

M23

[czardas]

It is an unfortunate fact that the binary range 0x80000000 - 0xFFFFFFFF is somewhat ambigous to the interpreter unless we deal with these values very carefully. Some older AutoIt functions are still not fully compatible with Int64. There are ways to sidestep some of these problems. For example:

;

MsgBox(0, "", Dec(StringTrimLeft(Binary('0xffd88d8a'), 2), 2)) ; Int64

Edited February 23, 2015 by czardas

[timsky]

Melba23,

Where do you see this? All I see is "An expression to convert into an integer" - no mention of binary data, and that does not surprise me at all given that I would not expect to work.

Then why Int works fine with string data except 4-byte case? Why Int works fine with both binary and string data if data is 1 byte (8bits)? Same with Number.

Number

A string beginning with letters has a numeric value of zero. A string beginning with digits has non-numeric characters stripped.

In case of Number: Why hexademical (binary) data is correctly interpreted and starting 2+ bytes it get broken for binary but works fine for string... just like Int does?

Double flag breaks Number (double) for binary data always and for string data starting from 5+ bytes.

Dec with double flag set returns wrong data always.

 

If you are right then Int and Number must return anything else but not "correct" results almost any time

Edited February 23, 2015 by timsky

[czardas]

Dec with double flag set returns wrong data always.

 

It may appear that way, but what does the binary data inside a double look like? I think this is normal interpretation of the binary where the hex simply has a different meaning.

Edited February 23, 2015 by czardas

[BrewManNH]

http://www.binaryconvert.com/result_double.html?hexadecimal=00000000000000FF

[czardas]

Cool website BrewManNH.

[BrewManNH]

The OP obviously doesn't understand floating point numbers and binary representations of them.

[Melba23]

timsky,

Back from the gym, and I still cannot decide if you are being deliberately obtuse or just trolling us. What is hard to understand about the fact that you need to convert your binary data into a datatype that the functions you claim do not work correctly will understand? AutoIt does a fair bit of datatype conversion to try and help the user - as you can see from the code I posted above where the function deals quite happily with various different datatypes - but binary data appears to be a case where it does not and so you need to ensure that the datatype passed to the function is accepted.

One last try. Why do you read the file in binary and why are you trying to use Int, Dec, etc on the returned bytes? If the answer is just "because I wanted to see if it worked" then fine, because you have your answer (it does not) - if it for a specific reason then please explain what you are trying to do so we can help you write some code that does do what you need. Over to you...

M23

BrewManNH,

I could not agree more with your comment above - in fact I would extend it to datatypes in general.

Edited February 23, 2015 by Melba23

[jchd]

Endianness.

[timsky]

Melba23, 

Why do you read the file in binary and why are you trying to use Int, Dec, etc on the returned bytes?

Because I need to see how binary data will "look" in decimal. That's it.

as you can see from the code I posted above where the function deals quite happily with various different datatypes - but binary data appears to be a case where it does not and so you need to ensure that the datatype passed to the function is accepted

Binary - yes, but string (hex) works in most cases.

I tried various combinations and you can see that these functions work as I expected but in some cases are limited by binary length of data.

So how can I convert binary data to make Int and Number work with binary/string (hex) data?

As I see now only Dec does the job as required by me.

[czardas]

Stay safe! Use Number() with numeric strings. Use Int() for floats less than 1e+015. Using Int() on Int32, or Int64, is meaningless anyway.

The binary range I mentioned earlier, is the range where things started to go wrong in your first example. The binary does not know what it represents (Int32, Int64, double, ansi, little endian, UTF8). It first has to be interpreted as one type, or another: otherwise the interpreter will cast the data internally - defaulting to one of a number of possibilities.

Endianness is also a bit like throwing a spanner in the works, but I believe such complications are partly a consequence of historical divergence. There are also benefits to be had from there being so many possible interpretations of binary data.

Edited February 24, 2015 by czardas

[Geir1983]

Melba23, 

Because I need to see how binary data will "look" in decimal. That's it.

 

If you read the file in binary mode you also need to know how to convert it to sensible data. For example if your binary data is 4 bytes long, this data could be 4 separate variables with values 0-255 or it could be 4 separate variables with values -128-127 or again it could be a single floating point variable. There are a dozen possible datatypes that could be used on the same binary data, it is not possible to find this by using VarGetType on a string of binary data. You will need to know the structure of this binary data to make sense of it.

[timsky]

Geir1983,

 For example if your binary data is 4 bytes long, this data could be 4 separate variables with values 0-255 or it could be 4 separate variables with values -128-127 or again it could be a single floating point variable.

I know that but thank you for explanation.

czardas, BrewManNH, Melba23, thank you too