Calc double and integer

[Ascer]

Hello

Need technical support with math in AutoIt.

Result code below should be 0. but is 2.8421709430404e-014

$a = 3.99 - 3
ConsoleWrite(VarGetType($a) & ": " & $a & @CRLF)

$a *= 100
ConsoleWrite(VarGetType($a) & ": " & $a & @CRLF)

$a -= 99
ConsoleWrite(VarGetType($a) & ": " & $a & @CRLF)

 

[Melba23]

Ascer,

Not an AutoIt problem, but one that affects all computers that do not have a specialist maths app/CPU. This has been discussed many, many times - search the forum for "floating point arithmetic".

M23

[Ascer]

@Melba23

Thans for advice but this is important problem in my opinion

[Melba23]

Ascer,

I agree, but it is beyond AutoIt to do anything about it. Did you read anything about floating point arithmetic? it is a product of the way all computers store real numbers (i.e. non-integers) and, as I mentioned above, requires a special maths program or CPU do deal with it correctly.

In your case, just convert everything to integers:

$a = 3.99

$a = $a * 100
ConsoleWrite(VarGetType($a) & ": " & $a & @CRLF)

$a -= 300
ConsoleWrite(VarGetType($a) & ": " & $a & @CRLF)

$a -= 99
ConsoleWrite(VarGetType($a) & ": " & $a & @CRLF)

$a = $a/100
ConsoleWrite(VarGetType($a) & ": " & $a & @CRLF)

M23

Edited March 5, 2018 by Melba23
Added link

[Bowmore]

One way of dealing with this issue is to round the final result or before doing any comparisons to a sensible number of decimal places.

 

$a = 3.99 - 3
ConsoleWrite(VarGetType($a) & ": " & $a & @CRLF)

$a *= 100
ConsoleWrite(VarGetType($a) & ": " & $a & @CRLF)

$a -= 99
ConsoleWrite(VarGetType($a) & ": " & $a & @CRLF)
ConsoleWrite(VarGetType($a) & ": " & Round($a,12) & @CRLF)

If $a = 0 Then
    ConsoleWrite(VarGetType($a) & ": $a = zero is True" & @CRLF)
Else
    ConsoleWrite(VarGetType($a) & ": $a = zero is False" & @CRLF)
EndIf

If Round($a,12) = 0 Then
    ConsoleWrite(VarGetType($a) & ": $a = zero is True" & @CRLF)
Else
    ConsoleWrite(VarGetType($a) & ": $a = zero is False" & @CRLF)
EndIf

 

[LarsJ]

It's not an AutoIt problem, but we still can do something about it by using variants of type decimal:

expandcollapse popup

#include "Variant.au3"

Opt( "MustDeclareVars", 1 )

Example1()
Example2()

Func Example1()
  ConsoleWrite( "Example1" & @CRLF )

  Local $a = 3.99 - 3
  ConsoleWrite(VarGetType($a) & ": " & $a & @CRLF)

  $a *= 100
  ConsoleWrite(VarGetType($a) & ": " & $a & @CRLF)

  $a -= 99
  ConsoleWrite(VarGetType($a) & ": " & $a & @CRLF)
EndFunc

Func Example2()
  ConsoleWrite( @CRLF & "Example2" & @CRLF )

  Local $a = "3.99", $b = "3", $c = "100", $d = "99"
  ; The strings ensures that the numbers are not converted to floats

  Local $ta = StrToDec( $a ), $pa = DllStructGetPtr( $ta )
  Local $tb = StrToDec( $b ), $pb = DllStructGetPtr( $tb )
  Local $tc = StrToDec( $c ), $pc = DllStructGetPtr( $tc )
  Local $td = StrToDec( $d ), $pd = DllStructGetPtr( $td )

  VarDecSub( $pa, $pb, $pa ) ; $a = $a - $b = 3.99 -   3
  $a = DecToStr( $pa )
  ConsoleWrite( $a & @CRLF )
  VarDecMul( $pa, $pc, $pa ) ; $a = $a * $c =  $a  * 100
  $a = DecToStr( $pa )
  ConsoleWrite( $a & @CRLF )
  VarDecSub( $pa, $pd, $pa ) ; $a = $a - $d =  $a  -  99
  $a = DecToStr( $pa )
  ConsoleWrite( $a & @CRLF )
EndFunc

Func StrToDec( $str )
  Local $tStr = DllStructCreate( $tagVARIANT )
  Local $pStr = DllStructGetPtr( $tStr )
  $tStr.vt   = $VT_BSTR
  $tStr.data = SysAllocString( $str )
  VariantChangeType( $pStr, $pStr, 0, $VT_DECIMAL )
  Local $tDec = DllStructCreate( $tagDEC, $pStr )
  Local $l = StringInStr( $str, "." )
  $tDec.scale = $l ? StringLen( $str ) - $l : 0
  Return $tStr
EndFunc

Func VarDecAdd( $pDecLeft, $pDecRight, $pDecResult )
  Local $aRet = DllCall( "OleAut32.dll", "long", "VarDecAdd", "ptr", $pDecLeft, "ptr", $pDecRight, "ptr", $pDecResult )
  If @error Then Return SetError(1,0,1)
  Return $aRet[0]
EndFunc

Func VarDecDiv( $pDecLeft, $pDecRight, $pDecResult )
  Local $aRet = DllCall( "OleAut32.dll", "long", "VarDecDiv", "ptr", $pDecLeft, "ptr", $pDecRight, "ptr", $pDecResult )
  If @error Then Return SetError(1,0,1)
  Return $aRet[0]
EndFunc

Func VarDecMul( $pDecLeft, $pDecRight, $pDecResult )
  Local $aRet = DllCall( "OleAut32.dll", "long", "VarDecMul", "ptr", $pDecLeft, "ptr", $pDecRight, "ptr", $pDecResult )
  If @error Then Return SetError(1,0,1)
  Return $aRet[0]
EndFunc

Func VarDecSub( $pDecLeft, $pDecRight, $pDecResult )
  Local $aRet = DllCall( "OleAut32.dll", "long", "VarDecSub", "ptr", $pDecLeft, "ptr", $pDecRight, "ptr", $pDecResult )
  If @error Then Return SetError(1,0,1)
  Return $aRet[0]
EndFunc

Func DecToStr( $pDec )
  Local $tStr = DllStructCreate( $tagVARIANT )
  VariantChangeType( DllStructGetPtr( $tStr ), $pDec, 0, $VT_BSTR )
  Return SysReadString( $tStr.data )
EndFunc

You can get Variant.au3 in this post.