Decimal errors on exact mathematical formules

[marko001]

Hi all,

i'm becoming crazy since i'm finding several errors when relating numeric expressions each others.

Example:

$var1 = 2
$var2 = 0.50
$sum = $var1 * $var2

 

Result is NOT exactly 1 but something like 1.000000000000000000123 and using round() doesn't solve.

Generally speaking i realize that too often simple calcs generates micro-decimals that invalid relations.

I also tried string() but with no success.

Is there a way to *exactly* compare numbers with no margin of errors?

Thanks a lot,

Marco

[UEZ]

Well, for me it works as expected:

ConsoleWrite(1 / 2 & @CRLF)
ConsoleWrite(0.5 * 2 & @CRLF)
ConsoleWrite(1 / 3 * 3 & @CRLF)
ConsoleWrite(5 / 9 & @CRLF)
ConsoleWrite(677 / 1375 & @CRLF)
ConsoleWrite(ACos(-1) & @CRLF)
ConsoleWrite(Sin(ACos(-1) / 2) & @CRLF)

Results:

0.5
1
1
0.555555555555556
0.492363636363636
3.14159265358979
1

Br,

UEZ

Edited October 9, 2014 by UEZ

[Spider001]

for me it works as expected on xp

[marko001]

This obviously leads to correct values if the computation is simple.

What I say is that after lot of calculations on same iniital value, I loose some decimals.

If I iterate calculations, it happens.

Usually it's multiplying an integer with a 2 decimal value (i.e 2*.25)

M.

[Spider001]

that gives a wrong number

#include <MsgBoxConstants.au3>

 $var1 = 2
 $var2 = 0.00000000001
 $sum = $var1 * $var2
MsgBox($MB_SYSTEMMODAL, "Title", StringFormat ("%.40f", $sum))

[Melba23]

marko001,

This is a problem that affects ALL computers regardless of the language used - google "floating point arithmetic" to learn all about it.

One of the normal workarounds when, as you say you are, dealing with 2 decimal place values (often currency) is to convert all the decimal values into integer values and then reconvert the result once the calculation has been made.

M23

[marko001]

This is what I thought but hoped in a possible workaround.

I'll try what you suggested and I'll be back with results.

(I knew an aswer would have come from you   )

[UEZ]

For a workaround it depends what result you want from the calculation. 1.000000000000000000123 = 1? or 0.000000000019999999999999999 = 0?

Can you provide some examples what the calculation is and what result you are expecting.

Br,

UEZ

[marko001]

I need precision on 2nd decimal.

Based on your example:

1.000000000000000000123 = 1

0.000000000019999999999999999 = 0

I'm trying to replicate errors and I'll post you values and functions.

M.

[UEZ]

Then this should do your job:

 

ConsoleWrite(StringFormat ("%.2f", 0.065934857935793) & @CRLF)
ConsoleWrite(StringFormat ("%.2f", 10.000023) & @CRLF)
ConsoleWrite(StringFormat ("%.2f", ACos(-1)) & @CRLF)
ConsoleWrite(StringFormat ("%.2f", 1.009) & @CRLF)

 

Br,

UEZ

[jchd]

As Melba just said, there is nothing wrong here. You just can't expect an exact representation of most real values, let alone the result of any computation involving those values that can't represent exactly in limited precision binary floating point.