Jump to content

problem with decimals


V23
 Share

Recommended Posts

Why

7.04 - 7 = 0.04

and

8.04 - 8 = 0.0399999999999991

???

I not found the answer of problem in forum.

Please help me.

hmmm. i'm posing a bug report

edit:http://www.autoitscript.com/trac/autoit/ticket/716

Edited by mmavipc

[size="10"]Pure Au3 crypt funcs(I'm currently also working on making a dll from this)[/size][Y] Be more active in the community[Y] Get 200 posts[N] Get 300 posts[N] Make a Topic in the example scripts forum with at least 50 replies.People who currently hate me:ValikSmOke_N

Link to comment
Share on other sites

hmmm. i'm posing a bug report

edit:http://www.autoitscript.com/trac/autoit/ticket/716

heh. I'm too. (#715)

but the real cool man told me

"It NOT a bug, you should try to ask in forum"

and deleted my report... =\

Link to comment
Share on other sites

Could you tell me more in detail, please?

Floating-point values (decimals) are prone to being inaccurate. The more precise (the more numbers behind the decimal) the more of a chance the value will be inaccurate. AutoIt, internally, uses a double to store floating-point values. Since a double can store more decimal places than a 'float', there is more of a chance the value will be inaccurate. If you can see the real values of some floating-point values before some operation is done to them, you'll see a lot of them are inaccurate.

Here's an example of the same problem in C++:

Posted Image

Link to comment
Share on other sites

Floating-point values (decimals) are prone to being inaccurate. The more precise (the more numbers behind the decimal) the more of a chance the value will be inaccurate. AutoIt, internally, uses a double to store floating-point values. Since a double can store more decimal places than a 'float', there is more of a chance the value will be inaccurate. If you can see the real values of some floating-point values before some operation is done to them, you'll see a lot of them are inaccurate.

Here's an example of the same problem in C++:

Posted Image

Thank you very much!

I think I understood.

Link to comment
Share on other sites

Yes, floating point values are inaccurate. Because I was not aware of this before, I had a real headache when I discovered it for the first time with another programming language :)

The solution is to always use a formatted string for the outputs. Or, you can round the number first.

$i = 8.04 - 8
$f = StringFormat("%.14f",$i)
$r = Round($i,14)
MsgBox(0, "Format Number", $i & @LF & $f & @LF & $r)

And to be most accurate, you should take only the 14 first SIGNIFICANT digits. This means that when you have 1 digit before the decimal point, get the first 13 digits after the decimal point. If you have 2 digits before the decimal point, take the first 12 digits after the decimal point, and so on.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...