Sign in to follow this  
Followers 0
V23

problem with decimals

11 posts in this topic

Why

7.04 - 7 = 0.04

and

8.04 - 8 = 0.0399999999999991

???

I not found the answer of problem in forum.

Please help me.

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

Why

7.04 - 7 = 0.04

and

8.04 - 8 = 0.0399999999999991

???

I not found the answer of problem in forum.

Please help me.

hmmm. i'm posing a bug report

edit:http://www.autoitscript.com/trac/autoit/ticket/716

Edited by mmavipc

[size="10"]Pure Au3 crypt funcs(I'm currently also working on making a dll from this)[/size][Y] Be more active in the community[Y] Get 200 posts[N] Get 300 posts[N] Make a Topic in the example scripts forum with at least 50 replies.People who currently hate me:ValikSmOke_N

Share this post


Link to post
Share on other sites

hmmm. i'm posing a bug report

edit:http://www.autoitscript.com/trac/autoit/ticket/716

heh. I'm too. (#715)

but the real cool man told me

"It NOT a bug, you should try to ask in forum"

and deleted my report... =\

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

It isn't a bug. It is an error with floating-point values.

Edited by cppman

Share this post


Link to post
Share on other sites

It isn't a bug. It is an error with floating-point values.

Could you tell me more in detail, please?

Share this post


Link to post
Share on other sites

Could you tell me more in detail, please?

Floating-point values (decimals) are prone to being inaccurate. The more precise (the more numbers behind the decimal) the more of a chance the value will be inaccurate. AutoIt, internally, uses a double to store floating-point values. Since a double can store more decimal places than a 'float', there is more of a chance the value will be inaccurate. If you can see the real values of some floating-point values before some operation is done to them, you'll see a lot of them are inaccurate.

Here's an example of the same problem in C++:

Posted Image

Share this post


Link to post
Share on other sites

Floating-point values (decimals) are prone to being inaccurate. The more precise (the more numbers behind the decimal) the more of a chance the value will be inaccurate. AutoIt, internally, uses a double to store floating-point values. Since a double can store more decimal places than a 'float', there is more of a chance the value will be inaccurate. If you can see the real values of some floating-point values before some operation is done to them, you'll see a lot of them are inaccurate.

Here's an example of the same problem in C++:

Posted Image

Thank you very much!

I think I understood.

Share this post


Link to post
Share on other sites

Yes, floating point values are inaccurate. Because I was not aware of this before, I had a real headache when I discovered it for the first time with another programming language :)

The solution is to always use a formatted string for the outputs. Or, you can round the number first.

$i = 8.04 - 8
$f = StringFormat("%.14f",$i)
$r = Round($i,14)
MsgBox(0, "Format Number", $i & @LF & $f & @LF & $r)

And to be most accurate, you should take only the 14 first SIGNIFICANT digits. This means that when you have 1 digit before the decimal point, get the first 13 digits after the decimal point. If you have 2 digits before the decimal point, take the first 12 digits after the decimal point, and so on.

Share this post


Link to post
Share on other sites

yes, realy thanx everybody for info!

I'v got the point)

Share this post


Link to post
Share on other sites

yes, realy thanx everybody for info!

I'v got the point)

You mean... floating-point.

♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0