Jump to content

Numbers - Rounding Off ?


Recommended Posts

Hey guys/girls, semi new user here, sorry if this question has been asked before but I couldn't find a similar topic:

Sometimes when I am working with numbers as variables, such as subtracting or adding, something strange happens. A 1 will automatically become a .99999, or a 4 - 2 will become a 2.000001 instead of just plain 2. Where does this come from and is there an easy way to correct it?

Thank you,

Brad

Link to comment
Share on other sites

Hey guys/girls, semi new user here, sorry if this question has been asked before but I couldn't find a similar topic:

Sometimes when I am working with numbers as variables, such as subtracting or adding, something strange happens. A 1 will automatically become a .99999, or a 4 - 2 will become a 2.000001 instead of just plain 2. Where does this come from and is there an easy way to correct it?

Thank you,

Brad

I do not know why other than the default no of decimal places being 6. If you know that you are going to have whole nos type 0.9999 into

$input = InputBox('enter a number', 'moo')
$input = StringFormat("%.0f", $input)
MsgBox(0, '', $input)


Time you enjoyed wasting is not wasted time ......T.S. Elliot
Suspense is worse than disappointment................Robert Burns
God help the man who won't help himself, because no-one else will...........My Grandmother

Link to comment
Share on other sites

  • Moderators

Hey guys/girls, semi new user here, sorry if this question has been asked before but I couldn't find a similar topic:

Sometimes when I am working with numbers as variables, such as subtracting or adding, something strange happens. A 1 will automatically become a .99999, or a 4 - 2 will become a 2.000001 instead of just plain 2. Where does this come from and is there an easy way to correct it?

Thank you,

Brad

Must admit... I've never had 4 - 2 become 2.000001 :)

Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Link to comment
Share on other sites

It's perfectly normal and you can learn to correct it by researching floating point numbers which explains why the issue occurs and provides some solutions to some common problems when dealing with floating point numbers.

Link to comment
Share on other sites

It's perfectly normal and you can learn to correct it by researching floating point numbers which explains why the issue occurs and provides some solutions to some common problems when dealing with floating point numbers.

Thank you for your help everyone, I think I have figured it out.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...