bradley1 Posted March 23, 2006 Share Posted March 23, 2006 Hey guys/girls, semi new user here, sorry if this question has been asked before but I couldn't find a similar topic: Sometimes when I am working with numbers as variables, such as subtracting or adding, something strange happens. A 1 will automatically become a .99999, or a 4 - 2 will become a 2.000001 instead of just plain 2. Where does this come from and is there an easy way to correct it? Thank you, Brad Link to comment Share on other sites More sharing options...
BigDod Posted March 23, 2006 Share Posted March 23, 2006 Hey guys/girls, semi new user here, sorry if this question has been asked before but I couldn't find a similar topic: Sometimes when I am working with numbers as variables, such as subtracting or adding, something strange happens. A 1 will automatically become a .99999, or a 4 - 2 will become a 2.000001 instead of just plain 2. Where does this come from and is there an easy way to correct it? Thank you, BradI do not know why other than the default no of decimal places being 6. If you know that you are going to have whole nos type 0.9999 into $input = InputBox('enter a number', 'moo') $input = StringFormat("%.0f", $input) MsgBox(0, '', $input) Time you enjoyed wasting is not wasted time ......T.S. Elliot Suspense is worse than disappointment................Robert Burns God help the man who won't help himself, because no-one else will...........My Grandmother Link to comment Share on other sites More sharing options...
Moderators SmOke_N Posted March 23, 2006 Moderators Share Posted March 23, 2006 Hey guys/girls, semi new user here, sorry if this question has been asked before but I couldn't find a similar topic:Sometimes when I am working with numbers as variables, such as subtracting or adding, something strange happens. A 1 will automatically become a .99999, or a 4 - 2 will become a 2.000001 instead of just plain 2. Where does this come from and is there an easy way to correct it? Thank you,BradMust admit... I've never had 4 - 2 become 2.000001 Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer. Link to comment Share on other sites More sharing options...
Valik Posted March 23, 2006 Share Posted March 23, 2006 It's perfectly normal and you can learn to correct it by researching floating point numbers which explains why the issue occurs and provides some solutions to some common problems when dealing with floating point numbers. Link to comment Share on other sites More sharing options...
bradley1 Posted March 25, 2006 Author Share Posted March 25, 2006 It's perfectly normal and you can learn to correct it by researching floating point numbers which explains why the issue occurs and provides some solutions to some common problems when dealing with floating point numbers.Thank you for your help everyone, I think I have figured it out. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now