cycotron Posted January 31, 2012 Share Posted January 31, 2012 (edited) Please forgive me if this has been answered before, because I'm sure it has, but i couldn't find the information. Why is it that when I do the following: ConsoleWrite(.817 - .79 & @crlf) The result is "0.0269999999999999" Should the result not be "0.027"? Is there a standard way of working with decimals other than having to Round() every one of my results? Edited January 31, 2012 by cycotron Link to comment Share on other sites More sharing options...
Moderators Melba23 Posted January 31, 2012 Moderators Share Posted January 31, 2012 cycotron,Welcome to the wonderful world of "binary floating point". This is not an AutoIt problem - all computers suffer the same way. Google will tell you more than you ever wanted to know if you look. I am afraid you will have to Round your results - at least you have that function to use. M23 Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind Open spoiler to see my UDFs: Spoiler ArrayMultiColSort ---- Sort arrays on multiple columnsChooseFileFolder ---- Single and multiple selections from specified path treeview listingDate_Time_Convert -- Easily convert date/time formats, including the language usedExtMsgBox --------- A highly customisable replacement for MsgBoxGUIExtender -------- Extend and retract multiple sections within a GUIGUIFrame ---------- Subdivide GUIs into many adjustable framesGUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView itemsGUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeViewMarquee ----------- Scrolling tickertape GUIsNoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxesNotify ------------- Small notifications on the edge of the displayScrollbars ----------Automatically sized scrollbars with a single commandStringSize ---------- Automatically size controls to fit textToast -------------- Small GUIs which pop out of the notification area Link to comment Share on other sites More sharing options...
cycotron Posted January 31, 2012 Author Share Posted January 31, 2012 Thank you Melba23. I thought I remembered something along those lines that I read a year or two ago. I will use Round, it's simple enough Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now