Jump to content
Sign in to follow this  
chun914

calculation error

Recommended Posts

I have some numbers like below and the final $temp amount should be 450 exact but the calculation ended up with 450.00000000001..

i have no idea what's going wrong ?

$p = 4592.05
$1 = 679.41
$2 = 661.31
$3 = 660.04
$4 = 648.06
$5 = 963.85
$6 = 529.38

$childsum = 0

$childsum += $1
msgbox(0,"",$childsum)
$childsum += $2
msgbox(0,"",$childsum)
$childsum += $3
msgbox(0,"",$childsum)
$childsum += $4
msgbox(0,"",$childsum)
$childsum += $5
msgbox(0,"",$childsum)
$childsum += $6
msgbox(0,"",$childsum)

$temp = $p -$childsum
msgbox(0,"",$temp)

Share this post


Link to post
Share on other sites

chun914,

As all the amounts have 2 decimal places I would convert all of them to integers (i.e. multiply them by 100) so that I could use integer arithmetic to sum them. Then I would divide by 100 and use Round when I wanted the total back in 2 decimal place format. ;)

M23


Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×
×
  • Create New...