Jump to content

calculation error


Recommended Posts

I have some numbers like below and the final $temp amount should be 450 exact but the calculation ended up with 450.00000000001..

i have no idea what's going wrong ?

$p = 4592.05
$1 = 679.41
$2 = 661.31
$3 = 660.04
$4 = 648.06
$5 = 963.85
$6 = 529.38

$childsum = 0

$childsum += $1
msgbox(0,"",$childsum)
$childsum += $2
msgbox(0,"",$childsum)
$childsum += $3
msgbox(0,"",$childsum)
$childsum += $4
msgbox(0,"",$childsum)
$childsum += $5
msgbox(0,"",$childsum)
$childsum += $6
msgbox(0,"",$childsum)

$temp = $p -$childsum
msgbox(0,"",$temp)
Link to comment
Share on other sites

  • Moderators

chun914,

As all the amounts have 2 decimal places I would convert all of them to integers (i.e. multiply them by 100) so that I could use integer arithmetic to sum them. Then I would divide by 100 and use Round when I wanted the total back in 2 decimal place format. ;)

M23

Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...