Sign in to follow this  
Followers 0
marko001

Decimal errors on exact mathematical formules

11 posts in this topic

Hi all,

i'm becoming crazy since i'm finding several errors when relating numeric expressions each others.

Example:

$var1 = 2
$var2 = 0.50
$sum = $var1 * $var2

 

Result is NOT exactly 1 but something like 1.000000000000000000123 and using round() doesn't solve.

Generally speaking i realize that too often simple calcs generates micro-decimals that invalid relations.

I also tried string() but with no success.

Is there a way to *exactly* compare numbers with no margin of errors?

Thanks a lot,

Marco

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

Well, for me it works as expected:

ConsoleWrite(1 / 2 & @CRLF)
ConsoleWrite(0.5 * 2 & @CRLF)
ConsoleWrite(1 / 3 * 3 & @CRLF)
ConsoleWrite(5 / 9 & @CRLF)
ConsoleWrite(677 / 1375 & @CRLF)
ConsoleWrite(ACos(-1) & @CRLF)
ConsoleWrite(Sin(ACos(-1) / 2) & @CRLF)

Results:

0.5
1
1
0.555555555555556
0.492363636363636
3.14159265358979
1

Br,

UEZ

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

for me it works as expected on xp

Share this post


Link to post
Share on other sites

This obviously leads to correct values if the computation is simple.

What I say is that after lot of calculations on same iniital value, I loose some decimals.

If I iterate calculations, it happens.

Usually it's multiplying an integer with a 2 decimal value (i.e 2*.25)

M.

Share this post


Link to post
Share on other sites

that gives a wrong number

#include <MsgBoxConstants.au3>

 $var1 = 2
 $var2 = 0.00000000001
 $sum = $var1 * $var2
MsgBox($MB_SYSTEMMODAL, "Title", StringFormat ("%.40f", $sum))

Share this post


Link to post
Share on other sites

marko001,

This is a problem that affects ALL computers regardless of the language used - google "floating point arithmetic" to learn all about it. ;)

One of the normal workarounds when, as you say you are, dealing with 2 decimal place values (often currency) is to convert all the decimal values into integer values and then reconvert the result once the calculation has been made. :)

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites

This is what I thought but hoped in a possible workaround.

I'll try what you suggested and I'll be back with results.

(I knew an aswer would have come from you :)  )

Share this post


Link to post
Share on other sites

For a workaround it depends what result you want from the calculation. 1.000000000000000000123 = 1? or 0.000000000019999999999999999 = 0?

Can you provide some examples what the calculation is and what result you are expecting.

Br,

UEZ


Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

I need precision on 2nd decimal.

Based on your example:

1.000000000000000000123 = 1

0.000000000019999999999999999 = 0

I'm trying to replicate errors and I'll post you values and functions.

M.

Share this post


Link to post
Share on other sites

Then this should do your job:

 

ConsoleWrite(StringFormat ("%.2f", 0.065934857935793) & @CRLF)
ConsoleWrite(StringFormat ("%.2f", 10.000023) & @CRLF)
ConsoleWrite(StringFormat ("%.2f", ACos(-1)) & @CRLF)
ConsoleWrite(StringFormat ("%.2f", 1.009) & @CRLF)
 

Br,

UEZ


Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

As Melba just said, there is nothing wrong here. You just can't expect an exact representation of most real values, let alone the result of any computation involving those values that can't represent exactly in limited precision binary floating point.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0