marko001 Posted October 9, 2014 Share Posted October 9, 2014 Hi all, i'm becoming crazy since i'm finding several errors when relating numeric expressions each others. Example: $var1 = 2 $var2 = 0.50 $sum = $var1 * $var2 Result is NOT exactly 1 but something like 1.000000000000000000123 and using round() doesn't solve. Generally speaking i realize that too often simple calcs generates micro-decimals that invalid relations. I also tried string() but with no success. Is there a way to *exactly* compare numbers with no margin of errors? Thanks a lot, Marco Link to comment Share on other sites More sharing options...
UEZ Posted October 9, 2014 Share Posted October 9, 2014 (edited) Well, for me it works as expected:ConsoleWrite(1 / 2 & @CRLF) ConsoleWrite(0.5 * 2 & @CRLF) ConsoleWrite(1 / 3 * 3 & @CRLF) ConsoleWrite(5 / 9 & @CRLF) ConsoleWrite(677 / 1375 & @CRLF) ConsoleWrite(ACos(-1) & @CRLF) ConsoleWrite(Sin(ACos(-1) / 2) & @CRLF)Results:0.5 1 1 0.555555555555556 0.492363636363636 3.14159265358979 1Br,UEZ Edited October 9, 2014 by UEZ Please don't send me any personal message and ask for support! I will not reply! Selection of finest graphical examples at Codepen.io The own fart smells best! ✌Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!¯\_(ツ)_/¯ ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ Link to comment Share on other sites More sharing options...
Spider001 Posted October 9, 2014 Share Posted October 9, 2014 for me it works as expected on xp Link to comment Share on other sites More sharing options...
marko001 Posted October 9, 2014 Author Share Posted October 9, 2014 This obviously leads to correct values if the computation is simple. What I say is that after lot of calculations on same iniital value, I loose some decimals. If I iterate calculations, it happens. Usually it's multiplying an integer with a 2 decimal value (i.e 2*.25) M. Link to comment Share on other sites More sharing options...
Spider001 Posted October 9, 2014 Share Posted October 9, 2014 that gives a wrong number #include <MsgBoxConstants.au3> $var1 = 2 $var2 = 0.00000000001 $sum = $var1 * $var2 MsgBox($MB_SYSTEMMODAL, "Title", StringFormat ("%.40f", $sum)) Link to comment Share on other sites More sharing options...
Moderators Melba23 Posted October 9, 2014 Moderators Share Posted October 9, 2014 marko001,This is a problem that affects ALL computers regardless of the language used - google "floating point arithmetic" to learn all about it. One of the normal workarounds when, as you say you are, dealing with 2 decimal place values (often currency) is to convert all the decimal values into integer values and then reconvert the result once the calculation has been made. M23 Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind Open spoiler to see my UDFs: Spoiler ArrayMultiColSort ---- Sort arrays on multiple columnsChooseFileFolder ---- Single and multiple selections from specified path treeview listingDate_Time_Convert -- Easily convert date/time formats, including the language usedExtMsgBox --------- A highly customisable replacement for MsgBoxGUIExtender -------- Extend and retract multiple sections within a GUIGUIFrame ---------- Subdivide GUIs into many adjustable framesGUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView itemsGUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeViewMarquee ----------- Scrolling tickertape GUIsNoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxesNotify ------------- Small notifications on the edge of the displayScrollbars ----------Automatically sized scrollbars with a single commandStringSize ---------- Automatically size controls to fit textToast -------------- Small GUIs which pop out of the notification area Link to comment Share on other sites More sharing options...
marko001 Posted October 9, 2014 Author Share Posted October 9, 2014 This is what I thought but hoped in a possible workaround. I'll try what you suggested and I'll be back with results. (I knew an aswer would have come from you ) Link to comment Share on other sites More sharing options...
UEZ Posted October 9, 2014 Share Posted October 9, 2014 For a workaround it depends what result you want from the calculation. 1.000000000000000000123 = 1? or 0.000000000019999999999999999 = 0?Can you provide some examples what the calculation is and what result you are expecting.Br,UEZ Please don't send me any personal message and ask for support! I will not reply! Selection of finest graphical examples at Codepen.io The own fart smells best! ✌Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!¯\_(ツ)_/¯ ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ Link to comment Share on other sites More sharing options...
marko001 Posted October 9, 2014 Author Share Posted October 9, 2014 I need precision on 2nd decimal. Based on your example: 1.000000000000000000123 = 1 0.000000000019999999999999999 = 0 I'm trying to replicate errors and I'll post you values and functions. M. Link to comment Share on other sites More sharing options...
UEZ Posted October 9, 2014 Share Posted October 9, 2014 Then this should do your job: ConsoleWrite(StringFormat ("%.2f", 0.065934857935793) & @CRLF) ConsoleWrite(StringFormat ("%.2f", 10.000023) & @CRLF) ConsoleWrite(StringFormat ("%.2f", ACos(-1)) & @CRLF) ConsoleWrite(StringFormat ("%.2f", 1.009) & @CRLF) Br,UEZ Please don't send me any personal message and ask for support! I will not reply! Selection of finest graphical examples at Codepen.io The own fart smells best! ✌Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!¯\_(ツ)_/¯ ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ Link to comment Share on other sites More sharing options...
jchd Posted October 9, 2014 Share Posted October 9, 2014 As Melba just said, there is nothing wrong here. You just can't expect an exact representation of most real values, let alone the result of any computation involving those values that can't represent exactly in limited precision binary floating point. This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe hereRegExp tutorial: enough to get startedPCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta. SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt) Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now