goldenix Posted April 22, 2009 Share Posted April 22, 2009 the title says all. I also tried to use my pocet calc & my PC calc & both result in 0.2045 is my calculator calculating wrong? what can I do about this? & why is this happening? $a = 93.4112 $b = 93.6157 ConsoleWrite($b - $a & @CRLF) Exit My Projects:[list][*]Guide - ytube step by step tut for reading memory with autoitscript + samples[*]WinHide - tool to show hide windows, Skinned With GDI+[*]Virtualdub batch job list maker - Batch Process all files with same settings[*]Exp calc - Exp calculator for online games[*]Automated Microsoft SQL Server 2000 installer[*]Image sorter helper for IrfanView - 1 click opens img & move ur mouse to close opened img[/list] Link to comment Share on other sites More sharing options...
Moderators Melba23 Posted April 22, 2009 Moderators Share Posted April 22, 2009 goldenix, A well-known problem with floating point number arithmetic. Search the forum or Google it and all (except an easy solution!) will be revealed. M23 Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind Open spoiler to see my UDFs: Spoiler ArrayMultiColSort ---- Sort arrays on multiple columnsChooseFileFolder ---- Single and multiple selections from specified path treeview listingDate_Time_Convert -- Easily convert date/time formats, including the language usedExtMsgBox --------- A highly customisable replacement for MsgBoxGUIExtender -------- Extend and retract multiple sections within a GUIGUIFrame ---------- Subdivide GUIs into many adjustable framesGUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView itemsGUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeViewMarquee ----------- Scrolling tickertape GUIsNoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxesNotify ------------- Small notifications on the edge of the displayScrollbars ----------Automatically sized scrollbars with a single commandStringSize ---------- Automatically size controls to fit textToast -------------- Small GUIs which pop out of the notification area Link to comment Share on other sites More sharing options...
AlmarM Posted April 22, 2009 Share Posted April 22, 2009 Round will do the trick. $a = 93.4112 $b = 93.6157 ConsoleWrite(Round($b - $a, 4) & @CRLF) Exit AlmarM Minesweeper A minesweeper game created in autoit, source available. _Mouse_UDF An UDF for registering functions to mouse events, made in pure autoit. 2D Hitbox Editor A 2D hitbox editor for quick creation of 2D sphere and rectangle hitboxes. Link to comment Share on other sites More sharing options...
goldenix Posted April 22, 2009 Author Share Posted April 22, 2009 what if the end sesult looks like this? I also thought about round, but what if the result looks like this: 0.20450000000002 or 0.204500092610 or 0.204506004008008001 the calculations wont be correct then would they? Yes I understand, if you want to display final result & if the nr is not that important this is ok, but if you need to continue calculations & you want to showthe user correct nr. then I cant use this now can I? My Projects:[list][*]Guide - ytube step by step tut for reading memory with autoitscript + samples[*]WinHide - tool to show hide windows, Skinned With GDI+[*]Virtualdub batch job list maker - Batch Process all files with same settings[*]Exp calc - Exp calculator for online games[*]Automated Microsoft SQL Server 2000 installer[*]Image sorter helper for IrfanView - 1 click opens img & move ur mouse to close opened img[/list] Link to comment Share on other sites More sharing options...
Malkey Posted April 23, 2009 Share Posted April 23, 2009 (edited) goldenixMaybe something like this would help.; $a = 93.4112 $b = 93.61570001 ConsoleWrite(Round($b - $a, _MaxLen($a, $b)) & @CRLF) Func _MaxLen($sStr1, $sStr2) Local $iLen1 = StringLen(StringRegExpReplace($sStr1, "(.*)\.(.*)", "\2")) Local $iLen2 = StringLen(StringRegExpReplace($sStr2, "(.*)\.(.*)", "\2")) If $iLen1 >= $iLen2 Then Return $iLen1 Else Return $iLen2 EndIf EndFunc ;==>_MaxLen ;or using "BigNum.au3" UDF fromhttp://www.autoitscript.com/forum/index.ph...st&p=597491; #include "BigNum.au3" ;Get BigNum.au3 from ; http://www.autoitscript.com/forum/index.php?showtopic=83529&st=0&p=597491&hl=_BigNum_Add&#entry597491 $a = '-93.4112' $b = '93.4112000001' ConsoleWrite(_BigNum_Add($b, $a) & @CRLF) ; Edited April 23, 2009 by Malkey Link to comment Share on other sites More sharing options...
Richard Robertson Posted April 23, 2009 Share Posted April 23, 2009 Floating point numbers are pretty much fail. They can be used approximately but will never be accurate. Values like 0.1 actually don't even have a representation. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now