Jump to content
Sign in to follow this  
goldenix

93.6157 - 93.4112 = 0.20450000000001

Recommended Posts

goldenix

the title says all. I also tried to use my pocet calc & my PC calc & both result in 0.2045

is my calculator calculating wrong?

what can I do about this? & why is this happening?

$a = 93.4112
$b = 93.6157

ConsoleWrite($b - $a & @CRLF)
Exit

My Projects:[list][*]Guide - ytube step by step tut for reading memory with autoitscript + samples[*]WinHide - tool to show hide windows, Skinned With GDI+[*]Virtualdub batch job list maker - Batch Process all files with same settings[*]Exp calc - Exp calculator for online games[*]Automated Microsoft SQL Server 2000 installer[*]Image sorter helper for IrfanView - 1 click opens img & move ur mouse to close opened img[/list]

Share this post


Link to post
Share on other sites
Melba23

goldenix,

A well-known problem with floating point number arithmetic. Search the forum or Google it and all (except an easy solution!) will be revealed.

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites
AlmarM

Round will do the trick.

$a = 93.4112
$b = 93.6157

ConsoleWrite(Round($b - $a, 4) & @CRLF)
Exit

AlmarM


Minesweeper

A minesweeper game created in autoit, source available.

_Mouse_UDF

An UDF for registering functions to mouse events, made in pure autoit.

2D Hitbox Editor

A 2D hitbox editor for quick creation of 2D sphere and rectangle hitboxes.

Share this post


Link to post
Share on other sites
goldenix

what if the end sesult looks like this?

I also thought about round, but what if the result looks like this:

0.20450000000002

or

0.204500092610

or

0.204506004008008001

the calculations wont be correct then would they?

Yes I understand, if you want to display final result & if the nr is not that important this is ok, but if you need to continue calculations & you want to showthe user correct nr. then I cant use this now can I?


My Projects:[list][*]Guide - ytube step by step tut for reading memory with autoitscript + samples[*]WinHide - tool to show hide windows, Skinned With GDI+[*]Virtualdub batch job list maker - Batch Process all files with same settings[*]Exp calc - Exp calculator for online games[*]Automated Microsoft SQL Server 2000 installer[*]Image sorter helper for IrfanView - 1 click opens img & move ur mouse to close opened img[/list]

Share this post


Link to post
Share on other sites
Malkey

goldenix

Maybe something like this would help.

;
$a = 93.4112
$b = 93.61570001

ConsoleWrite(Round($b - $a, _MaxLen($a, $b)) & @CRLF)


Func _MaxLen($sStr1, $sStr2)
    Local $iLen1 = StringLen(StringRegExpReplace($sStr1, "(.*)\.(.*)", "\2"))
    Local $iLen2 = StringLen(StringRegExpReplace($sStr2, "(.*)\.(.*)", "\2"))
    If $iLen1 >= $iLen2 Then
        Return $iLen1
    Else
        Return $iLen2
    EndIf
EndFunc  ;==>_MaxLen
;

or using "BigNum.au3" UDF from

http://www.autoitscript.com/forum/index.ph...st&p=597491

;
#include "BigNum.au3"

;Get  BigNum.au3 from
; http://www.autoitscript.com/forum/index.php?showtopic=83529&st=0&p=597491&hl=_BigNum_Add&#entry597491

$a = '-93.4112'
$b = '93.4112000001' 

ConsoleWrite(_BigNum_Add($b, $a) & @CRLF)
;
Edited by Malkey

Share this post


Link to post
Share on other sites
Richard Robertson

Floating point numbers are pretty much fail. They can be used approximately but will never be accurate. Values like 0.1 actually don't even have a representation.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.