Modify

Opened 11 years ago

Closed 11 years ago

#2810 closed Bug (No Bug)

handling of some float numbers

Reported by: anonymous Owned by:
Milestone: Component: AutoIt
Version: 3.3.12.0 Severity: None
Keywords: Cc:

Description

wierd internal rounding.

$n1=0.9999999999999995
$n2=0.9999999999999996
$n3=0.99999999999999994449919099999
$n4=0.999999999999999944499191
ConsoleWrite('IsInt($n1)='&IsInt($n1)&'; $n1='&$n1&@cr) ;IsInt($n1)=0; $n1=0.999999999999999
ConsoleWrite('IsInt($n2)='&IsInt($n2)&'; $n2='&$n2&@cr) ;IsInt($n2)=0; $n2=1
ConsoleWrite('IsInt($n3)='&IsInt($n3)&'; $n3='&$n3&@cr) ;IsInt($n3)=0; $n3=1
ConsoleWrite('IsInt($n4)='&IsInt($n4)&'; $n4='&$n4&@cr) ;IsInt($n4)=1; $n4=1

Change History (1)

comment:1 Changed 11 years ago by jchd18

  • Resolution set to No Bug
  • Status changed from new to closed

Again there is no bug here.

You can invent an infinite number of cases where the result of a floating point calculation differs from what you'd expect. Again it has nothing to do with AutoIt, which uses the relies on C++ compiler to provide "correct" results, "correct" being taken modulo what IEEE754 dictates. Please read http://en.wikipedia.org/wiki/Floating_point

IsInt($n) = 0 applied to a floating point value $n means that $n is not exactly represented as an integer inside the double.

Now, displaying $n by default conversion to string implies rounding to a fix number of decimals, which is not based on the machine epsilon (http://en.wikipedia.org/wiki/Machine_epsilon), nor on the ULP nearby (http://en.wikipedia.org/wiki/Unit_in_the_last_place).

Would you seriously want the following to display anything else than 0.1?

$n = 1/10
ConsoleWrite('Hex($n) = ' & Hex($n) & '; IsInt($n) = ' & IsInt($n) & '; $n = ' & $n & @LF)

Guidelines for posting comments:

  • You cannot re-open a ticket but you may still leave a comment if you have additional information to add.
  • In-depth discussions should take place on the forum.

For more information see the full version of the ticket guidelines here.

Add Comment

Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.