Jump to content

1.003 * 1000 = 1002.99999989


Recommended Posts

First i don't know if this is something that you already know. But eater i have some error in coding or AutoIT wasn't designed to do this way.

Its a simple function that calculates four digits in decimal number, eg. 1.256 returns 1256, at least what it suppose to do.

But when i run the code below instead of numbers 1003 i get 1002. It turned out that when 1.003 gets multiplied with 1000 it becomes 1002.99999999989 instead of 1003.

So is this normal because of float number representation in PC or something else I'm not aware of?

Func Temperature($T)
Local $C0, $C1, $C2, $C3
$T = $T *1000
$C0 = Int($T / 1000)
$C1 = Int(($T - ($C0 * 1000)) / 100)
$C2 = Int(($T - ($C0 * 1000) - ($C1 *100))/10)
$C3 = Int($T - ($C0 * 1000) - ($C1 *100) - ($C2 *10))
ToolTip($C0 & $C1 & $C2 & $C3,500,0)
Sleep(3000)
EndFunc

Temperature(1.003)
Link to comment
Share on other sites

That's a problem with the internal representation of decimal numbers as binary. There has been a lengthy discussion about this lately. I will post the link as soon as I find it.

Edit: Here is the

Edited by water

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2022-02-19 - Version 1.6.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2017-07-21 - Version 0.4.0.1) - Download - General Help & Support - Example Scripts
OutlookEX (2021-11-16 - Version 1.7.0.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX_GUI (2021-04-13 - Version 1.4.0.0) - Download
Outlook Tools (2019-07-22 - Version 0.6.0.0) - Download - General Help & Support - Wiki
PowerPoint (2021-08-31 - Version 1.5.0.0) - Download - General Help & Support - Example Scripts - Wiki
Task Scheduler (NEW 2022-07-28 - Version 1.6.0.1) - Download - General Help & Support - Wiki

Standard UDFs:
Excel - Example Scripts - Wiki
Word - Wiki

Tutorials:
ADO - Wiki
WebDriver - Wiki

 

Link to comment
Share on other sites

from help file

Example

Local $var = Int(10.793) ;$var is the integer 10

so why are you using Int? shudnt you use

$C3 = Round($somedata, 0) ?!

Edited by bogQ

TCP server and client - Learning about TCP servers and clients connection
Au3 oIrrlicht - Irrlicht project
Au3impact - Another 3D DLL game engine for autoit. (3impact 3Drad related)



460px-Thief-4-temp-banner.jpg
There are those that believe that the perfect heist lies in the preparation.
Some say that it’s all in the timing, seizing the right opportunity. Others even say it’s the ability to leave no trace behind, be a ghost.

 
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...