Jump to content
Sign in to follow this  
luciffer

1.003 * 1000 = 1002.99999989

Recommended Posts

luciffer

First i don't know if this is something that you already know. But eater i have some error in coding or AutoIT wasn't designed to do this way.

Its a simple function that calculates four digits in decimal number, eg. 1.256 returns 1256, at least what it suppose to do.

But when i run the code below instead of numbers 1003 i get 1002. It turned out that when 1.003 gets multiplied with 1000 it becomes 1002.99999999989 instead of 1003.

So is this normal because of float number representation in PC or something else I'm not aware of?

Func Temperature($T)
Local $C0, $C1, $C2, $C3
$T = $T *1000
$C0 = Int($T / 1000)
$C1 = Int(($T - ($C0 * 1000)) / 100)
$C2 = Int(($T - ($C0 * 1000) - ($C1 *100))/10)
$C3 = Int($T - ($C0 * 1000) - ($C1 *100) - ($C2 *10))
ToolTip($C0 & $C1 & $C2 & $C3,500,0)
Sleep(3000)
EndFunc

Temperature(1.003)

Share this post


Link to post
Share on other sites
water

That's a problem with the internal representation of decimal numbers as binary. There has been a lengthy discussion about this lately. I will post the link as soon as I find it.

Edit: Here is the

Edited by water

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2018-06-01 - Version 1.4.9.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (2018-01-27 - Version 1.3.3.1) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites
bogQ

from help file

Example

Local $var = Int(10.793) ;$var is the integer 10

so why are you using Int? shudnt you use

$C3 = Round($somedata, 0) ?!

Edited by bogQ

TCP server and client - Learning about TCP servers and clients connection
Au3 oIrrlicht - Irrlicht project
Au3impact - Another 3D DLL game engine for autoit. (3impact 3Drad related)



460px-Thief-4-temp-banner.jpg
There are those that believe that the perfect heist lies in the preparation.
Some say that it’s all in the timing, seizing the right opportunity. Others even say it’s the ability to leave no trace behind, be a ghost.

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×