Jump to content

possible math inconsistency


Recommended Posts

Oh ok - i searched before but didn't found this ticket.
Anyway - the argumentation there is simply wrong.
It has nothing to do with a "max precision 15/16 digits" as it stated there.
There is no rounding needed because 2^49 can be mapped completely without rounding errors in double.
As jchd clearly explained, the rounding errors are to be expected only from 2^53. So here is a different problem.

Also, the internal type conversion in C/C++ does not show this problem.
So if it's possible to cast double to int64 without errors in C/C++ then it should be possible in AutoIt too.
I have explained all this there also so plausibly and think therefore that it is recognized this time correctly as a bug.
 

Edited by AspirinJunkie
Link to comment
Share on other sites

8 hours ago, AspirinJunkie said:

I have explained all this there also so plausibly and think therefore that it is recognized this time correctly as a bug.

Yes, however I think there is merit in referencing this ticket in your new ticket in so it doesn’t get marked as a duplicate.

Code hard, but don’t hard code...

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...