Sign in to follow this  
Followers 0
rabit

calculation errors?

10 posts in this topic

can anyone help?

I need to know every digit of a number which is larger than 10^15 and which could not be shown by calculator or AutoIt3. So I use an array for storing each digit and then for showing all the elements of the array.

but I found one thing hard to understand. For $x = 100000001 but not other some numbers, why are the calculation results of $x^2 different, of which the latter is obviously right?

Below is the script I wrote.

====================================

$x = 100000001

$d = $x^2

$t= Int(Log($d)/Log(10))

dim $array[$t+1]

$out = ""

For $i = 0 To $t

$array[$i] = Int($d/10^($t-$i))

$out = $out & $array[$i]

$d = $d - Int($d/10^($t-$i))*10^($t-$i)

Next

Run("calc.exe")

WinWaitActive("Calculator")

AutoItSetOption("SendKeyDelay", 200)

ClipPut($x)

Send("^v")

Send("*")

Send("^v")

Send("=")

Send("^c")

;Send("100000001*100000001=")

$out2 = ClipGet()

WinClose("Calculator")

WinWaitClose("Calculator")

msgbox(0,"Comparison","by array: " & @CRLF &_

" " & $x & "^2=" & $out & @CRLF &_

@CRLF & @CRLF &_

"by calculator: " & @CRLF &_

" " & $x & "^2=" & $out2)

Share this post


Link to post
Share on other sites



and here is the result of the script.

still can not figure out why.

Share this post


Link to post
Share on other sites

if $x is changed to 10012547593376, the result attached still showed this error.

Share this post


Link to post
Share on other sites

but if $x = 100000002, both results are right.

Share this post


Link to post
Share on other sites

it looks like the power computation have a small precision problem

$x^2 not equal to $x*$x :">

Share this post


Link to post
Share on other sites

it looks like the power computation have a small precision problem

$x^2 not equal to $x*$x :">

<{POST_SNAPBACK}>

sounds possible. I then changed $d = $x^2 to $d = $x*$x. however, another result appeared (see attachement).

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

You know there's an edit button, Right rabit?

Edited by the_lord_mephy

My site for HTML Help :)[quote name='Valik' date='Oct 15 2004, 12:29 PM']Maybe nobody is an "elite uber-coder" like me because thinking is a capital offense in today's online-world?[right][snapback]36427[/snapback][/right][/quote]

Share this post


Link to post
Share on other sites

You get a precision/rounding problem during the x^2 operation because 100 000 001*100 000 001 will need a precision of more than 16 digits. (I am not sure of more but at least). This rounding operation is not done during x*x so that the reason of the difference.

You can notice that 100 000 002 does not need this extra digit so the result look ok.

But 100 000 003 will produce the same discrepancy.

as mention in the help (Language reference/datatype)

you cannot expect a 16 digits precision.

A "double precision" number which is a 15 digit precision number in the range 1.7E308 to 1.7E+308.  (Stored internally as 8 bytes)

Share this post


Link to post
Share on other sites

You know there's an edit key, Right rabit?

<{POST_SNAPBACK}>

sorry, I can not understand what did you mean by "there's an edit key". :)

I edited the script with UltraEdit.

Share this post


Link to post
Share on other sites

You get a precision/rounding problem during the x^2 operation because 100 000 001*100 000 001 will need a precision of more than 16 digits. (I am not sure of more but at least). This rounding operation is not done during x*x so that the reason of the difference.

You can notice that 100 000 002 does not need this extra digit so the result look ok.

But  100 000 003 will produce the same discrepancy.

as mention in the help (Language reference/datatype)

you cannot expect a 16 digits precision.

<{POST_SNAPBACK}>

I know the limitations. now it seems i can not avoid the precision/rounding problem by cutting a digit from left (highest) to an array element. :)

then do you know a way to show all digits of a large number, for example, 2^100?

thanks!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0