Sign in to follow this  
Followers 0
andybiochem

Simple number problem

3 posts in this topic

#1 ·  Posted (edited)

Hi!

I have searched the forum for topics similar to this problem and couldn't find anything related.

I'm writing a program that uses some very large numbers, and I need to define these numbers as variables.

Why does this happen when I try to define a variable??:

post-29091-1215879492_thumb.gif

This doesn't happen when I define the number as a string inside speech marks....is it just that I cannot define such a large number as a number?

In the help-file FAQ it says:

Number range (floating point): 1.7E–308 to 1.7E+308 with 15-digit precision

what is meant by 15 digit precision exactly?

Edited by andybiochem

- Table UDF - create simple data tables - Line Graph UDF GDI+ - quickly create simple line graphs with x and y axes (uses GDI+ with double buffer) - Line Graph UDF - quickly create simple line graphs with x and y axes (uses AI native graphic control) - Barcode Generator Code 128 B C - Create the 1/0 code for barcodes. - WebCam as BarCode Reader - use your webcam to read barcodes - Stereograms!!! - make your own stereograms in AutoIT - Ziggurat Gaussian Distribution RNG - generate random numbers based on normal/gaussian distribution - Box-Muller Gaussian Distribution RNG - generate random numbers based on normal/gaussian distribution - Elastic Radio Buttons - faux-gravity effects in AutoIT (from javascript)- Morse Code Generator - Generate morse code by tapping your spacebar!

Share this post


Link to post
Share on other sites



Hi!

I have searched the forum for topics similar to this problem and couldn't find anything related.

I'm writing a program that uses some very large numbers, and I need to define these numbers as variables.

Why does this happen when I try to define a variable??:

post-29091-1215879492_thumb.gif

This doesn't happen when I define the number as a string inside speech marks....is it just that I cannot define such a large number as a number?

In the help-file FAQ it says:

Number range (floating point): 1.7E308 to 1.7E+308 with 15-digit precision

what is meant by 15 digit precision exactly?

This means that a number can only have 15 digits.

You are using 21 digits in the example shown.

Jos


Visit the SciTE4AutoIt3 Download page for the latest versions  - Beta files                                How to post scriptsource        Forum Rules
 
Live for the present,
Dream of the future,
Learn from the past.
  :)

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

This means that a number can only have 15 digits.

You are using 21 digits in the example shown.

Jos

cool, I get it. Thanks!

Edited by andybiochem

- Table UDF - create simple data tables - Line Graph UDF GDI+ - quickly create simple line graphs with x and y axes (uses GDI+ with double buffer) - Line Graph UDF - quickly create simple line graphs with x and y axes (uses AI native graphic control) - Barcode Generator Code 128 B C - Create the 1/0 code for barcodes. - WebCam as BarCode Reader - use your webcam to read barcodes - Stereograms!!! - make your own stereograms in AutoIT - Ziggurat Gaussian Distribution RNG - generate random numbers based on normal/gaussian distribution - Box-Muller Gaussian Distribution RNG - generate random numbers based on normal/gaussian distribution - Elastic Radio Buttons - faux-gravity effects in AutoIT (from javascript)- Morse Code Generator - Generate morse code by tapping your spacebar!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0