Jump to content

Scientific notation problems


Recommended Posts

I am having some issues with AutoIt outputting scientific notation as opposed to decimal figures.

Here is my problem...

0 + 0.0000001 returns the value in scientific notation (1e-007)

anything other than zero added to the value returns the value in decimal form

0.01 + 0.0000001 returns 0.0100001

I know there are workarounds, but shouldn't this be consistent?

Link to comment
Share on other sites

Link to comment
Share on other sites

I thnk, this is because of more compact representation... At least this is the term which is used for the 'g'-Flag in StringFormat

g Float Signed value printed in f or e format, whichever is more compact for the given value and precision. The e format is used only when the exponent of the value is less than -4 or greater than or equal to the precision argument. Trailing zeros are truncated, and the decimal point appears only if one or more digits follow it.

0.0001001 -> 9 characters

1.001e-004 -> 10 characters

Edited by ProgAndy

*GERMAN* [note: you are not allowed to remove author / modified info from my UDFs]My UDFs:[_SetImageBinaryToCtrl] [_TaskDialog] [AutoItObject] [Animated GIF (GDI+)] [ClipPut for Image] [FreeImage] [GDI32 UDFs] [GDIPlus Progressbar] [Hotkey-Selector] [Multiline Inputbox] [MySQL without ODBC] [RichEdit UDFs] [SpeechAPI Example] [WinHTTP]UDFs included in AutoIt: FTP_Ex (as FTPEx), _WinAPI_SetLayeredWindowAttributes

Link to comment
Share on other sites

I thnk, this is because of more compact representation... At least this is the term which is used for the 'g'-Flag in StringFormat

0.0001001 -> 9 characters

1.001e-004 -> 10 characters

I think you are right, ProgAndy.
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...