Jump to content

Failing basic math

Recommended Posts

I have a problem with writing a script - seemingly not related to functions, syntax or anything relating to actual code... but rather, basic math errors. After a run of simple addition/subtraction routines, the script suddenly starts spouting gibberish, missing the mark by a tiny fraction {10^-12 ish}; normally, for whatever reason that happened, I'd simply round the number and to hell with it, but it happens even then.

The attached script and output file should explain the results better - all three values are supposed to gradually increase {by a factor of 0.1} to the rate of 1 per line, and after 100 steps, decrease by the same method. Unfortunately, a bit after the dropoff point, things pretty much get shot to hell and I can't figure out why.

Any ideas?


Link to comment
Share on other sites

Okay, so I'm guessing that I should round it down because... stuff. What is the "tolerance of a Double Word Floating Point number" in AutoIt, i.e., what number of decimal places should I round it down to? Googling says 11, but I should probably double-check.

EDIT: Also, is there a way to force the script to calculate only up to a set precision? Adding Round(....) to every math function in the script is going to get real messy real fast.

Edited by Quinch
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Create New...