Failing basic math

[Quinch]

I have a problem with writing a script - seemingly not related to functions, syntax or anything relating to actual code... but rather, basic math errors. After a run of simple addition/subtraction routines, the script suddenly starts spouting gibberish, missing the mark by a tiny fraction {10^-12 ish}; normally, for whatever reason that happened, I'd simply round the number and to hell with it, but it happens even then.

The attached script and output file should explain the results better - all three values are supposed to gradually increase {by a factor of 0.1} to the rate of 1 per line, and after 100 steps, decrease by the same method. Unfortunately, a bit after the dropoff point, things pretty much get shot to hell and I can't figure out why.

Any ideas?

Desktop.rar

[Richard Robertson]

When reporting a problem, it's best to present the smallest error producing code possible. I haven't the time to sit and understand this example, sorry.

[weaponx]

See here:

http://www.autoitscript.com/forum/index.ph...c=62590&hl=

#478133

[Quinch]

See here:

http://www.autoitscript.com/forum/index.ph...c=62590&hl=

#478133

Okay, so I'm guessing that I should round it down because... stuff. What is the "tolerance of a Double Word Floating Point number" in AutoIt, i.e., what number of decimal places should I round it down to? Googling says 11, but I should probably double-check.

EDIT: Also, is there a way to force the script to calculate only up to a set precision? Adding Round(....) to every math function in the script is going to get real messy real fast.

Edited May 27, 2008 by Quinch