# Help With Math Accuracy.

## Recommended Posts

Guys,

I'm putting a library of permutation routines together and I've stumbled on a problem that only occurs at the 'higher' end of the numeric values. For example, I'm converting a number from decimal to factoradic format. Not wanting to bore you too much but 533 can be converted to 4(5!) + 2(4!) + 0(3!) + 2(2!) + 1(1!). You'll have to accept that in AutoIT the highest factorial you can calculate is 20! before it exceeds the maximum precision the data type can accomodate. This is all well.

My problem occurs when I try to do the same conversion but using 20! - 1. Using .Net as my 'gold' standard where the same routine works 20! - 1 would convert to 1(19!) + 2(18!) + 3(17!) ... + 19(1!).

My first two numbers in the sequence would be 243290200817663999 / 121645100408832000.

Multiply 121645100408832000 * 20 and you get 243290200817664000 so the division should be 19.9999 or something like it. In my .Net I truncate the division to get 19 but in AutoIT the truncation yields 20 and that renders the rest of the sequence invalid. I know that AutoIT uses variants but I'd really like to ask if there is a coding solution that would give me the result I'm after?

TIA.

Edited by Peter Hamilton-Scott

##### Share on other sites

use Int() it lowers any number to an integer thus 19.9999 becomes 19

My Programs:AInstall - Create a standalone installer for your programUnit Converter - Converts Length, Area, Volume, Weight, Temperature and Pressure to different unitsBinary Clock - Hours, minutes and seconds have 10 columns each to display timeAutoIt Editor - Code Editor with Syntax Highlighting.Laserix Editor & Player - Create, Edit and Play Laserix LevelsLyric Syncer - Create and use Synchronised Lyrics.Connect 4 - 2 Player Connect 4 Game (Local or Online!, Formatted Chat!!)MD5, SHA-1, SHA-256, Tiger and Whirlpool Hash Finder - Dictionary and Brute Force FindCool Text Client - Create Rendered ImageMy UDF's:GUI Enhance - Enhance your GUIs visually.IDEA File Encryption - Encrypt and decrypt files easily! File Rename - Rename files easilyRC4 Text Encryption - Encrypt text using the RC4 AlgorithmPrime Number - Check if a number is primeString Remove - remove lots of strings at onceProgress Bar - made easySound UDF - Play, Pause, Resume, Seek and Stop.

##### Share on other sites

use Int() it lowers any number to an integer thus 19.9999 becomes 19

I have tried that. Here's a code sample you could try for me.

```local \$a = 2432902008176639999; 20! - 1
local \$b = 121645100408832000; 19!
local \$c = Int(\$a / \$b)
MsgBox(0, "Calculated.", \$a & " / " & \$b & " = " & \$c)```

I still get the result 20 with or without the Int. Perhaps there is some precision creepage that either yields 20 on the dot or it transitions from 19.9999 to 20.0001 or something like that?

Edit #1: I changed the code to factor some precision into the calculation. If I use this:

```local \$a = 2432902008176639999; 20! - 1
local \$b = 121645100408832000; 19!
local \$c = Int(\$a / \$b - 0.0000000000001)
MsgBox(0, "Calculated.", \$a & " / " & \$b & " = " & \$c)```

I get the result I'm after. I'll have to see if this artificial nudge affects the other calculations.

Edit #2: Attached screenshot shows the code I'm using and the result of the calculation I'm getting.

Edited by Peter Hamilton-Scott

##### Share on other sites

I found a workaround. I just need to subtract 1 from the count by checking if the count * factorial exceeds the value I'm using. This works and I now get consistent results (so far).

```local \$upper = UBound(\$result) - 1, \$count
for \$i = 0 to \$upper
\$factorial = Factorial(UBound(\$result) - \$i)
\$count = Int(\$p_n / \$factorial)
if \$count * \$factorial > \$p_n then
\$count -= 1
endif
\$p_n -= \$count * \$factorial
\$result[\$i] = \$count
if \$p_n = 0 then
exitLoop
endif
next```

Apologies for hijacking my own thread. I just thought you'd like to know how it progressed.

Edited by Peter Hamilton-Scott

##### Share on other sites

AutoIt has some auto-correction built-in to fix floating point numbers that are within a certain representational distance from the nearest whole number when using Int(). There were some circumstances where using Int() on a "whole" floating point number would produce an integer that was one below the expected value. The obvious side effect to the auto-correction is that sometimes, the value really does need truncated to that value.

## Create an account

Register a new account

×

• Wiki

• Back

• Git