# precision problem...

This topic is 3828 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

I'm doing some scientific calcs that need at least decimal type precision. But the System.Math namespace only deals in doubles. What can I do? -Devin

##### Share on other sites
Some quick googling shows that their are some high-precision and arbitrary-precision math libraries freely available, although all the ones I've seen are C/C++ libraries. However it's very easy to p/invoke into DLL's, so that shouldn't be a problem if you're working in a managed environment.

##### Share on other sites
Great, thanks, I'm thinking about creating my own number from one of those examples. Is there a guide like from ieee or something for doing such? Perhaps in binary?

##### Share on other sites
There is of course System.Decimal which gives around 28 decimal digits of precision.

##### Share on other sites
Ya know I would love to use system.decimal and tried. It worked super but when I went to do a square root, I found there was no such method at that precision. Does anyone know how to do a square root at this precision?

Thanks again,

Devin

##### Share on other sites
It only deals in floats/doubles because that's all the CPU can do natively. To perform a square root on a decimal type, you'll have to do it yourself.

One very simple way to do it is to use the Babylonian method. Start with an approximation, and iteratively apply a formula to approximate the square root.

For example:
class Program{	static Decimal Sqrt(Decimal s)	{		const int Iterations = 2;		Decimal x = (Decimal)Math.Sqrt((Double)s); // Start with a good approximation		// Iteratively apply Babylonian method, using the arithmetic mean as an approximation for the geometric mean		for (int i = 0; i < Iterations; ++i)		{			x = (x + s / x) * 0.5m;		}		return x;	}	static void Main(string[] args)	{		Decimal s = 2.0m;		Console.WriteLine("Square root of 2");		Console.WriteLine("Double:       {0:F28}", Math.Sqrt((Double)s));		Console.WriteLine("Decimal:      {0:F28}", Sqrt(s));		Console.WriteLine("Actual value: 1.414213562373095048801688724209698...");		Console.ReadKey();	}}

The more iterations you do, the more accurate the results. Although realistically, I think you'd only need to do maybe 1 or 2 iterations before you reach the maximum precision of the decimal type.

##### Share on other sites
Great, this seems to work although I have not verified the calc yet. But that is super and adds sqrt to decimal which handles my calcs.

Thanks Much.

• ### What is your GameDev Story?

In 2019 we are celebrating 20 years of GameDev.net! Share your GameDev Story with us.

• 11
• 11
• 15
• 11
• 11
• ### Forum Statistics

• Total Topics
634149
• Total Posts
3015834
×