I'm having a very weird problem. It's extremely simple to explain. I'm passing the float value 0.01f as a hard-coded parameter to a function. When I place a breakpoint in that function, the value comes up as 0.0099999998. Out of desperation to understand what is going on, I've even tried entering the number as 0.0100000000f. No difference. It's causing a very nasty alignment problem in my editor while working with transformation gizmos. Does anyone understand why this is happening? I'm using Visual Studio 7 .net with C++. Thanks

I was aware of accuracy problems when storing small changes in large numbers, but I didn't know it could happen in such simple numbers.

The values are modifiers for translation/scale gizmos. I could try to align the final value as each gizmo is dragged. But I don't see how even that would be possible, since I have no number to align it to.

I'm afraid that using the translation gizmo to align seperate objects will cause problems if the neighboring vertices are not exactly equal. It would cause visible cracks in the polygons. But perhaps all of the objects will be off by the same error. I can only hope until I test it.

Thanks for your help [smile]

Quote:Original post by Jiia
I was aware of accuracy problems when storing small changes in large numbers, but I didn't know it could happen in such simple numbers.

But actually, it's perfectly sensible.

Quick, write out one third exactly in decimal notation. What, isn't that a simple number?

Quote:Original post by Zahlman
But actually, it's perfectly sensible.

Quick, write out one third exactly in decimal notation. What, isn't that a simple number?

Bah, it's a terrible flaw. I say we start over. ìÜèçìá: 1 + 1

But I'll go ahead and play. Why are you bringing up 1/3 for 0.01? One third isn't a simple number. One hundredth is.
Or at least I thought it was.

edit: Oh yeah. To clarify, by "accuracy problems", I was referring to float accuracy problems. Not math in general.

Quote:But I'll go ahead and play. Why are you bringing up 1/3 for 0.01? One third isn't a simple number. One hundredth is. Or at least I thought it was.

One third is a perfectly "simple number" in trinary: it is precisely 0.1.

Now, because ten is divisible by two, every rational number that has a finite binary expansion also has a finite decimal representation, but it doesn't work the opposite way. The decimal 0.1 is one of the large set of numbers that do not have an exact binary representation in mantissa-exponent form.

Quote:Original post by Jiia

Quote:Original post by Zahlman
But actually, it's perfectly sensible.

Quick, write out one third exactly in decimal notation. What, isn't that a simple number?

Bah, it's a terrible flaw. I say we start over. ìÜèçìá: 1 + 1

But I'll go ahead and play. Why are you bringing up 1/3 for 0.01? One third isn't a simple number. One hundredth is.
Or at least I thought it was.

edit: Oh yeah. To clarify, by "accuracy problems", I was referring to float accuracy problems. Not math in general.

One hundredth is a "simple" number in base 10, if you define "simple" as the ability to write the number in decimal notation without large numbers of or recurring decimal digits. It is simply 0.01. One third is a "difficult" number in base 10. It is 0.3 recurring. But change to base 3 and suddenly everything is different. One third is now a "simple" number: 0.1 but one hundredth is a "difficult" number: 0.000021 recurring (3 digits recur). Computers use base 2. Every number which is "simple" in base 2 is also "simple" in base 10, but the converse is not true. A number which is "simple" in base 10 may not be "simple" in base 2. One hundredth is one such number. It is "simple" in base 10, but "difficult" in base 2: 0.0000001010001 etc.

Enigma

Quote:Original post by Jiia
It's causing a very nasty alignment problem in my editor while working with transformation gizmos.

This is the actual problem. The cause is not the floating-point precision (which is in fact very good). So I don't think we need a discussion about floating-point representation here (though it's still interesting to be aware of).

If I understand the alignment issue correctly, it's very similar to the Directly Mapping Texels to Pixels problem. All you have to do is offset your 2D coordinates by half a pixel.

If this is not the problem, please give us more details about it (screenshot)?

For completeness, 0.1 (decimal) is equal to 0.0 0011 0011 0011 ... in binary, the 0011 part being repeated. Since a float only stores a few bits of manitissa, the number precisely stored is not exactly 0.1 but the 0.0999... which you got.

Now it's a very good precision, especially if all you want to do is map this to a screen coordinate. The problem is somewhere else.

I can't really give a screenshot showing what the problem will be without a bit of work. I'm not mapping to screen coordinates. I'm positioning and orienting 3D objects in my world editor using a gizmo like that in 3D studio or such. Errors in the translation and scaling will normally not be a problem.

But if I were to build.. say, a castle wall section. I could plot that in one location, plot another right beside it, move over, plot another, and so on. I could build an entire castle wall out of 2 mesh parts. If the end vertices in one wall section do not exactly match the coordinates of the vertices in the section beside it, a visible tear will continuously jump at you as the camera is rotated around in the scene. This is how 3D rendering works. Vertices being "welded" simply means they share the exact same space. When the graphics card fills those polygons, the method it uses to fill it will exactly match the method it uses in the wall next to it if the vertices are "welded". So if the walls are aligned perfectly, there is no problem. If not, big ugly problems.

But even my GUI float-number control has it's own modifier to set the gizmo snap value. So if I use 0.01 increments to increase the gizmo snap value, the snap value will be flawed as well. Guess it's time to program that GUI float control type-in feature.

Thanks for all of the information [smile]