• FEATURED

View more

View more

View more

### Image of the Day Submit

IOTD | Top Screenshots

### The latest, straight to your Inbox.

Subscribe to GameDev.net Direct to receive the latest updates and exclusive content.

# Float Precision Issue?

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

13 replies to this topic

### #1DJTN  Members

Posted 17 November 2012 - 01:40 PM

Previously I posted about a rotation issue with my camera. I thought I had solved the problem by adding frame time to the camera’s movement calculation but unfortunately this only masked the real problem, which I think is a precision issue with floats. I noticed this when I rotate not only the camera but other objects as well.

I striped my render code down to the basics in an effort to identify the issue:

[source lang="cpp"]Camera CalculationsCameraTarget.X = CameraPosition.X + Sin(vRadians)CameraTarget.Y = CameraPosition.Y + (Cos(vRadians) * Cos(hRadians))CameraTarget.Z = CameraPosition.Z + (Cos(vRadians) * Sin(hRadians))CameraUp.X = 0;CameraUp.Y = 1;CameraUp.Z = 0;Update View Matrix:D3DXMatrixLookAtLH(View_Matrix, CameraPosition, CameraTarget, CameraUp);[/source]

To make things as simple as possible I’ll add 0.002 to the hRadians float each frame. This makes the camera rotate to the left but when I slow the frame rate down I can clearly see the camera is not turning the same amount each frame. This also happens when I calculate rotations for the world matrix before rendering 3D objects too. When rendering at 60 FPS and adding an easing calculation to movements it’s not as noticeable for moving forward, backwards, left and right but when rotating, it is very noticeable. Even with the easing code added you can tell the rotations are accelerating and decelerating, sometimes jerky.

I’ve tried to truncate the calculations for the CameraTarget and CameraPosition to regain some precision but to no avail, the issue still persists.

Is this a precision issue? And if so, how can I get more precision for rotations? If it’s a not a precision problem then what is it?

Edited by DJTN, 17 November 2012 - 01:43 PM.

### #2C0lumbo  Members

Posted 17 November 2012 - 02:13 PM

What are those trig functions you're using - do they map to sinf/cosf?

What is the absolute value of hRadians? Do you reset it back to the zero to 2Pi range or to the -PI to PI range after it leaves it? (if you let a float get too far from zero the precision drops significantly).

Posted 17 November 2012 - 02:27 PM

How exactly do you increase hRadians? If you add a constant amount every frame, the rotation will of course accelerate/decelerate depending on the frame rate.

### #4DJTN  Members

Posted 17 November 2012 - 03:32 PM

What are those trig functions you're using - do they map to sinf/cosf?

What is the absolute value of hRadians? Do you reset it back to the zero to 2Pi range or to the -PI to PI range after it leaves it? (if you let a float get too far from zero the precision drops significantly).

Yes they are sinf and cosf, the code is pseudo code from memory, I'm not at my computer. hRadians starts out at 0 and I'm adding 0.002 to it every frame in an attempt to find the problem. It accumulative I do not set it back to zero. When I get home I'll check to see how I can implement locking it down to as close to zero as I can get and see if that helps.

### #5DJTN  Members

Posted 17 November 2012 - 03:38 PM

How exactly do you increase hRadians? If you add a constant amount every frame, the rotation will of course accelerate/decelerate depending on the frame rate.

I've removed the easing (dampening) code as well as the addition of the timestamp each frame so that I can get to the bottom of the issue. Basically my render loop is just adding 0.002 to hRadians and creating the view matrix. This slowly turns the camera. When I slow the frame rate down in the debugger I can clearly see the ratation is different each frame. Some frames the rotation amount is more obvious than others.

### #6DJTN  Members

Posted 17 November 2012 - 04:41 PM

What is the absolute value of hRadians?

hRadians is set to zero and 0.002 is added each frame. 6.0 in hRadians is a complete circle. It doesn't appear to be related with the inaccuracies coming from the distance from zero.

### #7Tispe  Members

Posted 17 November 2012 - 04:52 PM

### #8DJTN  Members

Posted 17 November 2012 - 07:40 PM

A lot of overhead making that change. I'll need to figure out a way I cant test it before going all the way.

### #9Hodgman  Moderators

Posted 17 November 2012 - 08:22 PM

6.0 in hRadians is a complete circle

Shouldn't it be 6.283....?

### #10C0lumbo  Members

Posted 18 November 2012 - 02:25 AM

Another thing that occurs to me is that if your CameraPosition vector was a long way from the origin, then there'd be a lot of error when you create your CameraTarget vector by adding the unit vector, which could explain some oddness. Could you check that's not the case?

### #11DJTN  Members

Posted 18 November 2012 - 09:22 AM

Shouldn't it be 6.283....?

Yes Hodge, you're correct. I was truncating the float in my code to gain precision.

Another thing that occurs to me is that if your CameraPosition vector was a long way from the origin, then there'd be a lot of error when you create your CameraTarget vector by adding the unit vector, which could explain some oddness. Could you check that's not the case?

My camera position is (0,8,0). My target is (1,8,0) at the start. When I break on the code after the first value of 0.003 for hRadians has been calculated the target becomes (0.9999955, 8, 0.002999996) with the position vector staying the same.

The target for the next 90+ frames:

Frame: 0 Target: (1 , 8 , 0)
Frame: 1 Target: (0.9999955 , 8 , 0.002999996)
Frame: 2 Target: (0.999982 , 8 , 0.005999964)
Frame: 3 Target: (0.9999595 , 8 , 0.008999879)
Frame: 4 Target: (0.999928 , 8 , 0.01199971)
Frame: 5 Target: (0.9998875 , 8 , 0.01499944)
Frame: 6 Target: (0.999838 , 8 , 0.01799903)
Frame: 7 Target: (0.9997795 , 8 , 0.02099846)
Frame: 8 Target: (0.999712 , 8 , 0.0239977)
Frame: 9 Target: (0.9996355 , 8 , 0.02699672)
Frame: 10 Target: (0.99955 , 8 , 0.0299955)
Frame: 11 Target: (0.9994556 , 8 , 0.03299401)
Frame: 12 Target: (0.9993521 , 8 , 0.03599223)
Frame: 13 Target: (0.9992396 , 8 , 0.03899011)
Frame: 14 Target: (0.9991181 , 8 , 0.04198765)
Frame: 15 Target: (0.9989877 , 8 , 0.04498481)
Frame: 16 Target: (0.9988482 , 8 , 0.04798157)
Frame: 17 Target: (0.9986998 , 8 , 0.05097789)
Frame: 18 Target: (0.9985424 , 8 , 0.05397375)
Frame: 19 Target: (0.998376 , 8 , 0.05696913)
Frame: 20 Target: (0.9982005 , 8 , 0.059964)
Frame: 21 Target: (0.9980162 , 8 , 0.06295833)
Frame: 22 Target: (0.9978228 , 8 , 0.06595208)
Frame: 23 Target: (0.9976205 , 8 , 0.06894525)
Frame: 24 Target: (0.9974091 , 8 , 0.0719378)
Frame: 25 Target: (0.9971888 , 8 , 0.07492969)
Frame: 26 Target: (0.9969596 , 8 , 0.07792092)
Frame: 27 Target: (0.9967213 , 8 , 0.08091144)
Frame: 28 Target: (0.9964741 , 8 , 0.08390123)
Frame: 29 Target: (0.9962179 , 8 , 0.08689027)
Frame: 30 Target: (0.9959527 , 8 , 0.08987853)
Frame: 31 Target: (0.9956786 , 8 , 0.09286598)
Frame: 32 Target: (0.9953955 , 8 , 0.09585259)
Frame: 33 Target: (0.9951035 , 8 , 0.09883834)
Frame: 34 Target: (0.9948025 , 8 , 0.1018232)
Frame: 35 Target: (0.9944926 , 8 , 0.1048071)
Frame: 36 Target: (0.9941736 , 8 , 0.1077901)
Frame: 37 Target: (0.9938458 , 8 , 0.1107722)
Frame: 38 Target: (0.9935091 , 8 , 0.1137532)
Frame: 39 Target: (0.9931633 , 8 , 0.1167332)
Frame: 40 Target: (0.9928086 , 8 , 0.1197122)
Frame: 41 Target: (0.9924451 , 8 , 0.1226901)
Frame: 42 Target: (0.9920725 , 8 , 0.1256668)
Frame: 43 Target: (0.9916911 , 8 , 0.1286425)
Frame: 44 Target: (0.9913006 , 8 , 0.131617)
Frame: 45 Target: (0.9909014 , 8 , 0.1345903)
Frame: 46 Target: (0.9904931 , 8 , 0.1375624)
Frame: 47 Target: (0.9900759 , 8 , 0.1405333)
Frame: 48 Target: (0.9896499 , 8 , 0.1435029)
Frame: 49 Target: (0.989215 , 8 , 0.1464712)
Frame: 50 Target: (0.9887711 , 8 , 0.1494382)
Frame: 51 Target: (0.9883183 , 8 , 0.1524038)
Frame: 52 Target: (0.9878566 , 8 , 0.1553681)
Frame: 53 Target: (0.9873861 , 8 , 0.1583309)
Frame: 54 Target: (0.9869066 , 8 , 0.1612924)
Frame: 55 Target: (0.9864184 , 8 , 0.1642524)
Frame: 56 Target: (0.9859211 , 8 , 0.1672109)
Frame: 57 Target: (0.9854151 , 8 , 0.1701679)
Frame: 58 Target: (0.9849001 , 8 , 0.1731234)
Frame: 59 Target: (0.9843763 , 8 , 0.1760773)
Frame: 60 Target: (0.9838437 , 8 , 0.1790297)
Frame: 61 Target: (0.9833022 , 8 , 0.1819804)
Frame: 62 Target: (0.9827518 , 8 , 0.1849295)
Frame: 63 Target: (0.9821926 , 8 , 0.1878769)
Frame: 64 Target: (0.9816245 , 8 , 0.1908226)
Frame: 65 Target: (0.9810476 , 8 , 0.1937666)
Frame: 66 Target: (0.980462 , 8 , 0.1967089)
Frame: 67 Target: (0.9798674 , 8 , 0.1996494)
Frame: 68 Target: (0.979264 , 8 , 0.2025881)
Frame: 69 Target: (0.9786519 , 8 , 0.205525)
Frame: 70 Target: (0.9780309 , 8 , 0.20846)
Frame: 71 Target: (0.9774011 , 8 , 0.2113932)
Frame: 72 Target: (0.9767625 , 8 , 0.2143244)
Frame: 73 Target: (0.9761152 , 8 , 0.2172538)
Frame: 74 Target: (0.975459 , 8 , 0.2201811)
Frame: 75 Target: (0.9747941 , 8 , 0.2231065)
Frame: 76 Target: (0.9741204 , 8 , 0.2260299)
Frame: 77 Target: (0.9734379 , 8 , 0.2289513)
Frame: 78 Target: (0.9727467 , 8 , 0.2318705)
Frame: 79 Target: (0.9720467 , 8 , 0.2347877)
Frame: 80 Target: (0.9713379 , 8 , 0.2377028)
Frame: 81 Target: (0.9706205 , 8 , 0.2406158)
Frame: 82 Target: (0.9698942 , 8 , 0.2435265)
Frame: 83 Target: (0.9691593 , 8 , 0.2464351)
Frame: 84 Target: (0.9684156 , 8 , 0.2493415)
Frame: 85 Target: (0.9676632 , 8 , 0.2522456)
Frame: 86 Target: (0.9669021 , 8 , 0.2551475)
Frame: 87 Target: (0.9661323 , 8 , 0.258047)
Frame: 88 Target: (0.9653539 , 8 , 0.2609442)
Frame: 89 Target: (0.9645667 , 8 , 0.2638391)
Frame: 90 Target: (0.9637709 , 8 , 0.2667316)
Frame: 91 Target: (0.9629663 , 8 , 0.2696217)
Frame: 92 Target: (0.9621531 , 8 , 0.2725094)
Frame: 93 Target: (0.9613312 , 8 , 0.2753946)
Frame: 94 Target: (0.9605008 , 8 , 0.2782773)
Frame: 95 Target: (0.9596616 , 8 , 0.2811576)
Frame: 96 Target: (0.9588138 , 8 , 0.2840353)
Frame: 97 Target: (0.9579574 , 8 , 0.2869104)
Frame: 98 Target: (0.9570924 , 8 , 0.289783)

### #12DJTN  Members

Posted 19 November 2012 - 03:38 PM

Wow, I'm running out of ideas here.

Since my View Matrix is accumalitive (I never reset my axis) - could this be the problem? I update every frame using the D3DXMatrixLookAtLH to build the View Matrix. My camera's Up never changes, only the position and the target.

Posted 19 November 2012 - 03:49 PM

Honestly, to me it sounds like you are overanalyzing the "problem". I never encountered anything along those lines that was really noticeable and without seeing this myself I can't imagine what the problem actually looks like. Are you sure the wrong motion isn't caused by something else?

### #14DJTN  Members

Posted 19 November 2012 - 04:14 PM

Honestly, to me it sounds like you are overanalyzing the "problem". I never encountered anything along those lines that was really noticeable and without seeing this myself I can't imagine what the problem actually looks like. Are you sure the wrong motion isn't caused by something else?

I've stripped my code down to nothing more than a camera and one object being rendered. I'm not overanalyzing the "problem". If the camera's rotations are not smooth, they are not smooth. Some might be ok with a randomly jerking camera, I'm not one of those people.

Thanks?

I'll revisit this at a later time.

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.