# max coord values and fp precision

This topic is 2003 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

what does one do when you want really big world coordinates, and lots of accuracy, say coords going up to 1 million or so, and 2-3 decimal places of accuracy? this is more that a float can do.

is this avoided in level or cell based design because levels are never that big?

in a title i'm working on (a ww1 era flight sim), the world is about 10,000 miles across, and movement rates are on the order of inches per turn. i've gone to a sector-coordinate (cell-coord?) system.  the world is 2500x2500 sectors, each with its own world axis. and a sector is 10,000 x 10,000 D3D units in size. in another title i'm doing (fps/rpg), the world is 500x500 map squares (sectors/cells), and each map square is 26400 x 26400 D3D units.

the sector-coordinate approach requires special routines that work with sectors and coordinates for calculating range, heading to target, angle to target, normalizing locations to coordinate ranges of 0 to 10000, and converting locations to camera relative coordinates for drawing.

##### Share on other sites

i guess another question along the same line would be what scale do most people use, IE how many feet per d3d unit? Ive used 10 d3d units per foot in the past, and currently use 1 d3d unit per foot, with sector-coordinate systems. i tried to use 100 feet per d3d unit and do the ww1 flight sim in one huge world coordinate system, but floating point accuracy was an issue.

##### Share on other sites

what does one do when you want really big world coordinates, and lots of accuracy, say coords going up to 1 million or so, and 2-3 decimal places of accuracy? this is more that a float can do.

These cases are easily handled using either 32 bit integers (both signed and unsigned would work) or floating-point doubles.

I personally prefer using integers / fixed point (for determinism/reproducibility/constant precision reasons.)

You still need to convert coordinates to camera relative before sending them to D3D / OpenGL in these cases.

I would probably only resort to a sector based system if a 64 bit integer wasn't satisfactory.

Edited by scniton

##### Share on other sites

These cases are easily handled using either 32 bit integers (both signed and unsigned would work) or floating-point doubles.

I personally prefer using integers / fixed point (for determinism/reproducibility/constant precision reasons.)

wow fixed point. been a long time since i heard that term or used fixed point. takes me back to the days of trig lookup tables.

i do see what you mean about the predictability of fixed point over float.

so do it all in fixed point ints and convert eh? that should work. the game world is about 50 million feet across. at 1/10 foot precision, that's still +/-200 millon with an int, and there's always long long. what about compatibility, is a long long available on a 32 bit machine? i never even looked.

##### Share on other sites

what scale do most people use, IE how many feet per d3d unit? Ive used 10 d3d units per foot in the past, and currently use 1 d3d unit per foot, with sector-coordinate systems.

I've worked on one game that used 1unit = 1inch, one that used 1unit = 1cm, and the rest all used 1unit = 1metre, which seems very common.

##### Share on other sites

what about compatibility, is a long long available on a 32 bit machine? i never even looked.

Depends on the language / compiler:

Assuming C/C++:
- long long is present in all C99 and C++11 compilers
- long long is often present in many popular compilers, possibly in extension form (e.g. __int64)

Since I use gcc mostly, I've always just include stdint.h / cstdint.h and assumed the 64bit types were there.
Some projects make things easier:
http://www.boost.org/doc/libs/1_53_0/libs/integer/doc/html/boost_integer/cstdint.html

##### Share on other sites

now, what abut loss of precision when converting to camera relative? just have to make sure you have enough decimal places in your fixed point eh? and your integer parts don't get too big?

another issue: true range checks (pythagorean) require squaring dx and dz (and dy for 3d range). that could lead to overflow with signed int and values on the order of 50 million. 50 meg ^2 = 250 trillion (?).  time for long long, eh?

##### Share on other sites

now, what abut loss of precision when converting to camera relative? just have to make sure you have enough decimal places in your fixed point eh? and your integer parts don't get too big?

another issue: true range checks (pythagorean) require squaring dx and dz (and dy for 3d range). that could lead to overflow with signed int and values on the order of 50 million. 50 meg ^2 = 250 trillion (?).  time for long long, eh?

do not check distance like this. If you want to find out wheather object falls into interactive scene, just examine its X value with global scene position value, then Y value with scene value, or also Z if you are not CPU overheaded and have also up down complexity, (like an universe fly game). The best scale for me, since I needed fingers on hands moving , or even better details, was adopted from 3ds MAX, 1 float is 1 meter. Heavenly precision leaving me to bunch of meters

##### Share on other sites
for distance checks, just make one of the points the origin (works as long as points are close to each other)

If the points are far away so that even that would cause problems, scale the point down (after making the other the origin), do the check and scale the result back up.

##### Share on other sites

another issue: true range checks (pythagorean) require squaring dx and dz (and dy for 3d range). that could lead to overflow with signed int and values on the order of 50 million. 50 meg ^2 = 250 trillion (?).  time for long long, eh?

Yes, and depending on the range of height values (dy) a 64bit unsigned may not be enough for the intermediate results depending on the chosen representation.
Using doubles is the easier path if it satisfies your requirements.

now, what abut loss of precision when converting to camera relative? just have to make sure you have enough decimal places in your fixed point eh? and your integer parts don't get too big?

Converting to relative doesn't usually cause significant loss of precision.
If the integer parts are big, then your objects are at extreme distances from the camera and so any differences introduced from converting from fixed point to floating point should be insignificant.

Edited by scniton

1. 1
2. 2
3. 3
Rutin
18
4. 4
JoeJ
14
5. 5

• 14
• 10
• 23
• 9
• 33
• ### Forum Statistics

• Total Topics
632632
• Total Posts
3007541
• ### Who's Online (See full list)

There are no registered users currently online

×