• Create Account

Banner advertising on our site currently available from just \$5!

# max coord values and fp precision

Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

11 replies to this topic

### #1Norman Barrows  Crossbones+   -  Reputation: 2933

Like
0Likes
Like

Posted 24 March 2013 - 02:06 PM

what does one do when you want really big world coordinates, and lots of accuracy, say coords going up to 1 million or so, and 2-3 decimal places of accuracy? this is more that a float can do.

is this avoided in level or cell based design because levels are never that big?

in a title i'm working on (a ww1 era flight sim), the world is about 10,000 miles across, and movement rates are on the order of inches per turn. i've gone to a sector-coordinate (cell-coord?) system.  the world is 2500x2500 sectors, each with its own world axis. and a sector is 10,000 x 10,000 D3D units in size. in another title i'm doing (fps/rpg), the world is 500x500 map squares (sectors/cells), and each map square is 26400 x 26400 D3D units.

the sector-coordinate approach requires special routines that work with sectors and coordinates for calculating range, heading to target, angle to target, normalizing locations to coordinate ranges of 0 to 10000, and converting locations to camera relative coordinates for drawing.

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php

### #2Norman Barrows  Crossbones+   -  Reputation: 2933

Like
0Likes
Like

Posted 24 March 2013 - 02:15 PM

i guess another question along the same line would be what scale do most people use, IE how many feet per d3d unit? Ive used 10 d3d units per foot in the past, and currently use 1 d3d unit per foot, with sector-coordinate systems. i tried to use 100 feet per d3d unit and do the ww1 flight sim in one huge world coordinate system, but floating point accuracy was an issue.

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php

### #3scniton  Members   -  Reputation: 252

Like
0Likes
Like

Posted 25 March 2013 - 12:21 AM

what does one do when you want really big world coordinates, and lots of accuracy, say coords going up to 1 million or so, and 2-3 decimal places of accuracy? this is more that a float can do.

These cases are easily handled using either 32 bit integers (both signed and unsigned would work) or floating-point doubles.

I personally prefer using integers / fixed point (for determinism/reproducibility/constant precision reasons.)

You still need to convert coordinates to camera relative before sending them to D3D / OpenGL in these cases.

I would probably only resort to a sector based system if a 64 bit integer wasn't satisfactory.

Edited by scniton, 25 March 2013 - 12:44 AM.

### #4Norman Barrows  Crossbones+   -  Reputation: 2933

Like
0Likes
Like

Posted 25 March 2013 - 01:12 AM

These cases are easily handled using either 32 bit integers (both signed and unsigned would work) or floating-point doubles.

I personally prefer using integers / fixed point (for determinism/reproducibility/constant precision reasons.)

wow fixed point. been a long time since i heard that term or used fixed point. takes me back to the days of trig lookup tables.

i do see what you mean about the predictability of fixed point over float.

so do it all in fixed point ints and convert eh? that should work. the game world is about 50 million feet across. at 1/10 foot precision, that's still +/-200 millon with an int, and there's always long long. what about compatibility, is a long long available on a 32 bit machine? i never even looked.

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php

### #5Hodgman  Moderators   -  Reputation: 40013

Like
0Likes
Like

Posted 25 March 2013 - 01:36 AM

what scale do most people use, IE how many feet per d3d unit? Ive used 10 d3d units per foot in the past, and currently use 1 d3d unit per foot, with sector-coordinate systems.

I've worked on one game that used 1unit = 1inch, one that used 1unit = 1cm, and the rest all used 1unit = 1metre, which seems very common.

### #6scniton  Members   -  Reputation: 252

Like
0Likes
Like

Posted 25 March 2013 - 02:28 AM

what about compatibility, is a long long available on a 32 bit machine? i never even looked.

Depends on the language / compiler:

Assuming C/C++:
- long long is present in all C99 and C++11 compilers
- long long is often present in many popular compilers, possibly in extension form (e.g. __int64)

Since I use gcc mostly, I've always just include stdint.h / cstdint.h and assumed the 64bit types were there.
Some projects make things easier:
http://www.boost.org/doc/libs/1_53_0/libs/integer/doc/html/boost_integer/cstdint.html

### #7Norman Barrows  Crossbones+   -  Reputation: 2933

Like
0Likes
Like

Posted 26 March 2013 - 01:39 PM

now, what abut loss of precision when converting to camera relative? just have to make sure you have enough decimal places in your fixed point eh? and your integer parts don't get too big?

another issue: true range checks (pythagorean) require squaring dx and dz (and dy for 3d range). that could lead to overflow with signed int and values on the order of 50 million. 50 meg ^2 = 250 trillion (?).  time for long long, eh?

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php

### #8JohnnyCode  Members   -  Reputation: 578

Like
0Likes
Like

Posted 28 March 2013 - 03:47 AM

now, what abut loss of precision when converting to camera relative? just have to make sure you have enough decimal places in your fixed point eh? and your integer parts don't get too big?

another issue: true range checks (pythagorean) require squaring dx and dz (and dy for 3d range). that could lead to overflow with signed int and values on the order of 50 million. 50 meg ^2 = 250 trillion (?).  time for long long, eh?

do not check distance like this. If you want to find out wheather object falls into interactive scene, just examine its X value with global scene position value, then Y value with scene value, or also Z if you are not CPU overheaded and have also up down complexity, (like an universe fly game). The best scale for me, since I needed fingers on hands moving , or even better details, was adopted from 3ds MAX, 1 float is 1 meter. Heavenly precision leaving me to bunch of meters

### #9Waterlimon  Crossbones+   -  Reputation: 3394

Like
0Likes
Like

Posted 28 March 2013 - 04:15 AM

for distance checks, just make one of the points the origin (works as long as points are close to each other)

If the points are far away so that even that would cause problems, scale the point down (after making the other the origin), do the check and scale the result back up.

o3o

### #10scniton  Members   -  Reputation: 252

Like
0Likes
Like

Posted 28 March 2013 - 02:08 PM

another issue: true range checks (pythagorean) require squaring dx and dz (and dy for 3d range). that could lead to overflow with signed int and values on the order of 50 million. 50 meg ^2 = 250 trillion (?).  time for long long, eh?

Yes, and depending on the range of height values (dy) a 64bit unsigned may not be enough for the intermediate results depending on the chosen representation.
Using doubles is the easier path if it satisfies your requirements.

now, what abut loss of precision when converting to camera relative? just have to make sure you have enough decimal places in your fixed point eh? and your integer parts don't get too big?

Converting to relative doesn't usually cause significant loss of precision.
If the integer parts are big, then your objects are at extreme distances from the camera and so any differences introduced from converting from fixed point to floating point should be insignificant.

Edited by scniton, 28 March 2013 - 02:09 PM.

### #11Mr_F_  Members   -  Reputation: 279

Like
0Likes
Like

Posted 28 March 2013 - 03:00 PM

I've just read a paper about how related problems were handled in Just Cause 2. May be useful (from page 9):

Edited by Mr_F_, 28 March 2013 - 03:02 PM.

### #12Norman Barrows  Crossbones+   -  Reputation: 2933

Like
0Likes
Like

Posted 31 March 2013 - 02:38 AM

it occurred to me that longer range distances could be done with BBox distances (greater of dx,dz), and any place where you'd need true Euclidean distance, the range would be short (visual range).

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php

Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

PARTNERS