Jump to content
  • Advertisement
Sign in to follow this  

2d glOrtho - Low precision

This topic is 2112 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi guys, 
I am trying to draw an axis for a 2d-plot.
Specifically I am trying to draw the ticks on that axis.
Unfortunately I ran into some precision problems. As soon as the range between min and max exceeds single precision (32 bit float)
the ticks are fucked up or all drawn at the same position.

Example: m_NiceMin = 1.1111111 and m_NiceMax = 1.1111112
In double precision there is no problem, in single precision it is all 1.1111112 and therefore drawn in the same place.

Here is my code. Hope you can help me out. 

// Set drawing area
_u32 AxisWidth = m_ControlRect.Width() - 2*BorderSize - YAxisWidth;
glViewport(BorderSize + YAxisWidth, // X0
BorderSize, // Y0
AxisWidth, // Width
m_BaseSize + AxisTitleSize // Height

glOrtho(m_NiceMin, m_NiceMax, 0, 1, -1, 1); 

// Draw the ticks on the right side of the axis
double CurrentTick = m_NiceMin + m_TickSpacing;
for (_u32 i = 1; i < Ticks; i++)
// Draw tick line
glVertex2d(CurrentTick, 1);
glVertex2d(CurrentTick, 0.8);

CurrentTick += m_TickSpacing;

Share this post

Link to post
Share on other sites

I'm not sure what it is you want assistance with. You are aware that the problem happens when you pass the precision of a float, and you have a working solution by switching to double. If I understand that right, then it sounds like there is no problem.

Share this post

Link to post
Share on other sites

Ah sorry I didn't say that explicitly.


m_NiceMin and m_NiceMax are double as well. I am not using float in a single location - its all double.

I can see all that in the debugger. All variables such as m_NiceMin, m_NiceMax and CurrentTick are double and have the correct value throughout the loop. Until it is drawn to the screen. There it is all fucked up.


I added two images where you can see what the problem looks like.



Share this post

Link to post
Share on other sites

If your driver is working with floats internally instead of doubles, which is more than likely, then you are ultimately limited by the float even if your calculations are in higher precision. There really is nothing you can do about that in itself.


If double gives you enough precision for internal calculations, then I suggest you do it all in doubles and skip all of OpenGL's internal calculations. In this case, it basically means you do the coordinate transform yourself in double precision, and then pass coordinates with a more sensible range to OpenGL. Something along these lines:

glOrtho(0, 1, 0, 1, -1, 1); 
double offset = m_NiceMin;
double range = (m_NiceMax-m_NiceMin);

glVertex2d((CurrentTick-offset)/range, 1);

edit: Clarification; I don't mean to skip OpenGL's internal calculations as in entirely bypassing it, but skipping the precision-demanding calculations and passing data that has enough precision for a float that OpenGL can continue to work with.

Edited by Brother Bob

Share this post

Link to post
Share on other sites


That really solved my problem. I thought I tried something along those lines before and had other problems.

But this time it worked!


Thanks a million!

Edited by xift

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!