# OpenGL float to unorm conversion

## Recommended Posts

Hi,

my question is related (more or less the same) to this question:

https://devtalk.nvidia.com/default/topic/887223/float-to-unorm-conversion/

When I write a program in DirectX or OpenGL which renders into a 32-bit, 4-channel (4x8-bit) non-srgb texture of unorm unsigned char type, the conversion that should be used (according to the OpenGL documentation) is:

u = round(f * 255.0)

however if I write something like:

65.55 / 255.0 in f, I still get 65 for u, and only for 65.56 I get 66. I tested on an NVidia 780 Ti and an NVidia 980 graphics card. Could somebody explain this behavior?

Kind regards,

##### Share on other sites
8 bits in range 0...1 is not enough to precisely store 65.55/255.0 .

In general, you should not assume anything about precision when handling floating-point values and their conversions to fixed point. All you are guaranteed to get is an approximation of the correct result.

##### Share on other sites

Of course, but as the OpenGL documentation says, the stored value should be computed using the formula:

u = round(f * 255.0)

so, what should happen is the following:

float temp1 = 65.55f / 255.0f;
float temp2 = roundf(temp1 * 255.0f);
unsigned char storedValue = (unsigned char)temp2;

so casting the value to unsigned char should be the last step in the execution. (If you compile this with a C-compiler and execute it, you indeed get 66 as expected.)

##### Share on other sites
If you're doing everything right, then that sounds like a NV bug :(

## Create an account

Register a new account

• ## Partner Spotlight

• ### Forum Statistics

• Total Topics
627681
• Total Posts
2978611
• ### Similar Content

• Both functions are available since 3.0, and I'm currently using glMapBuffer(), which works fine.
But, I was wondering if anyone has experienced advantage in using glMapBufferRange(), which allows to specify the range of the mapped buffer. Could this be only a safety measure or does it improve performance?
Note: I'm not asking about glBufferSubData()/glBufferData. Those two are irrelevant in this case.
• By xhcao
Before using void glBindImageTexture(    GLuint unit, GLuint texture, GLint level, GLboolean layered, GLint layer, GLenum access, GLenum format), does need to make sure that texture is completeness.
• By cebugdev
hi guys,
are there any books, link online or any other resources that discusses on how to build special effects such as magic, lightning, etc. in OpenGL? i mean, yeah most of them are using particles but im looking for resources specifically on how to manipulate the particles to look like an effect that can be use for games,. i did fire particle before, and I want to learn how to do the other 'magic' as well.
Like are there one book or link(cant find in google) that atleast featured how to make different particle effects in OpenGL (or DirectX)? If there is no one stop shop for it, maybe ill just look for some tips on how to make a particle engine that is flexible enough to enable me to design different effects/magic
let me know if you guys have recommendations.
• By dud3
How do we rotate the camera around x axis 360 degrees, without having the strange effect as in my video below?
Mine behaves exactly the same way spherical coordinates would, I'm using euler angles.
Tried googling, but couldn't find a proper answer, guessing I don't know what exactly to google for, googled 'rotate 360 around x axis', got no proper answers.

References:
Code: https://pastebin.com/Hcshj3FQ
The video shows the difference between blender and my rotation:

• By Defend
I've had a Google around for this but haven't yet found some solid advice. There is a lot of "it depends", but I'm not sure on what.
My question is what's a good rule of thumb to follow when it comes to creating/using VBOs & VAOs? As in, when should I use multiple or when should I not? My understanding so far is that if I need a new VBO, then I need a new VAO. So when it comes to rendering multiple objects I can either:
* make lots of VAO/VBO pairs and flip through them to render different objects, or
* make one big VBO and jump around its memory to render different objects.
I also understand that if I need to render objects with different vertex attributes, then a new VAO is necessary in this case.
If that "it depends" really is quite variable, what's best for a beginner with OpenGL, assuming that better approaches can be learnt later with better understanding?

• 13
• 12
• 10
• 12
• 22