Hello! In my current project i need to implement refraction effect, when the refracted direction is calculated in eye space(or view space), it's supposed to indexed into an environment map to obtain the color. The environment map is oriented relative to world space, so transformation is needed to convert the vector back into world space. I'm not quite sure which transformations to take, would i multiply the vector with the inverse of ModelView matrix or the inverse of ModelViewInverseTranspose matrix? I think the latter case is more likely becasue normals are transformed from world space to eye space by multiply the ModelViewInverseTranspose matrix. Any suggestions? Thanks in advance. [Edited by - hmcen on May 3, 2007 7:40:29 AM]

To transform normals, the matrix inverse(transpose(M)) is used, where M is the matrix that would be used to transform points. To transform points from view space to world space, one would use the inverse of the world transformation matrix. Thus, you need inverse(transpose(inverse(world matrix))) in general, although this can typically be reduced significantly in terms of how many inverses you need to calculate by taking advantage of the orthogonality properties of matrices; you also only need to use the upper-left 3x3 of the matrix, as the translation component would wash out when you renormalized the normal anyway.

In OpenGL, the object-to-world matrix and the world-to-view matrix are combined into the GL_MODELVIEW stack. So you can't neccessarily use the modelview matrix itself, unless you actually want to get yourself back into object space.

The ModelView matrix transforms from model-space to view-space, so the ModelViewInverse matrix will transform from view-space to model-space. In many cases model-space and world-space are the same (i.e. static geometry), so that's not a problem. However you need to keep careful track of what space your transformations put you in.

One must be careful to distinguish transforming a normal and transforming a vector. Only use inverse transpose for normals. For vectors (like eye rays) just use the upper left 3x3 inverse of the camera matrix. If your normal has w=0 than you can use the inverse matrix directly. However that will do 4 MADs. So if you cast your vector to float3 you will save one instruction.

Thanks jpetrie & Zipster & MannyK :) your replies are really helpful.
Multiply the eye space vector with the inverse of camera matrix do solve the problem. It seems to me that i have to write my own functions to construct the camera matrix and inverse it in OpenGL. I wonder whether OpenGL has its built-in functions to do this? Thanks again:)