what does a normal do and an RMA, I assume a normal adjusts the amount of light being reflected? does RMA determine what wavelength is reflected more?

A normal is a normalized vector. Vectors represent directions and amounts tied together permanently. However, sometimes you don't care about the amount and only want to represent a direction, especially when doing vector math such as this case.

When you model you probably model in quads or maybe n-gons occasionally. But the graphics card doesn't understand that geometry. It only understands triangles. So, it may help to think of your model as nothing but a series of triangles. It __will__ be reduced to that by the time it reaches the graphics card.

Each triangle in the mesh has a normal that determines what direction that triangle faces. This normal is used in even the most basic lighting calculations. Basically, if it faces into the light it receives more of the light. If it faces away from the light it receives less light. Fresnel and some other things can affect this, but even they will use the normal of the surface to do their calculations.

Back in the days when this is all they had models were very faceted. Blinn and Phong got the idea of doing normals per vertex instead of per face and interpolating (averaging) their values across the face of the surface. This gives us smooth shading on faceted models. It also gives us texturing because the UV coordinates are interpolated across the face of the triangle.

Now taking this idea one step further, if you assign normals per pixel instead of per vertex, you can make the surfaces look much more complicated than averaging between the vertices all in one smooth gradient.

So, you create a high-poly count mesh and a matching low-poly mesh. In Blender there is a Sub-D modifier where the high-poly mesh is literally built on top of the low-poly mesh. Then you "bake" a normal map. Basically, what it's doing is using a photograph just like for the albedo (colored texture). It's using those same UV coordinates and determining which pixel by interpolating the UV coordinates across the face of the triangle. Then it samples from the photograph to determine the color value of that pixel for an albedo(color) map. But for a normal map, it will sample a color that represents a 3 dimensional vector normal. So, in the baking process it takes the low poly model and determines the position of every pixel on the surface of the model as it will UV map that back to the maps. Then it projects the place on the high-poly model on to the low-poly model to find the direction that pixel faces in the high poly model.

Put more simply, for a given triangle in the low-poly model every pixel will be assigned a direction according to the surface of the high-poly model at that point on the model and that direction it faces will be stored as pixels in a normal map.

Or put even more simply, a normal map simulates the details of a high-poly model when in reality it is a low-poly model.

Roughness mapping is a very similar concept except that it is for surface detail smaller than a pixel on your model (which is the smallest truly measurable unit). I'm new to PBR, so I'm still trying to get the difference between Roughness and Specular straight in my head. I "believe" Specular is reflective of the surface and Roughness is about sub-surface light scattering. As I get better with PBR I will probably be able to tell you more, but pretty much all maps work on similar principles.

I think roughness on the metalic work-flow is actually "adjusting the amount of light being reflected" as the micro-surface is assumed to be rougher and rougher and reflecting more and more light in random directions rather than at the camera. The less "rough" it is the more the light reflects directly at the camera and thus becomes metalic reflection. In it's map it's probably a gray scale value between 0 and 1 representing a percentage where 0 is black and 1 is white in the image. But again, these are per-pixel values when the pixel is drawn to the computer screen just like the color of a pixel when drawing it's color to the screen using an albedo or color map.

The way a normal map is calculated is that you calculate a normal for a pixel on the surface of the model which is just an arrow that represents it's direction of facing. Normals in game programming are stored as a single triplet (X,Y,Z) coordinate. The tail end of the vector is understood to always be at the origin (0,0,0) and thus you only need one set of numbers to represent the position of the arrow's head. Since it's a direction, the actual position of the 3D arrow is irrelevant. So, you can imagine the origin being at the pixel in question and the position of the head as being an offset from that position.

Since you want to reduce this to a color you need to put the numbers into the color space. The definition of a vector normal is that it has a length of one. Having a length of one means we do not care about the length (amount) and only care about the direction. So, the position of the head will always be in the range of -1 to +1. Colors have no negative values. So, you add 1 to X,Y, and Z putting it in the range of 0 to 2. Colors only have a range of 0 to 1. So, you divide by 2 to put the numbers in that range. Now you have X,Y,Z values that can be stored as RGB color values in a photograph like a .PNG. or JPeg. And those color values represent the direction in 3D space that pixel faces. Reverse the math to turn the color back into a 3D mathematical vector that describes what direction the pixel faces.

These pixel normals are relative to the direction the triangle itself faces. So, the direction that triangle or quad in the model faces is irrelevant for these calculations. Basically, a flat triangle is going to have a normal of 0,0,1 where the normal faces straight down the Z axis (relative to the triangle it lives on) regardless of what direction the triangle actually faces. (That will be factored into the math when creating the normals and converting colors back to normals, but it doesn't affect the colors in the map.) So, a perfectly flat shaded model will give you a perfectly blue normal map because Z of X,Y,Z maps to Blue in RGB. So, it will be 0,0,1 in X,Y,Z and 0,0,1 in RGB. Any colors you see in the normal map are deviations from the direction the triangle faces that the pixel lives.

To make things a bit more complicated, DirectX and OpenGl use two different coordinate systems, left handed and right handed. I find this especially confusing because both allow you to set this. So, I don't associate one or the other with DX or OGL. And so I never know which is which. I think I use the OGL coordinate system for everything including DX and off the top of my head I couldn't tell you if that's left or right handed. The only difference is the direction Z faces. If you have an X,Y 2D plane there's only one way you can configure that. But when you add a Z axis to make it 3D it can face in or out of the plane, either one and the choice will give different results.

I'm not certain how this affects normal maps, but it would seem to me that if you reverse the Z values most normals would be 0,0,0 when facing straight forward instead of 0,0,1 which would make the map black except where the directions deviate from straight forward. That's a guess. I'm not certain exactly how it's done in this case, but I do know you will see the question of whether to use DX or OGL normals come up a lot. The main thing is to know that they are different.

Trying to think of a way to summarize what I just said: A Normal Map stores color values that represent the direction a pixel faces. Remember that in reality everything is flat surfaced triangles in your mesh. Everything else is just mathematical "tricks" to work around this. You can do a uber high-poly mesh and get more detail but that's expensive on the graphics card. But one trick is to bake that uber high-poly mesh down on a low-poly mesh of the same model. This allows the calculations for drawing to work at the pixel level rather than the vertex or triangle level, because the maps store information per-pixel rather than per-vertex or per-triangle. So, the end result looks almost like you have an uber high-poly mesh, when in reality what you have is a low-poly mesh and the maps allow mathematical tricks to do the lighting and drawing calculations per-pixel rather than per-vertex or per-triangle. Ambient Occlusion, Roughness, Specular, and other values can be stored per-pixel this same way by baking a high-poly mesh down against a low-poly mesh. Think of your basic texture(color) map and how that draws pixels across the face of the triangle based on the triangle surface being mapped to a photograph. Now do that same process with other information other than color such as normals or specular values.

**Edited by BBeck, 30 January 2017 - 08:22 AM.**