currently I'm working on a ocean shader with some vertex shader math to simulate the waves. For a more convincing effect I'd like to recalculate the surface normals after the vertex displacement has happenend, which is where I'm struggling at the moment. It's worth mentioning what I'm working with the UDK, which means there are several restrictions for the material editor that I won't be able to bypass.
So far I've come across 2 different approaches.
One is to use the derivative of the wave height function for recalculating the normals. Whilst this is looking pretty good it's adding lots of pixel shader instructions, depending on the wave simultion complexity. For instance:
The other one uses partial derivates of the WorldPosition. This is has a fixed impact on instruction count, which is nice, but as expected the resulting normals are flat shaded.
Since an ocean wave algorithm might get pretty complex I'd rather go with approach #2. So I'm wondering if there are any techniques to calculate smooth normals in the pixel shader with a fixed instruction count?
Edited by OMH, 21 February 2014 - 11:21 AM.