Jump to content

  • Log In with Google      Sign In   
  • Create Account


Member Since 12 Jun 2013
Offline Last Active Aug 11 2015 09:13 AM

#5245442 Question about nDotL across LOD levels

Posted by on 10 August 2015 - 06:26 AM

I know I've been spamming a bit the forums, but please bare with me.


I have this old DX9 CPU based terrain LOD system and I'm updating it to a modern DX11 GPU based one. Progress has been slow but very fun, since I get to replace hundreds of lines of code doing expensive CPU LOD operations with a few GPU lines and I also get a performance boost out of it.


I am not going to talk a bout terrain height across LOD and morphing because these questions have fairly solid answers in my head.


It is about lighting. Even with physically based rendering, the good old cosine factor, the nDotL plays a huge role since we multiply by it. So I want to get it right, but I am getting some weird results as I move across LOD/MIP levels in my normal map generated based on the height map.


Supposing that we have a sampling rate of 1, meaning for NxN vertices we have NxN texels in our height map and a direction of float3(0.80, -0.40, 0.0), I have a first question.


1. Is the following output of nDotL correct, useful for physically based rendering and must be used as a general guideline across all LOD levels, meaning that whatever the sampling rate, your nDotL should roughly follow this curve?




This looks correct for a low resolution (128x128) input texture.


But if I double the height map and normal map resolution, but adjust the sample rate (half it) so that the amount of vertices remains the same, I get this result, which changes the the nDotL curve:




This because I am creating the normal map based on sampling of 4 points in the height map:

float4 GenNM(VertexShaderOutput input) : SV_TARGET {
	float ps = 1 / size;
	//ps *= size / 128;

	float3 n;
	n.x = permTexture2d.SampleLevel(permSampler2d, float4(input.texCoord.x - ps, input.texCoord.y, 0, 0), 0).x * 30 - 
			permTexture2d.SampleLevel(permSampler2d, float4(input.texCoord.x + ps, input.texCoord.y, 0, 0), 0).x  * 30;
	n.z = -(permTexture2d.SampleLevel(permSampler2d, float4(input.texCoord.x, input.texCoord.y - ps, 0, 0), 0).x * 30 - 
			permTexture2d.SampleLevel(permSampler2d, float4(input.texCoord.x, input.texCoord.y + ps, 0, 0), 0).x * 30);
	n.y = 2;
	n = normalize(n);
	n = n * 0.5 + 0.5;	

	return float4(n, 1);

permTexture2d is the wrongly named heightMapTexture and 30 is the world scale. When I double the height map resolution, it becomes finer and the height points closer, so some of the overall curve of the lower resolution height map is lost, rather than adding detail to it. Am I understanding this right?


Doubling the height and normal resolution again and halving the sample rate so that we have the same amount of vertices I get this result:




By the time I get to the desired resolution, the normals become quite flat and so does the lighting.


The reason for generating a high resolution map is that this is the input for LOD 0 close to the camera terrain. LOD 0 is rendered using a lot of vertices. As I move away from the camera, I start using less vertices, spaced further apart. The final LOD will have the 128x128 vertices, like in the first low resolution screenshot.


So the next question is:


2. How should the normal map on full resolution look? More like the one in the first screenshot, but with more detail, or more like in the last screenshot, flat.


3. How consistent should be the various MIP levels of the normal map as I go down in resolution?


#5244956 I guess I manged to mess up GPU perlin?

Posted by on 07 August 2015 - 09:41 AM

Sorry, it is indeed that I misunderstood how to populate data for DXGI_FORMAT_R8G8B8A8_SNORM.


Multiple octave noise is looking as expected now:




On the plus side, I know now how to do a weird procedural square snaky texture!

#5206166 OpenGL to DirectX

Posted by on 23 January 2015 - 04:17 AM


But if you want the highest of the high level of graphics, super AAA quality rendering, then OpenGl is inferior. Maybe the latest version is great, but the ones I tried were a lot behind. For starters, a big chunk of OpenGL is implemented in the GPU driver. There is a huge quality gap between supper cheap GPUs never meant to do real rendering work and high quality stuff.
[citation needed]



I can't offer you a citation, only anecdotal evidence.


In historic order:

1. Linux. This is probably the problem of early Linux drivers and not a real problem with OpenGL, but OpenGL reliability on Linux was very bad. Sure, if you ran glgears there were no problems. But if you tried to do more complicated stuff, some things would not render exactly as expected with no way to fix it except to either try to change X setting and mess around with the driver or move to another PC. There were small bugs that effected only parts of it, allowing you to render the scene but some effects would not show up.

2. Windows Vista + some ATI cards. When Vista came out, some ATI cards had broken OpenGL drivers and it took so many months for them to fix it that I gave up on Vista. The performance was very bad.

3. Even today, there can be subtle differences. Things like Toksvig specular AA are highly dependent on the exact behavior of the GPU filtering system. Even right now if I tried, I can get at least 3 different results on 3 different PCs. Same code, same resolution, different GPUs. The probability of getting it pixel perfect is less than in DirectX and OpenGL has no mechanic to report to you that it is doing stuff differently than you would expect. It says: OK, rendered corectly! And indeed there is output on the screen. And if you examine it superficially it looks OK. Maybe just by a little and I had bad luck. But I'm pretty sure it is not higher.


What I got out of all these experiences is a common theme of slightly less reliability. This is of course an issue only on the PC. If you target some hardware that only has OpenGL and historically has only had OpenGL and everybody is using OpenGL do do 100% of the graphics stuff, there will be probably no such issues.

#5205860 OpenGL to DirectX

Posted by on 21 January 2015 - 03:38 PM

Hi avadhon!


No, you won't find a simple tutorial for what you are seeking: something that does not render a triangle or a cube and resembles a simple game. Those are long a extremely complicated. I am a bit in the same shoes as you: I am experiencing DirectX 11 for the first time. Now granted, I am a self assessed DirectX 9 expert, but DirectX 11 is completely new for me and I am writing a full engine (actually more like and incremental port) from scratch, minus physics. And I can tell you that this engine will be around 500 KiB of code at minimum. Probably 700 KiB. Add to that an external physics engine. There is no way for a tutorial to even cover 100 KiB of code in an easy manner.


But don't let this discourage you! There is a reason most tutorials only show a triangle. That is the base of what you need. You need to learn once the basics of window and device creation. This can be done in DirectX or OpenGl. Not important at all. But once you have this done, I would recommend to abstract away and ignore all this setup and busy work and focus on that triangle. The ideal 3D program that displays a triangle should have one line of code that creates the triangle buffer, one line of code for each vertex in the triangle and one line of code to render it using a camera.


Once you are very comfortable with creating a triangle from scratch, you can do basically anything. Learning to create meshes and what can be done with it is far more important that how to set up a 3D device.


As for OpenGL or DirectX, if you are a beginner and only want to do simple to medium stuff, the choice is irrelevant. Don't let fan boys sway you one way or the other. They are both good in their own way. And if you go with a right handed coordinate system, the same camera and mesh code can run on both.


But if you want the highest of the high level of graphics, super AAA quality rendering, then OpenGl is inferior. Maybe the latest version is great, but the ones I tried were a lot behind. For starters, a big chunk of OpenGL is implemented in the GPU driver. There is a huge quality gap between supper cheap GPUs never meant to do real rendering work and high quality stuff. DirectX if capable of rendering what you wish will have a pretty consistent quality, but of course the low end GPU will run like shit. You will have a beautiful picture that runs a 2 FPS. OpenGL is really not universally capable of of pixel prefect rendering, but does scale OK.


Second, OpenGL does not generally allow you to load precompiled shaders. You need to compile them at runtime. This is a huge disadvantage since modern heavy weight engines use hundreds of KiB of shader code. One of my DirectX 9 projects can spend literally 5 minutes just compiling shaders, but is able to load those precompiled shaders in seconds from a binary blog.


The real question is what you want to do. Do you want to learn the core of 3D rendering? That is procedural meshes and shaders. Do you want to learn rendering in general? Choose one: DirectX or OpenGL. Do you want to write a real game? Don't use either. Start with something like Unity.

#5178822 How to implement globe style mouse only camera control?

Posted by on 08 September 2014 - 04:28 AM

You can also find a simple orbit/pan camera in my demos: http://dl.dropboxusercontent.com/u/45638513/sdx/Tut08.zip

#5145586 Max buffer size

Posted by on 09 April 2014 - 01:23 AM

With 16bit indices you can't have such a large index count.


There are 16bit and 32bit indices. XNA Reach only supports 16bit. HiDef supports 32bit. 

#5139294 I love shader permutations! (not sure if ironic or not)

Posted by on 15 March 2014 - 01:58 PM

I'm not a game developer, so take my advice for what its worth smile.png  You have already made the case for why you need to use permutations in the beginning of your post.  Max performance, and support for a number of different lighting techniques pretty much mandates that you have a bunch of different permutations of your shaders.  I would also suggest that if you can auto generate your shaders, then absolutely you should - with one caveat.  If you want max performance, then you should allow special customization for cases where you find that the generated code isn't all that efficient.


Other than that, you should be pre-compiling your shaders anyways, so really that doesn't affect your end product.  It seems to me that you have already made a strong, logical case for going the way that you have, so I agree with your approach!


OK, thanks for the feedback! One good thing I noticed is that my prototyping time is way down. I can write a new sub-shader, run my generation and try it out with it affecting all permutations.


The generated code for an individual shader is not that short but very readable. So I can track down bugs, do changes in the generated version and if needed feed them back to the generator.


So, what are you then? You have a huge rep...




And I also want them to perform at maximum speed, so run-time shader dynamic branching is out of the question.


Ironically, on modern hardware, you are likely going to shoot yourself in the food with this mindest. Switching shaders, from what I recall, can be almost as, if not even more, expensive than dynamic branching. Especially when, as in permutations, all branches will take the same path for all pixels of the same mesh.


Regarding your actual question, it appears to me that you are putting too much weight at those "shader profiles". From the number of techniques you statet, am I correct to assume all those settings are part of one uber-shader? In my own implementation, stuff like SSAO are all different shaders applied in different stages. I am using a deferred renderer, so my implementation might vary, but especially HDR (and maybe SSAO too) can be done in a post-process-effect, therefore be their own shader. This not only reduces the number of permutations, but also removes some - no need for an "off" permutation, off simply equals to not rendering the pass.



No uber shaders, only relevant settings are accessed in one permutation/render profile. I had uber shaders in the dynamic branching version. They greatly vary in size, with the flat ambient shader being one line of code and the mixed metal dielectric lerp environment mapped lighter being quite long. The former only depends on the material ambient color set as a pixel shader constant, while the alter has a ton of material properties and sampler dependencies.


Every permutation can be easily copied and pasted into any FX file as long as the 200 lines of code of common structure and variables are included.


I am using forward rendering, so I pretty much need to either change change shader technique or lighting constants for every single object, but the total pool of techniques is low. One disadvantage is that you can't use instancing, but you can't really use instancing with forward rendering only with simple lighting schemes.


I wrote the whole thing today, so it is not perfect yet. Some things shouldn't be there.


HDR shouldn't result in a permutation, and once I add it to the post processing pipeline I will reduce my permutations by half. But today is the first time I wrote a HDR shader, so I went with the simplest solution.


SSAO will again be added to the post processor, so no further permutations. I first need to figure out how the hell to do SSAO in a forward renderer. I tried before but it is very grainy.


SSAA is a weird beast. I'm using fixed buffer size SSAA so you must run it every single pixel shader. To keep the permutations down I only included three settings out of the 6 I have implemented. The final version will have off, almost medium and high since it is so expensive, leaving out things like the super duper ultra high version (11x SSAA). That is only there as a reference when I need to compare to maximum achievable quality.


And If I decide to drop SSAA I will reduce permutation by 66%, but I do hope to keep it. I don't care about surface aliasing, but SSAA makes the render rock solid when objects are moving around. I use it for it's temporal aliasing properties and high frequency data shimmering reduction.


And I plan to keep it around for a secret screenshot mode, with even more hardcore options. Maybe 30x SSAA...

#5139268 I love shader permutations! (not sure if ironic or not)

Posted by on 15 March 2014 - 11:35 AM

So I decided to support most reasonable lighting setups. And I also want them to perform at maximum speed, so run-time shader dynamic branching is out of the question.


So I came up with the concept of render profiles. Each combination of render profiles results in a unique pixel shader. All permutations are automatically generated.


Render profiles support the following settings for now:

  • Ambient mode. Can be off, flat color modulation, spherical harmonics or environment map ambient lighting. For metallic or mixed objects off ambient lighting is the same as flat, because of reasons and PBR.
  • Baked ambient AO. On or off. Baked AO only gets displayed in the ambient component because you shouldn't have AO in strong direct light.
  • SSAA mode: three settings. Off, medium and super duper ultra for now.
  • HDR mode. Currently only off and a single tone-mapping operator is supported. I'll add things like filmic later.
  • Material nature: metallic or dielectric. Or mixed, where you can lerp between metallic and dielectric.

This is pretty comprehensive. I found that a handful of render profiles are enough to render scenes.


The only problem is the number of permutations. With this limited setup there are 120 different techniques. I can easily see this getting over 1000. They are autogenerated so not a big problem, but I was wondering how others do this.


Manual shader source management is out of the question. Even a custom tailored solution that only woks with exactly what I want to render and that shader is compiled for my setup only will have dozens of permutations, so generated seems to win. The 120 techniques do occupy 100 KiB of source code and take 10 second to compile under FXC, but precompiled loads very fast.


So my question  for more experienced folk: is this a good approach? Half live 2 uses almost 2000 permutations, so I'm not the only one doing this. And pretty much everyone uses permutations to handle different light setups and numbers. Unless they write those fancy shaders with for loops.

#5137068 DirectX 12 Announced

Posted by on 06 March 2014 - 11:38 PM


XP->Vista introduced a new driver model. D3D10/11 still could've been implemented for XP if they cared about supporting it though... e.g. GL4 exposes all the D11 functionality on XP...

I'm sure it could have been, but there's a lot more than marketing going on there. Agree on 11.1 restrictions being a bit silly, even if it's not just marketing either (new WDDM and DXGI versions).



Marketing isn't the right word for it. There are multiple. "Agenda" is one for it.


It is perfectly fine to introduce new models and break compatibility once in a while. The driver model change in Vista was sorely needed. When something really need improvement, it should be improved.


But their current actions betray an agenda, not them seeing a need for improvement. They are practically strong arming you into upgrading. Sure, it is always easier to not back-port changes, but I'm sure the current management would even prevent those back-ports is possible. They need to sell Windows 9. They need to reach with again numbers like in the glory periods, even though with current PC sales trends that is not possible even if Windows 9 is the best Windows yet. I have Windows 8 and it is a piece of crap, especially for gaming and engine development. Whatever support for miss-behaving fullscreen applications was in the OD previously, "Metro" gutted it. I often need to log out because the desktop app can't handle a full-screen app doing stupid stuff. Well, at least I could never crash the GPU driver like I could under 7 :).


But being tied down to a version of a software sucks. I'm not die hard by principle, but I'll probably end up the last DirectX 9 user in the world. Every time I try to port stuff and do dual maintenance, something does not work.


Anyway, screw stubborn principles. I'm upgrading right now to Windows 8.1. The reason I  did not update until now was I that mistakenly believed you needed an account for it. I was just about ready to create my account, when I happily noticed it would let me download the update without it.


I have over 40 account that are all important. Does anybody else have problems with modern social media and services presence and the number of accounts you need?

#5136877 DirectX 12 Announced

Posted by on 06 March 2014 - 11:02 AM


Maybe it's taking ideas from AMD's Mantle (or developed alongside...) to give better direct access to the hardware more akin to consoles.


Edit: and possibly a Windows 9 exclusive?



Yeah, that's my guess too. Looking at the history of DirectX from 9 to 11 that would be a likely direction for them. This applies for both performance and control and exclusivity


But if by some miracle they make the new DirectX available under Windows 7 and up, I will officially bury the hatchet, forget my recent animosity gained towards Microsoft and they will also acquire a huge buffer of good will, enough for one Microsoft employee to shit weekly on my doorstep for at least a year.

#5136063 Wrote some simple SharpdDX tutorials, what do you think?

Posted by on 03 March 2014 - 05:55 AM

This is why I don't like releasing stuff until it is 100% done :). I found some CSAA 32x and multithreading bugs. I fixed them and back-ported the fixes to each version.


I also split up the tutorials, each in its own archive to make it easier to edit one. I edited the first post with links.


DirectX 10 is not progressing at all. I'm getting super strange bugs and abysmal fullscreen MSAA performance, so I'll leave it for later.


The next tutorial is going to be on GUI. Unless someone can point me to a good lightweight but powerful GUI system for SharpDX. I couldn't find one. If you know of one please let me know.


Oh, and I should stop calling these tutorials. They are far too complex for that title. It is more like a DIY step-by-step iterative toy engine and shaders for each common rendering task.

#5135031 Wrote some simple SharpdDX tutorials, what do you think?

Posted by on 27 February 2014 - 07:47 AM



I wrote the 10th one: http://dl.dropboxusercontent.com/u/45638513/sdx/Tut10.zip


It should have the sources and a working exe. If you want to compile it you'll need this too: http://dl.dropboxusercontent.com/u/45638513/sdx/Dependencies.zip


To take control of my running away back log I'll write up the text of the tutorial before I continue!


Now that I remember, I forgot to give the controls for the demo:

  • Mouse buttons: camera control
  • P: Toggle cube rotation
  • Alt-Enter: toggle fullscreen
  • F2: Toggle VSync
  • F3: Cycle MSAA/CSAA
  • F4: Cycle FXAA/SMAA
  • F11: Toggle master post processing override. This way you can force to off the entire post processing framework. Good for debugging.
  • F12: Toggle master AA override. This way you can force to off the entire MSAA/CSAA framework. Good for debugging.
  • T: Toggle debug text.
  • +/-: Adjust bloom threshold
  • O: Toggle bloom debug overlays
  • B: Cycle though bloom modes. Mode 0 is no bloom. Mode 1 is default bloom. The rest of modes are not that useful for realistic rendering.
  • E: Cycle though render modes:
  1. Low
  2. Medium
  3. High: Default
  4. Debug: Ambient spherical harmonics lighting
  5. Debug: Diffuse lighting
  6. Debug: Specular lighting


edit: typo

#5134099 Wrote some simple SharpdDX tutorials, what do you think?

Posted by on 24 February 2014 - 08:12 AM

Hello everybody!


I did not find some proper SharpDX tutorials so I started writing a few. I know that SharpDX is basically DirectX, but porting your knowledge from one to the other is not as straightforward and I did have to spend quite a while searching for things in the documentation. I finished tutorial 9 back in December and wanted to write another one, but then I got into physically based BRDF and that is a huge topic so I couldn't finish the 10th one. Sorry that I did not post them before but I needed to clean them up badly and I've been super busy in 2014.


So a few words about the tutorials: they are built on the idea that you take a very simple sample and keep adding small features to it until you have a basic but rich renderer. The code is written for people who like DirectX (i.e. having full control, manually setting device states, sampler states, shader variables, etc.) but at the same time would like to have a full set of features pretty much out of the box. So you will manually fill in buffers and set variables to achieve simple things like rendering a cube, but that cube will be rendered using normal mapping, bloom, AA. etc.


The code is for DirectX 9.


I have the scripts written for the first 10 tutorials, but I did not have time to write the articles for them yet.


The tutorials do a lot of seemingly random shit, but all of them are there to fix some strange behaviors or instabilities. The written articles would contain explanations for those sections.


What is implemented:

  - master control for post processing and MSAA

  - MSAA and CSAA

  - FXAA and SMAA

  - bloom

  - directional light normal mapping with optional spherical harmonics hack ambient lighting

  - good window control and input (screen capture friendly)


What I would like to add in the future:

  - DirectX 10 mode

  - physically based BRDF 

  - HDR rendering

  - custom resolve for HDR with MSAA (only DirectX 10)

  - flexible light composition scheme (almost done)

  - depth of field


I hope I'll have time to write the articles and the rest of the tutorials, but until then I'm dropping a link here for anybody who might be interested: 

Dependencies (needed only for compiling): http://dl.dropboxusercontent.com/u/45638513/sdx/Dependencies.zip

Media (needed for Tut11+)http://dl.dropboxusercontent.com/u/45638513/sdx/Media.zip

Tut01: http://dl.dropboxusercontent.com/u/45638513/sdx/Tut01.zip

Tut02: http://dl.dropboxusercontent.com/u/45638513/sdx/Tut02.zip

Tut03: http://dl.dropboxusercontent.com/u/45638513/sdx/Tut03.zip

Tut04: http://dl.dropboxusercontent.com/u/45638513/sdx/Tut04.zip

Tut05: http://dl.dropboxusercontent.com/u/45638513/sdx/Tut05.zip

Tut06: http://dl.dropboxusercontent.com/u/45638513/sdx/Tut06.zip

Tut07: http://dl.dropboxusercontent.com/u/45638513/sdx/Tut07.zip

Tut08: http://dl.dropboxusercontent.com/u/45638513/sdx/Tut08.zip

Tut09: http://dl.dropboxusercontent.com/u/45638513/sdx/Tut09.zip

Tut10: http://dl.dropboxusercontent.com/u/45638513/sdx/Tut10.zip





Important note: Tut11 though Tut13 need the contents of Media.zip. All Zip achieves should be unarchived. If you unarchive in foo, you should have foo/Media and foo/TutXX. If you also want to compile, you should have foo/Dependencies.


A few notes:

  - you need .Net 4.0 or compatible

  - you can find ready to run binaries for each tutorial in the TutXX/TutXX/bin folders (this is why the download is so big)

  - the code should be ready to be compiled and run, but running will only work in release mode. For debug mode you need to copy the contents from bin/Release to /bin/Debug. I did not copy it for you in order to keep the download size smaller.

  - there may be bugs. Some coding bug or some resulting from my lacking understanding of the topic. In particular bloom may seem kind of blocky in some samples.


If you find any bugs or have any feedback please let me know smile.png.


I'm learning DirectX 10 now for future tutorials. Did not manage to make any meaningful progress here because DXGI is just not cooperating and even without it rendering fullscreen with MSAA is 3-4 time slower than with DirectX 9.

#5117156 I love all the random crap when porting from XNA to SharpDX

Posted by on 15 December 2013 - 01:59 PM

It seems that this is due to the fact that DirectX uses a left-handed coordinate system and XNA a right-handed one.


For now I have multiplied the Z axis by -1 to fix this, but I might have to change the order of vertices in triangles. And I need to update my binary mesh importers to do the same.


Do I need tot do this to the translation matrices of bones too?

#5116698 I love all the random crap when porting from XNA to SharpDX

Posted by on 13 December 2013 - 09:59 AM

I love all the random crap when porting from XNA to SharpDX:




Inverted faces on that "procedural" cube :).