Jump to content

  • Log In with Google      Sign In   
  • Create Account


Member Since 03 Jan 2000
Offline Last Active Yesterday, 05:37 AM

#5292586 multi-light , shadow system

Posted by Ingenu on 20 May 2016 - 01:28 AM

A spatial acceleration structure should help you finding the closest lights, brute force is also a viable option depending on the number of lights you have in your DataBase.

If you write a deferred renderer you won't need to find the closest lights at all.


A directionnal light is a point light at infinity, ie like the sun or moon or star lights, so you will always consider it visible (unless in a cave or such).


Without light the lighting part of your shader won't trigger, no light means pure black unless you have some ambient light to avoid the issue, but sure your per light function won't be called, I don't really see a problem with that...


There are many shadow tech, shadow buffers (usually called shadow maps, even though maps are typically used for precomputed things and not runtime computed things) are a popular tech, and there are plenty of variations, I'd try straightforward shadow mapping (ie depth buffer) + Parallel Split Shadows for the sun/moon/stars.

#5291121 Render Queue Question

Posted by Ingenu on 11 May 2016 - 06:57 AM

Go for deferred shading directly and add a light bucket, that will be much easier.

Otherwise go for the light list, if you plan to go deferred I don't really see the point in wasting my time on the forward proof of concept step, so it will be fine.

#5290966 Best way to abstract shaders in a small engine?

Posted by Ingenu on 10 May 2016 - 08:32 AM

Yes that's closer.


Basically as you write your GPU program code, you decide where you put your data and what data you need, and if you write say "cbuffer Object : register(cb0) float4x4 WorldMatrix};" for your fragment subprogram you have just explicitely decided to put that constant buffer to constant buffer slot 0, you must therefore write the corresponding glue code engine-side, which would something like "gfx.PSSetConstantBuffers( 0, 1, pCB );" in your setParameters(...) procedure, and you must also write the code that will fill the data in that CB (mapping it, casting it and writing the data such as "pCBMatrix = instance->GetWorldMatrix();")

#5290934 Best way to abstract shaders in a small engine?

Posted by Ingenu on 10 May 2016 - 01:48 AM

To some extent, I wouldn't have a "CommandQueue" described in XML because maintaining it would turn bad rather quickly (adding features, never removing deprecated stuff)...

I would rather have the Effect PlugIn system I described above that does that in one of the callable functions instead, because it's way more versatile, much easier to read, understand and modify, and if it's a plugin/dll pretty much as flexible.

You'll need some glue between your engine code and the GPU setup/program.


I would however extend the program to contain meta informations regarding texture binding and sampler descriptions as they do.

#5290795 Best way to abstract shaders in a small engine?

Posted by Ingenu on 09 May 2016 - 08:16 AM

I don't think so, maybe you can find something on the net, however we used to say multi pass to mean rendering the same mesh more than once because we were limited in number of instructions, so we couldn't light a mesh properly rendering it once, that limitation vanished a while ago, but you'll likely still find a lot of references about it.
The way I present a pass is different, think it more a list of logically ordered rendering steps/groups. (I'll use step instead to differentiate with the old definition.)
If you have shadows, do GPU animation, or procedural sky generation on the GPU using Perlin noise, you'll need to run that before you need the data, since those operations are expensive and reused by several "items" in the world, you want to compute them first to know they are ready past that point, hence having a Pre-Process step.
After that, there are logical groups you'll want to render at different times, for correctness you need to render opaque before translucent, so you need two different steps, you also want to be optimal when rendering, so you want to render your opaque either front to back (if you don't do a depth fill step), or in effect order (to use instanced rendering if you have a depth fill step), but the translucent step needs all its meshes rendered in back to front order to be correct.
I will urge you to limit the number of GPU programs to the minimum, to that effect you should have a data driven GPU program, have a look at Disney's BRDF to see a single program that can do a lot with only a few parameters.
It's easier to get one program right, it means you can batch/instance a lot more, and even do better with indirect rendering (since usually indirect API don't allow you to change the GPU program).
The Effect I talked about before is also the glue between your C++ code and the GPU program language, the one that sets data in the right place (slots in D3D11/OGL parlance) before the draw call.
There are so many things that are intertwined when making an engine it's difficult to go into meso description, it's either macro or micro ^^
Anyway, when it comes to mesh, I shared geometry (vb/ib) data between instances, but could have unique textures, so I had a mesh description containing an enum/key+pathfilename such as : "albedo /gfx/monster/joe.tex" so when that mesh is created the associated textures are loaded (well unless the TextureManager has them in memory in which case they are only reference counted.).
To go back to the shadow effect, it's linked to a light, you don't necessarily need a Mesh object but rather a Drawable/Renderable for the effect.
I also subdivided my Effects into DrawEffect, ComputeEffect and a third I can't remember atm ^^

#5290761 Best way to abstract shaders in a small engine?

Posted by Ingenu on 09 May 2016 - 02:08 AM

What about having a Material that's made of effects (for multi pass, think water which needs reflection & refraction for exemple).



array<Effect*> m_Effects;



EPass m_Pass;   //enum RenderingPass { Pre_Process, Reflection, Refraction, Depth_Fill, Opaque, Translucent, Post_Processing, Final }

Pipeline m_Pipeline;   //That's the whole pipeline setup, similar to OpenGL Program Object (which was very well thought)

array<TextureBinding> m_TextureBindings;


void fillCBData( const Mesh& mesh ); //fill CB

void fillGeomData( const Mesh& mesh );  //fill IB & VB (only if you want more flexibility, like having fallback on shaders using only a subset of disk data, I don't do this anymore.)

void setData( const ConstantBuffer& cb, const IndexBuffer& ib, const VertexBuffer& vb, const Camera& camera ); //Set CB, IB, VB, TB, Textures (using the previous array)...

void render();  //Either standalone, or integrated in setData, which name you should then change to reflect it.




uint32_t m_ID; //Could be an enum TextureBindings { Albedo, Normal, Specular, Gloss, ... }

uint32_t m_Slot; //Where to bind it on the GPU



Rendering Process:

Go through you Spatial Graph and gather visible Mesh, for each of them access its Material and then its Effects, store the Mesh & Effect in an array for the pass.

(Something conceptually like : array<array<Thingy>, RenderingPass:COUNT> with Thingy { Effect* m_Effect, Mesh* m_Mesh; })

Generate a key and sort by that key (sorting will be different given the RenderingPass you are in, for exemple you'll sort by material for the Opaque pass, roughly front to back in the Depth_Fill pass and Back to Front in the Translucent pass...), then render.


You'll soon discover that you can change a few things to make it better/faster/more to your liking, it's just a broad presentation of the idea.

I'd strongly advice to use the Pipeline (= Program Object) abstraction as it's how hardware really works and the basis of low level API (Mantle/Vulkan, D3D12)

#5239228 How Does Unreal Engine 4's Rendering Engine Stand Out

Posted by Ingenu on 09 July 2015 - 09:04 AM

Unreal engine's strength has historically been editors/tools, and being feature rich, not really the engine quality which was about average.


I'm not sure about UE4, the tools are once again impressive, I haven't read the code so I don't really know how it is.


The problem isn't whether a programmer can do as good or better, the problem is how long it will take...

AFAIK, UE4 doesn't contain any tech beyond what is already publicly available if you survey the field. (ie read all GDC/SIGGRAPH... papers, and forums.)

#5231430 Graphics engines: what's more common; standard vertex structure or dynami...

Posted by Ingenu on 28 May 2015 - 01:54 AM

For shadow passed I know a few games which used a stream specifically for that. (As Hodgman said, first stream is position + texcoord[if you have alpha testing/punchthrough], second is remaining attributes, both are interleaved inside.)

When it comes to high performance you don't really have a choice, any waste is bad, so tailor your vertex format to what your shader needs.

Often you'll end up with only a few different vertex format, but it doesn't really matter, it's easy enough to support any format anyway...


I can tell you for having tested it back then that having a stream for shadow buffers helped performance a lot (~30%).

You should support both and test on your targets what's giving you the best performance.

There's no magic in programming, it's also science, you suppose, then you test, they you draw conclusions and use whatever is the best trade-off in your case.

(Which might not be performance if it varies only slightly, but ease of use... everyone has different goals.)

#5229587 What the difference between these two books? which one would you recommend?

Posted by Ingenu on 18 May 2015 - 07:35 AM

I'm not sure,I bought #1 on your list and threw it in a bin shortly after, many years ago.


I would recommend "Game Engine Architecture, Second Edition" from Jason Gregory, the best book on the topic I know of.


(For the record I've been a AAA lead 3D programmer and have done GPU design R&D.)

#5182342 DX12 - Documentation / Tutorials?

Posted by Ingenu on 23 September 2014 - 02:49 AM

Pretty much yes.

In fullscreen mode you may expect the rules to be bent in your favour, so things won't be swapped out of VRAM.

#5180949 DX12 - Documentation / Tutorials?

Posted by Ingenu on 17 September 2014 - 02:26 AM

It's pretty much VirtualAlloc & VirtualFree for the GPU.

You still have to manually manage the memory yourself, flagging pages as appropriate and loading/inflating data from disk/RAM to VRAM on need.


Virtual Textures were available in hardware on 3DLabs cards 10 years ago, long before "Mega Textures" ever existed...


Doing manually allows you to predict what you'll need, and keep things compressed in RAM, that's not the case for Managed which needs you to upload everything in final format and let the system page in/out on use [which is, too late to avoid a performance hit].

#5090092 what is a scene graph ?

Posted by Ingenu on 29 August 2013 - 06:32 AM

In 2007 I wrote a series of articles (that really need an update) about 3D engines, in which I talk about SceneTree, SpatialGraph & RenderingQueue...





(Although they got published over time I wrote the whole series in a week-end in September 2007.)


I hope it will help you.

#5078083 The future of graphic APIs

Posted by Ingenu on 16 July 2013 - 02:53 AM

I did write a simple design for a graphics API on Beyond3D : http://forum.beyond3d.com/showthread.php?t=63565

Unfortunately no-one publicly commented on it (it's not nearly finished though) :(


I really want something way simpler and way more empowering than what we have, anyone who knows how GPU are working can just see that OpenGL/D3D are major obstacles in using them efficiently. Unfortunately some of that is due to OS stability/security :(

(Also if you worked on current gen consoles, you probably know of smaller API)

#5027855 Anyone here a self-taught graphics programmer?

Posted by Ingenu on 01 February 2013 - 08:40 AM

I started programming when I was a teen, but it's only when 3D accelerators (PowerVR, 3Dfx Voodoo, Rendition Verite, S3 Savage, nVidia TNT...) became widely available  that I made my dive into 3D, and only because I wanted to make a breath-taking, living environment, and there was no freely available engine that could match what I wanted.

Took me years to learn by myself, today it's much better with the internet and a lot of literature available, but at first I was almost drowning in the middle of an incredible lot of things (algorithms, API, GPU, CPU, cache, memory allocation/management, BRDF, BSSRDF, photons...) to be learnt. At some point though a few things started to make sense and things went into forming a picture becoming clearer and clearer. [Although, in truth, the more you know the more you realize how little you do.]


I also got the chance to work in a very talented team at Funcom, and it was a blast, in the short time I spent there I tremendously improved. Working with great people has an amazing impact, it's like your brain is blooming ; It's an experience I sincerely wish to everyone. (There's a downside though, working in  team of average people feels like suffocating to some extent. sad.png )


I'm currently working on algorithms for upcoming hardware in a GPU company.



I think it would be great to make a list of books we think are really good in our area of expertise including a note about why and which level of expertise is required to read them.

#4966946 OpenGL 4.3 - compute shaders and much more

Posted by Ingenu on 07 August 2012 - 02:59 AM

Still waiting for OpenGL 2 Lean & Mean, and Long Peaks...