It's pretty much VirtualAlloc & VirtualFree for the GPU.
You still have to manually manage the memory yourself, flagging pages as appropriate and loading/inflating data from disk/RAM to VRAM on need.
Virtual Textures were available in hardware on 3DLabs cards 10 years ago, long before "Mega Textures" ever existed...
Doing manually allows you to predict what you'll need, and keep things compressed in RAM, that's not the case for Managed which needs you to upload everything in final format and let the system page in/out on use [which is, too late to avoid a performance hit].
Unfortunately no-one publicly commented on it (it's not nearly finished though)
I really want something way simpler and way more empowering than what we have, anyone who knows how GPU are working can just see that OpenGL/D3D are major obstacles in using them efficiently. Unfortunately some of that is due to OS stability/security
(Also if you worked on current gen consoles, you probably know of smaller API)
I started programming when I was a teen, but it's only when 3D accelerators (PowerVR, 3Dfx Voodoo, Rendition Verite, S3 Savage, nVidia TNT...) became widely available that I made my dive into 3D, and only because I wanted to make a breath-taking, living environment, and there was no freely available engine that could match what I wanted.
Took me years to learn by myself, today it's much better with the internet and a lot of literature available, but at first I was almost drowning in the middle of an incredible lot of things (algorithms, API, GPU, CPU, cache, memory allocation/management, BRDF, BSSRDF, photons...) to be learnt. At some point though a few things started to make sense and things went into forming a picture becoming clearer and clearer. [Although, in truth, the more you know the more you realize how little you do.]
I also got the chance to work in a very talented team at Funcom, and it was a blast, in the short time I spent there I tremendously improved. Working with great people has an amazing impact, it's like your brain is blooming ; It's an experience I sincerely wish to everyone. (There's a downside though, working in team of average people feels like suffocating to some extent. )
I'm currently working on algorithms for upcoming hardware in a GPU company.
I think it would be great to make a list of books we think are really good in our area of expertise including a note about why and which level of expertise is required to read them.
In additions to the two good books you already got I recommend reading: -Fundamentals of Computer Graphics by Peter Shirley, Michael Ashikhmin, Steve Marschner. (Know about the Ashikhmin-Shirley BRDF ? It's them.) -Game Engine Architecture by Jason Gregory. (AAA game developer.) -Mathematics for 3D Game Programming and Computer Graphics from Eric Lengyel. (He was one of the first to push for the correct "bitangent" name over "binormal".)
I assume you already know aglorithms, CPU architecture, how memory works, GPU architecture, C++ and at least one graphics API.
If it's not the case I can recommend a few more books on those topics.
2048*2048*36=144MiB hardly a problem.
Modern games rely on depth shadow buffers, a popular tech is parallel split shadow maps, you'll use up to 4096² for the depth shadow buffer, for 4 2048² splits, that's only 64MiB.
(That's only for high-end cards, you'd reduce the number of split an the depth shadow buffer resolution to scale on slower hardware. Try starting with 2 1024² splits to begin with.)
mmh I think the articles I wrote I relevant. I'll try to keep an eye on the topic should you have any more questions. (Keep in mind I wrote that in 2007 even though it got published over time, some things would need an update ^^)
So there are 3 different representations: -a simple scene tree for hiearchical animation. (Note that I only have nodes for some joints and not the whole skeleton anymore.) -a spatial tree/graph for visibility culling/selection. (That can be whatever you need, Sector/Portal, BSP, Octree, Quadtree, kd-tree... Or a combination of them...) -a rendering queue/flat tree for rendering. (In which you sort by state prior to rendering, includes batching/instancing support.)
I consider the Scene Tree to be the main representation. atm I have an Entity class that inherits from SceneNode & Spatial to "join" the Scene & Spatial representation, you could decide to do it differently. (for example associate Spatial & Renderable instead) My Spatial class have a cull(...) function in which it adds its renderables to the Rendering Queue passed as a parameter.
I would like to point out that Scene Management and Visibility/Culling are two different things.
The first is usually a way to uniformely manage scene entities, and often handle hierarchical animation. (Typically a sword in an hand.)
The second is about finding the best algorithm to find out what's inside a convex area.
There are many solutions to that one, and an engine could be abstract enough to replace one by another to answer the specific needs of a game.
(A RTSG would not have the same requirements as a FPS.)
Popular systems are Quadtrees, Portals/Sectors...
BSP are not about culling anyway, they are about finding where you are in the world, it's the PVS that does the culling, and in Doom 3, the BSP is used along with Sectors and Portals.
(You can just download Doom 3 demo and open the files they are in text format. They call Sectors "Areas" AFAIR.)
Today you will most likely end up with a quadtree, occlusion culling and maybe portals/sectors (if you mix indoor/outdoor with complex indoor), which is pretty much what Phantom already said.