I've been at this for hours and IMO my code looks pretty clear-cut. Nevertheless, I'm obviously missing something because two things are happening that I'm unable to explain:
1) first off, I'm trying to blur my shadowmap cascades (I'm using four)
2) my source is a four-layer float 32 GL_TEXTURE_2D_ARRAY RGB target (call it texVSM), which is updated properly in the VSM projective phase
3) Peculiarity 1: for some reason my VSM FBO requires me to bind the target array (texVSM) to a texture (input) although no texture look-ups are being done inside the shader (it just calculates the moments and stores them in the active slice); if I don't bind the texture, the depth buffer is updated, but the color attachment (texVSM) is not. As usual, I'm selecting my output slice by calling:
I'm relatively sure you don't need to bind a texture for it to work as an FBO attachment.
4) rendering shadows as-is at this point works nicely (save for the first peculiarity)
5) my second FBO expects texVSM as its input at TU 0 and writes to another 3D texture attachment (call it texBlurVSM) (I've also given it a go by binding the output target as its primary input)
6) Peculiarity 2: nothing is ever written to texBlurVSM, even if I set the shader to output a constant color. This is very weird, because the code is pretty much identical to that of the first FBO, except for the fact that it's a post-processing effect and just draws a single quad. In other words, I'm unable to copy anything from one 3D texture to another through an FBO proxy. I've disable depth testing and masking for the blur FBO and it makes no difference
At this point I'm rather out of ideas as to what to even try or change. So far I have something like 6 FBO's in use and they all work just fine save for the blur one.
Here's a stitched-together mock-up of the code I'm using for both FBOs (these calls are pseudocode for actual OpenGL calls):
fbo = CreateFBO();
fbo->AddDepthBuffer();
fbo->AddAttachement(new Texture);
fbo->AddInputs(...); //basically adds all texture handles to the FBO's internal list for automated binding
fbo->Enable(); //updates attachments and calls glBindTexture() for all inputs (texture targets and TU's are automatically taken care of)
for(i to numCascades)
{
fbo->ActivateLayer(i); //calls glFramebufferTextureLayerEXT() as above for all attached output targets
ClearDepthAndColor();
//RenderShadowCasters() for the projective phase
//Draw a quad over the viewport for the blur phase
}
fbo->Disable();
I think you might need a call to glDrawBuffer or glDrawBuffers, if you want to render to something other than GL_COLOR_ATTACHMENT0.
Incidentally, from what I remember of rendering to a 3d / array texture layer, the call to glFramebufferTextureLayer attaches one layer of the texture to the attachment point. So if you're only rendering to one layer at a time, you might as well use GL_COLOR_ATTACHMENT0 for all of them, instead of GL_COLOR_ATTACHMENT0 + i. (If I'm not misunderstanding what you're doing with iIndex / iLayerIndex here).
On the other hand, if you want to render to several layers at once, you should attach them all to separate attachment points once, then output multiple values from the shader (using gl_FragData or different out locations). I'm not sure what your AddAttachment(new Texture) call does -> if this is the whole 3D texture, it's not necessary, you just need to attach the individual layers. (Binding the whole 3D texture is instead used with gl_Layer in a geometry shader to render to each layer.)
Could you try and explain what's happening and why you think there's anything wrong. If you're trying to validate your results using gDebugger you would find that it simply doesn't work when it comes to layered FBOs.
Also FBO's are very VERY picky when it comes to which calls you're using and I've personally experienced different behaviors using the EXT, ARB and core calls. So try to translate the pseudocode into actual ogl calls so we have a chance to see which you're using.
Also if you're trying to render to several layeres at once do as __sprite also said - Pick a layer using a geometry shader as that's the only way atm. If you try to do that you need to bind the layered texture to the FBO using the glFramebufferTexture() call NOT the layered one as that only binds a single slice.
Ugh, work never helps to reply faster; but I finally have the time to post the full code (the iterator iIndex for the attachment number is from a generic piece of code that binds any number of added attachments - it's not specific to this example). The GL code that is called is (note that I'm actually checking each and every GL FBO call for errors and doing rather strict compatibility and format/error checking, which I have removed for clarity):
//accepts an already propely created texture ref; additional error and compatibility checking is done here
void FBO:AddAttachment(TEXHANDLE tex)
{
output.push_back(tex);
int iIndex = output.size() - 1;
void FBO::Enable()
{
//set up the target list for glDrawBuffers(); the output[] array contains refs to actual target handles
UINT mrt[GD_FBO_MAX_TARGETS];
UINT iNumActiveTargets = 0;
for(UINT i = 0; i < GD_MAX_TEXTURE_UNITS; i++)
{
if(output)
{ mrt = GL_COLOR_ATTACHMENT0_EXT + i; iNumActiveTargets = i + 1; }
else
mrt = NULL;
}
I hope there are no mistakes in the above code as I had to copy these bits together from all over the place (my code is structured in an object-oriented manner on multiple levels). I'm not including my texture generation code, which is used to actually create all the targets as it's an all-around piece of code and contains some hacks (nevertheless, I use it all over the place and it works just fine, both for 2D and 3D targets). If there's any other code I can include, please let me know; I'm guessing that, if anything, the above bit should be to blame, though.
>> (Binding the whole 3D texture is instead used with gl_Layer in a geometry shader to render to each layer.)
My Cascaded VSM store depth code uses the above method of output layer selection and works just fine; why would I need to use a geometry shader here?
Also, I'm only rendering to a single slice at once (the calling code structure in the OP loops through each layer and does the magic individually on each one)!
Your ActivateLayer function looks incomplete / wrong, you might want to check you've copied it here correctly. (You're attaching a layer of some texture as a depth attachment, then iterating something and attaching that same layer as color outputs... even if that's allowed in the specs (which seems unlikely) it doesn't make any sense).
Your enable function also appears to be adding a draw buffer for every attachment, but you say you only want to render to one attachment. I'm not sure what the specs say happens if you only output to one drawbuffer having specified several, but it may be undefined (at least for the buffers you don't output to).
(I'm not sure that putting inputs in the fbo class really makes sense from a conceptual viewpoint of what a framebuffer object actually is either?)
In the end, you appear to want to render to one output attachment at a time. That output just happens to be a layer of a 3D texture. So you want something like this:
[font="Lucida Console"][size="1"]// At startup:
Create your fbo.
Attach the depth buffer (assuming this stays constant, if not, do so at render time).
// At render time:
For each layer of 3D texture
* Attach one layer of 3D texture to COLOR_ATTACHMENT0 (using glFramebufferTextureLayer).
* Render to that layer (just output gl_FragColor).
[/font]
OR something like this:
[font="Lucida Console"][size="1"]// At startup:
Create your fbo.
Attach depth buffer (assuming it's constant...)
Attach 3D texture to GL_COLOR_ATTACHMENT0 (using glFramebufferTexture).
// At render time:
Render once (use a geometry shader to output vertices once for each layer, using gl_Layer to specify which. Fragment shader outputs to gl_FragColor.)
[/font]
On the other hand, if you really wanted to render to multiple attachments, your code would look something like this:
[font="Lucida Console"][size="1"]// At startup:
Create fbo.
Attach depth buffer.
Attach one layer of 3D texture to each attachment point (using glFramebufferTextureLayer).
// At render time:
Render once (Fragment shader outputs a value to each attachment using gl_FragData[n]).[/font]
V-Man - my code automatically degenerates to rendering to just one attachment if there is just one color target attached . Generally working with 2 or more attachements made me write an FBO class that automatically takes care of any number of attached targets.
__sprite:
You're absolutely right - I made a mistake when copying the code. The layer activation code should be (with some more elaboration):
void FBO::ActivatLayer(int iLayerIndex)
{
//activate the desired layer for the depth buffer (assumed to be 3D)
glFramebufferTextureLayerEXT(GL_FRAMEBUFFER_EXT, GL_DEPTH_ATTACHMENT_EXT, iDepthHandle, 0, iLayerIndex);
//loop through all color attachments (assumed to be 3D and have the same number of layers as the depth buffer) and activate the desired layer
for(iIndex in all attachments)
glFramebufferTextureLayerEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT + iIndex, output[iIndex], 0, iLayerIndex);
}
Which assumes the FBO has any number of color attachments and a single depth attachment, all of which have the same number of layers. The reason I'm not just activating attachment 0 is because I'm looping through all color targets and activating them to their rightful attachments at the requested layer. Again, this degenerates to what V-Man wrote if there is only one attachment. The same applies to Enable() - the code assumes the FBO can have up to GD_MAX_TEXTURE_UNITS attachments, but only enables those that have actual targets attached to them. Unless I'm blind to boot, I'm pretty sure there are no logic problems with this code.
What is especially annoying is the fact that OpenGL is generating zero errors (as mentioned in the OP, I'm actually checking each and every FBO call for errors and none are generated - I've removed the checks for clarity from the posted code). Is there any way I can check if my GTX 260M is even supposed to support this? I mean, maybe the drivers are silently letting the code slip, but there's no actual hardware support.