With the design above, it forces the cbuffers defined in the FX section to have unique names so that a user could look up a cbuffer for a particular shader type.

Yeah at the moment, I force shader authors to give every cbuffer both a name and an ID.

Also, how does the mask fit in with your state group idea where you have cbuffer bind commands, are these commands necessary if you already have a mask to know what has to set?

The mask is separate from the binding commands.
After the user has issued a bunch of items to draw, it ends up as a list of state-changes (e.g. bind commands) and draw-calls. In the D3D9 renderer, a bind command simply writes some data into a cache and sets a dirty bit. When a draw-call is issued, this cbuffer cache is used to perform the actual Set*ShaderConstantF calls just prior to the Draw*Primitive call.

Before setting shader constants (before a draw-call), the renderer first has to select a shader permutation, which has one of these cbuffer masks. Only the cbuffers in the cache which are specified in this mask will be flushed through to D3D. Note that this is just a slight optimisation though, not a required feature.

There's a few reasons that the user may have issued a bind-cbuffer command that isn't actually needed --- perhaps the cbuffer is used by most permutations, but there's one permutation where it's not used. e.g. maybe your scene has a "flash of lightning" effect, where you simply disabling lighting calculations for that frame. The user might bind a light cbuffer regardless, but then some other layered render-state causes a "no lighting" permutation to be selected. In this situation, this allows the author of the "lighting" effect to quickly implement their idea without changing any of the "calculate/bind lighting cbuffers" code.

Another possibility is that you've got some global cbuffers, which you simply always bind out of convenience. e.g. maybe you define cbuffer #12 as holding fog/atmosphere settings, and bind this data to this slot by default (unless a particular renderable overrides that state with it's own binding). In this case, every draw-call would have something bound to cbuffer slot #12, but there might be some shaders/permutations that ignore fog, and hence don't need that buffer to be bound.

The CBuffer defined in the FX section is mainly just for creating the lookup table, the actual CBuffer layout is created in each shader type via reflection.
I'm not really sure this is the best way to go about it so I'm open to suggestions as I'm still in design/thinking stages

My only objection to that format is that you're repeating yourself (declaring cbuffers twice), which adds an extra place where mistakes can be made.

Would it be possibe to automatically generate one of these sets of information from the other? (e.g. generate the HLSL variables from the FX, or generate the FX by parsing/reflecting the HLSL?).

perObjectVSIndex = model->Effect()->FindCBuffer("cbPerObjectVS");
perObjectVSData = model->Effect()->CloneBuffer(perObjectVSIndex);

I've got an API like this, but for many cases I don't have to use it. For most engine-provided data, I can instead do:
perObjectVSData = CBuffer::Create( sizeof(StructThatIPromiseMatchesMyHLSL) );

And for debugging:
CBufferInfo* cb = model->Effect()->FindCBuffer("cbPerObjectVS");
ASSERT( cb->SizeInBytes() == sizeof(StructThatIPromiseMatchesMyHLSL) );
ASSERT( OFFSETOF(StructThatIPromiseMatchesMyHLSL::foo) == cb->OffsetOf("foo") );

By the way, seeing as you use C# for your content pipeline, does that mean you only target windows for your engine or do you have bindings for other languages too?

The engine is multi-platform, but the content tools are only designed to work on a Windows PC, because that's what we use for development ;)

There's a few options for integrating your windows-only content tools with your cross-platform engine that I've personally used:
1) Have some kind of "editor" build of the engine, which does more stuff than any of the single-platform builds (e.g. contains multiple different platform-specific data structures - so you can serialize your data for each different platform).
2) Link your content tools against your Windows build of your engine, and use formats that serialize identically for all platforms.
3) Don't directly link your content tools to your engine at all. Instead, define a specification for the input/output data formats, and implement a that spec once in the tools (as a producer) and once in the engine (as a consumer).

I currently use method #3 -- The C# tools uses easy-to-use-but-bloated data structures internally, and have hand-written binary serialisation code for outputting data to the engine. The C++ engine loads these data files into byte-arrays and casts them to hand-written structs that match the expected data layouts.

With the design above, it forces the cbuffers defined in the FX section to have unique names so that a user could look up a cbuffer for a particular shader type.

Yeah at the moment, I force shader authors to give every cbuffer both a name and an ID.

Also, how does the mask fit in with your state group idea where you have cbuffer bind commands, are these commands necessary if you already have a mask to know what has to set?

The mask is separate from the binding commands.
After the user has issued a bunch of items to draw, it ends up as a list of state-changes (e.g. bind commands) and draw-calls. In the D3D9 renderer, a bind command simply writes some data into a cache and sets a dirty bit. When a draw-call is issued, this cbuffer cache is used to perform the actual Set*ShaderConstantF calls just prior to the Draw*Primitive call.

Before setting shader constants (before a draw-call), the renderer first has to select a shader permutation, which has one of these cbuffer masks. Only the cbuffers in the cache which are specified in this mask will be flushed through to D3D. Note that this is just a slight optimisation though, not a required feature.

There's a few reasons that the user may have issued a bind-cbuffer command that isn't actually needed --- perhaps the cbuffer is used by most permutations, but there's one permutation where it's not used. e.g. maybe your scene has a "flash of lightning" effect, where you simply disabling lighting calculations for that frame. The user might bind a light cbuffer regardless, but then some other layered render-state causes a "no lighting" permutation to be selected. In this situation, this allows the author of the "lighting" effect to quickly implement their idea without changing any of the "calculate/bind lighting cbuffers" code.

Another possibility is that you've got some global cbuffers, which you simply always bind out of convenience. e.g. maybe you define cbuffer #12 as holding fog/atmosphere settings, and bind this data to this slot by default (unless a particular renderable overrides that state with it's own binding). In this case, every draw-call would have something bound to cbuffer slot #12, but there might be some shaders/permutations that ignore fog, and hence don't need that buffer to be bound.

The CBuffer defined in the FX section is mainly just for creating the lookup table, the actual CBuffer layout is created in each shader type via reflection.
I'm not really sure this is the best way to go about it so I'm open to suggestions as I'm still in design/thinking stages

My only objection to that format is that you're repeating yourself (declaring cbuffers twice), which adds an extra place where mistakes can be made.

Would it be possibe to automatically generate one of these sets of information from the other? (e.g. generate the HLSL variables from the FX, or generate the FX by parsing/reflecting the HLSL?).

perObjectVSIndex = model->Effect()->FindCBuffer("cbPerObjectVS");
perObjectVSData = model->Effect()->CloneBuffer(perObjectVSIndex);

I've got an API like this, but for many cases I don't have to use it. For most engine-provided data, I can instead do:
perObjectVSData = CBuffer::Create( sizeof(StructThatIPromiseMatchesMyHLSL) );

And for debugging:
CBufferInfo* cb = model->Effect()->FindCBuffer("cbPerObjectVS");
ASSERT( cb->SizeInBytes() == sizeof(StructThatIPromiseMatchesMyHLSL) );
ASSERT( OFFSETOF(StructThatIPromiseMatchesMyHLSL::foo) == cb->OffsetOf("foo") );

By the way, seeing as you use C# for your content pipeline, does that mean you only target windows for your engine or do you have bindings for other languages too?

The engine is multi-platform, but the content tools are only designed to work on a Windows PC, because that's what we use for development ;)

There's a few options for integrating your windows-only content tools with your cross-platform engine:
1) Have some kind of "editor" build of the engine, which does more stuff than any of the single-platform builds (e.g. contains multiple different platform-specific data structures - so you can serialize your data for each different platform).
2) Link your content tools against your Windows build of your engine, and use formats that serialize identically for all platforms.
3) Don't directly link your content tools to your engine at all. Instead, define a specification for the input/output data formats, and implement a that spec once in the tools (as a producer) and once in the engine (as a consumer).

I use method #3 -- The C# tools uses easy-to-use-but-bloated data structures internally, and have hand-written binary serialisation code for outputting data to the engine. The C++ engine loads these data files into byte-arrays and casts them to hand-written structs that match the expected data layouts.

With the design above, it forces the cbuffers defined in the FX section to have unique names so that a user could look up a cbuffer for a particular shader type.

Yeah at the moment, I force shader authors to give every cbuffer both a name and an ID.

Also, how does the mask fit in with your state group idea where you have cbuffer bind commands, are these commands necessary if you already have a mask to know what has to set?

The mask is separate from the binding commands.
After the user has issued a bunch of items to draw, it ends up as a list of state-changes (e.g. bind commands) and draw-calls. In the D3D9 renderer, a bind command simply writes some data into a cache and sets a dirty bit. When a draw-call is issued, this cbuffer cache is used to perform the actual Set*ShaderConstantF calls just prior to the Draw*Primitive call.

Before setting shader constants (before a draw-call), the renderer first has to select a shader permutation, which has one of these cbuffer masks. Only the cbuffers in the cache which are specified in this mask will be flushed through to D3D. Note that this is just a slight optimisation though, not a required feature.

There's a few reasons that the user may have issued a bind-cbuffer command that isn't actually needed --- perhaps the cbuffer is used by most permutations, but there's one permutation where it's not used. e.g. maybe your scene has a "flash of lightning" effect, where you simply disabling lighting calculations for that frame. The user might bind a light cbuffer regardless, but then some other layered render-state causes a "no lighting" permutation to be selected. In this situation, this allows the author of the "lighting" effect to quickly implement their idea without changing any of the "calculate/bind lighting cbuffers" code.

Another possibility is that you've got some global cbuffers, which you simply always bind out of convenience. e.g. maybe you define cbuffer #12 as holding fog/atmosphere settings, and bind this data to this slot by default (unless a particular renderable overrides that state with it's own binding). In this case, every draw-call would have something bound to cbuffer slot #12, but there might be some shaders/permutations that ignore fog, and hence don't need that buffer to be bound.

The CBuffer defined in the FX section is mainly just for creating the lookup table, the actual CBuffer layout is created in each shader type via reflection.
I'm not really sure this is the best way to go about it so I'm open to suggestions as I'm still in design/thinking stages

My only objection to that format is that you're repeating yourself (declaring cbuffers twice), which adds an extra place where mistakes can be made.

Would it be possibe to automatically generate one of these sets of information from the other? (e.g. generate the HLSL variables from the FX, or generate the FX by parsing/reflecting the HLSL?).

perObjectVSIndex = model->Effect()->FindCBuffer("cbPerObjectVS");
perObjectVSData = model->Effect()->CloneBuffer(perObjectVSIndex);

I've got an API like this, but for many cases I don't have to use it. For most engine-provided data, I can instead do:
perObjectVSData = CBuffer::Create( sizeof(StructThatIPromiseMatchesMyHLSL) );

And for debugging:
CBufferInfo cb =

model->Effect()->FindCBuffer("cbPerObjectVS");

ASSERT( cb->SizeInBytes() == sizeof(StructThatIPromiseMatchesMyHLSL) );

ASSERT( OFFSETOF(StructThatIPromiseMatchesMyHLSL::foo) ==

cb->OffsetOf("foo") );

By the way, seeing as you use C# for your content pipeline, does that mean you only target windows for your engine or do you have bindings for other languages too?

The engine is multi-platform, but the content tools are only designed to work on a Windows PC, because that's what we use for development ;)

There's a few options for integrating your windows-only content tools with your cross-platform engine:
1) Have some kind of "editor" build of the engine, which does more stuff than any of the single-platform builds (e.g. contains multiple different platform-specific data structures - so you can serialize your data for each different platform).
2) Link your content tools against your Windows build of your engine, and use formats that serialize identically for all platforms.
3) Don't directly link your content tools to your engine at all. Instead, define a specification for the input/output data formats, and implement a that spec once in the tools (as a producer) and once in the engine (as a consumer).

I use method #3 -- The C# tools uses easy-to-use-but-bloated data structures internally, and have hand-written binary serialisation code for outputting data to the engine. The C++ engine loads these data files into byte-arrays and casts them to hand-written structs that match the expected data layouts.

With the design above, it forces the cbuffers defined in the FX section to have unique names so that a user could look up a cbuffer for a particular shader type.

Yeah at the moment, I force shader authors to give every cbuffer both a name and an ID.

Also, how does the mask fit in with your state group idea where you have cbuffer bind commands, are these commands necessary if you already have a mask to know what has to set?

The mask is separate from the binding commands.
After the user has issued a bunch of items to draw, it ends up as a list of state-changes (e.g. bind commands) and draw-calls. In the D3D9 renderer, a bind command simply writes some data into a cache and sets a dirty bit. When a draw-call is issued, this cbuffer cache is used to perform the actual Set*ShaderConstantF calls just prior to the Draw*Primitive call.

Before setting shader constants (before a draw-call), the renderer first has to select a shader permutation, which has one of these cbuffer masks. Only the cbuffers in the cache which are specified in this mask will be flushed through to D3D. Note that this is just a slight optimisation though, not a required feature.

There's a few reasons that the user may have issued a bind-cbuffer command that isn't actually needed --- perhaps the cbuffer is used by most permutations, but there's one permutation where it's not used. e.g. maybe your scene has a "flash of lightning" effect, where you simply disabling lighting calculations for that frame. The user might bind a light cbuffer regardless, but then some other layered render-state causes a "no lighting" permutation to be selected. In this situation, this allows the author of the "lighting" effect to quickly implement their idea without changing any of the "calculate/bind lighting cbuffers" code.

Another possibility is that you've got some global cbuffers, which you simply always bind out of convenience. e.g. maybe you define cbuffer #12 as holding fog/atmosphere settings, and bind this data to this slot by default (unless a particular renderable overrides that state with it's own binding). In this case, every draw-call would have something bound to cbuffer slot #12, but there might be some shaders/permutations that ignore fog, and hence don't need that buffer to be bound.

The CBuffer defined in the FX section is mainly just for creating the lookup table, the actual CBuffer layout is created in each shader type via reflection.
I'm not really sure this is the best way to go about it so I'm open to suggestions as I'm still in design/thinking stages

My only objection to that format is that you're repeating yourself (declaring cbuffers twice), which adds an extra place where mistakes can be made.

Would it be possibe to automatically generate one of these sets of information from the other? (e.g. generate the HLSL variables from the FX, or generate the FX by parsing/reflecting the HLSL?).

perObjectVSIndex = model->Effect()->FindCBuffer("cbPerObjectVS");
perObjectVSData = model->Effect()->CloneBuffer(perObjectVSIndex);

I've got an API like this, but for many cases I don't have to use it. For most engine-provided data, I can instead do:
perObjectVSData = CBuffer::Create( sizeof(StructThatIPromiseMatchesMyHLSL) );


And for debugging:
CBufferInfo cb =

model->Effect()->FindCBuffer("cbPerObjectVS");

ASSERT( cb->SizeInBytes() ==

sizeof(StructThatIPromiseMatchesMyHLSL) );
ASSERT( OFFSETOF(StructThatIPromiseMatchesMyHLSL::foo) ==

cb->OffsetOf("foo") );

By the way, seeing as you use C# for your content pipeline, does that mean you only target windows for your engine or do you have bindings for other languages too?

The engine is multi-platform, but the content tools are only designed to work on a Windows PC, because that's what we use for development ;)

There's a few options for integrating your windows-only content tools with your cross-platform engine:
1) Have some kind of "editor" build of the engine, which does more stuff than any of the single-platform builds (e.g. contains multiple different platform-specific data structures - so you can serialize your data for each different platform).
2) Link your content tools against your Windows build of your engine, and use formats that serialize identically for all platforms.
3) Don't directly link your content tools to your engine at all. Instead, define a specification for the input/output data formats, and implement a that spec once in the tools (as a producer) and once in the engine (as a consumer).

I use method #3 -- The C# tools uses easy-to-use-but-bloated data structures internally, and have hand-written binary serialisation code for outputting data to the engine. The C++ engine loads these data files into byte-arrays and casts them to hand-written structs that match the expected data layouts.

With the design above, it forces the cbuffers defined in the FX section to have unique names so that a user could look up a cbuffer for a particular shader type.

Yeah at the moment, I force shader authors to give every cbuffer both a name and an ID.

Also, how does the mask fit in with your state group idea where you have cbuffer bind commands, are these commands necessary if you already have a mask to know what has to set?

The mask is separate from the binding commands.
After the user has issued a bunch of items to draw, it ends up as a list of state-changes (e.g. bind commands) and draw-calls. In the D3D9 renderer, a bind command simply writes some data into a cache and sets a dirty bit. When a draw-call is issued, this cbuffer cache is used to perform the actual Set*ShaderConstantF calls just prior to the Draw*Primitive call.

Before setting shader constants (before a draw-call), the renderer first has to select a shader permutation, which has one of these cbuffer masks. Only the cbuffers in the cache which are specified in this mask will be flushed through to D3D. Note that this is just a slight optimisation though, not a required feature.

There's a few reasons that the user may have issued a bind-cbuffer command that isn't actually needed --- perhaps the cbuffer is used by most permutations, but there's one permutation where it's not used. e.g. maybe your scene has a "flash of lightning" effect, where you simply disabling lighting calculations for that frame. The user might bind a light cbuffer regardless, but then some other layered render-state causes a "no lighting" permutation to be selected. In this situation, this allows the author of the "lighting" effect to quickly implement their idea without changing any of the "calculate/bind lighting cbuffers" code.

Another possibility is that you've got some global cbuffers, which you simply always bind out of convenience. e.g. maybe you define cbuffer #12 as holding fog/atmosphere settings, and bind this data to this slot by default (unless a particular renderable overrides that state with it's own binding). In this case, every draw-call would have something bound to cbuffer slot #12, but there might be some shaders/permutations that ignore fog, and hence don't need that buffer to be bound.

The CBuffer defined in the FX section is mainly just for creating the lookup table, the actual CBuffer layout is created in each shader type via reflection.
I'm not really sure this is the best way to go about it so I'm open to suggestions as I'm still in design/thinking stages

My only objection to that format is that you're repeating yourself (declaring cbuffers twice), which adds an extra place where mistakes can be made.

Would it be possibe to automatically generate one of these sets of information from the other? (e.g. generate the HLSL variables from the FX, or generate the FX by parsing/reflecting the HLSL?).

perObjectVSIndex = model->Effect()->FindCBuffer("cbPerObjectVS");
perObjectVSData = model->Effect()->CloneBuffer(perObjectVSIndex);

I've got an API like this, but for many cases I don't have to use it. For most engine-provided data, I can instead do:
perObjectVSData = CBuffer::Create( sizeof(StructThatIPromiseMatchesMyHLSL) );

By the way, seeing as you use C# for your content pipeline, does that mean you only target windows for your engine or do you have bindings for other languages too?

The engine is multi-platform, but the content tools are only designed to work on a Windows PC, because that's what we use for development ;)

There's a few options for integrating your windows-only content tools with your cross-platform engine:
1) Have some kind of "editor" build of the engine, which does more stuff than any of the single-platform builds (e.g. contains multiple different platform-specific data structures - so you can serialize your data for each different platform).
2) Link your content tools against your Windows build of your engine, and use formats that serialize identically for all platforms.
3) Don't directly link your content tools to your engine at all. Instead, define a specification for the input/output data formats, and implement a that spec once in the tools (as a producer) and once in the engine (as a consumer).

I use method #3 -- The C# tools uses easy-to-use-but-bloated data structures internally, and have hand-written binary serialisation code for outputting data to the engine. The C++ engine loads these data files into byte-arrays and casts them to hand-written structs that match the expected data layouts.

With the design above, it forces the cbuffers defined in the FX section to have unique names so that a user could look up a cbuffer for a particular shader type.

Yeah at the moment, I force shader authors to give every cbuffer both a name and an ID.

Also, how does the mask fit in with your state group idea where you have cbuffer bind commands, are these commands necessary if you already have a mask to know what has to set?

The mask is separate from the binding commands.
After the user has issued a bunch of items to draw, it ends up as a list of state-changes (e.g. bind commands) and draw-calls. In the D3D9 renderer, a bind command simply writes some data into a cache and sets a dirty bit. When a draw-call is issued, this cbuffer cache is used to perform the actual Set*ShaderConstantF calls just prior to the Draw*Primitive call.

Before setting shader constants (before a draw-call), the renderer first has to select a shader permutation, which has one of these cbuffer masks. Only the cbuffers in the cache which are specified in this mask will be flushed through to D3D. Note that this is just a slight optimisation though, not a required feature.

There's a few reasons that the user may have issued a bind-cbuffer command that isn't actually needed --- perhaps the cbuffer is used by most permutations, but there's one permutation where it's not used. e.g. maybe your scene has a "flash of lightning" effect, where you simply disabling lighting calculations for that frame. The user might bind a light cbuffer regardless, but then some other layered render-state causes a "no lighting" permutation to be selected. In this situation, this allows the author of the "lighting" effect to quickly implement their idea without changing any of the "calculate/bind lighting cbuffers" code.

Another possibility is that you've got some global cbuffers, which you simply always bind out of convenience. e.g. maybe you define cbuffer #12 as holding fog/atmosphere settings, and bind this data to this slot by default (unless a particular renderable overrides that state with it's own binding). In this case, every draw-call would have something bound to cbuffer slot #12, but there might be some shaders/permutations that ignore fog, and hence don't need that buffer to be bound.

The CBuffer defined in the FX section is mainly just for creating the lookup table, the actual CBuffer layout is created in each shader type via reflection.
I'm not really sure this is the best way to go about it so I'm open to suggestions as I'm still in design/thinking stages

My only objection to that format is that you're repeating yourself (declaring cbuffers twice), which adds an extra place where mistakes can be made.

Would it be possibe to automatically generate one of these sets of information from the other? (e.g. generate the HLSL variables from the FX, or generate the FX by parsing/reflecting the HLSL?).

By the way, seeing as you use C# for your content pipeline, does that mean you only target windows for your engine or do you have bindings for other languages too?

The engine is multi-platform, but the content tools are only designed to work on a Windows PC, because that's what we use for development ;)

There's a few options for integrating your windows-only content tools with your cross-platform engine:
1) Have some kind of "editor" build of the engine, which does more stuff than any of the single-platform builds (e.g. contains multiple different platform-specific data structures - so you can serialize your data for each different platform).
2) Link your content tools against your Windows build of your engine, and use formats that serialize identically for all platforms.
3) Don't directly link your content tools to your engine at all. Instead, define a specification for the input/output data formats, and implement a that spec once in the tools (as a producer) and once in the engine (as a consumer).

I use method #3 -- The C# tools uses easy-to-use-but-bloated data structures internally, and have hand-written binary serialisation code for outputting data to the engine. The C++ engine loads these data files into byte-arrays and casts them to hand-written structs that match the expected data layouts.