Glad to read that my code comes useful for someone but me
So, I'll give you a couple of advices, but before is the most important one. Don't spend time on writing The Fastest Possible Code if you don't have a performance bottleneck or if it isn't your aim. While the performance is acceptable (say, 30-60 FPS), develop new functionality without micro-optimization.
Ok, now let's switch from boring lectures to what you want to read:
if(!SetVertexShader(pD3dscene, ec, pTechnique, pCam)) return false; // 1x SetTechnique, 1x SetMatrix viewproj
You can use shared shader constants (google "HLSL shared") and effect pool and set cross-tech variables like ViewProjection once per frame.
shared float4x4 ViewProj;
In my code it works this way.
Here you save (NumberOfTechniques - 1) * SetMatrix
Also note, that you can pre-multiply World * ViewProj on CPU, if your shaders don't require separate World matrix.
Each pass sets render states you described for it. VertexShader, PixelShader, ZEnable, ZFunc and others. Also here shader constants are filled. Use PIX from DX SDK to dig deeper into the D3DXEffect calls. Here you can reduce state changes by writing passes effectively, especially when using D3DXFX_DONOTSAVESTATE. There is a good article: http://aras-p.info/texts/d3dx_fx_states.html
Instead of iterating through all meshes for all techs, you can (and probably should) sort your models. Using qsort or std::sort it is a trivial task and takes about 5 minutes.
Also for big scenes you may want to use spatial partitioning for visibility checks and avoid testing visibility of every object. Renderer will receive only visible objects, which leads to performance improvement (especially sorting, which depends on number of objects being sorted).
if(!pD3dscene->PreSelectMesh(pMeshIndex[oc], mD3ddev)) return false;// 2x SetMatrix, world/worldinvtransp, 1x SetStreamSource, 1x SetIndices
If you sort your models by geometry, you can do 1x SetStreamSource, 1x SetIndices once per all objects of this geometry (inside the same shader, but often objects of the same geometry DO use the same shader).
Again, shader is tightly coupled with material. Material is just a shader tech + shader variable values for this tech. So, set as much shaders param as you can after setting technique, and don't reset them for each mesh. Say, all golden objects have the same DiffuseColor. Use material "Gold" of shader "metal" and yellow DiffuseColor, set it once and render all golden objects. Sorting by material will help you a lot. Now you have to reset material for each mesh, even if it is the same for half of them.
Check for redundant sets. In my code you can see
called for each object, but inside these methods you will find:
if (CurrVB[Index].get_unsafe() == pVB && CurrVBOffset[Index] == OffsetVertex) return;
Early exits may save you a couple of sets the renderer didn't take care of.
Hope this helps.