As a general rule "people" seem to say you want to minimize the number of render state changes and draw calls, but how agreessive should someone be about this?

For example
you have lots of meshes each with multiple materials

you could
1 -have each mesh build it's own vertex/index buffers for each material. Sort by material and then issue one draw command per mesh-section.

2-
Have each mesh stores it's own vertex/index buffer in main memory
Sort by material
Dynamically build a vertex/index buffer on the GPU for all relevant mesh-sections
Issue a draw call

Both cases minimize the number of render state changes

The second case also minimizes the number of draw calls, but at the cost of the CPU constantly regenerating the vertex buffer each frame. Not reasonable to assume you could reuse the vertex buffer much from frame to frame as the specific meshes in view will change Yes frame coherence will generally keep the same meshes around. But again, to detect that case you are spending CPU time working out if the vertex buffer can be reused in whole or in part.

I recognize the specifics will vary. But "in general" is it a best practice to dynamically create vertex/index buffer each frame, or is the cost of doing it each frame not worth the potential gain?

Use many draw-calls. It would require a very extreme scenario for that to be worth it, like drawing many thousand quads where the 2 different triangles in each quad required a state change.