Let's say I define a spline-based coord system. Vertex positions are set relative to a straight line, and we view bending the spline as deforming the coordinate system. Or another example would be modeling a gravity well... situations where a coordinate system is not linear.

How can you move this kind of task to the (vertex?) shader, so it knows how to transform the vertex from its special coordinate system to a regular 3D cartesian space? You'd have to have a way to describe the coordinate system in some way - the spline example means the shader would need to understand how the spline works, etc.

Any good articles on this?

A shader is code you're writing, so if that's how you want to deform vertices then go for it. Nothing is stopping you, except possibly data representation. If you wanted to use a Bezier curve as your basis, you could have a constant array in the shader that defines the coefficients of the desired curve. The math for transforming the vertices will be no different than they would be anywhere else.

spline example means the shader would need to understand how the spline works, etc.[/quote]I think that's a given. The vertex shader is doing the transform, it has to be aware of the transformations being done. In most graphics work the transformation can be represented as a basis change with translation (yielding a 4x4 matrix), but it certainly doesn't have to be that way.

A spline can be a fairly complex bit of code, working out the axes at any point along the spline. So I suppose my question is how easily you can move all that to shaders... big arrays of the spline control points and so on.
Is it sensible to put all of that into the VS,,, or to pre-calculate a bunch of stuff on the CPU so the VS has much simpler calculations to do? I'm targeting SM3, which probably is relevant to this discussion.