Like Erik said,
in the specific example you specified, you'd need some kind of algorithm that generates an outline of the areas of the different colors, after that, it's not too hard.
So imagine you found some way of finding these areas, then you could check where there are edges, and add these points to an array, in order. Then after getting these points, you could generate a vertex structure, defining the area, and as constructing the triangles can be a bit harder, I could throw this into the post: https://code.google.com/p/poly2tri/ (A tool that converts polygons to triangles, also in c++) Then having this vertex structure, to get this smoothing effect, there's multiple ways of archiving it, like: Tessellation, Bezier Curves, and that's going to be your adventure .
Visualizing all of this would give this:
- We defined the yellow area, and the yellow area itself is a mesh, with ~32 Vertices
- The Blue being the outline of the area, only top though, you can figure the rest out.
- The yellow being the smooth/tessellated outline.
- NOTE: I forgot to highlight 1 edge...
Hope this may help you.