Even after all these years, terrain generation is still very much a field of experimentation. The book you mentioned even talks about the difficulty of exploring the procedural space in a directed fashion, due to the sheer unpredictability and complexity of the math involved. Really the only reliable trick is to work with functions a lot and get a feel for what certain functions do and how they will interact.
Sedimentary rock can be modeled by assigning a gradient of coloration based on the Y coordinate (or whichever axis is your vertical). You can model local tilt of a chunk of strata using a small scale randomized rotation around an arbitrary horizontal axis, perhaps defined by another set of functions. You can model cracks in rock, or caves, or any other kind of fractures by inverting the output of a ridged fractal function and using it multiplicatively. And so forth.
One trick that I frequently do is to build a program that will randomly generate a chain or tree of functions, then map the output to see what it looks like. Typically, I will script it to generate several hundred permutations and dump PNGs of the result, as well as an XML or YAML file describing the function chain, to browse at my leisure. Whenever I see a pattern I like I will file it away into a library, or open up the YAML descriptor to see exactly what it is doing and maybe figure out how I can tweak it to better accomplish what I have in mind. With a large enough library of these randomized chains, I can usually browse and find something that sort of fits what I'm looking for at any given time.
Here is an image I had posted in an earlier blog entry, showing a few of the results of one such set of randomized functions:
Alternatively, the texturing and modeling book also discusses (briefly) the idea of using some sort of genetic algorithm to construct noise functions, with input from a user (with plenty of time on his hands) to direct the model toward aesthetically pleasing results. This idea seems to have merit, but would involve a great deal of rather complicated coding and a large investment of time.
If you play with noise generation enough, you start to get a feel for things. Be prepared to pour a lot of time into just playing around. The up-side of this is that it's great fun.
Bear in mind that some things are difficult to accomplish using noise functions alone, such as the types of terrain formed by erosion. At this and this links, a couple of noise variants are described that utilize the first-order derivative of a particular layer of noise in order to modify the fractal contributions of successive layers. The results are decent approximations of certain types of eroded terrain, but if you want to accomplish any sort of realism in terrain generation you're going to have to write code to simulate water flow and erosion.
This is beyond helpful! You've definitely helped my research pick right back up. I do have one question though: When you talk about a tree of functions (this of which i understand), I have trouble thinking about the implementation. Are these functions you hardcode in and the program chooses a tree-like structure of chaining functions together? Or is there something I'm missing here? I apologize if this is a bad question . . .
Otherwise, thank you for this monumental post of yours. I've got lots of fun, amazing work set ahead of me!
Also, have you heard of generators? I've heard of people using "sand generators" when creating deserts, and other types of generators. Is this like the simulation you talk about for water flow and erosion, but instead for other things too, like sand and water?