Thanks, I'll look into the Haar stuff, sounds interesting.
I am not doing physics on the GPU, only visuals. The LOD is also only for the visuals. I need to do accurate physics for the whole level, since I am doing multiplayer. At least on the server.
I agree that the physics should be "fixed". I am using Bullet though, so the fixing will most likely be creating a smoother terrain, and more movement friendly physics meshes for the vehicles.
Wavelet transform might work for you, but you should know that it doesn't differ from the mean by much. Linear filters are notorious for eliminating a lot of important data (mostly transitions), while all you want is to eliminate small bumps with is as little effect on the other data as possible.
You should really consider non-linear filters, like the bilateral filter or the more general nlm filter. Efficient implementation of these two is a bit tricky but doable as long as you don't go wild with the filter radius. Also, don't even try to implement them in the naive way, they'll be terribly slow.