What you want is actually anti-aliasing since what you're making is closer to rasterizing than texture mapping. You can use any of the anti-aliasing techniques that people have been using for years, such as multisampling. You'll just have to implement them in your fragment shader, and depending on how it's written, could be very tricky.
Anti-aliasing in terms of today's APIs works on geometry edges not on shading.
Think of each pixel as a tiny little frustum instead of a ray.
Yup, that's closer to a sensible approach. du/dv tells you the frequency you're being sampled at... so theoretically you do "multisampling" at higher frequencies and average the results. But I want to know if anyone has smarter/faster ideas than this...
You need to take care of antialiasing in each fragment, which means calling a "smart" antialiasing-aware fragment shader instead of a naive one. Your shader can be less costly than brute-force pure multisampling; for example, it can generalize and abuse texture mipmapping for your specific needs (more lowpass filtering for increasing fragment depth).
The only possible way to get cheaper but lower quality antialiasing is cheating with intermediate frame buffers: scaling up a properly antialiased small buffer (saving a lot of shader invocations) or fixing the ugly results of your original shader with "morphological antialiasing" or similar postprocessing.
