Caustics is the effect of light being bent by a surface and then somehow being absorbed by another surface.

That's not true. There are different types of caustics. In a water volume, caustics at the bottom of say, a river, come from (in order of importance):
- refractive caustics, caused by light being focused by refraction upon entering the water volume
- reflective caustics, which happens when light inside the water volume tries to escape the water but reflects back into it instead
- scattering caustics, in which light gets scattered inside the volume due to local changes in density

Here, only refractive caustics are being considered (as they are the most important, the easiest to represent, and the simplest to calculate). In the PDF, yes, the caustic triangle is the triangle upon which incident light refracting through the specular triangle falls, and the caustic volume is the volume subtended between those two triangles (see figure 2) as Luca says. It seems they assume incident light is directional, which makes sense if you consider sufficiently small triangles.