I would use heightmaps. You can edit them with an image editor, or easily in your own editor by manipulating the image data directly. I would suggest you to use 16bit image data because you will have smoother surfaces. You can also generate lightmaps with these heightmap. You dont have to load the whole image at once, you could split it into chunks and then load them individually.

As always, it depends...

As mentioned before caves and cliffs are easier with an actual mesh. However there are some hybrid method to do them with heightmaps.

I would also imagine that it will be easier to deform terrain that is based on a heightmap (e.g. for artillery craters).

However I find heightmaps often suffer from the fixed mesh resolution. E.g. on steep mountain surfaces, if you where using a mesh, you could just add more vertices.

One more thing, if you are building a shooter or another game where the player gets a close view of your terrain, I would use meshes. They can look much better and much less generic. E.g. if the artist is working on it in a skulpting application.

But if you wish to procedurally generate your terrain or just need much of it and don't have time I would recommend height maps.

There is always a third way to do it. For exaple you could use tileable meshes like they can look much better than just height maps but aren't as much work as individual meshes.

For more information on that take a look at Warcraft 3 or Starcraft 2, they used meshes for the cliffs and ramps and also support heightmaps (at least that is what I think they are using) for the rest of the terrain.

Good luck,

Christoph