what is the payoff (risk vs reward) for having whatever loot these caves may have in them ??
Since it is your own character who does the questing, death is actually off the table.
Without going too much into details, a player should be awarded for being first in investigating a cave/ruin. The resources are mainly actions turns, army and possibly one's own health.
To elaborate a bit, I'll use the Dragon Caves as an example that already is balanced:
- A player may attack a Dragon with their army. There are then two possible outcomes:
a) The dragon is stronger than the army: in this case, around 20% of the army is decimated, but they steal away with parts of the treasure, at around 80% of the value of the dead warriors. There is a slight chance to get an item or a spell, which is larger if a larger army is used.
b) The army is stronger than the dragon. In this case the dragon will die and a large part of the army will die. The reward is 80-150% of the lost army in terms of gold + a chance for a magical item or spell.
For dragon caves, each "try" will lose you around 4% of your army's value, plus the cost in action turns to make the attempt. Collecting a sufficiently large army will defeat the dragon, but will often cost more than the gold gained. However, there is the chance at a magical item, or that the dragon horde might actually be larger than the armies lost.
(There are other things to do in a dragon cave, but they're not relevant for the discussion)
A ruin/quest is similar: you try to defeat the monster to get an award of a magical item or spell. However, it differs from the dragon in two very significant ways:
a) It is intended that a weak character should be able to win against the weakest monsters
b) The army should not be used for questing.
It's especially b) which makes it difficult, and yet it's also this that makes it different from the dragon cave.