The simplest algorithm will work:

1) Send packets

2) Receive responses

3) Measure timing

4) Calculate RTT and variance

yeah, I already measured both of these things and now I thought I could combine them into 1 value and judging by it I could choose the best host, hmm? But how to combine those 2 values into one? Add them? In what ratio, etc? Or maybe that's not a good approach?

There is no "one size fits all" here. Adding them with some kind of weighting factors would probably work okay. If you really worry about jitter, you might weight the standard deviation by a factor 20 compared to the RTT, and choose the smallest value. If you don't worry as much about jitter, you might add standard deviation by a factor of 1.

For what it's worth, if the jitter is Gaussian distributed (it may or may not be,) adding one standard deviation gets you to about 65% of the curve; two standard deviations 95%, and three standard deviations to 99.7%, IIRC.

And Standard Deviation is the square root of variance, and a better unit to compare to the actual average, because it has the same unitless cardinality as the average -- variance is the square, by contrast.

So, to start somewhere, I'd calculate "RTT + 2 * SDEV" and pick the host that has the smallest value.