From my artistic point of view, using HSB is much more reliable.

If the saturation is really low, it means the only component to take in account to check the difference is the brightness.

If the saturation is medion-high, then you check based on HUE.

If the saturation is medion-high and the HUE is really close for two colors, then you check for the brightness.

RGB is a poor measure of color (in artistic terms). Good to represent color on monitors, poor in actual biological meaning (perception).

By HSB you mean HSI? (I'm not extremelly familiar with the terminology) Also, HSI=HSL=HSB (just called differently)?

Some of these means the same thing, some don't.

The two most common are HSL (also called HLS) and HSV (also called HSB). There's the third HSI (which you mentioned), but I rarely see it around.

I have a function to convert RGB-HSB/HSB-RGB here:

https://github.com/felipefs/libpixpaint/blob/master/pixpaint.c

Well, brightness probably would be always similar (artistic wise, since those represent same type of thing (country) then it should have similar brightness (so fro example I can safely use black text over it becuse I would assume the foreground is light)). So, I think I should check against HUE and SATURATION mostly/only?

If you are going to use black text, then yeah. HUE and saturation are the ones you will check mostly.

OK, HUE would be easy, if less than 0.25 difference in hue then assume it's "too similar. But how (and if I should) throw SATURATION to the mix as well? Like if same HUE could look different if SATURATION differs greatly (just guessing, I know only basics of colour theory)?

If the two countries have colors with low saturation (S), then both are gray-ish, and they are the same color (you don't even need to check HUE). Else, if saturation is medium-high, then check HUE.

Better to use LAB color space, which has been designed for color comparisons (hence its wide use in computer vision). In LAB space, the euclidean distance corresponds closely to the perceptual color difference.

Here is the conversion to/from LAB and sRGB:

/// Convert the gamma from sRGB to linear RGB space.
template < typename T >
T gamma_sRGB_RGB( T x )
{
	return (x <= T(0.0404482362771076)) ? (x / T(12.92)) : math::pow( (x + T(0.055))/T(1.055), T(2.4) );
}

/// Convert the gamma from linear RGB to sRGB space.
template < typename T >
T gamma_RGB_sRGB( T x )
{
	return (x <= T(0.0031306684425005883)) ? (x * T(12.92)) : (T(1.055)*math::pow( x, T(1)/T(2.4) ) - T(0.055));
}

template < typename T >
T LAB_f( T x )
{
	return (x > T(8.85645167903563082e-3)) ? math::pow( x, T(1)/T(3) ) : (T(841)/T(108))*x + T(4)/T(29);
}

template < typename T >
T LAB_f_inverse( T x )
{
	return (x > T(0.206896551724137931)) ? (x*x*x) : (T(108)/T(841))*(x - T(4)/T(29));
}

/// Convert from the sRGB color space to the linear RGB color space.
template < typename T >
void convert_sRGB_RGB( const T srgb[3], T rgb[3] )
{
	rgb[0] = gamma_sRGB_RGB(srgb[0]);
	rgb[1] = gamma_sRGB_RGB(srgb[1]);
	rgb[2] = gamma_sRGB_RGB(srgb[2]);
}

/// Convert from the linear RGB color space to the sRGB color space.
template < typename T >
void convert_RGB_sRGB( const T rgb[3], T srgb[3] )
{
	srgb[0] = gamma_RGB_sRGB(rgb[0]);
	srgb[1] = gamma_RGB_sRGB(rgb[1]);
	srgb[2] = gamma_RGB_sRGB(rgb[2]);
}

/// Convert from the linear RGB color space to the XYZ color space.
template < typename T >
void convert_RGB_XYZ( const T rgb[3], T xyz[3] )
{
	xyz[0] = T(0.4123955889674142161)*rgb[0] + T(0.3575834307637148171)*rgb[1] + T(0.1804926473817015735)*rgb[2];
	xyz[1] = T(0.2125862307855955516)*rgb[0] + T(0.7151703037034108499)*rgb[1] + T(0.07220049864333622685)*rgb[2];
	xyz[2] = T(0.01929721549174694484)*rgb[0] + T(0.1191838645808485318)*rgb[1] + T(0.9504971251315797660)*rgb[2];
}

/// Convert from the XYZ color space to the linear RGB color space.
template < typename T >
void convert_XYZ_RGB( const T xyz[3], T rgb[3] )
{
	rgb[0] = T(3.2406)*xyz[0] + T(-1.5372)*xyz[1] + T(-0.4986)*xyz[2];
	rgb[1] = T(-0.9689)*xyz[0] + T(1.8758)*xyz[1] + T(0.0415)*xyz[2];
	rgb[2] = T(0.0557)*xyz[0] + T(-0.2040)*xyz[1] + T(1.0570)*xyz[2];
	
	// Ensure positive numbers.
	const T minRGB = math::min( rgb[0], math::min( rgb[1], rgb[2] ) );
	
	if ( minRGB < T(0) )
	{
		rgb[0] -= minRGB;
		rgb[1] -= minRGB;
		rgb[2] -= minRGB;
	}
}

/// Convert from the XYZ color space to the CIE LAB color space.
template < typename T >
void convert_XYZ_LAB( const T xyz[3], T lab[3], const VectorND<T,3>& whitePoint = WHITE_POINT_D65 )
{
	// Divide by XYZ color of the D65 white point and apply the LAB f function.
	T temp0 = LAB_f(xyz[0] / whitePoint[0]);
	T temp1 = LAB_f(xyz[1] / whitePoint[1]);
	T temp2 = LAB_f(xyz[2] / whitePoint[2]);
	lab[0] = T(116)*temp1 - T(16);
	lab[1] = T(500)*(temp0 - temp1);
	lab[2] = T(200)*(temp1 - temp2);
}

/// Convert from the CIE LAB color space to the XYZ color space.
template < typename T >
void convert_LAB_XYZ( const T lab[3], T xyz[3], const VectorND<T,3>& whitePoint = WHITE_POINT_D65 )
{
	T y = (lab[0] + T(16))/T(116);
	T x = y + lab[1]/T(500);
	T z = y - lab[2]/T(200);
	xyz[0] = LAB_f_inverse(x)*whitePoint[0];
	xyz[1] = LAB_f_inverse(y)*whitePoint[1];
	xyz[2] = LAB_f_inverse(z)*whitePoint[2];
}

/// Convert from the linear RGB color space to the CIE LAB color space.
template < typename T >
void convert_RGB_LAB( const T rgb[3], T lab[3], const VectorND<T,3>& whitePoint = WHITE_POINT_D65 )
{
	T xyz[3];
	convert_RGB_XYZ( rgb, xyz );
	convert_XYZ_LAB( xyz, lab, whitePoint );
}

/// Convert from the CIE LAB color space to the linear RGB color space.
template < typename T >
void convert_LAB_RGB( const T lab[3], T rgb[3], const VectorND<T,3>& whitePoint = WHITE_POINT_D65 )
{
	T xyz[3];
	convert_LAB_XYZ( lab, xyz, whitePoint );
	convert_XYZ_RGB( xyz, rgb );
}

/// Convert from the sRGB color space to the CIE LAB color space.
template < typename T >
void convert_sRGB_LAB( const T srgb[3], T lab[3], const VectorND<T,3>& whitePoint = WHITE_POINT_D65 )
{
	T rgb[3];
	convert_sRGB_RGB( srgb, rgb );
	convert_RGB_LAB( rgb, lab, whitePoint );
}

/// Convert from the CIE LAB color space to the sRGB color space.
template < typename T >
void convert_LAB_sRGB( const T lab[3], T srgb[3], const VectorND<T,3>& whitePoint = WHITE_POINT_D65 )
{
	T xyz[3];
	T rgb[3];
	convert_LAB_XYZ( lab, xyz, whitePoint );
	convert_XYZ_RGB( xyz, rgb );
	convert_RGB_sRGB( rgb, srgb );
}

From an artistic perspective, RGB is possibly the worst color scheme out there. You're far better off comparing in practically any other color space. If you wanted to be super accurate about it, you'd want to find a color space that better represents the colors people can actually see. This is outside my expertise, but I know that people can generally see more shades of green than any other, for instance, so you'd want to put less weight on that than other colors.

Better to use LAB color space

This, most definitively. Not only was it designed for the purpose of being "perceptual", it also has the immense advantage of automatically solving the problem of color weakness/blindness for you. This is otherwise an extremely hard problem.

Deuteranomaly? Deuteranopia? No problem because red and green are on the opposite sides of the a axis. Protanomaly or protanopia? No problem because, you guessed it, blue and yellow are on the opposite sides of the b axis. For any colors to be used on adjacent areas, choose a pair of colors that are significantly different on these (or alternatively have a significantly different L), and you have both deficiencies covered. Try doing the same in RGB, it's a sheer unsolvable nightmare.

Boring answer: since any 2D planar map can be uniquely painted using only four colors, all you have to do is manually choose four you like!

It's that kind of thinking that led to CGA's black/white/cyan/magenta, what is sometimes called the worst colour palette ever inflicted on mankind.

Oh I don't know, I quite like the old black/white/cyan/magenta look of things... but then again I'm pretty weird. And in terms of Color I would agree that using LAB offers probably the best (or one of at least) ways to do color comparisons