I have a sphere formed of spherical triangles. Each triangle is determined by 3 normalized vectors:

I need to determine which triangles (spherical projcetions) I can see if I know: camera position, view normal vector and fov 45.0 deg. The problem is if the camera is very close to the sphere, the spherical projection of a triangle may be visible and its flat version may not and vice versa and I can't cast it to somethink simple.

Here is an example:

As the view is limited with fov, actual view area is not a rectangle , but a circle around the view rectangle.

In this example I need to determane if the orange area is visible:

So, I think I should somehow find a point in a triangle, which angle to the center of the sphere comes close to the view angle.

Ashaman73, it only determines possible visibility, without checking fov area intersection. I can't see all the triangles in front of the horizon because of the fov limitation (for example - a cant see triangles behind the camera).

It's possible that I don't see triangle vertices, but I see the triangle.

Lets assume the camera is looking at the sphere's center. You should be able to construct a plane such that it passes through the sphere's center, with the normal being the camera's forward view vector. This plane passes through the horizon circle that lays upon the horizon of the camera's view on the sphere.

Then you can take the triangles and transform them into the space of the sphere, such that the sphere center is the origin. Now you can treat your triangle vertices as vectors, and see if any of the triangle penetrates the plane we constructed earlier. This can be achieved by dotting the vertices with the plane's normal and checking the sign.

int Visible( v3 N, v3 a, v3 b, v3 c )
{
    float t0 = Dot( N, a );
    float t1 = Dot( N, b );
    float t2 = Dot( N, c );

    return t0 < 0 || t1 < 0 || t2 < 0;
}

This all assumes the camera is infinitely far away from the planet. Since your camera can be closer to the planet this becomes a matter of adjusting the plane along the negative direction of the camera's view vector. I'm not sure how to do this, but perhaps I've given enough information to figure the rest out

I think what you need is to find the colsest point on the spherical triangle to a plane.
Then you can test against frustum planes for culling and the sphere for self occlusion.

For the self occlusion, Randys answer assumes orthographic projection, this leads to large error if we ar close to the sphere.
To take perspective into account, we can use the solid angle of the sphere:

vec ray = (sphereCenter - camera).Unit();
float dist = ray.Length();
float solidAngle = asin (sphereRadius / dist);

From that, we can get a circle on the sphere, everything behind the plane of this circle is occluded by the sphere:

vec circlePosition = ray * cos(solidAngle) * cos(solidAngle) * dist;
vec circleNormal = ray;
float circleRadius = sin(solidAngle) * cos(solidAngle) * dist; // don't need this




Now, my vague idea for colsest point spherical triangle to plane:

First, calculate the closest point of the sphere to the plane.
Calculate 3 planes for each arc edge (by its 2 vertices and sphere center - the arc is guaranteed to lie on the plane, yes?)
Classify the closest sphere point by checking in front or back of each plane.
If it is inside all three, the point is already in the triangle, so we can do the final plane test by the closest point and the 3 vertices.

If not, the point is outside and need to move it closest to the triangle.
With a planar triangle, you could easily detect on the classification result, if the point belongs to a vertex or edge region.
If in edge region, find closest point on edge and we're done. I assume projecting this back to sphere is accurate and no closest point on arc is necessary.

I think th entire method works on the sphere too, but there may be special cases when point is on the other side of the sphere than the triangle - my imagination end's here
But it should be easy to figere out during developement.