Suppose we have an area of land, with height of each grid point stored in an array. You can write a function which calculates height at any x,y by interpolating. When you choose a camera angle you can create a matrix which will convert x,y,h(x,y) into screen coordinates. But is there a simple way to convert back the other way? Suppose you want to iterate through the whole screen and calculate the x,y,z using the matrix and the height function, so you can choose a colour for the pixel (based on a tilemap of x,y or height or slope or whatever). You can assume a continuous surface, although there may be steep slopes. This is essentially a mathematical problem - find the (first) intersection between a line and a surface. Point me to another thread or article if necessary, I imagine a lot of people have had to work this out before me.

You''re describing ray-tracing. Just type ''ray tracing'' into a search engine and you should get all the info you need.

it''s very simple

to find and intersection point of a ray (line) and a plane you have to do this :

lets consider a ray : X=A+t*V
where X is any point on the ray , t is a scalar and V is the direction vector of the ray.

now , lets consider a plane : X dot N + D=0
where X is any point on the plane, N is the normal to the plane and D is a scalar unique to the plane

all we have to do is find a point Xi that fits both eqations :
Xi dot N+D=0 , but Xi=A+t*V , so :
(A+t*V) dot N +D=0
A dot N + t*V dot N +D =0
t=-(D+A dot N)/(V dot N )

back to the first equation , replace t and there you go , the intersection point !!!

You gave a solution for a ray and a flat plane. I was talking about a surface described by a 2D array of height values.

I looked around for stuff about raytracing and most of it is about intersections of lines with geometric shapes, which gives you a mathematical solution. But looking at a height map is one which probably requires some iterative method to zoom in on the right point. Any ideas?

What you''re calling ''picking'' in this case is going to end up being ray tracing against polygons generated from the height map.
When you ''picking'' for every pixel on the screen, you are ray tracing or at least ray casting.

You could do that, but it''ll be faster to draw all the triangles with frustrum culling and maybe LOD. Even in a pure software renderer.

It sounds to me that AndyMan is trying to find an intersection between a ray and the surface that is defined by function h(x,y).
You can do this - for some types of h(x,y)

Unfortunately, while the surface may be continuous, it''s derivative is probably not. This is because your function h(x,y) is really selecting from a whole bunch of different plane equations depending on which x and y you''ve got. I think if you want to solve this mathematically in any sane manner, you would need to generate a more complicated surface from your height map that uses one equation and not switching between several distinct equations. Some type of spline function, perhaps.

Look into voxel rendering. Yu''ll get a better idea of how the kind of stuff you''re talking about can be done efficiently.

But, in pure 3d space with no restrictions, what you want to do is raytracing, pure and simple.

There is no heightmap-screen pixel equivalency that can be worked backwards from the screen pixel to the heightmap cell short of raytracing. Why? Well, for one thing, on the screen, some mountains are behind other mountains. You know that rule in math, that in a graph, if a vertical line can intersect more than 1 part of it at any location, it''s not a function? Well, you''re trying to essentially find a function that can be intersected by a vertical line. It can''t be done. Algorithmically, the only equivalency can be found through ray-tracing, and that''s rather brute-force. But, like I said, if you limit some freedom on the part of the player, you can start doing all kinds of cool things, and that''s called voxel rendering.

On a side note, I am positive that the key to efficient terrain rendering lies in the old voxel techniques - specifically wavesurfing. And if adapted, you can get more freedom.