Need help finding which tile my cursor is in?
That is a very regular shape so there is probably a perfect mathematical solution (The maths sub forum might be a better bet). I don't have a solution myself but I like your triangle idea, you could limit the amount of triangles you check by partitioning them. Slice the board vertically and horizontally and now you have 4 large triangle areas that are trivial to check against (if x < xSlice then it's one of the left triangles, if y < ySlice then it's one of the top triangles). You could then slice again diagonally.
Slice horizontally (is the cursor left or right of centre), easy check and halves the triangles you need to check.
Slice vertically (is the cursor top or bottom), easy to do and you half the triangles again
Then slice diagonally, halved again.
You are effectively doing a binary space partition.
That said I also feel there is a far better solution involving a single matrix (which transforms a screen space ray into a model space ray). You can then solve that for where it intersects the board plane (e.g. if your boards 'face' is at y = 1 (assuming y is up/down) then you solve for when the ray intersects the x/z plane at y=1. Once you have the x/z of where it is you can work out which square it is in (it might be 0.5/0.75 which is within square 0,0 or 4.7/3.8 which is in square 4, 3 and so on.
I should point out that the transformation matrix will also need to 'unrotate' that board so the squares follow a sensible axis. The model should be defined so it is easy to work with (you probably already have this) and then rotated and projected. You just need a matrix to unproject and unrotate which 'should' be easy to work out (the inverse of the model view projection matrix) but I'm unsure so hopefully someone else will be able to help more.
I guess you already know this seen as you are well on your way but if not try searching 'picking'. You can do it mathematically but also graphically by rendering. I think the mathematic approach would be best for this since it is so regular but if you later decide to add more irregular things to click (chess pieces for example) then you might be better off going straight for the solour/graphical type of picking.
There's generic solution to this.
Imagine your initial board was regular grid with square cells. To get its final "isometric" look, it was rotated and squished.
Now, to get cell coordinate under cursor you have to perform inverse transformation to cursor coordinates.
So, your initial matrix would be "rotate field 45 deg counterclockwise, scale it down 2 times by y"
In opengl-like notation:
[attachment=30068:rotatescale.gif]
Or in final numbers:
[attachment=30069:rotatescale_numbers.gif]
That matrix would transform coordinates from grid to screen space.
To get cursor coordinates back from screen to grid space, you need inverse transformation.
Inverse matrix would be:
[attachment=30070:inverse.gif]
Or in numbers:
[attachment=30071:inverse_numbers.gif]
Now you take cursor coordinates, subtract x/y offset you've applied to field when positioned it on screen, and multiply by that inverse matrix.
float x = cursor_x - field_screen_x;
float y = cursor_y - field_screen_y;
int cell_x = floor( 0.707107*x+1.41421*y)
int cell_y = floor(-0.707107*x+1.41421*y)
Those cell_x/cell_y is your final answer.
Note: cursor coordinates and screen coordinates origin assumed to be in left-bottom corner.
You can also treat it as just normal squares, and use an additional mask image. First, find the tile as normal, then use the mask image to determine if you need to offset you position.
This technique will also work with other tile shapes.
See this for more detail (described near the bottom).
You can also treat it as just normal squares, and use an additional mask image. First, find the tile as normal, then use the mask image to determine if you need to offset you position.
This technique will also work with other tile shapes.
See this for more detail (described near the bottom).
This is pretty much what I ended up going with. It works beatifically, the code is understandable, and it takes just as long to run the code if I have 10x10 or 10,000x10,000 tiles.