below is the portion of a recent iso demo that i made that deals with mapping mouse coordinates to tile positions. it uses a diagonal map (aka diamond shaped, the type used in SimCity2k, AoE, etc.)
//get mouse coordinates
int xPos = LOWORD(lparam); // horizontal position of cursor
int yPos = HIWORD(lparam); // vertical position of cursor
int fwKeys = wparam; // key flags
//adjust xpos to reference off of tile 0,0
//calc mousemap row and column
//calc mm row and col remainders
//adjust MM coordinates if a remainder is negative
//calc cursor position
//find color in mousemap
//make certain that cursor is in bounds
//force a repaint
in this code example, the map uses tile coordinates where tile (0,0) is the topmost tile of the map. x increases to the southeast, and y increases to the southwest.
this example also has the client area of the window the same size as the complete area of the tilemap, so there is no screen coordinate to world coordinate transformation, and in most cases, there would be.
so anyway, the example goes through the following steps:
1. Translate world coordinates to make (0,0) reference the upperleft corner of Tile(0,0)
to do this, we subtract (MAPROWS-1)*TILEWIDTH/2, because this sum is added when we transform tile coordinates into world coordinates
2. Determine MouseMap Column, Row, and Column and Row remainders.
2A. Fix any negative remainders
due to the way integer division works, we have to make sure that the remainder is a positive value, where 0<=MMCRso:
3. calculate rough tile coordinates based on MMC and MMR
//clear the coordinates out
now, for each mousemap column to the east, add 1 to x and subtract 1 from y, to the west, subtract 1 from x and add 1 to y
for each mousemaprow to the north, subtract 1 from x and subtract 1 from y, to the south, add 1 to each of x and y
4. we tweak this rough tile position to get the actual tile position, using the mousemap
since i store the mousemap as a bitmap on a system memory DC, we get the pixels color at (MMCR,MMRR)
next, adjust the TileX, TileY based on the value of crTest.
i made a few constants to specify certain colors found on the mousemap, to make lookup easier.
MM_CE: no adjustment
so, thats the method i use.
for fixed ratio tiles, as mine are(64x32), an alternate way is to use Jim's method, which is optimized for tiles with a ratio of 2:1.
if you want to get my isometric example program (the one i mention in this post is Isometric Example #5), the source code and workspace for VC++6, just email me for the time being. soon, these examples, workspaces, and articles about them will be published here on gamedev, so stay tuned.