Let's clean up the code a bit first:
For a square (rectangular) array in general, boost::multi_array is a better idea than std::vector. Also, let's take out the debugging stuff for now (because it's told us as much as we can really figure out from it) - although for future reference, it's better to send debug output to std::cerr instead; that way you don't have to worry about flushing at all, and it actually exists for the purpose of reporting errors and stuff like that. Finally, let's not repeat ourselves; factor out the common sub-expressions for adjacent cell coordinates. (That provides some insurance against typos, too.) Finally, there is no point in having a return value from the function, and writing the initial check as a guard clause probably is cleaner (makes things less indented) even if it doesn't impact the actual algorithm.
void followLand(boost::multi_array<unsigned int, 2>& tmpmap, unsigned int x, unsigned int y, unsigned short group) { if (tmpmap[x][y] != 1) return; tmpmap[x][y] = group; int size = tmpmap.shape()[0]; int next_x = (x + 1) % size; int prev_x = (x + size - 1) % size; int next_y = (y + 1) % size; int prev_y = (y + size - 1) % size; followLand(tmpmap, prev_x, prev_y, group); followLand(tmpmap, prev_x, y, group); followLand(tmpmap, prev_x, next_y, group); followLand(tmpmap, x, prev_y, group); followLand(tmpmap, x, next_y, group); followLand(tmpmap, next_x, prev_y, group); followLand(tmpmap, next_x, y, group); followLand(tmpmap, next_x, next_y, group);}
Then the next step is to convert to iteration with a stack. To do this we'll want to make a temporary structure to represent the data that changes with each currently recursive call, i.e. the coordinate.
void followLand(boost::multi_array<unsigned int, 2>& tmpmap, unsigned int x, unsigned int y, unsigned short group) { int size = tmpmap.shape()[0]; std::vector<std::pair<int, int> > coordinates; coordinates.push_back(std::make_pair(x, y)); while (!coordinates.empty()) { std::pair<int, int> coordinate = coordinates.pop_back(); x = coordinate.first; y = coordinate.second; if (tmpmap[x][y] != 1) { continue; } tmpmap[x][y] = group; int next_x = (x + 1) % size; int prev_x = (x + size - 1) % size; int next_y = (y + 1) % size; int prev_y = (y + size - 1) % size; // These will be processed in the reverse order compared to the // original code, but because we use the vector as a stack, // we'll still process "depth-first". To process // "breadth-first", use queue logic instead. For a queue you // probably don't want to use std::vector, though. Consider // std::deque instead. coordinates.push_back(std::make_pair(prev_x, prev_y)); coordinates.push_back(std::make_pair(prev_x, y)); coordinates.push_back(std::make_pair(prev_x, next_y)); coordinates.push_back(std::make_pair(x, prev_y)); coordinates.push_back(std::make_pair(x, next_y)); coordinates.push_back(std::make_pair(next_x, prev_y)); coordinates.push_back(std::make_pair(next_x, y)); coordinates.push_back(std::make_pair(next_x, next_y)); }}
BTW, why do you pass 'unsigned short group' if your group IDs are allowed to be 'unsigned int' within the actual storage?