Let us say that this is our rendered map. If we make a mask for each layer, with a different colour we get this (blended witht the original for clarity):
So now, lets say that we want to render a sprite, we just look at the mask and cut out accordingly... If we are rendereing the sprite to the OrangeY, then we need to cut out of the sprite the blue and the red. So now I have to go do some research on how to do this.... hmmmm...
Now making one operate is probably a whole lot harder... but I could use some fun!