I'm working on a small 3d engine, and I've been looking at the Unreal editor a bit. In this editor you can Subtract cubes from the world. But if you subtract 2 cubes next to each other, the polygon in between gets a hole in it. An example: How would I do something like this in OpenGL, would it be like splitting the big polygon into several smaller ones, or more like masking part of it?

You have to recalculate the new polygons on the CPU!