This is called boolean operations.

I can't point you towards any specific articles, but I can give you some guidance on how I'd approach this conceptually...

Rule #1: Try to understand the simplest form of the solution. We can always make things more complicated later.

Abiding by this rule, let's try to subtract a volume from a 2D rectangle using another 2D rectangle. If you have a rect with the lower left oriented at 0,0 with a height & width of 2, and then subtract a 1x1 rectangle from it, can we figure out how to do it on paper? What result should we try to get? Do we want a polygon, or do we want to create a bunch of rects? What's the easiest to work with? What if we subtract the 1x1 rect from an arbitrary location somewhere on or around the 2x2 rect?

I think ideally, we'd want to recreate the geometry by rebuilding it with new verticies. The new rect would certainly contain the verticies which are contained within its bounding area. We'd also be interested in the lines of the verticies which are within the bounding area since the new region will have to place its adjusted verts on that line.
Example:
Original rect verticies: {(0,0),(0,2),(2,2),(2,0)}
Subtraction rect verts: {(1,1),(1,3),(3,3),(3,1)}

Result should contain five verts:
{(0,0),(0,2),(1,2),(1,1),(2,1),(2,0)}
Note: (1,1) was the bounding vert but it was connected to (1,3) and (3,1). So, how do we get the resulting verts (1,2) and (2,1)? We do a line intersection test! Since we know that there is a vert contained within another vert, we MUST have intersecting lines. Although this is a math & physics forum, I'm going to leave the line intersection test mathematics out -- it's pretty simple; start with the "y = mx + b" equations and don't forget to account for floating point precision errors!!!!!!! Set a minimum epsilon and use testing to adjust its accuracy. Test this thoroughly since you don't want floating point precision errors to randomly creep up on you. (A more scientific approach would be to know exactly how large your epsilon tolerance needs to be for different parts of the float scale)

Anyways, once you've got this solved for integers in two dimensional space, you can add additional complexity by adding in additional verticies at arbitrary positions, the z-coordinate, and eventually rebuilding complex meshes (such as rebuilding a section of terrain to simulate a crater after an artillery shell impacted it).

If you get stuck on something, back up a step to its more simple form and make sure that its working as expected.

On a slightly more abstract/philosophical note, you may be able to avoid doing any of this all together if you can find a clever way to eliminate this requirement from your game play or simulate its effect (some games will use precalculated shattered meshes to simulate something like wood splintering). If this is vital to your game play (such as with mining out a tunnel), you may be able to get away with carving out blocks (like minecraft or terraria) instead of recreating meshes. Anyway, its something to keep in mind. If you spend an hour trying to think of a clever solution, it may be more cost effective than spending ten hours or more implementing a less clever solution.