The short answer: Yes, it's possible.
The explanation: You need a way to (comfortably) get back the data from the GPU to the CPU - if you can rely on modern APIs, that usually means a buffer, like a shader storage buffer in OpenGL, where you can write to from all of your shaders (geometry included, I think). Or transform feedback, but nowadays it's kind of outdated if you could use ssbos.
The question is: Does it make sense? The pipeline needs some sort of.... synchronistation between GPU and CPU. Not to talk about that you have to transfer your "modeling actions" somehow to the GPU. And if one action affects multiple vertices at once, things start to get complicated. There's a concept called "pinned memory" (OpenGL world calls it persistent mapped buffers), where you represent your vertex buffers in persistent memory and do synchronization by yourself. With this concept, you could easily work on your native buffers on the CPU side and synchronize before you draw - easiest, probably fastest way I could imagine.