I wouldn't really recommend it, but it is possible, yes. Even so you do need at least one VAO, so "entirely optional" isn't entirely true.
Well id do it, Valve do it, so it seems (at least on the surface) to be viable. Despite that, I do find VAOs useful as a way to avoid having to track lots of state, but I'm still looking forward to GL_ARB_vertex_attrib_binding being more widely available.
It is obviously doable/viable but why track all that state yourself, when the API can do it for you (which may have the side effect of being more efficient)?