Remember that your program does not live in isolation, and is not composed of just the code you write (in fact your code is a very small portion of the total). As well as your stuff, your program is also loading lots of DLLs, including your graphics driver, various Windows DLLs, and potentially sound, input and networking drivers too. These all make a contribution to the total memory used. In addition to this it needs to allocate stack space, heap space (every program has a default heap), and so on. Your graphics driver needs to allocate video RAM for it's framebuffer, which in OpenGL is typically backed up by a system RAM copy (the same applies to any textures you create). In other words there's lots of stuff going on behind the scenes which you don't have control over.

So even something like GLQuake (which originally needed to run on a first-gen Pentium with 16 MB RAM) uses about 76 MB of memory on my PC, for example.

Also note that the memory usage displayed in Windows Task Manager is not the actual physical RAM being used.

So don't sweat the details of this. Worry about the memory you explicitly allocate and deallocate in your own code for sure, but expect the OS to be also doing it's own memory allocations behind the scenes.