I think I'll just write the system in my comparability layer then using the OS API
Hello, fellow engine creator
I would strongly suggest you to stick to either Qt, or to something like wxWidgets or GTK. Don't listen to those, who saying that 'everything is garbage, write your own OS/GPU API/ UI system', they're only partially right Or you will end up creating your own GUI library, learning OS-level API, and will not learn about game engine programming Anyways...
I have some experience with Qt within application involving graphics and realtime data processing (not 3D though, but hey, it's realtime with threading ) ) and I can say that it has everything you need: a set of widgets of all sorts, threading, UI editor, IDE ( if you prefer ) and even a special widget to which you can give your OpenGL rendering stuff.
Most likely I'm going to stick to Qt with my own OpenGL-driven engine project
And one more advice - it's better for you to learn first the hard part - OpenGL, GLSL, script language integration, software architecture and game engine creation process itself. In my opinion it's preferable over first attempts to create your toolset, level editor or whatever.
How it was mentioned above - grab some windowing library ( SDL2/GLFW ) and go on trying You can even create switchable system with various windowing backends.
Anyways, good luck, will wait for amazing screenshots from you
Try to avoid OS-level API, if you want to keep it cross-platform. Better - not use them at all, or pack 'em carefully with macros invocations or configurable build system.