Thanks for the replies, in the meantime I've also been searching some more on the internet and it seems that newer version of dwarf fortress which can be run on linux seem to use SDL, so I'll look into SDL, which also would give me a system to handle input more easily, because I was planning to use getch, but it is a bit annoying when it comes to arrow keys and things like that.
Even if you were to use OpenGL directly, you would still want SDL (or something similar, like GLFW) to handle window creation and input in a cross-platform manner. I still recommend libtcod. It uses SDL for input and such. More importantly, it has several features that, if you start from scratch, you would need to implement yourself when you need them, such as a 'console' to draw your ASCII characters to, loading of ASCII fonts and tile sheets, pathfinding & line of sight, and more. You can choose between renderers based on SDL and OpenGL, and can implement any custom rendering you need on top.
Of course, if you want to do it all from scratch, that's fine, but libtcod will save you a good bit of time. I'm not a fan of its API myself, but it's the most complete solution I know that's usable in multiple languages.