This is kind of yet another "which engine" post (so recommendations welcome), but not quite.

I'm particularly interested in hearing what people have to say about productivity of modern commercial engines (ex. Unreal 4) vs that of popular open source engines for a single developer. That alone may be enough to rule out most or all commercial stuff, but maybe I'm wrong!

I've always assumed that a commercial engine would require a ton of "bootstrapping", have a huge learning curve, would assume your requirements are all AAA (thus making my more trivial requirements actually harder, vs a simpler engine designed for the little guys), and that it'd be completely infeasible for a single dev to ever use. But I've decided to re-visit that assumption.

A few constraints to be aware of:
-Not particularly interested in toolset-oriented engines (ex. Unity?)

-C++ should be a first class citizen - I don't want to spend my time learning tools, I'd rather spend my time learning a well documented, high quality API where I can stay in code world as much as possible.

-"Tools" like static scene or level builders are okay I guess, but probably not very helpful since I'm interested in space-sim type projects centered around travel, "building" your own scenes/bases, and some procedural content.
-Linux support (at least as a runtime platform)

Edit:

I should probably state that by "commercial" I mean proprietary, AAA capable, corporate backed, and so on. I guess that commercial/noncomm line is starting to blur a bit lately...

If I did try a commercial engine, UE4 is the one I'm probably most interested in. I am concerned that it might be too constraining to always have to do it the "engine's way", as I tend to be interested in building stuff which doesn't usually fit into the typical cookie cutter pattern, but that issue seems to crop up even with smaller engines. Only way to know is to try it out, I guess.

I find that with the open source engines it's easy to do basic things, when you start getting deeper into development and the closer you get to releasing the game the harder it gets to do things.

I'd say I've noticed the same.

Does anyone know how UE4 is with on the fly mesh generation and low level tasks like this? Ex. manually filling vertex buffers, texture coords, and so on. (Not for everything, just for say, terrain.) Or is that kind of thing way too low level for an engine like this?

I find that with the open source engines it's easy to do basic things, when you start getting deeper into development and the closer you get to releasing the game the harder it gets to do things.

I would say my experience is similar.

With UE4 I have found that the massive community surrounding the engine helps to find solutions to problems of every size quickly, and in the rare cases (which I have had) that there isn't already a solution posted the developers scan the forums and make posts themselves. This fact is boosted by an actual company looking to make money from the product and can only do so if you make money, provides a great system of aid.

Hope this helps.

Honestly, I can't claim that I've used a variety of engines, especially when you get in-depth enough to know how well they work out for a full and complete project. But that being said, I've pretty much only worked solo, so I can make some comments about that.

Using UE4 is, I think, completely viable for a one-man operation. First off, you don't *have* pursue a AAA visual design. As a solo developer myself, I have to honestly look at what I can produce by myself. What kind of code can I write? What kind of of art can I produce? Levels, sound, and so on down the line.
And while I don't have the art skill to make something that looks photo-realistic, I can still create something that looks stylized to a uniform direction. And while I can't make extensive ground-breaking changes in my code, I can follow examples and make relatively simple games.
And you know what? I can do both of those with UE4. Their material system makes it MUCH easier to build something to a cartoony style or a painted style or etc, as opposed to having to actually write shaders myself. And their blueprints system makes it easier to create basic programming changes; I can do a lot without having to actually compile code. And beyond that, I have a large community that I can turn to when I get stuck and get help on their forums.
Epic does these showcases every so often, and they include a number of projects that don't look like some AAA team of 30-40 people, but just talented hobbyists doing stuff at home. They don't get as much attention, but they are still there.

But I'm a bit confused as to why you would reject Unity for being a "toolset orientated engine" but not Unreal for the same reason. Perhaps I'm just not sure as to what you mean with that term, but they both come with their own interface and their own set of tools that you have to use to produce anything. And honestly, I'm not sure why you would want to avoid that. All development requires tools. All games are going to require you to use a variety of tools to create the game. And personally I would much rather be using Unity to get a significant starting point in what I'm doing plus have my code easy to write and implement. Being able to tweak variables in the game and instantly see the changed result without spending hours recompiling is a godsend.

Since the main purpose here was to determine if using something the size of UE4 was feasible for a one person team, and nobody has claimed otherwise here, I think I have my answer. Thanks

But I'm a bit confused as to why you would reject Unity for being a "toolset orientated engine" but not Unreal for the same reason.

Possibly just ignorance of these large engines (that's why I'm asking), but I was under the impression that Unity was more toolset oriented relatively. The real issue for me with Unity is that I'd rather work in C++.

Being able to tweak variables in the game and instantly see the changed result without spending hours recompiling is a godsend.

I wonder if this is this a big issue with UE4? I know how valuable that is, since I'm used to it.

Possibly just ignorance of these large engines (that's why I'm asking), but I was under the impression that Unity was more toolset oriented relatively. The real issue for me with Unity is that I'd rather work in C++.

Oh. Well in that case, yeah, Unity doesn't give you access to the source code.
You get to write scripts in C#, but you don't have full source control.
That being said, I never felt like I needed full source control; I have no need to re-write how the engine handles libraries or rendering control. I had to learn a few Unity-specific functions that I call within my scripts, but I never felt like there was anything I wanted to do that I couldn't in Unity. Except for creating custom shaders, which I could have but I would have had to learn a whole new language for that.
Oh yeah, and I believe you can write new dll's for the engine to use, I started to look into that while back but didn't get very far.

With that in mind, Unity and Unreal really give you the same experience. It's just that Unreal gives you an option for more control if you need it.
When describing the differences between Unity and Unreal to a layman, I said that if you are create a small and simple game, like most mobile games these days, it is easier in Unity, but you want something with a lot of content, it is easier in Unreal.

I wonder if this is this a big issue with UE4? I know how valuable that is, since I'm used to it.

Unreal 4 give you two significant options for adjusting your code. One, if you have only made simple changes (like no new variables I think) you can actually recompile the code while the game runs, and it will begin executing the new code as soon as it finishes. The game never drops or stops, it just recompiles in the background.
The other option they give you is to be able to adjust variables via console commands; you have make some functions be able to be called via console commands so you can easily set up your code to have certain functions run different versions or tweak variables on the fly. (Actually I have personally never used this in UE4 yet, but I assume it is still there.)

By contrast, in Unity any public variable in your script can have its value changed within the editor while the game is running (or before the game runs.)