I am working as a software engineer and would say my preferred working is in "rapid prototyping" but I am even creating software that goes into products and get sold.
One big difference is that for a product you "must" create lots of papers because all and everything must be checked, tracked, tested again.
Someone working as hobby developer need not all that things. But should create some simple papers that explain the ideas behind. That helps later to identify problems.

For a prototype it is essential that it runs quickly, after a short period of codeing. For a product it is essential that it works at the end.

If you have a running system and get stuck because your code cannot be changed without a big effort its time to "re-design" parts. Identify the problem and create something that will fit better with the new requirements.

For a prototype the programming language is some kind of important. Because you need to make changes easy. Restructuring should not take weeks but a day. So you will find OO languages convenient because you can introduce new abstractions easy without changing tons of code.

A prototype need not catch all faults and is allowed to crash. Its the job of the prototype to show you what can be done and what not. Sometimes you find that extending the code is a problem but something you need for further development. So change it.

In my spare time I wrote a Web-Server that shall work as a front-end for a game. I started in 2007 and now I am in the fifth iteration of this web server and at any iteration I thought it would be the last. The changes made to the code are going less and less. So I my guess is, that this will be the last iteration to the web-server.

As a matter of interest I use UML tools to structure my work and to let generate the code. This helps on one side because restructureing is only a drag-and-drop action, but it is another thing to learn to work with these tools and so it is not an easy task to create good structure even with UML.

Hope this helps a bit.

[quote name='Nairou' timestamp='1338934199' post='4946594']By comparison, I see people on these forums throwing together prototypes on a matter of days (I don't even understand how you can prototype a game, with all of the underlying framework that is required). People participate in the Ludum Dare competition, creating full games in under 48 hours. And full released games on Steam, like Terraria, which were developed in only 6 months.

I have a feeling that those people have a very, very long history as programmers and/or hobby game developers.

Probably a decade or so, before they had the necessary skills to produce decent games quickly.
[/quote]
I would fall into that group. It took me 5 years to develop a decent amount of skill, at which time I got snapped up by a mid size game studio. It wasn't until after completing a couple of games there that I was capable of competing in 48 hour competition type stuff. As Promit says, a lot of the work setting up a flexible architecture is bypassed, and only the exact parts necessary are written. The trick is, you have to have a pretty clear idea what the full version would look like in order to do that. Also, even when starting from scratch, I've witten and worked in the basic parts of a game enough that I know it by heart, so I'm limited more by my typing speed (which is fast) than my thinking speed for a lot of those 48 hours.

But even when writing at full quality and not on a time limit, it gets faster the more games you make. You'll develop instincts for how to approach new roblems, and a library of knowledge about what worked well and what turned into a mess... and a lot of the time, there is no way to get the game to behave the way you want it to without parts of the code turning into a mess. But you can at least try to minimize it.

In other words, experience is the best, and you're gaining experience right now, so just keep at it But doing a short game or two may be a good idea as well... even if you don't need a lot of the architecture of large projects, you'll still get to experience the end stage of tying it all together and fixing bugs. Starting a new project and whipping up the basics goes fast, the mid stage just drags on and on building and building every day, and the end stage is panic and chaos and fixing and finishing everything that was mostly-done and the light at the end of the tunnel just doesn't seem to be getting any closer. But eventually, the bugs run out, and you're just polishing up little things to make the game extra special, and you ship it whenever you decide it's good enough... because it will never be as good as it could be.

I think one of my problems is that I haven't been able to make the paradigm shift towards OOP yet. I know basic C++, what's a class, inheritance etc but still always think procedurally. I believe it's because I'm very much a bottom-up thinker and was exposed to assembler at the age of 11. I learned C much later and initially with great difficulty as I still thought in assembler and had trouble understanding how even a simple while loop tests the condition at the start of a block but the actual compare and branch in assembler happens at the end. After a while I 'got' structured programming by analysing lots of compiler output. I did something similar to understand classes in C++.

Thanks for all of the responses! It sounds like my development speed is due to lack of experience more than anything else, which I suppose is both good news and bad news.

In thinking about it, I'm starting to wonder if a small part of my problem is also due to poor code organization. I have my code divided into separate files based on what it affects, but this division is very rough (one file for server code, one file for client code, one file for common game logic, etc.). The files tend to be rather large, and I spend a good deal of time hunting around for a particular function I need to edit or refer to. Organizing them into groups of smaller files might make it easier for me to find what I need without getting overwhelmed. I just need to put some thought into how to split them up so they make sense.

Another thought, is that I spend a lot of time making changes to multiple parts of the code base for each small feature. For example, let's say I add the ability to change a unit's color. Sounds simple enough, but what actually happens is I write the code to change the unit color, then create a network command struct to specify unit color changes, then add code to the server to package up and broadcast the unit color change to the clients, then update the client to receive and unpack this command, and finally re-implement the code (or refer to a shared function) to change the unit color locally. Every little synchronized action takes a good bit of time because of all of these steps. I've considered trying to make shared data more modular, so that it can contain its own code for network packing/unpacking and transmit, so that adding a new feature is more self-contained, but I haven't come up with a good idea for doing this yet that doesn't feel limiting.

I appreciate all of the replies, and any further suggestions or ideas are very welcome.

@Karsten_:
My absolut favorite is not an opensource one, ArtisanStudio.
But another one where I am afraid the support will die at some time is OpenAmeos. http://www.openameos.org
Even it is not an UML-2.x tool it has the ability to run your own code generators, so you can create whatever text-based output you want. TeX for documentation would be one.
And it is capable of working with a group on the same database. So everyone in the group can see any changes in real-time. It is available for windows and linux but mixing it does not work well.

For my private projects, the web-server and later the game, I use ArtisanStudio which has the same capabilities as OpenAmeos, but is absolutly up-to-date. So you can share the development in a group and can create any text based output you want.

Pardon for advertising the tools. But UML can help to get projects managed and I am curious why no one is using them. But that is another topic.

With regard to productivity, Patrick deWitt (winner of the 2011 Canadian Governor General's Award for English language fiction for his totally awesome book The Sisters Brothers) gave out a really great tip during the 2011 Scotiabank Giller Prize awards telecast. It was something along the lines of: Turn off your Internet connection at 8am and don't turn it back on until 5pm.

IMHO, that's just genius advice.

One thing to help on re-design/re-structuring is to have some kind of nameing convention. This makes it easier to shift complete modules.
And try to use speaking names for variables.
From ancient times of programming you have i, j, and k for simple counting variables. So i use them only for that purpose.