VReality

Logging is the classic example of the thing you make that everything needs access to, because it's the only example (unless you're re-implementing other environmental features, like the memory manager, etc.)

Personally, I like to treat it specifically as an exception to the rule, because I don't see it so much as part of your application, but instead it's a part of the environment in which you're building your application.

I don't know how Hodgman manages to be right every single time, but once again, Hodgman's right.  Logging is the exception because it's diagnostic scaffolding which is unrelated to our code's actual functionality.

But even so, it's possible to come up with a reasonable design for it.

Our design goals might be:
- Support multiple logging channels, locally configurable to be sent to various destinations.
- No logger object needed at point of logging (i.e. no passing loggers to every single function).
- No globally accessible state.

class Logger
{
  public: // "Object part"
	void SendChannelToFile			  (string const & Channel, string const & FileName);
	void SendChannelToDebugConsole	  (string const & Channel);
	void SendChannelToOnScreenConsole (string const & Channel);
	...

  public: // "Global part"
	static void Log(string const & Channel, string const & Message);
  ...
};

The basic idea is that Logger::Log() has some sort of safe default functionality (like ignoring all log messages), but if a Logger object exists, then the function uses that object's configuration information to route any configured channels.

Note that the globally accessible Logger::Log() function neither sets nor gets state.

The appropriate part of the codebase can create a Logger object, and load a local configuration file to configure logging channels according to the local user's current preference.  Setup, access, lifetime, and cleanup of the object and its configuration data are all managed as with any normal object.  The Logger class can prevent more than one instance from being instantiated, to avoid mishaps.

The Logger object's internal state does remotely effect the behavior of Logger::Log(), but then Logger::Log() is generally designed to effect the environment outside of the application (like printf() and std::clog() ).

And yes, the connection between the global and object parts would be accomplished through some static pointer, and Logger::Log() may need to be thread-proofed.  But again, this system is the exception, and all of the above functions can be conditionally compiled to no-ops in published versions.

I agree that this issue can be addressed with component based design.

I currently favor a design in which components are those parts of entities which are associated with specific engine systems ("services").  They're created (and processed) by the engine systems themselves.  And entities are collections of components.

The components generally need to know about a couple of other requisite components types, and often don't even need to know about their own engine system, since the system knows about them.

Note that this still leaves us with entity constructors which know about components, and the engine systems which create them.

This can be resolved by designing an entity factory, with which each engine system registers itself (rather than one which already knows about them), that does data driven entity construction (is given a list of component type IDs and uses the registered systems to generate an entity containing instances of those components).

That should eliminate the need to recompile code for non-dependent entity types when adding new systems.  Although I must admit, I'm not sure how much this saves you in terms of re-compiling.  On the one hand, there will still be some high level code (AI) which will need to know about a lot of the systems/component types.  And on the other hand, adding systems doesn't seem like something which should be happening often enough for this to be an annoyance.

Here's a little meta-discussion for you.

Ideas are fun, we all have them. I'm not surprised that GameDev is full of people that are just kicking around ideas;  I am surprised ,however, over the constructive criticism and general helpful attitude. Kudos to you, gamedev community! While I'm not going to share my personal ideas...

I don't know if you think your ideas aren't worth discussing, or if you consider them potentially too valuable to share, so this is not necessarily about you...  But I think people in general tend to heavily over-value ideas.  

This applies to every industry.  Apple didn't invent the computer GUI, or the mouse, or the personal-digital music player, or the cell-phone combo-device, or the tablet computer, or the minimalist interface, or mobile web-browsing, or online commerce, or music as a digital commodity, etc.  What Apple did was execute better on those ideas than others who had them.

Facebook didn't invent the social network, and Microsoft didn't invent DOS or the windowed OS. Id didn't invent the FPS, Valve didn't invent the story driven FPS, and Blizzard didn't invent the RTS or the MMO.

A high-functioning, efficient game studio can produce a far superior game from a mediocre idea than the average studio can produce from a better idea.

People seem to think that the right idea will produce success, or that sharing good ideas is equivalent to giving away wealth.  Don't get me wrong.  Ideas are important.  But this industry is full of smart people, and as mentioned they all have ideas.  Success depends far, far less on the quality of the ideas one produces on their own than on their ability to execute effectively on the ideas available.  An idea held by someone not in the position to execute on it has almost zero potential value.  And someone who can execute on it derives far more value from that ability than from any one of the world of ideas they might choose to use.

The bottom line is that practically speaking, we loose effectively nothing by sharing ideas, even great ideas.

Microsoft didn't use their performance counter in implementing C++11's high resolution clock.  As a result it's not really high resolution and is probably an order of magnitude or so low for your frame timer.  (See the bug report.  They intend to roll out a fix in their next release.)

You can test a clock's resolution by getting a time from it repeatedly until it returns a different time.  Then you can decide for yourself if its resolution is sufficient for your purpose.

It's true that there's no need to get bogged down in terminology.  But sometimes the best way to avoid that is to get answers to your questions.  

Library
The term "Library" refers to a specific way functionality can be packaged for use by the application programmer.

Specifically, in a compiled language like C++, a set of functionality is compiled to object code (i.e., "link-able" .obj files), and made available in an "archive" or "library", which on windows is a .lib file (.lib files essentially contain a bunch of .obj files).  The application programmer generally "includes" provided header files which declare functions and types implemented in the library, uses them in code, and links the library into the executable.

The term "library" says nothing about what type of functionality is provided, or what it might be used for.

API
As mentioned, API means "Application Programming Interface".  "API" generally refers to a library, but is named for the interface (i.e., the contents of accompanying header files) to which the application code will be written, rather than the implementation in the library.

The phrase, "Application Programming" implies that the functionality provided is specific for creating some certain type of application.  The Windows API is used for making Windows applications; The OpenGL ("Open Graphics Library") API is used for making graphical applications; Etc.  APIs often deal with how the application interacts with the outside world - how it gets input and/or generates output.  

In general, an API is a library but a library is not necessarily an API.  The C++ standard library, for example, is mostly functionality for a program's internal processing.  It's not designed for giving an application access to, or control over, any specific platform/environment.

Engine
A game engine is a unified set of run-time software technology designed to provide a good portion of the core functionality of a game.  The thing that makes it an engine, is its frame-based multi-tasking nature. The thing that makes it a game engine is all the basic functionality it delivers that makes it useful for building games.  (OpenGL and Direct 3D are not engines, for example.  They provide nothing but a way to get graphics on screen.)

While not technically part of the engine itself, most engines would be useless without a set of accompanying tools for generating content specific to that engine, so a tool-set is sometimes thought of as part of the engine.

The term "engine" says nothing about how the functionality is delivered.  It could, for example, be delivered as a set of libraries, or as source code for a sample game "stub".  If it's a set of libraries, they could be considered an API for developing applications on that engine.

OpenGL
The "Open Graphics Library" is a cross-platform API for putting graphics on-screen.

Providers of all sorts of different platforms, that have graphics rendering ablilty, can implement the library for their platform.  All implementations of the library work with the published OpenGL interface.  That way, applications programmers can write applications using that interface, and they'll build and work on various different platforms, without alteration.

Direct X
Direct X is a Microsoft API for developing "multi-media" applications on Microsoft platforms.  In addition to Direct 3D (their graphics API), it includes other components like Direct Sound and Direct Input.

SDL & SFML
SDL & SFML are video games APIs.

They provide facilities for such things as audio output, input acquisition, networking, threading, timing, etc.  They work with, or along-side, OpenGL.

Unity
I don't really know much about Unity, but in general, any robust game engine will include all of the functionality provided by the graphics and video games APIs, and more (e.g., physics, collision, AI, game objects/entities, message passing, asset handling, load/save, GUI/menus, etc.).

No.  It's not.

I too have only worked on two cancelled projects in sixteen years.  But the majority of projects that are completed in this industry don't end up achieving much commercial or artistic success.  The majority of start-up studios fail within a handful of years.  And the large publishers routinely suffer large scale lay-offs.

That's life.  And it has nothing to do with me.

What did you wish you had known before getting into the industry?

I consider the first project I worked on to be a technical and creative achievement.  But a new console was released shortly before we finished.  It did alright in Japan, but no one has heard of it in the USA.  I consider the second project I worked on to be a technical and creative achievement.  But the publisher semi-collapsed near the end of the project and it got no marketing.  It was appreciated by the few who played it, but it was few indeed.

That studio closed, and I got hired by the biggest publisher in the industry (at that time).  I did great work on projects that made money.  But they were only an advertizing success.  Management and production were abysmal, the games weren't very good, half the studio got laid off, and development staff brought a massive class-action suit against the company.  Lawyers got rich.

The next studio I went to had proven themselves technically competent and reliable by picking up port projects that other studios didn't want.  They were working on graduating to actual development.  When the economy put the squeeze on the industry, their projects dried up.  They are no more.

I went to a new studio being launched by a large publisher, with a producer stolen from a competitor's successful franchise.  But the first-time studio management was way out of their depth and ultimately failed to produce anything.  The publisher pulled the plug and closed the studio.

I went to another start-up which was picking up contracting gigs, solving other studios' technical problems, with the intention of developing a couple large scale game concepts and landing publishing deals.  But the designer turned out to be a fraud, and the publishers wouldn't touch us.

Etc., etc., etc.  My experience is not unusual.

What people need to know about this industry is that video game projects are very large and expensive, with thousands of moving parts that need to come together just right.  99.999% of what happens on a project will be outside of your control.  It only takes one bad twist of economy, industry, business, etc., or incompetent person in a key position, to ultimately take a project down.  

Most projects don't amount to much, and most studios go broke.  Odds are, regardless of your talent, failed and mediocre projects will far outnumber successes, you will often be out of work, and likely forced to move to land new jobs.  This is not the industry to be in if you value stability.

Disclaimer:  I didn't Google to see if these are ok to use.

Fire Wire
Ride the Wire
Electric ICE
ICE Pick
System Subversion
Cyber-subversion
Hired Hack
Part-time Hack
Hack for Cash
Hack Saw
Sam Hack
Subnet Mask
Subnet Take-down
Net Control
Net Shadows
Cypher Fatigue
Super User
Root Access
Inside Job
Cubicle Commando
Protocol Hero
Router Raid(er)

• In fourth grade they had a Texas Instruments computer in the classroom.  If I finished my classwork early, I could mess around with it.  Taught myself Basic.
• In Junior High they had a computer class, but after a week I realized they weren't going to cover anything that anyone who had been within 100 yards of a computer before didn't already know.  I transferred to wood-shop.
• In High school they offered essentially that exact same computer class.  But they also offered Advanced Placement Pascal.  And a top grade on the advanced placement test would test me out of the entire first year of Computer Science classes in college.  That's money in the bank.
• In college they didn't really teach languages.  They'd say, "Your project this quarter is in Ada.  There's a book on it in the book store."  When I graduated, I had about ten languages on my résumé - not to claim I was proficient, but to start the conversation in which I pointed out that I had been taught to learn any language.
• There was one rogue professor who taught C++ under the guise of an object oriented design class.  So I learned C++ much more thoroughly than any other language.
• In college, I got a job making networked medical systems, using C++.  And I used it in aerospace for a year and a half during and after college.
• At my first job in the video games industry, they were on the first generation of games developed in C++.  We discovered all the classic pitfalls, such as the all consuming monolithic base class.  But I also arrived at the abstract factory pattern independently, while designing a physics system.
• Between that and my next job, I developed a functor class which was a precursor the the C++11 std::function and std::bind.
• But there were aspects of the language which for various reasons weren't used by any of the studios I was at.  There are those who would say that I was never truly proficient with the language until very recently, when I took some time to study those aspects of the language and the new C++11 features as well.

I'm not sure you're gaining much by using templates here, since you're using the loaded data, and you must handle each type differently.

I take a slightly different approach.

class asset_loader
{
	public:
		template<typename T> void Registrar(function<function<void()>(T *)> const & Register);
		unique_ptr<asset_package> LoadPackage(char const * PackageName);
	...
};

The basic ideas here are:

• Assets of various types are generally grouped together into a file to be loaded all at once.  I call that an "asset package".
• There are various engine sub-systems which will end up owning and using the loaded assets.  You associate member functions of those systems with specific asset types, using the templated member of asset_loader.  Then, when it's loading assets, it calls those functions to submit each loaded asset to its respective system.
• Loading assets produces an asset package object which acts as a sort of smart pointer to all the assets that got loaded.  When this object is deleted, it unregisters and deletes those assets.

It's used like this:

asset_loader AssetLoader;
AssetLoader.Registrar<model_asset>(bind(&renderer::Model, &Renderer, _1));
...
unique_ptr<asset_package> pAssetPackage(AssetLoader.LoadPackage("MyAssets"));
// Renderer.Model(model_asset &) will get passed each model_asset in the package.
...
pAssetPackage.reset(); // (or deletion of the unique_ptr) Signals for deletion of all associated assets.

Admittedly this approach is significantly more complicated.

But note that renderer::Model(model_asset &) - which is an example of a function which would receive actual loaded data, and perhaps create a driver owned resource from it - is a unique member of a unique class which owns and processes model data.  Loading and lifetime management is handled generically across asset types, but specific data processing is not.

lt;dr.  A couple of random thoughts:
.

• If you have a game concept/design and would like to assemble a team to build it, your worth is shown in the draw exhibited by you concept/design.  For the purposes of finding people, your worth is augmented by your connections.
  
  Ultimately, if you have the concept, and you have the design, and you're building the team, then it seems like you're not the one who would need to demonstrate your worth.  I'm not envisioning someone thinking, "I'd like to work on that project, but I'm not sure the designer is bringing enough value per body to the mix."
  
  Demonstrating value as a designer seems more like something to worry about when you want to get hired.  With that in mind...

.

• If you don't have a body of work to show, then you aren't likely to get hired as "game designer".  Level design is a much more realistic goal for breaking in.  As I have posted in a previous topic, the primary skill you need to demonstrate for that position is the ability to execute a design concept within given constraints.  So level/scenario construction is the more relevant skill, especially if you can show how your examples support a story arc, and deliver on various design goals.

.

• Keep in mind that, for legal reasons, existing teams do not want to see your game designs.  And if they do hire you, they'll generally own any design you show them.

.

• While being an engineer implies some attributes which happen to be invaluable in a designer - as an engineer, I do not have all the talents of a good game designer.  If you do, then allow me to apologize, on behalf of engineering, for any lack of respect for your skills conveyed by the more sophomoric members of our fraternity.
  
  Everyone seems to think they can design games, and engineers who are particularly proud of themselves for tackling a very challenging trade, are clearly not immune to this sentiment.

.
If you really think that potential contributors might pass your project by, for lack of respect for your past accomplishments, then maybe you could start by focusing on a person or two who you think would have the draw that you don't.

Best of luck to you.

Requirements change. [...snip...] What design you need is based of off need...

I couldn't agree more.  We need to be ready to evolve our design and implementation as we go.  

So we wouldn't reject, for instance, an implementation that gives us a single instance of some given data, if that's all that makes sense for our needs.  First, that's not really a "limiting" choice if it deliveres exactly what we need.  And second, we will change our design and implementation as our needs change, so we're not limiting ourselves in that sense either.

If we could actually use multiple instances of that data for our given situation, then we wouldn't choose an implementation that couldn't deliver them in the first place.

You're still limited to one instance...

If you need more than one instance, I can see how only having one instance might be a problem.  If we actually have this problem, we're doing something very wrong.

...you still need to make sure it's valid/reset it whenever you load a game or whatever.

Constructors and destructors should properly handle static data as well as dynamic storage.  Proper setup/shutdown/reset/etc. is business as usual for both static and non-static member data.  The two are not exactly the same, but managing static member data is not inherently tricky.

...shared state is the real problem with globals, not the global access.

Any time any object can trade information (get or set) with any other object, they're sharing state. Shared state is a given.  The question is how state sharing is restricted.  It's the fact that they're shared globally (i.e. sharing of state is not restricted) that makes globals so problematic.  Global access is the state sharing issue with globals.

When you know  (or can be sure within reasonable doubt) that something you are about  to make will be faster than the alternative, why would you not make it?

Because chances are you don't need it.

In general, there shouldn't be enough dynamic allocation going on that gaining a speed increase over a stock memory manager should make an appreciable difference to overall performance.  When one's use of a stock memory manager becomes a performance  bottleneck, they've made some fairly serious higher level design blunders.  And in general addressing issues in higher level technical design will produce far greater performance improvements.  

If  you waited around for every part of the engine to prove itself as being  a bottleneck, you will ultimately find yourself overloaded with  bottlenecks, or you won’t be able to find bottlenecks at all because you  won’t have a good basis for comparison.  Everything will be slow,  thereby making nothing seem like a specific bottleneck.

And yet a profiler will still give you your best targets for performance improvement.  You should always develop with performance in mind of course, but the "performance" of the vast majority of your code just won't matter to actual overall engine/game performance.

The idea isn't to use the slowest approach until it proves a problem, it's to avoid unnecessary work until it proves necessary.

The only reason to claim “premature optimization” is when either of the following is true:
#1:  You aren’t even sure it will work, and you are already trying to make  it optimized.  This falls under the “make it work, then make it fast”  mantra.
#2: You aren’t sure that the end result will be faster than the alternative.

If either of those are true, then you probably can't call it an optimization at all.  

The marks of premature optimization are that the anticipated enhancement hasn't proven necessary to success, or that its implementation hinders development.  Creating a custom heap meets both conditions, to different degrees (depending on the approach and the scope of the work).

Most programmers are not going to be able to  produce a heap allocator that is more robust, faster, and better-tooled  than existing allocators.

Back in the day, this wasn't such a big deal.  Operating systems (and their heap managers) were relatively newly multi-threaded, and game engines were not.  It wasn't too big a challenge to make a heap allocator that used the same basic allocation, block joining, and error detection schemes without the multi-threading overhead.  But in the end, it still proved unnecessary.  


Anyway, my point isn't that you should never write your own high performance custom heap manager.  It's that if you never do, you'll most likely never end up missing it.  The stock managers work, and work well.  Your time is probably better spent elsewhere.  But it is your time.

So basically, how important is this kind of thing?

In the most general, high level terms, writing your own heap manager is almost always an instance of "premature optimization".  In other words, you should really only be doing it if:

• you've developed an engine/game which turns out to "over-use" heap allocations to the point that the profiler has identified them as a performance bottleneck,

• you only need that little boost that you think you can get from a custom heap manager, and

• you can't find a (probably much more effective) higher level solution to the problem.

In other words, almost never.

As for memory corruption and leak debugging tools, stock heap managers (e.g., Microsoft's) generally provide them, and the first thing a custom manager does is break those tools, costing you development time on your own.  In fact, there's enough value in the built-in debugging tools, that even if you were to write your own manager, you'd probably want it disabled during development, and only swapped in for whatever performance boost it can give you, in release builds.