Jump to content
  • Advertisement
Sign in to follow this  
SeraphLance

Total Data "Reentrancy": Fool's Errand?

This topic is 2134 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Note:  The title may be a bit misleading, but I have no idea what to call it.

 

So I've been making a game on top of the SharpDX Toolkit, and I recently discovered that the Toolkit's Content Pipeline dynamically reloads shaders at runtime if changes have been made.  I haven't had the time (class projects and the like take priority) to investigate how it's done, but it got me thinking:  I could apply this elsewhere.

 

For instance, a configuration file that controls the physical properties (L/W/H) of the tiles in my map grid, or the textures and parameters to materials.  Both of the aforementioned data is -- or at least will eventually be -- in configuration files, so there's no reason this couldn't be done.  What I'm unsure of is...  is it worth it?

 

There's certainly utility in trying something like this.  Being able to rapidly experiment with things during runtime is always a plus.  However, overhead notwithstanding (it's easy enough to turn off for a release build), this would require some serious re-engineering.  I'm only three weeks into my game, so it's not insurmountable, but it's still a significant amount of time and effort, and I'm not even sure how to approach it yet.

 

Has anyone experimented with trying something like this?  If so, what were your experiences?

Edited by SeraphLance

Share this post


Link to post
Share on other sites
Advertisement
We've got a system which is both a mix of 'hot loading'/'hot swapping' and 'live link' of data - between the two, with properly written code, you can effectively change pretty much anything in the game and have the game react sanely, the definition of 'sanely' depends the subsystem.

The whole engine is built around the idea that at "any" point in time a resource you were using could be yanked from under you ("any" meaning during the start of a frame, not mid-frame) and you will have to react to that in some way; be it to stop rendering or fallback to a 'default' (if, for example, the material gets yanked our model rendering system will fallback to a default checkerboard material until the data comes back).

"Hot swap" is controlled via tools; on save a build step is run to condition on the data then the runtime is notified to flush and reload the correct resources.

"Live link" allows for 'over the wire' changes; so you can fiddle with the look of a material by changing its parameters and these are instantly transmitted across the wire and patched into the data for use the next frame. This applies to most properties, so you can open the world in the editor, load the same world in the game, then move the camera around in the editor and the game will reflect the changes (same with dynamic light sources, sounds, model locations etc) - we also have a hub in place which means you can broadcast the changes to multiple instances at once (useful for cross PC, Xbox and Playstation visualisation).

Artist are the biggest fans of this system; due to an engine rebuild we were without this functionality for a bit as things transitions and the guys who had used it on the last project in its primative form (live link only) were virtually battering down the door to make sure the functionality came back. (The guys who hadn't used it before didn't see what the big deal was, then they got it shown to them and joined the mob *chuckles*).

Share this post


Link to post
Share on other sites

I implemented such a system for the Molecule Engine I'm currently working on. Even though the engine is not completely finished yet, it has been such a time saver already. Being able to quickly iterate on things such as models, textures, shaders, etc. is paramount for fast iteration times. I've recently added support for runtime-compiled C++ code, so you can even change scripts/code on-the-fly, and see the changes immediately. I use this feature on a day-by-day basis to get things done, and move them into the engine core once I'm finished. I never want to go back to an engine that doesn't support hot-swapping of *everything*!

 

But as others already pointed out, it is quite a bit of work to get such a general system up-and-running, and of course it's easier if you planned for it right from the start. The way I do it is to use a completely separate content pipeline (= its own executable) that watches file system changes, compiles assets, and talks to the engine over TCP/IP. The engine only ever reads binary data (optimized for the platform), and all binary data must be prepared by the content pipeline - no parsing or similar is done at runtime!

 

This has several benefits: you can write your tools in any language, the engine executable is smaller (reading binary data is simpler than e.g. parsing XML), loading times are always fast, and you can change any content on-the-fly.

 

Almost all of the assets can be compiled in less than a second, so iteration times are really fast.

 

Be warned though: having such a system is addictive, you don't want to go back afterwards :). Finally, here are two videos that show the system in action: asset hot-swap, code hot-swap.

Share this post


Link to post
Share on other sites


I've usually seen this called "asset refresh" or "hot reloading", not "data reentrancy" 

 

Ah, but there's so much fun in trying to make up terms that nobody understands when you don't know the proper jargon. :)

 

Anyway, thanks for the information, guys.  It may be total overkill for a project of this scale, but I'm definitely inspired.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!