Jump to content

  • Log In with Google      Sign In   
  • Create Account


Member Since 08 Apr 2011
Offline Last Active Feb 19 2016 05:52 PM

#5276254 Procedurally-regenerating identical content on different machines (floating p...

Posted by on 17 February 2016 - 10:44 PM

I've been designing the architecture for a distributed procedural content simulation engine for some time now, and recently it's become apparent that floating point determinism is going to become a thorn in my side with respect to my efforts to enable distributed setups, so for those of you with experience in (or reasonable knowledge of) floating point determinism issues, I'd really like to know how you'd approach this scenario:


Imagine you have two machines. The machines can be reasonably guaranteed to use modern, common hardware and operating systems (PowerPCs, IBM mainframes and other gotchas are not relevant here). In other words, both machines are either going to provide modern commodity x64 architectures on a server rack, or be two relatively-modern gaming machines of two friends whose machines are sharing the workload I'll describe. The operating system types can be constrained to the standard three (Windows/OSX/Linux).


Anyway, imagine we have two machines, and one of them is going to procedurally-generate some content based on known inputs. Let's say the content is a simple perlin noise-based terrain. Taking this a step further, let's say we procedurally-generate a set of physical objects (for brevity, perhaps they're simple wooden cubes of varying sizes and weights). We then use physics algorithms to scatter them around the terrain and allow them to settle based on physical characteristics such as gravity and the shape of the terrain.


Here's the catch:

  1. We don't know in advance which of the two machines will do the work.
  2. The machine that generates the content will not be able to guarantee that the other machine is available at the time that the work needs to be performed.
  3. The other machine will, later on, have to do generate the exact same outputs under the same conditions.
  4. We don't want to store the overall result as there will likely be too much data.
  5. It'd be nice to be able to offload some of the compute work (where relevant and reasonable) to the GPU (whatever is available on the machine in question).

Some places that I am willing to compromise:

  1. Reducing floating point precision
  2. Storing a limited subset of source data, as long as bulkier derivative data can be regenerated as needed
  3. Using large integers instead of floats
  4. Using fixed point calculations

Some additional questions:

  1. Is it possible to write integer-only versions of the popular coherent noise algorithms? (perlin/simplex/brownian/etc.)
  2. Can I get away with forcing IEEE754 floating point accuracy or will that compromise speed too much?
  3. Are 64-bit integers a thing on GPUs yet, or likely to become a thing any time soon?

As an addendum to this, I'm considering the possibility that this problem doesn't really have a good solution at this point in time, and that perhaps I need to be willing to simply store a lot of data and ensure that it is available when a machine needs to generate new content.

#5255938 Thoughts on the Boost.Build system? (as opposed to CMake?)

Posted by on 06 October 2015 - 08:48 PM

Personally I like to place my build directory as a 'build'-subdirectory of my source directory and set that directory to be completely ignored by source control. Others require the build directory to be completely separate (for example 'D:\projects\myproject' and 'D:\projects\myproject-build'.


Yeah I do this too, irrespective of build system. I guess I just need to learn CMake a bit more than I have.

#5255723 Thoughts on the Boost.Build system? (as opposed to CMake?)

Posted by on 05 October 2015 - 05:14 PM

I don't think I ever had that problem. Maybe you are doing something wrong?


Very possibly. All I know is that when I run CMake on various different projects that are not mine, I end up with loads of solution files, with extra solutions like ZERO_CHECK and so forth, and a bunch of other accompanying files and folders, all cluttering up the root directory.


If you do an out of source build, you can isolate the generated cmake files to a build directory tree and avoid polluting your source.


That sounds ideal. I'll look into that, thanks.


I use the Qt Build Suite (Qbs) with Qt Creator on Windows


I must admit, the ugliness of the QT IDE makes my inner snobby designer twitch a bit... I bet it's really good though. Maybe I should look into it a bit more than I have...

#5255578 Thoughts on the Boost.Build system? (as opposed to CMake?)

Posted by on 04 October 2015 - 04:53 PM

For C++ on Windows, I've been preferring development in Sublime Text, rather than Visual Studio. It's more lightweight, there's less magic going on and I've got it configured the way I like. And I really hate VC++'s filtered folder project system. I just want to see my source directory and not have to screw around with the project file every time I want to add a child folder. Also, as someone from a non-C++ background, I feel I am learning more by running my own build script (just a custom PowerShell script at the moment) and having it run CL.exe. CMake is all well and good, but my project is built around a large number of plugins which each have to be built separately, and... well I guess the way CMake clutters up my folders with a million files just irks me a bit. That, and I've read a number of stories from other projects where the developers were complaining about CMake problems that made their lives harder rather than easier on Linux. I like things to be simple and clean.


Even so, I know that maintaining your own set of build scripts can get complicated and be a pain to maintain, particularly when you want to target the other major OS's as well. CMake's not the only show in town. What do you guys think about Boost's build system? I built Boost for the first time a few weeks ago and found the build process to be very clean and straightforward. Still, CMake seems to be what the majority of dev teams gravitate towards, so I'd like to get the opinions of those more experienced than myself with cross-platform C and C++ development. If you know of something else really solid as well, I'd be interested in that as well.

#5254910 Unpredictable, ubiquitously rng seed

Posted by on 30 September 2015 - 07:49 PM

Why not just use UTC date? It's the same for everyone no matter what, and changes exactly once per day.



#5250365 GDD Level of Detail and Completeness?

Posted by on 02 September 2015 - 10:24 PM

To me that seems like it would be an overwhelming burden before starting a game, and would prevent you from ever doing anything new and interesting in your games as you try to figure out every detail before even getting something playable. I think it would be better to try something new and follow the fun.


I often see this objection from newer/younger/less-experienced developers. Nobody is advocating that you work out the literal code of your whole system before you type it in. I also think that sounds like a huge, boring waste of time. What Norman (as I understand his message) is saying (and I agree with him) is that, assuming that you have a vision for your system/game/app/project; i.e. you're presumably embarking on your project because you've had a great idea and you want to make it real, then you're going to want to work out, in advance, what the primary systems and subsystems of your project are going to need, how they're going to connect together, how data is going to flow and so forth. You're going to want to build a strong mental model of all of the primary moving parts of your application, and in going through this process to build up your model, you're going to hit countless things you haven't thought of that would have messed up your architecture too late to fix easily, had you started coding straight away. These will include inefficiencies in your initial approach, deeper requirements that you didn't realise you had, cases where your initial assumptions turned out to be completely wrong, etc.


I'm not just parroting stuff I've read somewhere, I've been there. Countless times in my career I have started coding with little-to-no plan, and have worked in environments where that was the approach, and it is almost always a fantastic recipe for coding yourself into a corner and screwing up the project. Think of all of the failed projects; the games that got released after huge delays (if at all) with way more bugs than there should have been, the huge government and corporate website and intranet projects which went millions of dollars over budget and came out as a steaming pile of broken crap, and all of the indie projects which started out great and then ended up being cancelled or tossed in the "developer has lost interest" pile due to the immensity of the problems that the developers created for themselves.


I have been there and done that, over and over again. I've learned the hard way, and it changed my approach over time. And it's really not anywhere near as hard (or as unfun!) as you might think!


Screw design documents. If you have a large team and publisher-imposed deadlines, or you need VC buy-in or whatever, then sure, go the detailed, official design document route, but for yourself? Keep it simple and employ a divide-and-conquer approach. My approach is actually somewhat analogous to the widely-accepted, age-old process called "critical thinking". Yes, I also used to think that was just a term that meant "thinking about stuff carefully and without bias". It's more than that and there's actually a process, but I'll leave that to you to Google, if you're interested.


Anyway, start with bullet points and words. Write down a few sentences that you think explains your game to someone you're pitching it to. This may sound boring, but it's only a few sentences and it's kind of a catalyst that switches your brain from "shutup and let me code" mode into a more thoughtful, ready-for-brainstorming mode. If you're so inclined, you may expand this to a few paragraphs. In the process of doing this, your ideas tap is going to start to flow, and there's nothing you can do to stop it. And once it starts flowing, you're almost not going to be able to write fast enough to keep up with yourself. It's an enjoyable process, trust me! I have lost countless days to this process on numerous occasions just because I was so deep in design, trying to figure out the details of what I was trying to do.


Next pretend that this person you're explaining your game to is a developer who is contemplating working with you on the project, but wants to know more, so start putting down a few bullet points just quickly outlining the primary components you're probably going to need to include, and explain what those components do in a way that if you came back to it a month later (because hey, you got distracted with something else, or got a boy/girlfriend or something) then you'd be able to read through your notes, understand what you were trying to do and continue where you left off. Think about your game in terms of the data if you can. Imagine what sorts of data will exist in your game, and would need to be processed if your game were just a simulation of your idea, with graphics and inputs and stuff to be implemented later. Doing this will help you to separate minor implementation details (which you're not going to bother with until you're coding) from the actual logic and underlying architecture that describe the raw game itself. As a quick example, you might think of your player's spaceship as a sprite, and it needs to respond to input and then you'll think about key mappings and that sound effects will need to be played, etc. This is the wrong way to approach things. Instead, think about the spaceship in terms of its data. How big is it? How fast can it go? Can multiple players be on board? What sorts of upgrades and modifications can be performed on the ship? Can it dock anywhere? Can players get out, or is the player the ship itself? What other sorts of attributes might it have? Expand this to the other things in your game. What sorts of interactions with other game entities are possible? Why do you even want those interactions to be possible? And so on.


By now you should have quite a few bullet points and they'll probably be quite disorganized. That's fine! Every time you write down an idea, one of two things will be true. (1) you're going to already have a detailed mental model of exactly how you're going to approach it in code, and I don't mean line-by-line, I mean conceptually. If you were to sit down and implement this feature right now and you don't even know where to start, then you probably haven't thought about it enough in advance, which brings me to point (2) you don't really know how you're going to approach this just yet; you may have no idea at all, or maybe sort of a hazy concept of how you might go about it. If this is true, then hit the tab key for your next bullet point (you're indenting it from the idea bullet point you're breaking down), and literally write the question "How am I even going to approach this?", and then start thinking of rough ideas. Write them down as bullet points. Every time you write down an approach, question it and try to think of a reason why it might fail. Write that down. If you think it's a good idea, write down why.


What you're doing is breaking your approach down and coming up with the architecture of your game. Every time you pose a question to yourself that (a) is not a minor implementation detail, and (b) you don't know the answer to, then indent, add a new bullet point and start breaking it down by posing the question to yourself "how am i going to approach this?". After you have done all of this, you're going to find that you very rapidly start crystallizing your game idea into something that you have a clear mental model of. You're going to have inadvertently hit many, many brick walls in advance of having done so in code, and moved beyond them in a very short space of time before having ever coded up the suite of classes and functions that it would have otherwise taken for you to realise the flaws in your own approach.


To all those who say "there are some things you just can't know before you write the code", I say that either you're just being argumentative, or you haven't adequately figured out the difference between implementation details and architecture. If you're hitting fundamental architectural problems in code, then you failed to invest in what would have been valuable time spent figuring out your architecture in advance. If on the other hand, your architecture is sound, but the problems you're hitting are at the implementation detail level, then you're missing my point. Take this as far as you need to know that your approach, across the board, covers that which you wanted to achieve up front. If your requirements change halfway through, then, well there's not much you can do to plan for that other than being careful to balance consideration of the "what-ifs" against venturing too far into YAGNI territory. If you don't have enough information to know whether your approach is sound, then research! Find out what algorithms are available that could be used once you start coding. Learn about some new data structures that could solve your problem. Write a few quick sample apps using the frameworks or libraries you're thinking of so that you can get a sense of whether or not you really want to use them. Read a few articles. Ask some questions on StackOverflow. "I didn't know that in advance" is not a good enough excuse for wasting three months coding something only to realise that your approach was fundamentally flawed and destined for grand failure.


Finally, if one of your requirements is that you want the freedom to evolve, experiment and change your approach as you explore, then fine. Plan for that. Ask yourself the questions "what sorts of ways could my approach hypothetically diverge from what I'm planning at the moment?" and "how can design my code in a way that will preserve the important stuff without boxing me in?".


Take a leap of faith and try this process this once (I mean properly - you should end up with pages and pages of notes at a minimum), and notice how good you feel having encountered and moved passed six months or more of roadblocks in a tiny space of time without having wasted the time coding to get to those realisations. Feel how great it is coming up with a theoretical model of your game that you now have a really good idea of how to tackle and what architectural requirements you're going to need to deal with almost before you've written a line of code. I know coding is fun, but please. Take the time. Trust me on this. Architecture is architecture, whether you're writing a game or a stock trading system. Get it as right as you can, before you start to code, and you'll save a huge amount of time and have a result you're a lot more proud of and which is a lot easier to maintain and extend.


That is all.

#5244440 Voxel Ray Tracing

Posted by on 04 August 2015 - 12:16 AM

I thought the point of raytracing was to allow for more realistic light physics? The lighting in Voxelnauts is terribly washed out and dull.

#5241142 New Game Programmer!

Posted by on 17 July 2015 - 08:25 PM

Here you go:
function range(start, end) {
    var arr = [];
    for(var i = start; i < end; i++) arr[arr.length] = i;
    return arr;
var sorted = range(0, 52).sort(function() { return Math.random(); });

#5237438 Learning Direct3D Programming properly

Posted by on 29 June 2015 - 02:19 AM

I'm not sure I would even know where to begin with that. I wouldn't know how to interact with hardware without using Direct3D functions.

It was just a suggestion; and writing an engine doesn't mean not using Direct3D, it means moving the bits that directly interact with Direct3D into class and function wrappers that could conceivably be swapped out for, say OpenGL. Anyway, I realise my suggestion was beyond what you were looking to do. Just look for examples of games doing things you don't know how to do and learn how to do them. Write simple little games that do something that will stretch your skill set. For example if you're alright at basic lighting but haven't done anything fancy with it, challenge yourself to implement SSAO maybe. Or perhaps you've never used the geometry shader, so maybe use it to put grass into your game. Etc.

#5236656 What to focus ?

Posted by on 24 June 2015 - 08:46 PM

Here you go:


Go through that and watch the confusion slowly melt away smile.png

#5236287 Level design of a "Roguelike" set in outer space

Posted by on 23 June 2015 - 02:27 AM

Your idea reminds me of Approaching Infinity, which is a pretty new game, but I don't think it's that well known. https://ibol17.wordpress.com/the-game/

#5233486 Questions about the final stages of the graphics pipeline

Posted by on 08 June 2015 - 01:12 AM


#5233207 What should I do now that I implemented a game mechanic that is not popular a...

Posted by on 06 June 2015 - 01:30 PM

I'm going to go out on a limb and suggest that maybe the underlying code is a bit broken and should be refactored/improved anyway. Judging whether or not to apply damage to a character should not be a performance issue if the code is well-designed. Perhaps he is trying to determine in advance whether or not to perform the check to begin with on a per-character basis when in fact he should go ahead and do it as he is right now but simply calculate the damage per character hit, after the character has been hit? The damage calculation can simply be nullified if the character is friendly. Hard to say without seeing the code but I'd be very surprised if there aren't multiple candidates in the code for improvement that would make this issue go away entirely.

#5232874 Questions about the final stages of the graphics pipeline

Posted by on 04 June 2015 - 08:05 PM

Thanks, great response, and very educational! Engine programming definitely makes my life a lot more complex than if I were to just pick up Unity or UE4, but I'd never learn this kind of stuff otherwise, and I feel like it'll help me wield a lot more power in the long run.

#5231215 Graphics engines: what's more common; standard vertex structure or dynami...

Posted by on 27 May 2015 - 05:46 AM

What's more common in engines? Using a standard one-size-fits-all vertex structure, or a custom vertex structure depending on the context, and delegating the choice of vertex composition, and associated shader input layouts, to the game developer? Is there any significant downside to hardcoding a standard vertex structure? Is vertex structure flexibility a case of YAGNI?