Bluefirehawk

Members
  • Content count

    112
  • Joined

  • Last visited

Community Reputation

1232 Excellent

About Bluefirehawk

  • Rank
    Crossbones+
  1. When you are like me, as old as the Gulf Wars, you probably started programming in a managed language like Java, Ruby, Python or C#. As many of you before, you didn't really know what the language is doing with your objects, you create them when you need them and forget about it. As many programmers gain experience, they start to learn more and more about what happens behind the facade, about Garbage Collecting and overall memory management. Sadly, I see many programmers filled with half right knowledge and wrong conclusions especially in the field of Garbage Collection and Performance. I saw attempts to "optimize" C# code, which did, if anything, slow down the program. So let us take a closer look at the modern Garbage Collectors, how they work and what problems they want to solve. I am going to explain the specific Java implementation, but don't worry, most of them work fairly similar. Garbage Collection isn't a beginner's topic, I assume you have a decent knowledge of your programming language of choice. I also assume you know what Heaps and Stacks are, how you use them in C++ and how pointers work. If you feel unsure about any of those topics, you should probably read this article later. Unless you are writing performance critical code, you can live a happy programmers life without knowing anything about Garbage Collection. So take your time, this article won't run away. Reference Counting, or why would I need a Garbage Collector? This is maybe the oldest strategy to find unused resources. It is still widely used in C/C++ programs and OS code. This strategy does exactly what the name implies, it counts how many references a resource has. When it drops to zero it can be cleaned up, usually by the thread that decreased the counter to zero. It is a beautifully simple design. The resources a thread has to clean up before resuming are very small, meaning that there are no noticable "hiccups" for the user. It appears as if the program runs seamlessly. Sounds great, but sadly Reference Counting also has its downsides: Referemce Counting is rarely supported on the language level. The compiler doesn't help to enforce correct increase/decrease of the counters. Nothing stops the programmer from creating memory leaks. Free Space needs to be managed, typically by a Freelist that is able to merge neighboring free areas. It also takes time to allocate a new object, since the program has to go through the list and find an area big enough. Cyclic dependencies are hard to find. There are solutions to this problem, like weak pointers in C++, but the programmer has to identify the cyclic dependancy and resolve it manually, or the program has a memory leak. The Heap tends to fragment as time goes on. It is possible to make it less of a problem by allocating as much on the stack as possible, like you can in C++. But when you are developing a server application that should run 24/7, fragmentation can become a real issue. Simple Mark and Sweep Garbage Collector Java's Garbage Collectors are using a Mark and Sweep algorithm. Every time the GC kicks in, the JVM and all Applications running on it get suspended. After that it cleans the heap in two phases Mark Phase:The GC marks all objects that are reachable Sweep Phase:The GC sweeps all unreachable objects Fair enough, but how does the GC actually find unreachable objects? How to find reachable Objects, the Root Set Think of the current in memory objects as a graph, each Node represents an object and each edge a reference. All objects reachable from the Root Set should stay in memory. So if you traverse each edge, you find all reachable objects, the mark phase is basically a graph traversal. Each object is reachable as soon as it was visited once, so the GC can mark a visited object with a flag. The last missing puzzle piece is what the Root Set actually contains. All objects belong to the Root Set that are: References on the Callstack (A stackVar = new A(); ...) Static, global object referenes very short lived references ( return new Integer(3);) Java allows through the JNI (Java Native Interface) the invocation of C/C++ code which is also able to allocate Java objects. Those are also part of the Root Set Sweep the Heap Now it comes to actually cleaning up the heap. The GC has to somehow walk through the whole Heap, visiting each object. How does it do that? The GC knows the starting adress of the Heap and it knows the heapsize. Now the only thing it needs are the sizes of the individual objects, so it knows where an object starts and where it has to look for the next one. The object size together with the mark flag are typically stored as meta information in each object. Now we know how to find objects and we can clean them up. Awesome, let's have a look at how we improved compared to the Reference Counting strategy: Allocation, STW Phases and Other Bad Habits But how does it find a new space for an object? Sadly, we are still dependant on a Freelist for managing our free spaces. Even worse, now every object is allocated on the heap. Not only do we still have fragmentation issues, but we made them worse. If that isn't enough, now we have Stop The World (STW) phases, the time span from the GC kicking in to it giving control back to the actual program are very noticable. Which is pretty bad if you are the provider of a streaming service, or playing a video game. Note that the Mark and Sweep GC is doing the same work as Reference Counting, the only difference is the distribution of when and how much each strategy cleans up. But what do we get? The GC automatically solves cyclic dependancies, if a cluster of objects isn't reachable, it can get deleted no matter how many references they have internally. The GC also hides memory management from the programmer, it is one thing less he or she has to worry about. That sounds great, but the Mark and Sweep GC did so by making everything else worse. If you think this can't be worth it, then you are right! Just for legacy, the first experimental Garbage Collectors worked in this way. I don't know if they were ever thrown in a productive system in this form... I hope not. They were devastatingly impractical and required a reboot after five hours or so, because the program managed to fragment the heap to hell. The problem does not lie the actual Mark and Sweep algorithm itself. This is nice and tidy and is used in every Garbage Collector I know of. So let us analyze the problem we are trying to solve, how many objects are allocated and when exactly are they becoming garbage? Object Lifetime Typically a program allocates some objects at the start which stay alive during the whole runtime. There are also many objects that become useless shortly after their creation. The following diagram shows what a typical program usually does. The Axis are a bit puzzling, but think of the x axis as the age of a byte and the y axis the number of bytes for each age. For now, ignore the dashed lines What does this mean? The average computer program tends to have three different classes of objects. By far the largest amount of objects die shortly after they were created. Then follows the middle class, still moderately in size and stays useful for a little while. At last, we have the old class. Not as many objects as the other classes, but they stay in memory for a long while, some even as long as the program runs. Now it becomes clear why the simple Mark and Sweep GC fragmented even worse. At the start of the program, short, middle and long lived objects get allocated at roughly the same time. After a second, a big amount of bytes became garbage already, but they are all spread out between medium and long lived objects. In C++ for example, short lived objects could at least sometimes be allocated on the stack. As you see, in order to make a "good" Garbage Collector, we need to handle the classes differently. It makes sense to regularly clean up the short lived objects and make space for new ones. On the other hand we should be able to let the long lived objects alone as long as possible, only few objects die when they are already old. It may not surprise you, but modern Collectors can do that! When Java only cleans up young objects, it calls it a Minor Collection and when it cleans up the young and the old objects, it calls it a Major/Full Collection. But how do we only search through young objects? And how can we take care of fragmentation? The Generational GC So this should be it, the solution to all our problems. It is still a Mark and Sweep GC, but far more sophisticated. This GC uses Minor and Major Collections and it splits up the Heap in five different Generations or Spaces. Again, these are the Java-specific names, other Garbage Collectors work similarly but use different names: Eden Space: Here is where every object will be created. Every minor collection completely clears out the Eden Space. Survivor Space 1 and 2: Both spaces have to be exactly the same. As the name implies, the survivors of a minor collection get copied here. Why we exactly need two Survivor spaces gets clear when we talk about aging. After every Minor Collection, one Survivor Space is completely empty. Old Space (or Tenured Generation): When a object has lived long enough, it gets promoted to the Old Space. Here, it can only get cleaned out by a Major Collection. Permanent Space: No GC will be performed here, that is why it is called permanent. You are also unable to allocate here, this space is used by the JVM to store metadata and other implementation-specific stuff. Now let us have a look at the Generational GC in detail: Allocation As we talked about in Reference Counting, we don't want a Freelist to manage our free space. But we do have our heap divided in several different spaces. We can use a "Free" Pointer for each space. As the name suggests, it marks the beginning of the free space: The red squares are objects, the black arrow are refrences to other objects and the bright red triangle is the ominous free pointer. New objects are always allocated in the Eden Space, unless they are too big for it, then the JVM allocates it directly in the Old Space. Minor Collection Now, let us assume that the Eden Space is getting full. The Garbage Collector has to perform a Minor Collection: Each green diamond means that this object is part of the root set. Now, the Garbage Collector goes through the root set and marks all reachable objects. In this case, it marks objects 1, 3 and 5. It then copies the still-reachable objects in one of the survivor spaces: After each Minor Collection, the Eden Space and one of the Survivor Spaces have to be empty. This gets important when we have a look at when the Generational GC runs out of memory. All references of the surviving objects have to be adjusted, otherwise the reference in object 1 would point to something completely different, which we obviously don't want to happen. The Garbage Collector doesn't have to initialize the whole heap with zeros. In Java, when the Constructor gets called, every variable should be initialized to zero. It is not specified if the Garbage Collector or the new operator should be responsible for this. Aging As you recall from Object Lifetime, an object is less likely to become garbage the older it gets. Having an object die in Eden or the Survivor spaces is far cheaper than in the Old Generation, so the Garbage Collector tries to keep it outside of the Old Space as long as possible, so it is less likely to die in the Old Generation. This process is called Aging. Let us go back to our example, and imagine that the Eden Space is getting crowded again. The next new object doesn't fit in the Eden Space, so the Garbage Collector performs another Minor Collection: The mark phase now reaches object 1,3,5,7. All surviving objects are now copied in the free Survivor Space and all other objects get cleaned out. This can take a while, the Garbage Collector can perform serveral Minor Collections, swap the objects from one space to the other. This sounds like a very expensive operation. It can get out of hand when objects get promoted too late, or the Eden Space is too small and Minor Collections happen often. So aging is a good optimization, if the Garbage Collector has enough memory to work with. We can start to see a trend in this Generational strategy: The more memory we have, the less overhead it has to do for the memory management. If an object can't fit in a survivor space, it gets promoted directly to the Old Generation. Promotion to Old Generation There are two reasons for a Garbage Collector to promote objects: Objects in the Survivor Spaces are getting older, their chances of becoming garbage have sunk dramatically. The Survivor Space is getting full. Let us have a look what happens, when the survivor space is getting full: The mark phase reaches object 1,3,5,7 and 10. But you'll see quickly that they don't fit in one Survivor space. Somebody has to get promoted. The Generational GC always promotes the oldest objects first. In this example, object 1 and 3 are moved to the old space. It is possible to tune how old an object should get before it gets promoted. Fragmentation in Old Space We have looked at the Minor Collection in detail and can conclude: It is free from freelists and fragmentation. Awesome! But what about the Old Space? This is still the same as the Simple Mark and Sweep GC algorithm. So what magic concept can we use here? Since we try to let most of the objects die in Eden and the Survivor spaces, Major Collections are more rare and we can afford to perform expensive compacting algorithm here. While there are other reasons why managing Old Space is expensive, this is the most obvious and most important to know about. Any other reasons are beyond the scope of this article. This is not important for a developer, but very interesting to think about: How much memory does a Major Collection have at its disposal? Think about it, a major collection is called when the Heap has almost run out of free space. The Major Collection has almost no Memory to work with. OutOfMemoryException Now the final question, when does the heap actually run out of memory? Obviously, we would like the heap to be full when, well duh, it is full, when we can't allocate any more space. Sadly, this isn't the case. Let us make a freak example: Every object you allocate is 1 MiB in size, you have 4 Mib in Eden, 2* 0.5 MiB for Survivors and 5 MiB for Old Space. In total, your heap could hold 10 objects. Your program needs 7 objects at once in the heap, it regularly allocates a new 1 Mib object. Before it does it, your program removes the reference to an old object. Can the program run without an OutOfMemoryException? The answer is a clear yes and no! In theory at thee time of the crash, the Eden Space is full with 4 objects. The Old Space contains 3 garbage and 2 objects alive. A survivor space can not hold even a single object. The Generational GC has to clean up the Eden Space completely, but there is no space for all 4 objects, even after a Major Collection. So the Garbage Collector has no other choice but to hoist the white flag, to throw an OutOfMemoryException. But it CAN work, the Oracle implementation for example has a smarter Eden Space, it doesn't clean up the whole Eden Space because it would throw an OutOfMemoryException. So it can work, but don't count on it. Common Misconceptions Especially newer programmers seem to struggle with this. They hear about Garbage Collectors and the overhead. They think that good code therefore generates as less garbage as possible. They try to keep objects longer in memory than they should and maybe even dislike the Immutable Pattern. It is true that if you are using an object anyways later on, that you should probably reuse it, but don't keep objectsyou might use at some point in the future in memory. With modern garbage collectors, short lived objects produce much less overhead than long lived ones. The Collector might need to swap it around in the Survivor spaces or if you are unlucky, it might even get promoted. Then you really produced unnecessary overhead. So don't hesitate to throw away what you don't need, it will only matter if you are allocating and throwing away megabytes. If you are doing that, no matter if you are using a Garbage Collector or not, you should rethink your program logic. Performance When I read people arguing about why C++ has the better performance than higher-level languages, I always read one sentence that says: "because it doesn't use a Garbage Collector". Which I find a bit irritating, because it is in general not true. Let us discuss Garbage Collectors and Performance. First, we should think about what we mean when we say performance, there are mainly four factors when we talk about memory management: Memory Footprint Stop the World Phases Allocation Overhead Throughput Memory Footprint The Memory Footprint is generally worse when we use a Garbage Collector. As we concluded previously, in extreme cases both Survivor Spaces and big parts of the Eden Space are free but the Garbage Collector still runs out of memory. In other words, it uses more memory than it could. Stop The World Phases Here too the Garbage Collector performs worse than the simple Reference Counting strategy. This is neglectible for Applications like E-Banking Servers or the average desktop program. For time critical programs like Streaming Services or Games, this can be a dealbreaker. That doesn't mean Garbage Collectors are unsuitable for time critical applications. Thanks to multicore processors, tuning possibilities and cheap RAM, Stop the World Phases can be reduced to unnoticable lengths. Allocation Overhead Thanks to Generational GC, we don't need to search through a Freelist of an adequate sized memory block. We always know where to allocate the next object, no matter how big it is. Allocation is as cheap as it gets. Throughput First of all, throughput is measured in how much time the program spends cleaning up memory compared to how much time it is actually running. I think this is the most counter-intuitive point of all, but a Garbage Collector tends to spend less time cleaning up than the accumulated time in a Reference Counting program. A Garbage Collector has a favorable memory access pattern, plus the more garbage there is, the less work has to be done. A Garbage Collector gets more and more efficient the more memory it should take care of. Throughput tends to scale with the memory available, which makes it very favorable for Service Providers. When the load increases, you can add memory and handle it to some degree. Standard Configuration This is Java implememtation-specific stuff, I don't know about anything about the standard configuration from other Garbage Collectors, but they very likely have a similar strategy behind it: If you have installed Java on a desktop computer, the JVM sets the ratio Old Generation/New Generation to 1/3. The Old Space makes up to 3/4 of the total heap size, while the Eden Space and the two Survivor Spaces only get 1/4. Why is that the case? For throughput, the Eden Space should be well sized, to keep track of short lived objects, to reduce promotions in the more expensive Old Space. As we saw before, the Collector throws an OutOfMemoryException if it doesn't have enough space to clean out the Eden Space completely. On the average desktop, memory tends to be more scarce, since it has to be shared between many other programs. With a big Old Generation, the JVM sacrifices speed for a more efficient memory footprint. This is something you don't have to worry about if you are writing an everyday application. But if you are implementing a Server Application or a Computer Game, fiddling with the Garbage Collector settings may be worth your while. The End That's about it, these are the basics of modern Garbage Collectors. This is all you need to know for most of your programming life. But the road doesn't end here, there are many more advanced topics to talk about: programming pitfalls with Garbage Collection, paralell and concurrent implementations and maybe most interesting of all; Garbage Collector Tuning. I am certainly going expand on this topic at a later time and date. But first I want to write on a different topic and give Garbage Collection a small rest. References Java Garbage Collector Tuning Article Update Log 6 Jun 2013: Revised "OutOfMemoryException" 9 May 2013: Revised "Reference Counting, or why would I need a Garbage Collector?" and Introduction. 30 Apr 2013: Initial release
  2. The Introduction to an Introduction If you think of a small indie developmer team, many people imagine an Artist, a Programmer and a Game Designer. The Artists draws nice shapes with a pen, the Programmer punches his keyboard until the nice shapes show up and the Game Designer? What in the world does he do, a sacred design ritual? It is a bit hard to grasp for anybody new to Game Development, it is also not an easy job to explain. That's why I hope to shed some light the role of the Game Designer and what he actually does and maybe even give a starting point for aspiring Game Designers. I also want to touch upon Game Design, it obviously has to do something with this role and what a good design is. So without further ado, let's get right to business. The Role of a Game Designer Let's make a comparison, a game Designer and Level Designer have a lot in common. As a Level Designer, you are responsible to put the assets of an Artist into a nice level for a player. But that is nearly not enough, the player has to like the level he is playing, it has to fulfill a certain goal, for example introduce a new tool to the player. Your job is not to dream up nice levels, you are making levels for a specific purpose to the player. You design obstacles to overcome, you design the level so the player has learned something after completing it. Now you made a prediction how your Level affects the player, a good Level Designer tests these predictions,you are playtesting your level. With your new experience, you can make more accurate predictions, make a better level, playtest, rinse and repeat. That is basically what a Game Designer does, but in a bigger scheme of things. As a Game Designer, you are constantly testing predictions about how a mechanic or subpart affects the player. Even an experienced designer gets surprises on a regular basis. You should analyse the results and figure out why your predictions were inaccurate and how you can fix it. As a Game Designer, you don't design one level, you think about when and what the player should learn, how difficult the levels should become. You are responsible the pacing of the gameplay, how the story is told and progresses. You design the character skills and you are responsible for the balancing of the skill system. Everything the player can do, everything the game reacts to, is part of your job to work correctly. This is frankly a lot. But when does something "work"? Imagine that you are a Game Designer for Dota2, a team member comes up to you and says: "Hey, wouldn't it be nice to have a Necromancer Char with these cool skills?". And his ideas are indeed awesome, they fit perfectly in the visions of the game. If you are designing this character, you are responsible for two things: 1. The Necromancer needs to be balanced correctly, resurrecting a horde that kills every enemy player may break your game. 2. Make the mechanics meaningful. More often than not, you are playing a Hitman or a Ninja. And it's cool and all, but you somehow don't feel like a Hitman, like the Master Ninja you are supposed to be. Your job is that the player feels like she or he is walking in the shoes of a Dragonslayer, that you don't have to write "You are playing as the God of Thunder". When you did it right, the player knows. As the coder is responsible for "making stuff dance on the screen", you are responsible for creating the experience the player has. You have the vision of the game, you know when a mechanic does or doesn't fit in the game. You calculated the statistics the items/skills/whatever should have in the game, how it could work. Chances are, you are not the only one developing the game. Your team members are drawing, composing, programming and getting new, great ideas as they go along. The Game Designer isn't the only one full of knew concepts, anyone can suddenly have a spark of genious. As diverse your team mates are as diverse their ideas will be. But they can't come up with good ideas if you didn't manage to give them the same pictures you had in mind. As time goes on, you run into the danger of forgetting vital parts of your design ("Did it have 10% Fire Resistance of 25%?). Your awesome work is meaningless when it gets distorted and lost in the sand of time. You may not always be available for questions and wouldn't it be nice if the other team members also knew about the weapon system and how it is balanced? This calls for a way to write down your design. If you did it right, other people should understand your thoughts and decisions, why you added certain features or why you specifically left something out. Being able to communicate your plans, by talking or by writing, is maybe as important as planning itself. Only by communicating your plans, only if everybody understood what you planned, only then is your team able to build an outstanding game. This is a skill that also needs training if you want to become a Game Designer. If you have found yourself in an Indie project, the game designer tends to also have responsibilities of the industry's producer. You have the idea of the game, you know the most important mechanics, the must have and the extra features. With this, you have the burden to abandon cool features you simply don't have the ressources to implement. No game is ever finished, there is always this little feature that would make the game that little bit cooler. But you don't have limitless time and money, there is a point where you have to ship and throw it at the Game Reviewers. "Good" and "Bad" Game Design Those two terms get thrown around a lot, especially when you read game reviews. But many don't ask, when is your design actually "good" or "bad"? What is a good game? Take your favorite game for example, if you can't play it in your head, play it again for five to ten minutes. Why do you like it? Why is it your favorite game? I guess the most frequent answer would be: "Because it is fun!". You probably picked a game where this is partially true, but I argue it is not the reason why this game stands out. Many things are fun, many games have been fun but vanished from the face of the earth. I would say the same thing is true about movies. There are many entertaining movies, but the Batman reboot didn't stand out because they have been more entertaining than the old movies(well, not counting Batman the dark knight rises, my heart is still bleeding Christopher Nolan!). I say it is because of the story that we cared about. We could see the pain Batman went through. We saw a human, how he overcame his inner fears, how and why he chose to do the right thing. Batman Begins is one of the few movies I really cared about the protagonist. One of the few movies where the antagonist, the evil guy, wasn't far more interesting. It stood out because we could relate to Bruce Wayne and Batman, because it was an intriguing story. I argue the same thing is true for games. Let's take Ben "Yahtzee" Croshaw's favorite game and analyse what he says about it: Watch his Silent Hill 2 review I selected him because he is simply the hardest reviewer to please and does not shy away from pointing out flaws in the best game ever. To use Yahtzee's words: "the strangest thing about Silent Hill 2 is, from a cold critical non gushy standpoint, the actual gameplay of it is kinda shitty",in other words not fun, sometimes even outright bad. The reason you play Silent Hill 2 is for the story, but why should tell it in the game and not in a movie? Because Silent Hill 2 managed to tell a deep story without using words, it did it by solely through the Level Design, Enemies and the mechanics of the game (the combat is bad for a reason). Everything is a symbol for the Protagonist's psyche, everything comes together to tell a story in a way that hardly works in a different medium. There seems to be this doctrine in the industry that only "fun" games are good (see reference 2) and every game needs to be fun. This is like saying every movie needs to be entertaining. How would Schindler's List have looked like if the directors said: "We need to make it more entertaining". That luckily doesn't happen for movies, but it sadly does for games. To end this section on a positive note, it seeems like times are changing. If you haven't already, look at ExtraCreditz's explanation on Spec ops: The Line and why it is so important. Conclusion Your job as a Game Designer is not to make an entertaining piece of software, your job is to create experiences. It can tell a story, leave a question inside the players head, provide a way to relax the mind, or scare the living hell out of people. You plan how the player and the game interacts, you test if the player sees what you waned to show. If you have done a good job, the player will do something only possible in video games, to walks in somebody else's shoes, to travel an other world. To explore your own mind and conscious. Those are the games we remember, those are the games we aspire to. Further Reading I Have a Game Idea Mechanics Dynamics and Aestetics Developing Your Game Concept By Making A Design Document References So you want to be a Game Designer Beyond fun Article Update Log 5 Apr 2013: Updated Teaser, Deleted "The Team and the Game Desigenr" and rewrote part of "The Role of the Game Designer", rewrote part of the conclusion. 26 Mar 2013: Article Created
  3. I am a student myself, I agree that a mental boot in the ass is what many students need (again, those are not the people who have trouble with learning). But this article won't cut it, any article can not really give the boot somebody would need. A speech written by a Drill Sergeant won't have the effect when you have to read it. I am starting to repeat myself; what is the point reading this article?   On the other hand, many Educators think that it is MAINLY the boot people need, if the boot fails, you just haven't given enough boot! Which is completely and utterly wrong.  For example, I am currently in in the top 10% of my year, but I had particularly problem with a weirdly easy course. I got a bad exam back. So I learned harder, which went well because I was worse than before. So I learned harder with the reward of an even worse grade, while everybody else did less and was far more successful than me. Now THAT is demotivating and it was only one course. No matter how big your boot is, it won't help.   If you are training for something and you get rewarded with success is the easiest thing to deal with. But to be REALLY successful in life you have to cope with setbacks, how to deal with it when EVERYBODY flies past you with minimum effort, while you are struggling and making no progress whatsoever. No matter how hard you try.   THAT would be an article worth reading, hell it would be an article worth writing. But not this text.   This is the reason why I decided it was appropriate to write "And fuck you too". Because in the whole article he said "I had problems, I trained, I succeeded. See how easy it is?" . Nice that you were so lucky in your life, now take your metal and get out. This is not the reason why people are afraid of learning something, this is not the reason why people have low self-esteem. This is not what it takes to get out of this spiral. Again, it serves noone except his ego and maybe makes those feel superior who don't have a problem. It is the most disgusting, self-profiling piece of text I have read that didn't come out of the dark corners of the internet. And that is the nicest words I have found, if you message me, I will call it differently.
  4. It is a bit like saying: "Are you afraid of heights? Stop being afraid then. See how easy that is?"   "If you want to know something, you should learn it" should have been the message of the article, which is kinda as useless as the fear statement. Anybody who reacts to your appeal is not somebody who has trouble with learning something in the first place. So what is the point?   But here is where I really think you misstepped, the actual message of your article is more like: "If you are bad at something, you should learn to be better at it, or you are an irresponsible loser type of a human being!", the first sentence that comes to my mind is: "And fuck you too".   This article serves nothing to the people you are actually talking about. You are nauseatingly harsh against a group of people you are not fond of ("...such irresponsible loser types...") while you are over glorifying yourself ("… many intelligent, self-directed people are actually hearing this...").   In short: This article in this form teaches nobody, serves nobody except your ego. Please drop and rewrite.
  5. Modern Garbage Collectors under the Hood

    "Reference counting does not need free list, neither creates memory fragmentation.   Reference counting, specially as it is in C++ just helps decide when an object can be freed and frees it, but how the system manages memory is a diferent problem in my opinion."   I am afraid you are wrong. Just look at the custom memory allocation article. You say it is the job of the system, what system? The OS? Wouldn't you agree that you are using a reference counting strategy even without the C++11 pointers? If you disagree with me, please message me.   "Another item: I am not sure, but for Java, before the generation GC, I beleive they do not had the fragmentation issue, because every time an object was freed, the heap was "compacted", filling all the holes and references to objects where updated ."   Honestly, I don't know, the 'first' experimental GC's didn't compact and I don't know the exact history from there on. As I wrote, I HOPE the basic Mark & Sweep GC wasn't used in any productive system. I only mentioned the the basic Mark & Sweep GC for two reasons: 1. The idea is basically the same, but a bit more sophisticated. 2. Getting the idea what problems the Generational GC actually tries to solve. The Mark & Compact is for this article just an other piece of the puzzle. In this scope, it wasn't interesting enough to explicitly mention it.
  6. I remember reading a whole youtube page full of comments arguing if the Linux Kernel IS the OS or simply a part of it.    I agree with TheChubu, those two words are so related, they are almost interchangable. They add nothing to any conversation, calling a Programmer a Software Engineer makes no difference whatsoever. Better yet, the usage of the words tend to change with time and culture. In my Region for example, you tend to call Programmers with a Bachelor/Master in Computer Science a Software Engineer.   IMO there are better ways to waste your time than with irrelevant word definitions.
  7. Documentation for Indie Studios: Why do you need it?

    It is a nice article, and I think an important one for aspiring indie teams.   I have two suggestions: With the text "– leave them be for now. It’s better to plan this out before someone actually starts to work on it. After the Game Design documentation is done, the Technical Documentation goes into play.", an inexperienced developer might think you are propagating a Waterfall Strategy. I am very sure this wasn't your intention, or a version doc would not make sense. Waterfall seems to be the bane of programming, especially younger developers are still trying to work this way. Could you make it clear somewhere?   The second suggestion is about the paragraph after "... the Technical Documentation goes into play", you are basically talking about software design and what a software designer should specify. I think this is impossible to cover in one paragraph. I agree, the most important things to document are the interfaces, or else the code of team members Alice and Bob won't work together. But the interface of what, where do you draw the line? Sometimes, if the developers have split up working in different "modules", it is enough to specify the interfaces they offer and the responsibilities they have. Leaving the actual classes with the implementation to the developer.  I also think that, in general, a software designer should not need to specify the algorithms to use, that is an implementation detail of the class/module. If the interface needs to know about the algo, you should probably rethink the design.   In short, I think the paragraph hides the powerful choices you have in the early software drafts. If you tell a young developer to plan out the classes and interfaces, he or she tends to make huge decicions without even realizing that he/she had a choice. There are good reasons for a designer to accurately specify some classes, but there are also good reasons to leave whole project parts undefined.
  8. If you write feedback welcome, it is a bit hard to give you feedback. Like somebody gives you a story excerpt and asks: "What do you think?". It isn't easy to give specific answers to general questions.   About your System: I think you made a solution for a problem you still have to find. I don't get why it should be for multiple genres. On the surface, different genres seem to have similar leveling systems, but they can serve a vastly different purpose, solve a completely different problem. You made an elaborate system and I haven't read what you want to do with it, how it should interact with other systems to form a specific experience. So I think as a learning task, this was very good. It sure as hell wasn't easy coming up with this system. Also, it can serve as an inspirational well for future leveling systems. But I don't think you should implement it as it is in any game, because there you have a specific problem to solve.   On a general note, your system seems very complex, I guess for giving the player enough choices and replay value. But a system doesn't need to be complex to allow a lot of depth: Extra Creditz depth vs complexity
  9. What makes an RTS game stand out?

    What makes any game stand out? Game Design is only part science, the other part is art. There is no checklist on how to make an RTS exceptional, or any game for that matter. Games tend to stand out, that either have exceptional good storytelling or a novel idea. That is the most general statement I could come up with.   You aren't a senior game developer I presume, if you are planning to produce a sellable game, I'd suggest to rethink your position. You would either have to compete with multi million dollar projects or search for your own niche. Both choices aren't easy.   I d suggest you think about your game, what is special and why people would like to play it and discuss it in the forums.
  10. Modern Garbage Collectors under the Hood

    Ah damn, I hate those words and names that are ALMOST the same in german and english, like "Author" and "Autor". Thanks for the correction.   @Thurok Maybe I didn't write it as clear as I thought: C++ is of course devoid of a Garbage Collector, but you have to clean up your memory somehow. As you said, shared_ptr is just a wrapper class and that is all what Reference Counting is. It isn't supported by the language, that is exactly the point. On the language level, you could help enforcing correct counter increase/decrease (I think Objective C does this, but not sure). But here, there is nothing stopping the programmer from creating memory leaks. I would also argue that even if you aren't explicitly using shared_ptr/unique_ptr, you are still using reference counting, then you are responsible for counting when the resouce is not used anymore.   I am having a look at the passage again, I won't explicitly write that you shouldn't simply set a pointer to null in C++, this isn't a beginners article. I am probably going to write this more clearly in the introduction.   E: Yes, you are right "...still used in mid level programming languages..." is misleading, it is used in mid level programs but not in the language itself.
  11. AI Bots- Why are they hardly used anymore?

    At first glance, you might think that games like COD would already have a decent AI implementation for their Single Player Campaign, at least that was my first thought.   But these implementations are probably less sophisticated than you think: the only thing they have to do is run to cover and shoot at you. And maybe a very trivial "spotted" method for stealth missions. Everything else, any tactics behind it, is probably done via level design, backspawns for flanking etc. So multiplayer AI takes up a lot more resources than you might think.   The last game I played that had multiplayer AI was Counter Strike, here you really don't have a Single Player Campagin. Multiplayer matches are sort of the only thing. On the other hand you have games like MOW/COD that have a full blown SP Campaign, the single players are likely already satisfied with this.
  12. D.F battlefield project feedback

    It is a bit like you just put up a picture of a meal and ask us how we like it.   If you want feedback, write a bit what you wanted to achieve, what the player should like about your game. What makes your game special from any other top down shooter. Maybe you could ask specific questions you are unsure or want feedback about.   There is almost no helpful feedback I can give you if I don't even know the target of your project. Money, know-how acquirement, experimentation?
  13. Improve on Java or preapre for C++?

    When I read about why C++ is faster than higher language X, I always hear Garbage Collectors. I don't know why it is such a common misconception. While I do agree with you in general, I want to look a bit closer at the details:   Garbage Collectors do not make the program slower, quite the contrary. If you use a GC or not, you still have the same amount of memory to clean up. The biggest difference here is that C++ cleans it bit by bit, every time an object isn't referenced anymore, while a GC does it in one fellow swoop. Garbage Collection has it's downsides, like you said when it kicks in, it tends to stop the whole program for a bit until it's finished, which isn't very nice. It also needs more memory. The Java GC can throw an OutOfMemoryException while a third of it's heap is technically free. Where a GC is better however is throughput, a program tends to spend less time cleaning it's memory with a GC than with reference counting. It finds cyclic dependancies on it's own, throughput tends to scale with the amount of memory you give plus object allocation is faster than in C++. A modern garbage collector shines when you have a program that has many objects that become garbage quickly and a hand full of objects that stay alive during the whole runtime of the program.   So no, a Garbage Collector does not in general negatively impact performance. It all comes down to what task there is to do. And that's where I agree again  . I wouldn't write a high performance math library in Java, nor would I implement a high performance streaming service in C++.   E: also with today's runtime optimization of JVM/CLR, you don't necessarily get a performance advantage when you just switch to C++. You can be quite a bit faster, but you have to know what can be optimized and how.
  14. Slender is one of the few games that truly delivers a gripping, horrifying experience. While there is a so called "Survivor Horror" genre in the AAA games industry, when it comes to scaring your playerbase very few games of late actually play in the same league as Slender. Which is weird because Slender is more of a first pencil sketch of something than a full sized painting, it doesn't even feel finished, yet it seems to do a better job. That is why Slender is a great game to analyze, because it seems unfinished and bare bones, you can see each individual part more clearly. So if you too are a sadistic human being, who only finds pleasure in torturing peasants, then take my metaphorical hand and let us wreak havoc! Links and Stuff From this point on, you should have played Slender (It is free!) at least once, also you should have read the Article about Mechanics Dynamics Aesthetics or this dissection won't be as useful to you as it could. While I am trying to not needlessly compare other games with Slender, I couldn't help myself to take Dead Space 2 in the equation. You don't need to have played the game, you just need to know the start of the game which you can watch in a Let's Play youtube video. It helps a lot to watch people play the game you are trying to take apart. In this article, I am going to referene Nova's first Slender video. He played the game while he talked about his current gaming experience, GREAT! Usability technicians have to specifically ask a test user to talk out loud, that they talk about their experience. And here on Youtube there are hundreds of testpersons doing this all by themselves. Now that we got that out of the way, let's get cracking! Concepts and Inner Fears If you are out to scare your players, you need a scary concept, a setting for your game to play in. This can be almost everything from an abandoned spaceship to a haunted castle. Many horror projects just come up with an arbitrary dark place and venture on. But you could do a lot more with it, some concepts are more powerful than others. While you can horrify your gamers on an intergalactic cruiser, you can not haunt them in their real world, the gamers can simply turn off the computer and leave the horror behind. The reason why they can do that is they probably don't live on a huge space ship, there is nothing in the real world that would remind them of their harrowing experience. Most game designers would shrug their shoulders and leave it at that, but we are too evil to just walk away. So let's have a look how Slender did it: You are trying to scare a human being, a human being with primal fears. Evolutionary speaking, we are not one step away from our tribal ancestors, we are hard wired to feel threatened or scared by certain things. You are in a fenced in forest, searching for sketches and walk away from a skinny guy in a business suit. At first glance this doesn't really look like a good idea for a horror game, but believe me it is a spark of genius. First, in order to be scared, the player needs to immerse into the world. It is hard to do that when you have no clue about what you are doing and why you are here. Since Slender is plain simple, it doesn't require a lot of exposition to get you strapped in, the exposition it needs can actually be fed to the player via gameplay mechanics - but more to that later. The other great thing about it is, we are already scared in forests at night. If you have been strolling around in the woods after sunset, you know what I am talking about. In a dark wood, your mind starts to play tricks on you, connects loose contours to monsters, your heart is beating faster, you are on alert. You have been hard wired to feel uncomfortable in such an area, it was beneficial to humans the first couple of thousand years they existed. We still have this primal fear in us, and Slender exploits for its personal enjoyment. Plus you have a chance that the player is visiting a forest at dusk after he or she played the game, the forest might remind them of their experience in Slender. Starting off right Time to immerse and the Monster Compare the beginning of Slender to the first ten minutes of Dead Space 2 (walkthrough video here). The first 3 minutes of the game are plain cutscene exposition thrown at you. After that, a guy transforms into a monster you should be scared of right in front of your face! It would be an understatement to say that Dead Space lacks subtlety, horror doesn't work when the game tells you: "Alright I am going to scare you now; BOOOO! Are you scared yet? BOOOO, BOOOOOOOOH. How about now?" Horror needs subtlety, horror needs time. That is why Slender lets you walk around in the forest for several minutes before showing anything remotely scary (other than the forest itself). When you watch Nova playing Slender, you see how he first screws around with the controls, how he tries to get his bearings before he can start concentrating on the game, and even then he needs some time before he starts connecting with the game, until Nova forgets that he is sitting in front of a PC screen and starts walking in the shoes of the protagonist, only then he starts to be sensitive to our game mechanics. So trying to scare the player before is just a waste of time and may even turn the uncomfortable setting to a ridicolous one. And we definitely don't want the player laughing at us. It also doesn't help to show your monster cinematically lit right at the start of your game. No matter how scary your monster is, no matter how hideous it is, it isn't as scary as the monsters our immagination gives us. So instead of showing the monster before you even started the game, you should rather take the Amnesia approach. Let the player see your monster rarely in the dark, from a distance, distorted and loosely shaped. An unknown monster is usually more scary than one you know. If you do it right, if you just show bits and pieces of your monster, the player will create his own image that scares him or her to the bones. If you do it right, you can startle your player by just playing a faint growl of your monster. Exposition with no words Slender does a very good job at explaining the small settings in it without any words. Let us analyze Nova's first experience with Slender's exposition: While he is busy reading the title screen, he hears somebody walking around in the grass. Note that Nova says: "What's that noise, that's a fence". Then he already has control over the game, the first thing Nova does is turn around. He sees the fence and concludes that he climbed the fence and can't go back. That's a beautiful way to tell the player something without using one single word. He also doesn't ask why he climbed the fence in the first place. Climbing a fence is not abnormal, maybe the protagonist was curious, maybe he or she tried to cut through the area and save some time on the way home. After all that, the game shows the only text message "Collect all 8 pages". Now you might think the exposition ends, but no, the game hasn't even started yet. Now the player has time to figure out the controls, you see Nova screwing around with the flashlight and searching for a sprint function. Slender also introduces the player to trails and split ups, Nova will be encountering those later on on a regular basis. At 2:40, Nova finds his first page, he hears the heartbeat sound and concludes: "I bet that means the game has started", yes it does. Now the game has finally started, but I'd say the exposition is still not finished, Slender has to introduce Slender Man. With only one page found, Slender Man always watches you from a distance, that is how you usually see him the first time. With the heartbeat sound, you know that he isn't here for your tax return, he doesn't walk towards you, but you know you should better run away. Tools of Terror The HUD of Nothingness In Slender, there is almost nothing that resembles a HUD, and it is missing for a reason. You are used to In-Game HUDs, it is a small reminder that you are still playing a game. So they did the most minimalistic HUD they could. The only thing the HUD ever tells you is how to pick up pages. You don't necessarily need to know how to toggle the flashlight or how to sprint but picking up pages is essential. Everything else though is communicated via mechanics and sound effects. With practically no HUD, there is also nothing to stop you from forgetting that you are playing a game, that you are not in a forest with a business consultant. Static The static has an obvious mechanic, it shows how close you are to losing of course. But it also has a function that is very helpful for a horror game: It obscures the monster. We touched upon this before, a monster stays scarier the less you see about it and Slender Man is no exception. If you see this guy in a brightly lit room with time to examine him, he loses his scary potential as soon as you can start laughing at him. That's why the static is a very good tool for Slender Man, it ensures that you don't get a good look at him even when he is standing right in front of your face. The Map of Disorientation I guess you have already figured out that the Level Design tries to disorient the player. But how did they do that? Now let's play try dissecting the map design. Go and play the game, don't take any pages. Instead, try to draw a map, what way you should go to get from one note to an other. Have you caught their little tricks? In general, you feel lost when you can't make any sense where you should be going, but you are only truly lost when everything looks the same to you. The developers played right into that. First by only using a limited number of models and textures for the trees and by reusing different set pieces with slight modifications. Not having the resources of a AAA studio can be beneficial too. I've counted at least two different bathroom buildings both looking very similar, but their pages are in very different places. Chances are the player has found the page in the first bathroom in the first playthrough. In the second round he or she has found the second building and searches at the place it was in the first building. This only helps to confuse the player and when they are being chased by the Slender Man. But the set pieces are not the only tools to create confusion. The paths are also an essential piece of the puzzle. Each fork of the pathway looks almost indistinguishable. Together with the similar set pieces, they create another feeling of confusion, a general lost of direction: "I am walking in circles". The beauty behind that is, sometimes the player isn't, but when he or she can't distinguish a set piece from an other, it creates this feeling. It makes it a lot harder to keep track of your path "OK, I was at the truck and I took a left, then I was at the bathroom stall. I am now at the truck again so let's take a left again" and the player finds him or herself somewhere completely different. As if the paths are changing. Sound of Immersion Let's break up the sound design of Slender, I think the sound design mainly serves three different purposes: Immersion As we concluded already, the silent beginning is here to let the player immerse. The tricky part is to make it feel organic, real. Go back and play the beginning again, before you have collected any pages, you hear a familiar chirping. Also very important, there are different sound effects for when you are walking in the grass or on a dirt path. Unsettlement and Pressure The sounds you hear when the game has started is basically for this, unsettling the player. The first sound effect you hear is the famous heartbeat track. It is good at telling the player "something is here with you, you are not alone!". As you collect pages, the game puts in more and more sound effects. First a weird humming noise, which enforces the strange feeling the player already has, that Slender Man isn't from this world. Later on, Slender introduces the howling wind noise which makes the player feel as if the whole forest is after him. It is also important to note that because the sound effects come over time, they create a feeling of "everything is getting worse, everything gets more dangerous" as you get closer and closer to your goal, it feels like you are going straight towards your doom. Which is an excellent feeling for a horror game. The Piano of Shock There is sadly very little to say about the sound effect itself. It is an awesome sound effect that fits perfectly, done. It is worth considering however, when the effect is played. The guys behind Slender did an awesome job to analyze when a player actually sees Slender Man, when it is a shock moment that needs to be enforced by this sound for maximum shock effect. Decisions of Doom While playing Slender, you are constantly making decisions. Yes you are, you just don't notice it. Go back to Nova and watch him again strolling away from the fence the first time. He says once: "Oh wait, this shit splits". Yes indeed it does, you have to make an arbitrary decision. This mechanic alone seems a bit werid, but have patience, it will become clear when we talk about Slender Man. How the Accountant brings it all together Now, let's talk about Slender Man. He alone isn't actually as interesting as for example the Level Design, but when you look at him together with the other Tools of Terror, you start to see what the Dynamics of the game are. Now, Slender Man has two completely different mechanics that they may be considered different Slender Men: The Accountant of Stalking This is the normal Slender Man, always watching you from a distance when you turn around. That is to say, almost always. He sometimes isn't there, which makes him a bit more strange and confusing. Together with the unsettling sound, he provides pressure to the player. The player doesn't actually need to sprint as much as he does, but it feels necessary. The sound effects and Slender Man together with the decisions the player has to make, forms a dynamic of increasing panic. You have to decide what way to take, one, both or none can lead you to a place you have already been. You don't know, you can't know, but you have to take the right way. The player has to do such decisions regularly, with the flashlight, with the path, even with the sprint function all while he feels increasingly uncomfortable, while he gets more and more panicky. Increasing panic means the player gets more and more sensitive to when Slender Man goes in for the kill. Accountant of Instant Transmission I guess you have noticed already, Slender Man is able to teleport around the map as much as he likes. As the game progresses, he gets more and more aggressive with his teleportations. It is a bit counter intuitive, but Slender is actualy scarier when the player knows about it. Together with the trees or claustrophobic bathroom buildings,it forms tension. Slender Man could at any moment be around the next corner. You know he is going to be behind a tree at some point, you look again but he isn't there this time, maybe around the next tree... it gives tension similar to russian roulette, it builds up together with the panic. Now the player is ready for the shock moment, when Slender Man is suddenly standing in front of him. Together with the sound effect, it forms a shocking experience that doesn't lose its power even as the player knows what is happening. The End of all Slender really ends when you collected all eight pages, but what does happen? All the sound effects stop, for a brief moment you think you have survived this, but then Slender Man takes you. No matter what you do, no matter how good you are, Slender Man will take you. This is very important, because if the game would end differently, you could feel accomplished, it would be easier to leave this unpleasant memory behind. The next time you pick up Slender, you know you are doomed, you can't get away no matter what. And that's it, Slender works with simple tools but the genius behind the game is how each piece plays together. They knew exactly what their horror experience is made of, what dynamics they needed and how they can deliver on those with their mechanics. From a Game Design perspective, horror games are extremely interesting. Because their mechanics are not fun, you don't like collecting pages. It is solely how the mechanics play together, what feelings they manage to bring up, why the gamers play Slender. So horror games should interest any aspiring Game Designer, they are a great platform for experimenting with how different mechanics play together and what effect they have on the player. In the end, that is what we are creating. Experiences. Article Update Log 19 April 2013: updated "HUD of Nothingness" 18 April 2013: Initial release
  15. No. @Avaro or Vortez: also Block Cipher Chaining is also something very similar. If you are really that interested, I can ask my Cryptology professor (I hope that's the word in english and not some strange religion) how breakable your algorithm is, but I would guess that it is pretty bad even compared to RSA. (Today, nobody would be using RSA encryption, there are just too many problems with it. But it is widely used, so until somebody comes up with the death strike for RSA. That happens as soon as the quantum computers can factorize bigger numbers). Let's examine quickly where your cryptographic power lies: You generate a hash from your password. No power is hidden here, it is the same as if you would use a random 32 bit value. Then you create a pseudo random number with the Mersenne Twister. After that you make an XOR operation with the pseudo random number. So your whole power lies in the strenght of Mersenne Twister. I don't now much about it, but it isn't used in cryptography because the next pseudo number can easily be guessed when you have some. Plus as the link suggests, when you reuse a password, you are pretty much done, which isn't a very useful cipher in the modern age. There is a reason why you tend to have a bachelor in mathematics before you specialize in Cryptography. While it can happen that you find the next super cipher, the chances that you do are very slim. There are a LOT of ciphers out there that can fit your need, RSA, modern DES algos, even modified Vignere Ciphers may be more secure. For a Programmer, there is absolutely NO reason to not use a standard cipher. They have been very well researched and are VERY secure ("the hidden world" may know how to break it, but that is always a chance for people like us).