Jump to content
  • Advertisement
Sign in to follow this  
  • entries
  • comments
  • views

About this blog

A journal describing my journey through creating a Game Engine alongside my games.

Entries in this blog


WIP: Week 6 -- Node.js, Scripting, Networking

Hey all!

As a continuation of my last post, I decided to move onto creating some debugging/feature tools, as well as implementing a scripting language. This was basically a successful 2-for-1 special!! For me, networking and general client/server/browser work would be much simpler to do in Javascript, and should be performant enough. Though I have littler experience with Javascript, the experience I've had and the research I've done seems like it'll be a good choice. I also had chosen the language as my scripting language, since it's simple to learn and easily applicable to a variety of platforms and peoples. I've also been looking for excuses to use/learn the language

This past week, I downloaded Node.js, which is backed by Google's V8 Javascript Engine. The code is definitely a bit convoluted, but seems very cautious and safe, and makes sense none the less! I delved right in, and got some simple examples running:
Simple Chat Program, all Javascript. Uses browser for the clients.
C++ to Javascript code, where I could use C++ objects and call C++ methods from my Javascript code. Easy enough!
Combined the two above: Chat program, where messages got printed to the console and appended with the sender's information on the C++ side.

The code looked like the following:[code=:0]// C++// Point example.//...static v8::Persistent constructor;// Everything's static except the internal field of the class instance.static void init(v8::Handle exports) { v8::Local tpl = v8::FunctionTemplate::New(New); tpl->SetClassName(v8::String::NewSymbol("Point")); // This internal field is a pointer to the class instance. tpl->InstanceTemplate()->SetInternalFieldCount(1); // These methods need work, handling different method parameters is HARD. // These methods add to the "New" method a function pointer to a callback. // These method callbacks grab a "this" argument, the parameter arguments, // then grab the underlying object and call the same method, with the given // parameter arguments. I've hard-coded the arguments for now, but I need to // template-ize these methods and get it all working generically. addMethod(tpl, "getX", getX); addMethod(tpl, "getY", getY); addMethod(tpl, "setX", setX); addMethod(tpl, "setY", setY); constructor = v8::Persistent::New(tpl->getFunction()); exports->Set(v8::String::NewSymbol("Point"), constructor);} //...void InitAll(v8::Handle exports) { JSPoint::init(exports);}NODE_MODULE(addon, InitAll);// End C++ // Javascript var addon = require('./build/Release/addon.node'); var p1 = new addon.Point(129576, 165.567); var p2 = new addon.Point(124.9, 235.1423); console.log("P1: x=" + p1.getX() + ",y=" + p1.getY()); console.log("P2: x=" + p2.getX() + ",y=" + p2.getY());console.log(p1 instanceof addon.Point);// End Javascript
These examples were brief and simple, but they got the main parts I needed to figure out down pat. From here, the C++ objects and methods lay in the engine, and Javascript runs the game and networking logic. The last issue I need to solve is creating an extensible wrapper for classes that I can wrap for the Javascript code -- surprisingly more difficult than one might think I hope I can get there in the next week or so!





WIP 5 -- Transparency Woes, Debugging and Refactoring


[font=arial][size=2]So I posted the mid-week update of my transparency work earlier this past week. The results looked nice, but shortly afterwards, I realized I had some memory issues and it became a mangled mess. What was happening was I had my INDEXARRAY pointing to the array of indices stored in a VertexBufferObject in the mesh. When I passed that same array to itself for copying, weird glitches and a lot of memory access violations occured, mangling the meshes pretty bad. I cleaned that up, and fixed some of my sorting code to only perform sorting when crossing a block boundary (x-, y-, or z-plane). There's quite some optimization to be done there, but for now, it's good![/font]

[font=arial][size=2]Two things popped up during this process this past week however, and I need to address those before getting too deep into the engine. The first is Debugging. Currently, I have the most simplest form: logging. This has been fine so far, albeit with some print statements here and there while ironing out details, but when it got to the point of debugging some of the finer issues of transparency debugging, I realized my setup isn't ideal. The main issue I had was the lack of a "needle" to poke around into the executable. I wanted to peek in, grab a variable, change it's value, and see what happens. I wanted to set up the game at a certain point and see how things traverse from there. These are all useful things for debugging various features, now and later.[/font]

[font=arial][size=2]I also ran into the issue of transparency causing such a big change to the engine. I hacked this solution together; great! But now I have to clean it up. That'll take some time! I need to work on a better habit of programming in order to handle the situations of big structural changes with better ease. I'll have to try different things and see where I end up.[/font]

[font=arial][size=2]All in all, this experience was a wake-up call. I need to work on a couple tools before moving forward (luckily I've got quite some interest in these tools!) and then focus back on gameplay. Looking ahead at some big changes (World generation, chunk state management, disk handling, etc.) this change and these tools can be very instrumental in creating a good system. Heck; what's an engine without it's tools and flexibility???[/font]

[font=arial][size=2]And here we leave at what to work on. I'll be working on incorporating an HTTP server into my engine, in order to flesh out debugging and feature addition. This will be a simple data server coupled with a couple HTML pages for displaying and manipulating data. These have been done before and I'm looking for good libraries and examples, so I'm hoping this won't be a terribly long diversion! Once it's done, onto more memory related things (disk I/O, paging, world gen, etc).[/font]





Mid-Week Update: Transparency Woes (Pictures!)

Well....been working on transparency, and here's some of the hilarious results I've had through the process:

Aaaaaannnndddd.....that last one shows static sorting -- one round of sorting on generation, and that's it. Looks not too bad! When I work with movement, I've gotten a lot of flickering issues (see the crazy 4th image!); just some bugs to fix. Until next time!





WIP: Week 4 -- Transparency


I've never done transparency before. I've dealt with it in extremes (either opaque or completely transparent) but never anything in between. Because of that, I definitely had my work cut out for me this week. I posted in the forums, as well as on a couple of other places, looking for advice in simplifications I could make to the normal process of just sorting all transparent faces.

What I ended coming up with was two things: I can limit the sorting two 8 different options (+x+y+z, -x+y+z, +x-y+z, +x+y-z, -x-y+z, -x+y-z, +x-y-z, -x-y-z), and I can sort only within chunks, since my chunks are already being sorted for rendering. The first option requires one calculation of an index, making distance calculations and such unnecessary and computationally expensive compared to an index calculation, and the other makes it easy to sort in parallel -- pushing these tasks to the thread pool during the pre-render stage.

Creating this algorithm, I noticed the effects of it didn't work well with greedy meshing! That wasn't a good thing....what ended up happening was the sorting didn't take into account that some faces extend a decent distance, and that issue unfortunately led to the failure of this simplified method. The method assumes every face is its own, and isn't merged with other faces around it. Instead (for now), I am keeping the per-chunk sorting, but depth sorting using a distance vector form the player to the face. The structure is shown below:[code=:0]typedef struct INDEXARRAY { //pointer to actual data in VBO (CPU side) unsigned int* _indices; int _length;} INDEXARRAY;typedef struct TRANSPARENTFACE { // Indices for a face are in groups // of 6 -- from _indexStart to // _indexStart + 3, inclusive. unsigned int _indexStart; // hopefully center-point will work fine // for sorting! If not, I can expand here // for something that works better. With // greedy meshing, I can also limit the // extent faces merge to try and help too. float _x, _y, _z;} TRANSPARENTFACE;class TransparentMesh : public Mesh {private:protected: INDEXARRAY _indexArray; std::vector _transparentFaces;public: void addTransparentFace(unsigned int firstIndex, float x, float y, float z); void setIndices(std::unique_ptr &&ind); void sort(Camera& cam);};
My current meshing strategy requires the following strategy:[code=:0]// 1. std::vector for each thing (position, index, color, normal, lighting, etc.)//2. get Type* and copy into VertexBuffer (stores in a Buffer class, // much like nio.Buffer in Java)//3. Sends unique ptr of VertexBuffer to the Mesh instance (passing ownership,// and you can still get the data with get() functions).
From there, the [font='courier new']TransparentMesh[/font] class overrides the virtual method [font='courier new']setIndices[/font] to make sure to extract the indices and store the pointer to the array in the [font='courier new']_indexArray[/font] variable. Now when sorting occurs, at the end of the method, it marks the mesh and the indices VertexBuffer for updating during the rendering method! It's a little hack-y, but it works for now.

Also, the mesher needs to call the [font='courier new']addFace()[/font] method to add the face information used in sorting. This is useful and allows for users to decide when/how/if they want to sort faces, meaning they only use the method when they do want to sort faces. Since this is a voxel engine, i doubt this code will be touched much anyways, but I'll come back to it later when/if it presents an issue.

At this point I haven't written the depth-sorting by distance yet since school started this week, but I should have time after I finish my homework to get it finished, and hopefully get to more stuff this week, like terrain generation (in the works already!) as well as the beginnings of something new...? I'll have to see what I want to do next!

Until Next Time,




WIP: Week 3 -- Threading, Chunking, and Optimizations!

Hello all!

This week, I begun work on the threading model for the engine. Since having a thread for each component (rendering, physics, main game loop, input, etc) wasn't feasible and left a lot to be desired for the engine, I changed it to a Fork/Join or Job model. This allows easy use of a thread pool, which is easy and intuitive to use. I created a ThreadPool class that manages the queueing of tasks and the management of threads.

This was new territory to me, since I wanted to play with C++0x features like condition variables, return_type, std::futures/std::forward, etc. I also learned about more than the typical lambda function, and made the ThreadPool class accept any type of method: class method, static methods, functions, lambda expressions, etc. The interface looks like the following, and returns an std::future for the result:[code=:0]ThreadPool::create(std::thread::hardware_concurrency() - 1);// ...//auto = std::futureauto result1 = ThreadPool::enqueue([]{return "test"});//getInt returns 42.//auto = std::futureauto result2 = ThreadPool::enqueue(&getInt);// ..std::cout
I have the threads sleep on the condition_variable in ThreadPool until there's tasks for them to complete. Each call returns an std::future, from which the result is received.

Once I had the threading model, I started using it! I chunked out the world, and threaded different aspects of it (loading, unloading, generation, etc.). Below is an example of it working:

Lastly, I noticed some issues with my mesh generation. It was taking 125-160ms, and that is WAY too long -- my Java version managed to generate chunk meshes in about 30ms. After a long time looking through the code, unrolling loops, etc., I was able to get it down to a whopping 22-25ms!! Now I can generate chunks (threaded) about 2-4 per frame, rather than one at a time! AND since I've got a Job framework going, I can have mesh generation take most of the frame time, and have the renderer grab the result and render it at the end!

This next week, I hope to start putting together a noise library for creating terrain. Alongside that, I need to start working on transparency (UGH It's already giving me a headache...) in order to handle water, liquids, and translucent materials.





WIP: Week 2 -- Meshing, Shapes, and Worlds!

This past week has been quite productive for the engine! I updated my RenderingEngine to support rendering buckets, which is basically a set of efficient priority queues to sort meshes every frame. I also started working on frustum culling, to help cull meshes using their bounds from going through the rendering pipeline. Currently the supported bounds are AABBs and spheres, but I hope to also add OBBs as a "originally" supported bounds. Anything else shouldn't apply to a basic voxel engine yet, anyways!

I then started working on meshing chunks for the engine. Though I've done this already on my Java version of the game, I needed to change some things in order to *mesh* well with my engine (pardon the pun). With that in mind, I rewrote my greedy mesher to be not only modular, but more efficient and data-driven. It seamlessly molded into the voxel corner of the engine, and I now can mesh chunks efficiently, with AO, as can be seen here:

Not only that, but I began creating methods that create circles, spheres, boxes, and other simple shapes in order to make modeling easier. Yay!

Next I wanted to tackle what puts me off these types of projects all the time: world management. For some reason, I can logically break apart that topic into a large group of manageable tasks, but I've never managed to do more than meshing management. Now that I have an engine structure I'm happy with and I've got a momentum carrying me along, I hope I can accomplish just that! I haven't done much yet, so there's nothing to show, but the general layout of a world's "tick" looks a little like this:[code=:0]//...In VoxelWorld.cppvoid tick(float dt) { // Update Chunk Management Logic // --> Chunks can move between different states: // a. Loading (Fire event with chunk pointer, // wait until loaded ) // b. Setup (generate, decorate, mesh, etc) // c. Physics/Water/Player/LOD/etc updates // d. Unloading (Fire event with chunk pointer, // wait until unloaded ) _chunkStateManager.updateChunkStates(dt, this); // Update visibility // --> Read front to back for solid rendering, // read back to front for transparent rendering :) // _chunkSorter is a modified std::priority_queue // (underlying vector for iterating ease). _chunkSorter.update(_camera.getPosition()); // Update player inputs and movement. _player.update(dt);}//...
Whoo!! The nice thing about this is all cross-thread communication is done through the EventManager, and each event is fired asynchronously, making everything easy on the developer. Having [font='courier new']_chunkStateManager[/font] be an object has allowed me to try different options for state management, like dumb systems that are step-based (run 'x' amount of chunk updates per frame), as well as smart systems that are AI based (extend chunk updates in a smart fashion that allows for more realistic results that should take into account how it's taxing the computer). All of this together creates a flexible system that I can use with ease.

This next week, I'll be continuing with the World Management in the game, and will focus on possibly scripting events following that. I also want to start thinking about vegetation decorations and weather effects, in order to bring the world alive.

Until Next Time,




WIP: Week 1 -- INI Files and Resources!


What I hope to accomplish are weekly updates about my engine build, and go over some of the things I've been working on lately. This first one will cover what I did today, but future ones will cover an entire weeks worth of work.

Since last time I created an event system to tie my engine together, I began this weekend working on beginning my resource allocators. Since I wanted to start small and work my way up, I began with INI file types (with my own twist).

In reference to the voxel game I'm working on, I moved my voxel types from classes into a resource to load at runtime. No longer do I have to recompile my code just to change the color of a voxel; I can now fix the .conf file and re-run the executable! It was incredibly satisfying to get this working, and it allows an explosion of different voxel types to emerge rather quickly.

The BlockDef.conf file looks as follows (roughly):## Block Data information is in this file. Format (copy, paste):# [BlockName]# id = (int)# transparent = (bool -- true, false)# material = solid, liquid, gas (string)# color = (4 length array of floats)# [AirBlock] id = 0 transparent = true mat = "gas" color = {1, 1, 1, 1}[NullBlock] id = 1 transparent = false material = "solid" color = {0, 0, 0, 1} [StoneBlock] id = 2 transparent = false material = "solid" color = {0.2, 0.3, 0.4, 1}
For those of you that know the INI file structure, you'll notice one big difference -- there's arrays! I added this simple functionality in order to make defining colors, texture coordinates, and other such things easier to add. Overall it looks solid and is simple enough for newbies to learn!

In the engine, the results are viewable using nested C++ hashtables. Here's an example for gathering the color and name of a voxel from the results:[code=nocode:0]INIFILEMAP map = INIParser::parseFile("Resources/Blocks/BlockDef.conf");for (auto kv : map) { std::string voxel_name = kv.first; INIMAP voxel_data = kv.second; //... std::vector voxel_color = voxel_data["color"]; //...}
In the above code INIFILEMAP is the overall nested hashtable, and INIMAP is the interior labeled data from a specific entry in the INI file. Pretty sweet!

I'm not sure of what to work on next, but I've got the following list of things I can work on currently:
rendering buckets
resource management
python scripting (or lua.....not 100% sure yet on the pros/cons)
start physics engine

Next time we'll see where I'm at with this list!




New Journal, Old Project, Present State.

Hello Everyone!

I started over my journal entries based on many things, but mainly this: I have recently altered my view on what I enjoy doing. I thought I wanted to make full-fledged game (and I do!) but the inner workings of things behind the scenes and closer to the metal has drawn me in more than anything else. I enjoy taking things apart, figuring out how they work, and putting it all back together in my own way.

My journey through programming games and computer graphics in general has been an interesting one; I've seemingly worked my way form the bottom up rather than the top down. My first program was a 2.5D game, much like DOOM (Yay raycasting!). From there, I focused on 3D programming techniques and created some 3D software renderers. I even created some terrain editors using those renderer's, and learned a lot about what goes on in the GPU.

After that, I moved on to using OpenGL and began looking into making a voxel game. This has been me for the past year and a half or so, and it's been amazing! I've learned a lot on pushing a ton of data to the GPU, dealing with simplified physics and collision detection, and work with existing engines (JMonkeyEngine and Unity) in order to see how it all comes together.

Lately, I've been delving deep into the issues I've had with these engine's I've used. As nice as they've been to get off my a** and get some decent work going, they also set some limitations to the work that I've been trying to do. As I've worked deeper into why that is and what needs to be done to fix the problem for myself, I learned more and more that I don't truly understand what goes on under the hood, and I wanted to understand it all. As a lesson to myself, I thought it would be good to try to do it myself -- learn about it all, put it all together, and more importantly, rely on others only for code that I already truly understand and write everything I don't understand. Therefore I set off on my quest to start a game engine. This wasn't without doubts, however. I've focused on my other work and started this project on the side; and only now am I sure that this is something that can be completed, something I can follow through on.

Now let's get to the meat:

Third Party Libraries:
libPNG and other simple I/O libraries
GLFW for window, context, and input management
OpenGL for rendering with the GPU
GLEW for handling OpenGL version issues
GLM for vectors, matrices and math

With that, I started designing the layout of the engine, based off of the current and past engine's I've used, and the plethora of reading I've done over the past four years. I decided to do a component-based design, all connected through a multi-threaded event system. I broke down the structure into the following parts:

Game Engine Structure:
Core - The core functionality of the engine. Math, System, Logging, Input, etc.
Rendering - Window and OpenGL management, as well as culling, shaders, effects, and anything else connected to rendering.
Physics - Adding functionality to a game to make it more realistic. This can include collision, gravity, force and moment generation, etc.
Game - This isn't a game engine component, but an actual game! This is helpful for testing and prototyping engine parts before adding it to the engine.

Something not included here is external modules I'll make as I'm working on stuff; for example, I've created a Voxel external module for the voxel work I'm doing, and this allows others to reuse that module if they also want to make a voxel game -- like an engine, but not core to ALL games. These external modules fit better with certain genres.

And with this general structure, I began work! I'm now about 6 months into this engine build, and I've got many of the core systems fleshed out. I created the bare bones for the Rendering Engine and the Core Engine, and the voxel/game portion is already rendering a world onto the screen that the player can explore. Currently, I just implemented the EventManager class and with the lambda function capabilities and templated member functions for classes, I've laid the groundwork for an event system capable of handling events and listeners of ANY class -- something I'm very, very happy with. It looks like this for implementation:#include #include "EventManager.h"typedef struct Event { int a;} Event;class Listener {public: void operator()(const Event& e) { std::cout (listener); Event event; event.a = 10; EventManager::fire(event);}// OUTPUT://// >>// >> Event Recieved! A = 10// >>
In future posts I'll cover more details about the engine; for now, this is a good introduction. I look forward to see where this goes, and I'll be back soon with some posts about the current implementation I've got with the engine!



Sign in to follow this  
  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!