• entries
8
10
• views
11049

Documenting the creation of an action (M)MORPG.

## Refactoring persistence

I had a bit of stress at my dayjob this spring with a lot of overtime, so I kinda lost the motivation for working on my game. This last week I've been getting into the game again. Since I've kinda forgotten a lot about working in Unity I wanted to start on something that I could do purely serverside (C#) - refactoring how storage is being handled. I was already performing database transactions asynchronously, but it was a bit cumbersome and not even implemented safely. DB tasks were (sometimes) being enqueued in a thread-safe way, but they were referencing live game entities themselves, thereby sharing them between the game logic and database layer, without any locks, mutexes or other syncronization constructs - OUCH!
Other times blocking DB queries were being issued directly by the code that handled network messages from the client (login, create player etc.). Obviously this would starve the networking thread pool for no reason, and also result in DB queries waiting on locks - also ouch...

Basically it was all a bit of a mess. I wanted to move more towards a share-nothing architecture, and a single writing DB thread. I have to give credits to It-Hare (Sergey Ignatchenko) for his inspirational articles. He's writing a multi-volume book on MMO development and has a lot of early versions of the articles for his book freely available on his website.
A couple of articles with relevance to what I'm doing atm:
http://ithare.com/eight-ways-to-handle-non-blocking-returns-in-message-passing-programs-with-script/

I also wanted to be able to batch up logical transactions in a single database transaction, enabling me to:

1. Issue batch updates and inserts

2. Cut down on the amount of transaction logfile thrashing

This would provide me with a higher potential throughput at the cost of some latency which doesn't bother me as long as the player is able to log in within a few seconds. The game itself never waits for database transactions anyway, so these "few seconds" are an acceptable upper limit for latency of DB transactions.

In addition to that I wanted an architecture that could support moving the DB server to another computer.

This increased latency would of course make any blocking calls to the DB an even bigger No-No than it already was, so reading the articles (among others) mentioned before inspired me to have a single thread for persistence, with logical database transactions being put into a queue (coupled with continuation lambdas) to be batched up and executed.

So I made a persistence server with an app-level cache where transactions can be queued and a single DB update thread that just pulls transactions from the queue, performs them, commits the batch and returns the results of the transactions to the caller.
To be honest this has not (yet) been implemented as a server but just as an in-process object with an Enqueue() method and an OnResult() event. I don't know which communication protocol I would use - Rest, homebrew TCP, something totally different? But I don't want to bother with it now anyway since it is not important at this stage and whatever I choose would probably be changed anyway. I also decided not to decide on a storage technology. My old DAL layer used SQLite but I want to get rid of that third part dependency for now so I'll just be saving JSON files to avoid having to update schemas with frequent data changes. I'm actually not even doing that yet - I'm just using the app-level cache itself which means nothing is saved on program exit
I then made a PersistenceController that will run on the gameserver.
It has methods like LoginAccount, UpdatePlayerCharacter etc. In addition to the normal parameters they also take Action/ Action<T> parameters so you can supply a lambda expression to act as a continuation on completion of a transaction.
These methods just enqueue the transactions, letting another thread handle the sending of the transactions to the persistence server. This transaction sending is also being done in batches from the controller to cut down on ping-pong - these batches do not nece
Mutable reference-type parameters (basically anything heavier than a String or an int) are mapped to DTOs before being enqueued, so that no live game data is shared between threads.

Time for a little code, showing how the handling of a request for availability of an account name resulted in a blocking call to the database. Fine for serving an individual client with low latency but blocking a thread on the iocp pool for several milliseconds? A horrible idea in a multi-player game.

Notice SendMessage doesn't in fact send anything - it just queues the message for sending.

public override void Execute(NetworkMessage message, ServerConnection con)
{
bool accountNameAvailable = !PlayerDataMapper.AccountExists(name);
con.SendMessage(ServerMessageFactory.CreateAccountNameAvailableResponse(name, accountNameAvailable));
}

Here is the same code using the persistence controller, with the continuation as a lambda expression.

        public override void Execute(NetworkMessage message, ServerConnection con)
{
PersistenceController.AccountExists(name, exists =>
{
bool accountNameAvailable = !exists;
con.SendMessage(ServerMessageFactory.CreateAccountNameAvailableResponse(name, accountNameAvailable));
});
}

## The importance of a starting area

A decision has been made - instead of focusing primarily on coding features the immediate goal is prototyping a starting area.
Obviously this is a very important thing to get right - it might actually be the most important area of the entire game since this is where players decide within the first 15 minutes if they want to keep playing the game or look somewhere else.
I thought about the games I've played in the past, and how they handled things. It's important that players are given a chance of action without too much tutoring - and especially without too much OOC, like a "Tutorial Island" would be - yuck.
Not wanting to start the player in a city I've decided to dump the player in a campground in the middle of a forest, with opportunities to find a few easy opponents before being introduced to the first "dungeon" - a small cave.
I haven't exactly made the cave yet... the trees and the grass kinda sucks, and the opponents are still those pesky skeletons, but at least it's a bit better than my first tech-demo area.

Basically the idea is to limit the immediate scope of the huge task this game is to a small "vertical slice" that will also serve well as a demo if I want to go for greenlight.

## Skeletons go down!

But not without a fight...

A bit of gameplay video + code snippet

Sunday was to be the big day when I finally killed the first skeleton. Then I moved on to have skeletons fight back and kill players.
Still the AI simply walks around randomly while checking to see if any players are in melee range and then stops to hit them.
It actually surprised me how easy it was to make the skeletons attack and how few lines of code it took. Having the engine handle the death of a player character, "respawn" him (resetting health and position) and then making sure messages are sent correctly to the client was actually more of a challenge: even though that also took only a few lines of code it needed a bit more tinkering that I expected.
I'm thinking about trying out a behaviour tree for the AI but I really don't think I need it yet.
Instead I'm structuring my behaviour code in such a way that it can be converted to a tree with only a little work. Explicit methods are a bit easier to work with and I also expect them to be quite a bit faster. But code reuse is harder because the "tree" formed by the calling structure is hardcoded. While reuse of branching is just as easy you can't just make a new tree based on an old one by substituting a few child branches.

An example of the simple AI - a custom Behaviour class that is attached to an Entity - already somewhat structurally prepared for added pathfinding logic.
Notice this is server code, but I still have an animation class. It has a duration and an Animate() method that is called each game loop as long as it isn't finished. No actual "animation" actually takes place on the server of course but a running animation stops the player from thinking, and also enables running code at specified points of time during the animation. That way weapon attacks can be implemented that do hit checks halfway through the swing - or at the end - or do collision checks every frame while taking care not to hit the same entity twice.
In addition to MeleeAnimation I also had to implement BeenHitAnimation (simply prevents the player from thinking or acting while it plays) and DeathAnimation.

[source]
public override void Think(GameUpdate update)
{
if (Attack(update))
return;
if (MoveToPlayer(update))
return;
if (_entity.NextGeometry.Speed == 0 || _entity.Geometry.Pos.Equals(_entity.Geometry.Destination) && (_entity.Path == null || _entity.Path.Count == 0))
{
Point2D newDestination = GetRandomPosition();
try
{
paths++;
_entity.Path = _entity.Section.Spatial.ConnectivityIndex.GetShortestPathAStarWithGridLineOfSightMaxLengthGridReflex(_entity.Geometry.Pos, newDestination, maxDistance);
if (_entity.Path == null)
failedPaths++;
}
catch (Exception ex)
{
Console.WriteLine(_entity.Geometry.Pos + "->" + newDestination);
Console.WriteLine(ex);
}
_entity.NextGeometry.Speed = _entity.MaxSpeed;
}
}

private bool MoveToPlayer(GameUpdate update)
{
return false;//TODO: Implement
}

private bool Attack(GameUpdate update)
{
Entity victim = FindVictim();
if (victim == null)
return false;
_entity.TurnTo(victim.Geometry.Pos);
_entity.Animation = new MeleeAnimation(_entity);
return true;

}

private Entity FindVictim()
{
var victims = _entity.Section.Spatial.GetEntities(_entity.Geometry.Pos, _entity.AttackRange);

foreach (var victim in victims)
{
//check if enemy is targetable. Don't hit yourself...
if (victim == _entity || victim.IsDead)
continue;
if (victim is PlayerCharacter)
return victim;
}
return null;
}

private Point2D GetRandomPosition()
{
var grid = _entity.Section.Spatial.GridIndex;
int x = (int)(_entity.Geometry.Pos.X);
int y = (int)(_entity.Geometry.Pos.Y);
int xMin = Math.Max(x - maxDistance, grid.xMin);
int yMin = Math.Max(y - maxDistance, grid.yMin);
int xMax = Math.Min(x + maxDistance, grid.xMax);
int yMax = Math.Min(y + maxDistance, grid.yMax);

Point2D pos;
do
{
pos = new Point2D(r.Next(xMin, xMax), r.Next(yMin, yMax));
} while ( VectorMath.DistanceSquared(_entity.Geometry.Pos, pos) > maxDistance * maxDistance ||
!grid.IsOnMesh(pos));
return pos;
}

[/source]

I also went ahead and bought the Barbarian RPG pack from InfinityPBR - a big thank you to Fidelum Games for pointing me in that direction.
Mecanim is totally new for me and a bit confusing so I haven't worked on incorporating the characters into my game but it's gonna be great when it happens.

## Get work done and kill a skeleton - 2017 New Year's Resolution

Since my last post I've unfortunately had a pretty quiet period without getting much work done on the project. I simply wasn't able to find the motivation to get things done.
I recently got a new dayjob much closer to my home which saves me 10 hours of driving every week - I really ought to be able to translate this into more development time...
Anyway - in my last post I mentioned that I wanted to use a data-driven approach. The plan was to use text files or XML to load definition files for classes, items, NPCs, spawners etc. but I have decided not to bother with this at the moment because it's a bit of a pain to write parsers etc. for all of this. Since I'm gonna be the only only one editing all of this in the foreseeable future I've decided to pretty much scrap that idea and instead define all of this in code. Eventually of course I want the spawners to be exported from Unity using editor-only gameobjects so I don't have to hard-code coordinates but for now I can live with it. All of the other definitions could in principle stay C# until the release of the game.

So - back to work on the fun stuff.
The milestone I'm heading for at the moment is still being able to actually fight skeletons in the game.
This includes:
Performing attacks serverside - showing animations clientside (done)
Handling collision detection on server (work in progress)
Hitpoints - and stats in general (work in progress)
Damage dealing (TODO)
Death (TODO)
Simple NPC Melee AI using fluent behaviour trees (TODO)
Simple Player "AI" (TODO - Clicking on a monster should make the player go to the monster and hit it)

## Die Skeleton!

Well - not quite yet. There is no such thing as damage or death implemented yet - and also the skeletons have no AI except for picking a random point on the map, finding a path, then walking to it.

Still it's more than what I had a week ago, and development is progressing steadily.

Right now I'm working on the basics of how I want my stats to work and I just made the base of a layer-based stat representation. I really want to get to the point fast where I can actually damage and kill the skeletons but at the same time I want to do things properly without cutting too many corners. Unfinished work is ok but I want most of my code to form a healthy base on which I can continue building.

Next stop should be damage mechanics - but at the same time I really want to get started laying a base for data driven entity definition.
Once I have monster/class definition files down and as soon as I have killed my first skeleton I will move on to AI and try to build a simple AI behaviour tree.

## So Far So Good...

So far most of the development has been on the server side and implementation of the protocol to be used.
The current state of the project is that you can create an account, log on, create a character, choose a character from your list and finally start a game.
The only thing you can do in the game though is to walk around and see a bunch of enemies walking around randomly. I'm actually developing a simplified WinForms client and a Unity client in parallel - both clients share a lot of code. This allows me to test changes quickly at this stage of development, although it might be something I have to stop once I really get some Unity work done - still to be decided. At the moment it's serving me very well because I'm working with server code, and code shared between the 2 clients.
In the past week my focus has been on state update messages sent from the server to the client.
In particular a basic "interest manager" has been implemented.
The interest manager is responsible for maintaining knowledge about what the client knows - and what the client should know.
It keeps score of the entities already known by the client, and makes sure that unknown entities are published to the client if they come in range. It also takes care of telling the client if an entity comes out of range. Known entities are kept updated if speed or destination changes - but no update is sent when the position changes because of the speed already known to the client.
The following events are sent in the update message:
Spawn: Tells client to construct an entity
Update: Tells client an entity should be updated.
An article worth reading on the subject: http://ithare.com/mmog-world-states-and-reducing-traffic/

Another area I've been working on is "the grid" serverside - a grid of square cells indexing both the entities of a zone and the static obstructions/walls etc. of the zone itself for fast lookup.

I'm having a hard time deciding if I should get started on Unity animations and dynamic object loading now, or if I should continue serverside and implement combat moves, serverside colliders and simple enemy AI.

## An Mmo? - Really...

Well... not really.
I'm not envisioning creating the next WoW-killer.
A MASSIVE multiplayer online game is obviously outside the scope of a lone wolf as myself who is only prepared to spend a few thousand \$ on assets (in the foreseeable future), and isn't able to commit to working full time on the project.

I just don't know any other word that covers a shared online world supporting hundreds (or perhaps a couple of thousands) of concurrent players.

I guess "graphical MUD" would in fact be the best term - but it might discourage players who might think it's just a shallow graphical layer on top of a text-based game - or players who don't even know what a MUD is.

So what is the vision for this game?

The technical side:
The game will be a 3D game rendered in an isometric-like view (Like Diablo 3). Unity is the chosen game engine for the client, while the server is written from scratch using C#/.Net. Performance benchmarks will determine at a later time if Mono/Linux or .Net/Windows is to be used for production servers.
While the game is rendered in 3D the logical spatial model is just a 2D plane - this simplifies a lot of server logic but of course also restrains the client - no 2 locations can be on top of each other.
The server will be fully authoritative, but the client will implement smoothening logic to hide lag to some degree.
The server code must take full advantage of multi-core architectures. The game will not be finished tomorrow. Quadcore dedicated servers are already accessible at a reasonable price - octacore might very well be the norm soon. Contrary to the golden rule of "code first, optimize later" I believe this is better handled in advance by having lock-free and low-lock concurrency in mind when developing.

Gameplay:

The gameplay will be very Diablo-like, except taking place in a shared world.
Combat will be real-time, and collision detection will be used to determine hits, as opposed to the typical WoW "homing missiles" where hit/miss is a simple dice-roll determined as soon as an arrow is fired. This means a skilled player will be able to sidestep an incoming missile.
Obviously this will put a strain on the server - but it is an important feature and if it means limiting the maximum allowable number of concurrent players on a server to a few hundred then it's a price I'm willing to pay - time will tell.
A lower player count will also increase the chances of meeting someone you've met before.
Monster density will be lower than Diablo and individual enemies will be challenging - don't expect to take on hordes of monsters (If you want to live). This is a conscious decision for gameplay reasons - although it might alleviate some of the server strain the real reason is that I want the players to be forced to concentrate on the immediate oppenents and use their wits to survive.
Powerful items are very important if you want to be a powerful player, and are mainly obtained by killing monsters or from trading with other players. Basic items that will get you through the first levels along with somewhat adequate items for higher levels will be available in shops.
Account-bound and character-bound items will not appear - at least not as drops. Perhaps some quest rewards should be personal but it's not a priority.
Trading is crucial in developing a community - therefore drops will not be tailor-made for the class you're playing.
Main "official" currency is gold. Inflation is probably something that must be accepted - I won't mind too much if the economy in player trades shifts towards bartering (think "Stone of Jordan").
Crafting is not a planned feature - although it's on the wish-list it's not important enough to delay a release.

Clans on the other hand is something I really want to see happening as soon as basic gameplay is implemented. An online multiplayer game depends on the community and I really want the game to be a place where people feel they belong.

## Why an MMO?

Greetings weary traveller...

Something like the above line was how my entry into the world of online gaming looked like, when I first discovered the internet, and with that, MUDs, around 1998.
For those of you who weren't around back then - a MUD (Multi User Dungeon) is a text based multiplayer role-playing game, played over the internet via Telnet or specialized MUD clients.
My fate was sealed - too much time playing MUD and too little studying (chemistry) caused me to pursue software development instead.

At the same time I always knew what I really wanted to do was make games.
I started writing MUD zones - one made it to my mud of choice - "Land of Legends" - but unfortunately it closed down for good several years ago.
At the same time I also started trying to write an online game on my own. First a MUD, but soon I wanted to go graphical and started working on a 3D isometric engine. This was around the year 2002, and I was programming in Java.
I lost motivation on and off, tried a little C++ and OpenGL and as I was using .Net professionally also took a look at XNA.
One of my main reasons for giving up several times was how hard it was to get acceptable game art. I knew I would never be able to draw or model an even remotely acceptable monster, let alone a main character...

Then I discovered Unity and the Asset Store - it is now perfectly feasible to make a complete game demo with high quality graphical assets bought for tens of dollars, as opposed to the thousands of dollars one would have to spend to have a 3D artist create something custom.
This gave me hope - even though a finished commercial game would of course need a real investment in custom assets a complete and polished demo can be made for a few hundred dollars.

I also had a personal breakthough in pathfinding. Where I have been implementing A* several times in the past it has always been on graphs - discrete node based structures. How to do this in a continuous space like a polygon puzzled me.
Then one day when reading on the internet I found a few ressources that explained it to me, and I quickly implemented a simple polygonal pathfinding algorithm.