About this blog
Attempts at creating an MMO environment that can be reused.
Entries in this blog
I finally decided after much thought to show a screenie of the over all control system. I've been starting the opposite way in developing a game system by designing the backend first.
Well, in order to have a backend of servers and processes running across multiple server, you have to have a way to control them. So here the screenie and I'll post later on tonight or tomorrow with a detailed explaination about what, how, why.
Over the past year I have designed, redesigned, and trashed mutiple backend systems simply because of the lack of planning and control. I have finally hit upon a method that will lead my project into success.....hopefully. For those of you that might remember, I have been working on a backend system that will support mulitplayer online games. My dream was, and still is, to be able to use this flexible design to help independent game developers have a place to start with a strong backend.
The new method that I'm pursuing at the moment is to design a monitoring backend system that will control, monitor, and support the necessary components for a strong backend. So I've pulled back in my development and started the logical design (using Visio) with a full design document. I've made leaps and bounds using the logical design because I can create scenarios, run them through the system logically and have been able to determine bottlenecks, possible issues, and elimminated several flaws. Since starting this method at the beginning of July, I'm now finally ready to start the database design for the system. I'm still planning on supporting several popular database engines as to give the developers a choice in what they want and can afford to power their backend systems.
Because this is all usually written in documents, I have no pretty pictures, but I will keep the journal up to date every week in case someone is reading.
But in the future, I may issue limited screen shots of the system design as long as it doesn't give too much away. The pretty screen shots will come when I finally start developing the game server and need a base client for the front end.
Thanks for reading this dry and uneventful entry, but I promise it will get more exciting as I move along the design and coding steps.
Writing a system that requires maximum flexability surely is very hard work, but I'm getting closer. I'm planning on having the management tools for the clusters (realms), servers (physical) and processes finished by 2/1/2006. I'll be posting a screenshot with a listing of features this weekend. Hopefully someone will take a look and possibly have some interest in the project, but I doubt it. heh
So far we have the ways to monitor the clusters, server and processes. I'm working on getting the control application defined and coded right now.
After this, I can finally move into the design of the servers (software) that will support MMOArchitect.
Thanks for viewing
The team (myself and another) discussed the new architecture and actually came up an even more robust modified version that would allow for a very dynamic load balancing system for the back end. But it's so good that I don't want to even detail it until we can get it completed and hopefully some sort of intellectual property protection. So with that said I'll post again tomorrow and will expand on other areas thet we are currently woring on.
Again Thanks for reading and comment if feel so inclined!
Well, after posting yesterday, I picked up a development book I hadn't read in a while, Massively Multiplayer Game Development 2, and noticed an excellent method of monitoring the processes and providing a great fault tolerance for keeping your processes up. While I will have to modify the base architecture to use it, these methods will be a life save for providing almost 100% uptimes.
The method I'm refering to is as follows:
There are two "Big Brother" Windows services that are running on the physical server. The first to start up is registered in the database as being the Primary Big Brother. It stores it's name, process ID, and last check in the database table.
The Primary "Big Brother" is responsible for monitoring the processes (logical servers) running on the physical server. It checks their statuses and last update in the database table to validate the process(es) are still running. If the Primary "Big Brother" detects that a process has stopped (Big Brother checks every 5 seconds), it will take the steps needed to restart the process and will notify the Logging server that the process stopped so that system operators can check to see if there are any issues.
The Secondary "Big Brother" is responsible only for monitoring the Primary "Big Brother". If it notices that the Primary "Big Brother" has stopped working, it will promote itself to Primary "Big Brother", demote the original Primary "Big Brother" to Secondary, and then will restart the original Primary "Big Brother", so that it will start monitoring the new Primary "Big Brother".
I will be modifying the base architecture to include this fault tolerance method, but I will be modifying it to allow for intelligent restarts of processes to support the immediate notification and sending of trace logs to the server monitoring application.
While most people here on GameDev are staunch nay-sayers about MMO's and the actual ability to create an Indie MMO, I think that by creating a great base architecture that is re-usable and configurable, you can and will be able to create an Indie MMO. That's why I'm toiling at the keyboard every night documenting, designing and coding. So hopefully someone who actually reads my journal and is interested will have a chance to voice some input and insight into what they would expect this architecture to do for them.
If you have a great architecture already in place then you can concentrate on the actual game development and content, which is 90% of the battle.
Thanks again for reading and please comment!
Another Wow! I posted twice within the same month.
I have decided to break the logging component up so that it will have two possible operating area: Local and Remote.
Local will be used if the component is unable to connect to the remote logging server and will provide only text file logging [txt|csv|xml]. When the connection is re-established, the logging component will switch from local to remote and catch up the log entries to the remote logging server.
Why we decided to break this so far apart is the segregating of processes so that if logging fails, the application (server) using the component can still continue to function without causing crashes to the application. This also allows us to provide a common system monitoring application that can connect to multiple logging servers and retrieve log information wiythout disrupting or putting any additional strain on the applications (servers).
Now that I've mentioned the system monitoring application, I'll break open the Xmas presents early and give a little insight into the other monitoring processes that will be used in the MMO Architect project.
We will be using MySQL, MSDE, and MSSQL initially for our supported databases. We will provide a heartbeat/status monitoring for keeping track of the databases.
We will be providing a process that will monitor a directory for changes including new files, changed files, new directory.
We will be providing a process that will monitor 1:M files on a specified server for changes.
We will be providing a process that will monitor the network traffic based on IP address and port.
We will be providing a process monitoring for specified processes.
All of the monitoring systems will be designed to be accessed remotely and will allow mutiple machines (servers) to be watched. Alerting will be written in to the system monitoring application.
So that's it for today. I expect to have the base component for logging and it's logging server written by next Tuesday.
Oh by the way, all of the servers will be written for either remote or local commands for control and written in C#.
Thanks for reading :)
Wow. It's been a while since my last post and while I have no real tangible progress (other than a lot of documentation), I know that I have made great strides in bringing my framework into the design/development stages.
MMOArchitect has been re-designed from the ground floor to the penthouse and is coming along quite nicely. We (the other team member and I) have really broken functionality out into re-usable components which is making life a whole lot easier for us.
We started concentrating on making the components needed to support the creation of a base server (that could be re-used and modified for other uses). We have design and started coding the logging component (there's a whole lot of functionality here from file based to DBs to streaming) and the XML File handler (which is used to handle our config files and standard XML data files).
The next step is designing a core DB component that will abstract a data layer so we can use any of our supported DB engines without sacrificing the core requirements.
I know there's no pictures or pretty graphics, but we really have accomplished a lot from a flexible design standpoint.
Look for our next post about the core server template, which will bring together our needs for authentication, encryption, scripting (using Noaktree's C# scripting), and networking.
Ta Ta for now!
Eric 'Wackatronic' Tomlinson
Even though it just steams me, I've dropped all current code and have gone back to the design phase and will be working on the backend design more thoroughly to support robustness and configurability features.
I want the system to be used by developers and designers to come up with their ideas for an MMO and provide them with a great base unit that they can configure to their needs.
In order to do this, I've gone back to the drawing board and will be coming up with a more complete design and requirements document that will make me focus on the areas that were lacking in the past. I will concentrate on the back end servers and database design.
In addition to just database configuration, I want to allow the use of XML and binary files for configuration on both the backend servers and the clients (binary only for protecting and guaranteeing the secure client).
With that all said, I will be trying to update this journal and the website on a weekly basis to keep my self moving forward with development.
And if I can just keep myself from playing WoW too much!!!
Thanks for reading.
Well, if you've read my last post and have been waiting on pins and needles, this is my latest entry "Even Longer" (besides the huge entry I spent over 30 minutes writing that somehow vanished when I posted it).
Yes, it has been a long time since I posted last and I have finally made seom headway.
I have been working on both the logging server and a internal tool (may be released to public) for localizing your strings, images and custom (must be serializable (.NET)) objects into a satellite DLL, that is used to pull in language independent resources.
So without further ado, here's the screen shots and brief explanations.
This is the main screen. It is used to either create a new file or open an existing file. The original "master" file for an application is used to provide information and a base for all required translations for an application.
This is the open screen showing the naming for the files.
This screen shows the loaded config file. Notice the Original Translations, Comment and Description fields. With these, you should be able to distribute the application to a translator, provide them with a R/O version of the master file and they should return a language file that can be compiled into a satellite resource DLL.
Added a new string value for editing.
Save As features!
In the next several days, I'll put up the feature list and instructions on how to use the application. Then hopefully, I can generate enough interest to put the application on my site for downloading and beta testing. I may also release the source for the general public to use freely.
-- Eric 'Wackatronic' Tomlinson
Well, it's been a long time since my last post. I went back to the design board after finding a potentially fatal flaw in my design. So I've spent the last two months proving each and every component will work together and at maximum efficency.
So up to this date. I've done the following so far.
1. Create a scripting DLL that can be used by all the server components that provides LUA scripting. This is only for the server side development. We will be providing a client side scripting component that will allow users to modify their GUI and events/actions on the client.
2. Started the programming of the Logging server and now have a 70% complete server that has multiple threads for each connection and can log to the database (MS SQL and MySQL so far). This took longer to code because of the next point.
3. Completed a generic data DLL that will provide connectivity to the following databases (MS SQL, MSDE, MySQL, PostgreSQL, and Oracle). It is designed so you set the property of which database type you are connecting to and it will use the appropriate drivers and methodology to connect and provide access. We are currently fully tested on MS SQL, MSDE, and MySQL due to the space on my server. We will be removing MSDE and MySQL and installing PostgreSQL and Oracle to complete the supported databases. This component will be a life saver because it doesn't need to know anything about the actual DB structure. We have designed it to either run statements (stored procedures or SQL script) that either return data or just modifies data.
4. Completed a test client that simulates multiple connections to the logging server to test robustness and effectiveness. So far so good.
The next server on my plate is the Login server. I will be using ticket authentication (was pointed in this direction here by someone (sorry I forgot who, but I do appreciate it) to access the system after a successful login. This will in turn pass off a list of possible Master servers, who in turn use the ticket to authenticate the client accessing them.
Well, that's it for now. I hope I'll be able to post more frequently now that I'm back on track and moving forward.
Thanks for reading!
Eric 'Wackatronic' Tomlinson
Prinicipal, ImagiNet Games, Inc.
Well, its been a while since my last post and its not because I'm being lazy (this time), but I've been out sick since 2/4 and finally made it back to work this morning.
This being sick is for the birds. I actually didn't log onto my PC for the entire time and oh biy did the Spam Monster visit my e-mail address. I had over 2,000 e-mails most of which wanted to expand my breast size. While I'm like most hot blooded men, I do not want bigger boobs.
So after digressing for the past minute, I now move onto what's happening with the MMO Architect project......
Nothing, Didn't you pay attention above or are you just too mentally broke.
But I am getting back to the logging issue and solution today and hopefully within the next two days, I'll have a lengthy journal entry to share the solution and the next steps we'll be going through.
I'm kicking around the idea to post the base code to some of the servers that MMO Architect will have so I can give back to the game development community. Let me know what you think.
Don't you just hate scope creep, especially when it's during the development phase and in the middle of programming?
Well I'm starting to, but at least the scope creep is a good thing in this case.
While working on the server template (If you don't know what this is, read previous journal entries), I realized that instead of copying and pasting tons of code later on, I should be implementing the socket thread pool and it's basic handling.
So with this I really opened a big can of whoop @$$ on myself. Not that I'm a beginning programmer or anything, but I haven't worked much with multi-threading or evn multi-threading sockets from a single listening socket. So I just headed to my favorite (for now) code site (Code Project) to look up some sample code to get me started. Whew. This topic is not only complex, but there are 20 people all with 20 ways of doing it. So back to the web search. Now I went to an old favorite (MSDN) and remembered why I quit going there. Oh yes, they have code but you have to locate 10 snippets to get any idea of what's going on and the recommended way of doing it.
So to cut the post short, I trying here to get some sample code or at least to get some questions answered to point me in the right direction.
But hey! If you've done this and want to help me (please help me), I could use a good code sample on threading with windows sockets in .NET.
See ya soon and you probably won't recognize me, but I'll be the person with less hair (from pulling it out). :-)
This weekend was spent working on the first portion of the back end for MMO Architect and boy oh boy do my knees smart.
I was able to get the database portion for logins designed and ran into problems when I posed the question to my self "Why you unassuming bas#$%d, why assume that all people using this system for designing their MMO will speak English".
So after taking myself outside for a few quick slaps to the back of the head, I came back in and spent several hours trying to come up with a feasible design that would allow me to localize the backend system to any language. I know I wouldn't have problems with graphics and sounds, so I was concerning my self with how the text could be displayed. In my past I have designed only one system that was database driven to control a business system for people residing in Mexico, Canada and the US. That implementation was absolutely the worst thing I had gone through, so I knew I wouldn't be using the database for storing the actual string conversions. So most of Saturday (1/29) was spent coming up with a feasible, yet fast design.
So by Sunday morning, I had come up with a flexible low level design for making the MMO Architect backend language independent. (Reminder to myself to discuss what some of the options to making the actual game servers and client locale independent).
Now I was free to go back to the login process and try to get started again. Well another wrench for the works. I started desiging a specific server just for logins and came to the conclusion that most of the servers have quite a bit in common, so again I pulled myself out of the loop and started thinking generically again.
So once I had defined exactly what a server for MMO Architect will require for all servers, I started working on a C# template project and that's where I spent all my time on Sunday doing.
I didn't have a chance to finish the template project, but I know this will save at least 10+ hours per server so it's not wasted work.
Anyway, a server within MMO Architect will have to provide the following functionality:
1. One common private port behind the firewall and on the network to communicate securely with the configuration server.
2. Minimum 1 public unsecured port for communication outside the firewall and network.
3. Basic dynamic script loading/unloading.
4. Basic logging functionality for seperate log files as well as Windows event logging.
5. Database connection support for all supported database servers.
6. Basic cryptography for use in the server to server authentication process.
So I'll be spending part of this week in the evenings getting this template designed and tested for use.
Check back this week and I should have the template finished and the login server started.
'The tired and frustrated designer'
Well, since becoming a member, I haven't been very contributing except for bugging people for answers. Well now it's my time to try and give back to the community. I bet you are wondering how writing a journal will give back to the community.
Well here's your answer!
I have been kicking around ideas on how to make a fully re-usable MMO architecture (and before you start screaming 'heretic', please read on) that could be used by independent developers and studios to give them a leg up on the competition (and we all know there's a lot of competition). I have finally gotten my ideas in order and will be starting the logical design (actually started two weeks ago) for the MMO Architect system.
So before going into the ideas and what I'll be profiling in my journal, let me give you come background on who I am and what I've done.
Age: 39 (yes, I'm old but I'm good)
Profession: Senior Business Systems Architect
Training: BS in Computer Information Systems from DeVry (Phoenix, AZ).
Programming Background: Started professionally in 1995 upon graduation and have worked in the industry (business systems) since.
Languages: VB (all flavors), C++, C#
Interests: I'm here what does that tell you
So you now know I'm not a youngster trying to clone a current MMO. I have background in the computer industry and I love writing systems.
Now for MMO Architect. The system will be designed as a base turnkey system for providing all aspects of a MMO system. We are working on the backend system that support server configuration, customer service, and content management.
While I will not reveal all, I will start off giving base bits of information about each portion of the system as we comeplete it.
MMO Architect backend system will be written on C# with connections available into the most popular database servers. The base client will be written in C++ and written in a generic flavor so support/rollout to Windows, Mac and Linux will be possible.
We are estimating on having a complete working backend system that will support a MMO done within 6 months. After that, the fun starts. We will start designing the generic game server and the generic client with C++ and expect to have a working system (not great, but working with the client, game server and backend system) with the six months following the completion of the backend portion of MMO Architect.
Finally a shameless plug and call out for interested parties.
We are looking for people who would be interested in working with us on the project. If you would like to see more about us, please visit ImagiNet Games and if you like what you've heard, please contact the Admin and let him know.
Thanks and I'll see you next week.
Eric 'Wackatronic' Tomlinson