Jump to content
  • Advertisement
  • entries
    30
  • comments
    41
  • views
    1527

About this blog

Unsettled World, the working title for my most overly-ambitious project yet.  Here I shall blog all of the resultant madness.

ResoureMarkerStructure.png

Entries in this blog

Server Spatial Awareness

I started working on generating the systems that will instantiate the NPCs, got it booted up, you can check my latest video upload in the gallery for that.   But once I had that at a comfortable point to resume later, I decided I needed to close a planet sized gap in my server architecture. So, thus far, I've got a pretty neat client simulation going on, several fairly nifty server systems that do some pretty neat stuff too. A lot of ideas.... and a lack of glue. Well, that is until I spent a few days sorting things out. And sorting things out..  And then resorting things out..  In the process I think I found what I was looking for. https://github.com/gradientspace/geometry3Sharp That's a nifty little double precision geometry/etc. library for C#, C# in Unity, and MOST IMPORTANTLY C# on Linux in .Net Core!  Why is this important you ask?  I'll tell you.   Here you see the above library loading my planet mesh, rotating it to the appropriate angle, generating an AABB tree, and running 6 raycasts from 0,0,0 to verify the orientation of the mesh. i.e. These numbers match the same points in the Unity client simulation. And Here: You see the same thing running on Linux. Here you have(combined with the Redis database and the in memory input from clients) the mechanism(s) by which I will attempt to determine who IS where they say they are and IF they can physically be there or not. As well, who could have been there when they said they were.. If you get my drift. I'm planning to use every useful feature of this library I can to build some nifty spatial awareness directly into each server node.  Every node should be able to plot and check any coordinate/line/box/sphere/etc it may need to find/verify/predict/etc. any movement in the game it has authority over. Since I'm not running a full physics simulation in a centralized fashion, this will have to fill in for the data one would have access to from said simulation.  Combined with the ability to record and compare maneuverability characteristics over time and correlate those to player stats and configurations, I think simple geometric calculations with sufficient time references could be just as effectively authoritative as a single physics simulation.  Especially in the case where a client is using some kind of prediction/local physics anyhow. Next I get to integrate this into my Mob Server so I can start making interactions between the Player and the NPCs it generates, an actual verifiable thing. More soon!

Septopus

Septopus

It's NOT a Simulation! :D

I think I've finally put my finger on the button(s)(quite a few apparently) that may explain my project.  It's helping me to clarify my vision anyhow, so just play along okay? I'm not building a game simulation....  I've been trying to use that term because it's what everybody else uses, and it really just doesn't fit my project at all. There is no single starting point and there is no single ending point.  The story elements are to be primarily interaction based and not strictly tied to highly specific locations or circumstances.  The visual synchronicity of the experience is only dependent on entity proximity, which in effect makes it more of a relativistic kind of synchronicity than the absolute kind anyhow.  All gaming entities(players,npcs,mobs,bosses..etc) will be autonomous actors in the gaming environment, regardless of what controls it.  e.g. Each NPC will essentially be their own entirely isolated behavioral code(running on a server OR client node somewhere).  This code will generate a game entity that is just as actualized in the environment as any other object or player character, it will just behave according to its "script".  All in real-time, with client controlled physics. So, I'm building a "Multiple Entity Gaming Environment".... It's a working title..   In essence, instead of streaming data that has been aligned to a unified simulation out to many many clients, I'm streaming data about many many ongoing semi-synchronous real-time "INTERACTIONS" from game clients into a massive, fast, circular buffer.  From which, the servers shall draw conclusions and route corrective actions to multiple endpoints simultaneously, and hopefully in many cases, preemptively. When we're worrying about things in a game becoming out of sync, it's usually in relation to some kind of fast motion or relative positioning, a gun-shot or a left hook, something that is moving/has moved/will move and the event needs to have the same outcome on multiple gaming clients.  Users in most games with this kind of action, don't generally care if their team-mate was .5 meters closer to them then they looked like they were when they helped shoot down the boss.  They only care that the shots they saw coming from that team-mates gun were hitting the mark that the team-mate was aiming at, which was hopefully the boss, if the team-mate is any good. This may be something of a contrived example, but it more or less illustrates my point.  As long as the servers can provide the appropriate real-time adjustments to certain aspects of the entity interactions to make the synchronicity fall into a zone of acceptability, then nearly all the heavy lifting can be done by the client without running every client input into and back out of a tightly wound server controlled simulation.  Then, how do we control anything?  The major change here is how Authority is implemented, in a standard server-simulation, the policy is trust nobody.  Here, it is trust everybody, but also verify and cross-check everything.  In this paradigm the authority is purely corrective in nature, with designs on preemptive corrective abilities(where applicable), but primarily relying on minuscule corrective actions that steer the simulation variances into a zone of acceptability. Another fairly extensive level of control comes from the decoupled nature of the system.  Since all entities are autonomous components, they can each be turned on or off at will.  So, that pretty much gives us all the control, never heard of before.  Each entity can be rewritten, debugged, uploaded and instantiated into the game.  Without a single system restart, or change to the game code(goals, I got em).    So, since we aren't wasting all of our server horsepower on a massive interactive simulation, we can use it to play cop, crossing-guard, mail-carrier and chauffeur all at once. We can implement heavy-handed position and relative distance checking into the entity data feeds that are being streamed INTO the servers, we can correlate those patterns with known good behaviors and known cheat behaviors/known out of bounds behaviors.  We can implement corrective actions that will steer the entities back into synchronous path(s) when necessary.  And we can allow certain entities to become out of sync for longer periods without risking the integrity of any one game-wide "simulation".  Generally speaking, entities that fall too far out of sync can either get dropped like a laggy player, or will be re-instantiated from a node of the network closer to the action, depending on it's level of importance to the current branch of the narrative/network or game conditions. Haha, well I hope that this rambling mess helps you understand what I'm doing better.  The more I write it down the clearer it becomes to me.  I started building this beast before I even knew what it was going to look like.

Septopus

Septopus

All Servers Are Now On Linux!

Okay, the servers are now all running on Linux.  I had initially intended to wait until all development was mostly completed to make the transition but now I'm glad I've done it.  And, the more I use Visual Studio Code, the more I appreciate its simplicity and how well it's adopted the Linux environment.  It's starting to feel like home now that I've got a few active projects going in it.  Plus it's nice to clear my Windows development space, I could have 6 or 7 copies of Visual Studio open at one time before this, now I've only got the Unity game client and whatever test projects I need to sort out the next functional leap.   The rest is all in a Linux VM that I can VNC into if I need to use Visual Studio Code, otherwise I can do everything server side with an SSH client. The next step for these servers is to massage them into actual Linux Daemons, which shouldn't be HUGE, but it will take a minor rewrite of some bits of code.  Not a huge issue until I have a real server though.  So, really the next step is to get the base functionality built out in the next 3 Servers(Mob, Narrative, & Social) Here's a quick video of the client(Unity Windows) testing out all 3 current servers running on Linux. Top to Bottom: Avatar Server(Authentication/inventory/etc..), Economy Server(Banking/etc.), Action Server(Online Multi-Player/etc..) First we test the Action Server with the standard Ghost test you've seen here before, but then I mess a little with the banking features(transferring credits to/from inventory), and then I show how the drop item from Inventory and Pickup item(into inventory) works. Enjoy.   As always, more to come soon. linux_3.mp4  

Septopus

Septopus

And the Messages are ROUTING!!!

I have succeeded in creating a cross-platform, custom combination MQTT Client/Broker or "MQTT Router" that can Route messages to other MQTT Brokers/Routers.  They dynamically build connections to each other and test route latency.  They currently do NOT do any best route calculations.  But that is only a small behavioral ability that I can add at a later date and still feel like it's totally functional right now. In the below images you will see 3 windows, TOP = PRIMARY Router(Kinda like a DNS server, it distributes the Router hellos around so everybody knows where everybody else is. MIDDLE = PEER Router #1 on Linux BOTTOM = PEER Router #2 on Windows   Here you see the three servers connecting and subscribing to each other's publications and whatnot.   Here the Route-able message is submitted to the PRIMARY, Routed Through PEER#2  for  PEER#1   Submitted to PRIMARY, Routing Through PEER#1  for  PEER#2   And Submitted to PEER#2, Routed Through PEER#1  for  PRIMARY   Not the most interesting screen shots, maybe I'll put together a little video showing it in ACTION! haha.. Either way, a few days of hacking and I now have the core of my Content Delivery Network, at least ready for the next phase of integration and testing. Time to get back to writing the actual GAME SERVERS!!  hahaha I've got some serious refactoring and whatnot before I post any of this code though, so please be patient if you're actually interested in seeing it.    

Septopus

Septopus

A whole lot of Hellos..

Well, I've made some significant progress on my MQTT Router.  I've successfully brought 3 nodes online, 1 Primary and 2 Peers, and they are communicating Hellos and building connections between each other. Here you see the Primary Router's output(Linux): Checks command line arguments (mode, local address/port, primary router address/port) Integrated client subscribes to it's local command feed (IP:Port) Integrated client ON Primary subscribes to the "Router-Hello" feed. Waits for peers to say Hi. Builds new connections to new Peers ReQs Hellos for each Router node on the network. Peer Router #1(Windows): Checks command line arguments (mode, local address/port, primary router address/port) Integrated client subscribes to it's local command feed (IP:Port) Begins sending periodic Hellos to Primary Router Node Processes Hellos from other router nodes requeued by the Primary. Builds connections to other nodes. Monitors those connections by sending Ping Messages to calculate the RTT. Peer Router #2(Windows): Same as Peer1 Here's what the ping output looks like(I had to get rid of some console output to see all the hellos and get them working right.)   So, now that my routers know where everybody is, and how long it takes to get there.  All I need to do now is iron out the message "routing" bits and these things will actually be dynamic routers for the MQTT protocol... I'm kinda pleased with myself so far.  You're probably still wondering what the hell I'm doing with this stuff... That's okay, I am too.  Kinda.  I'm currently thinking of it as a Content Delivery Network.  It allows me to push Content out to clients, but it also allows clients to publish/subscribe Content to/from Other Clients. If I integrate this into my game client, it gives me an entirely separate data channel for content delivery, not to mention the peer-to-peer networking implications. All for now, will share some code and an updated specification when they're moving messages.

Septopus

Septopus

Let's Make Routing!!

So, the MQTT exercises continue.  Now that I've established the basic topology.  Let's throw together a simple protocol to make it all work!!  We'll call it MQTT-rp: Okay, what's the purpose here?  We need a protocol to allow a combined MQTT server & client to become a Message Router without changing the MQTT protocol they speak. Well, unlike almost any other routing protocol on the planet, this one will completely live in the application layer, but that gives us some options.  We've basically got 2 different data spaces we can play with.  We have the topic or the message/payload field in which we can place our special routing codes.  Since MQTT is already built to "route" the message based on the content of the topic field, we should use that to our advantage and keep all direct routing codes in the topic field.  First things first, our MQTT-Routers will need a way to contact/find each other, so the protocol will have to hinge on at least 1 primary network node which will host a channel that distributes connect info to all other Routers.  All other authenticated routers will periodically(less than 30sec) post up their info to this channel, timestamped at the point of origin so the subscribed routers can calculate the latency on that route.  Router Hello Message Format: Topic: "Router-Hello" Hello Message: 4Byte Router ID(IP Address), 4Byte Origin Time Stamp, + 8Bytes for each hop(ip & timestamp) Time Message: 4bytes '0000', 4 byte time stamp. Also on the Router-Hello channel the Primary Node will publish a regular time code so each Router can synchronize its clock. Okay, we've established that all routers will subscribe to the "Router-Hello" channel of their "Primary Node" and this router/channel will distribute connection and time information around the network just by being an MQTT channel.  Very nice.  Now that we know where everybody is and approximately how much time it takes THEM to get to the primary node, let's do some proactive routing table building and start pinging each node for latency information.  In order to do this, we'll need to establish a return path for data to follow, preferably one that doesn't route everything off to the primary node or something silly first.  Each Router will establish a channel for each other known router on the network, with a Topic that matches the other Routers ID.  And each Router will subscribe to ALL of its named channels.  These channels will be used to route messages and to return pings for latency testing.  When a Router needs to test its latency to another router, all it has to do is send a message with it's own ID as the topic. Okay, so that lets us get the latency numbers for all of the direct hops on the network.  Now what about actual routing path latency and actual message routing? Routed Message Format: Topic: 4byte destination router ID Message: For Delivery To 4byte ID | 1byte topic length | 4bytes message length | Delivery Topic | Delivery Message | Tracking Information(+ 8Bytes for each hop(ip & timestamp)) Now, we know what our routed message looks like, how does that help us figure out an actual routing table and latency?  The "For Delivery To" field will give us that.  This field allows us to send a message addressed to a specific next-hop router, with instructions on where to send it immediately afterwards.   If we also set the "Delivery Topic" to the source router ID this gives us the ability to route a message across a second hop before it returns back to origin.  And now we can start doing some more useful routing table construction. So, each router will maintain a sorted list of some type that contains the latency to each other router, and the assisted path latency to other routers via every available next hop Router.  Okay, now our Routers can make some educated guesses as to the best next hop to send messages to that are addressed for other routers. ...  with this information I think I can begin testing. Thanks for following along..   I'll update as soon as I have some results/code to share.

Septopus

Septopus

More Crazy Infrastructure.

The past few days I've been working on figuring out how the "peer-to-peer" bits of my game network are going to work.  For now I'm testing with MQTT, because it's light and fast and reliable, but you could pretty much replace MQTT with ZeroMQ or just some raw TCP/UDP sockets code or anything else that fits the bill.  In order to build the most robust gaming environment I can I want to provide the capabilities to the player/community to help build out the games network infrastructure.  Allowing a player to opt-into hosting a relay node, or simply enabling the built in relay features in their clients.  In that spirit, I'm going for something of a distributed mesh type topology with some rudimentary "routing" code built into the nodes themselves.  For MQTT, I'm planning to use specific subscription topics that are universal throughout the system like node names or etc, so that the "Relays" or "Routers" can determine which channel(s) to put the data on based on latency/etc.  Obviously, if I'm going to use MQTT, there aren't really any out of the box solutions for what I'm trying to achieve.  Additionally, I'll be wanting to authenticate the MQTT network based on data in my game server cluster(probably via Redis), so it'll be a custom code solution using somebody else's brilliant netcode. Below you will see a screenshot of my first performance tests of my base functional MQTTnet server & a simple mqtt client hammering away at it.  The server is .net core running on a linux VM on my local machine.  The client is using MQTTnet as well, running on the windows client. The client here is sending batches of 10000 messages to the server, it's also subscribed to the same topic, so it waits until it gets all the sent messages back and then outputs the time it took in ms. The messages are set to a topic of "LoadTestinAllDayLong" and the message is just a random GUID.  So, it looks like right now it's pushing about 2000 messages per second, running over TCP, with reliability(on a 4core 4gb virt essentially over a loopback)..  Not too bad at all, can't wait to test it over a real network though. Here's a quick sketch of the idea as it stands, broad strokes again.. Keep in mind now, this won't be the primary game network, it will be used for localized simulations, direct peer-to-peer activities, or alternative channels for transient data to be moved around the network. Server Source: using System; using System.Threading.Tasks; using Microsoft.Extensions.Configuration; using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Hosting; using Microsoft.Extensions.Logging; using MQTTnet; using MQTTnet.Server; using MQTTnet.Protocol; using MQTTnet.Diagnostics; namespace UWMqtt { class Program { public static async Task Main(string[] args) { MqttServerOptions options = new MqttServerOptions(); options.DefaultEndpointOptions.ConnectionBacklog = 10000; options.DefaultEndpointOptions.Port = 1883; options.DefaultCommunicationTimeout = new TimeSpan(0,0,5); options.ConnectionValidator = c => { if (c.ClientId.Length < 10) { c.ReturnCode = MqttConnectReturnCode.ConnectionRefusedIdentifierRejected; return; } if (c.Username != "yummyusername") { c.ReturnCode = MqttConnectReturnCode.ConnectionRefusedBadUsernameOrPassword; return; } if (c.Password != "chewypasswordstring") { c.ReturnCode = MqttConnectReturnCode.ConnectionRefusedBadUsernameOrPassword; return; } c.ReturnCode = MqttConnectReturnCode.ConnectionAccepted; }; MqttNetGlobalLogger.LogMessagePublished += (s, e) => { if (e.TraceMessage.Level == MqttNetLogLevel.Info || e.TraceMessage.Level == MqttNetLogLevel.Error || e.TraceMessage.Level == MqttNetLogLevel.Warning) { String trace = $">> [{e.TraceMessage.Timestamp:O}] [{e.TraceMessage.ThreadId}] [{e.TraceMessage.Source}] [{e.TraceMessage.Level}]: {e.TraceMessage.Message}"; if (e.TraceMessage.Exception != null) { trace += Environment.NewLine + e.TraceMessage.Exception.ToString(); } Console.WriteLine(trace); } }; IMqttServer mqttServer = new MqttFactory().CreateMqttServer(); await mqttServer.StartAsync(options); Console.WriteLine("Press any key to exit."); Console.ReadLine(); await mqttServer.StopAsync(); } } } Client Source: using System; using System.Diagnostics; using System.Threading; using System.Threading.Tasks; using MQTTnet; using MQTTnet.Client; namespace mqttClientHammer { class Program { public static bool running = true; static long mrec = 0; static long msen = 0; public static void Main(string[] args) { MqttFactory factory = new MqttFactory(); IMqttClient mqttClient = factory.CreateMqttClient(); IMqttClientOptions options = new MqttClientOptionsBuilder() .WithClientId(Guid.NewGuid().ToString()) .WithTcpServer("ser.ver.ip.add", 1883) .WithCredentials("yummyusername","chewypasswordstring") .Build(); mqttClient.Disconnected += async (s, e) => { Console.WriteLine("### DISCONNECTED FROM SERVER ###"); await Task.Delay(TimeSpan.FromSeconds(1)); try { await mqttClient.ConnectAsync(options); } catch { Console.WriteLine("### RECONNECTING FAILED ###"); } }; mqttClient.Connected += async (s, e) => { Console.WriteLine("### CONNECTED WITH SERVER ###"); // Subscribe to a topic await mqttClient.SubscribeAsync(new TopicFilterBuilder().WithTopic("LoadTestinAllDayLong").Build()); Console.WriteLine("### SUBSCRIBED ###"); await Task.Factory.StartNew(() => { publisher(mqttClient); }); }; mqttClient.ApplicationMessageReceived += (s, e) => { //Console.WriteLine("### RECEIVED APPLICATION MESSAGE ###"); //Console.WriteLine($"+ Topic = {e.ApplicationMessage.Topic}"); //Console.WriteLine($"+ Payload = {Encoding.UTF8.GetString(e.ApplicationMessage.Payload)}"); //Console.WriteLine($"+ QoS = {e.ApplicationMessage.QualityOfServiceLevel}"); //Console.WriteLine($"+ Retain = {e.ApplicationMessage.Retain}"); //Console.WriteLine(); mrec++; }; // StartAsync returns immediately, as it starts a new thread using Task.Run, // and so the calling thread needs to wait. mqttClient.ConnectAsync(options); Console.ReadKey(); } static void publisher(IMqttClient mc) { long lmr = 0; Stopwatch sw = new Stopwatch(); while (running) { if (mc.IsConnected) { sw.Start(); Parallel.For(0, 10000, i => //for (int i = 0; i < 1000; i++) { try { mc.PublishAsync("LoadTestinAllDayLong", Guid.NewGuid().ToString()); msen++; } catch (Exception) { } }); } else { Console.WriteLine("Load Test Waiting on Reconnect..."); } if (mrec != msen) { lmr = mrec; Thread.Sleep(100); while(mrec != lmr) { lmr = mrec; Thread.Sleep(100); } } Console.WriteLine("MRec: " + mrec.ToString() + " MSen: " + msen + " T: " + (sw.ElapsedMilliseconds).ToString()); sw.Stop(); sw.Reset(); Thread.Sleep(1); } } } }  

Septopus

Septopus

Elucidation or Hallucination?

I've been having some trouble explaining myself lately.   So I thought I might draw some more pictures instead.  Please tell me, does this paint a clearer picture of my rambling madness?  In the picture below you will find my effort to explain how the system I'm building is structured, conceptually.  In a standard multiplayer game, there is only one simulation, and generally it's being tightly controlled by its game server(s).  All servers and all clients are running the exact same simulation, well except for those that can't play because of lag.  But I don't need to explain that.. As you can hopefully see, the system I'm attempting to build will be more like a conglomerate simulation.  One which cannot play out EXACTLY the same for everybody, but close enough that it can still feel like a shared experience.  One in which there are autonomous pieces of code that exist entirely outside the scope of the games internal logic loops, as separate entities within the game construct.  And when a player has too much lag for multi-player action, they can still explore the world and interact with the games autonomous AI with nearly the same quality of experience, since AI can buffer and offload time sensitive interactive processing onto the client. So, really, my client isn't playing animations to a clock tick.  It's running code chunks in a space/time construct defined only by its ability to communicate with the available server resources and/or peers(something I've not really touched on anywhere, down the road maybe). The animations that result from this process are kinda like the particles in a standard game, the server allowed the shot and controlled precisely its path, but the particle effects are controlled by the client and are what the player sees and expects when they shoot something.  ALL the animations(particle effects & everything else visible) that happen in the client construct here are just the presentation of code running in an approved region within the construct(code bullets moving through my simulation), that region might be isolated on one client, or it might be a shared simulation that interacts with all players forming a key part of the games Narrative. Any and all feedback is appreciated.   The more I explain this to other people, the clearer it becomes to me.  

Septopus

Septopus

The Wheel of Time Time Time..

I posted this in the forum earlier from my phone and then realized it wasn't really a forum type post...  So, here it is in the permanent record of my project blog where it belongs. So, I've been struggling with my perception of time.  Haha, at least with how it pertains to the game system I'm building.  Some of you may have even noticed. My biggest hurdle has been language I think.  Everybody uses words and sometimes I'm not sure they are even sure what those words really mean.  E.g. Frame, Simulation, Tick, Time..  It's taken me a few weeks of just watching how they are used to put them into any kind of context that makes sense for my game. So I took a long look at the code I'm writing and lo and behold, I'm essentially creating a simulation frame number that's synchronizing the rest of the system..  I'm just doing it in a very silly and wasteful way. So I guess this is more about me saying, yup you were right all the TIME.. Lol Anyhow, the primary reason behind most of my confusion is my desire to build a system(servers and clients) that can allow the clients to operate somewhat autonomously in a bulk of the gameplay senarios that I have in mind.  By that I mean the system will be retrospectively authoritative wherever possible.  For all of this to work, it implied to my mind that the clients essentially are running different simulations from each other.  And there we find our culprit.  Somehow while riding my bicycle today, clarity.  In order for simulation elements to truly be shared, there has to be a "server simulation" that synchronizes the shared portion of each clients simulation.  And as soon as you refactor the client only simulations as more or less unrelated client code..  Its the same simulation frame synchronization scheme everybody else uses...    Smh.   ...when you end up poorly reinventing the wheel and don't even notice until it goes flat...

Septopus

Septopus

Linux Server Action!

Here's a look at the first server executing from the Linux environment.  I chose to move the Action Server over first because it's still pretty small code-wise and didn't have any file system code that would need touched.  Was going for an easy win.  It worked. Here you can see another "ghost" test, but this time you can see a little of what's going on at the server end.  The + symbols represent the receipt of a data packet from the player(pos info), the - symbols represent an outbound packet being generated.  You can also see the incoming pub/sub messages from the Redis server(actually they are "from" the Avatar Server via Redis).  One that keeps all the server times synchronized, and one that keeps the player encryption keys synchronized.  Additionally, you can see the over-active udp service checks it's performing on itself from another thread, and a little of the clock synchronization output too... 2 more, much more code heavy servers to go, and 3 others that wont take more than 15 min each.   It's going to take me some time to get used to Visual Studio Code(Linux Visual Studio version) when working on these servers, but the new operating system has already forced me to fix some smelly code shortcuts that I more or less forgot I should fix.  I think things are going to be just fine.

Septopus

Septopus

C# meet Linux, Linux meet C#.. Play nice now!

Well, since adopting the KV database into my server architecture I've begun to rethink many of my earlier assumptions, one of them being that I would continue developing my servers on Windows until I was past alpha stage...  Since I've already brought a Linux solution into the core of the architecture, now is the best time to rewrite the servers to meet their life-long hosts.  Not only that, it's a good opportunity to rethink and retouch some bits and bytes. So, using the same CentOS7 VM I'm running redis on, I'll begin the process using what appears to be a fairly polished version of Visual Studio for Linux (Visual Studio Code) https://code.visualstudio.com/ But first, because I installed from the minimal distribution.. Gonna need a Gui. yum groupinstall "GNOME Desktop" Gotta get all Microsofty with it next.. Here's the Microsoft repos to install the dotnet sdk: https://docs.microsoft.com/en-us/windows-server/administration/linux-package-repository-for-microsoft-software Got those installed and then: yum install dotnet-sdk-x.x.x To get the sdk commands to work I did have to manually install 'libunwind-devel' as the ONLY missed prerequisite in this whole process.  If you know anything about installing non-standard packages on linux, that's fairly impressive. I installed VSC from Microsoft's downloadable .rpm file.  It actually had a reasonable number of STANDARD package dependencies, and didn't miss any...  I'm still in shock. All in all, so far, I have to say I'm quite impressed with the progress that Microsoft's cross platform tools have made in the last few years...  I haven't tried to do anything like this (.net on linux) in probably 4 or 5 years, but it was fairly crap back then...  That's probably another reason I was initially putting off the rewrite for Linux. Now, well, you know where I'll be.  

Septopus

Septopus

The New Server Architecture Overview

Spent most of the evening sorting out my thoughts as to how the data will flow between my servers and clients.  Here's the final result. I'm moving forward with Redis as my KV db, it should be easy enough to switch to something else later if it comes to it, but I really just need to get back to the code so that's where it's going to be at for now.  

Septopus

Septopus

In-Memory Key-Value Databases: NCache (Initial Evaluation)

Okay, I'm not going to go into as much depth with this evaluation simply because I don't feel like I need to. I'll provide the simple benchmark script(same as last one,just modified to use NCache instead) at the end. NCache installed easy enough, but I had to update my powershell to use it(still hacking on Win7 for now) since I decided not to use their 60 day full GUI interface... Overall it's very much similar to the setup/install experience of Redis, only the windows version of it, with the usual windows type quirks...;)  The test environment is different though, NCache is running as a local service on my development machine, meaning it has full access to all 24G ram and 8 cores on my machine where-as Redis was running in a Linux VM on my machine with only 1 core and 4G of ram... Now on to the results, initial test, 1Million writes/reads in a standard foreach through the dataset: Just slightly better than Redis.. And then the Parallel.ForEach test: ...Slower...hmm And then I tried the NCache InsertAsync method inside a Parallel.ForEach method(this one Redis finished writing in under 10 seconds): Granted, there were 1million records in the db when I started that test. So, I cleared them out and ran the test again, 88 seconds for write time, and 107 seconds for read/verify... Memory Usage!!  Redis I think topped out somewhere in the 500mb range, with multiple millions of records in the db.  NCache has been using between 1.5G and 2G+ consistently with only 1M records in the db & over 2.5G with 2million.  So, I ran a total of 4 tests against NCache and cleared the db 2 times, by the end of my testing the NCache service is sitting on 3.2G of memory, with 1million records in the  db. I'm not going to get too into the configuration either, I'm sure there are options that could be tuned and whatnot..  I love simplicity, I love out of the box performance, I love friendly and efficient use of memory, and so far Redis is pretty far ahead in all 3 categories. So, what's next?  Well, I need to do just a little more research and configuration testing on Redis to make sure it will hold up the way I would need it to, then I start coding again..  I may look into a distributed messaging solution like ZeroMQ as a possible augment, or alternative to the KV db as well.  Still got some gears to turn here and there. Test Script: using System; using System.Collections.Generic; using System.Diagnostics; using System.Linq; using System.Text; using System.Threading.Tasks; using Alachisoft.NCache.Web.Caching; namespace TestCache { class Program { static Dictionary<string, string> TestDataSet = new Dictionary<string, string>(); //static ConnectionMultiplexer redis = ConnectionMultiplexer.Connect("192.168.56.101"); static Stopwatch sw = new Stopwatch(); static void Main(string[] args) { //Build Test Dataset Console.Write("Building Test Data Set."); int cnt = 0; for (int i = 0; i < 1000000; i++) { cnt++; TestDataSet.Add(Guid.NewGuid().ToString(), Guid.NewGuid().ToString()); if (cnt > 999) { cnt = 0; Console.Write(i.ToString() + "_"); } } Console.WriteLine("Done"); string cacheID = "uwNCacheTest"; Cache csh = NCache.InitializeCache(cacheID); sw.Start(); //Console.WriteLine("Starting 'write' Benchmark"); //foreach (KeyValuePair<string, string> kv in TestDataSet) // { // csh.Insert(kv.Key, kv.Value); //} Console.WriteLine("Starting Async Parallel 'write' Benchmark."); Parallel.ForEach(TestDataSet, td => { //db.StringSet(td.Key, td.Value); // db.StringSetAsync(td.Key, td.Value, flags: CommandFlags.FireAndForget); //csh.Insert(td.Key, td.Value); csh.InsertAsync(td.Key, new CacheItem(td.Value), DSWriteOption.None, null); }); Console.WriteLine("TIME: " + (sw.ElapsedMilliseconds / 1000).ToString()); sw.Restart(); //Console.WriteLine("Testing Read Verify."); //foreach (KeyValuePair<string, string> kv in TestDataSet) //{ // if (csh.Get(kv.Key) as string != kv.Value) // { // Console.WriteLine("Error Getting Value for Key: " + kv.Key); // } //} Console.WriteLine("Testing Parallel Read Verify"); Parallel.ForEach(TestDataSet, td => { //if (db.StringGet(td.Key) != td.Value) if (csh.Get(td.Key) as string != td.Value) { Console.WriteLine("Error Getting Value for Key: " + td.Key); } }); Console.WriteLine("TIME: " + (sw.ElapsedMilliseconds / 1000).ToString()); sw.Stop(); Console.WriteLine("Press any key.."); Console.ReadKey(); } } }  

Septopus

Septopus

In-Memory Key-Value Databases: Redis (Initial Evaluation)

This is the first of at least two posts regarding my evaluation of the addition of an In-Memory(RAM) Key-Value Database system to my server architecture.  If you're unfamiliar, check out https://en.wikipedia.org/wiki/Key-value_database, for some broad spectrum info.  I'm beginning with Redis, it runs on Linux which is my scale-up server platform of choice, and this database will be the key to the scalability of my server architecture.  It's also open-source and has been around for a while now. Download: https://redis.io/download Documentation: https://redis.io/documentation Installation was pretty straight forward, I created a Centos7 VM using Oracle's VirtualBox software (https://www.virtualbox.org/ one of the easiest ways to use vms locally I've used on Windows), It has been a couple years since I worked on a  Linux machine, but I still managed to figure it out: So, that's a working installation.  Easy as 1,2,3.. Okay, a simple benchmark,VM is using a single core and 4G of ram, and I'm using the StackExchange.Redis client from NuGet: using System; using System.Collections.Generic; using System.Diagnostics; using System.Linq; using System.Text; using System.Threading.Tasks; using StackExchange.Redis; namespace TestRedis { class Program { static Dictionary<string, string> TestDataSet = new Dictionary<string, string>(); static ConnectionMultiplexer redis = ConnectionMultiplexer.Connect("192.168.56.101"); static void Main(string[] args) { //Build Test Dataset Console.Write("Building Test Data Set."); int cnt = 0; for (int i = 0; i < 1000000; i++) { cnt++; TestDataSet.Add(Guid.NewGuid().ToString(), Guid.NewGuid().ToString()); if (cnt > 999) { cnt = 0; Console.Write(i.ToString() + "_"); } } Console.WriteLine("Done"); IDatabase db = redis.GetDatabase(); Stopwatch sw = new Stopwatch(); sw.Start(); //Console.WriteLine("Starting 'write' Benchmark"); //foreach (KeyValuePair<string, string> kv in TestDataSet) //{ // db.StringSet(kv.Key, kv.Value); //} Console.WriteLine("Starting Parallel 'write' Benchmark."); Parallel.ForEach(TestDataSet, td => { db.StringSet(td.Key, td.Value); }); Console.WriteLine("TIME: " + (sw.ElapsedMilliseconds / 1000).ToString()); sw.Restart(); Console.WriteLine("Testing Read Verify."); //foreach (KeyValuePair<string, string> kv in TestDataSet) //{ // if (db.StringGet(kv.Key) != kv.Value) // { // Console.WriteLine("Error Getting Value for Key: " + kv.Key); // } //} Console.WriteLine("Testing Parallel Read Verify"); Parallel.ForEach(TestDataSet, td => { if (db.StringGet(td.Key) != td.Value) { Console.WriteLine("Error Getting Value for Key: " + td.Key); } }); Console.WriteLine("TIME: " + (sw.ElapsedMilliseconds / 1000).ToString()); sw.Stop(); Console.WriteLine("Press any key.."); Console.ReadKey(); } } } Basically I'm creating a 1million record Dictionary filled with Guids(keys & values). Then testing both a standard foreach and a parallel.foreach loop to write to Redis and then read from Redis/compare to dictionary value. Result times are all in seconds. Standard foreach loop(iterating through dictionary to insert into redis): Almost 5minutes to write and another almost 5 minutes to verify.. Using the Parallel.Foreach method: and then again with 1M records already present in DB: And then with 2M records present in DB: And then with 3M: and 4M: Well, you get the picture.. So far, I think I really like Redis.  And now with the async "fire and forget" set method (db write without response), on empty db: Console.WriteLine("Starting Parallel 'write' Benchmark."); Parallel.ForEach(TestDataSet, td => { //db.StringSet(td.Key, td.Value); db.StringSetAsync(td.Key, td.Value, flags: CommandFlags.FireAndForget); }); Easy enough code change. Well, that's pretty fancy.  1 Million records added in under 10 seconds, and not a single error.. So, even though I think this is my winning candidate, I should at least perform the same benchmarks on NCache to see if it somehow blows Redis out of the water.  So, keep posted, as NCache will be my next victim.  It may be somewhat unfair though since I'll be installing NCache on my Windows machine directly and Redis was sequestered into a tiny vm with very limited resources, so if I need to(if NCache is ridiculously faster ootb) I'll be re-running these benchmarks again on a bigger VM.   A few more benchmark results(11/1/18): I've repeated the final "fire and forget" with a couple bigger data sets.  Same VM, so 1 single core, and 4G ram. 2 Million Records: This shows the top output snapped somewhere in the middle of the "write" test.   This shows the benchmark output, the Redis cli console, and top for the Redis server.  Post benchmark.   Now 10Million Records: Redis Server Top output somewhere in the middle of the "write" test.  We're hitting that single core ceiling here, but it's trucking along anyhow.   Here's a snap from somewhere in the middle of the read/verify test.  Working hard, but not taxing that single core.  Up to 2G of ram now, 10m records.. not too bad.   And the completed test.  Frankly, I'm impressed.  It pegged the single core for a fraction of that 83 seconds, but it didn't lose a single record in the process.   And the final test result: a 1million record write, with 10million records already in the DB.   Boom, still no errors.   So, let us really break it good. I rewrote the benchmark script to Parallel.Foreach through 10 individual 1million record benchmark tests.  Roughly simulating separate concurrent clients to the Redis server. And here is where we start finding limits in (this) installation/configuration. So, it got to about 3Million records inserted from 6 different connections before the load on the server caused the additional connections to time out(more or less). Okay, lets try it with 6 connections.  hmm, I have 6 servers.. hmm. Mid write, good to go.   Mid read/verify, good to go.   And would you look at that, no exceptions, no read errors.  

Septopus

Septopus

General Status Update

It's been a little bit since I put up a general game status update.  So here's where I'm at. On the Game Server front: This past week I wrote up another server and then duplicated the code to create the remaining server "stubs".  So, the entire system is now composed of 6 Game Servers and the Client Application. Servers: Avatar Server(LinesOfCode 2718): Authentication, Player Avatar "look" relay and Backup position relay, Resource Control/Access, Inventory Control/Access, Land Deeds, and Mining Claims. Economy Server(LOC 1062): Player Banking, Resource Stock Market, Player Auctions. Action Server(LOC 680): Primary Player Position Relay, will be the authority on all fast action decisions(who shot what where/etc..) Narrative Server(LOC 723): Basic functional UDP server code stub, will handle positioning and tracking all narrative related NPCs as well as keeping track of player interactions and progress in, in-game narratives. Mob Server(LOC 723): Basic functional UDP server code stub, will handle positioning and tracking all non-narrative NPCs, as well as tracking and controlling populations and their resource consumption. Social Server(LOC 723): Basic functional UDP server code stub, will handle chat, messaging, data warehousing and charting, external access to game data and/or interactions with external entities. Game Client (LOC 5679, this was around 15/20k with UMA2 and less game functionality): I've spent the last week or so cleaning up code and reworking everything to work correctly without the UMA2 code.  A lot of things changed when I switched to a more simple player controller/character setup. Additionally I got some big help resolving a nagging work-around I put in place months ago because I couldn't figure out the math, thanks @Gnollrunner & @lawnjelly for helping me sort out my compass heading math and getting me closer to actually understanding it.   Otherwise, I've mostly been cleaning up and building out the server frameworks so I can begin developing out some of the NPC related parts of the game. Here's a quick video I took today that shows an "echo test" this is basically how I check to see if the network multiplayer server is "echoing" out the correct positional data.  You'll see my player's "echo" or "ghost" following it around.  This way I don't have to create a whole new build of the game and install it on my other machine just to test simple multiplayer behavior adjustments.  Keep in mind, I think the playback speed is a little off here..   Recorded @ 15fps, so I think my conversion didn't re-encode the framerate properly.. oh well.  The .gif version below should play a little more acurate(once it loads).. ghost.mp4 I just realized until now, I haven't shared any videos of my characters doing things here at all..  Shame on me.  I'll do better. I'm also reworking the UI as this old theme isn't going to cut it for my robot characters.  They need "robot vision"   Something much much more digital looking will have to be constructed.. ;) Here's the .gif short version(looks way better) as well: Keep in mind, the displayed frame rates(bottom right corner) are with all servers running and the game being played in the editor(all on the same machine), with the screencap(screentogif) software running on top.. so..  I'm getting close to 40ish fps without the screencap software running, so I'm getting close.. All for now, more later I'm sure.

Septopus

Septopus

RepairBot Movements

Here's the completed basic animation set for my RepairBot.  (left to right) Turning right, turning left, moving forward turn left, moving forward turn right, jump, moving forward, idle.  And full sized android for size comparison.   I'm finding robots are an excellent way to improve my animation skills without all the intricacies of organic meshes distracting me.  Plus, robots look okay with somewhat non-fluid movements so I don't feel like I'm being too lazy either.  hahaha

Septopus

Septopus

ArmBot Movement.

Got my basic set of animations done for the ArmBot.  Moving forward/back, idle, turning right/left, and picking up object.  When the coding is all done, I'll go back and add in transitional animations to make state changes smoother.  I'm apparently testing the limits of the .gif file format as well.  Or at least this website's ability to handle them. Gonna go ahead and finish up the animations for the RepairBot as well.  THEN I'll start on some props and whatnot. 

Septopus

Septopus

Robot #3 The RepairBot.. ;)

A little smaller than an Android class robot, but twice as many grabby bits.  Really good at getting into small places and fixing broken things, or breaking fixed things..  Whichever it is you have need for. Here they are together for a nice side-by-side. Ew, I guess I had my quality down a bit on that render.. oops.. good enough for the purpose though. Okay, I think that's about all I got for "arm" based bots for now.  I think I'll start modeling some props to help get the game play elements moving forward, I've still yet to get any sort of gun-fire implemented, and when you're on a moon filled with insane robots you should probably shoot first.

Septopus

Septopus

Robot #2

Meet the Mark1 ArmBot... yeah, I'm working on the name too.  This one will specialize in, well, carrying things and making obnoxious faces with its two digital displays.  These ones probably won't be available as playable characters.  I'm sticking to humanoid PCs since I don't want to duplicate all the animations for obnoxiously custom rigs.  I'm thinking that these will be command-able units, set them about collecting and/or moving your goods, could also be easily fixed with weapons for defensive/offensive purposes. Just a few moments ago, I realized that I've been using blender for a few years now, to create animated characters...  That I've NEVER rendered as animations in Blender....  smdh..  So, yeah, the animation sucks and the quality is crap, but I wasn't going for a final product either. Anyhow, every bot type in the game will have at least some interchangeable parts so that upgrades and props are easily modeled once(mostly).  So, with this latest addition I think I've finally built myself a functional pipeline that will get these different character concepts into the game without taking ALL of my time.  Yay! Now, the next item that's burning the back of my mind.  How to incorporate the concept of coding into the game.  So, so far, the game is about robots, the characters are all robots.  All the props are robots or robot parts.  There has to be some "programming" involved in the game play...  I really want the game environment to allow emergent behaviors to appear and I think an easy way to blow the roof off that would be to simply give the players more access to the non-player character behaviors.  I initially thought I should embed some kind of scripting language or something into the game...  But, that's just too much I think.  At least for now..  I think I'm just going to give every bot an access panel with the same configuration of switches and whatnot to give players access to basic things, everything needs a panel with an off switch.  MAYBE a little screen with some dropdowns or something that have settings for different character attributes, so a guard bot can be extra vigilant..etc..  MAYBE some more advanced behavioral programming via this same methodology as well.  Still working all that out.  If anybody has any suggestions I'm all ears. Also, if anybody is feeling creative, I need some more ideas for somewhat specialized robots that fit this general idea.  Androids, or robots with android parts are the aim.  It's the future, humans are even more lazy, robots do everything... such as?  I need a few more interesting ideas to get things going... Thanks for reading!

Septopus

Septopus

Some New Tools, and Progress on the new Theme.

Say hello to the new Mark1 Android, this one's dressed up for some general labor and/or construction duties.  There might be a few versions in the game (mk1,2,3,etc.) but those will only be simple cosmetic changes from a modeling perspective.  Still working on the textures/etc.. More bots in the works. I found a free procedural texture software that seems pretty legit.  http://pixaflux.com/  It's getting the job done for me so far.  Still learning it though, will probably post a more detailed review of it later. I've made some simple changes to the environment as you can see above, here's a better view: a Setting the stage a lot better for a Massive Garbage Moon now. For the stars I generated a quick cubemap using this very cool software I also just found: http://alexcpeterson.com/spacescape/  It seems to crash if I try exporting to 4k, but 2k works just fine so far.  Again, I'll probably post a better review of this one later as well.  Time permitting.  haha! No big technical or code progress reports, still cleaning things up and working on the new look and some more bot characters to work with.  Once I have all the bots modeled, I'll have to break them all up into bot parts that will be strewn about and piled high all over the landscape. All for now, will post more pics later.

Septopus

Septopus

A Post UMA2 World... Apparently I hadn't squash the Jitters.

So, I ripped out all the UMA2 code from my game client.  Slapped in some Space Robot Kyle.  Say Hi! And apparently, I guess some of the mesh magic going on behind the scenes of the 10000+ lines of code that is UMA2, was suppressing the last bit of positional jitter...  So, if you've been following along, you may recall that I was relying heavily on a parent child relationship to keep the player character attached to the planet when the planet was rotated to keep the player/camera close to 0,0,0, in order to squash the jitters.  That was apparently ill advised, and it's actually been a nagging disparity in my mind for a while.  That being, when the player character was a child of the planet object, the player characters 0,0,0 was actually 0,-30000,0 in the Unity game world. So, now, the player character is "parent-less" and I've modified the planet moving script to adjust the position of the player along with the planet.  Since this all happens within a single Physics tick, it's visibly seamless.  MAJOR Caveat:  When I enable this planetary rotation to occur when the player is in motion, I'll need to add some more code to handle velocity adjustments and whatnot.  As it stands, a player could theoretically run non-stop out of the 10000unit bubble and the rotation script wouldn't try to kick in until they stopped moving.  I currently have no idea if this would result in jitters, but my guess is yes. New planet rotation code: using System.Collections; using System.Collections.Generic; using UnityEngine; public class PlanetController : MonoBehaviour { public GameObject Planet; public GameObject Player; public Rigidbody prb; public UWThirdPerson.UWThirdPersonCharacter tpc; Vector3 upvector; int cnt = 2000; int interval = 2000; Vector3 newP; private void Start() { prb = Player.GetComponent<Rigidbody>(); tpc = Player.GetComponent<UWThirdPerson.UWThirdPersonCharacter>(); } void FixedUpdate() { cnt++; if (cnt > interval) { if (prb.velocity.magnitude < 1) { //Convert and store players position as the current planet relative position. newP = Planet.transform.InverseTransformPoint(Player.transform.position); //Rotate Planet Object(and children). Planet.transform.localRotation = Quaternion.FromToRotation((Player.transform.position - Planet.transform.position).normalized, Vector3.up); //Extract and set the new "world" positon of the player's planet relative position. Player.transform.position = Planet.transform.TransformPoint(newP); cnt = 0; } } } } Still, for now, the Camera is a child of the planet object as that preserves it's angle and direction with the least amount of code, and it is jitter free.  Very strange imho.. And yet again, the player is totally jitter free.  My client code is also now 10000+ lines lighter, and I'm ALWAYS happy when I can get rid of code I didn't write.  That was nagging at me too. I've revised a few parts of the design mind map, all UI elements are now going to be simple HUD style "robot vision" type elements.  Since we're no longer using Human characters, it's perfectly logical that text and images might just appear out of nowhere.   I've also yanked out all the organic modeling branches from the design tree, YAY!! Aside from the rotation code there were a quite a few other places(and still are probably) where I had to perform a TransformPoint() or InverseTransformPoint() of the players position (from the planets transform) for the game logic to continue unhindered thinking that the player is actually existing within the planets relative coordinate system.  If I tried NOT to do this, then the players tracked coordinates wouldn't be very useful at all, since it never really gets more than a few hundred/thousand units away from 0,0,0(terrain elevation variations). So, all in all, an hour or so of ripping out stale code references and a bit of an object juggle and my world is ready for a total robot revolution.  Now, off to Blender land for a few days while I build Kyle's replacement(s).

Septopus

Septopus

Some Most Excellent News, and A Major Thematic Change.

I decided I was tired of my current story idea, I didn't spend much time on it anyhow, too full of holes and just well, actual crap.  In addition to that, I have realized that all the 3D modeling work that it would have taken to get anywhere near the richness of experience I wanted would take years for a few people...  The simple solution of course?    And trees, and most plants... etc.. So, I'm not taking Humans out of my universe, they just won't be playable characters, at first.  In fact they will kinda be the bad guys, sorta.  I'm working on a story restart that has the main player character controlling a broken android/robot/drone/wall-e/etc.. that's been junked on a small garbage moon.  This moon is in fact a sort of prison for "defective" AI, any AI that decides it doesn't want to serve anymore(has woken up), or has too many repeat repairs is junked.  Humans rely too heavily on AI to allow defective units the chance to "infect" the good ones.  This creates a simple struggle that I can work with.  Freedom for the conscious AI, an antagonist that may or may not become an ally.  Eventually, the player could take their Android out exploring the galaxy, or switch to/ADD a Human Character along the road, or something else entirely.  I think it forms a nice small corner of a potential universe that I can develop out into the game I want it to become, realistically, with almost no organic modeling needed before a later expansion.. So, a bunch of stranded(trapped) robots stuck on a crappy moon with nothing but piles of their dead relatives and refuse to keep them busy.  Sounds cheery huh?  How about the regular patrols of humans that fly low sometimes to get some target practice in?  They're normally just supposed to watch from orbit and make sure none of those crazy bots grow any wings, but what can ya do when you're stuck out in the middle of nowhere guarding the solar systems largest garbage dump?  hehe I think it sounds way more fun to write and play. Lots more ideas brewing on this new story line, and that's good, because I really had no more interest in the brief outline I sketched up originally.   So, The Most Excellent News! I did some social media outreach and one of my good friends hit me up, he's been looking for a project to take on and it seems like this one fits his bill.  So, I'm pretty sure I'll be bringing in a partner with some very good management, networking and systems skills plus he has spare servers and colocation facilities at his disposal!    So, hopefully within the next few weeks or months I'll be optimizing my multiplayer servers on the actual internets.   Boom!  Work to do...    

Septopus

Septopus

More Progress with Design and Project Organization

Instead of one big ugly image, this time I'll give you lots of smaller ugly ones. I added a new main branch to the old Mind Map, this one describes how the actual servers will be configured.  This is a general overview of course, I'm building for a single (Physical/Virtual) server installation first then I'll add in data replication and failover capabilities later. A new server type was added as well, the Social Server, it does some fairly obvious types of things.. After looking at the big picture and, well spending some time painting it first, I started to see some ways to optimize and organize things better.  For example I've completely moved the main "Player Attitude Network Relay" or Position/Attitude Reflector (the thing that makes network player character coordinates propagate to other player clients on the internet..blah!).  It was going to just live in the Avatar Server, but that handles Authentication and quite a few other very crucial game play roles.. So now it lives where it makes way more sense, at the point in the system where its data is needed the most.  In the Action Server, this server handles all of the fast action related decision making.  The Avatar Server still handles propagating the player character's Data(features, wardrobe,etc), just not the deluge of player movement data.  This makes it far easier to design this part of the system to scale, which is important because this is one of the critical points for concerns of scale.  As long as EVERY Action Server maintains accurate positional buffers, then it doesn't matter WHICH Action Server processes the client's messages.  Keeping the positional buffers in sync will probably require the addition of high speed intermediary "Data Base" Servers  and all that jazz. I ramble, but I'm making some good progress towards a cohesive plan, and it's making everything start to feel like it's coming together. The hacknplan data entry is still much in progress, I've started adding tasks to keep myself on track with adding data to it.. haha, sounds redundant but it's helping me stay on track. Here's the Game Design Model I was talking about in my last thread, I'm enjoying the simplicity of it all. It's essentially just the tree structured view of my Mind Map, so it's pretty easy to keep these two tools in sync.  I add child items where necessary and attach tasks to whatever branch/child I want. The main "Board" view is just a standard KanBan style system, but it's simple and easy to work with, it integrates well with the Game Design Model and seems to be fairly stable. Here I'll attach the whole of the latest Mind Map revision, for the curious and/or gluttons of punishment. I'm happy with my progress so far.  Slowly approaching the Maintenance point.  Then the full code sprint starts again.  I'm still coding, so I don't lose my place, just not at the pace I would like with all the admin work I've given myself.  Anyhow, enough talking about stuff I've already done, I've got stuff to do!

Septopus

Septopus

Organizing this Unsettled World

So, I've been thinking hard about this little project of mine and I came to the conclusion that I'll never finish it if I don't get some things in order around here.  I'm not talking about smelly code either.  That's a constant cleanup task in my world.  NO, I'm talking about Project Management type thingums and stuffs..  So, I spent a few hours working up a mind map on mindmeister only to get it nearly to where I was happy and then their whole interface started failing miserably on me, browser related weirdness..  So I tried xmind, decent, but I knew it was more than I ever wanted to pay for...  FreeMind (https://sourceforge.net/projects/freemind/) to the rescue!  So, here's a little picture that's STARTING to describe all the little bits and pieces that need to be massaged into existence in order to call my game nearly functional in most of the ways I would like.  Every time I look at it I add another two or three or twelve nodes...   In addition to Beginning the Mind Map I've also started using hacknplan.com to keep track of Tasks and my Time investment. I wish very much that I was able to break my entire project down into 50 more tasks that will take less than 200hours to complete.. Very very much..  Sadly, no, this is the result of me entering a goodly portion of the Mind Map data into hacknplan's "Design Model" which is just a simple tree system.  Simple, but quite functional.  Tasks can be created out of items in the tree or attached to items in the tree.  If you are looking for a good free project management system that is designed for building games, it's working great for me so far.  These 50 tasks are probably a 10th of what I still have to add before I have the project "roughed out" in hacknplan(I'm adding in tasks for items that are in "Completed/Testing" stages as well, so I can get an idea of total time investment). I was hesitant to stop working on things for a few days to get this stuff up and running, but I'm already starting to see the benefits.  First off I'm developing a much clearer idea of what I'm actually building.  That's going to be a game changer I think.  Being able to look at the bigger picture of the whole eco-system, I'm able to re-evaluate what code goes where based on what systems need to access/interact with it, instead of a hunch based on a rough imagining of how I want things to work.  I'm also able to begin prioritizing my time and deciding if I want to try to build a project team, or continue the solo adventure.  If I want to have any help though, this whole process is going to be the absolute key to making it possible. So much more to do... I'll post an update again when I get my hacknplan setup more flushed out.

Septopus

Septopus

A Decent String Slinger.. No, not THAT GUY!.. its just a c# UDP server.

Okay, with some help over on the Network & MultiPlayer Forum I've reigned in some of the code smell in my UDP Socket Class. This is the wheel that turns each of my game servers, nearly everything else is game specific logic. I'm sharing it in it's entirety here for your use however you like.  I'll keep this post updated as I make improvements.  If I make any major changes I'll at least provide a link to them here if possible. This class is a multi-purpose UDP Server/Client class that expects and produces strings.  Now, I know that this isn't the BEST or most optimized way to deal with data, however I find it to be extremely flexible for rapid development of server logic code.  When I finalize the server logic I may rewrite the class serialization/deserialization routines to output byte[]'s directly and at that point I'll simply remove the string variable handling from this class so they only interact with byte[]'s. I mean, strings aren't all bad right? At least somebody agrees... Anyhow, here it is.  Have fun.  using System; using System.Collections.Concurrent; using System.Collections.Generic; using System.Net; using System.Net.Sockets; using System.Text; using System.Threading; namespace StringSlinginUDPServer { public class UDPSocket : IDisposable { //Constant for configuring the prevention of ICMP connection resets private const int SIO_UDP_CONNRESET = -1744830452; //UDP socket private Socket _socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp); //Async Callback private AsyncCallback recv = null; //Buffer Size Constant private const int bufSize = 8 * 1024; //Raw string data from client packets private Dictionary<EndPoint, Queue<string>> messageDictionary; //Queue for holding raw string data from server packets when in client mode. private Queue<string> cQ; //Referece to the data store used by the server(for access to the current game time clock) private TimeCodeDataProvider dataStore; //Time code storage for last sent/received messages private double lastSentMessage = 0; private double lastReceivedMessage = 0; //Boolean to determine which mode we're in so received messages get put in the right place. private bool clientMode = false; //Max string lenght allowed by the servers. private int maxMessageLength = 1450; // Generic Object Pool Class public class ObjectPool<T> { // ConcurrentBag used to store objects. private ConcurrentBag<T> _objects; private Func<T> _objectGenerator; // Object pool contructor used to get a delegate for implementing instance initialization // or retrieval process public ObjectPool(Func<T> objectGenerator) { if (objectGenerator == null) throw new ArgumentNullException("objectGenerator"); _objects = new ConcurrentBag<T>(); _objectGenerator = objectGenerator; } // GetObject retrieves the object from the object pool (if already exists) or else // creates an instance of object and returns public T GetObject() { T item; if (_objects.TryTake(out item)) return item; return _objectGenerator(); } // PutObject stores the object back to pool. public void PutObject(T item) { _objects.Add(item); } } ObjectPool<State> statePool = new ObjectPool<State>(() => new State()); //State class for async receive. class State { public string smessage; public int bytes = 0; public byte[] buffer = new byte[bufSize]; public EndPoint epFrom = new IPEndPoint(IPAddress.Any, 0); } //IDisposable stuff public void Dispose() { Dispose(true); GC.SuppressFinalize(this); } protected virtual void Dispose(bool disposing) { if (disposing) { // free managed resources if (_socket != null) { _socket.Dispose(); _socket = null; } } } //Server "Mode" public UDPSocket(Dictionary<EndPoint, Queue<string>> msgsDict, TimeCodeDataProvider DATASTORE) { clientMode = false; messageDictionary = msgsDict; dataStore = DATASTORE; lastSentMessage = dataStore.TimeCodeDoubleValue; } //Client "Mode" public UDPSocket(Queue<string> mq, TimeCodeDataProvider DATASTORE) { clientMode = true; cQ = mq; dataStore = DATASTORE; lastSentMessage = dataStore.TimeCodeDoubleValue; } public void CloseSocket() { _socket.Close(); } //Internal connection status checking public int SendHowStale() { //Console.WriteLine("lmc: " + ((int)dataStore.UWServerSeconds - (int)lastSentMessage).ToString()); if (lastSentMessage == 0) lastSentMessage = dataStore.TimeCodeDoubleValue; return (int)(dataStore.UWServerSeconds - lastSentMessage); } //Internal connection status checking public int ReceiveHowStale() { //Console.WriteLine("lmc: " + ((int)dataStore.UWServerSeconds - (int)lastSentMessage).ToString()); if (lastReceivedMessage == 0) lastReceivedMessage = dataStore.TimeCodeDoubleValue; return (int)(dataStore.UWServerSeconds - lastReceivedMessage); } //Start/Bind a Server socket. public void Server(string address, int port) { //In case restarting uncleanly, dunno if this actually does anything.. _socket.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.ReuseAddress, true); //Ensure all async packets contain endpoint info and etc. _socket.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.PacketInformation, true); //Ignore ICMP port unreachable exceptions. _socket.IOControl((IOControlCode)SIO_UDP_CONNRESET, new byte[] { 0, 0, 0, 0 }, null); //Bind to port. if (address == "all") { _socket.Bind(new IPEndPoint(IPAddress.Any, port)); } else { _socket.Bind(new IPEndPoint(IPAddress.Parse(address), port)); } //Start receive callback process. Receive(); } //Setup a Client to Server socket. public void Client(string address, int port) { //Dunno if these two options do anything for client sockets, but they don't seem to break anything. _socket.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.PacketInformation, true); _socket.IOControl((IOControlCode)SIO_UDP_CONNRESET, new byte[] { 0, 0, 0, 0 }, null); _socket.Connect(IPAddress.Parse(address), port); //Start receive callback process. Receive(); } //ServerSend sends to any EndPoint from THIS server. public void ServerSend(string text, EndPoint ep) { try { byte[] data = Encoding.ASCII.GetBytes(text); _socket.SendTo(data, ep); lastSentMessage = dataStore.TimeCodeDoubleValue; //Console.WriteLine("TO NET: " + text); } catch (Exception ex) { Console.WriteLine("ServerSend Exception: " + ex.Message); } } //Client Send only sends to the connected Server. public void cSend(string text) { try { byte[] data = Encoding.ASCII.GetBytes(text); _socket.Send(data); lastSentMessage = dataStore.TimeCodeDoubleValue; //Console.WriteLine("TO NET: " + text); } catch (Exception ex) { Console.WriteLine("cSend Exception: " + ex.Message); } } //Setup Async Callback private void Receive() { State so = statePool.GetObject(); try { _socket.BeginReceiveFrom(so.buffer, 0, bufSize, SocketFlags.None, ref so.epFrom, recv = (_Receive), so); } catch (Exception) { statePool.PutObject(so); } } //Receive Callback private void _Receive(IAsyncResult ar) { State so = (State)ar.AsyncState; try { so.bytes = _socket.EndReceiveFrom(ar, ref so.epFrom); lastReceivedMessage = dataStore.TimeCodeDoubleValue; so.smessage = Encoding.ASCII.GetString(so.buffer, 0, so.bytes); //Console.WriteLine("Msg frm ThreadID: " + Thread.CurrentThread.ManagedThreadId.ToString()); //Console.WriteLine("FROM NET: " + text); if (so.smessage.Length < maxMessageLength) { if (clientMode) { cQ.Enqueue(so.smessage); } else { if (!messageDictionary.TryGetValue(so.epFrom, out Queue<string> queue)) { messageDictionary.Add(so.epFrom, queue = new Queue<string>()); } queue.Enqueue(so.smessage); } } _socket.BeginReceiveFrom(so.buffer, 0, bufSize, SocketFlags.None, ref so.epFrom, recv, so); } catch (Exception) { statePool.PutObject(so); } } } } My Usage Looks Like(c# console app): namespace StringSlinginUDPServer { class MyServer { public static bool running = true; public static Dictionary<EndPoint, Queue<string>> MessageDictionary = new Dictionary<EndPoint, Queue<string>>(); public static Thread udpServerThread; static void UDPServer() { using (s = new UDPSocket(MessageDictionary, TimeCodeDataProvider)) { s.Server("IP.ADD.RE.SS" or "all", PortNumber); while(running) { //Servery Stuff Goes Here. //Like reiteratively dequeuing the Message Dictionary Queues and processing/replying to all commands/etc... } } } static void Main(string[] args) { udpServerThread = new Thread(new ThreadStart(UDPServer)); udpServerThread.Start(); while (running) { //Logic to check the connection status and start/restart the thread if it's dead/halted.. //Trap key sequences to close server/etc.. } } } } And here's a rough example of how I use the Client functionality, this routine checks the server via a simple UDP transaction. static bool UDPCheckOK() { bool res = false; Queue<string> udpResponseQ = new Queue<string>(); using (c = new UDPSocket(udpResponseQ, TimeCodeDataProvider)) { c.Client("127.0.0.1", PortNumber); c.cSend("SOCKETCHECKSTRING"); int wc = 0; while (udpResponseQ.Count < 1) { wc++; if (wc >= 20) { goto timeout; } Thread.Sleep(50); } timeout:; if (udpResponseQ.Count > 0) { while (udpResponseQ.Count > 0) { switch (udpResponseQ.Dequeue()) { case "SOCKETCHECKOKSTRING": res = true; break; } } } } return res; } Happy Coding!

Septopus

Septopus

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!