• Create Account

# Unreal Networking design questions

Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

18 replies to this topic

### #1Angus Hollands  Members   -  Reputation: 823

Like
0Likes
Like

Posted 10 February 2013 - 06:38 AM

Hey everyone.
I'm redesigning my multiplayer architecture to work in a more manageable structure. At present, client and server classes are separately defined and ultimately share little in common. Simple functions that interface with additional data must be separated for the sake of perhaps one function call, and it is explicitly serialised. Anyway, this is mostly network terms. What I wish to understand is how Unreal (the solution I'm building off) handles the entry points for server and client routines.

For the most part, there is little difference between how the server and client actors run; certain functions are suppressed because they're not client-side or server-side, and certain functions are simply invoked at the other end. However, the fundamental difference exists somewhere in the loop. I understand that both the Server and Client have predefined events that are called onCollision etc.

My first question is how are server and client-side functions separated if they both need to be called but do different things? For example, collision events would do different things on the server and client, If the client-side actor was autonomous it would play a sound before the server called playSound, but on the server it would tell nearby actors but the current actor to hearSound. How is this achieved?

Secondly, how are events sent to the server? It seems like they are sent in an RPC after the client's autonomous actor runs the simulation client-side. But if this is the case, how does the simulation get called? I know of a playerTick function, but how does this choose which function to run? Is it a simple if-switch? If ROLE == ROLE_AUTONOMOUS server_func() else client_func()?

Lastly, simulated functions. I know that the question has been asked many a time, but I would like to clarify their purpose.

I'm writing this solution in Python, so I have a lot of duck-typing possibilities I can exploit. I intend to use decorators to define when functions should be called on the remote side (@remote(world_info, target=SERVER)). I therefore assume that any function without a remote call can be called by either side. I think once I clarify the above points I may answer my own question, but until then what does the Simulated keyword actually mean for the client and server?

Angus.

### #2Kylotan  Moderators   -  Reputation: 3376

Like
0Likes
Like

Posted 10 February 2013 - 09:21 AM

My first question is how are server and client-side functions separated if they both need to be called but do different things? For example, collision events would do different things on the server and client, If the client-side actor was autonomous it would play a sound before the server called playSound, but on the server it would tell nearby actors but the current actor to hearSound. How is this achieved?

http://udn.epicgames.com/Three/NetworkingOverview.html#Function call replication

Each program knows its current role, whether as server, client, etc. Certain code only executes when that program is running in that specific role. Servers know to send off function execution to clients if the function is marked for the client.

Just below that part, you have: http://udn.epicgames.com/Three/NetworkingOverview.html#Simulated functions and states

Marking a function with 'simulated' means it's only executed on simulated proxies - on all other roles, it's skipped. It's not strictly about clients and servers - it's about entities on a client that are not controlled by the client. So the 'simulated' functions don't usually make decisions about gameplay, just about presentation (eg. graphical and sound effects).

So basically it's just 2 simple things:
- mark parts of the code as server or client only
- ensure the server sends appropriate messages to the client whenever it encounters some client-only code, to ensure the code is called

In terms of implementation detail, Unrealscript can basically just generate a remote procedure call for each time it sees a function that needs calling on the other side.

The key thing is that there is flexibility in how you implement it. You could choose to have 1 big function for each event that calls out to other smaller functions, and those smaller functions may or may not be implemented on the client or server. Or you might split it up so that you call one client function and one server function, and only the things in the client function will get replicated. There's no right or wrong because it's just 2 routes to 1 goal, ie. getting different bits of script to execute in different places based on process roles.

Edited by Kylotan, 10 February 2013 - 09:24 AM.

### #3hplus0603  Moderators   -  Reputation: 8344

Like
1Likes
Like

Posted 10 February 2013 - 07:13 PM

how are server and client-side functions separated if they both need to be called but do different things?

It varies between games. Some games simply stub out the graphics and sound and input parts on the server, and stub out the network-broadcast part on the client.

In general, the best thing you can do is keep your code highly modular. Define a central data structure for what a "player" is, that everybody shares. Then define optional data structures specific to each subsystem -- sound, AI, graphics, networking, etc. Create a "factory" that knows how to create the right flavor of each object depending on whether it's server or client (you could have a ClientEntitySource and a ServerEntitySource if you want.)
enum Bool { True, False, FileNotFound };

### #4Kylotan  Moderators   -  Reputation: 3376

Like
1Likes
Like

Posted 11 February 2013 - 07:38 AM

I merged the topics so that my answer came into this thread; hope that's ok for everyone.

Just to recap and to keep it simpler than my previous answer: UnrealScript handles the division between client and server code by running the same script on all machines but by annotating certain functions so that they only run when the role is appropriate. This is fine for them since they have their own scripting language and can fairly easily implement such a thing.

In Python it would certainly be possible to decorate functions so that they only run when executed under a certain role, so that aspect can be reproduced if you like. The other thing to do is have a decorator able to package up a remote function call and send it off (eg. when a server needs to tell a client to execute something).

The final part is to be able to accept incoming remote procedure calls and execute them. This would not form part of the normal update/tick script but would be done in an event-handling section before or after that.

More info on this is up on Unreal's site: http://udn.epicgames.com/Three/FunctionReplication.html - in fact it's best to read the whole of their networking and replication docs if you want to implement the system their way: http://udn.epicgames.com/Three/ReplicationHome.html

### #5Angus Hollands  Members   -  Reputation: 823

Like
0Likes
Like

Posted 11 February 2013 - 03:42 PM

Thanks to everyone. I've considered all of the points. I have read the entire documentation countless times, but certain things were not clear. I'm still not fully sure how best to define replication conditions. I don't know If I will want to evaluate a condition to determine which variables to send every time I want to replicate variables because for any serious number of clients this will become slow.

I have some ideas regarding speeding up the system. All of this is conceptual, but in my old system objects would be iterated through again when they were serialised. Instead, I plan to now use the network attributes as part of the state storage (every tick store the state of the object by copying it's network values) and only converting that to bytes every network tick, but doing so in the same iteration.

One question I have; Would it be a worse design to declare replicating attributes by name, or to use a special declare object that looks more like attribute lookup?

e.g:

class DeclaredProxy:
def __init__(self, replicator):
self.tags = set()
self.replicator = replicator

def __getattr__(self, key):
network_variables = self.replicator.network_variables
if key in network_variables:
return getattr(self.replicator, key)

class Declared:
def __init__(self, replicator):
self.proxy = DeclaredProxy(replicator)

def __enter__(self):
return self.proxy

def __exit__(self, *a, **kw):
pass

class Actor(Replicator):
a = Variable(1)
b = Variable(1)
c = Variable(1)
d = Variable(1)

e = Variable(1)
f = Variable(1)
g = Variable(1)
h = Variable(1)

def replication(self, caller):
dirty_variables = self.dirty_variables(caller)
is_initial = self.is_initial(caller)

with self.declarer as declare:
if is_initial:
declare.a
declare.b
declare.c
declare.d


Here is the yield example. It's likely less overhead, but less pretty in my opinion

class Actor(Replicator):
a = Variable(1)
b = Variable(1)
c = Variable(1)
d = Variable(1)

e = Variable(1)
f = Variable(1)
g = Variable(1)
h = Variable(1)

def replication(self, caller):
dirty_variables = self.dirty_variables(caller)
is_initial = self.is_initial(caller)

if is_initial:
yield "a"
yield "b"
yield "c"


Edited by Angus Hollands, 11 February 2013 - 04:35 PM.

### #6Kylotan  Moderators   -  Reputation: 3376

Like
0Likes
Like

Posted 11 February 2013 - 05:33 PM

The time it takes to have a CPU decide whether to send a variable or not is always going to be dwarfed by the time it takes for the network adapter to put it onto the network. So bear that in mind when worrying about what is efficient and what is not.

As for your code examples, I don't really understand what the difference is. Looks like you're worried about micro-optimisations. The most important thing is how simple it is for the user to mark which variables must be replicated or not. You shouldn't need any special code in the Actor class apart from whatever marks or tracks those relevant variables.

### #7Angus Hollands  Members   -  Reputation: 823

Like
0Likes
Like

Posted 13 February 2013 - 06:02 AM

I do care about the time spent checking attributes for changed values etc, because that shouldn't be the largest limiting factor.

Also, I am a little perplexed after checking the UT3 source unrealscripts. I am looking at WorldInfo.uc and it has a replication block. Until now I assumed replication was for actors only. As this seems not to be the case, I was wondering how it is so seamlessly consistent between different class types.

1. Specifically, I wish to know how it distinguishes between already instanced and non-instanced classes. Take Actors. I would imagine that when receiving a replication packet on the client for an actor would simply be directed to the actor manager, and if the id wasn't instantiated already it would create the actor. However, this cannot apply to non-actors such as WorldInfo which don't inherently have an ID.
2. To achieve automatic replication, would it be the case that every time something inherits from the Replication base class, it registers the channel ID to a special mapping; so that every time something is instantiated of that type it adds its network ID to that map? This means that replication would actually interface with replicated classes by itself, without needing to be performed explicitly for each class.

Continuing on. The main issue that I face is that UnrealScript is built on top of the engine, and so it doesn't need to define entry points to certain functions like collisions and physics updates. Therefore I believe I need to create an interface layer between the various aspects of my Game Engine and the networked "game" that exists separately to this. For example, defining a network role as Simulated will not be implemented as any functional movement code. Therefore I have to define that myself. This means that I will need to keep a history of "states" of the actors for each game tick to allow for extrapolation using EPIC

I was reading into Unreal and I realised that they send the inputs, and expected outcome and predict between confirmation. At the moment I just send the inputs and skew the time so that it matches closely to when the inputs are received. I think that the first method would be better because then it allows me to avoid guessing and get fewer prediction errors, but at the cost sending three floats every network tick to the server. Does anyone have any ideas if this is a better method? It seems like I've been thinking about movement code wrongly and this is actually how most people do this. I would also send the time taken to process the inputs, but this of course opens the possibility for speedhacking. Are there better solutions?

Also, some other questions about Unreal network attributes.

1. There is bNetDirty which I believe is a simple attribute stored on each actor instance and changes when any networked values are changed. If this is the case how do they stop position and orientation from setting it as it seems to suggest it's optionally triggered? As well as this, It doesn't seem to handle specific attributes, instead it seems that if any attributes change, all would be sent if the replication block checks for bNetDirty only. I assume that the system only sends the changed attributes, so is that caught after the replication block?
2. There is also bNetOwner and bNetInitial. These are only available inside the replication block according to the documentation, and so I assume that bNetDirty is available outside. My hunch is that because attributes are reliably transmitted, if they are told to be replicated because they are dirty, they will eventually get to the other side. Why is it then that bNetInitial is only set False when it receives an ACK? Because if its reliable, the initial attributes will eventually get to the other side? It seems contradictory to me.

Edited by Angus Hollands, 13 February 2013 - 08:48 AM.

### #8Kylotan  Moderators   -  Reputation: 3376

Like
1Likes
Like

Posted 13 February 2013 - 07:50 AM

I do care about the time spent checking attributes for changed values etc, because that shouldn't be the largest limiting factor.

But it never will be. A modern computer can perform 100,000,000,000 operations per second but can typically only write 1,000,000 bytes per second to the network adapter. Deciding what to send if that decision is a simple yes/no as in this case is never going to be the limiting factor.

Also, I am a little perplexed after checking the UT3 source unrealscripts. I am looking at WorldInfo.uc and it has a replication block. Until now I assumed replication was for actors only. As this seems not to be the case, I was wondering how it is so seamlessly consistent between different class types.

I don't own UC3 nor have I seen the source code so anything I say here would just be speculation. But I suspect WorldInfo and GameInfo are subclasses of Actor, or of some other base class that actually permits replication. I doubt there's any hidden magic here.

Specifically, I wish to know how it distinguishes between already instanced and non-instanced classes. Take Actors. I would imagine that when receiving a replication packet on the client for an actor would simply be directed to the actor manager, and if the id wasn't instantiated already it would create the actor. However, this cannot apply to non-actors such as worldInfo which don't inherently have an ID.

It's likely to be a lot simpler than that - when the server creates a new Actor, it tells each client to create a new one of the same type with the same ID. The same goes for destroying them.

### #9Angus Hollands  Members   -  Reputation: 823

Like
0Likes
Like

Posted 13 February 2013 - 08:56 AM

Thanks Kylotan.

Well, when I started looking at logic usage that started to be the largest factor. If you have 6 players and 100 Actors in total, thats 6 * 100 checks per update. Which is a lot if its dict checking etc.

In terms of creating a new actor, what happens if a client joins after the actor is created? I'm used to just having some check on the other side to see if we have the player registered otherwise we create it. Would you suggest that when the server evaluates the replication for the new client it invokes a create function on the client? I'm not really wanting to add RPC functionality beyond the scope of the actors themselves (using directed methods mentioned before) so I would imagine the client would simply deduce if the actor existed and if not create it. The reason I asked this question was because I originally believed that the WorldInfo class did not derive from Actor and So I wondered what sort of black magic was taking place to match the instance on the server to the instance on the client. I think I shall let that wait for now.  (the replication of world info).

Another question about reliability. can I confirm that whenever a reliable packet is attempted to be sent to a peer, it stores it in a reliable buffer, and checks the ACK packets to see if it got there, else it will resend it. If there is not enough bandwidth, it will ramp up the netpriority until it forcibly enters the outgoing queue. How does this deal with out of order delivery? I've yet to notice any signficant packet loss in my experience, so I've never worried considerably about order of packets, but if you resend a packet it then creates an out of order sequence on the receiving end. Should I drop the packet but ACK its receipt?

Also, I updated my post above to reflect my evolving thought processes! I shall quote it here, if you would be as kind as to read it!

Continuing on. The main issue that I face is that UnrealScript is built on top of the engine, and so it doesn't need to define entry points to certain functions like collisions and physics updates. Therefore I believe I need to create an interface layer between the various aspects of my Game Engine and the networked "game" that exists separately to this. For example, defining a network role as Simulated will not be implemented as any functional movement code. Therefore I have to define that myself. This means that I will need to keep a history of "states" of the actors for each game tick to allow for extrapolation using EPIC

I was reading into Unreal and I realised that they send the inputs, and expected outcome and predict between confirmation. At the moment I just send the inputs and skew the time so that it matches closely to when the inputs are received. I think that the first method would be better because then it allows me to avoid guessing and get fewer prediction errors, but at the cost sending three floats every network tick to the server. Does anyone have any ideas if this is a better method? It seems like I've been thinking about movement code wrongly and this is actually how most people do this. I would also send the time taken to process the inputs, but this of course opens the possibility for speedhacking. Are there better solutions?

Also, some other questions about Unreal network attributes.

Aside from bNetDirty there is also bNetOwner and bNetInitial. These are only available inside the replication block according to the documentation, and so I assume that bNetDirty is available outside. My hunch is that because attributes are reliably transmitted, if they are told to be replicated because they are dirty, they will eventually get to the other side. So, bNetDirty can be set to False immediately after replication because reliability is just a delay in when attributes arrive. Why is it then that bNetInitial is only set False when it receives an ACK? Because if its reliable, the initial attributes will eventually get to the other side? It seems contradictory to me.

Edited by Angus Hollands, 13 February 2013 - 02:05 PM.

### #10Inverness  Members   -  Reputation: 115

Like
0Likes
Like

Posted 15 February 2013 - 01:37 PM

Thanks Kylotan.

Well, when I started looking at logic usage that started to be the largest factor. If you have 6 players and 100 Actors in total, thats 6 * 100 checks per update. Which is a lot if its dict checking etc.

In terms of creating a new actor, what happens if a client joins after the actor is created? I'm used to just having some check on the other side to see if we have the player registered otherwise we create it. Would you suggest that when the server evaluates the replication for the new client it invokes a create function on the client? I'm not really wanting to add RPC functionality beyond the scope of the actors themselves (using directed methods mentioned before) so I would imagine the client would simply deduce if the actor existed and if not create it. The reason I asked this question was because I originally believed that the WorldInfo class did not derive from Actor and So I wondered what sort of black magic was taking place to match the instance on the server to the instance on the client. I think I shall let that wait for now.  (the replication of world info).

Another question about reliability. can I confirm that whenever a reliable packet is attempted to be sent to a peer, it stores it in a reliable buffer, and checks the ACK packets to see if it got there, else it will resend it. If there is not enough bandwidth, it will ramp up the netpriority until it forcibly enters the outgoing queue. How does this deal with out of order delivery? I've yet to notice any signficant packet loss in my experience, so I've never worried considerably about order of packets, but if you resend a packet it then creates an out of order sequence on the receiving end. Should I drop the packet but ACK its receipt?

Also, I updated my post above to reflect my evolving thought processes! I shall quote it here, if you would be as kind as to read it!

I'm also using Unreal's networking model as inspiration for my own project, though I'm using C# for it.

Anyhow, a few points:

Actor checking: I suggest adding actors that could possibly be replicated (that aren't already provided by the level itself) to a separate list to be checked. You would further eliminate unnecessary checks by allowing actors to specify a replication rate so less important actors need not use as many resources.

And yes, if a client joins, the server would evaluate replication for that client and send it any actors that are relevant to it and they have not already been replicated to it.

As for reliability, I am using the Lidgren library for C# and used reliable sequenced delivery so packets always arrive but old packets are dropped in favor of newer ones.

As for Unreal networking attributes, bNetDirty is set whenever a property is set on an object in UnrealScript, it is a helper variable so you can skip replication if its false. bNetOwner is true when the client you're replicating to owns that actor, and bNetInitial is true when it is the first time you're replicating so you can send variables that only need to be sent once.

My current source code is available, it might be of some help or it might confuse you.  ReplicaManager handles replicating my actors through the IReplica interface. And you can look at my MovementBehavior component to see how player movement is handled. I just copied how Unreal did it in that case.

Edited by Inverness, 15 February 2013 - 01:42 PM.

### #11Kylotan  Moderators   -  Reputation: 3376

Like
0Likes
Like

Posted 16 February 2013 - 03:55 PM

Well, when I started looking at logic usage that started to be the largest factor. If you have 6 players and 100 Actors in total, thats 6 * 100 checks per update. Which is a lot if its dict checking etc.

It doesn't matter how many characters you have - whatever extra work you do checking which values need to be sent should always be outweighed by the actual sending. If that isn't what you were seeing, then I suspect you were either doing something very strange or testing under quite odd conditions.

In terms of creating a new actor, what happens if a client joins after the actor is created?

When a client joins, you send them all the actors that they need to know about.

I'm used to just having some check on the other side to see if we have the player registered otherwise we create it.

How can you create the actor without having all the information necessary for them yet?

Another question about reliability. can I confirm that whenever a reliable packet is attempted to be sent to a peer, it stores it in a reliable buffer, and checks the ACK packets to see if it got there, else it will resend it. If there is not enough bandwidth, it will ramp up the netpriority until it forcibly enters the outgoing queue. How does this deal with out of order delivery?

This isn't strictly a question about the Unreal networking system, is it? How best to implement reliable messaging over an unreliable transport
is best asked in another thread. You do need to ACK whatever you receive though, because you don't want the other side resending messages you already have.

...but at the cost sending three floats every network tick to the server.

Three floats per network tick is nothing. It sounds like you're prematurely optimising.

Does anyone have any ideas if this is a better method? It seems like I've been thinking about movement code wrongly and this is actually how most people do this. I would also send the time taken to process the inputs, but this of course opens the possibility for speedhacking. Are there better solutions?

Why is it then that bNetInitial is only set False when it receives an ACK? Because if its reliable, the initial attributes will eventually get to the other side? It seems contradictory to me.

Presumably the engine needs to be sure that the data has arrived before it can move forward with other logic. Exactly why, I don't know.

### #12Angus Hollands  Members   -  Reputation: 823

Like
0Likes
Like

Posted 17 February 2013 - 08:33 AM

For client side replication: If we receive a replication "packet" from the server, it is denoted by the ID of the entity replicating, and any replicated variables.

If we don't have the ID instantiated locally on the client, we create it and set the replication data.

Of course, I would have to have the server send the full replication because it wouldn't send any previously modified values.

You mention actors that exist already in the level. This isn't a concept that has really applied to me as yet, because I have not considered a scenario where I would want level actors that aren't added depending upon the client connected. Could you possibly elaborate? I saw mention in the Unreal source but I wasn't sure when this would be useful to me.

The question reliability was specific to Unreal (or Tribes) with net priority; I was asking if that is the correct understanding of their model. Out of order delivery was asking how Unreal handles it (although this is related to the typical "How do I order an unreliable protocol"! I wouldn't mind simply dropping out of order packets except when there are delta variables (e.g the difference vector between two velocities) for more precision, which require access to the last state. Additionally, this does not seem to be how Unreal handles it.

My other vague question was about the age of variables. It seems as though there is no specific support for determining the time for which the attributes were valid. This is fine but for extrapolation where I need to know the age of the position and velocity vectors to be able to use EPIC. I think that It would make sense to be able to lookup past states at a particular game tick (a synchronised concept of time) so should I implement this within the replication system, so the networking layer has its packet sequence number and the replication also sends the game tick with the replicated data on top of that protocol?

Let me add some explanation as to why I appear to be so concerned with premature optimisation. I aim at sixty frames per second game frame rate. That leaves 16ms for all processes to execute in. Assuming 1ms is used for the render pipeline on average, that leaves 15ms. That's a lot of time if you write good code, and typically you'll have a fraction of a ms per client. However, this scales geometrically (I believe) with connection of new game-observing clients, as they have to be sent all other actors. Therefore .2 ms may seem small, but when that starts growing with just the scraper logic that seems dangerous. This said, these numbers are merely demonstrating the idea and are not representative of the actual model and this is without relevancy checks.

Seeing as you mentioned your own progress, do you allow any non-actor classes to use replication? or have them handle their own data (e.g the scoreboard, clock)

My final question so far is should the server store the state of all actors every tick? I cannot think immediately of why it would save for replaying, but does Unreal store the "state" of each actor in a lookup by game time?  And if so, is it every game tick or every network tick? It seems now would be the time to think about how that would be done, not further down the line.

And thanks for the example source, I'm sure it will be very useful!!

Edited by Angus Hollands, 17 February 2013 - 08:46 AM.

### #13Inverness  Members   -  Reputation: 115

Like
1Likes
Like

Posted 17 February 2013 - 12:27 PM

You mention actors that exist already in the level. This isn't a concept that has really applied to me as yet, because I have not considered a scenario where I would want level actors that aren't added depending upon the client connected. Could you possibly elaborate? I saw mention in the Unreal source but I wasn't sure when this would be useful to me.

Actors that already exist on the client would be those that are added to a level in the editor, which both the server and client would have a copy of in Unreal games.

The question reliability was specific to Unreal (or Tribes) with net priority; I was asking if that is the correct understanding of their model. Out of order delivery was asking how Unreal handles it (although this is related to the typical "How do I order an unreliable protocol"! I wouldn't mind simply dropping out of order packets except when there are delta variables (e.g the difference vector between two velocities) for more precision, which require access to the last state. Additionally, this does not seem to be how Unreal handles it.

I recommend using RakNet if possible because it can handle reliability issues for you. Unreal compresses its vectors send sends the full value over the Internet, not a delta from the last state. This way the packet can be ignored if it arrives late or not at all.

My other vague question was about the age of variables. It seems as though there is no specific support for determining the time for which the attributes were valid. This is fine but for extrapolation where I need to know the age of the position and velocity vectors to be able to use EPIC. I think that It would make sense to be able to lookup past states at a particular game tick (a synchronised concept of time) so should I implement this within the replication system, so the networking layer has its packet sequence number and the replication also sends the game tick with the replicated data on top of that protocol?

Unreal handles player movement separately from the rest of its replication using RPC. See this section: http://udn.epicgames.com/Three/NetworkingOverview.html#Player Movement and Prediction

Let me add some explanation as to why I appear to be so concerned with premature optimisation. I aim at sixty frames per second game frame rate. That leaves 16ms for all processes to execute in. Assuming 1ms is used for the render pipeline on average, that leaves 15ms. That's a lot of time if you write good code, and typically you'll have a fraction of a ms per client. However, this scales geometrically (I believe) with connection of new game-observing clients, as they have to be sent all other actors. Therefore .2 ms may seem small, but when that starts growing with just the scraper logic that seems dangerous. This said, these numbers are merely demonstrating the idea and are not representative of the actual model and this is without relevancy checks.

This is why you need to be conservative in determining which actors in a game need to be replicated and how often they need to be replicated.

Seeing as you mentioned your own progress, do you allow any non-actor classes to use replication? or have them handle their own data (e.g the scoreboard, clock)

In Unreal, the score board would be send to the client in WorldInfo which is a special actor representing the game world. You could also simply not use replication for things like that.

Right now I have 3 modes of communication in my game:

Normal replication happens only between actors and 5 times per second. This is done reliably, but order is not guaranteed and older packets can be ignored by the client. Example:
public override void Serialize(DeltaSerializer s, IReplicaClient destination)
{
base.Serialize(s, destination);

s.Write(_collideTiles);
s.Write(_collideActors);
s.Write(_blockActors);
s.Write(_layer);
}

public override void Deserialize(DeltaSerializer s, IReplicaClient source)
{
base.Deserialize(s, source);

}


Event messages can be sent manually between actors, which is what I use for my complex things like player movement or updating the inventory of an actor. For movement, this is done for every game update, which means 30 times per second as opposed to 5 for normal replication.
private void MoveAutonomous(Vector2 delta)
{
// Only call this on the client when it is autonomous
Debug.Assert(Actor.Role == NetRole.Autonomous);

Move(delta);
Vector2 newPos = Actor.Position;

// Save moves so we can replay them later if the server corrects us

// Tell the server to do the same move and confirm or correct our own
NetOutgoingMessage message = CreateEventMessage(EventServerMove);
message.Write(delta);
message.Write(newPos);
message.Write(_time);
message.Write((byte)_direction);
SendEvent(message, NetDeliveryMethod.UnreliableSequenced);
}

Player messages can be sent through through the player objects that represent clients in a game. The client only has its own local player object that can send to the server, but the server has one for each player logged in the game to communicate with its local counterpart.

My final question so far is should the server store the state of all actors every tick? I cannot think immediately of why it would save for replaying, but does Unreal store the "state" of each actor in a lookup by game time?  And if so, is it every game tick or every network tick? It seems now would be the time to think about how that would be done, not further down the line.

And thanks for the example source, I'm sure it will be very useful!!

For each combination of a client and an actor relevant to it Unreal stores the values previously sent to that client. Replication is done by comparing the current value in an actor to the last sent value for each connection. This happens during network ticks only.

Edited by Inverness, 17 February 2013 - 12:35 PM.

### #14Kylotan  Moderators   -  Reputation: 3376

Like
0Likes
Like

Posted 17 February 2013 - 01:46 PM

For client side replication: If we receive a replication "packet" from the server, it is denoted by the ID of the entity replicating, and any replicated variables.
If we don't have the ID instantiated locally on the client, we create it and set the replication data.
Of course, I would have to have the server send the full replication because it wouldn't send any previously modified values.

So the server needs to know that the client doesn't have the actor, in order to send the full replication the first time around. Which means it's not sufficient to just create an actor whenever you get a replication message, because not every replication message is going to be a full set of data. This is why I always have explicit create/update/destroy messages.

You mention actors that exist already in the level.

If I join a game late, what are you going to send? I need some way of knowing who is playing and what their current properties are.

My other vague question was about the age of variables. It seems as though there is no specific support for determining the time for which the attributes were valid.

You can assume the attributes were valid at the time the message was sent. So timestamp the message.

But there is a good argument for handling movement with a separate RPC system, as Inverness has alluded to. Those values change so often and are so important to games that it's often worthwhile to have a special case for them.

Seeing as you mentioned your own progress, do you allow any non-actor classes to use replication? or have them handle their own data (e.g the scoreboard, clock)

If something has properties that need replicating on a regular and/or arbitrary basis, I designate them as an entity that replicates. Otherwise I'll handle them with an RPC (request for the other side to do something) or an event (notification that something has happened). I wouldn't handle a clock as a replicating entity - latency and jitter make that rather unreliable.

### #15Angus Hollands  Members   -  Reputation: 823

Like
0Likes
Like

Posted 17 February 2013 - 03:58 PM

I intend to handle Autonomous movement with an RPC. However "simulated proxy actors" can run using extrapolation between ticks, and so I need to handle that, hence needing previous state access.

Here are my initial ideas about how the system should network data:

1. Nearly (?) everything that should be able to interface over the network  should derive from the Replicable base class

2. This will establish a network ID on the authority for that actor (server-side for nearly all cases) which is replicated to the non-authorities

3. A Generic actor network handling layer should exist. This will handle  network scraping / RPC invocation. Essentially it will scrape attributes and rpc calls (which are invoked, not collected) and bundle the network ID of the actor calling / sending the data. This is then reconstructed on the other side of the network.

If this is the case, there are some issues I can foresee:

1. How should I support RPC / replication from actors that are static. Take for example the clock. Both server and client need a clock instance, but the server is the authority on game time. How would you designate a network id here? (Ironic that I answered my previous question relating to pre-existing actors). The idea at the moment is that the ROLE_AUTHORITY assigns the network ID. In non-static cases actors will not exist with that ID and so they will be recreated by the replication system. Yet, in the case of statics, they will exist but they will not be aware of their ID outside of the authority. One method I could consider is simply not implementing it client side. If everything that should be replicated is created on the server and not on the client I would enforce an authority-instantiates principle, and clients would delete all actors, including bStatic. However, in doing this I would leave an incomplete system client side. How would I access game-specific actors (like game rules)? Just check if they exist somewhere? I suppose another solution is to preset the network ID on the actor, so it can be matched. Any ideas on how you / Unreal does it would be most welcome.
2. I have been asking the same question about global access for some time now. I think I need to break some fundamental ideas I have had from my time spent programming. In Python I have always been averse to using global scope namespaces. globals() repulses me in a lot of ways (it seems like most of the time if you're using it you shouldn't be). I think It is that I prefer direct passing of data (invoking functions etc), The same applies to using a module namespace. Is this wrong? I like to think of the scripts has being totally modular. Before the entry point is called, no instances of any data exist. But, the entry point has its own local namespace (it's in a function) so It usually means I have to pass a direct references to the rest of the functions that require these instances. The reason that I ask this is because when an Actor is created, it has to be told about the network interface instance, which will usually have been created in a local namespace and a reference stored inside an instantiated class.

Firstly, let me assume that I am correct in stating the following about RPC calls:

1. Simulated functions can be executed by an actor with any ROLE
2. Non-simulated functions require ROLE_AutonomousProxy or higher to be executed

3. Functions can only be replicated between two actors (with a ROLE / REMOTEROLE == ROLE_AUTHORITY). And thus servers cannot broadcast.

Therefore, I ask, According to point 3, replicated functions can only be replicated from actors of the same net ID and with one of ROLE, REMOTEROLE == ROLE_AUTHORITY. So, how is this controlled to prevent cheating? I could just check the source of the call on the server, and match the call address to the address that the server believes controls that client.This would prevent the client from invoking calls on other actors it doesn't control, but it would still be able to call the function on the server actor of its local agent from any other actor in its world. This would mean someone could write a special bot that would invoke a server_move() call when the bot saw the local player, allowing the local player to avoid getting shot as easily. How would one create a secure cheat-proof (at the basic level at least) RPC system? Is this something that is enforced client side?

Separation:

My last, and arguably biggest question is about design. I'm not sure how to structure the system; where to set the entry point; should it be the same for client and server. What rule should I follow to know when to separate or join logic? This is the most difficult question I face, and It is rather disconcerting if not upsetting that I cannot seem to think around it.

Fundamentally, I struggle to understand why you would go to the extra effort of writing an abstraction system that enables the use of the same Actor class in each game mode, and then explicitly define server and client code.

1. Should I think of Server and Client as fundamentally different, with no common ground
2. Should I think of Actors and replication as a singleton; providing an easy scripting system for people extending games

The specific issue is with the netmode enum. It seems stupid to have it on the client when you would just create a client flavour with the netmode set already. This seems to suggest that there is more of a factory going on than explicit differences between client server. This applies to the network layer (which has basic protocols and responses, which are only called by certain peers (for example request_connect is only received by a server). Or, should everything make use of the netmode and there is just a single game class (With the conditional... I think I will do this unless you offer advice against it). Any detailed help here would be much appreciated. Much appreciated.

Edited by Angus Hollands, 18 February 2013 - 05:39 PM.

### #16Kylotan  Moderators   -  Reputation: 3376

Like
0Likes
Like

Posted 19 February 2013 - 09:49 AM

I intend to handle Autonomous movement with an RPC. However "simulated proxy actors" can run using extrapolation between ticks, and so I need to handle that, hence needing previous state access.

Previous state and RPCs aren't mutually exclusive. RPCs are just another way of getting information across the wire - how you store it at the other end is up to you.

How should I support RPC / replication from actors that are static. Take for example the clock. Both server and client need a clock instance, but the server is the authority on game time. How would you designate a network id here? (Ironic that I answered my previous question relating to pre-existing actors). The idea at the moment is that the ROLE_AUTHORITY assigns the network ID. In non-static cases actors will not exist with that ID and so they will be recreated by the replication system. Yet, in the case of statics, they will exist but they will not be aware of their ID outside of the authority.

I don't know exactly what you mean by 'static' here but I don't see any reason why a clock would not work exactly the same as any other entity you want to replicate. If, for some reason, it needs to exist on both sides before communication happens and therefore they have different IDs for it, just have some sort of mapping from one to the other. eg. Clients create a clock proxy which can update the local clock. Or just assign the network ID at some point after creation when the server tells the client what the clock's ID is. You can have local IDs and remote (ie. server) IDs and map one to the other when the server takes control of something. It's a non-issue really.

The reason that I ask this is because when an Actor is created, it has to be told about the network interface instance, which will usually have been created in a local namespace and a reference stored inside an instantiated class.

Sounds like a bad idea to me. Actors don't need to know about networks. A separate object can mediate between actors and networks.

how is this controlled to prevent cheating?

The same way everybody checks security of data from client -> server - look at the arguments to the function and see if they make sense.

The fact that it is wrapped up in a replication or RPC abstraction doesn't change the underlying situation, which is that an untrusted remote client has sent a message which may or may not be legitimate. So you treat it as a request with potentially malicious data and handle it conservatively.

I could just check the source of the call on the server, and match the call address to the address that the server believes controls that client.This would prevent the client from invoking calls on other actors it doesn't control, but it would still be able to call the function on the server actor of its local agent from any other actor in its world. This would mean someone could write a special bot that would invoke a server_move() call when the bot saw the local player, allowing the local player to avoid getting shot as easily. How would one create a secure cheat-proof (at the basic level at least) RPC system? Is this something that is enforced client side?

There's no foolproof way to stop a player from modifying their client to instantly react based on incoming game data. This is exactly what bots are, really.

Fundamentally, I struggle to understand why you would go to the extra effort of writing an abstraction system that enables the use of the same Actor class in each game mode, and then explicitly define server and client code.

The Unreal system is just one way of approaching the issue. There is no right and wrong. I don't do it the Unreal way, partly because it is heavily tied in to having your own scripting system, and also because it is too flexible with many ways of achieving the same goals, which I don't like. Personally I try to just have a shared data model, not a shared behaviour model.

The Unreal system is almost 20 years of legacy code. Lots of it is probably only still in there because it's too expensive to remove or rework it.

Ultimately though you will always have some behaviour that is shared across client and server and some that is not. Deciding how to factor that out is something that all developers face. In a way I am lucky - since my client is in C# and the server is in Python, the 2 are necessarily completely separate. The price is a small bit of extra work, but the benefit is that there is no doubt how it needs to be implemented nor any time lost to considering the architectural issues.

### #17Angus Hollands  Members   -  Reputation: 823

Like
0Likes
Like

Posted 20 February 2013 - 10:14 AM

I don't know exactly what you mean by 'static' here but I don't see any reason why a clock would not work exactly the same as any other entity you want to replicate. If, for some reason, it needs to exist on both sides before communication happens and therefore they have different IDs for it, just have some sort of mapping from one to the other. eg. Clients create a clock proxy which can update the local clock. Or just assign the network ID at some point after creation when the server tells the client what the clock's ID is. You can have local IDs and remote (ie. server) IDs and map one to the other when the server takes control of something. It's a non-issue really.

Static is the Unreal term for actors that are not "removed" from the world on startup, and so exist without a network ID (they're not created through replication). By assigning the network ID later, the point is null. I need to have a matching Network ID because that is how the owning peer maps the data to the receiving peer. This should be possible in both directions. I like the proposition of a proxy. Alternatively, I could just go with a static_id and be done with it. That would perhaps make things easier. I don't want to introduce a special case for a specific actor class. I would like it to work for more than one instance. I shall have to think about it.

I think I will work with a registration system for actors, whenever the Replicator base class (or whatever I called it) is instantiated it adds the current class reference to the base class list. I could even add methods to get by type etc.

Do you think I should add a special replication case for Actor references? So that it can map them across the network? Or should I just stick with using their NET ID and allow the client to figure out using the getter methods.

### #18Sudi  Members   -  Reputation: 761

Like
0Likes
Like

Posted 20 February 2013 - 12:25 PM

May i ask, is your game object the actor with the network id? if so you might want to rethink that. Somewhere in this post i saw python code with a "proxy" declaration, that seams to be the better idea. Seperate your actual game objects from the networked objects. That way you don't have to worry about special cases like "static" actors. They just exist...have different ids but the actor syncs to it.

It as well saves you a lot of trouble with your local entities.

### #19Angus Hollands  Members   -  Reputation: 823

Like
0Likes
Like

Posted 21 February 2013 - 09:22 AM

At the moment It won't be the case. Actors are just replicated instances with methods and data. They can hold a reference to a child object as part of their data but the object isn't itself the actor.

I'm a little unnerved by a dependancy that I've introduced. Allow me to explain.

Nearly everything is replicated using the actor replication methods. This includes the worldinfo class. However, there is a problem. The WorldInfo class is needed by the replication system to know whether to call variable replication methods that pipe the serialised variables to the peer (only servers should be able to replicate variables) and it is also needed for function replication.

I planned on making the static (already instantiated) worldinfo class available by using a class attribute (__actors__) which can store all actors that inherit the replicable base class. This avoids the need for a handler to store the actors, and we can take the base class and read the registered actors from it.

Anyway, regardless of how I map the worldinfo client instance to the server instance, does anyone see any qualms with this. In some ways, it makes sense because replication can still occur to the peer, just not from it, and that would ensure that nothing was invoked until the client knew it was a client...like a setup routine. But if anyone can offer a nicer idea, please, by all means!

definition: game-object = Physical Game-Engine object; mesh, objects can also run gamelogic but I don't like it over python

Going back to the local game-objects, How would someone else handle them?

The method that I am considering is a Physics component. Essentially, it's an optimised way of packing and unpacking position, velocity and rotation (see here http://www.pasteall.org/39896/python) that I could have controlling the mesh. For example, instantiating the physics component as a network attribute like this:

physics = ReplicatedVariable("ObjName")


and this would create an instance of that GameObject for this actor's physics component. When the actor is replicated, it will do the same for the other side of course. Then when you need to have the object update its position you can call various methods on the physics component. (Component probably isn't the best name for it). Whenever the component is updated through replication the gameobject can be updated at the same time.

Unreal has a lot of behind the scenes work that can serialise references to actors, and so forth. I was considering doing this; converting to and from network IDs (it may be more hassle than its worth, but I was thinking that this may be useful for replicated weapons). Do you think that it would be better to recreate actor relationships automatically (which wouldn't take a whole lot of work) or force the user to handle the network id? I think it's a bit of a no-brainer but I may as well ask.

Edited by Angus Hollands, 21 February 2013 - 09:23 AM.

Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

PARTNERS