Jump to content

  • Log In with Google      Sign In   
  • Create Account

#ActualAngus Hollands

Posted 23 January 2013 - 11:04 AM

I think, partially because of the language barrier, I have some undefined terms that I need to justify.
Firstly, It's pretty easy to serialise in Python. Attribute lookups aren't too difficult, and in most cases I'd use class attributes that store such identifiers of attributes.
"Serialising" is a loosely defined term. I am not using conventional serialising libraries such as Pickle or JSON because they provide too much extra padding and information that I do not need. Because all data in this system is built from defined attributes that are available to all parts of the system, it is easy to create a format string that can run the struct library unpack and pack methods.

Events and packets. Well, as I've said, I'm using Python, and Python includes a socket wrapper in the standard library. The nature of this means that there is little immediate concept of anything beneath the socket layer. Simply input a destination and a payload and it will arrive (hopefully) at the other end. If I didn't send have data that needed to be queued, and only had data that was needed to be sent every network tick, then this might be an alright method: I can call a socket.sendto function and send the data, but when sending a reasonable amount of data, it is faster to store it in a bytes buffer and send that in one socket call. (this does increase the damage done by packet loss). Because of this, things that are handled by the library that allow me to read each payload from the buffer as individually sent bytes are no longer applicable. They simply ensure my "container" gets from A to B (hopefully). So, I introduced the concept of a "packet". A packet essentially consists of: Default header: size in bytes, Optional "protocol" header (defined by user): packet type, Packet payload. Each packet can optionally nest a sub-packet as its payload. There is no point to allow recursive unpacking because it will always be a fixed depth, and you would need to access the protocol from somewhere. Omitted from my sample code, the nested packets would have a protocol argument which accepts a protocol instance with read/write methods. A depth of three would do the following:

  • Uppermost layer - contains the full integer game tick that the packet started "filling" from
     
  • Centre layer - contains N entries of tick-container packets: packets which have the delta tick since the base tick determining their creation tick
     
  • Lower layer - contains the event data for each game tick specified in the centre layer .

This is all represented by the XML diagram. Furthermore, using automatic detection for nested packets would add to overhead and packet size. (Overhead required to find the protocol for the packet, and then extra bytes to add a type for the packet (e.g 0 = EmptyContainer, 1=MiddleContainer ...). However, as I state this, I would be interested to see how much overhead. It would still make the process more convoluted though.

Prioritisation and state
I am trying to write a system that allows for some deviation from the FPS genre. It's quite easy to do in a lot of ways, save for some genre specific stuff.
Firstly, I intend to have automatic attribute flags. In other words, when the entity class is created:

  • It reads all the attributes that are networkable, and sorts them by type (and name within type).
     
  • It saves the ordered names of these attributes to a defined variable.
     
  • It creates a bitfield for any booleans to pack them more efficiently. Each boolean class is registered with the index and the bitfield that will store their values, and their set and get methods of the descriptor will point to that value.
     
  • It creates a format string for struct using the type of the initial value for each attribute, When booleans are found to exist, it adds a Char formatter for each 8 bools. The format string then removes booleans and uses the boolean bitfield(s). I also prepend a contents bitfield (or two) which indicates which attributes are "dirty". The dirty flag is set whenever the attribute is set.

In writing this, I intend to demonstrate that dirty flags would be relatively simple to implement. Whenever the properties are changed, we just read the dirty bitfield, and if it is > 0 it needs updating. At present, updating would require sending all the data, but I could modify it so that it can dynamically send different parts of the attributes

The reason that I am going through all of this "complex design" is twofold.
Firstly, there are few resources implemented in Python. There are basic parts in C++ that are simply not reproducible with the same simplicity as C/C++ and in some cases it would be slower to try and do so. Furthermore, I want the system to be flexible. I already have a working game, but it is very chunky. It doesn't feel clean and I don't like that. For example, at the moment I cannot set "dirty flags" automatically. The entities house the to/from bytes methods, and they don't use networked attributes, instead they just scrape the data. This makes a client entity and a server entity very different, with little code in common (when it really should be in many parts).
As well as this, I want to do things correctly. I've been learning about authoritative networking for under a year, and after having rewritten things a number of times, I want to do them the best way I can. I feel like it is more appealing to a C++ workflow than Python in a lot of ways. I'm not sure where this underlying feeling comes from. Perhaps it is the ease of which one can do something badly and have it work.

There are a few more things I need to ask.

  1. Generically speaking, I have two options regarding the client. Firstly, I could attempt to run the same logic as on the server, but using server corrections. This would include all AI calculations etc. The other method is just to extrapolate data. The second method would be harder to do automatically, and It may be best to add some form of manual defined extrapolation that I can easily hook in without modifying the system too much. I suspect that the second option is the one I should choose, simply for calculations sake. Which is the most commonly used for an FPS? (Please note that I have used these words a few times before, and it is for the reason that I don't believe you can improve upon someone else's ideas until you know why they need improving. By implementing conventional methods, you are provided with some idea that it will work effectively, and then you can consider a new iteration solving bugs!)
  2. Accessing utilities. How do you share references to instanced classes in C++? If I instance a SoundManager in Python, I still need to find a way of getting it from its namespace to other instances potentially separated as far as it would be out of scope. (E.g self.parent.x.y.soundManager is a little out of place). One cannot simply pass a named reference because If one wants to extend the class with inheritance, it would still point to the old BaseClass name, and you'd be forced then to use classes and not class instances. I just fundamentally hate this part of Python, because there are lots of ways around this (module attributes, special sharer modules etc...) but all seemed wrong in some way.

#1Angus Hollands

Posted 23 January 2013 - 10:52 AM

I think, partially because of the language barrier, I have some undefined terms that I need to justify.

Firstly, It's pretty easy to serialise in Python. Attribute lookups aren't too difficult, and in most cases I'd use class attributes that store such identifiers of attributes.

"Serialising" is a loosely defined term. I am not using conventional serialising libraries such as Pickle or JSON because they provide too much extra padding and information that I do not need. Because all data in this system is built from defined attributes that are available to all parts of the system, it is easy to create a format string that can run the struct library unpack and pack methods. 

 

Events and packets. Well, as I've said, I'm using Python, and Python includes a socket wrapper in the standard library. The nature of this means that there is little immediate concept of anything beneath the socket layer. Simply input a destination and a payload and it will arrive (hopefully) at the other end. If I didn't send have data that needed to be queued, and only had data that was needed to be sent every network tick, then this might be an alright method: I can call a socket.sendto function and send the data, but when sending a reasonable amount of data, it is faster to store it in a bytes buffer and send that in one socket call. (this does increase the damage done by packet loss). Because of this, things that are handled by the library that allow me to read each payload from the buffer as individually sent bytes are no longer applicable. They simply ensure my "container" gets from A to B (hopefully). So, I introduced the concept of a "packet". A packet essentially consists of: Default header: size in bytes, Optional "protocol" header (defined by user): packet type, Packet payload. Each packet can optionally nest a sub-packet as its payload. There is no point to allow recursive unpacking because it will always be a fixed depth, and you would need to access the protocol from somewhere. Omitted from my sample code, the nested packets would have a protocol argument which accepts a protocol instance with read/write methods. A depth of three would do the following:

  1. Uppermost layer - contains the full integer game tick that the packet started "filling" from
  2. Centre layer - contains N entries of tick-container packets: packets which have the delta tick since the base tick determining their creation tick
  3. Lower layer - contains the event data for each game tick specified in the centre layer . 

This is all represented by the XML diagram. Furthermore, using automatic detection for nested packets would add to overhead and packet size. (Overhead required to find the protocol for the packet, and then extra bytes to add a type for the packet (e.g 0 = EmptyContainer, 1=MiddleContainer ...). However, as I state this, I would be interested to see how much overhead. It would still make the process more convoluted though.

 

Prioritisation and state

I am trying to write a system that allows for some deviation from the FPS genre. It's quite easy to do in a lot of ways, save for some genre specific stuff.

Firstly, I intend to have automatic attribute flags. In other words, when the entity class is created:

  1. It reads all the attributes that are networkable, and sorts them by type (and name within type).
  2. It saves the ordered names of these attributes to a defined variable.
  3. It creates a bitfield for any booleans to pack them more efficiently. Each boolean class is registered with the index and the bitfield that will store their values, and their set and get methods of the descriptor will point to that value.
  4. It creates a format string for struct using the type of the initial value for each attribute,  When booleans are found to exist, it adds a Char formatter for each 8 bools. The format string then removes booleans and uses the boolean bitfield(s). I also prepend a contents bitfield (or two) which indicates which attributes are "dirty". The dirty flag is set whenever the attribute is set.

In writing this, I intend to demonstrate that dirty flags would be relatively simple to implement. Whenever the properties are changed, we just read the dirty bitfield, and if it is > 0 it needs updating. At present, updating would require sending all the data, but I could modify it so that it can dynamically send different parts of the attributes

 

The reason that I am going through all of this "complex design" is twofold.

Firstly, there are few resources implemented in Python. There are basic parts in C++ that are simply not reproducible with the same simplicity as C/C++ and in some cases it would be slower to try and do so. Furthermore, I want the system to be flexible. I already have a working game, but it is very chunky. It doesn't feel clean and I don't like that. For example, at the moment I cannot set "dirty flags" automatically. The entities house the to/from bytes methods, and they don't use networked attributes, instead they just scrape the data. This makes a client entity and a server entity very different, with little code in common (when it really should be in many parts).

 

There are a few more things I need to ask.

Generically speaking, I have two options regarding the client. Firstly, I could attempt to run the same logic as on the server, but using server corrections. This would include all AI calculations etc. The other method is just to extrapolate data. The second method would be harder to do automatically, and It may be best to add some form of manual defined extrapolation that I can easily hook in without modifying the system too much. I suspect that the second option is the one I should choose, simply for calculations sake.


PARTNERS