Hello,

I am using async c# code for my client. Now I wonder how you should implent this since sometimes I just read a packet from the server and you need to have information about the game. Do you need to place it inside a queue for the mainthread to check it and let him do the reading and then sending? Or are there better designs, this feels like the same as sync but without handling the thread yourself.

For a ciient, async network completion isn't that helpful, because the amount of I/O will not be enough to matter. However, there's nothing wrong with using async even on the client. In the end, a game loop typically looks like:

1) read inputs

2) calculate simulation

3) render graphics

With networking, the "read inputs" step includes both keyboard and network, and you typically send the inputs to others right before going to the calculate simulation step.

More specifics can't really be known without also knowing what your network mechanism is -- input synchronous? Forward extrapolated? Dumb client? Something else?

I handle everything sync from the input :)

Now the weird thing is, in my game you can start more games vs other players at the same time. I use for every game a new connection, would this mean that async starts to scale better when there are many clients. Else I create 5 or more threads when a user is playing 5 games.

On Windows, select() works fine up to 64 sockets (this is an internal implementation detail.)

On Linux, select() works fine up to a little over 1000 sockets (somewhat depending on how many other file descriptors you use.)

The networking part is very, very, seldom an actual bottleneck for games.

On the client side, physics simulation (if used) and graphics will typically be a limitation long before networking even shows up on a profile.

On the server side, interest management (who should see what data,) as well as physics simulation (if used) is typically a lot heavier work than the networking bit, too.

This is my code so far, do you see anything weird?

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Net.Sockets;
using System.Net;

namespace clb
{
    class Connection
    {
        public enum PacketType
        {
            INIT_PACKET_HEADER,
            INIT_PACKET_DATA,
            PACKET_HEADER,
            PACKET_DATA
        }

        public struct PacketInfo
        {
            public PacketType type;
            public byte[] buffer;
            public int bytesRead;

            public PacketInfo(int len, PacketType pt)
            {
                type = pt;
                buffer = new byte[len];
                bytesRead = 0;
            }
        }

        TcpClient m_client;

        protected GameThread m_thread;

        // constructors
        public Connection()
        {
            m_client = new TcpClient();
        }

        // virtual fcuntions, they get called from my queue again
        public virtual void OnInitPacketReceived(short version)
        {

        }

        public virtual void OnPacketReceived(PacketReader pr)
        {
        }

        // Interface functions
        protected async void Connect(string ip, int port)
        {
            try
            {
                await m_client.ConnectAsync(ip, port);

                if(m_client.Connected)
                    StartReadingAsync(new PacketInfo(2, PacketType.INIT_PACKET_HEADER)); 
            }
            catch (Exception e)
            {
                Console.WriteLine("[exception] in Connect:" + e.ToString());
            }
        }

        public void Disconnect()
        {
            if(m_client.Connected)
                m_client.Close();
        }

        public void SendPacket(PacketWriter pw)
        {
            SendPacket(pw.ToArray());
        }

        // The real functions

        private void InitPacketHeader(PacketInfo pi)
        {
            PacketReader pr = new PacketReader(pi.buffer);
            short header = pr.ReadShort();

            StartReadingAsync(new PacketInfo(header, PacketType.INIT_PACKET_DATA));
        }

        private void InitPacketData(PacketInfo pi)
        {
            // read stuff
            StartReadingAsync(new PacketInfo(4, PacketType.PACKET_HEADER));
        }

        private void PacketHeader(PacketInfo pi)
        {
            // read stuff

            StartReadingAsync(new PacketInfo(packetLength, PacketType.PACKET_DATA));
        }

        private void PacketData(PacketInfo pi)
        {
           // read stuff

            if (data.Length != 0)
            {
                m_thread.AddMessage(new Message(MessageID.PacketRecv, new PacketReader(data)));
            }
            StartReadingAsync(new PacketInfo(4, PacketType.PACKET_HEADER));
        }

        private async void StartReadingAsync(PacketInfo pi)
        {
            if (!m_client.Connected)
            {
                Console.WriteLine("Trying to read, but not connected anymore????" );
                return;
            }

            try
            {
                int recv = await m_client.GetStream().ReadAsync(pi.buffer, pi.bytesRead, pi.buffer.Length - pi.bytesRead);

                if(recv == 0)
                {
                    Console.WriteLine("recv 0 in reading, dc?");
                    return;
                }

                pi.bytesRead += recv;

                if (pi.bytesRead != pi.buffer.Length)
                {
                    StartReadingAsync(pi);
                    return;
                }

                switch (pi.type)
                {
                    case PacketType.INIT_PACKET_DATA:
                        InitPacketData(pi);
                        break;

                    case PacketType.INIT_PACKET_HEADER:
                        InitPacketHeader(pi);
                        break;

                    case PacketType.PACKET_DATA:
                        PacketData(pi);
                        break;

                    case PacketType.PACKET_HEADER:
                        PacketHeader(pi);
                        break;
                }
                
            }
            catch (Exception e)
            {
                Console.WriteLine("[exception] in StartReadingAsync:" + e.ToString());
            }
        }

        private async void SendPacket(byte[] buffer)
        {
           // cripto and stuff

            try
            {
               await m_client.GetStream().WriteAsync(sendData, 0, sendData.Length);
            }
            catch (Exception e)
            {
                Console.WriteLine("[exception] in SendPacket:" + e.ToString());
            }
        }

    }
}

I am with you on async design. However what you need is more than just threading and polling. In my upcoming distributed interface called Flow I have made it transparent to the caller or sender of messages if the recipient is on the network or in local memory. Like this you can seamlessly distribute tasks and responsibilities as the different message handlers work independently and asynchronously (unless the synchronization is done explicitly with some wait on a response message).

You also positively should never have a loop where you poll input with a sub 60 or even 120 Hz frequency as you will miss key events such as a quick key up and down press. The most important here is to always offload heavy calculations to separate threads. Typically simulation can run at 10-20 Hz, visualisation at 25-60 Hz and input at 120 Hz, this makes it a rather poor design choice to use a shared thread for the three of them,

It does require that you design your program differently as you will deal with messages rather than state variables in some global shared memory.

My current game loop looks like this

while (continue)

{

sleep(1000);

}

:-)

All the code and logic is distributed and kick started with a set of message handlers.

It is rather a beautiful thing to abstract communication be it intra process (same process), inter process (between processes) or inter machine network communication. It also allows parts of your logic run on a mac and another part run on a windows machine as it allows heterogeneous platform processing.

The water fall or multi-pass execution model is a technology of ancient times and maladapted to multi-core and multi-processing methodologies.

spinningcube

Why would you use a polling loop for main? This means that, after I decide to quit, "nothing" may be happening for up to a second. I'd presume that, if you have a good queuing API, the main loop could blocking wait on a queue where you send it a "quit" message (rather than setting a boolean to false.) And if you don't have blocking waits, you could at least use the OS-specific mutex/condition variable/event to achieve the same thing.

Other than that, mostly-transparent network/memory messaging APIs have some nice properties. ZeroMQ for example took that to heart and had significant success with it. The two main drawbacks, in my mind, are that sometimes, you NEED to know, for example when implementing and requiring authentication. And sometimes, you NEED to know, for performance reasons. As long as those don't get in the way, the code can be very nice and clean.

Seriously? It is an example and a joke at that to exemplify how a truly distributed system does not care about order as the components negotiate their communication without a central authority. I like anarchy :-) But sure you could make the main thread just wait or sleep on a signal. I guess you will gain 0.0000001 seconds performance ;-) But ok if your aim is to exit as fast as possible then sure a long sleep and then a poll will delay that. I give you that.

Nice straw man argument. Here is a pat on the back.

Transparent message passing is the way to go and I am quite amased why not everyone has transitioned to the paradigm already. I guess laziness. And I totally can't stand Intel TBB it just plainly over complicates things (yes all programmers know this is not a message passing interface, next...)

What do you mean that you NEED to know? I don't understand. Do you mean knowing the clock cycles that were spent? Curious.

spinningcube

PS - and people wonder why coders have zero personal skills. You hung up on a detail on a polling loop when 99% of the post was about how to design a proper distributed game environment. Sheesh.

PS2 - the moderator gives obvious BS advice and gets plussed and I get a negative when I tell the truth. Awesome. Shows how many here are actual professional coders...

Seriously? It is an example and a joke at that to exemplify how a truly distributed system does not care about order as the components negotiate their communication without a central authority. I like anarchy :-) But sure you could make the main thread just wait or sleep on a signal. I guess you will gain 0.0000001 seconds performance ;-) But ok if your aim is to exit as fast as possible then sure a long sleep and then a poll will delay that. I give you that.

Polling for main game loops may be fine for turn-based or event-based games and can be used for them. However, most modern games do not rely on that method, instead operating on a tight main loop that runs "as fast as possible". These tight main loops decouple rendering, animation, simulation, and input because that allows them to bypass steps that are not required during that instance of the loop.

As for gaining 0.0000001 seconds (100 nanoseconds), there have absolutely been times when profiling and optimizing code that a 100 nanosecond improvement was a godsend. When you start multiplying things by 60 or 75 or 120 frames per second, multiplied again by hundreds or thousands of elements to update every frame, those nanoseconds become important pretty quickly.

Nice straw man argument. Here is a pat on the back.

Transparent message passing is the way to go and I am quite amased why not everyone has transitioned to the paradigm already. I guess laziness. And I totally can't stand Intel TBB it just plainly over complicates things (yes all programmers know this is not a message passing interface, next...)

What do you mean that you NEED to know? I don't understand. Do you mean knowing the clock cycles that were spent? Curious.

PS - and people wonder why coders have zero personal skills. You hung up on a detail on a polling loop when 99% of the post was about how to design a proper distributed game environment. Sheesh.

True, a few coders have zero personal skills. Fortunately they make up a small portion of the community. There are many books that might be useful on that subject to such people, such as this famous one, that teach techniques such as avoiding direct criticism and condemnation, being sympathetic with others' ideas even when they are different from yours, and allowing other people to save face when they (rightly or wrongly) feel wronged.

Seriously? It is an example and a joke at that to exemplify how a truly distributed system does not care about order as the components negotiate their communication without a central authority. I like anarchy :-) But sure you could make the main thread just wait or sleep on a signal. I guess you will gain 0.0000001 seconds performance ;-) But ok if your aim is to exit as fast as possible then sure a long sleep and then a poll will delay that. I give you that.

Polling for main game loops may be fine for turn-based or event-based games and can be used for them. However, most modern games do not rely on that method, instead operating on a tight main loop that runs "as fast as possible". These tight main loops decouple rendering, animation, simulation, and input because that allows them to bypass steps that are not required during that instance of the loop. As for gaining 0.0000001 seconds (100 nanoseconds), there have absolutely been times when profiling and optimizing code that a 100 nanosecond improvement was a godsend. When you start multiplying things by 60 or 75 or 120 frames per second, multiplied again by hundreds or thousands of elements to update every frame, those nanoseconds become important pretty quickly.

Nice straw man argument. Here is a pat on the back. Transparent message passing is the way to go and I am quite amased why not everyone has transitioned to the paradigm already. I guess laziness. And I totally can't stand Intel TBB it just plainly over complicates things (yes all programmers know this is not a message passing interface, next...) What do you mean that you NEED to know? I don't understand. Do you mean knowing the clock cycles that were spent? Curious. PS - and people wonder why coders have zero personal skills. You hung up on a detail on a polling loop when 99% of the post was about how to design a proper distributed game environment. Sheesh.

True, a few coders have zero personal skills. Fortunately they make up a small portion of the community. There are many books that might be useful on that subject to such people, such as this famous one, that teach techniques such as avoiding direct criticism and condemnation, being sympathetic with others' ideas even when they are different from yours, and allowing other people to save face when they (rightly or wrongly) feel wronged.

Again with the poll. It was shown as an example that you no longer have to use a main thread at all. Yes you can replace that with whatever other sleep/signal that you prefer.

It is not always true though. Unfortunately when working on OS X and iPad some parts have to be polled in the main thread. I hate that :-)

For the rest. Yeah maybe I should have been kinder but it puts my pants on fire when I spend a long post to explain the benefits of asynch and distributed design, to then make an obvious fun remark about how the main loop can now just sleep, that hplus spends all his energy and focus on that joke of a line. It's ironic and black humour. But it also shows how some think they can get out of a stupid comment (by advocating a multi-pass approach for things that should not, like hplus did) by attacking something without rime nor reason. It's weak and he should be the one pointing out his own mistake, not spreading bad information.

This site still is about better not worse coding practices right?

spinningcube

PS I've read that book and usually self help is just some honey for the soul, not really the solution. Sometimes anger is good and helps you get things done. I am sure hplus knows his shit, but why then play stupid and point out something like that, when he should be re-assessing his first answer. We all make mistakes and mine is sometimes to be a bit too confrontational, of which if I hurt anyone, apologize.

PS2 - and I am genuinely curious what would be the NEED in the message passing approach. I am designing such a system and would like to know. Maybe hplus has a good insight on this.