Jump to content

  • Log In with Google      Sign In   
  • Create Account


Member Since 03 Jun 2003
Offline Last Active Yesterday, 07:27 PM

#5316490 How should my Networking class send and receive info from my other classes?

Posted by on Yesterday, 12:05 PM

when I hear the word 'hashtable' I want to throw up

Why? The hash table is one of the most core data structures in software engineering, and almost all modern languages provide them built-in.
(Lua tables, PHP arrays, Python dicts, JavaScript objects are all basically a wrapper on top of a hash table.)
In C++, it's std::unordered_map<>.

#5316395 Running into what I believe is a networking issue regarding a server/client t...

Posted by on 23 October 2016 - 07:55 PM

It sounds like you have a code bug somewhere.
Use Wireshark to look for packets sent to your particular port.

The app will continuously request for updates from the server

If a "connection" exists, you can assume that the player wants the world state.
You don't need to have the client "request" the data.
Instead, the client should send "I'm alive, and here's my next input commands" messages X times per second, and the server should send "here's the latest world update and commands from all the other players" Y times per second.
The server can then look at the incoming stream, and if it hasn't seen anything from the player for something between 3 and 30 seconds, assume the client has disconnected.
Same thing for the client; if it doesn't see anything from the server for a timeout period, assume the connection has dropped.

#5316393 How should my Networking class send and receive info from my other classes?

Posted by on 23 October 2016 - 07:52 PM

Use a hash table.
Give each "thing" in your program an ID. Maybe the "scoreboardmanager" is ID 13, and objects are created with ID from 100 and up.
Write the ID that a message is intended for at the front of the message, together with length of message.
Then your network code can say:

while (!packet_is_empty(packet)) {
    auto id = read_an_int(packet);
    auto size = read_an_int(packet);
    auto data = read_packet_message(packet, size);
    auto entity = all_entities.find(id);
    if (!entity) {
        error("Entity ID " / id / " doesn't exist! Received from " / packet->source);
    } else {

#5315959 Commiting persistent server data

Posted by on 20 October 2016 - 09:47 AM

there's no good reason to keep data in memory that you're not using, and may never use again (e.g. if a player never comes back).

Shh! Don't tell the Redis folks!

(FWIW: We run the biggest single Redis instance I know of, with 768 GB of RAM. This turned out to be a mistake, because the entire kernel locks up for 10 seconds each time it forks to checkpoint the data.)

How would I go about implementing

At that point, you're building the lowest-level component of a database (or, for that matter, file system) which is "indirect block allocation and management."
A very simple way to do that is to split your file into chunks, say 1 MB each, and have each chunk link to the next chunk when it gets full. To read all the data, you follow the chain of links and concatenate all the data.
A slightly more sophisticated way is to make the first chunk be an array of chunk offset, and each time you need another 1 MB chunk, add a new offset to the table, and when the table runs out, you either say "file is full," or you apply the linked-list of table chunks, or you add a second layer of indirection.
(Chunk size varies by application -- 1 MB may be way too big or not big enough, depending on what you're doing.)
An even more sophisticated way of doing this is to structure your data in an ordered index -- at this point, you'll want to read up on B-trees, B*-trees, and other such structures, because you're well on your way to building your own database!

Simple math example:
Let's assume 1 MB chunks. Let's assume 64 bit file offset.
1 MB can fit 128K of file offset pointers. Each pointer references a 1 MB chunk of file data.
Maximum size of data stored in file: 128K * 1M == 128 GB of data.

#5315887 Commiting persistent server data

Posted by on 19 October 2016 - 06:41 PM

if I were to keep the data in a file in disk and if the data is unsorted, wouldn't it take too long to query what I need from that file?

If you don't save/load everything, but keep everything in an unstructured file on disk, then yes, that is kind-of the slowest of both worlds :-)

If you save/load pieces, then you typically either put each piece in some kind of database. This can be something heavyweight like MySQL or DB/2, or something simple like Berekeley DB or just a file with a table of contents index at the front, or something super-simple like a file per user/entity (with some recursive directory hierarchy to avoid 100,000 files in one directory.)

Also if the memory usage isn't a concern, is it still a bad idea to keep all data in memory?

If the performance is fine (maybe you can save the data asynchronously?) and you have no better use for the RAM, then that's fine.

Another option you may want to consider is memory mapping the file, which makes the file be automatically saved all the time without you having to worry about it.
mmap() on Linux; CreateFileMapping() on Windows.

#5315843 Commiting persistent server data

Posted by on 19 October 2016 - 11:15 AM

Some random points:

On writing binary data:
If you have a big struct of "everything" without any pointers in it, a single call to write() will write that out just fine.
Typically, you'll want to write() to a new file (foo.data.tmp) and then use rename() to replace the old copy with the new in an atomic operation (rename(foo.data.tmp, foo.data))
Typically, you also sync() and fdatasync() after closing the file before renaming, and then after renaming.
If you find that you need to be selective about what you write to disk, writing by appending to a big memory buffer, and then writing that big buffer to disk all at once, is another way to get good I/O performance.

On keeping data in memory:
You need to keep all ACTIVE data in memory, so that you don't block gameplay on accessing the database or disk.
However, when a user logs out, it's totally fine to purge that data from memory. When a user logs in again, re-load the data.

On arranging data for querying:
Whenever you find that you have to "look for data," you typically need to use a hash table (such as std::unordered_map<>).
Linear scans through arrays is fine for small arrays that fit in a couple of cache lines, but will suck fiercely once the array gets big.
If you need the data to be sorted, use a tree of some sort instead (such as std::map<>).

#5315202 Running both server and client on one PC causes lag despite ping < 1 ms.

Posted by on 14 October 2016 - 09:44 AM

I hope there aren't any floating point errors that can accumulate and mess up my timers with time.

If you use "float" then, yes, after hours of uptime, you may accumulate drift.
If you use "double," it will happen after years of uptime, which probalby is an OK trade-off.
An alternative is to count time in quanta (either "microseconds" or "simulation ticks" or whatever) and use an int32 or int64.

Separately: It's more likely that the rate of time progress on your PC will be slightly different from the rate of time progress on the server.
Thus, it is good to include timing information in each packet, and if you find that the PC is "too different" from the server, adjust the PC.
(This also happens when network conditions change and latency increases/decreases.)
Typically, you will never allow the PC to be ahead of the server, so if it is, immediately adjust the clock backward to match.
If the PC is too far behind the server, adjust the clock forwards by 1/10th of the delta. That way, it will be reasonably smooth when catching up, without making a large jump.

#5314881 Running both server and client on one PC causes lag despite ping < 1 ms.

Posted by on 12 October 2016 - 12:08 PM

I need to time perfectly when I press 'enter' and start the client app

Your "what time is it" (read the clock) function should include an "adjustment" parameter.
The clock you return to the game should be SystemClock() + Offset.
Then, your server should send "the time is now X" data in each packet it sends to the clients.
The clients can then compare the timestamp they get from the server, to the timestamp they have locally, and adjust the Offset to make them line up.

#5314753 Running both server and client on one PC causes lag despite ping < 1 ms.

Posted by on 11 October 2016 - 04:40 PM

The problem might not be that the laptop is "slow," but that scheduling of threads happens. If the code puts some data to the network library, then renders a frame, and then tells the network library to flush data to network if any is available, for example, then the time for rendering will be included in your measured latency.
If the three or four clients each render, and have to wait for the others to finish rendering before they can start their frame, then you may also be bound on the speed of your GPU's ability to work on many different things.

I believe that the internals of RakNet do not send a message immediately when you write some data, but instead collects possibly multiple sends, until it actually "flushes" the data into a packet.
Thus, your latency will always show whatever internal scheduling latency and batching happens in both your client, and the networking library. 20 milliseconds is not a terribly bad amount of latency.

#5314737 Running both server and client on one PC causes lag despite ping < 1 ms.

Posted by on 11 October 2016 - 03:15 PM

It depends. What else is the computer doing at the same time?
Is it rendering three different windows?
Is the graphics loop involved?
Given that you're using RakNet, and RakNet schedules its own packet sends (a send isn't necessarily generated immediately when you write some data,) I would expect that kind of latency even for local round-trips.

#5314727 Running both server and client on one PC causes lag despite ping < 1 ms.

Posted by on 11 October 2016 - 02:26 PM

I can't filter packets by processes or programs, how do you even use this stuff

You can filter packets by protocol type (TCP, UDP, etc) and port number.
Presumably your game uses a known port, so filtering by that should be easy!
Then, each separate process will have a separate source port for the connection (in addition to the server port.)

#5314542 Running both server and client on one PC causes lag despite ping < 1 ms.

Posted by on 10 October 2016 - 09:29 AM

I agree with "Kylotan," "lagging" is not a sufficiently deep description to draw any conclusions from.

A few pieces of data you should log/plot from each of your server/clients:
- Average number of ticks/steps/frames per second, every 5 seconds or so
- Maximum time between successive ticks/steps/frames per 5 seconds or so
- Received and sent number of packets per second
- Received and sent client/server clock values (you should have each packet include the current local clock, and the last-received remote clock)

Making sure that the log starts each line with date/time and milliseconds-since-game-start, will make it easier to match the log files up later.

Another great tool for debugging is Wireshark. You should absolutely get a copy of Wireshark and install it (if you don't have it,) and use it to capture all the traffic to verify that the data and packets are what you think they are!

#5313927 Need a simple way to store all received packets and then pass them to the cor...

Posted by on 04 October 2016 - 04:01 PM

Typically, you will have a hash table (or map, or dictionary) that uses the remote IP address/port as key, and uses the player ID (or player object) as value.
Then, you can look up which player it is, from the source of the incoming data.

There are some session hijacking risks involved, so using a secondary method to authenticate the connection (using encryption keys and signatures) is useful for bigger games with more risk, but you don't need to worry about that for now.

#5313926 Create player accounts on custom server (for steam users)

Posted by on 04 October 2016 - 04:00 PM

Is there any kind of critical user-data that is stored on the server (behind the scenes) of which i'm not aware of?

I can't read your mind, so I don't know what you're aware of :-)

Is the serversoftware (as an example Apache) storing the IPs of incoming traffic somewhere on the system?

This typically goes into access.log. There may be other Apache logging, too, such as Referer headers. Finally, once you scale the system, you will start getting logging from your proxies and firewalls as well.
That being said, a list of IP addresses without email addresses or names is not particularly sensitive for most use cases (especially for games.)

what to do if A) the player (somehow) loses his URL or B) someone gets hold of this URL

There is no solution that solves for all of the things you want at once. Engineering is all about making the correct trade-offs for the problem at hand!

#5313268 What are the typical requirements of browser-based MMORPG architectures?

Posted by on 29 September 2016 - 10:38 AM

But they also got an interesting GDC talk out of the effort, so it wasn't a total loss :-)