hplus0603Member Since 03 Jun 2003
Offline Last Active Yesterday, 10:36 PM
- Group Moderators
- Active Posts 11,698
- Profile Views 28,703
- Submitted Links 0
- Member Title Moderator - Multiplayer and Network Programming
- Age Age Unknown
- Birthday Birthday Unknown
Redwood City, CA
Posted by hplus0603 on Yesterday, 10:24 PM
That depends on what firewall you're using.
Which in turn depends on what OS you're using.
You're not giving us much to go on, here :-)
If this is Windows, and you're running your game server as a service, then perhaps this is a problem where it runs under different credentials and the firewall doesn't apply your config -- that depends on what firewall you're using I guess.
Posted by hplus0603 on Yesterday, 10:23 PM
libcurl is pretty good.
You can then "log in" the C++ client by posting name/password to the web service, which might return a Set-Cookie header with a random, hard-to-guess session ID, and you can provide that Cookie header in subsequent requests to the web service.
The good news with that is that you can then use the same login mechanism if you build a web app to manage your data :-)
Posted by hplus0603 on 24 August 2016 - 12:14 PM
how do I resolve the problem of sync between client and server?
You have two main options:
1) Temporarily display the player in an incorrect/extrapolated position, and keep updating the position to be "more correct" based on what you receive from the server. This gives immediate command response on the client, but will display the client slightly out-of-sync with the world.
2) Only send commands to the server, and update the player to "walking" on the screen only when the server sends the new state back. This will always display a world that's in sync, but it will cause command latency between "start moving button pressed" and "actually starts moving on screen."
Exactly how you implement each of these options varies based on game mechanics and other specifics.
In most cases, the client won't ever be out of sync with the server, because there is no discrepancy.
Only when something is different (a door has opened on the server, not on the client yet; another player is in the way on the server, not on the client yet; etc) is the correction actually visible.
You can choose to lerp that correction, or just snap the player.
The simplest way to correct the player is to store the commands the player has received, and "wind them forward" each time you receive a new state update from the server.
Posted by hplus0603 on 23 August 2016 - 03:36 PM
The issue is that not everyone is that careful and looks at the url or the SSL certificate before typing this information in.
This is true, but there is no better solution :-(
Well, you can have players purchase a cryptographic token from you, but that's not a low-friction onboarding experience :-)
Or i use the current method which doesn't store personal userdata but is by no means secure due to the fact that the master-key is stored in the client
One option is to start there, and then give the user the option of signing in with a Steam ID if they want to be able to play from other places, able to upload files, etc.
Let the user decide.
Note: If you have that "easy onboarding" option, then if there is any kind of abuse possible of your system (using it to host file uploads, etc,) then that's what the bad people will use, so be sure to take that into consideration!
Are they (the majority of them) using SQL injections to do this?
There are about three attack vectors:
1) SQL injection, or other hosted-software vulnerability (WordPress, Drupal, ImageMagic, etc.) This generally gives you database dumps and perhaps admin interface access to the site.
2) Host vulnerability (Heartbleed, etc) This generally gives you command-line access to the site, which you can use to discover databases that you can then dump, insert payloads in hosted pages, etc.
3) Social engineering ("I am Robert, the County Password Inspector!") This generally gives you some kind of employee access to the site from the command line, again.
The question, then, is what the bad guys are after. If you store credit card details, absolutely that! If not, perhaps a list of emails and passwords to try on other sites. Or perhaps just another box they can run DDoS, email spam, and fradulent web sites from.
If you keep your development code on another system, with tight security, and use good source control, and good automated deployment methodology, you will minimize the impact from most such events, once you can detect that they occurred.
Wipe the hard disk, re-deploy to new OS image, restart servers; done!
Posted by hplus0603 on 23 August 2016 - 03:16 PM
Just Say No to blocking network calls!
Second, for an RPG, I bet that TCP will work well enough. That way, you don't have to worry about whether a client got the update that there's a rock in a particular place or not.
Just send out a packet whenever the client desires to do something (move to a place, move in a direction, cast a spell, whatever) and on the server, collate these and send to all clients X times a second.
For an RPG, I bet 10 times a second would be plenty.
Just make sure to pack the entire set of updates into a single frame/packet/send() call, and turn on SO_NODELAY.
If the client has the same map as the server, the client will 99% of the time predict correctly what the server does.
It can just go ahead and render whatever it it believes should happen.
When it receives updates from the server, it can compare that response with what it, itself, thought the state was at that time (the time the updated pertained to.)
It can then re-play whatever inputs the player provided between that time and now, to show the next player position.
For "important" actions, like the result of spells, the result of fighting, purchasing/transactions, etc, you should play a local "wind-up" animation when the player first initiates it, but you should wait to show the result until the server sends "fireball blasted here" or whatever back.
That way, you will never show the player "you succeeded" only to take it back again 300 milliseconds later.
Billions of rows in a database is generally a bad idea, not because you couldn't store it on a hard disk (a 4 terabyte hard disk can store a lot!) but because the height of the index becomes very tall. And one component of database performance is number-of-indices times height-of-indices.
For most game character data, I would just store the character stats/abilities as one big JSON blob that's stored as inline text (a long varchar or maybe text field.)
This reduces the index height (number of rows) to the number of characters, which is better.
If you really plan on having millions of players, though, at some point, you're going to want to horizontally shard your data -- characters with ID 0 - 999,999 live on instance A; ID 1,000,000 - 1,999,999 live on instance B; ...
Then keep a table of ID range to instance mapping, so you talk to the right database instance for the given customer.
You will of course need a central table that goes from "customer name" to "customer id" so that you know which database to look at; this may have to live in a central database, but should be a much smaller table with a single index.
If you have really quite a lot of customers, even the mapping from "customer name" to "character id" will be too big to keep on a central server; at that point, you may shard that initial table based on "first character of customer name" or "hash of customer name modulo 20" or whatever. But you'll likely never get to that point, as pretty much Google and Facebook have that problem :-)
Posted by hplus0603 on 21 August 2016 - 10:28 AM
dropped TCP packets are just waited for at the router to be resent again
Routers, in general, do not wait for re-transmission before they forward, because they work at the IP layer, not the TCP layer.
For "reverse NAT servers" this changes, as they play the role of end-points, but when you talk about "router" in general internet parlance, you typically mean the boxes that sit in the middle of the network.
The re-transmission in TCP is entirely handled by the endpoints -- the machine doing the sending, and the machine doing the receiving on the other end.
And if one packet is dropped or delayed, ALL THE PACKETS AFTER THAT PACKET ARE HELD when they are received, waiting for the dropped/delayed packet to be re-sent.
By the time the dropped packet is detected, re-transmitted, and received on the other end, anything that was sent after that is also late, because it's been sitting in the kernel, received by the host, but not delivered to the application, because of the in-order guarantee.
Regarding spraying 100 packets at the same time, that doesn't really help, because the main reason a packet is dropped, is that there is congestion at some node on the network.
Once there is congestion, if the node receives 1 packet from you, or 100 packets from you, they will all be dropped at that time until the congestion recovers.
If you want to try duplication, consuming bandwidth in an attempt at higher robustness, it's much more robust to perhaps include the last 2-3 packets as copies in the next packets you send, so there is some time between each. This will add a bit of overhead, but if you structure your data right, it will RLE compress really well, and thus won't actually consume 200-300% of the bandwidth.
If the network drops three successive packets with a packet-send delay between each, the congestion is likely so bad that you're going to have a bad playing experience no matter what.
Posted by hplus0603 on 21 August 2016 - 10:19 AM
If the turns could conceivably be delievered over email, then using MySQL to store/forward them is fine.
If the turns need to be interactive, then you need an interactive game server/persistence system of some sort.
If you have to live with the PHP "no global state persists past a web request" model, then an in-RAM storage like Redis can help, but of course you still have to pay the cost of inflating world state and then re-saving world state after each request.
If the game is truly interactive, then HTTP is the wrong protocol, and you want a persistent process server, a la most AAA 3D multiplayer games out there.
Posted by hplus0603 on 19 August 2016 - 07:01 PM
I actually do secure passwords by using sha256(password+uniqueSalt).
You should use bcrypt() or scrypt(), not just plain hashing.
the redirect from my website to steam (for the openID verification) could be intercepted and compromised
Presumably, the user will notice that the URL in the login window doesn't say "steampowered.com" but instead something like "steampwored.com" when looking at the SSL certificate validity key (green/red mark.)
This is in general a problem on the web -- some impostors are quite good. I guess the good news is that your game is probably not so super-successful as to be the biggest target for an attacker to go after.
That, in addition to "don't get hacked to the point where they can change your site."
Also, good site operations include making sure website code isn't surprisingly changed, perhaps re-deploying fresh code every so often and such.
Using a simple encryption key hard-coded into your site code to store the email address may be all you need to protect against most hackers -- most hackers just try to dump a database table to harvest info, and aren't super-coders with the resources of a nation state :-)
Posted by hplus0603 on 14 August 2016 - 03:05 PM
Either of those cases is actually, in the grand scheme of things, a good thing :-)
Second, because you're using LUA, you can actually prevent the users from doing anything "really bad" to the machine where it runs.
However, you have to make sure that the default LUA libraries for things like file I/O are not available.
Instead, only make functions available that respond to things that happen in your game. That way, you can build a "LUA sandbox" that doesn't let a script reach outside where it's supposed to be.
Designing secure, tight, sandbox APIs is actually pretty hard, unfortunately, but it's possible, assuming LUA doesn't have a bug in itself that's exploitable.
Also, user ratings of scripts ought to be able to alert you to any particular problems.
Server software for an upload/browse/download/rate system for text scripts? Literally any web application framework in the world will be able to do this.
(Static site generators don't count, as they don't generate applications :-)
Define a few REST services (GET/POST/PUT) that lets a user create a new file, upload the file, set metadata about the file, as well as post comments/ratings about it, and searching the database of files.
What's your favorite language?
Erlang with Webmachine?
Haskell with Warp?
PHP just as it is?
Python with Flask?
For a few dozen files, you don't need more storage than a file system (that makes sure users can't overwrite existing files or create files with bad file names like "." or "/etc/passwd" or whatnot.)
For many thousand files, a simple MySQL (or Postgres, or MSSQL, or Cassandra, or RIAK, or ...) server will work just fine. Use MEDIUMTEXT or similar data type for the payload script.
Another good thing to do is set an upper limit to the size of the script (30 kB?) and make sure that the entire script parses like a proper LUA file (throw it at a sandbox LUA interpreter and look for errors) so that nobody uploads child porn or whatever.
Finally, you will need some what for users to log into your game server, both from a web interface (if you want to support web-based management/ratings) and from the client.
An alternative is to just create an account with email address and password, and make the user verify the email address with a link you mail them.
Most web application frameworks should be able to do this for you.
Posted by hplus0603 on 10 August 2016 - 05:33 PM
when all IP addresses were right there on the public internet
IPv6 (now reaching about 10% of the internet!) gives you that again.
The IETF currently believes that IPv6 doesn't need any NAT.
The official recommendation is that "correct firewall configuration is the user's responsibility."
I think that's short-sighted, because one of the main benefits of NAT is that users by default have a reasonable stateful firewall preventing their Windows network shares from being accidentally exposed to the internet.
Posted by hplus0603 on 09 August 2016 - 04:07 PM
On the server, you do not use connect() at all.
So, let's assume your router's external address is 18.104.22.168, and that the external port is 5432.
Let's assume that your PC's internal address is 192.168.1.18, and that the internal port is 3800.
You should set up your router to port forward external port 5432 to internal address 192.168.1.18, port 3800.
You should then, in the client that connects to your router from the internet, put in the address 22.214.171.124 and the port 5432.
Now, if you try to connect to your PC from inside your network, your router may not support "hairpin NAT," and thus only when you are inside your network, you need to connect to 192.168.1.18 port 3800 for it to work.
Posted by hplus0603 on 08 August 2016 - 10:40 AM
Opening a port gives the external world access to whatever is available on that port.
when I actually open a port without any security, I open myself to some bad stuff they can do to harm my pc
If it's your chat program, and it has no bugs, then it only gives the world access to that port.
If it's your chat program, and you have bugs (such as buffer overflows,) then a skilled attacker may be able to break out of the server executable that you are running.
However, in practice, it's unlikely that a skilled attacker will be interested in your program while it's small, so in reality, the risk is minimal, unless you do something really crazy like allow users to run shell commands through chat or whatever.
That being said, hosting on your home computer on your home network is always troublesome -- you can't keep it up forever, your IP address may change, your computer is also needed for other things, ...
Instead, I recommend a cheap hosting provider of some sort. I myself use $6 virtual private servers from interserver.net but there are others -- $10 from linode -- $15 from dreamhost -- micro instances from amazon.
If you want to run Windows, they usually cost a bit more (and you can also look at [ulr=https://azure.microsoft.com/en-us/]Microsoft Azure[/url])
The benefit with these is that you can pay for them for as long as you need them (Amazon: hourly; most others: monthly) and then stop paying for them.
Also, if the machine gets hacked, just destroy the instance, fix the bug, and spin up a new instance.
Posted by hplus0603 on 04 August 2016 - 12:49 PM
why can't just people use simple words
Because distributed systems and networking is, unfortunately, not a simple problem.
Posted by hplus0603 on 03 August 2016 - 04:08 PM
They are a little higher level than SDL_net, boost::asio or other "socket wrappers."
Posted by hplus0603 on 31 July 2016 - 08:05 PM
what potential alternatives - other than Erlang - there are for any serious network programmer?
You can write a scalable system in Node.js. Just look at eBay -- they rewrote their web page front-ends on Node.js.
It just takes more work. And, because it's not a statically checked language, you need more unit tests.
For distributed server projects for games with real-time components (simulation servers or similar,) I would personally look at the following:
- C++ -- you will never, ever, be blocked by some bug in some infrastructure, because you can always re-implement it yourself. Also, if you need to go "to the metal," there's no change in tools or build systems -- you're already there! Also, easily links whatever third party libraries you want.
- Java (ecosystem) -- whether you use Clojure, Java proper, or Scala, this is a suite of tools that a lot of people know how to run, and it's easy to find help here. It's a little more ghetto than C++, and a little less real-time (lots of Hadoop / Kafka / Spark type systems) but not to be ignored.
- C# (ecosystem) -- this wasn't on my list before, because mono is hilariously non-good for servers, but with the release of .NET Core open source, I'd love to take a look at that and see what it can do. C# is a more efficient language than Java by design, and Visual Studio is the world's best development environment.
- Erlang, as I said. It has some limitations and bugs, and you need to really learn and internalize the tools that come with it (OTP and the various distribution support libraries.) But excellent systems can be built.
I could conceivably talk about others. I love the Haskell web service infrastructure we have at work, but getting to that point was a lot of work -- there are less batteries included in that part of the world.
Python, Node, and Go are all possible, if the project is smaller. Neither is particularly good at threading, though, and neither is particularly fast; probably Node is fastest because it benefits from the massive optimizations in V8.
Perl, PHP, Ruby -- not really suitable for scalable, multiplayer, real-time game servers. I mean, you could, if you REALLY wanted to. But why would you?
Wow. Seeing it all in a single place makes it seem like ... there are a lot of languages and environments out there!