hplus0603Member Since 03 Jun 2003
Online Last Active Today, 06:22 PM
- Group Moderators
- Active Posts 11,719
- Profile Views 28,974
- Submitted Links 0
- Member Title Moderator - Multiplayer and Network Programming
- Age Age Unknown
- Birthday Birthday Unknown
Redwood City, CA
Posted by hplus0603 on 21 August 2016 - 10:19 AM
If the turns could conceivably be delievered over email, then using MySQL to store/forward them is fine.
If the turns need to be interactive, then you need an interactive game server/persistence system of some sort.
If you have to live with the PHP "no global state persists past a web request" model, then an in-RAM storage like Redis can help, but of course you still have to pay the cost of inflating world state and then re-saving world state after each request.
If the game is truly interactive, then HTTP is the wrong protocol, and you want a persistent process server, a la most AAA 3D multiplayer games out there.
Posted by hplus0603 on 19 August 2016 - 07:01 PM
I actually do secure passwords by using sha256(password+uniqueSalt).
You should use bcrypt() or scrypt(), not just plain hashing.
the redirect from my website to steam (for the openID verification) could be intercepted and compromised
Presumably, the user will notice that the URL in the login window doesn't say "steampowered.com" but instead something like "steampwored.com" when looking at the SSL certificate validity key (green/red mark.)
This is in general a problem on the web -- some impostors are quite good. I guess the good news is that your game is probably not so super-successful as to be the biggest target for an attacker to go after.
That, in addition to "don't get hacked to the point where they can change your site."
Also, good site operations include making sure website code isn't surprisingly changed, perhaps re-deploying fresh code every so often and such.
Using a simple encryption key hard-coded into your site code to store the email address may be all you need to protect against most hackers -- most hackers just try to dump a database table to harvest info, and aren't super-coders with the resources of a nation state :-)
Posted by hplus0603 on 14 August 2016 - 03:05 PM
Either of those cases is actually, in the grand scheme of things, a good thing :-)
Second, because you're using LUA, you can actually prevent the users from doing anything "really bad" to the machine where it runs.
However, you have to make sure that the default LUA libraries for things like file I/O are not available.
Instead, only make functions available that respond to things that happen in your game. That way, you can build a "LUA sandbox" that doesn't let a script reach outside where it's supposed to be.
Designing secure, tight, sandbox APIs is actually pretty hard, unfortunately, but it's possible, assuming LUA doesn't have a bug in itself that's exploitable.
Also, user ratings of scripts ought to be able to alert you to any particular problems.
Server software for an upload/browse/download/rate system for text scripts? Literally any web application framework in the world will be able to do this.
(Static site generators don't count, as they don't generate applications :-)
Define a few REST services (GET/POST/PUT) that lets a user create a new file, upload the file, set metadata about the file, as well as post comments/ratings about it, and searching the database of files.
What's your favorite language?
Erlang with Webmachine?
Haskell with Warp?
PHP just as it is?
Python with Flask?
For a few dozen files, you don't need more storage than a file system (that makes sure users can't overwrite existing files or create files with bad file names like "." or "/etc/passwd" or whatnot.)
For many thousand files, a simple MySQL (or Postgres, or MSSQL, or Cassandra, or RIAK, or ...) server will work just fine. Use MEDIUMTEXT or similar data type for the payload script.
Another good thing to do is set an upper limit to the size of the script (30 kB?) and make sure that the entire script parses like a proper LUA file (throw it at a sandbox LUA interpreter and look for errors) so that nobody uploads child porn or whatever.
Finally, you will need some what for users to log into your game server, both from a web interface (if you want to support web-based management/ratings) and from the client.
An alternative is to just create an account with email address and password, and make the user verify the email address with a link you mail them.
Most web application frameworks should be able to do this for you.
Posted by hplus0603 on 10 August 2016 - 05:33 PM
when all IP addresses were right there on the public internet
IPv6 (now reaching about 10% of the internet!) gives you that again.
The IETF currently believes that IPv6 doesn't need any NAT.
The official recommendation is that "correct firewall configuration is the user's responsibility."
I think that's short-sighted, because one of the main benefits of NAT is that users by default have a reasonable stateful firewall preventing their Windows network shares from being accidentally exposed to the internet.
Posted by hplus0603 on 09 August 2016 - 04:07 PM
On the server, you do not use connect() at all.
So, let's assume your router's external address is 22.214.171.124, and that the external port is 5432.
Let's assume that your PC's internal address is 192.168.1.18, and that the internal port is 3800.
You should set up your router to port forward external port 5432 to internal address 192.168.1.18, port 3800.
You should then, in the client that connects to your router from the internet, put in the address 126.96.36.199 and the port 5432.
Now, if you try to connect to your PC from inside your network, your router may not support "hairpin NAT," and thus only when you are inside your network, you need to connect to 192.168.1.18 port 3800 for it to work.
Posted by hplus0603 on 08 August 2016 - 10:40 AM
Opening a port gives the external world access to whatever is available on that port.
when I actually open a port without any security, I open myself to some bad stuff they can do to harm my pc
If it's your chat program, and it has no bugs, then it only gives the world access to that port.
If it's your chat program, and you have bugs (such as buffer overflows,) then a skilled attacker may be able to break out of the server executable that you are running.
However, in practice, it's unlikely that a skilled attacker will be interested in your program while it's small, so in reality, the risk is minimal, unless you do something really crazy like allow users to run shell commands through chat or whatever.
That being said, hosting on your home computer on your home network is always troublesome -- you can't keep it up forever, your IP address may change, your computer is also needed for other things, ...
Instead, I recommend a cheap hosting provider of some sort. I myself use $6 virtual private servers from interserver.net but there are others -- $10 from linode -- $15 from dreamhost -- micro instances from amazon.
If you want to run Windows, they usually cost a bit more (and you can also look at [ulr=https://azure.microsoft.com/en-us/]Microsoft Azure[/url])
The benefit with these is that you can pay for them for as long as you need them (Amazon: hourly; most others: monthly) and then stop paying for them.
Also, if the machine gets hacked, just destroy the instance, fix the bug, and spin up a new instance.
Posted by hplus0603 on 04 August 2016 - 12:49 PM
why can't just people use simple words
Because distributed systems and networking is, unfortunately, not a simple problem.
Posted by hplus0603 on 03 August 2016 - 04:08 PM
They are a little higher level than SDL_net, boost::asio or other "socket wrappers."
Posted by hplus0603 on 31 July 2016 - 08:05 PM
what potential alternatives - other than Erlang - there are for any serious network programmer?
You can write a scalable system in Node.js. Just look at eBay -- they rewrote their web page front-ends on Node.js.
It just takes more work. And, because it's not a statically checked language, you need more unit tests.
For distributed server projects for games with real-time components (simulation servers or similar,) I would personally look at the following:
- C++ -- you will never, ever, be blocked by some bug in some infrastructure, because you can always re-implement it yourself. Also, if you need to go "to the metal," there's no change in tools or build systems -- you're already there! Also, easily links whatever third party libraries you want.
- Java (ecosystem) -- whether you use Clojure, Java proper, or Scala, this is a suite of tools that a lot of people know how to run, and it's easy to find help here. It's a little more ghetto than C++, and a little less real-time (lots of Hadoop / Kafka / Spark type systems) but not to be ignored.
- C# (ecosystem) -- this wasn't on my list before, because mono is hilariously non-good for servers, but with the release of .NET Core open source, I'd love to take a look at that and see what it can do. C# is a more efficient language than Java by design, and Visual Studio is the world's best development environment.
- Erlang, as I said. It has some limitations and bugs, and you need to really learn and internalize the tools that come with it (OTP and the various distribution support libraries.) But excellent systems can be built.
I could conceivably talk about others. I love the Haskell web service infrastructure we have at work, but getting to that point was a lot of work -- there are less batteries included in that part of the world.
Python, Node, and Go are all possible, if the project is smaller. Neither is particularly good at threading, though, and neither is particularly fast; probably Node is fastest because it benefits from the massive optimizations in V8.
Perl, PHP, Ruby -- not really suitable for scalable, multiplayer, real-time game servers. I mean, you could, if you REALLY wanted to. But why would you?
Wow. Seeing it all in a single place makes it seem like ... there are a lot of languages and environments out there!
Posted by hplus0603 on 31 July 2016 - 12:46 PM
The only draw-back of single-threading is that it doesn't make good use of existing hardware resources -- but if you're already using a dynamically tag-checked language, rather than a statically compiled language, that's probably not your main priority!
Regarding Elixir: I'm happy that it draws some attention to the Erlang ecosystem, but in my opinion, it's a solution to a problem that doesn't exist.
"Hey, let's take a functional language and pretend it looks like Ruby" is just not a good idea, except for a few initial days if you happen to already know Ruby.
Shortly thereafter, you will find that the syntax they provide (like re-assigning variable names) actually gets in the way, and causes potentially significant performance problems.
I would recommend just learning the Erlang syntax, if you want to live in that ecosystem.
Btw: If you want to build threaded, distributed, servers, the Erlang ecosystem helps a lot. It has tools for many of the harder things, like "leader election" or "distributed workers" or whatnot.
Also, the cost of learning a new language/syntax (with Erlang) is much smaller, long term, than the cost of learning to live in a distributed world. That's the real challenge!
Anyway: If max scalability and efficient hardware use isn't your top priority, and your project is small so dynamic type checking is okay, then Node.js is fine.
Posted by hplus0603 on 24 July 2016 - 04:54 PM
User A buys something on City 1 and so the price changed.
Users B,C,D and E, who the server has in memory that are looking at that city right now, recieve a pushed message from the server notifying of said price change.
You can totally do that over long-polling over HTTP. Our in-RAM message queue / bus system can push messages to online users through TCP sockets, websocket sockets, or HTTP long-polling, through different gateways for each method. To the server, it all looks the same: "post this message on this topic."
Separately, if a user app is not in the foreground, then on iOS, you *have* to use polled communications and/or mobile push; you don't get a persistent connection while backgrounded. On Android, you could have a persistent connection from a Service, but doing so will likely drain the battery faster.
I cannot loose a user buying/selling if the server goes down
I did suggest that important actions like trade should go to the database, so it sounds like we agree. As long as players don't trade "often" (dozens of times per second, or whatever,) then a database would be reasonable. They can always be horizontally sharded if necessary (at a cost, of course -- it's reducing cost that's the main goal here.)
Separately, if all players lose the same amount of time in a crash, and that amount of time is never more than 15 minutes, how bad is that? Only you can decide how much those kinds of trade-offs are worth.
Posted by hplus0603 on 24 July 2016 - 02:34 PM
Once you run out of that, you can presumably horizontally shard the game.
Second, if the game is turn based, you can also look to in-RAM storage, like Redis, or even memcached, to get much faster querying. The draw-back is that you have to denormalize your data; ideally to the point where all state for a particular game instance is in a single data blob.
Third, why do you say HTTP won't work for the new Unity based game? Unity has the WWW and the UnityWebClient classes, so you can reasonably easily make it work. Especially if you serve on node.js or a similar server that makes it easy to "pend" incoming HTTP calls until there is actually data, or it times out. (Long polling)
Fifth, and now we get to the question you're asking, assuming the previous bits are already dead ends:
When building games with "current" game state, you don't want to store "current" game state in a database at all; you want to store it in RAM.
If the state is trivial to marshal/de-marshal (as it would be for a game of chess, or poker, say,) then you can store it in network-attached RAM, such as memcached or Redis. Each time someone wants to mutate the state, they do a transaction on the state by slurping, mutating, and writing back. This is trivial to shard horizontally if you get very successful, both the front end servers, and the back-end storage.
Once a game has an outcome, user information should be updated in a persistent database of some sort; similar for if you have consumables in the game.
If the rule set is more complex, or the simulation data is more complex, or you iterate the simulation in real time (think: physically simulated world,) then you don't want to demarshal/re-marshal for each simulation.
At that point, you inflate the game world/level into RAM in a game server, and route all player connections to the correct server process. The easiest way to do this is have users send their traffic/connect to a gateway, which knows where each game instance runs on the back end (again, assuming you're going to be successful and need to shard the game hosts.)
The hosts can periodically checkpoint state back to databases; they can also delegate to databases for very important stuff like player trade, commerce, etc, but most game updates happen in RAM, and if the server instance crashes, you lose whatever changes happened since the last save/checkpoint.
You can implement this on top of raw TCP sockets, or on top of websockets, or on top of HTTP long-poll, and it will work approximately the same (with some increased latency for each step to the right along that chain.)
Note that you can still implement a "message queue" of messages from server to client over HTTP. The client will simply make sure to always have a HTTP request outstanding to the server. When it comes in to the server, if the queue is empty, the server pends the request (doesn't send a response) until there is data; if there is data, the server immediately removes/returns it.
This doesn't mean the clients are "asking for it" -- it just means that you build an event transport on top of a double-buffered polled underlying network protocol.
Which option is the best? Depends on where you want to run clients, what client libraries you want to use, what server libraries you want to use, and how low latency your game really needs.
Finally, management games typically do NOT want to keep an instance per player running. Thus, they aren't really able to push messages to clients, except perhaps if you have a queue of alarms/timers where something happens on behalf of the player at a certain point in time.
Specifically, if the player starts action X, which will take 6 minutes, you don't actually keep that around for six minutes in a game server; instead you store "this thing will be done at time Y" and use the client to draw the progress bar. When the user asks for "what's the current state," you can wind time forward from the last stored state to the current time, return that, and store it back in the database. The wind-forward will be super fast for any typical management game.
Posted by hplus0603 on 17 July 2016 - 11:14 AM
Then, subtract start time from current high-precision time, divide by time step length, this gives you the time step number you should be at.
Now, if your step counter is less than the target, step physics repeatedly until you catch up. You may want to put an upper limit so you never step more than, say, 30 steps before rendering again.
You can do this on the server, as well as on the client. However, because client clocks aren't generally in sync with the server, it's typically better to just send the target step number to clients from the server as part of the update packets.
There's another bit of optimization you'll typically do where the client will send its actual step number (or clock) to the server in each packet, and the server will send its step number (or clock) to the client in each packet, as well as the timestep/clock that the client provided in the last-received packet.
This allows the client to calculate approximately how much latency it should compensate for, or, put another way, now much offset to add to its local clock.
To read a high-resolution clock:
On Linux, use clock_gettime().
On Windows, use QueryPerformanceCounter().
Posted by hplus0603 on 16 July 2016 - 10:21 PM
When the player takes too long to render, you must step simulation more than once.
If you can't step simulation at real time speed, the player's computer is too slow, and needs to be dropped from the game.
Posted by hplus0603 on 16 July 2016 - 07:32 PM
Can I still use client prediction with the first approach so I can continue to have smooth movement?
Well, kind-of. You are no longer deterministic. The player will move based on delayed information about the world.
If two players race for a ground pick-up, they will both think that they are way ahead of the other (by one RTT) even if they are neck-to-neck.
You will need to detect and correct any cases where the forward-simulated player is out of sync with the world, which is generally any time that player interacts with another player.
The deterministic model isn't that great if you don't go all in, IMO. If you can't take the latency, and if the players interact a lot, then model 1 kind-of doesn't fit.
That being said, model 1 is actually the model I have the most experience with, including forward-projecting players, and including doing crazy stuff like letting the player upper body / aim / head move locally at ahead time, but the lower body / feet / center of mass move based on another player, which is useful for sitting as a passenger in a vehicle.
If I were to build a new networked game from scratch, and I couldn't take the latency of "pure" model 1, I'd use model 2 if I didn't need to worry about cheating (which most hobby games don't, actually!) and #3 if it were a competitive FPS type game or something like it.
However, your game may be different. Only you can make the choice for how you want your game to play, and which of the options work best for you. The fact that you actually know you can't make it perfect is more than half the battle won already :-)