Jump to content

  • Log In with Google      Sign In   
  • Create Account


Member Since 03 Jun 2003
Offline Last Active Yesterday, 07:27 PM

#5307495 Create player accounts on custom server (for steam users)

Posted by on 23 August 2016 - 03:36 PM

The issue is that not everyone is that careful and looks at the url or the SSL certificate before typing this information in.

This is true, but there is no better solution :-(
Well, you can have players purchase a cryptographic token from you, but that's not a low-friction onboarding experience :-)

Or i use the current method which doesn't store personal userdata but is by no means secure due to the fact that the master-key is stored in the client

One option is to start there, and then give the user the option of signing in with a Steam ID if they want to be able to play from other places, able to upload files, etc.
Let the user decide.
Note: If you have that "easy onboarding" option, then if there is any kind of abuse possible of your system (using it to host file uploads, etc,) then that's what the bad people will use, so be sure to take that into consideration!

Are they (the majority of them) using SQL injections to do this?

There are about three attack vectors:

1) SQL injection, or other hosted-software vulnerability (WordPress, Drupal, ImageMagic, etc.) This generally gives you database dumps and perhaps admin interface access to the site.

2) Host vulnerability (Heartbleed, etc) This generally gives you command-line access to the site, which you can use to discover databases that you can then dump, insert payloads in hosted pages, etc.

3) Social engineering ("I am Robert, the County Password Inspector!") This generally gives you some kind of employee access to the site from the command line, again.

The question, then, is what the bad guys are after. If you store credit card details, absolutely that! If not, perhaps a list of emails and passwords to try on other sites. Or perhaps just another box they can run DDoS, email spam, and fradulent web sites from.
If you keep your development code on another system, with tight security, and use good source control, and good automated deployment methodology, you will minimize the impact from most such events, once you can detect that they occurred.
Wipe the hard disk, re-deploy to new OS image, restart servers; done!

#5307491 Questions about "reliable" UDP protocol and DB interface for MOG

Posted by on 23 August 2016 - 03:16 PM

First, any kind of "force" or "wait for response" function will end up blocking the UI of your game and be a terrible user experience.
Just Say No ™ to blocking network calls!

Second, for an RPG, I bet that TCP will work well enough. That way, you don't have to worry about whether a client got the update that there's a rock in a particular place or not.
Just send out a packet whenever the client desires to do something (move to a place, move in a direction, cast a spell, whatever) and on the server, collate these and send to all clients X times a second.
For an RPG, I bet 10 times a second would be plenty.
Just make sure to pack the entire set of updates into a single frame/packet/send() call, and turn on SO_NODELAY.

If the client has the same map as the server, the client will 99% of the time predict correctly what the server does.
It can just go ahead and render whatever it it believes should happen.
When it receives updates from the server, it can compare that response with what it, itself, thought the state was at that time (the time the updated pertained to.)
It can then re-play whatever inputs the player provided between that time and now, to show the next player position.

For "important" actions, like the result of spells, the result of fighting, purchasing/transactions, etc, you should play a local "wind-up" animation when the player first initiates it, but you should wait to show the result until the server sends "fireball blasted here" or whatever back.
That way, you will never show the player "you succeeded" only to take it back again 300 milliseconds later.

Billions of rows in a database is generally a bad idea, not because you couldn't store it on a hard disk (a 4 terabyte hard disk can store a lot!) but because the height of the index becomes very tall. And one component of database performance is number-of-indices times height-of-indices.
For most game character data, I would just store the character stats/abilities as one big JSON blob that's stored as inline text (a long varchar or maybe text field.)
This reduces the index height (number of rows) to the number of characters, which is better.
If you really plan on having millions of players, though, at some point, you're going to want to horizontally shard your data -- characters with ID 0 - 999,999 live on instance A; ID 1,000,000 - 1,999,999 live on instance B; ...
Then keep a table of ID range to instance mapping, so you talk to the right database instance for the given customer.
You will of course need a central table that goes from "customer name" to "customer id" so that you know which database to look at; this may have to live in a central database, but should be a much smaller table with a single index.
If you have really quite a lot of customers, even the mapping from "customer name" to "character id" will be too big to keep on a central server; at that point, you may shard that initial table based on "first character of customer name" or "hash of customer name modulo 20" or whatever. But you'll likely never get to that point, as pretty much Google and Facebook have that problem :-)

#5307032 Friends living abroad have really laggy connection to me, why?(using raknet,...

Posted by on 21 August 2016 - 10:28 AM

dropped TCP packets are just waited for at the router to be resent again

Routers, in general, do not wait for re-transmission before they forward, because they work at the IP layer, not the TCP layer.
For "reverse NAT servers" this changes, as they play the role of end-points, but when you talk about "router" in general internet parlance, you typically mean the boxes that sit in the middle of the network.

The re-transmission in TCP is entirely handled by the endpoints -- the machine doing the sending, and the machine doing the receiving on the other end.
And if one packet is dropped or delayed, ALL THE PACKETS AFTER THAT PACKET ARE HELD when they are received, waiting for the dropped/delayed packet to be re-sent.
By the time the dropped packet is detected, re-transmitted, and received on the other end, anything that was sent after that is also late, because it's been sitting in the kernel, received by the host, but not delivered to the application, because of the in-order guarantee.

Regarding spraying 100 packets at the same time, that doesn't really help, because the main reason a packet is dropped, is that there is congestion at some node on the network.
Once there is congestion, if the node receives 1 packet from you, or 100 packets from you, they will all be dropped at that time until the congestion recovers.
If you want to try duplication, consuming bandwidth in an attempt at higher robustness, it's much more robust to perhaps include the last 2-3 packets as copies in the next packets you send, so there is some time between each. This will add a bit of overhead, but if you structure your data right, it will RLE compress really well, and thus won't actually consume 200-300% of the bandwidth.
If the network drops three successive packets with a packet-send delay between each, the congestion is likely so bad that you're going to have a bad playing experience no matter what.

#5307030 Multiplayer Web Game - Is SQL fast enough?

Posted by on 21 August 2016 - 10:19 AM

Think of it like this:
If the turns could conceivably be delievered over email, then using MySQL to store/forward them is fine.
If the turns need to be interactive, then you need an interactive game server/persistence system of some sort.
If you have to live with the PHP "no global state persists past a web request" model, then an in-RAM storage like Redis can help, but of course you still have to pay the cost of inflating world state and then re-saving world state after each request.
If the game is truly interactive, then HTTP is the wrong protocol, and you want a persistent process server, a la most AAA 3D multiplayer games out there.

#5306815 Create player accounts on custom server (for steam users)

Posted by on 19 August 2016 - 07:01 PM

I actually do secure passwords by using sha256(password+uniqueSalt).

You should use bcrypt() or scrypt(), not just plain hashing.

the redirect from my website to steam (for the openID verification) could be intercepted and compromised

Presumably, the user will notice that the URL in the login window doesn't say "steampowered.com" but instead something like "steampwored.com" when looking at the SSL certificate validity key (green/red mark.)
This is in general a problem on the web -- some impostors are quite good. I guess the good news is that your game is probably not so super-successful as to be the biggest target for an attacker to go after.
That, in addition to "don't get hacked to the point where they can change your site."
Also, good site operations include making sure website code isn't surprisingly changed, perhaps re-deploying fresh code every so often and such.

Using a simple encryption key hard-coded into your site code to store the email address may be all you need to protect against most hackers -- most hackers just try to dump a database table to harvest info, and aren't super-coders with the resources of a nation state :-)

#5305796 Custom Level Upload-Server

Posted by on 14 August 2016 - 03:05 PM

First, nobody will DDoS you for any length of time unless you have either a very profitable game, or a game with a very rabidly emotionally attached user base, or both.
Either of those cases is actually, in the grand scheme of things, a good thing :-)

Second, because you're using LUA, you can actually prevent the users from doing anything "really bad" to the machine where it runs.
However, you have to make sure that the default LUA libraries for things like file I/O are not available.
Instead, only make functions available that respond to things that happen in your game. That way, you can build a "LUA sandbox" that doesn't let a script reach outside where it's supposed to be.
Designing secure, tight, sandbox APIs is actually pretty hard, unfortunately, but it's possible, assuming LUA doesn't have a bug in itself that's exploitable.
Also, user ratings of scripts ought to be able to alert you to any particular problems.

Server software for an upload/browse/download/rate system for text scripts? Literally any web application framework in the world will be able to do this.
(Static site generators don't count, as they don't generate applications :-)
Define a few REST services (GET/POST/PUT) that lets a user create a new file, upload the file, set metadata about the file, as well as post comments/ratings about it, and searching the database of files.

What's your favorite language?
Erlang with Webmachine?
Haskell with Warp?
PHP just as it is?
Python with Flask?
JavaScript with Nodejs?

For a few dozen files, you don't need more storage than a file system (that makes sure users can't overwrite existing files or create files with bad file names like "." or "/etc/passwd" or whatnot.)
For many thousand files, a simple MySQL (or Postgres, or MSSQL, or Cassandra, or RIAK, or ...) server will work just fine. Use MEDIUMTEXT or similar data type for the payload script.

Another good thing to do is set an upper limit to the size of the script (30 kB?) and make sure that the entire script parses like a proper LUA file (throw it at a sandbox LUA interpreter and look for errors) so that nobody uploads child porn or whatever.

Finally, you will need some what for users to log into your game server, both from a web interface (if you want to support web-based management/ratings) and from the client.
"Log in with Facebook/Google/Twitter/Whatever" is quite handy for web servers, but much harder to do for game clients, because it assumes the game client is a full-fledged browser that can talk HTTPS and follow HTTP redirects and show HTML/JavaScript applications to the user.
An alternative is to just create an account with email address and password, and make the user verify the email address with a link you mail them.
Most web application frameworks should be able to do this for you.

#5305188 How To See A Router's Forwarding Table?

Posted by on 10 August 2016 - 05:33 PM

when all IP addresses were right there on the public internet

IPv6 (now reaching about 10% of the internet!) gives you that again.
The IETF currently believes that IPv6 doesn't need any NAT.
The official recommendation is that "correct firewall configuration is the user's responsibility."
I think that's short-sighted, because one of the main benefits of NAT is that users by default have a reasonable stateful firewall preventing their Windows network shares from being accidentally exposed to the internet.

#5304985 How To See A Router's Forwarding Table?

Posted by on 09 August 2016 - 04:07 PM

On the client, you should put the router's IP address, and the external port (being forwarded,) into the connect() function.
On the server, you do not use connect() at all.

So, let's assume your router's external address is, and that the external port is 5432.
Let's assume that your PC's internal address is, and that the internal port is 3800.

You should set up your router to port forward external port 5432 to internal address, port 3800.
You should then, in the client that connects to your router from the internet, put in the address and the port 5432.

Now, if you try to connect to your PC from inside your network, your router may not support "hairpin NAT," and thus only when you are inside your network, you need to connect to port 3800 for it to work.

#5304692 Should I Get A Low-Level Or A High-Level Lib For A Really Basic, Not Secured,...

Posted by on 08 August 2016 - 10:40 AM

when I actually open a port without any security, I open myself to some bad stuff they can do to harm my pc

Opening a port gives the external world access to whatever is available on that port.
If it's your chat program, and it has no bugs, then it only gives the world access to that port.
If it's your chat program, and you have bugs (such as buffer overflows,) then a skilled attacker may be able to break out of the server executable that you are running.
However, in practice, it's unlikely that a skilled attacker will be interested in your program while it's small, so in reality, the risk is minimal, unless you do something really crazy like allow users to run shell commands through chat or whatever.

That being said, hosting on your home computer on your home network is always troublesome -- you can't keep it up forever, your IP address may change, your computer is also needed for other things, ...
Instead, I recommend a cheap hosting provider of some sort. I myself use $6 virtual private servers from interserver.net but there are others -- $10 from linode -- $15 from dreamhost -- micro instances from amazon.
If you want to run Windows, they usually cost a bit more (and you can also look at [ulr=https://azure.microsoft.com/en-us/]Microsoft Azure[/url])
The benefit with these is that you can pay for them for as long as you need them (Amazon: hourly; most others: monthly) and then stop paying for them.
Also, if the machine gets hacked, just destroy the instance, fix the bug, and spin up a new instance.

#5304041 Should I Get A Low-Level Or A High-Level Lib For A Really Basic, Not Secured,...

Posted by on 04 August 2016 - 12:49 PM

why can't just people use simple words

Because distributed systems and networking is, unfortunately, not a simple problem.

#5303866 Should I Get A Low-Level Or A High-Level Lib For A Really Basic, Not Secured,...

Posted by on 03 August 2016 - 04:08 PM

RakNet and Enet are both fine libraries for C / C++ (and some other languages with bindings.)
They are a little higher level than SDL_net, boost::asio or other "socket wrappers."

#5303394 Best Client/server Architecture For A Mobile Management Game?

Posted by on 31 July 2016 - 08:05 PM

what potential alternatives - other than Erlang - there are for any serious network programmer?

You can write a scalable system in Node.js. Just look at eBay -- they rewrote their web page front-ends on Node.js.
It just takes more work. And, because it's not a statically checked language, you need more unit tests.

For distributed server projects for games with real-time components (simulation servers or similar,) I would personally look at the following:

- C++ -- you will never, ever, be blocked by some bug in some infrastructure, because you can always re-implement it yourself. Also, if you need to go "to the metal," there's no change in tools or build systems -- you're already there! Also, easily links whatever third party libraries you want.
- Java (ecosystem) -- whether you use Clojure, Java proper, or Scala, this is a suite of tools that a lot of people know how to run, and it's easy to find help here. It's a little more ghetto than C++, and a little less real-time (lots of Hadoop / Kafka / Spark type systems) but not to be ignored.
- C# (ecosystem) -- this wasn't on my list before, because mono is hilariously non-good for servers, but with the release of .NET Core open source, I'd love to take a look at that and see what it can do. C# is a more efficient language than Java by design, and Visual Studio is the world's best development environment.
- Erlang, as I said. It has some limitations and bugs, and you need to really learn and internalize the tools that come with it (OTP and the various distribution support libraries.) But excellent systems can be built.

I could conceivably talk about others. I love the Haskell web service infrastructure we have at work, but getting to that point was a lot of work -- there are less batteries included in that part of the world.
Python, Node, and Go are all possible, if the project is smaller. Neither is particularly good at threading, though, and neither is particularly fast; probably Node is fastest because it benefits from the massive optimizations in V8.
Perl, PHP, Ruby -- not really suitable for scalable, multiplayer, real-time game servers. I mean, you could, if you REALLY wanted to. But why would you?

Wow. Seeing it all in a single place makes it seem like ... there are a lot of languages and environments out there!

#5303351 Best Client/server Architecture For A Mobile Management Game?

Posted by on 31 July 2016 - 12:46 PM

Single-threading is always easier than multi-threading, because a whole class of design bugs (synchronization and race conditions) just doesn't exist.
The only draw-back of single-threading is that it doesn't make good use of existing hardware resources -- but if you're already using a dynamically tag-checked language, rather than a statically compiled language, that's probably not your main priority!

Regarding Elixir: I'm happy that it draws some attention to the Erlang ecosystem, but in my opinion, it's a solution to a problem that doesn't exist.
"Hey, let's take a functional language and pretend it looks like Ruby" is just not a good idea, except for a few initial days if you happen to already know Ruby.
Shortly thereafter, you will find that the syntax they provide (like re-assigning variable names) actually gets in the way, and causes potentially significant performance problems.
I would recommend just learning the Erlang syntax, if you want to live in that ecosystem.

Btw: If you want to build threaded, distributed, servers, the Erlang ecosystem helps a lot. It has tools for many of the harder things, like "leader election" or "distributed workers" or whatnot.
Also, the cost of learning a new language/syntax (with Erlang) is much smaller, long term, than the cost of learning to live in a distributed world. That's the real challenge!

Anyway: If max scalability and efficient hardware use isn't your top priority, and your project is small so dynamic type checking is okay, then Node.js is fine.

#5302381 Best Client/server Architecture For A Mobile Management Game?

Posted by on 24 July 2016 - 04:54 PM

User A buys something on City 1 and so the price changed.
Users B,C,D and E, who the server has in memory that are looking at that city right now, recieve a pushed message from the server notifying of said price change.

You can totally do that over long-polling over HTTP. Our in-RAM message queue / bus system can push messages to online users through TCP sockets, websocket sockets, or HTTP long-polling, through different gateways for each method. To the server, it all looks the same: "post this message on this topic."

Separately, if a user app is not in the foreground, then on iOS, you *have* to use polled communications and/or mobile push; you don't get a persistent connection while backgrounded. On Android, you could have a persistent connection from a Service, but doing so will likely drain the battery faster.

I cannot loose a user buying/selling if the server goes down

I did suggest that important actions like trade should go to the database, so it sounds like we agree. As long as players don't trade "often" (dozens of times per second, or whatever,) then a database would be reasonable. They can always be horizontally sharded if necessary (at a cost, of course -- it's reducing cost that's the main goal here.)

Separately, if all players lose the same amount of time in a crash, and that amount of time is never more than 15 minutes, how bad is that? Only you can decide how much those kinds of trade-offs are worth.

#5302341 Best Client/server Architecture For A Mobile Management Game?

Posted by on 24 July 2016 - 02:34 PM

So, first, a dozen queries every 5 seconds isn't all that much. You can easily do 1000 parallel players on a single MySQL instance that way, making sure that you have SSDs and/or enough RAM for caching/indexing.
Once you run out of that, you can presumably horizontally shard the game.

Second, if the game is turn based, you can also look to in-RAM storage, like Redis, or even memcached, to get much faster querying. The draw-back is that you have to denormalize your data; ideally to the point where all state for a particular game instance is in a single data blob.

Third, why do you say HTTP won't work for the new Unity based game? Unity has the WWW and the UnityWebClient classes, so you can reasonably easily make it work. Especially if you serve on node.js or a similar server that makes it easy to "pend" incoming HTTP calls until there is actually data, or it times out. (Long polling)

Fourth, just embedding a webview may not be fast enough, but there might be other options. Anything from "compile javascript to Objective C" to "make your core website responsive and just install a shortcut used by the system browser on mobile." You may have pushed far enough along this path that you can tell for sure this won't work; or not; it's hard to tell from the data you gave.

Fifth, and now we get to the question you're asking, assuming the previous bits are already dead ends:

When building games with "current" game state, you don't want to store "current" game state in a database at all; you want to store it in RAM.
If the state is trivial to marshal/de-marshal (as it would be for a game of chess, or poker, say,) then you can store it in network-attached RAM, such as memcached or Redis. Each time someone wants to mutate the state, they do a transaction on the state by slurping, mutating, and writing back. This is trivial to shard horizontally if you get very successful, both the front end servers, and the back-end storage.
Once a game has an outcome, user information should be updated in a persistent database of some sort; similar for if you have consumables in the game.

If the rule set is more complex, or the simulation data is more complex, or you iterate the simulation in real time (think: physically simulated world,) then you don't want to demarshal/re-marshal for each simulation.
At that point, you inflate the game world/level into RAM in a game server, and route all player connections to the correct server process. The easiest way to do this is have users send their traffic/connect to a gateway, which knows where each game instance runs on the back end (again, assuming you're going to be successful and need to shard the game hosts.)
The hosts can periodically checkpoint state back to databases; they can also delegate to databases for very important stuff like player trade, commerce, etc, but most game updates happen in RAM, and if the server instance crashes, you lose whatever changes happened since the last save/checkpoint.

You can implement this on top of raw TCP sockets, or on top of websockets, or on top of HTTP long-poll, and it will work approximately the same (with some increased latency for each step to the right along that chain.)
Note that you can still implement a "message queue" of messages from server to client over HTTP. The client will simply make sure to always have a HTTP request outstanding to the server. When it comes in to the server, if the queue is empty, the server pends the request (doesn't send a response) until there is data; if there is data, the server immediately removes/returns it.
This doesn't mean the clients are "asking for it" -- it just means that you build an event transport on top of a double-buffered polled underlying network protocol.

Which option is the best? Depends on where you want to run clients, what client libraries you want to use, what server libraries you want to use, and how low latency your game really needs.

Finally, management games typically do NOT want to keep an instance per player running. Thus, they aren't really able to push messages to clients, except perhaps if you have a queue of alarms/timers where something happens on behalf of the player at a certain point in time.
Specifically, if the player starts action X, which will take 6 minutes, you don't actually keep that around for six minutes in a game server; instead you store "this thing will be done at time Y" and use the client to draw the progress bar. When the user asks for "what's the current state," you can wind time forward from the last stored state to the current time, return that, and store it back in the database. The wind-forward will be super fast for any typical management game.