Or you could look at a whole different approach - use steam (even though they will take a cut), you can develop your dedicated servers that the public will be able to host to verify each player is a valid steam client with a copy of the game (You should be able to do this, if not my bad. I havn't played around with the steamworks API main due to the problem being I havnt released a game on steam yet).
I thought of doing that as well...
I would agree with hplus. Just make the game first. You can develop the game with efficiency and reliability in mind but, once you release the game you can then expand (improve hardware) as the game grows. 10,000 connections per second wont happen overnight.
I think you guys are missing the point. I have a game.... the point now is to work on the connections to enter the game. I wouldnt be coming here and asking these things if I did not have a game already in place. I am looking for ideas on how best to tackle the situation not advice on where to start the game development; while I appreciate the concern, it is more or less pointless to attempt to try and tell me "dont bother with this till much later". I am bothering with it since it is where we are going next. As for 10,000 requests... i have no clue what is or is not a bit amount. I have come to this section looking for knowledge... If this is the improper practice I am sorry for my misunderstanding.
Wouldn't it be an even huger waste of time to spend time writing support for a game that nobody wants to play? Or spending time optimizing something which won't actually turn out to be a bottleneck in the end?
It would be, assuming I had nothing... an assumption that is very much unfounded.
But that doesn't mean that login servers are a bottleneck at all. A login server needs to verify user name/password, generate a signed ticket with some lifetime, and write an entry into a log file. You can do thousands of that per second on a single virtual instance, and if you have a thousand users per second logging in, you're bigger than any indie game company I know of (except possibly for Steam.)
This is good information to have, something that I did not know and something that is important to realize. Scope is the whole point of this and if the scope I was looking at is too grand than I can certainly scale that back a bit. This is constructive information, much more than the "dont bother with this for now" comments.
I think Minecraft is a phenomenal success and I would have been very proud have I created it. Sure it can be better, but 99.9% of all projects are actually failures. Usually, the problem is that developers spend way too much time building things that don't matter *yet* instead of making sure they figure out what the successful game will look like. Building robust infrastructure as you need it is much easier than building a compelling experience that users want to pay for.
That being said: If you can spend six hours instead of two hours on some task, and the difference is that with the six hours, you solve a problem right, and with two hours, it will fail when you have a hundred users, then spend the six hours. Just don't spend six days if you can unblock the "fun finding" with two hours of work.
I never said that minecraft wasnt a success, i said I do not want to fail on the aspect of not being able to login, as they have had that issue on several occasions. The point of this thread is not for me to get advice on what I should be working on. The point of the thread is to get advice on how best to handle a login server. I am not the main coder of my game and because of this I have all the time in the world to waste on asking these questions and learning. Without knowing my development process and my development team such advice seems unfounded and more or less a waste of all of our time. If you have something of value to add to the post please let me know so I can learn and start to do some research. Otherwise, I would very much appreciate the comments about "not a valid use of time" to cease. Part of the development process of any project is research and development.... to get there you have to know how things work and how they are built. That is what I am doing with this thread... finding out the information with the help of people who are, much like yourselves, experienced. Walking through the jungle can be a very dangerous path... but if you have a tour guide they can show you the things you need to know to get through and focus on what you are looking for.
Thus, the creation of this post. Finding those people who know much more about a thing that I have barley scratched the surface on. Again I will stress that I have all the time in the world to be doing this so please indulge me with your comments on how best to approach this, otherwise dont post about how I should or shouldnt handle my time.
If you want to understand load generation, set up a user database in Redis or something. Spin up an Apache or nginx or Node or something. Write a little PHP script or JavaScript or whatever, where all it does is:
0) decode the user name (email?) and password from the request
1) look up the user record based on the user name / email in Redis
2) hash the provided password with a salt
3) compare the password hash with the hash in the user record
4) if mismatch, return error
5) if match, create a ticket consisting of (username:expirytime:hash(username:expirytime:serversecret)) and return success
Now, your client can present the ticket to any server, and it can verify that the user is who he/she says he/she is by simply verifying the hash (all servers should be configured with the same server secret.)
Now, use "ab" or a similar load testing program, running on another machine or two, to generate load for this service, and see how far it goes.
Note that, if average session time is 1 hour, and average login request service time is 10 ms, then you can serve 360,000 simultaneous online users from a single login server, and I'd be quite surprised if the login mechanism described above couldn't be made to run much faster than that.
And, if you're using a simple key/value store for user records, and a simple signed-ticket service for login/authentication, then you can scale horizontally with very little effort -- use consistent hashing for the storage instances, and any HTTP load balancer for the front end. This is a solved problem, and as long as you have a reasonable path towards the scalability you need, you don't actually need to develop and test it until you actually need it. Time is the most precious resource we have, and using it to address the biggest risk first is, IMO, the best use of it.
This is exactly what I like to read as it is informative and supports the idea you wish to express. Informative and it shares your point of.... its not needed right now in a way that seems less hostile. Thank you for this information as it will certainly help in me finding the path I am searching for.
You are wasting your time, first write a game that demands 10,000 requests per second, and then you will be able to hire a whole team of hplus types to solve all your problems
Read the above wall of text. Post is pointless and I have no need to hire teams of hplus's. The point of this forum ( as far as I understood it ) was for would be game developers to help one another grow and develop. I am not asking for direct answers or for people to do the work for me, what I did ask is how people would do it or what ideas they might have when dealing with such a request. Information like that can lead me to the answers I am really looking for and save me wasted time of wading through thousands of hours of bullshit that has no relevance to what I am looking for.
Again, thank you for the help hplus, your most recent post was very helpful.