I am not trying to guard against any particular type of attack, I just want to have some general security measures for account information protection (login/password/CD-Key) that are being sent across the client and server.

I am attempting to implement WSASetSocketSecurity but I am having a difficult time of it due to a lack of documentation and examples. I apparently have the security settings enabled, the client connects to local computer on the same system, but when I connect to remote server it fails to connect.

I must admit I don't even know if I am going in the right direction with this. It almost literally like seems one step forward two steps back. If I cannot get this security socket protocol working I may just proceed with the salt/hashing algorithms on a non-secure socket and just call it there.

Okay, so after much more research and running into brick walls I have come up with the solution of just holding off on upgrading to secure sockets, and just focus on the packet hash and encryption for now. Based on what I have read and researched, this seems to be the simplest answer for me until I really want to go full speed into securing sockets for money transfers,etc (which I am currently not doing nor interested in at this time).

So for my account protection I do this:

For Account creation:

1. Player enters email,login,password on the client

2. Send only the login name to the server

3. The server creates the required information for the account, including a salt, and a time-based mask string (to prevent replay attacks)

4. Server sends the salt, and the mask to the client

5. Client adds the salt to the password string

6. Client performs a SHA256 hash on the new string using the OpenSSL SHA256 function.

7. Client applies the time-based mask to the hash

8. Client sends all the account information (login,email,hash) email is masked as well for posterity

9. Server de-masks the information and places it into the account database

A similar method is used for the login process as well.

Any suggestions for what to improve on this are appreciated. And again thanks for all the replies, it was all helpful.

"holding off on secure sockets" is the wrong solution.

There are two levels of security:

1) You don't really care if a determined man in the middle can read the data. Such a man in the middle must have access to the wires to snoop on the network (or an open wireless network.) At this point, XOR encryption, or no encryption, or your own cryptosystem based on hashes, is approximately the same level of security. Spending any time on this is time you don't have to spend on more important things, so don't do it.

2) You care about determined men in the middle (the most common being open wireless network sniffers.) At this point, you use TLS. If you can't get the Microsoft TLS wrapper to work, then use a library like openssl or whatever.

Note that, to guard against men in the middle, you have to also do remote host authentication, which requires you to get a SSL certificate. You can create a self-signed certificate, and provide the public part of that with your installed client, if you don't want to pay for a "real" SSL certificate.

(I'm sloppily using "SSL" and "TLS" interchangeably above, whereas you really should be using TLS for everything these days.)

Well it was a monsterous effort for me, but I decided to take your suggestions and I have integrated OpenSSL into my code and I have implemented Secure Sockets. I even got it working with non-blocking sockets (so far I had no problems with it, but research tells me its a complicated issue to implement non-blocking sockets with OpenSSL but, knock on wood).

I searched a huge amount of sites trying to get this all implemented, but for those who are in the same boat as I am, these websites were the most helpful:

http://simplestcodings.blogspot.com/2010/08/secure-server-client-using-openssl-in-c.html - A great set of code to test secure socket openSSL connection, even tell you how to make your own certificates.

http://slproweb.com/products/Win32OpenSSL.html - a Win32 C++ version of the OpenSSL source with libs ready to go.

So hopefully with this implemented and the previous methods for password protection I have a decent basis of network security, and I have OpenSSL all ready to go if I need to continue along this path.

I am assuming my self-made certificate will be acceptable for now since I am still in the testing phases of my game.

The worst part of this process was the lack of documentation and examples I was running into. Some examples even looked completely foreign from others, where they almost looked like another API.

Again I am appreciating all this advice from you all. I hope I am done posting any major questions about this, but who knows :)

I decided to take your suggestions and I have integrated OpenSSL

Great!

I am assuming my self-made certificate will be acceptable for now

As long as you control the game client, so you can provide your own root certificate to the client, it will be good enough for production, too.

I hopefully have one final question regarding this. Now that I have the secure connection to pass account information back and forth, I want to remove that secure connection afterwards so I can use an unsecure socket for the game packets since none of the data that will be sent will be of any compromising value (I am assuming that using an unsecure socket is much faster to send the data).

Would I be able to open two sockets on the client, one Secure and one non-secure, then when the secure socket authorizes the login, It sends a unique key to the client for the unsecure socket to connect to the server to initialize the unsecure socket?

I am afraid that I may do something that would compromise all the security I just implemented in some way.

UPDATE: What I am going to plan to do is have the client connect with two sockets on two different ports, one secure, one normal. I will compare the IP address of the two incoming sockets and "link" them together on the server. Once the authentications are done, I will stop the secure socket, and only use the normal unsecure socket for the game. If there is any other way to accomplish this I would appreciate any ideas. Again, I searched for many hours about this and found no answer for this. I feel I am missing something simple here but I cant seem to put my finger on it.

Generally speaking, network latency is going to dwarf any time spent on encryption/decryption of the actual packet stream. It is entirely possible to have a realtime network simulation using encrypted traffic.

I am assuming that using an unsecure socket is much faster to send the data

In 1995, that might have been the case.

These days, not so much. CPUs are very fast, and wires are very, very, slow by comparison. I'd just keep using the secure socket.