• Announcements

    • khawk

      Download the Game Design and Indie Game Marketing Freebook   07/19/17

      GameDev.net and CRC Press have teamed up to bring a free ebook of content curated from top titles published by CRC Press. The freebook, Practices of Game Design & Indie Game Marketing, includes chapters from The Art of Game Design: A Book of Lenses, A Practical Guide to Indie Game Marketing, and An Architectural Approach to Level Design. The GameDev.net FreeBook is relevant to game designers, developers, and those interested in learning more about the challenges in game development. We know game development can be a tough discipline and business, so we picked several chapters from CRC Press titles that we thought would be of interest to you, the GameDev.net audience, in your journey to design, develop, and market your next game. The free ebook is available through CRC Press by clicking here. The Curated Books The Art of Game Design: A Book of Lenses, Second Edition, by Jesse Schell Presents 100+ sets of questions, or different lenses, for viewing a game’s design, encompassing diverse fields such as psychology, architecture, music, film, software engineering, theme park design, mathematics, anthropology, and more. Written by one of the world's top game designers, this book describes the deepest and most fundamental principles of game design, demonstrating how tactics used in board, card, and athletic games also work in video games. It provides practical instruction on creating world-class games that will be played again and again. View it here. A Practical Guide to Indie Game Marketing, by Joel Dreskin Marketing is an essential but too frequently overlooked or minimized component of the release plan for indie games. A Practical Guide to Indie Game Marketing provides you with the tools needed to build visibility and sell your indie games. With special focus on those developers with small budgets and limited staff and resources, this book is packed with tangible recommendations and techniques that you can put to use immediately. As a seasoned professional of the indie game arena, author Joel Dreskin gives you insight into practical, real-world experiences of marketing numerous successful games and also provides stories of the failures. View it here. An Architectural Approach to Level Design This is one of the first books to integrate architectural and spatial design theory with the field of level design. The book presents architectural techniques and theories for level designers to use in their own work. It connects architecture and level design in different ways that address the practical elements of how designers construct space and the experiential elements of how and why humans interact with this space. Throughout the text, readers learn skills for spatial layout, evoking emotion through gamespaces, and creating better levels through architectural theory. View it here. Learn more and download the ebook by clicking here. Did you know? GameDev.net and CRC Press also recently teamed up to bring GDNet+ Members up to a 20% discount on all CRC Press books. Learn more about this and other benefits here.
Sign in to follow this  
Followers 0
BinaryStorm

On a scale from 1 to 10 how bad of an idea would it be to use a JSON like format for game networking?

24 posts in this topic

It's actually lua, but there's not much difference there.

 

So I'm making a turned based strategy game and it does a lot of work with lua scripts server side. I created a wrapper class that I really like to use. So far I've been using it to interact with objects in the lua environment and save and load data etc. Then I thought: 'why not just use lua to send data across the network too?'

 

I figure the biggest bottleneck would be interpreting the code at the client and that doesn't worry since each individual client won't be receiving that many messages.

 

I know it's more bulky than binary, but it seems pretty convenient. Do you think it'll have a big impact on how many users the server can support? Sorry if this is a dumb question.

0

Share this post


Link to post
Share on other sites

[quote name='BinaryStorm' timestamp='1357848257' post='5019982']

I know it's more bulky than binary
[/quote]

 

My friend, you've answered your own question. If you can use binary, then use binary... or maybe you should try both and do some performance testing.

0

Share this post


Link to post
Share on other sites

A JSON like format - great. It will make debugging easier, that's for sure.

 

But Lua? Unless you have a Lua parser that will treat it purely as data and not try to execute it, it would be extremely dangerous. Anybody on the internet could craft some malicious Lua code and send it to your server to be executed.

2

Share this post


Link to post
Share on other sites

Didn't think about security. For what it's worth, messages would probably be executed in their own lua states, and I won't even load the standard libraries into those. I use states like that to load data.

0

Share this post


Link to post
Share on other sites

Why not use google protocol buffers instead? It'll be much faster and it has inter-language communication capabilities. The only drawback is the awkward build process.

2

Share this post


Link to post
Share on other sites

I use json all the time for turn based networking for mobile games.  I only use simple dictionaries though.  One thing to watchout for with LUA though is if you are targeting iPhone.  I know somebody who tried sending lua scripts as part of in app purchase downloadable content and his app was rejected.

0

Share this post


Link to post
Share on other sites

Agreed on the security angle, however you should have more robust security in place than simple obscurity. What people seem to be worried about is, whatever protection scheme you have, the consequences become really dire, really immediately, if you're sending executable instructions around.

 

However, if you're truly using lua as a data-passing layer only, it probably wouldn't be too difficult to write a parser for that subset of the language (which would allow you continue using it as you are), convert to JSON (at least maintain ease of debugging), or use some other compact format, even binary.

2

Share this post


Link to post
Share on other sites

The points are:

- Worse performance

- Away higher bandwith consuption.

- Easier to hack.

 

You can counter the second one by compressing the packages and the third one by encrypting the protocol (it still will be easier than binary approach, but not THAT easy). Both those counters will make the first point even worst.

 

So, here are your cons, weight then and decide if it is worth it. Personally I wouldn't unless it is something like school project, round based with small player base game.

0

Share this post


Link to post
Share on other sites

I guess there isn't that much to gain. I just really love using my Lua wrapper class. It has a lot of cool functionality, but it's probably not the tool for this job. Thanks for all the answers.

0

Share this post


Link to post
Share on other sites

Existing (and even moderately successful) real-life games do that. I know at least two Facebook MOs that do this, one recently added XOR-encryption (*cough*) for security. I remember one (successful) MMO that I played years ago that would simply send the output of Java's Serialize() over the network (unencrypted). No, it didn't break their neck. They eventually added encryption after 5 or 6 years to thwart proxy botting. They're still in business.

 

How good/bad an idea is it on a scale from 1 to 10? Depends, it can be anything from 1 to 15.

 

If you use a properly encrypted stream, the only disadvantage to using JSON is that you waste a bit if extra bandwidth copared to something binary. If you can afford that extra bandwidth, there's no issue. The huge advantage is that JSON already works, and is dead easy to use.

 

Now if you don't encrypt your traffic and you are already struggling with bandwidth, then 10 is not a high enough number to express the trouble you're getting in.

0

Share this post


Link to post
Share on other sites

[quote name='KnolanCross' timestamp='1357911300' post='5020298']
You can counter the second one by compressing the packages and the third one by encrypting the protocol (it still will be easier than binary approach, but not THAT easy). Both those counters will make the first point even worst.
[/quote]

 

Actually, neither of those counters make the first worse -- the biggest bottleneck here is bandwidth over the wire, compression reduces the amount of info, thereby making better use of bandwidth, and encryption has no real effect. A computer can decompress and decrypt a packet 100 times over and still take Sunday off in the time it took that same packet to traverse the wire.

 

Heck, AAA console games compress data on the disc because its faster to read less data and decompress it than it is to just read the raw data -- and the disc transfer speed is probably an order of magnitude faster than an average internet connection, and the console CPUs less than half as fast as a typical modern computer.

 

I suppose on the server side, if you've got tens of thousands of players sending packets into your data center, decompression and decryption *might* be a noticeable in your profiling data, but if you're running something that large and successful you can afford to throw hardware resources at the problem.

 

There are plenty of valid cons here, but that you'd have to compress and encrypt the packets to make it viable is far from one of them.

1

Share this post


Link to post
Share on other sites
This is a severely bad idea, unless you really want malicious players to have free run of your game - both client and server.

The format of the packets -- whether JSON, XML, ASN.1-BER, or your custom binary serialization format -- does not change your security model at all.

JSON is fine as long as throughput doesn't matter. I know several turn-based games that use JSON, including some that I've worked on myself.

Once you measure your data rate in "packets per second" rather than "seconds per packet," you may find that JSON is too verbose -- or, more likely, that HTTP is too verbose, if that's your transport. At that point, a direct, persistent connection with binary payloads starts making a lot of sense.
0

Share this post


Link to post
Share on other sites
The format of the packets -- whether JSON, XML, ASN.1-BER, or your custom binary serialization format -- does not change your security model at all.

JSON is fine as long as throughput doesn't matter. I know several turn-based games that use JSON, including some that I've worked on myself.

Once you measure your data rate in "packets per second" rather than "seconds per packet," you may find that JSON is too verbose -- or, more likely, that HTTP is too verbose, if that's your transport. At that point, a direct, persistent connection with binary payloads starts making a lot of sense.


I never said anything to the contrary.

My point is transmitting executable code as a communication mechanism does change your security profile. To make it secure you have to basically strip out all execution potential from your data stream and the interpretation mechanisms, at which point you may as well just use an existing serialization mechanism in the first place.


[Edit] For further clarity: I'm not talking about the subject line (which talks about JSON) but rather about the actual question the OP asked, which was using Lua as a JSON replacement. JSON is not executable by default; Lua is. That's all I'm saying is a bad idea. Edited by ApochPiQ
1

Share this post


Link to post
Share on other sites
My point is transmitting executable code as a communication mechanism does change your security profile. To make it secure you have to basically strip out all execution potential from your data stream and the interpretation mechanisms, at which point you may as well just use an existing serialization mechanism in the first place.

Every web browser and web site sends gobs of executable code to your client -- Javascript is the de facto application language of the future. Yet, security holes don't come from Javascript; they come from Java or Flash plug-ins written in native code.
0

Share this post


Link to post
Share on other sites
I really don't understand what that has to do with the present discussion.

JavaScript is heavily sandboxed but still has security implications (were you not on the Internet in the late 1990's?). It's not like someone just says "herp derp have some executable code!" and magically it's safe because we called it JavaScript. I really can't believe that you'd be trying to imply something that disingenuous, either, so I don't follow your point at all.

I also have a hard time believing that you're unaware of the vast differences in security model between web services and client/server games. So I have no idea why you feel that one-way transmission of executable data via the web is a relevant point of comparison for bidirectional communication where a client can submit unverified executable data back to a server.


If you really want to use the web as a comparison, fix your analogy. The OP is suggesting creating a web site where I can upload arbitrary PHP or ASP.Net or JSP code and execute it as a superuser. All I'm saying here is that to ensure security you need to take a lot of precautions - things which you seem to be implying don't exist on the web, when I think you honestly know a lot better.



So yeah. I'm confused now.
0

Share this post


Link to post
Share on other sites
You're right, this is side tracking. The argument that sending executable code is unsafe isn't particularly strong IMO.

Sending code from the server to the client is done all the time, even for native code: That's what a patcher does!

You can sandbox whatever you execute to the set of instructions that make sense, and not give access to anything that would be dangerous, for code sent from client to server. Many games do this, too -- any game with a mod system, any game with user scripts (like Second Life,) any web site that lets you add javascript to your pages, etc.

Regarding "you have to encrypt to avoid tha haxx OMGWFTBBQ!1!" argument, we've had that so many times. Encrypting data that the client generates is pretty useless, as the potential attacker has full control of the client anyway, and can simply use your own encryption function to send whatever data he wants to. There is a narrow middle ground where some encryption dissuades the most casual of hackers, and your game isn't successful enough that anyone real wants to attack it, where it might help, but I feel that you're better off making your game good enough that you'll exceed that narrow window :-)
0

Share this post


Link to post
Share on other sites
1: If it's truly turn based, this should be completely fine, and will make life a lot easier I'd say. I've even considering using compressed json strings for packets on real time games just because JSON rocks.

 

Seconded. If you're writing abstractly enough, then swapping out portions of your network data-transmission module when your game reaches a point where security and/or performance are under scrutiny shouldn't be a big deal. The up-side is you've got a working game in a shorter amount of time, which lets you focus on more important things -- like finishing the game.

0

Share this post


Link to post
Share on other sites
Existing (and even moderately successful) real-life games do that. I know at least two Facebook MOs that do this, one recently added XOR-encryption (*cough*) for security.
Everybody knows that you need 2 rounds of XOR-encryption for it to work...


I remember one (successful) MMO that I played years ago that would simply send the output of Java's Serialize() over the network (unencrypted). No, it didn't break their neck. They eventually added encryption after 5 or 6 years to thwart proxy botting. They're still in business.
I often lament the fact that software development is heavily influenced by the law of diminishing returns. It's so much more costly to make something correct than it is to make something adequate, so there's rarely a business case for the former. If it all blows up later, you can almost always fix it later. Edited by Kylotan
formatting, once more
0

Share this post


Link to post
Share on other sites

I'm afraid I have to disagree with your conclusions on serialization/deserialization risk.

 

This is a severely bad idea, unless you really want malicious players to have free run of your game - both client and server.

The format of the packets -- whether JSON, XML, ASN.1-BER, or your custom binary serialization format -- does not change your security model at all.

 

This isn't actually true. The more generalized your serialization/deserialization is, the worse its security posture is. The worst being of course, deserialization into arbitrary code (ala JSONP or YAML). The recent issues in the Rails community should highlight that.

 

My point is transmitting executable code as a communication mechanism does change your security profile. To make it secure you have to basically strip out all execution potential from your data stream and the interpretation mechanisms, at which point you may as well just use an existing serialization mechanism in the first place.

Every web browser and web site sends gobs of executable code to your client -- Javascript is the de facto application language of the future. Yet, security holes don't come from Javascript; they come from Java or Flash plug-ins written in native code.

 

There are whole classes of browser exploits that are only possible because of the ability of an attacker to run arbitrary Javascript. So again, bad example.

 

You're right, this is side tracking. The argument that sending executable code is unsafe isn't particularly strong IMO.

Sending code from the server to the client is done all the time, even for native code: That's what a patcher does!

 

Again, this is a bad example. There are so many badly written patchers out there that there are a whole class of "Evilgrade" vulnerabilities, where an attacker can impersonate the update service and run arbitrary code.

 

To pull things back to the original question, JSON is actually a reasonable choice, with a couple of caveats.

 

The first is never use JSONP. The second is to make sure that the JSON parser you are using is not calling a more dangerous parser behind the scenes (as in a recent Rails issue).

 

I would certainly use JSON over a languages generic object serialize/deserialize functionality; you want to make sure that you can strictly control the output of the parsing of untrusted data, and anything that looks like "object Deserialize( byte[] rawNetworkData)" (in whatever language you are using) is inherently unwise.

0

Share this post


Link to post
Share on other sites

You could give the MessagePack protocol (http://msgpack.org/) a go.  It's like a binary JSON with readers and writers in many languages.  I've successfully used it on a server network running LuaJIT to transfer data around, and it's quite compact.

1

Share this post


Link to post
Share on other sites

A JSON like format - great. It will make debugging easier, that's for sure.

 

But Lua? Unless you have a Lua parser that will treat it purely as data and not try to execute it, it would be extremely dangerous. Anybody on the internet could craft some malicious Lua code and send it to your server to be executed.

 

Lua is sandboxed. It's an important point you make though: Choose carefully the Lua functions that your network packets can execute; don't give network packets generic functions with filesystem access, for starters (No DeleteFile() or CreateFile() or anything like that).

 

Aside from that, I don't see a problem with sending Lua scripts - but I'm no security expert. As for the 'text is larger than binary' - this is absolutely true. HTML is text when it's sent over the network - but it's not sent 10-30 times a second like a real-time competitive FPS or MMO might require, just three or four times every few minutes - and the delay in receiving and processing the HTML and CSS and such is visible (just hit 'refresh' on your browser - it's not instantaneous). So, like everything, the pros and cons need to be weighed depending on the project requirements.

0

Share this post


Link to post
Share on other sites

The more generalized your serialization/deserialization is, the worse its security posture is. The worst being of course, deserialization into arbitrary code (ala JSONP or YAML). The recent issues in the Rails community should highlight that.

 

To be fair, the Ruby/Rails people were doing some REALLY STUPID things with their data, as there is absolutely no good reason that any parser, be it JSON, YAML, or anything else, should be attempting to deserialise and construct arbitrary objects looked up by name. The formats do not inherently support this feature - it was an incredibly stupid addition that was added by web developers to make their life simpler, for which they (and thousands of others) are now paying the price. There's nothing inherently unsafe to YAML or JSON parsing. You do have to understand your parser though (because evidently the people writing them can't be trusted to keep them safe).

Lua is an interesting case as it is obviously executable code. Yes, it can be sandboxed, but at some point you need to pull the data from the sandbox into the rest of the app, which is the risky part. So I'd just prefer to go with a plain data format like JSON. Lua is fine for trusted data.

0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0