• FEATURED

View more

View more

### Image of the Day Submit

IOTD | Top Screenshots

## Best Language for Cross-Platform Content Pipeline

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

19 replies to this topic

### #1IncidentRay  Members

Posted 07 January 2013 - 02:29 PM

I'd like to make a content pipeline for some of my games/engines/etc I'm working on, and I'm not quite sure what would be the best language to use.  I'm using C++ for the actual runtime, but this seems very unproductive for tools.  I've seen C# recommended several times for Windows-only content pipelines, but I'd like to know if there's anything more cross-platform.

Here are some features that an ideal language would provide:

• It also should provide some abstraction of OS-specific functionality such as directory iteration, etc.  It should also be easy to use high-level features such as regular expressions and other text-manipulation techniques.
• Integrating this language with Lua should be simple, as I want to use Lua as a data-description language for several resource types.
• Calling external processes shouldn't be too hard -- for example, I might want to use the Cg toolkit's cgc program to compile shaders.

I suppose C++ could work if I used heaps of boost libraries -- this does seem a bit cumbersome though, and I imagine it would cause a large hit on compile times.

Anyone have any suggestions for a good language to use?

Edited by IncidentRay, 07 January 2013 - 02:30 PM.

### #2Nypyren  Members

Posted 07 January 2013 - 02:47 PM

C# works fine cross platform.

### #3Dan Mayor  Members

Posted 07 January 2013 - 03:00 PM

C# requires very heavy weight dependencies, .NET on Windows, and Mono on everything else.  You are looking at increasing your install process upwards of a gigabyte or more.  Although it is possible to use C# on non windows systems I wouldn't really call it a practical solution.  There is always Java but my personal opinion is that this can cause some slight lag in the loading process (traditionally Java performs a bit slower than C++ due to the interpreter that must intercept and parse the byte code).  If you are using lightweight graphics and don't mind adding a few extra seconds to your loading screens I would recommend java.  If performance is critical your a little stuck with C++.

Dan Mayor

Professional Programmer & Hobbyist Game Developer

Seeking team for indie development opportunities, see my classifieds post

### #4uglybdavis  Members

Posted 07 January 2013 - 03:39 PM

If you're just making offline tools and load time doesn't matter use Java or C#. If youre going to be using it in game, suck it up and C++ your way trough it.

### #5ApochPiQ  Moderators

Posted 07 January 2013 - 03:47 PM

Out of raw curiosity, why do you need cross-platform tools?

Wielder of the Sacred Wands

### #6Hodgman  Moderators

Posted 07 January 2013 - 04:00 PM

I use C# for that -- the huge standard library does all your directory/text/regex stuff, LuaInterface makes integrating with Lua amazingly simple (I use Lua to describe a lot of my input data), and calling external compilers, etc is easy.

I don't need it to be x-platform, as a lot of our other tools are Windows-only anyway. We compile resources for all platforms from a Windows development PC.
However Mono does make it portable if you need that, and the Mono/.NET runtime dependency doesnt bother me because only my devs need it, not my players/customers.

### #7IncidentRay  Members

Posted 07 January 2013 - 04:53 PM

Thanks for the responses!

If you're just making offline tools and load time doesn't matter use Java or C#. If youre going to be using it in game, suck it up and C++ your way trough it.

Yes, I'm just making offline tools, so as you say, load time isn't too important.  I don't like Java much, but I'm interested in the possibility of using C# as a cross-platform language.

Out of raw curiosity, why do you need cross-platform tools?

Mostly because I don't have a PC.  Also I just think it's theoretically a bit nicer.

I use C# for that -- the huge standard library does all your directory/text/regex stuff, LuaInterface makes integrating with Lua amazingly simple (I use Lua to describe a lot of my input data), and calling external compilers, etc is easy.

Wow, that stuff looks great; I'm quite intrigued by the idea of using C# now.

However Mono does make it portable if you need that, and the Mono/.NET runtime dependency doesnt bother me because only my devs need it, not my players/customers.

Okay, I'll look into Mono.

Edited by IncidentRay, 07 January 2013 - 04:54 PM.

### #8ApochPiQ  Moderators

Posted 07 January 2013 - 05:14 PM

There's still a major difference between "I need this to work on {some platform besides Windows}" and "I need this to be cross-platform."

I assume you mean you run a Mac, or maybe even a Linux machine; in which case C# is still totally viable if you target Mono from day one. The only real limiter to using C# in a cross-platform setting is quirks between .NET and Mono, and those are rapidly diminishing these days anyhow.

Wielder of the Sacred Wands

### #9IncidentRay  Members

Posted 07 January 2013 - 05:37 PM

Yes, what I meant is that I use a Mac.  I'm not sure what you mean by your first sentence; are you saying that I could use a Mac-specific programming language?

I had heard of Mono, but I wasn't sure how practical it was, so thanks for the input.  From what you've said, using C# with Mono sounds like a good choice then.

EDIT: My line of reasoning for wanting to make it cross-platform was that most people seem to make their tools Windows-only, so if I need to support another platform, it's probably better to support Windows as well.

Edited by IncidentRay, 07 January 2013 - 05:47 PM.

### #10ApochPiQ  Moderators

Posted 07 January 2013 - 07:31 PM

Whether or not you "need" to support any given platform depends entirely on who is using your tools.

If you're making tools for you, make them for you. At the risk of being overly blunt, don't waste your time on people who will never touch your software. What everyone else does with their tools is only a relevant thing to care about if you have the exact same use cases and needs as "Everyone Else" - which you probably don't.

Do what makes sense for your particular situation. If you're writing tools to produce games on a Mac, why not use the existing (and fairly robust and rich) Objective-C ecosystem?

Wielder of the Sacred Wands

### #11IncidentRay  Members

Posted 07 January 2013 - 09:51 PM

Yes, I am writing this only for me, but as this is also intended as a learning exercise, I would prefer if I could write something cross-platform.  As to the suggestion of using Objective-C, I find Objective-C overly verbose and hard to read.  I agree that it might be the best choice if I need something with a complicated GUI, but I don't, so I don't think it's worth it.

### #12ApochPiQ  Moderators

Posted 08 January 2013 - 01:24 AM

Let me be even more frank, then:

Trying to learn to write cross-platform software when you only run one platform is kind of like trying to learn to drive a car by bouncing around your yard on a pogo stick. Entertaining, sure, but ultimately you're just wasting a lot of time.
Wielder of the Sacred Wands

### #13Karsten_  Members

Posted 08 January 2013 - 04:03 AM

I would probably suggest using the same language as whatever the majority of libraries you use are written in.
Also, since my games are generally in C++, I tend to just use that so I can reuse code between the two if I need to. (i.e If I decide to move some things from the pipeline into being loaded by the game at runtime).

If your games are written in Microsoft C# (Which I think is the norm in hobby development) then it would be hard to suggest any other language. (Especially since C# has very poor support for using libraries from other languages (Such as Java libs, python etc...) so you are kinda stuck with it anyway.

Edited by Karsten_, 08 January 2013 - 04:04 AM.

http://tinyurl.com/shewonyay - Thanks so much for those who voted on my GF's Competition Cosplay Entry for Cosplayzine. She won! I owe you all beers

Mutiny - Open-source C++ Unity re-implementation.
Defile of Eden 2 - FreeBSD and OpenBSD binaries of our latest game.

### #14dougbinks  Members

Posted 08 January 2013 - 07:33 AM

One fully cross platform way to do tools is via using a browser for the UI, see http://www.insomniacgames.com/new-generation-of-insomniacgames-tools-as-webapp/. Wolfire also do this using embedded Webkit so you get the editor in the game engine.

Although I've not tried this method myself, it seems a valid and fairly future proof approach.

### #15IncidentRay  Members

Posted 08 January 2013 - 10:01 PM

I would probably suggest using the same language as whatever the majority of libraries you use are written in.
Also, since my games are generally in C++, I tend to just use that so I can reuse code between the two if I need to. (i.e If I decide to move some things from the pipeline into being loaded by the game at runtime).

If your games are written in Microsoft C# (Which I think is the norm in hobby development) then it would be hard to suggest any other language. (Especially since C# has very poor support for using libraries from other languages (Such as Java libs, python etc...) so you are kinda stuck with it anyway.

Yeah, availability of useful libraries is definitely an important factor -- that's why I was mentioning text manipulation and Lua integration in the original post.

I actually am writing the game itself in C++, but in this case I think reusing code between the engine and tools isn't too important. These are mostly data-conversion tools, which will write files in a format known to the engine, so the tools and the engine don't need to interact directly.  The idea you mentioned of being able to move some asset conditioning steps from the pipeline to the game could be useful, but for the types of files I am converting, this would probably not be too practical.  For example, I would prefer if the game didn't have a dependency on the Cg framework, so this means the shaders will have to be compiled offline.  So what I'm saying is that in my case I think it's okay if the tools are written in a different language to the game.

One fully cross platform way to do tools is via using a browser for the UI, see http://www.insomniacgames.com/new-generation-of-insomniacgames-tools-as-webapp/. Wolfire also do this using embedded Webkit so you get the editor in the game engine.

Although I've not tried this method myself, it seems a valid and fairly future proof approach.

Hmm, that looks like a reasonable idea; I don't know too much about web development though, and getting the interaction right between the UI and the back-end looks tricky.

### #16Flimflam  Members

Posted 09 January 2013 - 10:45 PM

C# requires very heavy weight dependencies, .NET on Windows, and Mono on everything else.  You are looking at increasing your install process upwards of a gigabyte or more.  Although it is possible to use C# on non windows systems I wouldn't really call it a practical solution.  There is always Java but my personal opinion is that this can cause some slight lag in the loading process (traditionally Java performs a bit slower than C++ due to the interpreter that must intercept and parse the byte code).  If you are using lightweight graphics and don't mind adding a few extra seconds to your loading screens I would recommend java.  If performance is critical your a little stuck with C++.

I'm curious under what circumstances you would ever need even close to 1GB or more space using C# under any OS? The Client Profile for 4.0 which is all you need to do most projects (unless you need some really obscure .NET features--unlikely), is a 30MB download in its entirety, and mono can be even smaller.

C# won't require a bigger set of dependencies than Java would. It's more than a practical solution these days.

Edited by Flimflam, 09 January 2013 - 10:48 PM.

### #17pinebanana  Members

Posted 09 January 2013 - 11:47 PM

C# requires very heavy weight dependencies, .NET on Windows, and Mono on everything else.  You are looking at increasing your install process upwards of a gigabyte or more.  Although it is possible to use C# on non windows systems I wouldn't really call it a practical solution.  There is always Java but my personal opinion is that this can cause some slight lag in the loading process (traditionally Java performs a bit slower than C++ due to the interpreter that must intercept and parse the byte code).  If you are using lightweight graphics and don't mind adding a few extra seconds to your loading screens I would recommend java.  If performance is critical your a little stuck with C++.

I'm curious under what circumstances you would ever need even close to 1GB or more space using C# under any OS? The Client Profile for 4.0 which is all you need to do most projects (unless you need some really obscure .NET features--unlikely), is a 30MB download in its entirety, and mono can be even smaller.

C# won't require a bigger set of dependencies than Java would. It's more than a practical solution these days.

.NET 30mb? Are you mad? The .NET framework is huge.

If you're referring to this:

The installer may be "30mb" (actually 48.1mb, it's the online installer I think), but If you look at the system-requirements, the x86 version requires 850mb and the x64 version requires 2GB.

anax - An open source C++ entity system

### #18Mike.Popoloski  Members

Posted 13 January 2013 - 11:52 AM

C# requires very heavy weight dependencies, .NET on Windows, and Mono on everything else.  You are looking at increasing your install process upwards of a gigabyte or more.  Although it is possible to use C# on non windows systems I wouldn't really call it a practical solution.  There is always Java but my personal opinion is that this can cause some slight lag in the loading process (traditionally Java performs a bit slower than C++ due to the interpreter that must intercept and parse the byte code).  If you are using lightweight graphics and don't mind adding a few extra seconds to your loading screens I would recommend java.  If performance is critical your a little stuck with C++.

I'm curious under what circumstances you would ever need even close to 1GB or more space using C# under any OS? The Client Profile for 4.0 which is all you need to do most projects (unless you need some really obscure .NET features--unlikely), is a 30MB download in its entirety, and mono can be even smaller.

C# won't require a bigger set of dependencies than Java would. It's more than a practical solution these days.

.NET 30mb? Are you mad? The .NET framework is huge.

If you're referring to this:

The installer may be "30mb" (actually 48.1mb, it's the online installer I think), but If you look at the system-requirements, the x86 version requires 850mb and the x64 version requires 2GB.

False. That link points to the standalone offline installer. If you wanted to bundle the .NET framework with your app, it'd take only an additional 48 MB. In fact, that link includes x86 and x64 components. The one that's x86 only is slightly above 30 MB.

The system requirements are just what they want you to have available during the install process, probably so there's sufficient space for unpacking, decompressing, coping things around, etc. Unless of course you think they managed to get a 0.000000024 lossless compression ratio on their download for the x64 version (48 MB -> unpacks to 2 GB of data?!?)

Mike Popoloski | Journal | SlimDX

### #19Dan Mayor  Members

Posted 13 January 2013 - 08:35 PM

Actually, and I'm sorry that this one will sound as if I am arguing my validity but I did need to point out that even those contradicting me about the dependency weight are linking and proving my comments.  I probably should have been more clear and mentioned that POST install can be 1GB or more and even with that I was partially wrong.  As pointed out by pinebanana after installing .net for x86 you will require 850mb of disk space.  That's a whole hell of a lot of space just to get a tool to work especially if you need that just to launch the window.

64bit installs can take up to 2GB as pointed out, however it's unlikely you will use or need 64bit functionality in an editing tool.  Long story short you have to have your user's have or install the .NET libraries which can vary depending on which install you use but in most cases range in the 850mb area.  For anyone who remembers this install process it's quite time consuming even on a fast system.  When all is said and done and your asking your users to install 850mb (or even upwards of 2gb!) just to use a map editor, which in actuality doesn't even use 20% of the stuff you installed...  That's my point.  When you get away from windows and you have the comparable sizes of Mono (post install) it's even less likely the end user already has it or that they would ever need it for anything besides your editor.

So java has similar sizes post install, true.  Difference?  Most if not all of your end users most likely already have it installed.  Does your team use blender?  They have java, do they use open office?  They have java.  Have they ever used anything from Oracle?  They have java...  There are some things to take into account such as which JRE is installed and what you are building against but in most cases it's safe to assume that your end user probably already has java.  On non windows platforms it's even safer to bet they have it.  This equates to less install time, less disk space getting used and overall just less garbage that your end user has to deal with just to use your one or two editors.

With that said I should note that I am a huge C# and .Net advocate, I am not attempting to talk anyone out of using C# (as long as it's for windows), however considering the question relates to cross platforming we're talking more about forcing non windows users to install things that emulate windows natural behavior in a foreign platform to use a tool.  These extra mono libraries that you are asking your user to install have nothing to do with their day to day computing and until we start seeing more and more mono ports of C# programs in the main stream they aren't likely to ever see or use it again.

Dan Mayor

Professional Programmer & Hobbyist Game Developer

Seeking team for indie development opportunities, see my classifieds post

### #20Mike.Popoloski  Members

Posted 13 January 2013 - 11:28 PM

Your whole post is basically rubbish. Let's see why.

64bit installs can take up to 2GB as pointed out, however it's unlikely you will use or need 64bit functionality in an editing tool.

If you didn't need 64-bit functionality, why in the bloody blue blazes did you install the 64-bit runtime? And if you didn't install the 64-bit runtime, why are you complaining about how large it is? Also, to say it's unlikely that you'll need it is as hilarious as it is condescending. You can't conceive of a game editor tool needing to work on large assets and large quantities of content, such that larger address space would be useful?

Long story short you have to have your user's have or install the .NET libraries which can vary depending on which install you use but in most cases range in the 850mb area.  For anyone who remembers this install process it's quite time consuming even on a fast system.  When all is said and done and your asking your users to install 850mb (or even upwards of 2gb!) just to use a map editor, which in actuality doesn't even use 20% of the stuff you installed...

On Windows, the .NET framework is installed automatically as part of the core OS, and later versions come down as automatic updates. The operating system has many components that aren't used by every single program; that doesn't make them pointless or a waste of space. The space used by a shared library is amortized by having many programs sharing the library.

When you get away from windows and you have the comparable sizes of Mono (post install) ...

False. If you had done the most cursory of searches before posting, you would see that a usable Mono install can be much smaller than a .NET or Java install.

So java has similar sizes post install, true.  Difference?  Most if not all of your end users most likely already have it installed.  Does your team use blender?  They have java, do they use open office?  They have java.  Have they ever used anything from Oracle?  They have java...  There are some things to take into account such as which JRE is installed and what you are building against but in most cases it's safe to assume that your end user probably already has java.

Do your users use newer versions of Windows? They automatically have some version of .NET. Do they use Visual Studio, widely considered to be the industry standard in development tools? They have the absolute latest in .NET. Ever heard of Unity3D? Pretty popular among indies these days, and runs on an impressive array of platforms. They use Mono for all of their scripting needs.

Java, on the other hand? I know lots of people who don't have Java installed. But that's neither here nor there; we can point out useful applications that use either framework. If we're talking about on non-Windows platforms only, then I've already shown that Mono is a much smaller dependency than the JDK. Perhaps we should start advocating that nobody use Java if a few megabytes of disk space is suddenly the most important consideration in computing.

With that said I should note that I am a huge C# and .Net advocate, I am not attempting to talk anyone out of using C# (as long as it's for windows), however considering the question relates to cross platforming we're talking more about forcing non windows users to install things that emulate windows natural behavior in a foreign platform to use a tool.

Uh, no. Mono isn't forcing people to "emulate windows natural behavior" anymore than people using C++'s boost are being forced to emulate some specific bit of behavior on platforms that don't natively support it. The whole point of a cross platform library is that it abstracts the differences between platforms so that the programmer isn't writing separate code paths. Mono is as "natural" to its target platforms as Java is; which is to say, not at all.

These extra mono libraries that you are asking your user to install have nothing to do with their day to day computing and until we start seeing more and more mono ports of C# programs in the main stream they aren't likely to ever see or use it again.

If the user of your tool is a designer working with you to develop a game, then whatever libraries your tool needs to run are *EXACTLY* relevant to what they need to do with their day to day computing. Furthermore, you don't ask your users to install random components in order to make your tool work; you install them all automatically and no end user is even going to care as long as they can get their work done. At least, that is, until the runtime you've installed starts opening up security vulnerabilities left and right

Mike Popoloski | Journal | SlimDX

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.