Need scripting advice for my project

This topic is 4897 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

I'm currently working on a game called Hero of Allacrost and I'd like some outside opinions on our scripting methods. I've never done game scripting before, but I have written some small programs in Python and Perl. The game is a 2D RPG and is written in C++ and SDL. Currently, the plan is to use XML configuration files for data storage like character attributes, etc. Additionally we are thinking of using Lua for scripting events like when the game takes control of the character(s). What do you guys think about that? Oh, also we are making the game cross-platform (Linux, Windows, Mac) so we are trying to write the code as platform-independent as possible. Thank you.

Share on other sites
Sounds good to me. That's how I do a lot of my scripting, with the XML/Script Language combination. You can even customise the Xml by specifying which function is called for a given event. For example, two characters may wish to have different AI scripts...

 // XML  <character name="Gandalf" think="Gandalf_think" class="mage" />  <character name="Thor" think="Thor_think" class="warrior" /> // Script file function Gandalf_think() {   me.examineArea();   if ( me.mana > 100 )   {     me.castSpell( "somecoolspell" );   } } function Thor_think() {   me.examineArea();   if ( me.health > 10 )   {     me.say("raaaa");     me.attackEverythingInSight();   } }

Share on other sites
Personally I'd just use the script file for those XML-based attributes too.

Share on other sites
I would also be inclined to agree, not knowing the complexity of your level file.

Share on other sites
Hmm, well the only scripting I can imagine us doing for now are things like "move the character 4 tiles left on the map, then stop and say something". The reason we are using both is because we started out with XML at another person's advice and we've already gotten a little work done with it. XML is currently being strictly used for storing class attributes, like weapons, armor, map data, etc. There is still enough time to turn around and do configuration files and scripting in 100% lua. Would this be a wise step or does XML provide something that lua does not? Thanks you guys.

Share on other sites
Quote:
 Original post by RootsWould this be a wise step or does XML provide something that lua does not?

Structure.

I'd much rather see my data in nice, cohesive storage container files (such as XML) than have to create everything in script. For starters you can easily load the Xml into your level editor, something which would be difficult to do as easily with just pure script. I prefer Xml for Data and script for logic, but both sides have their merits. I'd say stick with what you have planned, but others will present valid arguments to the contrary which will have as much merit as my method. What it comes down to is personal choice IMHO.

Share on other sites
I will put it this way: if you represent your data in XML, you have to write some code to translate from XML to your internal data structures. Whether that's reading a DOM tree or setting up SAX handlers or writing your own parser.

On the other hand, if you represent your data in Lua, you get to re-use your existing Lua integration. If you're gonna be scripting with Lua then Lua will have access to Gandalf and Thor. So why not let Lua handle the initialisation rather than duplicating your effort to let your XML parser access it too?

There's not much difference between these two:
<character name="Gandalf" think="Gandalf_think" class="mage" />
CreateCharacter("Gandalf", "mage", Gandalf_think)

Or you could use a global character table; there are several ways you could approach it in Lua.

If you weren't using Lua, I'd say stick with XML. But since you do plan on using the scripting language, my personal recommendation would be to kill 2 birds with the 1 stone.

Share on other sites
Quote:
 does XML provide something that lua does not?

Yes, you get to write all tags twice.

Quote:
 Structure.

Lua has structure, and is actually designed to be read and written. Example. That you also get a very powerful programming language for free is that much better.

There are so many things one could do with Lua scripts that XML can't dream of.

Share on other sites
I recommened Simkin: http://www.simkin.co.uk/

It's a scripting language integrated inside of XML. I think it would suit your project very well since you already plan to use XML for your data storage.

Share on other sites
Quote:
 Original post by Kylotan If you're gonna be scripting with Lua then Lua will have access to Gandalf and Thor. So why not let Lua handle the initialisation rather than duplicating your effort to let your XML parser access it too?

A valid point, however I doubt people would create their entire game's level content in Lua, for that you'd need a data container such as a custom file format or Xml. The benefit of Xml is that parsing it is trivial with the plethora of good libraries out there.

Quote:
 There's not much difference between these two:CreateCharacter("Gandalf", "mage", Gandalf_think)

The difference here is that you would need to code up a custom extraction routine to integrate such content in your game/level editor. With Xml you can serialise your internal game structures quite easily, something that would be much more difficult in script alone - especially when your game content doesn't revolve around just scripting.

Quote:
 Original post by DiodorYes, you get to write all tags twice.

That often seems to be the typical anti-Xml argument, that all tags need entering twice. That's only true if the tag is a container element for other elements. Even then, the amount of time you'd be hard coding the xml will be minimal if you create a tool to edit your content. Xml has the benefit of being hand tweakable too.

Quote:
 There are so many things one could do with Lua scripts that XML can't dream of.

Xml doesn't pretend to be a scripting language and hence shouldn't have the capability of Lua - unless of course you consider the hideous xml-script hybrid that is "Water".

As ever, I'm inclined to argue the case for using both as I believe they can both be used together sucessfully, however as other have pointed out there are solutions availbale using one or the other, I guess it's up to your development team and game as to which one of the solutions you take.

Share on other sites
Quote:
 Original post by evolutionalThat often seems to be the typical anti-Xml argument, that all tags need entering twice. That's only true if the tag is a container element for other elements. Even then, the amount of time you'd be hard coding the xml will be minimal if you create a tool to edit your content. Xml has the benefit of being hand tweakable too.

Let's leave the tools out - making them costs time and makes the data format unflexible. Not having to use tools other than a text editor is a strong point of data description languages, compared to binary formats.

For instance, let's compare how one would describe a set in lua and in XML:
--luaset={1,2,5,12,"foobar",20};XML<set><element>1</element><element>2</element><element>5</element><element>12</element><element>foobar</element><element>20</element></set>

Quote:
 Xml doesn't pretend to be a scripting language and hence shouldn't have the capability of Lua

XML in facts pretends to be nothing useful at all. It doesn't even has separate types for numerical values and strings. To get to do anything at all with XML you have to either write extra code or start shopping for tools and extensions, from DTD to Schema to XSLT to Xwhatever (== work).

Quote:
 As ever, I'm inclined to argue the case for using both as I believe they can both be used together sucessfully, however as other have pointed out there are solutions availbale using one or the other, I guess it's up to your development team and game as to which one of the solutions you take.

My point is that a scripting language and a data description languages are both useful in a program, but lua _is_ both to begin with. It's better than xml at plain data description and it's that much better because one can mix data description and code in various ways.

Share on other sites
Wow, it seems I've sparked quite the debate. There have been so many good arguments made for both sides that I honestly can't say that I'm closer to a decision than I was before. But I can say that I feel much more comfortable with the knowledge that both parties can be used effectively. I feel like a kid in a ice cream store trying to choose between vanilla and chocolate, lol. Thanks for all the responses and constructive arguments everyone.

Share on other sites
Quote:
 Original post by evolutionalA valid point, however I doubt people would create their entire game's level content in Lua, for that you'd need a data container such as a custom file format or Xml. The benefit of Xml is that parsing it is trivial with the plethora of good libraries out there.

Of course, but that's not a benefit over Lua (if already embedded) because Lua not only provides the parser but the way to get that data into internal structures. Assuming the internal game data is going to be reflected into Lua for scripting purposes anyway, you've essentially got all your parsing and assigning pre-written. That wouldn't be the case with any XML parser in C or C++, where you're going to need to write functions to map entities to objects and attributes to member variables purely for loading and saving purposes, when all this has already been done for the Lua interface.

If Lua wasn't already part of the system I would consider XML to be an equally attractive option.

Share on other sites

Quote:
 Original post by DiodorLet's leave the tools out - making them costs time and makes the data format unflexible.

Quote:
 Original post by KylotanAssuming the internal game data is going to be reflected into Lua for scripting purposes anyway, you've essentially got all your parsing and assigning pre-written.

Using the example cited by Diodor, in a purely Lua vs Xml contest it's obvious that Lua would win as it's more than Xml claims to be. Lua is a runtime scripting language as well as a data description language, Xml is a meta language for describing data. However, what sort of development environment wouldn't use external tools? When you start to bring in such tools as level editors and modellers you begin to open up more and more uses for Xml for describing data, especially when the external tools need to perform trasnformations on that data or parse it when a Lua environment doesn't exist.

Quote:
 Original post by RootsBut I can say that I feel much more comfortable with the knowledge that both parties can be used effectively. I feel like a kid in a ice cream store trying to choose between vanilla and chocolate

To me this comment sums up the whole debate.

I have never once argued that you should not use Lua, nor said that you should use just Xml. I've argued that both systems have their merits and it's possible to use them both in combination and I still hold to that fact.

Xml and Lua are what you make of them and what you make of them is entirely down to personal choice.

[smile]

Share on other sites
the nice thing about Lua is that it can be compiled to byte code. so it *should* be faster than XML. I don't know for sure. I keep my scripts in their ASCII form. Additionally, you can use Lua as a config file. If you want, i could give ya the code that uses the LUA VM to parse the lua script directly.

So something like this:

Engine={
UseHWSkinning=true,
UseHWVP=true
}

you could get these attributes from the script fairly easy.

Btw, someone suggested Simkin scripting language above. Well, i;ve had the misfortune to work on a game that used Simkin. it didn't use the XML component however- it just used it as a script language. Simkin is the worst script language I have ever seen. it crashes pretty often and is really really incomplete. For instance if a designer wants to store a reference to an object in his script to a treenode- they can;t do this unless an engineer adds this capability to it.

Once again i want to emphasize we didn't use the XML component of simkin though. My advice to you is that whatever you pick, make sure it's something that's proven and has been around. That might help reduce risk.

Share on other sites
Quote:
 Original post by evolutional However, what sort of development environment wouldn't use external tools? When you start to bring in such tools as level editors and modellers you begin to open up more and more uses for Xml for describing data, especially when the external tools need to perform trasnformations on that data or parse it when a Lua environment doesn't exist.

When you say tools, do you mean tools written by the developer or general use tools available for XML-based solutions?

If it's the former, I stand by Lua - Lua scripts can easily parse, transform and serialize data. Not knowing much about the XML way, some examples of things XML could do better than Lua may further the debate.

EDIT: [smile]

[Edited by - Diodor on September 20, 2004 1:32:43 AM]

Share on other sites
From what I've heard open source XML parsers are "meh", but I have no idea.

Share on other sites
Quote:
 Original post by RootsFrom what I've heard open source XML parsers are "meh", but I have no idea.

TinyXML is my choice of Xml parser. Small, lightweight, stable and pretty quick. It doesn't have any of the extra stuff such as Validation, XSLT, XPath or XQuery though, as it's mainly just a quick parser for grabbing Xml data into your program. If you need other features, I believe that libXml2 has them (or at least ways of adding them).

C# an .NET languages also have access to Microsoft's optimised Xml classes, so if you're using a .NET language you have support already built in.

Quote:
 Original post by DiodorNot knowing much about the XML way, some examples of things XML could do better than Lua may further the debate.

I'm not really one for 'vs' debates; but Xml and Xml technologies are foundation to technologies such as SOAP and .NET web services, they can also be used in many back-office scenarios from data exchange (think legacy EDI systems) and data querying. Xml also powers many websites (including this one [wink]) because of it's flexibility.

Being a web systems programmer and data analyst it's hard for me to ignore Xml in all of it's forms, it is a logical conclusion that it can be used in games - and it is, my games use it extensively as do several hobby games I've seen (don't ask me about commerical games as I seldom play them). A lot of my work/research lately is about bolstering games with scripting technologies and Xml, granted I'm not using Lua in this. Lua could probably do all that and more given the chance, I'm not saying it couldn't - I'm just saying that there is another way which is what I was trying to show the OP.

Lua and Xml are what you make of them, that's the crux in this matter. It's not what you use, it's how you use it.

Share on other sites
Quote:
 Original post by RootsFrom what I've heard open source XML parsers are "meh", but I have no idea.

Since I've been the one inside the project pushing lua, I'll respond by saying that the open source xml parsers are flakey and generally slow. I can back this up with empircal data, but I'm really lazy at the moment.

The big advantage to Lua is that if we were to define data, that data would not necessarily be static. Wherein some xml data file you'd have to hand edit a bunch of crap to make a big system wide change, you can dynamically do things in lua, do calculations at runtime, etc. It allows the application to be extendable in an insane number of ways.

Also, lua allows things like dictionaries and other data structures to be created to store the data - data structures which are much harder to manage and utilize properly when written strictly in c.

This in turn allows the data to be passed to c in a single simplified way with an underlying structure which could be quite advanced. Lua simplifies this all.

Unlike using xml, lua also allows things like great regular expression searching and such. I know there are c libraries, but this just adds to the complexity of the application. Lua will take care of all of this and more, while allowing true extendability within the application itself.

On top of that, I'm one of the developers on the project and I know lua pretty damned well :P. So there's that advantage to using lua.

Share on other sites
Quote:
Original post by Diodor
Quote:
 Original post by evolutionalThat often seems to be the typical anti-Xml argument, that all tags need entering twice. That's only true if the tag is a container element for other elements. Even then, the amount of time you'd be hard coding the xml will be minimal if you create a tool to edit your content. Xml has the benefit of being hand tweakable too.

Let's leave the tools out - making them costs time and makes the data format unflexible. Not having to use tools other than a text editor is a strong point of data description languages, compared to binary formats.

For instance, let's compare how one would describe a set in lua and in XML:
*** Source Snippet Removed ***

Quote:
 Xml doesn't pretend to be a scripting language and hence shouldn't have the capability of Lua

XML in facts pretends to be nothing useful at all. It doesn't even has separate types for numerical values and strings. To get to do anything at all with XML you have to either write extra code or start shopping for tools and extensions, from DTD to Schema to XSLT to Xwhatever (== work).

Quote:
 As ever, I'm inclined to argue the case for using both as I believe they can both be used together sucessfully, however as other have pointed out there are solutions availbale using one or the other, I guess it's up to your development team and game as to which one of the solutions you take.

My point is that a scripting language and a data description languages are both useful in a program, but lua _is_ both to begin with. It's better than xml at plain data description and it's that much better because one can mix data description and code in various ways.

exactly. with lua, you are able to add logic in with the data rather then just having static data. people could add/change functionality in the game to suit their own tastes (as seen with homeworld 2) via lua. there are a TON of examples of lua being a successful scripting/data description language in gaming.

Monkey Island, Baldurs Gate (which has a lot lot lot of weapon/monster/etc type of data, similar to the data we'll be dealing with), Homeworld 2, Mythica (which was cancelled, not due to quality or because of lua), and many many others.

Not many to my knowledge embrace xml.

Share on other sites
Although I generally agree with the use of Lua in this case, I would defend TinyXML as I have always found that to be a solid library for what it does and not really 'flaky'. For projects without embedded scripting languages, TinyXML is a great way to read and write structured configuration files with minimal effort. XML is not entirely without support; Age Of Mythology is one game that relies heavily on XML files, making it easy for mod developers.

Share on other sites
Quote:
 Original post by KylotanAlthough I generally agree with the use of Lua in this case, I would defend TinyXML as I have always found that to be a solid library for what it does and not really 'flaky'. For projects without embedded scripting languages, TinyXML is a great way to read and write structured configuration files with minimal effort. XML is not entirely without support; Age Of Mythology is one game that relies heavily on XML files, making it easy for mod developers.

I honestly haven't tried TinyXML. I've used libXML2 though, and from my experience with it, it was a slow parser and quite flakey at times. If I have the need for XML I'll definitely give TinyXML a try. As of yet though, I haven't found anything (at least for gaming) where XML gives me a pure advantage over something else. However, for structured online content XML is kinda hard to beat :).

Share on other sites
Quote:
 Original post by thesadjesterI've used libXML2 though, and from my experience with it, it was a slow parser and quite flakey at times. If I have the need for XML I'll definitely give TinyXML a try :).

I have use libXML2 quite extensively, and IMHO it is plenty fast for data loading, and has never fallen down, except when my code is at fault. However, the flexibility with which it has been defined makes it very easy to write bad code, and many users never really seem to learn the fast and simple ways to parse xml data.

I have to admit, that while I use xml for most of my games, it does not make a good final runtime solution, mainly because the extra disk space used by all those tags, etc. slows down file access and parsing. I noticed the Ogre engine has gotten around this by using xml files during development, and then using a tool to convert them to binary files for shipping with the finished product. This seems a bit of a hack, and I would advocate using binary files and a simple editor app throughout the process, but this, of course, doesn't allow text editing, although it is very fast.

Share on other sites
I wanted to bring this decision on my team to a close, so I googled for "XML lua versus" and found 3 useful sources of information about the matter. Here are the three links and important excerpts I've copied from the documents.

"GDC 2004 - Lua in the Gaming Industry Roundtable Report"
http://www.gdconf.com/archives/2004/burns_jon.doc

"A question was posed regarding using Lua or XML for data definition. The general consensus was that Lua is much more compact and readable than XML."

"XML is good for describing small sets of data and web-related data, and it was suggested that a team could use XML as a meta-language to express game data in binary, which would be good for cross-platform development."

"Most agreed it is much easier to find XML-savvy editors than Lua editors."

"Most developers heavily experienced with Lua said Lua is a language that developers can learn in about 10 minutes and that non-programmers can pick up at a fairly quick pace."

Scripting Programs with Lua - Gallaudet Research Institute
http://gri.gallaudet.edu/~cvogler/ research/data/yhslug021304.pdf

"Some projects that use Lua: Baldur's Gate, Escape from Monkey Island, MDK, Grim Fandango. Mostly used to control AI and behavior"

"Lua for data description

Lua is ideally suited to describe data and configuration files.
XML is all the rage, but Lua has three advantages over it:
- Much easier to use than an XML parser
- Much smaller than the monstrous XML libraries out there
- The third one will become apparent in this talk"

"Processing Sequence Annotation Data Using the Lua Programming Language"
http://www.jsbi.org/journal/GIW03/GIW03F016.pdf

"To avoid having to program in 2 languages simultaneously, the ”document object model” method allows a program to access XML data as a structured variable on-memory. This is the same as our approach with the data warehouse."

"In addition, while XML usually describes a well-formatted static data schema with rigid tags, an active research project sometimes requires modifying and adding new tags on demand. Although XML can provide consensus annotation data file formats, the use of such tags would yield conflicting differentiations from the original XML schema."

I can say that after doing this research, I am officially pro-Lua. I would like to see any documents that can find the inherit advantages of XML over Lua (especially in gaming applications), but I have not found any such material.

Share on other sites
I was thinking about going witu xml and javascript my self, however, lua has been proven many times like was said earlier, it can be used for all sorts of things, its syntex is easy to learn and theres tools for binding it with C++.

Hehe, but the coolest thing about lua that most of you guys may not know, is that it was used to manage the oxygen mixtures in one of the space shuttles! If Nasa can use it, I think I can ;)

All together though,IMO: with any project, you should use what best fits your needs and not use something just because its the trendy thing to do.