brief history (approximately of the past 14 years):
at one point, I wrote a Scheme interpreter, and it naturally uses S-Expressions.
later on, this project partly imploded (at the time, the code became mostly unmaintainable, and Scheme fell a bit short in a few areas).
by its later stages, it had migrated to a form of modified S-Expressions, where essentially:
macros were expanded; built-in operations used operation-numbers rather than symbols; lexical variables were replaced with variable-indices; ...
there was also a backend which would spit out Scheme code compiled to globs of C.
Quite the array of language projects you have there!
I too am fond of the use of S-expressions over that of XML, and have had experience using them for data and DSLs in a number of projects. You can't beat the terseness and expressive power, and it's not hard to roll your own parser to handle them.
I share many of the opinions from: http://c2.com/cgi/wiki?XmlIsaPoorCopyOfEssExpressions
As for my own projects, I've also built a custom R6RS parser in C++, and have done some interesting things with it. For specifying data as maps/sets/vectors, I added support for handling special forms which yield new data-structure semantics, added Closure-like syntactic sugar to the lexer/parser where braces and square brackets can be used to define such data structures, and added a quick tree-rewriting pass to the data compiler to convert from the internal list AST node representation to the appropriate container type.
For simple data, sometimes I just go with simple key-value text files if I can get away with it (less is more! strtok_r does the job good enough), and I've recently been experimenting with using parsing expression grammar generators to quickly create parser combinators for custom DSLs that generate more complex data or code as s-expressions or C++.
A shame that many of the "big iron" game studios still use XML for a lot of things, although I've managed to convince a number people that it's time to move on. I dread the days where I am tasked with working on anything touching the stuff.
In short, if you're still using XML, you're needlessly wading through an endless swamp of pain, suffering and obtuse complexity. Things can be better.
I was working at the time with R5RS.
by the time R6RS came out, I had mostly stopped using Scheme, and looking at it briefly, it looked like a bit of a jump from what R5RS was.
the AST format later used for BGBScript was based partly on R5RS, but differs in a lot of ways, namely in those which make it a better fit for an HLL with a more C/JS/AS3/... like syntax, like different special-forms for defining things, ones representing control-flow constructs (for/while/switch/...), ...
also, generally it moved to the use of explicit special-forms for things like function calls and using operators, ...
some elements of Scheme also were worked into the HLL design as well (tail-calls / tail-position, implicit return values, lists, ...).
early on, both Self and Erlang were also influences for the language design.
later on, Java, C#, and AS3 became influences.
basically, while it started out dynamic and prototype based, static-types, classes, packages, ... were later glued on, partly for performance reasons, and also because they are more effective for a lot of use-cases (can do stronger compile-time checking, ...).
though, the language still retains most of its dynamic funkiness (including a Self-derived scoping model, scoping semantics are fun in my language...). not going to try to explain the type-system and scoping model here though.
for parsers, I have most often used hand-written recursive-descent.
I started out with RD, and pretty much every non-trivial syntax I have encountered seems to work fine with RD.
XML and S-Expressions both have some use-cases.
granted, my XML APIs have since diverged somewhat from DOM, becoming generally a lot more operation-centric, and much less about treating XML nodes as objects (and generally, the "Document" metaphor is all but absent in-use). basically, the API focuses a lot more on composition and decomposition of data, rather than on node manipulation. ironically, it isn't used much at all with external tools (typically about the only time most of this is actually seen is in debugging dumps).
theoretically, it could also matter if/when I needed to interact with other things which use XML, or if by some off-chance I decide to use XML-RPC again (currently unlikely...).
granted, from an ease-of-use perspective, lists are hard to beat, as they are generally a lot easier to work with with a lot less code.
granted, my approach to this (C-side) has been to build a big chunk of Lisp-like APIs in C (basically, a bunch of Lisp and CLOS-like stuff glued onto C).
granted, it took several iterations before really settling on a usable set of tradeoffs (getting something that is both usable and performs well).
a lot of the infrastructure is shared between my script-language and C parts of the project.
I had considered (binary) XML for my network protocol, but ended opting instead with lists.
basically, my network protocol consists of basically large nested list structures, generally passed along to/from specific "targets" (such as between client-side and server-side versions of an entity, ...). initial versions had used Deflated textual serializations, but I later implemented a direct entropy-coded binary serialization.
this protocol is also used for my voxel-terrain, though it is sort of a hybrid (generally, the actual voxel-chunk data is passed using large byte arrays, with the chunk-data being flattened out and RLE compressed). partly this is because passing every voxel as a list-based message would be a bit of a stretch...
(chunk-delta (origin -240 416 48) (size 16 16 16) ... (voxeldata (voxel :type dirt :aux 0 :slight 240 :vlight 0 ...) (voxel :type dirt ...) ...))
it is basically a problem of 16x16x16 * 32*32*8 * 4 * ... which would take some fairly absurd numbers of cons-cells...
so, passing the chunk data in a byte-serialized format seemed like a "reasonable" compromise here.
so, instead it is something more like:
(wdelta
...
(voxdelta ...
(rgndelta ... #Ah( ... ))
(rgndetla ...)
...)
...)
where wdeta=world-delta, voxdelta=voxel-delta, rgndelta=region-delta, and #Ah(...) is a 1D byte array.
