Third Party Library Gripes

Published June 08, 2011
Advertisement
Following on from my last post, I want to gripe about something. With a lot of third party libraries, one of the goals of people who use the library is to bind it to another language than the one it is developed in. Lots and lots of people want to do this. Consequently, it annoys me (almost as much as people who are easily annoyed annoy me) when the library is written in a fashion that seems deliberately designed to complicate binding to other languages. They utilize things in the public interface to the API that are well-supported in the native language, but not so much in the language being bound to.

If you do a Google search for " + Lua binding" you're tossing yourself onto the craps table. Odds are good that what you are looking for exists, kind of. It's six versions behind, hasn't been updated since 2005, and didn't work 100% even then. Odds are even better that it doesn't exist at all, but that there was a forum post in 2008 from some guy who said he's working on a binding and it'll be released soon, followed by seventeen replies saying "awesome! let us know when it's done". Followed by radio silence. The guy no doubt discovered that the API isn't at all well suited for binding to Lua, and silently gave it up, leaving seventeen other people in the same boat.


A couple gems I've run across in my years of using Lua:

APIs that expose naughty bits of the C++ standard library.
One culprit: SFML:

A lot of SFML is very simple to bind, which is why it baffles me when I run across such gems as the VideoMode::GetFullscreenModes() function, a function which pretty much any kind of game project that reaches finish/polish is going to use, if the game allows fullscreen modes. Consequently, it's a function that needs to be bound to the language in question. So why, oh why, would they have it return a std::vector? Now, instead of a simple binding, you have to add additional code to bind to a std::vector, plus all that entails, including binding iterators and all the other detritus of std::vector. This can be hacked, of course, but it's messy. And for no good reason.
A solution might be to populate a fixed-length array instead. Lua can deal with arrays; Lua deals with arrays very well. In GLFW, the equivalent query for video modes will populate an array passed in by the user, making it simple as pie to enumerate valid desktop modes. No munging around with binding std::vector and std::vector::iterator for the sole purpose of iterating video modes.

APIs that expose raw void pointers:
One culprit: Horde3D:

Horde3D was ostensibly designed to ease binding to other languages, and as I mentioned in the previous post, doing so is a dream. With some caveats, however. I suppose that there comes a time when there just is no elegant way to do something, and that is when the library writers take the easy way out and do something that is not easily bindable. In Horde3D, one of those places was the h3dMapResStream() function. This is a function used to map the data associated with a resource (be it a texture, a mesh, or what-have-you) to a buffer so that modifications can be performed on the resource. If you were generating meshes or textures procedurally, this is the function for you. However, the function maps the resource data to a void pointer, which Lua doesn't really like. Lua doesn't really know how to deal with void pointers. How do you tell Lua that the stream is supposed to be an array of integers as opposed to an array of doubles? To Lua, a number is a number, and it is the responsibility of the binding code to correctly cast to the proper type. Another problem with the h3dMapResStream() function is that there is no immediately available way to tell how large the mapped array is. You need to run previous queries on the resource in question to obtain size, element types, etc... and from that information you calculate the size of the array. This is dangerous, I think. It'd be really easy to do a buffer overrun in this setup.
I tend to dislike raw access to arrays via pointer at all, for the obvious reasons. I think that the API writer should avoid this at all costs, and not just for making binding easier. It's unsafe. In my own stuff, whenever I have to map something to an array, I enclose the pointer to the array in a proxy structure that includes array size, and all access to the array goes through the proxy, which will prevent overruns, whether unintentional or malicious. It's not hard to write a proxy, although it might slightly be complicated if the possible formats for the data stream are complex and variable. Nevertheless, I do not think it is the job of the guy trying to write the binding to figure this out; I think it's the responsibility of the guy writing the API to provide a clean and safe interface.
In fact, returning a pointer to an array of any sort, void or otherwise, should be avoided. Wrap it up, people.


APIs that use callbacks.
One culprit: GLFW

Callbacks. Yuck. I understand the idea, and I've used them before, but I personally do not believe they ever belong in an API that is intended for general public use, especially if many of the users are going to want to bind it to something else. A callback by its very nature is tightly coupled to the language in which it is being developed. If an API does use callbacks, I think it should provide an alternate system that is more easily bound.
GLFW is a serious violator of this. In GLFW, all input events are handled via callback. If you want to receive mouse move updates, you gotta set a callback. Want keystrokes? It's a callback. Want mouse buttons? You bet, it's a callback. And for what purpose? What benefit does a callback system provide over a traditional message queue accessed via poll/get event type commands?
When I wrote a Lua binding for GLFW I ended up writing an extension that hides the callbacks behind a message queue instead. An init function would hook up the callbacks to functions that would put together an event message structure and push it onto a global queue. It's a hack, but it sure simplifies the binding. Now instead of having to figure out how to bind a callback, I just call a GetEvent function to pop the next event off the queue. It's simple, the binding is easy, and it works better with my traditional input/update/render game-loop anyway. Even if I weren't binding to Lua, I would still wrap up the cruddy callback system in a more easily dealt with message queue.


Now, some of these problems are correctable with a little work on the binding coder's behalf. I fixed the GLFW callbacks problem with a couple of extensions, and the Horde3D problem with specialized functions tailored for the task at hand. Fixing the GetFullscreenModes issue in SFML can be done by writing a wrapper for a std::vector that can be easily exported to Lua. But all of these require stepping beyond the bounds outline by the official API docs. Your extensions, your new functions, your wrappers, none of them will exist in the official documentation, requiring your own addendum. With an "easy" binding, the API docs are fully relevant, but with these hacks you now have unsupported things and entry points for new bugs.

Of course, there is always the argument that binding directly to these libraries is non-optimal anyway, and that instead you should write a higher-level API more specifically tailored to your application. This is valid, although sometimes it would still be nice to have a direct binding to the lower level APIs for the sake of more rapid prototyping.
1 likes 5 comments

Comments

nhold
I think an API developed in C++ should take advantage of anything C++ has to offer. It isn't the job of an API to make it easy to bind things to a specific scripting language or even a scripting language (Unless that's it's goal). You can however write functions that wrap up the functions they have and bind that.

These problems you are mentioning is because different people like different programming styles and paradigms, which while I agree are problems(Albeit very small) they are unavoidable in the usage of third party libraries. If you really don't like how one works, find another, write your own or wrap it up (Which is what the majority of people do).
June 09, 2011 10:12 PM
evolutional
This is Drew posting as Oli to test out his account...
June 10, 2011 02:15 AM
JTippetts
I think my gripe may really be that I don't like inconsistency. If an API is obviously designed to "take advantage of anything C++ has to offer" then that's fine; I'll either write my own or find another, or write another abstraction layer. I guess what I really mean is this:

Take SFML again, for example. The API for SFML is extremely "generic" for the most part, and very easy to bind to Python, or Lua, or whatever. All except for 2 places. The VideoMode::GetFullscreenModes exposes an std::vector and the Packet class in the Networking module exposes operator<< and operator>>. That's it. In a library of dozens of classes and hundreds of methods that all bind extremely easily, the presence of these two little hangups is strange and inconsistent. In a library that is supposedly designed for binding it seems unusual that the inconsistencies remain, given that they complicate the binding process. The maintainer of SFML provides bindings for C, D, C# and Python, although in the release I have, the included bindings are out of date and actually incompatible with the source version in the same package; this seems to happen, I've noticed, when a library is difficult to bind.

It just seemed odd and gripe-worthy to me. YMMV.
June 10, 2011 01:35 PM
JTippetts
Oli is possessed by Drew! Ack!
June 10, 2011 01:35 PM
Gaiiden
mwaaahahaha
June 11, 2011 09:03 AM
You must log in to join the conversation.
Don't have a GameDev.net account? Sign up!
Profile
Author
Advertisement
Advertisement