Jump to content

  • Log In with Google      Sign In   
  • Create Account

achild's Journal



Slow progress

Posted by , 13 September 2013 - - - - - - · 2,030 views

Well, it has been just over 2 months since my last entry. A lot has happened since then...

Basis released!
First and foremost, Basis, our (completely free) 2d skeletal animation tool was released for public feedback and testing. We've produced an SDK for a few frameworks, with many more to come, and though there is still a ton of work to do, it is quite usable. It can be found at http://achild.wikidot.com/basis.

Basis themed!
Second of all, we themed nearly every aspect of the software. I think it makes quite a difference - it never looked bad before, but it caught me off guard when my wife saw the newly themed version one night and said "Oh okay, now it looks professional!". I really didn't have any words for that at the time, but it was definitely helpful feedback outside of my own biased opinion. See:

before
Spoiler


after
Spoiler

Spoiler


I mean she's right - yes it's just a cosmetic change and doesn't alter functionality in any way - but it really does make a difference. (I really did think it "looked professional" before, but now with a before and after comparison, well, there is no comparison)

Kickstarter failure!
We tried. And we failed miserably. The interesting thing is we hardly got any feedback, even though we had hundreds of views. There were many points we now feel which could have been better. One contender for the biggest issue of all was that we had not built a community yet. We thought the kickstarter would build the community and get people to know about it. Sometimes this can be true, but I believe more often than not you have to have some kind of following to build that initial inertia. Otherwise, you are almost doomed from the start, in most cases. We definitely failed here.

Unfortunately we also needed it to keep going at such a high pace, so development has been tremendously slow since then. So it is with side-projects!

We may try again in the future, but for now the focus is on improving Basis, and continuing to improve the Basis SDK and expand it to other frameworks and libraries and software, as slow as it may happen for now.

Which brings us finally to:

Basis SDK!
Okay so the SDK has been written and re-written. I want to get it right for these few libraries before expanding to other languages and frameworks. The hardest part has been getting it right on so many platforms, for instance we have to support many of the Amiga flavors. This can be difficult when you don't own an Amiga! Some systems such as OS3 can be emulated, but OS4 and WarpOS definitely need original hardware. Naturally, Basis SDK worked great on OS3, but not on OS4 and debugging it has been troublesome to say the least.

Also, the Amiga build system was moved from cross compiling on Windows using gcc, to full blown Amiga compilation using VBCC. This allows us to even support WarpOS in the first place, and really just makes the whole build process much smoother (after the many hours of getting it set up properly with all the different Amiga SDKs and what-not).

Another interesting issue was the need to use software based rendering on the Amiga. This presented a couple weeks of head-scratching in itself. Even with the available sources on the internet, you'd be surprised how hard it is to get it right, accurate, and fast. But still, thank goodness for the old Chris Hecker's articles and the FATMAP articles. They were both very helpful, though neither one of them fit the bill perfectly. A nice side effect is software rendering in SDL when OpenGL isn't available.

After this, the other SDK implementations were very easy.

However, what's an SDK without a game? We have demos to show it off on each SDK on each system, but you really need a small game to see what is going to work and not work. Or at least what's going to work best. It's just the nature of the beast. So, next on the list is to build a small game using http://hollywood-mal.com/ because it was sponsored by someone and test it on the zillion systems is supports.

Then I really want to make a game for the Wii. There's a functioning Wii demo, but it's been so long since I had fun coding something up, this must be done. Then back to work on Basis and more SDKs.
functioning Wii demo
Spoiler

But first, a game in Hollywood-MAL and it's slightly butchered Lua. Hmm, what to make...


Experiment in Kickstarter

Posted by , 11 July 2013 - - - - - - · 1,145 views

Woo!

We finally reached the point that we are ready to release Basis. As I mentioned before it is free in all ways possible, private or commercial use. It is proven to be usable in-game, and easy to implement into various frameworks and libraries. So, we have decided to start a kickstarter for it.

Here it is.

The big question

Why kickstarter if it's free? Is that sort of an oxymoron?

We have worked very hard on this, and lately it's come to the point that it is like working a second job. Combine that with our every day lives with our churches and community, families, and actual job, and it's not certain we can keep it going like this. So, basically we have 2 options in front of us:

1) We can put it on the back burner, working on it a few lucky hours a week. We fully intend to build Basis all the way to our final stretch goals, but it may take a very, very long time like this.

2) We can try to get the support of the community, asking for funding and for others to spread the word. Basically, the more we get funded, the longer we can put off doing #1.

There is really no in between for us. I personally have really enjoyed building this to where it is at even now, but it's a point that we'll have to let the online community decide where it goes from here. Honestly, we have no idea what to expect!

Our own questions

You've probably heard the expression before that something is worth what you pay for it.

If we're giving something away for free, will people even feel it is worth anything? The intention from the beginning has always been to give this away, and we don't plan on budging there.

When a beginning photographer saves up his or her money for, say, Photoshop... with the money they worked so hard to earn, chances are they'll take the time to learn it and value it very much. On the other hand, the things we get for free - we put no sacrifice into, we invested nothing of our own into it - their value is much less to us... typically. It's very plain human psychology.

That doesn't mean that putting a price tag on something makes it worth more, but if you put that price on it and people are willing to pay it, it's worth more to them, which in turn gives your product perceivable value.

Even the reward tiers were hard to come up with - what more do you offer people when you're giving the product away after all!

Do people still want this? We started working on this 2.5 years ago. Since then, a couple other amazing products have been released, namely Spriter and Spine. Both of these are shaping up to be fantastic packages with a lot of innovations. They also were both kickstarted successfully. They also both have a value attached to them.

So, being #3, will people even want this? We feel we are bringing enough new things to the table for there to at least be a chance, not to mention the free-thing could actually be an advantage (we have no idea how this will play out), and our workflow is definitely different than the others. Naturally we are biased and feel it is a better, faster workflow. Will other people?

Is Windows only going to hurt? Often there is a complaint if something isn't cross platform. This is understandable - everyone wants it native to their system. It does run very well under WINE, but I know it's not the same. This was a hard decision to make, but it seemed that since it won't really add anything to the software itself to make it cross platform, and since it is already well under way, it would be better to make it fully featured and work very well as it is. The OS and Window code is pretty well segregated, so we made it a stretch goal to port it. It shouldn't be too much work, but it will require some intense research, testing, and in the case of OSX, we'll have to buy a computer and learn some things we wouldn't have had to otherwise. That is no problem, but we decided making a great animation tool would come first.

Our hope

Our hope is to get the support of the game development community, as well as get the word spreading around to others. Again, at this point, we really can't predict one way or the other what kind of reaction we are going to get to this. So, we are prepared to more than abundantly succeed, as well as completely fail, and anything in between.

Do feel free to check out Basis though, and whether you support the campaign or not, enjoy it and use it and even contribute to it! We have forums for discussion, and we're always looking for people to contribute runtime implementations for other frameworks, libraries, and software.

And if you like it, let other people know about it!

I'm sure I will be doing another journal at the end about the success/failure and our take on it all. Thanks for reading Posted Image


90 + 90 = 100

Posted by , 05 July 2013 - - - - - - · 2,834 views

If you've been in development long, you've heard the phrase or something similar: "The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time."

Sometimes even this seems overly optimistic.

Are we there yet?

After roughly 2 1/2 years of development, we're looking at our initial beta release in the next couple days. The funny thing is that we've been saying this for about 2 years! "90% done".

When Basis first started development, it was called something else, and there were no other 2d skeletal animation programs worth much of anything out there (publicly). That was the motivation for it. So "there" was "the bare minimum of what I can use to make a game", and I was excited to share it with others as well. Then I used it, was happy and excited and all those good things... and tried to use it in a small game. Animations worked great! And then I needed to know when my character punched, or ran into a wall, or ... uh oh... back to work on the tool.

At this point, there were periods of disinterest, as well as periods of heavily committed working. There was a lot to do. While I was at it, it was time to take it seriously, knock out some of these bugs, and fix some of the "systems" that were being implicitly defined by the software logic.


Are we there yet?

So, around a little more than a year ago, I was determined to release Basis. Started telling other people. But something happened...

I let someone else use it. And then another someone else. That was all it took. It was quickly clear there were some fundamental issues that needed changing in the workflow. This was developed by a programmer after all, from a programmer's perspective. Artists often have a different world-view...

All this time it was becoming more and more evident that the current undo/redo/command system was inadequate, and shoehorning new features in was making the whole software less and less stable. Bugs were harder to fix, and fixing them were more likely to create new ones... you know the drill. I'm sure you do. So I was going to rebuild the system after our first release, but fixing our newly discovered workflow issues was going to dig deep into the action system anyway... it was clearly a bad decision to put it off any longer.

So the central nervous system of the program was redesigned, then reconstructed piece by piece for the next 6 months, all while adding some of those new features and workflow enhancements simultaneously.

Are we there yet?

This part was like the monotonous many-hours drive on a very straight highway with nothing interesting to look at... at night time. But it's done now, and the benefits are shining so brightly pretty much every day! Everything is more stable, the code is better organized, things are so much easier to tweak and refactor... the whole software didn't change that much but it some how actually felt tremendously more solid! And the fundamental issues with the first testers were resolved. So again I determined to release in a couple days.

Then I let someone else use it, again. Only this time it was for more than a couple minutes to just goof around. This time they were really set to use it for a purpose. It quickly became clear that those big workflow issues were just the beginning, now we were going to get into the little details... 90% done right?


ARE WE THERE YET?

So, we did it again. This time, however, there were 2 key differences:
1) Starting roughly 3 months ago, my friend used it the entire time it was being developed. I can not explain with words how helpful this was.
2) After about 6 months redoing the action system (yes I had bouts of neglect), there was not much to show for it, besides it actually being functional again. In complete contrast to this, and thanks to those 6 months' work, the last 3 months have been spent implementing feature after feature after feature... it has been very nice! Exciting actually! But it has been much like a second full-time job.

With so many ideas and plans and freedom to develop and feedback, it was hard to come to a decision about when it was "good enough" for others to use, especially after the first 2 or 3 blunders. Finally, it has come again to that point.

Posted Image

Now are we there?

So here we are again... ready to release in a couple days. Is it real? Will there be any unexpected surprises? Are we making the same naive judgment we made so many other times?

In many ways, yes!

I'm sure there will be a myriad of issues we haven't encountered before. Another friend tested it out to make a quick 3 hour game-jam-like game (speed challenge). We had mixed results. On the plus side, he figured out the software for the first time and used it in his game all within the 3 hour time period! On the minus side, he used windows paint, so his background was a color mask instead of an alpha channel. Basis automatically detects this, but the Basis plugin developed for his software did not. Naturally, this had to be fixed for his game to work.

At some point you realize the question isn't "are we there yet?". At some point you realize it is "Where is there?"

So... where is "there"?

This has been a moving target, honestly. Not usually a good thing, but unavoidable sometimes. Especially with iterative development.

Eventually, software called "Spriter" was shown to the world and it showed a lot of promise. Had this happened a year or 2 earlier, I probably would have funded it instead of starting my own. (We donated a bit anyway). They thought they had found their "there", only to soon realize they basically had to start over. It is just now coming to fruition.

Then, a few months ago, software called "Spine" was released. This software is very good, and a lot of developers have made it a part of their workflow. They are getting a great deal of much deserved recognition. In a different universe, we probably would have switched from Spriter to Spine at this point, though they each have different strengths and weaknesses. They found their "there".

Then we have our software. Some things are complete, and have undergone enough iterations that they are easier/better to use (subjectiveI know). Some things are in their infancy. Some features are totally different then our brother softwares have to offer. Some things they are just getting, we had 1-2 years ago! We even have a couple nice innovations (again subjective)... so at what point do we let it out there? Does it even matter at this point? I didn't answer the question about where "there" is yet, did I?

Posted Image

We're there!

We didn't really know it until we found it.

Often times this is a horrible way to develop. It is harder to meet goals, and it raises the chances of "vapourware" and things such as that. However, because of the iterative nature of this type of development, and the fact we had no real deadline and no publisher or bosses pressing us to finish, we were free to let it run its own course. The "it's done when it's done" methodology.

I wouldn't necessarily recommend doing software development like this to anyone. Not with a clear conscious anyway. And especially not with actual games. You're creating an alternate world with games, and there are never enough details. Even our world is still in development. You have to have a clear goal in mind. Unless you're Blizzard. Or Valve. But even when you're one of them, you may never find your "there". Halflife 3, right?

Iterative software development works very differently, whether it happens intentionally or naively. We are probably one of the lucky ones in this case.

I'm not saying the program is complete. Again, there are many very exciting features we are working on or will be working on soon. It is, however, quite usable, and quite efficient. And hopefully it is as intuitive as we believe it is. The only direction to go from here is up. It's a wonderful and encouraging place to be. But what is different this time?

How do we know it is usable? We've used it! How do we know it is intuitive? Others have used it! How do we know it is efficient? Experience! And carefully crafted workflow and design. And there are many more evolutionary changes in the works to increase all 3 of these main factors - not just a little, but very very much.

I have to wait on my friend to finish fine tuning some example artwork (another where is "there" issue). 1 more day? 2? In the mean time, we're taking the opportunity to implement some reference examples on other systems. We hit SFML, SDL, Allegro, HGE, this Amiga environment called Hollywood....

... and then for fun I downloaded Devkitpro and attempted a Wii implementation. Hmm.. If more than 2 nights of research and work then just forget it. Amazingly, in only a few hours, it worked. What a cool feeling.

Posted Image
(my family's Wii)


Tools Dev - What's it DO

Posted by , 29 April 2013 - - - - - - · 1,111 views

A Tool's Purpose

The purpose of tools, by definition, is to perform or at least help with some kind of work. When it comes to development tools, as with all software, one can quickly lose focus of the work the tool was to be purposed for! Especially after a few months of development, or even a few years!

That is one advantage quick 'n dirty "hack job" tools actually have - they strike right at the purpose and help you get your work done. They may be crude, and you wouldn't want someone who doesn't know all the "don't do that"s to get all click-happy, but without question they will help you get your game finished much faster.

On the other hand, it limits their re-usability, and even re-purposing them can actually be more time consuming and inefficient than starting (mostly) from scratch on a new tool.

So, if we go back 8 months from today, here I am, quite satisfied with a 2d skeletal animation tool I have been working on. I've been working on it for about a year and a half at this point, and I'm finally ready to make a little game with it!! WHOO! And then I actually used it.

...

Don't blind yourself because it's easier - it's NOT

Okay so this software works, and it works quite well, and IK is working well, and animations are being created, and then I put it in the game, and ... oh. The game? Yeah. The game. The tool was not meant to make animations (only). Plenty of software makes animations. Toon boom anime 3000 and all that stuff. And they do a pretty good job at it. This tool was meant to make animations... for games. Sprite animations, 2d skeletal animations, fully animated levels a la Rayman, you name it. ... for games!!!!

I had lost sight of that. I put the animation in the game, and would ya know it, I couldn't do collision detection. I could hack around that though. But then I couldn't trigger events in the animation. How do we know when the jump animation is on the ground? How to know when the punch animation is capable of causing damage? Should I hard code it? Eek! What happened to my tool?

I was ready to release it to the world, ready for other people to be able to animate their characters, their games, their worlds. And for free, at that! Similar software, Spriter, came out eventually, and it looked (and still looks) promising, but has fallen to the wayside, for now. But I had to take a step back and remember what I was making a tool for. It was then I realized if I didn't make internal changes now, every feature it needed would be hacked on to a (now obvious) defective base and would inevitably one day explode. *sigh*

...

The Hard Road

So over the last very many months it has undergone a massive internal transformation. I just finished re-factoring the entire action system into the undo/redo command system I spoke of before, as well as a number of other time consuming internal improvements for program structure to prepare it to do useful things. 8 months later, Basis could do less than it could before, was more buggy, and had nothing new to show for it besides a window that says "history" with unlimited undo/redo.

But it was fantastic.

6 weeks later... yes, 6 weeks later, today, it has gone from "will this ever get done or at least back to where it was before?" to "wow this is awesome!". So I will show some screenshots (or it didn't happen right), but just remember, when you're making something general purpose that is supposed to be useful and intuitive for everyone, you will be challenged. Your engineering skills, psychological analysis skills, deduction skills, design skills, coding skills, and more skills will all be put to the test. It will tax your brain. And you might even find you got it wrong after all that! But through it all, try not to lose focus of what your purpose is. Always build towards that!


Evidence of Hard Work!

Sprite export
Posted Image

"Attach" bones - these do not work as actual bones, as they are more of a static extension. But they do more than simple "parenting". They solve a whole list of problems, actually. (attach hats/items, connect skins while limiting rotation/scaling, mark positions, simple parenting, etc)
Posted Image

Document notes and image - a simple thing but useful none the less in the content creation stage
Posted Image

Animated OBB Collision - define multiple states for the collision shapes too, so you know what is "hot", "normal", or whatever else you can imagine
Also events! Define any arbitrary events with variables, trigger-able per frame.
Posted Image

Animated AABB Collision - If OBB is for skeletal animations, this is purposed for simple sprite animations. Tools for both workflows are similar so why not be in the same software package?
Posted Image

I showed texture auto-detection and texture packing in my last entry. The gif animations weren't that great (sorry), but I think they got the point across.

Animated Z Order - It's subtle but notice the tail go behind and in front of the head as it swings (don't be harsh, it is a still WIP animation)
http://imageshack.us/a/img17/948/basisz.gif


More coming soon! Yay!
-Scott


Forging Tools - Close to release!

Posted by , 12 April 2013 - - - - - - · 1,253 views

Well! After over 2 years of development, working countless nights after the kids and wife have gone to bed or fallen asleep, we are just about ready. I am going to make a crazy attempt to finish a new major feature a night. As they come together, they'll be posted here.

Tonight: Texture packing.

For traditional sprites - Posted Image

Also for skeletal animation (also see the automatic texture detection on load) - Posted Image

and anything in between. Shooting for a very important feature tomorrow. Wish me luck!


Tools Dev - Undo Systems

Posted by , 17 January 2013 - - - - - - · 1,094 views

(warning: this post is purely about code)

Do not, I repeat, Do not wait to add undo/redo to your tool/editor "later"...
... and even when you plan ahead, expect it not to be perfect the first time. In fact, it can even affect the user experience.

I have created 3 totally different types of undo systems from scratch.

#1 was for this calendar application and was completely action based: it saved your action and it's parameters and had a small virtual machine of sorts that could do and undo these actions. There were only a handful of them after all.

It was super-lightweight, but insanely hard to debug. I doubt I will ever go that route again. Partly my fault, but it caught me off guard the complexities this created. Also, this was "the time" I tried to implement undo/redo after-the-fact.

#2's system tried (implemented in Stickimator) to be clever. #2 was combined with an action callback system using predefined enums so that it could be automated. "boost::any" was used to store the undo and redo data, and some template-magic was used to automatically undo or redo the change. As long as the different data types were specialized, it would handle everything for me without any extra code. Neat!

All actions were in one place in this giant switch statement, again much like a VM. This kept everything in one nice place, and all actions were simply done like so
eventAction.Fire(Action(ACTION_FOO, fooData), true);
As long as fooData's type was specialized, the rest was automatic. And ACTION_FOO had to be in the giant switch statement.

This got unwieldy fast. Adding features became hacks (like accumulating multiple actions into a single undo), and the code for actions tended to get messy. It became clear that it wasn't a horrible attempt, but it wasn't going to suffice. Changing it would take a good bit of refactoring, and there would be no immediate obvious benefit in the software once all the crippled parts were fixed. So, I was going to wait for the first real release of Stickimator, but once I started adding more to the old version to get the release out, it seemed a better idea to just rewrite what there was rather than rewrite more later.

#3 is based on, yes, the command pattern. Sort of. Very similar to QT's command/undo structure, it supports:
  • Command logic is encapsulated per command instead of one-place-for-all-commands
  • Undo/Redo logic per command instead of forcing a single method for the entire system. Undo/redo based on action logic OR data OR a mix! Again, encapsulation benefits (versus specialization here, data structure there, command logic over there...). I wonder that I didn't go this way from the get-go?
  • Accumulate (like QT's merge), and Freeze to stop accumulating. Accumulation is automatic when a command supports it. Freeze is explicit, of course.
  • Cancel! If you are doing some command which accumulates with the left mouse button, and you right-click, you might expect it to cancel. Not just undo, but undo without a redo. No trace left. That's what cancel is for.
  • Get/SetText for undo/redo menu descriptions, history list, logging, etc.
  • ApplyTo *
  • Command Stacks to keep undo/redo list
  • Command Macros
* ApplyTo is neat but is Stickimator-specific. Stickimator edits works on a per frame basis. So, what if you want your changes to affect multiple frames? Multiple animations? Some sort of grouping functionality (that can get messy!!)? Well, ApplyTo can be overridden in supported commands to take the most recent command(s) on the CommandStack and apply them to selected frames and/or animations. This will be a super-neat feature in Stickimator for quick editing and fixes.

The best part? Tons of Stickimator's code was magically cleaned up in all this process. I had not realized how much command-specific code had intermingled with other parts of the code, such as input-logic, making it very complex. And it still is triggered very simply:
// These events are handled by the main application and can be called from any controls/views that
// have them. The main application links eventCommand directly to CommandStack::AddCommand, and again,
// accumulation is automatic if supported and the most recent command isn't frozen.
eventCommand.Fire(make_shared<CommandFoo>(...));

// or in some cases
shared_ptr<CommandFoo> cmd(make_shared<CommandFoo>(...));
if (cmd->IsValid())  // do this; don't throw from CommandFoo's constructor if there is a problem
    eventCommand.Fire(cmd);

// also there's this - the ID ensures that the action only occurs on the expected command
eventCommandAccum.Fire(COMMAND_ID_FOO, ACCUM_FREEZE);  // or ACCUM_CANCEL
Also, my undo/redo didn't have command descriptions before. Posted Image

So that is why there are no new neat screenshots this time. Hopefully next time I will have some fun stuff to show! Until then!

[edit] clarified a couple small things; fixed a code typo


Tools Dev - Using it Wrong but Right

Posted by , 06 December 2012 - - - - - - · 1,393 views

Hi!

So, a while back, a good friend tested Stickimator. He is the perfect first tester. Why? Well, my friend is an amazing artist, but his computer experience goes just far enough to shoot his friends online on Halo 1-zillion. He is, for all practical purposes, clueless when it comes to digital art. In fact, I have been working with him on it - which is scary because I am almost as clueless as he is!

So, we scanned a drawing into Gimp and attempted to color it, and import the textures into Stickimator. I'm showing him the ropes in this tool: attach a skin to this bone here; press your mouse button to rotate that bone there; press ctrl+mouse button to freely move that bone right there.

The thing about the ctrl+free-move, though, is that it affects the attached skin in 2 ways. It rotation AND scales the skin because the bone is doing the same thing (rotation + adjust bone length).

So, armed with a good hour of me talking and doing far too much (he should have been just trying it out I think), he eventually gives it a whirl.

Of course, he used it exactly like I did not expect, using the ctrl+move action for everything. I believe the idea of posing bones by rotation only must have been quite foreign to him... so he just free-moved it all. This is when I realized that IK was going to be very important, not as an extra feature for the future, but as a necessity for now. And here in lies an important key in tool design and usability.

If you're willing, watch a customer/client/user use your software for the first time, even if they do it the "wrong way". This is an incredibly useful way to improve your application. I'll grant you that sometimes we figure out the best way the first time, and they really should take the time to learn it (RTM anyone?), but this is a rarity. Usually, we are too close to the software to really see. Just as the end user is often not close enough to really see. It takes both points of view, as well as a healthy does of humility.

Anyway, check it out! I am glad this came about, because it was fun to implement Posted Image

Posted Image

[edit - content basically the same but it was revised for clarity - should not be typing these up at 5 am!!]


Tools Dev - Handling file paths

Posted by , 29 October 2012 - - - - - - · 492 views

As my second public blog post, bear with me as I learn how to share some of the things I have learned in tools development.

Yes, you have to think about it.

UI design is not the only issue of usability when it comes to tools. There are many unseen details which affect it just as much. Case in point: "Windows Notepad" is a very simple, quick tool for jotting down a couple of notes. But have you ever noticed how its undo system works? It is very simple, and yet it really throws one off. The first time you undo, it undoes your latest text entry. However, hit undo again, and it puts it all back! It undoes the undo, instead of undoing more.

I do plan on talking about undo systems soon. Today, I want to talk about another unseen detail - handling file paths in your tools and data files.

There are no details in tools development (any software really) that can be implemented without thinking about them. We all know this. Sometimes it can be surprising, however, what those details are. Even after many years, we can take very simple things for granted. How often do we consider file paths? I'm sure they exist, but I don't recall ever seeing an article or paper on this problem.

It can be a problem.

The Problem

The issue is actually two-fold, with some details underneath.
  • How to store them. Should they be relative or absolute? How do we know the difference?
  • How to store them, again. But now, how do we support multiple languages? Unicode, right? So which encoding?

Keep in mind that our focus is mainly tools which create or manipulate game data. Therefore, it is important to keep in mind what will happen on multiple platforms in multiple countries, using software other than the tool to read the data. While this problem has been addressed again and again, there really is no one best solution. Opinions will vary on this - each with sensible arguments. When it comes down to it, though, it will ultimately depend on the application.

The Theory of Relativity

So let's address the first question. Should the paths be stored as relative paths, absolute paths, or perhaps support both? Again, this entirely depends on the application. We will look at 3 examples, each with an entirely different solution.

At work, our software captures a number of images from an external camera/microscope system. It may do measurements or analysis on these images, as well as any other type of processing to make the images usable. One can end up with a few small images totaling a few MB maximum, or one can end up with hundreds of thousands of very large multi-plane images totaling many GB. An additional requirement - thanks to the interesting process in determining what is "accepted" and what is "wrong" - is that all image data must remain completely uncompressed. Posted Image

So, we store all data within a single data file which essentially incorporates its own file system (based on FAT32). All images, measurements, blobs/shapes, results, processes, camera settings, tables, and so on are stored in this data file. Everything is set to a fixed location (or 2 for historical purposes) so that software just knows where to find the various "files" it is looking for. There are paths, but they need not be determined - only checked for.

CodeSync (here) is a very specific tool for comparing two or more folder structures at a time. Naturally, it would not make much sense to store relative paths for this. All saved paths in CodeSync are absolute. If it can't find a path any more, it simply highlights it in red so that you know there is an issue. I do not think much else needs to be said here.

Syncing relative paths would be ... interesting!
Attached Image

Stickimator is an interesting example, as it is our main focus: Game data! Stickimator creates 2d bone-based animations, and therefore has a fair amount of both internal data (bones, frames, etc) and external data (texture images for skinning, background reference images). It uses a main data file containing internal data combined with relative paths to the external data.

The first issue is about internal and external data. Why not just put it all into a stickimator data file? And the answer is very simple: you may wish to reuse textures.

As for the second issue - relative paths, let's imagine a complex scenario. Our super awesome game will have a main data file with all its resources. However, we also want it to be able to override with external files on disk for fast iterative development. We also want to be able to mod our super awesome game. Our mods should follow the same pattern: a main data file with all resources which can be overridden by external disk files.

Now our super awesome game has 4 possible search paths when running a mod. It will check, first, the disk path from the mod itself. Then it might check the mod's main data file to see if the path exists within it. Then it may check the main game folder. And finally it might check the main game's data file for the resource. This can only be done with relative paths.

As an additional proof, even when just sending your stickimator data file + texture resources to someone else, the paths to the texture resources will have to be relative or it likely won't work anywhere but your own computer. Imagine your coworker: "F: drive not found? I don't even have an F drive! Why is it looking for an F: drive to load these textures?!" Oops!

Localization-Unicode-UTF-32-UCS-2-ASCII-Locale-Encoding-Endianness-Code-points-Diacritical-marks-Normalization-AHHHHHHH

For now, we do not need to get too far into the details of internationalization. We will simply decide which unicode encoding to use. Our major options here are UTF-8, UTF-16, or UTF-32. Many environments (Java, Windows, Python...) were built at some level or another to support UCS-2 rather than UTF-16 traditionally, but for the most part each has resolved this in its own way.

And therein lies the problem. Did you see that? In its own way.

Win32 favors UTF-16. It also supports basic ascii, but not UTF-8 directly. Linux handles UTF-8 most naturally and UTF-32 in second place. Mac OSX seems to be UTF-8 then UTF-16, though I have no experience there yet. Java uses UTF-16 though at higher levels supports most any encoding. And the list goes on and on...

At work, we needed to at least support Japanese and English. The entire 20 year old codebase used win32 ASCII functions, so we had to convert the whole source to the wide versions. Because our software is Windows only and very niche, it made the most sense to us to just save our new paths and other text data as UTF-16 strings. This is great and works very well for us, but were someone to want to read our data files on, say, a linux system, they would now have some extra work on their hands, and their application would probably have an additional dependency.

Some games may choose to use UTF-32 because of its simplicity. Especially if your game does a lot of text processing and/or searching, UTF-32 makes things easier because their is no ambiguity as to number of characters vs number of double-words. One double-word equals one character! It is very simple. As long as you have code that supports this (reading file systems, displaying text, accessing internet addresses, etc) you get to do less thinking, and concentrate on something else. The only real drawbacks are limited support if you don't already have code that handles it, and the fact that often your text data will take up much more space than they would otherwise. Between 2x and 4x in the average case.

In Stickimator, after much back-and-forth, I have resolved to use UTF-8. Again, it is not the only "right" answer, but it won in my own mind in the end. Although the application is Windows only right now, it is intended on creating data that should be easily read in any environment. Because my own second priority is linux, UTF-16 becomes a bit uncomfortable. While win32 doesn't directly support UTF-8, it does include a couple functions for converting to/from UTF-8 and UTF-16, so conversion on loading/saving is very straightforward. Besides this, most file paths' characters tend to be in the basic latin character range, so the majority of our strings will only use 1-byte UTF-8 characters, thus saving much space. Since OSX also supports UTF-8 very easily, this is win-win. I also didn't have to change the actual file format when adding UTF-8 support to Stickimator. Now it's win-win-win!

Of course, a full unicode library such as ICU will handle most of your problems. You can pick whichever encoding you want then. Still, if your data file will be read by other people, or if your tool is not part of a closed asset pipeline (ie it is not for just one specific game), you will need to consider the above factors and figure out what truly makes the most sense.

Bonus Caveat

In regards to unicode encoding - there is one more thing that you may catch you off guard. If, for some reason, you include any textual data directly in your code, you will actually have to consider your compiler and how it handles unicode text data! Take this interesting line of code, for example:
mTimeL = make_shared<GuiLabel>(CONTROL_ID_STATIC, 230, 3, 10, 18, this,
  L"≈", GdiFont::GUI_FONT, MakeDelegate(this, &GuiFrames::OnDefaultGui));

In MSVC, you have to make this code a wide-string literal. If you assume it will put a UTF-8 string into a normal string literal, you are wrong. I was wrong when I assumed this! The compiler will whine and complain, and when you run your program only the 1st (or is it the last?) byte in each UTF-8 character will be present.

In GCC, it will gladly take this string literal without the 'L' and gladly include all of the utf-8 characters' bytes.

Can you find the unicode squiggly?
Attached Image


Freebie - CodeSync

Posted by , 25 October 2012 - - - - - - · 1,088 views
codesync, free, sync
Freebie

First, I would like to share a tool developed a year or two ago to aid in sync'ing files - especially code.
You can find it at http://achild.wikidot.com/codesync

"Another file sync'ing tool?" you ask?

Well, sort of.

"But why?!" you ask?

I suppose it is better described as a folder comparison tool. It doesn't even have it's own merge functionality (you set up an external merge tool). It would be silly to try to replicate what, say, WinMerge already does. (I like WinMerge).

Maybe my google-fu stinks, but at the time I could not find a simple, quick, and straightforward tool that would compare more than 2 folder hierarchies simultaneously. ... and save my paths so I can do it again ... easily. I'm sure they exist. But I couldn't find one. So here one is. And I can be sure it has features I need. And it can be quick and easy to use. And it backs up every change in case I mess up and don't realize it until weeks later.

So if you have any need for such a thing, feel free to use this. I'm open to suggestions on any improvements, and especially to bug reports. Because I'm now releasing it publicly, there are a few minor issues that will be addresed in the near future, but it is quite usable. Do note however that it requires .NET 2.0 and therefore is a Windows only application. If it becomes insanely popular some how I may port it, but we'll cross that bridge if and when that happens.

Of course, here's a screenshot or 2:
Posted Image

Posted Image

And on that note - the next journal entry will in fact be talking about the little detail of saving file paths to resources in your game data files.

PS Can anyone tell me how to resize an image on here? I have tried multiple things but apparently I'm missing it Posted Image


Don't forget the Tools Devs

Posted by , 16 October 2012 - - - - - - · 1,168 views
tools programmer
Wow.

I have been thinking of starting a dev journal here for a few years. Obviously never got around to it. Now I wonder why I waited so long! Wow. I really appreciate all the resources and tools we have to do this now that I'm doing it.

It makes the most sense for my little journal to be about what I actually do, so for now it will focus mostly on tool development. This is my job (though it is not for games) as well as the stage I am at in working on the usual hobby-side-project-dream-game, so it will fit the bill perfectly. With that said, let us address something right quick:

Tools development is hard.

Yes, game development is hard. It requires one to have a handle on many disciplines, even if they primarily focus in one area. But how often do we think about the tool developer? It is no secret that the quality of your tools can make or break the development cycle. It is an engineering challenge in its own rite. The better the tool is designed, the less its user notices it, and the more they can focus on producing content or whatever it is intended to do. Even if one is overly happy with their tool of choice at first and is all gaga about it (is that phrase still legal?), eventually their mind will be on using it - not working around it.

Not only that, but as programmers we tend to be lazy and unfair. We want our tools to read our mind so that a single click and maybe a drag always does what we are thinking. We then want our tools to read our mind again so that a single click and maybe a drag does the completely different thing we are now thinking. If we have to, we'll tell it "no, not that, read my mind again" by attempting a keyboard modifier such as alt or ctrl. If this fails, naturally the tool is "stupid" or "broken" (in other words we probably didn't read the manual).

And then, when we programmers decide we want to tweak some oddball parameter all of a sudden because we are just so clever, the tool that automatically reads our mind must stop and let us very specifically tell it what to do. But it should be very easy to do that.

And don't get me started on how we all are so certain of "how it should look". The feel of a tool actually does make a big difference. From the color scheme to the organization of the UI, the last thing you want is for it to interfere with someone's thought process and creativity. And speaking of UI - the workflow of a tool usually takes many iterations to get right. Even when you get it working "bug-free" after many days or weeks or months (or years!), you may discover there is a far better way. Often, this can't be seen until you've finished the previous iteration and spent a great deal of time with it.

Sounds a lot like game development doesn't it?

Don't forget the tools devs.

One must practically have a degree in both engineering and psychology! Often unnoticed until something is broken, a good tools developer is a very important asset to your team. Be their friend Posted Image

This journal

So, that wraps up my first attempt at this, for better or worse. Following entries will probably be a bit random, but they will simply cover details of tools programming. For now, I have in mind to write about various aspects of tools that are tricky to design and tricky to implement, and present one or more possible solutions.

These types of things don't really get talked about often, and perhaps sharing them can help someone else. I hope it will also be beneficial to myself to write these things out, as I usually have much trouble organizing my thoughts.

A screenshot

It just wouldn't be right to start a journal without posting a screenshot of ... you know ... something.
Often I will use the development of this software as a source of journal material.

Posted Image





August 2016 »

S M T W T F S
 123456
78910111213
14151617181920
21222324252627
28 293031   

Recent Entries

Recent Comments



PARTNERS