This series chronicles Catch the Monkey from ideation to sale worldwide in the App Store.
You can read other articles in this series here:
Part 1: Design & Prototyping
Part 2: Building the Core
Part 3: Balancing & Polishing

You can find out more about Mirthwerx and our projects at our website.

Intro
We had a game! Hurray! It worked, it played well, it has a beginning, middle, and ending. We were ready to get the sucker out the door. But before release, we had to go through the final stage of development: testing. Wow, what an eye opener!

No one Knows How to Play Your Game, and They Don't Care to Learn!
I submit I may be the strangest person alive.

I grew up in the 80's and early 90's. This is where many of my early gaming habits formed. Back then when I bought a new computer game, such as Civilization, Ultima, or Wing Commander, I would sit down and read the entire manual cover to cover before attempting to play the game. And this was when manuals were works of art: the civilization manual was over 100 pages and filled with fascinating sidebar historical facts. I continued this practice, though I've stopped now that the manuals are nothing more than epilepsy warnings and telling me where the square button is on my controller. (Fortunately board games still have amazing manuals, so I can enjoy those )


The good old days when games were hard and manuals were thicker than your arm!

Apparently no one else read manuals so the gaming industry moved away from them altogether. Players want to play, not learn how to play. I sort of knew this, I read about it in game design books, but I didn't have the experiential knowledge of it. It quickly came.

At our pre-release party (unfortunately 3 months before the actual release, but who's counting) we had several iOS devices with a 4 level demo version of Catch the Monkey installed on them for people to play. We watched people pick up the game and play it for the first time. We learned two things: people enjoyed playing with the monkeys, but they had no clue how to do it. So while they had fun, they were frustrated not knowing how to use the various tools.

It seems obvious as I write this, but we discovered our need for tutorials built into the game. All I can say is that when you work on something closely for 2 years you lose track of what is "intuitive" and what isn't. Observe strangers and reality will come crashing in. So, we used our character dialog system to retrofit in a tutorial system.


First Iteration of Tutorials. Nobody reads em.


Weeks before release, we tested with a focus group of teenage girls (our demographic) to see how they enjoyed the game. I squirmed in my seat as I watched them tap "next, next, next" and completely bypass the tutorial to get on to the game. Once there, they didn't know how to use certain features, started losing, and became frustrated.


Final version of tutorials. If you can't follow that, there's no helping you!!!

We learned valuable lessons as we went through three iterations of tutorials:

• People assume they already know how to play your game. I can't for the life of me figure out why they come with this belief, but they do. Work with it, not against it.
  
• People don't want to learn (because of the above), they want to play. So teach them one basic thing and set them off playing
  
• When teaching, we observed the average player's patience is two: Two screens of slides, two steps of interaction, two dialog boxes, then they don't care anymore and want to skip forward
  
• After playing, people want to learn. There is a correlation between how long they play and how interested they are in learning to play. In the first 2 minutes, they have 0% interest, at 5 minutes they have 10% interest, at 10 minutes they have 50% interest. You have to space your lessons appropriately
  
• Remove all flowery "in character" text from the tutorial, they want to learn as quickly and efficiently as possible and could care less if a character starts each sentence with ""
  
• They don't want to read, they want to do. Make the tutorial visual and interactive
  
• Pre-plan your tutorials into the main story/progression of the game. Don't do what we did and try to retrofit it in, it was a lot of work after the fact
  

Testing with Testflight
As previously mentioned, iOS locks down the devices to which you can install a binary. This requires the unique UDID of each device, registering it through the apple portal, including the provisioning profile at compile time into the binary, and then copying a provisioning profile to the device through iTunes (which provides zero feedback if it was done successfully) and finally copying the binary to the device through iTunes. I can't think of a process more antithetical to the apple "it just works" mantra. So, you can do all that OR you can use testflightapp.com.


Testflight for build distribution during testing; epic win!


With testflight, you send testers (family, friends, enemies) an email link and they go to it on their device. Testflight takes care of finding the device UDID, os version, make/model of device, and sending it to you the developer, installing the provisioning certificate. As a developer, all you need to do is register the device id to your binary. Now you can upload a build (with release notes) to testflight, and everyone you autohrize is sent an email a link to download it. It bypasses all the silly itunes file copying. Testflight's reporting allows you to see who has what installed. Valuable when they start reporting problems as you can definitively say "Oh, that's because you're on a build that was so yesterday! That build was terrible! Install the NEW build, it's wonderful."

Testing with Non-Gamers
Would you rather know about an issue during development, or once it's released? Of course during development!

Testing with people who play similar games as the one you are making is very helpful. You can be sure you are meeting your demographics' demands and they can often make suggestions or give articulate feedback on issues as they have a frame of reference.

However, we found great value in testing with people who have never ever played an electronic game in their life. You know who I'm talking about: your mother-in-law whose only game experience is yahtzee on the dining room table; the friend who didn't know brick breaker was pre-installed on his blackberry.

The official term is blackbox testing. These people can confirm if your tutorials work, but more importantly they do things you never in a million years would do. But make sure you watch them closely, they won't be able to tell you what they did or did not do. Here is an example:



We finished our final bug free never crashes build Dec 15. Over the Christmas holidays I showed a non-gamer friend the finished game. Within 2 minutes of playing he crashed it.

How? He never once lifted his finger from the screen. If you recall from part 3 about our gesture system, it tracks the current finger position every 50ms. Well if the player never ever lifts their finger from the screen, it becomes one giant gesture of 2,400 points after 2 minutes (affecting performance). Even worse, the initial target object of the gesture may be destroyed while waiting for the gesture to finish, resulting in a NULL reference and therefore a crash.

It was relatively easy to replicate and fix once I saw what he did, but I have to admit I never imagined someone not lifting their finger!

How Do You Know When You are Done Testing?
This may seem like a silly question, especially coming from a business software background. In business software you would already have all your test cases written before hand (Right?) and execute them. When they all pass, you know you are good to release. Typically we would then get the customer to test the application in a pilot project, and that is the "real world" test. If it's good, we release.

Well in games, it's different. There is no "customer" to sign off and take responsibility that the application is good, you just have to decide at some point: it's done.

Release too early, and the game will crash or misbehave for customers. Release too late, and you've squandered valuable effort that could have gone into another title.

I was listening to the MIT Open Courseware on Game Design and they asked this very question: how do you know testing is done?

Their answer: When you are out of time.


When you can't take another step forward, you might be done testing.

That seemed like a cheeky answer, but having now lived it, I agree. Now, of course, we made certain all features worked, it was as fun as we could make it, and it didn't crash. (Of course now as I write this we've heard reports of a bug in one of our tutorials, oops!). The game will never be perfect, there is always more to add, more to test. There comes a point where you have to draw a line in the sand and ship it. For perfectionists, this is a very difficult thing to do. I am fortunate that I'm not working solo on this project, both the artist and I together were able to agree the game was ready to go out. That gave me confidence I wasn't deluding myself or just fed up. J For those soloing it, I recommend you ask a friend to be your "quality control" and help give you the thumbs up for releasing.

Judging the Difficulty
I read in the book Level Up: Guide to Great Game Design that the game makers are the worst people for judging the difficulty. So, I knew this going in, but it is still difficult in practice. Obviously the first people that need to be happy are the ones making the game, that is your first quality control gate.

We also tested on young children (3, 8, and 9). Why? Because they'll test for free all day long and they have nothing better to do. (And they are the artist's children). We found that young kids love to play Catch the Monkey, but the mid game was too hard for them. So thinking this was a secondary market, we created an Easy mode that gives the player more energy and reduces the maximum number of monkeys in the field at a single time. The kids loved the easy mode and were able to finish the game, so we were happy.



Later, when doing focus group testing, two of the teenage girls couldn't get past a certain level and were getting frustrated. We recommended they restart the game in easy mode. As soon as they started playing in easy mode they said "Oh wow, this is much more fun."

At this point we had a dilemma: do we make easy mode normal mode, and normal mode hard mode? We did and made all the code changes to reflect this.

Then, a few days later, we fixed a bug in the Level GM AI and found it was working much better than previously. So we flipped it all back to Easy and Normal rather than Normal and Hard.

Big mistake.

Now that we have released and friends/family have been buying it, the most common complaint we hear from casual gamers is that it is too hard. When we tell them through facebook to try it on easy mode, they always come back with "Oh wow, this is much more fun." Even post release we're still learning things the hard way!

Here are the key lessons we've learned:

• Make the game too easy, rather than too hard. Too easy can still be enjoyable, too hard never is.
  
• Casual gamers are not looking for a challenge, they are looking to pass the time. Easy fits within this expectation.
  
• Don't make the game hard with the ability for the player to opt into easy. Make it easy with the ability to opt into hard.
  

Releasing to the App Store
After making it through the arduous testing phase, we were ready to release this sucker. This has several steps:




[indent=1]1) Making a build signed for App Store, as opposed to ad-hoc provisioning build
[indent=1]2) Uploading the binary to Apple
[indent=1]3) Waiting for approval
It was time to fire up Visual Studio one last time and create an app store build. It was relatively easy to do, I simply copied the deployment options from my test flight build in Marmalade and off I went. Now here's the thing: you cannot test your app store build before you upload it. Why? Because it can only be installed on a device through the app store. So better get it right!

And we didn't.


iTunes Connect is how you control your app in the app store

I made two blunders when doing the final build. The first was somehow I didn't copy the proper icon image settings from the internal build to the final build, so it went to the app store with the default app icon. Doh! The second was far, far worse.

We knew our game didn't work on anything below iPhone 3GS or iPod 3. I saw other games show this requirement in the app store along the left column. I didn't see any way to set this through marmalade deployment, so I assumed it was done in the app store itself.

Well when you upload an app, especially your first, iTunes Connect walks you through a wizard. The answers you provide can never be changed, you got one shot to do it right. These we did right. Again, I didn't see anywhere I could set the requirements, so I figured it was after I uploaded the binary. I uploaded the binary and it went into the queue for review and approval.

Well, 8 days later, it was approved. But still no way to set the system requirements. It wasn't until later that I found out you need to modify the plist.info file to include the OpenGL ES version to 2.0 to target the devices we wanted.

No biggie, I modified the plist.info file and proceeded to upload the new binary.
ERROR!

Apple does not allow a developer to set more narrow restrictions on an application update than the app first had. So in short: if your initial version says it will run on iPod 2, you can't later do an update that makes it no longer run on iPod 2. We screwed up the one thing you cannot correct through an app update. After much back and forth with apple support we had to resort to putting the requirement right into the app description. We already know people don't read, but it's the best we can do at this point.

And one final thing to know about releasing to the app store when you make your app on a PC: you REQUIRE a mac to upload the binary to apple.

ITunes Connect used to allow you to upload through http, but no longer. There is a binary uploader program in the iOS SDK that checks and uploads your binary to apple. That uploader program only works on a mac. So while you can build and test the entire app on a PC, you need a mac for 10 minutes to upload your final binary. I've seen someone suggest just going to an apple store and using a demo mac to upload. In my case I already had the mac mini, so it wasn't terribly inconvenient, but it was a real surprise.

Releasing to the World
By default, all apps uploaded to apple are released to every itunes store in the world, unless you specifically turn a store off.


There be a lot of iTunes Stores. Fortunately you only need 6 translations to cover them all.

We had always intended to release to multiple countries, so we tried to minimize the amount of text in game and use symbols where we could (the monkey story sequences use icons rather than text for this reason, although it actually made more sense conceptually too: how do you write "monkey speak" anyway?!).

The key is to get your app description translated from your native language into the various app store languages. It costs about $100 per language to use a translation service to translate our app description. Of course, the difficulty is we have no way to judge how good the translation is!

Initial Marketing
As I write this fourth part, our game has been out in the world for 22 days. Sales haven't skyrocketed, so we're in no position to advise on how to market a game. However, there are two things we can share.

First, Apple controls the app store. They make their decisions based on volume. The sections like "What's Hot" and "New and Noteworthy" are driven by volume. The more volume you can drive in the initial days, the more likely you are to appear in those sections. Obviously the key is to get into the "Top 100 Free" or "Top 100 Paid". The only way you get there is through volume.


We've made it to #45 in the Family What's Hot. Go little monkey! Go!


Secondly, we knew review sites are important to get initial buzz going, but how do you find all the review sites out there? A google search will return some of the biggies, but also blogs that haven't been updated in 2 years. So we devised a clever way to make a short list of review sites: most games put their reviews or quotes from review sites in the top part of their app description. By clicking through about 20 apps we were able to compile a list of 41 respectable mobile game review sites.

Most if not all review sites work from a backlog of about 3-4 weeks. And they all want a promo code to get the game for free, they won't pay money for your app. Apple allows you 50 promo codes per release. Once you make a new release, the unused promo codes are invalidated.

In the 3 weeks we've been waiting, we've had 1 review come back. Fortunately it was a good one.

For our next title we'll be doing more on the pre-release marketing side to get the game buzz out before release. As we were taking so long on Catch the Monkey and we didn't really know if or when we would be done, we had to forgo pre-release marketing.

Conclusion
Well, there you have it: a summary of our ups and downs over roughly 2 years trying to make an iphone game.

We set a goal, and despite great difficulties, achieved it. Beyond this, three things have brought great satisfaction:

1. Our first review came in:

We received 4/5 stars and an editor's choice award from the family mobile gaming site famigo.com. It's nice to know someone objective thinks what we made is good!

2. A review someone wrote on the US store:
How fun can catching monkeys be? Hours of fun! I love this game because it's something for my kids to do that's different from princess games and phonics--and it's something that I can do when I'm commuting, waiting for my next appointment, or just to relax. This has got to be one of the best non-violent games I've ever seen. Great graphics, good story, and entertaining for everyone.

3. The popularity of these articles.
When we first set out to talk about our experience, we didn't know who would be interested. Over 3,000 reads and counting on the first article makes all this writing effort worthwhile! Thanks!

What's next for Mirthwerx?
We're currently working on a few things:

• Playbook version (taking full advantage of having used Marmalade)
  
• Android version (dido)
  
• Free version (different from a lite version, it's a different but similar game)
  
• And our second title which is a puzzle game for the masses (remember, turn based!)
  

I've enjoyed writing these articles, I hope they've been of benefit. I have some ideas for maybe doing an "encore" 5[sup]th[/sup] article next week, but I'd be looking for questions or comments from people before I decide to do it.

Until next we meet,
Lord Yabo