Developing a Mobile Application with Flash

As  you have undoubtedly seen in recent months, you can build mobile  applications with Flash. And you have probably seen quite a bit of drama  and degrees of support from the mobile hardware-makers ranging from “we  do not want it on our platform” to “we love it and fully embrace it”.

For  this article I am going to skip all that drama apart from how it  affects the underlying technology and talk about the process of building  a Mobile Flash app using Flash, the AIR SDK, and the mobile packager.

First off, what is AIR?

I  must start with a description of Adobe AIR because it is not an obvious  thing. At its heart, AIR is a webkit-based HTML rendering engine. It is  not, however, a web browser. It’s best thought of as a “browser  construction kit” which is used as a base for applications that look and  behave like OS-native applications even though they are built using web  client technologies (HTML, Javascript, Flash, and Acrobat) rather than  native code.

For example, let’s  say that I wanted to build a simple text editing application like  Windows Notepad or Mac TextEdit, and I wanted to build it with  Javascript and/or Flash. With a little work I could put together such a  tool (as the Mozilla “Bespin” project shows). The only problem is that  such an application would have two big deficiencies that would make it a  poor cousin of a standalone text editor:

• It would have to run in a browser, so I am stuck with the browser’s Windows and menus and toolbar buttons and URL bar and such.
• The  Javascript and Flash sandbox would preclude me from accessing the  underlying OS filesystem, so I could not open or save text files on the  computer.

Adobe AIR is  designed to get around those limitations. Using some XML-based window  description code and a class library of cross-platform “native bridge”  functions exposed to Javascript and Flash, you can give your application  a native look as well as access to much of the underlying OS that is  denied to web client technologies. Popular desktop applications like  TweetDeck are actually written in Flash and use the AIR classes and  runtime to give the appearance of being OS-native applications.

If  you have downloaded an application like TweetDeck, you might have been  presented with a file with the extension .AIR and not quite understood  what that’s about. Well, that is how the AIR runtime deals with  cross-platform executable files. Rather than have the developer maintain  multiple executable files for multiple platforms, you build a single  AIR file that’s actually a bundle of all your assets (HTML, JS, SWF,  PDF), which the AIR runtime then converts into something resembling a  first-class executable on your target platform (Windows, Mac, or Linux).  Adobe’s AIR packager handles all of the building for you, and your user  just double-clicks the AIR file to install.

And, to quote Bill Cosby, “I told you that story so I can tell you this one”.

AIR on Mobile

AIR  on a mobile platform (specifically iOS, Android, and Playbook) is a bit  different from its desktop counterpart. Not only is it different from  desktop AIR, but each device deals with them differently. Thankfully,  the differences between mobile platform AIR implementations are internal  and, with a couple of exceptions, are not things you need to worry  about.

For developers, the  biggest difference between desktop AIR and mobile AIR is that mobile AIR  is Flash-only.  While you could build a desktop AIR application  entirely out of HTML and/or Javascript and/or PDF, mobile AIR only deals  with Flash. It’s best thought of as a runtime that runs your Flash  content full-screen and with access to the AIR native-OS class library  so you can talk to the underlying phone OS and pretend you are a native  mobile application.

In addition  to standard OS stuff (files, process control, system event  notifications), the AIR class library has been extended to include  mobile OS features that most desktops lack, like geolocation, the  accelerometer, and multi-touch gestures.

AIR on Android

Android  AIR is much more like its desktop counterpart than iPhone AIR. Like  desktop AIR, it is a free-to-download runtime (available in the Android  Market for Android 2.2 “Froyo” and later) containing a Flash interpreter  and native class library. While developers use the same command-line  packager utility as the one that builds .AIR files, Android files are  built into APK (native Android executable) files. These APK files look  and behave like native Android executable files, although they require  the AIR runtime to be installed (ala .NET). If you install an  AIR-enabled executable to an Android device that doesn’t currently have  the AIR runtime installed, you will be gently ushered to the Android  Market to install the runtime. While it is not a completely invisible  process, it is just a one-time thing. Once the runtime is installed, you  are good to go.

AIR on Playbook

Playbook  AIR is even simpler, mainly because the AIR runtime is already baked  into every shipping Playbook. You use a different command-line packager  (supplied by RIM for free) to build a .BAR file that Playbooks run as  first-class applications.

AIR on iPhone

You  have undoubtedly heard of the drama surrounding the Adobe iPhone  packager. In response to Apple’s “no third-party libraries (by which we  mean virtual machines)” decree, Adobe quietly put together a method to  pre-compile iPhone native executable apps using Flash. In retaliation,  Apple altered its license to specifically exclude Adobe-authored content  (as well as several other non-Xcode development tools) from the App  Store.

Apple did eventually  soften its stance regarding non-Xcode tools, likely following some  credible legal threats, and they did start to “bless” various  non-Xcode-authored applications for the App Store. Ultimately  Adobe-authored apps were quietly paroled.  So it is once again “game on”  for building iPhone apps with Flash.

Internally,  iPhone AIR applications are quite a bit different from the the other  mobile systems, and that is mainly due to Apple’s license and its ban on  shared runtime code or bundled VM’s. Adobe’s iPhone packager is built  around an “ahead of time compiler” (what we old-schoolers used to just  call a “compiler”) for SWF files. If you request the AIR packager build  an iPhone application, it will not build a bundle that will be executed  by a runtime engine. Instead, your SWF will be precompiled into a  runtime-free iPhone executable, ready to be signed and installed to a  hardware phone.

And this  developer workflow is not unique to the Adobe Packager. Novell’s Mono  for iPhone precompiles executable applications that would normally use a  shared .NET VM on other systems.

There  are a couple of things you lose with this ahead-of-time compile  approach, necessitated by using a native code compiler rather than a VM.  For one, you bypass Xcode so your application will not run on the  iPhone simulator. Internally, the iPhone simulator is an API emulator  and not an ARM processor emulator, so it cannot run executable files  created with the packager or any other authoring tool that directly  creates ARM executable files.

Secondly,  you lose the ability to stream in and execute external SWF files. And  this can be a pretty significant problem depending on the application  you are building. If you are building a game that is entirely  self-contained and does not stream in resources as SWF, then you should  not have a problem. Some Flash-authored content, though, like the  popular Club Penguin kids’  MMORPG, is built as a “shell” that streams in all of its content  (games, assets, locations) as SWF on an as-needed basis. Such a system  is only do-able if you have a VM on the client.

But  again, if your game doesn’t plan to stream in external SWF content and  you have an actual hardware iPhone or iPod Touch for testing, there’s  little reason you shouldn’t be able to make a game in Flash.

Furthermore, you should be able to make a game that can be compiled for several mobile platforms with few or no changes.

How to Package

While it’s done a bit differently between Android and iPhone, the packaging step is conceptually similar.

• Create  a SWF file using Flash or Flex. If you want to talk to the phone’s  features, use the AIR native class library. Note that AIR only contains  the second-generation AS3 VM, so it cannot work with old AS1 or AS2  code. Unless you have an old game you are porting to mobile, this should  not be much of a problem.
• Create  an XML file containing a description of the application. This file  contains things like the application’s name, version number, path to  icons, security descriptor, orientation, etc.
• Run  the packager. In all cases, the packager is a console application that  can be run from a build script or from within the Flash IDE. The  packager takes the XML description, SWF, and any other resources (like  the icon PNG files) and builds the runtime file.
• Copy your executable to your phone and run it.

For  the Android, you will need the Android SDK installed, as it includes  the utilities to sign the code. This SDK is a free download. Since  Androids are basically “self-jailbreaking” with a check of the “allow  non-Market applications” checkbox in your device’s settings, you can  generate your own signing certificate and then copy your application to  the phone.

For iOS devices  (iPhone, iPad, and iPod Touch), you will need to be a member of the  Apple Developer Program to generate Apple-blessed code-signing  certificates. While the build and process for iOS is similar to Android,  actually generating and downloading the certificate is a weird  multi-step process that I only figured out via trial and error. Once  packaged, you copy your game IPA file to your connected test device via  iTunes.

Also note that the  iPhone packager, since it contains an extra step of running an  ahead-of-time compiler written in Java, is a lot slower than the Android  packager. Packaging an Android or Playbook application basically  entails zipping up your SWF with an executable stub, some icon  resources, and a manifest file that the packager generates, so it is a  quick process.

Building An Application

Beyond  the build process, actually writing your game is exactly like writing a  web-based game in Flash. You can build and test your game using the  Flash IDE just as you would a web game. When you are ready to deploy to a  mobile device, use the appropriate packager to build a mobile app.

But  it isn’t all easy. You do have to take into account device performance  as well as screen limitations and limitations imposed by finger-based  input.

In my experience, input  imposes the most design limitations. While Flash will tell you “The user  just tapped on point (175, 192)”, you should assume that means “The  user intended to tap on something somewhat near point (175, 192), so  look for it”. Mice and Styluses are accurate to within a pixel or two,  but fingertips aren’t. So make sure you design and code with that in  mind.

And such design decisions  are really dictated more by the game than any programming. A Tic Tac  Toe game should work on any phone screen without any changes, as the  squares would be quite large. A 20x20 Go board, on the other hand, could  be quite frustrating. The individual board-squares would be quite a bit  smaller than the smallest fingertip, and your users would likely find  themselves accidentally tapping on the wrong square.

And  games tackle this problem differently. Some games zoom in on the board  when it is time to click on a cell. Some games scroll around. Some games  pop up tips explaining what you’re about to click (like most mobile  keyboards). It is probably best that you play a few mobile games before  you settle on your input scheme, as a natural feeling input scheme is  often the difference between an excellent game and a frustrating one.

For my example, I designed my game “Hexapies”, which is a simple “paint the board” game, to be mobile-friendly.

Figure 1: Hexapies

The  input in the game is quite simple, consisting of five large colored  pies at the bottom of the screen. Clicking on a pie colors the “seed”  pie, connecting to any adjacent pies. Pies that aren’t available for  tapping partially hide themselves so it’s obvious.

Note  that the click-able pies are along the bottom of the board. This  placement wasn’t chosen randomly. Keeping the input items at the bottom  keeps your hand from obscuring the board every time you make a move. If  you want an example of how placement can hamper gameplay, try out any of  a hundred Windows Solitaire knockoffs for mobile phones. Virtually all  of them keep the same layout as Windows Solitaire even though such a  layout is not mobile friendly. Every time you drag a card, you cover the  entire board with your hand, and you find yourself having to drag the  card off to the side to find your destination. A much better solution  would be to have the cards along the bottom and have them “grow”  upwards, but few games have that option, opting instead for familiarity  with the original.

Real Production!

Okay,  enough pontificating. Here is a real project. One game I have been  developing and re-developing forever is “Brain Bones”. It is a dice game  that I thought it would be a good fit, as the screen is about the right  size for mobile, and the controls are simple -- you have five dice to  swap around and a big “play this combination” button. So I decided to  make a mobile-friendly version as well as a version that I could move to  various platforms. Here is the game in production in the Flash IDE.

Figure  2: Brain Bones in Flash. My biggest tip for using the Flash IDE is to  put “one really bigass monitor” in your budget, as there are a lot of  panels you will want to use.

While  I like the idea of building stuff directly from the IDE, it really was  not practical for the amount of building I was doing. I ended up writing  a big batch file and some template XML files. My batch file then ran  all of the command-line bits, most notably the AIR packager ADT, to  package each game.

(and yes, I know I should be more state-of-the-art and use ANT. I’ll learn it soon. I promise)

And  then I tested. Most testing involved building features that I could  turn on and off for each platform. For example, you may notice that my  desktop games shown below have a volume-slider on the screen and the  mobile games do not. This is because most mobiles have hardware volume  buttons, and it is not good form to duplicate that mobile feature.

Also,  Androids have a hardware “back” button that I could override and use as  a “return to main menu” function, so I wrote a little code for that and  enabled it on those phones.

Brain Bones running on the Playbook Simulator.

Brain Bones running on a Motorola Droid

Brain Bones running on an iPod Touch 4

Brain Bones running on a Barnes & Noble Nook Color

Brain  Bones running on Windows as an AIR application. Note the volume slider  in the lower right. It’s not there on devices with hardware volume  buttons.

Brain Bones running on a Mac as an AIR application

And finally, just to be gratuitous, the Brain Bones SWF running in the Chrome web browser as a web app.

Things I Learned

ImageMagick Is A Good Thing -  One thing I learned early on is that every platform has its own icon  requirements. While all icons were the same format (PNG), every platform  had its own preferred sizes. If I didn’t want to spend a half hour in a  paint program resizing icons every time I made a change, I needed a way  to automate this.

And ImageMagick  fit the bill. ImageMagick is a free set of cross-platform command-line  tools to do stuff with images. And one of those tools is “convert”,  which resizes images and saves them back out.

The  largest icon size I would need is 512x512 pixels (No, really. The iPad  uses an icon this big), so I drew all my icons this big and made a  little batch file that shrunk that mega-icon to the other NINE sizes I  needed.

Dropbox is great for Android, iOS not so much- The final step in my batch-build was to copy every runtime, AIR, APK, BAR, and IPA file to my Dropbox  folder. From there I could open the Dropbox app on my Android test  devices and directly download and install the APK file, thus removing  the need to cable all my Android test devices to my computer every time I  wanted to test the game.

And  while the iOS Dropbox app could see my brainbones.IPA file just fine,  iOS flat refused to install or run it. Apple’s policy is that the only  way to put an IPA file on a test device is via iTunes, and they’re  sticking with that.

Installing  to the Playbook Simulator was a matter of putting the simulator into  “developer mode” and telling the Playbook BAR packager to send the BAR  file to the Simulator’s IP address. This process worked very simply,  although it RIM has made the process into a more complicated  “development certificate” affair with actual hardware devices.

(and if you don’t yet have a dropbox account, go get a free one here, as I get a little extra space as a referral bonus, thankyouverymuch)

Flash Professional CS5.5

Recently  shipped is the new Adobe CS5.5 suite, featuring improvements and new  features in several tools. The bulk of the new features and improvements  are designed to improve development for mobile devices. I recently got a  one-on-one with someone from the Flash team and got a sneak-peek at the  new features.

Here are the ones I found that stood out as being helpful for my own development process.

My Flash CS5.5 Likes

(note that Flash CS5.5 is being released as this article is being published. I should have a more in-depth review in a bit)

Incremental Compilation- While it’s a good idea to store your sound files in an uncompressed  high-bitrate format, that’s not good for deployment. 44 kHz stereo WAV  files sound great and don’t lose any sound information, but they’re way  too big for mobile or web-streaming.

Flash  has long had the ability to store uncompressed sounds in its project  file and then compress them at compile-time to a compact format. You can  tell the compiler to compress all sounds to the same format and  bitrate, or you can compress each sound individually. If you are setting  compression levels for each sound, you can listen to sounds without  recompiling, so you can find a good balance between quality and size for  each individual sound. Then the sounds are compressed into the SWF at  compile-time for production testing.

Problem  is, compressing and resampling big WAV files into compact lossy formats  is slow, and the tradeoff for this convenience is a long wait during  compiles. About 95% of the compile-time for my Bulldozer game is  compressing the game’s theme music into a low-bitrate mono MP3.

Flash  CS5.5 finally added incremental compilation for sound and font files.  Rather than re-rendering sounds and fonts every compile, the compressed  results are now cached in the project file. If you, like me, like the  convenience of letting Flash compress your sounds at compile-time, this  is most welcome.

Export As Bitmap -  If you read the development blogs from fellow mobile Flash developers,  you have learned to avoid doing a lot of vector stuff in mobile Flash.  While Flash was originally designed as a vector renderer, Flash also  handles bitmaps just fine.

Problem  is, Flash does not have a bitmap editor. If you want a game with bitmap  space invaders, you will need to draw them outside Flash and import  them. You can leave your invaders as vectors and use the Flash engine’s  “cache as bitmap” feature, but that still leaves the work of rasterizing  to the Flash runtime. The new “Export As Bitmap” feature converts  vector objects to bitmaps at design-time rather than run-time.

Visible Property -  This is one of those “why didn’t they implement this six versions ago”  features. While Flash DisplayObjects have had a “visible” attribute  since forever, the attribute was only something that you could set at  runtime. If you place an object on the stage at design-time, but you  want it to be invisible (like a wall at the edge of the screen for  collision detection), you had two options:

• Set  the alpha property to zero. This works, but it’s not good practice  because, while you can’t see the object, it is still in the rendering  pipeline and is included in rendering and bounding-box calculations. And  if you are developing for a CPU-constrained platform, you do not want  to spend time drawing things you can’t see.
• Set  the visible property to false at runtime. This is much better practice  than setting alpha to zero, but you have to remember to code for it.

You now have a “visible” checkbox in the property inspector, so this minor hassle is now gone.

Better Publish-For-Mobile dialogs- While I chose to build my XML descriptor files manually, Flash does  have a couple of “wizard” style dialogs where you can set your  descriptor properties in a friendlier fashion. The Android one has been  improved to allow Flash to directly deploy and run an APK file on a  USB-connected device without moving to the command line.

And if it seems odd that you can target Android, iOS, Windows, Mac, and Linux, but not Playbook, this will be addressed soon.

Shared Libraries- All of the graphics and sounds that make up your game exist in your  Flash project in the “Library”. While I ended up making all of my  platforms out of a single project, I could have broken it up into more  than one project. For example, if I wanted to “tune” my graphics for  each platform or have higher bitrate sounds on higher quality devices, I  could have done that with multiple projects. Problem is, each project  has its own assets, so anything updated in one project would need to be  updated in the rest. Flash CS5.5 addresses this by allowing you to share  and synchronize symbols among multiple documents inside a project  folder.

My Flash CS5.5 Wish List

Components -  While Flex is going to see much-improved mobile components (no surprise  there. 90% of Flex IS components), Flash’s current set of “works good  on the desktop, but not on your phone” components are still there.

I  was told that the plan is to usher people to some quality third-party  mobile components for Flash. I would like to see some more effort on  this, as things are currently fragmented. For example, AS3Flobile has a  nice Android skin (which I have tried), and MinimalComps has an iOS skin  (which I have not yet tried), but the two libraries are incompatible  and would require me to support two different codebases to use both.

These  third-party component libraries are nice, but I haven’t found one yet  that’s complete to my satisfaction. And by that, I mean skinned for  desktop, iOS, Android, and Playbook so that I can make my big collection  of Brain Bones’s look and work even better on each device with nice  native-looking controls on each.

Conclusion

Flash  has definitely arrived on mobile. Pretty-much every mobile device out  there supports Flash to one extent or another (except Windows Phone 7,  what gives?)

It’s not only possible to quickly build a high-quality multiplatform game with Flash, but it’s not all that difficult.