Introduction
It was only a matter of time until someone decided to put a 3d graphics API onto a phone handset. OpenGL has long been a graphics industry standard for 3d, and now OpenGL ES is fast becoming the standard for 3d on limited devices. Limited devices is an apt description though, even a high end phone might only have a 50Mhz ARM processor and 1MB of memory. But don't be put off, even with these limitations you can still create some very impressive games.

Writing games for mobile phones is unlike writing for the PC. With the design more limited by the platform restrictions you don't need a huge team with multiple programmers and an army of artists, its well within reason for a single person to turn out a quality title from the comfort of their bedroom.

This article will go from installing and setting up a BREW development environment and emulator, through to getting an Open GL ES system up and running and displaying a single triangle. From there existing OpenGL resources can take you further into the process of developing your 3d application.


Installing the BREW SDK
You really need to use Internet Explorer for this process. The BREW SDK is installed by an ActiveX control which only seems to work in Internet Explorer 6 or better. During this article I am going to assume you create a c:\BREW directory, and then install the BREW SDK into c:\BREW\BREW 3.0.1. If you want to install it somewhere else (like the default c:\program files\BREW 3.0.1), then just adapt the paths I mention as you proceed.

First, go here and register for a free BREW developer account, then install the BREW 3.0.1 SDK from here. Its a web based installer, just start it going, give it a directory to install to and wait. At around 20MB it won't take too long to install, even on a 56k modem. Towards the end it will ask if you want it to set a BREWDIR environment variable. Say yes or various things won't work correctly.

From this page, install the Visual C++ addon, and download the BREW SDK Extension for OpenGL ES. Extract the OpenGL ES zip file, and:

• Move all files from inc into c:\BREW\BREW 3.0.1\sdk\inc
• Move all files from src into c:\BREW\\BREW 3.0.1\sdk\src
• Move the dll from BREW 3.x into c:\BREW\BREW 3.0.1\sdk\bin\modules
• Move all files from devices into c:\BREW\BREW 3.0.1\sdk\devices

Directory structure
BREW is a bit tricky sometimes when it comes to where it expects to find various files, and tends to give the same cryptic error message for pretty much any case of missing or misplaced files. Below is how I have my machine setup. For this article I am assuming you have the same setup, again if you install BREW somewhere else just substitute your paths as appropriate.

c:\BREW\ My BREW root directory c:\BREW\BREW 3.0.1\ The 3.0.1 SDK. You can have several SDKs installed at once and chose between them by setting an environment variable c:\BREW\project1\ A project directory c:\BREW\project1.mif The MIF file for project 1 (note that its here and not inside the project1 directory, very important)
Create new project

• Select the Brew App Wizard (under Visual c++ projects)
• Set the "location" to c:\BREW
• Enter a project name. Make it lower case with no spaces or special symbols, lets choose "test_project1" (this is what I'll refer to through out the article as the project name)
• Hit OK, and then Finish without making any changes on the Wizard dialog or running the MIF editor
• It will probably say that the project has been modified outside Visual C++, so say OK to reload

MIF editor

• Run the MIF editor. On the Visual C++ BREW toolbar (which should be on by default, if not right click in the toolbar are and enable it) its third button
• Click the new button (NOT File -> New)
• Assuming your not a licensed BREW developer you will need to generate a class id locally. Select that option.
• Make up a class id, I started at A0000001 and went up as I created more projects. Pick anything, but if you create more projects they must have unique class id's
• Enter your project name as the class name, so in our case "test_project1"
• Click OK, and you will be prompted to save. Save into the project directory, so c:\BREW\test_project1\test_project1.bid
• File->save, save into the PARENT of the project directory, so c:\BREW\test_project1.mif
• Now compile the MIF by choosing Build -> compile MIF script. Click OK a couple of times and you're done.

Setting up to run and debug through Visual Studio

• Right click on the project in solution explorer to get up project properties
• Configuration properties -> Debugging -> Command, Select BREW_Simulator.exe in your BREW SDK bin directory (in my case, C:\BREW\BREW 3.0.1\sdk\bin\BREW_Simulator.exe)
• Configuration properties -> Linker -> Debugging, Change "Generate debug info" to "Yes (/DEBUG)"

Compile and run the project. It should compile with no errors and start the emulator. If you get compile errors you probably didn't set your class name in the MIF editor to exactly the same as the project name.

Now if you set a break point in your code it will get triggered correctly when the emulator is running your dll.


The emulator
Select Blackcap16 as your emulator profile. From File -> Load Device browse to the devices directory of the SDK and select Blackcap16.qsc. It remembers which device you are using, so you will only have to do this the first time you run the emulator.

In the emulator, File -> Change applet directory. Set it to the directory that contains your .mif file and your project directory, for me thats c:\BREW.

You should now see the emulator with two icons, your projects and Settings. Use the arrow keys to select which application you want to run, and enter to start it. When you run your project it looks like nothing happens! Thats because the app wizard only generates boiler plate start up code for you. You should see in the output window of Visual C++ a message saying that your dll was loaded, assuming it does and you get no errors, success!

If you get a message saying "This app was disabled to save space. Would you like to restore it now?", that's the cryptic message I mentioned earlier. It almost always means you have your files in the wrong places (probably the .dll in the wrong place.) Assuming you used the app wizard to generate your initial code, check you saved the .mif file into the right place.


The best laid plans...
Coding for limited devices like mobile phones can be a nightmare, especially if you come from a PC background and are used to luxuries like having more than a few hundred kb of heap, and more than a few hundred bytes of stack space.

Although it must be a Design Patterns advocates wildest fantasy, there is no static or global data on BREW. Also, BREW is completly event driven. Unlike "normal" programming where you typically have a while(..) loop to do your stuff, with BREW you can only respond to events like key presses or timers going off. There's also no floating point math, GL ES expected its values in 16.16 fixed point format. I'll address each of these in turn.


Storage space
So with no global or static data, where do we store our variables? BREW will store a single struct for us, which must first contain an AEEAplet structure, but can then contain any other data we want. Check out the main .c file the app wizard created for you. Right at the top is a structure named after your application, and in the AEEClsCreateInstance function is a call to AEEApplet_New which allocates heap space for it. BREW will look after a pointer to this data for us, and will pass it to us as a parameter to most things. I am going to refer to this as "the global BREW structure".

Some people like to just put all their data straight into that structure. However I prefer a slightly more oo approach.

Assuming you are going to write more than one BREW application you want to structure your startup/shutdown/event handling code into a shell so you dont have to rewrite it for every single application you create. First, change your main .c file to a .cpp file so it compiles as C++ (else you will get errors using classes). Create a class called Game with functions boolean Create(), void Destroy() and void Tick(int timeElapsed). Add an instance of Game into your global BREW data structure, right after AEEApplet, and remove the other data from the struct. I have also added int mOldTime which will will use later to track the elapsed time between frames.

// From test_project1.cpp struct test_project1 { AEEApplet a; // The compulsory applet structure Game mGame; // Our game class int mOldTime; // used to track the speed we are running at }; // From Game.h class Game { private: IShell * mShell; IDisplay * mDisplay; AEEDeviceInfo mDeviceInfo; public: boolean Create(IShell * shell, IDisplay * display); void Destroy(); void Tick(int timeElapsed); IShell * GetShell() { return mShell; } IDisplay * GetDisplay() { return mDisplay; } int GetWidth() { return mDeviceInfo.cxScreen; } int GetHeight() { return mDeviceInfo.cyScreen; } }; The AEEDeviceinfo structure contains various information about the current phone and operating environment, most importantly for now it contains the width and height of the screen. Given that virtually all phones have different sized screens you should try and adapt to the screen size at run time. That way your program will have a chance to work on several phones without recompiling.

// From Game.cpp boolean Game::Create(IShell * shell, IDisplay * display) { mShell = shell; mDisplay = display; mDeviceInfo.wStructSize = sizeof(mDeviceInfo); ISHELL_GetDeviceInfo(mShell, &mDeviceInfo); DBGPRINTF(" *** Width %d, Height %d", GetWidth(), GetHeight()); return TRUE; } void Game::Destroy() { } void Game::Tick(int timeElapsed) { // Uncomment this if you want proof the timer callback is working //DBGPRINTF("TICK! %d", timeElapsed); } The DBGPRINTF function to output text, either to the Visual C++ output pane if you are running in the debugger, or to a window within the emulator. To get access to it you need to include AEEStdLib.h. For now Destroy() doesn't do anything, as you add more functionality you can use it to clean up any resources you allocate.

Now to wire these up. Replace the contents of test_project1_InitAppData with a call to Game::Create, and make a call to Game::Destroy in test_project1_FreeAppData. Both of these functions are passed a pointer to the global BREW data structure, so you have easy access to the instance of your Game class. The other parameters you need are available through the AEEApplet stored within the global BREW structure.


Timers
Every thing in BREW is event based. If you were to try and remain in the startup function forever with a while loop after a few seconds the phone would reboot. BREW detects applications that have stopped responding (in its opinion) and forces a full reboot to try and clear the problem.

To get an application to run in a style resembling a real time game we use a timer to repetedly execute our game loop. BREW makes it really easy to set up a timer to callback a function of our choice with ISHELL_SetTimer. ISHELL_SetTimer takes four parameters, a pointer to the applications IShell (which is now contained in our Game class), the number of miliseconds in the future you want the function called, a pointer to a function to call, and finally a void * to some data you want passed to the callback.

The callback function needs to take a void pointer as a parameter and return void. I usually cast the address of global BREW structure to a void * and use that as my user data, that way in the callback function I can call Game::Tick(int timeElapsed). One thing to note is that timer callback functions are one shot wonders. If you want the callback to happen again you need to set the timer again.

// From test_project1.cpp void SetTimer(test_project1 * data); void timer_tick(void * data) { test_project1 * tp1 = static_cast < test_project1 * > (data); int TimeElapsed = GETUPTIMEMS() - tp1->mOldTime; tp1->mOldTime = GETUPTIMEMS(); tp1->mGame.Tick(TimeElapsed); SetTimer(tp1); } void SetTimer(test_project1 * data) { int result = ISHELL_SetTimer(data->mGame.GetShell(), TickTime, timer_tick, static_cast < void * > (data)); if (result != SUCCESS) { DBGPRINTF(" *** SetTimer failed"); } } GETUPTIMEMS() returns the number of miliseconds the phone has been on. TickTime is a constant that specifies how often (again in miliseconds) to call the main loop. Its calculated based on the FPS you want, like this:

// From test_project1.cpp const int WantedFPS = 20; const int TickTime = 1000 / WantedFPS; The only thing that remains is to set the timer going for the first time. Do this from the event handler funtion in test_project1.cpp. The function is called test_project1_HandleEvent. Add a call to SetTimer(pMe); to the EVT_APP_START case. This will get called by BREW when (if) your create function has successfully completed.


Fixed Point Math
The ARM chips that power most BREW phones have no floating point units. Instead they use a format called 16.16 fixed point. The 16.16 refers to taking a 32 bit variable, using the first 16 bits for the whole part of a number, and the last 16 bits for the fractional part.

To convert an int to 16.16 format, simply shift it left 16 places. To convert it back, shift the other way. A full fixed point tutorial is outside the scope of this article, but there are plenty of resources on the internet. All you need for this article is a macro to convert numbers to fixed point.

// From Game.h #define ITOFP(x) ((x)<<16)
Input
We recieve an event to our event handler function when a key is pressed, and another when it is released. Its up to us to track which keys are down at any given time.

To make things slightly more intresting the key codes used dont start at 0. They start at a constant called AVK_FIRST, and end at AVK_END. AVK is the prefix for the key codes too, so the 3 key would be AVK_3, the direction keys are AVK_UP, AVK_DOWN, etc. Check out aeevcodes.h for a complete list.

Lets add an array to our Game class to track the state of keys, and two functions to be called when we recieve key press and release events.

// From game.h class Game { ... boolean mKeysDown[AVK_LAST - AVK_FIRST]; ... void KeyPressed(int keyCode); void KeyReleased(int keyCode); ... In Game::Create loop through and set all the mKeysDown[..] to false so we start with a blank slate. The implementation of KeyPressed and KeyReleased is simple enough, just remember to take into consideration the key codes starting at AVK_FIRST not 0.

In your startup file, in your event handler function test_project1_HandleEvent, in the switch statement replace the whole case EVT_KEY with the following to route key events into the Game class.

// From test_project1.cpp ... case EVT_KEY_PRESS: pMe->mGame.KeyPressed(wParam); return TRUE; case EVT_KEY_RELEASE: pMe->mGame.KeyReleased(wParam); return TRUE; ... Now in game code you can test if (mKeysDown[AVK_UP - AVK_FIRST] == TRUE). Again, don't forget to take into account the offset of AVK_FIRST. It would probably be best to write a wrapper function to do the test which handles the offset internally.

If you compile and run now you should see the code from Game::Create printing out the width and height of the screen to the Visual C++ output pane.


OpenGL ES 1.0
At last we come to the actual topic of this article, OpenGL ES. It taken a while to get here mainly because for a lot of people this will be their first non-PC programming target.

If you have any experience with writing Open GL code you will find your knowledge translates nearly directly to GL ES. There are however a few differences. The startup and shutdown sequence is different from PC based Open GL. Given that there are no floating point functions things have slightly different names. Most of the names simply replace the trailing f (for float), with an x (for fixed). The most significant difference is GL ES does away with the immediate mode glVertexf interface. All renderering is done through the batched glDrawElements interface for improved efficiency.

To get access to the GL ES functions and data types you need to include IGL.h in your code. You will also need to add the file GL.c, which came with the GL ES SDK to your project. Its located at c:\BREW\BREW 3.0.1\sdk\src\GL.c.


The Renderer class
Keeping with the oo theme, all the setup and shutdown code is gathered into a class called Renderer. Take a look at the class definition.

// From Renderer.h class Renderer { private: IGL * mIGL; IEGL * mIEGL; EGLDisplay mDisplay; EGLConfig mConfig; EGLSurface mSurface; EGLContext mContext; public: boolean Create(IShell * shell, IDisplay * display); void Destroy(); void FlipToScreen(); }; IGL is an Interface to GL, while IEGL is a platform specific layer to sit between IGL and the underlying architecture.

The other parameters are just as their names suggest. EGLDisplay is the graphics display, EGLConfig is the video mode (there is normally only one mode available, as opposed to a PC graphics card which might have several to choose from). EGLSurface is the actual surface rendering operations write to. EGLContext represents the current state of the GL environment that will be used when you execute commands.


Renderer::Create(..)
Throughout this function its very important to check every function call for errors, and to completely clean up if anything goes wrong. On a PC its maybe a bit annoying if a program spews garbage and you have to reboot, but I have heard several stories of phones locking up and having to be sent for repair after particularly nasty code errors.

// From Renderer.cpp if (ISHELL_CreateInstance(shell, AEECLSID_GL, (void **)&mIGL) != SUCCESS) { Destroy(); return FALSE; } if (ISHELL_CreateInstance(shell, AEECLSID_EGL, (void **)&mIEGL) != SUCCESS) { Destroy(); return FALSE; } IGL_Init(mIGL); IEGL_Init(mIEGL); Using the ISHELL interface we get BREW to create IGL and IEGL objects for us. The IGL_Init() and IEGL_Init() functions are part of a wrapper system that stores pointers to the IGL and IEGL so we can just call the more usual glClear(..) rather than IGL_glClear(mIGL, ...).

// From Renderer.cpp mDisplay = eglGetDisplay(display); if (mDisplay == EGL_NO_DISPLAY) { Destroy(); return FALSE; } Get the GL display, based on the current BREW display.

// From Renderer.cpp EGLint major = 0; EGLint minor = 0; if (eglInitialize(mDisplay, &major, &minor) == FALSE) { Destroy(); return FALSE; } DBGPRINTF(" *** ES version %d.%d", major, minor); Initialize GL ES, which also sets major and minor to the major and minor version numbers of the current GL ES implementation. At the moment that is going to always say 1.0, but version 1.1 is coming soon. In the future it will be worth checking this the same way you check for various extensions in GL to be able to use more advanced features if they are available. If you really don't care, you can pass NULL for the last two parameters to not retrieve the version information.

// From Renderer.cpp EGLint numConfigs = 1; if (eglGetConfigs(mDisplay, &mConfig, 1, &numConfigs) == FALSE) { Destroy(); return false; } Retrieve a valid configuration based on the display.

// From Renderer.cpp IBitmap * DeviceBitmap = NULL; IDIB * DIB = NULL; if (IDISPLAY_GetDeviceBitmap(display, &DeviceBitmap) != SUCCESS) { Destroy(); return FALSE; } if (IBITMAP_QueryInterface(DeviceBitmap, AEECLSID_DIB, (void**)&DIB) != SUCCESS) { IBITMAP_Release(DeviceBitmap); Destroy(); return FALSE; } Using the BREW IDISPLAY interface, get the current device bitmap. From this, use the IBITMAP interface to query for a device dependant bitmap (a bitmap in the native phone format). This will be our front buffer.

// From Renderer.cpp mSurface = eglCreateWindowSurface(mDisplay, mConfig, DIB, NULL); IDIB_Release(DIB); IBITMAP_Release(DeviceBitmap); if (mSurface == EGL_NO_SURFACE) { Destroy(); return FALSE; } Create the surface we will be rendering to. This is our back buffer which when we issue an eglSwapBuffers will be copied to the font buffer. We can release the bitmaps we acquired earlier, they have served their purpose.

// From Renderer.cpp mContext = eglCreateContext(mDisplay, mConfig, NULL, NULL); if (mContext == EGL_NO_CONTEXT) { Destroy(); return FALSE; } if (eglMakeCurrent(mDisplay, mSurface, mSurface, mContext) == FALSE) { Destroy(); return FALSE; } Create a context, and then lastly make our display, surface and context current so they are the target of any rendering we do.

Assuming we got this far with no errors, the basic GL ES system is up and ready to be used.


Renderer::Destroy
I have mentioned the importance of cleaning up correctly several times, so lets take a look at the Destroy function that takes care of shutting everything down.

// From Renderer.cpp eglMakeCurrent(EGL_NO_DISPLAY, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT); if (mContext) { eglDestroyContext(mDisplay, mContext); mContext = NULL; } if (mSurface) { eglDestroySurface(mDisplay, mSurface); mSurface = NULL; } if (mDisplay) { eglTerminate(mDisplay); mDisplay = NULL; } if (mIEGL) { IEGL_Release(mIEGL); mIEGL = NULL; } if (mIGL) { IGL_Release(mIGL); mIGL = NULL; } First we deactivate our display, surface and context, then take each in turn and destroy or release them depending on how they were created.


Renderer::FlipToScreen
We are nearly finished with the Renderer class now, lets take a look at the final function FlipToScreen, and then move onto actually getting something on screen.

// From Renderer.cpp void Renderer::FlipToScreen() { eglSwapBuffers(mDisplay, mSurface); } That is the entire function, it just calls eglSwapBuffers to copy our backbuffer to the screen.


A Spinning Triangle
Add an instance of Renderer to the Game class. Also add an int called mRotateAngle to record the current rotation of the triangle. In Game::Create, at the end, we have this:

mRenderer.Create(mShell, mDisplay); // Enable the zbuffer glEnable(GL_DEPTH_TEST); // Set the view port size to the window size glViewport(0, 0, GetWidth(), GetHeight()); // Setup the projection matrix glMatrixMode(GL_PROJECTION); glLoadIdentity(); // Diable lighting and alpha blending glDisable(GL_LIGHTING); glDisable(GL_BLEND); // Set the fustrum clipping planes glFrustumx(ITOFP(-5), ITOFP(5), ITOFP(-5), ITOFP(5), ITOFP(10), ITOFP(100)); // Set the model view to identity glMatrixMode(GL_MODELVIEW ); glLoadIdentity(); // Enable the arrays we want used when we glDrawElements(..) glEnableClientState(GL_VERTEX_ARRAY); glEnableClientState(GL_COLOR_ARRAY); mRotateAngle = 0; This initializes our renderer with our stored ISHELL and IDISPLAY. It sets various initial GL states. Note the use of the ITOFP macro to convert values into 16.16 fixed point. Start the triangle with no rotation, facing the camera. Don't forget to add a matching call to Renderer::Destroy to Game::Destroy to clean up when the program exits.

glClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT ); glPushMatrix(); glLoadIdentity(); glTranslatex(0, 0, ITOFP(-15)); if (mKeysDown[AVK_LEFT - AVK_FIRST] == TRUE) { mRotateAngle -= 3; } if (mKeysDown[AVK_RIGHT - AVK_FIRST] == TRUE) { mRotateAngle += 3; } if (mRotateAngle < 0) mRotateAngle += 360; if (mRotateAngle > 360) mRotateAngle -= 360; glRotatex(ITOFP(mRotateAngle), ITOFP(0), ITOFP(1), ITOFP(0)); int FaceData[9] = { -ITOFP(2), -ITOFP(2), ITOFP(0), // First vertex position ITOFP(2), -ITOFP(2), ITOFP(0), // Second vertex position -ITOFP(0), ITOFP(2), ITOFP(0) // Third vertex position }; int ColorData[12] = { ITOFP(1), ITOFP(0), ITOFP(0), ITOFP(0), // First vertex color ITOFP(0), ITOFP(1), ITOFP(0), ITOFP(0), // Second vertex color ITOFP(0), ITOFP(0), ITOFP(1), ITOFP(0) // Third vertex color }; uint8 IndexData[3] = {0, 1, 2}; glVertexPointer(3, GL_FIXED, 0, FaceData); // Set the vertex (position) data source glColorPointer(4, GL_FIXED, 0, ColorData); // Set the color data source glDrawElements(GL_TRIANGLES, 3, GL_UNSIGNED_BYTE, IndexData); // Draw the triangle glPopMatrix(); mRenderer.FlipToScreen(); As I said earlier if you have any existing OpenGL knowledge this should look very familiar to you. Note again the use of the ITOFP macro to convert values to 16.16 fixed point, and the introduction of the new data type GL_FIXED as a parameter to various functions.


Adding Text
We can use the BREW functions to draw text to screen, as long as we do it after ending GL rendering for the frame. At the end of Game::Tick, after calling mRenderer.EndFrame(), add this:

AECHAR buffer[16]; WSPRINTF(buffer, 16, L"FPS: %d", 1000 / timeElapsed); IDISPLAY_DrawText(mDisplay, AEE_FONT_BOLD, buffer, -1, 5, 5, NULL, 0); IDISPLAY_Update(mDisplay); The last call, IDISPLAY_Update only needs to be called once at the very end for however much text or other data you want to put on screen. BREW is entirely UNICODE (except for filenames), so we need to use the wide version of sprintf. To declare a string constant as a wide string simply precede it with an L.


Run!
If you compile and run you should have a triangle with three different colored corners. The FPS should be in the top left corner, and pressing the left and right arrows (either use the keyboard, or click the emulator buttons) should rotate the triangle. Congratulations, you just wrote your first Open GL ES program!


Conclusion
Hopefully if you have been following along you have managed to install the BREW SDK, set up the emulator and have built your first OpenGL ES program.

Once you have OpenGL ES up and running you can use nearly any existing OpenGL books or websites for information. Just bear in mind the restrictions of the hardware, and don't forget to convert all your values to 16.16 fixed point!


Source
The source code, including a Visual C++ 2003 project file, that accompanies this article is available here (10k).


Where now?
I hope you are aware of the great contest Gamedev.net and Qualcomm are running. This article provides enough information to get you started writing the contest winner, and next cult classic 3d game for mobile phones.


Further reading
Books
OpenGL ES Game Development by Dave Astle and Dave Durnil

Contest
Gamedev.net & Qualcomm OpenGL ES development contest

BREW
Register for a free BREW developer account
Install the BREW SDK
Get the BREW GL ES SDK (and Visual C++ addon)
BREW developer forums

Open GL ES
Official OpenGL ES web site


Open GL
OpenGL.org
NeHe OpenGL tutorials

Fixed Point Math
Fixed point math article
More fixed point math

"The road must be trod, but it will be very hard. And neither strength nor wisdom will carry us far upon it. This quest may be attempted by the weak with as much hope as the strong. Yet such is oft the course of deeds that move the wheels of the world: small hands do them because they must, while the eyes of the great are elsewhere." J.R.R. Tolkein, The Fellowship of the Ring

[email="gamedev@alankemp.com?subject=GL%20ES%20Article"]Email me[/email] | 0AD: A Historical RTS