Okay, this is my perspective on game engine design (at least the graphic part). I'm assuming you already have mastered your desired programming language (i.e. C/C++, C#, Java, Objective-C, whatever) and already know how to use your desired compiler to build programs/apps for your target platform (if you don't know these things, please specify), so IMO the best thing to do when starting off is to do a bit of brainstorming about what functionality you'd like in your 2D rendering engine. Examples, 2D sprites, animated sprites, scaling, rotation, z-sorting, sprites that flash a certain colour when you shoot it, transparency, colour keying, etc. After you have all of that down, create like a flow-chart or some pseudocode to get a visual idea of what your engine and APIs would look like. Then you'll be better prepared to write your 2D engine.
What I won't assume is that you are familiar with OpenGL (or at least not an advanced user yet), but it would help if you could share with us how much programming knowledge and/or OpenGL experience you have already... so I'll go ahead and give you some source examples. I'll start off by giving you a fixed-pipeline example of how to set up a 2D orthographic view:
//--------------------------------------------------------------------------------------
// Name: glEnable2D
// Desc: Enables 2D rendering via Ortho projection.
//--------------------------------------------------------------------------------------
GLvoid glEnable2D( GLvoid )
{
GLint iViewport[4];
// Get a copy of the viewport
glGetIntegerv( GL_VIEWPORT, iViewport );
// Save a copy of the projection matrix so that we can restore it
// when it's time to do 3D rendering again.
glMatrixMode( GL_PROJECTION );
glPushMatrix();
glLoadIdentity();
// Set up the orthographic projection
glOrtho( iViewport[0], iViewport[0]+iViewport[2],
iViewport[1]+iViewport[3], iViewport[1], -1, 1 );
glMatrixMode( GL_MODELVIEW );
glPushMatrix();
glLoadIdentity();
// Make sure depth testing and lighting are disabled for 2D rendering until
// we are finished rendering in 2D
glPushAttrib( GL_DEPTH_BUFFER_BIT | GL_LIGHTING_BIT );
glDisable( GL_DEPTH_TEST );
glDisable( GL_LIGHTING );
}
//--------------------------------------------------------------------------------------
// Name: glDisable2D
// Desc: Disables the ortho projection and returns to the 3D projection.
//--------------------------------------------------------------------------------------
void glDisable2D( GLvoid )
{
glPopAttrib();
glMatrixMode( GL_PROJECTION );
glPopMatrix();
glMatrixMode( GL_MODELVIEW );
glPopMatrix();
}
This code should be really easy to understand once you understand OpenGL well enough. Starting in glEnable2D, it grabs a copy of the current viewport and uses it to set up the orthographic projection. Before setting up the orthographic projection, it saves a copy of the previous matrix (assuming it's your 3D perspective matrix) and pushes it onto the stack. Then it saves a copy of the modelview matrix and also pushes it to the stack before setting it to the identity matrix. Lastly, it disables lighting and depth testing so that it doesn't interfere with any 3D rendering that has already taken place (i.e. HUD). After that you're ready to start drawing in 2D! When finished, call glDisable2D. It restores the previous render states that were modified by glEnable2D and also restores the projection and modelview matrices to their original state and taking them off the stack.
I'll give you an example of how to draw a 2D sprite also.
//--------------------------------------------------------------------------------------
// Name: draw_quad
// Desc: Renders a 2D quad to the screen (using a triangle fan).
//--------------------------------------------------------------------------------------
void draw_quad( GLuint texture, float x, float y, float w, float h )
{
float vertices[] = { x, y, x+w, y, x+w, y+h, x, y+h };
float tex[] = { 0.0f, 1.0f, 1.0f, 1.0f, 1.0f, 0.0f, 0.0f, 0.0f };
glBindTexture( GL_TEXTURE_2D, texture );
glEnableClientState( GL_VERTEX_ARRAY );
glEnableClientState( GL_TEXTURE_COORD_ARRAY );
glTexCoordPointer( 2, GL_FLOAT, 0, tex );
glVertexPointer( 2, GL_FLOAT, sizeof(float)*2, vertices );
glDrawArrays( GL_QUADS, 0, 4 );
glDisableClientState( GL_TEXTURE_COORD_ARRAY );
glDisableClientState( GL_VERTEX_ARRAY );
}
//--------------------------------------------------------------------------------------
// Name: draw_quad2
// Desc: Renders a 2D quad to the screen (using a triangle fan), but allows you to specify
// the texture coordinates yourself.
//--------------------------------------------------------------------------------------
void draw_quad2( GLuint texture, float* tex, float x, float y, float w, float h )
{
float vertices[] = { x, y, x+w, y, x+w, y+h, x, y+h };
glBindTexture( GL_TEXTURE_2D, texture );
glEnableClientState( GL_VERTEX_ARRAY );
glEnableClientState( GL_TEXTURE_COORD_ARRAY );
glTexCoordPointer( 2, GL_FLOAT, 0, tex );
glVertexPointer( 2, GL_FLOAT, sizeof(float)*2, vertices );
glDrawArrays( GL_QUADS, 0, 4 );
glDisableClientState( GL_TEXTURE_COORD_ARRAY );
glDisableClientState( GL_VERTEX_ARRAY );
}
Keep in mind that these two functions aren't optimized, so I encourage you to do better! So, these functions should be rather straight forward, but the difference between the 1st and the second is that the first one simply maps the entire texture to the quad, the second takes in custom texture coordinates for a sprite sheet. One thing I recommend you NOT do is constantly enable/disable certain client states for vertex arrays. Second, if possible, implement a basic sprite batching routine to draw multiple sprites in one draw call if your game has very heavy sprite usage. Also, keep in mind that if you're using OpenGL ES, then you can't use GL_QUADS (which kinda sucks IMO), so you'll have to change it to use GL_TRIANGLE_FAN or split it up into 2 triangles if you're going to batch sprites.
Also before I for get, always remember that OpenGL uses texture coordinates that stem from the bottom left hand corner instead of the top left hand corner like Direct3D does. I wrote a basic function that takes care of generating OpenGL compatible texture coords for me.
void get_tex_coords( struct texture_t* t, struct Rect* r, float* tc )
{
tc[0] = r->x1 / t->width;
tc[1] = 1.0f - ( r->y1 / t->height );
tc[2] = r->x2 / t->width;
tc[3] = 1.0f - ( r->y1 / t->height );
tc[4] = r->x2 / t->width;
tc[5] = 1.0f - ( r->y2 / t->height );
tc[6] = r->x1 / t->width;
tc[7] = 1.0f - ( r->y2 / t->height );
}
The first two structure parameters are custom, but the necessary fields should be common sense (forgive me if I'm overwhelming you already). So with all this you can render 2D textured sprites in OpenGL. But what about rotation and scaling? That's a good question. If you plan on rotating and scaling your sprites, use the Z-axis to do it. But before rotating and scaling anything, you should translate the sprites to your point of origin and "draw around" that point, not necessarily "draw at" that point. What I mean by this is to center your sprite around the destination point. If you don't, it's not going to work right and will likely piss you off. Here's another example.
Given some random sprite dimensions:
Position (X = 300, Y = 200)
Size (Width = 64, Height = 64)
draw_quad( texture, X-(Width/2.0), Y-(Height/2.0), Width, Height );
This will center your sprite and it will rotate properly. When I was new to writing 2D OpenGL games, I had to figure this out on my own, so learn from my trial and error.
Now, there's one more thing I want to share with you. If you decide you want to use OpenGL ES 2.0 or OpenGL 3.x or OpenGL 4.x, then the above implementation of glEnable/Disable2D isn't going to work on it's own. You'll have to manually create your orthographic matrix and feed it to your vertex program. You can use this function to create a custom orthographic projection:
void glOrtho(float* out, float left, float right,float bottom, float top,float near, float far)
{
float a = 2.0f / (right - left);
float b = 2.0f / (top - bottom);
float c = -2.0f / (far - near);
float tx = - (right + left)/(right - left);
float ty = - (top + bottom)/(top - bottom);
float tz = - (far + near)/(far - near);
float ortho[16] = {
a, 0, 0, 0,
0, b, 0, 0,
0, 0, c, 0,
tx, ty, tz, 1
};
memcpy(out, ortho, sizeof(float)*16);
}
Just use the resulting matrix to multiply against your vertices in your vertex program the same way you'd do 3D with a perspective matrix. I don't know if you've touched vertex or fragment programs (aka shaders) with GLSL yet, but if you haven't don't worry about it too much just yet and focus on the basics until you're ready. The code and examples I've given you should be enough to get started (at least I hope so).
I use this code in my own 2D OpenGL games (and I have a handful of them) and so far it's portable enough to where I don't have to modify it much. OpenGL ES 1.1 was a challenge though. I hope this helps you in some way (or at least a random person who finds this on Google) and feel free to ask anything I might have left out. Also if possible please share with us your skill level as a programmer if you haven't already to avoid any possible redundancy.
Shogun
EDIT: I wrote an article about 2D sprite rendering for OpenGL back in November 2007. It uses a specific extension called GL_NV_texture_rectangle. If at all possible, I recommend using the ARB version instead (if this is relevant to you) unless you checked your extension list first. Even if your card does suppor it, it's best to have a more compatible fallback. I didn't use it in my latest game engine because I was initially trying to keep compatibility with OpenGL ES, but if your game is for Windows, MacOSX or Linux, it should be beneficial.
