• entries
    232
  • comments
    1462
  • views
    957177

Summary status of the ICP

Sign in to follow this  

399 views

You're not dreaming, a new update :)

I haven't posted progress in the past month, because i've simply been too busy releasing patch after patch after patch for the Infinity combat prototype. Yesterday, i released version 2.1 which is fully public. A lot of players have downloaded and tested it, and although far from perfect, it's a giant step up from the old 1.1 version.

I'd like to take a bit of room in this journal to make a summary of the ICP, what went right, what went wrong, and how much of it will go in the final game, what is experimental and will get improved, and the "unknowns".

On textures loading:

Version 2.1 added "unified" textures. I wrote one line about it in the change logs, but since we're in the dev journal which is more technically oriented, i'll give a few additional details here: previously, all textures were using bump + specular (gloss) + diffuse + self-illumination maps. Self-illum are for pixels emitting light ( like the small windows on the hull of the ships ). If you count well, that's up to 4 textures per surface, and that's also 4 texture 2D samples in the pixel shaders.

Since 2.1, the textures are "unified". The goal was to use only 2 textures per surface instead of 4. How is that possible ? Simply by using the alpha channel, as specular and self-illumination maps are grayscale. Another thing was the optimization of textures: converting bump maps to normal map was done at loading time, on-the-fly before. Now, the bump maps are pre-processed into normal maps. That helps loading times.. a bit.

Loading times would be blazingly fast if it wasn't for one thing: texture compression is enabled by default. It could of course be disabled ( and i might add a switch in a future patch ), but it would require a lot more video memory. Texture compression is automatically done by the OpenGL driver when you upload a texture to the video card, and let me tell you something: it's horribly slow. With my timings, i found that usually, a 1024x1024 texture takes 10-40 milliseconds to load, and 200 milliseconds to compress. So without texture compression, loading would be 10 times faster or almost. Unfortunately, there isn't much i can do to reduce this compression time since it's a video card driver thing. Maybe i could pre-compress them, but then i'd loose the ability to store my textures in jpeg2000, the format that i'm planning to use for the full game..

On networking:

Right, that's probably the most controversial part of the ICP. I'm not very happy with the networking model. There are many sides of the network model, and none of them work perfectly.

First of all, there's the network interpolation and prediction. You know, firing at a ship, seeing a hit on the client but nothing on the server.. most first-person shooters ala Quake/Unreal use techniques that can't easily be applied to something like Infinity/ICP, so i came up with my own technique. It looks okay when you've got a good ping ( < 100 ms ) but when it starts to lag, ships randomly jump around or move in strange ways, making it hard to hit anything.. i definately have to review from the grounds up the networking code for the game.

Another controversial point is what i called the "bubble system". Bubbles will likely play a fundamental role in the full game ; they're supposed to act more or less transparently to the players, but in the ICP it doesn't work so well yet. Bubbles are used to align movement of a ship to another one - the closer you are to your "parent" ship, the more you stick to its movement -. In the case where you're in the hangar of your parent ship, or very close to the hull, any movement ( translations or rotations ) your parent ship will do, you will follow *perfectly*. And it better work that way.. when you're docked in the hangar of your parent ship, you don't want to have to adjust your velocity constantly because your parent ship is maneuvring or accelerating ( remember, in space there's no friction ).

Another goal of the bubble system is to avoid seeing weird effects due to the network latency. If your 20 meters long spacecraft is moving at 1000 Km/sec, that means it travels 10 Km in 10 milliseconds. Now imagine an average latency that fluctuates between 100 and 150 milliseconds.. the ship will jump around by many kilometers at each "tick".

At this stage you're probably wondering: why don't other games ( like Quake/Unreal ) suffer from this ? The answer is simple: but they do ! However, their entities don't travel that fast. While in Infinity you'd have a 20m ship traveling at 1000 Km/sec, in Quake you've got a 2m character traveling at.. 0.01 Km/sec. Relatively speaking, it means that the effect is
*one million times* less of a problem.

Bubbles to the rescue! Bubbles won't solve the problem of ships jumping around due to those huge velocities, but they'll make your own ship/camera follow the same movements, meaning that visually, they cancel each other.. or at least, in theory.

In practise in the ICP it doesn't work so well. The reason to this is that it's only working client based, and that it only affects your own ship. What it means is that:

1. Another player spawning in a hangar should in theory stick to its parent ship, and perfectly follow its movement. Because the bubble only affects yourself, it doesn't and so.. you'll often see in hangars other player's ships jumping around, or even entering walls.

2. It is client based, so anything that involves server calculations will be slightly messed up. Plasma shots are one thing - which partially explain the hit misses and the difficulty of playing when you're close to a battleship -. Another thing is for example when you fire a missile and have your own ship collide with it and go mad. That's because the bubble system doesn't affect the missile: at the moment you fire it, your own ship will follow the movement of the battleship, but the missile will not.. and if those two movements oppose each other, the missile ends up hitting your ship.

Those problems can be fixed by unifiying the bubble system; in other words, making bubbles for every single entity in the game, and for both the client and the server. Unfortunately that was too much work for the ICP, so i'll do it for the full game.. and pray that it works as expected.

The A.I.

The A.I. is quite flawed. I used something based on "boids" ( flocks ) to simulate the movement of battleships. Unfortunately, they use different physics than player ships. It makes them possible to rotate by 180? in a second for example ( you can notice this strange behavior sometimes ). I will have to unify that too, and to make all ships use the same physics.

Because at level generation time, the position of all ships and entities ( asteroids, mines ) are random, it is possible to see ( that's rare, but i've seen it happen no less than one hour ago ) the battleship spawn inside the giant asteroid.. huh.

Performance

Last but not least, i'm not satisfied at all with the performance, both of the client and the server.

On the client side: old video cards have serious troubles to keep up with the complexity of the pixel shaders; that makes them slow down a lot in the hangars or near the biggest ships, when a lot of pixels are filled on the screen. But that's expected and there isn't much i can do here, except optimizing the shaders and providing a shader quality option. What's less expected is the CPU usage of the ICP.. it's the main bottleneck at the moment.

Why such a high CPU usage ? There are two main reasons:

1. The amount of objects to update every frame. A fighter is made of 2-3 objects for the body/wings, and 2 to 8 turrets. Frigates or medium-sized ships can have from 4 to 10s of turrets. The battleships have 30 turrets. Each turret is usually made of 3 objects ( they can pitch/yaw freely, count the barrels and the base too.. ). Since teams are usually balanced, a decent battle has around 15 ships per team. Finally, count a good hundred of asteroids and a few mines. Do the sum: in total, there are more than a thousand objects to update every frame. Moving objects indirectly means dynamically updating the octree for frustum culling, and updating the whole hierarchy of transformation matrices ( barrles attached to turrets, turrets attached to ships, etc.. ).

2. The collision tests - usually, the ray casting used for various reasons: plasma shots are the main one, but also testing occlusions ( ships behind asteroids, thruster glow effect visible by camera, lens-flares occluded, etc.. ).

On the server side, you find similar bottlenecks - excepting the video card related ones of course. But unlike the client, the server has to run the A.I. and a lot of additional checks, meaning that all in all, the server isn't really "lighter" than the client..

In the full game, i'm going to use an entity LOD system: merge together objects that are static or with unrelevant animations ( the server doesn't need to track the tens of the objects in gears of ships for example ). I'm also going to use a LOD system in the ray-casts and plasma shots.

The good!

Not everything is bad, of course. The public reaction to the ICP is very good. A lot of people are playing, enjoying and having fun with it. It has a "Star Wars" feeling to it that you can't find in modern games IMO. Probably the influence of oldby games such as "XWing" on me :) When it's not lagging too much and when there's enough players, flying around, passing close to battleships firing at each other, seeing explosions in the distance, all against a nice planetary background.. that's quite epic !

The physics and controls are now pretty good and are unlikely to fundamentally change in the full game. The question of the maximum speed is still floating, though.. if i remove it, i'm afraid most fighters will pass very quickly near each other and never fight. The maximum speed ensures that dogfights are possible.

Controls or HUD will get improved, especially in the way of customization. But it's unlikely to have very different controls: you'll still find keys for rolling, firing, controling your thrusters.. and the "target velocity" with automatic adjustment ( that you can disable for full control ) will likely stay. Newtonian physics are not going to be changed either, although ships will for sure have different behaviors/parameters.

The future

I might patch a few bugs in the ICP in the coming months, but nothing urgent. I now need a small break from the ICP - i'm leaving in vacation in France next week -, and i will now continue the work and finish the planetary engine, as was initially planed.

The next step is in more or less correct order: cascading shadow maps, ground texturing, vegetation and volumetric clouds.
Sign in to follow this  


11 Comments


Recommended Comments

Better and better every time we see it! Congrats and looking forward to the next update. In the meanwhile we have the ICP to keep us occupied ;)

Share this comment


Link to comment
Cool stuff, glad to see you are improving further than where other indie developer has been.

Just one question, why do you plan to use jpeg2000 instead a format where the mipmaps are enveded? bad quality for dds?

Thanks

Share this comment


Link to comment
Mipmaps are calculated on the GPU, there's no need to store them on disk :)

Jpeg2000 because it uses less disk space than Jpeg and has a much higher quality. It also supports an alpha channel, while Jpeg doesn't.

Share this comment


Link to comment
You don't store your normal or bump maps in jpeg2000, do you? I'd figure that even though you won't get those typical jpeg artifacts it still will distort the results.
I'm really looking forward to some technical updates about your planet simulation! :-)

Share this comment


Link to comment
Oooh, public release! I'm downloading right now.

I'm running 2.8GHz dual Xeons, each dual-core and hyperthreaded, w/ 2 GB mem and a NVidia Quadro FX 540, 128 MB. Looking forward to seeing how it handles the ICP. =D

Share this comment


Link to comment
Just how big are the textures when you put them in DXT-compressed format (e.g. DDS), compared to the jpeg2000 versions? (For a fair comparison you'd need to exclude mipmaps).

I'd figure that DXT is what they're being compressed to on the hardware side...

Share this comment


Link to comment
An uncompressed 1024x1024 texture, 4 channels, would be 4 MB. With DXT the compression ratio you get is between 1:4 and 1:6 - let's assume 1:5 for an average. That'd be 800 KB. The same texture stored in Jpeg2000 would be 100-200 KB (at a better quality than DXT).

Share this comment


Link to comment
Breathtaking! I was having some extreme choppiness while observing and haven't had the time to actually join in battle. When not choppy, it's smooth as glass and feels very solid.

Share this comment


Link to comment
Got it. Ran it. Claimed to be running at 12-15 FPS, but honestly it appeared a LOT faster than that, aside occasional from network lag it ran VERY smoothly.

And I thought it might be interesting to note that it runs quite well under Vista (even in Windowed mode with Aero enabled).

Again, I'm jealous, and I must steal your brain.

Share this comment


Link to comment
Got it. Ran it. Claimed to be running at 12-15 FPS, but honestly it appeared a LOT faster than that, aside occasional from network lag it ran VERY smoothly.

And I thought it might be interesting to note that it runs quite well under Vista (even in Windowed mode with Aero enabled).

Again, I'm jealous, and I must steal your brain.


THANK YOU 500.13!

Share this comment


Link to comment
I don't believe there is any run-time or load-time benefit to simply restricting your engine to use Jpeg2000 just because of the reduced size footprint. Particularly since the images would still have to be stored in memory as uncompressed or in DXT format to be usable by GPU hardware. Unless GPU hardware supports jpeg2000 - but I haven't heard of it :D In the Jpeg 2000 case you will definitely get the Jpeg 2000 decompress cpu-hit as well as the dxt encode cpu-hit. (And by cpu-hit I include memory bottlenecks, which image processing apps are quite susceptible to)

All this you already know, but if you were to precompress your images as a DXT format (eg. to a DDS file), you would avoid the Jpeg2000 decoding, and dxt-encoding steps, but you'd have a slightly larger fileIO. Furthermore, my private experiments have shown DDS is further compressible - a compressed size likely to be around 60%-70% of the uncompressed size. So if you've got a general data compression/decompression scheme you could get a benefit by running that while you still have pending IO operations.

Obvious tricks for improving actual fileio operations is to sequentialize your file operations (maybe even from a pack file which can be defragmented), reduce small reads, reduce seeks, avoid re-opening and closing files - and avoid interleaved file operations (eg. read from one file, read from the other, read again from the first file)

Another idea for improving your loads is to kick off the next file's io asynchronously while you're doing some massive CPU processing on the current file (eg. driver dxt compression or general data decompression). At least you should be able to amortize file opening and seeking costs against the cpu time you have to perform anyway :)

Just a comment: mipmap generation isn't neccessarily done on the GPU - it will vary according to the driver and hardware you're running it on. If the driver can do it on GPU it will, but some hardware might not support those operations. Additionally, remember that if you request mipmap generation, then more time will be chewed up during your load times within the driver (rather than on fileio which might actually be cheaper at times). But it's not even as easy problem as that! Especially considering that that some driver/GPU combos (eg. ATI) use ripmaps for aniso filtering and will sometimes generate up to twice the number of ripmaps, so that they can reduce the number of samples taken during the actual aniso filtering. Blah, I'll shutup now :)

Share this comment


Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now