SourceForge.net is a decent host for open source projects. They are a pretty common starting point for searching for OSS, so you may want to register there even if you don't use all of their services (e.g. my project hosts our website offsite, but we use SF.net's SVN repository, file release system, etc.). They have a ton of mirrors which is nice if you are releasing good sized binaries. That way you don't have to pay for that bandwidth. They have occasional reliability issues and do have some restrictions about what you can do, but in general I have found them to reasonable and you can't complain about the price. There are others, but SF.net is the one that I am familiar with. Wikipedia has a comparison of free software hosting services that you may want to look at too. I have some other thoughts, I'll post those separately...

First off, I founded and admin Crown and Cutlass which is an open source game. We have a tiny (sometimes almost non-existent) community, and have hardly made a splash in the open source world. However, I would say we have been fairly successful so far. I wish we were farther along of course, but I think so far it has been a really good experience.

Think over your choice of license carefully. It can have a serious impact on how you run your project, and different licenses may be better depending on what you are looking to do. Do you have a license picked out? If so, which one are you using? Why? If not, what are you looking to get out of the project? What do you want to allow your users to do or not do?

You don't really absolutely need to do anything to open source your project except for releasing the source under a suitable license. Documenting the project, having consistent coding style, dealing with bug reports are all good things and they will certainly help encourage a community. However, I think the most important thing is that _you_ remain active. Nothing turns off potential users/contributors like a dead (or apparently dead) project.

You should be ready to have very little community involvement for a long time. You maybe lucky and get a huge community right away, but it'll probably be more of a slow process. Even though Crown and Cutlass has slowly been making progress and we have gotten significant contributions from people (art!), most contributors have been people that come in for a while and then drop out. We haven't had a lot of code contributions, partially because we haven't been quick to accept new developers without seeing that they are committed and capable. If you are writing an engine, you may be able to attract more developers since the users are all developers.

Take a look at Producing Open Source Software. You can buy it or read it for free. It is a good read. I'm not sure if you need to read all of it, but it is definitely worth skimming.

Anyway, welcome to the world of open source projects. Post a link to your project once it is set up!