Sign in to follow this  
aero68

Two questions, for lead programmers ???

Recommended Posts

aero68    122
I was leadprogrammer for our current graphics engine. Now we will start up our next generation DX10 graphics engine but I have hard time letting them understand that it is needs alot of time and research. So my first question is, how big is you graphics team ? Next is about if you should rewrite your code or not ? If you have developed a component that fills your needs, one day you gonne need to add more and more functionallity. I believe you actually win time if you rewrite that component, using all experience you got from writing it first time and you also get a better architecture that will live longer then you first version. Any comments about team sizes, and rewrite or not, is most grateful. Thanks, Anders Eriksson.

Share this post


Link to post
Share on other sites
Our core low-level engine team has varied over time from 1 to about 4. Usually only 1 or 2 of these are graphics related. For atleast half of my 6 years, I've been the sole person on the engine team, mainly going graphics. The team increased in size for few months when we did the DX8/XBox to DX9 switch. The most important part of that switch was attempting to move anything API specific out into seperate modules, leaving nearly all of the remaining code to use useful for any platform.

We have a somewhat API agnostic interface, and underneath we can implement D3D8, D3D9, D3D10, OpenGL, or anything else that sticks fairly closely to some simple concepts. We also allow the game and other parts of the engine to access the underlying API to include platform specific features. When switching APIs there are only a few core D3D specific things. Textures, Surfaces, Vertex/Index storage, Drawing, and effects. These concepts will exist in every API, but only the implmentation details change. Our animation system, collisions, cameras, math libraries, etc, don't care about D3D. Even mesh loading doesn't care. It loads data and uses the platform agnostic interface to request somewhere to put the mesh. The renderer calls an agnostic API to activate whatever effect is needed, and calls another agnostic API to actually draw from the storage.

Rewriting code can be wasteful, or very good. If the code is slow, it may need a rewrite, or even a re-engineering, using another technique to solve the same problem. If the code has become so convoluted that it's hard to follow, for debugging or adding new features, perhaps a cleanup is in order. Try to keep in mind all those features you've been meaning to add and ensure the new design accomodates both the old and new features.

Share this post


Link to post
Share on other sites
Radan    404
Quote:
Original post by Namethatnobodyelsetook
We also allow the game and other parts of the engine to access the underlying API to include platform specific features.


Hm, isn't allowing other parts of the engine to access the API directly going against encapsulation and abstraction of the API. Isn't some kind of API agnostic wrapping needed even in that case.(at least a really simple one)

Ideally API change should include just a rewrite of that API agnostic module. No other part of the code should need to be changed.

Share this post


Link to post
Share on other sites
Saruman    4339
Quote:
Original post by Radan
Ideally API change should include just a rewrite of that API agnostic module. No other part of the code should need to be changed.

I take it you have never worked on a console? :)

Share this post


Link to post
Share on other sites
Editors and tools were other people, not the core low-level team. The amount of platform specific things in the editor is very limited, and platform specific code is mainly isolated to a few CPP files.

The reason to allow a game to get past the abstraction is because, as an abstraction, it's taking some detail away, somewhere. (Read more). There will always be something super cool that one platform or API has that isn't available elsewhere. Should I arbitrarily decide games can't use that feature because it's not available on all hardware? Should I write an abstraction for something which may never be seen on another platform/API?

The code MUST be tailored to the target API/platform to some degree. I try to keep 95% of that hidden behind the engine API, but certain things make that impossible. There are common rendering ideas that if used on an XBox360 would kill performance. There are other things possible on the 360 that a PC would have trouble with. A game is going to have to understand how to load-balance it's work on the Cell on the PS3. I can't hide these facts. The majority of code doesn't have to change though, and that's what we aim for. Platform neutral whenever possible.

Share this post


Link to post
Share on other sites
Sphet    631
Quote:
Original post by Namethatnobodyelsetook
The code MUST be tailored to the target API/platform to some degree. I try to keep 95% of that hidden behind the engine API, but certain things make that impossible. There are common rendering ideas that if used on an XBox360 would kill performance. There are other things possible on the 360 that a PC would have trouble with. A game is going to have to understand how to load-balance it's work on the Cell on the PS3. I can't hide these facts. The majority of code doesn't have to change though, and that's what we aim for. Platform neutral whenever possible.


Nice to hear I'm not the only one doing this - our company has very specific holes punched through the API to get access at platform features - the consoles are just too powerful/wacky/useful to give up on them.

Share this post


Link to post
Share on other sites
remigius    1172
Quote:
Original post by NamethatnobodyelsetookThe code MUST be tailored to the target API/platform to some degree. I try to keep 95% of that hidden behind the engine API, but certain things make that impossible. There are common rendering ideas that if used on an XBox360 would kill performance. There are other things possible on the 360 that a PC would have trouble with. A game is going to have to understand how to load-balance it's work on the Cell on the PS3. I can't hide these facts. The majority of code doesn't have to change though, and that's what we aim for. Platform neutral whenever possible.


Sorry 'bout the thread hijack, but this got me wondering. Exactly how do you deal with varying numbers of processors or cores in keeping platform neutrality? Obviously you could schedule more tasks on a single hardware thread if you have fewer processors available like on a typical PC, taking advantage of the OS scheduler. But still, I'd imagine you might need to write bigger or smaller tasks depending on whether you're dealing with 6 or 2 hardware threads. At some point, this could cut deep into your engine's API.

I don't know if it's really an issue, but it sounds like it might be a big pitfall in 'nextgen' cross-platform development. Has anyone already encountered problems with this? If so, is it a serious issue and what would be a good way to tackle it?

Share this post


Link to post
Share on other sites
Radan    404
Yeah, I never worked on the console. I'm stuck with the PC.

Actually, punching holes to the API makes sense. It would be nearly impossible to abstract everything away.

How do you generally go about "punching holes"?
For example, in a higher level class that needs access to the API, would you just add "#include <d3d.h>" or would you add a passtrough inlined function in the abstraction API that actually just passed the data trough to the actual API function?
First is simpler but the second one should be easier to maintain as the API code is still kept in the same namespace/class even though it actually isn't abstracted.

Do you have a specific system to mark the places in code where API abstraction is bypassed for easier maintance or API switching?

Share this post


Link to post
Share on other sites
Demirug    884
Quote:
Original post by remigius
Sorry 'bout the thread hijack, but this got me wondering. Exactly how do you deal with varying numbers of processors or cores in keeping platform neutrality? Obviously you could schedule more tasks on a single hardware thread if you have fewer processors available like on a typical PC, taking advantage of the OS scheduler. But still, I'd imagine you might need to write bigger or smaller tasks depending on whether you're dealing with 6 or 2 hardware threads. At some point, this could cut deep into your engine's API.

I don't know if it's really an issue, but it sounds like it might be a big pitfall in 'nextgen' cross-platform development. Has anyone already encountered problems with this? If so, is it a serious issue and what would be a good way to tackle it?


This is an issue for sure and the general solution is to use your own lightweight job management.

“taking advantage of the OS scheduler” is a bad idea as it requires that you use more threads than cores and threads are heavy kernel objects. If you force a core to switch between multiple threads all the time you will lost CPU power.
But a user mode job manager generates some overhead per job too. This is a dilemma as if you made jobs large there is a risk that you would not have enough jobs for all cores. If you made them to small the overhead would be larger than the additional CPU power you gain. There are even more problems especial when it comes to window handling and graphics APIs.
Consoles can be even trickier but at least you have a fixed hardware in exchange.


Share this post


Link to post
Share on other sites
remigius    1172
Thanks for the insights Ralf.

The situation you described is indeed what I was getting at. It seems that you're either stuck with too many small jobs for the PC, or too few big jobs for consoles. I think I understand what you mean with the usermode lightweight job manager. I guess I'll have to give it a go in XNA as soon as the XBox360 wireless lan adapters finally ship again over here ;)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this