Sign in to follow this  
ChuckE

architecture of game engine

Recommended Posts

ChuckE    124

Hi there!

 

I am currently not developing game engine because I do not have enough knowedge for know. However I'm planning to do it one day. So I want to make some preparations till that time.

 

However I've been wondering about something.

 

Is it possible to implement something like...

Virtual Machine for game engine with unified VRAM and RAM? I mean during its life cycle, game object does not know that it has some data being operated via GPU and some data being operated via CPU & RAM. So basically it means that even for me during game creation or programming I have no need to know that there is GPU and CPU tasks - I simply create object and all required duties is being done my virtual machine.

 

As far as I can understand if it is possible to implement it means that there is a need to create custom set of instructions, which will do GPU and CPU & RAM tasks?

I don't know whether it can be called virtual machine or not. Maybe it's better be called APU emulator. But software emulator of APU - is basically a virtual machine?

 

The talk is simply hypothetical for know. But it would be interesting to create some kind of game engine with its own virtual machine.

 

As a background aka "why do I need this" I'll answer -

I enjoy creating tools and utilities without particular reason. For example due to religious reasons I perform compilation using gcc or g++ (because of the newest c++ standart and its complete or almost complete support there) so I developed for myself .cpp template generator using Python (created simple markup template language) because I got tired of creation files manually. Maybe I'll wrap it into simple WinForms UI and create some kind of IDE for myself.

 

Or for example I am planning to develop source control tool (something like Liquibase but for .cpp files) using Groovy.

 

Or the simpliest CAD to create 1,2, 3 point perspective projections(Don't want to draw it manually neither want to learn some software) and import it to .png (I think I'll use either SharpDX(interested) or Tao Framework(used it years ago) + UI using WinForms+C# is easy to create + import to png is easier to implement that than using c++ I think (though not sure)).

 

It's all about interest and I the idea to implement your own VM is exciting + add your own assembly-like language - what can be better?

Edited by ChuckE

Share this post


Link to post
Share on other sites
Irlan    4067

Game engines are made in layers that relatively depends of another layers.

 

A graphics module knows what a mathematics module is because they're related.

 

 

 


Is it possible to implement something like...
Virtual Machine for game engine with unified VRAM and RAM? I mean during its life cycle, game object does not know that it has some data being operated via GPU and some data being operated via CPU & RAM. So basically it means that even for me during game creation or programming I have no need to know that there is GPU and CPU tasks - I simply create object and all required duties is being done my virtual machine.

 

Yes. A memory module it's implemented to take care of RAM management, and the graphics module can take care of VRAM management (if VRAM stands for Video Random Acess Memory).

 

 

 


As far as I can understand if it is possible to implement it means that there is a need to create custom set of instructions, which will do GPU and CPU & RAM tasks?
I don't know whether it can be called virtual machine or not. Maybe it's better be called APU emulator. But software emulator of APU - is basically a virtual machine?

 

If you're talking about VRAM being the virtual memory, if we could manage that we'd be doing duplicated wor because the OS already does. A memory manager of a engine operates in a way that can be used in any other related level in the application, allocating objects, providing profiling statistics information, etc, and generally it's worried about the heap and the stack.

 

 

 


It's all about interest and I the idea to implement your own VM is exciting + add your own assembly-like language - what can be better?

 

What you're saying here it's completely unrelated to game architeture. You're trying to implement a virtual machine, but such software it's unrelated to a game engine, and you want to manage virtual memory in a game engine. huh.png

 

I'm not saying that it's not impossible to manage virtual memory in modern architectures, but it's unrelated to a game engine architeture. 

Edited by Irlan

Share this post


Link to post
Share on other sites
ChuckE    124

What you're saying here it's completely unrelated to game architeture. You're trying to implement a virtual machine, but such software it's unrelated to a game engine, and you want to manage virtual memory in a game engine. huh.png

 

Well....I see. I thought to game engine architecture depends of the hardware it uses. And it's being created with knowledge of hardware it will run.

 

It seems I need to implement something like JVM(more or less) and provide some kind of language, which will allow to develop game. And the game will run inside virtual machine.

Edited by ChuckE

Share this post


Link to post
Share on other sites
Promit    13246

It seems I need to implement something like JVM(more or less) and provide some kind of language, which will allow to develop game. And the game will run inside virtual machine.

Unreal works sort of along these lines, as did Quake way back in the day. Unity is obviously based around this approach. Still, I don't recommend doing it in general.

Edited by Promit

Share this post


Link to post
Share on other sites
ChuckE    124

 


It seems I need to implement something like JVM(more or less) and provide some kind of language, which will allow to develop game. And the game will run inside virtual machine.

Unreal works sort of along these lines, as did Quake way back in the day. Unity is obviously based around this approach. Still, I don't recommend doing it in general.

 

why ?

Share this post


Link to post
Share on other sites
ChuckE    124

 


For example due to religious reasons I perform compilation using gcc or g++

 

Which religion is that?  That might be a religion I can get behind! ;)

 

Nothing special. I sincerely believe that compiler should support all current language features - because only in that case it will be complete. Thus perfect.

Share this post


Link to post
Share on other sites
Avilius    710

Game engines are made in layers that relatively depends of another layers.

 

A graphics module knows what a mathematics module is because they're related.

Well, a graphics module is a pretty vague term. I'd say that the scene manager would know about mathematics. The renderer doesn't really need to do any transformations, as it should be taken care of before hand.

 

To the OP:

A lot of renderers are indeed glorified VMs. Generally the renderer will take in a stream of render commands, which are essentially just opcodes. Implementing an assembly language is a bit excessive IMO.

Share this post


Link to post
Share on other sites
Irlan    4067

Well, a graphics module is a pretty vague term. I'd say that the scene manager would know about mathematics. The renderer doesn't really need to do any transformations, as it should be taken care of before hand.

 

While you're correct about the graphics module can live without a mathematics module, you're forgeting that a graphics module it's all about bringing the low-level graphics API to our users interface with it.

 

A graphics module should know about the mathematics module to facilitate its interface with the graphics wrapper, and it should be called CXApiWrapper or similar, instead of a renderer because it's not a software-side renderer nor rasterizer. The graphics wrapper it's a interface that needs to be pre-determined before re-building the engine and running the simulation depending of your current platform or some criteria. Example: DirectX® runs faster on Windows, so you should use that instead of OpenGL®—altough both are available. In iOS, we're only limited to OpenGL® ES and it's mandatory to use that.

 

A scene manager it's at the same level of the engine and knows what a graphics module, mathematics, physics, etc. are.

Edited by Irlan

Share this post


Link to post
Share on other sites
Avilius    710

While you're correct about the graphics module can live without a mathematics module, you're forgeting that a graphics module it's all about bringing the low-level graphics API to our users interface with it.

 

A graphics module should know about the mathematics module to facilitate its interface with the graphics wrapper, and it should be called CXApiWrapper or similar, instead of a renderer because it's not a software-side renderer nor rasterizer. The graphics wrapper it's a interface that needs to be pre-determined before re-building the engine and running the simulation depending of your current platform or some criteria. Example: DirectX® runs faster on Windows, so you should use that instead of OpenGL®—altough both are available. In iOS, we're only limited to OpenGL® ES and it's mandatory to use that.

I don't think the renderer is the same thing as a graphics wrapper. Sure, a renderer could utilize a graphics wrapper, or even double as one, but the two terms are not mutually exclusive.

 

A renderer should be above a wrapper, if any. All it should do is take relatively high-level commands and transform them to commands for the underlying graphics library, whether that be a wrapper or raw D3D/OpenGL. There doesn't need to be any abstraction with the renderer. The renderer is more of a convenience system rather than one necessary for an engine. It's not essential for the engine to do tasks without it. If you want multiple render backends then you should write the renderer to transform engine commands into the wrapper's commands.

 

 

 

A scene manager it's at the same level of the engine and knows what a graphics module, mathematics, physics, etc. are.

The scene manager should not be at the same level as a graphics module.

Share this post


Link to post
Share on other sites
Irlan    4067

I don't think the renderer is the same thing as a graphics wrapper. Sure, a renderer could utilize a graphics wrapper, or even double as one, but the two terms are not mutually exclusive.

 

The OP mentioned GPU management, so it's basically what a graphics wrapper brings to a high-level visibility. If by "renderer" you mean a render queue or related, of course it's higher level than the graphics library, but still lives inside the graphics module.

 

A render queue has render items that can be texture IDs, shaders IDs, or even a shader technique type.

 

 

 


A renderer should be above a wrapper, if any. All it should do is take relatively high-level commands and transform them to commands for the underlying graphics library, whether that be a wrapper or raw D3D/OpenGL.

 

The user asks the graphics wrapper to set the current ambient color to the current shader, so the graphics library high-level interface ("renderer") provides that. The graphis API wrapper doesn't know what a renderer is. The correct it's that the "renderer" uses the graphics low-level interface.

 

 

 


There doesn't need to be any abstraction with the renderer.

 

Yes. As I stated before, the renderer should not be related to a scene manager and it is a high-level interface of the low-level graphics wrapper.

 

 

 


The renderer is more of a convenience system rather than one necessary for an engine.

 

The "renderer" it's not necessary in non cross-platform situations (even if it should be). If only have DirectX® 11 and you need to set a shader variable, just pick its constant buffer and update its data directly interfacing with the API, but even if you're not running, it is a good idea to keep this separation. 

 

 

 


The scene manager should not be at the same level as a graphics module.

 

Check again. I said that it's at the same level of the engine. In the engine module you have the game, the game entities like animated skeletons, etc., and the scene manager has lights, cameras, and actors (it has game entities). The scene manager can use all modules because it's at the same level of the engine.

Edited by Irlan

Share this post


Link to post
Share on other sites
kalle_h    2464

 

 


For example due to religious reasons I perform compilation using gcc or g++

 

Which religion is that?  That might be a religion I can get behind! ;)

 

Nothing special. I sincerely believe that compiler should support all current language features - because only in that case it will be complete. Thus perfect.

 

Only those features that you actually use are important for compiler selection and there are also other reasons to compare which is best compiler for you. Compilers are never perfect nor they are complete.(Every complex piece of code contains bugs.) This just sounds superstition.

Share this post


Link to post
Share on other sites
Avilius    710

The OP mentioned GPU management, so it's basically what a graphics wrapper brings to a high-level visibility. If by "renderer" you mean a render queue or related, of course it's higher level than the graphics library, but still lives inside the graphics module.

 

A render queue has render items that can be texture IDs, shaders IDs, or even a shader technique type.

 

No no no... a render queue is just that - a queue. It doesn't take care of managing resources, the renderer itself does that. The render queue is just a stream of renderer commands.

 

The graphics wrapper itself should provide a low-level common interface for the renderer to interact with. I think your renderer and graphics wrappers are strongly coupled.

 

 

 

The user asks the graphics wrapper to set the current ambient color to the current shader, so the graphics library high-level interface ("renderer") provides that. The graphis API wrapper doesn't know what a renderer is. The correct it's that the "renderer" uses the graphics low-level interface.

That should be the renderer's job. If the engine has a graphics wrapper, then it should aim to try to provide a near 1:1 interface with the underlying API while still keeping a common interface between other APIs.

 

 

 

The "renderer" it's not necessary in non cross-platform situations (even if it should be). If only have DirectX® 11 and you need to set a shader variable, just pick its constant buffer and update its data directly interfacing with the API, but even if you're not running, it is a good idea to keep this separation. 

Well, the renderer shouldn't be necessary at all in my way of doing it, even in cross-platform situations. If you want cross-platform capabilities, then just use a wrapper instead of D3D.

 

 

Check again. I said that it's at the same level of the engine. In the engine module you have the game, the game entities like animated skeletons, etc., and the scene manager has lights, cameras, and actors (it has game entities). The scene manager can use all modules because it's at the same level of the engine.

 

Ah, thanks for clearing that up. I misread it.

 

That's how I do it anyways. I guess everyone has a different way of doing things.

Edited by Avilius

Share this post


Link to post
Share on other sites
Irlan    4067

No no no... a render queue is just that - a queue. It doesn't take care of managing resources, the renderer itself does that. The render queue is just a stream of renderer commands.

 

 

I didn't say managing resources. I said using a ID in order to bind a texture or a shader to a specific model to be rendered—a unique identifier. Resource management it's a competely separate concept, and for each type of loaded resource you should have manager, and that is when you bring proper GPU high-level vision and management to the project to enfatize what the OP said.

 

 

The graphics wrapper itself should provide a low-level common interface for the renderer to interact with. I think your renderer and graphics wrappers are strongly coupled.

 

 

That's what I clearly said. Check again:

 

 

the renderer should not be related to a scene manager and it is a high-level interface of the low-level graphics wrapper.

 

 

 

The user asks the graphics wrapper to set the current ambient color to the current shader, so the graphics library high-level interface ("renderer") provides that. The graphis API wrapper doesn't know what a renderer is. The correct it's that the "renderer" uses the graphics low-level interface.

 

 

I said that the "renderer" uses the graphics wrapper. Coupling it's a bad idea since the high-level exposed functionality depends of the low-level one.

 

 

Well, the renderer shouldn't be necessary at all in my way of doing it, even in cross-platform situations. If you want cross-platform capabilities, then just use a wrapper instead of D3D.

 

 

The correct way of doing it is having a separate functionality for the low-level graphics API.

 

If you ever used the Effects Framework or looked at it you know that it provides high-level functionality to the user. The user asks to set specific material color, and the wrapper translates that into DirectX functionality. I'm saying that because the "renderer" (if by that you mean, a high-level graphics wrapper) works in a similar way, but the only difference it's that it doesn't care which API you're using. Take a look here.

 

That said, a "renderer" isn't a scene manager, nor a render queue manager, nor a model manager, nor a compiled shader manager. The "renderer" only fits better in being a facade to a graphics interface.

 

There are tons of possibilities of something being rendered. If you have a "renderer", which of the following objects can be rendered and be related with it?

 

Renderable;

RenderableModel;

RenderItem; 

RenderableText;

RenderableInstance;

 

You see? It  gets more complicated to differentiate between renderables, game entities, render items, UI, while in the end, each of these classes would fit better in its appropriate managers or even modules.

Edited by Irlan

Share this post


Link to post
Share on other sites
Hodgman    51326
On this new topic - my pet hate with most low-level graphics APIs is that they are horrible state machines, with that state almost being global... Which is evil ;)

So my cross-platform wrapper slightly raises the abstraction to a stateless API, while still being very low-level, i.e. still dealing with buffers, textures, draw-calls, etc..
I also implement an "Effect"/cgfx/etc style higher-level shader-collection object at this level, allowing you to select an actual shader program from an "fx file" containing a collection of passes/permutations.

Then the level above this - the "Renderer" - deals with models, materials, lights, culling, rendering pipelines (e.g. deferred rendering)...

Share this post


Link to post
Share on other sites
Avilius    710

-snip-

 

...So I pretty much misread your entire post then...

 

Lol, sorry. I was arguing about mostly the same thing that you were saying then. My bad.

 

 

The correct way of doing it is having a separate functionality for the low-level graphics API.

 

If you ever used the Effects Framework or looked at it you know that it provides high-level functionality to the user. The user asks to set specific material color, and the wrapper translates that into DirectX functionality. I'm saying that because the "renderer" (if by that you mean, a high-level graphics wrapper) works in a similar way, but the only difference it's that it doesn't care which API you're using. Take a look here.

 

That said, a "renderer" isn't a scene manager, nor a render queue manager, nor a model manager, nor a compiled shader manager. The "renderer" only fits better in being a facade to a graphics interface.

Yeah... we shared the same views. I wasn't reading what you said correctly and now I feel like an idiot for wasting both of our times.

 

EDIT: Edited out the last section because I of course read everything wrong, lol. 

Edited by Avilius

Share this post


Link to post
Share on other sites
Avilius    710

yes. been there, done that.

 

good idea?  

 

no.   not recommended for mission critical code.   less performant than native machine code.

 

Huh? I don't understand what you're saying...

He isn't talking about implementing a full blown virtual machine... I think he's referring to the rendering engine.

Edited by Avilius

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this