Okay, for starters, it's been quite a while since I've been on GD, so take pity on me, and the possible uselessness of my tags and other things :) GD.net has been enhanced significantly since I was regularly posting last, and I'm excited to see it :)


At the moment I'm in the middle of enhancing the asset management in my engine.

I find myself needing to make a distinction between "internal" assets and "external" assets.

Internal assets are assets that are compiled specifically for my engine using my own tools, to a basic data chunk format which defines a set of data chunks and pointer references between those chunks. Nothing super complicated, just something to get compiled static assets up and loaded with a minimum of fuss without having to write a loader or interpreter for each and every different kind of asset.

External assets are assets such as textures and shaders which I want to be able to deal with generically as assets, but are actually directly loaded and dealt with by Direct X.

So, I have a base class "Asset", so for every asset I can quickly and easily just cast to an Asset and release it's reference count, etc, etc, all the basic stuff that one would want to do to a basic asset. But I also want to deal with external asset through the same Asset interface, to make things simpler for my whole engine.

At the moment, most internal assets are composed of basic C structures, where the "Asset" structure is the first.

Example:

struct ModelTemplate{
    Asset m_Asset;
    // blah other Model and geometry related data
};

But most of my engine had been written in C style, and not C++ style, and I recently decided that it was stupid, and have spent a bit of time converting things to C++ OO style, including the Asset Managers, and the Asset struct into proper classes.

My current problem is that I can have the base Asset Manager class quite easily load from my internal data chunk format, but if I want to keep the same Asset Manager -> Asset interface, then I'll start having issues, because with C++ objects, it's quite likely that I'll at some point be dealing with virtual functions in my derived Asset classes. Which means that with External assets, I can always create an Asset object and set a ptr to the Direct 3D asset (ie. texture or shader, etc) without an issue, but then if I ever have a virtual function on a non-external assets, then I immediately have problems, because the power of the Internal asset system (chunk based format) is also it's weakness, the fact that it just copies data around and links pointers, and has no knowledge of virtual function tables, whereas when referencing data members of a class, the size of the virtual function table is automatically accounted for and offset from the beginning of the object.

I'm certain that at some point, I'll end up overwriting a virtual table of function pointers. And I don't want to have to remember and re-compile all the assets of a certain type whenever I add another virtual function to an Asset derived class.

I'm trying to think of my options, and I can only think of 2:

• Make the asset compilation much more complicated, much more akin to a full serialization, de-serialization set of methods.
• Go back to the old way of having all the assets be simple data structs without any virtual functions and write all their functionality as C-style functions which accept a ptr to the struct as the first parameter.

I'm leaning towards the second option even though it means that I'll be mixing C and C++ style code, I think that it will work better overall.

Does anyone have any suggestions? Have I been clear enough, or did I ramble a bit? Is there a standard way to handle something like this? I mean, I'm not dealing with legacy code, just an Asset Manager generic asset loader which is blind to type, and other Asset Managers which are specific and don't load the assets themselves ...

I'm not sure I get the problem. What I am sure I feel like there's a lot of confusion.

What can I say... not much besides sharing my findings and beliefs I guess.

So 1st thing. Don't focus on this basic Asset class. It bought me really little. World resources are a thing, textures another, meshes another. Instanced resources are especially quirky to manage without dissecting them in detail and although this functionality can be supported by properly designed interfaces doing so requires quite some effort in my opinion.

As 2nd, consider you might have a different representation for disk and runtime. I'm surprised you want to go (2), which is basically sending type-safety down the drain.

What I suspect is that your asset system is seriously garbled, possibly as a result of putting it in a single inheritance tree.

Sure thing, there must be no distinction between "internal" and "external" assets. If you think the renderer is something "external" that your logic has nothing to deal with you sure have to reconsider!

Regardless of what you're using, C or C++, dumping live objects to disk is a recipe for trouble. C++ at least makes this more evident by letting you know a vftable exist. No idea why the disk-format should influence your runtime structures. Think at it. If you think you can just pull an arbitrary data structure and dump its memory blob to disk, reconsider: this is a recipe for trouble even in C. Your intuition is correct: stuff going to disk must be POD and if the runtime representation is not POD there will have to be some logic to take care of the trasformation.

It's probably a better idea if I describe what type of assets I have and how I compile and deal with them.

First of all, when I "compile" an asset, I'm basically converting it into a generic data chunk format, which can set up pointers between different chunks. So, when it is loaded from this generic data chunk format, it is in it's runtime format.

These assets are static and are not intended to change. These assets are basic C structures compiled in C++, but with no member functions or virtual functions. As their first data member, they have an Asset struct so a pointer to the asset can be casted as an Asset and we can do ref counting and other basic things.

There are some bits of data that need transformation, like when loading a model (Asset type is called ModelTemplate), it needs to convert an array of vertices into a Direct X vertex buffer. This is handled specifically by the derived Asset Manager, and creates the DX vertex buffers, and then frees the data chunks that comprise the vertex and index buffers.

So, I have a base Asset Manager which does this generic deserialization, which then calls a post-load virtual function in the ModelAssetManager which does the vertex buffer conversion.

All the data chunks are POD essentially (or made up of other structures that are), and the deserialization process will create ptrs between the different chunks. These required ptrs are registered at compile time, by the source chunk, the destination chunk, and an offset into each.

Then, in order to use the loaded asset, I have a separate object that references the Asset. So, I'll create a Model object with a reference to the ModelTemplate, and this instanced object does NOT inherit or contain Asset.

So, for another example, I have an EntityTemplate which is compiled into this generic format and loaded by the Asset Manager. Then, when I want to use it to create a runtime Entity, I create a separate Entity object which refers to the EntityTemplate and use the unchanging data inside the EntityTemplate to create the Entity.

Then I have assets such as Textures and Shaders which will just be loaded directly by Direct X, but I still want to create an Asset object with a ptr to the DX texture or shader object so I can still do it through my Asset Manager interface, it's just that the derived asset manager will create a Texture object (essentially a struct with an Asset object, and a ptr to a DX texture) and get DX to load the texture instead of loading everything from my generic data chunk format. I mean, I could compile a very small header object which, as a result loads the texture using DX, but that seems unwieldy and unnecessary to have what will amount to a secondary 40 byte "Asset" header that has no real data inside it to load a texture.

I guess my issue is that I was converting everything in my engine (fairly mechanically and obviously without enough forethought) to C++ objects because previously everything (Renderer, Asset Manager, etc) were C structs and separate functions instead of C++ objects with member functions. But because for my assets everything needs to be POD, I can't use C++ objects at all because of the virtual function table issue. Well, technically I could, but every time I added a virtual function I would have to rebuild all the assets, and that's just silly.

So I was wondering if anyone had a better idea, or if there was a standard way of being able to have C++ objects with virtual functions as assets that are loaded from file without having a really complex, member by member serialize/deserialize process.