Sign in to follow this  
Angelic Ice

Abstract Classes and returning a varying Type

Recommended Posts

Hello forum!

 

I have a base class, this base class shall have a virtual get-method but its type shall vary based on the deriving class.

Therefore, I thought I could define this as get-method as template-type and let the deriving classes just return the type they want.

 

On the one side, know what type I will get when I call the get-method, however I'm not sure if this how-it-should-be-done.

There might be about 3 or 4 deriving classes in the future and they share a similar structure method-structure but own different types of objects.

For example:

Deriving class 1 owns a buffered sound file.

Deriving class 2 owns a texture.

And so on.

 

They all will have their load-method and returning all kinds of meta-data. However, letting them return their specific loaded content of data is the problem, as it varies.

 

Does this point out that I should prefer create one complete class that does not derive instead of forcing abstract classes?

 

Thanks for taking your time to read my thread.

I would be happy if you could help me out!

 

Share this post


Link to post
Share on other sites
If the caller will know what type of data they're returning, that can be a templated method that returns the exact type you ask for.

If the caller won't know what type of data they're returning, and the generic functions on that object aren't very useful, it's probably worthless having a generic method that returns all those different things.

I could probably give a better answer if your question was a bit more concrete. It's not clear why you want to do this.

Share this post


Link to post
Share on other sites

I want to manage my loaded assets, for every asset there shall be one derived class that holds it. These derived classes are allocated by another class.

Textures would be one deriving class, sounds another one, and so on.

 

If an objects needs to use a texture, they can use the appropriate shared pointer that they own. This pointer shall point to one of those mentioned derived class.

The object always knows what it needs, requests it from a class that manages the assets and obtains a shared pointer or even creates a new entry for a new asset.

Therefore, the object using the assets (textures, sound, ..) will have a shared pointer to the derived class of the needed asset.

 

Now, the objects needs to actually use the loaded texture at some point.

Hence the object that wants to use it, needs to use its shared pointer and call the get-method in order to obtain the allocated asset.

 

I hope my case is clearer now : ) Nonetheless, I think this pretty much would suit the template-variant?

Share this post


Link to post
Share on other sites

You may consider making it a bit more generic - with some template programming you could have resources of any type and not coupled with some common base class which is not really a good solution (like deriving from Resource class or similar). Not only it will have broader use (for example you could use some external classes that you can't derive from common base without modifying some library), but also should be faster because you won't use virtual calls for something so crucial as getting resource id, something that will happen a lot each frame.

 

Think of a handle like Resource<Texture>, Resource<Shader>, Resource<Config> etc. You could use resources that don't necessarily use the same key to fetch them (so for example instead of std::string some more sophisticated struct or maybe uint?). 

This way you don't have the problem you seem to describe - that you have several "semi similar" classes, but they all have their own quirks and data. If you have Resource<Texture> you know the object you're dealing with is a texture. If you have Resource<Shader> you know it's a shader and may have completly different interface than texture. You are tying your hands with a common interface for things that may be completly unrelated in terms of their functionality, and only share one thing - that they're loaded and managed in a way that minimizes duplicates. 

 

In terms of loading, that's where you specify what resource you're querying:

 

auto texture = resources.Get<Texture>("mytexture");

auto shader = resources.Get<Shader>(ShaderKey("myshader", shaderBitMask));

auto framebuffer = resources.Get<Framebuffer>(2); 

 

etc.

Edited by noizex

Share this post


Link to post
Share on other sites

I hope my case is clearer now : ) Nonetheless, I think this pretty much would suit the template-variant?

 

Usually that's a good smell that you're doing something wrong.

 

Usually different types of resources are generated by different types of factory methods.  If you use a factory method for game models and are passing in a texture resource, that's generally a problem.

 

In larger engines games tend to work in the pattern of a collection of a store, loader, cache, and proxy for whatever resource type you need.  Each is unique for the interface, such as a texture type, a shader type, a model type, and each can be specialized.  A texture factory may be able to provide many types of concrete texture types (DXT3, DXT5, etc) that all implement the same base class and are interchangeable as far as class users go. 

 

 

If you really believe in using a unified resource loader, the model suggested by noizex above can work.  You're still going to implement all the behavior for each resource type, it will be specialized via template specialization so all that work still needs to happen.  It's just that instead of TextureStore.Get and ModelStore.Get that return a pointer to their corresponding data types, you'll have resources.Get<Texture> and resources.Get<Model> that do exactly the same thing.

Share this post


Link to post
Share on other sites

Thanks a lot, everyone : )

 

Noizex, I understood the implementation but the key-identifier part. While I understood that std::strings can be a bit of a power consuming thing to compare, how would I do it with integers?

Well, example:

A game object gets created, it knows a file path of its texture. Now it requests the asset-manager if this has been allocated already. The asset-manager would iterate through a list of collected assets with all their <type>s.

What I thought about is to compare the strings with each other, as every asset-item knows its original path. Storing std::strings is not that lovely for such an undertaking, especially comparing them. If I would use something like you suggested, I could simply compare numbers, is that right?

 

The new game object would simply send the type it is looking for together with an ID, maybe send the file-path as well, to directly allocate if it does not exist.

But how would I define what asset would get what number? Or is there something better?

 

 

 

In larger engines games tend to work in the pattern of a collection of a store, loader, cache, and proxy for whatever resource type you need. Each is unique for the interface, such as a texture type, a shader type, a model type, and each can be specialized. A texture factory may be able to provide many types of concrete texture types (DXT3, DXT5, etc) that all implement the same base class and are interchangeable as far as class users go.

That sounds really interesting! Does this indicate, they do not use templates for this case but have separated classes?

Could you maybe elaborate what a "texture factory"-class would do? You mentioned DXT3 and 5, compression types, is that right? Would this part make the textures usable? Decompress them?

It would be wonderful if you could give me an example for how these classes (store, loader, cache, and proxy) interact with each other, as this sounds really interesting to me!

Edited by Angelic Ice

Share this post


Link to post
Share on other sites

Could you maybe elaborate what a "texture factory"-class would do?

 

Search for the term "factory method" or "abstract factory".  You might pass in a string, and the method builds something and returns it.

 

In this kind of example, you might pass a string labeled "someimage.dds". The method would crack open the file, figure out what kind of file it is, and do whatever creation work is necessary. Then it gives you back a pointer to a texture that represents whatever the contents of the image happen to be.

 

You mentioned DXT3 and 5, compression types, is that right? Would this part make the textures usable? Decompress them?

 

That would defeat the purpose. Those texture types are designed to go on to the card directly so you don't need to spend the time, compute power, or memory to decompress them.

 

It would be wonderful if you could give me an example for how these classes (store, loader, cache, and proxy) interact with each other, as this sounds really interesting to me!

 

The store is the thing that controls all the access.  You say "give me a resource" and the store returns something.  The thing it returns is not actually the final object at all, but is a proxy for the final object. A proxy allows for the actual data to be loaded as needed.  For example, a proxy for a texture may not be the actual loaded texture because there might be a limited amount of memory.  The proxy contains enough information that other systems can use the object and intelligently swap it out, which is why it needs to be custom for each resource type. The proxy object can be a placeholder initially, no other resources attached, so that other systems don't need to wait around. The loader is a thing that loads the data into the proxy, it might load from a file, load from a network stream, load from a database, wherever. The loader is able to get the resource into memory.  The cache is the thing that actually contains the data as long as it is loaded.  When resources are tight something identifies the pressure --- maybe another resource needs to be loaded and there isn't room, or maybe the system is facing other memory pressure --- the cache unloads a resource it isn't used any more and the proxy points back to the placeholder.

 

 

An example:

 

Let's say you are loading a level. Your rendering system needs models and textures.  The code looks up the names of the models and textures from your art asset information.  It calls the model store and requests a Model* for every one of the file names. The model store immediately returns a proxy object that implements the Model interface. (An alternative would be to wait several milliseconds for the data to load from disk.) Since the actual model data isn't available yet, it gives you a default placeholder object, maybe a small gray box. Similarly calls are made for all the textures, and the texture store returns texture proxies in the form of a Texture* for each one rather than waiting for the data to load from disk. Perhaps the default proxy value gives you a default placeholder object of a pink solid value.  You begin to add the items in your scene, and in the background the engine starts loading resources, probably from disk.  As the loader completes, it will internally provide the actual models and textures and replace the pointers inside the proxy object. The loader puts the object inside the cache, possibly evicting (unloading) other objects from the cache.  When the renderer renders the Model* and Texture* items it is written to be friendly with the other systems so they don't accidentally break anything.  Later, when you request another texture from the store, the store can check to see if the texture is already loaded or in use.  If it is, the store can immediately return a proxy that points to the existing model rather than pointing to the solid pink placeholder value.

 

Repeat the process for any other kind of resource.  Textures and models work with the graphics cards and need to be coordinated one way.  Audio clips need to be coordinated another way, and audio streams may need yet another way.  You may have other resource types, such as script files, shader files, level map files, or whatever else you need.

Share this post


Link to post
Share on other sites

Thanks a lot, everyone : )
 
Noizex, I understood the implementation but the key-identifier part. While I understood that std::strings can be a bit of a power consuming thing to compare, how would I do it with integers?
Well, example:
A game object gets created, it knows a file path of its texture. Now it requests the asset-manager if this has been allocated already. The asset-manager would iterate through a list of collected assets with all their <type>s.
What I thought about is to compare the strings with each other, as every asset-item knows its original path. Storing std::strings is not that lovely for such an undertaking, especially comparing them. If I would use something like you suggested, I could simply compare numbers, is that right?
 

The new game object would simply send the type it is looking for together with an ID, maybe send the file-path as well, to directly allocate if it does not exist.
But how would I define what asset would get what number? Or is there something better?

 

I think you're confusing several concepts here and complicate unnecessarily. When you load the resource you want to supply whatever ID you need to use to identify this resource and its version on a medium that you store it. It may be file path, it may be just a name or numeric id, it may be something more complex like a struct that contains both, filename and version. It doesn't matter what type it will be, as long as your loader knows how to deal with it. One thing matters - type should be hashable because you want to store references to your resources in a unique_map or similar container for fast lookup. You don't want iterate over a vector and look up for the resource comparing strings, as you suggest above.

 

So for resource key just use whatever type is most convenient for you. Strings are perfectly fine for this, as they're hashable. You can make it all compile-time even with some C++ magic, but that's probably only needed if you do a lot of lookups on thousands of resources.

In my case, I don't ask manager each time I need the resource (so, each frame just before rendering), I use a simple refcount and don't return plain pointer but a handle that can be resolved to a pointer (this also acts as a proxy and allows returning placeholder when doing async load). This resolving doesn't use a map but rather a vector, and a handle keeps index to the resource in that vector (this is the main container for my resources, map just holds references in case the key-lookup is needed through Get<T> method). This requires designing a good policy of when the handle is valid, what happens when resources in vector need to be moved to other indices and so on, but otherwise is fast where it's needed and allows easy lookup when it's needed.

 

In terms of code it looks a bit like this snippet:

struct Model {
   Resource<Shader> shader; 
   Resource<Texture> texture;
   Resource<Mesh> mesh;

   void LoadResources(ResourceManager& resources)
   {
      # Here it does the std::unique_map lookup on the Store class (mind that Store class is provided by default
      # but also can be user-defined with completly different storage implementation, so this unique_map is just 
      # what is the usual case)

      shader = resources.Get<Shader>(ShaderKey("base_shader", E_NormalMapped | E_Lighting));
      texture = resources.Get<Texture>("texture.dds");
      mesh = resource.Get<Mesh>("mesh.iqm");
   }
}

Now a bit on what happens when you Get<T>:

  1. Find appropriate meta data for resource type T - meta data contains load/store/retrieve function callbacks, Store object and Loader object
  2. Call Store->Retrieve(key) to check if resource exists in the cache
    1. If resource exists, Store returns Resource<T> handle to it and increases refcount
    2. If resource doesn't exist Store returns empty handle
  3. If handle is empty, proceed to loading the object, so Loader->Load(key) is called
    1. Depending on the loader, it either fetches resource from somewhere, or creates it using factory class, this all happens inside Loader and other classes like Store or ResourceManager itself don't know anything about it. Loader just has to return the resource (this time it's unique_ptr) or nullptr
  4. When loader returns and the handle is not empty, it gets stored into Store, increasing refcount to 1 (so the store callback is called with the unique_ptr, which is then moved into the Store, as the store is the owning entity for resources)
  5. Handle is returned to user

Later on, when I actually need to use the resource I do it as if it were a normal pointer to resource it holds:

RenderModel(mesh->GetVertexBuffer(), mesh->GetIndexBuffer(), texture->GetId(), shader.get());

When -> is called, the handle does the call on the owning Store (to which it holds a pointer) like:

m_Owner->Get(m_ResourceIndex);

where m_ResourceIndex is index into resource vector. This way it can return placeholder, or do all sort of things, because it has this indirection which you don't get if you point straight to Resource* pointer.

 

So to rendering system I usually pass straight numbers of resources (GLuints in my case), but sometimes I need resource object itself like in Shader case (I need to call few more methods and Shader keeps the state for this, like caching uniforms per shader type). This is why last argument is passed as shader.get(), which returns plain pointer to Shader*, just like std::unique_ptr and similar smart pointers. There is a policy that resource manager will never free any resource during rendering phase, so passing raw pointer and holding to it for a while (during the journey of RenderTask through queues, sorting, and into actual render call) is fine because we guarantee that the resource will stay alive during that time.

 

It's still a naive implementation so there may be some problems, but it worked for me so far and it's quite flexible so I hope it helps you a bit and not adds more confusion :)

Edited by noizex

Share this post


Link to post
Share on other sites

Thanks a lot, this helps me a lot : )

 

I somehow forgot hash tables, that suits this case way better than simple lists.

 

Do you, by chance, know a project that uses this kind of implementation (proxy, cache, loader, and store)? I would love to take a look at how they made it work.

 

It calls the model store and requests a Model* for every one of the file names.

Would you implement the store as a template or create one full class per needed type of resource? This counts for the other three ones as well.

 

The model store immediately returns a proxy object that implements the Model interface. (An alternative would be to wait several milliseconds for the data to load from disk.) Since the actual model data isn't available yet, it gives you a default place-holder object, maybe a small gray box.

So, the store creates a proxy class and this proxy class has a default place holder variable. However... does not this require async loading? I feel like that would an extreme overkill for my project. As my needed textures are way too fast to load. It would also add all the struggle and issues that come with multi-threading.

Would the store check whether the proxy is done and then move it to the cache?

I'm not sure how the ownership works between the store and the cache.

 

Later, when you request another texture from the store, the store can check to see if the texture is already loaded or in use.

This sounds like the store looks in the cache's hash map. But where is the proxy as long as it is not fully loaded? Is it still in the cache? Does the store check for hash map elements that return true to be fully loaded?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this