I have an online game with unknown count of object types, so we cannot load before game process all the meshes and textures (someone can create objects while game runs, like in SecondLife).

So I think about smart resource manager implementation. I'll explain how I make it and please fix my mistakes...

I have a separate thread, which processes the queue of requests (we see new object, so we need to load mesh, texture and animation). If there are no such mesh/texture/animation on client, then ask server to send it.

All the requests are processed one by one, because object needs model->texture->animation to do not get animation until mesh is loaded (we cannot assign texture to null mesh). Also what to do if we need to change a mesh, but current mesh is still loading. Sometimes objects load very slow if we wait for server data, and there is about hundred objects must be loaded from local filesystem, but they don't because we cannot assign texture to null mesh, or animation to null mesh.

Maybe I need to make one-by-one queue per object? Or make this functionality inside mesh or texture? But what to do if I have a man with a gun in his hand? Until I get the man skinned mesh, I cannot assign a gun there.

Or I need to remove my one-by-one queue and check like this:

Creature::Creature(int mesh, int texture){    loadModel(mesh);    loadTexture(texture);    m_meshID = mesh;    m_textureID = texture;}void Creature::onLoad(int type, int id, Object* obj){    if (type == MESH)    {        // We got mesh, but soneone assigned new one in setMesh-like function        if(id != m_meshID)        {            return;        }        m_mesh = (Mesh*) obj;        if(m_texture)        {            m_mesh->setTexture(m_texture);        }    }    else if (type == TEXTURE)    {        // We got texture, but soneone assigned new one in setTexture-like function        if(id != m_textureID)        {            return;        }        m_texture = (Texture*) obj;        if(m_mesh)        {            m_mesh->setTexture(m_texture);        }    }}

An "object" as seen by the game has a bunch of things:
- zero or more meshes
- zero or more materials (which may have textures)
- zero or more skeletons
- zero or more animations
- zero or more particle effects
- zero or more rigid body properties
- zero or more collision shapes
- zero or more behaviors
- ...

You probably want a component object model, where each component is added to the object as it gets downloaded.
You may also want a dependency model, where each piece can know what it depends on, and whether that dependency is satisfied yet or not.
For example, you don't want the rigid body to start simulating before you have the collision shapes, or you'd fall through the ground, never to be found again.
In the simplest case, you simply send a name, a rough size estimation (bounding box?) and a list of all the things that will be there, and then the object just displays as a question mark with a label, until all the components have fully loaded, at which point it goes operating.

Yes, but what order should be for loading:
1. Single queue. If some object needs loading from internet then next ones will "wait". So we see a lot of invisible (placeholder) items sometimes. It's easy to destroy object - just place destroy "command" and all the object parts will be unloaded because on this queue state all the items are loaded.
2. Queue per object. Hard to detect what is the object. Objects are simple entities, skinned entities, menus (html-like pages).
3. We load object as they go. We ask to load mesh, texture and animation, but first get animation, then mesh, then texture. So we apply animation to placeholder object, then replace placeholder mesh, and then change texture.

Just what solution do you preffer?

Because if I choose 3rd, then how to process that:
1. Entity asked about mesh loading.
2. Entity asked about texture loading.
3. Got mesh.
4. Entity asked about texture change.
5. Texture change arrived.
6. Got old texture that replaced new one...

Presumably, there's a server-side copy of the object that knows the "full state" of the object. It should not generally be the case that, after loading some mesh file, the object suddenly discovers it needs a texture that it previously didn't know. If you're doing it that way, you're not pre-processing your data sufficiently. Thus, you should know all the assets needed to get an object operating when you start loading that object (see the example I posted above).

If you don't want to pre-process to have all knowledge about each object, then you will need to know, for each object, what it is waiting for. When something it's waiting for arrives, the object has a chance to unpack whatever it is, and perhaps request more assets -- doing so adds those assets to the queue. The object is considered complete when a loaded asset is returned to the object, and after delivering the asset data, the queue for that object is empty.

An alternative is to create stand-in assets for each asset kind (texture, shader, mesh, etc), and then replace the data in-flight when the real data comes online.

A network/disk/cache scheduler that slurps stuff from file servers or whatever is separate from a queue per object. The fact that an asset is loading within an object, just means that the object isn't "done." Separately, each such request per-object goes to a central asset loading system, which can schedule disk, network and memory load. Thus, you likely need both queues, because they contain subtly different data, and they represent different things.

What is the easiest:
1. Make server to know what every client needs (what textures are stored there, what meshes are stored...)
2. Every client asks server when it needs some data.

I would choose 1, but in games where administration has created some city, when you entered some part of it you need some part of data and if client had never been there, then send pack of it's data.

But here, I as user can load texture from my hard disk right now, I place it on my favourite object, and everybody must see it! And there's no need to load more than once.

It's like a browser. Server doesn't send all the images, videos if you have never been there. Maybe there's a video or image that you saw on another site so why do you need to reload it?

About queues... I didn't understand... Do you advise queue per object? Or load resources in order they come (local textures will come faster than server-meshes)...

And what about object releasing? We don't need this object anymore, so it should be deleted. And as resource manager stores just one instance for one unique data (two similar textures are only two in world, but one in ResMan). So object needs to decrement some ResMan counter. But how to do this if texture is still loading from server?

Quote:Original post by Anton Vatchenko
What is the easiest:
1. Make server to know what every client needs (what textures are stored there, what meshes are stored...)
2. Every client asks server when it needs some data.

I would choose 1, but in games where administration has created some city, when you entered some part of it you need some part of data and if client had never been there, then send pack of it's data.

I would say number 2 is easier. Imagine you have 1000 clients each of them creating 100 resources of their own (a total of 100,000 different resources), suddenly you have to keep track of 100,000 resources for all 1000 clients, that results in 100,000,000 posts or whatever you call it.

Compare this to the server requesting a client to load resource X from disk (perhaps comparing a checksum to detect updates), if client does not have it, it downloads it from the server (perhaps P2P would be a good alternative).
This way you do not have to keep track of hundreds of millions of resources.

I know that better to ask server about new objects. I ask about the type of resource managing in this way. Use one queue, or use queue per game entity, or something else? Maybe make no queues and have problems as I've already told, please read accurately.

I already answered this question.

You have one queue per object, so that each object can know whether it's "done."

You then have one queue for the network/disk/cache subsystem, where the object queue elements will say "please deliver this data to me when available."

Typically, you'll want to have some number (4?) outstanding network requests, and some number (4?) outstanding disk requests at any one time, to maximize overall throughput.

The object queue works something like:

object.startup:  add all known assets to per-object queueobject.onAssetFetched:  load the asset  add any new references to the per-object queue  if the per-object queue is empty, the object is readyobject.addNewReference:  ask central loader for asset  if available, just use it and return  add a pending taskloader.getAsset:  if asset in RAM cache:    return asset  queue disk cache requestloader.onDiskCacheResult:  if asset was on disk, return to requester  else start network requestloader.onNetworkRequestResult:  put asset onto disk  return asset to requester

Additionally, you need each "asset" to be capable of serving more than one user, because object A may say "I need texture/foobar.jpg," and while that's loading from network, object B may say the same thing; that should not cause a second network request, beacuse one is already pending.