Hi all,
I'm just pondering asset management and how it fits in with game development. At work I find our in-house 'solution' pretty awful, but I doubt I'd be allowed to speak about it so I'll keep it general.
One of the main problems I see with most asset management systems is that they work in parallel to source control rather than together with it. Programmers commit new versions of code to the source repository, everybody else commits new versions of individual assets back to the asset repository, but neither of these are in sync. Generally you'll want the concept of a 'version' - for testing, distribution, blame and so on - so what we do is take a snapshot of the assets when code is committed and call that a version. The problem this creates (a lot more often than you'd expect) is that non-technical types commit back bad assets just before a code commit, resulting in a bad version and a lot of head-scratching, and a lot of fingers pointing at the programmer.
One way round this would be to have some system in place whereby the version numbers of all assets are taken when the programmer is doing his final building and testing before a code commit, and those assets are then used in the version, so if it goes wrong the programmer has time to chase up the artist before committing. That seems a bit heavyweight to me though, I generally don't like having too many scripts floating around in addition to the core build system.
The ideal option to me seems to be to include all assets in source control, but that throws up several more problems.
The main one cited by others is that assets are generally large binaries and there is a lot of churn, which results in bloated repositories. I can't see how that's an issue since if you want complete histories for all assets you'll have to store that information somewhere.
Another issue is that due to the churn, you'll end up with a *lot* of versions, often very similar, perhaps only differing in the shade of brown on a certain texture or a one character spelling fix in a dialogue asset. I'm all for committing small changes often to combat integration headaches, but I could see that getting ridiculous.
A third problem is to do with the way we work - I don't know if it's typical, but day-to-day if I'm adding a new feature as a programmer, I'll be including a couple of assets from artists, musicians and so on, and will want to commit those along with my code as one chunk of work. Having a version with the code but not the assets would be broken, and having the assets but not the code would result in redundancy, albeit for a short time. I don't see the point in splitting one commit into two.
So, that's one set of problems - how to store assets in relation to code. I'd love to hear your thoughts on the matter!
The other thing I want to talk about is how to separate (or not) assets that are used by the game from other assets. One example would be source and built assets; let's say you have a model asset that can be loaded by the game and is built from a geometry asset and a texture asset, neither of which should be directly loadable by the game.
Would all three of these exist in the same repository? If so, you'd presumably need to label which assets are directly used by the game so you'd know which assets to package up to create a version, else you'd end up with redundant data. Also I don't like this because it increases required storage, especially in terms of history. You should only need to store source assets in history; the latest built assets are for convenience when grabbing a version.
Would the source assets live outside the repository, only versioning built assets? I could see that being a huge pain; not having source assets version controlled forces you to abandon a huge amount of potential history data. History data on built assets isn't nearly as meaningful. That said, it would be nice to be able to grab a version and have that contain exactly the built assets you need to run the game.
Or would just the source assets live in the repository, and the built asset only gets built from these on demand? This approach seems the most elegant to me, but I could see it being a huge resource hog. Any time someone wants to build the latest version they'd have to build all assets, compared with just checking out built assets from version control. Perhaps some cache of built assets on a network somewhere would be useful?
What do you chaps think about this?
Thanks in advance!
0
Asset management
Started by hymerman, Nov 04 2008 10:21 AM
13 replies to this topic
Ad:
#2 Moderators - Reputation: 3293
Posted 05 November 2008 - 01:08 AM
I haven't put a lot of thought into it, mainly because all my think-energy is drained off by other tasks at the moment, but I definitely agree that there's a gaping hole in asset version control. We use a very ad hoc and fragile system that constantly causes issues, but it's extremely hard to come up with a viable alternative because of the problems you've pointed out.
This sounds like something that would make a terrific roundtable at GDC [smile]
This sounds like something that would make a terrific roundtable at GDC [smile]
#3 Members - Reputation: 130
Posted 06 November 2008 - 11:40 AM
What about storing the assets in a separate asset management system and the code in the source code version control system with a resource description stating which versions of the assets goes with the code in the commit?
This way the artists can commit whatever they want, including work-in-progress versions they know will not play well with the current code, whenever they want, without risking to change anything behind the back of the coders and causing chaos.
Then later on when a new and shiny asset is "ready", notice is given to someone to test it with the current code (could be someone in QA or a coder), and if everything plays well the asset is promoted to approved and the resource description(s) are updated and committed.
As for source vs. built assets I would store them all (disk space is cheap), but the resource description(s) would only reference the built and final assets. I'd also make sure that all assets are stored with information about which assets went into building them. If given a final asset, for example an image sequence, you could follow the trail, trough every single image in the sequence, every model and texture and script, back to the very first concept art.
My apologies if this sounds silly (my excuse would be that I've never worked with asset management in a game development setting and I've never been required to tie code and assets together).
This way the artists can commit whatever they want, including work-in-progress versions they know will not play well with the current code, whenever they want, without risking to change anything behind the back of the coders and causing chaos.
Then later on when a new and shiny asset is "ready", notice is given to someone to test it with the current code (could be someone in QA or a coder), and if everything plays well the asset is promoted to approved and the resource description(s) are updated and committed.
As for source vs. built assets I would store them all (disk space is cheap), but the resource description(s) would only reference the built and final assets. I'd also make sure that all assets are stored with information about which assets went into building them. If given a final asset, for example an image sequence, you could follow the trail, trough every single image in the sequence, every model and texture and script, back to the very first concept art.
My apologies if this sounds silly (my excuse would be that I've never worked with asset management in a game development setting and I've never been required to tie code and assets together).
#4 Members - Reputation: 1202
Posted 06 November 2008 - 01:33 PM
After looking at what we do at work, i think my dream system would be code / art in one repository. All built data in the repository, but only updated from an automated build system.
My biggest concern that I don't know how to conquer is built code dependent on other built code, built art dependent on code, and built code dependent on art. There is a lot of dependency that causes some people to just not update because then they are broken till a programmer can sort out the issue. And that non-updating causes problems. A working build system and repository would mean that if the build server works right, the most anyone has to wait is however long it takes the server to rebuild stuff (once a day?). But even that could be a pain if the build server doesn't have a good version to build (approved auto build branch that somehow passes by QA or something?).
I think the real issue is redoing work flow around the fact that art / design / programming all do different tasks on the same asset base. But that means more tools / engine work to allow the three tasks to be seperated so art doesn't need the gameplay code to view their level, and so design doesn't need new code if they add something to a level.
Maybe the solution is to have 3 repos. One code. One assets. One buildbox. And have someone in charge of merging changes from the first two into the third?
My biggest concern that I don't know how to conquer is built code dependent on other built code, built art dependent on code, and built code dependent on art. There is a lot of dependency that causes some people to just not update because then they are broken till a programmer can sort out the issue. And that non-updating causes problems. A working build system and repository would mean that if the build server works right, the most anyone has to wait is however long it takes the server to rebuild stuff (once a day?). But even that could be a pain if the build server doesn't have a good version to build (approved auto build branch that somehow passes by QA or something?).
I think the real issue is redoing work flow around the fact that art / design / programming all do different tasks on the same asset base. But that means more tools / engine work to allow the three tasks to be seperated so art doesn't need the gameplay code to view their level, and so design doesn't need new code if they add something to a level.
Maybe the solution is to have 3 repos. One code. One assets. One buildbox. And have someone in charge of merging changes from the first two into the third?
#5 Members - Reputation: 132
Posted 06 November 2008 - 02:26 PM
For the project I'm working on, I ended up having to develop a system for asset management, since we felt that the same Code version control tools (CVS, SVN or whatever you use) didn't work that well for asset guys and also when it came to spinning off builds of assets.
Backend relies on MSSQL (with a C# .NET DLL acting as wrapper around it so you don't have to know SQL to use the system), which users being able to checkout specific assets, work on them and recommit them. While that asset is checked out, its "locked", meaning noone else can modify it and commit it back, however they can freely load it and look at the asset. Editing Tools directly communicate with this system, for example you load up the map editor, and you see a list of map assets, and who (if anyone) has a map checked out, you then click and it'll pull the map data from the backend asset system, you work on it and commit it when your done. For other things (like storing max files), we use a front end tool which allows you to browse all non editor bound assets, check them out and work with them as above. Including in all of this is a build manager, where you can click a built and perform a build (for release) of all of the assets, these builds can be perma stored (ie versioned) and the build will be saved (a build bascially stores a list of what files and what version of files are used), its during this time that if any extra preprocessing is required (ie format conversion based on platform) is done right there.
The entire system took about a month to develop, although intergration with tools did take longer and as you'd expect more functionality was added over time.
Overall I was pleased with the system apart from the fact that its very much a "1 user" lock system with the assets (meaning 2 people can't work on 1 asset and "merge" the result, but thats more something coders would do, not asset workers, well at least in my opinion).
Backend relies on MSSQL (with a C# .NET DLL acting as wrapper around it so you don't have to know SQL to use the system), which users being able to checkout specific assets, work on them and recommit them. While that asset is checked out, its "locked", meaning noone else can modify it and commit it back, however they can freely load it and look at the asset. Editing Tools directly communicate with this system, for example you load up the map editor, and you see a list of map assets, and who (if anyone) has a map checked out, you then click and it'll pull the map data from the backend asset system, you work on it and commit it when your done. For other things (like storing max files), we use a front end tool which allows you to browse all non editor bound assets, check them out and work with them as above. Including in all of this is a build manager, where you can click a built and perform a build (for release) of all of the assets, these builds can be perma stored (ie versioned) and the build will be saved (a build bascially stores a list of what files and what version of files are used), its during this time that if any extra preprocessing is required (ie format conversion based on platform) is done right there.
The entire system took about a month to develop, although intergration with tools did take longer and as you'd expect more functionality was added over time.
Overall I was pleased with the system apart from the fact that its very much a "1 user" lock system with the assets (meaning 2 people can't work on 1 asset and "merge" the result, but thats more something coders would do, not asset workers, well at least in my opinion).
#6 Members - Reputation: 212
Posted 06 November 2008 - 11:58 PM
Quote:
Original post by mm78
What about storing the assets in a separate asset management system and the code in the source code version control system with a resource description stating which versions of the assets goes with the code in the commit?
That's what I'm beginning to think is the best solution. Minimum source control clutter, maximum control over a build, maximum flexibility for asset management tools. Also it would be dead easy to write a tool to modify this resource description, which artists could use to test the game with new assets, then commit back a new version with the changed description rather than having to get programmers to integrate it. Even better, it allows programmers to pick out new assets to work with their new code and commit back a version including both, without a messy intermediate version, without having to pass files between each other outside of version control, and without the possibility of accidental unintended asset commits.
The thing is though, that it still strikes me that the asset management system would itself just be a version control system, much like the source repository...
Quote:
Original post by KulSeran
My biggest concern that I don't know how to conquer is built code dependent on other built code, built art dependent on code, and built code dependent on art
That's been on my mind too, I can't think of a way to solve that other than explicitly marking up dependencies like these.
Code relying on assets can be sort of solved by making the system heavily data-driven, meaning the code reads in the game config file and everything from then on is done via assets (the game asset specifies which level assets to load, which specify a background and objects, which specify components, which specify models and animations and so on). That way there shouldn't be any code that requires an asset to exist, and all dependencies can be worked out by parsing assets.
The other way round is pretty difficult though, there are some cases where assets need code to function correctly. Class overrides in UI files for example, or more commonly, bindings in scripts.
Quote:
Maybe the solution is to have 3 repos. One code. One assets. One buildbox. And have someone in charge of merging changes from the first two into the third?
I don't like that at all, that person would have to be pretty senior to make those kinds of decisions, which would use up time they could be spending on much more important matters. Separating code from assets using a resource description as above would be fine, and that way it'd be the code repository that is where you take the build from.
Quote:
Original post by DensitY
For the project I'm working on, I ended up having to develop a system for asset management, since we felt that the same Code version control tools (CVS, SVN or whatever you use) didn't work that well for asset guys...
Why is that? Are the tools just too programmer-oriented or is it something more fundamental to the way artist-types work?
Your approach sounds like a better-engineered version of what we do at work. It has its merits, but it suffers from the same problem with locking and a lack of merging as our system does. I find that a lot of our assets are actually text or XML, and a lot of the time people will have them locked to make tiny changes, so people have to check out unlocked, make changes out of version control, then merge by hand when the other person unlocks after committing. Proper version control would solve this silly situation very well. We haven't tried making artists use version control though, perhaps you could tell us what problems your asset people had with it? In our case I suppose the biggest problem would be that our current asset management tool provides quick access to building, editing properties and all the update/commit/lock/whatever version control stuff, all in one neat GUI, and replacements would have to be found if proper version control were to be used.
Thanks for your responses, guys, they've been very helpful :)
#7 Members - Reputation: 132
Posted 07 November 2008 - 12:45 AM
Our art guys didn't like it so much firstly because They didn't like working with CVS. Their work flow went between 3dsmax, photoshop (for textures skins, 2d art work) and the custom mapping tool, which is all gui based and going down command line level for checking stuff out, messing around with folders it just didn't work for them, Also our assets do have to go through special processing during final export (The assets are for a resourced limited system, some assets get broken up into multiple files in its final format) and having that stored in a CVS would make that process annoying on anyone wanting todo a fresh asset build.
Secondly the Artists store both the art source files (.max/.psd etc) and our exported format for all of the assets in there. when they work they generally always work with the art source files and in the case of say 3dmax they use your standard custom export plug to convert it to our working format which our build manager can take and bind the final binary versions for X platform. if 2 artists happen to update the same .max there was no easy way for them to merge it (unlike programmers source code) besides firing up versions of the asset in 3dsmax or photoshop etc. This actually created a rather big mess initially so the 'lock' was a annoying trade off for bigger problems. I won't say it was smooth sailing but once we got things like assets dropped into the system automatically showing up in the tools for them to use work flow really boost when it came to integration into the game (and for us programmers since everything was data driven, we could load specific versions of assets if we needed it was rather nice).
That seemed to work well for us. However if you're using alot of text based assets, which could merge rather well (goodbye locks). I agree with ApochPiQ, this topic would be interesting at GDC :)
Secondly the Artists store both the art source files (.max/.psd etc) and our exported format for all of the assets in there. when they work they generally always work with the art source files and in the case of say 3dmax they use your standard custom export plug to convert it to our working format which our build manager can take and bind the final binary versions for X platform. if 2 artists happen to update the same .max there was no easy way for them to merge it (unlike programmers source code) besides firing up versions of the asset in 3dsmax or photoshop etc. This actually created a rather big mess initially so the 'lock' was a annoying trade off for bigger problems. I won't say it was smooth sailing but once we got things like assets dropped into the system automatically showing up in the tools for them to use work flow really boost when it came to integration into the game (and for us programmers since everything was data driven, we could load specific versions of assets if we needed it was rather nice).
That seemed to work well for us. However if you're using alot of text based assets, which could merge rather well (goodbye locks). I agree with ApochPiQ, this topic would be interesting at GDC :)
#8 Members - Reputation: 1202
Posted 07 November 2008 - 04:10 PM
Heavy data driven is a good approach if you can.
But, if you don't have a scripting system, then code related to assets has to be compiled in. And you don't want to compile everything into a level if that level doesn't have those things, as that is just random code bloat. So that makes for some major dependency pains as you have to load assets associated with X code, and at the same time only compile X code if that asset is in the level.
But, if you don't have a scripting system, then code related to assets has to be compiled in. And you don't want to compile everything into a level if that level doesn't have those things, as that is just random code bloat. So that makes for some major dependency pains as you have to load assets associated with X code, and at the same time only compile X code if that asset is in the level.
#9 Members - Reputation: 130
Posted 09 November 2008 - 01:34 AM
Quote:
Original post by hymerman Quote:
Original post by mm78
What about storing the assets in a separate asset management system and the code in the source code version control system with a resource description stating which versions of the assets goes with the code in the commit?
That's what I'm beginning to think is the best solution. Minimum source control clutter, maximum control over a build, maximum flexibility for asset management tools. Also it would be dead easy to write a tool to modify this resource description, which artists could use to test the game with new assets, then commit back a new version with the changed description rather than having to get programmers to integrate it. Even better, it allows programmers to pick out new assets to work with their new code and commit back a version including both, without a messy intermediate version, without having to pass files between each other outside of version control, and without the possibility of accidental unintended asset commits.
The thing is though, that it still strikes me that the asset management system would itself just be a version control system, much like the source repository...
Well, maybe or maybe not, it would be some sort of version control system, but focusing on artwork rather than code.
Artwork and code are quite different and I imagine that an asset management system would be quite different from a version control system as artists and coders have different workflows. For example coders like to see diffs and patches and are happy with command line tools, while artists probably want to be able to browse the their artwork. Coders don't (or shouldn't) commit broken code (broken being won't compile or failing auto-tests), while with artwork "broken" isn't that simple. Coders branch and merge code, but I don't think artists care much about branching or merging.
#10 Members - Reputation: 212
Posted 10 November 2008 - 02:47 AM
Quote:
Original post by DensitY
Our art guys didn't like it so much firstly because They didn't like working with CVS.
... if 2 artists happen to update the same .max there was no easy way for them to merge it (unlike programmers source code) besides firing up versions of the asset in 3dsmax or photoshop etc.
Quote:
Original post by mm78
Artwork and code are quite different and I imagine that an asset management system would be quite different from a version control system as artists and coders have different workflows. For example coders like to see diffs and patches and are happy with command line tools, while artists probably want to be able to browse the their artwork
That suggests that the underlying system should be very similar or identical to source control, but it should have a happy friendly GUI over the top of it that will display thumbnails, show histories graphically, allow easy browsing and an easy interface to version control, including probably automatic updating and automatic locking for assets that don't support a merge. I like that, since it's not too big a deal and can be easily replaced since it'd just a layer over an existing and widely used system.
Quote:
Original post by mm78
Coders don't (or shouldn't) commit broken code (broken being won't compile or failing auto-tests), while with artwork "broken" isn't that simple. Coders branch and merge code, but I don't think artists care much about branching or merging.
Now that's (sort of) another thing I was going to mention... generally programmers won't be committing unless their commit means or does something. Artists will probably have files dotted around that are bits of concept art, reference pics and so on, that aren't required by the game at all but that probably still want to be versioned and made available to other people on the team. It'd be silly to have a whole separate system for such things, but at the same time keeping them together seems kind of wasteful if developers have to fetch the whole lot. Keeping a list of assets the game needs to run more or less fixes that though - as part of a build, the asset list would be read, any assets that are explicitly (or implicitly?) listed would be fetched, and everything else would be left alone.
The more I think about it, the more convinced I am that that's the best solution :)
#11 Members - Reputation: 100
Posted 24 November 2008 - 02:56 AM
Only the programmers should have code commit privelages. Noone else should be able to modify the code that others use. If an internal customer makes changes that they think should be incorporated into the base library, they should submit a request for change doc to the programmers. Too often management treats internal customers differently from external customers, which is inapropriate.
#12 Members - Reputation: 212
Posted 24 November 2008 - 03:07 AM
Artists aren't exactly 'customers' though - they're still developers, just less technical than programmers. They should still be able to modify the game. If they make a new background and test that it works they should be able to commit that. Similarly designers tweaking values in assets should be able to commit those back once they've tested ok. Frankly I'd hate it if the non-technical members of the team had to go through the programmers to get anything committed, it'd bog me down and I'm sure nobody would enjoy it, and all that would be gained is that there would be a little more testing done before the commit.
If commits are done as described above though, where even asset changes have the committer's name stamped on them, the attitude of the team would soon change and non-technical types would accept responsibility for changes and for breaks. Or not, I don't know. Arty people don't think as logically as I want them to.
If commits are done as described above though, where even asset changes have the committer's name stamped on them, the attitude of the team would soon change and non-technical types would accept responsibility for changes and for breaks. Or not, I don't know. Arty people don't think as logically as I want them to.
#13 Moderators - Reputation: 1188
Posted 28 November 2008 - 10:21 PM
I never really looked at it, but this sounds like the sort of thing Alien Brain was designed to handle.
#14 Members - Reputation: 841
Posted 28 November 2008 - 11:54 PM
A very interesting discussion here.
I'm of the mind that all source should go into Source Control, both code and assets. After all, that is code other than an asset? Intermediary files (output from 3d editor export for example) should also go in. Final assets should not, they should be generated as part of the build process from as early in the export process as possible.
It's unclear to me why anyone would want it differently really, as this is the only decent way I see to do this. A nice gui on top of the SC software would be fine, and indeed Alienbrain has nice image previewing functionality, but that can be added over the top of any SC solution.
Concept art and such doesn't need to be in the main branch of the project, and as mentioned you can always follow dependencies and fetch needed assets as you build.
And I'm not sure I fully see the issue with dependencies between code and assets. A coder making a new feature needs some assets to test with, those can be checked in (either in the main tree, or in an automated test tree). Since everything is built from sources unsupported new features in source can be ignored if there is no code to handle that yet.
I'm of the mind that all source should go into Source Control, both code and assets. After all, that is code other than an asset? Intermediary files (output from 3d editor export for example) should also go in. Final assets should not, they should be generated as part of the build process from as early in the export process as possible.
It's unclear to me why anyone would want it differently really, as this is the only decent way I see to do this. A nice gui on top of the SC software would be fine, and indeed Alienbrain has nice image previewing functionality, but that can be added over the top of any SC solution.
Concept art and such doesn't need to be in the main branch of the project, and as mentioned you can always follow dependencies and fetch needed assets as you build.
And I'm not sure I fully see the issue with dependencies between code and assets. A coder making a new feature needs some assets to test with, those can be checked in (either in the main tree, or in an automated test tree). Since everything is built from sources unsupported new features in source can be ignored if there is no code to handle that yet.
