Sign in to follow this  

Game content source repository?

This topic is 972 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

What do you gentlemen/ladies use for keeping histories of your game content?

I mean things like game maps, image files, and so on?

 

I use Hg for source code, and was previously committing my artwork into the same repository, but I've come to the conclusion that this isn't a good idea, because the repository is growing too large.

Share this post


Link to post
Share on other sites

Most professional teams that I know of keep data on source control too, even if it's a separate system (which is often a good idea, as data and code have different requirements).

 

As a hobbyist though, I doubt many people need actual versioning for their data, but rather just a backup.  Having some cloud storage space somewhere is probably sufficient.

Share this post


Link to post
Share on other sites

Perforce. I don't think there is anything out there that is going to touch it for this purpose.

 

What's special about perforce besides basic versioning and bindiffs?  I'm not familiar with the product, but I can't imagine anything else really being needed, and those two things alone are pretty easy.

Share this post


Link to post
Share on other sites


I use Hg for source code, and was previously committing my artwork into the same repository, but I've come to the conclusion that this isn't a good idea, because the repository is growing too large.

I use SVN for source and art. Yes, it get bigger over time(currently ~8-10G), but I don't mind as long as it doesn't surpase the capacitiy of a blu-ray. Once it hits a critical mass I will most likely archive the art-repository and create a fresh new one from the last snapshot, but I never reached it yet smile.png

 

There are commercial tools (perforce), but does a small team (1-2 devs) really need such a tool ?

Share this post


Link to post
Share on other sites

I started off only putting source in Git, out of fear of bloating the repository, though this had the downside that people couldn't run the game from source just from the repository, they'd have to download the latest released archive of the game - which is more hassle for them, but also a problem if changes have been made to support changes to the game code (obvious example being new graphics being added).

I didn't want the hassle of having to upload art separately; I looked into the various Git add-ons, but preferred something that worked with standard Git. So now I just stick the media that's distributed with the game into Git, and it seems to work fine. I sometimes leave large files like music out of it, as these are optional. Media that isn't distributed with the game ("source" files for art, concept art, and so on) can also be left out.

Share this post


Link to post
Share on other sites

It's also both centralized (like SVN) or distributed (like Git) or set up as a hybrid of both, and there's a tool to let it integrate with git so that your coders can use git but the server is actually a perforce database.

 

Is there a similiar tool for the SVN client where the server is still Perforce?

Share this post


Link to post
Share on other sites
If you pay, Dropbox does give you the ability to retrieve old versions. We use it for our source assets at the moment because it's so easy to use, and easy to collaborate with outsourced contractors - no servers/passwords/VPNs for us to manage ourselves.
It does make me anxious though, having another company promising to keep all our core data safe though... I'd recommend local backups/archives in case anything goes wrong!

One downside to dropbox's ease of use, is that it's rediculously easy for someone to delete everything. If they just try to move the "Assets" dir out of "D:/Dropbox" (etc) then it's the same as deleting each asset one by one. Everyone else on the team will start seeing their files disappear as Dropbox dutifully replicates those changes!

Share this post


Link to post
Share on other sites
Correct me if I'm wrong, but doesn't Dropbox move all remotely deleted files into a local cache directory? At least at some point they did it like that. I noticed that because I used Dropbox to send some largish files to someone else with the recommendation to just delete them when they were done. After a while I noticed the partition in question getting awfully full until I found that cache directory...

Share this post


Link to post
Share on other sites
We use perforce here, on a large project with multiple teams around the world, to store both the code and the data.

Some things that make perforce good for this are:

- the possibility to have different locking policies per file type (you want to allow multiple checkout for sources, but exclusive checkout only for binary data such as art assets)

- the possibility (again, per file type) to limit the number of revision for which you keep the actual data. For instance, you can get it to store only the last 5 revisions of PNG files and discard earlier ones. This is vital for very large project s that really deal with a lot of data, to keep the size of the repository under control.

- perforce allows to setup proxy servers, and it works really well and allows the dozens or hundreds of people working at each studio to just talk with their local proxy, which in turn incrementally synchronize themselves with the central repository. This way the (large) data being committed elsewhere in the world is only downloaded once by your local proxy, and then everyone gets them on their PC through the lan. Imagine if a team of 100 persons had to download the same latest art assets through the internet directly...

Despite of all this it is very responsive in practice, someone on the other side of the world pushes a commit through their local proxy and you see it almost immediately. Of course when large operations are underway such as branching or large commits it tends to create some slow downs but nothing really crippling.

Share this post


Link to post
Share on other sites

I keep all my source code on a server machine hosted in my utility room.

 

It runs 24/7 and is a debian linux based machine running Git and Atlassian Stash. I also have some other bits and bobs on there like a mail server, apache2, etc.

 

I've used cvs originally (absolutely atrocious) and then subversion for many years, only just making the move into Git.

 

I tend to keep all my assets and the game code together in the same repository usually and just split it by directories, there isn't much need to split it into seperate repositories as it stands with the size of the project. I can see how you might want to on larger projects though, to avoid for example cluttering up the artist's hard disk with source code he won't ever use...

Share this post


Link to post
Share on other sites

I use Dropbox and SVN.  My project dir is in my Dropbox folder so I get all the benefits of cloud storage.  We also use SVN with Tortoise cause I like to know who did what and diff management.  I have my own Redmine server which has SVN installed on it.  I use Turnkey Linux - Redmine.  Once it's setup it's awesome.

Edited by SpiderJack

Share this post


Link to post
Share on other sites

Github is coming out with large file support, and there is a commercial git based platform that supports large files, can't remember the name.  It's based on git-annex.   So the future is looking brighter.

 

Even though perforce handles binary/large files really well, I just can't give up github workflow for code.  So I've been using amazon S3 for large binary files.  Managing versions gets tricky when working in teams.  Let me rephrase, it outright sucks.  But it's been manageable.

Share this post


Link to post
Share on other sites

This topic is 972 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this