Sign in to follow this  
jbadams

Replication RAID functionality (symlinks)

Recommended Posts

jbadams    25712
I'm trying to solve an odd little problem and would like some feedback on my proposed solution. Given a large collection of files on multiple disk drives, I need to present a single point-of-access for the files as if they were on a single volume. I'm not allowed to use RAID to achieve this. The directory structure needs to be maintained, but there are no duplicate directories across different drives; the complete contents of any given directory are all on the same disk and no other disk contains a directory with the same name. Once created files will not be removed unless the entire dataset is scrapped, but new files may be added. After a quick bit of research my proposed solution is to create a utility that does the following:
  1. Duplicate the complete directory structure from all of the disks on a single volume; this will be done on an additional disk, not one of the ones containing the data.
  2. Populate the newly created directory structure with Symbolic Links to the original files.
  3. Given the above has already been done, the utility will search through the existing set of files and symlinks and create additional links to any newly added files.
As I understand it the symbolic links will act as the original files for purposes of I/O, and as files won't be removed individually (only if the whole set is removed - in which case all the symlinks can go) I shouldn't have to worry about the source files not being there, which seems to be a potential problem with symlinks. I'm confident that the above will work, but does anyone else see any potential issues or perhaps have suggestions (other than the obvious "use RAID you idiot!") for a more elegant solution? Thanks guys. [smile]

Share this post


Link to post
Share on other sites
cache_hit    614
Why don't you just use a directory junction to the original directory rather than symlinks to the individual files inside the directory? This makes maintenance a non-issue, any changes that happen on the original disk will automatically be reflected.

Share this post


Link to post
Share on other sites
jbadams    25712
Because I hadn't heard of them, I'll look into it. Thanks for the tip, I suspected there might be some simpler alternatives I wasn't aware of!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this