Jump to content
  • Advertisement
Sign in to follow this  
jeff8j

Filesystem for compiling over network

This topic is 2507 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Im looking for a protocol to use thats easy to implement I like webdav but im worried that about requirements of programs and compilers.

I want to run several machines sharing the same drive and I think webdav locks can handle that right?

I have programs that read files from the middle and append data to files and im thinking that some programs overwrite certain data without rewriting the whole file. Can web dav handle all that?

Share this post


Link to post
Share on other sites
Advertisement
I want to use it generically for various things but the most immediate problem is compiling and debugging. I want to compile debug and run all from the network drive.

Share this post


Link to post
Share on other sites
Just use whatever your OS natively supports. For Windows that is SMB/CIFS, for Linux it's probably NFS, and for Mac it's AFP (in a mixed Windows/Linux/Mac environment you can use any of the above, provided you install a few drivers).

Share this post


Link to post
Share on other sites
For distributed builds, ccache and distcc are typical free options.

For other forms of builds it gets trickier.

Locks of any kind are simply not possible. Since tools are generic, any file they touch becomes locked. So "shared.hpp" will be locked and all other services that attempt to open it will fail. Simple shared file systems are also intended merely for decentralized access, not distributed computing. File systems which work transparently in this way are in domain of high-end mainframe solutions.

Other options is obviously something like git/hg. By using versioning and some replication method, perhaps in combination with Hudson/Jenkins, one can run distributed builds, but with emphasis on distribution rather than concurrency.

Anything else and you hit ACID-related issues that databases solve, but aren't directly usable as a file system.

Share this post


Link to post
Share on other sites
Im using multiple operating systems ubuntu and windows right now but plan on adding a mac into the mix to bring my products to mac. I would like to avoid installing anything thats what drove me to webdav.

Im not concerned with distributed builds building on each computer is fine but I want the results and sources shared. I want to skip the whole checkout process with git/hg/svn.

I was thinking with locking the computers would be forced to take turns waiting for each other and since I dont have many users it wouldnt be long I dont really see many people writing to the same files at once but I would like multiple reads at once.

"Anything else and you hit ACID-related issues that databases solve, but aren't directly usable as a file system."
Its funny you say that because I was looking into a webdav system built in php that uses mysql as the internal filesystem thats why I was asking if web dav supported everything I need

Share this post


Link to post
Share on other sites
So you want a shared directory, that multiple computers can access? I'm really not sure why you are getting into such complexities in your solution (hint: webdav has horrible performance, compared to real network shares).

Just create a SMB share on whichever computer you want to physically host the files. Mac and Windows have support for mounting SMB out of the box, on Ubuntu you'll have to 'apt-get install smbfs', and that's about it.

Share this post


Link to post
Share on other sites
I use samba now and I feel like its more headache than its worth I probably dont know enough about it but its horribly slow and not reliable. To access lots of small files for compiling its horribly slow and sometimes it doesnt work at all and takes a restart and waiting 20minutes or so before it shows up I cant have that.

Share this post


Link to post
Share on other sites

I use samba now and I feel like its more headache than its worth I probably dont know enough about it but its horribly slow and not reliable

That sounds more like a network issue. I run a 40-computer lab with their home directories on a samba server in a remote location, and it's fine for even heavy research/development tasks.

I assume you are using a wired network over cat6 cabling via a gigabit switch (no WiFi, no 10/100 switches), with no other major traffic over the wire?

Share this post


Link to post
Share on other sites
Its a 10/100 router with wifi its never really maxed out bandwidth wise and I need wifi for laptops and presentations. It does have cat6 though and all the main computers are wired. I may be trying to get rid of samba when I should be looking more into whats going on.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!