Im looking for a protocol to use thats easy to implement I like webdav but im worried that about requirements of programs and compilers.
I want to run several machines sharing the same drive and I think webdav locks can handle that right?
I have programs that read files from the middle and append data to files and im thinking that some programs overwrite certain data without rewriting the whole file. Can web dav handle all that?
Filesystem for compiling over network
Author
I want to use it generically for various things but the most immediate problem is compiling and debugging. I want to compile debug and run all from the network drive.
Just use whatever your OS natively supports. For Windows that is SMB/CIFS, for Linux it's probably NFS, and for Mac it's AFP (in a mixed Windows/Linux/Mac environment you can use any of the above, provided you install a few drivers).
For distributed builds, ccache and distcc are typical free options.
For other forms of builds it gets trickier.
Locks of any kind are simply not possible. Since tools are generic, any file they touch becomes locked. So "shared.hpp" will be locked and all other services that attempt to open it will fail. Simple shared file systems are also intended merely for decentralized access, not distributed computing. File systems which work transparently in this way are in domain of high-end mainframe solutions.
Other options is obviously something like git/hg. By using versioning and some replication method, perhaps in combination with Hudson/Jenkins, one can run distributed builds, but with emphasis on distribution rather than concurrency.
Anything else and you hit ACID-related issues that databases solve, but aren't directly usable as a file system.
For other forms of builds it gets trickier.
Locks of any kind are simply not possible. Since tools are generic, any file they touch becomes locked. So "shared.hpp" will be locked and all other services that attempt to open it will fail. Simple shared file systems are also intended merely for decentralized access, not distributed computing. File systems which work transparently in this way are in domain of high-end mainframe solutions.
Other options is obviously something like git/hg. By using versioning and some replication method, perhaps in combination with Hudson/Jenkins, one can run distributed builds, but with emphasis on distribution rather than concurrency.
Anything else and you hit ACID-related issues that databases solve, but aren't directly usable as a file system.
Author
Im using multiple operating systems ubuntu and windows right now but plan on adding a mac into the mix to bring my products to mac. I would like to avoid installing anything thats what drove me to webdav.
Im not concerned with distributed builds building on each computer is fine but I want the results and sources shared. I want to skip the whole checkout process with git/hg/svn.
I was thinking with locking the computers would be forced to take turns waiting for each other and since I dont have many users it wouldnt be long I dont really see many people writing to the same files at once but I would like multiple reads at once.
"Anything else and you hit ACID-related issues that databases solve, but aren't directly usable as a file system."
Its funny you say that because I was looking into a webdav system built in php that uses mysql as the internal filesystem thats why I was asking if web dav supported everything I need
Im not concerned with distributed builds building on each computer is fine but I want the results and sources shared. I want to skip the whole checkout process with git/hg/svn.
I was thinking with locking the computers would be forced to take turns waiting for each other and since I dont have many users it wouldnt be long I dont really see many people writing to the same files at once but I would like multiple reads at once.
"Anything else and you hit ACID-related issues that databases solve, but aren't directly usable as a file system."
Its funny you say that because I was looking into a webdav system built in php that uses mysql as the internal filesystem thats why I was asking if web dav supported everything I need
So you want a shared directory, that multiple computers can access? I'm really not sure why you are getting into such complexities in your solution (hint: webdav has horrible performance, compared to real network shares).
Just create a SMB share on whichever computer you want to physically host the files. Mac and Windows have support for mounting SMB out of the box, on Ubuntu you'll have to 'apt-get install smbfs', and that's about it.
Just create a SMB share on whichever computer you want to physically host the files. Mac and Windows have support for mounting SMB out of the box, on Ubuntu you'll have to 'apt-get install smbfs', and that's about it.
Author
I use samba now and I feel like its more headache than its worth I probably dont know enough about it but its horribly slow and not reliable. To access lots of small files for compiling its horribly slow and sometimes it doesnt work at all and takes a restart and waiting 20minutes or so before it shows up I cant have that.
I use samba now and I feel like its more headache than its worth I probably dont know enough about it but its horribly slow and not reliable
That sounds more like a network issue. I run a 40-computer lab with their home directories on a samba server in a remote location, and it's fine for even heavy research/development tasks.
I assume you are using a wired network over cat6 cabling via a gigabit switch (no WiFi, no 10/100 switches), with no other major traffic over the wire?
Author
Its a 10/100 router with wifi its never really maxed out bandwidth wise and I need wifi for laptops and presentations. It does have cat6 though and all the main computers are wired. I may be trying to get rid of samba when I should be looking more into whats going on.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement