[ Date Index ] [ Thread Index ] [ <= Previous by date / thread ] [ Next by date / thread => ]
On 27/12/13 17:41, Mesar Hameed wrote: > On Fri 27/12/13,16:45, bad apple wrote: >> On 27/12/13 07:35, Mesar Hameed wrote: >>> I am doing something like this with git-annex, across multiple >>> sites for personal data (approx 800 gb). It helps by not having >>> a single point of failure (central server),, and I can easely >>> ensure that each file is in at least x distinct locations. >>> >>> git-annex does support encryption, so all that is required from >>> a fellow data sharer is an open ssh port and git-annex on >>> their machine. >>> >>> People just starting with git-annex might want to try the >>> assistant, but it is still somewhat buggy. I would definitely >>> recommend trying out the command line interface for any >>> serious usage. >>> >>> In the last two or so years,, it really has simplified my file >>> management life including redundancy worries. >>> >>> For more info please see: http://git-annex.branchable.com/ >> >> Whilst this does look quite interesting, how has it solved the >> notorious git issues with large files and binaries? > > Think of it as an content/location management layer on top of rsync > ( other transport layers available), the git repo only contains > meta information about the files, and their locations and symlinks > to blobs. These blobs are stored in a known location, but not in > git. If not available in the current repo, it will try to get it > from one of the other configured remotes, or tell you to make the > repo available. > > http://git-annex.branchable.com/how_it_works/ > >> If you're shunting 800Gb of data about happily presumably this >> has either been ignored or solved somehow (I'm guessing the >> former). > > solved, see above. It is good for static content files, such as > music, videos etc, but need additional attension when trying to > share changing content files such as vm images. > >> Surely you're not just moving 800Gb of text files between >> machines. > > Binary in the real sence. > > >> Love to see what your commit logs look like - completely >> unreadable I imagine. > > If you use the assistant probably, but if you use the command line > it is as good or as bad as you make them. The attached files > contains some commits. The git-annex branch which is used by the > software to sync and to track metadata is less readable, but there > is no need to read those commits. > > I recommend having a look at the walkthrough and setting up two > test repos. > > > thanks, Mesar Yeah, even a cursory read through the actual site (which I have now done) would have told me all of that - serves me right for not RTFM I guess. Knowing how git fails to handle even relatively large files and binaries in general should really have clued me in that git-annex would simply be hashing the files and versioning metadata rather than git'ing the actual files themselves... This is looking to solve a problem that already has many solutions available but it still looks interesting enough that I am indeed going to fire it up and give it a test. Thanks for the info. Regards -- The Mailing List for the Devon & Cornwall LUG http://mailman.dclug.org.uk/listinfo/list FAQ: http://www.dcglug.org.uk/listfaq