Velocity Reviews - Computer Hardware Reviews

Velocity Reviews > Newsgroups > Programming > Python > [ANN] git peer-to-peer bittorrent experiment: first milestone reached

Reply
Thread Tools

[ANN] git peer-to-peer bittorrent experiment: first milestone reached

 
 
Luke Kenneth Casson Leighton
Guest
Posts: n/a
 
      09-01-2010
http://gitorious.org/python-libbittorrent/pybtlib

this is to let people know that a first milestone has been reached in
an experiment to combine git with a file-sharing protocol, thus making
it possible to use git for truly distributed software development and
other file-revision-management operations (such as transparently
turning git-configured ikiwiki and moinmoin wikis into peer-to-peer
ones).

the milestone reached is to transfer git commit "pack objects", as if
they were ordinary files, over a bittorrent network, and have them
"unpacked" at the far end. the significance of being able to transfer
git commit pack objects is that this is the core of the "git fetch"
command.

the core of this experiment comprises a python-based VFS layer,
providing alternatives to os.listdir, os.path.exists, open and so on -
sufficient to make an interesting experiment itself by combining that
VFS layer with e.g. python-fuse.

the bittornado library, also available at the above URL, has been
modified to take a VFS module as an argument to all operations, such
that it would be conceivable to share maildir mailboxes, mailing list
archives, .tar.gz archives, .deb and .rpm archives and so on, as if
they were files and directories within a file-sharing network.

as the core code has only existed for under three days, and is only
400 lines long, there are rough edges:

* all existing commit objects are unpacked at startup time and are
stored in-memory (!). this is done so as to avoid significant
modification of the bittorrent library, which will be required.
* all transferred commit objects are again stored in-memory before
being unpacked. so, killing the client will lose all transfers
received up to that point.

on the roadmap:

* make things efficient! requires modification of the bittornado library.
* create some documentation!
* explore how to make git use this code as a new URI type so that it
will be possible to just do "git pull"
* explore how to use PGP/GPG to sign commits(?) or perhaps just
tags(?) in order to allow commits to be pulled only from trusted
parties.
* share all branches and tags as well as just refs/heads/*
* make "git push" re-create the .torrent (make_torrent.py) and work out
how to notify seeders of a new HEAD (name the torrent after the HEAD ref,
and just create a new one rather than delete the old?)

so there is quite a bit to do, with the priority being on making a new
URI type and a new "git-remote-{URI}" command, so that this becomes
actually useable rather than just an experiment, and the project can
be self-hosting as a truly distributed peer-to-peer development
effort.

if anyone would like to assist, you only have to ask and (ironically)
i will happily grant access to the gitorious-hosted repository.

if anyone would like to sponsor this project, that would be very
timely, as if i don't get some money soon i will be unable to pay for
food and rent.

l.
 
Reply With Quote
 
 
 
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Eclipse Web Tools Platform (WTP) project Milestone M5 Vikram Sorathia Java 3 07-05-2005 01:28 PM
Eclipse Web Tools Platform (WTP) project Milestone M5 Vikram Sorathia XML 0 07-05-2005 12:10 PM
Blu-ray Disc Association Surpasses 100th Member Milestone. Allan DVD Video 0 02-16-2005 01:17 PM
woohoo SETI, another milestone reached. Bret NZ Computing 16 11-22-2003 10:40 PM
Milestone DVD Sale Mile Films DVD Video 0 11-11-2003 05:36 PM



Advertisments