Using jgit To Publish on Amazon S3

Recent versions of jgit, the 100% pure Java implementation of the Git version control system, support fetch and push directly over Amazon S3 .

It behaves like http push does in C git in that it is transparent to the end-user. Transparent client-side encryption can also be enabled, in case the repository data must be protected from the operators of S3.

First you need to create a bucket using some sort of standard S3 tools. I used jets3t's cockpit tool to create "gitney". A bucket may hold any number of repositories and acts as a root directory. It may also be a domain name if you want to use S3 based virtual hosting.

Next you need to create a properties file containing your AWSAccessKeyId and AWSSecretAccessKey so that jgit can authenticate itself with the S3 service. Since the AWSSecretAccessKey should be maintained privately its a good idea to store this in a protected file within your home directory.

  $ touch ~/.jgit_s3_public
  $ chmod 600 ~/.jgit_s3_public
  $ cat >>~/.jgit_s3_public
  accesskey: AWSAccessKeyId
  secretkey: AWSSecretAccessKey
  acl: public
  EOF

We also include acl: public so all objects (files) created by jgit through this configuration file are readable by anyone. The default (if not specified) is acl: private, making the objects readable only by yourself, and those who manage the S3 service.

Next we configure the remote in Git and push to the S3 bucket:

  $ git remote add s3 amazon-s3://.jgit_s3_public@gitney/projects/egit.git/
  $ jgit push s3 refs/heads/master
  $ jgit push --tags s3

Future updates are just as easy:

  $ jgit push s3 refs/heads/master

(or)

  $ git config --add remote.s3.push refs/heads/master
  $ jgit push s3

Pushes are always incremental and consequently there is relatively little bandwidth usage during subsequent pushes.

Our repository is now cloneable directly over HTTP (assuming we used acl: public):

  $ git clone http://gitney.s3.amazonaws.com/projects/egit.git

A jgit amazon-s3 URL is organized as:

  amazon-s3://$config@$bucket/$prefix
  http://$bucket.s3.amazonaws.com/$prefix

where the three major components are:

• $config is the name of the configuration properties file stored in $GIT_DIR/$config or $HOME/$config (searched for in that order).


• $bucket is the name of the Amazon S3 bucket holding the objects.


• $prefix is the prefix to apply to all objects (files) within this repository. It implicitly ends in "/". You may omit this portion of the URI if you want the bucket to contain only one repository.

This is something of an abuse of URI syntax as the traditional username field is holding the name of a file in either $GIT_DIR or $HOME, however it permits hiding the secret access key from prying eyes as well as supplies a way to carry more information (such as acl or encryption settings) than what can appear in a URI.

Transparent client-side encryption for a repository stored on S3 can be enabled by adding a password to the properties file:

  $ cp ~/.jgit_s3_public ~/.jgit_s3_private
  $ echo password: Sup3rS3cr3t >>~/.jgit_s3_private

and using .jgit_s3_private in the $config field of an amazon-s3:// URL. The encryption algorithm can also be specified in property crypto.algorithm, which defaults to PBEWithMD5AndDES.

The encryption format currently used by jgit matches the format used by jets3t (specifically format version 2), making it possible to download and decrypt a repository through cockpit in the event that jgit is not readily available.