Package org.eclipse.jgit.transport

Class AmazonS3

  • public class AmazonS3
extends java.lang.Object
    A simple HTTP REST client for the Amazon S3 service.

    This client uses the REST API to communicate with the Amazon S3 servers and read or write content through a bucket that the user has access to. It is a very lightweight implementation of the S3 API and therefore does not have all of the bells and whistles of popular client implementations.

    Authentication is always performed using the user's AWSAccessKeyId and their private AWSSecretAccessKey.

    Optional client-side encryption may be enabled if requested. The format is compatible with jets3t, a popular Java based Amazon S3 client library. Enabling encryption can hide sensitive data from the operators of the S3 service.

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      private static class  AmazonS3.KeyInfo
      KeyInfo enables sorting of keys by lastModified time
      (package private) static interface  AmazonS3.Keys
      Property names used in amazon connection configuration file.
      private class  AmazonS3.ListParser  

    • Field Summary

      Fields 
      Modifier and Type Field Description
      private java.lang.String acl
      ACL to apply to created objects.
      private java.lang.String domain
      S3 Bucket Domain.
      private WalkEncryption encryption
      Encryption algorithm, may be a null instance that provides pass-through.
      private static java.lang.String HMAC  
      (package private) int maxAttempts
      Maximum number of times to try an operation.
      private javax.crypto.spec.SecretKeySpec privateKey
      Decoded form of the private AWSSecretAccessKey, to sign requests.
      private java.net.ProxySelector proxySelector
      Our HTTP proxy support, in case we are behind a firewall.
      private java.lang.String publicKey
      AWSAccessKeyId, public string that identifies the user's account.
      private static java.util.Set<java.lang.String> SIGNED_HEADERS  
      private java.io.File tmpDir
      Directory for locally buffered content.
      private static java.lang.String X_AMZ_ACL  
      private static java.lang.String X_AMZ_META  

    • Constructor Summary

      Constructors 
      Constructor Description
      AmazonS3​(java.util.Properties props)
      Create a new S3 client for the supplied user information.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      (package private) void authorize​(java.net.HttpURLConnection c)  
      java.io.OutputStream beginPut​(java.lang.String bucket, java.lang.String key, ProgressMonitor monitor, java.lang.String monitorTask)
      Atomically create or replace a single large object.
      java.io.InputStream decrypt​(java.net.URLConnection u)
      Decrypt an input stream from get(String, String).
      void delete​(java.lang.String bucket, java.lang.String key)
      Delete a single object.
      (package private) java.io.IOException error​(java.lang.String action, java.lang.String key, java.net.HttpURLConnection c)  
      java.net.URLConnection get​(java.lang.String bucket, java.lang.String key)
      Get the content of a bucket object.
      private static java.lang.String httpNow()  
      private static boolean isSignedHeader​(java.lang.String name)  
      java.util.List<java.lang.String> list​(java.lang.String bucket, java.lang.String prefix)
      List the names of keys available within a bucket.
      (package private) java.io.IOException maxAttempts​(java.lang.String action, java.lang.String key)  
      private static java.security.MessageDigest newMD5()  
      private java.net.HttpURLConnection open​(java.lang.String method, java.lang.String bucket, java.lang.String key)  
      (package private) java.net.HttpURLConnection open​(java.lang.String method, java.lang.String bucket, java.lang.String key, java.util.Map<java.lang.String,​java.lang.String> args)  
      (package private) static java.util.Properties properties​(java.io.File authFile)  
      void put​(java.lang.String bucket, java.lang.String key, byte[] data)
      Atomically create or replace a single small object.
      (package private) void putImpl​(java.lang.String bucket, java.lang.String key, byte[] csum, TemporaryBuffer buf, ProgressMonitor monitor, java.lang.String monitorTask)  
      private static java.lang.String remove​(java.util.Map<java.lang.String,​java.lang.String> m, java.lang.String k)  
      private static java.lang.String toCleanString​(java.util.List<java.lang.String> list)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • SIGNED_HEADERS

        private static final java.util.Set<java.lang.String> SIGNED_HEADERS

      • publicKey

        private final java.lang.String publicKey
        AWSAccessKeyId, public string that identifies the user's account.

      • privateKey

        private final javax.crypto.spec.SecretKeySpec privateKey
        Decoded form of the private AWSSecretAccessKey, to sign requests.

      • proxySelector

        private final java.net.ProxySelector proxySelector
        Our HTTP proxy support, in case we are behind a firewall.

      • acl

        private final java.lang.String acl
        ACL to apply to created objects.

      • maxAttempts

        final int maxAttempts
        Maximum number of times to try an operation.

      • encryption

        private final WalkEncryption encryption
        Encryption algorithm, may be a null instance that provides pass-through.

      • tmpDir

        private final java.io.File tmpDir
        Directory for locally buffered content.

      • domain

        private final java.lang.String domain
        S3 Bucket Domain.

    • Constructor Detail

      • AmazonS3

        public AmazonS3​(java.util.Properties props)
        Create a new S3 client for the supplied user information.

        The connection properties are a subset of those supported by the popular jets3t library. For example: 

         # AWS Access and Secret Keys (required)
 accesskey: <YourAWSAccessKey>
 secretkey: <YourAWSSecretKey>

 # Access Control List setting to apply to uploads, must be one of:
 # PRIVATE, PUBLIC_READ (defaults to PRIVATE).
 acl: PRIVATE

 # S3 Domain
 # AWS S3 Region Domain (defaults to s3.amazonaws.com)
 domain: s3.amazonaws.com

 # Number of times to retry after internal error from S3.
 httpclient.retry-max: 3

 # End-to-end encryption (hides content from S3 owners)
 password: <encryption pass-phrase>
 crypto.algorithm: PBEWithMD5AndDES
        Parameters:
        props - connection properties.

    • Method Detail

      • isSignedHeader

        private static boolean isSignedHeader​(java.lang.String name)

      • toCleanString

        private static java.lang.String toCleanString​(java.util.List<java.lang.String> list)

      • remove

        private static java.lang.String remove​(java.util.Map<java.lang.String,​java.lang.String> m,
                                       java.lang.String k)

      • httpNow

        private static java.lang.String httpNow()

      • newMD5

        private static java.security.MessageDigest newMD5()

      • get

        public java.net.URLConnection get​(java.lang.String bucket,
                                  java.lang.String key)
                           throws java.io.IOException
        Get the content of a bucket object.
        Parameters:
        bucket - name of the bucket storing the object.
        key - key of the object within its bucket.
        Returns:
        connection to stream the content of the object. The request properties of the connection may not be modified by the caller as the request parameters have already been signed.
        Throws:
        java.io.IOException - sending the request was not possible.

      • decrypt

        public java.io.InputStream decrypt​(java.net.URLConnection u)
                            throws java.io.IOException
        Decrypt an input stream from get(String, String).
        Parameters:
        u - connection previously created by get(String, String)}.
        Returns:
        stream to read plain text from.
        Throws:
        java.io.IOException - decryption could not be configured.

      • list

        public java.util.List<java.lang.String> list​(java.lang.String bucket,
                                             java.lang.String prefix)
                                      throws java.io.IOException
        List the names of keys available within a bucket.

        This method is primarily meant for obtaining a "recursive directory listing" rooted under the specified bucket and prefix location. It returns the keys sorted in reverse order of LastModified time (freshest keys first).

        Parameters:
        bucket - name of the bucket whose objects should be listed.
        prefix - common prefix to filter the results by. Must not be null. Supplying the empty string will list all keys in the bucket. Supplying a non-empty string will act as though a trailing '/' appears in prefix, even if it does not.
        Returns:
        list of keys starting with prefix, after removing prefix (or prefix + "/")from all of them.
        Throws:
        java.io.IOException - sending the request was not possible, or the response XML document could not be parsed properly.

      • delete

        public void delete​(java.lang.String bucket,
                   java.lang.String key)
            throws java.io.IOException
        Delete a single object.

        Deletion always succeeds, even if the object does not exist.

        Parameters:
        bucket - name of the bucket storing the object.
        key - key of the object within its bucket.
        Throws:
        java.io.IOException - deletion failed due to communications error.

      • put

        public void put​(java.lang.String bucket,
                java.lang.String key,
                byte[] data)
         throws java.io.IOException
        Atomically create or replace a single small object.

        This form is only suitable for smaller contents, where the caller can reasonable fit the entire thing into memory.

        End-to-end data integrity is assured by internally computing the MD5 checksum of the supplied data and transmitting the checksum along with the data itself.

        Parameters:
        bucket - name of the bucket storing the object.
        key - key of the object within its bucket.
        data - new data content for the object. Must not be null. Zero length array will create a zero length object.
        Throws:
        java.io.IOException - creation/updating failed due to communications error.

      • beginPut

        public java.io.OutputStream beginPut​(java.lang.String bucket,
                                     java.lang.String key,
                                     ProgressMonitor monitor,
                                     java.lang.String monitorTask)
                              throws java.io.IOException
        Atomically create or replace a single large object.

        Initially the returned output stream buffers data into memory, but if the total number of written bytes starts to exceed an internal limit the data is spooled to a temporary file on the local drive.

        Network transmission is attempted only when close() gets called at the end of output. Closing the returned stream can therefore take significant time, especially if the written content is very large.

        End-to-end data integrity is assured by internally computing the MD5 checksum of the supplied data and transmitting the checksum along with the data itself.

        Parameters:
        bucket - name of the bucket storing the object.
        key - key of the object within its bucket.
        monitor - (optional) progress monitor to post upload completion to during the stream's close method.
        monitorTask - (optional) task name to display during the close method.
        Returns:
        a stream which accepts the new data, and transmits once closed.
        Throws:
        java.io.IOException - if encryption was enabled it could not be configured.

      • putImpl

        void putImpl​(java.lang.String bucket,
             java.lang.String key,
             byte[] csum,
             TemporaryBuffer buf,
             ProgressMonitor monitor,
             java.lang.String monitorTask)
      throws java.io.IOException
        Throws:
        java.io.IOException

      • error

        java.io.IOException error​(java.lang.String action,
                          java.lang.String key,
                          java.net.HttpURLConnection c)
                   throws java.io.IOException
        Throws:
        java.io.IOException

      • maxAttempts

        java.io.IOException maxAttempts​(java.lang.String action,
                                java.lang.String key)

      • open

        private java.net.HttpURLConnection open​(java.lang.String method,
                                        java.lang.String bucket,
                                        java.lang.String key)
                                 throws java.io.IOException
        Throws:
        java.io.IOException

      • open

        java.net.HttpURLConnection open​(java.lang.String method,
                                java.lang.String bucket,
                                java.lang.String key,
                                java.util.Map<java.lang.String,​java.lang.String> args)
                         throws java.io.IOException
        Throws:
        java.io.IOException

      • authorize

        void authorize​(java.net.HttpURLConnection c)
        throws java.io.IOException
        Throws:
        java.io.IOException

      • properties

        static java.util.Properties properties​(java.io.File authFile)
                                throws java.io.FileNotFoundException,
                                       java.io.IOException
        Throws:
        java.io.FileNotFoundException
        java.io.IOException