diff options
Diffstat (limited to 'src/ceph/doc/radosgw/s3')
| -rw-r--r-- | src/ceph/doc/radosgw/s3/authentication.rst | 75 | ||||
| -rw-r--r-- | src/ceph/doc/radosgw/s3/bucketops.rst | 377 | ||||
| -rw-r--r-- | src/ceph/doc/radosgw/s3/commons.rst | 111 | ||||
| -rw-r--r-- | src/ceph/doc/radosgw/s3/cpp.rst | 331 | ||||
| -rw-r--r-- | src/ceph/doc/radosgw/s3/csharp.rst | 199 | ||||
| -rw-r--r-- | src/ceph/doc/radosgw/s3/java.rst | 212 | ||||
| -rw-r--r-- | src/ceph/doc/radosgw/s3/objectops.rst | 403 | ||||
| -rw-r--r-- | src/ceph/doc/radosgw/s3/perl.rst | 192 | ||||
| -rw-r--r-- | src/ceph/doc/radosgw/s3/php.rst | 208 | ||||
| -rw-r--r-- | src/ceph/doc/radosgw/s3/python.rst | 171 | ||||
| -rw-r--r-- | src/ceph/doc/radosgw/s3/ruby.rst | 364 | ||||
| -rw-r--r-- | src/ceph/doc/radosgw/s3/serviceops.rst | 39 |
12 files changed, 2682 insertions, 0 deletions
diff --git a/src/ceph/doc/radosgw/s3/authentication.rst b/src/ceph/doc/radosgw/s3/authentication.rst new file mode 100644 index 0000000..b187538 --- /dev/null +++ b/src/ceph/doc/radosgw/s3/authentication.rst @@ -0,0 +1,75 @@ +========================= + Authentication and ACLs +========================= + +Requests to the RADOS Gateway (RGW) can be either authenticated or +unauthenticated. RGW assumes unauthenticated requests are sent by an anonymous +user. RGW supports canned ACLs. + +Authentication +-------------- +Authenticating a request requires including an access key and a Hash-based +Message Authentication Code (HMAC) in the request before it is sent to the +RGW server. RGW uses an S3-compatible authentication approach. + +:: + + HTTP/1.1 + PUT /buckets/bucket/object.mpeg + Host: cname.domain.com + Date: Mon, 2 Jan 2012 00:01:01 +0000 + Content-Encoding: mpeg + Content-Length: 9999999 + + Authorization: AWS {access-key}:{hash-of-header-and-secret} + +In the foregoing example, replace ``{access-key}`` with the value for your access +key ID followed by a colon (``:``). Replace ``{hash-of-header-and-secret}`` with +a hash of the header string and the secret corresponding to the access key ID. + +To generate the hash of the header string and secret, you must: + +#. Get the value of the header string. +#. Normalize the request header string into canonical form. +#. Generate an HMAC using a SHA-1 hashing algorithm. + See `RFC 2104`_ and `HMAC`_ for details. +#. Encode the ``hmac`` result as base-64. + +To normalize the header into canonical form: + +#. Get all fields beginning with ``x-amz-``. +#. Ensure that the fields are all lowercase. +#. Sort the fields lexicographically. +#. Combine multiple instances of the same field name into a + single field and separate the field values with a comma. +#. Replace white space and line breaks in field values with a single space. +#. Remove white space before and after colons. +#. Append a new line after each field. +#. Merge the fields back into the header. + +Replace the ``{hash-of-header-and-secret}`` with the base-64 encoded HMAC string. + +Access Control Lists (ACLs) +--------------------------- + +RGW supports S3-compatible ACL functionality. An ACL is a list of access grants +that specify which operations a user can perform on a bucket or on an object. +Each grant has a different meaning when applied to a bucket versus applied to +an object: + ++------------------+--------------------------------------------------------+----------------------------------------------+ +| Permission | Bucket | Object | ++==================+========================================================+==============================================+ +| ``READ`` | Grantee can list the objects in the bucket. | Grantee can read the object. | ++------------------+--------------------------------------------------------+----------------------------------------------+ +| ``WRITE`` | Grantee can write or delete objects in the bucket. | N/A | ++------------------+--------------------------------------------------------+----------------------------------------------+ +| ``READ_ACP`` | Grantee can read bucket ACL. | Grantee can read the object ACL. | ++------------------+--------------------------------------------------------+----------------------------------------------+ +| ``WRITE_ACP`` | Grantee can write bucket ACL. | Grantee can write to the object ACL. | ++------------------+--------------------------------------------------------+----------------------------------------------+ +| ``FULL_CONTROL`` | Grantee has full permissions for object in the bucket. | Grantee can read or write to the object ACL. | ++------------------+--------------------------------------------------------+----------------------------------------------+ + +.. _RFC 2104: http://www.ietf.org/rfc/rfc2104.txt +.. _HMAC: http://en.wikipedia.org/wiki/HMAC diff --git a/src/ceph/doc/radosgw/s3/bucketops.rst b/src/ceph/doc/radosgw/s3/bucketops.rst new file mode 100644 index 0000000..c7cd5b4 --- /dev/null +++ b/src/ceph/doc/radosgw/s3/bucketops.rst @@ -0,0 +1,377 @@ +=================== + Bucket Operations +=================== + +PUT Bucket +---------- +Creates a new bucket. To create a bucket, you must have a user ID and a valid AWS Access Key ID to authenticate requests. You may not +create buckets as an anonymous user. + +.. note:: We do not support request entities for ``PUT /{bucket}`` in this release. + +Constraints +~~~~~~~~~~~ +In general, bucket names should follow domain name constraints. + +- Bucket names must be unique. +- Bucket names must begin and end with a lowercase letter. +- Bucket names may contain a dash (-). + +Syntax +~~~~~~ + +:: + + PUT /{bucket} HTTP/1.1 + Host: cname.domain.com + x-amz-acl: public-read-write + + Authorization: AWS {access-key}:{hash-of-header-and-secret} + +Parameters +~~~~~~~~~~ + ++---------------+----------------------+-----------------------------------------------------------------------------+------------+ +| Name | Description | Valid Values | Required | ++===============+======================+=============================================================================+============+ +| ``x-amz-acl`` | Canned ACLs. | ``private``, ``public-read``, ``public-read-write``, ``authenticated-read`` | No | ++---------------+----------------------+-----------------------------------------------------------------------------+------------+ + + + +HTTP Response +~~~~~~~~~~~~~ + +If the bucket name is unique, within constraints and unused, the operation will succeed. +If a bucket with the same name already exists and the user is the bucket owner, the operation will succeed. +If the bucket name is already in use, the operation will fail. + ++---------------+-----------------------+----------------------------------------------------------+ +| HTTP Status | Status Code | Description | ++===============+=======================+==========================================================+ +| ``409`` | BucketAlreadyExists | Bucket already exists under different user's ownership. | ++---------------+-----------------------+----------------------------------------------------------+ + +DELETE Bucket +------------- + +Deletes a bucket. You can reuse bucket names following a successful bucket removal. + +Syntax +~~~~~~ + +:: + + DELETE /{bucket} HTTP/1.1 + Host: cname.domain.com + + Authorization: AWS {access-key}:{hash-of-header-and-secret} + +HTTP Response +~~~~~~~~~~~~~ + ++---------------+---------------+------------------+ +| HTTP Status | Status Code | Description | ++===============+===============+==================+ +| ``204`` | No Content | Bucket removed. | ++---------------+---------------+------------------+ + +GET Bucket +---------- +Returns a list of bucket objects. + +Syntax +~~~~~~ + +:: + + GET /{bucket}?max-keys=25 HTTP/1.1 + Host: cname.domain.com + +Parameters +~~~~~~~~~~ + ++-----------------+-----------+-----------------------------------------------------------------------+ +| Name | Type | Description | ++=================+===========+=======================================================================+ +| ``prefix`` | String | Only returns objects that contain the specified prefix. | ++-----------------+-----------+-----------------------------------------------------------------------+ +| ``delimiter`` | String | The delimiter between the prefix and the rest of the object name. | ++-----------------+-----------+-----------------------------------------------------------------------+ +| ``marker`` | String | A beginning index for the list of objects returned. | ++-----------------+-----------+-----------------------------------------------------------------------+ +| ``max-keys`` | Integer | The maximum number of keys to return. Default is 1000. | ++-----------------+-----------+-----------------------------------------------------------------------+ + + +HTTP Response +~~~~~~~~~~~~~ + ++---------------+---------------+--------------------+ +| HTTP Status | Status Code | Description | ++===============+===============+====================+ +| ``200`` | OK | Buckets retrieved | ++---------------+---------------+--------------------+ + +Bucket Response Entities +~~~~~~~~~~~~~~~~~~~~~~~~ +``GET /{bucket}`` returns a container for buckets with the following fields. + ++------------------------+-----------+----------------------------------------------------------------------------------+ +| Name | Type | Description | ++========================+===========+==================================================================================+ +| ``ListBucketResult`` | Entity | The container for the list of objects. | ++------------------------+-----------+----------------------------------------------------------------------------------+ +| ``Name`` | String | The name of the bucket whose contents will be returned. | ++------------------------+-----------+----------------------------------------------------------------------------------+ +| ``Prefix`` | String | A prefix for the object keys. | ++------------------------+-----------+----------------------------------------------------------------------------------+ +| ``Marker`` | String | A beginning index for the list of objects returned. | ++------------------------+-----------+----------------------------------------------------------------------------------+ +| ``MaxKeys`` | Integer | The maximum number of keys returned. | ++------------------------+-----------+----------------------------------------------------------------------------------+ +| ``Delimiter`` | String | If set, objects with the same prefix will appear in the ``CommonPrefixes`` list. | ++------------------------+-----------+----------------------------------------------------------------------------------+ +| ``IsTruncated`` | Boolean | If ``true``, only a subset of the bucket's contents were returned. | ++------------------------+-----------+----------------------------------------------------------------------------------+ +| ``CommonPrefixes`` | Container | If multiple objects contain the same prefix, they will appear in this list. | ++------------------------+-----------+----------------------------------------------------------------------------------+ + +Object Response Entities +~~~~~~~~~~~~~~~~~~~~~~~~ +The ``ListBucketResult`` contains objects, where each object is within a ``Contents`` container. + ++------------------------+-----------+------------------------------------------+ +| Name | Type | Description | ++========================+===========+==========================================+ +| ``Contents`` | Object | A container for the object. | ++------------------------+-----------+------------------------------------------+ +| ``Key`` | String | The object's key. | ++------------------------+-----------+------------------------------------------+ +| ``LastModified`` | Date | The object's last-modified date/time. | ++------------------------+-----------+------------------------------------------+ +| ``ETag`` | String | An MD-5 hash of the object. (entity tag) | ++------------------------+-----------+------------------------------------------+ +| ``Size`` | Integer | The object's size. | ++------------------------+-----------+------------------------------------------+ +| ``StorageClass`` | String | Should always return ``STANDARD``. | ++------------------------+-----------+------------------------------------------+ + +Get Bucket Location +------------------- +Retrieves the bucket's region. The user needs to be the bucket owner +to call this. A bucket can be constrained to a region by providing +``LocationConstraint`` during a PUT request. + +Syntax +~~~~~~ +Add the ``location`` subresource to bucket resource as shown below + +:: + + GET /{bucket}?location HTTP/1.1 + Host: cname.domain.com + + Authorization: AWS {access-key}:{hash-of-header-and-secret} + +Response Entities +~~~~~~~~~~~~~~~~~~~~~~~~ + ++------------------------+-----------+------------------------------------------+ +| Name | Type | Description | ++========================+===========+==========================================+ +| ``LocationConstraint`` | String | The region where bucket resides, empty | +| | | string for defult region | ++------------------------+-----------+------------------------------------------+ + + + +Get Bucket ACL +-------------- +Retrieves the bucket access control list. The user needs to be the bucket +owner or to have been granted ``READ_ACP`` permission on the bucket. + +Syntax +~~~~~~ +Add the ``acl`` subresource to the bucket request as shown below. + +:: + + GET /{bucket}?acl HTTP/1.1 + Host: cname.domain.com + + Authorization: AWS {access-key}:{hash-of-header-and-secret} + +Response Entities +~~~~~~~~~~~~~~~~~ + ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| Name | Type | Description | ++===========================+=============+==============================================================================================+ +| ``AccessControlPolicy`` | Container | A container for the response. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``AccessControlList`` | Container | A container for the ACL information. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Owner`` | Container | A container for the bucket owner's ``ID`` and ``DisplayName``. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``ID`` | String | The bucket owner's ID. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``DisplayName`` | String | The bucket owner's display name. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Grant`` | Container | A container for ``Grantee`` and ``Permission``. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Grantee`` | Container | A container for the ``DisplayName`` and ``ID`` of the user receiving a grant of permission. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Permission`` | String | The permission given to the ``Grantee`` bucket. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ + +PUT Bucket ACL +-------------- +Sets an access control to an existing bucket. The user needs to be the bucket +owner or to have been granted ``WRITE_ACP`` permission on the bucket. + +Syntax +~~~~~~ +Add the ``acl`` subresource to the bucket request as shown below. + +:: + + PUT /{bucket}?acl HTTP/1.1 + +Request Entities +~~~~~~~~~~~~~~~~ + ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| Name | Type | Description | ++===========================+=============+==============================================================================================+ +| ``AccessControlPolicy`` | Container | A container for the request. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``AccessControlList`` | Container | A container for the ACL information. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Owner`` | Container | A container for the bucket owner's ``ID`` and ``DisplayName``. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``ID`` | String | The bucket owner's ID. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``DisplayName`` | String | The bucket owner's display name. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Grant`` | Container | A container for ``Grantee`` and ``Permission``. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Grantee`` | Container | A container for the ``DisplayName`` and ``ID`` of the user receiving a grant of permission. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Permission`` | String | The permission given to the ``Grantee`` bucket. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ + +List Bucket Multipart Uploads +----------------------------- + +``GET /?uploads`` returns a list of the current in-progress multipart uploads--i.e., the application initiates a multipart upload, but +the service hasn't completed all the uploads yet. + +Syntax +~~~~~~ + +:: + + GET /{bucket}?uploads HTTP/1.1 + +Parameters +~~~~~~~~~~ + +You may specify parameters for ``GET /{bucket}?uploads``, but none of them are required. + ++------------------------+-----------+--------------------------------------------------------------------------------------+ +| Name | Type | Description | ++========================+===========+======================================================================================+ +| ``prefix`` | String | Returns in-progress uploads whose keys contains the specified prefix. | ++------------------------+-----------+--------------------------------------------------------------------------------------+ +| ``delimiter`` | String | The delimiter between the prefix and the rest of the object name. | ++------------------------+-----------+--------------------------------------------------------------------------------------+ +| ``key-marker`` | String | The beginning marker for the list of uploads. | ++------------------------+-----------+--------------------------------------------------------------------------------------+ +| ``max-keys`` | Integer | The maximum number of in-progress uploads. The default is 1000. | ++------------------------+-----------+--------------------------------------------------------------------------------------+ +| ``max-uploads`` | Integer | The maximum number of multipart uploads. The range from 1-1000. The default is 1000. | ++------------------------+-----------+--------------------------------------------------------------------------------------+ +| ``upload-id-marker`` | String | Ignored if ``key-marker`` is not specified. Specifies the ``ID`` of first | +| | | upload to list in lexicographical order at or following the ``ID``. | ++------------------------+-----------+--------------------------------------------------------------------------------------+ + + +Response Entities +~~~~~~~~~~~~~~~~~ + ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| Name | Type | Description | ++=========================================+=============+==========================================================================================================+ +| ``ListMultipartUploadsResult`` | Container | A container for the results. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``ListMultipartUploadsResult.Prefix`` | String | The prefix specified by the ``prefix`` request parameter (if any). | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Bucket`` | String | The bucket that will receive the bucket contents. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``KeyMarker`` | String | The key marker specified by the ``key-marker`` request parameter (if any). | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``UploadIdMarker`` | String | The marker specified by the ``upload-id-marker`` request parameter (if any). | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``NextKeyMarker`` | String | The key marker to use in a subsequent request if ``IsTruncated`` is ``true``. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``NextUploadIdMarker`` | String | The upload ID marker to use in a subsequent request if ``IsTruncated`` is ``true``. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``MaxUploads`` | Integer | The max uploads specified by the ``max-uploads`` request parameter. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Delimiter`` | String | If set, objects with the same prefix will appear in the ``CommonPrefixes`` list. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``IsTruncated`` | Boolean | If ``true``, only a subset of the bucket's upload contents were returned. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Upload`` | Container | A container for ``Key``, ``UploadId``, ``InitiatorOwner``, ``StorageClass``, and ``Initiated`` elements. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Key`` | String | The key of the object once the multipart upload is complete. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``UploadId`` | String | The ``ID`` that identifies the multipart upload. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Initiator`` | Container | Contains the ``ID`` and ``DisplayName`` of the user who initiated the upload. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``DisplayName`` | String | The initiator's display name. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``ID`` | String | The initiator's ID. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Owner`` | Container | A container for the ``ID`` and ``DisplayName`` of the user who owns the uploaded object. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``StorageClass`` | String | The method used to store the resulting object. ``STANDARD`` or ``REDUCED_REDUNDANCY`` | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Initiated`` | Date | The date and time the user initiated the upload. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``CommonPrefixes`` | Container | If multiple objects contain the same prefix, they will appear in this list. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``CommonPrefixes.Prefix`` | String | The substring of the key after the prefix as defined by the ``prefix`` request parameter. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ + +ENABLE/SUSPEND BUCKET VERSIONING +-------------------------------- + +``PUT /?versioning`` This subresource set the versioning state of an existing bucket. To set the versioning state, you must be the bucket owner. + +You can set the versioning state with one of the following values: + +- Enabled : Enables versioning for the objects in the bucket, All objects added to the bucket receive a unique version ID. +- Suspended : Disables versioning for the objects in the bucket, All objects added to the bucket receive the version ID null. + +If the versioning state has never been set on a bucket, it has no versioning state; a GET versioning request does not return a versioning state value. + +Syntax +~~~~~~ + +:: + + PUT /{bucket}?versioning HTTP/1.1 + +REQUEST ENTITIES +~~~~~~~~~~~~~~~~ + ++-----------------------------+-----------+---------------------------------------------------------------------------+ +| Name | Type | Description | ++=============================+===========+===========================================================================+ +| ``VersioningConfiguration`` | Container | A container for the request. | ++-----------------------------+-----------+---------------------------------------------------------------------------+ +| ``Status`` | String | Sets the versioning state of the bucket. Valid Values: Suspended/Enabled | ++-----------------------------+-----------+---------------------------------------------------------------------------+ diff --git a/src/ceph/doc/radosgw/s3/commons.rst b/src/ceph/doc/radosgw/s3/commons.rst new file mode 100644 index 0000000..ca848bc --- /dev/null +++ b/src/ceph/doc/radosgw/s3/commons.rst @@ -0,0 +1,111 @@ +================= + Common Entities +================= + +.. toctree:: + :maxdepth: -1 + +Bucket and Host Name +-------------------- +There are two different modes of accessing the buckets. The first (preferred) method +identifies the bucket as the top-level directory in the URI. :: + + GET /mybucket HTTP/1.1 + Host: cname.domain.com + +The second method identifies the bucket via a virtual bucket host name. For example:: + + GET / HTTP/1.1 + Host: mybucket.cname.domain.com + +To configure virtual hosted buckets, you can either set ``rgw_dns_name = cname.domain.com`` in ceph.conf, or add ``cname.domain.com`` to the list of ``hostnames`` in your zonegroup configuration. See `Ceph Object Gateway - Multisite Configuration`_ for more on zonegroups. + +.. tip:: We prefer the first method, because the second method requires expensive domain certification and DNS wild cards. + +Common Request Headers +---------------------- + ++--------------------+------------------------------------------+ +| Request Header | Description | ++====================+==========================================+ +| ``CONTENT_LENGTH`` | Length of the request body. | ++--------------------+------------------------------------------+ +| ``DATE`` | Request time and date (in UTC). | ++--------------------+------------------------------------------+ +| ``HOST`` | The name of the host server. | ++--------------------+------------------------------------------+ +| ``AUTHORIZATION`` | Authorization token. | ++--------------------+------------------------------------------+ + +Common Response Status +---------------------- + ++---------------+-----------------------------------+ +| HTTP Status | Response Code | ++===============+===================================+ +| ``100`` | Continue | ++---------------+-----------------------------------+ +| ``200`` | Success | ++---------------+-----------------------------------+ +| ``201`` | Created | ++---------------+-----------------------------------+ +| ``202`` | Accepted | ++---------------+-----------------------------------+ +| ``204`` | NoContent | ++---------------+-----------------------------------+ +| ``206`` | Partial content | ++---------------+-----------------------------------+ +| ``304`` | NotModified | ++---------------+-----------------------------------+ +| ``400`` | InvalidArgument | ++---------------+-----------------------------------+ +| ``400`` | InvalidDigest | ++---------------+-----------------------------------+ +| ``400`` | BadDigest | ++---------------+-----------------------------------+ +| ``400`` | InvalidBucketName | ++---------------+-----------------------------------+ +| ``400`` | InvalidObjectName | ++---------------+-----------------------------------+ +| ``400`` | UnresolvableGrantByEmailAddress | ++---------------+-----------------------------------+ +| ``400`` | InvalidPart | ++---------------+-----------------------------------+ +| ``400`` | InvalidPartOrder | ++---------------+-----------------------------------+ +| ``400`` | RequestTimeout | ++---------------+-----------------------------------+ +| ``400`` | EntityTooLarge | ++---------------+-----------------------------------+ +| ``403`` | AccessDenied | ++---------------+-----------------------------------+ +| ``403`` | UserSuspended | ++---------------+-----------------------------------+ +| ``403`` | RequestTimeTooSkewed | ++---------------+-----------------------------------+ +| ``404`` | NoSuchKey | ++---------------+-----------------------------------+ +| ``404`` | NoSuchBucket | ++---------------+-----------------------------------+ +| ``404`` | NoSuchUpload | ++---------------+-----------------------------------+ +| ``405`` | MethodNotAllowed | ++---------------+-----------------------------------+ +| ``408`` | RequestTimeout | ++---------------+-----------------------------------+ +| ``409`` | BucketAlreadyExists | ++---------------+-----------------------------------+ +| ``409`` | BucketNotEmpty | ++---------------+-----------------------------------+ +| ``411`` | MissingContentLength | ++---------------+-----------------------------------+ +| ``412`` | PreconditionFailed | ++---------------+-----------------------------------+ +| ``416`` | InvalidRange | ++---------------+-----------------------------------+ +| ``422`` | UnprocessableEntity | ++---------------+-----------------------------------+ +| ``500`` | InternalError | ++---------------+-----------------------------------+ + +.. _`Ceph Object Gateway - Multisite Configuration`: ../../multisite diff --git a/src/ceph/doc/radosgw/s3/cpp.rst b/src/ceph/doc/radosgw/s3/cpp.rst new file mode 100644 index 0000000..3451aeb --- /dev/null +++ b/src/ceph/doc/radosgw/s3/cpp.rst @@ -0,0 +1,331 @@ +.. _cpp: + +C++ S3 Examples +=============== + +Setup +----- + +The following contains includes and globals that will be used in later examples: + +.. code-block:: cpp + + #include "libs3.h" + #include <stdlib.h> + #include <iostream> + #include <fstream> + + const char access_key[] = "ACCESS_KEY"; + const char secret_key[] = "SECRET_KEY"; + const char host[] = "HOST"; + const char sample_bucket[] = "sample_bucket"; + const char sample_key[] = "hello.txt"; + const char sample_file[] = "resource/hello.txt"; + + S3BucketContext bucketContext = + { + host, + sample_bucket, + S3ProtocolHTTP, + S3UriStylePath, + access_key, + secret_key + }; + + S3Status responsePropertiesCallback( + const S3ResponseProperties *properties, + void *callbackData) + { + return S3StatusOK; + } + + static void responseCompleteCallback( + S3Status status, + const S3ErrorDetails *error, + void *callbackData) + { + return; + } + + S3ResponseHandler responseHandler = + { + &responsePropertiesCallback, + &responseCompleteCallback + }; + + +Creating (and Closing) a Connection +----------------------------------- + +This creates a connection so that you can interact with the server. + +.. code-block:: cpp + + S3_initialize("s3", S3_INIT_ALL, host); + // Do stuff... + S3_deinitialize(); + + +Listing Owned Buckets +--------------------- + +This gets a list of Buckets that you own. +This also prints out the bucket name, owner ID, and display name +for each bucket. + +.. code-block:: cpp + + static S3Status listServiceCallback( + const char *ownerId, + const char *ownerDisplayName, + const char *bucketName, + int64_t creationDate, void *callbackData) + { + bool *header_printed = (bool*) callbackData; + if (!*header_printed) { + *header_printed = true; + printf("%-22s", " Bucket"); + printf(" %-20s %-12s", " Owner ID", "Display Name"); + printf("\n"); + printf("----------------------"); + printf(" --------------------" " ------------"); + printf("\n"); + } + + printf("%-22s", bucketName); + printf(" %-20s %-12s", ownerId ? ownerId : "", ownerDisplayName ? ownerDisplayName : ""); + printf("\n"); + + return S3StatusOK; + } + + S3ListServiceHandler listServiceHandler = + { + responseHandler, + &listServiceCallback + }; + bool header_printed = false; + S3_list_service(S3ProtocolHTTP, access_key, secret_key, host, 0, NULL, &listServiceHandler, &header_printed); + + +Creating a Bucket +----------------- + +This creates a new bucket. + +.. code-block:: cpp + + S3_create_bucket(S3ProtocolHTTP, access_key, secret_key, NULL, host, sample_bucket, S3CannedAclPrivate, NULL, NULL, &responseHandler, NULL); + + +Listing a Bucket's Content +-------------------------- + +This gets a list of objects in the bucket. +This also prints out each object's name, the file size, and +last modified date. + +.. code-block:: cpp + + static S3Status listBucketCallback( + int isTruncated, + const char *nextMarker, + int contentsCount, + const S3ListBucketContent *contents, + int commonPrefixesCount, + const char **commonPrefixes, + void *callbackData) + { + printf("%-22s", " Object Name"); + printf(" %-5s %-20s", "Size", " Last Modified"); + printf("\n"); + printf("----------------------"); + printf(" -----" " --------------------"); + printf("\n"); + + for (int i = 0; i < contentsCount; i++) { + char timebuf[256]; + char sizebuf[16]; + const S3ListBucketContent *content = &(contents[i]); + time_t t = (time_t) content->lastModified; + + strftime(timebuf, sizeof(timebuf), "%Y-%m-%dT%H:%M:%SZ", gmtime(&t)); + sprintf(sizebuf, "%5llu", (unsigned long long) content->size); + printf("%-22s %s %s\n", content->key, sizebuf, timebuf); + } + + return S3StatusOK; + } + + S3ListBucketHandler listBucketHandler = + { + responseHandler, + &listBucketCallback + }; + S3_list_bucket(&bucketContext, NULL, NULL, NULL, 0, NULL, &listBucketHandler, NULL); + +The output will look something like this:: + + myphoto1.jpg 251262 2011-08-08T21:35:48.000Z + myphoto2.jpg 262518 2011-08-08T21:38:01.000Z + + +Deleting a Bucket +----------------- + +.. note:: + + The Bucket must be empty! Otherwise it won't work! + +.. code-block:: cpp + + S3_delete_bucket(S3ProtocolHTTP, S3UriStylePath, access_key, secret_key, host, sample_bucket, NULL, &responseHandler, NULL); + + +Creating an Object (from a file) +-------------------------------- + +This creates a file ``hello.txt``. + +.. code-block:: cpp + + #include <sys/stat.h> + typedef struct put_object_callback_data + { + FILE *infile; + uint64_t contentLength; + } put_object_callback_data; + + + static int putObjectDataCallback(int bufferSize, char *buffer, void *callbackData) + { + put_object_callback_data *data = (put_object_callback_data *) callbackData; + + int ret = 0; + + if (data->contentLength) { + int toRead = ((data->contentLength > (unsigned) bufferSize) ? (unsigned) bufferSize : data->contentLength); + ret = fread(buffer, 1, toRead, data->infile); + } + data->contentLength -= ret; + return ret; + } + + put_object_callback_data data; + struct stat statbuf; + if (stat(sample_file, &statbuf) == -1) { + fprintf(stderr, "\nERROR: Failed to stat file %s: ", sample_file); + perror(0); + exit(-1); + } + + int contentLength = statbuf.st_size; + data.contentLength = contentLength; + + if (!(data.infile = fopen(sample_file, "r"))) { + fprintf(stderr, "\nERROR: Failed to open input file %s: ", sample_file); + perror(0); + exit(-1); + } + + S3PutObjectHandler putObjectHandler = + { + responseHandler, + &putObjectDataCallback + }; + + S3_put_object(&bucketContext, sample_key, contentLength, NULL, NULL, &putObjectHandler, &data); + + +Download an Object (to a file) +------------------------------ + +This downloads a file and prints the contents. + +.. code-block:: cpp + + static S3Status getObjectDataCallback(int bufferSize, const char *buffer, void *callbackData) + { + FILE *outfile = (FILE *) callbackData; + size_t wrote = fwrite(buffer, 1, bufferSize, outfile); + return ((wrote < (size_t) bufferSize) ? S3StatusAbortedByCallback : S3StatusOK); + } + + S3GetObjectHandler getObjectHandler = + { + responseHandler, + &getObjectDataCallback + }; + FILE *outfile = stdout; + S3_get_object(&bucketContext, sample_key, NULL, 0, 0, NULL, &getObjectHandler, outfile); + + +Delete an Object +---------------- + +This deletes an object. + +.. code-block:: cpp + + S3ResponseHandler deleteResponseHandler = + { + 0, + &responseCompleteCallback + }; + S3_delete_object(&bucketContext, sample_key, 0, &deleteResponseHandler, 0); + + +Change an Object's ACL +---------------------- + +This changes an object's ACL to grant full control to another user. + + +.. code-block:: cpp + + #include <string.h> + char ownerId[] = "owner"; + char ownerDisplayName[] = "owner"; + char granteeId[] = "grantee"; + char granteeDisplayName[] = "grantee"; + + S3AclGrant grants[] = { + { + S3GranteeTypeCanonicalUser, + {{}}, + S3PermissionFullControl + }, + { + S3GranteeTypeCanonicalUser, + {{}}, + S3PermissionReadACP + }, + { + S3GranteeTypeAllUsers, + {{}}, + S3PermissionRead + } + }; + + strncpy(grants[0].grantee.canonicalUser.id, ownerId, S3_MAX_GRANTEE_USER_ID_SIZE); + strncpy(grants[0].grantee.canonicalUser.displayName, ownerDisplayName, S3_MAX_GRANTEE_DISPLAY_NAME_SIZE); + + strncpy(grants[1].grantee.canonicalUser.id, granteeId, S3_MAX_GRANTEE_USER_ID_SIZE); + strncpy(grants[1].grantee.canonicalUser.displayName, granteeDisplayName, S3_MAX_GRANTEE_DISPLAY_NAME_SIZE); + + S3_set_acl(&bucketContext, sample_key, ownerId, ownerDisplayName, 3, grants, 0, &responseHandler, 0); + + +Generate Object Download URL (signed) +------------------------------------- + +This generates a signed download URL that will be valid for 5 minutes. + +.. code-block:: cpp + + #include <time.h> + char buffer[S3_MAX_AUTHENTICATED_QUERY_STRING_SIZE]; + int64_t expires = time(NULL) + 60 * 5; // Current time + 5 minutes + + S3_generate_authenticated_query_string(buffer, &bucketContext, sample_key, expires, NULL); + diff --git a/src/ceph/doc/radosgw/s3/csharp.rst b/src/ceph/doc/radosgw/s3/csharp.rst new file mode 100644 index 0000000..af1c6e4 --- /dev/null +++ b/src/ceph/doc/radosgw/s3/csharp.rst @@ -0,0 +1,199 @@ +.. _csharp: + +C# S3 Examples +============== + +Creating a Connection +--------------------- + +This creates a connection so that you can interact with the server. + +.. code-block:: csharp + + using System; + using Amazon; + using Amazon.S3; + using Amazon.S3.Model; + + string accessKey = "put your access key here!"; + string secretKey = "put your secret key here!"; + + AmazonS3Config config = new AmazonS3Config(); + config.ServiceURL = "objects.dreamhost.com"; + + AmazonS3Client s3Client = new AmazonS3Client( + accessKey, + secretKey, + config + ); + + +Listing Owned Buckets +--------------------- + +This gets a list of Buckets that you own. +This also prints out the bucket name and creation date of each bucket. + +.. code-block:: csharp + + ListBucketsResponse response = client.ListBuckets(); + foreach (S3Bucket b in response.Buckets) + { + Console.WriteLine("{0}\t{1}", b.BucketName, b.CreationDate); + } + +The output will look something like this:: + + mahbuckat1 2011-04-21T18:05:39.000Z + mahbuckat2 2011-04-21T18:05:48.000Z + mahbuckat3 2011-04-21T18:07:18.000Z + + +Creating a Bucket +----------------- +This creates a new bucket called ``my-new-bucket`` + +.. code-block:: csharp + + PutBucketRequest request = new PutBucketRequest(); + request.BucketName = "my-new-bucket"; + client.PutBucket(request); + +Listing a Bucket's Content +-------------------------- + +This gets a list of objects in the bucket. +This also prints out each object's name, the file size, and last +modified date. + +.. code-block:: csharp + + ListObjectsRequest request = new ListObjectsRequest(); + request.BucketName = "my-new-bucket"; + ListObjectsResponse response = client.ListObjects(request); + foreach (S3Object o in response.S3Objects) + { + Console.WriteLine("{0}\t{1}\t{2}", o.Key, o.Size, o.LastModified); + } + +The output will look something like this:: + + myphoto1.jpg 251262 2011-08-08T21:35:48.000Z + myphoto2.jpg 262518 2011-08-08T21:38:01.000Z + + +Deleting a Bucket +----------------- + +.. note:: + + The Bucket must be empty! Otherwise it won't work! + +.. code-block:: csharp + + DeleteBucketRequest request = new DeleteBucketRequest(); + request.BucketName = "my-new-bucket"; + client.DeleteBucket(request); + + +Forced Delete for Non-empty Buckets +----------------------------------- + +.. attention:: + + not available + + +Creating an Object +------------------ + +This creates a file ``hello.txt`` with the string ``"Hello World!"`` + +.. code-block:: csharp + + PutObjectRequest request = new PutObjectRequest(); + request.BucketName = "my-new-bucket"; + request.Key = "hello.txt"; + request.ContentType = "text/plain"; + request.ContentBody = "Hello World!"; + client.PutObject(request); + + +Change an Object's ACL +---------------------- + +This makes the object ``hello.txt`` to be publicly readable, and +``secret_plans.txt`` to be private. + +.. code-block:: csharp + + PutACLRequest request = new PutACLRequest(); + request.BucketName = "my-new-bucket"; + request.Key = "hello.txt"; + request.CannedACL = S3CannedACL.PublicRead; + client.PutACL(request); + + PutACLRequest request2 = new PutACLRequest(); + request2.BucketName = "my-new-bucket"; + request2.Key = "secret_plans.txt"; + request2.CannedACL = S3CannedACL.Private; + client.PutACL(request2); + + +Download an Object (to a file) +------------------------------ + +This downloads the object ``perl_poetry.pdf`` and saves it in +``C:\Users\larry\Documents`` + +.. code-block:: csharp + + GetObjectRequest request = new GetObjectRequest(); + request.BucketName = "my-new-bucket"; + request.Key = "perl_poetry.pdf"; + GetObjectResponse response = client.GetObject(request); + response.WriteResponseStreamToFile("C:\\Users\\larry\\Documents\\perl_poetry.pdf"); + + +Delete an Object +---------------- + +This deletes the object ``goodbye.txt`` + +.. code-block:: csharp + + DeleteObjectRequest request = new DeleteObjectRequest(); + request.BucketName = "my-new-bucket"; + request.Key = "goodbye.txt"; + client.DeleteObject(request); + + +Generate Object Download URLs (signed and unsigned) +--------------------------------------------------- + +This generates an unsigned download URL for ``hello.txt``. This works +because we made ``hello.txt`` public by setting the ACL above. +This then generates a signed download URL for ``secret_plans.txt`` that +will work for 1 hour. Signed download URLs will work for the time +period even if the object is private (when the time period is up, the +URL will stop working). + +.. note:: + + The C# S3 Library does not have a method for generating unsigned + URLs, so the following example only shows generating signed URLs. + +.. code-block:: csharp + + GetPreSignedUrlRequest request = new GetPreSignedUrlRequest(); + request.BucketName = "my-bucket-name"; + request.Key = "secret_plans.txt"; + request.Expires = DateTime.Now.AddHours(1); + request.Protocol = Protocol.HTTP; + string url = client.GetPreSignedURL(request); + Console.WriteLine(url); + +The output of this will look something like:: + + http://objects.dreamhost.com/my-bucket-name/secret_plans.txt?Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXX&Expires=1316027075&AWSAccessKeyId=XXXXXXXXXXXXXXXXXXX + diff --git a/src/ceph/doc/radosgw/s3/java.rst b/src/ceph/doc/radosgw/s3/java.rst new file mode 100644 index 0000000..057c09c --- /dev/null +++ b/src/ceph/doc/radosgw/s3/java.rst @@ -0,0 +1,212 @@ +.. _java: + +Java S3 Examples +================ + +Setup +----- + +The following examples may require some or all of the following java +classes to be imported: + +.. code-block:: java + + import java.io.ByteArrayInputStream; + import java.io.File; + import java.util.List; + import com.amazonaws.auth.AWSCredentials; + import com.amazonaws.auth.BasicAWSCredentials; + import com.amazonaws.util.StringUtils; + import com.amazonaws.services.s3.AmazonS3; + import com.amazonaws.services.s3.AmazonS3Client; + import com.amazonaws.services.s3.model.Bucket; + import com.amazonaws.services.s3.model.CannedAccessControlList; + import com.amazonaws.services.s3.model.GeneratePresignedUrlRequest; + import com.amazonaws.services.s3.model.GetObjectRequest; + import com.amazonaws.services.s3.model.ObjectListing; + import com.amazonaws.services.s3.model.ObjectMetadata; + import com.amazonaws.services.s3.model.S3ObjectSummary; + + +If you are just testing the Ceph Object Storage services, consider +using HTTP protocol instead of HTTPS protocol. + +First, import the ``ClientConfiguration`` and ``Protocol`` classes. + +.. code-block:: java + + import com.amazonaws.ClientConfiguration; + import com.amazonaws.Protocol; + + +Then, define the client configuration, and add the client configuration +as an argument for the S3 client. + +.. code-block:: java + + AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey); + + ClientConfiguration clientConfig = new ClientConfiguration(); + clientConfig.setProtocol(Protocol.HTTP); + + AmazonS3 conn = new AmazonS3Client(credentials, clientConfig); + conn.setEndpoint("endpoint.com"); + + +Creating a Connection +--------------------- + +This creates a connection so that you can interact with the server. + +.. code-block:: java + + String accessKey = "insert your access key here!"; + String secretKey = "insert your secret key here!"; + + AWSCredentials credentials = new BasicAWSCredentials(accessKey, secretKey); + AmazonS3 conn = new AmazonS3Client(credentials); + conn.setEndpoint("objects.dreamhost.com"); + + +Listing Owned Buckets +--------------------- + +This gets a list of Buckets that you own. +This also prints out the bucket name and creation date of each bucket. + +.. code-block:: java + + List<Bucket> buckets = conn.listBuckets(); + for (Bucket bucket : buckets) { + System.out.println(bucket.getName() + "\t" + + StringUtils.fromDate(bucket.getCreationDate())); + } + +The output will look something like this:: + + mahbuckat1 2011-04-21T18:05:39.000Z + mahbuckat2 2011-04-21T18:05:48.000Z + mahbuckat3 2011-04-21T18:07:18.000Z + + +Creating a Bucket +----------------- + +This creates a new bucket called ``my-new-bucket`` + +.. code-block:: java + + Bucket bucket = conn.createBucket("my-new-bucket"); + + +Listing a Bucket's Content +-------------------------- +This gets a list of objects in the bucket. +This also prints out each object's name, the file size, and last +modified date. + +.. code-block:: java + + ObjectListing objects = conn.listObjects(bucket.getName()); + do { + for (S3ObjectSummary objectSummary : objects.getObjectSummaries()) { + System.out.println(objectSummary.getKey() + "\t" + + objectSummary.getSize() + "\t" + + StringUtils.fromDate(objectSummary.getLastModified())); + } + objects = conn.listNextBatchOfObjects(objects); + } while (objects.isTruncated()); + +The output will look something like this:: + + myphoto1.jpg 251262 2011-08-08T21:35:48.000Z + myphoto2.jpg 262518 2011-08-08T21:38:01.000Z + + +Deleting a Bucket +----------------- + +.. note:: + The Bucket must be empty! Otherwise it won't work! + +.. code-block:: java + + conn.deleteBucket(bucket.getName()); + + +Forced Delete for Non-empty Buckets +----------------------------------- +.. attention:: + not available + + +Creating an Object +------------------ + +This creates a file ``hello.txt`` with the string ``"Hello World!"`` + +.. code-block:: java + + ByteArrayInputStream input = new ByteArrayInputStream("Hello World!".getBytes()); + conn.putObject(bucket.getName(), "hello.txt", input, new ObjectMetadata()); + + +Change an Object's ACL +---------------------- + +This makes the object ``hello.txt`` to be publicly readable, and +``secret_plans.txt`` to be private. + +.. code-block:: java + + conn.setObjectAcl(bucket.getName(), "hello.txt", CannedAccessControlList.PublicRead); + conn.setObjectAcl(bucket.getName(), "secret_plans.txt", CannedAccessControlList.Private); + + +Download an Object (to a file) +------------------------------ + +This downloads the object ``perl_poetry.pdf`` and saves it in +``/home/larry/documents`` + +.. code-block:: java + + conn.getObject( + new GetObjectRequest(bucket.getName(), "perl_poetry.pdf"), + new File("/home/larry/documents/perl_poetry.pdf") + ); + + +Delete an Object +---------------- + +This deletes the object ``goodbye.txt`` + +.. code-block:: java + + conn.deleteObject(bucket.getName(), "goodbye.txt"); + + +Generate Object Download URLs (signed and unsigned) +--------------------------------------------------- + +This generates an unsigned download URL for ``hello.txt``. This works +because we made ``hello.txt`` public by setting the ACL above. +This then generates a signed download URL for ``secret_plans.txt`` that +will work for 1 hour. Signed download URLs will work for the time +period even if the object is private (when the time period is up, the +URL will stop working). + +.. note:: + The java library does not have a method for generating unsigned + URLs, so the example below just generates a signed URL. + +.. code-block:: java + + GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest(bucket.getName(), "secret_plans.txt"); + System.out.println(conn.generatePresignedUrl(request)); + +The output will look something like this:: + + https://my-bucket-name.objects.dreamhost.com/secret_plans.txt?Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXX&Expires=1316027075&AWSAccessKeyId=XXXXXXXXXXXXXXXXXXX + diff --git a/src/ceph/doc/radosgw/s3/objectops.rst b/src/ceph/doc/radosgw/s3/objectops.rst new file mode 100644 index 0000000..b8a2475 --- /dev/null +++ b/src/ceph/doc/radosgw/s3/objectops.rst @@ -0,0 +1,403 @@ +Object Operations +================= + +Put Object +---------- +Adds an object to a bucket. You must have write permissions on the bucket to perform this operation. + + +Syntax +~~~~~~ + +:: + + PUT /{bucket}/{object} HTTP/1.1 + +Request Headers +~~~~~~~~~~~~~~~ + ++----------------------+--------------------------------------------+-------------------------------------------------------------------------------+------------+ +| Name | Description | Valid Values | Required | ++======================+============================================+===============================================================================+============+ +| **content-md5** | A base64 encoded MD-5 hash of the message. | A string. No defaults or constraints. | No | ++----------------------+--------------------------------------------+-------------------------------------------------------------------------------+------------+ +| **content-type** | A standard MIME type. | Any MIME type. Default: ``binary/octet-stream`` | No | ++----------------------+--------------------------------------------+-------------------------------------------------------------------------------+------------+ +| **x-amz-meta-<...>** | User metadata. Stored with the object. | A string up to 8kb. No defaults. | No | ++----------------------+--------------------------------------------+-------------------------------------------------------------------------------+------------+ +| **x-amz-acl** | A canned ACL. | ``private``, ``public-read``, ``public-read-write``, ``authenticated-read`` | No | ++----------------------+--------------------------------------------+-------------------------------------------------------------------------------+------------+ + + +Copy Object +----------- +To copy an object, use ``PUT`` and specify a destination bucket and the object name. + +Syntax +~~~~~~ + +:: + + PUT /{dest-bucket}/{dest-object} HTTP/1.1 + x-amz-copy-source: {source-bucket}/{source-object} + +Request Headers +~~~~~~~~~~~~~~~ + ++--------------------------------------+-------------------------------------------------+------------------------+------------+ +| Name | Description | Valid Values | Required | ++======================================+=================================================+========================+============+ +| **x-amz-copy-source** | The source bucket name + object name. | {bucket}/{obj} | Yes | ++--------------------------------------+-------------------------------------------------+------------------------+------------+ +| **x-amz-acl** | A canned ACL. | ``private``, | No | +| | | ``public-read``, | | +| | | ``public-read-write``, | | +| | | ``authenticated-read`` | | ++--------------------------------------+-------------------------------------------------+------------------------+------------+ +| **x-amz-copy-if-modified-since** | Copies only if modified since the timestamp. | Timestamp | No | ++--------------------------------------+-------------------------------------------------+------------------------+------------+ +| **x-amz-copy-if-unmodified-since** | Copies only if unmodified since the timestamp. | Timestamp | No | ++--------------------------------------+-------------------------------------------------+------------------------+------------+ +| **x-amz-copy-if-match** | Copies only if object ETag matches ETag. | Entity Tag | No | ++--------------------------------------+-------------------------------------------------+------------------------+------------+ +| **x-amz-copy-if-none-match** | Copies only if object ETag doesn't match. | Entity Tag | No | ++--------------------------------------+-------------------------------------------------+------------------------+------------+ + +Response Entities +~~~~~~~~~~~~~~~~~ + ++------------------------+-------------+-----------------------------------------------+ +| Name | Type | Description | ++========================+=============+===============================================+ +| **CopyObjectResult** | Container | A container for the response elements. | ++------------------------+-------------+-----------------------------------------------+ +| **LastModified** | Date | The last modified date of the source object. | ++------------------------+-------------+-----------------------------------------------+ +| **Etag** | String | The ETag of the new object. | ++------------------------+-------------+-----------------------------------------------+ + +Remove Object +------------- + +Removes an object. Requires WRITE permission set on the containing bucket. + +Syntax +~~~~~~ + +:: + + DELETE /{bucket}/{object} HTTP/1.1 + + + +Get Object +---------- +Retrieves an object from a bucket within RADOS. + +Syntax +~~~~~~ + +:: + + GET /{bucket}/{object} HTTP/1.1 + +Request Headers +~~~~~~~~~~~~~~~ + ++---------------------------+------------------------------------------------+--------------------------------+------------+ +| Name | Description | Valid Values | Required | ++===========================+================================================+================================+============+ +| **range** | The range of the object to retrieve. | Range: bytes=beginbyte-endbyte | No | ++---------------------------+------------------------------------------------+--------------------------------+------------+ +| **if-modified-since** | Gets only if modified since the timestamp. | Timestamp | No | ++---------------------------+------------------------------------------------+--------------------------------+------------+ +| **if-unmodified-since** | Gets only if not modified since the timestamp. | Timestamp | No | ++---------------------------+------------------------------------------------+--------------------------------+------------+ +| **if-match** | Gets only if object ETag matches ETag. | Entity Tag | No | ++---------------------------+------------------------------------------------+--------------------------------+------------+ +| **if-none-match** | Gets only if object ETag matches ETag. | Entity Tag | No | ++---------------------------+------------------------------------------------+--------------------------------+------------+ + +Response Headers +~~~~~~~~~~~~~~~~ + ++-------------------+--------------------------------------------------------------------------------------------+ +| Name | Description | ++===================+============================================================================================+ +| **Content-Range** | Data range, will only be returned if the range header field was specified in the request | ++-------------------+--------------------------------------------------------------------------------------------+ + +Get Object Info +--------------- + +Returns information about object. This request will return the same +header information as with the Get Object request, but will include +the metadata only, not the object data payload. + +Syntax +~~~~~~ + +:: + + HEAD /{bucket}/{object} HTTP/1.1 + +Request Headers +~~~~~~~~~~~~~~~ + ++---------------------------+------------------------------------------------+--------------------------------+------------+ +| Name | Description | Valid Values | Required | ++===========================+================================================+================================+============+ +| **range** | The range of the object to retrieve. | Range: bytes=beginbyte-endbyte | No | ++---------------------------+------------------------------------------------+--------------------------------+------------+ +| **if-modified-since** | Gets only if modified since the timestamp. | Timestamp | No | ++---------------------------+------------------------------------------------+--------------------------------+------------+ +| **if-unmodified-since** | Gets only if not modified since the timestamp. | Timestamp | No | ++---------------------------+------------------------------------------------+--------------------------------+------------+ +| **if-match** | Gets only if object ETag matches ETag. | Entity Tag | No | ++---------------------------+------------------------------------------------+--------------------------------+------------+ +| **if-none-match** | Gets only if object ETag matches ETag. | Entity Tag | No | ++---------------------------+------------------------------------------------+--------------------------------+------------+ + +Get Object ACL +-------------- + +Syntax +~~~~~~ + +:: + + GET /{bucket}/{object}?acl HTTP/1.1 + +Response Entities +~~~~~~~~~~~~~~~~~ + ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| Name | Type | Description | ++===========================+=============+==============================================================================================+ +| ``AccessControlPolicy`` | Container | A container for the response. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``AccessControlList`` | Container | A container for the ACL information. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Owner`` | Container | A container for the object owner's ``ID`` and ``DisplayName``. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``ID`` | String | The object owner's ID. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``DisplayName`` | String | The object owner's display name. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Grant`` | Container | A container for ``Grantee`` and ``Permission``. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Grantee`` | Container | A container for the ``DisplayName`` and ``ID`` of the user receiving a grant of permission. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Permission`` | String | The permission given to the ``Grantee`` object. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ + + + +Set Object ACL +-------------- + +Syntax +~~~~~~ + +:: + + PUT /{bucket}/{object}?acl + +Request Entities +~~~~~~~~~~~~~~~~ + ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| Name | Type | Description | ++===========================+=============+==============================================================================================+ +| ``AccessControlPolicy`` | Container | A container for the response. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``AccessControlList`` | Container | A container for the ACL information. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Owner`` | Container | A container for the object owner's ``ID`` and ``DisplayName``. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``ID`` | String | The object owner's ID. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``DisplayName`` | String | The object owner's display name. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Grant`` | Container | A container for ``Grantee`` and ``Permission``. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Grantee`` | Container | A container for the ``DisplayName`` and ``ID`` of the user receiving a grant of permission. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ +| ``Permission`` | String | The permission given to the ``Grantee`` object. | ++---------------------------+-------------+----------------------------------------------------------------------------------------------+ + + + +Initiate Multi-part Upload +-------------------------- + +Initiate a multi-part upload process. + +Syntax +~~~~~~ + +:: + + POST /{bucket}/{object}?uploads + +Request Headers +~~~~~~~~~~~~~~~ + ++----------------------+--------------------------------------------+-------------------------------------------------------------------------------+------------+ +| Name | Description | Valid Values | Required | ++======================+============================================+===============================================================================+============+ +| **content-md5** | A base64 encoded MD-5 hash of the message. | A string. No defaults or constraints. | No | ++----------------------+--------------------------------------------+-------------------------------------------------------------------------------+------------+ +| **content-type** | A standard MIME type. | Any MIME type. Default: ``binary/octet-stream`` | No | ++----------------------+--------------------------------------------+-------------------------------------------------------------------------------+------------+ +| **x-amz-meta-<...>** | User metadata. Stored with the object. | A string up to 8kb. No defaults. | No | ++----------------------+--------------------------------------------+-------------------------------------------------------------------------------+------------+ +| **x-amz-acl** | A canned ACL. | ``private``, ``public-read``, ``public-read-write``, ``authenticated-read`` | No | ++----------------------+--------------------------------------------+-------------------------------------------------------------------------------+------------+ + + +Response Entities +~~~~~~~~~~~~~~~~~ + ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| Name | Type | Description | ++=========================================+=============+==========================================================================================================+ +| ``InitiatedMultipartUploadsResult`` | Container | A container for the results. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Bucket`` | String | The bucket that will receive the object contents. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Key`` | String | The key specified by the ``key`` request parameter (if any). | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``UploadId`` | String | The ID specified by the ``upload-id`` request parameter identifying the multipart upload (if any). | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ + + +Multipart Upload Part +--------------------- + +Syntax +~~~~~~ + +:: + + PUT /{bucket}/{object}?partNumber=&uploadId= HTTP/1.1 + +HTTP Response +~~~~~~~~~~~~~ + +The following HTTP response may be returned: + ++---------------+----------------+--------------------------------------------------------------------------+ +| HTTP Status | Status Code | Description | ++===============+================+==========================================================================+ +| **404** | NoSuchUpload | Specified upload-id does not match any initiated upload on this object | ++---------------+----------------+--------------------------------------------------------------------------+ + +List Multipart Upload Parts +--------------------------- + +Syntax +~~~~~~ + +:: + + GET /{bucket}/{object}?uploadId=123 HTTP/1.1 + +Response Entities +~~~~~~~~~~~~~~~~~ + ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| Name | Type | Description | ++=========================================+=============+==========================================================================================================+ +| ``ListPartsResult`` | Container | A container for the results. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Bucket`` | String | The bucket that will receive the object contents. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Key`` | String | The key specified by the ``key`` request parameter (if any). | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``UploadId`` | String | The ID specified by the ``upload-id`` request parameter identifying the multipart upload (if any). | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Initiator`` | Container | Contains the ``ID`` and ``DisplayName`` of the user who initiated the upload. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``ID`` | String | The initiator's ID. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``DisplayName`` | String | The initiator's display name. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Owner`` | Container | A container for the ``ID`` and ``DisplayName`` of the user who owns the uploaded object. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``StorageClass`` | String | The method used to store the resulting object. ``STANDARD`` or ``REDUCED_REDUNDANCY`` | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``PartNumberMarker`` | String | The part marker to use in a subsequent request if ``IsTruncated`` is ``true``. Precedes the list. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``NextPartNumberMarker`` | String | The next part marker to use in a subsequent request if ``IsTruncated`` is ``true``. The end of the list. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``MaxParts`` | Integer | The max parts allowed in the response as specified by the ``max-parts`` request parameter. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``IsTruncated`` | Boolean | If ``true``, only a subset of the object's upload contents were returned. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Part`` | Container | A container for ``LastModified``, ``PartNumber``, ``ETag`` and ``Size`` elements. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``LastModified`` | Date | Date and time at which the part was uploaded. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``PartNumber`` | Integer | The identification number of the part. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``ETag`` | String | The part's entity tag. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ +| ``Size`` | Integer | The size of the uploaded part. | ++-----------------------------------------+-------------+----------------------------------------------------------------------------------------------------------+ + + + +Complete Multipart Upload +------------------------- +Assembles uploaded parts and creates a new object, thereby completing a multipart upload. + +Syntax +~~~~~~ + +:: + + POST /{bucket}/{object}?uploadId= HTTP/1.1 + +Request Entities +~~~~~~~~~~~~~~~~ + ++----------------------------------+-------------+-----------------------------------------------------+----------+ +| Name | Type | Description | Required | ++==================================+=============+=====================================================+==========+ +| ``CompleteMultipartUpload`` | Container | A container consisting of one or more parts. | Yes | ++----------------------------------+-------------+-----------------------------------------------------+----------+ +| ``Part`` | Container | A container for the ``PartNumber`` and ``ETag``. | Yes | ++----------------------------------+-------------+-----------------------------------------------------+----------+ +| ``PartNumber`` | Integer | The identifier of the part. | Yes | ++----------------------------------+-------------+-----------------------------------------------------+----------+ +| ``ETag`` | String | The part's entity tag. | Yes | ++----------------------------------+-------------+-----------------------------------------------------+----------+ + + +Response Entities +~~~~~~~~~~~~~~~~~ + ++-------------------------------------+-------------+-------------------------------------------------------+ +| Name | Type | Description | ++=====================================+=============+=======================================================+ +| **CompleteMultipartUploadResult** | Container | A container for the response. | ++-------------------------------------+-------------+-------------------------------------------------------+ +| **Location** | URI | The resource identifier (path) of the new object. | ++-------------------------------------+-------------+-------------------------------------------------------+ +| **Bucket** | String | The name of the bucket that contains the new object. | ++-------------------------------------+-------------+-------------------------------------------------------+ +| **Key** | String | The object's key. | ++-------------------------------------+-------------+-------------------------------------------------------+ +| **ETag** | String | The entity tag of the new object. | ++-------------------------------------+-------------+-------------------------------------------------------+ + +Abort Multipart Upload +---------------------- + +Syntax +~~~~~~ + +:: + + DELETE /{bucket}/{object}?uploadId= HTTP/1.1 diff --git a/src/ceph/doc/radosgw/s3/perl.rst b/src/ceph/doc/radosgw/s3/perl.rst new file mode 100644 index 0000000..f12e5c6 --- /dev/null +++ b/src/ceph/doc/radosgw/s3/perl.rst @@ -0,0 +1,192 @@ +.. _perl: + +Perl S3 Examples +================ + +Creating a Connection +--------------------- + +This creates a connection so that you can interact with the server. + +.. code-block:: perl + + use Amazon::S3; + my $access_key = 'put your access key here!'; + my $secret_key = 'put your secret key here!'; + + my $conn = Amazon::S3->new({ + aws_access_key_id => $access_key, + aws_secret_access_key => $secret_key, + host => 'objects.dreamhost.com', + secure => 1, + retry => 1, + }); + + +Listing Owned Buckets +--------------------- + +This gets a list of `Amazon::S3::Bucket`_ objects that you own. +We'll also print out the bucket name and creation date of each bucket. + +.. code-block:: perl + + my @buckets = @{$conn->buckets->{buckets} || []}; + foreach my $bucket (@buckets) { + print $bucket->bucket . "\t" . $bucket->creation_date . "\n"; + } + +The output will look something like this:: + + mahbuckat1 2011-04-21T18:05:39.000Z + mahbuckat2 2011-04-21T18:05:48.000Z + mahbuckat3 2011-04-21T18:07:18.000Z + + +Creating a Bucket +----------------- + +This creates a new bucket called ``my-new-bucket`` + +.. code-block:: perl + + my $bucket = $conn->add_bucket({ bucket => 'my-new-bucket' }); + + +Listing a Bucket's Content +-------------------------- + +This gets a list of hashes with info about each object in the bucket. +We'll also print out each object's name, the file size, and last +modified date. + +.. code-block:: perl + + my @keys = @{$bucket->list_all->{keys} || []}; + foreach my $key (@keys) { + print "$key->{key}\t$key->{size}\t$key->{last_modified}\n"; + } + +The output will look something like this:: + + myphoto1.jpg 251262 2011-08-08T21:35:48.000Z + myphoto2.jpg 262518 2011-08-08T21:38:01.000Z + + +Deleting a Bucket +----------------- + +.. note:: + The Bucket must be empty! Otherwise it won't work! + +.. code-block:: perl + + $conn->delete_bucket($bucket); + + +Forced Delete for Non-empty Buckets +----------------------------------- + +.. attention:: + + not available in the `Amazon::S3`_ perl module + + +Creating an Object +------------------ + +This creates a file ``hello.txt`` with the string ``"Hello World!"`` + +.. code-block:: perl + + $bucket->add_key( + 'hello.txt', 'Hello World!', + { content_type => 'text/plain' }, + ); + + +Change an Object's ACL +---------------------- + +This makes the object ``hello.txt`` to be publicly readable and +``secret_plans.txt`` to be private. + +.. code-block:: perl + + $bucket->set_acl({ + key => 'hello.txt', + acl_short => 'public-read', + }); + $bucket->set_acl({ + key => 'secret_plans.txt', + acl_short => 'private', + }); + + +Download an Object (to a file) +------------------------------ + +This downloads the object ``perl_poetry.pdf`` and saves it in +``/home/larry/documents/`` + +.. code-block:: perl + + $bucket->get_key_filename('perl_poetry.pdf', undef, + '/home/larry/documents/perl_poetry.pdf'); + + +Delete an Object +---------------- + +This deletes the object ``goodbye.txt`` + +.. code-block:: perl + + $bucket->delete_key('goodbye.txt'); + +Generate Object Download URLs (signed and unsigned) +--------------------------------------------------- +This generates an unsigned download URL for ``hello.txt``. This works +because we made ``hello.txt`` public by setting the ACL above. +Then this generates a signed download URL for ``secret_plans.txt`` that +will work for 1 hour. Signed download URLs will work for the time +period even if the object is private (when the time period is up, the +URL will stop working). + +.. note:: + The `Amazon::S3`_ module does not have a way to generate download + URLs, so we are going to be using another module instead. Unfortunately, + most modules for generating these URLs assume that you are using Amazon, + so we have had to go with using a more obscure module, `Muck::FS::S3`_. This + should be the same as Amazon's sample S3 perl module, but this sample + module is not in CPAN. So, you can either use CPAN to install + `Muck::FS::S3`_, or install Amazon's sample S3 module manually. If you go + the manual route, you can remove ``Muck::FS::`` from the example below. + +.. code-block:: perl + + use Muck::FS::S3::QueryStringAuthGenerator; + my $generator = Muck::FS::S3::QueryStringAuthGenerator->new( + $access_key, + $secret_key, + 0, # 0 means use 'http'. set this to 1 for 'https' + 'objects.dreamhost.com', + ); + + my $hello_url = $generator->make_bare_url($bucket->bucket, 'hello.txt'); + print $hello_url . "\n"; + + $generator->expires_in(3600); # 1 hour = 3600 seconds + my $plans_url = $generator->get($bucket->bucket, 'secret_plans.txt'); + print $plans_url . "\n"; + +The output will look something like this:: + + http://objects.dreamhost.com:80/my-bucket-name/hello.txt + http://objects.dreamhost.com:80/my-bucket-name/secret_plans.txt?Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXX&Expires=1316027075&AWSAccessKeyId=XXXXXXXXXXXXXXXXXXX + + +.. _`Amazon::S3`: http://search.cpan.org/~tima/Amazon-S3-0.441/lib/Amazon/S3.pm +.. _`Amazon::S3::Bucket`: http://search.cpan.org/~tima/Amazon-S3-0.441/lib/Amazon/S3/Bucket.pm +.. _`Muck::FS::S3`: http://search.cpan.org/~mike/Muck-0.02/ + diff --git a/src/ceph/doc/radosgw/s3/php.rst b/src/ceph/doc/radosgw/s3/php.rst new file mode 100644 index 0000000..40542e0 --- /dev/null +++ b/src/ceph/doc/radosgw/s3/php.rst @@ -0,0 +1,208 @@ +.. _php: + +PHP S3 Examples +=============== + +Creating a Connection +--------------------- + +This creates a connection so that you can interact with the server. + +.. code-block:: php + + <?php + define('AWS_KEY', 'place access key here'); + define('AWS_SECRET_KEY', 'place secret key here'); + define('AWS_CANONICAL_ID', 'your DHO Username'); + define('AWS_CANONICAL_NAME', 'Also your DHO Username!'); + $HOST = 'objects.dreamhost.com'; + + // require the amazon sdk for php library + require_once 'AWSSDKforPHP/sdk.class.php'; + + // Instantiate the S3 class and point it at the desired host + $Connection = new AmazonS3(array( + 'key' => AWS_KEY, + 'secret' => AWS_SECRET_KEY, + 'canonical_id' => AWS_CANONICAL_ID, + 'canonical_name' => AWS_CANONICAL_NAME, + )); + $Connection->set_hostname($HOST); + $Connection->allow_hostname_override(false); + + // Set the S3 class to use objects.dreamhost.com/bucket + // instead of bucket.objects.dreamhost.com + $Connection->enable_path_style(); + + +Listing Owned Buckets +--------------------- +This gets a list of CFSimpleXML objects representing buckets that you +own. This also prints out the bucket name and creation date of each +bucket. + +.. code-block:: php + + <?php + $ListResponse = $Connection->list_buckets(); + $Buckets = $ListResponse->body->Buckets->Bucket; + foreach ($Buckets as $Bucket) { + echo $Bucket->Name . "\t" . $Bucket->CreationDate . "\n"; + } + +The output will look something like this:: + + mahbuckat1 2011-04-21T18:05:39.000Z + mahbuckat2 2011-04-21T18:05:48.000Z + mahbuckat3 2011-04-21T18:07:18.000Z + + +Creating a Bucket +----------------- + +This creates a new bucket called ``my-new-bucket`` and returns a +``CFResponse`` object. + +.. note:: + + This command requires a region as the second argument, + so we use ``AmazonS3::REGION_US_E1``, because this constant is ``''`` + +.. code-block:: php + + <?php + $Connection->create_bucket('my-new-bucket', AmazonS3::REGION_US_E1); + + +List a Bucket's Content +----------------------- + +This gets an array of ``CFSimpleXML`` objects representing the objects +in the bucket. This then prints out each object's name, the file size, +and last modified date. + +.. code-block:: php + + <?php + $ObjectsListResponse = $Connection->list_objects($bucketname); + $Objects = $ObjectsListResponse->body->Contents; + foreach ($Objects as $Object) { + echo $Object->Key . "\t" . $Object->Size . "\t" . $Object->LastModified . "\n"; + } + +.. note:: + + If there are more than 1000 objects in this bucket, + you need to check $ObjectListResponse->body->isTruncated + and run again with the name of the last key listed. + Keep doing this until isTruncated is not true. + +The output will look something like this if the bucket has some files:: + + myphoto1.jpg 251262 2011-08-08T21:35:48.000Z + myphoto2.jpg 262518 2011-08-08T21:38:01.000Z + + +Deleting a Bucket +----------------- + +This deletes the bucket called ``my-old-bucket`` and returns a +``CFResponse`` object + +.. note:: + + The Bucket must be empty! Otherwise it won't work! + +.. code-block:: php + + <?php + $Connection->delete_bucket('my-old-bucket'); + + +Forced Delte for Non-empty Buckets +---------------------------------- + +This will delete the bucket even if it is not empty. + +.. code-block:: php + + <?php + $Connection->delete_bucket('my-old-bucket', 1); + + +Creating an Object +------------------ + +This creates an object ``hello.txt`` with the string ``"Hello World!"`` + +.. code-block:: php + + <?php + $Connection->create_object('my-bucket-name', 'hello.txt', array( + 'body' => "Hello World!", + )); + + +Change an Object's ACL +---------------------- + +This makes the object ``hello.txt`` to be publicly readable and +``secret_plans.txt`` to be private. + +.. code-block:: php + + <?php + $Connection->set_object_acl('my-bucket-name', 'hello.txt', AmazonS3::ACL_PUBLIC); + $Connection->set_object_acl('my-bucket-name', 'secret_plans.txt', AmazonS3::ACL_PRIVATE); + + +Delete an Object +---------------- + +This deletes the object ``goodbye.txt`` + +.. code-block:: php + + <?php + $Connection->delete_object('my-bucket-name', 'goodbye.txt'); + + +Download an Object (to a file) +------------------------------ + +This downloads the object ``poetry.pdf`` and saves it in +``/home/larry/documents/`` + +.. code-block:: php + + <?php + $FileHandle = fopen('/home/larry/documents/poetry.pdf', 'w+'); + $Connection->get_object('my-bucket-name', 'poetry.pdf', array( + 'fileDownload' => $FileHandle, + )); + + +Generate Object Download URLs (signed and unsigned) +--------------------------------------------------- + +This generates an unsigned download URL for ``hello.txt``. +This works because we made ``hello.txt`` public by setting +the ACL above. This then generates a signed download URL +for ``secret_plans.txt`` that will work for 1 hour. +Signed download URLs will work for the time period even +if the object is private (when the time period is up, +the URL will stop working). + +.. code-block:: php + + <?php + my $plans_url = $Connection->get_object_url('my-bucket-name', 'hello.txt'); + echo $plans_url . "\n"; + my $secret_url = $Connection->get_object_url('my-bucket-name', 'secret_plans.txt', '1 hour'); + echo $secret_url . "\n"; + +The output of this will look something like:: + + http://objects.dreamhost.com/my-bucket-name/hello.txt + http://objects.dreamhost.com/my-bucket-name/secret_plans.txt?Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXX&Expires=1316027075&AWSAccessKeyId=XXXXXXXXXXXXXXXXXXX + diff --git a/src/ceph/doc/radosgw/s3/python.rst b/src/ceph/doc/radosgw/s3/python.rst new file mode 100644 index 0000000..a2c6a59 --- /dev/null +++ b/src/ceph/doc/radosgw/s3/python.rst @@ -0,0 +1,171 @@ +.. _python: + +Python S3 Examples +================== + +Creating a Connection +--------------------- + +This creates a connection so that you can interact with the server. + +.. code-block:: python + + import boto + import boto.s3.connection + access_key = 'put your access key here!' + secret_key = 'put your secret key here!' + + conn = boto.connect_s3( + aws_access_key_id = access_key, + aws_secret_access_key = secret_key, + host = 'objects.dreamhost.com', + #is_secure=False, # uncomment if you are not using ssl + calling_format = boto.s3.connection.OrdinaryCallingFormat(), + ) + + +Listing Owned Buckets +--------------------- + +This gets a list of Buckets that you own. +This also prints out the bucket name and creation date of each bucket. + +.. code-block:: python + + for bucket in conn.get_all_buckets(): + print "{name}\t{created}".format( + name = bucket.name, + created = bucket.creation_date, + ) + +The output will look something like this:: + + mahbuckat1 2011-04-21T18:05:39.000Z + mahbuckat2 2011-04-21T18:05:48.000Z + mahbuckat3 2011-04-21T18:07:18.000Z + + +Creating a Bucket +----------------- + +This creates a new bucket called ``my-new-bucket`` + +.. code-block:: python + + bucket = conn.create_bucket('my-new-bucket') + + +Listing a Bucket's Content +-------------------------- + +This gets a list of objects in the bucket. +This also prints out each object's name, the file size, and last +modified date. + +.. code-block:: python + + for key in bucket.list(): + print "{name}\t{size}\t{modified}".format( + name = key.name, + size = key.size, + modified = key.last_modified, + ) + +The output will look something like this:: + + myphoto1.jpg 251262 2011-08-08T21:35:48.000Z + myphoto2.jpg 262518 2011-08-08T21:38:01.000Z + + +Deleting a Bucket +----------------- + +.. note:: + + The Bucket must be empty! Otherwise it won't work! + +.. code-block:: python + + conn.delete_bucket(bucket.name) + + +Forced Delete for Non-empty Buckets +----------------------------------- + +.. attention:: + + not available in python + + +Creating an Object +------------------ + +This creates a file ``hello.txt`` with the string ``"Hello World!"`` + +.. code-block:: python + + key = bucket.new_key('hello.txt') + key.set_contents_from_string('Hello World!') + + +Change an Object's ACL +---------------------- + +This makes the object ``hello.txt`` to be publicly readable, and +``secret_plans.txt`` to be private. + +.. code-block:: python + + hello_key = bucket.get_key('hello.txt') + hello_key.set_canned_acl('public-read') + plans_key = bucket.get_key('secret_plans.txt') + plans_key.set_canned_acl('private') + + +Download an Object (to a file) +------------------------------ + +This downloads the object ``perl_poetry.pdf`` and saves it in +``/home/larry/documents/`` + +.. code-block:: python + + key = bucket.get_key('perl_poetry.pdf') + key.get_contents_to_filename('/home/larry/documents/perl_poetry.pdf') + + +Delete an Object +---------------- + +This deletes the object ``goodbye.txt`` + +.. code-block:: python + + bucket.delete_key('goodbye.txt') + + +Generate Object Download URLs (signed and unsigned) +--------------------------------------------------- + +This generates an unsigned download URL for ``hello.txt``. This works +because we made ``hello.txt`` public by setting the ACL above. +This then generates a signed download URL for ``secret_plans.txt`` that +will work for 1 hour. Signed download URLs will work for the time +period even if the object is private (when the time period is up, the +URL will stop working). + +.. code-block:: python + + hello_key = bucket.get_key('hello.txt') + hello_url = hello_key.generate_url(0, query_auth=False, force_http=True) + print hello_url + + plans_key = bucket.get_key('secret_plans.txt') + plans_url = plans_key.generate_url(3600, query_auth=True, force_http=True) + print plans_url + +The output of this will look something like:: + + http://objects.dreamhost.com/my-bucket-name/hello.txt + http://objects.dreamhost.com/my-bucket-name/secret_plans.txt?Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXX&Expires=1316027075&AWSAccessKeyId=XXXXXXXXXXXXXXXXXXX + diff --git a/src/ceph/doc/radosgw/s3/ruby.rst b/src/ceph/doc/radosgw/s3/ruby.rst new file mode 100644 index 0000000..435b3c6 --- /dev/null +++ b/src/ceph/doc/radosgw/s3/ruby.rst @@ -0,0 +1,364 @@ +.. _ruby: + +Ruby `AWS::SDK`_ Examples (aws-sdk gem ~>2) +=========================================== + +Settings +--------------------- + +You can setup the connection on global way: + +.. code-block:: ruby + + Aws.config.update( + endpoint: 'https://objects.dreamhost.com.', + access_key_id: 'my-access-key', + secret_access_key: 'my-secret-key', + force_path_style: true, + region: 'us-east-1' + ) + + +and instantiate a client object: + +.. code-block:: ruby + + s3_client = Aws::S3::Client.new + +Listing Owned Buckets +--------------------- + +This gets a list of buckets that you own. +This also prints out the bucket name and creation date of each bucket. + +.. code-block:: ruby + + s3_client.list_buckets.buckets.each do |bucket| + puts "#{bucket.name}\t#{bucket.creation_date}" + end + +The output will look something like this:: + + mahbuckat1 2011-04-21T18:05:39.000Z + mahbuckat2 2011-04-21T18:05:48.000Z + mahbuckat3 2011-04-21T18:07:18.000Z + + +Creating a Bucket +----------------- + +This creates a new bucket called ``my-new-bucket`` + +.. code-block:: ruby + + s3_client.create_bucket(bucket: 'my-new-bucket') + +If you want a private bucket: + +`acl` option accepts: # private, public-read, public-read-write, authenticated-read + +.. code-block:: ruby + + s3_client.create_bucket(bucket: 'my-new-bucket', acl: 'private') + + +Listing a Bucket's Content +-------------------------- + +This gets a list of hashes with the contents of each object +This also prints out each object's name, the file size, and last +modified date. + +.. code-block:: ruby + + s3_client.get_objects(bucket: 'my-new-bucket').contents.each do |object| + puts "#{object.key}\t#{object.size}\t#{object.last-modified}" + end + +The output will look something like this if the bucket has some files:: + + myphoto1.jpg 251262 2011-08-08T21:35:48.000Z + myphoto2.jpg 262518 2011-08-08T21:38:01.000Z + + +Deleting a Bucket +----------------- +.. note:: + The Bucket must be empty! Otherwise it won't work! + +.. code-block:: ruby + + s3_client.delete_bucket(bucket: 'my-new-bucket') + + +Forced Delete for Non-empty Buckets +----------------------------------- +First, you need to clear the bucket: + +.. code-block:: ruby + + Aws::S3::Bucket.new('my-new-bucket', client: s3_client).clear! + +after, you can destroy the bucket + +.. code-block:: ruby + + s3_client.delete_bucket(bucket: 'my-new-bucket') + + +Creating an Object +------------------ + +This creates a file ``hello.txt`` with the string ``"Hello World!"`` + +.. code-block:: ruby + + s3_client.put_object( + key: 'hello.txt', + body: 'Hello World!', + bucket: 'my-new-bucket', + content_type: 'text/plain' + ) + + +Change an Object's ACL +---------------------- + +This makes the object ``hello.txt`` to be publicly readable, and ``secret_plans.txt`` +to be private. + +.. code-block:: ruby + + s3_client.put_object_acl(bucket: 'my-new-bucket', key: 'hello.txt', acl: 'public-read') + + s3_client.put_object_acl(bucket: 'my-new-bucket', key: 'private.txt', acl: 'private') + + +Download an Object (to a file) +------------------------------ + +This downloads the object ``poetry.pdf`` and saves it in +``/home/larry/documents/`` + +.. code-block:: ruby + + s3_client.get_object(bucket: 'my-new-bucket', key: 'poetry.pdf', response_target: '/home/larry/documents/poetry.pdf') + + +Delete an Object +---------------- + +This deletes the object ``goodbye.txt`` + +.. code-block:: ruby + + s3_client.delete_object(key: 'goodbye.txt', bucket: 'my-new-bucket') + + +Generate Object Download URLs (signed and unsigned) +--------------------------------------------------- + +This generates an unsigned download URL for ``hello.txt``. This works +because we made ``hello.txt`` public by setting the ACL above. +This then generates a signed download URL for ``secret_plans.txt`` that +will work for 1 hour. Signed download URLs will work for the time +period even if the object is private (when the time period is up, the +URL will stop working). + +.. code-block:: ruby + + puts Aws::S3::Object.new( + key: 'hello.txt', + bucket_name: 'my-new-bucket', + client: s3_client + ).public_url + + puts Aws::S3::Object.new( + key: 'secret_plans.txt', + bucket_name: 'hermes_ceph_gem', + client: s3_client + ).presigned_url(:get, expires_in: 60 * 60) + +The output of this will look something like:: + + http://objects.dreamhost.com/my-bucket-name/hello.txt + http://objects.dreamhost.com/my-bucket-name/secret_plans.txt?Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXX&Expires=1316027075&AWSAccessKeyId=XXXXXXXXXXXXXXXXXXX + +.. _`AWS::SDK`: http://docs.aws.amazon.com/sdkforruby/api/Aws/S3/Client.html + + + +Ruby `AWS::S3`_ Examples (aws-s3 gem) +===================================== + +Creating a Connection +--------------------- + +This creates a connection so that you can interact with the server. + +.. code-block:: ruby + + AWS::S3::Base.establish_connection!( + :server => 'objects.dreamhost.com', + :use_ssl => true, + :access_key_id => 'my-access-key', + :secret_access_key => 'my-secret-key' + ) + + +Listing Owned Buckets +--------------------- + +This gets a list of `AWS::S3::Bucket`_ objects that you own. +This also prints out the bucket name and creation date of each bucket. + +.. code-block:: ruby + + AWS::S3::Service.buckets.each do |bucket| + puts "#{bucket.name}\t#{bucket.creation_date}" + end + +The output will look something like this:: + + mahbuckat1 2011-04-21T18:05:39.000Z + mahbuckat2 2011-04-21T18:05:48.000Z + mahbuckat3 2011-04-21T18:07:18.000Z + + +Creating a Bucket +----------------- + +This creates a new bucket called ``my-new-bucket`` + +.. code-block:: ruby + + AWS::S3::Bucket.create('my-new-bucket') + + +Listing a Bucket's Content +-------------------------- + +This gets a list of hashes with the contents of each object +This also prints out each object's name, the file size, and last +modified date. + +.. code-block:: ruby + + new_bucket = AWS::S3::Bucket.find('my-new-bucket') + new_bucket.each do |object| + puts "#{object.key}\t#{object.about['content-length']}\t#{object.about['last-modified']}" + end + +The output will look something like this if the bucket has some files:: + + myphoto1.jpg 251262 2011-08-08T21:35:48.000Z + myphoto2.jpg 262518 2011-08-08T21:38:01.000Z + + +Deleting a Bucket +----------------- +.. note:: + The Bucket must be empty! Otherwise it won't work! + +.. code-block:: ruby + + AWS::S3::Bucket.delete('my-new-bucket') + + +Forced Delete for Non-empty Buckets +----------------------------------- + +.. code-block:: ruby + + AWS::S3::Bucket.delete('my-new-bucket', :force => true) + + +Creating an Object +------------------ + +This creates a file ``hello.txt`` with the string ``"Hello World!"`` + +.. code-block:: ruby + + AWS::S3::S3Object.store( + 'hello.txt', + 'Hello World!', + 'my-new-bucket', + :content_type => 'text/plain' + ) + + +Change an Object's ACL +---------------------- + +This makes the object ``hello.txt`` to be publicly readable, and ``secret_plans.txt`` +to be private. + +.. code-block:: ruby + + policy = AWS::S3::S3Object.acl('hello.txt', 'my-new-bucket') + policy.grants = [ AWS::S3::ACL::Grant.grant(:public_read) ] + AWS::S3::S3Object.acl('hello.txt', 'my-new-bucket', policy) + + policy = AWS::S3::S3Object.acl('secret_plans.txt', 'my-new-bucket') + policy.grants = [] + AWS::S3::S3Object.acl('secret_plans.txt', 'my-new-bucket', policy) + + +Download an Object (to a file) +------------------------------ + +This downloads the object ``poetry.pdf`` and saves it in +``/home/larry/documents/`` + +.. code-block:: ruby + + open('/home/larry/documents/poetry.pdf', 'w') do |file| + AWS::S3::S3Object.stream('poetry.pdf', 'my-new-bucket') do |chunk| + file.write(chunk) + end + end + + +Delete an Object +---------------- + +This deletes the object ``goodbye.txt`` + +.. code-block:: ruby + + AWS::S3::S3Object.delete('goodbye.txt', 'my-new-bucket') + + +Generate Object Download URLs (signed and unsigned) +--------------------------------------------------- + +This generates an unsigned download URL for ``hello.txt``. This works +because we made ``hello.txt`` public by setting the ACL above. +This then generates a signed download URL for ``secret_plans.txt`` that +will work for 1 hour. Signed download URLs will work for the time +period even if the object is private (when the time period is up, the +URL will stop working). + +.. code-block:: ruby + + puts AWS::S3::S3Object.url_for( + 'hello.txt', + 'my-new-bucket', + :authenticated => false + ) + + puts AWS::S3::S3Object.url_for( + 'secret_plans.txt', + 'my-new-bucket', + :expires_in => 60 * 60 + ) + +The output of this will look something like:: + + http://objects.dreamhost.com/my-bucket-name/hello.txt + http://objects.dreamhost.com/my-bucket-name/secret_plans.txt?Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXX&Expires=1316027075&AWSAccessKeyId=XXXXXXXXXXXXXXXXXXX + +.. _`AWS::S3`: http://amazon.rubyforge.org/ +.. _`AWS::S3::Bucket`: http://amazon.rubyforge.org/doc/ + diff --git a/src/ceph/doc/radosgw/s3/serviceops.rst b/src/ceph/doc/radosgw/s3/serviceops.rst new file mode 100644 index 0000000..c55ce98 --- /dev/null +++ b/src/ceph/doc/radosgw/s3/serviceops.rst @@ -0,0 +1,39 @@ +Service Operations +================== + +List Buckets +------------ +``GET /`` returns a list of buckets created by the user making the request. ``GET /`` only +returns buckets created by an authenticated user. You cannot make an anonymous request. + +Syntax +~~~~~~ +:: + + GET / HTTP/1.1 + Host: cname.domain.com + + Authorization: AWS {access-key}:{hash-of-header-and-secret} + +Response Entities +~~~~~~~~~~~~~~~~~ + ++----------------------------+-------------+-----------------------------------------------------------------+ +| Name | Type | Description | ++============================+=============+=================================================================+ +| ``Buckets`` | Container | Container for list of buckets. | ++----------------------------+-------------+-----------------------------------------------------------------+ +| ``Bucket`` | Container | Container for bucket information. | ++----------------------------+-------------+-----------------------------------------------------------------+ +| ``Name`` | String | Bucket name. | ++----------------------------+-------------+-----------------------------------------------------------------+ +| ``CreationDate`` | Date | UTC time when the bucket was created. | ++----------------------------+-------------+-----------------------------------------------------------------+ +| ``ListAllMyBucketsResult`` | Container | A container for the result. | ++----------------------------+-------------+-----------------------------------------------------------------+ +| ``Owner`` | Container | A container for the bucket owner's ``ID`` and ``DisplayName``. | ++----------------------------+-------------+-----------------------------------------------------------------+ +| ``ID`` | String | The bucket owner's ID. | ++----------------------------+-------------+-----------------------------------------------------------------+ +| ``DisplayName`` | String | The bucket owner's display name. | ++----------------------------+-------------+-----------------------------------------------------------------+ |