diff options
Diffstat (limited to 'src/ceph/doc/install/install-ceph-gateway.rst')
| -rw-r--r-- | src/ceph/doc/install/install-ceph-gateway.rst | 605 |
1 files changed, 0 insertions, 605 deletions
diff --git a/src/ceph/doc/install/install-ceph-gateway.rst b/src/ceph/doc/install/install-ceph-gateway.rst deleted file mode 100644 index cf599cd..0000000 --- a/src/ceph/doc/install/install-ceph-gateway.rst +++ /dev/null @@ -1,605 +0,0 @@ -=========================== -Install Ceph Object Gateway -=========================== - -As of `firefly` (v0.80), Ceph Object Gateway is running on Civetweb (embedded -into the ``ceph-radosgw`` daemon) instead of Apache and FastCGI. Using Civetweb -simplifies the Ceph Object Gateway installation and configuration. - -.. note:: To run the Ceph Object Gateway service, you should have a running - Ceph storage cluster, and the gateway host should have access to the - public network. - -.. note:: In version 0.80, the Ceph Object Gateway does not support SSL. You - may setup a reverse proxy server with SSL to dispatch HTTPS requests - as HTTP requests to CivetWeb. - -Execute the Pre-Installation Procedure --------------------------------------- - -See Preflight_ and execute the pre-installation procedures on your Ceph Object -Gateway node. Specifically, you should disable ``requiretty`` on your Ceph -Deploy user, set SELinux to ``Permissive`` and set up a Ceph Deploy user with -password-less ``sudo``. For Ceph Object Gateways, you will need to open the -port that Civetweb will use in production. - -.. note:: Civetweb runs on port ``7480`` by default. - -Install Ceph Object Gateway ---------------------------- - -From the working directory of your administration server, install the Ceph -Object Gateway package on the Ceph Object Gateway node. For example:: - - ceph-deploy install --rgw <gateway-node1> [<gateway-node2> ...] - -The ``ceph-common`` package is a dependency, so ``ceph-deploy`` will install -this too. The ``ceph`` CLI tools are intended for administrators. To make your -Ceph Object Gateway node an administrator node, execute the following from the -working directory of your administration server:: - - ceph-deploy admin <node-name> - -Create a Gateway Instance -------------------------- - -From the working directory of your administration server, create an instance of -the Ceph Object Gateway on the Ceph Object Gateway. For example:: - - ceph-deploy rgw create <gateway-node1> - -Once the gateway is running, you should be able to access it on port ``7480`` -with an unauthenticated request like this:: - - http://client-node:7480 - -If the gateway instance is working properly, you should receive a response like -this:: - - <?xml version="1.0" encoding="UTF-8"?> - <ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> - <Owner> - <ID>anonymous</ID> - <DisplayName></DisplayName> - </Owner> - <Buckets> - </Buckets> - </ListAllMyBucketsResult> - -If at any point you run into trouble and you want to start over, execute the -following to purge the configuration:: - - ceph-deploy purge <gateway-node1> [<gateway-node2>] - ceph-deploy purgedata <gateway-node1> [<gateway-node2>] - -If you execute ``purge``, you must re-install Ceph. - -Change the Default Port ------------------------ - -Civetweb runs on port ``7480`` by default. To change the default port (e.g., to -port ``80``), modify your Ceph configuration file in the working directory of -your administration server. Add a section entitled -``[client.rgw.<gateway-node>]``, replacing ``<gateway-node>`` with the short -node name of your Ceph Object Gateway node (i.e., ``hostname -s``). - -.. note:: As of version 11.0.1, the Ceph Object Gateway **does** support SSL. - See `Using SSL with Civetweb`_ for information on how to set that up. - -For example, if your node name is ``gateway-node1``, add a section like this -after the ``[global]`` section:: - - [client.rgw.gateway-node1] - rgw_frontends = "civetweb port=80" - -.. note:: Ensure that you leave no whitespace between ``port=<port-number>`` in - the ``rgw_frontends`` key/value pair. The ``[client.rgw.gateway-node1]`` - heading identifies this portion of the Ceph configuration file as - configuring a Ceph Storage Cluster client where the client type is a Ceph - Object Gateway (i.e., ``rgw``), and the name of the instance is - ``gateway-node1``. - -Push the updated configuration file to your Ceph Object Gateway node -(and other Ceph nodes):: - - ceph-deploy --overwrite-conf config push <gateway-node> [<other-nodes>] - -To make the new port setting take effect, restart the Ceph Object -Gateway:: - - sudo systemctl restart ceph-radosgw.service - -Finally, check to ensure that the port you selected is open on the node's -firewall (e.g., port ``80``). If it is not open, add the port and reload the -firewall configuration. If you use the ``firewalld`` daemon, execute:: - - sudo firewall-cmd --list-all - sudo firewall-cmd --zone=public --add-port 80/tcp --permanent - sudo firewall-cmd --reload - -If you use ``iptables``, execute:: - - sudo iptables --list - sudo iptables -I INPUT 1 -i <iface> -p tcp -s <ip-address>/<netmask> --dport 80 -j ACCEPT - -Replace ``<iface>`` and ``<ip-address>/<netmask>`` with the relevant values for -your Ceph Object Gateway node. - -Once you have finished configuring ``iptables``, ensure that you make the -change persistent so that it will be in effect when your Ceph Object Gateway -node reboots. Execute:: - - sudo apt-get install iptables-persistent - -A terminal UI will open up. Select ``yes`` for the prompts to save current -``IPv4`` iptables rules to ``/etc/iptables/rules.v4`` and current ``IPv6`` -iptables rules to ``/etc/iptables/rules.v6``. - -The ``IPv4`` iptables rule that you set in the earlier step will be loaded in -``/etc/iptables/rules.v4`` and will be persistent across reboots. - -If you add a new ``IPv4`` iptables rule after installing -``iptables-persistent`` you will have to add it to the rule file. In such case, -execute the following as the ``root`` user:: - - iptables-save > /etc/iptables/rules.v4 - -Using SSL with Civetweb ------------------------ -.. _Using SSL with Civetweb: - -Before using SSL with civetweb, you will need a certificate that will match -the host name that that will be used to access the Ceph Object Gateway. -You may wish to obtain one that has `subject alternate name` fields for -more flexibility. If you intend to use S3-style subdomains -(`Add Wildcard to DNS`_), you will need a `wildcard` certificate. - -Civetweb requires that the server key, server certificate, and any other -CA or intermediate certificates be supplied in one file. Each of these -items must be in `pem` form. Because the combined file contains the -secret key, it should be protected from unauthorized access. - -To configure ssl operation, append ``s`` to the port number. Currently -it is not possible to configure the radosgw to listen on both -http and https, you must pick only one. So:: - - [client.rgw.gateway-node1] - rgw_frontends = civetweb port=443s ssl_certificate=/etc/ceph/private/keyandcert.pem - -Migrating from Apache to Civetweb ---------------------------------- - -If you are running the Ceph Object Gateway on Apache and FastCGI with Ceph -Storage v0.80 or above, you are already running Civetweb--it starts with the -``ceph-radosgw`` daemon and it's running on port 7480 by default so that it -doesn't conflict with your Apache and FastCGI installation and other commonly -used web service ports. Migrating to use Civetweb basically involves removing -your Apache installation. Then, you must remove Apache and FastCGI settings -from your Ceph configuration file and reset ``rgw_frontends`` to Civetweb. - -Referring back to the description for installing a Ceph Object Gateway with -``ceph-deploy``, notice that the configuration file only has one setting -``rgw_frontends`` (and that's assuming you elected to change the default port). -The ``ceph-deploy`` utility generates the data directory and the keyring for -you--placing the keyring in ``/var/lib/ceph/radosgw/{rgw-intance}``. The daemon -looks in default locations, whereas you may have specified different settings -in your Ceph configuration file. Since you already have keys and a data -directory, you will want to maintain those paths in your Ceph configuration -file if you used something other than default paths. - -A typical Ceph Object Gateway configuration file for an Apache-based deployment -looks something similar as the following: - -On Red Hat Enterprise Linux:: - - [client.radosgw.gateway-node1] - host = {hostname} - keyring = /etc/ceph/ceph.client.radosgw.keyring - rgw socket path = "" - log file = /var/log/radosgw/client.radosgw.gateway-node1.log - rgw frontends = fastcgi socket\_port=9000 socket\_host=0.0.0.0 - rgw print continue = false - -On Ubuntu:: - - [client.radosgw.gateway-node] - host = {hostname} - keyring = /etc/ceph/ceph.client.radosgw.keyring - rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock - log file = /var/log/radosgw/client.radosgw.gateway-node1.log - -To modify it for use with Civetweb, simply remove the Apache-specific settings -such as ``rgw_socket_path`` and ``rgw_print_continue``. Then, change the -``rgw_frontends`` setting to reflect Civetweb rather than the Apache FastCGI -front end and specify the port number you intend to use. For example:: - - [client.radosgw.gateway-node1] - host = {hostname} - keyring = /etc/ceph/ceph.client.radosgw.keyring - log file = /var/log/radosgw/client.radosgw.gateway-node1.log - rgw_frontends = civetweb port=80 - -Finally, restart the Ceph Object Gateway. On Red Hat Enterprise Linux execute:: - - sudo systemctl restart ceph-radosgw.service - -On Ubuntu execute:: - - sudo service radosgw restart id=rgw.<short-hostname> - -If you used a port number that is not open, you will also need to open that -port on your firewall. - -Configure Bucket Sharding -------------------------- - -A Ceph Object Gateway stores bucket index data in the ``index_pool``, which -defaults to ``.rgw.buckets.index``. Sometimes users like to put many objects -(hundreds of thousands to millions of objects) in a single bucket. If you do -not use the gateway administration interface to set quotas for the maximum -number of objects per bucket, the bucket index can suffer significant -performance degradation when users place large numbers of objects into a -bucket. - -In Ceph 0.94, you may shard bucket indices to help prevent performance -bottlenecks when you allow a high number of objects per bucket. The -``rgw_override_bucket_index_max_shards`` setting allows you to set a maximum -number of shards per bucket. The default value is ``0``, which means bucket -index sharding is off by default. - -To turn bucket index sharding on, set ``rgw_override_bucket_index_max_shards`` -to a value greater than ``0``. - -For simple configurations, you may add ``rgw_override_bucket_index_max_shards`` -to your Ceph configuration file. Add it under ``[global]`` to create a -system-wide value. You can also set it for each instance in your Ceph -configuration file. - -Once you have changed your bucket sharding configuration in your Ceph -configuration file, restart your gateway. On Red Hat Enteprise Linux execute:: - - sudo systemctl restart ceph-radosgw.service - -On Ubuntu execute:: - - sudo service radosgw restart id=rgw.<short-hostname> - -For federated configurations, each zone may have a different ``index_pool`` -setting for failover. To make the value consistent for a region's zones, you -may set ``rgw_override_bucket_index_max_shards`` in a gateway's region -configuration. For example:: - - radosgw-admin region get > region.json - -Open the ``region.json`` file and edit the ``bucket_index_max_shards`` setting -for each named zone. Save the ``region.json`` file and reset the region. For -example:: - - radosgw-admin region set < region.json - -Once you have updated your region, update the region map. For example:: - - radosgw-admin regionmap update --name client.rgw.ceph-client - -Where ``client.rgw.ceph-client`` is the name of the gateway user. - -.. note:: Mapping the index pool (for each zone, if applicable) to a CRUSH - ruleset of SSD-based OSDs may also help with bucket index performance. - -Add Wildcard to DNS -------------------- -.. _Add Wildcard to DNS: - -To use Ceph with S3-style subdomains (e.g., bucket-name.domain-name.com), you -need to add a wildcard to the DNS record of the DNS server you use with the -``ceph-radosgw`` daemon. - -The address of the DNS must also be specified in the Ceph configuration file -with the ``rgw dns name = {hostname}`` setting. - -For ``dnsmasq``, add the following address setting with a dot (.) prepended to -the host name:: - - address=/.{hostname-or-fqdn}/{host-ip-address} - -For example:: - - address=/.gateway-node1/192.168.122.75 - - -For ``bind``, add a wildcard to the DNS record. For example:: - - $TTL 604800 - @ IN SOA gateway-node1. root.gateway-node1. ( - 2 ; Serial - 604800 ; Refresh - 86400 ; Retry - 2419200 ; Expire - 604800 ) ; Negative Cache TTL - ; - @ IN NS gateway-node1. - @ IN A 192.168.122.113 - * IN CNAME @ - -Restart your DNS server and ping your server with a subdomain to ensure that -your DNS configuration works as expected:: - - ping mybucket.{hostname} - -For example:: - - ping mybucket.gateway-node1 - -Add Debugging (if needed) -------------------------- - -Once you finish the setup procedure, if you encounter issues with your -configuration, you can add debugging to the ``[global]`` section of your Ceph -configuration file and restart the gateway(s) to help troubleshoot any -configuration issues. For example:: - - [global] - #append the following in the global section. - debug ms = 1 - debug rgw = 20 - -Using the Gateway ------------------ - -To use the REST interfaces, first create an initial Ceph Object Gateway user -for the S3 interface. Then, create a subuser for the Swift interface. You then -need to verify if the created users are able to access the gateway. - -Create a RADOSGW User for S3 Access -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -A ``radosgw`` user needs to be created and granted access. The command ``man -radosgw-admin`` will provide information on additional command options. - -To create the user, execute the following on the ``gateway host``:: - - sudo radosgw-admin user create --uid="testuser" --display-name="First User" - -The output of the command will be something like the following:: - - { - "user_id": "testuser", - "display_name": "First User", - "email": "", - "suspended": 0, - "max_buckets": 1000, - "auid": 0, - "subusers": [], - "keys": [{ - "user": "testuser", - "access_key": "I0PJDPCIYZ665MW88W9R", - "secret_key": "dxaXZ8U90SXydYzyS5ivamEP20hkLSUViiaR+ZDA" - }], - "swift_keys": [], - "caps": [], - "op_mask": "read, write, delete", - "default_placement": "", - "placement_tags": [], - "bucket_quota": { - "enabled": false, - "max_size_kb": -1, - "max_objects": -1 - }, - "user_quota": { - "enabled": false, - "max_size_kb": -1, - "max_objects": -1 - }, - "temp_url_keys": [] - } - -.. note:: The values of ``keys->access_key`` and ``keys->secret_key`` are - needed for access validation. - -.. important:: Check the key output. Sometimes ``radosgw-admin`` generates a - JSON escape character ``\`` in ``access_key`` or ``secret_key`` - and some clients do not know how to handle JSON escape - characters. Remedies include removing the JSON escape character - ``\``, encapsulating the string in quotes, regenerating the key - and ensuring that it does not have a JSON escape character or - specify the key and secret manually. Also, if ``radosgw-admin`` - generates a JSON escape character ``\`` and a forward slash ``/`` - together in a key, like ``\/``, only remove the JSON escape - character ``\``. Do not remove the forward slash ``/`` as it is - a valid character in the key. - -Create a Swift User -^^^^^^^^^^^^^^^^^^^ - -A Swift subuser needs to be created if this kind of access is needed. Creating -a Swift user is a two step process. The first step is to create the user. The -second is to create the secret key. - -Execute the following steps on the ``gateway host``: - -Create the Swift user:: - - sudo radosgw-admin subuser create --uid=testuser --subuser=testuser:swift --access=full - -The output will be something like the following:: - - { - "user_id": "testuser", - "display_name": "First User", - "email": "", - "suspended": 0, - "max_buckets": 1000, - "auid": 0, - "subusers": [{ - "id": "testuser:swift", - "permissions": "full-control" - }], - "keys": [{ - "user": "testuser:swift", - "access_key": "3Y1LNW4Q6X0Y53A52DET", - "secret_key": "" - }, { - "user": "testuser", - "access_key": "I0PJDPCIYZ665MW88W9R", - "secret_key": "dxaXZ8U90SXydYzyS5ivamEP20hkLSUViiaR+ZDA" - }], - "swift_keys": [], - "caps": [], - "op_mask": "read, write, delete", - "default_placement": "", - "placement_tags": [], - "bucket_quota": { - "enabled": false, - "max_size_kb": -1, - "max_objects": -1 - }, - "user_quota": { - "enabled": false, - "max_size_kb": -1, - "max_objects": -1 - }, - "temp_url_keys": [] - } - -Create the secret key:: - - sudo radosgw-admin key create --subuser=testuser:swift --key-type=swift --gen-secret - -The output will be something like the following:: - - { - "user_id": "testuser", - "display_name": "First User", - "email": "", - "suspended": 0, - "max_buckets": 1000, - "auid": 0, - "subusers": [{ - "id": "testuser:swift", - "permissions": "full-control" - }], - "keys": [{ - "user": "testuser:swift", - "access_key": "3Y1LNW4Q6X0Y53A52DET", - "secret_key": "" - }, { - "user": "testuser", - "access_key": "I0PJDPCIYZ665MW88W9R", - "secret_key": "dxaXZ8U90SXydYzyS5ivamEP20hkLSUViiaR+ZDA" - }], - "swift_keys": [{ - "user": "testuser:swift", - "secret_key": "244+fz2gSqoHwR3lYtSbIyomyPHf3i7rgSJrF\/IA" - }], - "caps": [], - "op_mask": "read, write, delete", - "default_placement": "", - "placement_tags": [], - "bucket_quota": { - "enabled": false, - "max_size_kb": -1, - "max_objects": -1 - }, - "user_quota": { - "enabled": false, - "max_size_kb": -1, - "max_objects": -1 - }, - "temp_url_keys": [] - } - -Access Verification -^^^^^^^^^^^^^^^^^^^ - -Test S3 Access -"""""""""""""" - -You need to write and run a Python test script for verifying S3 access. The S3 -access test script will connect to the ``radosgw``, create a new bucket and -list all buckets. The values for ``aws_access_key_id`` and -``aws_secret_access_key`` are taken from the values of ``access_key`` and -``secret_key`` returned by the ``radosgw-admin`` command. - -Execute the following steps: - -#. You will need to install the ``python-boto`` package:: - - sudo yum install python-boto - -#. Create the Python script:: - - vi s3test.py - -#. Add the following contents to the file:: - - import boto.s3.connection - - access_key = 'I0PJDPCIYZ665MW88W9R' - secret_key = 'dxaXZ8U90SXydYzyS5ivamEP20hkLSUViiaR+ZDA' - conn = boto.connect_s3( - aws_access_key_id=access_key, - aws_secret_access_key=secret_key, - host='{hostname}', port={port}, - is_secure=False, calling_format=boto.s3.connection.OrdinaryCallingFormat(), - ) - - bucket = conn.create_bucket('my-new-bucket') - for bucket in conn.get_all_buckets(): - print "{name} {created}".format( - name=bucket.name, - created=bucket.creation_date, - ) - - - Replace ``{hostname}`` with the hostname of the host where you have - configured the gateway service i.e., the ``gateway host``. Replace ``{port}`` - with the port number you are using with Civetweb. - -#. Run the script:: - - python s3test.py - - The output will be something like the following:: - - my-new-bucket 2015-02-16T17:09:10.000Z - -Test swift access -""""""""""""""""" - -Swift access can be verified via the ``swift`` command line client. The command -``man swift`` will provide more information on available command line options. - -To install ``swift`` client, execute the following commands. On Red Hat -Enterprise Linux:: - - sudo yum install python-setuptools - sudo easy_install pip - sudo pip install --upgrade setuptools - sudo pip install --upgrade python-swiftclient - -On Debian-based distributions:: - - sudo apt-get install python-setuptools - sudo easy_install pip - sudo pip install --upgrade setuptools - sudo pip install --upgrade python-swiftclient - -To test swift access, execute the following:: - - swift -A http://{IP ADDRESS}:{port}/auth/1.0 -U testuser:swift -K '{swift_secret_key}' list - -Replace ``{IP ADDRESS}`` with the public IP address of the gateway server and -``{swift_secret_key}`` with its value from the output of ``radosgw-admin key -create`` command executed for the ``swift`` user. Replace {port} with the port -number you are using with Civetweb (e.g., ``7480`` is the default). If you -don't replace the port, it will default to port ``80``. - -For example:: - - swift -A http://10.19.143.116:7480/auth/1.0 -U testuser:swift -K '244+fz2gSqoHwR3lYtSbIyomyPHf3i7rgSJrF/IA' list - -The output should be:: - - my-new-bucket - -.. _Preflight: ../../start/quick-start-preflight |