summaryrefslogtreecommitdiffstats
path: root/src/ceph/doc/rados/configuration/ceph-conf.rst
diff options
context:
space:
mode:
Diffstat (limited to 'src/ceph/doc/rados/configuration/ceph-conf.rst')
-rw-r--r--src/ceph/doc/rados/configuration/ceph-conf.rst629
1 files changed, 0 insertions, 629 deletions
diff --git a/src/ceph/doc/rados/configuration/ceph-conf.rst b/src/ceph/doc/rados/configuration/ceph-conf.rst
deleted file mode 100644
index df88452..0000000
--- a/src/ceph/doc/rados/configuration/ceph-conf.rst
+++ /dev/null
@@ -1,629 +0,0 @@
-==================
- Configuring Ceph
-==================
-
-When you start the Ceph service, the initialization process activates a series
-of daemons that run in the background. A :term:`Ceph Storage Cluster` runs
-two types of daemons:
-
-- :term:`Ceph Monitor` (``ceph-mon``)
-- :term:`Ceph OSD Daemon` (``ceph-osd``)
-
-Ceph Storage Clusters that support the :term:`Ceph Filesystem` run at least one
-:term:`Ceph Metadata Server` (``ceph-mds``). Clusters that support :term:`Ceph
-Object Storage` run Ceph Gateway daemons (``radosgw``). For your convenience,
-each daemon has a series of default values (*i.e.*, many are set by
-``ceph/src/common/config_opts.h``). You may override these settings with a Ceph
-configuration file.
-
-
-.. _ceph-conf-file:
-
-The Configuration File
-======================
-
-When you start a Ceph Storage Cluster, each daemon looks for a Ceph
-configuration file (i.e., ``ceph.conf`` by default) that provides the cluster's
-configuration settings. For manual deployments, you need to create a Ceph
-configuration file. For tools that create configuration files for you (*e.g.*,
-``ceph-deploy``, Chef, etc.), you may use the information contained herein as a
-reference. The Ceph configuration file defines:
-
-- Cluster Identity
-- Authentication settings
-- Cluster membership
-- Host names
-- Host addresses
-- Paths to keyrings
-- Paths to journals
-- Paths to data
-- Other runtime options
-
-The default Ceph configuration file locations in sequential order include:
-
-#. ``$CEPH_CONF`` (*i.e.,* the path following the ``$CEPH_CONF``
- environment variable)
-#. ``-c path/path`` (*i.e.,* the ``-c`` command line argument)
-#. ``/etc/ceph/ceph.conf``
-#. ``~/.ceph/config``
-#. ``./ceph.conf`` (*i.e.,* in the current working directory)
-
-
-The Ceph configuration file uses an *ini* style syntax. You can add comments
-by preceding comments with a pound sign (#) or a semi-colon (;). For example:
-
-.. code-block:: ini
-
- # <--A number (#) sign precedes a comment.
- ; A comment may be anything.
- # Comments always follow a semi-colon (;) or a pound (#) on each line.
- # The end of the line terminates a comment.
- # We recommend that you provide comments in your configuration file(s).
-
-
-.. _ceph-conf-settings:
-
-Config Sections
-===============
-
-The configuration file can configure all Ceph daemons in a Ceph Storage Cluster,
-or all Ceph daemons of a particular type. To configure a series of daemons, the
-settings must be included under the processes that will receive the
-configuration as follows:
-
-``[global]``
-
-:Description: Settings under ``[global]`` affect all daemons in a Ceph Storage
- Cluster.
-
-:Example: ``auth supported = cephx``
-
-``[osd]``
-
-:Description: Settings under ``[osd]`` affect all ``ceph-osd`` daemons in
- the Ceph Storage Cluster, and override the same setting in
- ``[global]``.
-
-:Example: ``osd journal size = 1000``
-
-``[mon]``
-
-:Description: Settings under ``[mon]`` affect all ``ceph-mon`` daemons in
- the Ceph Storage Cluster, and override the same setting in
- ``[global]``.
-
-:Example: ``mon addr = 10.0.0.101:6789``
-
-
-``[mds]``
-
-:Description: Settings under ``[mds]`` affect all ``ceph-mds`` daemons in
- the Ceph Storage Cluster, and override the same setting in
- ``[global]``.
-
-:Example: ``host = myserver01``
-
-``[client]``
-
-:Description: Settings under ``[client]`` affect all Ceph Clients
- (e.g., mounted Ceph Filesystems, mounted Ceph Block Devices,
- etc.).
-
-:Example: ``log file = /var/log/ceph/radosgw.log``
-
-
-Global settings affect all instances of all daemon in the Ceph Storage Cluster.
-Use the ``[global]`` setting for values that are common for all daemons in the
-Ceph Storage Cluster. You can override each ``[global]`` setting by:
-
-#. Changing the setting in a particular process type
- (*e.g.,* ``[osd]``, ``[mon]``, ``[mds]`` ).
-
-#. Changing the setting in a particular process (*e.g.,* ``[osd.1]`` ).
-
-Overriding a global setting affects all child processes, except those that
-you specifically override in a particular daemon.
-
-A typical global setting involves activating authentication. For example:
-
-.. code-block:: ini
-
- [global]
- #Enable authentication between hosts within the cluster.
- #v 0.54 and earlier
- auth supported = cephx
-
- #v 0.55 and after
- auth cluster required = cephx
- auth service required = cephx
- auth client required = cephx
-
-
-You can specify settings that apply to a particular type of daemon. When you
-specify settings under ``[osd]``, ``[mon]`` or ``[mds]`` without specifying a
-particular instance, the setting will apply to all OSDs, monitors or metadata
-daemons respectively.
-
-A typical daemon-wide setting involves setting journal sizes, filestore
-settings, etc. For example:
-
-.. code-block:: ini
-
- [osd]
- osd journal size = 1000
-
-
-You may specify settings for particular instances of a daemon. You may specify
-an instance by entering its type, delimited by a period (.) and by the instance
-ID. The instance ID for a Ceph OSD Daemon is always numeric, but it may be
-alphanumeric for Ceph Monitors and Ceph Metadata Servers.
-
-.. code-block:: ini
-
- [osd.1]
- # settings affect osd.1 only.
-
- [mon.a]
- # settings affect mon.a only.
-
- [mds.b]
- # settings affect mds.b only.
-
-
-If the daemon you specify is a Ceph Gateway client, specify the daemon and the
-instance, delimited by a period (.). For example::
-
- [client.radosgw.instance-name]
- # settings affect client.radosgw.instance-name only.
-
-
-
-.. _ceph-metavariables:
-
-Metavariables
-=============
-
-Metavariables simplify Ceph Storage Cluster configuration dramatically. When a
-metavariable is set in a configuration value, Ceph expands the metavariable into
-a concrete value. Metavariables are very powerful when used within the
-``[global]``, ``[osd]``, ``[mon]``, ``[mds]`` or ``[client]`` sections of your
-configuration file. Ceph metavariables are similar to Bash shell expansion.
-
-Ceph supports the following metavariables:
-
-
-``$cluster``
-
-:Description: Expands to the Ceph Storage Cluster name. Useful when running
- multiple Ceph Storage Clusters on the same hardware.
-
-:Example: ``/etc/ceph/$cluster.keyring``
-:Default: ``ceph``
-
-
-``$type``
-
-:Description: Expands to one of ``mds``, ``osd``, or ``mon``, depending on the
- type of the instant daemon.
-
-:Example: ``/var/lib/ceph/$type``
-
-
-``$id``
-
-:Description: Expands to the daemon identifier. For ``osd.0``, this would be
- ``0``; for ``mds.a``, it would be ``a``.
-
-:Example: ``/var/lib/ceph/$type/$cluster-$id``
-
-
-``$host``
-
-:Description: Expands to the host name of the instant daemon.
-
-
-``$name``
-
-:Description: Expands to ``$type.$id``.
-:Example: ``/var/run/ceph/$cluster-$name.asok``
-
-``$pid``
-
-:Description: Expands to daemon pid.
-:Example: ``/var/run/ceph/$cluster-$name-$pid.asok``
-
-
-.. _ceph-conf-common-settings:
-
-Common Settings
-===============
-
-The `Hardware Recommendations`_ section provides some hardware guidelines for
-configuring a Ceph Storage Cluster. It is possible for a single :term:`Ceph
-Node` to run multiple daemons. For example, a single node with multiple drives
-may run one ``ceph-osd`` for each drive. Ideally, you will have a node for a
-particular type of process. For example, some nodes may run ``ceph-osd``
-daemons, other nodes may run ``ceph-mds`` daemons, and still other nodes may
-run ``ceph-mon`` daemons.
-
-Each node has a name identified by the ``host`` setting. Monitors also specify
-a network address and port (i.e., domain name or IP address) identified by the
-``addr`` setting. A basic configuration file will typically specify only
-minimal settings for each instance of monitor daemons. For example:
-
-.. code-block:: ini
-
- [global]
- mon_initial_members = ceph1
- mon_host = 10.0.0.1
-
-
-.. important:: The ``host`` setting is the short name of the node (i.e., not
- an fqdn). It is **NOT** an IP address either. Enter ``hostname -s`` on
- the command line to retrieve the name of the node. Do not use ``host``
- settings for anything other than initial monitors unless you are deploying
- Ceph manually. You **MUST NOT** specify ``host`` under individual daemons
- when using deployment tools like ``chef`` or ``ceph-deploy``, as those tools
- will enter the appropriate values for you in the cluster map.
-
-
-.. _ceph-network-config:
-
-Networks
-========
-
-See the `Network Configuration Reference`_ for a detailed discussion about
-configuring a network for use with Ceph.
-
-
-Monitors
-========
-
-Ceph production clusters typically deploy with a minimum 3 :term:`Ceph Monitor`
-daemons to ensure high availability should a monitor instance crash. At least
-three (3) monitors ensures that the Paxos algorithm can determine which version
-of the :term:`Ceph Cluster Map` is the most recent from a majority of Ceph
-Monitors in the quorum.
-
-.. note:: You may deploy Ceph with a single monitor, but if the instance fails,
- the lack of other monitors may interrupt data service availability.
-
-Ceph Monitors typically listen on port ``6789``. For example:
-
-.. code-block:: ini
-
- [mon.a]
- host = hostName
- mon addr = 150.140.130.120:6789
-
-By default, Ceph expects that you will store a monitor's data under the
-following path::
-
- /var/lib/ceph/mon/$cluster-$id
-
-You or a deployment tool (e.g., ``ceph-deploy``) must create the corresponding
-directory. With metavariables fully expressed and a cluster named "ceph", the
-foregoing directory would evaluate to::
-
- /var/lib/ceph/mon/ceph-a
-
-For additional details, see the `Monitor Config Reference`_.
-
-.. _Monitor Config Reference: ../mon-config-ref
-
-
-.. _ceph-osd-config:
-
-
-Authentication
-==============
-
-.. versionadded:: Bobtail 0.56
-
-For Bobtail (v 0.56) and beyond, you should expressly enable or disable
-authentication in the ``[global]`` section of your Ceph configuration file. ::
-
- auth cluster required = cephx
- auth service required = cephx
- auth client required = cephx
-
-Additionally, you should enable message signing. See `Cephx Config Reference`_ for details.
-
-.. important:: When upgrading, we recommend expressly disabling authentication
- first, then perform the upgrade. Once the upgrade is complete, re-enable
- authentication.
-
-.. _Cephx Config Reference: ../auth-config-ref
-
-
-.. _ceph-monitor-config:
-
-
-OSDs
-====
-
-Ceph production clusters typically deploy :term:`Ceph OSD Daemons` where one node
-has one OSD daemon running a filestore on one storage drive. A typical
-deployment specifies a journal size. For example:
-
-.. code-block:: ini
-
- [osd]
- osd journal size = 10000
-
- [osd.0]
- host = {hostname} #manual deployments only.
-
-
-By default, Ceph expects that you will store a Ceph OSD Daemon's data with the
-following path::
-
- /var/lib/ceph/osd/$cluster-$id
-
-You or a deployment tool (e.g., ``ceph-deploy``) must create the corresponding
-directory. With metavariables fully expressed and a cluster named "ceph", the
-foregoing directory would evaluate to::
-
- /var/lib/ceph/osd/ceph-0
-
-You may override this path using the ``osd data`` setting. We don't recommend
-changing the default location. Create the default directory on your OSD host.
-
-::
-
- ssh {osd-host}
- sudo mkdir /var/lib/ceph/osd/ceph-{osd-number}
-
-The ``osd data`` path ideally leads to a mount point with a hard disk that is
-separate from the hard disk storing and running the operating system and
-daemons. If the OSD is for a disk other than the OS disk, prepare it for
-use with Ceph, and mount it to the directory you just created::
-
- ssh {new-osd-host}
- sudo mkfs -t {fstype} /dev/{disk}
- sudo mount -o user_xattr /dev/{hdd} /var/lib/ceph/osd/ceph-{osd-number}
-
-We recommend using the ``xfs`` file system when running
-:command:`mkfs`. (``btrfs`` and ``ext4`` are not recommended and no
-longer tested.)
-
-See the `OSD Config Reference`_ for additional configuration details.
-
-
-Heartbeats
-==========
-
-During runtime operations, Ceph OSD Daemons check up on other Ceph OSD Daemons
-and report their findings to the Ceph Monitor. You do not have to provide any
-settings. However, if you have network latency issues, you may wish to modify
-the settings.
-
-See `Configuring Monitor/OSD Interaction`_ for additional details.
-
-
-.. _ceph-logging-and-debugging:
-
-Logs / Debugging
-================
-
-Sometimes you may encounter issues with Ceph that require
-modifying logging output and using Ceph's debugging. See `Debugging and
-Logging`_ for details on log rotation.
-
-.. _Debugging and Logging: ../../troubleshooting/log-and-debug
-
-
-Example ceph.conf
-=================
-
-.. literalinclude:: demo-ceph.conf
- :language: ini
-
-.. _ceph-runtime-config:
-
-Runtime Changes
-===============
-
-Ceph allows you to make changes to the configuration of a ``ceph-osd``,
-``ceph-mon``, or ``ceph-mds`` daemon at runtime. This capability is quite
-useful for increasing/decreasing logging output, enabling/disabling debug
-settings, and even for runtime optimization. The following reflects runtime
-configuration usage::
-
- ceph tell {daemon-type}.{id or *} injectargs --{name} {value} [--{name} {value}]
-
-Replace ``{daemon-type}`` with one of ``osd``, ``mon`` or ``mds``. You may apply
-the runtime setting to all daemons of a particular type with ``*``, or specify
-a specific daemon's ID (i.e., its number or letter). For example, to increase
-debug logging for a ``ceph-osd`` daemon named ``osd.0``, execute the following::
-
- ceph tell osd.0 injectargs --debug-osd 20 --debug-ms 1
-
-In your ``ceph.conf`` file, you may use spaces when specifying a
-setting name. When specifying a setting name on the command line,
-ensure that you use an underscore or hyphen (``_`` or ``-``) between
-terms (e.g., ``debug osd`` becomes ``--debug-osd``).
-
-
-Viewing a Configuration at Runtime
-==================================
-
-If your Ceph Storage Cluster is running, and you would like to see the
-configuration settings from a running daemon, execute the following::
-
- ceph daemon {daemon-type}.{id} config show | less
-
-If you are on a machine where osd.0 is running, the command would be::
-
- ceph daemon osd.0 config show | less
-
-Reading Configuration Metadata at Runtime
-=========================================
-
-Information about the available configuration options is available via
-the ``config help`` command:
-
-::
-
- ceph daemon {daemon-type}.{id} config help | less
-
-
-This metadata is primarily intended to be used when integrating other
-software with Ceph, such as graphical user interfaces. The output is
-a list of JSON objects, for example:
-
-::
-
- {
- "name": "mon_host",
- "type": "std::string",
- "level": "basic",
- "desc": "list of hosts or addresses to search for a monitor",
- "long_desc": "This is a comma, whitespace, or semicolon separated list of IP addresses or hostnames. Hostnames are resolved via DNS and all A or AAAA records are included in the search list.",
- "default": "",
- "daemon_default": "",
- "tags": [],
- "services": [
- "common"
- ],
- "see_also": [],
- "enum_values": [],
- "min": "",
- "max": ""
- }
-
-type
-____
-
-The type of the setting, given as a C++ type name.
-
-level
-_____
-
-One of `basic`, `advanced`, `dev`. The `dev` options are not intended
-for use outside of development and testing.
-
-desc
-____
-
-A short description -- this is a sentence fragment suitable for display
-in small spaces like a single line in a list.
-
-long_desc
-_________
-
-A full description of what the setting does, this may be as long as needed.
-
-default
-_______
-
-The default value, if any.
-
-daemon_default
-______________
-
-An alternative default used for daemons (services) as opposed to clients.
-
-tags
-____
-
-A list of strings indicating topics to which this setting relates. Examples
-of tags are `performance` and `networking`.
-
-services
-________
-
-A list of strings indicating which Ceph services the setting relates to, such
-as `osd`, `mds`, `mon`. For settings that are relevant to any Ceph client
-or server, `common` is used.
-
-see_also
-________
-
-A list of strings indicating other configuration options that may also
-be of interest to a user setting this option.
-
-enum_values
-___________
-
-Optional: a list of strings indicating the valid settings.
-
-min, max
-________
-
-Optional: upper and lower (inclusive) bounds on valid settings.
-
-
-
-
-Running Multiple Clusters
-=========================
-
-With Ceph, you can run multiple Ceph Storage Clusters on the same hardware.
-Running multiple clusters provides a higher level of isolation compared to
-using different pools on the same cluster with different CRUSH rulesets. A
-separate cluster will have separate monitor, OSD and metadata server processes.
-When running Ceph with default settings, the default cluster name is ``ceph``,
-which means you would save your Ceph configuration file with the file name
-``ceph.conf`` in the ``/etc/ceph`` default directory.
-
-See `ceph-deploy new`_ for details.
-.. _ceph-deploy new:../ceph-deploy-new
-
-When you run multiple clusters, you must name your cluster and save the Ceph
-configuration file with the name of the cluster. For example, a cluster named
-``openstack`` will have a Ceph configuration file with the file name
-``openstack.conf`` in the ``/etc/ceph`` default directory.
-
-.. important:: Cluster names must consist of letters a-z and digits 0-9 only.
-
-Separate clusters imply separate data disks and journals, which are not shared
-between clusters. Referring to `Metavariables`_, the ``$cluster`` metavariable
-evaluates to the cluster name (i.e., ``openstack`` in the foregoing example).
-Various settings use the ``$cluster`` metavariable, including:
-
-- ``keyring``
-- ``admin socket``
-- ``log file``
-- ``pid file``
-- ``mon data``
-- ``mon cluster log file``
-- ``osd data``
-- ``osd journal``
-- ``mds data``
-- ``rgw data``
-
-See `General Settings`_, `OSD Settings`_, `Monitor Settings`_, `MDS Settings`_,
-`RGW Settings`_ and `Log Settings`_ for relevant path defaults that use the
-``$cluster`` metavariable.
-
-.. _General Settings: ../general-config-ref
-.. _OSD Settings: ../osd-config-ref
-.. _Monitor Settings: ../mon-config-ref
-.. _MDS Settings: ../../../cephfs/mds-config-ref
-.. _RGW Settings: ../../../radosgw/config-ref/
-.. _Log Settings: ../../troubleshooting/log-and-debug
-
-
-When creating default directories or files, you should use the cluster
-name at the appropriate places in the path. For example::
-
- sudo mkdir /var/lib/ceph/osd/openstack-0
- sudo mkdir /var/lib/ceph/mon/openstack-a
-
-.. important:: When running monitors on the same host, you should use
- different ports. By default, monitors use port 6789. If you already
- have monitors using port 6789, use a different port for your other cluster(s).
-
-To invoke a cluster other than the default ``ceph`` cluster, use the
-``-c {filename}.conf`` option with the ``ceph`` command. For example::
-
- ceph -c {cluster-name}.conf health
- ceph -c openstack.conf health
-
-
-.. _Hardware Recommendations: ../../../start/hardware-recommendations
-.. _Network Configuration Reference: ../network-config-ref
-.. _OSD Config Reference: ../osd-config-ref
-.. _Configuring Monitor/OSD Interaction: ../mon-osd-interaction
-.. _ceph-deploy new: ../../deployment/ceph-deploy-new#naming-a-cluster