initial code repo

This patch creates initial code repo.

For ceph, luminous stable release will be used for base code,
and next changes and optimization for ceph will be added to it.

For opensds, currently any changes can be upstreamed into original
opensds repo (https://github.com/opensds/opensds), and so stor4nfv
will directly clone opensds code to deploy stor4nfv environment.
And the scripts for deployment based on ceph and opensds will be
put into 'ci' directory.

Change-Id: I46a32218884c75dda2936337604ff03c554648e4
Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com>

4 files changed, 242 insertions, 0 deletions

diff --git a/src/ceph/doc/dev/ceph-volume/index.rst b/src/ceph/doc/dev/ceph-volume/index.rst
new file mode 100644
index 0000000..b6f9dc0
--- /dev/null
+++ b/src/ceph/doc/dev/ceph-volume/index.rst
@@ -0,0 +1,13 @@
+===================================
+ceph-volume developer documentation
+===================================
+
+.. rubric:: Contents
+
+.. toctree::
+   :maxdepth: 1
+
+
+   plugins
+   lvm
+   systemd
diff --git a/src/ceph/doc/dev/ceph-volume/lvm.rst b/src/ceph/doc/dev/ceph-volume/lvm.rst
new file mode 100644
index 0000000..f89424a
--- /dev/null
+++ b/src/ceph/doc/dev/ceph-volume/lvm.rst
@@ -0,0 +1,127 @@
+
+.. _ceph-volume-lvm-api:
+
+LVM
+===
+The backend of ``ceph-volume lvm`` is LVM, it relies heavily on the usage of
+tags, which is a way for LVM to allow extending its volume metadata. These
+values can later be queried against devices and it is how they get discovered
+later.
+
+.. warning:: These APIs are not meant to be public, but are documented so that
+             it is clear what the tool is doing behind the scenes. Do not alter
+             any of these values.
+
+
+.. _ceph-volume-lvm-tag-api:
+
+Tag API
+-------
+The process of identifying logical volumes as part of Ceph relies on applying
+tags on all volumes. It follows a naming convention for the namespace that
+looks like::
+
+    ceph.<tag name>=<tag value>
+
+All tags are prefixed by the ``ceph`` keyword do claim ownership of that
+namespace and make it easily identifiable. This is how the OSD ID would be used
+in the context of lvm tags::
+
+    ceph.osd_id=0
+
+
+.. _ceph-volume-lvm-tags:
+
+Metadata
+--------
+The following describes all the metadata from Ceph OSDs that is stored on an
+LVM volume:
+
+
+``type``
+--------
+Describes if the device is a an OSD or Journal, with the ability to expand to
+other types when supported (for example a lockbox)
+
+Example::
+
+    ceph.type=osd
+
+
+``cluster_fsid``
+----------------
+Example::
+
+    ceph.cluster_fsid=7146B649-AE00-4157-9F5D-1DBFF1D52C26
+
+``data_device``
+---------------
+Example::
+
+    ceph.data_device=/dev/ceph/data-0
+
+``journal_device``
+------------------
+Example::
+
+    ceph.journal_device=/dev/ceph/journal-0
+
+``encrypted``
+-------------
+Example for enabled encryption with ``luks``::
+
+    ceph.encrypted=luks
+
+For plain dmcrypt::
+
+    ceph.encrypted=dmcrypt
+
+For disabled encryption::
+
+    ceph.encrypted=0
+
+``osd_fsid``
+------------
+Example::
+
+    ceph.osd_fsid=88ab9018-f84b-4d62-90b4-ce7c076728ff
+
+``osd_id``
+----------
+Example::
+
+    ceph.osd_id=1
+
+``block``
+---------
+Just used on :term:`bluestore` backends.
+
+Example::
+
+    ceph.block=/dev/mapper/vg-block-0
+
+``db``
+------
+Just used on :term:`bluestore` backends.
+
+Example::
+
+    ceph.db=/dev/mapper/vg-db-0
+
+``wal``
+-------
+Just used on :term:`bluestore` backends.
+
+Example::
+
+    ceph.wal=/dev/mapper/vg-wal-0
+
+
+``lockbox_device``
+------------------
+Only used when encryption is enabled, to store keys in an unencrypted
+volume.
+
+Example::
+
+    ceph.lockbox_device=/dev/mapper/vg-lockbox-0
diff --git a/src/ceph/doc/dev/ceph-volume/plugins.rst b/src/ceph/doc/dev/ceph-volume/plugins.rst
new file mode 100644
index 0000000..95bc761
--- /dev/null
+++ b/src/ceph/doc/dev/ceph-volume/plugins.rst
@@ -0,0 +1,65 @@
+.. _ceph-volume-plugins:
+
+Plugins
+=======
+``ceph-volume`` started initially to provide support for using ``lvm`` as
+the underlying system for an OSD. It is included as part of the tool but it is
+treated like a plugin.
+
+This modularity, allows for other device or device-like technologies to be able
+to consume and re-use the utilities and workflows provided.
+
+Adding Plugins
+--------------
+As a Python tool, plugins ``setuptools`` entry points. For a new plugin to be
+available, it should have an entry similar to this in its ``setup.py`` file:
+
+.. code-block:: python
+
+    setup(
+        ...
+        entry_points = dict(
+            ceph_volume_handlers = [
+                'my_command = my_package.my_module:MyClass',
+            ],
+        ),
+
+The ``MyClass`` should be a class that accepts ``sys.argv`` as its argument,
+``ceph-volume`` will pass that in at instantiation and call them ``main``
+method.
+
+This is how a plugin for ``ZFS`` could look like for example:
+
+.. code-block:: python
+
+    class ZFS(object):
+
+        help_menu = 'Deploy OSDs with ZFS'
+        _help = """
+    Use ZFS as the underlying technology for OSDs
+
+    --verbose   Increase the verbosity level
+        """
+
+        def __init__(self, argv):
+            self.argv = argv
+
+        def main(self):
+            parser = argparse.ArgumentParser()
+            args = parser.parse_args(self.argv)
+            ...
+
+And its entry point (via ``setuptools``) in ``setup.py`` would looke like:
+
+.. code-block:: python
+
+        entry_points = {
+            'ceph_volume_handlers': [
+                'zfs = ceph_volume_zfs.zfs:ZFS',
+            ],
+        },
+
+After installation, the ``zfs`` subcommand would be listed and could be used
+as::
+
+    ceph-volume zfs
diff --git a/src/ceph/doc/dev/ceph-volume/systemd.rst b/src/ceph/doc/dev/ceph-volume/systemd.rst
new file mode 100644
index 0000000..8553430
--- /dev/null
+++ b/src/ceph/doc/dev/ceph-volume/systemd.rst
@@ -0,0 +1,37 @@
+.. _ceph-volume-systemd-api:
+
+systemd
+=======
+The workflow to *"activate"* an OSD is by relying on systemd unit files and its
+ability to persist information as a suffix to the instance name.
+
+``ceph-volume`` exposes the following convention for unit files::
+
+    ceph-volume@<sub command>-<extra metadata>
+
+For example, this is how enabling an OSD could look like for the 
+:ref:`ceph-volume-lvm` sub command::
+
+    systemctl enable ceph-volume@lvm-0-8715BEB4-15C5-49DE-BA6F-401086EC7B41
+
+
+These 3 pieces of persisted information are needed by the sub-command so that
+it understands what OSD it needs to activate.
+
+Since ``lvm`` is not the only subcommand that will be supported, this
+is how it will allow other device types to be defined.
+
+At some point for example, for plain disks, it could be::
+
+    systemctl enable ceph-volume@disk-0-8715BEB4-15C5-49DE-BA6F-401086EC7B41
+
+At startup, the systemd unit will execute a helper script that will parse the
+suffix and will end up calling ``ceph-volume`` back. Using the previous
+example for lvm, that call will look like::
+
+    ceph-volume lvm activate 0 8715BEB4-15C5-49DE-BA6F-401086EC7B41
+
+
+.. warning:: These workflows are not meant to be public, but are documented so that
+             it is clear what the tool is doing behind the scenes. Do not alter
+             any of these values.