dovetail tool: developer userguide document

JIRA: DOVETAIL-28

1) documents are reorganized, all dovetail tool related documents are moved to /docs/dovetailtool
2) some sections are rewritten to make it more clear

Change-Id: Ib0c23b8c1771eb6de7d6cc264c101e8e26ed3d1f
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>

5 files changed, 357 insertions, 0 deletions

diff --git a/docs/dovetailtool/dovetail.tool.configtemplate.rst b/docs/dovetailtool/dovetail.tool.configtemplate.rst
new file mode 100644
index 00000000..9c0748a9
--- /dev/null
+++ b/docs/dovetailtool/dovetail.tool.configtemplate.rst
@@ -0,0 +1,125 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+======================
+Config Template Syntax
+======================
+
+For Dovetail tool, the config files are located in ``dovetail/dovetail/conf``, which are written
+in yaml format. As both functest and yardstick are utilized by Dovetail, their configuration files
+should be configured as follows, within the ``functest_config.yml`` and ``yardstick_config.yml`` files,
+respectively.
+
+Functest config template syntax
+-------------------------------
+
+An example functest configuration is shown as follows:
+
+::
+
+  functest:
+    image_name: opnfv/functest
+    docker_tag: latest
+    envs: '-e INSTALLER_TYPE=compass -e INSTALLER_IP=192.168.200.2
+           -e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn
+           -e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal'
+    opts: '-id --privileged=true'
+    pre_condition:
+      cmds:
+        - 'echo test for precondition'
+    testcase:
+      cmds:
+        - 'python /home/opnfv/repos/functest/ci/prepare_env.py start'
+        - 'python /home/opnfv/repos/functest/ci/run_tests.py -t {{script_testcase}} -r'
+    post_condition:
+      cmds:
+        - ''
+    result:
+      dir: '/home/opnfv/functest/results'
+      store_type: 'file'
+      file_path: 'tempest/tempest.log'
+      db_url: 'http://testresults.opnfv.org/test/api/v1/results?case=%s&last=1'
+
+- ``image_name`` and ``docker_tag`` sections define the docker image pulled from the public dockerhub.
+- ``envs`` should be correctly configed according to the SUT(System Under Test).
+- ``pre_condition`` represents some cleanups or preparations.
+  ``testcase`` represents the testcases running cmds.
+  ``post_condition`` represents some cleanups needed after all testcases finished.
+- ``result`` section gives the directory of the dovetail tool test result.
+  ``db_url`` gives the database URL of the dovetail results to be stored.
+
+Yardstick config template syntax
+---------------------------------
+
+The configuration in ``yardstick_config.yml`` is similiar to ``functest_config.yml``,
+and an example is shown as follows:
+
+::
+
+  yardstick:
+    image_name: opnfv/yardstick
+    docker_tag: latest
+    envs: '-e INSTALLER_TYPE=compass -e INSTALLER_IP=192.168.200.2
+           -e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn
+           -e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal
+           -e EXTERNAL_NETWORK=ext-net'
+    opts: '-id --privileged=true'
+    pre_condition:
+      cmds:
+        - 'source /home/opnfv/repos/yardstick/tests/ci/prepare_env.sh &&
+           source /home/opnfv/repos/yardstick/tests/ci/clean_images.sh && cleanup'
+        - 'source /home/opnfv/repos/yardstick/tests/ci/prepare_env.sh &&
+           cd /home/opnfv/repos/yardstick && source tests/ci/load_images.sh'
+    testcase:
+      cmds:
+        - 'mkdir -p /home/opnfv/yardstick/results/'
+        - 'cd /home/opnfv/repos/yardstick && source tests/ci/prepare_env.sh &&
+           yardstick task start tests/opnfv/test_cases/{{script_testcase}}.yaml
+           --output-file /home/opnfv/yardstick/results/{{script_testcase}}.out &>
+           /home/opnfv/yardstick/results/yardstick.log'
+    post_condition:
+      cmds:
+        - ''
+    result:
+      dir: '/home/opnfv/yardstick/results'
+      store_type: 'file'
+      file_path: 'yardstick.log'
+      db_url: 'http://testresults.opnfv.org/test/api/v1/results?case=%s&last=1'
+
+The main differences between ``yardstick_config.yml`` and ``functest_config.yml``
+are the ``cmds`` subsection.
+
+Jinja2 template syntax
+----------------------
+
+Jinja2 module can be used to config the ``{{script_testcase}}``. The
+parameters are defined in ``dovetail/dovetail/conf/dovetail_config.yml``:
+
+::
+
+  parameters:
+  - name: testcase
+    path: '("name",)'
+  - name: script_testcase
+    path: '("scripts", "testcase")'
+
+Here ``path`` is the path defined in the testcase configuration files.
+Take ``script_testcase`` as an example. For testcase ``dovetail.ipv6.tc001``:
+
+::
+
+  dovetail.ipv6.tc001:
+    name: dovetail.ipv6.tc001
+    objective: VIM ipv6 operations, to create/delete network, port and subnet in bulk operation
+    scripts:
+      type: functest
+      testcase: tempest_smoke_serial
+      sub_testcase_list:
+        - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network
+        - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port
+        - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet
+
+The path ("scripts", "testcase") means 'testcase' is the subsection of 'scripts' section. From above,
+by following the path ("scripts", "testcase") we can get the value of ``{{script_testcase}}`` is 'tempest_smoke_serial'.
diff --git a/docs/dovetailtool/dovetail.tool.configuration.rst b/docs/dovetailtool/dovetail.tool.configuration.rst
new file mode 100644
index 00000000..8e97e73c
--- /dev/null
+++ b/docs/dovetailtool/dovetail.tool.configuration.rst
@@ -0,0 +1,35 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+=========================
+Testcase Template Syntax
+=========================
+
+The testcases used for compliance and certification are defined in the ``dovetail/testcase`` directory,
+which are defined in yaml format. Take the testcase ``ipv6.tc001.yml`` as an example, it is shown as:
+
+::
+
+  dovetail.ipv6.tc001:
+    name: dovetail.ipv6.tc001
+    objective: VIM ipv6 operations, to create/delete network, port and subnet in bulk operation
+    scripts:
+      type: functest
+      testcase: tempest_smoke_serial
+      sub_testcase_list:
+        - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network
+        - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port
+        - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet
+
+- At least three sections named 'name', 'objective', 'scripts' must be included
+- Section 'name' distinguishes different test cases used for compliance and certification,
+  and should start with ``dovetail.``
+- Section 'objective' describes what this testcase does
+- Section 'scripts' has subsections such as 'type', 'testcase' and 'sub_testcase_list'
+- Two kinds of 'type' is supported by now, functest and yardstick
+- For functest, the 'testcase' represents the testcases in slicing/tier,
+  'sub_testcase_list' represents the testcases in this slicing compliance and certification will use.
+  For yardstick, since it is not sliced by now, 'sub_testcase_list' is not needed, only to edit the 'testcase' part
+  such as ``yardstick_tc027``
diff --git a/docs/dovetailtool/dovetail.tool.installation.rst b/docs/dovetailtool/dovetail.tool.installation.rst
new file mode 100644
index 00000000..dcd856bc
--- /dev/null
+++ b/docs/dovetailtool/dovetail.tool.installation.rst
@@ -0,0 +1,154 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+===========================
+Dovetail Tool Installation
+===========================
+
+Abstract
+########
+
+The Dovetail tool supports installation either directly on the Linux host, or within a Docker container.
+The detailed installation procedure on the Linux host or via the Docker container are shown
+in the following sections.
+
+There is a need to config the following parameters before Dovetail tool
+running on the SUT (System Under Test):
+
+::
+
+  SUT_TYPE, SUT type, e.g., apex, compass, fuel, joid, etc
+  SUT_IP, SUT external network IP, e.g., 192.168.200.2
+  NODE_NAME, this can be shown in the test result for users to see which pod the dovetail tool runs
+  DEPLOY_SCENARIO, deployment scenario, e.g., os-nosdn-nofeature-ha
+  BUILD_TAG, this can be shown in the test result for users to identify logs
+  CI_DEBUG, true for debug information printed and false for not printed
+  DEPLOY_TYPE, baremetal or virtual
+
+Dovetail tool installation on local Linux host environment
+##########################################################
+
+In order to install Dovetail into a local Linux host environment, the following steps should
+be performed:
+
+Downloading Dovetail tool
+--------------------------
+
+Source code downloading:
+
+::
+
+  git clone https://gerrit.opnfv.org/gerrit/dovetail
+
+Environment preparation
+-----------------------
+
+Dovetail configuration:
+
+::
+
+  cd {dovetail_path}/dovetail/conf
+  vim functest_config.yml
+  vim yardstick_config.yml
+
+To modify the ``envs`` parameters, e.g., SUT_TYPE, SUT_IP, NODE_NAME,
+DEPLOY_SCENARIO, BUILD_TAG, CI_DEBUG, DEPLOY_TYPE, in the yml files.
+
+System dependencies are installed via the ``prepare_env.py`` file, and you will need ``sudo``
+access to complete the installation.
+
+::
+
+  cd {dovetail_path}/dovetail/dovetail
+  python prepare_env.py
+
+At this point the environment is now ready for Dovetail execution.
+
+Compliance and certification test cases
+----------------------------------------
+
+The compliance and certification test cases can be defined under the ``/dovetail/cert``
+directory, which is defined in yaml format.
+A sample file named ``basic.yml`` is provided as follows:
+
+::
+
+  certification_basic:
+    name: certification_basic
+    testcase_list:
+      - dovetail.ipv6.tc001
+
+The testcase listed here is dovetail.ipv6.tc001, defined within ``dovetail/testcase``.
+
+Note: if a new test case yaml file is created, its name should start with ``certification_``,
+in similiar fashion as the sample file ``certification_basic``.
+
+Running Dovetail tool
+---------------------
+
+After environment preparation is complete and test cases added, the Dovetail tool can be run with
+
+::
+
+  python run.py --scenario basic
+
+The value ``basic`` passed to the ``scenario`` flag can be replaced with the test cases yaml file.
+If not argument is given, the basic scenario will be run as the default.
+
+Running Dovetail in a Docker container
+########################################
+
+The Dovetail tool can be run in a Docker container by utilizing the following steps:
+
+Pull Dovetail Docker image from public Dockerhub
+------------------------------------------------
+
+::
+
+  sudo docker pull opnfv/dovetail:<Tag>
+
+<Tag> here is the version, 'latest' is used for the master branch.
+
+Dovetail Docker container creation
+----------------------------------
+
+Next, create the ``dovetail-docker-env`` file to define the environment parameters ::
+
+  INSTALLER_TYPE=compass
+  INSTALLER_IP=192.168.200.2
+  DEPLOY_TYPE=baremetal
+  DEPLOY_SCENARIO=ha-nosdn
+  CI_DEBUG=true
+
+Then to instantiate the Dovetail Docker container, execute::
+
+    sudo docker run --privileged=true --rm -t \
+         --env-file dovetail-docker-env \
+         -v /home/opnfv/dovetail/results:/home/opnfv/dovetail/results \
+         -v /var/run/docker.sock:/var/run/docker.sock \
+         --name <Dovetail_Container_Name> \
+         opnfv/dovetail:<Tag> /bin/bash
+
+To attach dovetail container and Running test cases
+----------------------------------------------------
+
+Before connecting to the container, you can check the container status by running ::
+
+   docker ps -a
+
+Attach to the container by starting it and obtaining a bash prompt with ::
+
+   docker exec -it <Dovetail_Container_Name> bash
+
+Inside the container the following commands can be executed to trigger the testcases ::
+
+   cd /home/opnfv/dovetail/dovetail
+   python run.py --scenario basic
+
+Results Output
+###############
+
+The running log is stored in ``/home/opnfv/dovetail/results/dovetail.log``.
+The certification report is stored in ``/home/opnfv/dovetail/results/dovetail_report.txt``.
diff --git a/docs/dovetailtool/dovetail.tool.overview.rst b/docs/dovetailtool/dovetail.tool.overview.rst
new file mode 100644
index 00000000..a1133f34
--- /dev/null
+++ b/docs/dovetailtool/dovetail.tool.overview.rst
@@ -0,0 +1,27 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+==============
+Dovetail Tool
+==============
+
+What is Dovetail tool
+#####################
+
+A toolset for providing an evaluation of a deployed cloud environment for OPNFV certification.
+
+Overview
+########
+
+Dovetail tool intends on being THE source of tools to provide evaluation of the use of OPNFV trademarks.
+
+Dovetail tool provides users with a wrapper which leverage other test projects, such as Functest,
+Yardstick, etc, that helps to provide evaluation of the use of OPNFV trademarks.
+It does so by validating any NFV(Network Function Virtualization) implementation against the test cases in Dovetail.
+
+Get involved!
+#############
+- Wiki: https://wiki.opnfv.org/display/dovetail/Dovetail+Home
+- Git Repo: https://git.opnfv.org/dovetail
diff --git a/docs/dovetailtool/index.rst b/docs/dovetailtool/index.rst
new file mode 100644
index 00000000..a025cd50
--- /dev/null
+++ b/docs/dovetailtool/index.rst
@@ -0,0 +1,16 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+==================
+Dovetail Overview
+==================
+
+.. toctree::
+   :maxdepth: 2
+
+   dovetail.tool.overview.rst
+   dovetail.tool.installation.rst
+   dovetail.tool.configuation.rst
+   dovetail.tool.configtemplate.rst