Add documentation for dovetail tool

JIRA:DOVETAIL-28

Change-Id: I2478655d951a65a11f39e9ceba10969e56274cc3
Signed-off-by: Leo Wang <grakiss.wanglei@huawei.com>

3 files changed, 137 insertions, 2 deletions

diff --git a/docs/userguide/01-introduction.rst b/docs/userguide/01-introduction.rst
index f3d57be8..dbe33d5d 100644
--- a/docs/userguide/01-introduction.rst
+++ b/docs/userguide/01-introduction.rst
@@ -2,7 +2,7 @@
 Dovetail Introduction
 ======================
 
-**Welcome to Dovetail' documentation !**
+**Welcome to Dovetail's documentation !**
 
 .. _Dovetail: https://wiki.opnfv.org/dovetail
 
@@ -11,7 +11,7 @@ Dovetail_ is an OPNFV Project.
 This project intends to define and provide a set of OPNFV related validation
 criteria that will provide input for the evaluation of the use of OPNFV trademarks.
 The dovetail project will be executed with the guidance and oversight of the Compliance
-and Certification committee and work to secure the goals of the C&C committee for each release
+and Certification committee and work to secure the goals of the C&C committee for each release.
 
 This project intends to incrementally define qualification criteria
 that establish the foundations of how we are able to measure the ability
diff --git a/docs/userguide/04-dovetail_config_template.rst b/docs/userguide/04-dovetail_config_template.rst
new file mode 100644
index 00000000..fce99d21
--- /dev/null
+++ b/docs/userguide/04-dovetail_config_template.rst
@@ -0,0 +1,134 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+Config Template Syntax
+======================
+
+Dovetail uses Functest/Yardstick Docker container to run its testcases. So you need to give
+configurations for building the container and the commands it needs to do. In dovetail,
+all of these are defined in config yaml files ``dovetail/dovetail/conf/functest_config.yml``
+and ``dovetail/dovetail/conf/yardstick_config.yml``.
+
+Functest template syntax
+------------------------
+
+For example, you can define your ``functest_config.yml`` as:
+
+::
+
+  functest:
+    image_name: opnfv/functest
+    docker_tag: latest
+    envs: '-e INSTALLER_TYPE=compass -e INSTALLER_IP=192.168.200.2
+           -e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn
+           -e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal'
+    opts: '-id --privileged=true'
+    pre_condition:
+      cmds:
+        - 'echo test for precondition'
+    testcase:
+      cmds:
+        - 'python /home/opnfv/repos/functest/ci/prepare_env.py start'
+        - 'python /home/opnfv/repos/functest/ci/run_tests.py -t {{script_testcase}} -r'
+    post_condition:
+      cmds:
+        - ''
+    result:
+      dir: '/home/opnfv/functest/results'
+      store_type: 'file'
+      file_path: 'tempest/tempest.log'
+      db_url: 'http://testresults.opnfv.org/test/api/v1/results?case=%s&last=1'
+
+First, you need to give the image that you want to use for building functest/yardstick container.
+Besides, there also need some envirnment parameters such as ``INSTALLER_TYPE`` and ``INSTALLER_IP``
+and the options for you container. Then the functest/yardstick container can be build with your
+settings.
+
+Second, there need three kinds of commands, ``pre_condition``, ``testcase`` and ``post_condition``.
+If you want to do some cleanups or preparations, the commands can be put into ``pre_condition``
+section orderly. All commands in this section will just be executed once in the begining.
+The ``testcase`` section does the main jobs of the testing. All functest testcases will use the
+container to execute these commands one by one. After finishing that, the test is accomplished
+and the results are stored in files or uploaded to database. The ``post_condition`` section
+does some work such as clean Docker images or something else after all testcases finished.
+All commands in this section will just execute once.
+
+Besides, there need a ``result`` section and it gives the directory of the functest/yardstick
+results. The ``store_type`` should be the same with the cmds in ``testcase``. That means if the
+test results are stored in files, then store_type need to be file and the file_path is also
+needed. If the test results are uploaded to database, then a db_url is needed for acquiring the results.
+
+Yardstick template syntax
+-------------------------
+
+The framework of ``yardstick_config.yml`` is almost the same as ``functest_config.yml``.
+
+::
+
+  yardstick:
+    image_name: opnfv/yardstick
+    docker_tag: latest
+    envs: '-e INSTALLER_TYPE=compass -e INSTALLER_IP=192.168.200.2
+           -e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn
+           -e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal
+           -e EXTERNAL_NETWORK=ext-net'
+    opts: '-id --privileged=true'
+    pre_condition:
+      cmds:
+        - 'source /home/opnfv/repos/yardstick/tests/ci/prepare_env.sh &&
+           source /home/opnfv/repos/yardstick/tests/ci/clean_images.sh && cleanup'
+        - 'source /home/opnfv/repos/yardstick/tests/ci/prepare_env.sh &&
+           cd /home/opnfv/repos/yardstick && source tests/ci/load_images.sh'
+    testcase:
+      cmds:
+        - 'mkdir -p /home/opnfv/yardstick/results/'
+        - 'cd /home/opnfv/repos/yardstick && source tests/ci/prepare_env.sh &&
+           yardstick task start tests/opnfv/test_cases/{{script_testcase}}.yaml
+           --output-file /home/opnfv/yardstick/results/{{script_testcase}}.out &>
+           /home/opnfv/yardstick/results/yardstick.log'
+    post_condition:
+      cmds:
+        - ''
+    result:
+      dir: '/home/opnfv/yardstick/results'
+      store_type: 'file'
+      file_path: 'yardstick.log'
+      db_url: 'http://testresults.opnfv.org/test/api/v1/results?case=%s&last=1'
+
+The main differences between ``yardstick_config.yml`` and ``functest_config.yml`` are the commands.
+
+Jinja2 template syntax
+----------------------
+
+Note that you can use jinja2 template for your parameters such as ``{{script_testcase}}``. The
+parameters are defined in ``dovetail/dovetail/conf/dovetail_config.yml``:
+
+::
+
+  parameters:
+  - name: testcase
+    path: '("name",)'
+  - name: script_testcase
+    path: '("scripts", "testcase")'
+
+Here ``path`` is the path in testcase config files that you can find the value of parameters. Take
+``script_testcase`` as the example. For testcase dovetail.ipv6.tc001:
+
+::
+
+  dovetail.ipv6.tc001:
+    name: dovetail.ipv6.tc001
+    objective: VIM ipv6 operations, to create/delete network, port and subnet in bulk operation
+    scripts:
+      type: functest
+      testcase: tempest_smoke_serial
+      sub_testcase_list:
+        - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network
+        - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port
+        - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet
+
+The path ("scripts", "testcase") means 'testcase' subsection of 'scripts' section. So follow
+the path ("scripts", "testcase") we can get the value of ``{{script_testcase}}`` that is
+'tempest_smoke_serial'.
diff --git a/docs/userguide/index.rst b/docs/userguide/index.rst
index 03246e7b..0dcf55ce 100644
--- a/docs/userguide/index.rst
+++ b/docs/userguide/index.rst
@@ -14,3 +14,4 @@ Dovetail - user guide
    ./01-introduction.rst
    ./02-certification_criteria.rst
    ./03-certification_progress.rst
+   ./04-dovetail_config_template.rst