diff options
Diffstat (limited to 'docs/testing/developer/testframework/dovetail.tool.template.rst')
| -rw-r--r-- | docs/testing/developer/testframework/dovetail.tool.template.rst | 174 |
1 files changed, 174 insertions, 0 deletions
diff --git a/docs/testing/developer/testframework/dovetail.tool.template.rst b/docs/testing/developer/testframework/dovetail.tool.template.rst new file mode 100644 index 00000000..1a483dd9 --- /dev/null +++ b/docs/testing/developer/testframework/dovetail.tool.template.rst @@ -0,0 +1,174 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Technologies Co.,Ltd and others. + +================== +Template Syntax +================== + +Testcase Template Syntax +========================= + +The testcases used for compliance and certification are defined in the +``dovetail/testcase`` directory, which are written in yaml format. +Take the testcase ``ipv6.tc001.yml`` as an example. It is shown as: + +:: + + --- + dovetail.ipv6.tc001: + name: dovetail.ipv6.tc001 + objective: Bulk creation and deletion of IPv6 networks, ports and subnets + validate: + type: functest + testcase: tempest_smoke_serial + pre_condition: + - 'echo test for precondition in testcase' + cmds: + - 'functest env prepare' + - 'functest testcase run {{validate_testcase}}' + post_condition: + - 'echo test for precondition in testcase' + report: + sub_testcase_list: + - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network + - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port + - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet + + +- At least three sections named ``name``, ``objective``, ``validate``, + ``report`` must be included +- Section ``name`` distinguishes different test cases used for compliance, + it is composed of 3 parts, ``dovetail.``, belongs to which test area, + and the serial number +- Section ``objective`` briefly describes what this testcase does +- Section ``validate`` defines the scripts and configurations for the + validation of the test case. ``type`` defines which method is used to validate, + 3 ways, i.e., functest, yardstick and shell is supported currently. + ``testcase`` represents the testcases in slicing/tier. +- Section ``report`` defines the sub_testcases to be run. + For ``yardstick``, since it is not sliced by now, + ``sub_testcase_list`` is not needed, only to edit the ``testcase`` part in + section ``validate``, such as ``yardstick_tc027`` + + +Config Template Syntax +======================= + +For Dovetail tool, the config files are located in ``dovetail/dovetail/conf``, which are written +in yaml format. As both functest and yardstick are utilized by Dovetail, their configuration files +should be configured as follows, within the ``functest_config.yml`` and ``yardstick_config.yml`` files, +respectively. + +Functest config template syntax +------------------------------- + +An example of ``functest`` configuration is shown as follows: + +:: + + functest: + image_name: opnfv/functest + docker_tag: latest + envs: '-e INSTALLER_TYPE=compass -e INSTALLER_IP=192.168.200.2 + -e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn + -e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal' + opts: '-id --privileged=true' + pre_condition: + cmds: + - 'echo test for precondition' + testcase: + cmds: + - 'python /home/opnfv/repos/functest/ci/prepare_env.py start' + - 'python /home/opnfv/repos/functest/ci/run_tests.py -t {{script_testcase}} -r' + post_condition: + cmds: + - '' + result: + dir: '/home/opnfv/functest/results' + store_type: 'file' + file_path: 'tempest/tempest.log' + db_url: 'http://testresults.opnfv.org/test/api/v1/results?case=%s&last=1' + +- ``image_name`` and ``docker_tag`` sections define the docker image pulled from the public dockerhub. +- ``envs`` should be correctly configed according to the SUT(System Under Test). +- ``pre_condition`` represents some cleanups or preparations. + ``testcase`` represents the testcases running cmds. + ``post_condition`` represents some cleanups needed after all testcases finished. +- ``result`` section gives the directory of the dovetail tool test result. + ``db_url`` gives the database URL of the dovetail results to be stored. + +Yardstick config template syntax +--------------------------------- + +The configuration in ``yardstick_config.yml`` is similiar to ``functest_config.yml``, +and an example is shown as follows: + +:: + + yardstick: + image_name: opnfv/yardstick + docker_tag: latest + envs: '-e INSTALLER_TYPE=compass -e INSTALLER_IP=192.168.200.2 + -e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn + -e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal + -e EXTERNAL_NETWORK=ext-net' + opts: '-id --privileged=true' + pre_condition: + cmds: + - 'source /home/opnfv/repos/yardstick/tests/ci/prepare_env.sh && + source /home/opnfv/repos/yardstick/tests/ci/clean_images.sh && cleanup' + - 'source /home/opnfv/repos/yardstick/tests/ci/prepare_env.sh && + cd /home/opnfv/repos/yardstick && source tests/ci/load_images.sh' + testcase: + cmds: + - 'mkdir -p /home/opnfv/yardstick/results/' + - 'cd /home/opnfv/repos/yardstick && source tests/ci/prepare_env.sh && + yardstick task start tests/opnfv/test_cases/{{script_testcase}}.yaml + --output-file /home/opnfv/yardstick/results/{{script_testcase}}.out &> + /home/opnfv/yardstick/results/yardstick.log' + post_condition: + cmds: + - '' + result: + dir: '/home/opnfv/yardstick/results' + store_type: 'file' + file_path: 'yardstick.log' + db_url: 'http://testresults.opnfv.org/test/api/v1/results?case=%s&last=1' + +The main differences between ``yardstick_config.yml`` and ``functest_config.yml`` +are the ``cmds`` subsection. + +Jinja2 template syntax +---------------------- + +Jinja2 module can be used to config the ``{{script_testcase}}``. The +parameters are defined in ``dovetail/dovetail/conf/dovetail_config.yml``: + +:: + + parameters: + - name: testcase + path: '("name",)' + - name: script_testcase + path: '("scripts", "testcase")' + +Here ``path`` is the path defined in the testcase configuration files. +Take ``script_testcase`` as an example. For testcase ``dovetail.ipv6.tc001``: + +:: + + dovetail.ipv6.tc001: + name: dovetail.ipv6.tc001 + objective: VIM ipv6 operations, to create/delete network, port and subnet in bulk operation + scripts: + type: functest + testcase: tempest_smoke_serial + sub_testcase_list: + - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network + - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port + - tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet + +The path ("scripts", "testcase") means 'testcase' is the subsection of 'scripts' section. From above, +by following the path ("scripts", "testcase") we can get the value of ``{{script_testcase}}`` is 'tempest_smoke_serial'. |