1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
|
.. This work is licensed under a Creative Commons Attribution 4.0 International
.. License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
==================
Template Syntax
==================
Testcase Template Syntax
=========================
The testcases used for compliance and certification are defined in the
``dovetail/testcase`` directory, which are written in yaml format.
Take the testcase ``ipv6.tc001.yml`` as an example. It is shown as:
::
---
dovetail.ipv6.tc001:
name: dovetail.ipv6.tc001
objective: Bulk creation and deletion of IPv6 networks, ports and subnets
validate:
type: functest
testcase: tempest_smoke_serial
pre_condition:
- 'echo test for precondition in testcase'
cmds:
- 'functest env prepare'
- 'functest testcase run {{validate_testcase}}'
post_condition:
- 'echo test for precondition in testcase'
report:
sub_testcase_list:
- tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network
- tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port
- tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet
- At least three sections named ``name``, ``objective``, ``validate``,
``report`` must be included
- Section ``name`` distinguishes different test cases used for compliance,
it is composed of 3 parts, ``dovetail.``, belongs to which test area,
and the serial number
- Section ``objective`` briefly describes what this testcase does
- Section ``validate`` defines the scripts and configurations for the
validation of the test case. ``type`` defines which method is used to validate,
3 ways, i.e., functest, yardstick and shell is supported currently.
``testcase`` represents the testcases in slicing/tier.
- Section ``report`` defines the sub_testcases to be run.
For ``yardstick``, since it is not sliced by now,
``sub_testcase_list`` is not needed, only to edit the ``testcase`` part in
section ``validate``, such as ``yardstick_tc027``
Config Template Syntax
=======================
For Dovetail tool, the config files are located in ``dovetail/dovetail/conf``, which are written
in yaml format. As both functest and yardstick are utilized by Dovetail, their configuration files
should be configured as follows, within the ``functest_config.yml`` and ``yardstick_config.yml`` files,
respectively.
Functest config template syntax
-------------------------------
An example of ``functest`` configuration is shown as follows:
::
functest:
image_name: opnfv/functest
docker_tag: latest
envs: '-e INSTALLER_TYPE=compass -e INSTALLER_IP=192.168.200.2
-e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn
-e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal'
opts: '-id --privileged=true'
pre_condition:
cmds:
- 'echo test for precondition'
testcase:
cmds:
- 'python /home/opnfv/repos/functest/ci/prepare_env.py start'
- 'python /home/opnfv/repos/functest/ci/run_tests.py -t {{script_testcase}} -r'
post_condition:
cmds:
- ''
result:
dir: '/home/opnfv/functest/results'
store_type: 'file'
file_path: 'tempest/tempest.log'
db_url: 'http://testresults.opnfv.org/test/api/v1/results?case=%s&last=1'
- ``image_name`` and ``docker_tag`` sections define the docker image pulled from the public dockerhub.
- ``envs`` should be correctly configed according to the SUT(System Under Test).
- ``pre_condition`` represents some cleanups or preparations.
``testcase`` represents the testcases running cmds.
``post_condition`` represents some cleanups needed after all testcases finished.
- ``result`` section gives the directory of the dovetail tool test result.
``db_url`` gives the database URL of the dovetail results to be stored.
Yardstick config template syntax
---------------------------------
The configuration in ``yardstick_config.yml`` is similiar to ``functest_config.yml``,
and an example is shown as follows:
::
yardstick:
image_name: opnfv/yardstick
docker_tag: latest
envs: '-e INSTALLER_TYPE=compass -e INSTALLER_IP=192.168.200.2
-e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn
-e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal
-e EXTERNAL_NETWORK=ext-net'
opts: '-id --privileged=true'
pre_condition:
cmds:
- 'source /home/opnfv/repos/yardstick/tests/ci/prepare_env.sh &&
source /home/opnfv/repos/yardstick/tests/ci/clean_images.sh && cleanup'
- 'source /home/opnfv/repos/yardstick/tests/ci/prepare_env.sh &&
cd /home/opnfv/repos/yardstick && source tests/ci/load_images.sh'
testcase:
cmds:
- 'mkdir -p /home/opnfv/yardstick/results/'
- 'cd /home/opnfv/repos/yardstick && source tests/ci/prepare_env.sh &&
yardstick task start tests/opnfv/test_cases/{{script_testcase}}.yaml
--output-file /home/opnfv/yardstick/results/{{script_testcase}}.out &>
/home/opnfv/yardstick/results/yardstick.log'
post_condition:
cmds:
- ''
result:
dir: '/home/opnfv/yardstick/results'
store_type: 'file'
file_path: 'yardstick.log'
db_url: 'http://testresults.opnfv.org/test/api/v1/results?case=%s&last=1'
The main differences between ``yardstick_config.yml`` and ``functest_config.yml``
are the ``cmds`` subsection.
Jinja2 template syntax
----------------------
Jinja2 module can be used to config the ``{{script_testcase}}``. The
parameters are defined in ``dovetail/dovetail/conf/dovetail_config.yml``:
::
parameters:
- name: testcase
path: '("name",)'
- name: script_testcase
path: '("scripts", "testcase")'
Here ``path`` is the path defined in the testcase configuration files.
Take ``script_testcase`` as an example. For testcase ``dovetail.ipv6.tc001``:
::
dovetail.ipv6.tc001:
name: dovetail.ipv6.tc001
objective: VIM ipv6 operations, to create/delete network, port and subnet in bulk operation
scripts:
type: functest
testcase: tempest_smoke_serial
sub_testcase_list:
- tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network
- tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port
- tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet
The path ("scripts", "testcase") means 'testcase' is the subsection of 'scripts' section. From above,
by following the path ("scripts", "testcase") we can get the value of ``{{script_testcase}}`` is 'tempest_smoke_serial'.
|
