1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
|
.. This work is licensed under a Creative Commons Attribution 4.0 International
.. License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
Config Template Syntax
======================
Dovetail uses Functest/Yardstick Docker container to run its testcases. So you need to give
configurations for building the container and the commands it needs to do. In dovetail,
all of these are defined in config yaml files ``dovetail/dovetail/conf/functest_config.yml``
and ``dovetail/dovetail/conf/yardstick_config.yml``.
Functest template syntax
------------------------
For example, you can define your ``functest_config.yml`` as:
::
functest:
image_name: opnfv/functest
docker_tag: latest
envs: '-e INSTALLER_TYPE=compass -e INSTALLER_IP=192.168.200.2
-e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn
-e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal'
opts: '-id --privileged=true'
pre_condition:
cmds:
- 'echo test for precondition'
testcase:
cmds:
- 'python /home/opnfv/repos/functest/ci/prepare_env.py start'
- 'python /home/opnfv/repos/functest/ci/run_tests.py -t {{script_testcase}} -r'
post_condition:
cmds:
- ''
result:
dir: '/home/opnfv/functest/results'
store_type: 'file'
file_path: 'tempest/tempest.log'
db_url: 'http://testresults.opnfv.org/test/api/v1/results?case=%s&last=1'
First, you need to give the image that you want to use for building functest/yardstick container.
Besides, there also need some envirnment parameters such as ``INSTALLER_TYPE`` and ``INSTALLER_IP``
and the options for you container. Then the functest/yardstick container can be build with your
settings.
Second, there need three kinds of commands, ``pre_condition``, ``testcase`` and ``post_condition``.
If you want to do some cleanups or preparations, the commands can be put into ``pre_condition``
section orderly. All commands in this section will just be executed once in the begining.
The ``testcase`` section does the main jobs of the testing. All functest testcases will use the
container to execute these commands one by one. After finishing that, the test is accomplished
and the results are stored in files or uploaded to database. The ``post_condition`` section
does some work such as clean Docker images or something else after all testcases finished.
All commands in this section will just execute once.
Besides, there need a ``result`` section and it gives the directory of the functest/yardstick
results. The ``store_type`` should be the same with the cmds in ``testcase``. That means if the
test results are stored in files, then store_type need to be file and the file_path is also
needed. If the test results are uploaded to database, then a db_url is needed for acquiring the results.
Yardstick template syntax
-------------------------
The framework of ``yardstick_config.yml`` is almost the same as ``functest_config.yml``.
::
yardstick:
image_name: opnfv/yardstick
docker_tag: latest
envs: '-e INSTALLER_TYPE=compass -e INSTALLER_IP=192.168.200.2
-e NODE_NAME=dovetail-pod -e DEPLOY_SCENARIO=ha_nosdn
-e BUILD_TAG=dovetail -e CI_DEBUG=true -e DEPLOY_TYPE=baremetal
-e EXTERNAL_NETWORK=ext-net'
opts: '-id --privileged=true'
pre_condition:
cmds:
- 'source /home/opnfv/repos/yardstick/tests/ci/prepare_env.sh &&
source /home/opnfv/repos/yardstick/tests/ci/clean_images.sh && cleanup'
- 'source /home/opnfv/repos/yardstick/tests/ci/prepare_env.sh &&
cd /home/opnfv/repos/yardstick && source tests/ci/load_images.sh'
testcase:
cmds:
- 'mkdir -p /home/opnfv/yardstick/results/'
- 'cd /home/opnfv/repos/yardstick && source tests/ci/prepare_env.sh &&
yardstick task start tests/opnfv/test_cases/{{script_testcase}}.yaml
--output-file /home/opnfv/yardstick/results/{{script_testcase}}.out &>
/home/opnfv/yardstick/results/yardstick.log'
post_condition:
cmds:
- ''
result:
dir: '/home/opnfv/yardstick/results'
store_type: 'file'
file_path: 'yardstick.log'
db_url: 'http://testresults.opnfv.org/test/api/v1/results?case=%s&last=1'
The main differences between ``yardstick_config.yml`` and ``functest_config.yml`` are the commands.
Jinja2 template syntax
----------------------
Note that you can use jinja2 template for your parameters such as ``{{script_testcase}}``. The
parameters are defined in ``dovetail/dovetail/conf/dovetail_config.yml``:
::
parameters:
- name: testcase
path: '("name",)'
- name: script_testcase
path: '("scripts", "testcase")'
Here ``path`` is the path in testcase config files that you can find the value of parameters. Take
``script_testcase`` as the example. For testcase dovetail.ipv6.tc001:
::
dovetail.ipv6.tc001:
name: dovetail.ipv6.tc001
objective: VIM ipv6 operations, to create/delete network, port and subnet in bulk operation
scripts:
type: functest
testcase: tempest_smoke_serial
sub_testcase_list:
- tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network
- tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port
- tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet
The path ("scripts", "testcase") means 'testcase' subsection of 'scripts' section. So follow
the path ("scripts", "testcase") we can get the value of ``{{script_testcase}}`` that is
'tempest_smoke_serial'.
|
