1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
|
.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV
.. _cli-reference:
=========================================
Dovetail Command Line Interface Reference
=========================================
Dovetail command line is to have a simple command line interface in Dovetail to
make easier for users to handle the functions that dovetail framework provides.
Commands List
=============
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| Commands | Action |
| | |
+========================================================================+===================================================================================================+
| dovetail --help | -h | Show usage of command "dovetail" |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail --version | Show version number |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| **Dovetail List Commands** |
| |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail list --help | -h | Show usage of command "dovetail list" |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail list | List all available test suites and all test cases within each test suite |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail list <test_suite_name> | List all available test areas within test suite <test_suite_name> |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| **Dovetail Show Commands** |
| |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail show --help | -h | Show usage of command "dovetail show" |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail show <test_case_name> | Show the details of one test case |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| **Dovetail Run Commands** |
| |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail run --help | -h | Show usage of command "dovetail run" |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail run | Run Dovetail with all test cases within default test suite |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail run --testsuite <test_suite_name> | Run Dovetail with all test cases within test suite <test_suite_name> |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail run --testsuite <test_suite_name> --testarea <test_area_name> | Run Dovetail with test area <test_area_name> within test suite <test_suite_name>. |
| | Test area can be chosen from (vping, tempest, security, ha, stress, bgpvpn, vnf, snaps). |
| | Repeat option to set multiple test areas. |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail run --testcase <test_case_name> | Run Dovetail with one or more specified test cases. |
| | Repeat option to set multiple test cases. |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail run --mandatory --testsuite <test_suite_name> | Run Dovetail with all mandatory test cases within test suite <test_suite_name> |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail run --optional --testsuite <test_suite_name> | Run Dovetail with all optional test cases within test suite <test_suite_name> |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail run --debug | -d | Run Dovetail with debug mode and show all debug logs |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail run --offline | Run Dovetail offline, use local docker images instead of download online |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail run --report | -r <db_url> | Package the results directory which can be used to upload to OVP web portal |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail run --deploy-scenario <deploy_scenario_name> | Specify the deploy scenario having as project name 'ovs' |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail run --no-api-validation | Disable strict API response validation |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail run --no-clean | -n | Keep all Containers created for debuging |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
| dovetail run --stop | -s | Stop immediately when one test case failed |
| | |
+------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+
Commands Examples
=================
Dovetail Commands
-----------------
.. code-block:: bash
root@1f230e719e44:~/dovetail/dovetail# dovetail --help
Usage: dovetail [OPTIONS] COMMAND [ARGS]...
Options:
--version Show the version and exit.
-h, --help Show this message and exit.
Commands:
list list the testsuite details
run run the testcases
show show the testcases details
.. code-block:: bash
root@1f230e719e44:~/dovetail/dovetail# dovetail --version
dovetail, version 2018.9.0
Dovetail List Commands
----------------------
.. code-block:: bash
root@1f230e719e44:~/dovetail/dovetail# dovetail list --help
Usage: dovetail list [OPTIONS] [TESTSUITE]
list the testsuite details
Options:
-h, --help Show this message and exit.
.. code-block:: bash
root@1f230e719e44:~/dovetail/dovetail# dovetail list ovp.2018.09
- mandatory
functest.vping.userdata
functest.vping.ssh
functest.tempest.osinterop
functest.tempest.compute
functest.tempest.identity_v3
functest.tempest.image
functest.tempest.network_api
functest.tempest.volume
functest.tempest.neutron_trunk_ports
functest.tempest.ipv6_api
functest.security.patrole
yardstick.ha.nova_api
yardstick.ha.neutron_server
yardstick.ha.keystone
yardstick.ha.glance_api
yardstick.ha.cinder_api
yardstick.ha.cpu_load
yardstick.ha.disk_load
yardstick.ha.haproxy
yardstick.ha.rabbitmq
yardstick.ha.database
bottlenecks.stress.ping
- optional
functest.tempest.ipv6_scenario
functest.tempest.multi_node_scheduling
functest.tempest.network_security
functest.tempest.vm_lifecycle
functest.tempest.network_scenario
functest.tempest.bgpvpn
functest.bgpvpn.subnet_connectivity
functest.bgpvpn.tenant_separation
functest.bgpvpn.router_association
functest.bgpvpn.router_association_floating_ip
yardstick.ha.neutron_l3_agent
yardstick.ha.controller_restart
functest.vnf.vims
functest.vnf.vepc
functest.snaps.smoke
Dovetail Show Commands
----------------------
.. code-block:: bash
root@1f230e719e44:~/dovetail/dovetail# dovetail show --help
Usage: dovetail show [OPTIONS] TESTCASE
show the testcases details
Options:
-h, --help Show this message and exit.
.. code-block:: bash
root@1f230e719e44:~/dovetail/dovetail# dovetail show functest.vping.ssh
---
functest.vping.ssh:
name: functest.vping.ssh
objective: testing for vping using ssh
validate:
type: functest
testcase: vping_ssh
report:
source_archive_files:
- functest.log
dest_archive_files:
- vping_logs/functest.vping.ssh.log
check_results_file: 'functest_results.txt'
sub_testcase_list:
.. code-block:: bash
root@1f230e719e44:~/dovetail/dovetail# dovetail show functest.tempest.image
---
functest.tempest.image:
name: functest.tempest.image
objective: tempest smoke test cases about image
validate:
type: functest
testcase: tempest_custom
pre_condition:
- 'cp /home/opnfv/userconfig/pre_config/tempest_conf.yaml /usr/lib/python2.7/site-packages/functest/opnfv_tests/openstack/tempest/custom_tests/tempest_conf.yaml'
- 'cp /home/opnfv/userconfig/pre_config/testcases.yaml /usr/lib/python2.7/site-packages/xtesting/ci/testcases.yaml'
pre_copy:
src_file: tempest_custom.txt
dest_path: /usr/lib/python2.7/site-packages/functest/opnfv_tests/openstack/tempest/custom_tests/test_list.txt
report:
source_archive_files:
- functest.log
- tempest_custom/tempest.log
- tempest_custom/tempest-report.html
dest_archive_files:
- tempest_logs/functest.tempest.image.functest.log
- tempest_logs/functest.tempest.image.log
- tempest_logs/functest.tempest.image.html
check_results_file: 'functest_results.txt'
sub_testcase_list:
- tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_register_upload_get_image_file[id-139b765e-7f3d-4b3d-8b37-3ca3876ee318,smoke]
- tempest.api.image.v2.test_versions.VersionsTest.test_list_versions[id-659ea30a-a17c-4317-832c-0f68ed23c31d,smoke]
Dovetail Run Commands
----------------------
.. code-block:: bash
root@1f230e719e44:~/dovetail/dovetail# dovetail run --help
Usage: run.py [OPTIONS]
Dovetail compliance test entry!
Options:
--deploy-scenario TEXT Specify the DEPLOY_SCENARIO which will be used as input by each testcase respectively
--optional Run all optional test cases.
--offline run in offline method, which means not to update the docker upstream images, functest, yardstick, etc.
-r, --report Create a tarball file to upload to OVP web portal
-d, --debug Flag for showing debug log on screen.
--testcase TEXT Compliance testcase. Specify option multiple times to include multiple test cases.
--testarea TEXT Compliance testarea within testsuite. Specify option multiple times to include multiple test areas.
-s, --stop Flag for stopping on test case failure.
-n, --no-clean Keep all Containers created for debuging.
--no-api-validation disable strict API response validation
--mandatory Run all mandatory test cases.
--testsuite TEXT compliance testsuite.
-h, --help Show this message and exit.
.. code-block:: bash
root@1f230e719e44:~/dovetail/dovetail# dovetail run --testcase functest.vping.ssh --offline -r --deploy-scenario os-nosdn-ovs-ha
2017-10-12 14:57:51,278 - run - INFO - ================================================
2017-10-12 14:57:51,278 - run - INFO - Dovetail compliance: ovp.2018.09!
2017-10-12 14:57:51,278 - run - INFO - ================================================
2017-10-12 14:57:51,278 - run - INFO - Build tag: daily-master-b80bca76-af5d-11e7-879a-0242ac110002
2017-10-12 14:57:51,278 - run - INFO - DEPLOY_SCENARIO : os-nosdn-ovs-ha
2017-10-12 14:57:51,336 - run - WARNING - There is no hosts file /home/dovetail/pre_config/hosts.yaml, may be some issues with domain name resolution.
2017-10-12 14:57:51,336 - run - INFO - Get hardware info of all nodes list in file /home/cvp/pre_config/pod.yaml ...
2017-10-12 14:57:51,336 - run - INFO - Hardware info of all nodes are stored in file /home/cvp/results/all_hosts_info.json.
2017-10-12 14:57:51,517 - run - INFO - >>[testcase]: functest.vping.ssh
2017-10-12 14:58:21,325 - report.Report - INFO - Results have been stored with file /home/cvp/results/functest_results.txt.
2017-10-12 14:58:21,325 - report.Report - INFO -
Dovetail Report
Version: 2018.09
Build Tag: daily-master-b80bca76-af5d-11e7-879a-0242ac110002
Test Date: 2018-08-13 03:23:56 UTC
Duration: 291.92 s
Pass Rate: 0.00% (1/1)
vping: pass rate 100%
-functest.vping.ssh PASS
|
