blob: c215e05ec371b5389e6c911706e0432e85358530 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
|
.. SPDX-License-Identifier: CC-BY-4.0
Test results
============
Manual testing
--------------
In manual mode test results are displayed in the console and result files
are put in /home/opnfv/functest/results.
If you want additional logs, you may configure the logging.ini under
/usr/lib/python2.7/site-packages/functest/ci.
Automated testing
-----------------
In automated mode, test results are displayed in jenkins logs, a summary is
provided at the end of the job and can be described as follow::
+-------------------------+----------------------------------------------------------+
| ENV VAR | VALUE |
+-------------------------+----------------------------------------------------------+
| INSTALLER_TYPE | daisy |
| DEPLOY_SCENARIO | os-nosdn-nofeature-ha |
| BUILD_TAG | jenkins-functest-daisy-baremetal-daily-master-67 |
| CI_LOOP | daily |
+-------------------------+----------------------------------------------------------+
::
+------------------------------+------------------+---------------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+------------------------------+------------------+---------------------+------------------+----------------+
| connection_check | functest | healthcheck | 00:08 | PASS |
| api_check | functest | healthcheck | 04:22 | PASS |
| snaps_health_check | functest | healthcheck | 00:35 | PASS |
| vping_ssh | functest | smoke | 00:54 | PASS |
| vping_userdata | functest | smoke | 00:27 | PASS |
| tempest_smoke_serial | functest | smoke | 19:39 | FAIL |
| rally_sanity | functest | smoke | 15:16 | PASS |
| refstack_defcore | functest | smoke | 15:55 | PASS |
| snaps_smoke | functest | smoke | 26:45 | FAIL |
| cloudify_ims | functest | vnf | 23:56 | PASS |
| orchestra_openims | orchestra | vnf | 15:07 | PASS |
| orchestra_clearwaterims | orchestra | vnf | 19:10 | PASS |
| vyos_vrouter | functest | vnf | 00:00 | SKIP |
+------------------------------+------------------+---------------------+------------------+----------------+
Functest Kubernetes test result::
+--------------------------------------+------------------------------------------------------------+
| ENV VAR | VALUE |
+--------------------------------------+------------------------------------------------------------+
| INSTALLER_TYPE | compass |
| DEPLOY_SCENARIO | k8-nosdn-nofeature-ha |
| BUILD_TAG | jenkins-functest-compass-baremetal-daily-master-75 |
| CI_LOOP | daily |
+--------------------------------------+------------------------------------------------------------+
Kubernetes healthcheck suite::
+-------------------+------------------+---------------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+-------------------+------------------+---------------------+------------------+----------------+
| k8s_smoke | functest | healthcheck | 01:54 | PASS |
+-------------------+------------------+---------------------+------------------+----------------+
Kubernetes smoke suite::
+-------------------------+------------------+---------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+-------------------------+------------------+---------------+------------------+----------------+
| k8s_conformance | functest | smoke | 57:47 | PASS |
+-------------------------+------------------+---------------+------------------+----------------+
Kubernetes features suite::
+----------------------+------------------+------------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+----------------------+------------------+------------------+------------------+----------------+
| stor4nfv_k8s | stor4nfv | stor4nfv | 00:00 | SKIP |
| clover_k8s | clover | clover | 00:00 | SKIP |
+----------------------+------------------+------------------+------------------+----------------+
Results are automatically pushed to the test results database, some additional
result files are pushed to OPNFV artifact web sites.
Based on the results stored in the result database, a `Functest reporting`_
portal is also automatically updated. This portal provides information on the
overall status per scenario and per installer
.. _`Functest reporting`: http://testresults.opnfv.org/reporting/master/functest/status-apex.html
|
