blob: 092bcf39911101947b20d51bab4cf89a45a3a40f (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
|
# Xtesting
[Cédric Ollivier](mailto:cedric.ollivier@orange.com) &
[Morgan Richomme](mailto:morgan.richomme@orange.com)
2018/03/23
## Why Xtesting?
### Functest framework
- handle all interactions with OPNFV CI/CD components (entry points,
results publication, status codes, etc.)
- ease the development of third-party testcases by offering multiple drivers:
python, bash, unittest, robotframework and vnf.
**It has worked very well for E-release, but updates were required for
the next ones**
### Internal needs
- Functest has to verify Kubernetes deployment but
[its former framework](http://testresults.opnfv.org/functest/framework/) was
linked to OpenStack (e.g. credentials sourcing, rally verifiers, etc.)
- hosting both OpenStack and Kubernetes in the same python package would
increase dependencies and then complexify
[container slicing](http://testresults.opnfv.org/functest/dockerslicing/)
**Why not refactoring the first Functest framework?**
### External needs
- Functest python and containers framework could be very useful out of OPNFV
(ease developing testcases, manage requirements and offer lighweight docker
images)
- a new Functest design could simplify test integration in a complete
[OPNFV-based CI/CD toolchain](http://docs.opnfv.org/en/stable-euphrates/testing/ecosystem/overview.html)
(e.g. Testing Containers, Test API and dashboard)
**Let the developer only work on the test suites without diving into CI/CD
integration**
## What's been done for Fraser?
### Xtesting framework
- Functest framework were moved to a new xtesting repository
(functest only hosts OpenStack testcases)
- it has been updated and improved to follow all Xtesting technical
guidelines:
- unlink to **OpenStack** and **OPNFV**
- support both python2 and python3 (required by **Functional Gating**)
- be fully covered by unit tests and well rated by pylint (10/10)
### Xtesting deliverables
- Xtesting is released as [a python package](https://pypi.python.org/pypi/xtesting/)
and then is unlinked to OPNFV Milestones (Functest python package now depends
on it)
- [opnfv/xtesting](https://hub.docker.com/r/opnfv/xtesting/) is proposed to
build third-parties containers (both amd64 and arm64 architectures).
- the API documentation is automatically built
[online](http://xtesting.readthedocs.io/en/latest/apidoc/xtesting.html)
## Functest & Xtesting in ONAP OpenLab
### first verify the infrastructure via Functest
```
+----------------------------+------------------+---------------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+----------------------------+------------------+---------------------+------------------+----------------+
| connection_check | functest | healthcheck | 00:07 | PASS |
| api_check | functest | healthcheck | 07:46 | PASS |
| snaps_health_check | functest | healthcheck | 00:36 | PASS |
+----------------------------+------------------+---------------------+------------------+----------------+
```
<!-- .element: style="font-size: 0.34em" -->
```
+------------------------------+------------------+---------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+------------------------------+------------------+---------------+------------------+----------------+
| vping_ssh | functest | smoke | 00:57 | PASS |
| vping_userdata | functest | smoke | 00:33 | PASS |
| tempest_smoke_serial | functest | smoke | 13:22 | PASS |
| rally_sanity | functest | smoke | 24:07 | PASS |
| refstack_defcore | functest | smoke | 05:21 | PASS |
| patrole | functest | smoke | 04:29 | PASS |
| snaps_smoke | functest | smoke | 46:54 | PASS |
| odl | functest | smoke | 00:00 | SKIP |
| odl_netvirt | functest | smoke | 00:00 | SKIP |
| neutron_trunk | functest | smoke | 00:00 | SKIP |
+------------------------------+------------------+---------------+------------------+----------------+
```
<!-- .element: style="font-size: 0.34em" -->
```
+----------------------+------------------+--------------+------------------+----------------+
| TEST CASE | PROJECT | TIER | DURATION | RESULT |
+----------------------+------------------+--------------+------------------+----------------+
| cloudify_ims | functest | vnf | 28:15 | PASS |
| vyos_vrouter | functest | vnf | 17:59 | PASS |
| juju_epc | functest | vnf | 46:44 | PASS |
+----------------------+------------------+--------------+------------------+----------------+
```
<!-- .element: style="font-size: 0.34em" -->
### then run ONAP HealthCheck
- All tests are run by a specialized docker container(**<100 MB**) instead of
the classical ONAP testing virtual machine (**> 1GB**).
- the container mainly inherits from opnfv/xtesting and is completed by:
- python dependencies
- all ONAP robotframework files retrieved from the original repositories
- testcases.yaml describing the testcases
[Orange-OpenSource/xtesting-onap-robot](https://github.com/Orange-OpenSource/xtesting-onap-robot/)
<!-- .element: style="border: 0; width: 70%" -->
## Conclusion
## Benefits
- Xtesting allows a proper design inside OPNFV
- Xtesting and Functest help other LFN projects:
- verifying the infrastructure on top of which the components are deployed
- ease verifying the components as well in the same CI/CD toolchain
**All contributions coming from LFN projects are more than welcome!**
|
