1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
|
# From Paris to Stockholm
[Cédric Ollivier](mailto:cedric.ollivier@orange.com)
2019/06/09
## Agenda
### Agenda
- Functest and Xtesting in a nutshell
- what's new from Paris (Iruya and J-release)?
- code improvement
- functional gates
- switch to python3
- Raspberry PI
- what's next? GSMA, CNF, ETSI...
## Infrastructure verification
### Functest in a nutshell
- verify any kind of OpenStack and Kubernetes deployments (OPNFV model)
including production environments
- conform with upstream rules (OpenStack gate jobs and Kubernetes conformance
tests)
- ensure that the platforms meet Network Functions Virtualization requirements
### Functest suites
- all functional tests (2000+) as defined by the upstream communities
(e.g. Tempest, neutron-tempest-api, Barbican, Patrole...)
- upstream API and dataplane benchmarking tools (Rally, VMTP and Shaker)
- Virtual Network Function deployments and testing (vIMS, vRouter and vEPC)
### Network automation journey
### Xtesting in a nutshell
- allow the developer to work only on the test suites without diving into
CI/CD integration
- simplify test integration in a complete LFN-based CI/CD toolchain (e.g.
Jenkins, Testing Containers, Test API and dashboard)
- allow a proper design and verify multiple components in the same CI/CD
toolchain (OpenStack, Kubernetes, ONAP, etc.)
**Easy to use and very useful for any CI/CD toolchain**
### Chained CI
<!-- .element: style="border: 0" -->
### Testing [FTTH](https://en.wikipedia.org/wiki/Fiber_to_the_x) equipment
<!-- .element: style="border: 0; width: 90%" -->
### Demo
- describe the testcase execution
- build and publish your container
- list your test in the toolchain description
- deploy your own toolchain in a few commands
https://wiki.opnfv.org/pages/viewpage.action?pageId=32015004
## What's new from Paris (Iruya and J-release)
### Code improvement
- all patches are now verified by [bandit](https://github.com/PyCQA/bandit) (in
addition to yamllint, pep8, pylint, ansible-lint and bashate. Please see
[tox.ini](https://git.opnfv.org/functest/tree/tox.ini) for details).
- 10/10 by pylint is now enforced for all
Functest packages
**we still need python3.6 in build servers to run our unit tests vs python3**
### Functional Gates
- live-migration, VMTP and Shaker are now verified in all gates (pre and post
merge)
- Functest is continuously executed up to 4 tests in parallel verifying our
reference OpenStack/Kubernetes
- we continuously check the remaining resources after multiple runs (they only
remain a few region and volume leaks which are being fixed)
### Switch to python3
- all latest Functest containers switched to python3 as default (Functest has
supported both python2 and python3 via six for a while)
- all missing decode/encode calls are backported to Hunter and Iruya
- VMTP has been updated to keep the same testcase list
- our patch about Cloudify REST client will be published upstream if the
project is still active
### Raspberry PI
- travis-ci.org allows us to continuously cross-compile containers (Hunter,
Iruya and latest) for arm (Raspberry PI) and arm64
- no additional configuration is required (Docker manifest allows an unify way
to run the containers whatever the architectures)
- only testcases leveraging on go have not yet been ported (juju_epc and
Kubernetes conformance tests)
**Try it, and you will love it!**
### New testcases
- refstack_platform and refstack_object (already part of tempest_full) are now
dedicated testcases in Functest
- tempest_slow is on our agenda to keep conforming with OpenStack gates
### New opportunities
- a few indentified opportunities about
[CNF](https://www.cncf.io/announcement/2019/02/25/cncf-launches-cloud-native-network-functions-cnf-testbed/):
[Clearwater Ims via Kubernetes](https://github.com/Metaswitch/clearwater-docker/)
and [xrally-kubernetes](https://github.com/xrally/xrally-kubernetes)
- GSMA should come with new testcases needs (Functest can already be executed
vs the current profiles)
- ETSI could leverage on Xtesting to package its testcases
## Conclusion
### Takeaways
- Xtesting is a key helper from first tests to full end2end service testing
- Functest allows verifying production Infrastructure and is breaking the
classical preproduction/production model
- And we are still pushing the limits:
- python packages
- GNU/Linux packages
**Upstream first! Try them, and you will love them! **
## Thank you
|
