blob: aaf087743089788a1930bf9e5b5763a190941d3c (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
|
# Functest Gates
[Cédric Ollivier](mailto:cedric.ollivier@orange.com)
2018/04/19
### Why gating?
- maintain an overall high quality code whatever the skills involved in our
open community
- detect the errors as soon as possible (before merge operations)
- verify automatically most of Functest milestones
### Today's Verify -1
- one error is detected by yamllint, pylint or doc8
- one unit test fails (py27 and py35)
- our Framework is not fully covered
- specific modules are not rated 10/10 by pylint
- unix permissions are wrong
- one doc cannot be generated without warning
Please see [tox.ini](https://git.opnfv.org/functest/tree/tox.ini) for details
### Gambia's Verify -1
- enforce an overall 10/10 by pylint
- check the full coverage of all third-party interfaces
- integrate other linters like ShellScript (if they meet tox basics)
Please see
[Gambia Challenges](http://testresults.opnfv.org/functest/gambiachallenges/)
for details
### Great! But
- the current gate checks all coding rules, our Framework and all interfaces
to third-parties but **not our testcases**
- it doesn't break **the circular dependencies** between Installers and
Functest (Installers ask for a trustable healthcheck but we need results of
"Installer runs" as prerequisites)
Our testcases have still **to be verified by hand** what differs from
OpenStack Workflow
### functional gating
- we need to run all Functest patches before merge against reference
plateforms (OpenStack and Kubernetes)
- a voting job must forbid the merge if one test fails (as the workflow
process implemented by OpenStack)
### reference platform
- it could be baremetal or virtual if all testcases (including all VNFs) can
be tested successfully
- any compliant Installer could be selected as long as the appropriate target
VIM version is proposed at the beginning of the release:
- devstack (OpenStack gates)
- kolla-ansible (Orange ONAP OpenLab)
- XCI
- APEX
### Verify +2
Only the functional check will take hours once the patchset is accepted
by core reviewers (each patchset will be verified in ~10 minutes).
**Always stable!**
## Thank you!
|
