aboutsummaryrefslogtreecommitdiffstats
path: root/functest_kubernetes/cnf_conformance
AgeCommit message (Collapse)AuthorFilesLines
2025-01-30Remove useless call to self.corev1.list_namespacestable/v1.29Cédric Ollivier1-1/+0
It should have been removed in the previous change stopping creating namespaces. Change-Id: I1bb39ca9184a530b42f3cb4aa45423709e4090d3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit cfc203dc22b8cf4e6088f5e21119f5bab94ccf9d)
2025-01-30Upgrade cnf_testsuite to latest changeCédric Ollivier1-22/+2
uninstall_all is needed as well as pss support. note: we must keep deleting the namespaces until [1] is fixed [1] https://github.com/cnti-testcatalog/testsuite/issues/2213 Change-Id: Ife81a87d5cc0e7ae432c7397c1125adf595d2fa4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit d912f5609a0f743f6be2d3c235eb7656f96dc8b4)
2025-01-28Describe how to check how cloud native your CNFs areCédric Ollivier1-0/+5
It also adds an ugly sleep waiting https://github.com/cnti-testcatalog/testsuite/issues/2194 to be fixed. Change-Id: Id6c59c462a55d84543d1b1edc7e33d4e69a7f944 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8925b99f00044e38e942cf45f593c5ecbceaf6f5)
2025-01-28Add a new testcase: cnf_testsuite_workloadCédric Ollivier1-6/+14
It runs all the workload test plan vs the example coredns. Please note criteria is arbitrary set to 50%. Testing vs coredns is about 80% and envoy 70%. It also add extra processing to calculate results in the main 3 cases (cert, workload and single tag). Change-Id: I5e161727cab40228991ab97a061b13b335f87b4c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit b716ddbfd7efe66cf9fbe73838d9eec2e3350025)
2025-01-28Parameterize cnf-configCédric Ollivier1-2/+5
It eases switching from one CNF to another. It also removes obsolete extra commands in Dockerfile. Change-Id: I61f9dc1312a6d6b450ef0ac38e37fa11df614c66 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8766d98959e8df20eab5409ce10329bd4a930ea6)
2025-01-27Revert "Upgrade cnf_testsuite to latest change"Cédric Ollivier1-2/+22
v1.29 needs tarball releases so the previous change is delayed. This reverts commit 32504996002da9fb8f6df3bda7fd808dea592a0e. Change-Id: I6322ccc5206b6fac074bf922c5b6cd59a2f7b10d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2025-01-27Upgrade cnf_testsuite to latest changeCédric Ollivier1-22/+2
uninstall_all is needed as well as pss support. note: we must keep deleting the namespaces until [1] is fixed [1] https://github.com/cnti-testcatalog/testsuite/issues/2213 Change-Id: I65cd11bf02d8df049f964809eee1d51715e75473 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit d912f5609a0f743f6be2d3c235eb7656f96dc8b4)
2025-01-20Fix the cnf-testsuite callsCédric Ollivier1-2/+2
It takes [1] into account [1] https://github.com/cnti-testcatalog/testsuite/commit/bb1634895c6f9f119cdd3e543d3e110b08227c19 Change-Id: I96e625a14b5440fd9a039f5b87a22904a6444e8b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 7f1edb33f27d20e0d28fd9b391ccd333cd1bd032)
2024-09-29CNF TestSuite now uses cnf-default namespaceCédric Ollivier1-2/+2
Change-Id: Ia595665cc8f84a2aabd141ead7bb343758cb3d18 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 93c57b463c51116b99b3e59312cf8cf0c9bd6894)
2024-09-23Upgrade CNF TestSuite to v1.3.3Cédric Ollivier1-4/+3
Change-Id: I3ee3d1b49fc9efded0b3edc00a8ba67b5cc63bca Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2024-01-18Fix an invalid name issueCédric Ollivier1-2/+2
C0103: Variable name "ns" doesn't conform to snake_case naming style (invalid-name) Change-Id: I05a93badbe059cf868eade47a38699b6186c3528 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit ebf1398051e0b745fe5175b1d2807dfe5604789c)
2024-01-17Patch cnf_testsuite namespaces if they existCédric Ollivier1-6/+20
Change-Id: Ibace5356cb1f458fa8b929c0c84b2f06e5852ea0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2024-01-16Enforce baseline psp for cnf_testsuiteCédric Ollivier1-1/+18
It would be better not to change the default namespace. Change-Id: I9fde052d4dd7e0e4f6551213e2028c91c0296e42 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2023-12-21Fix CNF TestSuite's log levelCédric Ollivier1-3/+3
Change-Id: I9a7e1a9d17fd575b7e6077d8ccf5dd5d1a259717 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2023-12-20Exit if early failure in cnf_testsuiteCédric Ollivier1-7/+23
Change-Id: I3a6a264a73bdc73c1c90471948316d216cf6ad98 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2022-10-17cnf conformance now returns non zero valuesCédric Ollivier1-10/+5
Change-Id: I61cb69ef48b113571f791c7ee2ef7387c7421f32 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2022-10-17Use cnf certifaction criteriaCédric Ollivier1-4/+5
Change-Id: I7362827b600914ab032ba9657ac17e7bb157bdfc Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2022-10-17Update cnf_testsuite v0.34.0Cédric Ollivier1-4/+4
It now runs cert (replacing workload) It downgrades msg to warning as cnf_testsuite now returns non 0 values. Change-Id: I0ce4f201065bf601111d7154cb4afa31e9a4666c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 4338a2532600e659179335a5f5f71ab15f83f8d7)
2022-10-10Uninstall falco and cri_tools after testingCédric Ollivier1-4/+6
Functest runs several clusters in the same server. It quickly failes if falco is still running (dkms tries to unload the module which is in-use). Change-Id: I13b596a78ac40064cbed1aead8c785cc1cfc8626 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-09-11Set encoding utf-8 when opening fileCédric Ollivier1-1/+2
Change-Id: I4e756552173247499ba882bfee4fbe8738fbae3d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-09-10Fallback to item calculationCédric Ollivier1-0/+8
There is no final result if simple tag is called. Change-Id: If86c5115e917e5790799fac4308cf08d57e87905 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-08-18Update to Alpine 3.14Cédric Ollivier1-1/+1
It also disabled wrong order check as it fails vs kubernetes amongst others. [1] https://github.com/PyCQA/pylint/issues/2175 Change-Id: I3d641c213067428848212a148d25d78051c5674f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-07-07Print cnf_testsuite console on exceptionsCédric Ollivier1-0/+4
Change-Id: I3c20993e1cbb644546ac89a8b4b7d1cfd98b0a80 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-05-25Update cnf_testsuite to v0.11.2Cédric Ollivier1-22/+11
https://github.com/cncf/cnf-testsuite/releases/tag/v0.11.2 Change-Id: Id3a5ea98260859f7fe5aba6ecf02d09905610dda Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-05-03Fix cnf-conformance results filenameCédric Ollivier1-1/+1
It also decreases criteria to 1 due to massive changes (dockerd). Change-Id: Ic420139e6355debae183a77d6b0c574801b05076 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-05-03Update cnf-conformance to cnf-testsuiteCédric Ollivier1-10/+10
https://github.com/cncf/cnf-conformance https://github.com/cncf/cnf-testsuite Change-Id: I77983e0f04781a92b34afcf82f21366384782964 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-01-10Update cnf-conformance to v0.9.19Cédric Ollivier1-0/+6
It also adds configs needed by resilience. https://github.com/cncf/cnf-conformance/releases/tag/v0.9.19 Change-Id: I94b1efc47c5ceef96bc8879a6c71f92db1f63bd2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-11-21Update cnf_conformance to v0.9.4Cédric Ollivier1-25/+19
It removes all hacks bypassing a few former bugs (e.g. [1]) and adapts to the new cnf-conformance result output logic. It runs all workload tests as it's now proposed (k8s conformance is already part of smoke) and select the latest coredns cnf-conformance.yml taking latest helm repo changes into account. It sets criteria 95 as all steps are considered as passed and a overall result 100/105. [1] https://github.com/cncf/cnf-conformance/issues/388 Change-Id: I95b7889b21b86961e7a1d0c84b13e0ae367cae44 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-07Override default Xtesting logs in cnf-conformanceCédric Ollivier1-0/+5
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-latest-cnf_conformance-run/7/console Change-Id: Ia6845525e62274f7792b4cb1909f36204fb51418 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-04Allow skipping arm and arm64 if cnf-conformanceCédric Ollivier1-3/+16
cnf-conformance doesn't support this architectures. It also improve cleaning to publish artifacts [1]. [1] https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-latest-cnf_conformance-run/2/console Change-Id: Ie64e4a1b8b496a050a1798eaf4e5ef6397c011d0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-04Add cnf-conformance in functest-kubernetes-cnfCédric Ollivier2-0/+104
It selects cnf-conformance v0.7.2-beta1 (current latest release) and the default CNF as proposed by [1]. The end user is free to override the default yml files on purpose (e.g. docker volumes). Tag (default: all) could be modified too via testcases.yaml [1] https://hackmd.io/@vulk/SkY54QnsU Change-Id: Iec2a830c441e523e067f6eb114ecb205c49c6b93 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>