aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2021-03-31Disable sig_scheduling as serialCédric Ollivier2-19/+0
All remaining sig-scheduling tests are serial and for the time Functest doesn't run serial test cases. https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_scheduling-run/13/console Change-Id: I3f09077a1873b92120daa3a97cbb6f68558e2b4d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f28b4673a7b89f124d225e629f0c5267e6008c14)
2021-03-26Remove the useless Feature prefixCédric Ollivier1-1/+1
Change-Id: Ibec3e1cee170a91a4333b3c625ae39cd0c4bbe11 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-03-26Exlude Serial and Feature:Feature:StorageVersionAPICédric Ollivier1-12/+20
Serial asks to split the testcases into 2 runs. https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_api_machinery-run/7/console Change-Id: I58f9d9023f596141bbefae8339043f813a124854 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-03-26Blacklist all alpha testingCédric Ollivier1-11/+11
Change-Id: Ifb9edb0d70f285825eb2cfee05a53590367b2650 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-03-26Leverage py3-setuptools_scm as proposed by AlpineCédric Ollivier1-1/+1
Last version setuptools-scm asks for very new setuptools. https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-xrally_kubernetes-run/194/console Change-Id: Ib1f4b4b49c036e0018fd695b52013bccdaf4998f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-03-25Provide support for air gapped env for securitySylvain Desbureaux5-5/+11
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for security test cases. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: I699d065ee691596c4a5ccf06c22ea76ef00fe497 (cherry picked from commit af623f6b45c9357b0e33210f54db64b270d162bf)
2021-03-22Add missing blackslashingCédric Ollivier1-1/+1
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-leguer-sig_apps-run/2/console Change-Id: I5cd41f2da49e59acd6bd06ded0c4e2178e7402b7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 0b79c61af319c6f3beddf79e586e51e4675f3fc1)
2021-03-21Add sig_nodeCédric Ollivier2-0/+24
It runs sig-node tests which are not included in K8s conformance. It adds 38 single tests. Change-Id: Idbd172517e705402632a7fadb4d4639b5f1ee2d1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 56da8a0382889fa0558da48375d2319bfedd39a4)
2021-03-21Add sig_cluster_lifecycleCédric Ollivier2-0/+17
It runs sig-cluster-lifecycle tests which are not included in K8s conformance. It adds 4 single tests. Change-Id: I834523d753eec9bd570b8d4cc1cf3550b4a19bab Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f3d390ddba5021ce45dd763e2bf953994a487337)
2021-03-21Add sig_instrumentationCédric Ollivier2-0/+17
It runs sig-instrumentation tests which are not included in K8s conformance. It adds 6 single tests. Change-Id: Ic320f3bc7daed0aa2846a7ea672f13f3bec45f12 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 39511d8576f49b578882757fcc1e33a4bb2c9832)
2021-03-21Add sig_schedulingCédric Ollivier2-0/+18
It runs sig-scheduling tests which are not included in K8s conformance. It adds 19 single tests. Change-Id: I9b34c327cc1079e401771b1fa77a6baeef24251c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8bfdc8569b9eaffb96562e7753423d3ded218ed4)
2021-03-21Add sig_cliCédric Ollivier2-0/+17
It runs sig-cli tests which are not included in K8s conformance. It adds 34 single tests. Change-Id: I4beaf9d08d151d06fc3f3c05770c51dcc68f762a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 6eaf848638d7a1a7b4c4324837da9b8562ec7136)
2021-03-21Add sig-appsCédric Ollivier2-2/+22
It runs sig-apps tests which are not included in K8s conformance. Change-Id: Ic02e75b94e6cd923f05af16eb98706599fd26f59 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 985b7955d065065d191702f1355d7ca114f0c827)
2021-03-21Add sig-authCédric Ollivier2-0/+19
It runs sig-auth tests which are not included in K8s conformance. Change-Id: I5684dbcb5d61866f5ae1cb483f3a119b8d3c7807 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit b5c7bc0107a443722004ecced91118042965f327)
2021-03-21Add sig-api-machineryCédric Ollivier2-4/+21
It runs sig-api-machinery which are not included in K8s conformance. It adds 97 single tests. Change-Id: I76949b75e2d6c59d21f459f8fc358e806af6f23d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9c08bb471502b80df2470e577376f25400913939)
2021-03-19Provide support for air gapped env for rallySylvain Desbureaux5-80/+103
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for rally test case. There's two ways for providing the repository mirrors: - Give an environment variable (`MIRROR_REPO`) which gives a repository mirro with all needed images. - Gives an environment variable per needed repo: - `DOCKERHUB_REPO` for DockerHub repository (`docker.io`) - `GCR_REPO` for Google Cloud repository (`gcr.io`) - `K8S_GCR_REPO` for Kubernetes repository (`k8s.gcr.io`) Needed images list has also been extracted so Kubernetes administrator can easily upload these images to the mirror if the mirror also doesn't have access to Internet. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: I2ea6622b79f7e3c3c63c1441c4dab48e9bc4fb1a (cherry picked from commit b26ac308d4d441195fcd804d4d0ff36356fa8a90)
2021-03-15Fix k8s.gcr.io/etcd tagsCédric Ollivier1-1/+2
Kali is used to verify both v1.18 and v1.19 which respectively ask for k8s.gcr.io/etcd:3.4.3 and k8s.gcr.io/etcd:3.4.13-0. Co-Authored-By: Dominique Lagadec <dominique.lagadec@ericsson.com> Change-Id: I739202b0e58352fd3308d705807910cd1575d28f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-02-19Rename sig_network_feature -> sig_network_featuresCédric Ollivier2-11/+12
It also takes into account the last results (SCTP seems falky): https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_network-run/1/console https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_network_feature-run/3/console https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_storage-run/1/console Change-Id: I7d4b11462da33d20fd5ed00d870f1099b70f437b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 08558d2f47f7400176ad678ceaaff1b5be0b931c) (cherry picked from commit dcb442250783d97b2bbfc6e5671a18cc90106792)
2021-02-19Run rally firstCédric Ollivier1-12/+12
After all tests are based on e2e tests. Change-Id: I727f40255ccc165f9a7da7fa7bdff1a629614d21 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9e132759f6186672dd23dd31fac511bec128d386)
2021-02-19Add sig-storageCédric Ollivier1-0/+18
It runs the remaining sig-storage testing excluding the next labels: Conformance (part of k8s_conformance), all features, all drivers and Slow. Change-Id: I486d8ecdfc46287f4774ba8acaa64d47fb1da287 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8e6f69afa84ae428a013debcc7329be26a475695)
2021-02-19Add sig_networkCédric Ollivier2-0/+18
It runs the remaining sig-network testing excluding the next labels: Conformance (part of k8s_conformance), all features and load.balancer. Change-Id: I708ff4690205196e5f69382890a39ea4fd5b6942 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 80f0edf3194493620d1cfab79efbcdc8a136d799)
2021-02-19Add sig_network_featureCédric Ollivier2-0/+28
It verifies a couple of features by running a subset of e2e tests: Ingress, Networking-IPv4, Networking-Performance, PerformanceDNS, SCTP and NoSNAT. It excludes by default NetworkPolicy, Networking-IPv6, IPv6DualStackAlphaFeature and Example. A couple of features may be automatically skipped like NEG and kubemci. Change-Id: I6efac39bb99c1deb8efe8e82d976d69d6540a9d1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit e0d151ccaf4b267682feec0d38e8b7de3eb3d61c)
2021-02-17Move former wiki to README.mdCédric Ollivier1-0/+118
https://wiki.opnfv.org/pages/viewpage.action?pageId=20742412 Change-Id: I75a9303b2ee5b2df66e0169b37408bb9ad881d5a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 15f754678e4c8734c4600fea94fcbe66f2584515) (cherry picked from commit a33e2fe3956ce5dcf0893454131121af8e73f863)
2021-02-10Upgrade system firstCédric Ollivier1-1/+2
Change-Id: I0120e6a368b033f2fb47257ff9dcc3fd8a58d243 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9ebe47c8458c99066fd71a65007fff923004d209)
2021-02-04Allow parameterizing K8s versionsCédric Ollivier2-15/+18
ONAP integration team asks to support all K8s versions. A couple of OPNFV releases will allow testing 2 K8s versions. Change-Id: Idb43cff0cc03cc9c17b290f7f1867e2c380c6319 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 0163bb977604a32a0c46e10a97313ce0feea7b5f)
2021-01-10Update cnf-conformance to v0.9.19Cédric Ollivier2-7/+17
It also adds configs needed by resilience. https://github.com/cncf/cnf-conformance/releases/tag/v0.9.19 Change-Id: I94b1efc47c5ceef96bc8879a6c71f92db1f63bd2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 7385caf54c27b471bda8599d491aa3aee21b7b89)
2020-12-10Use pip==20.2.4 in tox venvCédric Ollivier3-2/+3
pip 20.3 is disruptive and would lead to big changes in stable branches. Change-Id: Ic3770f4098c52b0d26bc62291c9e79e8ee81d107 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 0c99749bb65cadd0142c90cd73e13802d48cb7f6)
2020-11-22Allow offline testing via xrally_kubernetesCédric Ollivier3-2/+36
It locally patches xrally_kubernetes before [1] is merged upstream. It adds docker.io/appropriate/curl:edge in download_images.sh too. [1] https://github.com/xrally/xrally-kubernetes/pull/51 Co-Authored-By: Tamas Lendvay <tamas.lendvay@nokia.com> Change-Id: I82ea2c777cd4d8f24c38dbf32d66081846e3e9af Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit ea2715240015a482c5321d309257d801a412f02d)
2020-11-22Reduce cnf-conformance criteriaCédric Ollivier1-1/+1
It's considered as passed vs 95 out of 105 [1]. [1] http://artifacts.opnfv.org/functest-kubernetes/ACI2B7JDLFHI/functest-kubernetes-opnfv-functest-kubernetes-cnf-kali-cnf_conformance-run-16/functest-kubernetes.log Change-Id: Ib16927cf269784daba1085a3e97fb611e869928f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 53b16e8f00e5a60db2e6db7f375bd06f184a5cc8)
2020-11-21Update cnf_conformance to v0.9.4Cédric Ollivier3-36/+32
It removes all hacks bypassing a few former bugs (e.g. [1]) and adapts to the new cnf-conformance result output logic. It runs all workload tests as it's now proposed (k8s conformance is already part of smoke) and select the latest coredns cnf-conformance.yml taking latest helm repo changes into account. It sets criteria 95 as all steps are considered as passed and a overall result 100/105. [1] https://github.com/cncf/cnf-conformance/issues/388 Change-Id: I95b7889b21b86961e7a1d0c84b13e0ae367cae44 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 854f164998793000fcf8019214b259b7283f5194)
2020-11-20Skip cnf_conformance till helm repos are fixedCédric Ollivier1-0/+1
Change-Id: Ie5b9aff6f56ca5cfd8501f142969f8a6246c4151 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 30a5a084fa9e0c8677d70d8b1d7a696ea06b49b9)
2020-11-15Add timeout as needed by Gitlab CI/CDCédric Ollivier3-0/+3
Gitlab CI/CD sets 1h as default timeout. The timeout is valid for both test cases even if only k8s_conformance reaches 1h (XtestingCI design). Change-Id: Ieaef1cefb3dfc7f9ca47b1ce8ec082409fa52604 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f38caec124b925f8c633f9d8b4474bd1085638a2)
2020-11-15Split Gates in a dedicated site.ymlCédric Ollivier2-34/+65
Change-Id: Iae0e61696987696fda3d9da05d75c5d87908b92a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-23Split kube_bench_master and kube_bench_node in site.cntt.ymlCédric Ollivier1-1/+2
Change-Id: Ifb9a5ac2fac7286d5f3f5e677fc2a7ecda6ade25 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 687723872c04c4b889cf4b529f6534776b16908d)
2020-09-23Stop hardcoding ims- as generate_nameCédric Ollivier2-2/+10
Change-Id: I3ea22a4050ff1eb609cffb61edc41c49fab44366 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 3a3bd294ea8101c22896d8601fe2723861f73124)
2020-09-20Redirect helm false warnings to logsCédric Ollivier1-2/+2
Helm fails when detecting arch if qemu. time="2020-09-17T16:25:15Z" level=error msg="failure getting variant" error="getCPUInfo for pattern: Cpu architecture: not found" https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-arm64-latest-helm_vims-run/1/console Change-Id: Ida37062a780f9e2acf60035bf2d41fb8fbb61173 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit aed4dd8a97dbfcb9d0558f2df8f8ef8696f2c421)
2020-09-16Add helm_vims in site.cntt.ymlCédric Ollivier1-0/+1
https://github.com/cntt-n/CNTT/pull/2022 Change-Id: Icb4bdfa0c1ecca9d0c5b4d5beef7ff4c5edb289e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 63476e09e51c5c847df47b0855ee9d1b0ea1add1)
2020-09-16Publish helm_vims also based on clearwater-dockerCédric Ollivier30-48/+892
It leverages helm rather than kubectl as proposed by k8s_vims. A new abstract class Vims is added to factorize code between both testcases. Change-Id: Ie5de8d62e25e74f73f8e32167228a08e82989abd Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 32dfc10f56ead9abea8798e1ce16fe87652ca480)
2020-09-16Increase timeout in vIMSCédric Ollivier1-1/+1
It depends on bandwidth. 30 minutes should be enough in all scenarios Change-Id: Ieaad053c8f8a270e5da08275de217b15fdf17f44 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f6e86e451a0f5bf0e9775482c487d86249b935bd)
2020-09-14Don't run disruptive hunter checksCédric Ollivier1-1/+1
Change-Id: I52cb8303950269946774546cf8e413166c70a33c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 4fa706f4b37d655f2239ad9381df8041e0701142)
2020-09-13Split kube-bench master and nodeCédric Ollivier5-23/+69
The former deployment asked for all-in-one. Change-Id: I12e470cec9e82b82c6f3ea5ff2431087f5deb9be Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit bced94b6fe24c7e939fb22834deb77477e4a9bb9)
2020-09-13self.details must be a dictCédric Ollivier1-2/+2
Change-Id: I4f65a9eeb7eda471371668db9abfa49e2875c5b0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit b866c73d70079fdb52e5fc999f49a32d2af82349)
2020-09-12Improve kube_bench outputCédric Ollivier3-0/+21
It also fills self.details. Change-Id: Ie73215ebcbd34de9d457fd364de4ab9cbdf64319 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 823b3a005ace9372a080421b2ae19152505201e4)
2020-09-12Enhance kube-hunter result postprocessingCédric Ollivier3-4/+66
It fills self.details and checks if the test case passes according to criteria (severity = high by default) Change-Id: Ib20779b4b5dca078c65b546c8703bc99856c6f41 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 4f197110710e9f148eae4533792e8e7e2d72f053)
2020-09-12Avoid running VNF testing if deployment failedCédric Ollivier1-3/+7
Change-Id: Id7642e47ec2cc438c5fe3a0ec9447f72b358adab Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 3afa7d700eebf2bfcaabab7bdd3aa09198ba4fe0)
2020-09-12Add clusterIP: None in ims svc.yamlCédric Ollivier11-0/+11
Change-Id: I0709e83f32261af232c20f94c1edef7423e05527 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 70d96ca61f1f11aafcf9e6975e61bce41febd371)
2020-09-10Remove clusterIP: None in ims svc.yamlCédric Ollivier11-11/+0
It now works on minikube in addition to Functest gates. Change-Id: I314989516f81dcddb3e615fe23480135f1d8cf9a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 757fcb55867aa8c0545543a87ec9741827e9ab50)
2020-09-10Allow setting e2e non blocking taints via env varsCédric Ollivier1-0/+3
Change-Id: I8132769ea23a376b2cae8b93beed27f5e66b9207 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit d4af425da9a60c0eb8e80cb74972b99e3deadeb8)
2020-09-10Add quay.io/coreos/etcd:v2.2.5 in download_images.shCédric Ollivier1-0/+1
Change-Id: I4b5da162dcb1efff810514498a20250c05afb3ba Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit ef4ef77dfda94d02e2d933318c1b0072da224e43)
2020-09-09Publish CNTT RC2 playbookCédric Ollivier1-0/+28
It runs the mandatory CNTT test cases [1]. [1] https://github.com/cntt-n/CNTT/commit/6af033c254727b7405fbfb2efbaa023e2b8ba876 Change-Id: I87d381c78049f7309c564791ed701cf446cc85d2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 463f233c0abb77f6758e7c39fa05b5c118fbaf1f)