aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2021-03-21Add sig-api-machineryCédric Ollivier2-4/+21
It runs sig-api-machinery which are not included in K8s conformance. It adds 97 single tests. Change-Id: I76949b75e2d6c59d21f459f8fc358e806af6f23d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9c08bb471502b80df2470e577376f25400913939)
2021-03-19Provide support for air gapped env for rallySylvain Desbureaux5-79/+102
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for rally test case. There's two ways for providing the repository mirrors: - Give an environment variable (`MIRROR_REPO`) which gives a repository mirro with all needed images. - Gives an environment variable per needed repo: - `DOCKERHUB_REPO` for DockerHub repository (`docker.io`) - `GCR_REPO` for Google Cloud repository (`gcr.io`) - `K8S_GCR_REPO` for Kubernetes repository (`k8s.gcr.io`) Needed images list has also been extracted so Kubernetes administrator can easily upload these images to the mirror if the mirror also doesn't have access to Internet. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: I2ea6622b79f7e3c3c63c1441c4dab48e9bc4fb1a (cherry picked from commit b26ac308d4d441195fcd804d4d0ff36356fa8a90)
2021-03-15Fix k8s.gcr.io/etcd tag (3.4.13-0)Cédric Ollivier1-1/+1
Change-Id: Idbc520f1b196fd9d66dc57d32a543a6af7b73598 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-02-18Rename sig_network_feature -> sig_network_featuresCédric Ollivier2-11/+12
It also takes into account the last results (SCTP seems falky): https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_network-run/1/console https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_network_feature-run/3/console https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_storage-run/1/console Change-Id: I7d4b11462da33d20fd5ed00d870f1099b70f437b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 08558d2f47f7400176ad678ceaaff1b5be0b931c)
2021-02-18Run rally firstCédric Ollivier1-12/+12
After all tests are based on e2e tests. Change-Id: I727f40255ccc165f9a7da7fa7bdff1a629614d21 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9e132759f6186672dd23dd31fac511bec128d386)
2021-02-18Add sig-storageCédric Ollivier1-0/+18
It runs the remaining sig-storage testing excluding the next labels: Conformance (part of k8s_conformance), all features, all drivers and Slow. Change-Id: I486d8ecdfc46287f4774ba8acaa64d47fb1da287 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8e6f69afa84ae428a013debcc7329be26a475695)
2021-02-18Add sig_networkCédric Ollivier2-0/+18
It runs the remaining sig-network testing excluding the next labels: Conformance (part of k8s_conformance), all features and load.balancer. Change-Id: I708ff4690205196e5f69382890a39ea4fd5b6942 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 80f0edf3194493620d1cfab79efbcdc8a136d799)
2021-02-18Add sig_network_featureCédric Ollivier2-0/+28
It verifies a couple of features by running a subset of e2e tests: Ingress, Networking-IPv4, Networking-Performance, PerformanceDNS, SCTP and NoSNAT. It excludes by default NetworkPolicy, Networking-IPv6, IPv6DualStackAlphaFeature and Example. A couple of features may be automatically skipped like NEG and kubemci. Change-Id: I6efac39bb99c1deb8efe8e82d976d69d6540a9d1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit e0d151ccaf4b267682feec0d38e8b7de3eb3d61c)
2021-02-17Move former wiki to README.mdCédric Ollivier1-0/+118
https://wiki.opnfv.org/pages/viewpage.action?pageId=20742412 Change-Id: I75a9303b2ee5b2df66e0169b37408bb9ad881d5a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 15f754678e4c8734c4600fea94fcbe66f2584515)
2021-02-10Upgrade system firstCédric Ollivier1-1/+2
Change-Id: I0120e6a368b033f2fb47257ff9dcc3fd8a58d243 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9ebe47c8458c99066fd71a65007fff923004d209)
2021-02-04Allow parameterizing K8s versionsCédric Ollivier2-15/+18
ONAP integration team asks to support all K8s versions. A couple of OPNFV releases will allow testing 2 K8s versions. Change-Id: Idb43cff0cc03cc9c17b290f7f1867e2c380c6319 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 0163bb977604a32a0c46e10a97313ce0feea7b5f)
2021-01-10Update cnf-conformance to v0.9.19Cédric Ollivier2-7/+17
It also adds configs needed by resilience. https://github.com/cncf/cnf-conformance/releases/tag/v0.9.19 Change-Id: I94b1efc47c5ceef96bc8879a6c71f92db1f63bd2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 7385caf54c27b471bda8599d491aa3aee21b7b89)
2021-01-10Switch to stable-1.20Cédric Ollivier1-1/+1
latest-1.20 is the last rc. Change-Id: I2587d1157c9274358f91e38f8991a349d4fc389a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-12-11Update to latest Kubernetes releaseCédric Ollivier1-1/+1
Change-Id: Id2c05f8b425eeba1924bc2bc2688233beeb7457b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8f70187ec85781e141b40dce050dba24d8134fe2)
2020-12-10Use pip==20.2.4 in tox venvCédric Ollivier3-2/+3
pip 20.3 is disruptive and would lead to big changes in stable branches. Change-Id: Ic3770f4098c52b0d26bc62291c9e79e8ee81d107 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 0c99749bb65cadd0142c90cd73e13802d48cb7f6)
2020-11-22Allow offline testing via xrally_kubernetesCédric Ollivier3-2/+37
It locally patches xrally_kubernetes before [1] is merged upstream. It adds docker.io/appropriate/curl:edge in download_images.sh too. [1] https://github.com/xrally/xrally-kubernetes/pull/51 Co-Authored-By: Tamas Lendvay <tamas.lendvay@nokia.com> Change-Id: I82ea2c777cd4d8f24c38dbf32d66081846e3e9af Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit ea2715240015a482c5321d309257d801a412f02d)
2020-11-22Reduce cnf-conformance criteriaCédric Ollivier1-1/+1
It's considered as passed vs 95 out of 105 [1]. [1] http://artifacts.opnfv.org/functest-kubernetes/ACI2B7JDLFHI/functest-kubernetes-opnfv-functest-kubernetes-cnf-kali-cnf_conformance-run-16/functest-kubernetes.log Change-Id: Ib16927cf269784daba1085a3e97fb611e869928f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 53b16e8f00e5a60db2e6db7f375bd06f184a5cc8)
2020-11-21Update cnf_conformance to v0.9.4Cédric Ollivier3-36/+32
It removes all hacks bypassing a few former bugs (e.g. [1]) and adapts to the new cnf-conformance result output logic. It runs all workload tests as it's now proposed (k8s conformance is already part of smoke) and select the latest coredns cnf-conformance.yml taking latest helm repo changes into account. It sets criteria 95 as all steps are considered as passed and a overall result 100/105. [1] https://github.com/cncf/cnf-conformance/issues/388 Change-Id: I95b7889b21b86961e7a1d0c84b13e0ae367cae44 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 854f164998793000fcf8019214b259b7283f5194)
2020-11-20Skip cnf_conformance till helm repos are fixedCédric Ollivier1-0/+1
Change-Id: Ie5b9aff6f56ca5cfd8501f142969f8a6246c4151 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 30a5a084fa9e0c8677d70d8b1d7a696ea06b49b9)
2020-11-12Add timeout as needed by Gitlab CI/CDCédric Ollivier3-0/+3
Gitlab CI/CD sets 1h as default timeout. The timeout is valid for both test cases even if only k8s_conformance reaches 1h (XtestingCI design). Change-Id: Ieaef1cefb3dfc7f9ca47b1ce8ec082409fa52604 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f38caec124b925f8c633f9d8b4474bd1085638a2)
2020-11-09Split Gates in a dedicated site.ymlCédric Ollivier2-34/+65
Change-Id: I8b31468e8244e63feed163134f25461d019dbdea Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-10-06Update upper-constraintsCédric Ollivier2-2/+2
It takes the new requirements just published into account. Change-Id: Iea0ba4113b44d3056f2f8f1b1516271d33e84e32 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-10-04Fix BRANCH and OPNFV_TAG in stable/leguerCédric Ollivier2-3/+3
Change-Id: I82797911296ac68701bbe1ced37c3c843360d92c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-10-04Prepare Functest Kubernetes Leguer containersCédric Ollivier9-45/+51
Change-Id: I91b31f2af0c2265cdda5715ae152558ffcbe003b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-10-04Set default branch in .gitreviewCédric Ollivier1-0/+1
Change-Id: I834255837e15487bc6ae52b0c2dc805f84fb82b3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-23Split kube_bench_master and kube_bench_node in site.cntt.ymlCédric Ollivier1-1/+2
Change-Id: Ifb9a5ac2fac7286d5f3f5e677fc2a7ecda6ade25 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-22Stop hardcoding ims- as generate_nameCédric Ollivier2-2/+10
Change-Id: I3ea22a4050ff1eb609cffb61edc41c49fab44366 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-20Redirect helm false warnings to logsCédric Ollivier1-2/+2
Helm fails when detecting arch if qemu. time="2020-09-17T16:25:15Z" level=error msg="failure getting variant" error="getCPUInfo for pattern: Cpu architecture: not found" https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-arm64-latest-helm_vims-run/1/console Change-Id: Ida37062a780f9e2acf60035bf2d41fb8fbb61173 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-16Add helm_vims in site.cntt.ymlCédric Ollivier1-0/+1
https://github.com/cntt-n/CNTT/pull/2022 Change-Id: Icb4bdfa0c1ecca9d0c5b4d5beef7ff4c5edb289e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-15Publish helm_vims also based on clearwater-dockerCédric Ollivier30-48/+892
It leverages helm rather than kubectl as proposed by k8s_vims. A new abstract class Vims is added to factorize code between both testcases. Change-Id: Ie5de8d62e25e74f73f8e32167228a08e82989abd Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-15Increase timeout in vIMSCédric Ollivier1-1/+1
It depends on bandwidth. 30 minutes should be enough in all scenarios Change-Id: Ieaad053c8f8a270e5da08275de217b15fdf17f44 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-14Don't run disruptive hunter checksCédric Ollivier1-1/+1
Change-Id: I52cb8303950269946774546cf8e413166c70a33c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-13Split kube-bench master and nodeCédric Ollivier5-23/+69
The former deployment asked for all-in-one. Change-Id: I12e470cec9e82b82c6f3ea5ff2431087f5deb9be Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-13self.details must be a dictCédric Ollivier1-2/+2
Change-Id: I4f65a9eeb7eda471371668db9abfa49e2875c5b0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-12Improve kube_bench outputCédric Ollivier3-0/+21
It also fills self.details. Change-Id: Ie73215ebcbd34de9d457fd364de4ab9cbdf64319 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-12Enhance kube-hunter result postprocessingCédric Ollivier3-4/+66
It fills self.details and checks if the test case passes according to criteria (severity = high by default) Change-Id: Ib20779b4b5dca078c65b546c8703bc99856c6f41 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-12Avoid running VNF testing if deployment failedCédric Ollivier1-3/+7
Change-Id: Id7642e47ec2cc438c5fe3a0ec9447f72b358adab Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-12Add clusterIP: None in ims svc.yamlCédric Ollivier11-0/+11
Change-Id: I0709e83f32261af232c20f94c1edef7423e05527 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-10Remove clusterIP: None in ims svc.yamlCédric Ollivier11-11/+0
It now works on minikube in addition to Functest gates. Change-Id: I314989516f81dcddb3e615fe23480135f1d8cf9a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-10Allow setting e2e non blocking taints via env varsCédric Ollivier1-0/+3
Change-Id: I8132769ea23a376b2cae8b93beed27f5e66b9207 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-10Add quay.io/coreos/etcd:v2.2.5 in download_images.shCédric Ollivier1-0/+1
Change-Id: I4b5da162dcb1efff810514498a20250c05afb3ba Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-09Publish CNTT RC2 playbookCédric Ollivier1-0/+25
It runs the mandatory CNTT test cases [1]. [1] https://github.com/cntt-n/CNTT/commit/6af033c254727b7405fbfb2efbaa023e2b8ba876 Change-Id: I87d381c78049f7309c564791ed701cf446cc85d2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-08Add helper to download all imagesCédric Ollivier3-0/+68
It eases offline testing. The default command leverages kind as in-use in Functest. Change-Id: I66a20ede5e857f80cf49b9b708843034bc3b9904 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-08Set all image tagsCédric Ollivier14-37/+37
It allows offline testing via xrally_kubernetes, k8s_vims, kube_bench and kube_hunter. Change-Id: I3084abec19f06a894d0083ecb3ed61882eddd785 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-08Remove latest in all imagesCédric Ollivier12-12/+12
The Kubernetes default pull policy is IfNotPresent unless the image tag is :latest in which case the default policy is Always. IfNotPresent causes the Kubelet to skip pulling an image if it already exists [1] [1] https://kind.sigs.k8s.io/docs/user/quick-start/ Change-Id: I83dac6165d2bbef165ca852dd03e5b76a5356f2f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-08Remove all imagePullPolicy: Always in ims depl yamlCédric Ollivier10-10/+0
It forbids offline testing. Change-Id: I6790f64fd1382671300374b339ae038ef015cfab Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-07Override default Xtesting logs in cnf-conformanceCédric Ollivier1-0/+5
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-latest-cnf_conformance-run/7/console Change-Id: Ia6845525e62274f7792b4cb1909f36204fb51418 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-06Decreate cnf-conformance criteriaCédric Ollivier1-1/+1
conf-confirmance is beta. Let's decrease criteria to allow gating [1]. [1] https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-latest-cnf_conformance-run/4/console Change-Id: I77068fd027b2ae042e0ff9574ed608c8a3d77ade Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-06Slightly generalize ginkgo paramsCédric Ollivier1-4/+2
Change-Id: I51b013056bb5b88ef2e1de31e9171129e8ce4475 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-06Publish k8s_quick as proposed by sonobuoyCédric Ollivier2-1/+16
https://github.com/vmware-tanzu/sonobuoy Change-Id: I7b1777cf7bc0d6cd04ea5cccc08b7c29887ae874 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>