aboutsummaryrefslogtreecommitdiffstats
path: root/functest_kubernetes
AgeCommit message (Collapse)AuthorFilesLines
2021-10-07Set defaut provider to local backCédric Ollivier1-1/+2
It apply the same rules as sonuoboy. Any user is free to set skeleton or something else via testcases.yml. Change-Id: Id999dfd5fbcf9bde9430e0f35c63c41e80c8044f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit b100b1254e06719bf1c779b9d53d0ac2da67fca4) (cherry picked from commit 329b58f2b897245dd91aedb280fec8db23ae25e1)
2021-09-10Fallback to item calculationCédric Ollivier1-0/+8
There is no final result if simple tag is called. Change-Id: If86c5115e917e5790799fac4308cf08d57e87905 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit e2f255aaaddeb0b7bebd7a158b535aa0ecdf55e0)
2021-07-07Print cnf_testsuite console on exceptionsCédric Ollivier1-0/+4
Change-Id: I3c20993e1cbb644546ac89a8b4b7d1cfd98b0a80 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f6d5010cd34db6cd639a7fcf5b6183aa117acacf)
2021-06-20Allow passing extra opts to e2e.testCédric Ollivier1-0/+12
E2E_TEST_OPTS now allows setting runtime opts. For instance, Functest K8s gates ask for -container-runtime containerd Change-Id: I4f6e1caf57590b16d0c01324dadb040f67f3cee4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 6ec9d5167032ed82e32cde592f98a4bcffca4c00)
2021-05-25Update cnf_testsuite to v0.11.2Cédric Ollivier1-22/+11
https://github.com/cncf/cnf-testsuite/releases/tag/v0.11.2 Change-Id: Id3a5ea98260859f7fe5aba6ecf02d09905610dda Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit c35c6d443eb93db318c3436d995809dfe7e8389f)
2021-05-10Benchmark Kubernetes Networking PerformanceCédric Ollivier2-0/+66
https://github.com/kubernetes/perf-tests/tree/master/network/benchmarks/netperf Change-Id: I6facd567f1c52c5949b53484a1fb107dcf34d622 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 3b5c1b115e234d636cb4f2a17d27ced872fee924)
2021-05-03Fix cnf-conformance results filenameCédric Ollivier1-1/+1
It also decreases criteria to 1 due to massive changes (dockerd). Change-Id: Ic420139e6355debae183a77d6b0c574801b05076 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-05-03Update cnf-conformance to cnf-testsuiteCédric Ollivier1-10/+10
https://github.com/cncf/cnf-conformance https://github.com/cncf/cnf-testsuite Change-Id: I77983e0f04781a92b34afcf82f21366384782964 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 959da00d3e1c4d406bc59995c0ea93add1ec48d3)
2021-04-21Mock os.makedirsCédric Ollivier1-2/+7
Change-Id: I5f9fd24421727cea474715f502670ae6f2c80b76 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 32d2ed700c449ba33a4060f55c2adadc2e76b070)
2021-04-03Fix etcd image repository in etcd-depl.yamlCédric Ollivier1-1/+1
Change-Id: I4447b4030b141b691b3a2cdf05c97c79de9eb72a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit b134d6735e722261bfe10c1ba83112e9cf95f85c)
2021-04-02Fix name in chronos-depl.yamlCédric Ollivier1-1/+1
Change-Id: Ia6972a216daaa9d807058b34b55615cab2c09b64 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 2eaa659eee7c83fd9221f88a207514e8e909943f)
2021-04-02Properly manage e2e parallel testingCédric Ollivier1-5/+7
It sets nodes=25 to run all tests in parallel. It also beautifies all testcases.yml Change-Id: I3ecd8899e957c0efbef34ab0409c2b4d557c9425 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 4004d776b01a1b038df52ca45b18430da1b6b1f5)
2021-04-02Rather use skeleton instead of localCédric Ollivier1-1/+1
“skeleton”: cluster is accessed via the Kubernetes API and nothing else. https://kubernetes.io/blog/2019/03/22/kubernetes-end-to-end-testing-for-everyone/ Change-Id: Iea8b65b2ac55d40572c0078b7a9a956d1c54d62a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 17929b0d015034bad2796fb733627fe5b23bbb2d)
2021-04-02 Provide support for air gapped env for e2eSylvain Desbureaux1-0/+35
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for e2e test case. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: Iaa2960ad03aae1fff859d41bcd6ddf3176658f8e (cherry picked from commit 27dbb7c14bac45a9af8d18e1913a2be6815cea43)
2021-04-02Export GINKGO_PARALLEL=yCédric Ollivier1-1/+3
It would speedup the testing. Change-Id: I084bcf0d44a060f65eb9549eab8dcbbf5a4cc9d5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 65d860c6ee36641ad7d248f7dc8a6f63f3173301)
2021-04-02Provide support for air gapped env for imsSylvain Desbureaux24-65/+80
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for ims test cases. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: I3b5e78fcf43ca7af6bcfd2d87fa4bb0705373697 (cherry picked from commit 9c95b5f46e57365c1f6e7e5c9073f7fe2ad48a66)
2021-03-25Provide support for air gapped env for securitySylvain Desbureaux4-4/+10
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for security test cases. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: I699d065ee691596c4a5ccf06c22ea76ef00fe497 (cherry picked from commit af623f6b45c9357b0e33210f54db64b270d162bf)
2021-03-19Provide support for air gapped env for rallySylvain Desbureaux4-80/+83
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for rally test case. There's two ways for providing the repository mirrors: - Give an environment variable (`MIRROR_REPO`) which gives a repository mirro with all needed images. - Gives an environment variable per needed repo: - `DOCKERHUB_REPO` for DockerHub repository (`docker.io`) - `GCR_REPO` for Google Cloud repository (`gcr.io`) - `K8S_GCR_REPO` for Kubernetes repository (`k8s.gcr.io`) Needed images list has also been extracted so Kubernetes administrator can easily upload these images to the mirror if the mirror also doesn't have access to Internet. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: I2ea6622b79f7e3c3c63c1441c4dab48e9bc4fb1a (cherry picked from commit b26ac308d4d441195fcd804d4d0ff36356fa8a90)
2021-03-15Fix k8s.gcr.io/etcd tagsCédric Ollivier1-1/+2
Kali is used to verify both v1.18 and v1.19 which respectively ask for k8s.gcr.io/etcd:3.4.3 and k8s.gcr.io/etcd:3.4.13-0. Co-Authored-By: Dominique Lagadec <dominique.lagadec@ericsson.com> Change-Id: I739202b0e58352fd3308d705807910cd1575d28f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-01-10Update cnf-conformance to v0.9.19Cédric Ollivier1-0/+6
It also adds configs needed by resilience. https://github.com/cncf/cnf-conformance/releases/tag/v0.9.19 Change-Id: I94b1efc47c5ceef96bc8879a6c71f92db1f63bd2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 7385caf54c27b471bda8599d491aa3aee21b7b89)
2020-11-22Allow offline testing via xrally_kubernetesCédric Ollivier1-0/+1
It locally patches xrally_kubernetes before [1] is merged upstream. It adds docker.io/appropriate/curl:edge in download_images.sh too. [1] https://github.com/xrally/xrally-kubernetes/pull/51 Co-Authored-By: Tamas Lendvay <tamas.lendvay@nokia.com> Change-Id: I82ea2c777cd4d8f24c38dbf32d66081846e3e9af Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit ea2715240015a482c5321d309257d801a412f02d)
2020-11-21Update cnf_conformance to v0.9.4Cédric Ollivier1-25/+19
It removes all hacks bypassing a few former bugs (e.g. [1]) and adapts to the new cnf-conformance result output logic. It runs all workload tests as it's now proposed (k8s conformance is already part of smoke) and select the latest coredns cnf-conformance.yml taking latest helm repo changes into account. It sets criteria 95 as all steps are considered as passed and a overall result 100/105. [1] https://github.com/cncf/cnf-conformance/issues/388 Change-Id: I95b7889b21b86961e7a1d0c84b13e0ae367cae44 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 854f164998793000fcf8019214b259b7283f5194)
2020-09-23Stop hardcoding ims- as generate_nameCédric Ollivier2-2/+10
Change-Id: I3ea22a4050ff1eb609cffb61edc41c49fab44366 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 3a3bd294ea8101c22896d8601fe2723861f73124)
2020-09-20Redirect helm false warnings to logsCédric Ollivier1-2/+2
Helm fails when detecting arch if qemu. time="2020-09-17T16:25:15Z" level=error msg="failure getting variant" error="getCPUInfo for pattern: Cpu architecture: not found" https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-arm64-latest-helm_vims-run/1/console Change-Id: Ida37062a780f9e2acf60035bf2d41fb8fbb61173 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit aed4dd8a97dbfcb9d0558f2df8f8ef8696f2c421)
2020-09-16Publish helm_vims also based on clearwater-dockerCédric Ollivier26-37/+859
It leverages helm rather than kubectl as proposed by k8s_vims. A new abstract class Vims is added to factorize code between both testcases. Change-Id: Ie5de8d62e25e74f73f8e32167228a08e82989abd Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 32dfc10f56ead9abea8798e1ce16fe87652ca480)
2020-09-16Increase timeout in vIMSCédric Ollivier1-1/+1
It depends on bandwidth. 30 minutes should be enough in all scenarios Change-Id: Ieaad053c8f8a270e5da08275de217b15fdf17f44 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f6e86e451a0f5bf0e9775482c487d86249b935bd)
2020-09-14Don't run disruptive hunter checksCédric Ollivier1-1/+1
Change-Id: I52cb8303950269946774546cf8e413166c70a33c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 4fa706f4b37d655f2239ad9381df8041e0701142)
2020-09-13Split kube-bench master and nodeCédric Ollivier3-16/+45
The former deployment asked for all-in-one. Change-Id: I12e470cec9e82b82c6f3ea5ff2431087f5deb9be Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit bced94b6fe24c7e939fb22834deb77477e4a9bb9)
2020-09-13self.details must be a dictCédric Ollivier1-2/+2
Change-Id: I4f65a9eeb7eda471371668db9abfa49e2875c5b0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit b866c73d70079fdb52e5fc999f49a32d2af82349)
2020-09-12Improve kube_bench outputCédric Ollivier2-0/+20
It also fills self.details. Change-Id: Ie73215ebcbd34de9d457fd364de4ab9cbdf64319 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 823b3a005ace9372a080421b2ae19152505201e4)
2020-09-12Enhance kube-hunter result postprocessingCédric Ollivier2-4/+64
It fills self.details and checks if the test case passes according to criteria (severity = high by default) Change-Id: Ib20779b4b5dca078c65b546c8703bc99856c6f41 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 4f197110710e9f148eae4533792e8e7e2d72f053)
2020-09-12Avoid running VNF testing if deployment failedCédric Ollivier1-3/+7
Change-Id: Id7642e47ec2cc438c5fe3a0ec9447f72b358adab Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 3afa7d700eebf2bfcaabab7bdd3aa09198ba4fe0)
2020-09-12Add clusterIP: None in ims svc.yamlCédric Ollivier11-0/+11
Change-Id: I0709e83f32261af232c20f94c1edef7423e05527 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 70d96ca61f1f11aafcf9e6975e61bce41febd371)
2020-09-10Remove clusterIP: None in ims svc.yamlCédric Ollivier11-11/+0
It now works on minikube in addition to Functest gates. Change-Id: I314989516f81dcddb3e615fe23480135f1d8cf9a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 757fcb55867aa8c0545543a87ec9741827e9ab50)
2020-09-10Allow setting e2e non blocking taints via env varsCédric Ollivier1-0/+3
Change-Id: I8132769ea23a376b2cae8b93beed27f5e66b9207 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit d4af425da9a60c0eb8e80cb74972b99e3deadeb8)
2020-09-10Add quay.io/coreos/etcd:v2.2.5 in download_images.shCédric Ollivier1-0/+1
Change-Id: I4b5da162dcb1efff810514498a20250c05afb3ba Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit ef4ef77dfda94d02e2d933318c1b0072da224e43)
2020-09-08Add helper to download all imagesCédric Ollivier1-0/+65
It eases offline testing. The default command leverages kind as in-use in Functest. Change-Id: I66a20ede5e857f80cf49b9b708843034bc3b9904 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9913456884a19525bae18226ef0a6233f00993b2)
2020-09-08Set all image tagsCédric Ollivier12-34/+34
It allows offline testing via xrally_kubernetes, k8s_vims, kube_bench and kube_hunter. Change-Id: I3084abec19f06a894d0083ecb3ed61882eddd785 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8d748b9efc9ed4ed3edb4eaaf09909e41fe29577)
2020-09-08Remove latest in all imagesCédric Ollivier11-11/+11
The Kubernetes default pull policy is IfNotPresent unless the image tag is :latest in which case the default policy is Always. IfNotPresent causes the Kubelet to skip pulling an image if it already exists [1] [1] https://kind.sigs.k8s.io/docs/user/quick-start/ Change-Id: I83dac6165d2bbef165ca852dd03e5b76a5356f2f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 44492e0da74473460ff6a9f1323c5df3a2af943e)
2020-09-08Remove all imagePullPolicy: Always in ims depl yamlCédric Ollivier10-10/+0
It forbids offline testing. Change-Id: I6790f64fd1382671300374b339ae038ef015cfab Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 6371491ee2bc134c60abaa6a1deec18b9ffe22e2)
2020-09-07Override default Xtesting logs in cnf-conformanceCédric Ollivier1-0/+5
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-latest-cnf_conformance-run/7/console Change-Id: Ia6845525e62274f7792b4cb1909f36204fb51418 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit e09c7eeab3a1bdc361769e469c0ec7a7b5718ebf)
2020-09-06Slightly generalize ginkgo paramsCédric Ollivier1-4/+2
Change-Id: I51b013056bb5b88ef2e1de31e9171129e8ce4475 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 0ec2aa15acbe7cd2b28d772165c22481b85bad79)
2020-09-04Allow skipping arm and arm64 if cnf-conformanceCédric Ollivier1-3/+16
cnf-conformance doesn't support this architectures. It also improve cleaning to publish artifacts [1]. [1] https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-cnf-latest-cnf_conformance-run/2/console Change-Id: Ie64e4a1b8b496a050a1798eaf4e5ef6397c011d0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 2cf6903c9a805b9c1b7702292bdf33b9485be319)
2020-09-04Add cnf-conformance in functest-kubernetes-cnfCédric Ollivier2-0/+104
It selects cnf-conformance v0.7.2-beta1 (current latest release) and the default CNF as proposed by [1]. The end user is free to override the default yml files on purpose (e.g. docker volumes). Tag (default: all) could be modified too via testcases.yaml [1] https://hackmd.io/@vulk/SkY54QnsU Change-Id: Iec2a830c441e523e067f6eb114ecb205c49c6b93 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 120daffd8d6d4be727d2ba52a76dbd8eacfe2628)
2020-08-30Increase cassandra timeoutsCédric Ollivier2-2/+2
homestead-prov sometimes fails reaching cassandra 30-08-2020 10:33:44.064 UTC ERROR base.py:327: Translating internal <class 'telephus.cassandra.ttypes.TimedOutException'> error into a 503 status code 30-08-2020 10:34:32.257 UTC ERROR base.py:327: Translating internal <class 'telephus.cassandra.ttypes.TimedOutException'> error into a 503 status code Change-Id: I48566e39dca3dc9644733a76140a76300bbfd84e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit e236b0f56f748c888df23eeaf65478967ad852f1)
2020-08-23Stop pushing all details in Test DBCédric Ollivier1-1/+1
It raises 413 Client Error: Request Entity Too Large for url [1] [1] https://build.opnfv.org/ci/job/functest-kubernetes-pi-ollivier-functest-kubernetes-benchmarking-latest-xrally_kubernetes_full-run/1/console Change-Id: I165f7aec53d8ce1e567491a5b5559f00b1fe4a4c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 1f720b0fabbcf162366160a4fa9a46807e53f8bc)
2020-08-23Set output logs in Rally testcasesCédric Ollivier1-0/+3
https://build.opnfv.org/ci/job/functest-kubernetes-pi-ollivier-functest-kubernetes-benchmarking-latest-xrally_kubernetes_full-run/1/console Change-Id: I13685d74ce9e286520af6f1e5cd6ce9dafc3d5ea Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 42c2bbed87779218a8fe43c4c5c3012c64bec17b)
2020-08-22Fix last hardcoded concurrenciesCédric Ollivier1-8/+8
Change-Id: Ib2b7bb6de8cd6524fcc52baf48ebf5c4835c396b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8a8f6eafc3c6268a3663852f7de8f0cc4c39b817)
2020-08-22Add xrally_kubernetes_fullCédric Ollivier2-60/+69
It iterates 10 times all operations run by xrally_kubernetes. Now times, concurrenty and count can be set via testcases.yaml. It creates a new containers functest-kubernetes-benchamrking as Functest. Change-Id: Ibfd493b987a359a3f16c2d8db0aef6b8e90b7e92 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit a0f76d884c1f13c27e906136d3418f12b5342fc4)
2020-08-17Add tempo waiting for ellis accountCédric Ollivier1-0/+1
Change-Id: I63c6af9d0e20aab426e7f875cdb4f86d206f49aa Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 3c7965bb37590eae9e968a530eaaa690ada903f2)