aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2021-05-03Reduce number of threadsCédric Ollivier1-11/+11
It passed successfully with 10 threads (more doesn't reduce the overall duration so much). Change-Id: I9013897883a40ebaf699c9a180f1bc8a6a34505c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-04-21Mock os.makedirsCédric Ollivier1-2/+7
Change-Id: I5f9fd24421727cea474715f502670ae6f2c80b76 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 32d2ed700c449ba33a4060f55c2adadc2e76b070)
2021-04-20Install setuptools_scm before xrallyCédric Ollivier1-0/+4
Else upper-constraints basically fails. Change-Id: I022f101e2ed44708afd0b88d47b77d2f7ca85458 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 75170fc0b528c7e182b8164d5905fcbaa6957faa)
2021-04-15Skip Network.should.set.TCP.CLOSE_WAIT.timeoutCédric Ollivier1-1/+2
https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-release/release-branch-jobs/1.21.yaml Change-Id: I9d8065fe84d3f6f5e74bf9b46bc8c85ad985c8b1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 16f4a0c29fcfd1c6dc577394d9627845f559f177)
2021-04-14Remove sig_node_serial as it's fully disruptiveCédric Ollivier2-25/+0
There is no other test when NoExecuteTaintManager is skipped. http://artifacts.opnfv.org/functest-kubernetes/T94WVHSMW8ZH/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-sig_scheduling_serial-run-6/sig_scheduling_serial/e2e.log Change-Id: I09a81ec973b26fc6de32df6788e73c7e529ecbf0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit a6dda83e6d6da08f776010a7cdebff77c3c7c2c0)
2021-04-14Skip SetHostnameAsFQDN in sig-nodeCédric Ollivier1-2/+3
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-sig_node-run/14/console Change-Id: I55c9d3404f9bca64354a6276881a71624281a59b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-04-08Disable Feature:StackdriverLogging in sig_instrumentationCédric Ollivier1-2/+4
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-sig_instrumentation-run/8/console Change-Id: I6634ec335b632ac6c7efb9b2078eb1c5f3d9be31 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-04-08Disable a few sig-auth featuresCédric Ollivier1-3/+5
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-sig_auth-run/8/console Change-Id: I8a1523ba293f40dc8533428f9d75b29560a817cc Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-04-04Update test case listCédric Ollivier1-0/+8
Change-Id: Ide5fdd5967ce11847f9672259b79b6068c228848 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit c1ea4de55608b931462cd0a8d2a3a8325e5888e5)
2021-04-04Add k8s_io and serial test casesCédric Ollivier1-6/+161
It completes the first sig test cases. Change-Id: I6b018177e93efe60f9de67f384e388d2a584536c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-04-04Merge sig_network_features in sig_networkCédric Ollivier2-28/+4
Change-Id: I33516a7b0f280355c40aa821738a0625e4fae93a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit adcbd42cabfa42a71410dd0c76ece0f367d338fd)
2021-04-03Spit conformance into 2 test casesCédric Ollivier2-0/+18
It hugely decreases the overall test duration. (10 + 15 << 120) Change-Id: I534f6d09956bb51c32ad35c5a1baff8ebbfabb3d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit a33e339a001d62352e15a3cc1546bb4d4d482e59)
2021-04-03Fix etcd image repository in etcd-depl.yamlCédric Ollivier1-1/+1
Change-Id: I4447b4030b141b691b3a2cdf05c97c79de9eb72a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit b134d6735e722261bfe10c1ba83112e9cf95f85c)
2021-04-02Fix name in chronos-depl.yamlCédric Ollivier1-1/+1
Change-Id: Ia6972a216daaa9d807058b34b55615cab2c09b64 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 2eaa659eee7c83fd9221f88a207514e8e909943f)
2021-04-02Run opnfv/functest-kubernetes-smoke sequentiallyCédric Ollivier2-0/+4
Now multithreading is enabled at test case level. Change-Id: I17a9bdb07216cd645483a0a893da502abee2ade0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit bbc8501437c1112d2e5c53281622d3dbe25b0d61)
2021-04-02Properly manage e2e parallel testingCédric Ollivier6-367/+366
It sets nodes=25 to run all tests in parallel. It also beautifies all testcases.yml Change-Id: I3ecd8899e957c0efbef34ab0409c2b4d557c9425 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 4004d776b01a1b038df52ca45b18430da1b6b1f5)
2021-04-02Rather use skeleton instead of localCédric Ollivier1-1/+1
“skeleton”: cluster is accessed via the Kubernetes API and nothing else. https://kubernetes.io/blog/2019/03/22/kubernetes-end-to-end-testing-for-everyone/ Change-Id: Iea8b65b2ac55d40572c0078b7a9a956d1c54d62a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 17929b0d015034bad2796fb733627fe5b23bbb2d)
2021-04-02 Provide support for air gapped env for e2eSylvain Desbureaux2-1/+42
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for e2e test case. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: Iaa2960ad03aae1fff859d41bcd6ddf3176658f8e (cherry picked from commit 27dbb7c14bac45a9af8d18e1913a2be6815cea43)
2021-04-02Export GINKGO_PARALLEL=yCédric Ollivier1-1/+3
It would speedup the testing. Change-Id: I084bcf0d44a060f65eb9549eab8dcbbf5a4cc9d5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 65d860c6ee36641ad7d248f7dc8a6f63f3173301)
2021-04-02Provide support for air gapped env for imsSylvain Desbureaux25-66/+82
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for ims test cases. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: I3b5e78fcf43ca7af6bcfd2d87fa4bb0705373697 (cherry picked from commit 9c95b5f46e57365c1f6e7e5c9073f7fe2ad48a66)
2021-03-31Disable sig_scheduling as serialCédric Ollivier2-19/+0
All remaining sig-scheduling tests are serial and for the time Functest doesn't run serial test cases. https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_scheduling-run/13/console Change-Id: I3f09077a1873b92120daa3a97cbb6f68558e2b4d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f28b4673a7b89f124d225e629f0c5267e6008c14)
2021-03-26Remove the useless Feature prefixCédric Ollivier1-1/+1
Change-Id: Ibec3e1cee170a91a4333b3c625ae39cd0c4bbe11 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-03-26Exlude Serial and Feature:Feature:StorageVersionAPICédric Ollivier1-12/+20
Serial asks to split the testcases into 2 runs. https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_api_machinery-run/7/console Change-Id: I58f9d9023f596141bbefae8339043f813a124854 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-03-26Blacklist all alpha testingCédric Ollivier1-11/+11
Change-Id: Ifb9edb0d70f285825eb2cfee05a53590367b2650 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-03-26Leverage py3-setuptools_scm as proposed by AlpineCédric Ollivier1-1/+1
Last version setuptools-scm asks for very new setuptools. https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-xrally_kubernetes-run/194/console Change-Id: Ib1f4b4b49c036e0018fd695b52013bccdaf4998f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-03-25Provide support for air gapped env for securitySylvain Desbureaux5-5/+11
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for security test cases. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: I699d065ee691596c4a5ccf06c22ea76ef00fe497 (cherry picked from commit af623f6b45c9357b0e33210f54db64b270d162bf)
2021-03-22Add missing blackslashingCédric Ollivier1-1/+1
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-leguer-sig_apps-run/2/console Change-Id: I5cd41f2da49e59acd6bd06ded0c4e2178e7402b7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 0b79c61af319c6f3beddf79e586e51e4675f3fc1)
2021-03-21Add sig_nodeCédric Ollivier2-0/+24
It runs sig-node tests which are not included in K8s conformance. It adds 38 single tests. Change-Id: Idbd172517e705402632a7fadb4d4639b5f1ee2d1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 56da8a0382889fa0558da48375d2319bfedd39a4)
2021-03-21Add sig_cluster_lifecycleCédric Ollivier2-0/+17
It runs sig-cluster-lifecycle tests which are not included in K8s conformance. It adds 4 single tests. Change-Id: I834523d753eec9bd570b8d4cc1cf3550b4a19bab Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f3d390ddba5021ce45dd763e2bf953994a487337)
2021-03-21Add sig_instrumentationCédric Ollivier2-0/+17
It runs sig-instrumentation tests which are not included in K8s conformance. It adds 6 single tests. Change-Id: Ic320f3bc7daed0aa2846a7ea672f13f3bec45f12 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 39511d8576f49b578882757fcc1e33a4bb2c9832)
2021-03-21Add sig_schedulingCédric Ollivier2-0/+18
It runs sig-scheduling tests which are not included in K8s conformance. It adds 19 single tests. Change-Id: I9b34c327cc1079e401771b1fa77a6baeef24251c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8bfdc8569b9eaffb96562e7753423d3ded218ed4)
2021-03-21Add sig_cliCédric Ollivier2-0/+17
It runs sig-cli tests which are not included in K8s conformance. It adds 34 single tests. Change-Id: I4beaf9d08d151d06fc3f3c05770c51dcc68f762a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 6eaf848638d7a1a7b4c4324837da9b8562ec7136)
2021-03-21Add sig-appsCédric Ollivier2-2/+22
It runs sig-apps tests which are not included in K8s conformance. Change-Id: Ic02e75b94e6cd923f05af16eb98706599fd26f59 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 985b7955d065065d191702f1355d7ca114f0c827)
2021-03-21Add sig-authCédric Ollivier2-0/+19
It runs sig-auth tests which are not included in K8s conformance. Change-Id: I5684dbcb5d61866f5ae1cb483f3a119b8d3c7807 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit b5c7bc0107a443722004ecced91118042965f327)
2021-03-21Add sig-api-machineryCédric Ollivier2-4/+21
It runs sig-api-machinery which are not included in K8s conformance. It adds 97 single tests. Change-Id: I76949b75e2d6c59d21f459f8fc358e806af6f23d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9c08bb471502b80df2470e577376f25400913939)
2021-03-19Provide support for air gapped env for rallySylvain Desbureaux5-80/+103
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for rally test case. There's two ways for providing the repository mirrors: - Give an environment variable (`MIRROR_REPO`) which gives a repository mirro with all needed images. - Gives an environment variable per needed repo: - `DOCKERHUB_REPO` for DockerHub repository (`docker.io`) - `GCR_REPO` for Google Cloud repository (`gcr.io`) - `K8S_GCR_REPO` for Kubernetes repository (`k8s.gcr.io`) Needed images list has also been extracted so Kubernetes administrator can easily upload these images to the mirror if the mirror also doesn't have access to Internet. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: I2ea6622b79f7e3c3c63c1441c4dab48e9bc4fb1a (cherry picked from commit b26ac308d4d441195fcd804d4d0ff36356fa8a90)
2021-03-15Fix k8s.gcr.io/etcd tagsCédric Ollivier1-1/+2
Kali is used to verify both v1.18 and v1.19 which respectively ask for k8s.gcr.io/etcd:3.4.3 and k8s.gcr.io/etcd:3.4.13-0. Co-Authored-By: Dominique Lagadec <dominique.lagadec@ericsson.com> Change-Id: I739202b0e58352fd3308d705807910cd1575d28f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-02-19Rename sig_network_feature -> sig_network_featuresCédric Ollivier2-11/+12
It also takes into account the last results (SCTP seems falky): https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_network-run/1/console https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_network_feature-run/3/console https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_storage-run/1/console Change-Id: I7d4b11462da33d20fd5ed00d870f1099b70f437b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 08558d2f47f7400176ad678ceaaff1b5be0b931c) (cherry picked from commit dcb442250783d97b2bbfc6e5671a18cc90106792)
2021-02-19Run rally firstCédric Ollivier1-12/+12
After all tests are based on e2e tests. Change-Id: I727f40255ccc165f9a7da7fa7bdff1a629614d21 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9e132759f6186672dd23dd31fac511bec128d386)
2021-02-19Add sig-storageCédric Ollivier1-0/+18
It runs the remaining sig-storage testing excluding the next labels: Conformance (part of k8s_conformance), all features, all drivers and Slow. Change-Id: I486d8ecdfc46287f4774ba8acaa64d47fb1da287 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8e6f69afa84ae428a013debcc7329be26a475695)
2021-02-19Add sig_networkCédric Ollivier2-0/+18
It runs the remaining sig-network testing excluding the next labels: Conformance (part of k8s_conformance), all features and load.balancer. Change-Id: I708ff4690205196e5f69382890a39ea4fd5b6942 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 80f0edf3194493620d1cfab79efbcdc8a136d799)
2021-02-19Add sig_network_featureCédric Ollivier2-0/+28
It verifies a couple of features by running a subset of e2e tests: Ingress, Networking-IPv4, Networking-Performance, PerformanceDNS, SCTP and NoSNAT. It excludes by default NetworkPolicy, Networking-IPv6, IPv6DualStackAlphaFeature and Example. A couple of features may be automatically skipped like NEG and kubemci. Change-Id: I6efac39bb99c1deb8efe8e82d976d69d6540a9d1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit e0d151ccaf4b267682feec0d38e8b7de3eb3d61c)
2021-02-17Move former wiki to README.mdCédric Ollivier1-0/+118
https://wiki.opnfv.org/pages/viewpage.action?pageId=20742412 Change-Id: I75a9303b2ee5b2df66e0169b37408bb9ad881d5a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 15f754678e4c8734c4600fea94fcbe66f2584515) (cherry picked from commit a33e2fe3956ce5dcf0893454131121af8e73f863)
2021-02-10Upgrade system firstCédric Ollivier1-1/+2
Change-Id: I0120e6a368b033f2fb47257ff9dcc3fd8a58d243 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9ebe47c8458c99066fd71a65007fff923004d209)
2021-02-04Allow parameterizing K8s versionsCédric Ollivier2-15/+18
ONAP integration team asks to support all K8s versions. A couple of OPNFV releases will allow testing 2 K8s versions. Change-Id: Idb43cff0cc03cc9c17b290f7f1867e2c380c6319 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 0163bb977604a32a0c46e10a97313ce0feea7b5f)
2021-01-10Update cnf-conformance to v0.9.19Cédric Ollivier2-7/+17
It also adds configs needed by resilience. https://github.com/cncf/cnf-conformance/releases/tag/v0.9.19 Change-Id: I94b1efc47c5ceef96bc8879a6c71f92db1f63bd2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 7385caf54c27b471bda8599d491aa3aee21b7b89)
2020-12-10Use pip==20.2.4 in tox venvCédric Ollivier3-2/+3
pip 20.3 is disruptive and would lead to big changes in stable branches. Change-Id: Ic3770f4098c52b0d26bc62291c9e79e8ee81d107 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 0c99749bb65cadd0142c90cd73e13802d48cb7f6)
2020-11-22Allow offline testing via xrally_kubernetesCédric Ollivier3-2/+36
It locally patches xrally_kubernetes before [1] is merged upstream. It adds docker.io/appropriate/curl:edge in download_images.sh too. [1] https://github.com/xrally/xrally-kubernetes/pull/51 Co-Authored-By: Tamas Lendvay <tamas.lendvay@nokia.com> Change-Id: I82ea2c777cd4d8f24c38dbf32d66081846e3e9af Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit ea2715240015a482c5321d309257d801a412f02d)
2020-11-22Reduce cnf-conformance criteriaCédric Ollivier1-1/+1
It's considered as passed vs 95 out of 105 [1]. [1] http://artifacts.opnfv.org/functest-kubernetes/ACI2B7JDLFHI/functest-kubernetes-opnfv-functest-kubernetes-cnf-kali-cnf_conformance-run-16/functest-kubernetes.log Change-Id: Ib16927cf269784daba1085a3e97fb611e869928f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 53b16e8f00e5a60db2e6db7f375bd06f184a5cc8)
2020-11-21Update cnf_conformance to v0.9.4Cédric Ollivier3-36/+32
It removes all hacks bypassing a few former bugs (e.g. [1]) and adapts to the new cnf-conformance result output logic. It runs all workload tests as it's now proposed (k8s conformance is already part of smoke) and select the latest coredns cnf-conformance.yml taking latest helm repo changes into account. It sets criteria 95 as all steps are considered as passed and a overall result 100/105. [1] https://github.com/cncf/cnf-conformance/issues/388 Change-Id: I95b7889b21b86961e7a1d0c84b13e0ae367cae44 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 854f164998793000fcf8019214b259b7283f5194)