aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2021-07-07Print cnf_testsuite console on exceptionsCédric Ollivier1-0/+4
Change-Id: I3c20993e1cbb644546ac89a8b4b7d1cfd98b0a80 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f6d5010cd34db6cd639a7fcf5b6183aa117acacf)
2021-06-30Fix previous backportCédric Ollivier1-1/+1
It reverts IPv6DualStack to IPv6DualStackAlphaFeature Change-Id: If7795d6e7ec37087079b2b7b598cae607b2873bd Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 573073379f57ca95a60ff6fa56ddef103810e803)
2021-06-20Make clear all skipped Features according to CNTT RA2 Chapter6Cédric Ollivier1-64/+92
Change-Id: I57d414902e973d7bcbb67ba12d10b727b2ec4785 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9e025ffa82fad7a5b49663a30987dcf03bb44ca3)
2021-06-20Allow passing extra opts to e2e.testCédric Ollivier1-0/+12
E2E_TEST_OPTS now allows setting runtime opts. For instance, Functest K8s gates ask for -container-runtime containerd Change-Id: I4f6e1caf57590b16d0c01324dadb040f67f3cee4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 6ec9d5167032ed82e32cde592f98a4bcffca4c00)
2021-05-25Update cnf_testsuite to v0.11.2Cédric Ollivier2-32/+19
https://github.com/cncf/cnf-testsuite/releases/tag/v0.11.2 Change-Id: Id3a5ea98260859f7fe5aba6ecf02d09905610dda Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit c35c6d443eb93db318c3436d995809dfe7e8389f)
2021-05-11Install libstdc++Cédric Ollivier1-1/+1
Error loading shared library libstdc++.so.6 Change-Id: Ica164f5fddabd5508200ad0017e81de041c40b22 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-05-11Install matplotlib as needed by netperfCédric Ollivier1-2/+8
Change-Id: I55d03dbc5a35adcbf9b88d873b3f2bb7851d0d86 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-05-11Set --use-deprecated=legacy-resolver in pip callsCédric Ollivier1-2/+2
Change-Id: I3fbf72043b8c9145364eecfd091bba6553424b67 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-05-10Benchmark Kubernetes Networking PerformanceCédric Ollivier8-0/+138
https://github.com/kubernetes/perf-tests/tree/master/network/benchmarks/netperf Change-Id: I6facd567f1c52c5949b53484a1fb107dcf34d622 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 3b5c1b115e234d636cb4f2a17d27ced872fee924)
2021-05-04Update test case resultsCédric Ollivier1-17/+35
Change-Id: I728e4d81b86f5ec851b513fd88f7501230c5350a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-05-03Skip [Feature:Federation] in sig_network_serialCédric Ollivier1-0/+1
Change-Id: I005d6207311e661a0ce6695f24e3b91703926692 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-05-03Fix cnf-conformance results filenameCédric Ollivier3-3/+3
It also decreases criteria to 1 due to massive changes (dockerd). Change-Id: Ic420139e6355debae183a77d6b0c574801b05076 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-05-03Update cnf-conformance to cnf-testsuiteCédric Ollivier6-30/+30
https://github.com/cncf/cnf-conformance https://github.com/cncf/cnf-testsuite Change-Id: I77983e0f04781a92b34afcf82f21366384782964 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 959da00d3e1c4d406bc59995c0ea93add1ec48d3)
2021-05-03Reduce number of threadsCédric Ollivier1-11/+11
It passed successfully with 10 threads (more doesn't reduce the overall duration so much). Change-Id: I9013897883a40ebaf699c9a180f1bc8a6a34505c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-04-21Mock os.makedirsCédric Ollivier1-2/+7
Change-Id: I5f9fd24421727cea474715f502670ae6f2c80b76 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 32d2ed700c449ba33a4060f55c2adadc2e76b070)
2021-04-20Install setuptools_scm before xrallyCédric Ollivier1-0/+4
Else upper-constraints basically fails. Change-Id: I022f101e2ed44708afd0b88d47b77d2f7ca85458 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 75170fc0b528c7e182b8164d5905fcbaa6957faa)
2021-04-15Skip Network.should.set.TCP.CLOSE_WAIT.timeoutCédric Ollivier1-1/+2
https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-release/release-branch-jobs/1.21.yaml Change-Id: I9d8065fe84d3f6f5e74bf9b46bc8c85ad985c8b1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 16f4a0c29fcfd1c6dc577394d9627845f559f177)
2021-04-14Remove sig_node_serial as it's fully disruptiveCédric Ollivier2-25/+0
There is no other test when NoExecuteTaintManager is skipped. http://artifacts.opnfv.org/functest-kubernetes/T94WVHSMW8ZH/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-sig_scheduling_serial-run-6/sig_scheduling_serial/e2e.log Change-Id: I09a81ec973b26fc6de32df6788e73c7e529ecbf0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit a6dda83e6d6da08f776010a7cdebff77c3c7c2c0)
2021-04-14Skip SetHostnameAsFQDN in sig-nodeCédric Ollivier1-2/+3
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-sig_node-run/14/console Change-Id: I55c9d3404f9bca64354a6276881a71624281a59b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-04-08Disable Feature:StackdriverLogging in sig_instrumentationCédric Ollivier1-2/+4
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-sig_instrumentation-run/8/console Change-Id: I6634ec335b632ac6c7efb9b2078eb1c5f3d9be31 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-04-08Disable a few sig-auth featuresCédric Ollivier1-3/+5
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-sig_auth-run/8/console Change-Id: I8a1523ba293f40dc8533428f9d75b29560a817cc Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-04-04Update test case listCédric Ollivier1-0/+8
Change-Id: Ide5fdd5967ce11847f9672259b79b6068c228848 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit c1ea4de55608b931462cd0a8d2a3a8325e5888e5)
2021-04-04Add k8s_io and serial test casesCédric Ollivier1-6/+161
It completes the first sig test cases. Change-Id: I6b018177e93efe60f9de67f384e388d2a584536c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-04-04Merge sig_network_features in sig_networkCédric Ollivier2-28/+4
Change-Id: I33516a7b0f280355c40aa821738a0625e4fae93a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit adcbd42cabfa42a71410dd0c76ece0f367d338fd)
2021-04-03Spit conformance into 2 test casesCédric Ollivier2-0/+18
It hugely decreases the overall test duration. (10 + 15 << 120) Change-Id: I534f6d09956bb51c32ad35c5a1baff8ebbfabb3d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit a33e339a001d62352e15a3cc1546bb4d4d482e59)
2021-04-03Fix etcd image repository in etcd-depl.yamlCédric Ollivier1-1/+1
Change-Id: I4447b4030b141b691b3a2cdf05c97c79de9eb72a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit b134d6735e722261bfe10c1ba83112e9cf95f85c)
2021-04-02Fix name in chronos-depl.yamlCédric Ollivier1-1/+1
Change-Id: Ia6972a216daaa9d807058b34b55615cab2c09b64 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 2eaa659eee7c83fd9221f88a207514e8e909943f)
2021-04-02Run opnfv/functest-kubernetes-smoke sequentiallyCédric Ollivier2-0/+4
Now multithreading is enabled at test case level. Change-Id: I17a9bdb07216cd645483a0a893da502abee2ade0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit bbc8501437c1112d2e5c53281622d3dbe25b0d61)
2021-04-02Properly manage e2e parallel testingCédric Ollivier6-367/+366
It sets nodes=25 to run all tests in parallel. It also beautifies all testcases.yml Change-Id: I3ecd8899e957c0efbef34ab0409c2b4d557c9425 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 4004d776b01a1b038df52ca45b18430da1b6b1f5)
2021-04-02Rather use skeleton instead of localCédric Ollivier1-1/+1
“skeleton”: cluster is accessed via the Kubernetes API and nothing else. https://kubernetes.io/blog/2019/03/22/kubernetes-end-to-end-testing-for-everyone/ Change-Id: Iea8b65b2ac55d40572c0078b7a9a956d1c54d62a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 17929b0d015034bad2796fb733627fe5b23bbb2d)
2021-04-02 Provide support for air gapped env for e2eSylvain Desbureaux2-1/+42
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for e2e test case. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: Iaa2960ad03aae1fff859d41bcd6ddf3176658f8e (cherry picked from commit 27dbb7c14bac45a9af8d18e1913a2be6815cea43)
2021-04-02Export GINKGO_PARALLEL=yCédric Ollivier1-1/+3
It would speedup the testing. Change-Id: I084bcf0d44a060f65eb9549eab8dcbbf5a4cc9d5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 65d860c6ee36641ad7d248f7dc8a6f63f3173301)
2021-04-02Provide support for air gapped env for imsSylvain Desbureaux25-66/+82
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for ims test cases. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: I3b5e78fcf43ca7af6bcfd2d87fa4bb0705373697 (cherry picked from commit 9c95b5f46e57365c1f6e7e5c9073f7fe2ad48a66)
2021-03-31Disable sig_scheduling as serialCédric Ollivier2-19/+0
All remaining sig-scheduling tests are serial and for the time Functest doesn't run serial test cases. https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_scheduling-run/13/console Change-Id: I3f09077a1873b92120daa3a97cbb6f68558e2b4d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f28b4673a7b89f124d225e629f0c5267e6008c14)
2021-03-26Remove the useless Feature prefixCédric Ollivier1-1/+1
Change-Id: Ibec3e1cee170a91a4333b3c625ae39cd0c4bbe11 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-03-26Exlude Serial and Feature:Feature:StorageVersionAPICédric Ollivier1-12/+20
Serial asks to split the testcases into 2 runs. https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-latest-sig_api_machinery-run/7/console Change-Id: I58f9d9023f596141bbefae8339043f813a124854 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-03-26Blacklist all alpha testingCédric Ollivier1-11/+11
Change-Id: Ifb9edb0d70f285825eb2cfee05a53590367b2650 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-03-26Leverage py3-setuptools_scm as proposed by AlpineCédric Ollivier1-1/+1
Last version setuptools-scm asks for very new setuptools. https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-kali-xrally_kubernetes-run/194/console Change-Id: Ib1f4b4b49c036e0018fd695b52013bccdaf4998f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-03-25Provide support for air gapped env for securitySylvain Desbureaux5-5/+11
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for security test cases. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: I699d065ee691596c4a5ccf06c22ea76ef00fe497 (cherry picked from commit af623f6b45c9357b0e33210f54db64b270d162bf)
2021-03-22Add missing blackslashingCédric Ollivier1-1/+1
https://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-leguer-sig_apps-run/2/console Change-Id: I5cd41f2da49e59acd6bd06ded0c4e2178e7402b7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 0b79c61af319c6f3beddf79e586e51e4675f3fc1)
2021-03-21Add sig_nodeCédric Ollivier2-0/+24
It runs sig-node tests which are not included in K8s conformance. It adds 38 single tests. Change-Id: Idbd172517e705402632a7fadb4d4639b5f1ee2d1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 56da8a0382889fa0558da48375d2319bfedd39a4)
2021-03-21Add sig_cluster_lifecycleCédric Ollivier2-0/+17
It runs sig-cluster-lifecycle tests which are not included in K8s conformance. It adds 4 single tests. Change-Id: I834523d753eec9bd570b8d4cc1cf3550b4a19bab Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f3d390ddba5021ce45dd763e2bf953994a487337)
2021-03-21Add sig_instrumentationCédric Ollivier2-0/+17
It runs sig-instrumentation tests which are not included in K8s conformance. It adds 6 single tests. Change-Id: Ic320f3bc7daed0aa2846a7ea672f13f3bec45f12 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 39511d8576f49b578882757fcc1e33a4bb2c9832)
2021-03-21Add sig_schedulingCédric Ollivier2-0/+18
It runs sig-scheduling tests which are not included in K8s conformance. It adds 19 single tests. Change-Id: I9b34c327cc1079e401771b1fa77a6baeef24251c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8bfdc8569b9eaffb96562e7753423d3ded218ed4)
2021-03-21Add sig_cliCédric Ollivier2-0/+17
It runs sig-cli tests which are not included in K8s conformance. It adds 34 single tests. Change-Id: I4beaf9d08d151d06fc3f3c05770c51dcc68f762a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 6eaf848638d7a1a7b4c4324837da9b8562ec7136)
2021-03-21Add sig-appsCédric Ollivier2-2/+22
It runs sig-apps tests which are not included in K8s conformance. Change-Id: Ic02e75b94e6cd923f05af16eb98706599fd26f59 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 985b7955d065065d191702f1355d7ca114f0c827)
2021-03-21Add sig-authCédric Ollivier2-0/+19
It runs sig-auth tests which are not included in K8s conformance. Change-Id: I5684dbcb5d61866f5ae1cb483f3a119b8d3c7807 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit b5c7bc0107a443722004ecced91118042965f327)
2021-03-21Add sig-api-machineryCédric Ollivier2-4/+21
It runs sig-api-machinery which are not included in K8s conformance. It adds 97 single tests. Change-Id: I76949b75e2d6c59d21f459f8fc358e806af6f23d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9c08bb471502b80df2470e577376f25400913939)
2021-03-19Provide support for air gapped env for rallySylvain Desbureaux5-80/+103
Sometimes, tested Kubernetes doesn't have direct access to Internet but access through repository mirrors. This patch handles this case for rally test case. There's two ways for providing the repository mirrors: - Give an environment variable (`MIRROR_REPO`) which gives a repository mirro with all needed images. - Gives an environment variable per needed repo: - `DOCKERHUB_REPO` for DockerHub repository (`docker.io`) - `GCR_REPO` for Google Cloud repository (`gcr.io`) - `K8S_GCR_REPO` for Kubernetes repository (`k8s.gcr.io`) Needed images list has also been extracted so Kubernetes administrator can easily upload these images to the mirror if the mirror also doesn't have access to Internet. Signed-off-by: Sylvain Desbureaux <sylvain.desbureaux@orange.com> Change-Id: I2ea6622b79f7e3c3c63c1441c4dab48e9bc4fb1a (cherry picked from commit b26ac308d4d441195fcd804d4d0ff36356fa8a90)
2021-03-15Fix k8s.gcr.io/etcd tagsCédric Ollivier1-1/+2
Kali is used to verify both v1.18 and v1.19 which respectively ask for k8s.gcr.io/etcd:3.4.3 and k8s.gcr.io/etcd:3.4.13-0. Co-Authored-By: Dominique Lagadec <dominique.lagadec@ericsson.com> Change-Id: I739202b0e58352fd3308d705807910cd1575d28f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>