aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2020-03-03Blacklist tests which are not mandatory in CNTTCédric Ollivier2-0/+49
CNTT rally_jobs must conform with CNTT RC whatever the OpenStack version under tests. All tests added by neutron since Hunter must be skipped in conformance. Of course, the IaaS verification part is unchanged as this tests are relevant. Change-Id: I4cfb4ae725e5ce9125f4ed25489dd003e16695f4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-03-03Add OVN Controller Gateway agent in neutron_bind_l2_agent_typesCédric Ollivier2-6/+2
It avoids blacklisting NeutronNetworks.create_and_bind_ports in case of OVN. It conforms with the upstream testing [1] and then reverts the previous change [2]. [1] https://github.com/openstack/networking-ovn/blob/master/zuul.d/networking-ovn-jobs.yaml#L299 [2] https://git.opnfv.org/functest/commit/?id=8e297c49c481961959b29c2080cc059fcc84e9ee This reverts commit 8e297c49c481961959b29c2080cc059fcc84e9ee. Change-Id: I95f1b7146e636ae3ebe9f995733e53ad1b276d26 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-28Increate Neutron test count to 501Cédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/P5QXI9R2ULZQ/functest-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-run-29/neutron-tempest-plugin-api/tempest-report.html Change-Id: I0dee2ecea051e1caafd6437bc1766c920b3e1ff4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-28Remove /src/glance after generating policy.confCédric Ollivier1-2/+2
It also cleans former dirs. Change-Id: I924436652e3fe0f19d0755024839ecb5290aa2f3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-24Blacklist NeutronNetworks.create_and_bind_ports if ovnCédric Ollivier1-0/+5
It only works for Linux Bridge and Open vSwitch. Change-Id: I1a437145eb8140642d3c8297c3f154113e74505f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-22Blacklist tests asking for vxlanCédric Ollivier1-0/+5
http://artifacts.opnfv.org/functest/B4HOI3WWLNVK/functest-ovn-opnfv-functest-smoke-jerma-neutron-tempest-plugin-api-run-4/neutron-tempest-plugin-api/tempest-report.html Change-Id: I6ff82b92cb4c392ac1f453faf7e16ef587dc2060 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-22Publish tempest_conf.yaml for Functest OVN SUTCédric Ollivier1-0/+78
Change-Id: I96237e8bfe469f100dc72d022747ce0013b41284 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-21Update patrole regex (OVN)Cédric Ollivier1-2/+6
It takes into account last changes in Patrole. It was not covered since Apex stopped deploying OVN. Change-Id: I7ab8362f9998024ae65f49973b169b95bbfd4fb5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-21Skip a few tests in tempest_full and tempest_slowCédric Ollivier1-2/+5
It completes the first patch updating neutron-tempest-plugin-api [1]. The same dvr regex is used in tempest_slow, a new one is defined for tempest_full which will be added in CNTT doc (test_network_basic_ops.TestNetworkBasicOps.test_router_rescheduling). Close cntt-n/CNTT#1106 Change-Id: I9fd79c9a82bb1f84a622ccb9ea316b8e43273e2a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-18Merge "Update test case list in neutron-tempest-plugin-api"Cedric Ollivier1-2/+24
2020-02-18Blacklick test_add_subport if OVNCédric Ollivier1-0/+1
https://bugs.launchpad.net/neutron/+bug/1863707 Change-Id: Ieebe10ee2a9040887d84aca3e8fb5ee66294634b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-18Update test case list in neutron-tempest-plugin-apiCédric Ollivier1-2/+24
It takes into account latest changes in CNTT [1] [1] https://github.com/cntt-n/CNTT/pull/1107 Close cntt-n/CNTT#1106 Change-Id: I3212d1b765f7fd657ed603ed520d79fccae80bfa Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-17Select OpenStack requirements masterCédric Ollivier4-8/+8
https://github.com/openstack/requirements/commit/7a04f4e96205f16e74a9a08fe60b168c187516d6 Change-Id: I9ed510c45f63714b64dcb9fa90e4d7239f93ee3f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-17Fix incorrect test numbersCédric Ollivier1-1/+1
[1] http://artifacts.opnfv.org/functest/16S4PBMRW5HP/functest-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-run-23/neutron-tempest-plugin-api/tempest-report.html Change-Id: I9cb04c3abdbb899d2f1d4188a9fb9743a8f4c6d0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-17Bypass upstream requirement issuesCédric Ollivier4-11/+14
https://review.opendev.org/#/c/708041/ Change-Id: Iebbf0a720fe9d1c62083028d0bc0078bf7680753 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-16Remove refstack version tagCédric Ollivier1-1/+0
It has been unread since the files are cached. Change-Id: Iba74163ac486dddaee4e81a340fcbe2b9297071f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-16Exclude vxlan testing in CNTTCédric Ollivier1-1/+5
Close cntt-n/CNTT#1077 Change-Id: If9c0f52138e9f669f17933c42906936e2e196b2a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-15Run test_agent_management vs OVNCédric Ollivier1-2/+0
They must not longer be blacklisted as opposed to test_dhcp_agent_scheduler [1]. https://bugs.launchpad.net/networking-ovn/+bug/1863432 Change-Id: Ib08bb87580a10e2124faa1b3853f3e27dcb3a0a0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-15Set dhcp_agent_scheduler as optionalCédric Ollivier1-3/+5
dhcp_agent_scheduler [1] indirectly forces the Neutron backends as highlighted by OVN testing. [1] https://specs.openstack.org/openstack/neutron-specs/specs/api/agent_schedulers.html Close cntt-n/CNTT#1073 Change-Id: Iafa3ad7f15dc157c0362ffa59eed31b89bf368e7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-15Order regex in neutron-tempest-plugin-api (CNTT)Cédric Ollivier1-4/+5
It mostly adds admin as prefix. Change-Id: Ic2e723b858ff1f3e26478f6037a19db1fd52e4aa Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-15Decrease test_count to 1290 in tempest_fullCédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/2Y4502QRM7YM/functest-opnfv-functest-smoke-cntt-latest-tempest_full-run-16/tempest_full/tempest.log Change-Id: Ifa1ae117db7dff6471ab557d6f7f3bf03940de76 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-15Increase test_count to 547 in neutron-tempest-plugin-apiCédric Ollivier1-1/+1
It takes into account the new tests added in Neutron Tempest Plugin. Change-Id: I7994638c69ef6bd39bae3c6ded8409baabc27e9d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-04Add Functest CNTT-specific containers in release notesCédric Ollivier1-0/+2
Change-Id: I979a5e26caee893d772de484e7a61f2e82762ace Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-01-24Update Alpine to 3.11Cédric Ollivier19-48/+51
It also updates Python to 3.8 (default version). It selects arm32v6 et arm64v8 repos and switches multiarch binfmt. https://alpinelinux.org/posts/Alpine-3.11.0-released.html Change-Id: Ib41cf0ab805ce2137658f3ff656b0a88c7b4ebf5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-01-24Generate /etc/glance/policy.jsonCédric Ollivier1-2/+3
It takes into account the following change in glance [1] [1] https://github.com/openstack/glance/commit/5c17e4c7ef533b57ba15ead931b0452e23f46b4b#diff-335462891f7178710c6ca7185f9f877c Change-Id: I61e37cd2383620fa6db665f1858830b9c5d13938 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-01-14Slightly change Functest2020Cédric Ollivier1-2/+6
Change-Id: I7f82f5905536da593193d586a1b0655fcd44d6ad Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-01-14Update ODL git commit id (python3 migration)Cédric Ollivier1-1/+1
https://jira.opendaylight.org/browse/INTTEST-65 Co-Authored-By: Alexandru Avadanii <alexandru.avadanii@enea.com> Change-Id: I40c9df7c7d80fc8d9709f0b75dda333d9ed9da12 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-01-13Publish Functest 2020Cédric Ollivier2-0/+109
http://testresults.opnfv.org/functest/functest2020/ Change-Id: I4116b08bd2210231d78e2761a1089fbfc0d56a46 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-01-09Revert "Ease overriding tempest_conf.yaml"Cédric Ollivier1-1/+0
It raises side effects if the file is missing (trying to mount a directory onto a file) which is the default case. This reverts commit fe4c595edbaa31a43b94c5065fb14cab932ddffd. Change-Id: Ic24bbbbadfe0cd8901ed66357e95f466cee8da80 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-01-09Remove test_server_connectivity_cold_migration_revert from ComplianceCédric Ollivier1-1/+2
It's skipped due to an upstream bug [1] highlighted by Jerma runs [2]. [1] https://launchpad.net/bugs/1836595 [2] http://artifacts.opnfv.org/functest/UXR96KDNMPTP/functest-opnfv-functest-smoke-cntt-jerma-tempest_slow-run-4/tempest_slow/tempest-report.html Change-Id: Ib4cbf0e6e8e010febd39dec8dd8ecd773ad28c3e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-01-02Ease overriding tempest_conf.yamlCédric Ollivier1-0/+1
It works asis in Functest gates but its values mays be modified by users at their convenience. Change-Id: Id83c01693818802d11eb3822b59a508ac09990c3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-30Remove test_server_connectivity_cold_migration_revert from exclusion listCédric Ollivier1-2/+1
It has to be part of the Compliance testing Co-Authored-By: Deepanshu Bhatia <deepanshu@voereir.com> Change-Id: I16d35e3c3a14b6b18901b6b27e50fc709ce0559f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-27Set port_security: true in tempest_conf.yamlCédric Ollivier3-2/+9
It allows running test_boot_into_disabled_port_security_network_without_secgroup. It also sets a few default values. Change-Id: I851c5d13705e0419a7e664e05d32eec0efb417c6 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-27Fully remove Quotas.nova_update_and_deleteCédric Ollivier1-11/+0
It was blacklisted for a while and it's still failing [1][2]. It has to be removed from scenarios (blacklisting is useless and could mislead endusers). [1] http://artifacts.opnfv.org/functest/BYERCT6UZDWT/functest-opnfv-functest-benchmarking-iruya-rally_full-run-227/rally_full/rally_full.html [2] http://artifacts.opnfv.org/functest/LYQLHFQB7CTI/functest-opnfv-functest-benchmarking-hunter-rally_full-run-329/rally_full/rally_full.html Change-Id: I9ccea90a70024b6e54d873024cf550db5beea990 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-27Set create_shared_resources to trueCédric Ollivier3-1/+3
It takes last Neutron Tempest plugin change into account [1]. It also increases test count. [1] https://review.opendev.org/#/c/692822/ Change-Id: I5e7218dd808069a0be6d481447576099aa9c63e9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-26Pin rally-openstack in masterCédric Ollivier1-1/+1
The last patch seems buggy [1] TypeError: _discover_or_create_flavor() missing 1 required positional argument: 'flv_disk' [2] [1] https://github.com/openstack/rally-openstack/commit/e62315115af20e53f6c7205494ba0008634047ed [2] https://build.opnfv.org/ci/job/functest-opnfv-functest-healthcheck-latest-tempest_smoke-run/629/console Change-Id: Id0402afdf79d7a1081e817ea513d90c93b077ce4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-24Remove former Rally blacklistCédric Ollivier1-8/+0
Change-Id: Ibc9aa210846ad2163f7ef943d224374dfff6f1ec Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-22Update Xtesting to 0.86Cédric Ollivier1-1/+1
It eases dumping all logs for third-parties. Change-Id: I6055e85902eb41cce01f1502cd31d7ec489708d8 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-07Merge "Run tests sequentially in tempest_slow"Cedric Ollivier3-3/+3
2019-12-07Merge "Introduce CNTT Compliance by Functest"Cedric Ollivier2-0/+149
2019-12-07Merge "Disable false ansiblelint warnings"Cedric Ollivier2-2/+2
2019-12-07Run tests sequentially in tempest_slowCédric Ollivier3-3/+3
concurrency is set to 2 in the upstream job [1] but it conflicts with serial also passed. As it fails once without any other test in parallel, the number of executors should be decreased [2]. [0] https://799189c1bf6a1aa9c8de-48970bda2a30ee836a70a12e61555969.ssl.cf1.rackcdn.com/694893/1/check/tempest-slow/1bf5851/job-output.txt [1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-cntt-hunter-tempest_slow-run-7/tempest_slow/tempest-report.html Change-Id: I51bc9a584cf4b32a88b7a84d9387e7b8e90281c2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-07Introduce CNTT Compliance by FunctestCédric Ollivier2-0/+149
Change-Id: I77dd7d3e0d21573e76ec8d43a0b6ca5b64be2ad2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-07Disable false ansiblelint warningsCédric Ollivier2-2/+2
git commit has to be executed via shell module [1]. It also update ansible to conform with the latest changes in XtestingCI. [1] https://github.com/ansible/ansible/issues/50334 Change-Id: I3cc7a675eddabb0cbd9834a9e64fe7389a11865f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-07Increase tests_count (538) in CNTT neutron-tempest-plugin-api-runCédric Ollivier1-1/+1
New tests have been published in the upstream project [1]. [1] https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-run/4/console Change-Id: I46603ed78f865f89fe2a7ebcf6a72da38a02bb14 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-03Disable tempest serial_console testingCédric Ollivier1-1/+1
DevStack seems incorrect when configuring serial console in case of multinode deployment (missing proxyclient_address in master, missing socket in Rocky, etc). Then the testing is disabled. Change-Id: I9d1d43017b0935b23ea2a939748b5f709212c70d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-02Verify Shaker results via json reportCédric Ollivier1-0/+8
As shaker always returns 0, it verifies the records and returns false after the first failure. Change-Id: I9393be0c65d315c1ca055de9f7f994f9db22eadb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-02Return result = 0 if failure in juju_epcCédric Ollivier1-0/+1
Else it's falsy successfull [1] [1] https://build.opnfv.org/ci/job/airship-opnfv-functest-vnf-latest-juju_epc-run/42/console Change-Id: Ic9f7c37db998904833c90ae3ee912e9908617abd Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-30Update Xtesting to 0.85.0Cédric Ollivier1-1/+1
It fixes falsy hrefs returned by OPNFV Test API. Change-Id: Ib6e7e7e35cf6ad581fb67692dcba561769686007 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-29Switch xrally_kubernetes back to masterJuha Kosonen1-1/+1
The pull request [1] is now merged. [1] https://github.com/xrally/xrally-kubernetes/pull/46 Change-Id: Ida258c279314cdbbacf255c50b3a4d1224ea803b Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>