aboutsummaryrefslogtreecommitdiffstats
path: root/docker/smoke-cntt/testcases.yaml
AgeCommit message (Collapse)AuthorFilesLines
2020-07-14Enable back manage_snapshot testing in CNTTCédric Ollivier1-2/+1
Baraque selects Train: https://github.com/cntt-n/CNTT/pull/1782 https://github.com/cntt-n/CNTT/pull/1783 Change-Id: Ie715a86cf6129ab5b51bf9e0ec07f326c0f86ccb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-07-14Enable back RemoteStack testing from tempest_heat_cnttCédric Ollivier1-3/+1
Baraque selects Train: https://github.com/cntt-n/CNTT/pull/1782 https://github.com/cntt-n/CNTT/pull/1783 Change-Id: I9650f5b9041a59b68af34c7410c64515e498a226 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-07-03Fix tempest_scenarioCédric Ollivier1-1/+1
It completes "min_computes_nodes is also used as max values". Change-Id: I72c67e17015bc35ac5e14f20839cf74353695373 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-29Update tempest_neutron_cntt regex and test countCédric Ollivier1-0/+1
It takes latest updates into account [1][2]. [1] http://artifacts.opnfv.org/functest/4W4YF3954M7H/functest-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-85/tempest_neutron_cntt/tempest-report.html [2] http://artifacts.opnfv.org/functest/K4DV91F2Y9V0/functest-ovn-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-76/tempest_neutron_cntt/tempest-report.html Change-Id: Ia9b57e708522926fde9f8c44e6a0e936e97f304f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-12Skip EncryptionVolTypeTest in Iaas verification and CNTT ConformanceCédric Ollivier1-1/+2
Attaching an encryted volume is optional in CNTT [1]. This test fails if the volume type already exists and if it runs twice in parallel [2]. As this single test doesn't fullfil the Functest requirements, it's skiped in the classical verification as well. [1] https://github.com/cntt-n/CNTT/blob/master/doc/ref_arch/openstack/chapters/chapter05.md [2] https://storyboard.openstack.org/#!/story/2007804 Change-Id: Ib6fc843d71704aab1bfc536133544eb3fa4f3e72 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-12Remove RemoteStack testing from tempest_heat_cnttCédric Ollivier1-1/+3
Trusts redelegation is supported by Train [1][2] which forces to skip RemoteStack as CNTT Baldy is designed for Pike (the support is incomplete in case of trust [3]). [1] https://docs.openstack.org/releasenotes/heat/train.html#relnotes-13-0-0-stable-train [2] https://github.com/openstack/heat/commit/e377658586c737150dad1bfd80b7b2267d46be51 [3] https://bugs.launchpad.net/heat/+bug/1701498 Change-Id: Ib4f66170b9b1f50b8224705edb4820e5a4cb0167 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-07Update functest/ci/testcases.yamlCédric Ollivier1-2/+2
It takes all CNTT changes into account. Change-Id: Ib6ddbd12695266fc78fd8ae4cce451bf8a4b95ad Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-06Increase tempest_neutron's tests_countCédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/Q3047D5AS3HJ/functest-ovn-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-40/tempest_neutron_cntt/tempest-report.html http://artifacts.opnfv.org/functest/03PRDE6TZC14/functest-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-49/tempest_neutron_cntt/tempest-report.html Change-Id: I4be6d2f58e50572a3e8b90ede0055f36bc1d6886 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-05Switch back to concurrency = 4 in tempest testcasesCédric Ollivier1-4/+4
3 seems good enough and stable which allows trying 4 as before Change-Id: Id1f17ebf73f9faa13ed3ceb327ccb4ecf88c2bf2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-03Increase tests_count into tempest_neutron_cnttCédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/KJSOWXFF5CIQ/functest-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-41/tempest_neutron_cntt/tempest-report.html Change-Id: I04f0c3ae695f17e2bd56add11410e24ee25967d2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-28Take last upstream changes into account in the regexesCédric Ollivier1-0/+1
http://artifacts.opnfv.org/functest/1NCC8YUFKE7X/functest-opnfv-functest-smoke-cntt-latest-tempest_slow_cntt-run-64/tempest_slow_cntt/tempest-report.html Change-Id: I196421d96ada3ec1eb8fc718f72c3e796a0bd3b4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-27Skip manage_snapshot in CNTTCédric Ollivier1-1/+2
As detected by Orange's RC Field trial, Ceph doesn't support this feature before Rocky [1]. "AttributeError: 'RBDDriver' object has no attribute 'unmanage_snapshot'" manage_snapshot testing is fully passing in Functest SUT because CNTT is verified vs OpenStack Rocky and newer. Ceph is selected by RI as backend and is widely reused by the ecosytem. It's similar to [2]. [1] https://bugs.launchpad.net/cinder/+bug/1645288 [2] https://gerrit.opnfv.org/gerrit/c/functest/+/69888 Change-Id: I3b7f88f5bed9a47a1e26e16bbbf74fea24ca76c2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-25Update tests_count in tempest_full_cnttCédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/R1927JRCFJR5/functest-opnfv-functest-smoke-cntt-latest-tempest_full_cntt-run-62/tempest_full_cntt/tempest-report.html Change-Id: Ia4b5bbdb92e7877d81acdcf9c4f3a9745badfb36 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-14Prevent microversion conflictsCédric Ollivier1-1/+6
CNTT selects Pike as baseline and Hunter matches Rocky. Change-Id: I8a1aa8385adce1ebb9d74bcb897effaf5b434e5a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 82ceefd8f1609508a476aa2e4573e59926653165)
2020-05-10Increase tempest_neutron test_count to 512Cédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/V0A7B0J1PIRL/functest-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-17/tempest_neutron_cntt/tempest-report.html Change-Id: I95dc0d8dec370dedb501878cbedfe5567ab86afd Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-04Enable network_v6 testing in CNTTCédric Ollivier1-3/+2
req.inf.ntw.16 is a MUST requirement. Fixes cntt-n/CNTT#1556 Change-Id: Id48595f5f4710ea9cbf31edaa4ae0ed31f5278ba Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-02Skip latest tests in tempest_full_cnttCédric Ollivier1-0/+1
http://artifacts.opnfv.org/functest/NQ4IHN2NFL69/functest-opnfv-functest-smoke-cntt-latest-tempest_full_cntt-run-41/tempest_full_cntt/tempest-report.html Change-Id: Ia2a4e4229f02f5909ac87b5f6175230b194bc205 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-01Exclude tests.scenario.test_octavia_lbaas in CNTT regexesCédric Ollivier1-0/+1
It may be skipped as well (the behavior differs if it's enabled or not via config). Change-Id: Ic23119001b7478e466782a58d8757117bf16e5c3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-28Rename neutron_tempest_plugin_api to tempest_neutronCédric Ollivier1-1/+1
It conforms with the other test case names Change-Id: Ia8836375c26ea4f1bff3637f7078e5f70f83c5fb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-28Complete "Skip CfnInitIntegrationTest in tempest_heat"Cédric Ollivier1-1/+2
http://artifacts.opnfv.org/functest/SXA8X2BBP3L4/functest-opnfv-functest-smoke-cntt-latest-tempest_heat_cntt-run-1/tempest_heat_cntt/tempest-report.html Change-Id: Ie26d561fbb490bcaafe7c8ffddebc28bd72ff51c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-28Increase neutron-tempest-api test_count to 510Cédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/3L7CL8K6CU2N/functest-ovn-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-cntt-run-29/neutron-tempest-plugin-api-cntt/tempest-report.html Change-Id: I077464ed0fc734b97de089f549fb66ab137fccdb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-27Add tempest_heat in CNTT smokeCédric Ollivier1-0/+24
It increases Heat coverage [1]. [1] http://artifacts.opnfv.org/functest/24ELW4C5RKD4/functest-ovn-opnfv-functest-smoke-latest-tempest_heat-run-34/tempest_heat/tempest-report.html Change-Id: I6630e4827a59b2ded34e5c271b9155df3b25f588 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-24Increase test_count in tempest_cinderCédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/C5BKAKPJIZVJ/functest-opnfv-functest-smoke-cntt-latest-tempest_cinder_cntt-run-26/tempest_cinder_cntt/tempest-report.html Change-Id: I5c0a7db033024259180610b83b07c055f0393e4c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-19Skip new tempest_full tests (CNTT)Cédric Ollivier1-0/+1
http://artifacts.opnfv.org/functest/53FYHR0Q9VVJ/functest-opnfv-functest-smoke-cntt-latest-tempest_full_cntt-run-20/tempest_full_cntt/tempest-report.html Change-Id: Ia4bba0b14ed1d1058f0461e90207aa7530f2e0f3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-17Take the last neutron tempest tests into accountCédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/U0Y127OQNBF4/functest-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-cntt-run-17/neutron-tempest-plugin-api-cntt/tempest-report.html Change-Id: Ib19c63e1a7f6e4c050c19b520fe149ff7c81b156 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-16Update Neutron regexCédric Ollivier1-0/+1
It skips the latest tests added in neutron-tempest-plugin [1][2] [1] https://review.opendev.org/#/c/711656/ [2] http://artifacts.opnfv.org/functest/CQY2GVXZ1NTU/functest-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-cntt-run-13/neutron-tempest-plugin-api-cntt/tempest-report.html Change-Id: Iae9cd7a45f908c310133bf354b205a623f4c7476 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-10Update CNTT regexes and countsCédric Ollivier1-1/+2
It increases neutron-tempest-plugin-api test count to 502. It skips test_multiattach_rw_volume_update_failure which depends on volume multi-attach. http://artifacts.opnfv.org/functest/BVDINIZ8P8UC/functest-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-cntt-run-6/neutron-tempest-plugin-api-cntt/tempest-report.html http://artifacts.opnfv.org/functest/BVDINIZ8P8UC/functest-opnfv-functest-smoke-cntt-latest-tempest_full_cntt-run-5/tempest_full_cntt/tempest-report.html Change-Id: Id4d4918ee24ef15f9610e706d6e90dfa3ae2cc43 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-05Remove last skipped tests in neutron-tempest-plugin-api-cnttCédric Ollivier1-0/+2
It takes into account last results [1] [1] http://artifacts.opnfv.org/functest/LIF2Y3SVUWLD/functest-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-cntt-run-2/neutron-tempest-plugin-api-cntt/tempest-report.html Change-Id: Idd03c5aa42309eb5f254d6a99ea7e7206ee705ed Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-04Reduce Tempest multi threadsCédric Ollivier1-4/+4
It decreases the threads from 4 to 3 to allow running the testcases in parallel. Else a few tests sometimes fail. Change-Id: I268eff5a72d5de5b26d7549ac74ee0e2a05cb61e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-04Skip test_container_synchronizationCédric Ollivier1-1/+2
It's not supported by radosgw which is now used in Functest SUTs [1][2]. It's removed from CNTT which seems leveraging on ceph (at least RI) [3] [1] https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3.2/html/release_notes/bug-fixes [2] http://artifacts.opnfv.org/functest/E46RCZLBRYCC/functest-opnfv-functest-smoke-hunter-tempest_slow-run-127/tempest_slow/tempest-report.html [3] http://artifacts.opnfv.org/cntt/WKFRPQODPW34/cntt-opnfv-functest-smoke-latest-tempest_slow-run-37/tempest_slow/tempest-report.html Change-Id: I8127df786c41af0d0285ef0e5cd1b28901685f65 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-01Rename CNTT specific testcasesCédric Ollivier1-7/+7
Test DB and API don't support suites which raise side effects in Cachet if both IaaS verification and CNTT conformance are executed. Vmtp and Shaker haven't been modified for CNTT. They will be renamed once CNTT asks for new post processing. http://testresults.opnfv.org:8080/ Change-Id: I64e06694620027faa06b6d9d339684f5e8d733d7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-03-05Add oauth1 as optional Keystone featuresCédric Ollivier1-2/+3
It was not listed in Keystone which defacto sets it as optional. It may be considered as mandatory in a second step. It should be noted that it's not working in CNTT RI and there is no true plan to update CNTT RI from the time writing. Close cntt-n/CNTT#1253 Change-Id: If7ee4efeafc05638173a51ce997ecbdcaabfc79c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-28Increate Neutron test count to 501Cédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/P5QXI9R2ULZQ/functest-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-run-29/neutron-tempest-plugin-api/tempest-report.html Change-Id: I0dee2ecea051e1caafd6437bc1766c920b3e1ff4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-21Skip a few tests in tempest_full and tempest_slowCédric Ollivier1-2/+5
It completes the first patch updating neutron-tempest-plugin-api [1]. The same dvr regex is used in tempest_slow, a new one is defined for tempest_full which will be added in CNTT doc (test_network_basic_ops.TestNetworkBasicOps.test_router_rescheduling). Close cntt-n/CNTT#1106 Change-Id: I9fd79c9a82bb1f84a622ccb9ea316b8e43273e2a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-18Update test case list in neutron-tempest-plugin-apiCédric Ollivier1-2/+24
It takes into account latest changes in CNTT [1] [1] https://github.com/cntt-n/CNTT/pull/1107 Close cntt-n/CNTT#1106 Change-Id: I3212d1b765f7fd657ed603ed520d79fccae80bfa Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-17Fix incorrect test numbersCédric Ollivier1-1/+1
[1] http://artifacts.opnfv.org/functest/16S4PBMRW5HP/functest-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-run-23/neutron-tempest-plugin-api/tempest-report.html Change-Id: I9cb04c3abdbb899d2f1d4188a9fb9743a8f4c6d0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-16Exclude vxlan testing in CNTTCédric Ollivier1-1/+5
Close cntt-n/CNTT#1077 Change-Id: If9c0f52138e9f669f17933c42906936e2e196b2a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-15Set dhcp_agent_scheduler as optionalCédric Ollivier1-3/+5
dhcp_agent_scheduler [1] indirectly forces the Neutron backends as highlighted by OVN testing. [1] https://specs.openstack.org/openstack/neutron-specs/specs/api/agent_schedulers.html Close cntt-n/CNTT#1073 Change-Id: Iafa3ad7f15dc157c0362ffa59eed31b89bf368e7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-15Order regex in neutron-tempest-plugin-api (CNTT)Cédric Ollivier1-4/+5
It mostly adds admin as prefix. Change-Id: Ic2e723b858ff1f3e26478f6037a19db1fd52e4aa Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-15Decrease test_count to 1290 in tempest_fullCédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/2Y4502QRM7YM/functest-opnfv-functest-smoke-cntt-latest-tempest_full-run-16/tempest_full/tempest.log Change-Id: Ifa1ae117db7dff6471ab557d6f7f3bf03940de76 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-15Increase test_count to 547 in neutron-tempest-plugin-apiCédric Ollivier1-1/+1
It takes into account the new tests added in Neutron Tempest Plugin. Change-Id: I7994638c69ef6bd39bae3c6ded8409baabc27e9d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-01-09Remove test_server_connectivity_cold_migration_revert from ComplianceCédric Ollivier1-1/+2
It's skipped due to an upstream bug [1] highlighted by Jerma runs [2]. [1] https://launchpad.net/bugs/1836595 [2] http://artifacts.opnfv.org/functest/UXR96KDNMPTP/functest-opnfv-functest-smoke-cntt-jerma-tempest_slow-run-4/tempest_slow/tempest-report.html Change-Id: Ib4cbf0e6e8e010febd39dec8dd8ecd773ad28c3e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-30Remove test_server_connectivity_cold_migration_revert from exclusion listCédric Ollivier1-2/+1
It has to be part of the Compliance testing Co-Authored-By: Deepanshu Bhatia <deepanshu@voereir.com> Change-Id: I16d35e3c3a14b6b18901b6b27e50fc709ce0559f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-27Set port_security: true in tempest_conf.yamlCédric Ollivier1-2/+1
It allows running test_boot_into_disabled_port_security_network_without_secgroup. It also sets a few default values. Change-Id: I851c5d13705e0419a7e664e05d32eec0efb417c6 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-27Set create_shared_resources to trueCédric Ollivier1-1/+1
It takes last Neutron Tempest plugin change into account [1]. It also increases test count. [1] https://review.opendev.org/#/c/692822/ Change-Id: I5e7218dd808069a0be6d481447576099aa9c63e9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-07Run tests sequentially in tempest_slowCédric Ollivier1-1/+1
concurrency is set to 2 in the upstream job [1] but it conflicts with serial also passed. As it fails once without any other test in parallel, the number of executors should be decreased [2]. [0] https://799189c1bf6a1aa9c8de-48970bda2a30ee836a70a12e61555969.ssl.cf1.rackcdn.com/694893/1/check/tempest-slow/1bf5851/job-output.txt [1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-cntt-hunter-tempest_slow-run-7/tempest_slow/tempest-report.html Change-Id: I51bc9a584cf4b32a88b7a84d9387e7b8e90281c2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-07Increase tests_count (538) in CNTT neutron-tempest-plugin-api-runCédric Ollivier1-1/+1
New tests have been published in the upstream project [1]. [1] https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-run/4/console Change-Id: I46603ed78f865f89fe2a7ebcf6a72da38a02bb14 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-26Fix CNTT tempest_slow regexCédric Ollivier1-7/+8
It also deactivates ipv6 tests as for tempest_scenario. Change-Id: Ie383e9c5b96fa4b7c989979a8c0bc8d9875d95ef Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-26Add tempest_scenario and tempest_slow in smoke-cnttCédric Ollivier1-0/+48
The testcases related to ipv6 are currenty not executed because they cannot be covered by our SUTs. Close cntt-n/CNTT#696 Change-Id: Ic0ca3818011bede965702e778f37752059c86173 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-26Complete tempest_full for CNTT ComplianceCédric Ollivier1-1/+22
It forbids skipping any test and counts the testcases executed. It mosly avoids running tests accoring to the upstream skip tests (known bugs), related to nova-network, glance v1, keystone v2 and all features defined as optional by CNTT. It fixes an issue highlighted by yallint. Change-Id: Ib4678cb9e9fc37b16fcf21d8418931eae0f667d0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>