aboutsummaryrefslogtreecommitdiffstats
path: root/docker/smoke/testcases.yaml
AgeCommit message (Collapse)AuthorFilesLines
2023-05-13Update to RefStack 2022.11Cédric Ollivier1-2/+0
Change-Id: I7b7667987cb6369b27f2d15c670987ea99812335 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-12-06Skip test_volume_actions_rbac.VolumesActionsV310RbacTestCédric Ollivier1-0/+3
It completes the previous patch. http://artifacts.opnfv.org/functest/2BTYC9OZVCC2/functest-opnfv-functest-smoke-latest-patrole_member-run-188/patrole_member/tempest-report.html Change-Id: I03d3a21e72c5bc5335d77b8ce972601f8d2d273a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-12-06Exlude programming issues in Patrole test casesCédric Ollivier1-0/+27
http://artifacts.opnfv.org/functest/5R496QTS4IFG/functest-opnfv-functest-smoke-latest-patrole_admin-run-187/patrole_admin/tempest-report.html http://artifacts.opnfv.org/functest/5R496QTS4IFG/functest-opnfv-functest-smoke-latest-patrole_member-run-187/patrole_member/tempest-report.html http://artifacts.opnfv.org/functest/5R496QTS4IFG/functest-opnfv-functest-smoke-latest-patrole_reader-run-187/patrole_reader/tempest-report.html Change-Id: Icd494ed69efaea37818d5b63a9772c4625aa8e08 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-11-26Enable RefStack backCédric Ollivier1-2/+0
Change-Id: Ia87753bb12963c2da9515231b1241860546b7cfe Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-11-25Filter test_policy_minimum_bandwidth_rule_rbac in PatroleCédric Ollivier1-0/+3
https://artifacts.opnfv.org/functest/H5LU90NKXLQX/functest-opnfv-functest-smoke-xena-patrole_admin-run-13/patrole_admin/tempest-report.html Change-Id: Id4477c52b71147f47c590369a9030dd997c69da6 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-11-25Reduce Patrole test case listsCédric Ollivier1-0/+33
They seem related to Patrole obsolescence patrole_tempest_plugin.rbac_exceptions.RbacParsingException: Attempted to test an invalid policy file or action Details: Policy action "volume_extension:types_manage" not found in policy files: [] or among registered policy in code defaults for cinder service. Change-Id: I2e0bfeb150c8835b47bddac7130021963850397a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-10-15Update RefStack test case descriptionsCédric Ollivier1-4/+0
RefStack now allows skip tests. Change-Id: I32f57388be81e3ace326c76320a33ca16c01bdba Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2021-05-21Skip test_get_network_ip_availabilitiesCédric Ollivier1-0/+3
http://artifacts.opnfv.org/functest/T5HYYAOA3L02/functest-opnfv-functest-smoke-wallaby-patrole_admin-run-7/patrole_admin/tempest-report.html http://artifacts.opnfv.org/functest/H3A87GU7QKXC/functest-opnfv-functest-smoke-latest-patrole_reader-run-153/patrole_reader/tempest-report.html Change-Id: I7b2aabdec4bed6928e0672d73627f6ea67f811df Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-12-08Skip Restack compute and platform if NO_TENANT_NETWORKCédric Ollivier1-0/+4
Change-Id: Icdfd3ed23b6a5a47d95ae50aea88a46cd77be2ea Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-12-07Remove test_reset_network from Patrole test casesCédric Ollivier1-0/+3
Details: Policy action "os_compute_api:os-admin-actions:reset_network" not found in policy files: [] or among registered policy in code defaults for nova service. http://artifacts.opnfv.org/functest/6I7CUHIE1P9P/functest-opnfv-functest-smoke-latest-patrole_member-run-140/patrole_member/tempest-report.html Change-Id: I0ec24bab4fa3c692a4096b093d81b11dc5d1367c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-11-22Disable refstack platform and computeCédric Ollivier1-0/+2
RefStack 2020.06 is obsolete regarding latest OpenStack changes [1] "Metadata is read-only on non-Xen-based deployments." [1] http://artifacts.opnfv.org/functest/CX0VIHZZXPK9/functest-opnfv-functest-smoke-latest-refstack_compute-run-737/refstack_compute/tempest-report.html Change-Id: I6c86ae097d99f1af2cb6a58f3c14fd9b22afe3ed Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-16Enable back Patrole volume testingCédric Ollivier1-3/+9
It skips test_create_network_provider_segmentation_id as it can run in parallel (200 is hardcoded) and test_create_image_from_volume_backed_server which doesn't clean its resources. Change-Id: I8918ecc0a215a491bff9f4d507dacdd4a9d0b92c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-07Skip patrole volume testingCédric Ollivier1-3/+3
They don't clean volume and snapshot which raises side effects after lots of iterations. They may be obsolete too [1] [1] http://artifacts.opnfv.org/functest/4VZKU3ZR64DP/functest-ovn-opnfv-functest-smoke-latest-patrole_admin-run-9/patrole_admin/tempest-report.html Change-Id: I5562931a403dfdc5e3d75a1fcce52316d359482a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-07Skip test_backup_crossproject_*_negativeCédric Ollivier1-2/+4
They are currently failing and they don't seem executed in gates [1]. [1] http://artifacts.opnfv.org/functest/Y0L6XUXULRRL/functest-ovn-opnfv-functest-smoke-cntt-latest-tempest_cinder_cntt-run-108/tempest_cinder_cntt/tempest-report.html Change-Id: I397bad9feb79ecaea8477f2e931ab0b57237be0b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-09-05Update Xtesting to 0.91.0Cédric Ollivier1-1/+0
Change-Id: Icd467525f4b1b1f97d14a5ec3d5d78b94fd1d4a9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-08-30Skip obsolete Patrole testsCédric Ollivier1-0/+3
Details: Policy action "os_compute_api:os-floating-ips" not found in policy files: [] or among registered policy in code defaults for nova service. http://artifacts.opnfv.org/functest/OTNURZ1MTVG3/functest-opnfv-functest-smoke-latest-patrole_reader-run-86/patrole_reader/tempest-report.html Change-Id: I678b3c10d19006d9ed978c8fa1d9eed933012b84 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-07-20Add Patrole compute testingCédric Ollivier1-3/+3
It stops leveraging on policy json files as it fails for Nova. Change-Id: Ic70997d0ff8151956dc189739b8468def048aad7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-07-20Skip Patrole test_show_auto_allocated_topologyCédric Ollivier1-0/+3
test_show_auto_allocated_topology doesn't clean its resources: - network: auto_allocated_network - subnets: auto_allocated_subnet_v4 & auto_allocated_subnet_v6 - router: auto_allocated_router This bug easily breaks testing in a quick run when allocating the subnet due to insufficient prefix space to allocate subnet size /26. https://build.opnfv.org/ci/job/functest-latest-daily/746/ Change-Id: I1a1bd22068bca0720016dab8a15ad7dd576ee152 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-07-15Add tempest_cyborg based on cyborg tempest pluginCédric Ollivier1-0/+17
Change-Id: I61e982011040ee8b0540910939983777280445ca Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-07-05Reduce Patrole extra logicCédric Ollivier1-42/+21
It also reduces the skipped test list taking into account Patrole improvements. Change-Id: I23a8c0610a8a4258aa806451f52796f3e7bc46ca Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-07-04Patrole user -> memberCédric Ollivier1-2/+2
Change-Id: I050caff2d255118edff48403a87305cfe336e20a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-07-03Fix tempest_scenarioCédric Ollivier1-1/+1
It completes "min_computes_nodes is also used as max values". Change-Id: I72c67e17015bc35ac5e14f20839cf74353695373 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-07-03Add patrole user and reader testingCédric Ollivier1-1/+51
The regexes will be updated in a second change (chicken and egg between Functest and Releng) Change-Id: I4273a80799730fe30c941a7b2966cfee46f24d4a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-05Switch back to concurrency = 4 in tempest testcasesCédric Ollivier1-9/+11
3 seems good enough and stable which allows trying 4 as before Change-Id: Id1f17ebf73f9faa13ed3ceb327ccb4ecf88c2bf2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-26Remove test_gnocchi.GnocchiGabbiTest.test_liveCédric Ollivier1-1/+1
It sometimes fails in our gates and it seems unverified in OpenStack [1]. "Archive policy gabbilive is still in use" [1] http://artifacts.opnfv.org/functest/DET32CBF0L0F/functest-opnfv-functest-smoke-jerma-tempest_telemetry-run-29/tempest_telemetry/tempest-report.html Change-Id: Id5e5ad5ff0f7bade7932a085df28cc35b0ac7984 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-03Update octavia regexesCédric Ollivier1-3/+6
It stops using noop drivers and adds all scenarios except: - .*api.v2.test_availability_zone - .*api.v2.test_availability_zone_profile - .*api.v2.test_member.MemberAPITest.test_member_ipv4_create - .*api.v2.test_member.MemberAPITest.test_member_ipv6_create It doesn't run ipv6 which asks for a deep dive (ipv6 from containers?) It increases threads to 3 as the other testcases. Change-Id: I36d9d764a5ab92efe89196181bffa3ad0a57227c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-01Rename barbican->tempest_barbican & octavia->tempest_octaviaCédric Ollivier1-2/+2
It unifies all test case names. Change-Id: I38576f2d6bae6ef0236b6acfbd30c0446d3a2eca Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-01Remove neutron_trunk; networking_bgpvpn and networking_sfcCédric Ollivier1-55/+0
They were all merged in neutron-tempest-plugin. Change-Id: Iefe3bf98fc41d0dab5a7cec640337c672a7b1a69 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-29Add tempest_telemetryCédric Ollivier1-0/+20
It runs all telemetry-tempest-plugin tests except test_autoscaling which is currently failing in our gates. Change-Id: I2c712eda71799e96735419e1c547a775be659156 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-28Rename neutron_tempest_plugin_api to tempest_neutronCédric Ollivier1-1/+1
It conforms with the other test case names Change-Id: Ia8836375c26ea4f1bff3637f7078e5f70f83c5fb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-19Run sequentially tempest_heatCédric Ollivier1-1/+1
It's forced by design else it remains a few resources. It also sets fixed_subnet_name and network_for_ssh to avoid falsy discovery as it seems [1] [1] http://artifacts.opnfv.org/functest/X5GDYF3B0R2B/functest-opnfv-functest-smoke-jerma-tempest_heat-run-18/tempest_heat/tempest-report.html Change-Id: Ieb584ddc1c0b4ebb543e9d69277a20aaee0fbac0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-06Add tempest_heat (heat-tempest-plugin)Cédric Ollivier1-0/+15
It asks for the Fedora-Cloud-Base image and a second user account. It should be noted that heat-tempest-plugin duplicates all configs from tempest. Minimal image is still Cirros and the first account is the admin one. Then it conforms with the TempestCommon logic. AodhAlarmTest is skipped by default because pre conditions are missing. SoftwareConfigIntegrationTest is skipped as in gates because it requires a custom image [1] [1] https://github.com/openstack/heat/blob/master/devstack/lib/heat#L444 Change-Id: I8b39dc65ef3714411cd828e17b95e124f2e90f34 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-04Reduce Tempest multi threadsCédric Ollivier1-8/+8
It decreases the threads from 4 to 3 to allow running the testcases in parallel. Else a few tests sometimes fail. Change-Id: I268eff5a72d5de5b26d7549ac74ee0e2a05cb61e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-07Run tests sequentially in tempest_slowCédric Ollivier1-1/+1
concurrency is set to 2 in the upstream job [1] but it conflicts with serial also passed. As it fails once without any other test in parallel, the number of executors should be decreased [2]. [0] https://799189c1bf6a1aa9c8de-48970bda2a30ee836a70a12e61555969.ssl.cf1.rackcdn.com/694893/1/check/tempest-slow/1bf5851/job-output.txt [1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-cntt-hunter-tempest_slow-run-7/tempest_slow/tempest-report.html Change-Id: I51bc9a584cf4b32a88b7a84d9387e7b8e90281c2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-26Add keystone-tempest-pluginCédric Ollivier1-0/+15
It adds tempest_keystone which runs the tests proposed by keystone-tempest-plugin [1]. [1] https://opendev.org/openstack/keystone-tempest-plugin Change-Id: I83db584c37fe63a549ccf396b03e034778254b3a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-26Add cinder-tempest-pluginCédric Ollivier1-0/+16
It adds tempest_cinder which runs the tests proposed by cinder-tempest-plugin [1]. test_incremental_backup is disabled because it currently fails vs Functest SUT (under analysis). [1] https://opendev.org/openstack/cinder-tempest-plugin Change-Id: I85b255eb05b9ab1c74c3829148da5230d6c7647e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-26Update RefStack lists to 2019.06Cédric Ollivier1-3/+3
Change-Id: I9d3abc3996857d8905007640250b94025735809c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-23Allow checking Tests countCédric Ollivier1-0/+3
It meets CNTT requirements (verifying conformance with a reference implementation) and may protect vs synchronization issues (see gambia refstack_defcore). Change-Id: I842bc4ed91eec6bc545c489e3686d0d41dae7b44 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-09-14Remove any instance quota in tempest_scenarioCédric Ollivier1-1/+1
The next testcase can reach the default limit if lots of computes available [1]. tempest.scenario.test_server_multinode.TestServerMultinode.test_schedule_to_all_nodes [1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-hunter-tempest_scenario-run-325/results/tempest_scenario/test_raw_list.txt Change-Id: Ibf68e2a367199afee3f6198ebf683be6cd938d94 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-08-28Merge "Add tempest slow"Cedric Ollivier1-0/+17
2019-08-27Add Octavia testingCédric Ollivier1-0/+18
It integrates Octavia Tempest Plugin. The new testcase octavia currently covers API only [1]. [1] https://logs.opendev.org/46/674846/2/check/octavia-v2-dsvm-noop-api/fe37c58/testr_results.html.gz Change-Id: I2a11a19aa0c43701499dc9e17bd69b9387b53b58 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-08-26Add tempest slowCédric Ollivier1-0/+17
It conforms with tempest-slow-py3 executed in Neutron Gates [1]. [1] https://logs.opendev.org/68/672568/6/check/tempest-slow-py3/49fece6/testr_results.html.gz Change-Id: I0c1c6f7be38c9fcdd1a32e5cfa2bc4d8414885e0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-06-28Add Barbican scenario in RallyCédric Ollivier1-0/+1
It doesn't run BarbicanSecrets.create_symmetric_and_delete which is currently failing [1]. Barbican testing is optional as Gnocchi. [1] https://bugs.launchpad.net/rally/+bug/1819285 Change-Id: I9121ce3c9f29ba05069c905325327d4c4d0b9e71 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-06-09Blacklist test_list_metadef_namespaces (Patrole)Cédric Ollivier1-0/+1
It seems dependent on test order (404 if no namespace) and sometimes fails in gates [1][2][3]. It will be enabled as soon it's fixed upstream or if SUT is wrong here. [1] https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-latest-patrole-run/ [2] https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-iruya-patrole-run/ [3] https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-hunter-patrole-run/ Change-Id: I68e71e9b0753089e5620d6689ddff37fe66aeab7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-06-01Stop installing sfc and bgpvpn tempest pluginsCédric Ollivier1-2/+2
They are being integrated into neutron tempest plugin [1][2]. It will fix our gates [3]. [1] https://github.com/openstack/neutron-tempest-plugin/commit/1a987ecb4d310328cc4c2c5628dc5018dc7afc48 [2] https://github.com/openstack/neutron-tempest-plugin/commit/8dd49aac9fa27166d76755c7d62ea9f0553d8d74 [3] https://build.opnfv.org/ci/job/functest-latest-gate/199/ Change-Id: Iafcfd0cb9b33a271cf37f269d9a3a2086eb67414 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-05-16Add all RefStack listsCédric Ollivier1-3/+36
It adds platform and object targets. refstack_defcore is renamed to refstack_compute. Change-Id: Iad5bfb27be614aada98eef5b3148965e65210725 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-05-08Disable Patrole network.test_agents_rbacCédric Ollivier1-0/+1
They don't clean routers (tempest-L3AgentsExtRbacTest-Router) after testing. Change-Id: I78cfc783feb51d0f1b029c98e988c07e57e1e694 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-04-14Disable falsy Patrole subtestsCédric Ollivier1-0/+2
It disables the two subtests which are failing in OpenStack Gates [1]. [1] http://logs.openstack.org/51/651851/1/check/patrole-admin/48dbeb0/testr_results.html.gz Change-Id: I6814d003d80593d2a89fefa439f0775187eb6f4d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-04-02Remove 1 Patrole test with a hardcoded vxlan idCédric Ollivier1-0/+1
It may fail when testing OpenStack in production or when running testcases in parallel [1] because vxlan 200 is hardcoded [2] [1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-hunter-patrole-run-75/results/patrole/tempest-report.html [2] https://github.com/openstack/patrole/blob/master/patrole_tempest_plugin/tests/api/network/test_networks_rbac.py#L208 Change-Id: Ibfaed6f3a95d016790baafebd203e49e152a889b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-03-19Do not allow testcase skipping in refstackJuha Kosonen1-0/+1
Set the result of refstack_defcore as failed if there were skipped testcases in a run. Change-Id: Id782c2dee511cefbafd5b4440f386fd66f934a47 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>