aboutsummaryrefslogtreecommitdiffstats
path: root/functest/ci/testcases.yaml
AgeCommit message (Collapse)AuthorFilesLines
2020-12-09Rally_jobs asks for Router in several tasksCédric Ollivier1-0/+2
NeutronNetworks.create_and_list_ports can be executed when no L3 if the appropriate params are passed (see rally_sanity and rally_full). As rally_jobs is an official task which shouldn't be changed, it simply skips rally_jobs rather than NeutronNetworks.create_and_list_ports [1] [1] http://artifacts.opnfv.org/airship/WR7HM1BV0NA2/airship-opnfv-functest-benchmarking-latest-rally_jobs-run-184/rally_jobs/rally_jobs.html#/NeutronNetworks.create_and_list_ports Change-Id: I9bccb7417bbd514bd213a6b103f62165de497c00 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 80f380502edf592015df1adf542b6a143da98e7d)
2020-11-26Skip dataplane benchmarkign if no tenant resourcesCédric Ollivier1-2/+2
It asks for routers for instance. Change-Id: I7c7468b21ecf0578895735b5bc83b90a752813d0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f0cac210e048bae9b33180a799af041def5bfb05)
2020-11-26Allow running cloudify without tenant network resources.Cédric Ollivier1-0/+8
It fits the Airship deployment where there is neither tenant networks nor floating ips. It skips all VNF test cases as they ask for tenant resources. Change-Id: Ib5f2074314ff940595ce1387912055fa916a1f79 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 09be8310be708661f2782e6af1d1d84ac0a52946)
2020-11-14Allow connecting vm to the external networkCédric Ollivier1-0/+4
It fits the Airship deployment where there is neither tenant networks nor floating ips. It only updates the scenarios and must be completed by other changes for all the advanced testcases (Rally, Tempest, etc) Change-Id: Ia19f24d484b7b384c4aeeb3be35aac7aee31baab Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit d99e61f9eb1c4a582b71eb304259ddd3fdeb748f)
2020-09-05Update Xtesting to 0.91.0Cédric Ollivier1-6/+0
Change-Id: Icd467525f4b1b1f97d14a5ec3d5d78b94fd1d4a9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 6d1302779501918acdbbd4047a67a010901814b1)
2020-07-15Update functest/ci/testcases.yamlCédric Ollivier1-63/+302
It takes all CNTT changes into account. Change-Id: Ib6ddbd12695266fc78fd8ae4cce451bf8a4b95ad Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f36cc92bb628b53ed3a58fa84a24f94f006d6f30)
2020-07-03Fix tempest_scenarioCédric Ollivier1-1/+1
It completes "min_computes_nodes is also used as max values". Change-Id: I72c67e17015bc35ac5e14f20839cf74353695373 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit b765cf825b1a0d0853420b4a4bb7a154063f117e)
2020-06-05Switch back to concurrency = 4 in tempest testcasesCédric Ollivier1-7/+9
3 seems good enough and stable which allows trying 4 as before Change-Id: Id1f17ebf73f9faa13ed3ceb327ccb4ecf88c2bf2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit c7296454e80cea8273700da13e720e7f357f008e)
2020-05-01Rename barbican->tempest_barbicanCédric Ollivier1-1/+1
Change-Id: Ia62394795db3555fba56a7a333d9f0db226cedbe Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-01Remove neutron_trunkCédric Ollivier1-18/+0
It was merged in neutron-tempest-plugin. Change-Id: I87ca29154f727021c007a386badcadc9f3858506 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit c31996fccb822a8839857590f4fcd23a8511b279)
2020-04-28Rename neutron_tempest_plugin_api to tempest_neutronCédric Ollivier1-1/+1
It conforms with the other test case names Change-Id: Ia8836375c26ea4f1bff3637f7078e5f70f83c5fb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f74088bb1ae93feaf56c7ec7d1f1e78c97d8de9a)
2020-04-19Run sequentially tempest_heatCédric Ollivier1-1/+1
It's forced by design else it remains a few resources. It also sets fixed_subnet_name and network_for_ssh to avoid falsy discovery as it seems [1] [1] http://artifacts.opnfv.org/functest/X5GDYF3B0R2B/functest-opnfv-functest-smoke-jerma-tempest_heat-run-18/tempest_heat/tempest-report.html Change-Id: Ieb584ddc1c0b4ebb543e9d69277a20aaee0fbac0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit c2e3d911d9eaf987d18b3b895778c411e62efc4a)
2020-04-11Add healthcheck Horizon testsCédric Ollivier1-0/+15
It runs TestDashboardBasicOps as proposed by tempest-horizon [1]. It's skipped if DASHBOARD_URL is unset (default). [1] https://opendev.org/openstack/tempest-horizon Change-Id: I1b6b5609412770609f412da4927641588b173ed6 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 7b03643170974d4307e0f630e4c79340df98e06f) (cherry picked from commit 9441711c5a03a7a8c476dcc28e1fbace38b48654)
2020-04-11Add tempest_heat (heat-tempest-plugin)Cédric Ollivier1-0/+15
It asks for the Fedora-Cloud-Base image and a second user account. It should be noted that heat-tempest-plugin duplicates all configs from tempest. Minimal image is still Cirros and the first account is the admin one. Then it conforms with the TempestCommon logic. AodhAlarmTest is skipped by default because pre conditions are missing. SoftwareConfigIntegrationTest is skipped as in gates because it requires a custom image [1] [1] https://github.com/openstack/heat/blob/master/devstack/lib/heat#L444 Change-Id: I8b39dc65ef3714411cd828e17b95e124f2e90f34 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-04Reduce Tempest multi threadsCédric Ollivier1-7/+7
It decreases the threads from 4 to 3 to allow running the testcases in parallel. Else a few tests sometimes fail. Change-Id: I268eff5a72d5de5b26d7549ac74ee0e2a05cb61e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit c0e9550a76bd91322a6bb5dc4b7f9947e1794913)
2019-12-07Run tests sequentially in tempest_slowCédric Ollivier1-1/+1
concurrency is set to 2 in the upstream job [1] but it conflicts with serial also passed. As it fails once without any other test in parallel, the number of executors should be decreased [2]. [0] https://799189c1bf6a1aa9c8de-48970bda2a30ee836a70a12e61555969.ssl.cf1.rackcdn.com/694893/1/check/tempest-slow/1bf5851/job-output.txt [1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-cntt-hunter-tempest_slow-run-7/tempest_slow/tempest-report.html Change-Id: I51bc9a584cf4b32a88b7a84d9387e7b8e90281c2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 83c3cc3f9b71de33cd1e88c11da4879156be7cf3)
2019-11-26Add tempest slowCédric Ollivier1-0/+17
It conforms with tempest-slow-py3 executed in Neutron Gates [1]. [1] https://logs.opendev.org/68/672568/6/check/tempest-slow-py3/49fece6/testr_results.html.gz Change-Id: I0c1c6f7be38c9fcdd1a32e5cfa2bc4d8414885e0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit bbe35dc42e442fa2669a24e7927d389188e42b4c)
2019-11-14Add keystone-tempest-pluginCédric Ollivier1-0/+15
It adds tempest_keystone which runs the tests proposed by keystone-tempest-plugin [1]. [1] https://opendev.org/openstack/keystone-tempest-plugin Change-Id: I83db584c37fe63a549ccf396b03e034778254b3a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 22ec0c1008b81f368476d8cea84117a93d15dab2)
2019-11-14Add cinder-tempest-pluginCédric Ollivier1-0/+16
It adds tempest_cinder which runs the tests proposed by cinder-tempest-plugin [1]. test_incremental_backup is disabled because it currently fails vs Functest SUT (under analysis). [1] https://opendev.org/openstack/cinder-tempest-plugin Change-Id: I85b255eb05b9ab1c74c3829148da5230d6c7647e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit e46aa41172fa1bf63364505fbbf922323788ac98)
2019-10-26Allow checking Tests countCédric Ollivier1-0/+1
It meets CNTT requirements (verifying conformance with a reference implementation) and may protect vs synchronization issues (see gambia refstack_defcore). Change-Id: I842bc4ed91eec6bc545c489e3686d0d41dae7b44 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 7c67624b8ee7f7a9a7fb36bb25e9881a63c982d4)
2019-10-04Update Shaker image (1.3.0 + Debian Stretch)Cédric Ollivier1-0/+2
It udates Shaker to 1.3.0 and switches to Debian Stretch as base image. It also eases debugging as ssh basic auth is enabled (shaker/shaker). The arm image has to be built before enabling the testcase for arm. Change-Id: I894bae86c7e8f7cafbba2b4e00e3334afed0b6e2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 574243656df6acae28094c62f33410ef1fc3e7b5)
2019-09-14Remove any instance quota in tempest_scenarioCédric Ollivier1-1/+1
The next testcase can reach the default limit if lots of computes available [1]. tempest.scenario.test_server_multinode.TestServerMultinode.test_schedule_to_all_nodes [1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-hunter-tempest_scenario-run-325/results/tempest_scenario/test_raw_list.txt Change-Id: Ibf68e2a367199afee3f6198ebf683be6cd938d94 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit bbe8c3e48543eca31493cf33b02e1d9a48de2280)
2019-06-09Blacklist test_list_metadef_namespaces (Patrole)Cédric Ollivier1-0/+1
It seems dependent on test order (404 if no namespace) and sometimes fails in gates [1][2][3]. It will be enabled as soon it's fixed upstream or if SUT is wrong here. [1] https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-latest-patrole-run/ [2] https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-iruya-patrole-run/ [3] https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-hunter-patrole-run/ Change-Id: I68e71e9b0753089e5620d6689ddff37fe66aeab7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 02b9592a129472ffd26206541f8efefe1f759957)
2019-05-11Revert "Disable Patrole network.test_agents_rbac"Cédric Ollivier1-1/+0
This reverts commit 4fd59de2b5751437ca6faf71415129cf6f64cf8f. It should have been applied in master and iruya only. Change-Id: I8f467e5cbe2f230a0a3f119a3f4ad80da8a5a6bf Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-05-08Disable Patrole network.test_agents_rbacCédric Ollivier1-0/+1
They don't clean routers (tempest-L3AgentsExtRbacTest-Router) after testing. Change-Id: I78cfc783feb51d0f1b029c98e988c07e57e1e694 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit d4178d425312497412e9832fb6cb1a074e5099f3)
2019-04-25Enable stor4nfv in HunterCédric Ollivier1-1/+1
The branch is ready. Change-Id: I0b246cf3bb83eb7ef0151509d2e99c45032b7a7e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-04-02Remove 1 Patrole test with a hardcoded vxlan idCédric Ollivier1-0/+1
It may fail when testing OpenStack in production or when running testcases in parallel [1] because vxlan 200 is hardcoded [2] [1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-hunter-patrole-run-75/results/patrole/tempest-report.html [2] https://github.com/openstack/patrole/blob/master/patrole_tempest_plugin/tests/api/network/test_networks_rbac.py#L208 Change-Id: Ibfaed6f3a95d016790baafebd203e49e152a889b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 8ee279600186f2713b4c181965d8d290d2644cfb)
2019-03-19Do not allow testcase skipping in refstackJuha Kosonen1-0/+1
Set the result of refstack_defcore as failed if there were skipped testcases in a run. Change-Id: Id782c2dee511cefbafd5b4440f386fd66f934a47 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com> (cherry picked from commit cfd79ec4c39e215a25fc3d99fc5368e7d874f7bf)
2019-03-13Remove Snaps-based testcasesCédric Ollivier1-43/+0
Snaps hasn't been synchronized for a while regarding requirements. We do remove it due to the inactivity. All test result tabs will be updated in a second change. Change-Id: I834afd902829ed3883b0e88e92aa806ec43d6fcf Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit e44841210c53290207db9b958c66cb8019910cb7)
2019-03-08Remove SDNVPN as it doesn't participate to hunterCédric Ollivier1-12/+0
Change-Id: Iddeaa3890635103b66042a918ab66cabd8fdee60 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f799428f8839a59973ba7edb3b6e66819e27e5ee)
2019-02-18Only run bgpvpn tempest api testsCédric Ollivier1-1/+1
It conforms with SFC regex and fits OPNFV criteria. Running scenarios may force the use of bagpipe [1]. The SUT (functest-virtual) is being updated to verify SFC and BGPVPN apis. [1] https://docs.openstack.org/networking-bgpvpn/latest/user/drivers/bagpipe/index.html Change-Id: I1442b9cac7eaba1aeef04e36a2f790bd29f90976 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 2b4e7955815ce27284043c03730fff6ace029a0d)
2019-02-12Set gnocchi as optional in Rally testcasesCédric Ollivier1-0/+9
ci/testcases.yaml was copied/pasted when moving the rally testcases. It completes "Set Gnocchi scenario as optional in rally tests" [1]. [1] https://gerrit.opnfv.org/gerrit/#/c/66653/ Change-Id: Ia23ae9764750775fe8983660c56ced06f3de977a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 44db92e6135a573ed4bb23648ea1c73e8a70f91b)
2019-02-12Move rally_full in benchmarkingCédric Ollivier1-18/+12
It allows merging all benchmarking testcases in one tiers and saves lots of time as they could all run in parallel. Change-Id: I551179709d2ce862056c1f5fcc1e16ef0b776dd4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit cec721055631734d2d1d39bd09cfa1404ee1d32e)
2019-02-11Update and enable Cloudify-based testcasesCédric Ollivier1-2/+0
It switches from the Cloudify virtual machine to the community container which allows deploying vyos_vrouter and cloudify_ims. Last stable Cloudify OpenStack plugin (2.4.17) is now deployed. It also updates the Cloudify python package to latest release. It adds a delay before deleting the blueprint which may be improved in a second change (it allows enabling the testcase in the stable release). Change-Id: Ie48a2c9d2badab0d7ba0b9e7845f453fad335ef7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 97dde581982b9d49122aee3630328fb6064c8609)
2019-02-07Move testcases in the right tiersCédric Ollivier1-48/+48
Here are the moves: - tempest_smoke in healthcheck - tempest_full and tempest_scenario in smoke - rally_jobs in benchmarking It avoids conflicts between benchmarking tools and few functional tests (short timeouts). Then it should allow running in parallel all testcases in one tiers. It also allows cleaning the additional steps defined in releng. rally_full could move in benchmarking in a second step. Change-Id: I0bc657557a61c7d9bf7679b4f2dbda08c36728ca Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 1d0f0bbf773aaaf7139b76442a9e59d13bd32830)
2019-02-03Update Cloudify imagesCédric Ollivier1-0/+2
The former image can't be downloaded and then breaks all gates. It will be completed by changes in vnf if needed. Change-Id: Ie364aa2315257157a5e09eb4ba121926a60e7b26 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit f501b520f80d573a77cdae941fe19db216b94307)
2019-01-14Remove fds in all branchesCédric Ollivier1-17/+0
FDS didn't participate to Gambia. Removing it in all branches avoids compexifying the jjbs. This patch will be reverted as soon as the support is ready. Change-Id: I9fb293b2972e0deb318ac9d3d08377039ea247e4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 2fd07adb36288e9247ad73a7344340c36754132e)
2018-11-06Run vnfs in all scenariosCédric Ollivier1-8/+0
It allows fair comparisons between scenarios (e.g. ovs dpdk). os-nosdn-calipso-noha could be verified VNFs as well [1]. [1] https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-gambia/32/console Change-Id: I0490eade81c531e38ce7156a61446a9fccaffb17 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 61b708cc0987bf48d07df9d46ed25e4b42c15efc)
2018-10-09Enable bgpvpn and barometerCédric Ollivier1-2/+0
They are being synchronized to OpenStack Rocky. Change-Id: I58f45bac24714faaa1cc2738ef04b50cd179f7be Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-10-06Enable sfc and doctor in hunterCédric Ollivier1-2/+0
sfc [1] and doctor [2] master are being synced with OpenStack Rocky. [1] https://gerrit.opnfv.org/gerrit/#/c/63239/ [2] https://gerrit.opnfv.org/gerrit/#/c/63237/ Change-Id: Id5d41fa98ab5f00703d557ce82cb2037bc827c2b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-20Add support for Rally OpenStack CI test casesJuha Kosonen1-0/+11
Include Neutron rally-jobs. Change-Id: Ic1122962bd7d876d179729640a8a321bc36442d8 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com> (cherry picked from commit 5d54a533d25a989dc64a2789788ff2f2fd6ad5c6)
2018-09-15Disable test_auto_allocated_topology_rbac in patroleCédric Ollivier1-0/+1
It fails at least vs OpenStack Queens. OPNFV Gambia Installers are still falsy running latest then we do disable it. It has to be double checked as soon as Functest can be verified vs Rocky or newer. Change-Id: I7126c61d0c26b16f9058ed813af3344ca2930a86 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 9b941ea58c60827d563d266f7dedd0d95de3ca93)
2018-09-13Disable all OPNFV FeaturesCédric Ollivier1-0/+6
For the time being, none is synced to run vs OpenStack master or Rocky. Change-Id: Iad0b99cfc1407fb514ddf4580363e03d94b4a498 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit b6fdf896bf87ed23ff766fe59f2072fe8844f2ed)
2018-09-05Conform with Barbican gatesCédric Ollivier1-2/+2
tempest.conf conforms with Devstack post-actions [1]. One test is still disabled as it may force to sign all images. A deeper analysis has to be done before enabling it. [1] https://github.com/openstack/barbican-tempest-plugin/blob/master/tools/pre_test_hook.sh Change-Id: I163c0e7c4671ca67f0b6c127c29fdcb12a8e426b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-02Move shaker and vmtp into functest-benchmarkingCédric Ollivier1-30/+37
It also updates all jobs and fixes Dockerhub hooks. Change-Id: Ib182155736b4dbdb829293081741a9d5cb8ea6d7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-02Update Xtesting to 0.60Cédric Ollivier1-85/+40
Xtesting leverages on stevedore [1] which will bring much more flexibility (discovery, module or class renaming, etc). OPNFV Features have to be updated as well: - https://gerrit.opnfv.org/gerrit/61713 - https://gerrit.opnfv.org/gerrit/61715 - https://gerrit.opnfv.org/gerrit/61717 - https://gerrit.opnfv.org/gerrit/61719 [1] https://docs.openstack.org/stevedore/latest/ Change-Id: I0ce45eb6c9af66546c2c020eb962fa457f6c7e1a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-29Add concurrency parameter to refstack_defcore testsDimitrios Markou1-0/+3
In fraser release we observed that refstack tests are executed in parallel and that might causes problems in CI because without concurrency parameter it will try to use as many cpus as possible and that might lead to race condition issues. This patch is created for backporting purposes to fraser branch where the problem occurs. JIRA: DOVETAIL-662 Change-Id: I6cf0ad3a32e88f04abd00d69523aa8ac47c39470 Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com>
2018-08-25Integrate Stor4NFV feature openstack test caseQiaowei Ren1-0/+14
Change-Id: I245d0fbe8f026a24138ede4041caae4dccc15cc0 Signed-off-by: Qiaowei Ren <qiaowei.ren@intel.com>
2018-08-20Add vgpu in global testcases.yamlCédric Ollivier1-0/+12
It completes the patch introducing vgpu [1]. [1] https://gerrit.opnfv.org/gerrit/#/c/60713/ Change-Id: Ib2a22dd656d365e87a37a239f0a0ee747c95b117 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-17Update restack defcore syncCédric Ollivier1-1/+0
It stops depending on refstack_client which cannot work till OpenStack Queens. JIRA: FUNCTEST-977 Change-Id: I2340643de82ab10d8fb83fb3a7fbfd1f412b2d7b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>