aboutsummaryrefslogtreecommitdiffstats
path: root/docker/smoke/testcases.yaml
AgeCommit message (Collapse)AuthorFilesLines
2019-04-14Disable falsy Patrole subtestsCédric Ollivier1-0/+2
It disables the two subtests which are failing in OpenStack Gates [1]. [1] http://logs.openstack.org/51/651851/1/check/patrole-admin/48dbeb0/testr_results.html.gz Change-Id: I6814d003d80593d2a89fefa439f0775187eb6f4d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-04-02Remove 1 Patrole test with a hardcoded vxlan idCédric Ollivier1-0/+1
It may fail when testing OpenStack in production or when running testcases in parallel [1] because vxlan 200 is hardcoded [2] [1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-hunter-patrole-run-75/results/patrole/tempest-report.html [2] https://github.com/openstack/patrole/blob/master/patrole_tempest_plugin/tests/api/network/test_networks_rbac.py#L208 Change-Id: Ibfaed6f3a95d016790baafebd203e49e152a889b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-03-19Do not allow testcase skipping in refstackJuha Kosonen1-0/+1
Set the result of refstack_defcore as failed if there were skipped testcases in a run. Change-Id: Id782c2dee511cefbafd5b4440f386fd66f934a47 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2019-03-12Remove Snaps-based testcasesCédric Ollivier1-16/+0
Snaps hasn't been synchronized for a while regarding requirements. We do remove it due to the inactivity. All test result tabs will be updated in a second change. Change-Id: I834afd902829ed3883b0e88e92aa806ec43d6fcf Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-02-18Only run bgpvpn tempest api testsCédric Ollivier1-1/+1
It conforms with SFC regex and fits OPNFV criteria. Running scenarios may force the use of bagpipe [1]. The SUT (functest-virtual) is being updated to verify SFC and BGPVPN apis. [1] https://docs.openstack.org/networking-bgpvpn/latest/user/drivers/bagpipe/index.html Change-Id: I1442b9cac7eaba1aeef04e36a2f790bd29f90976 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-02-13Fix smoke testcase listCédric Ollivier1-14/+0
Rally_jobs were moved into benchmarking. It completes "Move testcases in the right tiers" [1] [1] https://gerrit.opnfv.org/gerrit/#/c/66875/ Change-Id: I1c2da77eff6682f85ab8504750077c9ee542edb8 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-02-11Remove tempest_smoke from smokeCédric Ollivier1-18/+0
It completes "Move testcases in the right tiers" [1] [1] https://gerrit.opnfv.org/gerrit/#/c/66863/ Change-Id: Ic2e6ebf9371de87caafbfdb54029a24440e6fd5b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-02-06Move testcases in the right tiersCédric Ollivier1-0/+34
Here are the moves: - tempest_smoke in healthcheck - tempest_full and tempest_scenario in smoke - rally_jobs in benchmarking It avoids conflicts between benchmarking tools and few functional tests (short timeouts). Then it should allow running in parallel all testcases in one tiers. It also allows cleaning the additional steps defined in releng. rally_full could move in benchmarking in a second step. Change-Id: I0bc657557a61c7d9bf7679b4f2dbda08c36728ca Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-01-26Set Gnocchi scenario as optional in rally testsJuha Kosonen1-0/+6
The execution of an optional scenario is skipped if the corresponding openstack service does not exist. Change-Id: I8721f26ebc253ce02138b85d6c88e659c5afae80 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-12-20Generate Neutron policy.json (Patrole)Cédric Ollivier1-0/+3
Keystone policy.json et al will be managed in other changes to allow cherry-picking. It blacklists two subtests which requires policy actions not found in policy files. Change-Id: I5365fb1b24f7f07563037634d00cb9898fbde0fc Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-12-19Add task arguments to rally_jobsJuha Kosonen1-1/+0
Define floating network as a Rally task argument. Enable rally_jobs subtest. Change-Id: I37d7c4718cfb3de3ad1980878e0529d652013ccb Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-12-15Temporarily disable rally_jobs and patrole subtestsCédric Ollivier1-1/+1
The public network now seems hardcoded (public) [1] which doesn't fit the OPNFV model [2]. Neutron also stops publishing policy.json [3]. Upstream files have to be updated or fixed in Functest. [1] http://artifacts.opnfv.org/functest/opnfv-functest-smoke-rally_jobs-latest-run-25/functest.log [2] https://review.openstack.org/#/c/620952/ [3] https://review.openstack.org/#/c/585037/ Change-Id: Ibae4ce71a45afa3ed4a987577937220b8a3a89f9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-10-23Revert "Temporarily disable test_address_scope_rbac"Cédric Ollivier1-1/+0
This reverts commit 6939526b9802539a62f71ca7759f7bd49c16aef4. Change-Id: Ieb6644b47cc1567d5312f27e66ce250193db1f62 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-10-21Temporarily disable test_address_scope_rbacCédric Ollivier1-0/+1
It will be safely reverted when it'is fixed in patrole [1]. [1] https://review.openstack.org/#/c/612167/ Change-Id: Ib072f833523456004b7f211ef951c4b053d82577 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-18Add support for Rally OpenStack CI test casesJuha Kosonen1-0/+11
Include Neutron rally-jobs. Change-Id: Ic1122962bd7d876d179729640a8a321bc36442d8 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-09-15Disable test_auto_allocated_topology_rbac in patroleCédric Ollivier1-0/+1
It fails at least vs OpenStack Queens. OPNFV Gambia Installers are still falsy running latest then we do disable it. It has to be double checked as soon as Functest can be verified vs Rocky or newer. Change-Id: I7126c61d0c26b16f9058ed813af3344ca2930a86 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-05Conform with Barbican gatesCédric Ollivier1-2/+2
tempest.conf conforms with Devstack post-actions [1]. One test is still disabled as it may force to sign all images. A deeper analysis has to be done before enabling it. [1] https://github.com/openstack/barbican-tempest-plugin/blob/master/tools/pre_test_hook.sh Change-Id: I163c0e7c4671ca67f0b6c127c29fdcb12a8e426b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-04Remove vmtp and shaker from docker/smoke/testcases.yamlCédric Ollivier1-27/+0
Change-Id: Ic5754821542e873bb7d33a8864a852993947b762 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-02Update Xtesting to 0.60Cédric Ollivier1-27/+12
Xtesting leverages on stevedore [1] which will bring much more flexibility (discovery, module or class renaming, etc). OPNFV Features have to be updated as well: - https://gerrit.opnfv.org/gerrit/61713 - https://gerrit.opnfv.org/gerrit/61715 - https://gerrit.opnfv.org/gerrit/61717 - https://gerrit.opnfv.org/gerrit/61719 [1] https://docs.openstack.org/stevedore/latest/ Change-Id: I0ce45eb6c9af66546c2c020eb962fa457f6c7e1a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-29Add concurrency parameter to refstack_defcore testsDimitrios Markou1-0/+3
In fraser release we observed that refstack tests are executed in parallel and that might causes problems in CI because without concurrency parameter it will try to use as many cpus as possible and that might lead to race condition issues. This patch is created for backporting purposes to fraser branch where the problem occurs. JIRA: DOVETAIL-662 Change-Id: I6cf0ad3a32e88f04abd00d69523aa8ac47c39470 Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com>
2018-08-17Update restack defcore syncCédric Ollivier1-1/+0
It stops depending on refstack_client which cannot work till OpenStack Queens. JIRA: FUNCTEST-977 Change-Id: I2340643de82ab10d8fb83fb3a7fbfd1f412b2d7b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-14Remove all lxd refsCédric Ollivier1-2/+0
All Joid runs are stopped, then these scenarios can't be maintained. Change-Id: Ida7c8c86cbd377024047d4abbbc1806ad65afc8a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-14Disable Vmtp if POD_ARCH = aarch64Cédric Ollivier1-0/+2
Vmtp package delivers amd64 binaries in both architectures. JIRA: ARMBAND-396 Change-Id: I4c6e010e47c8aa9adfafb373c261262d23ef382a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-09Add networking-sfc Tempest test suiteCédric Ollivier1-0/+20
Change-Id: I6f1452469cb5ffaf9b538ba87edaa174977b91c4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-09Add networking-bgpvpn Tempest test suiteCédric Ollivier1-0/+19
Change-Id: If3191d8e8c16854aa059e6483b3c0ea704777c05 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-08-08Conform with OpenStack gatesCédric Ollivier1-2/+22
It adds testcases and updates regexes to conform with OpenStack gates: - tempest-full - neutron-tempest-plugin-api It should be noted that tempest-smoke is no longer run in gates and two testcases are available for tempest-full as we can't manage Tempest combine. Criteria is set to 100% for tempest_full (it works in OpenStack gates). JIRA: FUNCTEST-1000 Change-Id: I77a0f533255483bdf61f0a5f60c9c437ce843591 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-18Temporarily disable test_image_signing testsCédric Ollivier1-1/+2
They don't pass in two different Queens deployments. The topic will continue upstream. Change-Id: I43d6b8306aee6934fa1545ec3ed8130bb9b23ee4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-16Fix typos in trunk-detailsCédric Ollivier1-1/+1
Else neutron_trunk is skipped [1]. [1] https://build.opnfv.org/ci/view/functest/job/functest-compass-baremetal-daily-master/184/console Change-Id: I5d63e84414d8fc3af3ef2ac5b4d023fc5e59ef81 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-16Rename Patrole config fieldsCédric Ollivier1-2/+2
Else it conflicts with the new tempest config fields [1]. It makes sense to switch to includes here. [1] https://build.opnfv.org/ci/view/functest/job/functest-compass-virtual-daily-master/566/console Change-Id: If7b3025b4003ea5704c6a3cdeae8cd00f1c5509e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-15Dynamically skip neutron_trunk and barbicanCédric Ollivier1-6/+7
It avoids fasly testcases in failure for end users. It also removes the neutron_trunk class which is now useless (Queens). Change-Id: I3f5a1ac84a50e061da6eadcc5c9bafb2e99050db Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-15Include neutron_tempest_plugin in tempest-[smoke,full]Cédric Ollivier1-2/+3
Change-Id: I19f9a6ff2f76a4684bf341b2f22ffc4d09dca1cc Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-12Leverage to Xtesting 0.51Cédric Ollivier1-26/+3
It removes ci_loop which has been useless from Docker slicing. It can filter now on any env var (e.g. CI_LOOP if required by XCI). Change-Id: Ibd685edf4636bd251c85466e8b4febcd1e20d312 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-09Merge "Refactor tempest common"Cedric Ollivier1-4/+10
2018-07-09Add Vmtp as a new testcaseCédric Ollivier1-0/+17
VMTP is a small python application that will automatically perform ping connectivity, round trip time measurement (latency) and TCP/UDP throughput. JIRA: FUNCTEST-980 Change-Id: I8e94fc52a429f505ac14e14bbb9d0a3b20996ba4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-08Refactor tempest commonCédric Ollivier1-4/+10
It also renames two testcases and updates all docs because tests run in parallel in case of tempest smoke. To force one thread as before: run: module: 'functest.opnfv_tests.openstack.tempest.tempest' class: 'TempestCommon' args: mode: '^tempest\.(api|scenario).*\[.*\bsmoke\b.*\]$' option: - '--concurrency=1' JIRA: FUNCTEST-989 Change-Id: I1348f5908400a5046be8edf983db4afa12e546c3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-06-26Disable all new testcases for MS3.1Cédric Ollivier1-1/+1
It has been decided in TSC release meeting. Change-Id: I0a62f18c5b11fe31ac72fc301c5444e8d89b6fff Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-06-25Move odl into healthcheckCédric Ollivier1-20/+0
It should be included in healthcheck as all basic tests. Change-Id: I48439fe6385a9056eed8b779290f20633d59b0fd Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-06-25Move vping_* and cinder_test into healthcheckCédric Ollivier1-49/+0
Change-Id: I193c6dcdf90fe8eac46884fe5a5a329b0e5a7322 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-06-21Add the Barbican test suiteCédric Ollivier1-0/+15
It eases verifying the functionality of a barbican installation. Change-Id: Iac2144ae6ba4d61b2b37bb5c7a0359edb3f222a6 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-06-21Integrate Shaker as a new testcaseCédric Ollivier1-0/+18
Shaker wraps around popular system network testing tools like iperf, iperf3 and netperf (with help of flent). Change-Id: Idd82ffd9642c90335ad156e899c0330473260f15 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-05-25Add cinder_test testcaseDelia Popescu1-0/+17
Create 2 instances with Floating IPs Attach data volume to instance 1 Write data on the volume Detach volume from instance 1 Attach volume to instance 2 Read data from volume JIRA: FUNCTEST-908 Change-Id: I10efdfb3b7f9d7e6ab5c6a538052437585d52fb0 Signed-off-by: Delia Popescu <delia.popescu@enea.com>
2018-05-07Simplify vping_sshCédric Ollivier1-2/+2
It directly calls ping from the ssh socket. It fixes a possible missing security rule. It stops blocking if vpings are in failure. Change-Id: I64092a155bb134fa8b7d31d0d13f695e019540d1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-04-15Remove odl_netvirt testcaseCédric Ollivier1-22/+0
Its dendencies have not been met since Danube [1]. [1] https://build.opnfv.org/ci/job/apex-os-odl_l3-nofeature-noha-baremetal-danube/ Change-Id: I1a5bb4acd9e7e14dbf519132847521b5138f4cd6 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-04-15Set same dependencies for all vping test casesCédric Ollivier1-1/+1
It allows a fair comparison between odl and nosdn scenarios. Gluon could also be safely removed as scenario filter. https://build.opnfv.org/ci/job/functest-compass-baremetal-daily-master/88/console Change-Id: I0a9268d2038c79320e372d1d35d276560555aa7d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-04-13Disable refstack_defcore due to testrCédric Ollivier1-0/+1
refstack_client is hardcoded to testr when normalizing tests list. It's disabled till refstack_client conforms with OpenStack Queens (.testr.conf was removed from tempest). Change-Id: I10c7059e049017db994003eb5d42da83e1d5a839 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-04-13Disable volume in Patrole 0.3 by defaultCédric Ollivier1-1/+0
Else it fails without /etc/cinder/policy.json. Change-Id: Iaa5fb5b42626f2fbe00c3a89fa3edf4eae4ac166 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-04-03Remove NetVirt from installer dependenciesCédric Ollivier1-2/+2
Change-Id: Idb937b3ed4896208048b90b705dc3e6db6104b60 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-15Set blocking of patrole falseLinda Wang1-1/+1
Change-Id: Ia6f0c6fec30af5c4124f21b2bdaa3084bced6f9e Signed-off-by: Linda Wang <wangwulin@huawei.com>
2018-03-13Implement trunk ports testcasePanagiotis Karalis1-0/+17
The tempest testcases have been integrated from Neutron trunk testcases. It also enables every Neutron extension which may raise side effects [1]. [1] https://docs.openstack.org/tempest/latest/sampleconf.html JIRA: FUNCTEST-926 Change-Id: I05015acf119f504ce0a5539d677719de874594ad Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2018-03-13Add patrole in smokeCédric Ollivier1-0/+22
It also removes the former container which had helped when checking possible regex issues. I should be noted that patrole is already part of opnfv/functest-restapi. Change-Id: I76783bc3d121dfcef5b709f21a5e474354e93705 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>