aboutsummaryrefslogtreecommitdiffstats
path: root/functest/opnfv_tests/openstack
AgeCommit message (Collapse)AuthorFilesLines
2020-04-04Skip test_container_synchronizationCédric Ollivier1-0/+5
It's not supported by radosgw which is now used in Functest SUTs [1][2]. It's removed from CNTT which seems leveraging on ceph (at least RI) [3] [1] https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3.2/html/release_notes/bug-fixes [2] http://artifacts.opnfv.org/functest/E46RCZLBRYCC/functest-opnfv-functest-smoke-hunter-tempest_slow-run-127/tempest_slow/tempest-report.html [3] http://artifacts.opnfv.org/cntt/WKFRPQODPW34/cntt-opnfv-functest-smoke-latest-tempest_slow-run-37/tempest_slow/tempest-report.html Change-Id: I8127df786c41af0d0285ef0e5cd1b28901685f65 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-03-28Increase Shaker timeout before init message in consoleCédric Ollivier1-0/+1
It failed in parallel of rally_full [1]. [1] https://build.opnfv.org/ci/job/functest-opnfv-functest-benchmarking-hunter-shaker-run/397/console [2] https://build.opnfv.org/ci/job/functest-opnfv-functest-benchmarking-hunter-shaker-run/395/console Change-Id: I0eab7374c5962d45405c5f4147798067526f6e70 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-03-27Check the login prompt in console in SingleVm1Cédric Ollivier1-0/+2
It also checks the second vm2 console log in case of vping_ssh. Change-Id: I13a5edfb3e19449a38d2f0478d549bd8fcc5cfa7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-03-23Doesn't redirect stderr when getting verifier idCédric Ollivier1-1/+1
The logic fails when rally produces warnings [1]. It also updates Rally OpenStack and checks if the previous error has been solved[2]. TypeError: _discover_or_create_flavor() missing 1 required positional argument: 'flv_disk' [3] [1] https://build.opnfv.org/ci/job/functest-opnfv-functest-healthcheck-latest-tempest_smoke-run/757/console [2] https://github.com/openstack/rally-openstack/commit/e62315115af20e53f6c7205494ba0008634047ed [3] https://build.opnfv.org/ci/job/functest-opnfv-functest-healthcheck-latest-tempest_smoke-run/629/console Change-Id: Ia1daaa76d7cef7f30904c1f3bd42e2f6a6ba10af Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-03-03Add OVN Controller Gateway agent in neutron_bind_l2_agent_typesCédric Ollivier1-5/+0
It avoids blacklisting NeutronNetworks.create_and_bind_ports in case of OVN. It conforms with the upstream testing [1] and then reverts the previous change [2]. [1] https://github.com/openstack/networking-ovn/blob/master/zuul.d/networking-ovn-jobs.yaml#L299 [2] https://git.opnfv.org/functest/commit/?id=8e297c49c481961959b29c2080cc059fcc84e9ee This reverts commit 8e297c49c481961959b29c2080cc059fcc84e9ee. Change-Id: I95f1b7146e636ae3ebe9f995733e53ad1b276d26 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-24Blacklist NeutronNetworks.create_and_bind_ports if ovnCédric Ollivier1-0/+5
It only works for Linux Bridge and Open vSwitch. Change-Id: I1a437145eb8140642d3c8297c3f154113e74505f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-22Blacklist tests asking for vxlanCédric Ollivier1-0/+5
http://artifacts.opnfv.org/functest/B4HOI3WWLNVK/functest-ovn-opnfv-functest-smoke-jerma-neutron-tempest-plugin-api-run-4/neutron-tempest-plugin-api/tempest-report.html Change-Id: I6ff82b92cb4c392ac1f453faf7e16ef587dc2060 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-22Publish tempest_conf.yaml for Functest OVN SUTCédric Ollivier1-0/+78
Change-Id: I96237e8bfe469f100dc72d022747ce0013b41284 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-21Update patrole regex (OVN)Cédric Ollivier1-2/+6
It takes into account last changes in Patrole. It was not covered since Apex stopped deploying OVN. Change-Id: I7ab8362f9998024ae65f49973b169b95bbfd4fb5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-18Blacklick test_add_subport if OVNCédric Ollivier1-0/+1
https://bugs.launchpad.net/neutron/+bug/1863707 Change-Id: Ieebe10ee2a9040887d84aca3e8fb5ee66294634b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-15Run test_agent_management vs OVNCédric Ollivier1-2/+0
They must not longer be blacklisted as opposed to test_dhcp_agent_scheduler [1]. https://bugs.launchpad.net/networking-ovn/+bug/1863432 Change-Id: Ib08bb87580a10e2124faa1b3853f3e27dcb3a0a0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-01-24Update Alpine to 3.11Cédric Ollivier1-3/+3
It also updates Python to 3.8 (default version). It selects arm32v6 et arm64v8 repos and switches multiarch binfmt. https://alpinelinux.org/posts/Alpine-3.11.0-released.html Change-Id: Ib41cf0ab805ce2137658f3ff656b0a88c7b4ebf5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-27Set port_security: true in tempest_conf.yamlCédric Ollivier1-0/+4
It allows running test_boot_into_disabled_port_security_network_without_secgroup. It also sets a few default values. Change-Id: I851c5d13705e0419a7e664e05d32eec0efb417c6 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-27Fully remove Quotas.nova_update_and_deleteCédric Ollivier1-11/+0
It was blacklisted for a while and it's still failing [1][2]. It has to be removed from scenarios (blacklisting is useless and could mislead endusers). [1] http://artifacts.opnfv.org/functest/BYERCT6UZDWT/functest-opnfv-functest-benchmarking-iruya-rally_full-run-227/rally_full/rally_full.html [2] http://artifacts.opnfv.org/functest/LYQLHFQB7CTI/functest-opnfv-functest-benchmarking-hunter-rally_full-run-329/rally_full/rally_full.html Change-Id: I9ccea90a70024b6e54d873024cf550db5beea990 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-27Set create_shared_resources to trueCédric Ollivier1-0/+1
It takes last Neutron Tempest plugin change into account [1]. It also increases test count. [1] https://review.opendev.org/#/c/692822/ Change-Id: I5e7218dd808069a0be6d481447576099aa9c63e9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-24Remove former Rally blacklistCédric Ollivier1-8/+0
Change-Id: Ibc9aa210846ad2163f7ef943d224374dfff6f1ec Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-03Disable tempest serial_console testingCédric Ollivier1-1/+1
DevStack seems incorrect when configuring serial console in case of multinode deployment (missing proxyclient_address in master, missing socket in Rocky, etc). Then the testing is disabled. Change-Id: I9d1d43017b0935b23ea2a939748b5f709212c70d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-02Verify Shaker results via json reportCédric Ollivier1-0/+8
As shaker always returns 0, it verifies the records and returns false after the first failure. Change-Id: I9393be0c65d315c1ca055de9f7f994f9db22eadb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-26Complete tempest_full for CNTT ComplianceCédric Ollivier1-1/+3
It forbids skipping any test and counts the testcases executed. It mosly avoids running tests accoring to the upstream skip tests (known bugs), related to nova-network, glance v1, keystone v2 and all features defined as optional by CNTT. It fixes an issue highlighted by yallint. Change-Id: Ib4678cb9e9fc37b16fcf21d8418931eae0f667d0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-26Update scheduler_available_filtersCédric Ollivier2-7/+1
Else few tests are skipped if DifferentHostFilter is not available. Change-Id: Idfc524216dc8503e590d4fed569a43e99ea22123 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-22Verify additional Nova featuresCédric Ollivier1-3/+26
Our SUT allows verifying the next Nova features - shelve - vnc_console - cold_migration attach_encrypted_volume is now False because of ceph. It writes the tempest default values to ease highlighting the overall configuration. Change-Id: Ib9ebc8e5b390ea99edf2c3aacbd873c5eb6e7abc Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-22Set manage_volume_ref and manage_snapshot_refCédric Ollivier1-0/+2
Ceph doesn't conform with the LVM driver and asks for modifying the manage volume and snapshop refs [1]. It allows passing test_unmanage_manage_snapshot [2]. [1] https://opendev.org/openstack/tempest/commit/ebc752b59dd3e2164807930e31b313a92bce70d5 [2] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-latest-tempest_full-run-480/tempest_full/tempest-report.html Change-Id: I8d3b2484ba5cbd203c32fa446fa27e17a80888d5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-21Conform with latest SUT updatesCédric Ollivier1-2/+2
Ceph is now used as backend which requires few adaptations for live migration. It differs from the main OpenStack gates but suits CNTT and most of the feedbacks received from enduser deployments. Change-Id: If6b76a7d4ec09f90467f3e63618c15a9b413bfbd Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-17Fix details in RallyCédric Ollivier1-5/+4
self.details is now a dict and summary is located under root [1]. [1] https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-hunter-rally_sanity-run/422/console Change-Id: Ic89e674b2f6660704b9dafe6070a224753e2af46 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-15Test identity project_tags and application_credentialsCédric Ollivier1-0/+5
Both are supported by OpenStack and Functest gates. It also sets the default values to highlight the coverage. Change-Id: Iae080a8bde0c411acc7811e7a0e70e3471db8c98 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-10Add features volume testingCédric Ollivier1-0/+8
The next features are supported by Functest SUT: - backup - snapshot - clone - manage_snapshot - manage_volume - volume_revert Change-Id: I0e83bb616a56b9e8651115bebf2373f85c642260 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-10Set provider_vlans in tempest Neutron_plugin_optionsCédric Ollivier1-0/+1
Few tests are skipped if this value is unset. Then it's needed to fully cover Neutron Trunk (Compliance). Change-Id: Ife4becb7ff8b47c2d90fc6c5eb4f585685205ed6 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-09Update to Xtesting 0.82.0Cédric Ollivier1-0/+2
Change-Id: Ie94b35c9107c851bf0a412c1361629e0cf4154a2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-26Add healthcheck Horizon testsCédric Ollivier1-0/+15
It runs TestDashboardBasicOps as proposed by tempest-horizon [1]. It's skipped if DASHBOARD_URL is unset (default). [1] https://opendev.org/openstack/tempest-horizon Change-Id: I1b6b5609412770609f412da4927641588b173ed6 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-25Merge "Add neutron_plugin_options section in tempest.conf"Cedric Ollivier1-0/+3
2019-10-24Merge "Allow checking Tests count"Cedric Ollivier1-0/+4
2019-10-24Add neutron_plugin_options section in tempest.confCédric Ollivier1-0/+3
It avoids skipping the next 2 tests [1]: neutron_tempest_plugin.api.admin.test_agent_availability_zone.AgentAvailabilityZoneTestCase.test_agents_availability_zone neutron_tempest_plugin.api.admin.test_networks.NetworksTestAdmin.test_create_tenant_network_vxlan [1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-latest-neutron-tempest-plugin-api-run-498/results/neutron-tempest-plugin-api/tempest-report.html Change-Id: I83f3f44302336045e47514e0aa0b67737161f0e1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-23Allow checking Tests countCédric Ollivier1-0/+4
It meets CNTT requirements (verifying conformance with a reference implementation) and may protect vs synchronization issues (see gambia refstack_defcore). Change-Id: I842bc4ed91eec6bc545c489e3686d0d41dae7b44 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-23Allow blacklisting tests if public endpoint onlyCédric Ollivier2-3/+24
It allows reusing the blacklisting system if only public endpoints are reachable (ONAP Openlab testing). It contains a first list of tempest-based tests which force the use of admin endpoints. Change-Id: Ic1ab07dec3ae1f2a0ca50b80e08cd0ac396f2e56 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-11Leverage on Rally task tagsCédric Ollivier1-23/+20
It avoids parsing rally task outputs which may hang (subprocess communicate) as seen in rally_full and rally_jobs. It simply selects test names as tags. Change-Id: I88b54a8f155e557f8a606fdbd7d86c1f4d5dae3b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-08Count all hypervisors by defaultCédric Ollivier4-4/+4
If one hypervisor is down, few tempest test fails. User is free to set SKIP_DOWN_HYPERVISORS if it's down on purpose. Change-Id: I7b6a4d0d8f67755d8c1550fd1bc6fd707634f68b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-04Update Shaker image (1.3.0 + Debian Stretch)Cédric Ollivier1-2/+2
It udates Shaker to 1.3.0 and switches to Debian Stretch as base image. It also eases debugging as ssh basic auth is enabled (shaker/shaker). The arm image has to be built before enabling the testcase for arm. Change-Id: I894bae86c7e8f7cafbba2b4e00e3334afed0b6e2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-02Ease debugging ShakerCédric Ollivier1-1/+1
It turns Shaker debug on to ease troubleshooting it. Change-Id: I7f24a8d3776742d569d2d47a60d6985584514014 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-10-01Merge "Count all active hypervisors"Cedric Ollivier4-5/+4
2019-09-30Count all active hypervisorsCédric Ollivier4-5/+4
Else tempest_slow fails if one hypervisor is down (detected in ONAP Openlab). It also improves Shaker, Rally and Vmtp which count them. Change-Id: Iee7c20e0357d9237501c5c451580d8f6409ac86b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-09-29List services via OS_INTERFACE in RallyCédric Ollivier1-1/+2
Rally now leverages on list_services as proposed in utils [1] [1] https://git.opnfv.org/functest/commit/?id=0837a745c81ed2a6e7df097f60d227d5997c805e Change-Id: Ib17f9b530c2933d4a49eb5b744c2ae80b24ad560 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-09-28Allow getting service via any endpointCédric Ollivier3-7/+5
It overrides the default implementation provided by Shade. Change-Id: I3ad2f8ba543be4483f108c4bd7765018dddc5b19 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-09-20Merge "Print OpenStack version in connection_check"Juha Kosonen1-0/+2
2019-09-20Print OpenStack version in connection_checkCédric Ollivier1-0/+2
Functest is becoming very used and sometimes vs older OpenStack releases (ONAP Openlab). It will ease debugging and easily find backward incompatibilities. Even if that value could be checked for OPNFV Release Management, testing older SUT is not considered as false from a Functest statepoint. Change-Id: Idd782cc98ec4d36f726e233fc5d011308e9d6b71 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-09-19Remove any cores quota in tempest_scenario and shakerCédric Ollivier2-2/+6
Both can reach the default limit if lots of computes available. Co-Authored-By: Vincent Huet <vincent.huet@orange.com> Change-Id: Ifd8a1080a08f55bf4b0efe1858e79fa4aff75895 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-09-15Add punctuations in password generatorCédric Ollivier1-1/+1
It adds a subset of string.punctuation to conform with a few password rules (see ONAP Openlab). Change-Id: I7751031bcb9b908d4162d70f8acaa4bdc87e3c79 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-09-14Harden OS_ env vars for VMTPCédric Ollivier1-0/+2
VMTP doesn't take into account domain ids and then forces the use of domain names (detected when verifying ONAP Openlab). It prints the next warnings and then fails. 2019-09-14 08:43:50,162 WARNING OS_PROJECT_DOMAIN_NAME is missing 2019-09-14 08:43:50,162 WARNING OS_USER_DOMAIN_NAME is missing Change-Id: I283d86ac0d96071480eefb571da9f2ff92705b08 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-09-14Remove any instance quota in tempest_scenarioCédric Ollivier1-0/+12
The next testcase can reach the default limit if lots of computes available [1]. tempest.scenario.test_server_multinode.TestServerMultinode.test_schedule_to_all_nodes [1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-hunter-tempest_scenario-run-325/results/tempest_scenario/test_raw_list.txt Change-Id: Ibf68e2a367199afee3f6198ebf683be6cd938d94 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-09-12Allow skipping list_servicesCédric Ollivier1-7/+15
Shade forces the use of admin endpoints when listing services [1]. A new env var is added to allow running Functest when only public endpoints are reachable (e.g. testing from a VM as proposed for ONAP integration). It seems the best approach as list_services provides key data and we do avoid blocking requests. It seems easier and more generic than listing all calls in testcases.yaml. [1] https://github.com/openstack/shade/blob/master/shade/openstackcloud.py#L9956 Change-Id: Idcbe5d364f4230ffe1da34e7da0e99e863f15c5e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-09-07Blacklist boot_and_associate_floating_ip if no floating ipCédric Ollivier1-0/+1
Change-Id: Id907abff44490a57e8ec2cdb00396d9e4b422815 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>