aboutsummaryrefslogtreecommitdiffstats
path: root/functest
AgeCommit message (Collapse)AuthorFilesLines
2020-05-03Update octavia regexesCédric Ollivier2-3/+6
It stops using noop drivers and adds all scenarios except: - .*api.v2.test_availability_zone - .*api.v2.test_availability_zone_profile - .*api.v2.test_member.MemberAPITest.test_member_ipv4_create - .*api.v2.test_member.MemberAPITest.test_member_ipv6_create It doesn't run ipv6 which asks for a deep dive (ipv6 from containers?) It increases threads to 3 as the other testcases. Change-Id: I36d9d764a5ab92efe89196181bffa3ad0a57227c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-02Avoid useless ssh in juju_epcCédric Ollivier1-10/+3
scp + ssh calls are replaced by the right scp call. Change-Id: Iac2be63fb380486393af65dda8f7fd80bdd41d5f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-01Rename barbican->tempest_barbican & octavia->tempest_octaviaCédric Ollivier1-2/+2
It unifies all test case names. Change-Id: I38576f2d6bae6ef0236b6acfbd30c0446d3a2eca Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-01Remove neutron_trunk; networking_bgpvpn and networking_sfcCédric Ollivier1-55/+0
They were all merged in neutron-tempest-plugin. Change-Id: Iefe3bf98fc41d0dab5a7cec640337c672a7b1a69 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-30Skip LoadBalancerTest in Functest GatesCédric Ollivier2-2/+4
It seems failing due to octavia noop driver as in-used. Change-Id: I53e1fe1cfe4930f92c1a4460e89ed6e21eb3f922 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-30Update OpenStack release detectionCédric Ollivier1-1/+3
Change-Id: Icad13836d88583ee676525d4743f12a5b888ebe2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-29Add tempest_telemetryCédric Ollivier1-0/+20
It runs all telemetry-tempest-plugin tests except test_autoscaling which is currently failing in our gates. Change-Id: I2c712eda71799e96735419e1c547a775be659156 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-28Rename neutron_tempest_plugin_api to tempest_neutronCédric Ollivier3-6/+6
It conforms with the other test case names Change-Id: Ia8836375c26ea4f1bff3637f7078e5f70f83c5fb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-28Move static data to tempest_conf.yamlCédric Ollivier3-1/+2
Change-Id: I99f33f2e5f27c22c597f7c4f2f864710ceb1350e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-28Skip CfnInitIntegrationTest in tempest_heatCédric Ollivier2-2/+2
It sometimes fails due to other test resources [1]. [1] http://artifacts.opnfv.org/functest/83IV2XRCU05K/functest-opnfv-functest-smoke-latest-tempest_heat-run-53/tempest_heat/tempest-report.html Change-Id: Ied91781c784370b424653e37c090a909f16b6585 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-20Protect password in juju_epcCédric Ollivier1-3/+3
It sometimes fails according to the special chars selected [1]. [1] https://build.opnfv.org/ci/job/functest-opnfv-functest-vnf-hunter-juju_epc-run/333/console Change-Id: Ie0af3f2fe8d28225296788a8fce73e6d2ec2a304 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-19Run sequentially tempest_heatCédric Ollivier2-1/+3
It's forced by design else it remains a few resources. It also sets fixed_subnet_name and network_for_ssh to avoid falsy discovery as it seems [1] [1] http://artifacts.opnfv.org/functest/X5GDYF3B0R2B/functest-opnfv-functest-smoke-jerma-tempest_heat-run-18/tempest_heat/tempest-report.html Change-Id: Ieb584ddc1c0b4ebb543e9d69277a20aaee0fbac0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-14Decrease volume size in boot_server_attach_created_volume_and_live_migrateCédric Ollivier2-2/+2
It now conforms with the other tasks which sets 1 Gb as default. Change-Id: Ibadb2bee2bea7a79db2eb27d0cc1867a823d7935 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-12Merge "Conform with the upstream create_and_delete_subnets"Cedric Ollivier1-2/+0
2020-04-11Disable VolumeBackupRestoreIntegrationTest if OVNCédric Ollivier1-1/+2
It completes "Skip VolumeBackupRestoreIntegrationTest in temptest_heat" Change-Id: I798cefa3a3648277cc321cc9e85be9b634aa9e20 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-10Set concurrency and containers_per_tenant to 1Cédric Ollivier1-6/+6
Most of the rally_full runs failed vs our new SUTs (radosgw) . This two params fail the testing when higher than 1 (verified independently). As the new test config works when hugely tested in //, it makes sens to merge the new config to fix the gates. This patch will be safely reverted if the issue is confirmed on SUT. Change-Id: I222652d484a5e06190c12d45d6c1882de5ab9eec Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-09Conform with the upstream create_and_delete_subnetsCédric Ollivier1-2/+0
Ovn fails vs rally_full [1] but passes vs rally_jobs. It reverts the small diff between rally_full and rally_jobs added by "Update network context in selected rally tests". Then it may raise side effects in case of L2-only testing [2] [1] http://artifacts.opnfv.org/functest/JDD0PPKUOYO5/functest-ovn-opnfv-functest-benchmarking-cntt-jerma-rally_full_cntt-run-9/rally_full_cntt/rally_full_cntt.html [2] https://github.com/opnfv/functest/commit/b4b2e7f9a5a3cb3379b45ac4ddd0ef949974a698 Change-Id: I5c7591a46de1b534b6b3dc511ca2caf20ac48e1f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-07Skip VolumeBackupRestoreIntegrationTest in temptest_heatCédric Ollivier1-1/+2
It's linked to vbd which is false in many cases [1]. e.g. IMAGE_PROPERTIES=hw_scsi_model:virtio-scsi,hw_disk_bus:scsi Enabling it requires upstream changes to configure the right device. [1] http://artifacts.opnfv.org/functest/FHHSBCPDUV6C/functest-opnfv-functest-smoke-latest-tempest_heat-run-7/tempest_heat/tempest-report.html Change-Id: I2632779e96b7b0850d1937a8ec3c7bf4607862a5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-06Add tempest_heat (heat-tempest-plugin)Cédric Ollivier5-0/+92
It asks for the Fedora-Cloud-Base image and a second user account. It should be noted that heat-tempest-plugin duplicates all configs from tempest. Minimal image is still Cirros and the first account is the admin one. Then it conforms with the TempestCommon logic. AodhAlarmTest is skipped by default because pre conditions are missing. SoftwareConfigIntegrationTest is skipped as in gates because it requires a custom image [1] [1] https://github.com/openstack/heat/blob/master/devstack/lib/heat#L444 Change-Id: I8b39dc65ef3714411cd828e17b95e124f2e90f34 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-06Add swift sections in OVN's tempest_conf.yamlCédric Ollivier1-0/+5
http://artifacts.opnfv.org/functest/4LM7FMUVFMML/functest-ovn-opnfv-functest-smoke-jerma-tempest_slow-run-41/tempest_slow/tempest-report.html Change-Id: I1a3f684a9059eac21695bd655df976bceebf26f0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-05Fix the Glance tasksCédric Ollivier2-4/+6
http://artifacts.opnfv.org/functest/ONA0SEFCMPR5/functest-opnfv-functest-smoke-latest-rally_sanity-run-688/functest.log Change-Id: Ied7bd209aedd713fbd64b169247de7931098ca8b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-05Update server_kwargs to boot_server_kwargsCédric Ollivier2-2/+2
It takes into account "Remove deprecated stuff. Part 1" [1] and then fixes rally_sanity and rally_full (NovaKeypair.boot_and_delete_server_with_keypair [2]) [1] https://github.com/openstack/rally-openstack/commit/4436258d8fbb758a1771f908726e78c29e309c8a#diff-a80e66732701c545c7fc540d02ee1066 [2] http://artifacts.opnfv.org/functest/HG7XUW226S1T/functest-opnfv-functest-smoke-latest-rally_sanity-run-687/rally_sanity/rally_sanity.html#/NovaKeypair.boot_and_delete_server_with_keypair/failures Change-Id: Id39b3231153efcccd58ed6bd988e26e2a672825b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-05Enable TrunkTestInheritJSONBase.test_add_subport if ovnCédric Ollivier1-1/+0
It has been fixed upstream by "Protect vs unordered results in TrunkTestInheritJSONBase" [1]. [1] https://review.opendev.org/#/c/708305/5 Change-Id: Ibbc23a53d9d20d9b3a192e15779ca0f3334e2a38 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-04Reduce Tempest multi threadsCédric Ollivier1-9/+9
It decreases the threads from 4 to 3 to allow running the testcases in parallel. Else a few tests sometimes fail. Change-Id: I268eff5a72d5de5b26d7549ac74ee0e2a05cb61e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-04Skip test_container_synchronizationCédric Ollivier1-0/+5
It's not supported by radosgw which is now used in Functest SUTs [1][2]. It's removed from CNTT which seems leveraging on ceph (at least RI) [3] [1] https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3.2/html/release_notes/bug-fixes [2] http://artifacts.opnfv.org/functest/E46RCZLBRYCC/functest-opnfv-functest-smoke-hunter-tempest_slow-run-127/tempest_slow/tempest-report.html [3] http://artifacts.opnfv.org/cntt/WKFRPQODPW34/cntt-opnfv-functest-smoke-latest-tempest_slow-run-37/tempest_slow/tempest-report.html Change-Id: I8127df786c41af0d0285ef0e5cd1b28901685f65 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-03-28Increase Shaker timeout before init message in consoleCédric Ollivier2-1/+6
It failed in parallel of rally_full [1]. [1] https://build.opnfv.org/ci/job/functest-opnfv-functest-benchmarking-hunter-shaker-run/397/console [2] https://build.opnfv.org/ci/job/functest-opnfv-functest-benchmarking-hunter-shaker-run/395/console Change-Id: I0eab7374c5962d45405c5f4147798067526f6e70 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-03-27Check the login prompt in console in SingleVm1Cédric Ollivier3-9/+28
It also checks the second vm2 console log in case of vping_ssh. Change-Id: I13a5edfb3e19449a38d2f0478d549bd8fcc5cfa7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-03-23Doesn't redirect stderr when getting verifier idCédric Ollivier1-1/+1
The logic fails when rally produces warnings [1]. It also updates Rally OpenStack and checks if the previous error has been solved[2]. TypeError: _discover_or_create_flavor() missing 1 required positional argument: 'flv_disk' [3] [1] https://build.opnfv.org/ci/job/functest-opnfv-functest-healthcheck-latest-tempest_smoke-run/757/console [2] https://github.com/openstack/rally-openstack/commit/e62315115af20e53f6c7205494ba0008634047ed [3] https://build.opnfv.org/ci/job/functest-opnfv-functest-healthcheck-latest-tempest_smoke-run/629/console Change-Id: Ia1daaa76d7cef7f30904c1f3bd42e2f6a6ba10af Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-03-03Add OVN Controller Gateway agent in neutron_bind_l2_agent_typesCédric Ollivier1-5/+0
It avoids blacklisting NeutronNetworks.create_and_bind_ports in case of OVN. It conforms with the upstream testing [1] and then reverts the previous change [2]. [1] https://github.com/openstack/networking-ovn/blob/master/zuul.d/networking-ovn-jobs.yaml#L299 [2] https://git.opnfv.org/functest/commit/?id=8e297c49c481961959b29c2080cc059fcc84e9ee This reverts commit 8e297c49c481961959b29c2080cc059fcc84e9ee. Change-Id: I95f1b7146e636ae3ebe9f995733e53ad1b276d26 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-24Blacklist NeutronNetworks.create_and_bind_ports if ovnCédric Ollivier1-0/+5
It only works for Linux Bridge and Open vSwitch. Change-Id: I1a437145eb8140642d3c8297c3f154113e74505f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-22Blacklist tests asking for vxlanCédric Ollivier1-0/+5
http://artifacts.opnfv.org/functest/B4HOI3WWLNVK/functest-ovn-opnfv-functest-smoke-jerma-neutron-tempest-plugin-api-run-4/neutron-tempest-plugin-api/tempest-report.html Change-Id: I6ff82b92cb4c392ac1f453faf7e16ef587dc2060 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-22Publish tempest_conf.yaml for Functest OVN SUTCédric Ollivier1-0/+78
Change-Id: I96237e8bfe469f100dc72d022747ce0013b41284 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-21Update patrole regex (OVN)Cédric Ollivier1-2/+6
It takes into account last changes in Patrole. It was not covered since Apex stopped deploying OVN. Change-Id: I7ab8362f9998024ae65f49973b169b95bbfd4fb5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-18Blacklick test_add_subport if OVNCédric Ollivier1-0/+1
https://bugs.launchpad.net/neutron/+bug/1863707 Change-Id: Ieebe10ee2a9040887d84aca3e8fb5ee66294634b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-02-15Run test_agent_management vs OVNCédric Ollivier1-2/+0
They must not longer be blacklisted as opposed to test_dhcp_agent_scheduler [1]. https://bugs.launchpad.net/networking-ovn/+bug/1863432 Change-Id: Ib08bb87580a10e2124faa1b3853f3e27dcb3a0a0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-01-24Update Alpine to 3.11Cédric Ollivier2-5/+5
It also updates Python to 3.8 (default version). It selects arm32v6 et arm64v8 repos and switches multiarch binfmt. https://alpinelinux.org/posts/Alpine-3.11.0-released.html Change-Id: Ib41cf0ab805ce2137658f3ff656b0a88c7b4ebf5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-27Set port_security: true in tempest_conf.yamlCédric Ollivier1-0/+4
It allows running test_boot_into_disabled_port_security_network_without_secgroup. It also sets a few default values. Change-Id: I851c5d13705e0419a7e664e05d32eec0efb417c6 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-27Fully remove Quotas.nova_update_and_deleteCédric Ollivier1-11/+0
It was blacklisted for a while and it's still failing [1][2]. It has to be removed from scenarios (blacklisting is useless and could mislead endusers). [1] http://artifacts.opnfv.org/functest/BYERCT6UZDWT/functest-opnfv-functest-benchmarking-iruya-rally_full-run-227/rally_full/rally_full.html [2] http://artifacts.opnfv.org/functest/LYQLHFQB7CTI/functest-opnfv-functest-benchmarking-hunter-rally_full-run-329/rally_full/rally_full.html Change-Id: I9ccea90a70024b6e54d873024cf550db5beea990 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-27Set create_shared_resources to trueCédric Ollivier1-0/+1
It takes last Neutron Tempest plugin change into account [1]. It also increases test count. [1] https://review.opendev.org/#/c/692822/ Change-Id: I5e7218dd808069a0be6d481447576099aa9c63e9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-24Remove former Rally blacklistCédric Ollivier1-8/+0
Change-Id: Ibc9aa210846ad2163f7ef943d224374dfff6f1ec Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-07Run tests sequentially in tempest_slowCédric Ollivier1-1/+1
concurrency is set to 2 in the upstream job [1] but it conflicts with serial also passed. As it fails once without any other test in parallel, the number of executors should be decreased [2]. [0] https://799189c1bf6a1aa9c8de-48970bda2a30ee836a70a12e61555969.ssl.cf1.rackcdn.com/694893/1/check/tempest-slow/1bf5851/job-output.txt [1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-cntt-hunter-tempest_slow-run-7/tempest_slow/tempest-report.html Change-Id: I51bc9a584cf4b32a88b7a84d9387e7b8e90281c2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-03Disable tempest serial_console testingCédric Ollivier1-1/+1
DevStack seems incorrect when configuring serial console in case of multinode deployment (missing proxyclient_address in master, missing socket in Rocky, etc). Then the testing is disabled. Change-Id: I9d1d43017b0935b23ea2a939748b5f709212c70d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-02Verify Shaker results via json reportCédric Ollivier1-0/+8
As shaker always returns 0, it verifies the records and returns false after the first failure. Change-Id: I9393be0c65d315c1ca055de9f7f994f9db22eadb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-12-02Return result = 0 if failure in juju_epcCédric Ollivier1-0/+1
Else it's falsy successfull [1] [1] https://build.opnfv.org/ci/job/airship-opnfv-functest-vnf-latest-juju_epc-run/42/console Change-Id: Ic9f7c37db998904833c90ae3ee912e9908617abd Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-26Complete tempest_full for CNTT ComplianceCédric Ollivier1-1/+3
It forbids skipping any test and counts the testcases executed. It mosly avoids running tests accoring to the upstream skip tests (known bugs), related to nova-network, glance v1, keystone v2 and all features defined as optional by CNTT. It fixes an issue highlighted by yallint. Change-Id: Ib4678cb9e9fc37b16fcf21d8418931eae0f667d0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-26Update scheduler_available_filtersCédric Ollivier2-7/+1
Else few tests are skipped if DifferentHostFilter is not available. Change-Id: Idfc524216dc8503e590d4fed569a43e99ea22123 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-22Verify additional Nova featuresCédric Ollivier1-3/+26
Our SUT allows verifying the next Nova features - shelve - vnc_console - cold_migration attach_encrypted_volume is now False because of ceph. It writes the tempest default values to ease highlighting the overall configuration. Change-Id: Ib9ebc8e5b390ea99edf2c3aacbd873c5eb6e7abc Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-22Set manage_volume_ref and manage_snapshot_refCédric Ollivier1-0/+2
Ceph doesn't conform with the LVM driver and asks for modifying the manage volume and snapshop refs [1]. It allows passing test_unmanage_manage_snapshot [2]. [1] https://opendev.org/openstack/tempest/commit/ebc752b59dd3e2164807930e31b313a92bce70d5 [2] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-latest-tempest_full-run-480/tempest_full/tempest-report.html Change-Id: I8d3b2484ba5cbd203c32fa446fa27e17a80888d5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-21Conform with latest SUT updatesCédric Ollivier2-3/+3
Ceph is now used as backend which requires few adaptations for live migration. It differs from the main OpenStack gates but suits CNTT and most of the feedbacks received from enduser deployments. Change-Id: If6b76a7d4ec09f90467f3e63618c15a9b413bfbd Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2019-11-17Fix details in RallyCédric Ollivier1-5/+4
self.details is now a dict and summary is located under root [1]. [1] https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-hunter-rally_sanity-run/422/console Change-Id: Ic89e674b2f6660704b9dafe6070a224753e2af46 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>