aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2020-05-04Enable network_v6 testing in CNTTCédric Ollivier1-3/+2
req.inf.ntw.16 is a MUST requirement. Fixes cntt-n/CNTT#1556 Change-Id: Id48595f5f4710ea9cbf31edaa4ae0ed31f5278ba Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-03Update all results printed in docsCédric Ollivier3-70/+151
Change-Id: If69f74031c218e513464fda5be4e6fb86266292a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-03Update octavia regexesCédric Ollivier3-6/+12
It stops using noop drivers and adds all scenarios except: - .*api.v2.test_availability_zone - .*api.v2.test_availability_zone_profile - .*api.v2.test_member.MemberAPITest.test_member_ipv4_create - .*api.v2.test_member.MemberAPITest.test_member_ipv6_create It doesn't run ipv6 which asks for a deep dive (ipv6 from containers?) It increases threads to 3 as the other testcases. Change-Id: I36d9d764a5ab92efe89196181bffa3ad0a57227c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-03Stop overriding tempest_conf.yaml in CNTTCédric Ollivier2-92/+0
It's useless as they are skipped by regexes (vnc_console). Change-Id: If3cea451c95179e7e200708e1f3b5e9534be9e1d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-02Avoid useless ssh in juju_epcCédric Ollivier1-10/+3
scp + ssh calls are replaced by the right scp call. Change-Id: Iac2be63fb380486393af65dda8f7fd80bdd41d5f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-02Skip latest tests in tempest_full_cnttCédric Ollivier1-0/+1
http://artifacts.opnfv.org/functest/NQ4IHN2NFL69/functest-opnfv-functest-smoke-cntt-latest-tempest_full_cntt-run-41/tempest_full_cntt/tempest-report.html Change-Id: Ia2a4e4229f02f5909ac87b5f6175230b194bc205 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-01Remove py27 in .travis-ciCédric Ollivier1-1/+1
py27 cannot be gated due to Rally. https://travis-ci.org/github/collivier/functest/jobs/681899038 Change-Id: Ifeef8b99050182c7629b0767920724d94cd86c48 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-01Exclude tests.scenario.test_octavia_lbaas in CNTT regexesCédric Ollivier1-0/+1
It may be skipped as well (the behavior differs if it's enabled or not via config). Change-Id: Ic23119001b7478e466782a58d8757117bf16e5c3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-01Rename barbican->tempest_barbican & octavia->tempest_octaviaCédric Ollivier4-8/+8
It unifies all test case names. Change-Id: I38576f2d6bae6ef0236b6acfbd30c0446d3a2eca Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-01Remove neutron_trunk; networking_bgpvpn and networking_sfcCédric Ollivier4-116/+0
They were all merged in neutron-tempest-plugin. Change-Id: Iefe3bf98fc41d0dab5a7cec640337c672a7b1a69 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-30Skip LoadBalancerTest in Functest GatesCédric Ollivier3-3/+6
It seems failing due to octavia noop driver as in-used. Change-Id: I53e1fe1cfe4930f92c1a4460e89ed6e21eb3f922 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-30Debug rally race conditionsCédric Ollivier7-2/+162
It will be reverted as soon as the root cause is found. Change-Id: Icbff74c5fb940c1a201cbdc3fb2137c91e1ed460 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-30Update OpenStack release detectionCédric Ollivier1-1/+3
Change-Id: Icad13836d88583ee676525d4743f12a5b888ebe2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-29Add tempest_telemetryCédric Ollivier5-2/+51
It runs all telemetry-tempest-plugin tests except test_autoscaling which is currently failing in our gates. Change-Id: I2c712eda71799e96735419e1c547a775be659156 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-29Updated from global requirementsCédric Ollivier2-4/+5
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> Change-Id: I3fc23ba64da0440a5fb571b34c561337415eba77
2020-04-28Rename neutron_tempest_plugin_api to tempest_neutronCédric Ollivier8-11/+11
It conforms with the other test case names Change-Id: Ia8836375c26ea4f1bff3637f7078e5f70f83c5fb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-28Complete "Skip CfnInitIntegrationTest in tempest_heat"Cédric Ollivier1-1/+2
http://artifacts.opnfv.org/functest/SXA8X2BBP3L4/functest-opnfv-functest-smoke-cntt-latest-tempest_heat_cntt-run-1/tempest_heat_cntt/tempest-report.html Change-Id: Ie26d561fbb490bcaafe7c8ffddebc28bd72ff51c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-28Increase neutron-tempest-api test_count to 510Cédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/3L7CL8K6CU2N/functest-ovn-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-cntt-run-29/neutron-tempest-plugin-api-cntt/tempest-report.html Change-Id: I077464ed0fc734b97de089f549fb66ab137fccdb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-28Move static data to tempest_conf.yamlCédric Ollivier4-1/+3
Change-Id: I99f33f2e5f27c22c597f7c4f2f864710ceb1350e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-28Skip CfnInitIntegrationTest in tempest_heatCédric Ollivier3-2/+8
It sometimes fails due to other test resources [1]. [1] http://artifacts.opnfv.org/functest/83IV2XRCU05K/functest-opnfv-functest-smoke-latest-tempest_heat-run-53/tempest_heat/tempest-report.html Change-Id: Ied91781c784370b424653e37c090a909f16b6585 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-27Revert "Increase Thread stack size"Cédric Ollivier7-94/+2
It still fails [1]. [1] https://build.opnfv.org/ci/job/functest-ovn-opnfv-functest-benchmarking-latest-rally_jobs-run/28/console Change-Id: I8560edb7dfbba804a94bb1453d5a489138d6a152 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-27Add tempest_heat in CNTT smokeCédric Ollivier2-0/+25
It increases Heat coverage [1]. [1] http://artifacts.opnfv.org/functest/24ELW4C5RKD4/functest-ovn-opnfv-functest-smoke-latest-tempest_heat-run-34/tempest_heat/tempest-report.html Change-Id: I6630e4827a59b2ded34e5c271b9155df3b25f588 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-26Increase Thread stack sizeCédric Ollivier7-2/+94
It sets Glibc default value. https://wiki.musl-libc.org/functional-differences-from-glibc.html https://bugs.python.org/issue32307 Change-Id: I647569354b3a30f7de4116cab53c9e2aeff91a59 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-24Increase test_count in tempest_cinderCédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/C5BKAKPJIZVJ/functest-opnfv-functest-smoke-cntt-latest-tempest_cinder_cntt-run-26/tempest_cinder_cntt/tempest-report.html Change-Id: I5c0a7db033024259180610b83b07c055f0393e4c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-22Use ujson version asked by OpenStack requirementCédric Ollivier2-2/+0
It's newer than the former tag selected [1]. [1] https://github.com/openstack/requirements/blob/master/upper-constraints.txt Change-Id: Ie89136da364d7e972a1d7f49c71bdfbbdd9671d8 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-21Publish "CNTT RC April 2020 and beyond"Cédric Ollivier2-0/+151
Please see the following rendering if it helps http://testresults.opnfv.org/functest/vevent202004/ Change-Id: Iaf8baa017666e2f36792c1daf48a721ff6f3f044 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-20Protect password in juju_epcCédric Ollivier1-3/+3
It sometimes fails according to the special chars selected [1]. [1] https://build.opnfv.org/ci/job/functest-opnfv-functest-vnf-hunter-juju_epc-run/333/console Change-Id: Ie0af3f2fe8d28225296788a8fce73e6d2ec2a304 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-19Skip new tempest_full tests (CNTT)Cédric Ollivier1-0/+1
http://artifacts.opnfv.org/functest/53FYHR0Q9VVJ/functest-opnfv-functest-smoke-cntt-latest-tempest_full_cntt-run-20/tempest_full_cntt/tempest-report.html Change-Id: Ia4bba0b14ed1d1058f0461e90207aa7530f2e0f3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-19Run sequentially tempest_heatCédric Ollivier3-2/+4
It's forced by design else it remains a few resources. It also sets fixed_subnet_name and network_for_ssh to avoid falsy discovery as it seems [1] [1] http://artifacts.opnfv.org/functest/X5GDYF3B0R2B/functest-opnfv-functest-smoke-jerma-tempest_heat-run-18/tempest_heat/tempest-report.html Change-Id: Ieb584ddc1c0b4ebb543e9d69277a20aaee0fbac0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-17Take the last neutron tempest tests into accountCédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/U0Y127OQNBF4/functest-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-cntt-run-17/neutron-tempest-plugin-api-cntt/tempest-report.html Change-Id: Ib19c63e1a7f6e4c050c19b520fe149ff7c81b156 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-16Update Neutron regexCédric Ollivier1-0/+1
It skips the latest tests added in neutron-tempest-plugin [1][2] [1] https://review.opendev.org/#/c/711656/ [2] http://artifacts.opnfv.org/functest/CQY2GVXZ1NTU/functest-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-cntt-run-13/neutron-tempest-plugin-api-cntt/tempest-report.html Change-Id: Iae9cd7a45f908c310133bf354b205a623f4c7476 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-14Decrease volume size in boot_server_attach_created_volume_and_live_migrateCédric Ollivier2-2/+2
It now conforms with the other tasks which sets 1 Gb as default. Change-Id: Ibadb2bee2bea7a79db2eb27d0cc1867a823d7935 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-12Merge "Conform with the upstream create_and_delete_subnets"Cedric Ollivier1-2/+0
2020-04-11Disable VolumeBackupRestoreIntegrationTest if OVNCédric Ollivier1-1/+2
It completes "Skip VolumeBackupRestoreIntegrationTest in temptest_heat" Change-Id: I798cefa3a3648277cc321cc9e85be9b634aa9e20 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-10Set concurrency and containers_per_tenant to 1Cédric Ollivier1-6/+6
Most of the rally_full runs failed vs our new SUTs (radosgw) . This two params fail the testing when higher than 1 (verified independently). As the new test config works when hugely tested in //, it makes sens to merge the new config to fix the gates. This patch will be safely reverted if the issue is confirmed on SUT. Change-Id: I222652d484a5e06190c12d45d6c1882de5ab9eec Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-10Update CNTT regexes and countsCédric Ollivier1-1/+2
It increases neutron-tempest-plugin-api test count to 502. It skips test_multiattach_rw_volume_update_failure which depends on volume multi-attach. http://artifacts.opnfv.org/functest/BVDINIZ8P8UC/functest-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-cntt-run-6/neutron-tempest-plugin-api-cntt/tempest-report.html http://artifacts.opnfv.org/functest/BVDINIZ8P8UC/functest-opnfv-functest-smoke-cntt-latest-tempest_full_cntt-run-5/tempest_full_cntt/tempest-report.html Change-Id: Id4d4918ee24ef15f9610e706d6e90dfa3ae2cc43 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-09Conform with the upstream create_and_delete_subnetsCédric Ollivier1-2/+0
Ovn fails vs rally_full [1] but passes vs rally_jobs. It reverts the small diff between rally_full and rally_jobs added by "Update network context in selected rally tests". Then it may raise side effects in case of L2-only testing [2] [1] http://artifacts.opnfv.org/functest/JDD0PPKUOYO5/functest-ovn-opnfv-functest-benchmarking-cntt-jerma-rally_full_cntt-run-9/rally_full_cntt/rally_full_cntt.html [2] https://github.com/opnfv/functest/commit/b4b2e7f9a5a3cb3379b45ac4ddd0ef949974a698 Change-Id: I5c7591a46de1b534b6b3dc511ca2caf20ac48e1f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-07Skip VolumeBackupRestoreIntegrationTest in temptest_heatCédric Ollivier1-1/+2
It's linked to vbd which is false in many cases [1]. e.g. IMAGE_PROPERTIES=hw_scsi_model:virtio-scsi,hw_disk_bus:scsi Enabling it requires upstream changes to configure the right device. [1] http://artifacts.opnfv.org/functest/FHHSBCPDUV6C/functest-opnfv-functest-smoke-latest-tempest_heat-run-7/tempest_heat/tempest-report.html Change-Id: I2632779e96b7b0850d1937a8ec3c7bf4607862a5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-06Install libstdc++ in functest-coreCédric Ollivier1-1/+1
import ujson ImportError: Error loading shared library libstdc++.so.6 Change-Id: I0d479d9d4344d4a5c93174c7b0162b5c0234f2c2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-06Add tempest_heat (heat-tempest-plugin)Cédric Ollivier10-2/+118
It asks for the Fedora-Cloud-Base image and a second user account. It should be noted that heat-tempest-plugin duplicates all configs from tempest. Minimal image is still Cirros and the first account is the admin one. Then it conforms with the TempestCommon logic. AodhAlarmTest is skipped by default because pre conditions are missing. SoftwareConfigIntegrationTest is skipped as in gates because it requires a custom image [1] [1] https://github.com/openstack/heat/blob/master/devstack/lib/heat#L444 Change-Id: I8b39dc65ef3714411cd828e17b95e124f2e90f34 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-06Add swift sections in OVN's tempest_conf.yamlCédric Ollivier1-0/+5
http://artifacts.opnfv.org/functest/4LM7FMUVFMML/functest-ovn-opnfv-functest-smoke-jerma-tempest_slow-run-41/tempest_slow/tempest-report.html Change-Id: I1a3f684a9059eac21695bd655df976bceebf26f0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-05Remove last skipped tests in neutron-tempest-plugin-api-cnttCédric Ollivier1-0/+2
It takes into account last results [1] [1] http://artifacts.opnfv.org/functest/LIF2Y3SVUWLD/functest-opnfv-functest-smoke-cntt-latest-neutron-tempest-plugin-api-cntt-run-2/neutron-tempest-plugin-api-cntt/tempest-report.html Change-Id: Idd03c5aa42309eb5f254d6a99ea7e7206ee705ed Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-05Swith to coreutils timeoutCédric Ollivier1-1/+1
juju is not correctly stopped when no internet. https://build.opnfv.org/ci/job/functest-ovn-opnfv-functest-vnf-jerma-juju_epc-run/21/console root 1 0 0 Apr04 ? 00:00:06 /usr/bin/python3.7 /usr/bin/run_tests -t juju_epc -r -p root 139 1 0 Apr04 ? 00:00:03 [juju] <defunct> root 141 1 0 Apr04 ? 00:00:00 [timeout] <defunct> Change-Id: Id06c6e829ab8a992107f655aac6ec4539e292d82 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit cda22d7db0bcca380e1caef76e171cd7753b6221)
2020-04-05Fix the Glance tasksCédric Ollivier2-4/+6
http://artifacts.opnfv.org/functest/ONA0SEFCMPR5/functest-opnfv-functest-smoke-latest-rally_sanity-run-688/functest.log Change-Id: Ied7bd209aedd713fbd64b169247de7931098ca8b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-05Update server_kwargs to boot_server_kwargsCédric Ollivier2-2/+2
It takes into account "Remove deprecated stuff. Part 1" [1] and then fixes rally_sanity and rally_full (NovaKeypair.boot_and_delete_server_with_keypair [2]) [1] https://github.com/openstack/rally-openstack/commit/4436258d8fbb758a1771f908726e78c29e309c8a#diff-a80e66732701c545c7fc540d02ee1066 [2] http://artifacts.opnfv.org/functest/HG7XUW226S1T/functest-opnfv-functest-smoke-latest-rally_sanity-run-687/rally_sanity/rally_sanity.html#/NovaKeypair.boot_and_delete_server_with_keypair/failures Change-Id: Id39b3231153efcccd58ed6bd988e26e2a672825b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-05Enable TrunkTestInheritJSONBase.test_add_subport if ovnCédric Ollivier1-1/+0
It has been fixed upstream by "Protect vs unordered results in TrunkTestInheritJSONBase" [1]. [1] https://review.opendev.org/#/c/708305/5 Change-Id: Ibbc23a53d9d20d9b3a192e15779ca0f3334e2a38 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-04Reduce Tempest multi threadsCédric Ollivier4-22/+22
It decreases the threads from 4 to 3 to allow running the testcases in parallel. Else a few tests sometimes fail. Change-Id: I268eff5a72d5de5b26d7549ac74ee0e2a05cb61e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-04Skip test_container_synchronizationCédric Ollivier3-1/+12
It's not supported by radosgw which is now used in Functest SUTs [1][2]. It's removed from CNTT which seems leveraging on ceph (at least RI) [3] [1] https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3.2/html/release_notes/bug-fixes [2] http://artifacts.opnfv.org/functest/E46RCZLBRYCC/functest-opnfv-functest-smoke-hunter-tempest_slow-run-127/tempest_slow/tempest-report.html [3] http://artifacts.opnfv.org/cntt/WKFRPQODPW34/cntt-opnfv-functest-smoke-latest-tempest_slow-run-37/tempest_slow/tempest-report.html Change-Id: I8127df786c41af0d0285ef0e5cd1b28901685f65 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-02Revert "Keep the modified tempest"Cédric Ollivier4-4/+0
It's simply useless and may raise side effects later. Change-Id: Idaeab672480597ed39a5e29719580fee38aeb4ad Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-04-02Keep the modified tempestCédric Ollivier4-0/+4
http://artifacts.opnfv.org/functest/CRY58ZS3031L/functest-ovn-opnfv-functest-healthcheck-jerma-tempest_smoke-run-37/tempest_smoke/tempest-report.html Change-Id: I9598a45b09151fa267294716f70e65e3050f0239 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 21b512e55eb36e38160b697baae44c464fce9034)