aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2020-07-03Fix tempest_scenarioCédric Ollivier3-3/+3
It completes "min_computes_nodes is also used as max values". Change-Id: I72c67e17015bc35ac5e14f20839cf74353695373 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-07-03Generate all policies for patroleCédric Ollivier1-0/+3
Change-Id: Ica6455e3467ef2695d489170bd8f94553a6fdf06 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-07-03min_computes_nodes is also used as max valuesCédric Ollivier2-16/+2
Its max value is now 10 according to the default quota. Change-Id: Ie7bfcb224e821c15d38d0cba1d9f080548613e79 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-07-03Prepare full patrole testingCédric Ollivier1-2/+9
It adds the remaining services which didn't work before. It may be needed for CNTT (see security discussion in vevent). Change-Id: I36b3ad8bbfc8ffdcc427c3d97b92c78da27915c0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-07-03Add patrole user and reader testingCédric Ollivier5-5/+109
The regexes will be updated in a second change (chicken and egg between Functest and Releng) Change-Id: I4273a80799730fe30c941a7b2966cfee46f24d4a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-29Update tempest_neutron_cntt regex and test countCédric Ollivier1-0/+1
It takes latest updates into account [1][2]. [1] http://artifacts.opnfv.org/functest/4W4YF3954M7H/functest-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-85/tempest_neutron_cntt/tempest-report.html [2] http://artifacts.opnfv.org/functest/K4DV91F2Y9V0/functest-ovn-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-76/tempest_neutron_cntt/tempest-report.html Change-Id: Ia9b57e708522926fde9f8c44e6a0e936e97f304f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-29Simplify hypervisor list logic in vmtpCédric Ollivier1-5/+4
It completes the previous change "Allow listing hypervisors in VMTP" [1]. [1] https://gerrit.opnfv.org/gerrit/c/functest/+/70359 Change-Id: I12a19fd473076cfc3262f096cac1615c183710b1 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-24Remove build data in CNTT playbookCédric Ollivier1-27/+0
It's useless from a CNTT point of view. It remains in the classical IaaS verification to ease creating Functest Jenkins Jobs. Change-Id: Icb13ab7d0e43b4cf968cab4041b6827c11c20cbc Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-18Allow listing hypervisors in VMTPCédric Ollivier2-2/+11
It allows listing the compute under tests via VMTP_HYPERVISORS. They are no additional log in Functest and the note in VMTP is still valid [1]. [1] https://opendev.org/x/vmtp/src/branch/master/vmtp/vmtp.py#L833 Change-Id: Ic9fbda3901d796c0ed94e42e2a14c468598f4311 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-15Increase timeout in shaker and vmtpCédric Ollivier2-0/+3
Running them in parallel of rally full and jobs asks for a increase. Change-Id: Ib2369d384ac839fb46031411e1985ac402da0c1e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-15Reuse OpenStack master upper-constaints.txtCédric Ollivier1-1/+1
https://github.com/openstack/requirements/commit/c2f72e26a42eceb4b64f80d04db8253a4fbdb793 Change-Id: I0318c8614b09dc7cff1ffba4b813e55483df7eb3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-15Stop leveraging on neutron plugins and extraCédric Ollivier2-22/+2
Neutron leverages on the upstream Rally task in Stein and newer. Leveraging on the upstream task may fix the side effects which sometimes occured on Vmtp and Shaker [1] and filled better the CNTT model. [1] https://build.opnfv.org/ci/view/functest/job/functest-hunter-daily/574/ Change-Id: I5ee2e3a3ca956cf57365bf4579534177e87dc08b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-15Install py3-wheel to avoid legacy installationCédric Ollivier1-1/+1
Change-Id: I5cfd3e370ef16785eeceaafefa18b45d2221c04b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-15Compile lxml with optimization in smokeCédric Ollivier2-1/+5
Else cross compilation raises travis-ci.org's timeouts [1] [1] https://travis-ci.org/github/collivier/functest/jobs/698227192 Change-Id: I985321ac68a7a94cb8483d831f03222d44b3be57 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-14Clone repositories instead of wgetting constraintsCédric Ollivier7-73/+49
It stops leveraging on wget to get constraints as it fails since opendev was created. Then functest and requirements are no longer removed for functest-core which simplifies all child Dockerfile. It also fix the current incompatibilities in OpenStack constraints ERROR: cliff 3.2.0 has requirement cmd2!=0.8.3,<0.9.0,>=0.8.0, but you'll have cmd2 1.1.0 which is incompatible. https://github.com/openstack/requirements/blob/master/upper-constraints.txt#L230 https://github.com/openstack/requirements/blob/master/upper-constraints.txt#L416 Change-Id: I7c044208ef26ad047fb841710b422acb6c9f0320 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-12Skip EncryptionVolTypeTest in Iaas verification and CNTT ConformanceCédric Ollivier4-2/+5
Attaching an encryted volume is optional in CNTT [1]. This test fails if the volume type already exists and if it runs twice in parallel [2]. As this single test doesn't fullfil the Functest requirements, it's skiped in the classical verification as well. [1] https://github.com/cntt-n/CNTT/blob/master/doc/ref_arch/openstack/chapters/chapter05.md [2] https://storyboard.openstack.org/#!/story/2007804 Change-Id: Ib6fc843d71704aab1bfc536133544eb3fa4f3e72 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-12Remove RemoteStack testing from tempest_heat_cnttCédric Ollivier2-2/+5
Trusts redelegation is supported by Train [1][2] which forces to skip RemoteStack as CNTT Baldy is designed for Pike (the support is incomplete in case of trust [3]). [1] https://docs.openstack.org/releasenotes/heat/train.html#relnotes-13-0-0-stable-train [2] https://github.com/openstack/heat/commit/e377658586c737150dad1bfd80b7b2267d46be51 [3] https://bugs.launchpad.net/heat/+bug/1701498 Change-Id: Ib4f66170b9b1f50b8224705edb4820e5a4cb0167 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-08Add missing tempest_octavia in patchesCédric Ollivier1-0/+9
Change-Id: I0fae7487955895bf8f41046043425849f8fd0bf2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-08Update all patches proposed by FunctestCédric Ollivier2-73/+245
It mostly takes into account the last changes in test case names and new cntt-related test cases. Change-Id: I271fd02095b57f3b19b5ff95299fb4b257163c1f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-08Update aarch64 configsCédric Ollivier2-18/+28
Change-Id: Ib53b9f97f30bf86d33857910ab9bffa1ce0d1e9e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-07Merge "Update functest/ci/testcases.yaml"Cedric Ollivier5-8/+334
2020-06-07Merge "Stop increase thread stack size"Cedric Ollivier1-26/+2
2020-06-07Update functest/ci/testcases.yamlCédric Ollivier5-8/+334
It takes all CNTT changes into account. Change-Id: Ib6ddbd12695266fc78fd8ae4cce451bf8a4b95ad Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-07Stop increase thread stack sizeCédric Ollivier1-26/+2
It seems useless and then it limits the diff to Rally. Change-Id: Ib745c7aad20ab5ab7fd4f8a5c7deb9d4aff30881 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-06Update to alpine3.12Cédric Ollivier5-9/+9
https://alpinelinux.org/posts/Alpine-3.12.0-released.html Change-Id: Ie7c72abd10d70b90d65f260d595c0ca47b69a8c0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-06Increase tempest_neutron's tests_countCédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/Q3047D5AS3HJ/functest-ovn-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-40/tempest_neutron_cntt/tempest-report.html http://artifacts.opnfv.org/functest/03PRDE6TZC14/functest-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-49/tempest_neutron_cntt/tempest-report.html Change-Id: I4be6d2f58e50572a3e8b90ede0055f36bc1d6886 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-05Switch back to concurrency = 4 in tempest testcasesCédric Ollivier4-23/+27
3 seems good enough and stable which allows trying 4 as before Change-Id: Id1f17ebf73f9faa13ed3ceb327ccb4ecf88c2bf2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-05Run VNF in parallel in Functest default playbookCédric Ollivier1-2/+0
They pass in parallel if the bandwidth is good enough. https://build.opnfv.org/ci/view/functest/job/functest-latest-daily/679/ Change-Id: I2f466f7f25898d8af3004da1d3ef2208e4410f7c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-04Rename patch name and removes debug logsCédric Ollivier3-110/+76
The debug logs works only vs py3.8 and breaks backport to Iruya. Change-Id: Ie91a8a876e263a83b6ae3fbfb5bb05644b87b970 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-03Switching to Threading instead of multiprocessingCédric Ollivier1-343/+74
Change-Id: Id5059a06447357f4c9b058bad374ed6cbe4d742c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-03Increase tests_count into tempest_neutron_cnttCédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/KJSOWXFF5CIQ/functest-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-41/tempest_neutron_cntt/tempest-report.html Change-Id: I04f0c3ae695f17e2bd56add11410e24ee25967d2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-03Switch to queue.Queue() in RallyCédric Ollivier1-21/+52
It's still failing. Change-Id: I20fdf5a8e05bccc30a760934f6c0e3cac16d763c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-01Switch to SimpleQueue()Cédric Ollivier1-14/+56
Change-Id: I1f3cb636813b9c3c10b5c829e35bbdeea02a318c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-06-01Switch to fork in Python multiprocessing (python2 default)Cédric Ollivier1-5/+8
Change-Id: Id93da8c0fc710e1491adeb2b42e6da99974a0580 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-28Take last upstream changes into account in the regexesCédric Ollivier1-0/+1
http://artifacts.opnfv.org/functest/1NCC8YUFKE7X/functest-opnfv-functest-smoke-cntt-latest-tempest_slow_cntt-run-64/tempest_slow_cntt/tempest-report.html Change-Id: I196421d96ada3ec1eb8fc718f72c3e796a0bd3b4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-28Retry increasing Stack sizeCédric Ollivier1-2/+17
Change-Id: I61cb70a74179dea6a02e81989695325fe6331e10 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-27Skip manage_snapshot in CNTTCédric Ollivier2-2/+3
As detected by Orange's RC Field trial, Ceph doesn't support this feature before Rocky [1]. "AttributeError: 'RBDDriver' object has no attribute 'unmanage_snapshot'" manage_snapshot testing is fully passing in Functest SUT because CNTT is verified vs OpenStack Rocky and newer. Ceph is selected by RI as backend and is widely reused by the ecosytem. It's similar to [2]. [1] https://bugs.launchpad.net/cinder/+bug/1645288 [2] https://gerrit.opnfv.org/gerrit/c/functest/+/69888 Change-Id: I3b7f88f5bed9a47a1e26e16bbbf74fea24ca76c2 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-26Remove test_gnocchi.GnocchiGabbiTest.test_liveCédric Ollivier2-2/+2
It sometimes fails in our gates and it seems unverified in OpenStack [1]. "Archive policy gabbilive is still in use" [1] http://artifacts.opnfv.org/functest/DET32CBF0L0F/functest-opnfv-functest-smoke-jerma-tempest_telemetry-run-29/tempest_telemetry/tempest-report.html Change-Id: Id5e5ad5ff0f7bade7932a085df28cc35b0ac7984 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-25Update tests_count in tempest_full_cnttCédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/R1927JRCFJR5/functest-opnfv-functest-smoke-cntt-latest-tempest_full_cntt-run-62/tempest_full_cntt/tempest-report.html Change-Id: Ia4b5bbdb92e7877d81acdcf9c4f3a9745badfb36 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-24Precise validation configs in NeutronCédric Ollivier4-0/+29
ServerActionsTestJSON.test_reboot_server_hard [1] sometimes fails in gates. We may increase timeouts if it stills fails. [1] http://artifacts.opnfv.org/functest/LIQH38CPR30X/functest-opnfv-functest-smoke-hunter-tempest_full-run-649/tempest_full/tempest-report.html Change-Id: Ia27c0faead1df5c2b33a3e1a272e5a1010a5a83e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 7ccc1b0793e32cad3912e63bbca971d5c3ecd724)
2020-05-14Prevent microversion conflictsCédric Ollivier3-1/+98
CNTT selects Pike as baseline and Hunter matches Rocky. Change-Id: I8a1aa8385adce1ebb9d74bcb897effaf5b434e5a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 82ceefd8f1609508a476aa2e4573e59926653165)
2020-05-13Add other debug messages in RallyCédric Ollivier1-9/+141
It would allow debugging our Rally failures. Change-Id: I749a040fe76261e997d3a8dff4965867d54f7c42 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-10Increase tempest_neutron test_count to 512Cédric Ollivier1-1/+1
http://artifacts.opnfv.org/functest/V0A7B0J1PIRL/functest-opnfv-functest-smoke-cntt-latest-tempest_neutron_cntt-run-17/tempest_neutron_cntt/tempest-report.html Change-Id: I95dc0d8dec370dedb501878cbedfe5567ab86afd Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-10Update shaker imageCédric Ollivier1-1/+1
It updates pyshaker to 1.3.4 and adds fio which should be useful for the next disk benchmarking testcases. Change-Id: I662599745590eae71922ea5ef3f3876ad1019387 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-04Fix load_balancer options if ovnCédric Ollivier1-1/+1
Change-Id: I362b70cdfe753f9c57f30d61662e9b08564ebfe9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-04Enable network_v6 testing in CNTTCédric Ollivier1-3/+2
req.inf.ntw.16 is a MUST requirement. Fixes cntt-n/CNTT#1556 Change-Id: Id48595f5f4710ea9cbf31edaa4ae0ed31f5278ba Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-03Update all results printed in docsCédric Ollivier3-70/+151
Change-Id: If69f74031c218e513464fda5be4e6fb86266292a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-03Update octavia regexesCédric Ollivier3-6/+12
It stops using noop drivers and adds all scenarios except: - .*api.v2.test_availability_zone - .*api.v2.test_availability_zone_profile - .*api.v2.test_member.MemberAPITest.test_member_ipv4_create - .*api.v2.test_member.MemberAPITest.test_member_ipv6_create It doesn't run ipv6 which asks for a deep dive (ipv6 from containers?) It increases threads to 3 as the other testcases. Change-Id: I36d9d764a5ab92efe89196181bffa3ad0a57227c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-03Stop overriding tempest_conf.yaml in CNTTCédric Ollivier2-92/+0
It's useless as they are skipped by regexes (vnc_console). Change-Id: If3cea451c95179e7e200708e1f3b5e9534be9e1d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2020-05-02Avoid useless ssh in juju_epcCédric Ollivier1-10/+3
scp + ssh calls are replaced by the right scp call. Change-Id: Iac2be63fb380486393af65dda8f7fd80bdd41d5f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>