Age | Commit message (Collapse) | Author | Files | Lines |
|
Change-Id: Icdfd3ed23b6a5a47d95ae50aea88a46cd77be2ea
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 8351a852c23fd68b04b037ff4d53980a85e896ab)
|
|
Change-Id: Icd467525f4b1b1f97d14a5ec3d5d78b94fd1d4a9
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 6d1302779501918acdbbd4047a67a010901814b1)
|
|
It completes "min_computes_nodes is also used as max values".
Change-Id: I72c67e17015bc35ac5e14f20839cf74353695373
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit b765cf825b1a0d0853420b4a4bb7a154063f117e)
|
|
3 seems good enough and stable which allows trying 4 as before
Change-Id: Id1f17ebf73f9faa13ed3ceb327ccb4ecf88c2bf2
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit c7296454e80cea8273700da13e720e7f357f008e)
|
|
It sometimes fails in our gates and it seems unverified in OpenStack [1].
"Archive policy gabbilive is still in use"
[1] http://artifacts.opnfv.org/functest/DET32CBF0L0F/functest-opnfv-functest-smoke-jerma-tempest_telemetry-run-29/tempest_telemetry/tempest-report.html
Change-Id: Id5e5ad5ff0f7bade7932a085df28cc35b0ac7984
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 38827ed1862e8e26d9f2a9eaf79c33a5f745b463)
|
|
It runs all telemetry-tempest-plugin tests except test_autoscaling
which is currently failing in our gates.
Change-Id: I2c712eda71799e96735419e1c547a775be659156
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 0d90068be4aa4076ec1af0bcede780d41eddbe01)
|
|
It stops using noop drivers and adds all scenarios except:
- .*api.v2.test_availability_zone
- .*api.v2.test_availability_zone_profile
- .*api.v2.test_member.MemberAPITest.test_member_ipv4_create
- .*api.v2.test_member.MemberAPITest.test_member_ipv6_create
It doesn't run ipv6 which asks for a deep dive (ipv6 from containers?)
It increases threads to 3 as the other testcases.
Change-Id: I36d9d764a5ab92efe89196181bffa3ad0a57227c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 069930982dbf9745e6bc84cc7d4476be54fdcbc7)
|
|
It unifies all test case names.
Change-Id: I38576f2d6bae6ef0236b6acfbd30c0446d3a2eca
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit a11051968f25bdcc9362be88907731582ac1ea8f)
|
|
They were all merged in neutron-tempest-plugin.
Change-Id: Iefe3bf98fc41d0dab5a7cec640337c672a7b1a69
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit c22979393a734760105f73b75a5f90f4f9aecebc)
|
|
It conforms with the other test case names
Change-Id: Ia8836375c26ea4f1bff3637f7078e5f70f83c5fb
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit f74088bb1ae93feaf56c7ec7d1f1e78c97d8de9a)
|
|
It's forced by design else it remains a few resources.
It also sets fixed_subnet_name and network_for_ssh to avoid falsy
discovery as it seems [1]
[1] http://artifacts.opnfv.org/functest/X5GDYF3B0R2B/functest-opnfv-functest-smoke-jerma-tempest_heat-run-18/tempest_heat/tempest-report.html
Change-Id: Ieb584ddc1c0b4ebb543e9d69277a20aaee0fbac0
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit c2e3d911d9eaf987d18b3b895778c411e62efc4a)
|
|
It asks for the Fedora-Cloud-Base image and a second user account.
It should be noted that heat-tempest-plugin duplicates all configs from
tempest.
Minimal image is still Cirros and the first account is the admin one.
Then it conforms with the TempestCommon logic.
AodhAlarmTest is skipped by default because pre conditions are missing.
SoftwareConfigIntegrationTest is skipped as in gates because it
requires a custom image [1]
[1] https://github.com/openstack/heat/blob/master/devstack/lib/heat#L444
Change-Id: I8b39dc65ef3714411cd828e17b95e124f2e90f34
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It decreases the threads from 4 to 3 to allow running the testcases in
parallel. Else a few tests sometimes fail.
Change-Id: I268eff5a72d5de5b26d7549ac74ee0e2a05cb61e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit c0e9550a76bd91322a6bb5dc4b7f9947e1794913)
|
|
concurrency is set to 2 in the upstream job [1] but it conflicts with
serial also passed.
As it fails once without any other test in parallel, the number of
executors should be decreased [2].
[0] https://799189c1bf6a1aa9c8de-48970bda2a30ee836a70a12e61555969.ssl.cf1.rackcdn.com/694893/1/check/tempest-slow/1bf5851/job-output.txt
[1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-cntt-hunter-tempest_slow-run-7/tempest_slow/tempest-report.html
Change-Id: I51bc9a584cf4b32a88b7a84d9387e7b8e90281c2
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 83c3cc3f9b71de33cd1e88c11da4879156be7cf3)
|
|
It adds tempest_keystone which runs the tests proposed by
keystone-tempest-plugin [1].
[1] https://opendev.org/openstack/keystone-tempest-plugin
Change-Id: I83db584c37fe63a549ccf396b03e034778254b3a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 22ec0c1008b81f368476d8cea84117a93d15dab2)
|
|
It adds tempest_cinder which runs the tests proposed by
cinder-tempest-plugin [1]. test_incremental_backup is disabled because
it currently fails vs Functest SUT (under analysis).
[1] https://opendev.org/openstack/cinder-tempest-plugin
Change-Id: I85b255eb05b9ab1c74c3829148da5230d6c7647e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit e46aa41172fa1bf63364505fbbf922323788ac98)
|
|
Change-Id: I9d3abc3996857d8905007640250b94025735809c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit de7a2d821890f8bf1c5a9215def91ee136e867e4)
|
|
It meets CNTT requirements (verifying conformance with a reference
implementation) and may protect vs synchronization issues (see gambia
refstack_defcore).
Change-Id: I842bc4ed91eec6bc545c489e3686d0d41dae7b44
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
(cherry picked from commit 7c67624b8ee7f7a9a7fb36bb25e9881a63c982d4)
|
|
The next testcase can reach the default limit if lots of computes
available [1].
tempest.scenario.test_server_multinode.TestServerMultinode.test_schedule_to_all_nodes
[1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-hunter-tempest_scenario-run-325/results/tempest_scenario/test_raw_list.txt
Change-Id: Ibf68e2a367199afee3f6198ebf683be6cd938d94
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
|
|
It integrates Octavia Tempest Plugin.
The new testcase octavia currently covers API only [1].
[1] https://logs.opendev.org/46/674846/2/check/octavia-v2-dsvm-noop-api/fe37c58/testr_results.html.gz
Change-Id: I2a11a19aa0c43701499dc9e17bd69b9387b53b58
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It conforms with tempest-slow-py3 executed in Neutron Gates [1].
[1] https://logs.opendev.org/68/672568/6/check/tempest-slow-py3/49fece6/testr_results.html.gz
Change-Id: I0c1c6f7be38c9fcdd1a32e5cfa2bc4d8414885e0
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It doesn't run BarbicanSecrets.create_symmetric_and_delete which is
currently failing [1].
Barbican testing is optional as Gnocchi.
[1] https://bugs.launchpad.net/rally/+bug/1819285
Change-Id: I9121ce3c9f29ba05069c905325327d4c4d0b9e71
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It seems dependent on test order (404 if no namespace) and sometimes
fails in gates [1][2][3].
It will be enabled as soon it's fixed upstream or if SUT is wrong here.
[1] https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-latest-patrole-run/
[2] https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-iruya-patrole-run/
[3] https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-hunter-patrole-run/
Change-Id: I68e71e9b0753089e5620d6689ddff37fe66aeab7
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
They are being integrated into neutron tempest plugin [1][2].
It will fix our gates [3].
[1] https://github.com/openstack/neutron-tempest-plugin/commit/1a987ecb4d310328cc4c2c5628dc5018dc7afc48
[2] https://github.com/openstack/neutron-tempest-plugin/commit/8dd49aac9fa27166d76755c7d62ea9f0553d8d74
[3] https://build.opnfv.org/ci/job/functest-latest-gate/199/
Change-Id: Iafcfd0cb9b33a271cf37f269d9a3a2086eb67414
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It adds platform and object targets.
refstack_defcore is renamed to refstack_compute.
Change-Id: Iad5bfb27be614aada98eef5b3148965e65210725
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
They don't clean routers (tempest-L3AgentsExtRbacTest-Router) after
testing.
Change-Id: I78cfc783feb51d0f1b029c98e988c07e57e1e694
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It disables the two subtests which are failing in OpenStack Gates [1].
[1] http://logs.openstack.org/51/651851/1/check/patrole-admin/48dbeb0/testr_results.html.gz
Change-Id: I6814d003d80593d2a89fefa439f0775187eb6f4d
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It may fail when testing OpenStack in production or when running
testcases in parallel [1] because vxlan 200 is hardcoded [2]
[1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-hunter-patrole-run-75/results/patrole/tempest-report.html
[2] https://github.com/openstack/patrole/blob/master/patrole_tempest_plugin/tests/api/network/test_networks_rbac.py#L208
Change-Id: Ibfaed6f3a95d016790baafebd203e49e152a889b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Set the result of refstack_defcore as failed if there were skipped testcases
in a run.
Change-Id: Id782c2dee511cefbafd5b4440f386fd66f934a47
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Snaps hasn't been synchronized for a while regarding requirements.
We do remove it due to the inactivity.
All test result tabs will be updated in a second change.
Change-Id: I834afd902829ed3883b0e88e92aa806ec43d6fcf
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It conforms with SFC regex and fits OPNFV criteria.
Running scenarios may force the use of bagpipe [1].
The SUT (functest-virtual) is being updated to verify SFC and BGPVPN
apis.
[1] https://docs.openstack.org/networking-bgpvpn/latest/user/drivers/bagpipe/index.html
Change-Id: I1442b9cac7eaba1aeef04e36a2f790bd29f90976
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Rally_jobs were moved into benchmarking.
It completes "Move testcases in the right tiers" [1]
[1] https://gerrit.opnfv.org/gerrit/#/c/66875/
Change-Id: I1c2da77eff6682f85ab8504750077c9ee542edb8
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It completes "Move testcases in the right tiers" [1]
[1] https://gerrit.opnfv.org/gerrit/#/c/66863/
Change-Id: Ic2e6ebf9371de87caafbfdb54029a24440e6fd5b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Here are the moves:
- tempest_smoke in healthcheck
- tempest_full and tempest_scenario in smoke
- rally_jobs in benchmarking
It avoids conflicts between benchmarking tools and few functional
tests (short timeouts).
Then it should allow running in parallel all testcases in one tiers.
It also allows cleaning the additional steps defined in releng.
rally_full could move in benchmarking in a second step.
Change-Id: I0bc657557a61c7d9bf7679b4f2dbda08c36728ca
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
The execution of an optional scenario is skipped if the corresponding
openstack service does not exist.
Change-Id: I8721f26ebc253ce02138b85d6c88e659c5afae80
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Keystone policy.json et al will be managed in other changes to allow
cherry-picking.
It blacklists two subtests which requires policy actions not found in
policy files.
Change-Id: I5365fb1b24f7f07563037634d00cb9898fbde0fc
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Define floating network as a Rally task argument.
Enable rally_jobs subtest.
Change-Id: I37d7c4718cfb3de3ad1980878e0529d652013ccb
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
The public network now seems hardcoded (public) [1] which doesn't
fit the OPNFV model [2]. Neutron also stops publishing policy.json
[3].
Upstream files have to be updated or fixed in Functest.
[1] http://artifacts.opnfv.org/functest/opnfv-functest-smoke-rally_jobs-latest-run-25/functest.log
[2] https://review.openstack.org/#/c/620952/
[3] https://review.openstack.org/#/c/585037/
Change-Id: Ibae4ce71a45afa3ed4a987577937220b8a3a89f9
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
This reverts commit 6939526b9802539a62f71ca7759f7bd49c16aef4.
Change-Id: Ieb6644b47cc1567d5312f27e66ce250193db1f62
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It will be safely reverted when it'is fixed in patrole [1].
[1] https://review.openstack.org/#/c/612167/
Change-Id: Ib072f833523456004b7f211ef951c4b053d82577
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Include Neutron rally-jobs.
Change-Id: Ic1122962bd7d876d179729640a8a321bc36442d8
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
It fails at least vs OpenStack Queens.
OPNFV Gambia Installers are still falsy running latest then we do
disable it.
It has to be double checked as soon as Functest can be verified vs
Rocky or newer.
Change-Id: I7126c61d0c26b16f9058ed813af3344ca2930a86
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
tempest.conf conforms with Devstack post-actions [1].
One test is still disabled as it may force to sign all images.
A deeper analysis has to be done before enabling it.
[1] https://github.com/openstack/barbican-tempest-plugin/blob/master/tools/pre_test_hook.sh
Change-Id: I163c0e7c4671ca67f0b6c127c29fdcb12a8e426b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Change-Id: Ic5754821542e873bb7d33a8864a852993947b762
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Xtesting leverages on stevedore [1] which will bring much more
flexibility (discovery, module or class renaming, etc).
OPNFV Features have to be updated as well:
- https://gerrit.opnfv.org/gerrit/61713
- https://gerrit.opnfv.org/gerrit/61715
- https://gerrit.opnfv.org/gerrit/61717
- https://gerrit.opnfv.org/gerrit/61719
[1] https://docs.openstack.org/stevedore/latest/
Change-Id: I0ce45eb6c9af66546c2c020eb962fa457f6c7e1a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
In fraser release we observed that refstack tests are executed in parallel
and that might causes problems in CI because without concurrency
parameter it will try to use as many cpus as possible and that might
lead to race condition issues.
This patch is created for backporting purposes to fraser branch
where the problem occurs.
JIRA: DOVETAIL-662
Change-Id: I6cf0ad3a32e88f04abd00d69523aa8ac47c39470
Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com>
|
|
It stops depending on refstack_client which cannot work till
OpenStack Queens.
JIRA: FUNCTEST-977
Change-Id: I2340643de82ab10d8fb83fb3a7fbfd1f412b2d7b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
All Joid runs are stopped, then these scenarios can't be maintained.
Change-Id: Ida7c8c86cbd377024047d4abbbc1806ad65afc8a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Vmtp package delivers amd64 binaries in both architectures.
JIRA: ARMBAND-396
Change-Id: I4c6e010e47c8aa9adfafb373c261262d23ef382a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|