Age | Commit message (Collapse) | Author | Files | Lines |
|
It adds a subset of string.punctuation to conform with a few
password rules (see ONAP Openlab).
Change-Id: I7751031bcb9b908d4162d70f8acaa4bdc87e3c79
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
VMTP doesn't take into account domain ids and then forces the use of
domain names (detected when verifying ONAP Openlab).
It prints the next warnings and then fails.
2019-09-14 08:43:50,162 WARNING OS_PROJECT_DOMAIN_NAME is missing
2019-09-14 08:43:50,162 WARNING OS_USER_DOMAIN_NAME is missing
Change-Id: I283d86ac0d96071480eefb571da9f2ff92705b08
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
The next testcase can reach the default limit if lots of computes
available [1].
tempest.scenario.test_server_multinode.TestServerMultinode.test_schedule_to_all_nodes
[1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-hunter-tempest_scenario-run-325/results/tempest_scenario/test_raw_list.txt
Change-Id: Ibf68e2a367199afee3f6198ebf683be6cd938d94
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Shade forces the use of admin endpoints when listing services [1].
A new env var is added to allow running Functest when only public
endpoints are reachable (e.g. testing from a VM as proposed for ONAP
integration).
It seems the best approach as list_services provides key data and
we do avoid blocking requests.
It seems easier and more generic than listing all calls in
testcases.yaml.
[1] https://github.com/openstack/shade/blob/master/shade/openstackcloud.py#L9956
Change-Id: Idcbe5d364f4230ffe1da34e7da0e99e863f15c5e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Change-Id: Id907abff44490a57e8ec2cdb00396d9e4b422815
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
One HeatStacks.create_and_delete_stack call asks for floating_network.
Change-Id: I75d01b4ecb1f484eac3af6efc53cf4a94206b158
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Set timeout on subprocess invocation instead of spawned child process.
Increase the timeout value for rally_full to 2h since the execution
typically takes ~90min [1].
[1] https://build.opnfv.org/ci/job/functest-opnfv-functest-benchmarking-latest-rally_full-run/22/
Change-Id: I0ca90bc2d85b4625336eb0396d8b2816a486b746
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
|
|
|
|
It integrates Octavia Tempest Plugin.
The new testcase octavia currently covers API only [1].
[1] https://logs.opendev.org/46/674846/2/check/octavia-v2-dsvm-noop-api/fe37c58/testr_results.html.gz
Change-Id: I2a11a19aa0c43701499dc9e17bd69b9387b53b58
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
The expression both in the test cases as well as applicable scenarios
support regex in blacklist.
Change-Id: Id76297a1cfbf908bda67641bc7ba686db83d76e6
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
It calls timeout (1 hour) to stop any hanging task [1].
It will be backported to Hunter and Iruya if similar failures are
detected.
[1] https://build.opnfv.org/ci/job/functest-opnfv-functest-benchmarking-latest-rally_full-run/9/console
Change-Id: I8fe89f9fa1c6c39cd8810096c1d586bdef7865a0
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
rally_jobs can verify network providers by skipping:
- NeutronNetworks.create_and_delete_floating_ips
- NeutronNetworks.create_and_list_floating_ips
- NeutronNetworks.associate_and_dissociate_floating_ips
Co-Authored-By: Juha Kosonen <juha.kosonen@nokia.com>
Change-Id: Ibda66e47f4d1101b53435ef4cd144dacbabff165
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Remove empty workload entries when applying blacklist in RallyJobs.
Rally considers task file as invalid if it contains entries whose
workload list empty.
Change-Id: I34867c0aeea9c5ecf1294959d680e6de535be9cd
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
It also allows building docs as doc8 is broken due to latest
OpenStack's upper-constraints.
pylint is updated to 2.3.1 (lastest py3.7 version)
It disables perm as umask is currently false on lf-virtual1.
Change-Id: If09d4796d48b7d0591e7926621d3bbf5ca1b6c24
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It allows creating testcases leveraging on Functest scenarios without
external network. Then Rally can verify network providers by only
skipping NovaServers.boot_server_associate_and_dissociate_floating_ip
Change-Id: I07a9348214f7557195c7201466f7e37b2b980206
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It doesn't run BarbicanSecrets.create_symmetric_and_delete which is
currently failing [1].
Barbican testing is optional as Gnocchi.
[1] https://bugs.launchpad.net/rally/+bug/1819285
Change-Id: I9121ce3c9f29ba05069c905325327d4c4d0b9e71
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
As for refstack_object, it forces swift in SUT.
Role is hardcoded to admin as for a few Neutron tests.
Change-Id: I0ae823fc8f31016d37831a82f7a50e92b2ed5f58
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
The default quota could be inadequate if lots of computes.
Change-Id: I510d3ebd14c2a910b7be2bacc0b99d3059a7a881
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
The last commit requires two additional task args [1][2].
[1] https://github.com/openstack/neutron/commit/ccef17605d1e28c418821c2235ba24c15893fff2#diff-720114afabbecd64333f4586a338f749
[2] https://build.opnfv.org/ci/job/functest-opnfv-functest-benchmarking-latest-rally_jobs-run/145/console
Change-Id: I4943f737be29bcadfead427a7852964e6438a018
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Both Rally and Tempest publish Rally debug logs.
It also moves the logics from tempest to rally.
Change-Id: I5c057d830202baddd73577ade1b09ce304e3c5a5
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-latest-refstack_compute-run/30/console
Change-Id: I77264525917af5f67cdecae85ef7a50439499db8
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It eases backporting in iruya and hunter as a few logs can't be
decoded as ascii [1].
[1] https://build.opnfv.org/ci/job/functest-opnfv-functest-vnf-iruya-juju_epc-run/56/console
Change-Id: I52a6f9a25130cbf62e8ce3b9972064ab73177665
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It fixes a few incorrect types in log calls [1].
[1] https://build.opnfv.org/ci/view/functest/job/functest-latest-daily/163/
Change-Id: I01790f6454694f434d31366f9c1cf9d98ff5a9cf
Signed-off-by: Cédric Ollivier <ollivier.cedric@gmail.com>
|
|
It leverages on last Vmtp review which adds py3 support [1].
[1] https://review.opendev.org/659970
Change-Id: I8319b9e7eb88b9015961188d47657a8c67e0f8da
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It adds platform and object targets.
refstack_defcore is renamed to refstack_compute.
Change-Id: Iad5bfb27be614aada98eef5b3148965e65210725
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Functest containers leverage on Python3 instead of python2.
https://mail.python.org/pipermail/python-dev/2018-March/152348.html
It also updates robotframework librairies to latest release and Vmtp
to master ([1] is needed)
It patches cloudify rest client to support python3.
Vmtp is currently disabled because it currently supports python2 only.
[1] https://github.com/openstack/vmtp/commit/a5d062881d91bf4f547d92c6e289bea30feb5d6e#diff-b4ef698db8ca845e5845c4618278f29a
Change-Id: I39964a212ec2d4dbf53c9ea7c63f02e0b6a05b48
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It conforms with OpenStack gates by default.
Change-Id: Icc1537b0f8342666d2a2a0de4655ba99f4f42470
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It creates a new env var BLOCK_MIGRATION (True by default) to allow
block migration for live migration in Rally.
It updates scenarios and blacklisting to take that env vars into account.
Change-Id: I41129432798ae9e40fe23bbe2c2162df4c0dff80
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
The default values conform with OpenStack and Functest gates [1].
It could be set to false if shared storage-based live migration is
supported [2].
It also currently disables cold migration.
[1] http://logs.openstack.org/84/632984/24/check/tempest-multinode-full-py3/4216037/controller/logs/tempest_conf.txt.gz
[2] https://docs.openstack.org/nova/pike/admin/configuring-migrations.html
Change-Id: I0faa415eba610c20ba54353ac7ba7b0e83722c32
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It updates refstack_defcore and vmtp to conform with yaml [1].
[1] https://build.opnfv.org/ci/job/functest-opnfv-functest-smoke-latest-refstack_defcore-run/186/console
Change-Id: I38e7a8d4c294c27f01c3e7385c4b716522a40334
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It also increases number of lines to warm about duplicated codes.
Change-Id: I3914da8a90cbfe8fd024e9944c3adc8a617330d3
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It sets max_microversion to latest because if both values are not
specified, Tempest avoids tests which require a microversion [1].
It completes the previous change about nova [2].
[1] https://docs.openstack.org/tempest/latest/sampleconf.html
[2] https://gerrit.opnfv.org/gerrit/#/c/67318/
Change-Id: Ieab56a239077476dd9a0e1ee97ac4293c4075f5c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
tempest fails vs devstack stable/stein and it passes successfully vs
devstack stable/rocky (Functest gates) without that option.
It may be added to tempest_conf.yaml if SUT requires it.
Change-Id: I4730c25b25403fd130d71166a8778e06fd479935
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Change-Id: I1dd51a54f39967fcf6b3a522e2a548f81e5646a9
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
As a precondition for executing scenarios requiring volume, Rally
expects the presence of service named as "cinder". In a case when
there is only cinderv2 and cinderv3 configured in the system, Rally
fails to validate a testcase even the volume service is working ok.
Testcases in question changed to use v3.
Change-Id: I42e7f846e80d3647df274b2e03add94c9f0a34d6
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Network context not needed in following scenarios:
NeutronNetworks.create_and_delete_routers
NeutronNetworks.create_and_list_routers
NeutronNetworks.create_and_list_subnets
NeutronNetworks.create_and_update_routers
NeutronNetworks.create_and_update_subnets
NeutronNetworks.create_and_show_routers
Router not needed in network context in following scenarios:
NeutronNetworks.create_and_update_ports
NeutronNetworks.create_and_delete_ports
NeutronNetworks.create_and_delete_subnets
NeutronNetworks.create_and_list_ports
NeutronNetworks.create_and_show_ports
NeutronNetworks.create_and_show_subnets
This completes the patch skipping router creation [1].
[1] https://gerrit.opnfv.org/gerrit/#/c/67429/
Change-Id: I80ea145a21168fd984cc9ef88d50af227f0d6ed4
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Do not create routers in those rally_sanity tests where the test does
not require it.
Change-Id: I6dd8a2b7579e11ca6ecc3b7383fa4d09a1e56ec3
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
|
|
It sets the shaker args to override ext net and name servers [1].
[1] https://pyshaker.readthedocs.io/en/latest/tools.html
Change-Id: I1cc6d52837e3c370db5ff59f024e96d7423be63a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It sets max_microversion to latest because if both values are not
specified, Tempest avoids tests which require a microversion [1].
Then test_keypairsv22_create_list_show_with_type is skipped and
refstack_defcore fails [2].
It also partially reverts "Allow building containers when
refstack.openstack.org is down" [3].
[1] https://docs.openstack.org/tempest/latest/sampleconf.html
[2] http://artifacts.opnfv.org/functest/functest-opnfv-functest-smoke-latest-refstack_defcore-run-147/results/refstack_defcore/tempest-report.html
[3] https://gerrit.opnfv.org/gerrit/#/c/67232/
Change-Id: Ia87543bb476d3047684e71ddfd05c2aa3f10bbfe
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Set the result of refstack_defcore as failed if there were skipped testcases
in a run.
Change-Id: Id782c2dee511cefbafd5b4440f386fd66f934a47
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Snaps hasn't been synchronized for a while regarding requirements.
We do remove it due to the inactivity.
All test result tabs will be updated in a second change.
Change-Id: I834afd902829ed3883b0e88e92aa806ec43d6fcf
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Change-Id: I597abdbb5bd3de023aa7b66fa27764ca2ec27f0d
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
It should be modified by tempest custom yaml file instead.
It's safe as the default timeout is bigger than the previous one.
It also cleans the obsolete tempest/rally data in config_functest.yaml.
[1] https://docs.openstack.org/tempest/latest/sampleconf.html
Change-Id: Ibad02d8aaf90f154c82f3b60a6d3f04fc37bd7de
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Member is the default value [1] and SwiftOperator is not created by
default in OpenStack gates (devstack)
This role should be modified by tempest custom yaml file instead.
[1] https://docs.openstack.org/tempest/latest/sampleconf.html
Change-Id: Ie77b91e36a7011e5730cbf5998e029c07ec7fe2f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
This patch applies a change to the regex responsible for
identifying Tempest test cases. With the current regex some
skipped test cases are erroneously ignored.
The condition has been updated in order to match both format of
skipped testcases ('skip:' and 'skip [xxx]:')
Change-Id: Ie7144b01e07cae92918eede0324f3f1f2582c4aa
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
This patch adds success/failure information of rally test cases to
summary output.
Change-Id: Ie4f9b4972a26e187bb7afdddfd5ce51b16f3db12
Signed-off-by: Stamatis Katsaounis <mokats@intracom-telecom.com>
|
|
It adds xunit reports for rally-based and tempest-based testcases.
It completes the reports provided by snaps (thanks to Xtesting).
All rally related operations are moved to rally.
It allows removing the rally dependency to tempest which was false.
Change-Id: Ia7d2476f58f4f68b7c88442e50cad844037a36e9
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
|