Age | Commit message (Collapse) | Author | Files | Lines |
|
This patch applies a change to the regex responsible for
identifying Tempest test cases. With the current regex some
skipped test cases are erroneously ignored.
The condition has been updated in order to match both format of
skipped testcases ('skip:' and 'skip [xxx]:')
Change-Id: Ie7144b01e07cae92918eede0324f3f1f2582c4aa
Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
|
|
It adds xunit reports for rally-based and tempest-based testcases.
It completes the reports provided by snaps (thanks to Xtesting).
All rally related operations are moved to rally.
It allows removing the rally dependency to tempest which was false.
Change-Id: Ia7d2476f58f4f68b7c88442e50cad844037a36e9
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It switches from tempest.log to rally.log to conform with rally/stestr.
Change-Id: I00c2a268a2bc312a22a8d70278fad618c1c25235
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It allows running test_user_update_own_password [1]
[1] http://artifacts.opnfv.org/functest/functest-opnfv-functest-components-latest-tempest_full-run-8/results/tempest_full/tempest-report.html
Change-Id: I9f273ba58871f1e16886af623dcbb91f5c20c2f6
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Change-Id: I46053c92660139f4004d9cec8d1cfb6817cc0e07
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Rally selects the first external network [1] which could raise side
effects if:
- another testcase is creating a dummy external network in //
- all external subnets are not reachable from jumphost
It's now overriden by EXTERNAL_NETWORK as defined in Functest
Rally creates a new shared network if one already exists [2].
As juju now allows other shared networks, it can be safely created by
Functest to allow only one logic and to handle provider networks,
different name servers, etc...
[1] https://github.com/openstack/rally-openstack/blob/master/rally_openstack/verification/tempest/config.py#L146
[2] https://github.com/openstack/rally-openstack/blob/master/rally_openstack/verification/tempest/context.py#L85
Change-Id: Icf8c08077d4b0a9eb1c2e1b7309c62957b0a3b63
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It conforms with best programming practices.
It allows muting console and increasing log level (debug).
Change-Id: I0e0cfef6abe3b354a43817fcb242c71da701e237
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Vmtp is skipped if only one hypervisor.
As Vtmp returns 0 when all is skipped, it protects vs issues when
generating charts.
Change-Id: I5ea7c31c0688ae773425d2192a4f30a22360185f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It blacklists agent testing and two tests asking for VXLAN as overlay
if OVN (OVN supports GENEVE) [1]
blacklist.txt is renamed blacklist.yaml and is verified by yamllint.
[1] https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-gambia/33/console
Change-Id: I73c1b8cbbd74973a7f15476feea6ee4532786660
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It allows removing OPNFV logics out of Functest (releng) and then
simplifies conf_utils.
tempest.conf is filled by the current default values.
functest-alpine.sh will be updated to allow modifying the default
values without breaking the OPNFV installers.
It deprecates the only tempest-related env var (STORAGE_PROTOCOL)
Change-Id: I14bcb715826ef2d85af87e7af600261a3c08e72c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
From the time being, all OPNFV installers [1][2][3] don't support
Cinder volumes backup.
We will find a better way to handle the lack of OPNFV requiremements
(these tests pass succesfully in Orange Openlab).
At least, it will allow verifying tempest_full in OPNFV gates.
[1] Apex: https://artifacts.opnfv.org/logs/functest/lf-pod1/2018-10-25_02-17-25/tempest_full/tempest-report.html
[2] Compass: https://artifacts.opnfv.org/logs/functest/huawei-virtual9/2018-10-24_04-54-25/tempest_full/tempest-report.html
[3] Fuel: https://artifacts.opnfv.org/logs/functest/lf-pod2/gambia/2018-10-25_05-57-54/tempest_full/tempest-report.html
Change-Id: I319536070aa77a5dbab6a692a1d76049f75e758b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
At least Compass and Fuel configure Spice as console [1][2].
It disables vnc testing as it could fail for endusers as well.
[1] https://artifacts.opnfv.org/logs/functest/huawei-virtual9/2018-10-24_04-54-25/tempest_full/tempest-report.html
[2] https://artifacts.opnfv.org/logs/functest/lf-pod2/gambia/2018-10-25_05-57-54/tempest_full/tempest-report.html
Change-Id: I2a1ecf345cb81170c1b1a3e49f8f3a58602fc3f0
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It has been deprecated since Newton [1].
Even if the related tests are skipped in latest tempest [2], API v1
has to be disabled in Gambia's tempest.conf [3].
[1] https://docs.openstack.org/glance/latest/user/glanceapi.html
[2] https://artifacts.opnfv.org/logs/functest/lf-pod1/2018-10-22_02-34-24/tempest_full/tempest-report.html
[3] https://artifacts.opnfv.org/logs/functest/lf-pod1/gambia/2018-10-23_10-34-07/tempest_full/tempest-report.html
Change-Id: I3b6bb632412c0fecaa659da262760af64d79fa71
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It converts html_file to str and raises all possible exceptions.
It sometimes fails when locally generating reports.
Change-Id: Idf6a12aaac4561800cd5e364f8bcc495aabbee1d
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It removes useless spaces and now conforms with oslo.conf dict.
New helpers and unit tests are available.
Change-Id: I20335c5e5c3840cc872e158c6072d97a3f2cb98c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
|
|
Any other env vars are reduced.
Change-Id: Ic2c37d6efd543603a2495f9d2621b66d61710a54
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
|
|
Use IMG_PROP environment variable to read extra image
properties.
Set image_extra_properties for scsi volume types
without file patch, but using IMG_PROP env variable
IMG_PROP is defined through releng.
Image extra_properties will continue to get updates from
functest config file
With this change, remove unnecessary parameters from arm patch
Related to:https://gerrit.opnfv.org/gerrit/#/c/63203/
JIRA:ARMBAND-402
Change-Id: I64139240751b1e109bbe24ba32f487ccd98b9a92
Signed-off-by: Delia Popescu <delia.popescu@enea.com>
|
|
rally_jobs is failing vs APEX which still sets OS_TENANT [1].
[1] https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-master/324/console
Change-Id: I672bfa93b37b1b773ea7d20fdd91bea0ff14b79f
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It completes the previous change [1].
[1] https://gerrit.opnfv.org/gerrit/#/c/62793/2
Change-Id: Ie4c4c5cb374f2bfac6397841f8ddc2cd3b2f524b
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It allows verifying via non-default domain.
Change-Id: I95ae26fd3bfd1c10421c02d2b4f6e8a9b6aa49e6
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It completes the previous commit [1].
It also sets cidr and fully forces Keystone v3.
[1] https://gerrit.opnfv.org/gerrit/#/c/62777/
Change-Id: Ida948a4ff051dc1d1325d29a815c201366a1e33e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It doesn't work as expected as highlighted in [1].
[1] https://review.openstack.org/#/c/570207/
Change-Id: Iaf64345c40e8d62349cd50a6861c1191d25c40b9
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It seems mandatory for test_list_projects_returns_only_authorized_projects
Change-Id: Id994e66b088d751c4f6d7f260f8e482c2ee83307
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
fixed_network_name is not used in neutron or nova gates [1].
It may bypass juju bug (subnet discovery).
[1] http://logs.openstack.org/46/603046/1/check/tempest-full/a8468a1/controller/logs/tempest_conf.txt
Change-Id: I820e8c723918ed683ed1f8016363eb70bad0d14a
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It adds a new env var STORAGE_PROTOCOL to override storage_protocol
in tempest. The default value is unchanged [1].
[1] https://docs.openstack.org/tempest/latest/sampleconf.html
Co-Authored-By: Viktor Tikkanen <viktor.tikkanen@nokia.com>
Change-Id: I19e4e25212b6f6176e3b967e01eb634c15e4d474
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It indicates all filters that are included with nova are enabled [1].
[1] https://docs.openstack.org/tempest/latest/sampleconf.html
Co-Authored-By: Viktor Tikkanen <viktor.tikkanen@nokia.com>
Change-Id: I0d1943941d62d2eee418200c69e8198cd858383e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It avoids remaining resources after skipping a tempest-based testcase.
Change-Id: I0a993486c6e9265d15166d5eed5a612875d29b08
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
|
|
https://github.com/openstack/oslo.config/blob/master/oslo_config/types.py#L600
https://build.opnfv.org/ci/view/fuel/job/functest-fuel-virtual-daily-master/817/consoleFull
Change-Id: Icde9938bb9f6c0c37d2504317f31b0af4ef9ee8e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Return a reasonable value for caller to process.
Change-Id: I5a83544aaebb4e53db37ddaa13033d77848722b8
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
It's required at least by Barbican tempest plugin.
Change-Id: I46a65ee5fee044917c5375adaa8ebb529efe86ec
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Rally warns from these deprecated conf sections.
Change-Id: I15933c4f2cda14384e53202ec22f438a586d7b05
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
|
|
JIRA: FUNCTEST-1009
Change-Id: I6b9bca3827ba6cc804c25689a896ca63eec6a78b
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
|
|
If USE_DYNAMIC_CREDENTIALS = False, Functest is expected for
/home/opnfv/functest/data/accounts.yaml which contains the set of
pre-existing credentials to use for testing [1].
It also sets admin_domain_scope and default_credentials_domain_name.
[1] https://docs.openstack.org/tempest/latest/configuration.html
Change-Id: Ie7b81f0c56049260ca99b8e3a6d58cb8a2660659
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
|
|
It may allow bypassing RBAC policies if required.
Change-Id: I3b7ad44d7b80f8ea757846afdcd787b686c52856
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Enables utilization of blacklist also on non-OPNFV environments since
there is no link to installer type.
JIRA: FUNCTEST-1008
Change-Id: I3a012b8b7e7dbd049f8794dff3c9d56dcc1cadad
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
The rally.conf gets copied before the target directory has been created.
JIRA: FUNCTEST-1007
JIRA: DOVETAIL-730
Change-Id: Ie774fce994ada644c1163600e6c62e1231fde443
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>
|
|
They are blocking in Orange Openlab as the support is missing.
A dynamic check has to be implemented before activating it.
Change-Id: I8d87251fa4589f96d3515e411aeec5e79b7e7960
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It also copies rally.conf and tempest.log in res_dir.
Change-Id: Ib907a29ba3f47ca861d3af8afc0ee74446222e89
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Else rally_sanity may print logs in tempest_smoke res dir.
Change-Id: Iae97d533e9b99dcf8da70229b0675228472bfe52
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
FUNCTEST-1002
Change-Id: Id19b9aa70aeded6d99978084b3de4a63e31a81ad
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
It also redirect stderr to stdout as tempest mainly prints in stderr.
JIRA: FUNCTEST-999
Change-Id: I64bfa31b01aa2ce5444b6e0f0ac46f8cde8dbaa3
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
[1] https://build.opnfv.org/ci/view/functest/job/functest-fuel-armband-baremetal-daily-master/35/console
Change-Id: I7236bf5a74f36941f499da32c5c77dcd0b98b6f1
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Test failes to parse /etc/rally/rally.conf
(RALLY_CONF_FILE) whith inplace=1 keyword argument
for fileinput
One new space is added in second line
JIRA: ARMBAND-395
Change-Id: I8adb333047f9962f4105a6a74e2f8ef83d5745e0
Signed-off-by: Delia Popescu <delia.popescu@enea.com>
|
|
Else it could raise unauthorized exceptions [1].
[1] https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-master/188/console
Change-Id: Ia9b857069f775e2c8ecc5e07cf39b3a1fb7e3540
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|