Age | Commit message (Collapse) | Author | Files | Lines |
|
This provides a more generic way to integrate test case lists which are
not hosted in Functest.
Also removes vm scenarios since they have never been a part of actual
rally runs.
Change-Id: Ib0a020fe72800915bbf2d10ecc690a248d33c246
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Enables utilization of blacklist also on non-OPNFV environments since
there is no link to installer type.
JIRA: FUNCTEST-1008
Change-Id: I3eb4377af153480794c1a71c5d7ef3e78dacacb9
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
FUNCTEST-1002
Change-Id: Id19b9aa70aeded6d99978084b3de4a63e31a81ad
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Utilize the output of Rally task execution since it internally
validates the task.
Use Popen.communicate to get the output from the running Rally task.
Change-Id: I9b1299bc26129ff4f89adccec611c78195a3ab02
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Else it could raise unauthorized exceptions [1].
[1] https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-master/188/console
Change-Id: Ia9b857069f775e2c8ecc5e07cf39b3a1fb7e3540
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Rally may print additional data to console which results in an
invalid json files. The json files are generated via rally instead of
dumping stdout.
It's mandatory to backport the next bugfix [1] in stable/fraser.
[1] https://gerrit.opnfv.org/gerrit/#/c/60265/
Change-Id: Ifbd90e685a7b52c70b3110a46e3c7991eddc05f4
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It checks all openstack resources before cleaning.
It allows removing images even if the test fail quickly.
Change-Id: I1763b8a70fa7ffeb03c435e8e775c7afd1a7adac
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Change-Id: If63c228f2844194279b560515f4ab24a4ee88be0
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
self.result can be positive due to intermediary steps (scenarios).
Else it can return false positives [1].
[1] https://build.opnfv.org/ci/view/functest/job/functest-apex-virtual-suite-master/113/console
Change-Id: Iecadee7f7380fda5273a0668aff1b441647c45aa
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It also fixes clean().
Change-Id: I42d1305d5cdb5a717cfa7fce26798ac7102310f7
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
They are highlighted by the last pylint version.
Change-Id: I473212a413013492716993fb536903b13bff66ed
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It applied the same operations as [1].
Only one config is used between rally, tempest and vping.
[1] https://gerrit.opnfv.org/gerrit/#/c/57613/
Change-Id: I4d5fd2117d404b7eaa79326e21a35272232d0b92
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Use data provided by report generation since the output of task
results is not available as valid JSON if task didn't finish
gracefully.
JIRA: FUNCTEST-966
Change-Id: Ieb906bc209cc26055babd5f4e6b2bb556724f9a8
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Use Shade for managing OpenStack resources in rally.
Change-Id: Ic8b7aaef6886f6ca4930b2ea463317b75ed8d3c9
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
It converts private subnet name to str as it's mandary till [1].
It could be reverted as soon as [2] is merged in snaps.
[1] https://gerrit.opnfv.org/gerrit/#/c/56091/
[2] https://gerrit.opnfv.org/gerrit/#/c/56171/
Change-Id: Ia728415207f8ecb5b5ac55982f82337c45671c4c
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Added scenarios for testing:
- capabilities
- status
- archive policy
- archive policy rule
- resource
- resource type
- metric
Change-Id: I79f4f0b2bd4d206588ea3edbce8037d1cb993646
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
The API utilized by Rally in Ceilometer scenarios was deprecated in
OpenStack Pike and scenarios were blacklisted. Remove these scenarios
from respository.
JIRA: FUNCTEST-949
Change-Id: I6f100cdb18853d177fd88ea7bd539f9837914f25
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Also
1. remove the obsolete function: generate_test_accounts_file
2. Use patch mechanism to get config
Change-Id: I455ac37de71519bcaf9a51e5f1de1b9bc83b9ba6
Signed-off-by: Linda Wang <wangwulin@huawei.com>
|
|
Only pylint warnings in vnf modules has to be fixed now.
Change-Id: Ie71230072a51bb72ec0b1d4fef2652494d29948e
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
We should still consider that the nameserver must be handled by all
installers instead [1]:
- by configuring one or more DNS resolvers in dhcp_agent.ini
- by asking a local resolv
It eases setting the nameserver in testcases and then allows gating
them. 8.8.8.8 is still defined by default.
[1] https://docs.openstack.org/neutron/pike/admin/config-dns-res.html
Co-Authored-By: Cédric Ollivier <cedric.ollivier@orange.com>
Change-Id: Id58dd5ebee612a0fce2ebc40f089769e60899416
Signed-off-by: Linda Wang <wangwulin@huawei.com>
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
Use printable format of string in scenario input parameters.
JIRA: FUNCTEST-946
Change-Id: Ic4826506a6f030e749fecdc7f880e9f911f9c987
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
It removes all the files which have moved to Xtesting.
Vnf inheritances and env management will be improved in other
changes.
It keeps the same tree thanks to a symlink to allow publishing
artifacts.
Change-Id: I551bbd3f344cdab0158a50b7b09e541576695631
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It also removes constants.CONST and env.Environment which are now
useless.
Depends-On: I764a0a2a24447c941d1e726f3116593b29dd1c1e
Depends-On: I6cfa832466dcefd737314633d807512e46267a69
Change-Id: Ife41c59d9f2e6ec4e49df38af962039f99554bc5
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
It defines env.get() as an unique way to get Functest env vars or
their default values. It can be considered as a wrapper above os.environ.
It enforces backward compatibility via CONST which mustn't be used
for that purpose. It should be noted that it also stops using CONST
for getting OpenStack env vars.
Change-Id: I333dc1afbc0123166a7eaff8b551370098efa341
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
- utilize env vars directly via os module
- use getattr to read CONST
JIRA: FUNCTEST-931
Change-Id: I893a91331e26846f8effe2d3d58522950bd096fd
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
JIRA: FUNCTEST-906
Change-Id: I484f9a291d5adbedb855450e4ff067dfa4a3b2ae
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Add support for providing credentials in snaps_utils.
JIRA: FUNCTEST-916
Change-Id: I3e027229d213d3791a115920f7012309cc027712
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
- statistic data based on Rally results report
- fix statistics calculations
- use PrettyTable in generating the report
Change-Id: Ic84adc5e30d446aea1f3045ba997c3ddf895e9f5
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Read stdout until nothing left instead of stopping once the child
process is terminated.
Change-Id: I46a2eb93fd614e2e7ff676727eb9dc132c29d03f
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
When the network_type is vlan, physical_network and
segmentation_id could be configured.
This also supports to config physical network when creating
rally net.
Change-Id: I9770968537c1509c5712eada66925a4bf8f6678c
Signed-off-by: Linda Wang <wangwulin@huawei.com>
|
|
The setting of hw:mem_page_size=large indicates larger page sizes
for guest RAM, eg., either 2MB or 1GB on x86.
When tested on compass, 1GB is required.
Change-Id: Iefd2e8ff30e0a496157e0bd118e63ab3c402abab
Signed-off-by: Linda Wang <wangwulin@huawei.com>
|
|
And also, remove the params "tempest_use_custom_images" and
"unique_names" of vping, rally and tempest.
Change-Id: Ic4243ef2f5e965910e5befa87db8d585cad34fa2
Signed-off-by: Linda Wang <wangwulin@huawei.com>
|
|
|
|
JIRA: SNAPS-202
Change-Id: I115b4927b287a5525f4c1b95128d91f5229c7856
Signed-off-by: spisarski <s.pisarski@cablelabs.com>
|
|
Replace obsolete os.popen with subprocess module usage in command
output retrieval. Do not use shell as the program to execute.
Change-Id: Ic63bb0f393750c3b85a35a37fecd46c1c1b96602
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
JIRA: FUNCTEST-889
Change-Id: I96776da7af50b2c33c34dd731b5500b891d263d6
Signed-off-by: Linda Wang <wangwulin@huawei.com>
|
|
The test NovaServers.boot_and_migrate_server needs at least 2 compute
nodes:
https://github.com/openstack/rally/blob/stable/0.9/rally/plugins/openstack/scenarios/nova/servers.py#L763-L765
Change-Id: I4ddf99188094ced57225d886037b00e91f3b68e6
Signed-off-by: Linda Wang <wangwulin@huawei.com>
|
|
Live migration test cases were excluded if tempest was not executed
before rally since tempest config file is created during the execution
of tempest.
Remove the dependency to config file and define the support based on
the number of computes as done in tempest.
Change-Id: I57c18b7c57e6a4bc8bfbdfb4beb5f8d67c6269c4
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Create flavors and use them in rally scenarios instead of expecting
certain flavors to pre-exist.
Change-Id: I77c94ab80fcabd7b80ffb36f9856a48121858009
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Change-Id: I0a78a16d44c1e0e0b2edec2e43954c1f08e8bb09
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Use SNAPS to create and delete resources.
Change-Id: Ic4d390382acea2be196e8a7518cb8cce25276a45
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
Rally scenarios don't require volume type to exist as a precondition.
Change-Id: Id735346e062af084635fd8f76f3686005b9b8add
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
For tempest or rally, if there are more than 1 compute node, live
migration related tests should not be skipped or blacklisted if
shared storage is enabled via ceph.
Change-Id: Ic93e4f508b597c657abe8d246f0cf9c2b5dfd020
Signed-off-by: Linda Wang <wangwulin@huawei.com>
|
|
Add extra properties to the cirros image
for ARM architecture.
In the glanceclient repository, the create method
expects **kwargs as parameters, thus an extra_properties
variable containing a dictionary of property-value pairs
can be added (if given) to the image to be created.
If the properties do not appear, the call should not affect
the image properties because the dictionary is initialized
as empty.
Change-Id: I690da90e6c6f250bd0a3d99cce39aa250e19b9c2
Signed-off-by: Catalina Focsa <catalina.focsa@enea.com>
|
|
Ceilometer scenarios were added.
Reasons for testcase blacklisting:
- Deprecated ceilometer client used by Rally. This depends on the
confuguration of the system, currently it seems to affect all
installers.
- A bug in ceilometer client (already fixed and merged in OpenStack
repository).
JIRA: FUNCTEST-849
Change-Id: I0bd7e1e9817538165b7b9705bcec213d7d275359
Signed-off-by: Martin Kulhavy <martin.kulhavy@nokia.com>
|
|
Test cases are:
- rally (sanity and full)
- refstack
- cloudify_ims
- vping_ssh
It also fix a bug on "energy_recording" decorator of energy module.
Change-Id: I625c214552ad1b89eaf7e196e706d7c69b37da33
Signed-off-by: Benoit HERARD <benoit.herard@orange.com>
|
|
Reasoning: Some blacklisted test cases apply to a range of scenarios,
e.g. both 'ha' and 'noha' and possibly even to all scenarios for
a specific installer -- therefore the list of excluded scenarios can
get long and unclear. For consistency with rally test case scenarios
also allow regex for installers specification.
Additionally, log the excluded test cases (if any) to debug log.
Change-Id: Ie0bec35d1a6443203d4719ea5428c437c12e5e12
Signed-off-by: Martin Kulhavy <martin.kulhavy@nokia.com>
|
|
HttpRequests scenario is not directly related to OpenStack and not
supported by Rally in https environment.
Change-Id: I2e91d129d82dfaba62eca25e3e4890ff54f52ac6
Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
|
|
It also removes relative (and incomplete) references which could be
defined in modules instead of this global configuration.
Change-Id: Ie20dc3547e49b6224aa8100cd380d37b87ece5a9
Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
|
|
|