aboutsummaryrefslogtreecommitdiffstats
path: root/functest/opnfv_tests/openstack/rally/rally.py
AgeCommit message (Collapse)AuthorFilesLines
2018-10-15Do not create files in site-package directoryJuha Kosonen1-5/+22
Use /home/opnfv/functest/data/rally/task directory for storing temporary files when running rally tasks. Also removes unused vm scenario from task service list. Change-Id: Icfd4f99ebe6194360bda8bd88e497e018a9541a0 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-09-23Protect vs former OS env varsCédric Ollivier1-0/+5
rally_jobs is failing vs APEX which still sets OS_TENANT [1]. [1] https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-master/324/console Change-Id: I672bfa93b37b1b773ea7d20fdd91bea0ff14b79f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-21Remove rally plugins and extra when cleaningCédric Ollivier1-2/+11
Change-Id: I15393b870d648432c900d429d17129c630ed4f58 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-18Add support for Rally OpenStack CI test casesJuha Kosonen1-13/+121
Include Neutron rally-jobs. Change-Id: Ic1122962bd7d876d179729640a8a321bc36442d8 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-09-06Add role to users created by rally if requiredCédric Ollivier1-0/+27
It sets [openstack] keystone_default_role if NEW_USER_ROLE differs from member. Change-Id: Ib6be5cf22654e882ea110298cba421c1e87c78fd Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-03Refactor RallyBaseJuha Kosonen1-35/+37
This provides a more generic way to integrate test case lists which are not hosted in Functest. Also removes vm scenarios since they have never been a part of actual rally runs. Change-Id: Ib0a020fe72800915bbf2d10ecc690a248d33c246 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-28Remove installer type from rally blacklistJuha Kosonen1-5/+2
Enables utilization of blacklist also on non-OPNFV environments since there is no link to installer type. JIRA: FUNCTEST-1008 Change-Id: I3eb4377af153480794c1a71c5d7ef3e78dacacb9 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-14Create new project/user for rally testJuha Kosonen1-2/+20
FUNCTEST-1002 Change-Id: Id19b9aa70aeded6d99978084b3de4a63e31a81ad Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-10Remove explicit Rally task validationJuha Kosonen1-20/+4
Utilize the output of Rally task execution since it internally validates the task. Use Popen.communicate to get the output from the running Rally task. Change-Id: I9b1299bc26129ff4f89adccec611c78195a3ab02 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-07-30Delete alt images before removing the accountCédric Ollivier1-1/+1
Else it could raise unauthorized exceptions [1]. [1] https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-master/188/console Change-Id: Ia9b857069f775e2c8ecc5e07cf39b3a1fb7e3540 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-27Stop redirecting output to json filesCédric Ollivier1-15/+10
Rally may print additional data to console which results in an invalid json files. The json files are generated via rally instead of dumping stdout. It's mandatory to backport the next bugfix [1] in stable/fraser. [1] https://gerrit.opnfv.org/gerrit/#/c/60265/ Change-Id: Ifbd90e685a7b52c70b3110a46e3c7991eddc05f4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-11Clean all possible resourcesCédric Ollivier1-1/+2
It checks all openstack resources before cleaning. It allows removing images even if the test fail quickly. Change-Id: I1763b8a70fa7ffeb03c435e8e775c7afd1a7adac Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-11Check the status of inherited stepsCédric Ollivier1-1/+2
Change-Id: If63c228f2844194279b560515f4ab24a4ee88be0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-11Force result = 0 if exceptionCédric Ollivier1-0/+1
self.result can be positive due to intermediary steps (scenarios). Else it can return false positives [1]. [1] https://build.opnfv.org/ci/view/functest/job/functest-apex-virtual-suite-master/113/console Change-Id: Iecadee7f7380fda5273a0668aff1b441647c45aa Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-06-30Inherit rally from VmReady1Cédric Ollivier1-125/+15
It also fixes clean(). Change-Id: I42d1305d5cdb5a717cfa7fce26798ac7102310f7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-06-19Fix pylint errorsCédric Ollivier1-4/+4
They are highlighted by the last pylint version. Change-Id: I473212a413013492716993fb536903b13bff66ed Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-05-24Set image metadata in tempestCédric Ollivier1-1/+1
It applied the same operations as [1]. Only one config is used between rally, tempest and vping. [1] https://gerrit.opnfv.org/gerrit/#/c/57613/ Change-Id: I4d5fd2117d404b7eaa79326e21a35272232d0b92 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-05-18Refactor task results manipulation in rallyJuha Kosonen1-58/+82
Use data provided by report generation since the output of task results is not available as valid JSON if task didn't finish gracefully. JIRA: FUNCTEST-966 Change-Id: Ieb906bc209cc26055babd5f4e6b2bb556724f9a8 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-05-05Switch to Shade in rallyJuha Kosonen1-76/+79
Use Shade for managing OpenStack resources in rally. Change-Id: Ic8b7aaef6886f6ca4930b2ea463317b75ed8d3c9 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-04-22Bypass NoneType errors when creating router in rallyCédric Ollivier1-1/+1
It converts private subnet name to str as it's mandary till [1]. It could be reverted as soon as [2] is merged in snaps. [1] https://gerrit.opnfv.org/gerrit/#/c/56091/ [2] https://gerrit.opnfv.org/gerrit/#/c/56171/ Change-Id: Ia728415207f8ecb5b5ac55982f82337c45671c4c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-04-19Introduce Rally scenarios for GnocchiJuha Kosonen1-1/+1
Added scenarios for testing: - capabilities - status - archive policy - archive policy rule - resource - resource type - metric Change-Id: I79f4f0b2bd4d206588ea3edbce8037d1cb993646 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-04-09Remove Ceilometer test scenariosJuha Kosonen1-1/+1
The API utilized by Rally in Ceilometer scenarios was deprecated in OpenStack Pike and scenarios were blacklisted. Remove these scenarios from respository. JIRA: FUNCTEST-949 Change-Id: I6f100cdb18853d177fd88ea7bd539f9837914f25 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-03-15Create 2 images and flavors for tempest testsLinda Wang1-1/+1
Also 1. remove the obsolete function: generate_test_accounts_file 2. Use patch mechanism to get config Change-Id: I455ac37de71519bcaf9a51e5f1de1b9bc83b9ba6 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2018-03-11Fix many pylint warningsCédric Ollivier1-1/+1
Only pylint warnings in vnf modules has to be fixed now. Change-Id: Ie71230072a51bb72ec0b1d4fef2652494d29948e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-07Add a new env var for nameserverLinda Wang1-1/+2
We should still consider that the nameserver must be handled by all installers instead [1]: - by configuring one or more DNS resolvers in dhcp_agent.ini - by asking a local resolv It eases setting the nameserver in testcases and then allows gating them. 8.8.8.8 is still defined by default. [1] https://docs.openstack.org/neutron/pike/admin/config-dns-res.html Co-Authored-By: Cédric Ollivier <cedric.ollivier@orange.com> Change-Id: Id58dd5ebee612a0fce2ebc40f089769e60899416 Signed-off-by: Linda Wang <wangwulin@huawei.com> Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-05Fix Rally scenario parameters representationJuha Kosonen1-7/+7
Use printable format of string in scenario input parameters. JIRA: FUNCTEST-946 Change-Id: Ic4826506a6f030e749fecdc7f880e9f911f9c987 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-03-02Leverage on XtestingCédric Ollivier1-10/+8
It removes all the files which have moved to Xtesting. Vnf inheritances and env management will be improved in other changes. It keeps the same tree thanks to a symlink to allow publishing artifacts. Change-Id: I551bbd3f344cdab0158a50b7b09e541576695631 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-02-22Switch from CONST to CONFCédric Ollivier1-20/+22
It also removes constants.CONST and env.Environment which are now useless. Depends-On: I764a0a2a24447c941d1e726f3116593b29dd1c1e Depends-On: I6cfa832466dcefd737314633d807512e46267a69 Change-Id: Ife41c59d9f2e6ec4e49df38af962039f99554bc5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-02-16Get properly env vars or their default valuesCédric Ollivier1-2/+3
It defines env.get() as an unique way to get Functest env vars or their default values. It can be considered as a wrapper above os.environ. It enforces backward compatibility via CONST which mustn't be used for that purpose. It should be noted that it also stops using CONST for getting OpenStack env vars. Change-Id: I333dc1afbc0123166a7eaff8b551370098efa341 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-02-13Enhance variable manipulation in rallyJuha Kosonen1-36/+22
- utilize env vars directly via os module - use getattr to read CONST JIRA: FUNCTEST-931 Change-Id: I893a91331e26846f8effe2d3d58522950bd096fd Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-01-30Fix pylint errors/warnings in rallyJuha Kosonen1-7/+14
JIRA: FUNCTEST-906 Change-Id: I484f9a291d5adbedb855450e4ff067dfa4a3b2ae Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-01-30Remove duplicated code related to snaps credsJuha Kosonen1-14/+1
Add support for providing credentials in snaps_utils. JIRA: FUNCTEST-916 Change-Id: I3e027229d213d3791a115920f7012309cc027712 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-01-26Refactor rally report generationJuha Kosonen1-111/+64
- statistic data based on Rally results report - fix statistics calculations - use PrettyTable in generating the report Change-Id: Ic84adc5e30d446aea1f3045ba997c3ddf895e9f5 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-01-16Fix Rally output retrievalJuha Kosonen1-4/+2
Read stdout until nothing left instead of stopping once the child process is terminated. Change-Id: I46a2eb93fd614e2e7ff676727eb9dc132c29d03f Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-12-27Config physical network when creating tempest netLinda Wang1-0/+18
When the network_type is vlan, physical_network and segmentation_id could be configured. This also supports to config physical network when creating rally net. Change-Id: I9770968537c1509c5712eada66925a4bf8f6678c Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-12-11Set the ram of flavor 1024Linda Wang1-3/+7
The setting of hw:mem_page_size=large indicates larger page sizes for guest RAM, eg., either 2MB or 1GB on x86. When tested on compass, 1GB is required. Change-Id: Iefd2e8ff30e0a496157e0bd118e63ab3c402abab Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-12-06Fix the yamllint errors in functest/ciLinda Wang1-3/+1
And also, remove the params "tempest_use_custom_images" and "unique_names" of vping, rally and tempest. Change-Id: Ic4243ef2f5e965910e5befa87db8d585cad34fa2 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-11-22Merge "Removal of deprecated SNAPS-OO classes."Cedric Ollivier1-10/+12
2017-11-21Removal of deprecated SNAPS-OO classes.spisarski1-10/+12
JIRA: SNAPS-202 Change-Id: I115b4927b287a5525f4c1b95128d91f5229c7856 Signed-off-by: spisarski <s.pisarski@cablelabs.com>
2017-11-21Fix "Broken pipe" IOErrors of rally runJuha Kosonen1-23/+21
Replace obsolete os.popen with subprocess module usage in command output retrieval. Do not use shell as the program to execute. Change-Id: Ic63bb0f393750c3b85a35a37fecd46c1c1b96602 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-11-13Move rally and tempest out of functest-coreLinda Wang1-0/+2
JIRA: FUNCTEST-889 Change-Id: I96776da7af50b2c33c34dd731b5500b891d263d6 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-10-17Update blacklist about migration tests for rallyLinda Wang1-4/+4
The test NovaServers.boot_and_migrate_server needs at least 2 compute nodes: https://github.com/openstack/rally/blob/stable/0.9/rally/plugins/openstack/scenarios/nova/servers.py#L763-L765 Change-Id: I4ddf99188094ced57225d886037b00e91f3b68e6 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-10-05Live migration support determination in rallyJuha Kosonen1-20/+11
Live migration test cases were excluded if tempest was not executed before rally since tempest config file is created during the execution of tempest. Remove the dependency to config file and define the support based on the number of computes as done in tempest. Change-Id: I57c18b7c57e6a4bc8bfbdfb4beb5f8d67c6269c4 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-09-20Dedicated flavors for rally testsJuha Kosonen1-2/+30
Create flavors and use them in rally scenarios instead of expecting certain flavors to pre-exist. Change-Id: I77c94ab80fcabd7b80ffb36f9856a48121858009 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-09-18Fix image name in rally scenariosJuha Kosonen1-6/+6
Change-Id: I0a78a16d44c1e0e0b2edec2e43954c1f08e8bb09 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-09-15Refactor resource creation and cleanup in rallyJuha Kosonen1-33/+79
Use SNAPS to create and delete resources. Change-Id: Ic4d390382acea2be196e8a7518cb8cce25276a45 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-09-11Remove volume type handlingJuha Kosonen1-21/+0
Rally scenarios don't require volume type to exist as a precondition. Change-Id: Id735346e062af084635fd8f76f3686005b9b8add Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-09-11Allow live migration related tests to runLinda Wang1-1/+1
For tempest or rally, if there are more than 1 compute node, live migration related tests should not be skipped or blacklisted if shared storage is enabled via ceph. Change-Id: Ic93e4f508b597c657abe8d246f0cf9c2b5dfd020 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-09-01Add UEFI support in functestCatalina Focsa1-1/+6
Add extra properties to the cirros image for ARM architecture. In the glanceclient repository, the create method expects **kwargs as parameters, thus an extra_properties variable containing a dictionary of property-value pairs can be added (if given) to the image to be created. If the properties do not appear, the call should not affect the image properties because the dictionary is initialized as empty. Change-Id: I690da90e6c6f250bd0a3d99cce39aa250e19b9c2 Signed-off-by: Catalina Focsa <catalina.focsa@enea.com>
2017-08-24Add Rally scenarios for telemetryMartin Kulhavy1-2/+2
Ceilometer scenarios were added. Reasons for testcase blacklisting: - Deprecated ceilometer client used by Rally. This depends on the confuguration of the system, currently it seems to affect all installers. - A bug in ceilometer client (already fixed and merged in OpenStack repository). JIRA: FUNCTEST-849 Change-Id: I0bd7e1e9817538165b7b9705bcec213d7d275359 Signed-off-by: Martin Kulhavy <martin.kulhavy@nokia.com>