aboutsummaryrefslogtreecommitdiffstats
path: root/functest/opnfv_tests/openstack/rally/rally.py
AgeCommit message (Collapse)AuthorFilesLines
2018-02-13Enhance variable manipulation in rallyJuha Kosonen1-36/+22
- utilize env vars directly via os module - use getattr to read CONST JIRA: FUNCTEST-931 Change-Id: I893a91331e26846f8effe2d3d58522950bd096fd Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-01-30Fix pylint errors/warnings in rallyJuha Kosonen1-7/+14
JIRA: FUNCTEST-906 Change-Id: I484f9a291d5adbedb855450e4ff067dfa4a3b2ae Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-01-30Remove duplicated code related to snaps credsJuha Kosonen1-14/+1
Add support for providing credentials in snaps_utils. JIRA: FUNCTEST-916 Change-Id: I3e027229d213d3791a115920f7012309cc027712 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-01-26Refactor rally report generationJuha Kosonen1-111/+64
- statistic data based on Rally results report - fix statistics calculations - use PrettyTable in generating the report Change-Id: Ic84adc5e30d446aea1f3045ba997c3ddf895e9f5 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-01-16Fix Rally output retrievalJuha Kosonen1-4/+2
Read stdout until nothing left instead of stopping once the child process is terminated. Change-Id: I46a2eb93fd614e2e7ff676727eb9dc132c29d03f Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-12-27Config physical network when creating tempest netLinda Wang1-0/+18
When the network_type is vlan, physical_network and segmentation_id could be configured. This also supports to config physical network when creating rally net. Change-Id: I9770968537c1509c5712eada66925a4bf8f6678c Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-12-11Set the ram of flavor 1024Linda Wang1-3/+7
The setting of hw:mem_page_size=large indicates larger page sizes for guest RAM, eg., either 2MB or 1GB on x86. When tested on compass, 1GB is required. Change-Id: Iefd2e8ff30e0a496157e0bd118e63ab3c402abab Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-12-06Fix the yamllint errors in functest/ciLinda Wang1-3/+1
And also, remove the params "tempest_use_custom_images" and "unique_names" of vping, rally and tempest. Change-Id: Ic4243ef2f5e965910e5befa87db8d585cad34fa2 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-11-22Merge "Removal of deprecated SNAPS-OO classes."Cedric Ollivier1-10/+12
2017-11-21Removal of deprecated SNAPS-OO classes.spisarski1-10/+12
JIRA: SNAPS-202 Change-Id: I115b4927b287a5525f4c1b95128d91f5229c7856 Signed-off-by: spisarski <s.pisarski@cablelabs.com>
2017-11-21Fix "Broken pipe" IOErrors of rally runJuha Kosonen1-23/+21
Replace obsolete os.popen with subprocess module usage in command output retrieval. Do not use shell as the program to execute. Change-Id: Ic63bb0f393750c3b85a35a37fecd46c1c1b96602 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-11-13Move rally and tempest out of functest-coreLinda Wang1-0/+2
JIRA: FUNCTEST-889 Change-Id: I96776da7af50b2c33c34dd731b5500b891d263d6 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-10-17Update blacklist about migration tests for rallyLinda Wang1-4/+4
The test NovaServers.boot_and_migrate_server needs at least 2 compute nodes: https://github.com/openstack/rally/blob/stable/0.9/rally/plugins/openstack/scenarios/nova/servers.py#L763-L765 Change-Id: I4ddf99188094ced57225d886037b00e91f3b68e6 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-10-05Live migration support determination in rallyJuha Kosonen1-20/+11
Live migration test cases were excluded if tempest was not executed before rally since tempest config file is created during the execution of tempest. Remove the dependency to config file and define the support based on the number of computes as done in tempest. Change-Id: I57c18b7c57e6a4bc8bfbdfb4beb5f8d67c6269c4 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-09-20Dedicated flavors for rally testsJuha Kosonen1-2/+30
Create flavors and use them in rally scenarios instead of expecting certain flavors to pre-exist. Change-Id: I77c94ab80fcabd7b80ffb36f9856a48121858009 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-09-18Fix image name in rally scenariosJuha Kosonen1-6/+6
Change-Id: I0a78a16d44c1e0e0b2edec2e43954c1f08e8bb09 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-09-15Refactor resource creation and cleanup in rallyJuha Kosonen1-33/+79
Use SNAPS to create and delete resources. Change-Id: Ic4d390382acea2be196e8a7518cb8cce25276a45 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-09-11Remove volume type handlingJuha Kosonen1-21/+0
Rally scenarios don't require volume type to exist as a precondition. Change-Id: Id735346e062af084635fd8f76f3686005b9b8add Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-09-11Allow live migration related tests to runLinda Wang1-1/+1
For tempest or rally, if there are more than 1 compute node, live migration related tests should not be skipped or blacklisted if shared storage is enabled via ceph. Change-Id: Ic93e4f508b597c657abe8d246f0cf9c2b5dfd020 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-09-01Add UEFI support in functestCatalina Focsa1-1/+6
Add extra properties to the cirros image for ARM architecture. In the glanceclient repository, the create method expects **kwargs as parameters, thus an extra_properties variable containing a dictionary of property-value pairs can be added (if given) to the image to be created. If the properties do not appear, the call should not affect the image properties because the dictionary is initialized as empty. Change-Id: I690da90e6c6f250bd0a3d99cce39aa250e19b9c2 Signed-off-by: Catalina Focsa <catalina.focsa@enea.com>
2017-08-24Add Rally scenarios for telemetryMartin Kulhavy1-2/+2
Ceilometer scenarios were added. Reasons for testcase blacklisting: - Deprecated ceilometer client used by Rally. This depends on the confuguration of the system, currently it seems to affect all installers. - A bug in ceilometer client (already fixed and merged in OpenStack repository). JIRA: FUNCTEST-849 Change-Id: I0bd7e1e9817538165b7b9705bcec213d7d275359 Signed-off-by: Martin Kulhavy <martin.kulhavy@nokia.com>
2017-07-11Add decorators in testcases for energy recording.Benoit HERARD1-70/+98
Test cases are: - rally (sanity and full) - refstack - cloudify_ims - vping_ssh It also fix a bug on "energy_recording" decorator of energy module. Change-Id: I625c214552ad1b89eaf7e196e706d7c69b37da33 Signed-off-by: Benoit HERARD <benoit.herard@orange.com>
2017-07-10Allow regex for blacklist scenarios/installersMartin Kulhavy1-9/+35
Reasoning: Some blacklisted test cases apply to a range of scenarios, e.g. both 'ha' and 'noha' and possibly even to all scenarios for a specific installer -- therefore the list of excluded scenarios can get long and unclear. For consistency with rally test case scenarios also allow regex for installers specification. Additionally, log the excluded test cases (if any) to debug log. Change-Id: Ie0bec35d1a6443203d4719ea5428c437c12e5e12 Signed-off-by: Martin Kulhavy <martin.kulhavy@nokia.com>
2017-06-27Remove Rally requests scenarioJuha Kosonen1-4/+1
HttpRequests scenario is not directly related to OpenStack and not supported by Rally in https environment. Change-Id: I2e91d129d82dfaba62eca25e3e4890ff54f52ac6 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-06-20Remove all references to /home/opnfv/repos/functestCédric Ollivier1-6/+9
It also removes relative (and incomplete) references which could be defined in modules instead of this global configuration. Change-Id: Ie20dc3547e49b6224aa8100cd380d37b87ece5a9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-30Merge "Define create_snapshot() and clean() in TestCase"Cedric Ollivier1-1/+1
2017-05-30Define create_snapshot() and clean() in TestCaseCédric Ollivier1-1/+1
They replace the former calls in run_tests which are not suitable for all test cases. Now any test case can define how to clean its resources. If the snapshot cannot be created, the test case is considered as failed. Only a message is printed if any failure during cleaning. It also defines a new class called OSGCTestCase useful for test cases which don't clean their OpenStack resources. All test cases located in opnfv_tests/openstack inherit from it to keep the global behavior unchanged. It also deletes exit instructions in openstack_clean and openstack_snapshot, removes clean flags in testcases.yaml and updates the related utils. All Docs are modified as well. JIRA: FUNCTEST-438 Change-Id: I8938e6255708012380389763a24059ace4ce45d8 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-28Switch shebangs to /usr/bin/env pythonCédric Ollivier1-1/+1
It modified the shebang to /usr/bin/env which is considered as more flexible. Change-Id: I9c8cdb4243d41a83d524499cc91060d3700e008b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-19Replace CONST.* by getattribute/setattr for rally and tempestLinda Wang1-15/+18
Directories affected: - functest/opnfv_tests/openstack/rally - functest/opnfv_tests/openstack/tempest and respective unit test JIRA: FUNCTEST-796 Change-Id: I963f5947663f0b660337f3304defb697f5d338f3 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-05-19Merge "Add a new directory to hold pre-downloaded images"Helen Yao1-1/+1
2017-05-18Remove check_success_ratejose.lausuch1-10/+10
After the introduction of is_successful method in TestCase class, this is not needed anymore. Change-Id: I70f6f9e10495601cd5443467347b97efe27b81a3 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-05-18Add a new directory to hold pre-downloaded imageshelenyao1-1/+1
Put all images in the new direcotry. If users want to use the local image, map the local directory to the ${HOME}/functest/images Change-Id: I0d309b93f52b3da23d6130056f1a19907313ef68 Signed-off-by: helenyao <yaohelan@huawei.com>
2017-05-04Delete functest.utils.functest_loggerCédric Ollivier1-2/+2
It deletes functest.utils.functest_logger and the related unit tests. Then it modifies all functest modules to get all loggers via logging.getLogger(). __name__ is mainly used as getLogger arg as proposed by logging [1]. All loggers and handlers are now defined via functest/ci/logging.ini instead of a dict loaded by an external json file. Now only warn messages and info messages from ci and core packages are printed in console. [1] https://docs.python.org/2/library/logging.html Change-Id: Ic192855e0f9bf94825d8f7ec73549a0f3b8d44c5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-26Switch TestCase attribute criteria to resultCédric Ollivier1-2/+2
It mainly avoids mixing input and output. Criteria is now an input set in functest/ci/testcases.yaml and then must be passed as __init__() args (which will be proposed in an additional change). Then it also renames the related TestCase method to check_result(). Change-Id: Ifc3c8e3ea6cde7e3edf7174bed4bf2bf0894e8e3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-11Modify TestCase constructor attributesCédric Ollivier1-6/+10
Every feature/testcase now allows receiving the data defined in testcases.yaml as args (name is renamed to case_name). From the time being, only project and case names are handled. Next pending patches will add criteria, cmd and repo to this list. It keeps the default values for case names except for features which will be aggregated into Feature. Change-Id: Id742d100b8183d7f10894c24ae6879d1b2b60ac9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-05Bugfix: Add case_name as constructor arg for snaps and rallyLinda Wang1-6/+4
When running healthcheck tier and rally, it complains " __init__() got an unexpected keyword argument 'case_name'" Change-Id: I078094921f08b457b88beabee60eabf40b657411 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-04-04Add case_name as constructor argCédric Ollivier1-2/+2
It allows managing multiple TestCase names with only one TestCase module. It is mainly required by odl which implements: - odl, - odl_netvirt, - fds. It also renames case to case_name in Features to conform with TestCases. JIRA: FUNCTEST-762 Change-Id: Ie254f754a0ea3077a8afda1c470528d38c79478f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-27Rename testcase_base to testcaseCédric Ollivier1-4/+4
It simply removes the useless suffix. Change-Id: Ib9a682b57d8f33518359d817a593bb3786577390 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-27Switch TestcaseBase to TestCaseCédric Ollivier1-3/+3
It partially reverts "Rename TestCasesBase into TestcaseBase" [1] and removes the useless suffix too. [1] https://gerrit.opnfv.org/gerrit/#/c/25391/ Change-Id: Iddf2ec8d6c1dafe6e4c6713c2e69969613317471 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-02-13Use Exception instead of return codehelenyao1-5/+4
Checking Return status code after each step can be wrapped by Exception catching Also, the return code is not as informative as exception Change-Id: Ic15791d1b8ee47e10cbae8bad46a2cb90ac2b46e Signed-off-by: helenyao <yaohelan@huawei.com>
2017-02-03Unit tests for openstack/rally.ashishk19941-1/+2
Few bug fix in rally code along with unit tests. Change-Id: Id39a5089ca6fce8aac5ba89eb61d31b02a5e9a98 Signed-off-by: ashishk1994 <ashishk.iiit@gmail.com>
2017-02-02bug fix: no status returned at the end of a Tempest/Rally runMorgan Richomme1-0/+4
the return shall only deal with test executed or not it shall not return the criteria through a return statement the criteria is managed with the criteria field previous patch https://gerrit.opnfv.org/gerrit/#/c/27807 corrected the fact that we should not return the status of the test which was preventing Failed results to be pushed but it removed the return needed statement JIRA: FUNCTEST-714 Change-Id: I3d4c609467837e4ce2353c67c0b8ea95e21ebd84 Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
2017-02-01[rally] Get keystone auth endpoint as request_urlMichael Polenchuk1-5/+2
JIRA: FUNCTEST-685 Change-Id: Id0d0bcf55451832d9aeb22c3e6b9c21aea4e0911 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-01-31Tempest and SNAPS not pushing results to DB in case of FAILMorgan Richomme1-5/+0
this problem was due to the fact that we add exit condition EX_ERROR when the result was FAIL EX_ERROR shall be used as exit condition when there is an execution error, i.e the test case cannot be run if it can be run, there is no execution error the result can be PASS or FAIL, but from the CI, execution is OK JIRA: FUNCTEST-714 Change-Id: Id434bb9aa88b50277c40c717259304f26f7b3966 Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
2017-01-25Refactor Rally to adopt OOhelenyao1-0/+559
JIRA: FUNCTEST-541 Change-Id: Ife04f58dc31ca9ba6316cc31577f81a6bf0e0405 Signed-off-by: helenyao <yaohelan@huawei.com>