aboutsummaryrefslogtreecommitdiffstats
path: root/functest/opnfv_tests/openstack/tempest
AgeCommit message (Collapse)AuthorFilesLines
2017-05-18Remove check_success_ratejose.lausuch1-6/+12
After the introduction of is_successful method in TestCase class, this is not needed anymore. Change-Id: I70f6f9e10495601cd5443467347b97efe27b81a3 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-05-05Merge "Delete functest.utils.functest_logger"Jose Lausuch2-4/+4
2017-05-04Delete functest.utils.functest_loggerCédric Ollivier2-4/+4
It deletes functest.utils.functest_logger and the related unit tests. Then it modifies all functest modules to get all loggers via logging.getLogger(). __name__ is mainly used as getLogger arg as proposed by logging [1]. All loggers and handlers are now defined via functest/ci/logging.ini instead of a dict loaded by an external json file. Now only warn messages and info messages from ci and core packages are printed in console. [1] https://docs.python.org/2/library/logging.html Change-Id: Ic192855e0f9bf94825d8f7ec73549a0f3b8d44c5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-03Update tempest.conf to enable tempest from external serverLinda Wang1-1/+8
1. The endpoint type to use for OpenStack Identity API v3 "v3_endpoint_type" is set to be "adminURL" by default, which should be updated to the env OS_ENDPOINT_TYPE with the value "publicURL" 2. Reset the identity region name to "RegionOne" 3. For keystone api v3 in Newton, 5 testcases aimming at identity admin v2 test should be omitted. JIRA: FUNCTEST-801 Change-Id: I47ae41a80de0a52d32d25c9783860955f465099c Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-04-26Catch os.makedirs() exceptions in tempest.pyCédric Ollivier1-4/+2
It can raise an error exception if the leaf directory already exists or cannot be created [1]. [1] https://docs.python.org/2/library/os.html JIRA: FUNCTEST-804 Change-Id: Ie410bccf1bb2dfd36df6e918c152ecf94844b5d7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-26Switch TestCase attribute criteria to resultCédric Ollivier1-2/+2
It mainly avoids mixing input and output. Criteria is now an input set in functest/ci/testcases.yaml and then must be passed as __init__() args (which will be proposed in an additional change). Then it also renames the related TestCase method to check_result(). Change-Id: Ifc3c8e3ea6cde7e3edf7174bed4bf2bf0894e8e3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-11Modify TestCase constructor attributesCédric Ollivier1-14/+26
Every feature/testcase now allows receiving the data defined in testcases.yaml as args (name is renamed to case_name). From the time being, only project and case names are handled. Next pending patches will add criteria, cmd and repo to this list. It keeps the default values for case names except for features which will be aggregated into Feature. Change-Id: Id742d100b8183d7f10894c24ae6879d1b2b60ac9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-07Bugfixing the testcases execusionNikolas Hermanns1-2/+1
tempest.api.compute.servers.test_server_actions\ .ServerActionsTestJSON.test_reboot_server_hard is working in fuel so do test it. The "-" was missing so tempest.scenario.test_server_basic_ops .TestServerBasicOps.test_server_basic_ops was executed on fuel which is known to be not working since metadata service is not available. That was added already long ago but through the not adding of the "-" it got added again. Change-Id: I6808ce9283271e2d2f8acf7c2587aed006d1cbc2 Signed-off-by: Nikolas Hermanns <nikolas.hermanns@ericsson.com>
2017-04-04Merge "Add case_name as constructor arg"valentin boucher1-20/+14
2017-04-04Add case_name as constructor argCédric Ollivier1-20/+14
It allows managing multiple TestCase names with only one TestCase module. It is mainly required by odl which implements: - odl, - odl_netvirt, - fds. It also renames case to case_name in Features to conform with TestCases. JIRA: FUNCTEST-762 Change-Id: Ie254f754a0ea3077a8afda1c470528d38c79478f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-30Merge "Clean up Tempest blacklist"Jose Lausuch1-20/+0
2017-03-30Clean up Tempest blacklistjose.lausuch1-20/+0
This list has been generated in Colorado and there are some scenarios that are deprecated and there is no need to list them here. Change-Id: I62a30b5026ed1df65ebdb7a68a6acf3364469865 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-03-29Disable tempest server reboot test for SDNVPN and GluonGeorg Kunz1-0/+12
Bug report: https://bugs.opendaylight.org/show_bug.cgi?id=5586 As a result, floating IPs are not working in Apex at the moment. Change-Id: I437c10b53dc01810ed757f040d3378c92320274b Signed-off-by: Georg Kunz <georg.kunz@ericsson.com> Signed-off-by: Nikolas Hermanns <nikolas.hermanns@ericsson.com>
2017-03-27Rename testcase_base to testcaseCédric Ollivier1-4/+4
It simply removes the useless suffix. Change-Id: Ib9a682b57d8f33518359d817a593bb3786577390 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-27Switch TestcaseBase to TestCaseCédric Ollivier1-3/+3
It partially reverts "Rename TestCasesBase into TestcaseBase" [1] and removes the useless suffix too. [1] https://gerrit.opnfv.org/gerrit/#/c/25391/ Change-Id: Iddf2ec8d6c1dafe6e4c6713c2e69969613317471 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-25Merge "Change api version"Helen Yao1-2/+2
2017-03-24Change api versionDimitri Mazmanov1-2/+2
The api_version had full API url instead of just the version. Change-Id: I78b39ea5d39e1ffab54d2bca41df96aadd47ac86 Signed-off-by: Dimitri Mazmanov <dimitri.mazmanov@ericsson.com>
2017-03-24Merge "Clean redundant error logs of refstack_defcore in the console"Jose Lausuch1-1/+1
2017-03-23Merge "Replace domain name from Default to default"Jose Lausuch1-1/+5
2017-03-23Merge "[fuel] Skip test_server_basic_ops tempest test"Jose Lausuch1-0/+16
2017-03-23Clean redundant error logs of refstack_defcore in the consoleLinda Wang1-1/+1
1. Remove the file "refstack-error.log", and all the stderr and stdout will be stored in refstack.log 2. Clean redundant error logs in the console and all the detailed error logs will be found in refstack.log. Change-Id: I7e71567c98dc89c529421eccbf1b497962b07405 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-03-23Replace domain name from Default to defaultDimitri Mazmanov1-1/+5
The tests don't pass if the domain name is Default. Instead the domain id 'default' should be used. Change-Id: Ife919f79f007ab3fea3d53744634a92ddb172c23 Signed-off-by: Dimitri Mazmanov <dimitri.mazmanov@ericsson.com>
2017-03-15Parse refstack_defcore resultsLinda Wang1-0/+2
JIRA: DOVETAIL-366 JIRA: FUNCTEST-758 Parse refstack_defcore results, including: 1. show number of tests passed, skipped and failed 2. record the detailed info about testcases passed, skipped and failed. Change-Id: I355a907c5c9ea2739f845447144c62613ec3bb6a Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-03-15[fuel] Skip test_server_basic_ops tempest testMichael Polenchuk1-0/+16
Tempest ssh client is hanging on opendaylight enabled envs (getting deadlock in paramiko recv_exit_status method) while trying to execute a command on a vm. Change-Id: I6660cebc6bc44a5f02e6a1e83ee9a61bae141746 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-03-15Merge "refstack client integration"Morgan Richomme2-57/+115
2017-03-15refstack client integrationMatthewLi2-57/+115
JIRA: DOVETAIL-366 JIRA: FUNCTEST-758 usage: 1,functest testcase run refstack_defcore, will run the default testcases in defcore_201608.txt 2,cd <path>/functest/functest/opnfv_tests/openstack/refstack_client python refstack_client.py -c <tempest-conf-file-path> --testlist <testlist-path> this has been worked with openstack-refstack experts, etc. tested workedi. result see http://paste.openstack.org/show/602173/ Change-Id: I1aaffea01dec9dc8d1c572885bdf516614a16894 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-03-14[tempest/defcore] Configure volume device nameMichael Polenchuk2-2/+5
Actually attach_detach_volume must be skipped because the device name ain't reliable and that's using a hard-coded device name in config. This test should just get the device info after the attach and use that rather than the hard-coded config option in Tempest that ain't likely to work - the test should have been written more dynamically. However, set volume device name to "vdc" since "vdb" is used for config drive when force_config_drive=True is in action. Change-Id: Ic70de93a609df932c7749a94f71505e927df1b79 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-03-10Merge "Unit tests added for openstack/tempest"Morgan Richomme1-5/+0
2017-03-09Show verification details in tempest logJuha Kosonen1-1/+2
JIRA: FUNCTEST-747 Change-Id: I85f15188315136cc44876914325e3693f7973af8 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-03-09Install kingbird verifier extension for tempestJuha Kosonen2-0/+25
JIRA: FUNCTEST-742 Change-Id: I5de8ed22932250c542751e38eae3185c4a263cd0 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-03-01Unit tests added for openstack/tempestashishk19941-5/+0
_parse_verification_id method is removed as it is not used anywhere Change-Id: Ia2bc75ffbc2527b04cbe2428e2a3822803bde215 Signed-off-by: ashishk1994 <ashishk.iiit@gmail.com>
2017-03-01Merge "Re-enable the tests for bgpvpn scenario wich use floating ips"Jose Lausuch1-19/+0
2017-02-25Re-enable the tests for bgpvpn scenario wich use floating ipsjose.lausuch1-19/+0
Change-Id: Ia468b9c9ab9f74a3255aa3647916d1018cef2902 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-02-25Add custom tempest test.jose.lausuch2-3/+7
Some users need to run a specific list of tempest test cases. This patch allows them to do that by running the command: functest testcase run tempest_custom Change-Id: If8e7726b98c3a71398c7abdf80aea79f10653ebf Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-02-24Add Tempest Defcore test casejose.lausuch2-80/+215
Refstack is the OpenStack project that provides the tooling to run the approved specific tempest test cases (Defcore guidelines) by the Interop working group and by the OpenStack board for OpenStack certification purposes. For more info: https://refstack.openstack.org/ This adds the possibility for a Functest user to run this list of tempest test cases by just issuing the command: functest testcase run tempest_defcore The test will be run on daily loops with the scenario nosdn-nofeature on any installer. Change-Id: I920b2a44cff418116008fdf514425e69dc959412 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-02-15Fix tempest multisite configJuha Kosonen2-27/+24
Add multisite parameters into tempest config file after the file have been (re)configured by rally verifier. JIRA: FUNCTEST-721 Change-Id: I95fd7e29b5a7d1d9f82e84e383eb23e8fa813d19 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-02-14Fix config file name settingJuha Kosonen1-2/+4
JIRA: FUNCTEST-725 Change-Id: Ic1e6a54c2dbfb1dee5f8f6e6afd60382749bcd09 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-02-13Use Exception instead of return codehelenyao2-60/+32
Checking Return status code after each step can be wrapped by Exception catching Also, the return code is not as informative as exception Change-Id: Ic15791d1b8ee47e10cbae8bad46a2cb90ac2b46e Signed-off-by: helenyao <yaohelan@huawei.com>
2017-02-08Refactor configure_tempest() functiontomsou1-15/+36
Create two new functions: configure_tempest_update_params and configure_verifier, distinguishing between tempest.conf configure verification and add/updating parameters into it this update is useful and it will be used by sdnvpn/tempest Change-Id: I76e0bc00e33492252cdecab224380ecca2ce275a Signed-off-by: tomsou <soth@intracom-telecom.com>
2017-02-07Merge "bugfix for finding tempest failed testcase"Morgan Richomme1-2/+6
2017-02-06Change endpoint type for KingbirdDimitri Mazmanov1-1/+1
Replaced 'kingbird' with 'multisite' as kingbird installation scripts register 'multisite'. JIRA: MULTISITE-720 Change-Id: Ib53bab46a1fa225e371912d74eecb929d84dff3a Signed-off-by: Dimitri Mazmanov <dimitri.mazmanov@ericsson.com>
2017-02-04bugfix for finding tempest failed testcaseMatthewLi1-2/+6
JIRA: FUNCTEST-719 1.in new tempest version, FAILED-->fail see log https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-debug-master/163/artifact/results/tempest/tempest.log 2.there is a need to show the skipped testcase in tempest Change-Id: If753c59971362625ae0797a0161e07b1e4ff94c8 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-02-02Bugfix: multisite test case failsJuha Kosonen1-0/+3
Ensure the target directory for tempest config file exists. JIRA: FUNCTEST-715 Change-Id: I7bfaab1faa3e664d84813cdbf9a830b0b56a1f60 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-02-02Add tempest test details to results reportJuha Kosonen1-0/+13
JIRA: FUNCTEST-711 Change-Id: Ib89ad63f0f97ff27e62fd6af200a15b9eed646c6 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-02-02bug fix: no status returned at the end of a Tempest/Rally runMorgan Richomme1-0/+4
the return shall only deal with test executed or not it shall not return the criteria through a return statement the criteria is managed with the criteria field previous patch https://gerrit.opnfv.org/gerrit/#/c/27807 corrected the fact that we should not return the status of the test which was preventing Failed results to be pushed but it removed the return needed statement JIRA: FUNCTEST-714 Change-Id: I3d4c609467837e4ce2353c67c0b8ea95e21ebd84 Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
2017-02-01Merge "Extract util functions from tempest.py to conf_utils"Jose Lausuch2-71/+79
2017-01-31Extract util functions from tempest.py to conf_utilsjose.lausuch2-71/+79
These functions are useful for other test cases trying to run tempest. For example BGPVPN tests cases located in SDNVPN repo. Change-Id: I67e671f4e9b0a2d81a3ab0d1957ce144fb1a0432 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-01-31Tempest and SNAPS not pushing results to DB in case of FAILMorgan Richomme1-5/+0
this problem was due to the fact that we add exit condition EX_ERROR when the result was FAIL EX_ERROR shall be used as exit condition when there is an execution error, i.e the test case cannot be run if it can be run, there is no execution error the result can be PASS or FAIL, but from the CI, execution is OK JIRA: FUNCTEST-714 Change-Id: Id434bb9aa88b50277c40c717259304f26f7b3966 Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
2017-01-30Support newton in tempestMichael Polenchuk2-20/+2
* newton release support is started from 13.0.0 * set swift operator role to existing one * unblacklist server_basic_ops test JIRA: FUNCTEST-712 Change-Id: Ieb3d9477db3dfa98f940c3732b64f6eb7515f6e6 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-01-16Rally keystone test scenarios for v3helenyao2-88/+151
JIRA: FUNCTEST-652 1. Updated the Rally repository to master the keystone test scenarios of TAG= 0.7.0 cannot run on Newton also, 0.7.0 is the last tag that exists 2. Update the Tempest installation with updated rally command Rally team said "there was no possibility to do it in backward compatible way(rally verify re-design)" 3. The `rally show` was deprecated[1] in the master and corresponding logic is updated 4. Update rally commands [1] https://github.com/openstack/rally/commit/f2fc3015beb4cfd8053db728765714500db6eeb2 Change-Id: Ief9e36a16dd92e4449c0f76579e7191a93aa3e21 Signed-off-by: helenyao <yaohelan@huawei.com>