aboutsummaryrefslogtreecommitdiffstats
path: root/functest/opnfv_tests/openstack/tempest/tempest.py
AgeCommit message (Collapse)AuthorFilesLines
2017-09-15Enable tempest run concurrentlyLinda Wang1-8/+19
Append a guid to the end of names of resources created for tempest, including project, user, network, images and flavors. Change-Id: I01fc7c5cb45cd5724a99ba1d220c3f3f7d6639cf Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-09-07Remove redundant tempest cleanup utilityLinda Wang1-41/+0
All the resources including the project, user, network, images and flavors to enable tempest running, are created by snaps-OO, which will definitely clean up those finally. Change-Id: Iea3e3f275cc6cfd2cee421fcafef90aaf2e6cf3e Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-09-02Enable tempest multinode testsJuha Kosonen1-1/+4
JIRA: FUNCTEST-774 Change-Id: I259a9a37e40dba6a6594941d989e5aa1858e4a04 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-08-28Merge "Obtain scenario by CONST instead of get function"Cedric Ollivier1-2/+2
2017-08-28Avoid exiting when snapshot initialization failedLinda Wang1-5/+8
Change-Id: Ia8162d6c64847090d94f97fdab92cad2b5f059f1 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-08-28Obtain scenario by CONST instead of get functionLinda Wang1-2/+2
JIRA: FUNCTEST-763 Change-Id: Ic9fa3243b50a09de8a5e531fed8298997367dd84 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-08-24Fix the tempest error when creating snapshotLinda Wang1-3/+6
Change-Id: I9682f174a835d2bdf1ef3da01e369037e5ad7247 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-08-23Refactor resource creation and cleanup in TempestMartin Kulhavy1-5/+219
Use Snaps to create resources before running a testcase and to clean up afterwards. Use Tempest Cleanup utility to clean other resources. Change-Id: Ic0f69d3bafb60dfb283d18ac507e9f5992e9ae38 Signed-off-by: Martin Kulhavy <martin.kulhavy@nokia.com>
2017-08-22Add the skipped testcases into detailsLinda Wang1-2/+5
When running tempest_custom, if all tests are skipped, record them in details. Change-Id: Ie18ec0911ff298f91c524a9cd039929c2d0f6425 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-08-18Remove multisite supportJuha Kosonen1-15/+0
JIRA: FUNCTEST-866 Change-Id: I55a4416688b2aca0897910cac69b33e154e73095 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-08-15Remove timestamp in the details for tempestLinda Wang1-3/+3
Example: "success": ["2017-08-15 06:59:30.445 1180 INFO opnfv-tempest [-] {0} tempest.sce nario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops"] which is updated to: "success": ["tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_serv er_basic_ops"] Change-Id: Iad13e380b3a1fdc37b7742929512098bccc6d460 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-08-09Record the test cases passed in tempestLinda Wang1-6/+10
Also, convert the test cases recorded with string into list Change-Id: Ie3980a555b4042e6fe9706320d33d4ec4c06ea0c Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-06-26Remove obsolete repos in functest/ci/config_functest.yamlCédric Ollivier1-1/+2
It completes "Use directly OpenStack project packages" [1]. kingbird becomes a dependency. [1] https://gerrit.opnfv.org/gerrit/#/c/36125/ Change-Id: I2cb9e3a2146a2b2bbae89dde9917c5ace70048a6 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-30Merge "Define create_snapshot() and clean() in TestCase"Cedric Ollivier1-1/+1
2017-05-30Define create_snapshot() and clean() in TestCaseCédric Ollivier1-1/+1
They replace the former calls in run_tests which are not suitable for all test cases. Now any test case can define how to clean its resources. If the snapshot cannot be created, the test case is considered as failed. Only a message is printed if any failure during cleaning. It also defines a new class called OSGCTestCase useful for test cases which don't clean their OpenStack resources. All test cases located in opnfv_tests/openstack inherit from it to keep the global behavior unchanged. It also deletes exit instructions in openstack_clean and openstack_snapshot, removes clean flags in testcases.yaml and updates the related utils. All Docs are modified as well. JIRA: FUNCTEST-438 Change-Id: I8938e6255708012380389763a24059ace4ce45d8 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-28Switch shebangs to /usr/bin/env pythonCédric Ollivier1-1/+1
It modified the shebang to /usr/bin/env which is considered as more flexible. Change-Id: I9c8cdb4243d41a83d524499cc91060d3700e008b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-23Bugfix: fix two typosLinda Wang1-1/+1
1. In run_tests.py, test case name is shown before running: logger.error(kwargs['test']) -> logger.info(kwargs['test']) 2. In tempest.py, Verication -> Verification Change-Id: I117838cdb4b0f920c4cf855645d910e623e06868 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-05-19Replace CONST.* by getattribute/setattr for rally and tempestLinda Wang1-6/+7
Directories affected: - functest/opnfv_tests/openstack/rally - functest/opnfv_tests/openstack/tempest and respective unit test JIRA: FUNCTEST-796 Change-Id: I963f5947663f0b660337f3304defb697f5d338f3 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-05-18Remove check_success_ratejose.lausuch1-6/+12
After the introduction of is_successful method in TestCase class, this is not needed anymore. Change-Id: I70f6f9e10495601cd5443467347b97efe27b81a3 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-05-04Delete functest.utils.functest_loggerCédric Ollivier1-2/+2
It deletes functest.utils.functest_logger and the related unit tests. Then it modifies all functest modules to get all loggers via logging.getLogger(). __name__ is mainly used as getLogger arg as proposed by logging [1]. All loggers and handlers are now defined via functest/ci/logging.ini instead of a dict loaded by an external json file. Now only warn messages and info messages from ci and core packages are printed in console. [1] https://docs.python.org/2/library/logging.html Change-Id: Ic192855e0f9bf94825d8f7ec73549a0f3b8d44c5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-26Catch os.makedirs() exceptions in tempest.pyCédric Ollivier1-4/+2
It can raise an error exception if the leaf directory already exists or cannot be created [1]. [1] https://docs.python.org/2/library/os.html JIRA: FUNCTEST-804 Change-Id: Ie410bccf1bb2dfd36df6e918c152ecf94844b5d7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-26Switch TestCase attribute criteria to resultCédric Ollivier1-2/+2
It mainly avoids mixing input and output. Criteria is now an input set in functest/ci/testcases.yaml and then must be passed as __init__() args (which will be proposed in an additional change). Then it also renames the related TestCase method to check_result(). Change-Id: Ifc3c8e3ea6cde7e3edf7174bed4bf2bf0894e8e3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-11Modify TestCase constructor attributesCédric Ollivier1-14/+26
Every feature/testcase now allows receiving the data defined in testcases.yaml as args (name is renamed to case_name). From the time being, only project and case names are handled. Next pending patches will add criteria, cmd and repo to this list. It keeps the default values for case names except for features which will be aggregated into Feature. Change-Id: Id742d100b8183d7f10894c24ae6879d1b2b60ac9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-04Add case_name as constructor argCédric Ollivier1-20/+14
It allows managing multiple TestCase names with only one TestCase module. It is mainly required by odl which implements: - odl, - odl_netvirt, - fds. It also renames case to case_name in Features to conform with TestCases. JIRA: FUNCTEST-762 Change-Id: Ie254f754a0ea3077a8afda1c470528d38c79478f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-27Rename testcase_base to testcaseCédric Ollivier1-4/+4
It simply removes the useless suffix. Change-Id: Ib9a682b57d8f33518359d817a593bb3786577390 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-27Switch TestcaseBase to TestCaseCédric Ollivier1-3/+3
It partially reverts "Rename TestCasesBase into TestcaseBase" [1] and removes the useless suffix too. [1] https://gerrit.opnfv.org/gerrit/#/c/25391/ Change-Id: Iddf2ec8d6c1dafe6e4c6713c2e69969613317471 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-23Clean redundant error logs of refstack_defcore in the consoleLinda Wang1-1/+1
1. Remove the file "refstack-error.log", and all the stderr and stdout will be stored in refstack.log 2. Clean redundant error logs in the console and all the detailed error logs will be found in refstack.log. Change-Id: I7e71567c98dc89c529421eccbf1b497962b07405 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-03-15refstack client integrationMatthewLi1-57/+6
JIRA: DOVETAIL-366 JIRA: FUNCTEST-758 usage: 1,functest testcase run refstack_defcore, will run the default testcases in defcore_201608.txt 2,cd <path>/functest/functest/opnfv_tests/openstack/refstack_client python refstack_client.py -c <tempest-conf-file-path> --testlist <testlist-path> this has been worked with openstack-refstack experts, etc. tested workedi. result see http://paste.openstack.org/show/602173/ Change-Id: I1aaffea01dec9dc8d1c572885bdf516614a16894 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-03-10Merge "Unit tests added for openstack/tempest"Morgan Richomme1-5/+0
2017-03-09Show verification details in tempest logJuha Kosonen1-1/+2
JIRA: FUNCTEST-747 Change-Id: I85f15188315136cc44876914325e3693f7973af8 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-03-09Install kingbird verifier extension for tempestJuha Kosonen1-0/+1
JIRA: FUNCTEST-742 Change-Id: I5de8ed22932250c542751e38eae3185c4a263cd0 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-03-01Unit tests added for openstack/tempestashishk19941-5/+0
_parse_verification_id method is removed as it is not used anywhere Change-Id: Ia2bc75ffbc2527b04cbe2428e2a3822803bde215 Signed-off-by: ashishk1994 <ashishk.iiit@gmail.com>
2017-02-25Add custom tempest test.jose.lausuch1-3/+3
Some users need to run a specific list of tempest test cases. This patch allows them to do that by running the command: functest testcase run tempest_custom Change-Id: If8e7726b98c3a71398c7abdf80aea79f10653ebf Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-02-24Add Tempest Defcore test casejose.lausuch1-0/+9
Refstack is the OpenStack project that provides the tooling to run the approved specific tempest test cases (Defcore guidelines) by the Interop working group and by the OpenStack board for OpenStack certification purposes. For more info: https://refstack.openstack.org/ This adds the possibility for a Functest user to run this list of tempest test cases by just issuing the command: functest testcase run tempest_defcore The test will be run on daily loops with the scenario nosdn-nofeature on any installer. Change-Id: I920b2a44cff418116008fdf514425e69dc959412 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-02-15Fix tempest multisite configJuha Kosonen1-3/+3
Add multisite parameters into tempest config file after the file have been (re)configured by rally verifier. JIRA: FUNCTEST-721 Change-Id: I95fd7e29b5a7d1d9f82e84e383eb23e8fa813d19 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-02-13Use Exception instead of return codehelenyao1-42/+25
Checking Return status code after each step can be wrapped by Exception catching Also, the return code is not as informative as exception Change-Id: Ic15791d1b8ee47e10cbae8bad46a2cb90ac2b46e Signed-off-by: helenyao <yaohelan@huawei.com>
2017-02-04bugfix for finding tempest failed testcaseMatthewLi1-2/+6
JIRA: FUNCTEST-719 1.in new tempest version, FAILED-->fail see log https://build.opnfv.org/ci/view/dovetail/job/dovetail-compass-virtual-debug-master/163/artifact/results/tempest/tempest.log 2.there is a need to show the skipped testcase in tempest Change-Id: If753c59971362625ae0797a0161e07b1e4ff94c8 Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
2017-02-02Add tempest test details to results reportJuha Kosonen1-0/+13
JIRA: FUNCTEST-711 Change-Id: Ib89ad63f0f97ff27e62fd6af200a15b9eed646c6 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-02-02bug fix: no status returned at the end of a Tempest/Rally runMorgan Richomme1-0/+4
the return shall only deal with test executed or not it shall not return the criteria through a return statement the criteria is managed with the criteria field previous patch https://gerrit.opnfv.org/gerrit/#/c/27807 corrected the fact that we should not return the status of the test which was preventing Failed results to be pushed but it removed the return needed statement JIRA: FUNCTEST-714 Change-Id: I3d4c609467837e4ce2353c67c0b8ea95e21ebd84 Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
2017-02-01Merge "Extract util functions from tempest.py to conf_utils"Jose Lausuch1-68/+8
2017-01-31Extract util functions from tempest.py to conf_utilsjose.lausuch1-68/+8
These functions are useful for other test cases trying to run tempest. For example BGPVPN tests cases located in SDNVPN repo. Change-Id: I67e671f4e9b0a2d81a3ab0d1957ce144fb1a0432 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-01-31Tempest and SNAPS not pushing results to DB in case of FAILMorgan Richomme1-5/+0
this problem was due to the fact that we add exit condition EX_ERROR when the result was FAIL EX_ERROR shall be used as exit condition when there is an execution error, i.e the test case cannot be run if it can be run, there is no execution error the result can be PASS or FAIL, but from the CI, execution is OK JIRA: FUNCTEST-714 Change-Id: Id434bb9aa88b50277c40c717259304f26f7b3966 Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
2017-01-16Rally keystone test scenarios for v3helenyao1-76/+137
JIRA: FUNCTEST-652 1. Updated the Rally repository to master the keystone test scenarios of TAG= 0.7.0 cannot run on Newton also, 0.7.0 is the last tag that exists 2. Update the Tempest installation with updated rally command Rally team said "there was no possibility to do it in backward compatible way(rally verify re-design)" 3. The `rally show` was deprecated[1] in the master and corresponding logic is updated 4. Update rally commands [1] https://github.com/openstack/rally/commit/f2fc3015beb4cfd8053db728765714500db6eeb2 Change-Id: Ief9e36a16dd92e4449c0f76579e7191a93aa3e21 Signed-off-by: helenyao <yaohelan@huawei.com>
2016-12-23Fix Tempest push results to DBjose.lausuch1-28/+24
JIRA: FUNCTEST-678 Change-Id: Ia4dca2d134e91b1647694d41972d979f9b70e967 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2016-12-22Refactor tempest using new constants providerSerenaFeng1-28/+47
JIRA: FUNCTEST-673 Change-Id: Ibc274edd6bc9e8aea9e124fae84d7dec481982ab Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
2016-12-20Adapt Tempest to the new templatejose.lausuch1-0/+316
JIRA: FUNCTEST-542 Change-Id: I5576ce5c1aecd67ad97999f412e8c049c20d8e94 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>