summaryrefslogtreecommitdiffstats
path: root/functest/tests/unit/openstack/tempest/test_tempest.py
AgeCommit message (Collapse)AuthorFilesLines
2019-03-19Do not allow testcase skipping in refstackJuha Kosonen1-0/+19
Set the result of refstack_defcore as failed if there were skipped testcases in a run. Change-Id: Id782c2dee511cefbafd5b4440f386fd66f934a47 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com> (cherry picked from commit cfd79ec4c39e215a25fc3d99fc5368e7d874f7bf)
2019-02-15Generate xunit reports (rally and tempest)Cédric Ollivier1-20/+5
It adds xunit reports for rally-based and tempest-based testcases. It completes the reports provided by snaps (thanks to Xtesting). All rally related operations are moved to rally. It allows removing the rally dependency to tempest which was false. Change-Id: Ia7d2476f58f4f68b7c88442e50cad844037a36e9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 3393f2016483555c27d612c69ec11274cc8aa72a)
2019-01-20Move logics out of TempestCommon.__init__()Cédric Ollivier1-0/+1
It conforms with best programming practices. It allows muting console and increasing log level (debug). Change-Id: I0e0cfef6abe3b354a43817fcb242c71da701e237 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit b482b494ce695e83cca0a10e7750200fd1be42ea)
2018-10-21Harden Rally generate reportCédric Ollivier1-4/+2
It converts html_file to str and raises all possible exceptions. It sometimes fails when locally generating reports. Change-Id: Idf6a12aaac4561800cd5e364f8bcc495aabbee1d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com> (cherry picked from commit 4d3627e56d333e313b67bbf1e7880b11883b5652)
2018-08-29Create new project/user for tempest testsJuha Kosonen1-3/+10
JIRA: FUNCTEST-1009 Change-Id: I6b9bca3827ba6cc804c25689a896ca63eec6a78b Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-08-28Remove installer type from tempest blacklistJuha Kosonen1-3/+0
Enables utilization of blacklist also on non-OPNFV environments since there is no link to installer type. JIRA: FUNCTEST-1008 Change-Id: I3a012b8b7e7dbd049f8794dff3c9d56dcc1cadad Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-07-11Check the status of inherited stepsCédric Ollivier1-1/+4
Change-Id: If63c228f2844194279b560515f4ab24a4ee88be0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-11Update rally image name regexCédric Ollivier1-1/+2
The default value doesn't conform with our image names. Else it downloads the image from internet which may be not possible (offline testing, proxy or deployment l3 issues [1]) [1] https://build.opnfv.org/ci/view/functest/job/functest-apex-virtual-suite-master/113/console Change-Id: I7d951967784d591e7155fe8582299aaa9b680b10 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-07-08Refactor tempest commonCédric Ollivier1-18/+13
It also renames two testcases and updates all docs because tests run in parallel in case of tempest smoke. To force one thread as before: run: module: 'functest.opnfv_tests.openstack.tempest.tempest' class: 'TempestCommon' args: mode: '^tempest\.(api|scenario).*\[.*\bsmoke\b.*\]$' option: - '--concurrency=1' JIRA: FUNCTEST-989 Change-Id: I1348f5908400a5046be8edf983db4afa12e546c3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-06-22Update tempest to inherit from VmReady1Cédric Ollivier1-17/+8
It also copies tempest.conf due to side effects raised by Barbican plugins. Change-Id: Ie87680f5d2e68527a5a61aac302838afc339dde8 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-06-14Add py3 support in tempest and rallyCédric Ollivier1-4/+7
Change-Id: I009d38a0db409ab4ec641cba9173ab2386d0ce2a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-04-30Rewrite TempestResourcesManagerCédric Ollivier1-32/+17
It now leverages on OpenStack Shade to allow gating OpenStack master as asked by TSC. Switching to OpenStack master (OpenStack SDK Rocky) requires to change only one line. It stops dedicating the network to the user possibly created because the support is incomplete (the users have to be registed in tempest.conf and the multiprocessing has to be enforced). It also writes the second image and the second flavor in tempest.conf. Change-Id: I9fe2c90a6073ebc240e9aa8fdf2ef93d5dd5f15f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-04-13Merge "Fix role processing in Patrole"Cedric Ollivier1-0/+16
2018-04-13Switch to stestrCédric Ollivier1-1/+1
It also updates rally and installs rally-openstack. It completes the update to OpenStack Queens. Change-Id: Iabf436ac6e7facfa21a3d7c8a3dfaffa9142da98 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-04-12Fix role processing in PatroleCédric Ollivier1-0/+16
Role has to be listed in tempest.conf before calling testr. It also saves the right config file. Change-Id: Ifd122f31d0bfe3e9b3f93c6d62526acce96953d3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-17Mute all rally messagesCédric Ollivier1-6/+4
The are now printed in functest.log only. JIRA: FUNCTEST-950 Change-Id: I1fdd0e0f1be19fc5e7536326acb6a66645610696 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-16Precise tempest regexCédric Ollivier1-1/+1
It stops running neutron tests which matched the former regex. Change-Id: Ica5a0d1f4858d01533156c98da87f9ed4662489f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-14Refactor TempestCommon to simplify subclassesCédric Ollivier1-5/+5
Change-Id: I8d265b6939861fd9d02a3d0f31aef53b9bfcb035 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-13Implement trunk ports testcasePanagiotis Karalis1-1/+5
The tempest testcases have been integrated from Neutron trunk testcases. It also enables every Neutron extension which may raise side effects [1]. [1] https://docs.openstack.org/tempest/latest/sampleconf.html JIRA: FUNCTEST-926 Change-Id: I05015acf119f504ce0a5539d677719de874594ad Signed-off-by: Panagiotis Karalis <pkaralis@intracom-telecom.com>
2018-03-10Improve tempest files managementCédric Ollivier1-3/+17
Calling apply_tempest_blacklist() is no longer mandatory. Change-Id: Id9a4e970c8acca066a36be89920745127867cd63 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-07Allow overriding tempest output dirCédric Ollivier1-4/+3
Change-Id: I31a5d2772bab168b3cb4e6b0713d6e95aa5b6fd7 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-03-02Leverage on XtestingCédric Ollivier1-15/+13
It removes all the files which have moved to Xtesting. Vnf inheritances and env management will be improved in other changes. It keeps the same tree thanks to a symlink to allow publishing artifacts. Change-Id: I551bbd3f344cdab0158a50b7b09e541576695631 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-02-24Automatically download defcore.txtCédric Ollivier1-10/+2
It stops writing files in Functest package dirs and cleans instance variables in refstack modules too. Change-Id: Iaddbe4fbaf12d1af207b86d4e44258efdc6d6f3a Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-02-19Generate tempest verification report (HTML)Juha Kosonen1-4/+19
JIRA: FUNCTEST-934 Change-Id: I2befbd3f2b2311f69b24321c7e9544fb43df3a9d Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-02-19Precise Tempest tests listCédric Ollivier1-2/+2
Else full mode could run tests defined in tempest plugins too [1]. [1] https://docs.openstack.org/tempest/latest/plugin-registry.html Change-Id: I4355b03ba1b9ba47938f748c4f1a8822fd412142 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-02-16Get properly env vars or their default valuesCédric Ollivier1-5/+5
It defines env.get() as an unique way to get Functest env vars or their default values. It can be considered as a wrapper above os.environ. It enforces backward compatibility via CONST which mustn't be used for that purpose. It should be noted that it also stops using CONST for getting OpenStack env vars. Change-Id: I333dc1afbc0123166a7eaff8b551370098efa341 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-02-12Enable invalid-name warnings in tempest modulesJuha Kosonen1-23/+22
Do not suppress pylint invalid-name warnings but conform the naming convention. Change-Id: I324d25b3081fe8a366249559b343c1b9cacadb21 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-02-09Fix pylint errors/warnings in tempest modulesJuha Kosonen1-26/+32
JIRA: FUNCTEST-913 Change-Id: I37b25bbcf43fb1d6f7de0c815ad49070d1e1af12 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-02-02Do not use shell to execute commands in tempestJuha Kosonen1-4/+3
Change-Id: I4d699481fdc63058c12ca9cfe18138e87025e9b3 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-01-31Use snaps_utils to get credentials in tempestJuha Kosonen1-1/+2
Change-Id: I1d240d5edd536d71c87a7dff4e2676715697d0e2 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2018-01-21Disable all missing-docstring warnings in unit testsCédric Ollivier1-0/+2
Change-Id: I22b7bbdab5f000b4ade6c2ba3a4029c146887cbd Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-09-02Enable tempest multinode testsJuha Kosonen1-0/+15
JIRA: FUNCTEST-774 Change-Id: I259a9a37e40dba6a6594941d989e5aa1858e4a04 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-08-23Refactor resource creation and cleanup in TempestMartin Kulhavy1-5/+13
Use Snaps to create resources before running a testcase and to clean up afterwards. Use Tempest Cleanup utility to clean other resources. Change-Id: Ic0f69d3bafb60dfb283d18ac507e9f5992e9ae38 Signed-off-by: Martin Kulhavy <martin.kulhavy@nokia.com>
2017-08-18Remove multisite supportJuha Kosonen1-8/+0
JIRA: FUNCTEST-866 Change-Id: I55a4416688b2aca0897910cac69b33e154e73095 Signed-off-by: Juha Kosonen <juha.kosonen@nokia.com>
2017-05-19Merge "Replace CONST.* by getattribute/setattr for rally and tempest"Jose Lausuch1-4/+4
2017-05-19Replace CONST.* by getattribute/setattr for rally and tempestLinda Wang1-4/+4
Directories affected: - functest/opnfv_tests/openstack/rally - functest/opnfv_tests/openstack/tempest and respective unit test JIRA: FUNCTEST-796 Change-Id: I963f5947663f0b660337f3304defb697f5d338f3 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-05-19Merge "Modify how to disable logging in unit test."Jose Lausuch1-2/+1
2017-05-18Modify how to disable logging in unit test.Cédric Ollivier1-2/+1
Logs are only disabled when the unit tests are run from cmdline. It allows running them via pytest_suite_runner.py and printing their results to functest.log. You should notice that the unit test must disable the logger of the tested TestCase (e.g. testcase.TestCase.logger.disabled = 1). Otherwise false logs will be printed in console. Change-Id: Ic171e4575c8bab2e52db1897ae026b61a03b6247 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-18Remove check_success_ratejose.lausuch1-18/+0
After the introduction of is_successful method in TestCase class, this is not needed anymore. Change-Id: I70f6f9e10495601cd5443467347b97efe27b81a3 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-04-26Fix false positives in test_tempestCédric Ollivier1-92/+74
It fixes the return values and the side effects expected from the next methods: - configure_tempest - create_tempest_resources - generate_test_list - apply_tempest_blacklist - run_verifier_tests - parse_verifier_result JIRA: FUNCTEST-803 Change-Id: Ib7dbd28e003cf0f27f497ad8e63cf7ae5ce564de Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-26Manage criteria in TestCaseCédric Ollivier1-1/+1
It converts all criteria values to the corresponding percent in functest/ci/testcases.yaml. Result is expected to be equal or greater than criteria. If both are 0, result is considered as false. It is compatible with the old behavior but warns to update. It will allow a safer remove. It also fixes a bug in test_tempest to allow merging [1] and tier_handler.py which required that type criteria was str. [1] https://gerrit.opnfv.org/gerrit/#/c/27949/ Change-Id: Ib6edcfa3103b7d51b0bdc83119f1cea2a8be9fbc Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-24Remove the useless opnfv_tests dir in testsCédric Ollivier1-0/+269
Change-Id: I41a8db181adf6c0c67b9de8380c3ccdd1ad3b529 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>