aboutsummaryrefslogtreecommitdiffstats
path: root/functest/core
AgeCommit message (Collapse)AuthorFilesLines
2017-05-18Fix 2 issues related to logger in pytest_suite_runer.pyCédric Ollivier1-3/+1
logging.ini requires the full name otherwise all log messages are lost. Calling basicConfig() is useless as it has been correctly configured by run_tests.py before. Change-Id: I03663c01f795b1b844239a9e24379faa529829aa Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-18Allow running unittest.TestCase via run_tests.pyCédric Ollivier1-0/+5
pytest_suite_runner can run any unittest.TestCase thanks to name passed as run arg. Here is the related functest/ci/testcases.yaml: run: module: 'functest.core.pytest_suite_runner' class: 'PyTestSuiteRunner' args: name: 'functest.tests.unit.odl.test_odl' Change-Id: I7c2c611c4afbad1e52bfdca4a3288ccc21491389 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-17Update self.result value to INTjose.lausuch1-4/+4
So far, the test cases are storing a string "PASS" "FAIL" in the self.result variable. The way the new framework works is with INT Values. Change-Id: I45e8693327740faadd8254b21569adfb2cefa6c8 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-05-17Merge "Fix an unorderable types error (if python 3)"Jose Lausuch1-0/+1
2017-05-17Fix an unorderable types error (if python 3)Cédric Ollivier1-0/+1
Change-Id: I07c5a548f10a1779faead4d9c596a1976aff8866 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-16Converted vPing to leverage the object-oriented SNAPS library.spisarski1-1/+1
JIRA: FUNCTEST-577 1. Replaced procedural openstack_utils with the object-oriented SNAPS library 2. Tests now clean themselves up. 3. Improved code documentation. 4. Removed dependency on file-based Cirros image and now leveraging the download URL. 5. Removed hardcoded location to the testcases.yaml file to leverage the value in config_functest.yaml 6. Included optional flavor metadata based on the scenario being tested 7. Included support for multi-part images. 8. Added enhanced support for configurable images. see SNAPS-76 Fixed line length verification problem with the previous submission Fixed retrieval of external network name to be dynamic. Changed external network name retrieval to leverage the newly merged snap_util.get_ext_net_name() function Added the creation and deletion of flavors. SNAPS now has the ability to add/remove security groups from running instances. Changed VMInstance constructor by moving userdata to configuration object. Change-Id: I0ef5e268eb153d696bb4d9a7533f780e5c603d26 Signed-off-by: spisarski <s.pisarski@cablelabs.com>
2017-05-16Merge "Get a logger before printing"Jose Lausuch1-6/+12
2017-05-16Switch from generate_report to PrettyTableCédric Ollivier2-2/+1
run_tests.py now relies on PrettyTable as most of the openstack clients. generate_report.py and its related unit tests are simply removed. It sets padding_width=5 in testcase.py too to conform with run_tests.py. Now report is printed in every case. Change-Id: Id9ce93f984503f25d6a2150482f397853fa3dd64 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-15Get a logger before printingCédric Ollivier1-6/+12
It also fixes pylint errors. Change-Id: I0d973360574a517724d0be3986c788c3ce7f54d4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-11Modify TestCase.__str__() to use PrettyTableCédric Ollivier1-6/+10
It adds PrettyTable as requirement even if it's already defined in OpenStack client dependencies. If the TestCase object is considered as invalid, it simply returns the default str. Change-Id: Iee788aef2a13694d9482560977cbbf21c7f2c967 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-10Merge "Conform ODL with last framework updates"Morgan Richomme1-4/+5
2017-05-10Merge changes from topic 'add_testcase_str'Morgan Richomme1-1/+16
* changes: Remove the former results in DB url Implement TestCase __str__()
2017-05-09Merge "Bugfix: initialize the logger for snaps"Cedric Ollivier1-0/+1
2017-05-09Conform ODL with last framework updatesCédric Ollivier1-4/+5
Now ODL TestCase calculates the ratio between critical tests passed and failed and saves it in result. Non-critical test cases can fail as result doesn't take them into account [1]. It also updates default attribute values in TestCase and allows result to be float. [1] http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#setting-criticality Change-Id: Id4a89271b5b1a90cd3c1e2b08591ff26ffaffee0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-09Rename VnfOnBoardingBase to VnfOnBoardingCédric Ollivier1-2/+2
Change-Id: I514340fc6180e29b31ef11bdea9f719aa2bbb738 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-09Rename vnf_base to vnfCédric Ollivier1-0/+0
Change-Id: I0b8b9b2e85717fc92a233e4f7344b3419421778d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-09Bugfix: initialize the logger for snapsLinda Wang1-0/+1
Testcases about snaps failed due to the absence of attribute 'logger'. Change-Id: I78c85e51f63afa767fa73aa12faea066ad432371 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-05-06Implement TestCase __str__()Cédric Ollivier1-1/+16
It's designed to be called by run_tests.py when printing the results of test cases and when generating the global report. Change-Id: If4a6f023ef2344bbc4f940d07dde4b776dce5d68 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-06Merge "Rename TestCase check_result() to is_successful()"Cedric Ollivier1-2/+2
2017-05-05Merge "Print the real testcase duration"Jose Lausuch1-0/+18
2017-05-05Rename TestCase check_result() to is_successful()Cédric Ollivier1-2/+2
Change-Id: I41cea33b4c1dae79e75231e7c7e069fc89736f34 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-05Print the real testcase durationCédric Ollivier1-0/+18
TestCase offers a new public method to calculate and print the duration. It also adds the related unit tests and adapts run_tests.py. Change-Id: Ib6a7e637dafacb9027146199aeb033c2dcb986c6 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-05Replace CONST.* by getattribute/setattrjose.lausuch1-9/+9
Directories affected: - ci - core and respective unit tests Change-Id: I6a3d5aa68de29fc5a37ae543a067ff797eba33e6 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-05-05Replace project_name by case_name in result file nameCédric Ollivier1-1/+1
It avoids erasing functest.log when project_name = functest. Change-Id: Icae898abb8b51c89b79bb1124adfadab8b0b3b99 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-04Define loggers as class-private membersCédric Ollivier3-32/+36
This mangling ensures that all info messages printed from core packages are shown in console. It also avoids sphinx to print them. Change-Id: I07db9f33060c195bce3b48b06a6640eb6c56c2eb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-04Delete functest.utils.functest_loggerCédric Ollivier3-6/+7
It deletes functest.utils.functest_logger and the related unit tests. Then it modifies all functest modules to get all loggers via logging.getLogger(). __name__ is mainly used as getLogger arg as proposed by logging [1]. All loggers and handlers are now defined via functest/ci/logging.ini instead of a dict loaded by an external json file. Now only warn messages and info messages from ci and core packages are printed in console. [1] https://docs.python.org/2/library/logging.html Change-Id: Ic192855e0f9bf94825d8f7ec73549a0f3b8d44c5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-26Manage criteria in TestCaseCédric Ollivier2-7/+19
It converts all criteria values to the corresponding percent in functest/ci/testcases.yaml. Result is expected to be equal or greater than criteria. If both are 0, result is considered as false. It is compatible with the old behavior but warns to update. It will allow a safer remove. It also fixes a bug in test_tempest to allow merging [1] and tier_handler.py which required that type criteria was str. [1] https://gerrit.opnfv.org/gerrit/#/c/27949/ Change-Id: Ib6edcfa3103b7d51b0bdc83119f1cea2a8be9fbc Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-26Switch TestCase attribute criteria to resultCédric Ollivier4-22/+22
It mainly avoids mixing input and output. Criteria is now an input set in functest/ci/testcases.yaml and then must be passed as __init__() args (which will be proposed in an additional change). Then it also renames the related TestCase method to check_result(). Change-Id: Ifc3c8e3ea6cde7e3edf7174bed4bf2bf0894e8e3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-21Switch testcase to test case in docstringsCédric Ollivier2-5/+5
Change-Id: Iea64673788a3d5ab5eb3028522f659de183440ed Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-21Merge "Minor changes in docstrings"Serena Feng2-16/+19
2017-04-20[VNF_base] Support existing tenant and userboucherv1-62/+45
Support existing tenant and user with a get_or_create_tenant method. If the tenant and user already exist it only get the id of those JIRA: FUNCTEST-799 Change-Id: Id5daf2904592e42199471bdd3547e861e6db6e32 Signed-off-by: boucherv <valentin.boucher@orange.com>
2017-04-20Minor changes in docstringsCédric Ollivier2-16/+19
It removes case_name from the list of attributes that run() must set because it's now managed by run_tests.py (see __init__()) Change-Id: Id6228880257d0e9fb27483c56aa9985197feb04e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-18Add docstings in feature.pyCédric Ollivier2-1/+53
It also modifies the testcase module docstring. Now features.py is rated 10/10 by pylint. Change-Id: I83f2ac385b2a713d116c1ae4b49ba9cc9b26a83c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-18Remove former cmd instance attributeCédric Ollivier1-1/+0
It's now useless as it's passed as run() args. Change-Id: I97e14bd431a8844cd90905bf2050c48fb00f1cc5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-18Fix Sphinx warnings/errorsCédric Ollivier1-1/+3
It takes into account the following warnings/errors: - Inline strong start-string without end-string - Unexpected indentation Change-Id: Idb05c7b04ba98ef8f3ef65e6ec7c9bb5b7571cad Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-14Add class attribute docstrings in testcase.pyCédric Ollivier1-0/+7
Change-Id: I91a5c6fb14f7c76a348bb4329d023bab53a3804c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-14Fix typos in vnf_base.pyCédric Ollivier1-2/+2
Change-Id: Idf0380930e97e1aa4584ba000f32d742a87d9486 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-14Add license in feature.pySerenaFeng1-0/+7
Change-Id: I75f51f216dcbf0db1c563cf381c28efc47241d3c Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
2017-04-14Remove deprecated feature modulesCédric Ollivier1-3/+13
All features running bash programs are now simply defined in testcases.yaml. All deprecated modules and unit tests are removed. Change-Id: I7047b6f7a1e43cb8ed5ba2d569d5dcecae68fb86 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-12Refactor the Feature frameworkCédric Ollivier1-42/+24
run() returns the expected status code (see following JIRA tickets). repo, pre() and post() are removed as they were quite useless. A dedicated class is proposed for bash programs. Unit tests have been added to fully cover this module. All features have been modified to conform with these modifications. It also removes the decorators which skipped several unit tests. JIRA: FUNCTEST-778 JIRA: FUNCTEST-779 JIRA: FUNCTEST-780 JIRA: FUNCTEST-781 Change-Id: Ifb1e4c4f68260a4e20d895f67f07f369ca959374 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-11Modify TestCase constructor attributesCédric Ollivier4-15/+14
Every feature/testcase now allows receiving the data defined in testcases.yaml as args (name is renamed to case_name). From the time being, only project and case names are handled. Next pending patches will add criteria, cmd and repo to this list. It keeps the default values for case names except for features which will be aggregated into Feature. Change-Id: Id742d100b8183d7f10894c24ae6879d1b2b60ac9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-05Bugfix: Add case_name as constructor arg for snaps and rallyLinda Wang1-2/+2
When running healthcheck tier and rally, it complains " __init__() got an unexpected keyword argument 'case_name'" Change-Id: I078094921f08b457b88beabee60eabf40b657411 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-04-04Add case_name as constructor argCédric Ollivier3-8/+7
It allows managing multiple TestCase names with only one TestCase module. It is mainly required by odl which implements: - odl, - odl_netvirt, - fds. It also renames case to case_name in Features to conform with TestCases. JIRA: FUNCTEST-762 Change-Id: Ie254f754a0ea3077a8afda1c470528d38c79478f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-29Merge "rename FeatureBase to Feature"Cedric Ollivier1-2/+2
2017-03-29Merge "rename feature_base to feature"Cedric Ollivier1-0/+0
2017-03-29Added test case for opera vimshelenyao1-11/+15
1. Created a ImsOnBoardingBase to wrap up all common ims operations 2. Created test case to test opera vims onboarding Change-Id: Idf3a82e10661271ad413f4dd5795c80ca4421d73 Signed-off-by: helenyao <yaohelan@huawei.com>
2017-03-28rename FeatureBase to FeatureSerenaFeng1-2/+2
delete useless suffix 'Base' Change-Id: Ie789da2e83cc90585fd281fe183e8430af498436 Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
2017-03-28rename feature_base to featureSerenaFeng1-0/+0
remove useless suffix _base Change-Id: I63a73fd6fef2fccd9b92c405f64dda50735b0026 Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
2017-03-27Add docstrings in testcase.pyCédric Ollivier1-0/+51
testcase.py is now rated at 10.00/10 by pylint. Change-Id: Ia90e53db7a25aac3827cd29bb9fd6baaa46cd487 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-27Rename testcase_base to testcaseCédric Ollivier4-3/+3
It simply removes the useless suffix. Change-Id: Ib9a682b57d8f33518359d817a593bb3786577390 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>