aboutsummaryrefslogtreecommitdiffstats
path: root/functest/core
AgeCommit message (Collapse)AuthorFilesLines
2017-05-10Merge "Conform ODL with last framework updates"Morgan Richomme1-4/+5
2017-05-10Merge changes from topic 'add_testcase_str'Morgan Richomme1-1/+16
* changes: Remove the former results in DB url Implement TestCase __str__()
2017-05-09Merge "Bugfix: initialize the logger for snaps"Cedric Ollivier1-0/+1
2017-05-09Conform ODL with last framework updatesCédric Ollivier1-4/+5
Now ODL TestCase calculates the ratio between critical tests passed and failed and saves it in result. Non-critical test cases can fail as result doesn't take them into account [1]. It also updates default attribute values in TestCase and allows result to be float. [1] http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#setting-criticality Change-Id: Id4a89271b5b1a90cd3c1e2b08591ff26ffaffee0 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-09Rename VnfOnBoardingBase to VnfOnBoardingCédric Ollivier1-2/+2
Change-Id: I514340fc6180e29b31ef11bdea9f719aa2bbb738 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-09Rename vnf_base to vnfCédric Ollivier1-0/+0
Change-Id: I0b8b9b2e85717fc92a233e4f7344b3419421778d Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-09Bugfix: initialize the logger for snapsLinda Wang1-0/+1
Testcases about snaps failed due to the absence of attribute 'logger'. Change-Id: I78c85e51f63afa767fa73aa12faea066ad432371 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-05-06Implement TestCase __str__()Cédric Ollivier1-1/+16
It's designed to be called by run_tests.py when printing the results of test cases and when generating the global report. Change-Id: If4a6f023ef2344bbc4f940d07dde4b776dce5d68 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-06Merge "Rename TestCase check_result() to is_successful()"Cedric Ollivier1-2/+2
2017-05-05Merge "Print the real testcase duration"Jose Lausuch1-0/+18
2017-05-05Rename TestCase check_result() to is_successful()Cédric Ollivier1-2/+2
Change-Id: I41cea33b4c1dae79e75231e7c7e069fc89736f34 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-05Print the real testcase durationCédric Ollivier1-0/+18
TestCase offers a new public method to calculate and print the duration. It also adds the related unit tests and adapts run_tests.py. Change-Id: Ib6a7e637dafacb9027146199aeb033c2dcb986c6 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-05Replace CONST.* by getattribute/setattrjose.lausuch1-9/+9
Directories affected: - ci - core and respective unit tests Change-Id: I6a3d5aa68de29fc5a37ae543a067ff797eba33e6 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-05-05Replace project_name by case_name in result file nameCédric Ollivier1-1/+1
It avoids erasing functest.log when project_name = functest. Change-Id: Icae898abb8b51c89b79bb1124adfadab8b0b3b99 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-04Define loggers as class-private membersCédric Ollivier3-32/+36
This mangling ensures that all info messages printed from core packages are shown in console. It also avoids sphinx to print them. Change-Id: I07db9f33060c195bce3b48b06a6640eb6c56c2eb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-05-04Delete functest.utils.functest_loggerCédric Ollivier3-6/+7
It deletes functest.utils.functest_logger and the related unit tests. Then it modifies all functest modules to get all loggers via logging.getLogger(). __name__ is mainly used as getLogger arg as proposed by logging [1]. All loggers and handlers are now defined via functest/ci/logging.ini instead of a dict loaded by an external json file. Now only warn messages and info messages from ci and core packages are printed in console. [1] https://docs.python.org/2/library/logging.html Change-Id: Ic192855e0f9bf94825d8f7ec73549a0f3b8d44c5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-26Manage criteria in TestCaseCédric Ollivier2-7/+19
It converts all criteria values to the corresponding percent in functest/ci/testcases.yaml. Result is expected to be equal or greater than criteria. If both are 0, result is considered as false. It is compatible with the old behavior but warns to update. It will allow a safer remove. It also fixes a bug in test_tempest to allow merging [1] and tier_handler.py which required that type criteria was str. [1] https://gerrit.opnfv.org/gerrit/#/c/27949/ Change-Id: Ib6edcfa3103b7d51b0bdc83119f1cea2a8be9fbc Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-26Switch TestCase attribute criteria to resultCédric Ollivier4-22/+22
It mainly avoids mixing input and output. Criteria is now an input set in functest/ci/testcases.yaml and then must be passed as __init__() args (which will be proposed in an additional change). Then it also renames the related TestCase method to check_result(). Change-Id: Ifc3c8e3ea6cde7e3edf7174bed4bf2bf0894e8e3 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-21Switch testcase to test case in docstringsCédric Ollivier2-5/+5
Change-Id: Iea64673788a3d5ab5eb3028522f659de183440ed Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-21Merge "Minor changes in docstrings"Serena Feng2-16/+19
2017-04-20[VNF_base] Support existing tenant and userboucherv1-62/+45
Support existing tenant and user with a get_or_create_tenant method. If the tenant and user already exist it only get the id of those JIRA: FUNCTEST-799 Change-Id: Id5daf2904592e42199471bdd3547e861e6db6e32 Signed-off-by: boucherv <valentin.boucher@orange.com>
2017-04-20Minor changes in docstringsCédric Ollivier2-16/+19
It removes case_name from the list of attributes that run() must set because it's now managed by run_tests.py (see __init__()) Change-Id: Id6228880257d0e9fb27483c56aa9985197feb04e Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-18Add docstings in feature.pyCédric Ollivier2-1/+53
It also modifies the testcase module docstring. Now features.py is rated 10/10 by pylint. Change-Id: I83f2ac385b2a713d116c1ae4b49ba9cc9b26a83c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-18Remove former cmd instance attributeCédric Ollivier1-1/+0
It's now useless as it's passed as run() args. Change-Id: I97e14bd431a8844cd90905bf2050c48fb00f1cc5 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-18Fix Sphinx warnings/errorsCédric Ollivier1-1/+3
It takes into account the following warnings/errors: - Inline strong start-string without end-string - Unexpected indentation Change-Id: Idb05c7b04ba98ef8f3ef65e6ec7c9bb5b7571cad Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-14Add class attribute docstrings in testcase.pyCédric Ollivier1-0/+7
Change-Id: I91a5c6fb14f7c76a348bb4329d023bab53a3804c Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-14Fix typos in vnf_base.pyCédric Ollivier1-2/+2
Change-Id: Idf0380930e97e1aa4584ba000f32d742a87d9486 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-14Add license in feature.pySerenaFeng1-0/+7
Change-Id: I75f51f216dcbf0db1c563cf381c28efc47241d3c Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
2017-04-14Remove deprecated feature modulesCédric Ollivier1-3/+13
All features running bash programs are now simply defined in testcases.yaml. All deprecated modules and unit tests are removed. Change-Id: I7047b6f7a1e43cb8ed5ba2d569d5dcecae68fb86 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-12Refactor the Feature frameworkCédric Ollivier1-42/+24
run() returns the expected status code (see following JIRA tickets). repo, pre() and post() are removed as they were quite useless. A dedicated class is proposed for bash programs. Unit tests have been added to fully cover this module. All features have been modified to conform with these modifications. It also removes the decorators which skipped several unit tests. JIRA: FUNCTEST-778 JIRA: FUNCTEST-779 JIRA: FUNCTEST-780 JIRA: FUNCTEST-781 Change-Id: Ifb1e4c4f68260a4e20d895f67f07f369ca959374 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-11Modify TestCase constructor attributesCédric Ollivier4-15/+14
Every feature/testcase now allows receiving the data defined in testcases.yaml as args (name is renamed to case_name). From the time being, only project and case names are handled. Next pending patches will add criteria, cmd and repo to this list. It keeps the default values for case names except for features which will be aggregated into Feature. Change-Id: Id742d100b8183d7f10894c24ae6879d1b2b60ac9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-04-05Bugfix: Add case_name as constructor arg for snaps and rallyLinda Wang1-2/+2
When running healthcheck tier and rally, it complains " __init__() got an unexpected keyword argument 'case_name'" Change-Id: I078094921f08b457b88beabee60eabf40b657411 Signed-off-by: Linda Wang <wangwulin@huawei.com>
2017-04-04Add case_name as constructor argCédric Ollivier3-8/+7
It allows managing multiple TestCase names with only one TestCase module. It is mainly required by odl which implements: - odl, - odl_netvirt, - fds. It also renames case to case_name in Features to conform with TestCases. JIRA: FUNCTEST-762 Change-Id: Ie254f754a0ea3077a8afda1c470528d38c79478f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-29Merge "rename FeatureBase to Feature"Cedric Ollivier1-2/+2
2017-03-29Merge "rename feature_base to feature"Cedric Ollivier1-0/+0
2017-03-29Added test case for opera vimshelenyao1-11/+15
1. Created a ImsOnBoardingBase to wrap up all common ims operations 2. Created test case to test opera vims onboarding Change-Id: Idf3a82e10661271ad413f4dd5795c80ca4421d73 Signed-off-by: helenyao <yaohelan@huawei.com>
2017-03-28rename FeatureBase to FeatureSerenaFeng1-2/+2
delete useless suffix 'Base' Change-Id: Ie789da2e83cc90585fd281fe183e8430af498436 Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
2017-03-28rename feature_base to featureSerenaFeng1-0/+0
remove useless suffix _base Change-Id: I63a73fd6fef2fccd9b92c405f64dda50735b0026 Signed-off-by: SerenaFeng <feng.xiaowei@zte.com.cn>
2017-03-27Add docstrings in testcase.pyCédric Ollivier1-0/+51
testcase.py is now rated at 10.00/10 by pylint. Change-Id: Ia90e53db7a25aac3827cd29bb9fd6baaa46cd487 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-27Rename testcase_base to testcaseCédric Ollivier4-3/+3
It simply removes the useless suffix. Change-Id: Ib9a682b57d8f33518359d817a593bb3786577390 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-27Switch TestcaseBase to TestCaseCédric Ollivier4-18/+18
It partially reverts "Rename TestCasesBase into TestcaseBase" [1] and removes the useless suffix too. [1] https://gerrit.opnfv.org/gerrit/#/c/25391/ Change-Id: Iddf2ec8d6c1dafe6e4c6713c2e69969613317471 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-26Fix pylint warnings in testcase_baseCédric Ollivier1-2/+3
It takes into account the following warnings: - bare-except - unused-argument - broad-except Change-Id: I3a50bf6d07823edc80a7f0f78f46681ace7f4a39 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2017-03-20[vnf_base] improve exceptionboucherv1-5/+10
Return on part of previous changes in commit: 30257 https://gerrit.opnfv.org/gerrit/#/c/30257/ Change-Id: I11b33ee2a6f9d4cbcf1449006b47be508fce655e Signed-off-by: boucherv <valentin.boucher@orange.com>
2017-03-10Remove raise exceptionsMorgan Richomme1-6/+8
until exceptions are better managed in abstraction Currently it triggers an exit from the jenkins job and prevent other tests to be run Change-Id: Id3b18c1d6d3b786fc78456b3ad51963d2cbb2cc1 Signed-off-by: Morgan Richomme <morgan.richomme@orange.com>
2017-03-08Improve error log into vnf test stepboucherv1-2/+4
Change-Id: Ied92224bc80ab4e02611e7b1f4bf33f276914e89 Signed-off-by: boucherv <valentin.boucher@orange.com>
2017-03-08Added support for scenario-based flavor creation.spisarski1-2/+0
As certain scenarios such as, DPDK, FDIO, and OVS use cases require special 'hw:' settings to flavors, SNAPS-OO has just implemented the ability to take in these configuration values to apply to its integration tests. This change checks the scenario and should be configuring the flavor metadata accordingly. Also see SNAPS-38 Used this as an opportunity to refactor the SNAPS test classes as well... JIRA: FUNCTEST-746 Change-Id: I63535776acde088a7461f602966a324b6207592d Signed-off-by: spisarski <s.pisarski@cablelabs.com>
2017-02-22Merge "Integrate BAROMETER feature test case"Jose Lausuch1-1/+9
2017-02-21Integrate BAROMETER feature test casejose.lausuch1-1/+9
Also, add a executor method in VNFBase to allow re-write it for some cases where the feature is another python script instead of shell script. Change-Id: Id734553dffe32fdc9a0befc3f4c0e29e5d56fc61 Signed-off-by: jose.lausuch <jose.lausuch@ericsson.com>
2017-02-21Fix neutron auth issueboucherv1-4/+11
Adding admin role to the new_user for the new tenant It will fix this bug for other tests based on this class JIRA: FUNCTEST-734 Change-Id: Ia3b8797fa391f528d24425a5b13c1b329cb992f4 Signed-off-by: boucherv <valentin.boucher@orange.com>
2017-02-16Fix auth errorboucherv1-4/+6
JIRA: FUNCTEST-734 Change-Id: I9e13b9b9459b9051981766fbf32126583a761656 Signed-off-by: boucherv <valentin.boucher@orange.com>