diff options
Diffstat (limited to 'docs/com/pres/framework/framework.md')
| -rw-r--r-- | docs/com/pres/framework/framework.md | 407 |
1 files changed, 0 insertions, 407 deletions
diff --git a/docs/com/pres/framework/framework.md b/docs/com/pres/framework/framework.md deleted file mode 100644 index 109d8a03..00000000 --- a/docs/com/pres/framework/framework.md +++ /dev/null @@ -1,407 +0,0 @@ -# Functest Framework - -created by [Cédric Ollivier](mailto:cedric.ollivier@orange.com) - -2017/06/05 - -Note: - -- Functest integrates lots of heterogeneous testcases: - - python vs bash - - internal vs external -- it aims to benefit from object programming - - to define common operations - - to avoid conditional instructions regarding the testcases - - to avoid duplicating code - - to ease the integration of third-party testcases (written in Bash or Python) - - - -## Quick overview - - -### Functest function calls - -- **CI** calls *run_tests.py* (please see [jenkins jobs](https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=tree;f=jjb/functest)) -- *run_tests.py* parses *functest/ci/testcases.yaml* to: - - check which testcase(s) must be run - - execute the common operations on every testcase (run, push its results to db...) -<!-- .element: class="fragment highlight-red"--> - - return the right status code to **CI** - - -### Our target - -- limit run_tests.py instructions by defining: - - the basic testcase attributes - - all common operations - - the status codes expected -- avoid duplicating codes between testcases -- ease the development of third-party testcases (aka features) - - - -## class TestCase - -base model for single test case - - -### instance attributes - -- project_name (default: 'functest') -- case_name -- criteria -- result -- start_time -- stop_time -- details - - -### methods - -| Method | Purpose | -|-------------------|------------------------------| -| run(**kwargs) | run the test case | -| is_successful() | interpret the results | -| get_duration() | return the duration | -| push_to_db() | push the results to the DB | -| clean() | clean the resources | - - -### run(**kwargs) - -- the subclasses must override the default implementation which is false on purpose -- the new implementation must set the following attributes to push the results to DB: - - result - - start_time - - stop_time - - -### class attributes - -| Status code | Returned when | -|--------------------|---------------------| -| EX_OK | everything is OK | -| EX_RUN_ERROR | run() failed | -| EX_TESTCASE_FAILED | results are false | -| EX_PUSH_TO_DB_ERROR| push_to_db() failed | - - -### run_tests.py - -```python -module = importlib.import_module(run_dict['module']) -cls = getattr(module, run_dict['class']) -test_dict = ft_utils.get_dict_by_test(test_name) -test_case = cls(**test_dict) -try: - kwargs = run_dict['args'] - result = test_case.run(**kwargs) -except KeyError: - result = test_case.run() -if result == testcase.TestCase.EX_OK: - if GlobalVariables.REPORT_FLAG: - test_case.push_to_db() - result = test_case.is_successful() -``` - - - -## Your first test case - - -### first.py - -```python -#!/usr/bin/env python - -import time - -from functest.core import testcase - -class Test(testcase.TestCase): - - def run(self, **kwargs): - self.start_time = time.time() - print "Hello World" - self.result = 100 - self.stop_time = time.time() - return testcase.TestCase.EX_OK -``` - - -### functest/ci/testcases.yaml - -```yaml -case_name: first -project_name: functest -criteria: 100 -blocking: true -description: '' -dependencies: - installer: '' - scenario: '' -run: - module: 'first' - class: 'Test' -``` - - - -## class Feature -bases: TestCase - -base model for single feature - - -### methods - -| Method | Purpose | -|-------------------|---------------------------| -| run(**kwargs) | run the feature | -| execute(**kwargs) | execute the Python method | - - -### run(**kwargs) - -- allows executing any Python method by calling execute() -- sets the following attributes required to push the results to DB: - - result - - start_time - - stop_time -- doesn't fulfill details when pushing the results to the DB. - - -### execute(**kwargs) - -- the subclasses must override the default implementation which is false on purpose -- the new implementation must return 0 if success or anything else if failure. - - - -## Your second test case - - -### second.py - -```python -#!/usr/bin/env python - -from functest.core import feature - -class Test(feature.Feature): - - def execute(self, **kwargs): - print "Hello World" - return 0 -``` - - -### functest/ci/testcases.yaml - -```yaml -case_name: second -project_name: functest -criteria: 100 -blocking: true -description: '' -dependencies: - installer: '' - scenario: '' -run: - module: 'second' - class: 'Test' -``` - - - -## class BashFeature -bases: Feature - -class designed to run any bash command - - -### execute(**kwargs) - -execute the cmd passed as arg. - - - -## Your third test case - - -### functest/ci/testcases.yaml - -``` -case_name: third -project_name: functest -criteria: 100 -blocking: true -description: '' -dependencies: - installer: '' - scenario: '' -run: - module: 'functest.core.feature' - class: 'BashFeature' - args: - cmd: 'echo Hello World; exit 0' -``` - - - -## class Suite -bases: TestCase - -base model for running unittest.TestSuite - - -### run(**kwargs) - -- allows running any unittest.TestSuite -- sets the following attributes required to push the results to DB: - - result - - start_time - - stop_time - - details - - - -## Your fourth test case - - -### fourth.py - -```python -#!/usr/bin/env python - -import unittest - -class TestStringMethods(unittest.TestCase): - - def test_upper(self): - self.assertEqual('Hello World'.upper(), - 'HELLO WORLD') -``` - - -### functest/ci/testcases.yaml - -``` -case_name: fourth -project_name: functest -criteria: 100 -blocking: true -description: '' -dependencies: - installer: '' - scenario: '' -run: - module: 'functest.core.unit' - class: 'Suite' - args: - name: 'fourth' -``` - - - -## class VNF -bases: TestCase - -base model for VNF onboarding testing - - -### methods - -| Method | Purpose | -|-----------------------|---------------------------------------------------| -| prepare() | prepare VNF env (user, tenant, security group,..) | -| run(**kwargs) | run VNF test case | -| deploy_orchestrator() | deploy cloudify, ONAP, OpenBaton,... (optional) | -| deploy_vnf() | deploy the VNF | -| test_vnf() | run tests on the VNF | - - -### run(**kwargs) - -- deploys an orchestrator if needed (e.g. heat, OpenBaton, Cloudify, ONAP, Juju) -- deploys the VNF -- performs tests on the VNF - - -### prepare() - -- creates a user -- creates a Tenant/Project -- allocates admin role to the user on this tenant - - -### deploy_orchestrator() - -- deploys an orchestrator (optional) -- if this function is overridden then raise orchestratorDeploymentException if error during orchestrator deployment - - -### deploy_vnf() - -- **MUST be implemented** by vnf test cases. The details section MAY be updated in the vnf test cases. -- The deployment can be executed via a specific orchestrator or using build-in orchestrators such as heat, openbaton, cloudify, juju, ONAP, ... -- returns: - True if the VNF is properly deployed - False if the VNF is not deployed -- raises VnfDeploymentException if error during VNF deployment - - -### test_vnf() - -- **MUST be implemented** by vnf test cases. The details section MAY be updated in the vnf test cases. -- Once a VNF is deployed, it is assumed that specific test suite can be run to validate the VNF. -- returns: - True if VNF tests are PASS - False if test suite is FAIL -- raises VnfTestException if error during VNF tests - - - -## Your fifth test case - - -### fifth.py - -```python -#!/usr/bin/env python - -from functest.core import vnf - -class Vnf(vnf.VnfOnBoarding): - - def deploy_vnf(self): - print "Deploy your VNF here" - print "Feed orchestrator with VNF descriptor" - return 0 - - def test_vnf(self): - print "Test your VNF here" - return 0 -``` - - -### functest/ci/testcases.yaml - -```yaml -case_name: fifth -project_name: functest -criteria: 100 -blocking: true -description: '' -dependencies: - installer: '' - scenario: '' -run: - module: 'fifth' - class: 'Vnf' -``` - - - -## Thank You! |