aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--docker/parser/Dockerfile2
-rw-r--r--docs/com/pres/framework/framework.md97
-rw-r--r--docs/testing/developer/devguide/index.rst17
-rw-r--r--docs/testing/user/configguide/configguide.rst48
-rw-r--r--docs/testing/user/userguide/index.rst13
-rw-r--r--docs/testing/user/userguide/test_details.rst45
-rw-r--r--docs/testing/user/userguide/test_overview.rst3
-rw-r--r--docs/testing/user/userguide/troubleshooting.rst37
-rw-r--r--functest/core/vnf.py19
9 files changed, 193 insertions, 88 deletions
diff --git a/docker/parser/Dockerfile b/docker/parser/Dockerfile
index a28a1f5b..b1c2981f 100644
--- a/docker/parser/Dockerfile
+++ b/docker/parser/Dockerfile
@@ -24,7 +24,7 @@ RUN apk --no-cache add --update \
pip install --no-cache-dir --src /src -cupper-constraints.txt \
-chttps://git.opnfv.org/functest/plain/upper-constraints.txt?h=$BRANCH \
git+https://gerrit.opnfv.org/gerrit/functest@$BRANCH#egg=functest \
- git+https://gerrit.opnfv.org/gerrit/parser#egg=heat-translator\&subdirectory=tosca2heat/heat-translator \
+ git+https://gerrit.opnfv.org/gerrit/parser#egg=nfv-heattranslator\&subdirectory=tosca2heat/heat-translator \
git+https://gerrit.opnfv.org/gerrit/parser#egg=nfv-toscaparser\&subdirectory=tosca2heat/tosca-parser \
-e git+https://gerrit.opnfv.org/gerrit/parser#egg=nfv-parser && \
rm -r upper-constraints.txt /src/rally/.git && \
diff --git a/docs/com/pres/framework/framework.md b/docs/com/pres/framework/framework.md
index dfd7a215..7979bede 100644
--- a/docs/com/pres/framework/framework.md
+++ b/docs/com/pres/framework/framework.md
@@ -306,15 +306,102 @@ run:
-## Euphrates
+## class VNF
+bases: TestCase
+
+base model for VNF onboarding testing
+
+
+### methods
+
+| Method | Purpose |
+|-----------------------|---------------------------------------------------|
+| prepare() | prepare VNF env (user, tenant, security group,..) |
+| run(**kwargs) | run VNF test case |
+| deploy_orchestrator() | deploy cloudify, ONAP, OpenBaton,... (optional) |
+| deploy_vnf() | deploy the VNF |
+| test_vnf() | run tests on the VNF |
+
+
+### run(**kwargs)
+
+- deploys an orchestrator if needed (e.g. heat, OpenBaton, Cloudify, ONAP, Juju)
+- deploys the VNF
+- performs tests on the VNF
+
+
+### prepare()
+
+- creates a user
+- creates a Tenant/Project
+- allocates admin role to the user on this tenant
+
+
+### deploy_orchestrator()
+
+- deploys an orchestrator (optional)
+- if this function is overridden then raise orchestratorDeploymentException if error during orchestrator deployment
+
+
+### deploy_vnf()
+
+- **MUST be implemented** by vnf test cases. The details section MAY be updated in the vnf test cases.
+- The deployment can be executed via a specific orchestrator or using build-in orchestrators such as heat, openbaton, cloudify, juju, ONAP, ...
+- returns:
+ True if the VNF is properly deployed
+ False if the VNF is not deployed
+- raises VnfDeploymentException if error during VNF deployment
+
+
+### test_vnf()
+
+- **MUST be implemented** by vnf test cases. The details section MAY be updated in the vnf test cases.
+- Once a VNF is deployed, it is assumed that specific test suite can be run to validate the VNF.
+- returns:
+ True if VNF tests are PASS
+ False if test suite is FAIL
+- raises VnfTestException if error during VNF tests
+
+
+## Your fifth test case
-### Next actions
-- __to finish VNF abstraction (coverage + pylint)__
-- to publish doc API
+### fifth.py
-Please see [Functest Euphrates page](https://wiki.opnfv.org/display/functest/Functest+Euphrates+page) for more details
+```python
+#!/usr/bin/env python
+
+from functest.core import vnf
+
+class Vnf(vnf.VnfOnBoarding):
+
+ def deploy_vnf(self):
+ print "Deploy your VNF here"
+ print "Feed orchestrator with VNF descriptor"
+ return 0
+
+ def test_vnf(self):
+ print "Test your VNF here"
+ return 0
+```
+
+
+### functest/ci/testcases.yaml
+
+```yaml
+case_name: fifth
+project_name: functest
+criteria: 100
+blocking: true
+description: ''
+dependencies:
+ installer: ''
+ scenario: ''
+run:
+ module: 'fifth'
+ class: 'Vnf'
+```
diff --git a/docs/testing/developer/devguide/index.rst b/docs/testing/developer/devguide/index.rst
index 1eae33f6..96ba53e6 100644
--- a/docs/testing/developer/devguide/index.rst
+++ b/docs/testing/developer/devguide/index.rst
@@ -197,12 +197,7 @@ The tiers are:
* smoke
* features
* components
- * performance
* vnf
- * stress
-
-Note Functest deals with healthcheck, smoke, features, components and vnf tiers.
-Performance and stress tiers are out of scope.
Functest abstraction classes
============================
@@ -217,7 +212,7 @@ introduced:
The goal is to unify the way to run tests in Functest.
-Feature, unit and vnf_base inherit from testcase:
+Feature, unit and vnf_base inherit from testcase::
+-----------------------------------------+
| |
@@ -225,12 +220,12 @@ Feature, unit and vnf_base inherit from testcase:
| |
| - init() |
| - run() |
- | - publish_report() |
- | - check_criteria() |
+ | - push_to_db() |
+ | - is_successful() |
| |
+-----------------------------------------+
- | |
- V V
+ | | |
+ V V V
+--------------------+ +--------------+ +--------------------------+
| | | | | |
| feature | | unit | | vnf |
@@ -280,7 +275,7 @@ follows::
functest/utils/
|-- config.py
|-- constants.py
- |-- decoratos.py
+ |-- decorators.py
|-- env.py
|-- functest_utils.py
|-- openstack_clean.py
diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst
index 9d0179c8..e41700b2 100644
--- a/docs/testing/user/configguide/configguide.rst
+++ b/docs/testing/user/configguide/configguide.rst
@@ -353,16 +353,14 @@ includes two main directories:
src and repos directories are used to host third party code used for the tests.
-The functest code is under /usr/lib/python2.7/site-packages/functest
-The structure can be described as follows::
+The structure of functest repo can be described as follows::
|-- INFO
|-- LICENSE
|-- api
| `-- apidoc
|-- build.sh
- |-- commons
- | |-- docker
+ |-- docker
| |-- Dockerfile
| |-- Dockerfile.aarch64.patch
| |-- components
@@ -373,7 +371,9 @@ The structure can be described as follows::
| |-- healthcheck
| |-- smoke
| |-- vnf
- | `-- thirdparty-requirements.txt
+ | |-- parser
+ | |-- restapi
+ | |-- thirdparty-requirements.txt
|-- docs
| |-- com
| |-- images
@@ -392,13 +392,20 @@ The structure can be described as follows::
| |-- urls.py
| |-- common
| | |-- api_utils.py
- | | `-- error.py
+ | | |-- thread.py
| `-- resources
| `-- v1
| |-- creds.py
| |-- envs.py
| |-- testcases.py
- | `-- tiers.py
+ | |-- tiers.py
+ | |-- tasks.py
+ | `-- database
+ | |-- db.py
+ | `-- v1
+ | |-- handlers.py
+ | |-- models.py
+ | `-- swagger
|-- ci
│   |-- check_deployment.py
│   |-- config_aarch64_patch.yaml
@@ -412,36 +419,32 @@ The structure can be described as follows::
│   |-- run_tests.py
│   |-- testcases.yaml
│   |-- tier_builder.py
- │   `-- tier_handler.py
+ │   |-- tier_handler.py
|-- cli
│   |-- cli_base.py
│   |-- commands
│   │   |-- cli_env.py
│   │   |-- cli_os.py
│   │   |-- cli_testcase.py
- │   │   `-- cli_tier.py
+ │   │   |-- cli_tier.py
│   |-- functest-complete.sh
|-- core
│   |-- feature.py
│   |-- testcase.py
│   |-- unit.py
- │   `-- vnf.py
+ │   |-- vnf.py
|-- energy
│   |-- energy.py
- │   `-- energy.pyc
|-- opnfv_tests
- │   |-- mano
- │   │   |-- orchestra.py
- │   |-- openstack
- │   │   |-- rally
- │   │   |-- refstack_client
- │   │   |-- snaps
- │   │   |-- tempest
- │   │   `-- vping
- │   |-- sdn
+ │   `-- openstack
+ │      |-- rally
+ │      |-- refstack_client
+ │      |-- snaps
+ │      |-- tempest
+ │      |-- vping
+ │   `-- sdn
│   │   `-- odl
│   `-- vnf
- │   |-- aaa
│   |-- ims
│   `-- router
|-- tests
@@ -490,7 +493,8 @@ We may distinguish several directories, the first level has 5 directories:
* **functest**: This directory contains all the code needed to run
functest internal cases and OPNFV onboarded feature or VNF test cases.
-Functest directory has 7 sub-directories:
+Functest directory has 7 sub-directories, which is located under
+/usr/lib/python2.7/site-packages/functest:
* **api**: This directory is dedicated for the internal Functest API and the
API (framework) documentations.
* **ci**: This directory contains test structure definition files
diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst
index c0c76365..2addfaea 100644
--- a/docs/testing/user/userguide/index.rst
+++ b/docs/testing/user/userguide/index.rst
@@ -10,19 +10,6 @@ Functest User Guide
.. toctree::
:maxdepth: 2
-Version history
-===============
-+------------+----------+------------------+----------------------------------+
-| **Date** | **Ver.** | **Author** | **Comment** |
-| | | | |
-+------------+----------+------------------+----------------------------------+
-| 2016-08-17 | 1.0.0 | Juha Haapavirta | Colorado release |
-| | | Column Gaynor | |
-+------------+----------+------------------+----------------------------------+
-| 2017-01-23 | 1.0.1 | Morgan Richomme | Adaptations for Danube |
-+------------+----------+------------------+----------------------------------+
-| 2017-08-16 | 1.0.2 | Morgan Richomme | Adaptations for Euphrates |
-+------------+----------+------------------+----------------------------------+
Introduction
diff --git a/docs/testing/user/userguide/test_details.rst b/docs/testing/user/userguide/test_details.rst
index c9ef63d3..56cdd3b6 100644
--- a/docs/testing/user/userguide/test_details.rst
+++ b/docs/testing/user/userguide/test_details.rst
@@ -426,28 +426,29 @@ The Clearwater architecture is described as follows:
:align: center
:alt: vIMS architecture
- cloudify_ims_perf
- ^^^^^^^^^^^^
- This testcase extends the cloudify_ims test case.
- The first part is similar but the testing part is different.
- The testing part consists in automating a realistic signaling load on the vIMS
- using an Ixia loader (proprietary tools)
- - You need to have access to an Ixia licence server
- defined in the configuration file.
-
- To start this test you need to have access to an Ixia licence server and have ixia image locally
- -
- case_name: cloudify_ims_perf
- project_name: functest
- criteria: 100
- blocking: false
- description: ''
- dependencies:
- installer: ''
- scenario: 'o'
- run:
- module: 'functest.opnfv_tests.vnf.ims.cloudify_ims_perf'
- class: 'CloudifyImsPerf'
+
+cloudify_ims_perf
+^^^^^^^^^^^^^^^^^
+This testcase extends the cloudify_ims test case.
+The first part is similar but the testing part is different.
+The testing part consists in automating a realistic signaling load on the vIMS
+using an Ixia loader (proprietary tools)
+ - You need to have access to an Ixia licence server defined in the configuration
+ file.
+
+To start this test you need to have access to an Ixia licence server and have ixia image locally
+ -
+ case_name: cloudify_ims_perf
+ project_name: functest
+ criteria: 100
+ blocking: false
+ description: ''
+ dependencies:
+ installer: ''
+ scenario: ''
+ run:
+ module: 'functest.opnfv_tests.vnf.ims.cloudify_ims_perf'
+ class: 'CloudifyImsPerf'
orchestra_openims
^^^^^^^^^^^^^^^^^
diff --git a/docs/testing/user/userguide/test_overview.rst b/docs/testing/user/userguide/test_overview.rst
index 2fca325a..a23df02d 100644
--- a/docs/testing/user/userguide/test_overview.rst
+++ b/docs/testing/user/userguide/test_overview.rst
@@ -77,9 +77,6 @@ validate the scenario for the release.
| | | rally_full | Run the OpenStack testing tool |
| | | | benchmarking OpenStack modules |
| | | | See the Rally documents `[3]`_ |
-| | +----------------+----------------------------------+
-| | | tempest_custom | Allow to run a customized list |
-| | | | of Tempest cases |
+-------------+---------------+----------------+----------------------------------+
| Controllers | smoke | odl | Opendaylight Test suite |
| | | | Limited test suite to check the |
diff --git a/docs/testing/user/userguide/troubleshooting.rst b/docs/testing/user/userguide/troubleshooting.rst
index 29facc29..3ceb914c 100644
--- a/docs/testing/user/userguide/troubleshooting.rst
+++ b/docs/testing/user/userguide/troubleshooting.rst
@@ -249,6 +249,43 @@ related REST API requests/responses are output to the console. More detailed deb
information can be found from tempest.log file stored into related Rally deployment
folder.
+Functest offers a possibility to test a customized list of Tempest test cases.
+To enable that, add a new entry in docker/components/testcases.yaml on the "components" container
+with the following content::
+
+ -
+ case_name: tempest_custom
+ project_name: functest
+ criteria: 100
+ blocking: false
+ description: >-
+ The test case allows running a customized list of tempest
+ test cases
+ dependencies:
+ installer: ''
+ scenario: ''
+ run:
+ module: 'functest.opnfv_tests.openstack.tempest.tempest'
+ class: 'TempestCustom'
+
+Also, a list of the Tempest test cases must be provided to the container or modify
+the existing one in
+/usr/lib/python2.7/site-packages/functest/opnfv_tests/openstack/tempest/custom_tests/test_list.txt
+
+Example of custom list of tests 'my-custom-tempest-tests.txt'::
+
+ tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops[compute,id-7fff3fb3-91d8-4fd0-bd7d-0204f1f180ba,network,smoke]
+ tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,id-f323b3ba-82f8-4db7-8ea6-6a895869ec49,network,smoke]
+
+This is an example of running a customized list of Tempest tests in Functest::
+
+ sudo docker run --env-file env \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
+ -v $(pwd)/images:/home/opnfv/functest/images \
+ -v $(pwd)/my-custom-testcases.yaml:/usr/lib/python2.7/site-packages/functest/ci/testcases.yaml \
+ -v $(pwd)/my-custom-tempest-tests.txt:/usr/lib/python2.7/site-packages/functest/opnfv_tests/openstack/tempest/custom_tests/test_list.txt \
+ opnfv/functest-components /bin/bash -c "prepare_env start && run_tests -t tempest_custom"
+
Rally
^^^^^
diff --git a/functest/core/vnf.py b/functest/core/vnf.py
index 517838a7..507e2765 100644
--- a/functest/core/vnf.py
+++ b/functest/core/vnf.py
@@ -55,16 +55,16 @@ class VnfOnBoarding(base.TestCase):
"""
Run of the VNF test case:
- * Deploy an orchestrator if needed (e.g. heat, cloudify, ONAP),
+ * Deploy an orchestrator if needed (e.g. heat, cloudify, ONAP,...),
* Deploy the VNF,
* Perform tests on the VNF
A VNF test case is successfull when the 3 steps are PASS
If one of the step is FAIL, the test case is FAIL
- Returns:
- TestCase.EX_OK if result is 'PASS'.
- TestCase.EX_TESTCASE_FAILED otherwise.
+ Returns:
+ TestCase.EX_OK if result is 'PASS'.
+ TestCase.EX_TESTCASE_FAILED otherwise.
"""
self.start_time = time.time()
@@ -137,9 +137,8 @@ class VnfOnBoarding(base.TestCase):
"""
Deploy an orchestrator (optional).
- If function overwritten
- raise orchestratorDeploymentException if error during orchestrator
- deployment
+ If this method is overriden then raise orchestratorDeploymentException
+ if error during orchestrator deployment
"""
self.__logger.info("Deploy orchestrator (if necessary)")
return True
@@ -152,10 +151,8 @@ class VnfOnBoarding(base.TestCase):
The details section MAY be updated in the vnf test cases.
The deployment can be executed via a specific orchestrator
- or using nuild-in orchestrators such as:
-
- * heat, openbaton, cloudify (available on all scenario),
- * open-o (on open-o scenarios)
+ or using build-in orchestrators such as heat, OpenBaton, cloudify,
+ juju, onap, ...
Returns:
True if the VNF is properly deployed