aboutsummaryrefslogtreecommitdiffstats
path: root/docs/testing
diff options
context:
space:
mode:
authorCédric Ollivier <cedric.ollivier@orange.com>2019-08-31 10:56:23 +0200
committerCédric Ollivier <cedric.ollivier@orange.com>2019-08-31 11:27:50 +0200
commit4ef1169759eaeb39144ac7e2a3f814ea86af7a67 (patch)
treef67fc17b2043e360c4567cc725a93ae53bb5b44f /docs/testing
parent22b978b7a3790ec393024152e0d8f2254635e81e (diff)
Remove Features containers
Most the features are stopped and Functest shouldn't host dead code. Functest is a framework (Docker and Python modules) which can be still reused by any third-party (OPNFV Feature or anything else) in its tree and CI. Change-Id: Ia8475ce295b2cdcddbfcfc5b17ea2ab9e51a8afb Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
Diffstat (limited to 'docs/testing')
-rw-r--r--docs/testing/developer/devguide/index.rst36
-rw-r--r--docs/testing/user/configguide/ci.rst4
-rw-r--r--docs/testing/user/configguide/configguide.rst45
-rw-r--r--docs/testing/user/userguide/test_details.rst47
-rw-r--r--docs/testing/user/userguide/test_overview.rst57
-rw-r--r--docs/testing/user/userguide/test_results.rst29
6 files changed, 5 insertions, 213 deletions
diff --git a/docs/testing/developer/devguide/index.rst b/docs/testing/developer/devguide/index.rst
index 4091a39f9..a0faf42c1 100644
--- a/docs/testing/developer/devguide/index.rst
+++ b/docs/testing/developer/devguide/index.rst
@@ -52,7 +52,6 @@ to the different tiers:
* functest-core: https://hub.docker.com/r/opnfv/functest-core/
* functest-healthcheck: https://hub.docker.com/r/opnfv/functest-healthcheck/
* functest-smoke: https://hub.docker.com/r/opnfv/functest-smoke/
- * functest-features: https://hub.docker.com/r/opnfv/functest-features/
* functest-vnf: https://hub.docker.com/r/opnfv/functest-vnf/
* functest-restapi: https://hub.docker.com/r/opnfv/functest-restapi/
@@ -108,41 +107,6 @@ repository, the internal test cases can be grouped by domain:
If you want to create a new test case you will have to create a new folder
under the testcases directory (See next section for details).
-Functest external test cases
-============================
-The external test cases are inherited from other OPNFV projects, especially the
-feature projects.
-
-The external test cases are:
-
- * barometer
- * bgpvpn
- * doctor
- * domino
- * fds
- * promise
- * refstack_defcore
- * functest-odl-sfc
- * orchestra_clearwaterims
- * orchestra_openims
- * vyos_vrouter
- * juju_vepc
-
-External test cases integrated in previous versions but not released in
-Euphrates:
-
- * copper
- * moon
- * netready
- * security_scan
-
-
-The code to run these test cases is hosted in the repository of the project.
-Please note that orchestra test cases are hosted in Functest repository and not
-in orchestra repository. Vyos_vrouter and juju_vepc code is also hosted in
-functest as there are no dedicated projects.
-
-
Functest framework
==================
diff --git a/docs/testing/user/configguide/ci.rst b/docs/testing/user/configguide/ci.rst
index aee51a835..ee815dbe1 100644
--- a/docs/testing/user/configguide/ci.rst
+++ b/docs/testing/user/configguide/ci.rst
@@ -7,8 +7,8 @@ container from Jenkins.
4 steps have been defined::
* functest-cleanup: clean existing functest dockers on the jumphost
- * functest-daily: run dockers opnfv/functest-* (healthcheck, smoke, features,
- vnf)
+ * functest-daily: run dockers opnfv/functest-* (healthcheck, smoke,
+ benchmarking, vnf)
* functest-store-results: push logs to artifacts
See `[1]`_ for details.
diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst
index 95aecb2df..c50fd1f1f 100644
--- a/docs/testing/user/configguide/configguide.rst
+++ b/docs/testing/user/configguide/configguide.rst
@@ -17,7 +17,6 @@ Docker images are available on the dockerhub:
* opnfv/functest-healthcheck
* opnfv/functest-smoke
* opnfv/functest-benchmarking
- * opnfv/functest-features
* opnfv/functest-vnf
@@ -146,31 +145,6 @@ Results shall be displayed as follows::
Note: if the scenario does not support some tests, they are indicated as SKIP.
See User guide for details.
-Testing features suite
-^^^^^^^^^^^^^^^^^^^^^^
-
-Run features suite::
-
- sudo docker run --env-file env \
- -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \
- -v $(pwd)/images:/home/opnfv/functest/images \
- opnfv/functest-features
-
-Results shall be displayed as follows::
-
- +-----------------------------+-------------------+------------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +-----------------------------+-------------------+------------------+------------------+----------------+
- | doctor-notification | doctor | features | 00:00 | SKIP |
- | functest-odl-sfc | sfc | features | 00:00 | SKIP |
- | barometercollectd | barometer | features | 00:00 | SKIP |
- | vgpu | functest | features | 00:00 | SKIP |
- | stor4nfv_os | stor4nfv | features | 00:00 | SKIP |
- +-----------------------------+-------------------+------------------+------------------+----------------+
-
-Note: if the scenario does not support some tests, they are indicated as SKIP.
-See User guide for details.
-
Testing vnf suite
^^^^^^^^^^^^^^^^^
@@ -200,7 +174,6 @@ Docker images are available on the dockerhub:
* opnfv/functest-kubernetes-core
* opnfv/functest-kubernetest-healthcheck
* opnfv/functest-kubernetest-smoke
- * opnfv/functest-kubernetest-features
Preparing your environment
^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -246,24 +219,6 @@ Results shall be displayed as follows::
| k8s_conformance | functest | smoke | 57:14 | PASS |
+-------------------------+------------------+---------------+------------------+----------------+
-Testing features suite
-^^^^^^^^^^^^^^^^^^^^^^
-
-Run features suite::
-
- sudo docker run -it --env-file env \
- -v $(pwd)/config:/root/.kube/config \
- opnfv/functest-kubernetes-features
-
-Results shall be displayed as follows::
-
- +----------------------+------------------+------------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +----------------------+------------------+------------------+------------------+----------------+
- | stor4nfv_k8s | stor4nfv | stor4nfv | 00:00 | SKIP |
- | clover_k8s | clover | clover | 00:00 | SKIP |
- +----------------------+------------------+------------------+------------------+----------------+
-
Environment variables
=====================
diff --git a/docs/testing/user/userguide/test_details.rst b/docs/testing/user/userguide/test_details.rst
index 0315498ce..6061ce9ef 100644
--- a/docs/testing/user/userguide/test_details.rst
+++ b/docs/testing/user/userguide/test_details.rst
@@ -272,53 +272,6 @@ Note: the checks in OpenDaylight are based on the returned HTTP status
code returned by OpenDaylight.
-Features
---------
-
-Functest has been supporting several feature projects since Brahmaputra:
-
-
-+-----------------+---------+----------+--------+-----------+-----------+
-| Test | Brahma | Colorado | Danube | Euphrates | Fraser |
-+=================+=========+==========+========+===========+===========+
-| barometer | | | X | X | X |
-+-----------------+---------+----------+--------+-----------+-----------+
-| bgpvpn | | X | X | X | X |
-+-----------------+---------+----------+--------+-----------+-----------+
-| copper | | X | | | |
-+-----------------+---------+----------+--------+-----------+-----------+
-| doctor | X | X | X | X | X |
-+-----------------+---------+----------+--------+-----------+-----------+
-| domino | | X | X | X | |
-+-----------------+---------+----------+--------+-----------+-----------+
-| fds | | | X | X | X |
-+-----------------+---------+----------+--------+-----------+-----------+
-| moon | | X | | | |
-+-----------------+---------+----------+--------+-----------+-----------+
-| multisite | | X | X | | |
-+-----------------+---------+----------+--------+-----------+-----------+
-| netready | | | X | | |
-+-----------------+---------+----------+--------+-----------+-----------+
-| odl_sfc | | X | X | X | X |
-+-----------------+---------+----------+--------+-----------+-----------+
-| opera | | | X | | |
-+-----------------+---------+----------+--------+-----------+-----------+
-| orchestra | | | X | X | X |
-+-----------------+---------+----------+--------+-----------+-----------+
-| parser | | | X | X | X |
-+-----------------+---------+----------+--------+-----------+-----------+
-| promise | X | X | X | X | X |
-+-----------------+---------+----------+--------+-----------+-----------+
-| security_scan | | X | X | | |
-+-----------------+---------+----------+--------+-----------+-----------+
-| clover | | | | | X |
-+-----------------+---------+----------+--------+-----------+-----------+
-| stor4nfv | | | | | X |
-+-----------------+---------+----------+--------+-----------+-----------+
-
-Please refer to the dedicated feature user guides for details.
-
-
VNF
---
diff --git a/docs/testing/user/userguide/test_overview.rst b/docs/testing/user/userguide/test_overview.rst
index e18e32748..5aef27c1c 100644
--- a/docs/testing/user/userguide/test_overview.rst
+++ b/docs/testing/user/userguide/test_overview.rst
@@ -11,12 +11,11 @@ infrastructure.
The current list of test suites can be distributed over 5 main domains:
* VIM (Virtualised Infrastructure Manager)
* Controllers (i.e. SDN Controllers)
- * Features
* VNF (Virtual Network Functions)
* Kubernetes
Functest test suites are also distributed in the OPNFV testing categories:
-healthcheck, smoke, features, components, performance, VNF, Stress tests.
+healthcheck, smoke, benchmarking, VNF, Stress tests.
All the Healthcheck and smoke tests of a given scenario must be succesful to
validate the scenario for the release.
@@ -88,44 +87,6 @@ validate the scenario for the release.
| | | | upstream testcases. See below |
| | | | for details |
+-------------+---------------+------------+----------------------------------+
-| Features | features | bgpvpn | Implementation of the OpenStack |
-| | | | bgpvpn API from the SDNVPN |
-| | | | feature project. It allows for |
-| | | | the creation of BGP VPNs. |
-| | | | See `SDNVPN User Guide`_ for |
-| | | | details |
-| | +------------+----------------------------------+
-| | | doctor | Doctor platform, as of Colorado |
-| | | | release, provides the three |
-| | | | features: |
-| | | | * Immediate Notification |
-| | | | * Consistent resource state |
-| | | | awareness for compute host down |
-| | | | * Valid compute host status |
-| | | | given to VM owner |
-| | | | See `Doctor User Guide`_ for |
-| | | | details |
-| | +------------+----------------------------------+
-| | | odl-sfc | SFC testing for odl scenarios |
-| | | | See `SFC User Guide`_ for details|
-| | +------------+----------------------------------+
-| | | parser | Parser is an integration project |
-| | | | which aims to provide |
-| | | | placement/deployment templates |
-| | | | translation for OPNFV platform, |
-| | | | including TOSCA -> HOT, POLICY ->|
-| | | | TOSCA and YANG -> TOSCA. it |
-| | | | deals with a fake vRNC. |
-| | | | See `Parser User Guide`_ for |
-| | | | details |
-| | +------------+----------------------------------+
-| | | fds | Test Suite for the OpenDaylight |
-| | | | SDN Controller when the GBP |
-| | | | features are installed. It |
-| | | | integrates some test suites from |
-| | | | upstream using Robot as the test |
-| | | | framework |
-+-------------+---------------+------------+----------------------------------+
| VNF | vnf | cloudify | Example of a real VNF deployment |
| | | \_ims | to show the NFV capabilities of |
| | | | the platform. The IP Multimedia |
@@ -154,13 +115,6 @@ validate the scenario for the release.
| | smoke | k8s\_ | Run a subset of Kubernetes |
| | | conformance| End-to-End tests, expected to |
| | | | pass on any Kubernetes cluster |
-| +---------------+------------+----------------------------------+
-| | stor4nfv | stor4nfv | Run tests necessary to |
-| | | \_k8s | demonstrate conformance of the |
-| | | | K8s+Stor4NFV deployment |
-| +---------------+------------+----------------------------------+
-| | clover | clover_k8s | Test functionality of |
-| | | | K8s+Istio+Clover deployment. |
+-------------+---------------+------------+----------------------------------+
@@ -219,9 +173,8 @@ combinations (which may change from one version to another):
Most of the tests are runnable by any combination, but some tests might have
restrictions imposed by the utilized installers or due to the available
-deployed features. The system uses the environment variables (INSTALLER_TYPE
-and DEPLOY_SCENARIO) to automatically determine the valid test cases, for each
-given environment.
+deployed services. The system uses the environment variables to automatically
+determine the valid test cases, for each given environment.
A convenience Functest CLI utility is also available to simplify setting up the
Functest evironment, management of the OpenStack environment (e.g. resource
@@ -234,8 +187,4 @@ section.
.. _`[2]`: https://docs.openstack.org/tempest/latest/
.. _`[3]`: https://rally.readthedocs.io/en/latest/index.html
-.. _`Doctor User Guide`: http://artifacts.opnfv.org/doctor/colorado/userguide/index.html
-.. _`SDNVPN User Guide`: http://artifacts.opnfv.org/sdnvpn/colorado/docs/userguide/index.html
-.. _`Parser User Guide`: http://artifacts.opnfv.org/parser/colorado/docs/userguide/index.html
.. _`Functest Dashboard`: http://testresults.opnfv.org/
-.. _`SFC User Guide`: http://artifacts.opnfv.org/sfc/colorado/userguide/index.html
diff --git a/docs/testing/user/userguide/test_results.rst b/docs/testing/user/userguide/test_results.rst
index be3765376..8023c4661 100644
--- a/docs/testing/user/userguide/test_results.rst
+++ b/docs/testing/user/userguide/test_results.rst
@@ -45,18 +45,6 @@ Smoke suite::
| neutron_trunk | functest | smoke | 00:00 | SKIP |
+------------------------------+------------------+---------------+------------------+----------------+
-Features suite::
-
- +-----------------------------+------------------------+------------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +-----------------------------+------------------------+------------------+------------------+----------------+
- | doctor-notification | doctor | features | 00:00 | SKIP |
- | bgpvpn | sdnvpn | features | 00:00 | SKIP |
- | functest-odl-sfc | sfc | features | 00:00 | SKIP |
- | barometercollectd | barometer | features | 00:00 | SKIP |
- | fds | fastdatastacks | features | 00:00 | SKIP |
- +-----------------------------+------------------------+------------------+------------------+----------------+
-
Components suite::
+-------------------------------+------------------+--------------------+------------------+----------------+
@@ -76,14 +64,6 @@ Vnf suite::
| juju_epc | functest | vnf | 46:44 | PASS |
+----------------------+------------------+--------------+------------------+----------------+
-Parser testcase::
-
- +-----------------------+-----------------+------------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +-----------------------+-----------------+------------------+------------------+----------------+
- | parser-basics | parser | features | 00:00 | SKIP |
- +-----------------------+-----------------+------------------+------------------+----------------+
-
Functest Kubernetes test result::
+--------------------------------------+------------------------------------------------------------+
@@ -111,15 +91,6 @@ Kubernetes smoke suite::
| k8s_conformance | functest | smoke | 57:47 | PASS |
+-------------------------+------------------+---------------+------------------+----------------+
-Kubernetes features suite::
-
- +----------------------+------------------+------------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +----------------------+------------------+------------------+------------------+----------------+
- | stor4nfv_k8s | stor4nfv | stor4nfv | 00:00 | SKIP |
- | clover_k8s | clover | clover | 00:00 | SKIP |
- +----------------------+------------------+------------------+------------------+----------------+
-
Results are automatically pushed to the test results database, some additional
result files are pushed to OPNFV artifact web sites.