diff options
Diffstat (limited to 'docs/testing')
| -rw-r--r-- | docs/testing/developer/devguide/index.rst | 36 | ||||
| -rw-r--r-- | docs/testing/user/configguide/ci.rst | 4 | ||||
| -rw-r--r-- | docs/testing/user/configguide/configguide.rst | 45 | ||||
| -rw-r--r-- | docs/testing/user/userguide/test_details.rst | 47 | ||||
| -rw-r--r-- | docs/testing/user/userguide/test_overview.rst | 57 | ||||
| -rw-r--r-- | docs/testing/user/userguide/test_results.rst | 29 |
6 files changed, 5 insertions, 213 deletions
diff --git a/docs/testing/developer/devguide/index.rst b/docs/testing/developer/devguide/index.rst index 4091a39f9..a0faf42c1 100644 --- a/docs/testing/developer/devguide/index.rst +++ b/docs/testing/developer/devguide/index.rst @@ -52,7 +52,6 @@ to the different tiers: * functest-core: https://hub.docker.com/r/opnfv/functest-core/ * functest-healthcheck: https://hub.docker.com/r/opnfv/functest-healthcheck/ * functest-smoke: https://hub.docker.com/r/opnfv/functest-smoke/ - * functest-features: https://hub.docker.com/r/opnfv/functest-features/ * functest-vnf: https://hub.docker.com/r/opnfv/functest-vnf/ * functest-restapi: https://hub.docker.com/r/opnfv/functest-restapi/ @@ -108,41 +107,6 @@ repository, the internal test cases can be grouped by domain: If you want to create a new test case you will have to create a new folder under the testcases directory (See next section for details). -Functest external test cases -============================ -The external test cases are inherited from other OPNFV projects, especially the -feature projects. - -The external test cases are: - - * barometer - * bgpvpn - * doctor - * domino - * fds - * promise - * refstack_defcore - * functest-odl-sfc - * orchestra_clearwaterims - * orchestra_openims - * vyos_vrouter - * juju_vepc - -External test cases integrated in previous versions but not released in -Euphrates: - - * copper - * moon - * netready - * security_scan - - -The code to run these test cases is hosted in the repository of the project. -Please note that orchestra test cases are hosted in Functest repository and not -in orchestra repository. Vyos_vrouter and juju_vepc code is also hosted in -functest as there are no dedicated projects. - - Functest framework ================== diff --git a/docs/testing/user/configguide/ci.rst b/docs/testing/user/configguide/ci.rst index aee51a835..ee815dbe1 100644 --- a/docs/testing/user/configguide/ci.rst +++ b/docs/testing/user/configguide/ci.rst @@ -7,8 +7,8 @@ container from Jenkins. 4 steps have been defined:: * functest-cleanup: clean existing functest dockers on the jumphost - * functest-daily: run dockers opnfv/functest-* (healthcheck, smoke, features, - vnf) + * functest-daily: run dockers opnfv/functest-* (healthcheck, smoke, + benchmarking, vnf) * functest-store-results: push logs to artifacts See `[1]`_ for details. diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst index 95aecb2df..c50fd1f1f 100644 --- a/docs/testing/user/configguide/configguide.rst +++ b/docs/testing/user/configguide/configguide.rst @@ -17,7 +17,6 @@ Docker images are available on the dockerhub: * opnfv/functest-healthcheck * opnfv/functest-smoke * opnfv/functest-benchmarking - * opnfv/functest-features * opnfv/functest-vnf @@ -146,31 +145,6 @@ Results shall be displayed as follows:: Note: if the scenario does not support some tests, they are indicated as SKIP. See User guide for details. -Testing features suite -^^^^^^^^^^^^^^^^^^^^^^ - -Run features suite:: - - sudo docker run --env-file env \ - -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/env_file \ - -v $(pwd)/images:/home/opnfv/functest/images \ - opnfv/functest-features - -Results shall be displayed as follows:: - - +-----------------------------+-------------------+------------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +-----------------------------+-------------------+------------------+------------------+----------------+ - | doctor-notification | doctor | features | 00:00 | SKIP | - | functest-odl-sfc | sfc | features | 00:00 | SKIP | - | barometercollectd | barometer | features | 00:00 | SKIP | - | vgpu | functest | features | 00:00 | SKIP | - | stor4nfv_os | stor4nfv | features | 00:00 | SKIP | - +-----------------------------+-------------------+------------------+------------------+----------------+ - -Note: if the scenario does not support some tests, they are indicated as SKIP. -See User guide for details. - Testing vnf suite ^^^^^^^^^^^^^^^^^ @@ -200,7 +174,6 @@ Docker images are available on the dockerhub: * opnfv/functest-kubernetes-core * opnfv/functest-kubernetest-healthcheck * opnfv/functest-kubernetest-smoke - * opnfv/functest-kubernetest-features Preparing your environment ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -246,24 +219,6 @@ Results shall be displayed as follows:: | k8s_conformance | functest | smoke | 57:14 | PASS | +-------------------------+------------------+---------------+------------------+----------------+ -Testing features suite -^^^^^^^^^^^^^^^^^^^^^^ - -Run features suite:: - - sudo docker run -it --env-file env \ - -v $(pwd)/config:/root/.kube/config \ - opnfv/functest-kubernetes-features - -Results shall be displayed as follows:: - - +----------------------+------------------+------------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +----------------------+------------------+------------------+------------------+----------------+ - | stor4nfv_k8s | stor4nfv | stor4nfv | 00:00 | SKIP | - | clover_k8s | clover | clover | 00:00 | SKIP | - +----------------------+------------------+------------------+------------------+----------------+ - Environment variables ===================== diff --git a/docs/testing/user/userguide/test_details.rst b/docs/testing/user/userguide/test_details.rst index 0315498ce..6061ce9ef 100644 --- a/docs/testing/user/userguide/test_details.rst +++ b/docs/testing/user/userguide/test_details.rst @@ -272,53 +272,6 @@ Note: the checks in OpenDaylight are based on the returned HTTP status code returned by OpenDaylight. -Features --------- - -Functest has been supporting several feature projects since Brahmaputra: - - -+-----------------+---------+----------+--------+-----------+-----------+ -| Test | Brahma | Colorado | Danube | Euphrates | Fraser | -+=================+=========+==========+========+===========+===========+ -| barometer | | | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| bgpvpn | | X | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| copper | | X | | | | -+-----------------+---------+----------+--------+-----------+-----------+ -| doctor | X | X | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| domino | | X | X | X | | -+-----------------+---------+----------+--------+-----------+-----------+ -| fds | | | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| moon | | X | | | | -+-----------------+---------+----------+--------+-----------+-----------+ -| multisite | | X | X | | | -+-----------------+---------+----------+--------+-----------+-----------+ -| netready | | | X | | | -+-----------------+---------+----------+--------+-----------+-----------+ -| odl_sfc | | X | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| opera | | | X | | | -+-----------------+---------+----------+--------+-----------+-----------+ -| orchestra | | | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| parser | | | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| promise | X | X | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| security_scan | | X | X | | | -+-----------------+---------+----------+--------+-----------+-----------+ -| clover | | | | | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| stor4nfv | | | | | X | -+-----------------+---------+----------+--------+-----------+-----------+ - -Please refer to the dedicated feature user guides for details. - - VNF --- diff --git a/docs/testing/user/userguide/test_overview.rst b/docs/testing/user/userguide/test_overview.rst index e18e32748..5aef27c1c 100644 --- a/docs/testing/user/userguide/test_overview.rst +++ b/docs/testing/user/userguide/test_overview.rst @@ -11,12 +11,11 @@ infrastructure. The current list of test suites can be distributed over 5 main domains: * VIM (Virtualised Infrastructure Manager) * Controllers (i.e. SDN Controllers) - * Features * VNF (Virtual Network Functions) * Kubernetes Functest test suites are also distributed in the OPNFV testing categories: -healthcheck, smoke, features, components, performance, VNF, Stress tests. +healthcheck, smoke, benchmarking, VNF, Stress tests. All the Healthcheck and smoke tests of a given scenario must be succesful to validate the scenario for the release. @@ -88,44 +87,6 @@ validate the scenario for the release. | | | | upstream testcases. See below | | | | | for details | +-------------+---------------+------------+----------------------------------+ -| Features | features | bgpvpn | Implementation of the OpenStack | -| | | | bgpvpn API from the SDNVPN | -| | | | feature project. It allows for | -| | | | the creation of BGP VPNs. | -| | | | See `SDNVPN User Guide`_ for | -| | | | details | -| | +------------+----------------------------------+ -| | | doctor | Doctor platform, as of Colorado | -| | | | release, provides the three | -| | | | features: | -| | | | * Immediate Notification | -| | | | * Consistent resource state | -| | | | awareness for compute host down | -| | | | * Valid compute host status | -| | | | given to VM owner | -| | | | See `Doctor User Guide`_ for | -| | | | details | -| | +------------+----------------------------------+ -| | | odl-sfc | SFC testing for odl scenarios | -| | | | See `SFC User Guide`_ for details| -| | +------------+----------------------------------+ -| | | parser | Parser is an integration project | -| | | | which aims to provide | -| | | | placement/deployment templates | -| | | | translation for OPNFV platform, | -| | | | including TOSCA -> HOT, POLICY ->| -| | | | TOSCA and YANG -> TOSCA. it | -| | | | deals with a fake vRNC. | -| | | | See `Parser User Guide`_ for | -| | | | details | -| | +------------+----------------------------------+ -| | | fds | Test Suite for the OpenDaylight | -| | | | SDN Controller when the GBP | -| | | | features are installed. It | -| | | | integrates some test suites from | -| | | | upstream using Robot as the test | -| | | | framework | -+-------------+---------------+------------+----------------------------------+ | VNF | vnf | cloudify | Example of a real VNF deployment | | | | \_ims | to show the NFV capabilities of | | | | | the platform. The IP Multimedia | @@ -154,13 +115,6 @@ validate the scenario for the release. | | smoke | k8s\_ | Run a subset of Kubernetes | | | | conformance| End-to-End tests, expected to | | | | | pass on any Kubernetes cluster | -| +---------------+------------+----------------------------------+ -| | stor4nfv | stor4nfv | Run tests necessary to | -| | | \_k8s | demonstrate conformance of the | -| | | | K8s+Stor4NFV deployment | -| +---------------+------------+----------------------------------+ -| | clover | clover_k8s | Test functionality of | -| | | | K8s+Istio+Clover deployment. | +-------------+---------------+------------+----------------------------------+ @@ -219,9 +173,8 @@ combinations (which may change from one version to another): Most of the tests are runnable by any combination, but some tests might have restrictions imposed by the utilized installers or due to the available -deployed features. The system uses the environment variables (INSTALLER_TYPE -and DEPLOY_SCENARIO) to automatically determine the valid test cases, for each -given environment. +deployed services. The system uses the environment variables to automatically +determine the valid test cases, for each given environment. A convenience Functest CLI utility is also available to simplify setting up the Functest evironment, management of the OpenStack environment (e.g. resource @@ -234,8 +187,4 @@ section. .. _`[2]`: https://docs.openstack.org/tempest/latest/ .. _`[3]`: https://rally.readthedocs.io/en/latest/index.html -.. _`Doctor User Guide`: http://artifacts.opnfv.org/doctor/colorado/userguide/index.html -.. _`SDNVPN User Guide`: http://artifacts.opnfv.org/sdnvpn/colorado/docs/userguide/index.html -.. _`Parser User Guide`: http://artifacts.opnfv.org/parser/colorado/docs/userguide/index.html .. _`Functest Dashboard`: http://testresults.opnfv.org/ -.. _`SFC User Guide`: http://artifacts.opnfv.org/sfc/colorado/userguide/index.html diff --git a/docs/testing/user/userguide/test_results.rst b/docs/testing/user/userguide/test_results.rst index be3765376..8023c4661 100644 --- a/docs/testing/user/userguide/test_results.rst +++ b/docs/testing/user/userguide/test_results.rst @@ -45,18 +45,6 @@ Smoke suite:: | neutron_trunk | functest | smoke | 00:00 | SKIP | +------------------------------+------------------+---------------+------------------+----------------+ -Features suite:: - - +-----------------------------+------------------------+------------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +-----------------------------+------------------------+------------------+------------------+----------------+ - | doctor-notification | doctor | features | 00:00 | SKIP | - | bgpvpn | sdnvpn | features | 00:00 | SKIP | - | functest-odl-sfc | sfc | features | 00:00 | SKIP | - | barometercollectd | barometer | features | 00:00 | SKIP | - | fds | fastdatastacks | features | 00:00 | SKIP | - +-----------------------------+------------------------+------------------+------------------+----------------+ - Components suite:: +-------------------------------+------------------+--------------------+------------------+----------------+ @@ -76,14 +64,6 @@ Vnf suite:: | juju_epc | functest | vnf | 46:44 | PASS | +----------------------+------------------+--------------+------------------+----------------+ -Parser testcase:: - - +-----------------------+-----------------+------------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +-----------------------+-----------------+------------------+------------------+----------------+ - | parser-basics | parser | features | 00:00 | SKIP | - +-----------------------+-----------------+------------------+------------------+----------------+ - Functest Kubernetes test result:: +--------------------------------------+------------------------------------------------------------+ @@ -111,15 +91,6 @@ Kubernetes smoke suite:: | k8s_conformance | functest | smoke | 57:47 | PASS | +-------------------------+------------------+---------------+------------------+----------------+ -Kubernetes features suite:: - - +----------------------+------------------+------------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +----------------------+------------------+------------------+------------------+----------------+ - | stor4nfv_k8s | stor4nfv | stor4nfv | 00:00 | SKIP | - | clover_k8s | clover | clover | 00:00 | SKIP | - +----------------------+------------------+------------------+------------------+----------------+ - Results are automatically pushed to the test results database, some additional result files are pushed to OPNFV artifact web sites. |