diff options
Diffstat (limited to 'docs/testing/user/userguide/test_overview.rst')
| -rw-r--r-- | docs/testing/user/userguide/test_overview.rst | 81 |
1 files changed, 10 insertions, 71 deletions
diff --git a/docs/testing/user/userguide/test_overview.rst b/docs/testing/user/userguide/test_overview.rst index 2ff14f732..bc3e79dcb 100644 --- a/docs/testing/user/userguide/test_overview.rst +++ b/docs/testing/user/userguide/test_overview.rst @@ -11,27 +11,19 @@ infrastructure. The current list of test suites can be distributed over 5 main domains: * VIM (Virtualised Infrastructure Manager) * Controllers (i.e. SDN Controllers) - * Features * VNF (Virtual Network Functions) * Kubernetes Functest test suites are also distributed in the OPNFV testing categories: -healthcheck, smoke, features, components, performance, VNF, Stress tests. +healthcheck, smoke, benchmarking, VNF, Stress tests. -All the Healthcheck and smoke tests of a given scenario must be succesful to +All the Healthcheck and smoke tests of a given scenario must be successful to validate the scenario for the release. +-------------+---------------+------------+----------------------------------+ | Domain | Tier | Test case | Comments | +=============+===============+============+==================================+ | VIM | healthcheck | connection | Check OpenStack connectivity | -| | | \_check | through SNAPS framework | -| | +------------+----------------------------------+ -| | | api_check | Check OpenStack API through | -| | | | SNAPS framework | -| | +------------+----------------------------------+ -| | | snaps | basic instance creation, check | -| | | \_health | DHCP | | | | \_check | | | +---------------+------------+----------------------------------+ | | smoke | vping_ssh | NFV "Hello World" using an SSH | @@ -62,9 +54,6 @@ validate the scenario for the release. | | | rally | Run a subset of the OpenStack | | | | \_sanity | Rally Test Suite in smoke mode | | | +------------+----------------------------------+ -| | | snaps\ | Run the SNAPS-OO integration | -| | | \_smoke | tests | -| | +------------+----------------------------------+ | | | refstack | Reference RefStack suite | | | | \_defcore | tempest selection for NFV | | | +------------+----------------------------------+ @@ -98,48 +87,10 @@ validate the scenario for the release. | | | | upstream testcases. See below | | | | | for details | +-------------+---------------+------------+----------------------------------+ -| Features | features | bgpvpn | Implementation of the OpenStack | -| | | | bgpvpn API from the SDNVPN | -| | | | feature project. It allows for | -| | | | the creation of BGP VPNs. | -| | | | See `SDNVPN User Guide`_ for | -| | | | details | -| | +------------+----------------------------------+ -| | | doctor | Doctor platform, as of Colorado | -| | | | release, provides the three | -| | | | features: | -| | | | * Immediate Notification | -| | | | * Consistent resource state | -| | | | awareness for compute host down | -| | | | * Valid compute host status | -| | | | given to VM owner | -| | | | See `Doctor User Guide`_ for | -| | | | details | -| | +------------+----------------------------------+ -| | | odl-sfc | SFC testing for odl scenarios | -| | | | See `SFC User Guide`_ for details| -| | +------------+----------------------------------+ -| | | parser | Parser is an integration project | -| | | | which aims to provide | -| | | | placement/deployment templates | -| | | | translation for OPNFV platform, | -| | | | including TOSCA -> HOT, POLICY ->| -| | | | TOSCA and YANG -> TOSCA. it | -| | | | deals with a fake vRNC. | -| | | | See `Parser User Guide`_ for | -| | | | details | -| | +------------+----------------------------------+ -| | | fds | Test Suite for the OpenDaylight | -| | | | SDN Controller when the GBP | -| | | | features are installed. It | -| | | | integrates some test suites from | -| | | | upstream using Robot as the test | -| | | | framework | -+-------------+---------------+------------+----------------------------------+ | VNF | vnf | cloudify | Example of a real VNF deployment | | | | \_ims | to show the NFV capabilities of | | | | | the platform. The IP Multimedia | -| | | | Subsytem is a typical Telco test | +| | | | Subsystem is a typical Telco test| | | | | case, referenced by ETSI. | | | | | It provides a fully functional | | | | | VoIP System | @@ -164,13 +115,6 @@ validate the scenario for the release. | | smoke | k8s\_ | Run a subset of Kubernetes | | | | conformance| End-to-End tests, expected to | | | | | pass on any Kubernetes cluster | -| +---------------+------------+----------------------------------+ -| | stor4nfv | stor4nfv | Run tests necessary to | -| | | \_k8s | demonstrate conformance of the | -| | | | K8s+Stor4NFV deployment | -| +---------------+------------+----------------------------------+ -| | clover | clover_k8s | Test functionality of | -| | | | K8s+Istio+Clover deployment. | +-------------+---------------+------------+----------------------------------+ @@ -193,7 +137,7 @@ An overview of the Functest Structural Concept is depicted graphically below: Some of the test cases are developed by Functest team members, whereas others are integrated from upstream communities or other OPNFV projects. For example, -`Tempest <http://docs.openstack.org/developer/tempest/overview.html>`_ is the +`Tempest <https://docs.openstack.org/tempest/latest/>`_ is the OpenStack integration test suite and Functest is in charge of the selection, integration and automation of those tests that fit suitably to OPNFV. @@ -229,12 +173,11 @@ combinations (which may change from one version to another): Most of the tests are runnable by any combination, but some tests might have restrictions imposed by the utilized installers or due to the available -deployed features. The system uses the environment variables (INSTALLER_TYPE -and DEPLOY_SCENARIO) to automatically determine the valid test cases, for each -given environment. +deployed services. The system uses the environment variables to automatically +determine the valid test cases, for each given environment. A convenience Functest CLI utility is also available to simplify setting up the -Functest evironment, management of the OpenStack environment (e.g. resource +Functest environment, management of the OpenStack environment (e.g. resource clean-up) and for executing tests. The Functest CLI organised the testcase into logical Tiers, which contain in turn one or more testcases. The CLI allows execution of a single specified @@ -242,10 +185,6 @@ testcase, all test cases in a specified Tier, or the special case of execution of **ALL** testcases. The Functest CLI is introduced in more details in next section. -.. _`[2]`: http://docs.openstack.org/developer/tempest/overview.html -.. _`[3]`: https://rally.readthedocs.org/en/latest/index.html -.. _`Doctor User Guide`: http://artifacts.opnfv.org/doctor/colorado/userguide/index.html -.. _`SDNVPN User Guide`: http://artifacts.opnfv.org/sdnvpn/colorado/docs/userguide/index.html -.. _`Parser User Guide`: http://artifacts.opnfv.org/parser/colorado/docs/userguide/index.html -.. _`Functest Dashboard`: http://testresults.opnfv.org/kibana_dashboards/ -.. _`SFC User Guide`: http://artifacts.opnfv.org/sfc/colorado/userguide/index.html +.. _`[2]`: https://docs.openstack.org/tempest/latest/ +.. _`[3]`: https://rally.readthedocs.io/en/latest/index.html +.. _`Functest Dashboard`: http://testresults.opnfv.org/ |