diff options
Diffstat (limited to 'docs/testing/user/userguide/test_details.rst')
-rw-r--r-- | docs/testing/user/userguide/test_details.rst | 175 |
1 files changed, 62 insertions, 113 deletions
diff --git a/docs/testing/user/userguide/test_details.rst b/docs/testing/user/userguide/test_details.rst index 543f66a19..98247d488 100644 --- a/docs/testing/user/userguide/test_details.rst +++ b/docs/testing/user/userguide/test_details.rst @@ -24,20 +24,10 @@ The tests are: * *connection_check* - * *api_check* - * *snaps_health_check* -Connection_check consists in 9 test cases (test duration < 5s) checking the +Connection_check consists in test cases (test duration < 5s) checking the connectivity with Glance, Keystone, Neutron, Nova and the external network. -Api_check verifies the retrieval of OpenStack clients: Keystone, Glance, -Neutron and Nova and may perform some simple queries. When the config value of -snaps.use_keystone is True, functest must have access to the cloud's private -network. This suite consists in 49 tests (test duration < 2 minutes). - -Snaps_health_check creates a VM with a single port with an IPv4 address that -is assigned by DHCP and then validates the expected IP with the actual. - Self-obviously, successful completion of the 'healthcheck' testcase is a necessary pre-requisite for the execution of all other test Tiers. @@ -161,17 +151,14 @@ console output is stored to a *log* file for further analysis. The Tempest testcases are distributed across three Tiers: - * Smoke Tier - Test Case 'tempest_smoke_serial' - * Components Tier - Test case 'tempest_full_parallel' + * Smoke Tier - Test Case 'tempest_smoke' + * Components Tier - Test case 'tempest_full' * Neutron Trunk Port - Test case 'neutron_trunk' * OpenStack interop testcases - Test case 'refstack_defcore' * Testing and verifying RBAC policy enforcement - Test case 'patrole' -NOTE: Test case 'tempest_smoke_serial' executes a defined set of tempest smoke -tests with a single thread (i.e. serial mode). Test case -'tempest_full_parallel' executes all defined Tempest tests using several -concurrent threads (i.e. parallel mode). The number of threads activated -corresponds to the number of available logical CPUs. +NOTE: Test case 'tempest_smoke' executes a defined set of tempest smoke +tests. Test case 'tempest_full' executes all defined Tempest tests. NOTE: The 'neutron_trunk' test set allows to connect a VM to multiple VLAN separated networks using a single NIC. The feature neutron trunk ports have @@ -226,17 +213,6 @@ NOTE: Test case 'rally_sanity' executes a limited number of Rally smoke test cases. Test case 'rally_full' executes the full defined set of Rally tests. -snaps_smoke ------------- - -This test case contains tests that setup and destroy environments with VMs with -and without Floating IPs with a newly created user and project. Set the config -value snaps.use_floating_ips (True|False) to toggle this functionality. -Please note that When the configuration value of snaps.use_keystone is True, -Functest must have access the cloud's private network. -This suite consists in 120 tests (test duration ~= 50 minutes) - - SDN Controllers --------------- @@ -296,53 +272,6 @@ Note: the checks in OpenDaylight are based on the returned HTTP status code returned by OpenDaylight. -Features --------- - -Functest has been supporting several feature projects since Brahmaputra: - - -+-----------------+---------+----------+--------+-----------+-----------+ -| Test | Brahma | Colorado | Danube | Euphrates | Fraser | -+=================+=========+==========+========+===========+===========+ -| barometer | | | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| bgpvpn | | X | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| copper | | X | | | | -+-----------------+---------+----------+--------+-----------+-----------+ -| doctor | X | X | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| domino | | X | X | X | | -+-----------------+---------+----------+--------+-----------+-----------+ -| fds | | | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| moon | | X | | | | -+-----------------+---------+----------+--------+-----------+-----------+ -| multisite | | X | X | | | -+-----------------+---------+----------+--------+-----------+-----------+ -| netready | | | X | | | -+-----------------+---------+----------+--------+-----------+-----------+ -| odl_sfc | | X | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| opera | | | X | | | -+-----------------+---------+----------+--------+-----------+-----------+ -| orchestra | | | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| parser | | | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| promise | X | X | X | X | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| security_scan | | X | X | | | -+-----------------+---------+----------+--------+-----------+-----------+ -| clover | | | | | X | -+-----------------+---------+----------+--------+-----------+-----------+ -| stor4nfv | | | | | X | -+-----------------+---------+----------+--------+-----------+-----------+ - -Please refer to the dedicated feature user guides for details. - - VNF --- @@ -370,36 +299,27 @@ The Clearwater architecture is described as follows: :align: center :alt: vIMS architecture +heat_ims +^^^^^^^^ +The IP Multimedia Subsystem or IP Multimedia Core Network Subsystem (IMS) is an +architectural framework for delivering IP multimedia services. -cloudify_ims_perf -^^^^^^^^^^^^^^^^^ -This testcase extends the cloudify_ims test case. -The first part is similar but the testing part is different. -The testing part consists in automating a realistic signaling load on the vIMS -using an Ixia loader (proprietary tools) - - - You need to have access to an Ixia licence server defined in the - configuration file and have ixia image locally. - -This test case is available but not declared in testcases.yaml. The declaration -of the testcase is simple, connect to your functest-vnf docker, add the -following section in -/usr/lib/python2.7/site-packacges/functest/ci/testcases.yaml:: - - - - case_name: cloudify_ims_perf - project_name: functest - criteria: 80 - blocking: false - description: >- - Stress tests based on Cloudify. Ixia loader images and access to Ixia - server license. - dependencies: - installer: '' - scenario: 'os-nosdn-nofeature-ha' - run: - module: 'functest.opnfv_tests.vnf.ims.cloudify_ims_perf' - class: 'CloudifyImsPerf' +vIMS has been integrated in Functest to demonstrate the capability to deploy a +relatively complex NFV scenario on the OPNFV platform. The deployment of a +complete functional VNF allows the test of most of the essential functions +needed for a NFV platform. + +The goal of this test suite consists of: + +* deploy a Clearwater vIMS (IP Multimedia Subsystem) VNF using + OpenStack Heat orchestrator based on a HOT template defined in `[17]`_ +* run suite of signaling tests on top of this VNF + +The Clearwater architecture is described as follows: + +.. figure:: ../../../images/clearwater-architecture-v2.png + :align: center + :alt: vIMS architecture vyos-vrouter ^^^^^^^^^^^^ @@ -422,13 +342,41 @@ The vyos-vrouter architecture is described in `[14]`_ juju_epc ^^^^^^^^ +The Evolved Packet Core (EPC) is the main component of the System Architecture +Evolution (SAE) which forms the core of the 3GPP LTE specification. + +vEPC has been integrated in Functest to demonstrate the capability to deploy a +complex mobility-specific NFV scenario on the OPNFV platform. The OAI EPC +supports most of the essential functions defined by the 3GPP Technical Specs; +hence the successful execution of functional tests on the OAI EPC provides a +good endorsement of the underlying NFV platform. + +This integration also includes ABot, a Test Orchestration system that enables +test scenarios to be defined in high-level DSL. ABot is also deployed as a +VM on the OPNFV platform; and this provides an example of the automation +driver and the Test VNF being both deployed as separate VNFs on the underlying +OPNFV platform. + +The Workflow is as follows: + * Deploy Orchestrator + Deploy Juju controller using Bootstrap command. + * Deploy VNF + Deploy ABot orchestrator and OAI EPC as Juju charms. + Configuration of ABot and OAI EPC components is handled through + built-in Juju relations. + * Test VNF + Execution of ABot feature files triggered by Juju actions. + This executes a suite of LTE signalling tests on the OAI EPC. + * Reporting + ABot test results are parsed accordingly and pushed to Functest Db. +Details of the ABot test orchestration tool may be found in `[15]`_ Kubernetes (K8s) ---------------- Kubernetes testing relies on sets of tests, which are part of the Kubernetes -source tree, such as the Kubernetes End-to-End (e2e) tests `[15]`_. +source tree, such as the Kubernetes End-to-End (e2e) tests `[16]`_. The kubernetes testcases are distributed across various Tiers: @@ -450,14 +398,15 @@ The kubernetes testcases are distributed across various Tiers: designed to be run with no cloud provider configured. -.. _`[2]`: http://docs.openstack.org/developer/tempest/overview.html -.. _`[3]`: https://rally.readthedocs.org/en/latest/index.html +.. _`[2]`: https://docs.openstack.org/tempest/latest/ +.. _`[3]`: https://rally.readthedocs.io/en/latest/index.html .. _`[5]`: https://github.com/Orange-OpenSource/opnfv-cloudify-clearwater/blob/master/openstack-blueprint.yaml .. _`[8]`: https://github.com/openstack/refstack-client -.. _`[9]`: https://github.com/openstack/defcore +.. _`[9]`: https://github.com/openstack/interop .. _`[10]`: https://github.com/openstack/interop/blob/master/2016.08/procedure.rst -.. _`[11]`: http://robotframework.org/ -.. _`[12]`: http://docs.opnfv.org/en/latest/submodules/functest/docs/testing/user/userguide/index.html -.. _`[13]`: https://wiki.opnfv.org/display/PROJ/SNAPS-OO +.. _`[11]`: https://robotframework.org/ +.. _`[13]`: https://git.opnfv.org/snaps/ .. _`[14]`: https://github.com/oolorg/opnfv-functest-vrouter -.. _`[15]`: https://github.com/kubernetes/community/blob/master/contributors/devel/e2e-tests.md +.. _`[15]`: https://github.com/RebacaInc/abot_charm +.. _`[16]`: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md +.. _`[17]`: https://github.com/Metaswitch/clearwater-heat/blob/release-129/clearwater.yaml |