summaryrefslogtreecommitdiffstats
path: root/docs/testsuites
diff options
context:
space:
mode:
authorChristopherPrice <christopher.price@ericsson.com>2016-08-08 16:07:05 +0200
committerChristopherPrice <christopher.price@ericsson.com>2016-08-09 15:50:39 +0200
commite7316237cc9f5b469c53682af6f57099d42a3279 (patch)
tree3cc4559e5c2de6c2b83b3d2563130a9c11c34983 /docs/testsuites
parent5e04ff0326fc4f6ba23b5ac7abfe3fc04ace18f7 (diff)
Proposing a Dovetail test suite structure for IPv6
Adding the first draft of the IPv6 test suite. This patch attempts to start the documentation according to the proposal on https://wiki.opnfv.org/pages/viewpage.action?pageId=6827269 The text is neither complete nor accurate at this stage, this patch is intended to establish a concrete dialog for how we structure our test suites and reference test cases. Change-Id: I156ddafa3e8b7630390523864490426fc742eb3a Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>
Diffstat (limited to 'docs/testsuites')
-rw-r--r--docs/testsuites/ipv6/designspecification.rst133
-rw-r--r--docs/testsuites/ipv6/index.rst18
-rw-r--r--docs/testsuites/ipv6/testplan.rst60
-rw-r--r--docs/testsuites/ipv6/testprocedure.rst9
-rw-r--r--docs/testsuites/ipv6/testspecification.rst54
5 files changed, 274 insertions, 0 deletions
diff --git a/docs/testsuites/ipv6/designspecification.rst b/docs/testsuites/ipv6/designspecification.rst
new file mode 100644
index 00000000..9e403472
--- /dev/null
+++ b/docs/testsuites/ipv6/designspecification.rst
@@ -0,0 +1,133 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Christopher Price (Ericsson AB) and others
+
+==============================
+IPv6 test design specification
+==============================
+
+This document outlines the approach and method for testing IPv6 in the OPNFV compliance test
+suite. Providing a brief outline of the features to be tested, the methodology for testing,
+schema's and criteria.
+
+Features to be tested
+=====================
+
+The IPv6 compliance test plan outlines the method for testing IPv6 compliance to the OPNFV
+platform behaviours and features of IPv6 enabled VNFi platforms. The specific features to
+be tested by the IPv6 compliance test suite is outlined in the following table.
+
+.. table::
+ :class: longtable
+
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Features / Requirements |Tests available | Test Cases |
++===========================================================+===================+====================================================================+
+|All topologies work in a multi-tenant environment |No | |
+| | | |
+| | | |
+| | | |
+| | | |
+| | | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|IPv6 VM to VM only |No | |
+| | | |
+| | | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|IPv6 external L2 VLAN directly attached to a VM |No | |
+| | | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|IPv6 subnet routed via L3 agent to an external IPv6 network|No | |
+| | | |
+|1. Both VLAN and overlay (e.g. GRE, VXLAN) subnet attached | | |
+| to VMs; | | |
+|2. Must be able to support multiple L3 agents for a given | | |
+| external network to support scaling (neutron scheduler | | |
+| to assign vRouters to the L3 agents) | | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Ability for a NIC to support both IPv4 and IPv6 (dual |No | |
+|stack) address. | | |
+| | | |
+|1. VM with a single interface associated with a network, | | |
+| which is then associated with two subnets. | | |
+|2. VM with two different interfaces associated with two | | |
+| different networks and two different subnets. | | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Support IPv6 Address assignment modes. |No | |
+| | | |
+|1. SLAAC | | |
+|2. DHCPv6 Stateless | | |
+|3. DHCPv6 Stateful | | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Ability to create a port on an IPv6 DHCPv6 Stateful subnet |No | |
+|and assign a specific IPv6 address to the port and have it | | |
+|taken out of the DHCP address pool. | | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Full support for IPv6 matching (i.e., IPv6, ICMPv6, TCP, |No | |
+|UDP) in security groups. Ability to control and manage all | | |
+|IPv6 security group capabilities via Neutron/Nova API (REST| | |
+|and CLI) as well as via Horizon. | | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|During network/subnet/router create, there should be an |No | |
+|option to allow user to specify the type of address | | |
+|management they would like. This includes all options | | |
+|including those low priority if implemented (e.g., toggle | | |
+|on/off router and address prefix advertisements); It must | | |
+|be supported via Neutron API (REST and CLI) as well as via | | |
+|Horizon | | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Security groups anti-spoofing: Prevent VM from using a |No | |
+|source IPv6/MAC address which is not assigned to the VM | | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Protect tenant and provider network from rogue RAs |No | |
+| | | |
+| | | |
+| | | |
+| | | |
+| | | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Support the ability to assign multiple IPv6 addresses to |No | |
+|an interface; both for Neutron router interfaces and VM | | |
+|interfaces. | | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Ability for a VM to support a mix of multiple IPv4 and IPv6|No | |
+|networks, including multiples of the same type. | | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Support for IPv6 Prefix Delegation. |No | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|IPv6 First-Hop Security, IPv6 ND spoofing |No | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|IPv6 support in Neutron Layer3 High Availability |No | |
+|(keepalived+VRRP). | | |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+
+
+Test approach for IPv6
+======================
+
+The most common approach for testing IPv6 capabilities in the test suite is through interaction with the SUT control plane.
+In this instance the test framework will exercise the NBI provided by the VIM to configure and leverage IPv6 related features
+in the platform, instantiate workloads, and invoke behaviours in the platform. The suite may also interact directly with the
+data plane to exercise platform capabilities and further invoke helper functions on the platform for the same purpose.
+
+Test result analysis
+--------------------
+
+All functional tests in the IPv6 test suite will provide a pass/fail result on completion of the test. In addition test logs
+and relevant additional information will be provided as part of the test log, available on test suite completion.
+
+Some tests in the compliance suite measure such metrics as latency and performance. At this time these tests are intended to
+provide a feature based pass/fail metric not related to system performance.
+These tests may however provide detailed results of performance and latency in the 'test report'_ document.
+
+Test identification
+===================
+
+TBD: WE need to identify the test naming scheme we will use in DoveTail in order that we can cross reference to the test
+projects and maintain our suite effectively. This naming scheme needs to be externally relevant to non-OPNFV consumers and as
+such some consideration is required on the selection.
+
+Pass Fail Criteria
+==================
+
+This section requires some further work with the test teams to identify how and where we generate, store and provide results.
diff --git a/docs/testsuites/ipv6/index.rst b/docs/testsuites/ipv6/index.rst
new file mode 100644
index 00000000..14825521
--- /dev/null
+++ b/docs/testsuites/ipv6/index.rst
@@ -0,0 +1,18 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Christopher Price (Ericsson AB) and others
+
+*******************************
+OPNFV IPv6 compliance test plan
+*******************************
+
+.. toctree::
+ :maxdepth: 2
+
+ ./testplan.rst
+ ./designspecification.rst
+.. <Start>
+.. We should iterate here to cover all test specifications in the test suite
+ ./testspecification.rst
+.. <End>
+ ./testprocedure.rst
diff --git a/docs/testsuites/ipv6/testplan.rst b/docs/testsuites/ipv6/testplan.rst
new file mode 100644
index 00000000..1d03d812
--- /dev/null
+++ b/docs/testsuites/ipv6/testplan.rst
@@ -0,0 +1,60 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Christopher Price (Ericsson AB) and others
+
+===============================
+OPNFV IPv6 compliance test plan
+===============================
+
+Introduction
+============
+
+The IPv6 compliance test plan outlines the method for testing IPv6 compliance to the OPNFV
+platform behaviours and features of IPv6 enabled VNFi platforms.
+
+Scope
+-----
+
+In this document, and throughout the test suite, the system under test (SUT) is defined as
+the combination of the VNFi and VIM as defined by the ETSI NVF ISG (need reference).
+
+Test suite scope and procedures
+===============================
+
+The IPv6 compliance test suite will evaluate the ability for a NVFi platform and VIM to provide
+needed IPv6 features and functionality for use as an IPv6 enabled platform.
+
+An IPv6 enabled platform needs to be able to demonstrate the ability to support for instance;
+the ability to assign IPv6 addresses using SLAAC or stateful and stateless DHCPv6, the ability
+to securely send a receive IPv6 traffic to an external network, support for multiple address
+and dual stack interfaces. For a complete list of the test cases refer to the test case specification [2]_.
+
+As the test suite runs as an external entity the system under test includes both the behaviours
+and interfaces specified for the test. It is expected that the system under test (SUT) interfaces
+are compliant with the specifications or references supplied in the test case design [1]_ document.
+In addition the behaviours exhibited by the platform should comply to the referred standards
+or behaviours as outlined in the test case design [1]_ document.
+
+Test suite execution
+====================
+
+The test suite expects to interact with a system in an IPv6 enabled and ready state. An IPv6
+enabled system can be installed using an IPv6 enabled OPNFV scenario, or if you are not using
+an OPNFV scenario ensure you have configured your system to use IPv6, in this case also pay
+specific attention to the details of specific test preconditions captured in the
+test procedure specification [3]_.
+
+The test suite is designed to be run from an external server, the jump host, with access to the
+control and network interfaces in order that it interacts with the SUT as an external entity.
+The test suite will be executed by the test suite operator via scripts to be executed on the
+jump host, or by leveraging the OPNFV infrastructure to execute an IPv6 compliance test job
+from the jump host. Test cases may instantiate additional workloads on the SUT as part of the
+test cases being executed, the system shall be returned to it's original state once the test
+suite has completed.
+
+
+
+.._[1]: http://www.opnfv.org
+.._[2]: http://www.opnfv.org
+.._[3]: http://www.opnfv.org
+
diff --git a/docs/testsuites/ipv6/testprocedure.rst b/docs/testsuites/ipv6/testprocedure.rst
new file mode 100644
index 00000000..2119ed61
--- /dev/null
+++ b/docs/testsuites/ipv6/testprocedure.rst
@@ -0,0 +1,9 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Christopher Price (Ericsson AB) and others
+
+===================
+IPv6 test procedure
+===================
+
+Draft to be patched this week, someone feel free to work on this in parallel.
diff --git a/docs/testsuites/ipv6/testspecification.rst b/docs/testsuites/ipv6/testspecification.rst
new file mode 100644
index 00000000..6f7caba8
--- /dev/null
+++ b/docs/testsuites/ipv6/testspecification.rst
@@ -0,0 +1,54 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Christopher Price (Ericsson AB) and others
+
+===============================================
+Test specification - Service VM as IPv6 vRouter
+===============================================
+
+Draft to be worked on, this represents the YardStick test but I would suggest we need to break
+this into a set of tests which provide more details per action with boundary validation.
+
+Test Item
+=========
+
+TBD -> IPv6 Ping...
+
+Identify the items or features to be tested by this test case. The item description and
+definition can be referenced from any one of several sources, depending on the level of the
+test case specification. It may be a good idea to reference the source documents as well.
+
+Environmental requirements
+==========================
+
+TBD
+
+Preconditions and procedural requirements
+=========================================
+
+TBD
+
+.. <Start>
+.. this section may be iterated over for a set of simillar test cases that would be run as one.
+
+Input Specifications
+====================
+
+TBD
+
+Output Specifications
+=====================
+
+TBD
+
+.. <End>
+
+Test Reporting
+==============
+
+The test report for this test case will be generated with links to relevant data sources.
+This section can be updated once we have a template for the report in place.
+
+http://testresults.opnfv.org/grafana/dashboard/db/yardstick-tc027
+
+