Proposing a Dovetail test suite structure for IPv6

Adding the first draft of the IPv6 test suite.
This patch attempts to start the documentation according to the
proposal on https://wiki.opnfv.org/pages/viewpage.action?pageId=6827269
The text is neither complete nor accurate at this stage, this patch
is intended to establish a concrete dialog for how we structure our
test suites and reference test cases.

Change-Id: I156ddafa3e8b7630390523864490426fc742eb3a
Signed-off-by: ChristopherPrice <christopher.price@ericsson.com>

7 files changed, 274 insertions, 0 deletions

diff --git a/docs/releases/colorado/certification_usecase.rst b/docs/releases/colorado/certification_usecase.rst
deleted file mode 100644
index e69de29b..00000000
--- a/docs/releases/colorado/certification_usecase.rst
+++ /dev/null
diff --git a/docs/releases/colorado/usecase_description.rst b/docs/releases/colorado/usecase_description.rst
deleted file mode 100644
index e69de29b..00000000
--- a/docs/releases/colorado/usecase_description.rst
+++ /dev/null
diff --git a/docs/testsuites/ipv6/designspecification.rst b/docs/testsuites/ipv6/designspecification.rst
new file mode 100644
index 00000000..9e403472
--- /dev/null
+++ b/docs/testsuites/ipv6/designspecification.rst
@@ -0,0 +1,133 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Christopher Price (Ericsson AB) and others
+
+==============================
+IPv6 test design specification
+==============================
+
+This document outlines the approach and method for testing IPv6 in the OPNFV compliance test
+suite.  Providing a brief outline of the features to be tested, the methodology for testing,
+schema's and criteria.
+
+Features to be tested
+=====================
+
+The IPv6 compliance test plan outlines the method for testing IPv6 compliance to the OPNFV
+platform behaviours and features of IPv6 enabled VNFi platforms.  The specific features to
+be tested by the IPv6 compliance test suite is outlined in the following table.
+
+.. table::
+   :class: longtable
+
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Features / Requirements                                    |Tests available    | Test Cases                                                         |
++===========================================================+===================+====================================================================+
+|All topologies work in a multi-tenant environment          |No                 |                                                                    |
+|                                                           |                   |                                                                    |
+|                                                           |                   |                                                                    |
+|                                                           |                   |                                                                    |
+|                                                           |                   |                                                                    |
+|                                                           |                   |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|IPv6 VM to VM only                                         |No                 |                                                                    |
+|                                                           |                   |                                                                    |
+|                                                           |                   |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|IPv6 external L2 VLAN directly attached to a VM            |No                 |                                                                    |
+|                                                           |                   |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|IPv6 subnet routed via L3 agent to an external IPv6 network|No                 |                                                                    |
+|                                                           |                   |                                                                    |
+|1. Both VLAN and overlay (e.g. GRE, VXLAN) subnet attached |                   |                                                                    |
+|   to VMs;                                                 |                   |                                                                    |
+|2. Must be able to support multiple L3 agents for a given  |                   |                                                                    |
+|   external network to support scaling (neutron scheduler  |                   |                                                                    |
+|   to assign vRouters to the L3 agents)                    |                   |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Ability for a NIC to support both IPv4 and IPv6 (dual      |No                 |                                                                    |
+|stack) address.                                            |                   |                                                                    |
+|                                                           |                   |                                                                    |
+|1. VM with a single interface associated with a network,   |                   |                                                                    |
+|   which is then associated with two subnets.              |                   |                                                                    |
+|2. VM with two different interfaces associated with two    |                   |                                                                    |
+|   different networks and two different subnets.           |                   |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Support IPv6 Address assignment modes.                     |No                 |                                                                    |
+|                                                           |                   |                                                                    |
+|1. SLAAC                                                   |                   |                                                                    |
+|2. DHCPv6 Stateless                                        |                   |                                                                    |
+|3. DHCPv6 Stateful                                         |                   |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Ability to create a port on an IPv6 DHCPv6 Stateful subnet |No                 |                                                                    |
+|and assign a specific IPv6 address to the port and have it |                   |                                                                    |
+|taken out of the DHCP address pool.                        |                   |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Full support for IPv6 matching (i.e., IPv6, ICMPv6, TCP,   |No                 |                                                                    |
+|UDP) in security groups. Ability to control and manage all |                   |                                                                    |
+|IPv6 security group capabilities via Neutron/Nova API (REST|                   |                                                                    |
+|and CLI) as well as via Horizon.                           |                   |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|During network/subnet/router create, there should be an    |No                 |                                                                    |
+|option to allow user to specify the type of address        |                   |                                                                    |
+|management they would like. This includes all options      |                   |                                                                    |
+|including those low priority if implemented (e.g., toggle  |                   |                                                                    |
+|on/off router and address prefix advertisements); It must  |                   |                                                                    |
+|be supported via Neutron API (REST and CLI) as well as via |                   |                                                                    |
+|Horizon                                                    |                   |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Security groups anti-spoofing: Prevent VM from using a     |No                 |                                                                    |
+|source IPv6/MAC address which is not assigned to the VM    |                   |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Protect tenant and provider network from rogue RAs         |No                 |                                                                    |
+|                                                           |                   |                                                                    |
+|                                                           |                   |                                                                    |
+|                                                           |                   |                                                                    |
+|                                                           |                   |                                                                    |
+|                                                           |                   |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Support the ability to assign multiple IPv6 addresses to   |No                 |                                                                    |
+|an interface; both for Neutron router interfaces and VM    |                   |                                                                    |
+|interfaces.                                                |                   |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Ability for a VM to support a mix of multiple IPv4 and IPv6|No                 |                                                                    |
+|networks, including multiples of the same type.            |                   |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|Support for IPv6 Prefix Delegation.                        |No                 |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|IPv6 First-Hop Security, IPv6 ND spoofing                  |No                 |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+|IPv6 support in Neutron Layer3 High Availability           |No                 |                                                                    |
+|(keepalived+VRRP).                                         |                   |                                                                    |
++-----------------------------------------------------------+-------------------+--------------------------------------------------------------------+
+
+
+Test approach for IPv6
+======================
+
+The most common approach for testing IPv6 capabilities in the test suite is through interaction with the SUT control plane.
+In this instance the test framework will exercise the NBI provided by the VIM to configure and leverage IPv6 related features
+in the platform, instantiate workloads, and invoke behaviours in the platform.  The suite may also interact directly with the
+data plane to exercise platform capabilities and further invoke helper functions on the platform for the same purpose.
+
+Test result analysis
+--------------------
+
+All functional tests in the IPv6 test suite will provide a pass/fail result on completion of the test.  In addition test logs
+and relevant additional information will be provided as part of the test log, available on test suite completion.
+
+Some tests in the compliance suite measure such metrics as latency and performance.  At this time these tests are intended to
+provide a feature based pass/fail metric not related to system performance.
+These tests may however provide detailed results of performance and latency in the 'test report'_ document.
+
+Test identification
+===================
+
+TBD:  WE need to identify the test naming scheme we will use in DoveTail in order that we can cross reference to the test
+projects and maintain our suite effectively.  This naming scheme needs to be externally relevant to non-OPNFV consumers and as
+such some consideration is required on the selection.
+
+Pass Fail Criteria
+==================
+
+This section requires some further work with the test teams to identify how and where we generate, store and provide results.
diff --git a/docs/testsuites/ipv6/index.rst b/docs/testsuites/ipv6/index.rst
new file mode 100644
index 00000000..14825521
--- /dev/null
+++ b/docs/testsuites/ipv6/index.rst
@@ -0,0 +1,18 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Christopher Price (Ericsson AB) and others
+
+*******************************
+OPNFV IPv6 compliance test plan
+*******************************
+
+.. toctree::
+   :maxdepth: 2
+
+   ./testplan.rst
+   ./designspecification.rst
+.. <Start>
+.. We should iterate here to cover all test specifications in the test suite
+   ./testspecification.rst
+.. <End>
+   ./testprocedure.rst
diff --git a/docs/testsuites/ipv6/testplan.rst b/docs/testsuites/ipv6/testplan.rst
new file mode 100644
index 00000000..1d03d812
--- /dev/null
+++ b/docs/testsuites/ipv6/testplan.rst
@@ -0,0 +1,60 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Christopher Price (Ericsson AB) and others
+
+===============================
+OPNFV IPv6 compliance test plan
+===============================
+
+Introduction
+============
+
+The IPv6 compliance test plan outlines the method for testing IPv6 compliance to the OPNFV
+platform behaviours and features of IPv6 enabled VNFi platforms.
+
+Scope
+-----
+
+In this document, and throughout the test suite, the system under test (SUT) is defined as
+the combination of the VNFi and VIM as defined by the ETSI NVF ISG (need reference).
+
+Test suite scope and procedures
+===============================
+
+The IPv6 compliance test suite will evaluate the ability for a NVFi platform and VIM to provide
+needed IPv6 features and functionality for use as an IPv6 enabled platform.
+
+An IPv6 enabled platform needs to be able to demonstrate the ability to support for instance;
+the ability to assign IPv6 addresses using SLAAC or stateful and stateless DHCPv6, the ability
+to securely send a receive IPv6 traffic to an external network, support for multiple address
+and dual stack interfaces.  For a complete list of the test cases refer to the test case specification [2]_.
+
+As the test suite runs as an external entity the system under test includes both the behaviours
+and interfaces specified for the test.  It is expected that the system under test (SUT) interfaces
+are compliant with the specifications or references supplied in the test case design [1]_ document.
+In addition the behaviours exhibited by the platform should comply to the referred standards
+or behaviours as outlined in the test case design [1]_ document.
+
+Test suite execution
+====================
+
+The test suite expects to interact with a system in an IPv6 enabled and ready state.  An IPv6
+enabled system can be installed using an IPv6 enabled OPNFV scenario, or if you are not using
+an OPNFV scenario ensure you have configured your system to use IPv6, in this case also pay
+specific attention to the details of specific test preconditions captured in the
+test procedure specification [3]_.
+
+The test suite is designed to be run from an external server, the jump host, with access to the
+control and network interfaces in order that it interacts with the SUT as an external entity.
+The test suite will be executed by the test suite operator via scripts to be executed on the
+jump host, or by leveraging the OPNFV infrastructure to execute an IPv6 compliance test job
+from the jump host. Test cases may instantiate additional workloads on the SUT as part of the
+test cases being executed, the system shall be returned to it's original state once the test
+suite has completed.
+
+
+
+.._[1]: http://www.opnfv.org
+.._[2]: http://www.opnfv.org
+.._[3]: http://www.opnfv.org
+
diff --git a/docs/testsuites/ipv6/testprocedure.rst b/docs/testsuites/ipv6/testprocedure.rst
new file mode 100644
index 00000000..2119ed61
--- /dev/null
+++ b/docs/testsuites/ipv6/testprocedure.rst
@@ -0,0 +1,9 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Christopher Price (Ericsson AB) and others
+
+===================
+IPv6 test procedure
+===================
+
+Draft to be patched this week, someone feel free to work on this in parallel.
diff --git a/docs/testsuites/ipv6/testspecification.rst b/docs/testsuites/ipv6/testspecification.rst
new file mode 100644
index 00000000..6f7caba8
--- /dev/null
+++ b/docs/testsuites/ipv6/testspecification.rst
@@ -0,0 +1,54 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) Christopher Price (Ericsson AB) and others
+
+===============================================
+Test specification - Service VM as IPv6 vRouter
+===============================================
+
+Draft to be worked on, this represents the YardStick test but I would suggest we need to break
+this into a set of tests which provide more details per action with boundary validation.
+
+Test Item
+=========
+
+TBD -> IPv6 Ping...
+
+Identify the items or features to be tested by this test case.  The item description and
+definition can be referenced from any one of several sources, depending on the level of the
+test case specification. It may be a good idea to reference the source documents as well.
+
+Environmental requirements
+==========================
+
+TBD
+
+Preconditions and procedural requirements
+=========================================
+
+TBD
+
+.. <Start>
+.. this section may be iterated over for a set of simillar test cases that would be run as one.
+
+Input Specifications
+====================
+
+TBD
+
+Output Specifications
+=====================
+
+TBD
+
+.. <End>
+
+Test Reporting
+==============
+
+The test report for this test case will be generated with links to relevant data sources.
+This section can be updated once we have a template for the report in place.
+
+http://testresults.opnfv.org/grafana/dashboard/db/yardstick-tc027
+
+