IPv6 test cases: Test case specification document example

JIRA: DOVETAIL-9

1. We are keeping the test methodology simple and easy.
   We are reusing existing tests as much as we can from
   IPv6 projects and Tempest API and Scenario tests, which
   are run in FuncTest and Yardstick.

2. We modified "testplan.rst" to briefly describe the test plan,
   i.e. current IPv6 test cases for platform compliance, which
   have passed FuncTest and Yardstick test in Arno and Brahmaputra
   releases. We also leave room for additional IPv6 test cases if
   there are more contributions down the road

3. Original "designspecification.rst", "testprocedure.rst" and
   "testspecification.rst" are not applicable any more here. The
   test design, setup, and execution procedure related specification
   are covered by each test case.

4. Thus "index.rst" only covers "testplan.rst" and the list of test cases.

5. Two examples of test case specification documents are provided
   "ipv6.tc001.specification.rst"
   "ipv6.tc026.specification.rst"

6. A complete set of current IPv6 test cases are listed in
   "ipv6_all_testcases.rst". Once template is agreed, we will converted
   it into specific test case document.

For more background of IPv6, and its features and technical details, please see:
   https://wiki.opnfv.org/display/ipv6/IPv6+Home
   http://artifacts.opnfv.org/ipv6/docs/installationprocedure/index.html

Change-Id: I7e47c73c0f02a407df278c93a3da2142b5cc86e5
Signed-off-by: Bin Hu <bh526r@att.com>

5 files changed, 381 insertions, 44 deletions

diff --git a/docs/testsuites/ipv6/index.rst b/docs/testsuites/ipv6/index.rst
index 14825521..72fd6a89 100644
--- a/docs/testsuites/ipv6/index.rst
+++ b/docs/testsuites/ipv6/index.rst
@@ -1,18 +1,15 @@
 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
 .. http://creativecommons.org/licenses/by/4.0
-.. (c) Christopher Price (Ericsson AB) and others
+.. (c) OPNFV
 
 *******************************
-OPNFV IPv6 compliance test plan
+OPNFV IPv6 Compliance Test Plan
 *******************************
 
 .. toctree::
    :maxdepth: 2
 
    ./testplan.rst
-   ./designspecification.rst
-.. <Start>
-.. We should iterate here to cover all test specifications in the test suite
-   ./testspecification.rst
-.. <End>
-   ./testprocedure.rst
+   ./ipv6.tc001.specification.rst
+   ./ipv6.tc026.specification.rst
+
diff --git a/docs/testsuites/ipv6/ipv6.tc001.specification.rst b/docs/testsuites/ipv6/ipv6.tc001.specification.rst
new file mode 100644
index 00000000..350ff73d
--- /dev/null
+++ b/docs/testsuites/ipv6/ipv6.tc001.specification.rst
@@ -0,0 +1,61 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV
+
+==================================================================================================
+Dovetail IPv6 tc001 specification - Bulk Creation and Deletion of IPv6 Networks, Ports and Subnets
+==================================================================================================
+
+.. table::
+   :class: longtable
+
++-----------------------+----------------------------------------------------------------------------------------------------+
+|test case name         |Bulk creation and deletion of IPv6 networks, ports and subnets                                      |
+|                       |                                                                                                    |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|id                     |dovetail.ipv6.tc001                                                                                 |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|objective              |To verify that platform is able to create/delete networks, ports and subnets in bulk operation      |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|test items             |tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network            |
+|                       |{idempotent_id('d4f9024d-1e28-4fc1-a6b1-25dbc6fa11e2')}                                             |
+|                       |tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port               |
+|                       |{idempotent_id('48037ff2-e889-4c3b-b86a-8e3f34d2d060')}                                             |
+|                       |tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet             |
+|                       |{idempotent_id('8936533b-c0aa-4f29-8e53-6cc873aec489')}                                             |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|environmental          |                                                                                                    |
+|requirements &         | environment can be deployed on bare metal of virtualized infrastructure                            |
+|preconditions          | deployment can be HA or non-HA                                                                     |
+|                       |                                                                                                    |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|scenario dependencies  | NA                                                                                                 |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|procedural             |Step 1: create/delete network:                                                                      |
+|requirements           |     create 2 networks in one request                                                               |
+|                       |     asserting that the networks are found in the list after creation                               |
+|                       |                                                                                                    |
+|                       |Step 2: create/delete subnet:                                                                       |
+|                       |     create 2 subnets in one request                                                                |
+|                       |     asserting that the subnets are found in the list after creation                                |
+|                       |                                                                                                    |
+|                       |Step 3: create/delete port:                                                                         |
+|                       |     create 2 ports in one request                                                                  |
+|                       |     asserting that the ports are found in the list after creation                                  |
+|                       |                                                                                                    |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|input specifications   |The parameters needed to execute Neutron network APIs.                                              |
+|                       |Refer to Neutron Networking API v2.0 [1]_ [2]_                                                      |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|output specifications  |The responses after executing Network network APIs.                                                 |
+|                       |Refer to Neutron Networking API v2.0 [1]_ [2]_                                                      |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|pass/fail criteria     |If normal response code 200 is returned, the test passes.                                           |
+|                       |Otherwise, the test fails with various error codes.                                                 |
+|                       |Refer to Neutron Networking API v2.0 [1]_ [2]_                                                      |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|test report            |TBD                                                                                                 |
++-----------------------+----------------------------------------------------------------------------------------------------+
+
+.._[1]: http://developer.openstack.org/api-ref/networking/v2/
+.._[2]: http://wiki.openstack.org/wiki/Neutron/APIv2-specification
diff --git a/docs/testsuites/ipv6/ipv6.tc026.specification.rst b/docs/testsuites/ipv6/ipv6.tc026.specification.rst
new file mode 100644
index 00000000..a4bb6586
--- /dev/null
+++ b/docs/testsuites/ipv6/ipv6.tc026.specification.rst
@@ -0,0 +1,56 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV
+
+==============================================================
+Dovetail IPv6 tc026 specification - Service VM as IPv6 vRouter
+==============================================================
+
+.. table::
+   :class: longtable
+
++-----------------------+--------------------------------------------------------------------------+
+|test case name         |Service VM as IPv6 vRouter                                                |
+|                       |                                                                          |
++-----------------------+--------------------------------------------------------------------------+
+|id                     |dovetail.ipv6.tc026                                                       |
++-----------------------+--------------------------------------------------------------------------+
+|objective              |IPv6 connnectivity, service VM as IPv6 vRouter                            |
++-----------------------+--------------------------------------------------------------------------+
+|modules under test     |neutron, nova, etc                                                        |
++-----------------------+--------------------------------------------------------------------------+
+|dependent test project |yardstick                                                                 |
++-----------------------+--------------------------------------------------------------------------+
+|test items             |yardstick_tc027                                                           |
++-----------------------+--------------------------------------------------------------------------+
+|environmental          | OpenStack-only environment                                               |
+|requirements &         | environment can be deplyed on bare metal of virtualized infrastructure   |
+|preconditions          | deployment can be HA or non-HA                                           |
+|                       | test case image needs to be installed into Glance with ping6 included    |
++-----------------------+--------------------------------------------------------------------------+
+|scenario dependencies  | nosdn                                                                    |
++-----------------------+--------------------------------------------------------------------------+
+|procedural             |step 1: to setup IPv6 testing environment                                 |
+|requirements           |     1.1 disable security group                                           |
+|                       |     1.2 create (ipv6, ipv4) router, network and subnet                   |
+|                       |     1.3 create vRouter, VM1, VM2                                         |
+|                       |step 2: to run ping6 to verify IPv6 connectivity                          |
+|                       |     2.1 ssh to VM1                                                       |
+|                       |     2.2 ping6 to ipv6 router from VM1                                    |
+|                       |     2.3 get the result and store the logs                                |
+|                       |step 3: to teardown IPv6 testing environment                              |
+|                       |     3.1 delete vRouter, VM1, VM2                                         |
+|                       |     3.2 delete (ipv6, ipv4) router, network and subnet                   |
+|                       |     3.3 enable security group                                            |
++-----------------------+--------------------------------------------------------------------------+
+|input specifications   |packetsize: 56                                                            |
+|                       |ping_count: 5                                                             |
+|                       |                                                                          |
++-----------------------+--------------------------------------------------------------------------+
+|output specifications  |output includes max_rtt, min_rtt, average_rtt                             |
++-----------------------+--------------------------------------------------------------------------+
+|pass/fail criteria     |ping6 connectivity success, no SLA                                        |
++-----------------------+--------------------------------------------------------------------------+
+|test report            | dovetail dashboard DB here                                               |
++-----------------------+--------------------------------------------------------------------------+
+
diff --git a/docs/testsuites/ipv6/ipv6_all_testcases.rst b/docs/testsuites/ipv6/ipv6_all_testcases.rst
new file mode 100644
index 00000000..462219a8
--- /dev/null
+++ b/docs/testsuites/ipv6/ipv6_all_testcases.rst
@@ -0,0 +1,249 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV
+
+==================================================
+IPv6 Compliance Testing Methodology and Test Cases
+==================================================
+
+IPv6 Compliance Testing focuses on overlay IPv6 capabilities, i.e. to validate that
+IPv6 capability is supported in tenant networks, subnets and routers. Both Tempest API
+testing and Tempest Scenario testing are reused  as much as we can in IPv6 Compliance
+Testing. In addition, Yardstick Test Case 027 is also used to validate a specific use case
+of using a Service VM as an IPv6 vRouter.
+
+IPv6 Compliance Testing test cases are described as follows:
+
+---------------------------------------------------------------
+Test Case 1: Create and Delete an IPv6 Network, Port and Subnet
+---------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network
+    tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port
+    tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet
+
+-----------------------------------------------------------------
+Test Case 2: Create, Update and Delete an IPv6 Network and Subnet
+-----------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_networks.NetworksIpV6Test.test_create_update_delete_network_subnet
+    tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_create_update_delete_network_subnet
+
+----------------------------------------------
+Test Case 3: Check External Network Visibility
+----------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_networks.NetworksIpV6Test.test_external_network_visibility
+    tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_external_network_visibility
+
+-------------------------------------------------------
+Test Case 4: List IPv6 Networks and Subnets of a Tenant
+-------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_networks.NetworksIpV6Test.test_list_networks
+    tempest.api.network.test_networks.NetworksIpV6Test.test_list_subnets
+    tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_list_networks
+    tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_list_subnets
+
+-----------------------------------------------------------
+Test Case 5: Show Information of an IPv6 Network and Subnet
+-----------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_networks.NetworksIpV6Test.test_show_network
+    tempest.api.network.test_networks.NetworksIpV6Test.test_show_subnet
+    tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_show_network
+    tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_show_subnet
+
+------------------------------------------------------------
+Test Case 6: Create an IPv6 Port in Allowed Allocation Pools
+------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_in_allowed_allocation_pools
+
+--------------------------------------------------------
+Test Case 7: Create an IPv6 Port without Security Groups
+--------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_with_no_securitygroups
+
+---------------------------------------------------
+Test Case 8: Create, Update and Delete an IPv6 Port
+---------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_update_delete_port
+
+----------------------------------------
+Test Case 9: List IPv6 Ports of a Tenant
+----------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_ports.PortsIpV6TestJSON.test_list_ports
+
+----------------------------------------------
+Test Case 10: Show Information of an IPv6 Port
+----------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_ports.PortsIpV6TestJSON.test_show_port
+
+--------------------------------------------------------
+Test Case 11: Add Multiple Interfaces for an IPv6 Router
+--------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_routers.RoutersIpV6Test.test_add_multiple_router_interfaces
+
+------------------------------------------------------------------
+Test Case 12: Add and Remove an IPv6 Router Interface with port_id
+------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_routers.RoutersIpV6Test.test_add_remove_router_interface_with_port_id
+
+--------------------------------------------------------------------
+Test Case 13: Add and Remove an IPv6 Router Interface with subnet_id
+--------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_routers.RoutersIpV6Test.test_add_remove_router_interface_with_subnet_id
+
+------------------------------------------------------------------
+Test Case 14: Create, Update, Delete, List and Show an IPv6 Router
+------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_routers.RoutersIpV6Test.test_create_show_list_update_delete_router
+
+--------------------------------------------------------------------------
+Test Case 15: Create, Update, Delete, List and Show an IPv6 Security Group
+--------------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_security_groups.SecGroupIPv6Test.test_create_list_update_show_delete_security_group
+
+----------------------------------------------------------
+Test Case 16: Create, Delete and Show Security Group Rules
+----------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_security_groups.SecGroupIPv6Test.test_create_show_delete_security_group_rule
+
+--------------------------------------
+Test Case 17: List All Security Groups
+--------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_security_groups.SecGroupIPv6Test.test_list_security_groups
+
+--------------------------------------------------------
+Test Case 18: IPv6 Address Assignment - DHCPv6 Stateless
+--------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_dhcp6_stateless_from_os
+
+--------------------------------------------------------------------
+Test Case 19: IPv6 Address Assignment - Dual Stack, DHCPv6 Stateless
+--------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_dhcp6_stateless_from_os
+
+---------------------------------------------------------------------------
+Test Case 20: IPv6 Address Assignment - Multiple Prefixes, DHCPv6 Stateless
+---------------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_dhcpv6_stateless
+
+---------------------------------------------------------------------------------------
+Test Case 21: IPv6 Address Assignment - Dual Stack, Multiple Prefixes, DHCPv6 Stateless
+---------------------------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_dhcpv6_stateless
+
+---------------------------------------------
+Test Case 22: IPv6 Address Assignment - SLAAC
+---------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_slaac_from_os
+
+---------------------------------------------------------
+Test Case 23: IPv6 Address Assignment - Dual Stack, SLAAC
+---------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_slaac_from_os
+
+----------------------------------------------------------------
+Test Case 24: IPv6 Address Assignment - Multiple Prefixes, SLAAC
+----------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_slaac
+
+----------------------------------------------------------------------------
+Test Case 25: IPv6 Address Assignment - Dual Stack, Multiple Prefixes, SLAAC
+----------------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_slaac
+
+-------------------------------------------
+Test Case 26: Service VM as an IPv6 vRouter
+-------------------------------------------
+
+.. code-block:: bash
+
+    # Refer to Yardstick Test Case 027
+    # Instruction: http://artifacts.opnfv.org/ipv6/docs/configurationguide/index.html
+    # Step 1: Set up Service VM as an IPv6 vRouter
+    #    1.1: Install OPNFV and Preparation
+    #    1.2: Disable Security Groups in OpenStack ML2 Setup
+    #    1.3: Create IPv4 and IPv6 Neutron routers, networks and subnets
+    #    1.4: Boot vRouter VM, and Guest VM1 and Guest VM2
+    # Step 2: Verify IPv6 Connectivity
+    #    2.1: ssh to Guest VM1
+    #    2.2: Ping6 from Guest VM1 to Guest VM2
+    #    2.3: Ping6 from Guest VM1 to vRouter VM
+    #    2.4: Ping6 from Guest VM1 to Neutron IPv6 Router Namespace
+    # Step 3: Tear down Setup
+    #    3.1: Delete Guest VM1, Guest VM2 and vRouter VM
+    #    3.2: Delete IPv4 and IPv6 Neutron routers, networks and subnets
+    #    3.3: Enable Security Groups
+
diff --git a/docs/testsuites/ipv6/testplan.rst b/docs/testsuites/ipv6/testplan.rst
index 1d03d812..3470e7a6 100644
--- a/docs/testsuites/ipv6/testplan.rst
+++ b/docs/testsuites/ipv6/testplan.rst
@@ -1,60 +1,34 @@
 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
 .. http://creativecommons.org/licenses/by/4.0
-.. (c) Christopher Price (Ericsson AB) and others
+.. (c) OPNFV
 
 ===============================
-OPNFV IPv6 compliance test plan
+OPNFV IPv6 Compliance Test Plan
 ===============================
 
 Introduction
 ============
 
-The IPv6 compliance test plan outlines the method for testing IPv6 compliance to the OPNFV
-platform behaviours and features of IPv6 enabled VNFi platforms.
+The IPv6 compliance test plan outlines the method for testing IPv6 Tenant Network feature
+compliance with the OPNFV platform.
 
 Scope
 -----
 
-In this document, and throughout the test suite, the system under test (SUT) is defined as
-the combination of the VNFi and VIM as defined by the ETSI NVF ISG (need reference).
+This test, and other tests in the test suite, are designed to verify an entire SUT,
+and not any individual component of the system.
 
 Test suite scope and procedures
 ===============================
 
-The IPv6 compliance test suite will evaluate the ability for a NVFi platform and VIM to provide
-needed IPv6 features and functionality for use as an IPv6 enabled platform.
+The IPv6 compliance test suite will evaluate the ability for a SUT to support IPv6
+Tenant Network features and functionality provided by OPNFV platform.
 
-An IPv6 enabled platform needs to be able to demonstrate the ability to support for instance;
-the ability to assign IPv6 addresses using SLAAC or stateful and stateless DHCPv6, the ability
-to securely send a receive IPv6 traffic to an external network, support for multiple address
-and dual stack interfaces.  For a complete list of the test cases refer to the test case specification [2]_.
-
-As the test suite runs as an external entity the system under test includes both the behaviours
-and interfaces specified for the test.  It is expected that the system under test (SUT) interfaces
-are compliant with the specifications or references supplied in the test case design [1]_ document.
-In addition the behaviours exhibited by the platform should comply to the referred standards
-or behaviours as outlined in the test case design [1]_ document.
+Please refer to the complete list of the test cases for details.
 
 Test suite execution
 ====================
 
-The test suite expects to interact with a system in an IPv6 enabled and ready state.  An IPv6
-enabled system can be installed using an IPv6 enabled OPNFV scenario, or if you are not using
-an OPNFV scenario ensure you have configured your system to use IPv6, in this case also pay
-specific attention to the details of specific test preconditions captured in the
-test procedure specification [3]_.
-
-The test suite is designed to be run from an external server, the jump host, with access to the
-control and network interfaces in order that it interacts with the SUT as an external entity.
-The test suite will be executed by the test suite operator via scripts to be executed on the
-jump host, or by leveraging the OPNFV infrastructure to execute an IPv6 compliance test job
-from the jump host. Test cases may instantiate additional workloads on the SUT as part of the
-test cases being executed, the system shall be returned to it's original state once the test
-suite has completed.
-
-
+Please refer to each test case for specific setup and execution procedure.
 
 .._[1]: http://www.opnfv.org
-.._[2]: http://www.opnfv.org
-.._[3]: http://www.opnfv.org
-