Merge "IPv6 test cases: Test case specification document example"

5 files changed, 381 insertions, 44 deletions

diff --git a/docs/testsuites/ipv6/index.rst b/docs/testsuites/ipv6/index.rst
index 14825521..72fd6a89 100644
--- a/docs/testsuites/ipv6/index.rst
+++ b/docs/testsuites/ipv6/index.rst
@@ -1,18 +1,15 @@
 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
 .. http://creativecommons.org/licenses/by/4.0
-.. (c) Christopher Price (Ericsson AB) and others
+.. (c) OPNFV
 
 *******************************
-OPNFV IPv6 compliance test plan
+OPNFV IPv6 Compliance Test Plan
 *******************************
 
 .. toctree::
    :maxdepth: 2
 
    ./testplan.rst
-   ./designspecification.rst
-.. <Start>
-.. We should iterate here to cover all test specifications in the test suite
-   ./testspecification.rst
-.. <End>
-   ./testprocedure.rst
+   ./ipv6.tc001.specification.rst
+   ./ipv6.tc026.specification.rst
+
diff --git a/docs/testsuites/ipv6/ipv6.tc001.specification.rst b/docs/testsuites/ipv6/ipv6.tc001.specification.rst
new file mode 100644
index 00000000..350ff73d
--- /dev/null
+++ b/docs/testsuites/ipv6/ipv6.tc001.specification.rst
@@ -0,0 +1,61 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV
+
+==================================================================================================
+Dovetail IPv6 tc001 specification - Bulk Creation and Deletion of IPv6 Networks, Ports and Subnets
+==================================================================================================
+
+.. table::
+   :class: longtable
+
++-----------------------+----------------------------------------------------------------------------------------------------+
+|test case name         |Bulk creation and deletion of IPv6 networks, ports and subnets                                      |
+|                       |                                                                                                    |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|id                     |dovetail.ipv6.tc001                                                                                 |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|objective              |To verify that platform is able to create/delete networks, ports and subnets in bulk operation      |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|test items             |tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network            |
+|                       |{idempotent_id('d4f9024d-1e28-4fc1-a6b1-25dbc6fa11e2')}                                             |
+|                       |tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port               |
+|                       |{idempotent_id('48037ff2-e889-4c3b-b86a-8e3f34d2d060')}                                             |
+|                       |tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet             |
+|                       |{idempotent_id('8936533b-c0aa-4f29-8e53-6cc873aec489')}                                             |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|environmental          |                                                                                                    |
+|requirements &         | environment can be deployed on bare metal of virtualized infrastructure                            |
+|preconditions          | deployment can be HA or non-HA                                                                     |
+|                       |                                                                                                    |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|scenario dependencies  | NA                                                                                                 |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|procedural             |Step 1: create/delete network:                                                                      |
+|requirements           |     create 2 networks in one request                                                               |
+|                       |     asserting that the networks are found in the list after creation                               |
+|                       |                                                                                                    |
+|                       |Step 2: create/delete subnet:                                                                       |
+|                       |     create 2 subnets in one request                                                                |
+|                       |     asserting that the subnets are found in the list after creation                                |
+|                       |                                                                                                    |
+|                       |Step 3: create/delete port:                                                                         |
+|                       |     create 2 ports in one request                                                                  |
+|                       |     asserting that the ports are found in the list after creation                                  |
+|                       |                                                                                                    |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|input specifications   |The parameters needed to execute Neutron network APIs.                                              |
+|                       |Refer to Neutron Networking API v2.0 [1]_ [2]_                                                      |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|output specifications  |The responses after executing Network network APIs.                                                 |
+|                       |Refer to Neutron Networking API v2.0 [1]_ [2]_                                                      |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|pass/fail criteria     |If normal response code 200 is returned, the test passes.                                           |
+|                       |Otherwise, the test fails with various error codes.                                                 |
+|                       |Refer to Neutron Networking API v2.0 [1]_ [2]_                                                      |
++-----------------------+----------------------------------------------------------------------------------------------------+
+|test report            |TBD                                                                                                 |
++-----------------------+----------------------------------------------------------------------------------------------------+
+
+.._[1]: http://developer.openstack.org/api-ref/networking/v2/
+.._[2]: http://wiki.openstack.org/wiki/Neutron/APIv2-specification
diff --git a/docs/testsuites/ipv6/ipv6.tc026.specification.rst b/docs/testsuites/ipv6/ipv6.tc026.specification.rst
new file mode 100644
index 00000000..a4bb6586
--- /dev/null
+++ b/docs/testsuites/ipv6/ipv6.tc026.specification.rst
@@ -0,0 +1,56 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV
+
+==============================================================
+Dovetail IPv6 tc026 specification - Service VM as IPv6 vRouter
+==============================================================
+
+.. table::
+   :class: longtable
+
++-----------------------+--------------------------------------------------------------------------+
+|test case name         |Service VM as IPv6 vRouter                                                |
+|                       |                                                                          |
++-----------------------+--------------------------------------------------------------------------+
+|id                     |dovetail.ipv6.tc026                                                       |
++-----------------------+--------------------------------------------------------------------------+
+|objective              |IPv6 connnectivity, service VM as IPv6 vRouter                            |
++-----------------------+--------------------------------------------------------------------------+
+|modules under test     |neutron, nova, etc                                                        |
++-----------------------+--------------------------------------------------------------------------+
+|dependent test project |yardstick                                                                 |
++-----------------------+--------------------------------------------------------------------------+
+|test items             |yardstick_tc027                                                           |
++-----------------------+--------------------------------------------------------------------------+
+|environmental          | OpenStack-only environment                                               |
+|requirements &         | environment can be deplyed on bare metal of virtualized infrastructure   |
+|preconditions          | deployment can be HA or non-HA                                           |
+|                       | test case image needs to be installed into Glance with ping6 included    |
++-----------------------+--------------------------------------------------------------------------+
+|scenario dependencies  | nosdn                                                                    |
++-----------------------+--------------------------------------------------------------------------+
+|procedural             |step 1: to setup IPv6 testing environment                                 |
+|requirements           |     1.1 disable security group                                           |
+|                       |     1.2 create (ipv6, ipv4) router, network and subnet                   |
+|                       |     1.3 create vRouter, VM1, VM2                                         |
+|                       |step 2: to run ping6 to verify IPv6 connectivity                          |
+|                       |     2.1 ssh to VM1                                                       |
+|                       |     2.2 ping6 to ipv6 router from VM1                                    |
+|                       |     2.3 get the result and store the logs                                |
+|                       |step 3: to teardown IPv6 testing environment                              |
+|                       |     3.1 delete vRouter, VM1, VM2                                         |
+|                       |     3.2 delete (ipv6, ipv4) router, network and subnet                   |
+|                       |     3.3 enable security group                                            |
++-----------------------+--------------------------------------------------------------------------+
+|input specifications   |packetsize: 56                                                            |
+|                       |ping_count: 5                                                             |
+|                       |                                                                          |
++-----------------------+--------------------------------------------------------------------------+
+|output specifications  |output includes max_rtt, min_rtt, average_rtt                             |
++-----------------------+--------------------------------------------------------------------------+
+|pass/fail criteria     |ping6 connectivity success, no SLA                                        |
++-----------------------+--------------------------------------------------------------------------+
+|test report            | dovetail dashboard DB here                                               |
++-----------------------+--------------------------------------------------------------------------+
+
diff --git a/docs/testsuites/ipv6/ipv6_all_testcases.rst b/docs/testsuites/ipv6/ipv6_all_testcases.rst
new file mode 100644
index 00000000..462219a8
--- /dev/null
+++ b/docs/testsuites/ipv6/ipv6_all_testcases.rst
@@ -0,0 +1,249 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV
+
+==================================================
+IPv6 Compliance Testing Methodology and Test Cases
+==================================================
+
+IPv6 Compliance Testing focuses on overlay IPv6 capabilities, i.e. to validate that
+IPv6 capability is supported in tenant networks, subnets and routers. Both Tempest API
+testing and Tempest Scenario testing are reused  as much as we can in IPv6 Compliance
+Testing. In addition, Yardstick Test Case 027 is also used to validate a specific use case
+of using a Service VM as an IPv6 vRouter.
+
+IPv6 Compliance Testing test cases are described as follows:
+
+---------------------------------------------------------------
+Test Case 1: Create and Delete an IPv6 Network, Port and Subnet
+---------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_network
+    tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_port
+    tempest.api.network.test_networks.BulkNetworkOpsIpV6Test.test_bulk_create_delete_subnet
+
+-----------------------------------------------------------------
+Test Case 2: Create, Update and Delete an IPv6 Network and Subnet
+-----------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_networks.NetworksIpV6Test.test_create_update_delete_network_subnet
+    tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_create_update_delete_network_subnet
+
+----------------------------------------------
+Test Case 3: Check External Network Visibility
+----------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_networks.NetworksIpV6Test.test_external_network_visibility
+    tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_external_network_visibility
+
+-------------------------------------------------------
+Test Case 4: List IPv6 Networks and Subnets of a Tenant
+-------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_networks.NetworksIpV6Test.test_list_networks
+    tempest.api.network.test_networks.NetworksIpV6Test.test_list_subnets
+    tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_list_networks
+    tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_list_subnets
+
+-----------------------------------------------------------
+Test Case 5: Show Information of an IPv6 Network and Subnet
+-----------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_networks.NetworksIpV6Test.test_show_network
+    tempest.api.network.test_networks.NetworksIpV6Test.test_show_subnet
+    tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_show_network
+    tempest.api.network.test_networks.NetworksIpV6TestAttrs.test_show_subnet
+
+------------------------------------------------------------
+Test Case 6: Create an IPv6 Port in Allowed Allocation Pools
+------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_in_allowed_allocation_pools
+
+--------------------------------------------------------
+Test Case 7: Create an IPv6 Port without Security Groups
+--------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_port_with_no_securitygroups
+
+---------------------------------------------------
+Test Case 8: Create, Update and Delete an IPv6 Port
+---------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_ports.PortsIpV6TestJSON.test_create_update_delete_port
+
+----------------------------------------
+Test Case 9: List IPv6 Ports of a Tenant
+----------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_ports.PortsIpV6TestJSON.test_list_ports
+
+----------------------------------------------
+Test Case 10: Show Information of an IPv6 Port
+----------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_ports.PortsIpV6TestJSON.test_show_port
+
+--------------------------------------------------------
+Test Case 11: Add Multiple Interfaces for an IPv6 Router
+--------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_routers.RoutersIpV6Test.test_add_multiple_router_interfaces
+
+------------------------------------------------------------------
+Test Case 12: Add and Remove an IPv6 Router Interface with port_id
+------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_routers.RoutersIpV6Test.test_add_remove_router_interface_with_port_id
+
+--------------------------------------------------------------------
+Test Case 13: Add and Remove an IPv6 Router Interface with subnet_id
+--------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_routers.RoutersIpV6Test.test_add_remove_router_interface_with_subnet_id
+
+------------------------------------------------------------------
+Test Case 14: Create, Update, Delete, List and Show an IPv6 Router
+------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_routers.RoutersIpV6Test.test_create_show_list_update_delete_router
+
+--------------------------------------------------------------------------
+Test Case 15: Create, Update, Delete, List and Show an IPv6 Security Group
+--------------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_security_groups.SecGroupIPv6Test.test_create_list_update_show_delete_security_group
+
+----------------------------------------------------------
+Test Case 16: Create, Delete and Show Security Group Rules
+----------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_security_groups.SecGroupIPv6Test.test_create_show_delete_security_group_rule
+
+--------------------------------------
+Test Case 17: List All Security Groups
+--------------------------------------
+
+.. code-block:: bash
+
+    tempest.api.network.test_security_groups.SecGroupIPv6Test.test_list_security_groups
+
+--------------------------------------------------------
+Test Case 18: IPv6 Address Assignment - DHCPv6 Stateless
+--------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_dhcp6_stateless_from_os
+
+--------------------------------------------------------------------
+Test Case 19: IPv6 Address Assignment - Dual Stack, DHCPv6 Stateless
+--------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_dhcp6_stateless_from_os
+
+---------------------------------------------------------------------------
+Test Case 20: IPv6 Address Assignment - Multiple Prefixes, DHCPv6 Stateless
+---------------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_dhcpv6_stateless
+
+---------------------------------------------------------------------------------------
+Test Case 21: IPv6 Address Assignment - Dual Stack, Multiple Prefixes, DHCPv6 Stateless
+---------------------------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_dhcpv6_stateless
+
+---------------------------------------------
+Test Case 22: IPv6 Address Assignment - SLAAC
+---------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_slaac_from_os
+
+---------------------------------------------------------
+Test Case 23: IPv6 Address Assignment - Dual Stack, SLAAC
+---------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_slaac_from_os
+
+----------------------------------------------------------------
+Test Case 24: IPv6 Address Assignment - Multiple Prefixes, SLAAC
+----------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_slaac
+
+----------------------------------------------------------------------------
+Test Case 25: IPv6 Address Assignment - Dual Stack, Multiple Prefixes, SLAAC
+----------------------------------------------------------------------------
+
+.. code-block:: bash
+
+    tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_multi_prefix_slaac
+
+-------------------------------------------
+Test Case 26: Service VM as an IPv6 vRouter
+-------------------------------------------
+
+.. code-block:: bash
+
+    # Refer to Yardstick Test Case 027
+    # Instruction: http://artifacts.opnfv.org/ipv6/docs/configurationguide/index.html
+    # Step 1: Set up Service VM as an IPv6 vRouter
+    #    1.1: Install OPNFV and Preparation
+    #    1.2: Disable Security Groups in OpenStack ML2 Setup
+    #    1.3: Create IPv4 and IPv6 Neutron routers, networks and subnets
+    #    1.4: Boot vRouter VM, and Guest VM1 and Guest VM2
+    # Step 2: Verify IPv6 Connectivity
+    #    2.1: ssh to Guest VM1
+    #    2.2: Ping6 from Guest VM1 to Guest VM2
+    #    2.3: Ping6 from Guest VM1 to vRouter VM
+    #    2.4: Ping6 from Guest VM1 to Neutron IPv6 Router Namespace
+    # Step 3: Tear down Setup
+    #    3.1: Delete Guest VM1, Guest VM2 and vRouter VM
+    #    3.2: Delete IPv4 and IPv6 Neutron routers, networks and subnets
+    #    3.3: Enable Security Groups
+
diff --git a/docs/testsuites/ipv6/testplan.rst b/docs/testsuites/ipv6/testplan.rst
index 1d03d812..3470e7a6 100644
--- a/docs/testsuites/ipv6/testplan.rst
+++ b/docs/testsuites/ipv6/testplan.rst
@@ -1,60 +1,34 @@
 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
 .. http://creativecommons.org/licenses/by/4.0
-.. (c) Christopher Price (Ericsson AB) and others
+.. (c) OPNFV
 
 ===============================
-OPNFV IPv6 compliance test plan
+OPNFV IPv6 Compliance Test Plan
 ===============================
 
 Introduction
 ============
 
-The IPv6 compliance test plan outlines the method for testing IPv6 compliance to the OPNFV
-platform behaviours and features of IPv6 enabled VNFi platforms.
+The IPv6 compliance test plan outlines the method for testing IPv6 Tenant Network feature
+compliance with the OPNFV platform.
 
 Scope
 -----
 
-In this document, and throughout the test suite, the system under test (SUT) is defined as
-the combination of the VNFi and VIM as defined by the ETSI NVF ISG (need reference).
+This test, and other tests in the test suite, are designed to verify an entire SUT,
+and not any individual component of the system.
 
 Test suite scope and procedures
 ===============================
 
-The IPv6 compliance test suite will evaluate the ability for a NVFi platform and VIM to provide
-needed IPv6 features and functionality for use as an IPv6 enabled platform.
+The IPv6 compliance test suite will evaluate the ability for a SUT to support IPv6
+Tenant Network features and functionality provided by OPNFV platform.
 
-An IPv6 enabled platform needs to be able to demonstrate the ability to support for instance;
-the ability to assign IPv6 addresses using SLAAC or stateful and stateless DHCPv6, the ability
-to securely send a receive IPv6 traffic to an external network, support for multiple address
-and dual stack interfaces.  For a complete list of the test cases refer to the test case specification [2]_.
-
-As the test suite runs as an external entity the system under test includes both the behaviours
-and interfaces specified for the test.  It is expected that the system under test (SUT) interfaces
-are compliant with the specifications or references supplied in the test case design [1]_ document.
-In addition the behaviours exhibited by the platform should comply to the referred standards
-or behaviours as outlined in the test case design [1]_ document.
+Please refer to the complete list of the test cases for details.
 
 Test suite execution
 ====================
 
-The test suite expects to interact with a system in an IPv6 enabled and ready state.  An IPv6
-enabled system can be installed using an IPv6 enabled OPNFV scenario, or if you are not using
-an OPNFV scenario ensure you have configured your system to use IPv6, in this case also pay
-specific attention to the details of specific test preconditions captured in the
-test procedure specification [3]_.
-
-The test suite is designed to be run from an external server, the jump host, with access to the
-control and network interfaces in order that it interacts with the SUT as an external entity.
-The test suite will be executed by the test suite operator via scripts to be executed on the
-jump host, or by leveraging the OPNFV infrastructure to execute an IPv6 compliance test job
-from the jump host. Test cases may instantiate additional workloads on the SUT as part of the
-test cases being executed, the system shall be returned to it's original state once the test
-suite has completed.
-
-
+Please refer to each test case for specific setup and execution procedure.
 
 .._[1]: http://www.opnfv.org
-.._[2]: http://www.opnfv.org
-.._[3]: http://www.opnfv.org
-