Test case spec for SDN controller resilience in non-HA config

This is the test case specification for verifying the HA capabilities of
a SDN controller running in a non-HA configuration.

JIRA: HA-31
JIRA: YARDSTICK-955

Change-Id: I10f9ce4a2888092c033d2c02bf9a5a950b8b12dc
Signed-off-by: Georg Kunz <georg.kunz@ericsson.com>

2 files changed, 183 insertions, 0 deletions

diff --git a/docs/testing/user/userguide/15-list-of-tcs.rst b/docs/testing/user/userguide/15-list-of-tcs.rst
index 47526cdda..cb99c49cf 100644
--- a/docs/testing/user/userguide/15-list-of-tcs.rst
+++ b/docs/testing/user/userguide/15-list-of-tcs.rst
@@ -83,6 +83,7 @@ H A
    opnfv_yardstick_tc056.rst
    opnfv_yardstick_tc057.rst
    opnfv_yardstick_tc058.rst
+   opnfv_yardstick_tc087.rst
 
 IPv6
 ----
diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc087.rst b/docs/testing/user/userguide/opnfv_yardstick_tc087.rst
new file mode 100644
index 000000000..99bfeebfc
--- /dev/null
+++ b/docs/testing/user/userguide/opnfv_yardstick_tc087.rst
@@ -0,0 +1,182 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Ericsson and others.
+
+*************************************
+Yardstick Test Case Description TC087
+*************************************
+
++-----------------------------------------------------------------------------+
+|SDN Controller resilience in non-HA configuration                            |
+|                                                                             |
++--------------+--------------------------------------------------------------+
+|test case id  | OPNFV_YARDSTICK_TC087: SDN controller resilience in          |
+|              | non-HA configuration                                         |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|test purpose  | This test validates that network data plane services are     |
+|              | highly available in the event of an SDN Controller failure,  |
+|              | even if the SDN controller is deployed in a non-HA           |
+|              | configuration. Specifically, the test verifies that          |
+|              | existing data plane connectivity is not impacted, i.e. all   |
+|              | configured network services such as DHCP, ARP, L2,           |
+|              | L3 Security Groups should continue to operate                |
+|              | between the existing VMs while the SDN controller is         |
+|              | offline or rebooting.                                        |
+|              |                                                              |
+|              | The test also validates that new network service operations  |
+|              | (creating a new VM in the existing L2/L3 network or in a new |
+|              | network, etc.) are operational after the SDN controller      |
+|              | has recovered from a failure.                                |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|test method   | This test case fails the SDN controller service running      |
+|              | on the OpenStack controller node, then checks if already     |
+|              | configured DHCP/ARP/L2/L3/SNAT connectivity is not           |
+|              | impacted between VMs and the system is able to execute       |
+|              | new virtual network operations once the SDN controller       |
+|              | is restarted and has fully recovered                         |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|attackers     | In this test case, an attacker called “kill-process” is      |
+|              | needed. This attacker includes three parameters:             |
+|              |  1. fault_type: which is used for finding the attacker's     |
+|              |     scripts. It should be set to 'kill-process' in this test |
+|              |                                                              |
+|              |  2. process_name: should be set to the name of the SDN       |
+|              |     controller process                                       |
+|              |                                                              |
+|              |  3. host: which is the name of a control node where the      |
+|              |     SDN controller process is running                        |
+|              |                                                              |
+|              | e.g. -fault_type: "kill-process"                             |
+|              |      -process_name: "opendaylight"                           |
+|              |      -host: node1                                            |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|monitors      | This test case utilizes two monitors of type "ip-status"     |
+|              | and one monitor of type "process" to track the following     |
+|              | conditions:                                                  |
+|              |  1. "ping_same_network_l2": monitor ICMP traffic between     |
+|              |     VMs in the same Neutron network                          |
+|              |                                                              |
+|              |  2. "ping_external_snat": monitor ICMP traffic from VMs to   |
+|              |     an external host on the Internet to verify SNAT          |
+|              |     functionality.                                           |
+|              |                                                              |
+|              |  3. "SDN controller process monitor": a monitor checking the |
+|              |     state of a specified SDN controller process. It measures |
+|              |     the recovery time of the given process.                  |
+|              |                                                              |
+|              | Monitors of type "ip-status" use the "ping" utility to       |
+|              | verify reachability of a given target IP.                    |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|operations    | In this test case, the following operations are needed:      |
+|              |  1. "nova-create-instance-in_network": create a VM instance  |
+|              |     in one of the existing Neutron network.                  |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|metrics       | In this test case, there are two metrics:                    |
+|              |  1. process_recover_time: which indicates the maximun        |
+|              |     time (seconds) from the process being killed to          |
+|              |     recovered                                                |
+|              |                                                              |
+|              |  2. packet_drop: measure the packets that have been dropped  |
+|              |     by the monitors using pktgen.                            |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|test tool     | Developed by the project. Please see folder:                 |
+|              | "yardstick/benchmark/scenarios/availability/ha_tools"        |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|references    | none                                                         |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files:                |
+|              |  1. test case file: opnfv_yardstick_tc087.yaml               |
+|              |     - Attackers: see above “attackers” discription           |
+|              |     - waiting_time: which is the time (seconds) from the     |
+|              |       process being killed to stoping monitors the monitors  |
+|              |     - Monitors: see above “monitors” discription             |
+|              |     - SLA: see above “metrics” discription                   |
+|              |                                                              |
+|              |  2. POD file: pod.yaml The POD configuration should record   |
+|              |     on pod.yaml first. the “host” item in this test case     |
+|              |     will use the node name in the pod.yaml.                  |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|test sequence | Description and expected result                              |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|pre-action    |  1. The OpenStack cluster is set up with a single SDN        |
+|              |     controller in a non-HA configuration.                    |
+|              |                                                              |
+|              |  2. One or more Neutron networks are created with two or     |
+|              |     more VMs attached to each of the Neutron networks.       |
+|              |                                                              |
+|              |  3. The Neutron networks are attached to a Neutron router    |
+|              |     which is attached to an external network towards the     |
+|              |     DCGW.                                                    |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|step 1        | Start IP connectivity monitors:                              |
+|              |  1. Check the L2 connectivity between the VMs in the same    |
+|              |     Neutron network.                                         |
+|              |                                                              |
+|              |  2. Check connectivity from one VM to an external host on    |
+|              |     the Internet to verify SNAT functionality.
+|              |                                                              |
+|              | Result: The monitor info will be collected.                  |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|step 2        | Start attacker:                                              |
+|              | SSH connect to the VIM node and kill the SDN controller      |
+|              | process                                                      |
+|              |                                                              |
+|              | Result: the SDN controller service will be shutdown          |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|step 3        | Verify the results of the IP connectivity monitors.          |
+|              |                                                              |
+|              | Result: The outage_time metric reported by the monitors      |
+|              | is zero.                                                     |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|step 4        | Restart the SDN controller.                                  |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|step 5        | Create a new VM in the existing Neutron network              |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|step 6        | Verify connectivity between VMs as follows:                  |
+|              |  1. Check the L2 connectivity between the previously         |
+|              |     existing VM and the newly created VM on the same         |
+|              |     Neutron network by sending ICMP messages                 |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|step 7        | Stop IP connectivity monitors after a period of time         |
+|              | specified by “waiting_time”                                  |
+|              |                                                              |
+|              | Result: The monitor info will be aggregated                  |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|step 8        | Verify the IP connectivity monitor results                   |
+|              |                                                              |
+|              | Result: IP connectivity monitor should not have any packet   |
+|              | drop failures reported                                       |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|test verdict  | This test fails if the SLAs are not met or if there is a     |
+|              | test case execution problem. The SLAs are define as follows  |
+|              | for this test:                                               |
+|              |  * SDN Controller recovery                                   |
+|              |    * process_recover_time <= 30 sec                          |
+|              |                                                              |
+|              |  * no impact on data plane connectivity during SDN           |
+|              |    controller failure and recovery.                          |
+|              |    * packet_drop == 0                                        |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+