aboutsummaryrefslogtreecommitdiffstats
path: root/docs/testing/user
diff options
context:
space:
mode:
Diffstat (limited to 'docs/testing/user')
-rw-r--r--docs/testing/user/userguide/13-nsb-installation.rst94
-rw-r--r--docs/testing/user/userguide/14-nsb-operation.rst36
-rw-r--r--docs/testing/user/userguide/15-list-of-tcs.rst2
-rw-r--r--docs/testing/user/userguide/code/pod_ixia.yaml31
-rw-r--r--docs/testing/user/userguide/opnfv_yardstick_tc050.rst52
-rw-r--r--docs/testing/user/userguide/opnfv_yardstick_tc088.rst129
-rw-r--r--docs/testing/user/userguide/opnfv_yardstick_tc089.rst129
-rw-r--r--docs/testing/user/userguide/opnfv_yardstick_tc092.rst196
-rw-r--r--docs/testing/user/userguide/opnfv_yardstick_tc093.rst184
9 files changed, 751 insertions, 102 deletions
diff --git a/docs/testing/user/userguide/13-nsb-installation.rst b/docs/testing/user/userguide/13-nsb-installation.rst
index 00f8cfd97..3e0ed0bfb 100644
--- a/docs/testing/user/userguide/13-nsb-installation.rst
+++ b/docs/testing/user/userguide/13-nsb-installation.rst
@@ -135,6 +135,15 @@ Ansible:
ansible_user=root
ansible_pass=root
+.. note::
+
+ SSH access without password needs to be configured for all your nodes defined in
+ ``yardstick-install-inventory.ini`` file.
+ If you want to use password authentication you need to install sshpass
+
+ .. code-block:: console
+
+ sudo -EH apt-get install sshpass
To execute an installation for a Bare-Metal or a Standalone context:
@@ -1049,40 +1058,8 @@ IxLoad
Config ``pod_ixia.yaml``
- .. code-block:: yaml
-
- nodes:
- -
- name: trafficgen_1
- role: IxNet
- ip: 1.2.1.1 #ixia machine ip
- user: user
- password: r00t
- key_filename: /root/.ssh/id_rsa
- tg_config:
- ixchassis: "1.2.1.7" #ixia chassis ip
- tcl_port: "8009" # tcl server port
- lib_path: "/opt/ixia/ixos-api/8.01.0.2/lib/ixTcl1.0"
- root_dir: "/opt/ixia/ixos-api/8.01.0.2/"
- py_bin_path: "/opt/ixia/ixload/8.01.106.3/bin/"
- py_lib_path: "/opt/ixia/ixnetwork/8.01.1029.14/lib/PythonApi"
- dut_result_dir: "/mnt/ixia"
- version: 8.1
- interfaces:
- xe0: # logical name from topology.yaml and vnfd.yaml
- vpci: "2:5" # Card:port
- driver: "none"
- dpdk_port_num: 0
- local_ip: "152.16.100.20"
- netmask: "255.255.0.0"
- local_mac: "00:98:10:64:14:00"
- xe1: # logical name from topology.yaml and vnfd.yaml
- vpci: "2:6" # [(Card, port)]
- driver: "none"
- dpdk_port_num: 1
- local_ip: "152.40.40.20"
- netmask: "255.255.0.0"
- local_mac: "00:98:28:28:14:00"
+ .. literalinclude:: code/pod_ixia.yaml
+ :language: console
for sriov/ovs_dpdk pod files, please refer to above Standalone Virtualization for ovs-dpdk/sriov configuration
@@ -1104,10 +1081,10 @@ IxLoad
IxNetwork
---------
-1. Software needed: ``IxNetworkAPI<ixnetwork verson>Linux64.bin.tgz``
- (Download from ixia support site)
- Install - ``IxNetworkAPI<ixnetwork verson>Linux64.bin.tgz``
-2. Update pod_ixia.yaml file with ixia details.
+IxNetwork testcases use IxNetwork API Python Bindings module, which is
+installed as part of the requirements of the project.
+
+1. Update ``pod_ixia.yaml`` file with ixia details.
.. code-block:: console
@@ -1115,44 +1092,12 @@ IxNetwork
Config pod_ixia.yaml
- .. code-block:: yaml
-
- nodes:
- -
- name: trafficgen_1
- role: IxNet
- ip: 1.2.1.1 #ixia machine ip
- user: user
- password: r00t
- key_filename: /root/.ssh/id_rsa
- tg_config:
- ixchassis: "1.2.1.7" #ixia chassis ip
- tcl_port: "8009" # tcl server port
- lib_path: "/opt/ixia/ixos-api/8.01.0.2/lib/ixTcl1.0"
- root_dir: "/opt/ixia/ixos-api/8.01.0.2/"
- py_bin_path: "/opt/ixia/ixload/8.01.106.3/bin/"
- py_lib_path: "/opt/ixia/ixnetwork/8.01.1029.14/lib/PythonApi"
- dut_result_dir: "/mnt/ixia"
- version: 8.1
- interfaces:
- xe0: # logical name from topology.yaml and vnfd.yaml
- vpci: "2:5" # Card:port
- driver: "none"
- dpdk_port_num: 0
- local_ip: "152.16.100.20"
- netmask: "255.255.0.0"
- local_mac: "00:98:10:64:14:00"
- xe1: # logical name from topology.yaml and vnfd.yaml
- vpci: "2:6" # [(Card, port)]
- driver: "none"
- dpdk_port_num: 1
- local_ip: "152.40.40.20"
- netmask: "255.255.0.0"
- local_mac: "00:98:28:28:14:00"
+ .. literalinclude:: code/pod_ixia.yaml
+ :language: console
for sriov/ovs_dpdk pod files, please refer to above Standalone Virtualization for ovs-dpdk/sriov configuration
-3. Start IxNetwork TCL Server
+2. Start IxNetwork TCL Server
You will also need to configure the IxNetwork machine to start the IXIA
IxNetworkTclServer. This can be started like so:
@@ -1161,6 +1106,5 @@ IxNetwork
``Start->Programs->Ixia->IxNetwork->IxNetwork 7.21.893.14 GA->IxNetworkTclServer``
(or ``IxNetworkApiServer``)
-4. Execute testcase in samplevnf folder e.g.
+3. Execute testcase in samplevnf folder e.g.
``<repo>/samples/vnf_samples/nsut/vfw/tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml``
-
diff --git a/docs/testing/user/userguide/14-nsb-operation.rst b/docs/testing/user/userguide/14-nsb-operation.rst
index 250f0a8e1..851c6528e 100644
--- a/docs/testing/user/userguide/14-nsb-operation.rst
+++ b/docs/testing/user/userguide/14-nsb-operation.rst
@@ -423,3 +423,39 @@ options section.
options:
tg_0:
queues_per_port: 2
+
+
+Standalone configuration
+------------------------
+
+NSB supports certain Standalone deployment configurations.
+Standalone supports provisioning a VM in a standalone visualised environment using kvm/qemu.
+There two types of Standalone contexts available: OVS-DPDK and SRIOV.
+OVS-DPDK uses OVS network with DPDK drivers.
+SRIOV enables network traffic to bypass the software switch layer of the Hyper-V stack.
+
+Standalone with OVS-DPDK
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+SampleVNF image is spawned in a VM on a baremetal server.
+OVS with DPDK is installed on the baremetal server.
+
+.. note:: Ubuntu 17.10 requires DPDK v.17.05 and higher, DPDK v.17.05 requires OVS v.2.8.0.
+
+Default values for OVS-DPDK:
+
+ * queues: 4
+ * lcore_mask: ""
+ * pmd_cpu_mask: "0x6"
+
+Sample test case file
+^^^^^^^^^^^^^^^^^^^^^
+
+ 1. Prepare SampleVNF image and copy it to ``flavor/images``.
+ 2. Prepare context files for TREX and SampleVNF under ``contexts/file``.
+ 3. Add bridge named ``br-int`` to the baremetal where SampleVNF image is deployed.
+ 4. Modify ``networks/phy_port`` accordingly to the baremetal setup.
+ 5. Run test from:
+
+.. literalinclude:: /submodules/yardstick/samples/vnf_samples/nsut/acl/tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml
+ :language: yaml
diff --git a/docs/testing/user/userguide/15-list-of-tcs.rst b/docs/testing/user/userguide/15-list-of-tcs.rst
index 678f0f9a9..37ce819f1 100644
--- a/docs/testing/user/userguide/15-list-of-tcs.rst
+++ b/docs/testing/user/userguide/15-list-of-tcs.rst
@@ -84,6 +84,8 @@ H A
opnfv_yardstick_tc057.rst
opnfv_yardstick_tc058.rst
opnfv_yardstick_tc087.rst
+ opnfv_yardstick_tc092.rst
+ opnfv_yardstick_tc093.rst
IPv6
----
diff --git a/docs/testing/user/userguide/code/pod_ixia.yaml b/docs/testing/user/userguide/code/pod_ixia.yaml
new file mode 100644
index 000000000..4ab56fe4e
--- /dev/null
+++ b/docs/testing/user/userguide/code/pod_ixia.yaml
@@ -0,0 +1,31 @@
+nodes:
+-
+ name: trafficgen_1
+ role: IxNet
+ ip: 1.2.1.1 #ixia machine ip
+ user: user
+ password: r00t
+ key_filename: /root/.ssh/id_rsa
+ tg_config:
+ ixchassis: "1.2.1.7" #ixia chassis ip
+ tcl_port: "8009" # tcl server port
+ lib_path: "/opt/ixia/ixos-api/8.01.0.2/lib/ixTcl1.0"
+ root_dir: "/opt/ixia/ixos-api/8.01.0.2/"
+ py_bin_path: "/opt/ixia/ixload/8.01.106.3/bin/"
+ dut_result_dir: "/mnt/ixia"
+ version: 8.1
+ interfaces:
+ xe0: # logical name from topology.yaml and vnfd.yaml
+ vpci: "2:5" # Card:port
+ driver: "none"
+ dpdk_port_num: 0
+ local_ip: "152.16.100.20"
+ netmask: "255.255.0.0"
+ local_mac: "00:98:10:64:14:00"
+ xe1: # logical name from topology.yaml and vnfd.yaml
+ vpci: "2:6" # [(Card, port)]
+ driver: "none"
+ dpdk_port_num: 1
+ local_ip: "152.40.40.20"
+ netmask: "255.255.0.0"
+ local_mac: "00:98:28:28:14:00"
diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc050.rst b/docs/testing/user/userguide/opnfv_yardstick_tc050.rst
index 8890c9d53..82a491b72 100644
--- a/docs/testing/user/userguide/opnfv_yardstick_tc050.rst
+++ b/docs/testing/user/userguide/opnfv_yardstick_tc050.rst
@@ -34,23 +34,20 @@ Yardstick Test Case Description TC050
| | 2) host: which is the name of a control node being attacked. |
| | 3) interface: the network interface to be turned off. |
| | |
-| | There are four instance of the "close-interface" monitor: |
-| | attacker1(for public netork): |
-| | -fault_type: "close-interface" |
-| | -host: node1 |
-| | -interface: "br-ex" |
-| | attacker2(for management netork): |
-| | -fault_type: "close-interface" |
-| | -host: node1 |
-| | -interface: "br-mgmt" |
-| | attacker3(for storage netork): |
-| | -fault_type: "close-interface" |
-| | -host: node1 |
-| | -interface: "br-storage" |
-| | attacker4(for private netork): |
-| | -fault_type: "close-interface" |
-| | -host: node1 |
-| | -interface: "br-mesh" |
+| | The interface to be closed by the attacker can be set by the |
+| | variable of "{{ interface_name }}" |
+| | |
+| | attackers: |
+| | - |
+| | fault_type: "general-attacker" |
+| | host: {{ attack_host }} |
+| | key: "close-br-public" |
+| | attack_key: "close-interface" |
+| | action_parameter: |
+| | interface: {{ interface_name }} |
+| | rollback_parameter: |
+| | interface: {{ interface_name }} |
+| | |
+--------------+--------------------------------------------------------------+
|monitors | In this test case, the monitor named "openstack-cmd" is |
| | needed. The monitor needs needs two parameters: |
@@ -61,17 +58,17 @@ Yardstick Test Case Description TC050
| | |
| | There are four instance of the "openstack-cmd" monitor: |
| | monitor1: |
-| | -monitor_type: "openstack-cmd" |
-| | -command_name: "nova image-list" |
+| | - monitor_type: "openstack-cmd" |
+| | - command_name: "nova image-list" |
| | monitor2: |
-| | -monitor_type: "openstack-cmd" |
-| | -command_name: "neutron router-list" |
+| | - monitor_type: "openstack-cmd" |
+| | - command_name: "neutron router-list" |
| | monitor3: |
-| | -monitor_type: "openstack-cmd" |
-| | -command_name: "heat stack-list" |
+| | - monitor_type: "openstack-cmd" |
+| | - command_name: "heat stack-list" |
| | monitor4: |
-| | -monitor_type: "openstack-cmd" |
-| | -command_name: "cinder list" |
+| | - monitor_type: "openstack-cmd" |
+| | - command_name: "cinder list" |
+--------------+--------------------------------------------------------------+
|metrics | In this test case, there is one metric: |
| | 1)service_outage_time: which indicates the maximum outage |
@@ -109,9 +106,9 @@ Yardstick Test Case Description TC050
+--------------+--------------------------------------------------------------+
|step 2 | do attacker: connect the host through SSH, and then execute |
| | the turnoff network interface script with param value |
-| | specified by "interface". |
+| | specified by "{{ interface_name }}". |
| | |
-| | Result: Network interfaces will be turned down. |
+| | Result: The specified network interface will be down. |
| | |
+--------------+--------------------------------------------------------------+
|step 3 | stop monitors after a period of time specified by |
@@ -133,3 +130,4 @@ Yardstick Test Case Description TC050
| | execution problem. |
| | |
+--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc088.rst b/docs/testing/user/userguide/opnfv_yardstick_tc088.rst
new file mode 100644
index 000000000..2423a6b31
--- /dev/null
+++ b/docs/testing/user/userguide/opnfv_yardstick_tc088.rst
@@ -0,0 +1,129 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Yin Kanglin and others.
+.. 14_ykl@tongji.edu.cn
+
+*************************************
+Yardstick Test Case Description TC088
+*************************************
+
++-----------------------------------------------------------------------------+
+|Control Node Openstack Service High Availability - Nova Scheduler |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC088: Control node Openstack service down - |
+| | nova scheduler |
++--------------+--------------------------------------------------------------+
+|test purpose | This test case will verify the high availability of the |
+| | compute scheduler service provided by OpenStack (nova- |
+| | scheduler) on control node. |
+| | |
++--------------+--------------------------------------------------------------+
+|test method | This test case kills the processes of nova-scheduler service |
+| | on a selected control node, then checks whether the request |
+| | of the related OpenStack command is OK and the killed |
+| | processes are recovered. |
+| | |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, an attacker called "kill-process" is |
+| | needed. This attacker includes three parameters: |
+| | 1) fault_type: which is used for finding the attacker's |
+| | scripts. It should be always set to "kill-process" in this |
+| | test case. |
+| | 2) process_name: which is the process name of the specified |
+| | OpenStack service. If there are multiple processes use the |
+| | same name on the host, all of them are killed by this |
+| | attacker. |
+| | In this case. This parameter should always set to "nova- |
+| | scheduler". |
+| | 3) host: which is the name of a control node being attacked. |
+| | |
+| | e.g. |
+| | -fault_type: "kill-process" |
+| | -process_name: "nova-scheduler" |
+| | -host: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|monitors | In this test case, one kind of monitor is needed: |
+| | 1. the "process" monitor check whether a process is running |
+| | on a specific node, which needs three parameters: |
+| | 1) monitor_type: which used for finding the monitor class and|
+| | related scripts. It should be always set to "process" |
+| | for this monitor. |
+| | 2) process_name: which is the process name for monitor |
+| | 3) host: which is the name of the node running the process |
+| | |
+| | e.g. |
+| | monitor: |
+| | -monitor_type: "process" |
+| | -process_name: "nova-scheduler" |
+| | -host: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|operations | In this test case, the following operations are needed: |
+| | 1. "nova-create-instance": create a VM instance to check |
+| | whether the nova-scheduler works normally. |
+| | |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there are one metric: |
+| | 1)process_recover_time: which indicates the maximum time |
+| | (seconds) from the process being killed to recovered |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI NFV REL001 |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1) test case file: opnfv_yardstick_tc088.yaml |
+| | -Attackers: see above "attackers" description |
+| | -waiting_time: which is the time (seconds) from the process |
+| | being killed to stopping monitors the monitors |
+| | -Monitors: see above "monitors" description |
+| | -SLA: see above "metrics" description |
+| | |
+| | 2)POD file: pod.yaml |
+| | The POD configuration should record on pod.yaml first. |
+| | the "host" item in this test case will use the node name in |
+| | the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | do attacker: connect the host through SSH, and then execute |
+| | the kill process script with param value specified by |
+| | "process_name" |
+| | |
+| | Result: Process will be killed. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | start monitors: |
+| | each monitor will run with independently process |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | create a new instance to check whether the nova scheduler |
+| | works normally. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | stop the monitor after a period of time specified by |
+| | "waiting_time" |
+| | |
+| | Result: The monitor info will be aggregated. |
+| | |
++--------------+--------------------------------------------------------------+
+|post-action | It is the action when the test cases exist. It will check the|
+| | status of the specified process on the host, and restart the |
+| | process if it is not running for next test cases |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc089.rst b/docs/testing/user/userguide/opnfv_yardstick_tc089.rst
new file mode 100644
index 000000000..0a8b2570b
--- /dev/null
+++ b/docs/testing/user/userguide/opnfv_yardstick_tc089.rst
@@ -0,0 +1,129 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Yin Kanglin and others.
+.. 14_ykl@tongji.edu.cn
+
+*************************************
+Yardstick Test Case Description TC089
+*************************************
+
++-----------------------------------------------------------------------------+
+|Control Node Openstack Service High Availability - Nova Conductor |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC089: Control node Openstack service down - |
+| | nova conductor |
++--------------+--------------------------------------------------------------+
+|test purpose | This test case will verify the high availability of the |
+| | compute database proxy service provided by OpenStack (nova- |
+| | conductor) on control node. |
+| | |
++--------------+--------------------------------------------------------------+
+|test method | This test case kills the processes of nova-conductor service |
+| | on a selected control node, then checks whether the request |
+| | of the related OpenStack command is OK and the killed |
+| | processes are recovered. |
+| | |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, an attacker called "kill-process" is |
+| | needed. This attacker includes three parameters: |
+| | 1) fault_type: which is used for finding the attacker's |
+| | scripts. It should be always set to "kill-process" in this |
+| | test case. |
+| | 2) process_name: which is the process name of the specified |
+| | OpenStack service. If there are multiple processes use the |
+| | same name on the host, all of them are killed by this |
+| | attacker. |
+| | In this case. This parameter should always set to "nova- |
+| | conductor". |
+| | 3) host: which is the name of a control node being attacked. |
+| | |
+| | e.g. |
+| | -fault_type: "kill-process" |
+| | -process_name: "nova-conductor" |
+| | -host: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|monitors | In this test case, one kind of monitor is needed: |
+| | 1. the "process" monitor check whether a process is running |
+| | on a specific node, which needs three parameters: |
+| | 1) monitor_type: which used for finding the monitor class and|
+| | related scripts. It should be always set to "process" |
+| | for this monitor. |
+| | 2) process_name: which is the process name for monitor |
+| | 3) host: which is the name of the node running the process |
+| | |
+| | e.g. |
+| | monitor: |
+| | -monitor_type: "process" |
+| | -process_name: "nova-conductor" |
+| | -host: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|operations | In this test case, the following operations are needed: |
+| | 1. "nova-create-instance": create a VM instance to check |
+| | whether the nova-conductor works normally. |
+| | |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there are one metric: |
+| | 1)process_recover_time: which indicates the maximum time |
+| | (seconds) from the process being killed to recovered |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI NFV REL001 |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1) test case file: opnfv_yardstick_tc089.yaml |
+| | -Attackers: see above "attackers" description |
+| | -waiting_time: which is the time (seconds) from the process |
+| | being killed to stopping monitors the monitors |
+| | -Monitors: see above "monitors" description |
+| | -SLA: see above "metrics" description |
+| | |
+| | 2)POD file: pod.yaml |
+| | The POD configuration should record on pod.yaml first. |
+| | the "host" item in this test case will use the node name in |
+| | the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | do attacker: connect the host through SSH, and then execute |
+| | the kill process script with param value specified by |
+| | "process_name" |
+| | |
+| | Result: Process will be killed. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | start monitors: |
+| | each monitor will run with independently process |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | create a new instance to check whether the nova conductor |
+| | works normally. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | stop the monitor after a period of time specified by |
+| | "waiting_time" |
+| | |
+| | Result: The monitor info will be aggregated. |
+| | |
++--------------+--------------------------------------------------------------+
+|post-action | It is the action when the test cases exist. It will check the|
+| | status of the specified process on the host, and restart the |
+| | process if it is not running for next test cases |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc092.rst b/docs/testing/user/userguide/opnfv_yardstick_tc092.rst
new file mode 100644
index 000000000..895074a85
--- /dev/null
+++ b/docs/testing/user/userguide/opnfv_yardstick_tc092.rst
@@ -0,0 +1,196 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Ericsson and others.
+
+*************************************
+Yardstick Test Case Description TC092
+*************************************
+
++-----------------------------------------------------------------------------+
+|SDN Controller resilience in HA configuration |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC092: SDN controller resilience and high |
+| | availability HA configuration |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | This test validates SDN controller node high availability by |
+| | verifying there is no impact on the data plane connectivity |
+| | when one SDN controller fails in a HA configuration, |
+| | i.e. all existing configured network services DHCP, ARP, L2, |
+| | L3VPN, Security Groups should continue to operate |
+| | between the existing VMs while one SDN controller instance |
+| | is offline and rebooting. |
+| | |
+| | The test also validates that network service operations such |
+| | as creating a new VM in an existing or new L2 network |
+| | network remain operational while one instance of the |
+| | SDN controller is offline and recovers from the failure. |
+| | |
++--------------+--------------------------------------------------------------+
+|test method | This test case: |
+| | 1. fails one instance of a SDN controller cluster running |
+| | in a HA configuration on the OpenStack controller node |
+| | |
+| | 2. checks if already configured L2 connectivity between |
+| | existing VMs is not impacted |
+| | |
+| | 3. verifies that the system never loses the ability to |
+| | execute virtual network operations, even when the |
+| | failed SDN Controller is still recovering |
+| | |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, an attacker called “kill-process” is |
+| | needed. This attacker includes three parameters: |
+| | 1. ``fault_type``: which is used for finding the attacker's |
+| | scripts. It should be set to 'kill-process' in this test |
+| | |
+| | 2. ``process_name``: should be set to sdn controller |
+| | process |
+| | |
+| | 3. ``host``: which is the name of a control node where |
+| | opendaylight process is running |
+| | |
+| | example: |
+| | - ``fault_type``: “kill-process” |
+| | - ``process_name``: “opendaylight-karaf” (TBD) |
+| | - ``host``: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|monitors | In this test case, the following monitors are needed |
+| | 1. ``ping_same_network_l2``: monitor pinging traffic |
+| | between the VMs in same neutron network |
+| | |
+| | 2. ``ping_external_snat``: monitor ping traffic from VMs to |
+| | external destinations (e.g. google.com) |
+| | |
+| | 3. ``SDN controller process monitor``: a monitor checking |
+| | the state of a specified SDN controller process. It |
+| | measures the recovery time of the given process. |
+| | |
++--------------+--------------------------------------------------------------+
+|operations | In this test case, the following operations are needed: |
+| | 1. "nova-create-instance-in_network": create a VM instance |
+| | in one of the existing neutron network. |
+| | |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there are two metrics: |
+| | 1. process_recover_time: which indicates the maximun |
+| | time (seconds) from the process being killed to |
+| | recovered |
+| | |
+| | 2. packet_drop: measure the packets that have been dropped |
+| | by the monitors using pktgen. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | TBD |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1. test case file: opnfv_yardstick_tc092.yaml |
+| | - Attackers: see above “attackers” discription |
+| | - Monitors: see above “monitors” discription |
+| | - waiting_time: which is the time (seconds) from the |
+| | process being killed to stoping monitors the |
+| | monitors |
+| | - SLA: see above “metrics” discription |
+| | |
+| | 2. POD file: pod.yaml The POD configuration should record |
+| | on pod.yaml first. the “host” item in this test case |
+| | will use the node name in the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | Description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-action | 1. The OpenStack cluster is set up with an SDN controller |
+| | running in a three node cluster configuration. |
+| | |
+| | 2. One or more neutron networks are created with two or |
+| | more VMs attached to each of the neutron networks. |
+| | |
+| | 3. The neutron networks are attached to a neutron router |
+| | which is attached to an external network the towards |
+| | DCGW. |
+| | |
+| | 4. The master node of SDN controller cluster is known. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | Start ip connectivity monitors: |
+| | 1. Check the L2 connectivity between the VMs in the same |
+| | neutron network. |
+| | |
+| | 2. Check the external connectivity of the VMs. |
+| | |
+| | Each monitor runs in an independent process. |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Start attacker: |
+| | SSH to the VIM node and kill the SDN controller process |
+| | determined in step 2. |
+| | |
+| | Result: One SDN controller service will be shut down |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | Restart the SDN controller. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | Create a new VM in the existing Neutron network while the |
+| | SDN controller is offline or still recovering. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 5 | Stop IP connectivity monitors after a period of time |
+| | specified by “waiting_time” |
+| | |
+| | Result: The monitor info will be aggregated |
+| | |
++--------------+--------------------------------------------------------------+
+|step 6 | Verify the IP connectivity monitor result |
+| | |
+| | Result: IP connectivity monitor should not have any packet |
+| | drop failures reported |
+| | |
++--------------+--------------------------------------------------------------+
+|step 7 | Verify process_recover_time, which indicates the maximun |
+| | time (seconds) from the process being killed to recovered, |
+| | is within the SLA. This step blocks until either the |
+| | process has recovered or a timeout occurred. |
+| | |
+| | Result: process_recover_time is within SLA limits, if not, |
+| | test case failed and stopped. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 8 | Start IP connectivity monitors for the new VM: |
+| | 1. Check the L2 connectivity from the existing VMs to the |
+| | new VM in the Neutron network. |
+| | |
+| | 2. Check connectivity from one VM to an external host on |
+| | the Internet to verify SNAT functionality. |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 9 | Stop IP connectivity monitors after a period of time |
+| | specified by “waiting_time” |
+| | |
+| | Result: The monitor info will be aggregated |
+| | |
++--------------+--------------------------------------------------------------+
+|step 10 | Verify the IP connectivity monitor result |
+| | |
+| | Result: IP connectivity monitor should not have any packet |
+| | drop failures reported |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc093.rst b/docs/testing/user/userguide/opnfv_yardstick_tc093.rst
new file mode 100644
index 000000000..31fa5d3d3
--- /dev/null
+++ b/docs/testing/user/userguide/opnfv_yardstick_tc093.rst
@@ -0,0 +1,184 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intracom Telecom and others.
+.. mardim@intracom-telecom.com
+
+*************************************
+Yardstick Test Case Description TC093
+*************************************
+
++-----------------------------------------------------------------------------+
+|SDN Vswitch resilience in non-HA or HA configuration |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC093: SDN Vswitch resilience in |
+| | non-HA or HA configuration |
++--------------+--------------------------------------------------------------+
+|test purpose | This test validates that network data plane services are |
+| | resilient in the event of Virtual Switch failure |
+| | in compute nodes. Specifically, the test verifies that |
+| | existing data plane connectivity is not permanently impacted |
+| | i.e. all configured network services such as DHCP, ARP, L2, |
+| | L3 Security Groups continue to operate between the existing |
+| | VMs eventually after the Virtual Switches have finished |
+| | rebooting. |
+| | |
+| | The test also validates that new network service operations |
+| | (creating a new VM in the existing L2/L3 network or in a new |
+| | network, etc.) are operational after the Virtual Switches |
+| | have recovered from a failure. |
+| | |
++--------------+--------------------------------------------------------------+
+|test method | This testcase first checks if the already configured |
+| | DHCP/ARP/L2/L3/SNAT connectivity is proper. After |
+| | it fails and restarts again the VSwitch services which are |
+| | running on both OpenStack compute nodes, and then checks if |
+| | already configured DHCP/ARP/L2/L3/SNAT connectivity is not |
+| | permanently impacted (even if there are some packet |
+| | loss events) between VMs and the system is able to execute |
+| | new virtual network operations once the Vswitch services |
+| | are restarted and have been fully recovered |
+| | |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, two attackers called “kill-process” are |
+| | needed. These attackers include three parameters: |
+| | 1. fault_type: which is used for finding the attacker's |
+| | scripts. It should be set to 'kill-process' in this test |
+| | |
+| | 2. process_name: should be set to the name of the Vswitch |
+| | process |
+| | |
+| | 3. host: which is the name of the compute node where the |
+| | Vswitch process is running |
+| | |
+| | e.g. -fault_type: "kill-process" |
+| | -process_name: "openvswitch" |
+| | -host: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|monitors | This test case utilizes two monitors of type "ip-status" |
+| | and one monitor of type "process" to track the following |
+| | conditions: |
+| | 1. "ping_same_network_l2": monitor ICMP traffic between |
+| | VMs in the same Neutron network |
+| | |
+| | 2. "ping_external_snat": monitor ICMP traffic from VMs to |
+| | an external host on the Internet to verify SNAT |
+| | functionality. |
+| | |
+| | 3. "Vswitch process monitor": a monitor checking the |
+| | state of the specified Vswitch process. It measures |
+| | the recovery time of the given process. |
+| | |
+| | Monitors of type "ip-status" use the "ping" utility to |
+| | verify reachability of a given target IP. |
+| | |
++--------------+--------------------------------------------------------------+
+|operations | In this test case, the following operations are needed: |
+| | 1. "nova-create-instance-in_network": create a VM instance |
+| | in one of the existing Neutron network. |
+| | |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there are two metrics: |
+| | 1. process_recover_time: which indicates the maximun |
+| | time (seconds) from the process being killed to |
+| | recovered |
+| | |
+| | 2. outage_time: measures the total time in which |
+| | monitors were failing in their tasks (e.g. total time of |
+| | Ping failure) |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | none |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1. test case file: opnfv_yardstick_tc093.yaml |
+| | - Attackers: see above “attackers” description |
+| | - monitor_time: which is the time (seconds) from |
+| | starting to stoping the monitors |
+| | - Monitors: see above “monitors” discription |
+| | - SLA: see above “metrics” description |
+| | |
+| | 2. POD file: pod.yaml The POD configuration should record |
+| | on pod.yaml first. the “host” item in this test case |
+| | will use the node name in the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | Description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-action | 1. The Vswitches are set up in both compute nodes. |
+| | |
+| | 2. One or more Neutron networks are created with two or |
+| | more VMs attached to each of the Neutron networks. |
+| | |
+| | 3. The Neutron networks are attached to a Neutron router |
+| | which is attached to an external network towards the |
+| | DCGW. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | Start IP connectivity monitors: |
+| | 1. Check the L2 connectivity between the VMs in the same |
+| | Neutron network. |
+| | |
+| | 2. Check connectivity from one VM to an external host on |
+| | the Internet to verify SNAT functionality. |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Start attackers: |
+| | SSH connect to the VIM compute nodes and kill the Vswitch |
+| | processes |
+| | |
+| | Result: the SDN Vswitch services will be shutdown |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | Verify the results of the IP connectivity monitors. |
+| | |
+| | Result: The outage_time metric reported by the monitors |
+| | is not greater than the max_outage_time. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | Restart the SDN Vswitch services. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 5 | Create a new VM in the existing Neutron network |
+| | |
++--------------+--------------------------------------------------------------+
+|step 6 | Verify connectivity between VMs as follows: |
+| | 1. Check the L2 connectivity between the previously |
+| | existing VM and the newly created VM on the same |
+| | Neutron network by sending ICMP messages |
+| | |
++--------------+--------------------------------------------------------------+
+|step 7 | Stop IP connectivity monitors after a period of time |
+| | specified by “monitor_time” |
+| | |
+| | Result: The monitor info will be aggregated |
+| | |
++--------------+--------------------------------------------------------------+
+|step 8 | Verify the IP connectivity monitor results |
+| | |
+| | Result: IP connectivity monitor should not have any packet |
+| | drop failures reported |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | This test fails if the SLAs are not met or if there is a |
+| | test case execution problem. The SLAs are define as follows |
+| | for this test: |
+| | * SDN Vswitch recovery |
+| | * process_recover_time <= 30 sec |
+| | |
+| | * no impact on data plane connectivity during SDN |
+| | Vswitch failure and recovery. |
+| | * packet_drop == 0 |
+| | |
++--------------+--------------------------------------------------------------+
+