diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/release/release-notes/release-notes.rst | 41 | ||||
| -rw-r--r-- | docs/testing/user/userguide/14-nsb-operation.rst | 36 | ||||
| -rw-r--r-- | docs/testing/user/userguide/15-list-of-tcs.rst | 2 | ||||
| -rw-r--r-- | docs/testing/user/userguide/opnfv_yardstick_tc088.rst | 129 | ||||
| -rw-r--r-- | docs/testing/user/userguide/opnfv_yardstick_tc089.rst | 129 | ||||
| -rw-r--r-- | docs/testing/user/userguide/opnfv_yardstick_tc092.rst | 196 | ||||
| -rw-r--r-- | docs/testing/user/userguide/opnfv_yardstick_tc093.rst | 184 |
7 files changed, 711 insertions, 6 deletions
diff --git a/docs/release/release-notes/release-notes.rst b/docs/release/release-notes/release-notes.rst index 6598a2751..7ea2616e4 100644 --- a/docs/release/release-notes/release-notes.rst +++ b/docs/release/release-notes/release-notes.rst @@ -36,6 +36,9 @@ Version History | *Date* | *Version* | *Comment* | | | | | +-------------------+-----------+---------------------------------+ +| May 25, 2018 | 6.1.0 | Yardstick for Fraser release | +| | | | ++-------------------+-----------+---------------------------------+ | April 27, 2018 | 6.0.0 | Yardstick for Fraser release | | | | | +-------------------+-----------+---------------------------------+ @@ -117,19 +120,19 @@ Release Data | **Project** | Yardstick | | | | +--------------------------------+-----------------------+ -| **Repo/tag** | yardstick/opnfv-6.0.0 | +| **Repo/tag** | yardstick/opnfv-6.1.0 | | | | +--------------------------------+-----------------------+ -| **Yardstick Docker image tag** | opnfv-6.0.0 | +| **Yardstick Docker image tag** | opnfv-6.1.0 | | | | +--------------------------------+-----------------------+ | **Release designation** | Fraser | | | | +--------------------------------+-----------------------+ -| **Release date** | April 27, 2018 | +| **Release date** | May 25, 2018 | | | | +--------------------------------+-----------------------+ -| **Purpose of the delivery** | OPNFV Fraser 6.0.0 | +| **Purpose of the delivery** | OPNFV Fraser 6.1.0 | | | | +--------------------------------+-----------------------+ @@ -148,7 +151,7 @@ Documents Software Deliverables --------------------- - - The Yardstick Docker image: https://hub.docker.com/r/opnfv/yardstick (tag: opnfv-6.0.0) + - The Yardstick Docker image: https://hub.docker.com/r/opnfv/yardstick (tag: opnfv-6.1.0) List of Contexts ^^^^^^^^^^^^^^^^ @@ -174,7 +177,7 @@ List of Contexts List of Runners ^^^^^^^^^^^^^^^ -Note: Yardstick Fraser 6.0.0 add two new Runners, "Dynamictp" and "Search". +.. note:: Yardstick Fraser 6.0.0 add two new Runners, "Dynamictp" and "Search". +---------------+-------------------------------------------------------+ | **Runner** | **Description** | @@ -271,6 +274,8 @@ List of Scenarios New Test cases -------------- +.. note:: Yardstick Fraser 6.1.0 added two new test cases, "TC092" and "TC093". + * Generic NFVI test cases * OPNFV_YARDSTICK_TCO84 - SPEC CPU 2006 for VM @@ -280,6 +285,8 @@ New Test cases * OPNFV_YARDSTICK_TC087 - SDN Controller resilience in non-HA configuration * OPNFV_YARDSTICK_TC090 - Control node Openstack service down - database instance * OPNFV_YARDSTICK_TC091 - Control node Openstack service down - heat-api + * OPNFV_YARDSTICK_TC092 - SDN Controller resilience in HA configuration + * OPNFV_YARDSTICK_TC093 - SDN Vswitch resilience in non-HA or HA configuration Version Change @@ -384,6 +391,28 @@ Known Issues/Faults Corrected Faults ---------------- +Fraser 6.1.0: + ++--------------------+--------------------------------------------------------------------------+ +| **JIRA REFERENCE** | **DESCRIPTION** | ++====================+==========================================================================+ +| YARDSTICK-995 | Test case spec for SDN Virtual Switch resilience | ++--------------------+--------------------------------------------------------------------------+ +| YARDSTICK-1097 | Add pod.yaml file for APEX installer | ++--------------------+--------------------------------------------------------------------------+ +| YARDSTICK-1122 | Remove unused code in SampleVNF | ++--------------------+--------------------------------------------------------------------------+ +| YARDSTICK-1125 | Update samples/test_suite.yaml | ++--------------------+--------------------------------------------------------------------------+ +| YARDSTICK-1132 | Document for Euphrates test case results | ++--------------------+--------------------------------------------------------------------------+ +| YARDSTICK-1138 | Support Restart Operation | ++--------------------+--------------------------------------------------------------------------+ +| YARDSTICK-1142 | start_service script fails to start openvswitch service in centos distro | ++--------------------+--------------------------------------------------------------------------+ +| YARDSTICK-1165 | Bugfix: openrc api dump should be safe_dump | ++--------------------+--------------------------------------------------------------------------+ + Fraser 6.0.0: +--------------------+--------------------------------------------------------------------------+ diff --git a/docs/testing/user/userguide/14-nsb-operation.rst b/docs/testing/user/userguide/14-nsb-operation.rst index 2e741822e..d157914a9 100644 --- a/docs/testing/user/userguide/14-nsb-operation.rst +++ b/docs/testing/user/userguide/14-nsb-operation.rst @@ -313,3 +313,39 @@ options section. options: tg_0: queues_per_port: 2 + + +Standalone configuration +------------------------ + +NSB supports certain Standalone deployment configurations. +Standalone supports provisioning a VM in a standalone visualised environment using kvm/qemu. +There two types of Standalone contexts available: OVS-DPDK and SRIOV. +OVS-DPDK uses OVS network with DPDK drivers. +SRIOV enables network traffic to bypass the software switch layer of the Hyper-V stack. + +Standalone with OVS-DPDK +^^^^^^^^^^^^^^^^^^^^^^^^ + +SampleVNF image is spawned in a VM on a baremetal server. +OVS with DPDK is installed on the baremetal server. + +.. note:: Ubuntu 17.10 requires DPDK v.17.05 and higher, DPDK v.17.05 requires OVS v.2.8.0. + +Default values for OVS-DPDK: + + * queues: 4 + * lcore_mask: "" + * pmd_cpu_mask: "0x6" + +Sample test case file +^^^^^^^^^^^^^^^^^^^^^ + + 1. Prepare SampleVNF image and copy it to ``flavor/images``. + 2. Prepare context files for TREX and SampleVNF under ``contexts/file``. + 3. Add bridge named ``br-int`` to the baremetal where SampleVNF image is deployed. + 4. Modify ``networks/phy_port`` accordingly to the baremetal setup. + 5. Run test from: + +.. literalinclude:: /submodules/yardstick/samples/vnf_samples/nsut/acl/tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml + :language: yaml diff --git a/docs/testing/user/userguide/15-list-of-tcs.rst b/docs/testing/user/userguide/15-list-of-tcs.rst index 678f0f9a9..37ce819f1 100644 --- a/docs/testing/user/userguide/15-list-of-tcs.rst +++ b/docs/testing/user/userguide/15-list-of-tcs.rst @@ -84,6 +84,8 @@ H A opnfv_yardstick_tc057.rst opnfv_yardstick_tc058.rst opnfv_yardstick_tc087.rst + opnfv_yardstick_tc092.rst + opnfv_yardstick_tc093.rst IPv6 ---- diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc088.rst b/docs/testing/user/userguide/opnfv_yardstick_tc088.rst new file mode 100644 index 000000000..2423a6b31 --- /dev/null +++ b/docs/testing/user/userguide/opnfv_yardstick_tc088.rst @@ -0,0 +1,129 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Yin Kanglin and others. +.. 14_ykl@tongji.edu.cn + +************************************* +Yardstick Test Case Description TC088 +************************************* + ++-----------------------------------------------------------------------------+ +|Control Node Openstack Service High Availability - Nova Scheduler | +| | ++--------------+--------------------------------------------------------------+ +|test case id | OPNFV_YARDSTICK_TC088: Control node Openstack service down - | +| | nova scheduler | ++--------------+--------------------------------------------------------------+ +|test purpose | This test case will verify the high availability of the | +| | compute scheduler service provided by OpenStack (nova- | +| | scheduler) on control node. | +| | | ++--------------+--------------------------------------------------------------+ +|test method | This test case kills the processes of nova-scheduler service | +| | on a selected control node, then checks whether the request | +| | of the related OpenStack command is OK and the killed | +| | processes are recovered. | +| | | ++--------------+--------------------------------------------------------------+ +|attackers | In this test case, an attacker called "kill-process" is | +| | needed. This attacker includes three parameters: | +| | 1) fault_type: which is used for finding the attacker's | +| | scripts. It should be always set to "kill-process" in this | +| | test case. | +| | 2) process_name: which is the process name of the specified | +| | OpenStack service. If there are multiple processes use the | +| | same name on the host, all of them are killed by this | +| | attacker. | +| | In this case. This parameter should always set to "nova- | +| | scheduler". | +| | 3) host: which is the name of a control node being attacked. | +| | | +| | e.g. | +| | -fault_type: "kill-process" | +| | -process_name: "nova-scheduler" | +| | -host: node1 | +| | | ++--------------+--------------------------------------------------------------+ +|monitors | In this test case, one kind of monitor is needed: | +| | 1. the "process" monitor check whether a process is running | +| | on a specific node, which needs three parameters: | +| | 1) monitor_type: which used for finding the monitor class and| +| | related scripts. It should be always set to "process" | +| | for this monitor. | +| | 2) process_name: which is the process name for monitor | +| | 3) host: which is the name of the node running the process | +| | | +| | e.g. | +| | monitor: | +| | -monitor_type: "process" | +| | -process_name: "nova-scheduler" | +| | -host: node1 | +| | | ++--------------+--------------------------------------------------------------+ +|operations | In this test case, the following operations are needed: | +| | 1. "nova-create-instance": create a VM instance to check | +| | whether the nova-scheduler works normally. | +| | | ++--------------+--------------------------------------------------------------+ +|metrics | In this test case, there are one metric: | +| | 1)process_recover_time: which indicates the maximum time | +| | (seconds) from the process being killed to recovered | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | Developed by the project. Please see folder: | +| | "yardstick/benchmark/scenarios/availability/ha_tools" | +| | | ++--------------+--------------------------------------------------------------+ +|references | ETSI NFV REL001 | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | This test case needs two configuration files: | +| | 1) test case file: opnfv_yardstick_tc088.yaml | +| | -Attackers: see above "attackers" description | +| | -waiting_time: which is the time (seconds) from the process | +| | being killed to stopping monitors the monitors | +| | -Monitors: see above "monitors" description | +| | -SLA: see above "metrics" description | +| | | +| | 2)POD file: pod.yaml | +| | The POD configuration should record on pod.yaml first. | +| | the "host" item in this test case will use the node name in | +| | the pod.yaml. | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | do attacker: connect the host through SSH, and then execute | +| | the kill process script with param value specified by | +| | "process_name" | +| | | +| | Result: Process will be killed. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | start monitors: | +| | each monitor will run with independently process | +| | | +| | Result: The monitor info will be collected. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | create a new instance to check whether the nova scheduler | +| | works normally. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | stop the monitor after a period of time specified by | +| | "waiting_time" | +| | | +| | Result: The monitor info will be aggregated. | +| | | ++--------------+--------------------------------------------------------------+ +|post-action | It is the action when the test cases exist. It will check the| +| | status of the specified process on the host, and restart the | +| | process if it is not running for next test cases | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | Fails only if SLA is not passed, or if there is a test case | +| | execution problem. | +| | | ++--------------+--------------------------------------------------------------+ diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc089.rst b/docs/testing/user/userguide/opnfv_yardstick_tc089.rst new file mode 100644 index 000000000..0a8b2570b --- /dev/null +++ b/docs/testing/user/userguide/opnfv_yardstick_tc089.rst @@ -0,0 +1,129 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Yin Kanglin and others. +.. 14_ykl@tongji.edu.cn + +************************************* +Yardstick Test Case Description TC089 +************************************* + ++-----------------------------------------------------------------------------+ +|Control Node Openstack Service High Availability - Nova Conductor | +| | ++--------------+--------------------------------------------------------------+ +|test case id | OPNFV_YARDSTICK_TC089: Control node Openstack service down - | +| | nova conductor | ++--------------+--------------------------------------------------------------+ +|test purpose | This test case will verify the high availability of the | +| | compute database proxy service provided by OpenStack (nova- | +| | conductor) on control node. | +| | | ++--------------+--------------------------------------------------------------+ +|test method | This test case kills the processes of nova-conductor service | +| | on a selected control node, then checks whether the request | +| | of the related OpenStack command is OK and the killed | +| | processes are recovered. | +| | | ++--------------+--------------------------------------------------------------+ +|attackers | In this test case, an attacker called "kill-process" is | +| | needed. This attacker includes three parameters: | +| | 1) fault_type: which is used for finding the attacker's | +| | scripts. It should be always set to "kill-process" in this | +| | test case. | +| | 2) process_name: which is the process name of the specified | +| | OpenStack service. If there are multiple processes use the | +| | same name on the host, all of them are killed by this | +| | attacker. | +| | In this case. This parameter should always set to "nova- | +| | conductor". | +| | 3) host: which is the name of a control node being attacked. | +| | | +| | e.g. | +| | -fault_type: "kill-process" | +| | -process_name: "nova-conductor" | +| | -host: node1 | +| | | ++--------------+--------------------------------------------------------------+ +|monitors | In this test case, one kind of monitor is needed: | +| | 1. the "process" monitor check whether a process is running | +| | on a specific node, which needs three parameters: | +| | 1) monitor_type: which used for finding the monitor class and| +| | related scripts. It should be always set to "process" | +| | for this monitor. | +| | 2) process_name: which is the process name for monitor | +| | 3) host: which is the name of the node running the process | +| | | +| | e.g. | +| | monitor: | +| | -monitor_type: "process" | +| | -process_name: "nova-conductor" | +| | -host: node1 | +| | | ++--------------+--------------------------------------------------------------+ +|operations | In this test case, the following operations are needed: | +| | 1. "nova-create-instance": create a VM instance to check | +| | whether the nova-conductor works normally. | +| | | ++--------------+--------------------------------------------------------------+ +|metrics | In this test case, there are one metric: | +| | 1)process_recover_time: which indicates the maximum time | +| | (seconds) from the process being killed to recovered | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | Developed by the project. Please see folder: | +| | "yardstick/benchmark/scenarios/availability/ha_tools" | +| | | ++--------------+--------------------------------------------------------------+ +|references | ETSI NFV REL001 | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | This test case needs two configuration files: | +| | 1) test case file: opnfv_yardstick_tc089.yaml | +| | -Attackers: see above "attackers" description | +| | -waiting_time: which is the time (seconds) from the process | +| | being killed to stopping monitors the monitors | +| | -Monitors: see above "monitors" description | +| | -SLA: see above "metrics" description | +| | | +| | 2)POD file: pod.yaml | +| | The POD configuration should record on pod.yaml first. | +| | the "host" item in this test case will use the node name in | +| | the pod.yaml. | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | do attacker: connect the host through SSH, and then execute | +| | the kill process script with param value specified by | +| | "process_name" | +| | | +| | Result: Process will be killed. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | start monitors: | +| | each monitor will run with independently process | +| | | +| | Result: The monitor info will be collected. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | create a new instance to check whether the nova conductor | +| | works normally. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | stop the monitor after a period of time specified by | +| | "waiting_time" | +| | | +| | Result: The monitor info will be aggregated. | +| | | ++--------------+--------------------------------------------------------------+ +|post-action | It is the action when the test cases exist. It will check the| +| | status of the specified process on the host, and restart the | +| | process if it is not running for next test cases | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | Fails only if SLA is not passed, or if there is a test case | +| | execution problem. | +| | | ++--------------+--------------------------------------------------------------+ diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc092.rst b/docs/testing/user/userguide/opnfv_yardstick_tc092.rst new file mode 100644 index 000000000..895074a85 --- /dev/null +++ b/docs/testing/user/userguide/opnfv_yardstick_tc092.rst @@ -0,0 +1,196 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Ericsson and others. + +************************************* +Yardstick Test Case Description TC092 +************************************* + ++-----------------------------------------------------------------------------+ +|SDN Controller resilience in HA configuration | +| | ++--------------+--------------------------------------------------------------+ +|test case id | OPNFV_YARDSTICK_TC092: SDN controller resilience and high | +| | availability HA configuration | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | This test validates SDN controller node high availability by | +| | verifying there is no impact on the data plane connectivity | +| | when one SDN controller fails in a HA configuration, | +| | i.e. all existing configured network services DHCP, ARP, L2, | +| | L3VPN, Security Groups should continue to operate | +| | between the existing VMs while one SDN controller instance | +| | is offline and rebooting. | +| | | +| | The test also validates that network service operations such | +| | as creating a new VM in an existing or new L2 network | +| | network remain operational while one instance of the | +| | SDN controller is offline and recovers from the failure. | +| | | ++--------------+--------------------------------------------------------------+ +|test method | This test case: | +| | 1. fails one instance of a SDN controller cluster running | +| | in a HA configuration on the OpenStack controller node | +| | | +| | 2. checks if already configured L2 connectivity between | +| | existing VMs is not impacted | +| | | +| | 3. verifies that the system never loses the ability to | +| | execute virtual network operations, even when the | +| | failed SDN Controller is still recovering | +| | | ++--------------+--------------------------------------------------------------+ +|attackers | In this test case, an attacker called “kill-process” is | +| | needed. This attacker includes three parameters: | +| | 1. ``fault_type``: which is used for finding the attacker's | +| | scripts. It should be set to 'kill-process' in this test | +| | | +| | 2. ``process_name``: should be set to sdn controller | +| | process | +| | | +| | 3. ``host``: which is the name of a control node where | +| | opendaylight process is running | +| | | +| | example: | +| | - ``fault_type``: “kill-process” | +| | - ``process_name``: “opendaylight-karaf” (TBD) | +| | - ``host``: node1 | +| | | ++--------------+--------------------------------------------------------------+ +|monitors | In this test case, the following monitors are needed | +| | 1. ``ping_same_network_l2``: monitor pinging traffic | +| | between the VMs in same neutron network | +| | | +| | 2. ``ping_external_snat``: monitor ping traffic from VMs to | +| | external destinations (e.g. google.com) | +| | | +| | 3. ``SDN controller process monitor``: a monitor checking | +| | the state of a specified SDN controller process. It | +| | measures the recovery time of the given process. | +| | | ++--------------+--------------------------------------------------------------+ +|operations | In this test case, the following operations are needed: | +| | 1. "nova-create-instance-in_network": create a VM instance | +| | in one of the existing neutron network. | +| | | ++--------------+--------------------------------------------------------------+ +|metrics | In this test case, there are two metrics: | +| | 1. process_recover_time: which indicates the maximun | +| | time (seconds) from the process being killed to | +| | recovered | +| | | +| | 2. packet_drop: measure the packets that have been dropped | +| | by the monitors using pktgen. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | Developed by the project. Please see folder: | +| | "yardstick/benchmark/scenarios/availability/ha_tools" | +| | | ++--------------+--------------------------------------------------------------+ +|references | TBD | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | This test case needs two configuration files: | +| | 1. test case file: opnfv_yardstick_tc092.yaml | +| | - Attackers: see above “attackers” discription | +| | - Monitors: see above “monitors” discription | +| | - waiting_time: which is the time (seconds) from the | +| | process being killed to stoping monitors the | +| | monitors | +| | - SLA: see above “metrics” discription | +| | | +| | 2. POD file: pod.yaml The POD configuration should record | +| | on pod.yaml first. the “host” item in this test case | +| | will use the node name in the pod.yaml. | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | Description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|pre-action | 1. The OpenStack cluster is set up with an SDN controller | +| | running in a three node cluster configuration. | +| | | +| | 2. One or more neutron networks are created with two or | +| | more VMs attached to each of the neutron networks. | +| | | +| | 3. The neutron networks are attached to a neutron router | +| | which is attached to an external network the towards | +| | DCGW. | +| | | +| | 4. The master node of SDN controller cluster is known. | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | Start ip connectivity monitors: | +| | 1. Check the L2 connectivity between the VMs in the same | +| | neutron network. | +| | | +| | 2. Check the external connectivity of the VMs. | +| | | +| | Each monitor runs in an independent process. | +| | | +| | Result: The monitor info will be collected. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Start attacker: | +| | SSH to the VIM node and kill the SDN controller process | +| | determined in step 2. | +| | | +| | Result: One SDN controller service will be shut down | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | Restart the SDN controller. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | Create a new VM in the existing Neutron network while the | +| | SDN controller is offline or still recovering. | +| | | ++--------------+--------------------------------------------------------------+ +|step 5 | Stop IP connectivity monitors after a period of time | +| | specified by “waiting_time” | +| | | +| | Result: The monitor info will be aggregated | +| | | ++--------------+--------------------------------------------------------------+ +|step 6 | Verify the IP connectivity monitor result | +| | | +| | Result: IP connectivity monitor should not have any packet | +| | drop failures reported | +| | | ++--------------+--------------------------------------------------------------+ +|step 7 | Verify process_recover_time, which indicates the maximun | +| | time (seconds) from the process being killed to recovered, | +| | is within the SLA. This step blocks until either the | +| | process has recovered or a timeout occurred. | +| | | +| | Result: process_recover_time is within SLA limits, if not, | +| | test case failed and stopped. | +| | | ++--------------+--------------------------------------------------------------+ +|step 8 | Start IP connectivity monitors for the new VM: | +| | 1. Check the L2 connectivity from the existing VMs to the | +| | new VM in the Neutron network. | +| | | +| | 2. Check connectivity from one VM to an external host on | +| | the Internet to verify SNAT functionality. | +| | | +| | Result: The monitor info will be collected. | +| | | ++--------------+--------------------------------------------------------------+ +|step 9 | Stop IP connectivity monitors after a period of time | +| | specified by “waiting_time” | +| | | +| | Result: The monitor info will be aggregated | +| | | ++--------------+--------------------------------------------------------------+ +|step 10 | Verify the IP connectivity monitor result | +| | | +| | Result: IP connectivity monitor should not have any packet | +| | drop failures reported | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | Fails only if SLA is not passed, or if there is a test case | +| | execution problem. | +| | | ++--------------+--------------------------------------------------------------+ + diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc093.rst b/docs/testing/user/userguide/opnfv_yardstick_tc093.rst new file mode 100644 index 000000000..31fa5d3d3 --- /dev/null +++ b/docs/testing/user/userguide/opnfv_yardstick_tc093.rst @@ -0,0 +1,184 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intracom Telecom and others. +.. mardim@intracom-telecom.com + +************************************* +Yardstick Test Case Description TC093 +************************************* + ++-----------------------------------------------------------------------------+ +|SDN Vswitch resilience in non-HA or HA configuration | +| | ++--------------+--------------------------------------------------------------+ +|test case id | OPNFV_YARDSTICK_TC093: SDN Vswitch resilience in | +| | non-HA or HA configuration | ++--------------+--------------------------------------------------------------+ +|test purpose | This test validates that network data plane services are | +| | resilient in the event of Virtual Switch failure | +| | in compute nodes. Specifically, the test verifies that | +| | existing data plane connectivity is not permanently impacted | +| | i.e. all configured network services such as DHCP, ARP, L2, | +| | L3 Security Groups continue to operate between the existing | +| | VMs eventually after the Virtual Switches have finished | +| | rebooting. | +| | | +| | The test also validates that new network service operations | +| | (creating a new VM in the existing L2/L3 network or in a new | +| | network, etc.) are operational after the Virtual Switches | +| | have recovered from a failure. | +| | | ++--------------+--------------------------------------------------------------+ +|test method | This testcase first checks if the already configured | +| | DHCP/ARP/L2/L3/SNAT connectivity is proper. After | +| | it fails and restarts again the VSwitch services which are | +| | running on both OpenStack compute nodes, and then checks if | +| | already configured DHCP/ARP/L2/L3/SNAT connectivity is not | +| | permanently impacted (even if there are some packet | +| | loss events) between VMs and the system is able to execute | +| | new virtual network operations once the Vswitch services | +| | are restarted and have been fully recovered | +| | | ++--------------+--------------------------------------------------------------+ +|attackers | In this test case, two attackers called “kill-process” are | +| | needed. These attackers include three parameters: | +| | 1. fault_type: which is used for finding the attacker's | +| | scripts. It should be set to 'kill-process' in this test | +| | | +| | 2. process_name: should be set to the name of the Vswitch | +| | process | +| | | +| | 3. host: which is the name of the compute node where the | +| | Vswitch process is running | +| | | +| | e.g. -fault_type: "kill-process" | +| | -process_name: "openvswitch" | +| | -host: node1 | +| | | ++--------------+--------------------------------------------------------------+ +|monitors | This test case utilizes two monitors of type "ip-status" | +| | and one monitor of type "process" to track the following | +| | conditions: | +| | 1. "ping_same_network_l2": monitor ICMP traffic between | +| | VMs in the same Neutron network | +| | | +| | 2. "ping_external_snat": monitor ICMP traffic from VMs to | +| | an external host on the Internet to verify SNAT | +| | functionality. | +| | | +| | 3. "Vswitch process monitor": a monitor checking the | +| | state of the specified Vswitch process. It measures | +| | the recovery time of the given process. | +| | | +| | Monitors of type "ip-status" use the "ping" utility to | +| | verify reachability of a given target IP. | +| | | ++--------------+--------------------------------------------------------------+ +|operations | In this test case, the following operations are needed: | +| | 1. "nova-create-instance-in_network": create a VM instance | +| | in one of the existing Neutron network. | +| | | ++--------------+--------------------------------------------------------------+ +|metrics | In this test case, there are two metrics: | +| | 1. process_recover_time: which indicates the maximun | +| | time (seconds) from the process being killed to | +| | recovered | +| | | +| | 2. outage_time: measures the total time in which | +| | monitors were failing in their tasks (e.g. total time of | +| | Ping failure) | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | Developed by the project. Please see folder: | +| | "yardstick/benchmark/scenarios/availability/ha_tools" | +| | | ++--------------+--------------------------------------------------------------+ +|references | none | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | This test case needs two configuration files: | +| | 1. test case file: opnfv_yardstick_tc093.yaml | +| | - Attackers: see above “attackers” description | +| | - monitor_time: which is the time (seconds) from | +| | starting to stoping the monitors | +| | - Monitors: see above “monitors” discription | +| | - SLA: see above “metrics” description | +| | | +| | 2. POD file: pod.yaml The POD configuration should record | +| | on pod.yaml first. the “host” item in this test case | +| | will use the node name in the pod.yaml. | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | Description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|pre-action | 1. The Vswitches are set up in both compute nodes. | +| | | +| | 2. One or more Neutron networks are created with two or | +| | more VMs attached to each of the Neutron networks. | +| | | +| | 3. The Neutron networks are attached to a Neutron router | +| | which is attached to an external network towards the | +| | DCGW. | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | Start IP connectivity monitors: | +| | 1. Check the L2 connectivity between the VMs in the same | +| | Neutron network. | +| | | +| | 2. Check connectivity from one VM to an external host on | +| | the Internet to verify SNAT functionality. | +| | | +| | Result: The monitor info will be collected. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Start attackers: | +| | SSH connect to the VIM compute nodes and kill the Vswitch | +| | processes | +| | | +| | Result: the SDN Vswitch services will be shutdown | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | Verify the results of the IP connectivity monitors. | +| | | +| | Result: The outage_time metric reported by the monitors | +| | is not greater than the max_outage_time. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | Restart the SDN Vswitch services. | +| | | ++--------------+--------------------------------------------------------------+ +|step 5 | Create a new VM in the existing Neutron network | +| | | ++--------------+--------------------------------------------------------------+ +|step 6 | Verify connectivity between VMs as follows: | +| | 1. Check the L2 connectivity between the previously | +| | existing VM and the newly created VM on the same | +| | Neutron network by sending ICMP messages | +| | | ++--------------+--------------------------------------------------------------+ +|step 7 | Stop IP connectivity monitors after a period of time | +| | specified by “monitor_time” | +| | | +| | Result: The monitor info will be aggregated | +| | | ++--------------+--------------------------------------------------------------+ +|step 8 | Verify the IP connectivity monitor results | +| | | +| | Result: IP connectivity monitor should not have any packet | +| | drop failures reported | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | This test fails if the SLAs are not met or if there is a | +| | test case execution problem. The SLAs are define as follows | +| | for this test: | +| | * SDN Vswitch recovery | +| | * process_recover_time <= 30 sec | +| | | +| | * no impact on data plane connectivity during SDN | +| | Vswitch failure and recovery. | +| | * packet_drop == 0 | +| | | ++--------------+--------------------------------------------------------------+ + |