summaryrefslogtreecommitdiffstats
path: root/docs/userguide
diff options
context:
space:
mode:
Diffstat (limited to 'docs/userguide')
-rw-r--r--docs/userguide/opnfv_yardstick_tc053.rst142
-rw-r--r--docs/userguide/opnfv_yardstick_tc054.rst125
-rw-r--r--docs/userguide/opnfv_yardstick_tc061.rst88
-rw-r--r--docs/userguide/opnfv_yardstick_tc074.rst137
4 files changed, 492 insertions, 0 deletions
diff --git a/docs/userguide/opnfv_yardstick_tc053.rst b/docs/userguide/opnfv_yardstick_tc053.rst
new file mode 100644
index 000000000..8808d12d9
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc053.rst
@@ -0,0 +1,142 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Yin Kanglin and others.
+.. 14_ykl@tongji.edu.cn
+
+*************************************
+Yardstick Test Case Description TC053
+*************************************
+
++-----------------------------------------------------------------------------+
+|OpenStack Controller Load Balance Service High Availability |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC053: OpenStack Controller Load Balance |
+| | Service High Availability- |
++--------------+--------------------------------------------------------------+
+|test purpose | This test case will verify the high availability of the |
+| | load balance service(current is HAProxy) that supports |
+| | OpenStack on controller node. When the load balance service |
+| | of a specified controller node is killed, whether other load |
+| | balancers on other controller nodes will work, and whether |
+| | the controller node will restart the load balancer are |
+| | checked. |
++--------------+--------------------------------------------------------------+
+|test method | This test case kills the processes of load balance service |
+| | on a selected control node, then checks whether the request |
+| | of the related Openstack command is OK and the killed |
+| | processes are recovered. |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, an attacker called "kill-process" is |
+| | needed. This attacker includes three parameters: |
+| | 1) fault_type: which is used for finding the attacker's |
+| | scripts. It should be always set to "kill-process" in this |
+| | test case. |
+| | 2) process_name: which is the process name of the specified |
+| | OpenStack service. If there are multiple processes use the |
+| | same name on the host, all of them are killed by this |
+| | attacker. |
+| | In this case. This parameter should always set to "swift- |
+| | proxy". |
+| | 3) host: which is the name of a control node being attacked. |
+| | |
+| | e.g. |
+| | -fault_type: "kill-process" |
+| | -process_name: "haproxy" |
+| | -host: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|monitors | In this test case, two kinds of monitor are needed: |
+| | 1. the "openstack-cmd" monitor constantly request a specific |
+| | Openstack command, which needs two parameters: |
+| | 1) monitor_type: which is used for finding the monitor class |
+| | and related scritps. It should be always set to |
+| | "openstack-cmd" for this monitor. |
+| | 2) command_name: which is the command name used for request. |
+| | |
+| | 2. the "process" monitor check whether a process is running |
+| | on a specific node, which needs three parameters: |
+| | 1) monitor_type: which used for finding the monitor class |
+| | and related scripts. It should be always set to "process" |
+| | for this monitor. |
+| | 2) process_name: which is the process name for monitor |
+| | 3) host: which is the name of the node runing the process |
+| | In this case, the command_name of monitor1 should be |
+| | services that is supported by load balancer and the process- |
+| | name of monitor2 should be "haproxy", for example: |
+| | |
+| | e.g. |
+| | monitor1: |
+| | -monitor_type: "openstack-cmd" |
+| | -command_name: "nova image-list" |
+| | monitor2: |
+| | -monitor_type: "process" |
+| | -process_name: "haproxy" |
+| | -host: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there are two metrics: |
+| | 1)service_outage_time: which indicates the maximum outage |
+| | time (seconds) of the specified Openstack command request. |
+| | 2)process_recover_time: which indicates the maximun time |
+| | (seconds) from the process being killed to recovered |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI NFV REL001 |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1) test case file: opnfv_yardstick_tc053.yaml |
+| | -Attackers: see above "attackers" discription |
+| | -waiting_time: which is the time (seconds) from the process |
+| | being killed to stoping monitors the monitors |
+| | -Monitors: see above "monitors" discription |
+| | -SLA: see above "metrics" discription |
+| | |
+| | 2)POD file: pod.yaml |
+| | The POD configuration should record on pod.yaml first. |
+| | the "host" item in this test case will use the node name in |
+| | the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | start monitors: |
+| | each monitor will run with independently process |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | do attacker: connect the host through SSH, and then execute |
+| | the kill process script with param value specified by |
+| | "process_name" |
+| | |
+| | Result: Process will be killed. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | stop monitors after a period of time specified by |
+| | "waiting_time" |
+| | |
+| | Result: The monitor info will be aggregated. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | verify the SLA |
+| | |
+| | Result: The test case is passed or not. |
+| | |
++--------------+--------------------------------------------------------------+
+|post-action | It is the action when the test cases exist. It will check |
+| | the status of the specified process on the host, and restart |
+| | the process if it is not running for next test cases. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/userguide/opnfv_yardstick_tc054.rst b/docs/userguide/opnfv_yardstick_tc054.rst
new file mode 100644
index 000000000..7f92be2bc
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc054.rst
@@ -0,0 +1,125 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Yin Kanglin and others.
+.. 14_ykl@tongji.edu.cn
+
+*************************************
+Yardstick Test Case Description TC054
+*************************************
+
++-----------------------------------------------------------------------------+
+|OpenStack Virtual IP High Availability |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC054: OpenStack Virtual IP High |
+| | Availability |
++--------------+--------------------------------------------------------------+
+|test purpose | This test case will verify the high availability for virtual |
+| | ip in the environment. When master node of virtual ip is |
+| | abnormally shutdown, connection to virtual ip and |
+| | the services binded to the virtual IP it should be OK. |
++--------------+--------------------------------------------------------------+
+|test method | This test case shutdowns the virtual IP master node with |
+| | some fault injection tools, then checks whether virtual ips |
+| | can be pinged and services binded to virtual ip are OK with |
+| | some monitor tools. |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, an attacker called "control-shutdown" is |
+| | needed. This attacker includes two parameters: |
+| | 1) fault_type: which is used for finding the attacker's |
+| | scripts. It should be always set to "control-shutdown" in |
+| | this test case. |
+| | 2) host: which is the name of a control node being attacked. |
+| | |
+| | In this case the host should be the virtual ip master node, |
+| | that means the host ip is the virtual ip, for exapmle: |
+| | -fault_type: "control-shutdown" |
+| | -host: node1(the VIP Master node) |
++--------------+--------------------------------------------------------------+
+|monitors | In this test case, two kinds of monitor are needed: |
+| | 1. the "ip_status" monitor that pings a specific ip to check |
+| | the connectivity of this ip, which needs two parameters: |
+| | 1) monitor_type: which is used for finding the monitor class |
+| | and related scripts. It should be always set to "ip_status" |
+| | for this monitor. |
+| | 2) ip_address: The ip to be pinged. In this case, ip_address |
+| | should be the virtual IP. |
+| | |
+| | 2. the "openstack-cmd" monitor constantly request a specific |
+| | Openstack command, which needs two parameters: |
+| | 1) monitor_type: which is used for finding the monitor class |
+| | and related scripts. It should be always set to |
+| | "openstack-cmd" for this monitor. |
+| | 2) command_name: which is the command name used for request. |
+| | |
+| | e.g. |
+| | monitor1: |
+| | -monitor_type: "ip_status" |
+| | -host: 192.168.0.2 |
+| | monitor2: |
+| | -monitor_type: "openstack-cmd" |
+| | -command_name: "nova image-list" |
+| | |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there are two metrics: |
+| | 1) ping_outage_time: which-indicates the maximum outage time |
+| | to ping the specified host. |
+| | 2)service_outage_time: which indicates the maximum outage |
+| | time (seconds) of the specified Openstack command request. |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI NFV REL001 |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1) test case file: opnfv_yardstick_tc054.yaml |
+| | -Attackers: see above "attackers" discription |
+| | -waiting_time: which is the time (seconds) from the process |
+| | being killed to stoping monitors the monitors |
+| | -Monitors: see above "monitors" discription |
+| | -SLA: see above "metrics" discription |
+| | |
+| | 2)POD file: pod.yaml |
+| | The POD configuration should record on pod.yaml first. |
+| | the "host" item in this test case will use the node name in |
+| | the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | start monitors: |
+| | each monitor will run with independently process |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | do attacker: connect the host through SSH, and then execute |
+| | the shutdown script on the VIP master node. |
+| | |
+| | Result: VIP master node will be shutdown |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | stop monitors after a period of time specified by |
+| | "waiting_time" |
+| | |
+| | Result: The monitor info will be aggregated. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | verify the SLA |
+| | |
+| | Result: The test case is passed or not. |
+| | |
++--------------+--------------------------------------------------------------+
+|post-action | It is the action when the test cases exist. It restarts the |
+| | original VIP master node if it is not restarted. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/userguide/opnfv_yardstick_tc061.rst b/docs/userguide/opnfv_yardstick_tc061.rst
new file mode 100644
index 000000000..1d424414e
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc061.rst
@@ -0,0 +1,88 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+*************************************
+Yardstick Test Case Description TC061
+*************************************
+
+.. _man-pages: http://linux.die.net/man/1/sar
+
++-----------------------------------------------------------------------------+
+|Network Utilization |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC061_Network Utilization |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | Network utilization |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | To evaluate the IaaS network capability with regards to |
+| | network utilization, including Total number of packets |
+| | received per second, Total number of packets transmitted per |
+| | second, Total number of kilobytes received per second, Total |
+| | number of kilobytes transmitted per second, Number of |
+| | compressed packets received per second (for cslip etc.), |
+| | Number of compressed packets transmitted per second, Number |
+| | of multicast packets received per second, Utilization |
+| | percentage of the network interface. |
+| | This test case should be run in parallel to other Yardstick |
+| | test cases and not run as a stand-alone test case. |
+| | Measure the network usage statistics from the network devices|
+| | Average, minimum and maximun values are obtained. |
+| | The purpose is also to be able to spot trends. |
+| | Test results, graphs and similar shall be stored for |
+| | comparison reasons and product evolution understanding |
+| | between different OPNFV versions and/or configurations. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | File: netutilization.yaml (in the 'samples' directory) |
+| | |
+| | * interval: 1 - repeat, pausing every 1 seconds in-between. |
+| | * count: 1 - display statistics 1 times, then exit. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | sar |
+| | |
+| | The sar command writes to standard output the contents of |
+| | selected cumulative activity counters in the operating |
+| | system. |
+| | sar is normally part of a Linux distribution, hence it |
+| | doesn't needs to be installed. |
+| | |
++--------------+--------------------------------------------------------------+
+|references | man-pages_ |
+| | |
+| | ETSI-NFV-TST001 |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | Test can be configured with different: |
+| | |
+| | * interval; |
+| | * count; |
+| | * runner Iteration and intervals. |
+| | |
+| | There are default values for each above-mentioned option. |
+| | Run in background with other test cases. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | The test case image needs to be installed into Glance |
+|conditions | with sar included in the image. |
+| | |
+| | No POD specific requirements have been identified. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | The host is installed as client. The related TC, or TCs, is |
+| | invoked and sar logs are produced and stored. |
+| | |
+| | Result: logs are stored. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | None. Network utilization results are fetched and stored. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/userguide/opnfv_yardstick_tc074.rst b/docs/userguide/opnfv_yardstick_tc074.rst
new file mode 100644
index 000000000..c938f5dfd
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc074.rst
@@ -0,0 +1,137 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+*************************************
+Yardstick Test Case Description TC074
+*************************************
+
+.. Storperf: https://wiki.opnfv.org/display/storperf/Storperf
+
++-----------------------------------------------------------------------------+
+|Storperf |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC074_Storperf |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | Storage performance |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | Storperf integration with yardstick. The purpose of StorPerf |
+| | is to provide a tool to measure block and object storage |
+| | performance in an NFVI. When complemented with a |
+| | characterization of typical VF storage performance |
+| | requirements, it can provide pass/fail thresholds for test, |
+| | staging, and production NFVI environments. |
+| | |
+| | The benchmarks developed for block and object storage will |
+| | be sufficiently varied to provide a good preview of expected |
+| | storage performance behavior for any type of VNF workload. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | file: opnfv_yardstick_tc074.yaml |
+| | |
+| | * agent_count: 1 - the number of VMs to be created |
+| | * agent_image: "Ubuntu-14.04" - image used for creating VMs |
+| | * public_network: "ext-net" - name of public network |
+| | * volume_size: 2 - cinder volume size |
+| | * block_sizes: "4096" - data block size |
+| | * queue_depths: "4" |
+| | * StorPerf_ip: "192.168.200.2" |
+| | * query_interval: 10 - state query interval |
+| | * timeout: 600 - maximum allowed job time |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Storperf |
+| | |
+| | StorPerf is a tool to measure block and object storage |
+| | performance in an NFVI. |
+| | |
+| | StorPerf is delivered as a Docker container from |
+| | https://hub.docker.com/r/opnfv/storperf/tags/. |
+| | |
++--------------+--------------------------------------------------------------+
+|references | Storperf_ |
+| | |
+| | ETSI-NFV-TST001 |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | Test can be configured with different: |
+| | |
+| | * agent_count |
+| | * volume_size |
+| | * block_sizes |
+| | * queue_depths |
+| | * query_interval |
+| | * timeout |
+| | * target=[device or path] |
+| | The path to either an attached storage device |
+| | (/dev/vdb, etc) or a directory path (/opt/storperf) that |
+| | will be used to execute the performance test. In the case |
+| | of a device, the entire device will be used. If not |
+| | specified, the current directory will be used. |
+| | * workload=[workload module] |
+| | If not specified, the default is to run all workloads. The |
+| | workload types are: |
+| | - rs: 100% Read, sequential data |
+| | - ws: 100% Write, sequential data |
+| | - rr: 100% Read, random access |
+| | - wr: 100% Write, random access |
+| | - rw: 70% Read / 30% write, random access |
+| | * nossd: Do not perform SSD style preconditioning. |
+| | * nowarm: Do not perform a warmup prior to |
+| | measurements. |
+| | * report= [job_id] |
+| | Query the status of the supplied job_id and report on |
+| | metrics. If a workload is supplied, will report on only |
+| | that subset. |
+| | |
+| | There are default values for each above-mentioned option. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | If you do not have an Ubuntu 14.04 image in Glance, you will |
+|conditions | need to add one. A key pair for launching agents is also |
+| | required. |
+| | |
+| | Storperf is required to be installed in the environment. |
+| | There are two possible methods for Storperf installation: |
+| | Run container on Jump Host |
+| | Run container in a VM |
+| | |
+| | Running StorPerf on Jump Host |
+| | Requirements: |
+| | - Docker must be installed |
+| | - Jump Host must have access to the OpenStack Controller |
+| | API |
+| | - Jump Host must have internet connectivity for |
+| | downloading docker image |
+| | - Enough floating IPs must be available to match your |
+| | agent count |
+| | |
+| | Running StorPerf in a VM |
+| | Requirements: |
+| | - VM has docker installed |
+| | - VM has OpenStack Controller credentials and can |
+| | communicate with the Controller API |
+| | - VM has internet connectivity for downloading the |
+| | docker image |
+| | - Enough floating IPs must be available to match your |
+| | agent count |
+| | |
+| | No POD specific requirements have been identified. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | The Storperf is installed and Ubuntu 14.04 image is stored |
+| | in glance. TC is invoked and logs are produced and stored. |
+| | |
+| | Result: Logs are stored. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | None. Storage performance results are fetched and stored. |
+| | |
++--------------+--------------------------------------------------------------+