summaryrefslogtreecommitdiffstats
path: root/docs
diff options
context:
space:
mode:
Diffstat (limited to 'docs')
-rw-r--r--docs/userguide/03-list-of-tcs.rst11
-rw-r--r--docs/userguide/opnfv_yardstick_tc001.rst17
-rw-r--r--docs/userguide/opnfv_yardstick_tc004.rst77
-rw-r--r--docs/userguide/opnfv_yardstick_tc011.rst6
-rw-r--r--docs/userguide/opnfv_yardstick_tc024.rst23
-rw-r--r--docs/userguide/opnfv_yardstick_tc042.rst87
-rw-r--r--docs/userguide/opnfv_yardstick_tc043.rst5
-rw-r--r--docs/userguide/opnfv_yardstick_tc050.rst135
-rw-r--r--docs/userguide/opnfv_yardstick_tc051.rst117
-rw-r--r--docs/userguide/opnfv_yardstick_tc052.rst141
-rw-r--r--docs/userguide/opnfv_yardstick_tc053.rst142
-rw-r--r--docs/userguide/opnfv_yardstick_tc054.rst125
-rw-r--r--docs/userguide/opnfv_yardstick_tc061.rst88
-rw-r--r--docs/userguide/opnfv_yardstick_tc063.rst81
-rw-r--r--docs/userguide/opnfv_yardstick_tc069.rst25
-rw-r--r--docs/userguide/opnfv_yardstick_tc073.rst81
-rw-r--r--docs/userguide/opnfv_yardstick_tc074.rst137
17 files changed, 1267 insertions, 31 deletions
diff --git a/docs/userguide/03-list-of-tcs.rst b/docs/userguide/03-list-of-tcs.rst
index de48c7bc7..96e5297a1 100644
--- a/docs/userguide/03-list-of-tcs.rst
+++ b/docs/userguide/03-list-of-tcs.rst
@@ -28,6 +28,7 @@ Generic NFVI Test Case Descriptions
opnfv_yardstick_tc001.rst
opnfv_yardstick_tc002.rst
+ opnfv_yardstick_tc004.rst
opnfv_yardstick_tc005.rst
opnfv_yardstick_tc008.rst
opnfv_yardstick_tc009.rst
@@ -38,6 +39,16 @@ Generic NFVI Test Case Descriptions
opnfv_yardstick_tc024.rst
opnfv_yardstick_tc037.rst
opnfv_yardstick_tc038.rst
+ opnfv_yardstick_tc042.rst
+ opnfv_yardstick_tc043.rst
+ opnfv_yardstick_tc044.rst
+ opnfv_yardstick_tc055.rst
+ opnfv_yardstick_tc061.rst
+ opnfv_yardstick_tc063.rst
+ opnfv_yardstick_tc069.rst
+ opnfv_yardstick_tc070.rst
+ opnfv_yardstick_tc071.rst
+ opnfv_yardstick_tc072.rst
OPNFV Feature Test Cases
========================
diff --git a/docs/userguide/opnfv_yardstick_tc001.rst b/docs/userguide/opnfv_yardstick_tc001.rst
index 4cf4b94a4..fac375d50 100644
--- a/docs/userguide/opnfv_yardstick_tc001.rst
+++ b/docs/userguide/opnfv_yardstick_tc001.rst
@@ -21,14 +21,14 @@ Yardstick Test Case Description TC001
+--------------+--------------------------------------------------------------+
|test purpose | To evaluate the IaaS network performance with regards to |
| | flows and throughput, such as if and how different amounts |
-| | of flows matter for the throughput between hosts on different|
-| | compute blades. Typically e.g. the performance of a vSwitch |
-| | depends on the number of flows running through it. Also |
-| | performance of other equipment or entities can depend |
+| | of flows matter for the throughput between hosts on |
+| | different compute blades. Typically e.g. the performance of |
+| | a vSwitch depends on the number of flows running through it. |
+| | Also performance of other equipment or entities can depend |
| | on the number of flows or the packet sizes used. |
| | The purpose is also to be able to spot trends. Test results, |
-| | graphs ans similar shall be stored for comparison reasons and|
-| | product evolution understanding between different OPNFV |
+| | graphs ans similar shall be stored for comparison reasons |
+| | and product evolution understanding between different OPNFV |
| | versions and/or configurations. |
| | |
+--------------+--------------------------------------------------------------+
@@ -37,7 +37,8 @@ Yardstick Test Case Description TC001
| | Packet size: 60 bytes |
| | Number of ports: 10, 50, 100, 500 and 1000, where each |
| | runs for 20 seconds. The whole sequence is run |
-| | twice. The client and server are distributed on different HW.|
+| | twice. The client and server are distributed on different |
+| | HW. |
| | For SLA max_ppm is set to 1000. The amount of configured |
| | ports map to between 110 up to 1001000 flows, respectively. |
| | |
@@ -60,7 +61,7 @@ Yardstick Test Case Description TC001
| | of flows and test duration. Default values exist. |
| | |
| | SLA (optional): max_ppm: The number of packets per million |
-| | packets sent that are acceptable to loose, not received. |
+| | packets sent that are acceptable to loose, not received. |
| | |
+--------------+--------------------------------------------------------------+
|pre-test | The test case image needs to be installed into Glance |
diff --git a/docs/userguide/opnfv_yardstick_tc004.rst b/docs/userguide/opnfv_yardstick_tc004.rst
new file mode 100644
index 000000000..301286126
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc004.rst
@@ -0,0 +1,77 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+*************************************
+Yardstick Test Case Description TC004
+*************************************
+
+.. _cachestat: https://github.com/brendangregg/perf-tools/tree/master/fs
+
++-----------------------------------------------------------------------------+
+|Cache Utilization |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC004_Cache Utilization |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | Cache Utilization |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | To evaluate the IaaS compute capability with regards to |
+| | cache utilization.This test case should be run in parallel |
+| | to other Yardstick test cases and not run as a stand-alone |
+| | test case. |
+| | Measure the cache usage statistics including cache hit, |
+| | cache miss, hit ratio, page cache size and page cache size. |
+| | Both average and maximun values are obtained. |
+| | The purpose is also to be able to spot trends. |
+| | Test results, graphs and similar shall be stored for |
+| | comparison reasons and product evolution understanding |
+| | between different OPNFV versions and/or configurations. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | File: cachestat.yaml (in the 'samples' directory) |
+| | |
+| | * interval: 1 - repeat, pausing every 1 seconds in-between. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | cachestat |
+| | |
+| | cachestat is not always part of a Linux distribution, hence |
+| | it needs to be installed. |
+| | |
++--------------+--------------------------------------------------------------+
+|references | cachestat_ |
+| | |
+| | ETSI-NFV-TST001 |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | Test can be configured with different: |
+| | |
+| | * interval; |
+| | * runner Duration. |
+| | |
+| | There are default values for each above-mentioned option. |
+| | Run in background with other test cases. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | The test case image needs to be installed into Glance |
+|conditions | with cachestat included in the image. |
+| | |
+| | No POD specific requirements have been identified. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | The host is installed as client. The related TC, or TCs, is |
+| | invoked and cachestat logs are produced and stored. |
+| | |
+| | Result: logs are stored. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | None. Cache utilization results are fetched and stored. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/userguide/opnfv_yardstick_tc011.rst b/docs/userguide/opnfv_yardstick_tc011.rst
index 1c643cd72..cf2fd5055 100644
--- a/docs/userguide/opnfv_yardstick_tc011.rst
+++ b/docs/userguide/opnfv_yardstick_tc011.rst
@@ -56,12 +56,12 @@ Yardstick Test Case Description TC011
| | ETSI-NFV-TST001 |
| | |
+--------------+--------------------------------------------------------------+
-|applicability | Test can be configured with different |
+|applicability | Test can be configured with different: |
| | |
| | * bandwidth: Test case can be configured with different |
-| | bandwidth |
+| | bandwidth. |
| | |
-| | * duration: The test duration can be configured |
+| | * duration: The test duration can be configured. |
| | |
| | * jitter: SLA is optional. The SLA in this test case |
| | serves as an example. |
diff --git a/docs/userguide/opnfv_yardstick_tc024.rst b/docs/userguide/opnfv_yardstick_tc024.rst
index ffdacb106..8d15e8d2f 100644
--- a/docs/userguide/opnfv_yardstick_tc024.rst
+++ b/docs/userguide/opnfv_yardstick_tc024.rst
@@ -22,17 +22,17 @@ Yardstick Test Case Description TC024
|test purpose | To evaluate the CPU load performance of the IaaS. This test |
| | case should be run in parallel to other Yardstick test cases |
| | and not run as a stand-alone test case. |
-| | |
-| | The purpose is also to be able to spot trends. Test results, |
-| | graphs ans similar shall be stored for comparison reasons and|
-| | product evolution understanding between different OPNFV |
-| | versions and/or configurations. |
+| | Average, minimum and maximun values are obtained. |
+| | The purpose is also to be able to spot trends. |
+| | Test results, graphs and similar shall be stored for |
+| | comparison reasons and product evolution understanding |
+| | between different OPNFV versions and/or configurations. |
| | |
+--------------+--------------------------------------------------------------+
|configuration | file: cpuload.yaml (in the 'samples' directory) |
| | |
-| | There is are no additional configurations to be set for this |
-| | TC. |
+| | * interval: 1 - repeat, pausing every 1 seconds in-between. |
+| | * count: 10 - display statistics 10 times, then exit. |
| | |
+--------------+--------------------------------------------------------------+
|test tool | mpstat |
@@ -46,7 +46,14 @@ Yardstick Test Case Description TC024
|references | man-pages_ |
| | |
+--------------+--------------------------------------------------------------+
-|applicability | Run in background with other test cases. |
+|applicability | Test can be configured with different: |
+| | |
+| | * interval; |
+| | * count; |
+| | * runner Iteration and intervals. |
+| | |
+| | There are default values for each above-mentioned option. |
+| | Run in background with other test cases. |
| | |
+--------------+--------------------------------------------------------------+
|pre-test | The test case image needs to be installed into Glance |
diff --git a/docs/userguide/opnfv_yardstick_tc042.rst b/docs/userguide/opnfv_yardstick_tc042.rst
new file mode 100644
index 000000000..8660d9297
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc042.rst
@@ -0,0 +1,87 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, ZTE and others.
+
+***************************************
+Yardstick Test Case Description TC0042
+***************************************
+
+.. _DPDK: http://dpdk.org/doc/guides/index.html
+.. _Testpmd: http://dpdk.org/doc/guides/testpmd_app_ug/index.html
+.. _Pktgen-dpdk: http://pktgen.readthedocs.io/en/latest/index.html
+
++-----------------------------------------------------------------------------+
+|Network Performance |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC042_DPDK pktgen latency measurements |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | L2 Network Latency |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | Measure L2 network latency when DPDK is enabled between hosts|
+| | on different compute blades. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | file: opnfv_yardstick_tc042.yaml |
+| | |
+| | * Packet size: 64 bytes |
+| | * SLA(max_latency): 100usec |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | DPDK_ |
+| | Pktgen-dpdk_ |
+| | |
+| | (DPDK and Pktgen-dpdk are not part of a Linux distribution, |
+| | hence they needs to be installed. |
+| | As an example see the /yardstick/tools/ directory for how to |
+| | generate a Linux image with DPDK and pktgen-dpdk included.) |
+| | |
++--------------+--------------------------------------------------------------+
+|references | DPDK_ |
+| | |
+| | Pktgen-dpdk_ |
+| | |
+| | ETSI-NFV-TST001 |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | Test can be configured with different packet sizes. Default |
+| | values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | The test case image needs to be installed into Glance |
+|conditions | with DPDK and pktgen-dpdk included in it. |
+| | |
+| | The NICs of compute nodes must support DPDK on POD. |
+| | |
+| | And at least compute nodes setup hugepage. |
+| | |
+| | If you want to achievement a hight performance result, it is |
+| | recommend to use NUAM, CPU pin, OVS and so on. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | The hosts are installed on different blades, as server and |
+| | client. Both server and client have three interfaces. The |
+| | first one is management such as ssh. The other two are used |
+| | by DPDK. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Testpmd_ is invoked with configurations to forward packets |
+| | from one DPDK port to the other on server. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | Pktgen-dpdk is invoked with configurations as a traffic |
+| | generator and logs are produced and stored on client. |
+| | |
+| | Result: Logs are stored. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/userguide/opnfv_yardstick_tc043.rst b/docs/userguide/opnfv_yardstick_tc043.rst
index 2f907e9ef..b6e557d86 100644
--- a/docs/userguide/opnfv_yardstick_tc043.rst
+++ b/docs/userguide/opnfv_yardstick_tc043.rst
@@ -13,8 +13,9 @@ Yardstick Test Case Description TC043
|Network Latency Between NFVI Nodes |
| |
+--------------+--------------------------------------------------------------+
-|test case id | OPNFV_YARDSTICK_TC043_Latency_between_ |
-| | NFVI_nodes_measurements |
+|test case id | OPNFV_YARDSTICK_TC043_Latency_between_NFVI_nodes_ |
+| | measurements |
+| | |
+--------------+--------------------------------------------------------------+
|metric | RTT, Round Trip Time |
| | |
diff --git a/docs/userguide/opnfv_yardstick_tc050.rst b/docs/userguide/opnfv_yardstick_tc050.rst
new file mode 100644
index 000000000..8890c9d53
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc050.rst
@@ -0,0 +1,135 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Yin Kanglin and others.
+.. 14_ykl@tongji.edu.cn
+
+*************************************
+Yardstick Test Case Description TC050
+*************************************
+
++-----------------------------------------------------------------------------+
+|OpenStack Controller Node Network High Availability |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC050: OpenStack Controller Node Network |
+| | High Availability |
++--------------+--------------------------------------------------------------+
+|test purpose | This test case will verify the high availability of control |
+| | node. When one of the controller failed to connect the |
+| | network, which breaks down the Openstack services on this |
+| | node. These Openstack service should able to be accessed by |
+| | other controller nodes, and the services on failed |
+| | controller node should be isolated. |
++--------------+--------------------------------------------------------------+
+|test method | This test case turns off the network interfaces of a |
+| | specified control node, then checks whether all services |
+| | provided by the control node are OK with some monitor tools. |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, an attacker called "close-interface" is |
+| | needed. This attacker includes three parameters: |
+| | 1) fault_type: which is used for finding the attacker's |
+| | scripts. It should be always set to "close-interface" in |
+| | this test case. |
+| | 2) host: which is the name of a control node being attacked. |
+| | 3) interface: the network interface to be turned off. |
+| | |
+| | There are four instance of the "close-interface" monitor: |
+| | attacker1(for public netork): |
+| | -fault_type: "close-interface" |
+| | -host: node1 |
+| | -interface: "br-ex" |
+| | attacker2(for management netork): |
+| | -fault_type: "close-interface" |
+| | -host: node1 |
+| | -interface: "br-mgmt" |
+| | attacker3(for storage netork): |
+| | -fault_type: "close-interface" |
+| | -host: node1 |
+| | -interface: "br-storage" |
+| | attacker4(for private netork): |
+| | -fault_type: "close-interface" |
+| | -host: node1 |
+| | -interface: "br-mesh" |
++--------------+--------------------------------------------------------------+
+|monitors | In this test case, the monitor named "openstack-cmd" is |
+| | needed. The monitor needs needs two parameters: |
+| | 1) monitor_type: which is used for finding the monitor class |
+| | and related scritps. It should be always set to |
+| | "openstack-cmd" for this monitor. |
+| | 2) command_name: which is the command name used for request |
+| | |
+| | There are four instance of the "openstack-cmd" monitor: |
+| | monitor1: |
+| | -monitor_type: "openstack-cmd" |
+| | -command_name: "nova image-list" |
+| | monitor2: |
+| | -monitor_type: "openstack-cmd" |
+| | -command_name: "neutron router-list" |
+| | monitor3: |
+| | -monitor_type: "openstack-cmd" |
+| | -command_name: "heat stack-list" |
+| | monitor4: |
+| | -monitor_type: "openstack-cmd" |
+| | -command_name: "cinder list" |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there is one metric: |
+| | 1)service_outage_time: which indicates the maximum outage |
+| | time (seconds) of the specified Openstack command request. |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI NFV REL001 |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1) test case file: opnfv_yardstick_tc050.yaml |
+| | -Attackers: see above "attackers" discription |
+| | -waiting_time: which is the time (seconds) from the process |
+| | being killed to stoping monitors the monitors |
+| | -Monitors: see above "monitors" discription |
+| | -SLA: see above "metrics" discription |
+| | |
+| | 2)POD file: pod.yaml |
+| | The POD configuration should record on pod.yaml first. |
+| | the "host" item in this test case will use the node name in |
+| | the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | start monitors: |
+| | each monitor will run with independently process |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | do attacker: connect the host through SSH, and then execute |
+| | the turnoff network interface script with param value |
+| | specified by "interface". |
+| | |
+| | Result: Network interfaces will be turned down. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | stop monitors after a period of time specified by |
+| | "waiting_time" |
+| | |
+| | Result: The monitor info will be aggregated. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | verify the SLA |
+| | |
+| | Result: The test case is passed or not. |
+| | |
++--------------+--------------------------------------------------------------+
+|post-action | It is the action when the test cases exist. It turns up the |
+| | network interface of the control node if it is not turned |
+| | up. |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/userguide/opnfv_yardstick_tc051.rst b/docs/userguide/opnfv_yardstick_tc051.rst
new file mode 100644
index 000000000..3402ccd92
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc051.rst
@@ -0,0 +1,117 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Yin Kanglin and others.
+.. 14_ykl@tongji.edu.cn
+
+*************************************
+Yardstick Test Case Description TC051
+*************************************
+
++-----------------------------------------------------------------------------+
+|OpenStack Controller Node CPU Overload High Availability |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC051: OpenStack Controller Node CPU |
+| | Overload High Availability |
++--------------+--------------------------------------------------------------+
+|test purpose | This test case will verify the high availability of control |
+| | node. When the CPU usage of a specified controller node is |
+| | stressed to 100%, which breaks down the Openstack services |
+| | on this node. These Openstack service should able to be |
+| | accessed by other controller nodes, and the services on |
+| | failed controller node should be isolated. |
++--------------+--------------------------------------------------------------+
+|test method | This test case stresses the CPU uasge of a specified control |
+| | node to 100%, then checks whether all services provided by |
+| | the environment are OK with some monitor tools. |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, an attacker called "stress-cpu" is |
+| | needed. This attacker includes two parameters: |
+| | 1) fault_type: which is used for finding the attacker's |
+| | scripts. It should be always set to "stress-cpu" in |
+| | this test case. |
+| | 2) host: which is the name of a control node being attacked. |
+| | e.g. |
+| | -fault_type: "stress-cpu" |
+| | -host: node1 |
++--------------+--------------------------------------------------------------+
+|monitors | In this test case, the monitor named "openstack-cmd" is |
+| | needed. The monitor needs needs two parameters: |
+| | 1) monitor_type: which is used for finding the monitor class |
+| | and related scritps. It should be always set to |
+| | "openstack-cmd" for this monitor. |
+| | 2) command_name: which is the command name used for request |
+| | |
+| | There are four instance of the "openstack-cmd" monitor: |
+| | monitor1: |
+| | -monitor_type: "openstack-cmd" |
+| | -command_name: "nova image-list" |
+| | monitor2: |
+| | -monitor_type: "openstack-cmd" |
+| | -command_name: "neutron router-list" |
+| | monitor3: |
+| | -monitor_type: "openstack-cmd" |
+| | -command_name: "heat stack-list" |
+| | monitor4: |
+| | -monitor_type: "openstack-cmd" |
+| | -command_name: "cinder list" |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there is one metric: |
+| | 1)service_outage_time: which indicates the maximum outage |
+| | time (seconds) of the specified Openstack command request. |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI NFV REL001 |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1) test case file: opnfv_yardstick_tc051.yaml |
+| | -Attackers: see above "attackers" discription |
+| | -waiting_time: which is the time (seconds) from the process |
+| | being killed to stoping monitors the monitors |
+| | -Monitors: see above "monitors" discription |
+| | -SLA: see above "metrics" discription |
+| | |
+| | 2)POD file: pod.yaml |
+| | The POD configuration should record on pod.yaml first. |
+| | the "host" item in this test case will use the node name in |
+| | the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | start monitors: |
+| | each monitor will run with independently process |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | do attacker: connect the host through SSH, and then execute |
+| | the stress cpu script on the host. |
+| | |
+| | Result: The CPU usage of the host will be stressed to 100%. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | stop monitors after a period of time specified by |
+| | "waiting_time" |
+| | |
+| | Result: The monitor info will be aggregated. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | verify the SLA |
+| | |
+| | Result: The test case is passed or not. |
+| | |
++--------------+--------------------------------------------------------------+
+|post-action | It is the action when the test cases exist. It kills the |
+| | process that stresses the CPU usage. |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/userguide/opnfv_yardstick_tc052.rst b/docs/userguide/opnfv_yardstick_tc052.rst
new file mode 100644
index 000000000..9514b6819
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc052.rst
@@ -0,0 +1,141 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Yin Kanglin and others.
+.. 14_ykl@tongji.edu.cn
+
+*************************************
+Yardstick Test Case Description TC052
+*************************************
+
++-----------------------------------------------------------------------------+
+|OpenStack Controller Node Disk I/O Block High Availability |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC052: OpenStack Controller Node Disk I/O |
+| | Block High Availability |
++--------------+--------------------------------------------------------------+
+|test purpose | This test case will verify the high availability of control |
+| | node. When the disk I/O of a specified disk is blocked, |
+| | which breaks down the Openstack services on this node. Read |
+| | and write services should still be accessed by other |
+| | controller nodes, and the services on failed controller node |
+| | should be isolated. |
++--------------+--------------------------------------------------------------+
+|test method | This test case blocks the disk I/O of a specified control |
+| | node, then checks whether the services that need to read or |
+| | wirte the disk of the control node are OK with some monitor |
+| | tools. |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, an attacker called "disk-block" is |
+| | needed. This attacker includes two parameters: |
+| | 1) fault_type: which is used for finding the attacker's |
+| | scripts. It should be always set to "disk-block" in this |
+| | test case. |
+| | 2) host: which is the name of a control node being attacked. |
+| | e.g. |
+| | -fault_type: "disk-block" |
+| | -host: node1 |
++--------------+--------------------------------------------------------------+
+|monitors | In this test case, two kinds of monitor are needed: |
+| | 1. the "openstack-cmd" monitor constantly request a specific |
+| | Openstack command, which needs two parameters: |
+| | 1) monitor_type: which is used for finding the monitor class |
+| | and related scripts. It should be always set to |
+| | "openstack-cmd" for this monitor. |
+| | 2) command_name: which is the command name used for request. |
+| | |
+| | e.g. |
+| | -monitor_type: "openstack-cmd" |
+| | -command_name: "nova flavor-list" |
+| | |
+| | 2. the second monitor verifies the read and write function |
+| | by a "operation" and a "result checker". |
+| | the "operation" have two parameters: |
+| | 1) operation_type: which is used for finding the operation |
+| | class and related scripts. |
+| | 2) action_parameter: parameters for the operation. |
+| | the "result checker" have three parameters: |
+| | 1) checker_type: which is used for finding the reuslt |
+| | checker class and realted scripts. |
+| | 2) expectedValue: the expected value for the output of the |
+| | checker script. |
+| | 3) condition: whether the expected value is in the output of |
+| | checker script or is totally same with the output. |
+| | |
+| | In this case, the "operation" adds a flavor and the "result |
+| | checker" checks whether ths flavor is created. Their |
+| | parameters show as follows: |
+| | operation: |
+| | -operation_type: "nova-create-flavor" |
+| | -action_parameter: |
+| | flavorconfig: "test-001 test-001 100 1 1" |
+| | result checker: |
+| | -checker_type: "check-flavor" |
+| | -expectedValue: "test-001" |
+| | -condition: "in" |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there is one metric: |
+| | 1)service_outage_time: which indicates the maximum outage |
+| | time (seconds) of the specified Openstack command request. |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI NFV REL001 |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1) test case file: opnfv_yardstick_tc052.yaml |
+| | -Attackers: see above "attackers" discription |
+| | -waiting_time: which is the time (seconds) from the process |
+| | being killed to stoping monitors the monitors |
+| | -Monitors: see above "monitors" discription |
+| | -SLA: see above "metrics" discription |
+| | |
+| | 2)POD file: pod.yaml |
+| | The POD configuration should record on pod.yaml first. |
+| | the "host" item in this test case will use the node name in |
+| | the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | do attacker: connect the host through SSH, and then execute |
+| | the block disk I/O script on the host. |
+| | |
+| | Result: The disk I/O of the host will be blocked |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | start monitors: |
+| | each monitor will run with independently process |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | do operation: add a flavor |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | do result checker: check whether the falvor is created |
+| | |
++--------------+--------------------------------------------------------------+
+|step 5 | stop monitors after a period of time specified by |
+| | "waiting_time" |
+| | |
+| | Result: The monitor info will be aggregated. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 6 | verify the SLA |
+| | |
+| | Result: The test case is passed or not. |
+| | |
++--------------+--------------------------------------------------------------+
+|post-action | It is the action when the test cases exist. It excutes the |
+| | release disk I/O script to release the blocked I/O. |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails if monnitor SLA is not passed or the result checker is |
+| | not passed, or if there is a test case execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/userguide/opnfv_yardstick_tc053.rst b/docs/userguide/opnfv_yardstick_tc053.rst
new file mode 100644
index 000000000..8808d12d9
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc053.rst
@@ -0,0 +1,142 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Yin Kanglin and others.
+.. 14_ykl@tongji.edu.cn
+
+*************************************
+Yardstick Test Case Description TC053
+*************************************
+
++-----------------------------------------------------------------------------+
+|OpenStack Controller Load Balance Service High Availability |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC053: OpenStack Controller Load Balance |
+| | Service High Availability- |
++--------------+--------------------------------------------------------------+
+|test purpose | This test case will verify the high availability of the |
+| | load balance service(current is HAProxy) that supports |
+| | OpenStack on controller node. When the load balance service |
+| | of a specified controller node is killed, whether other load |
+| | balancers on other controller nodes will work, and whether |
+| | the controller node will restart the load balancer are |
+| | checked. |
++--------------+--------------------------------------------------------------+
+|test method | This test case kills the processes of load balance service |
+| | on a selected control node, then checks whether the request |
+| | of the related Openstack command is OK and the killed |
+| | processes are recovered. |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, an attacker called "kill-process" is |
+| | needed. This attacker includes three parameters: |
+| | 1) fault_type: which is used for finding the attacker's |
+| | scripts. It should be always set to "kill-process" in this |
+| | test case. |
+| | 2) process_name: which is the process name of the specified |
+| | OpenStack service. If there are multiple processes use the |
+| | same name on the host, all of them are killed by this |
+| | attacker. |
+| | In this case. This parameter should always set to "swift- |
+| | proxy". |
+| | 3) host: which is the name of a control node being attacked. |
+| | |
+| | e.g. |
+| | -fault_type: "kill-process" |
+| | -process_name: "haproxy" |
+| | -host: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|monitors | In this test case, two kinds of monitor are needed: |
+| | 1. the "openstack-cmd" monitor constantly request a specific |
+| | Openstack command, which needs two parameters: |
+| | 1) monitor_type: which is used for finding the monitor class |
+| | and related scritps. It should be always set to |
+| | "openstack-cmd" for this monitor. |
+| | 2) command_name: which is the command name used for request. |
+| | |
+| | 2. the "process" monitor check whether a process is running |
+| | on a specific node, which needs three parameters: |
+| | 1) monitor_type: which used for finding the monitor class |
+| | and related scripts. It should be always set to "process" |
+| | for this monitor. |
+| | 2) process_name: which is the process name for monitor |
+| | 3) host: which is the name of the node runing the process |
+| | In this case, the command_name of monitor1 should be |
+| | services that is supported by load balancer and the process- |
+| | name of monitor2 should be "haproxy", for example: |
+| | |
+| | e.g. |
+| | monitor1: |
+| | -monitor_type: "openstack-cmd" |
+| | -command_name: "nova image-list" |
+| | monitor2: |
+| | -monitor_type: "process" |
+| | -process_name: "haproxy" |
+| | -host: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there are two metrics: |
+| | 1)service_outage_time: which indicates the maximum outage |
+| | time (seconds) of the specified Openstack command request. |
+| | 2)process_recover_time: which indicates the maximun time |
+| | (seconds) from the process being killed to recovered |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI NFV REL001 |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1) test case file: opnfv_yardstick_tc053.yaml |
+| | -Attackers: see above "attackers" discription |
+| | -waiting_time: which is the time (seconds) from the process |
+| | being killed to stoping monitors the monitors |
+| | -Monitors: see above "monitors" discription |
+| | -SLA: see above "metrics" discription |
+| | |
+| | 2)POD file: pod.yaml |
+| | The POD configuration should record on pod.yaml first. |
+| | the "host" item in this test case will use the node name in |
+| | the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | start monitors: |
+| | each monitor will run with independently process |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | do attacker: connect the host through SSH, and then execute |
+| | the kill process script with param value specified by |
+| | "process_name" |
+| | |
+| | Result: Process will be killed. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | stop monitors after a period of time specified by |
+| | "waiting_time" |
+| | |
+| | Result: The monitor info will be aggregated. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | verify the SLA |
+| | |
+| | Result: The test case is passed or not. |
+| | |
++--------------+--------------------------------------------------------------+
+|post-action | It is the action when the test cases exist. It will check |
+| | the status of the specified process on the host, and restart |
+| | the process if it is not running for next test cases. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/userguide/opnfv_yardstick_tc054.rst b/docs/userguide/opnfv_yardstick_tc054.rst
new file mode 100644
index 000000000..7f92be2bc
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc054.rst
@@ -0,0 +1,125 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Yin Kanglin and others.
+.. 14_ykl@tongji.edu.cn
+
+*************************************
+Yardstick Test Case Description TC054
+*************************************
+
++-----------------------------------------------------------------------------+
+|OpenStack Virtual IP High Availability |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC054: OpenStack Virtual IP High |
+| | Availability |
++--------------+--------------------------------------------------------------+
+|test purpose | This test case will verify the high availability for virtual |
+| | ip in the environment. When master node of virtual ip is |
+| | abnormally shutdown, connection to virtual ip and |
+| | the services binded to the virtual IP it should be OK. |
++--------------+--------------------------------------------------------------+
+|test method | This test case shutdowns the virtual IP master node with |
+| | some fault injection tools, then checks whether virtual ips |
+| | can be pinged and services binded to virtual ip are OK with |
+| | some monitor tools. |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, an attacker called "control-shutdown" is |
+| | needed. This attacker includes two parameters: |
+| | 1) fault_type: which is used for finding the attacker's |
+| | scripts. It should be always set to "control-shutdown" in |
+| | this test case. |
+| | 2) host: which is the name of a control node being attacked. |
+| | |
+| | In this case the host should be the virtual ip master node, |
+| | that means the host ip is the virtual ip, for exapmle: |
+| | -fault_type: "control-shutdown" |
+| | -host: node1(the VIP Master node) |
++--------------+--------------------------------------------------------------+
+|monitors | In this test case, two kinds of monitor are needed: |
+| | 1. the "ip_status" monitor that pings a specific ip to check |
+| | the connectivity of this ip, which needs two parameters: |
+| | 1) monitor_type: which is used for finding the monitor class |
+| | and related scripts. It should be always set to "ip_status" |
+| | for this monitor. |
+| | 2) ip_address: The ip to be pinged. In this case, ip_address |
+| | should be the virtual IP. |
+| | |
+| | 2. the "openstack-cmd" monitor constantly request a specific |
+| | Openstack command, which needs two parameters: |
+| | 1) monitor_type: which is used for finding the monitor class |
+| | and related scripts. It should be always set to |
+| | "openstack-cmd" for this monitor. |
+| | 2) command_name: which is the command name used for request. |
+| | |
+| | e.g. |
+| | monitor1: |
+| | -monitor_type: "ip_status" |
+| | -host: 192.168.0.2 |
+| | monitor2: |
+| | -monitor_type: "openstack-cmd" |
+| | -command_name: "nova image-list" |
+| | |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there are two metrics: |
+| | 1) ping_outage_time: which-indicates the maximum outage time |
+| | to ping the specified host. |
+| | 2)service_outage_time: which indicates the maximum outage |
+| | time (seconds) of the specified Openstack command request. |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI NFV REL001 |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1) test case file: opnfv_yardstick_tc054.yaml |
+| | -Attackers: see above "attackers" discription |
+| | -waiting_time: which is the time (seconds) from the process |
+| | being killed to stoping monitors the monitors |
+| | -Monitors: see above "monitors" discription |
+| | -SLA: see above "metrics" discription |
+| | |
+| | 2)POD file: pod.yaml |
+| | The POD configuration should record on pod.yaml first. |
+| | the "host" item in this test case will use the node name in |
+| | the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | start monitors: |
+| | each monitor will run with independently process |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | do attacker: connect the host through SSH, and then execute |
+| | the shutdown script on the VIP master node. |
+| | |
+| | Result: VIP master node will be shutdown |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | stop monitors after a period of time specified by |
+| | "waiting_time" |
+| | |
+| | Result: The monitor info will be aggregated. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | verify the SLA |
+| | |
+| | Result: The test case is passed or not. |
+| | |
++--------------+--------------------------------------------------------------+
+|post-action | It is the action when the test cases exist. It restarts the |
+| | original VIP master node if it is not restarted. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/userguide/opnfv_yardstick_tc061.rst b/docs/userguide/opnfv_yardstick_tc061.rst
new file mode 100644
index 000000000..1d424414e
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc061.rst
@@ -0,0 +1,88 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+*************************************
+Yardstick Test Case Description TC061
+*************************************
+
+.. _man-pages: http://linux.die.net/man/1/sar
+
++-----------------------------------------------------------------------------+
+|Network Utilization |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC061_Network Utilization |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | Network utilization |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | To evaluate the IaaS network capability with regards to |
+| | network utilization, including Total number of packets |
+| | received per second, Total number of packets transmitted per |
+| | second, Total number of kilobytes received per second, Total |
+| | number of kilobytes transmitted per second, Number of |
+| | compressed packets received per second (for cslip etc.), |
+| | Number of compressed packets transmitted per second, Number |
+| | of multicast packets received per second, Utilization |
+| | percentage of the network interface. |
+| | This test case should be run in parallel to other Yardstick |
+| | test cases and not run as a stand-alone test case. |
+| | Measure the network usage statistics from the network devices|
+| | Average, minimum and maximun values are obtained. |
+| | The purpose is also to be able to spot trends. |
+| | Test results, graphs and similar shall be stored for |
+| | comparison reasons and product evolution understanding |
+| | between different OPNFV versions and/or configurations. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | File: netutilization.yaml (in the 'samples' directory) |
+| | |
+| | * interval: 1 - repeat, pausing every 1 seconds in-between. |
+| | * count: 1 - display statistics 1 times, then exit. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | sar |
+| | |
+| | The sar command writes to standard output the contents of |
+| | selected cumulative activity counters in the operating |
+| | system. |
+| | sar is normally part of a Linux distribution, hence it |
+| | doesn't needs to be installed. |
+| | |
++--------------+--------------------------------------------------------------+
+|references | man-pages_ |
+| | |
+| | ETSI-NFV-TST001 |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | Test can be configured with different: |
+| | |
+| | * interval; |
+| | * count; |
+| | * runner Iteration and intervals. |
+| | |
+| | There are default values for each above-mentioned option. |
+| | Run in background with other test cases. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | The test case image needs to be installed into Glance |
+|conditions | with sar included in the image. |
+| | |
+| | No POD specific requirements have been identified. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | The host is installed as client. The related TC, or TCs, is |
+| | invoked and sar logs are produced and stored. |
+| | |
+| | Result: logs are stored. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | None. Network utilization results are fetched and stored. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/userguide/opnfv_yardstick_tc063.rst b/docs/userguide/opnfv_yardstick_tc063.rst
new file mode 100644
index 000000000..a77653aa5
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc063.rst
@@ -0,0 +1,81 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+*************************************
+Yardstick Test Case Description TC063
+*************************************
+
+.. _iostat: http://linux.die.net/man/1/iostat
+.. _fdisk: http://www.tldp.org/HOWTO/Partition/fdisk_partitioning.html
+
++-----------------------------------------------------------------------------+
+|Storage Capacity |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC063_Storage Capacity |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | Storage/disk size, block size |
+| | Disk Utilization |
++--------------+--------------------------------------------------------------+
+|test purpose | This test case will check the parameters which could decide |
+| | several models and each model has its specified task to |
+| | measure. The test purposes are to measure disk size, block |
+| | size and disk utilization. With the test results, we could |
+| | evaluate the storage capacity of the host. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | file: opnfv_yardstick_tc063.yaml |
+| | |
+| |* test_type: "disk_size" |
+| |* runner: |
+| | type: Iteration |
+| | iterations: 1 - test is run 1 time iteratively. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | fdisk |
+| | A command-line utility that provides disk partitioning |
+| | functions |
+| | |
+| | iostat |
+| | This is a computer system monitor tool used to collect and |
+| | show operating system storage input and output statistics. |
++--------------+--------------------------------------------------------------+
+|references | iostat_ |
+| | fdisk_ |
+| | |
+| | ETSI-NFV-TST001 |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | Test can be configured with different: |
+| | |
+| | * test_type: "disk size", "block size", "disk utilization" |
+| | * interval: 1 - how ofter to stat disk utilization |
+| | type: int |
+| | unit: seconds |
+| | * count: 15 - how many times to stat disk utilization |
+| | type: int |
+| | unit: na |
+| | There are default values for each above-mentioned option. |
+| | Run in background with other test cases. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | The test case image needs to be installed into Glance |
+|conditions | |
+| | No POD specific requirements have been identified. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | Output the specific storage capacity of disk information as |
+| | the sequence into file. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | The pod is available and the hosts are installed. Node5 is |
+| | used and logs are produced and stored. |
+| | |
+| | Result: Logs are stored. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | None. |
++--------------+--------------------------------------------------------------+
diff --git a/docs/userguide/opnfv_yardstick_tc069.rst b/docs/userguide/opnfv_yardstick_tc069.rst
index 51807e246..af0e64fbf 100644
--- a/docs/userguide/opnfv_yardstick_tc069.rst
+++ b/docs/userguide/opnfv_yardstick_tc069.rst
@@ -9,6 +9,9 @@ Yardstick Test Case Description TC069
.. _RAMspeed: http://alasir.com/software/ramspeed/
+.. table::
+ :class: longtable
+
+-----------------------------------------------------------------------------+
|Memory Bandwidth |
| |
@@ -25,9 +28,9 @@ Yardstick Test Case Description TC069
| | while reading and writing certain blocks of data (starting |
| | from 1Kb and further in power of 2) continuously through ALU |
| | and FPU respectively. |
-| | Measure different aspects of memory performance via synthetic|
-| | simulations. Each simulation consists of four performances |
-| | (Copy, Scale, Add, Triad). |
+| | Measure different aspects of memory performance via |
+| | synthetic simulations. Each simulation consists of four |
+| | performances (Copy, Scale, Add, Triad). |
| | Test results, graphs and similar shall be stored for |
| | comparison reasons and product evolution understanding |
| | between different OPNFV versions and/or configurations. |
@@ -37,12 +40,14 @@ Yardstick Test Case Description TC069
| | |
| | * SLA (optional): 7000 (MBps) min_bandwidth: The minimum |
| | amount of memory bandwidth that is accepted. |
-| | * type_id: 1 - runs a specified benchmark (by an ID number): |
+| | * type_id: 1 - runs a specified benchmark |
+| | (by an ID number): |
| | 1 -- INTmark [writing] 4 -- FLOATmark [writing] |
| | 2 -- INTmark [reading] 5 -- FLOATmark [reading] |
| | 3 -- INTmem 6 -- FLOATmem |
-| | * block_size: 64 Megabytes - the maximum block size per array|
-| | * load: 32 Gigabytes - the amount of data load per pass |
+| | * block_size: 64 Megabytes - the maximum block |
+| | size per array. |
+| | * load: 32 Gigabytes - the amount of data load per pass. |
| | * iterations: 5 - test is run 5 times iteratively. |
| | * interval: 1 - there is 1 second delay between each |
| | iteration. |
@@ -52,8 +57,8 @@ Yardstick Test Case Description TC069
| | |
| | RAMspeed is a free open source command line utility to |
| | measure cache and memory performance of computer systems. |
-| | RAMspeed is not always part of a Linux distribution, hence it|
-| | needs to be installed in the test image. |
+| | RAMspeed is not always part of a Linux distribution, hence |
+| | it needs to be installed in the test image. |
| | |
+--------------+--------------------------------------------------------------+
|references | RAMspeed_ |
@@ -83,8 +88,8 @@ Yardstick Test Case Description TC069
|test sequence | description and expected result |
| | |
+--------------+--------------------------------------------------------------+
-|step 1 | The host is installed as client. RAMspeed is invoked and logs|
-| | are produced and stored. |
+|step 1 | The host is installed as client. RAMspeed is invoked and |
+| | logs are produced and stored. |
| | |
| | Result: logs are stored. |
| | |
diff --git a/docs/userguide/opnfv_yardstick_tc073.rst b/docs/userguide/opnfv_yardstick_tc073.rst
new file mode 100644
index 000000000..a6499eabb
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc073.rst
@@ -0,0 +1,81 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+*************************************
+Yardstick Test Case Description TC073
+*************************************
+
+.. _netperf: http://www.netperf.org/netperf/training/Netperf.html
+
++-----------------------------------------------------------------------------+
+|Throughput per NFVI node test |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC073_Network latency and throughput between |
+| | nodes |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | Network latency and throughput |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | To evaluate the IaaS network performance with regards to |
+| | flows and throughput, such as if and how different amounts |
+| | of packet sizes and flows matter for the throughput between |
+| | nodes in one pod. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | file: opnfv_yardstick_tc073.yaml |
+| | |
+| | Packet size: default 1024 bytes. |
+| | |
+| | Test length: default 20 seconds. |
+| | |
+| | The client and server are distributed on different nodes. |
+| | |
+| | For SLA max_mean_latency is set to 100. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | netperf |
+| | Netperf is a software application that provides network |
+| | bandwidth testing between two hosts on a network. It |
+| | supports Unix domain sockets, TCP, SCTP, DLPI and UDP via |
+| | BSD Sockets. Netperf provides a number of predefined tests |
+| | e.g. to measure bulk (unidirectional) data transfer or |
+| | request response performance. |
+| | (netperf is not always part of a Linux distribution, hence |
+| | it needs to be installed.) |
+| | |
++--------------+--------------------------------------------------------------+
+|references | netperf Man pages |
+| | ETSI-NFV-TST001 |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | Test can be configured with different packet sizes and |
+| | test duration. Default values exist. |
+| | |
+| | SLA (optional): max_mean_latency |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | The POD can be reached by external ip and logged on via ssh |
+|conditions | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | Install netperf tool on each specified node, one is as the |
+| | server, and the other as the client. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Log on to the client node and use the netperf command to |
+| | execute the network performance test |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | The throughput results stored. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/userguide/opnfv_yardstick_tc074.rst b/docs/userguide/opnfv_yardstick_tc074.rst
new file mode 100644
index 000000000..c938f5dfd
--- /dev/null
+++ b/docs/userguide/opnfv_yardstick_tc074.rst
@@ -0,0 +1,137 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+*************************************
+Yardstick Test Case Description TC074
+*************************************
+
+.. Storperf: https://wiki.opnfv.org/display/storperf/Storperf
+
++-----------------------------------------------------------------------------+
+|Storperf |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC074_Storperf |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | Storage performance |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | Storperf integration with yardstick. The purpose of StorPerf |
+| | is to provide a tool to measure block and object storage |
+| | performance in an NFVI. When complemented with a |
+| | characterization of typical VF storage performance |
+| | requirements, it can provide pass/fail thresholds for test, |
+| | staging, and production NFVI environments. |
+| | |
+| | The benchmarks developed for block and object storage will |
+| | be sufficiently varied to provide a good preview of expected |
+| | storage performance behavior for any type of VNF workload. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | file: opnfv_yardstick_tc074.yaml |
+| | |
+| | * agent_count: 1 - the number of VMs to be created |
+| | * agent_image: "Ubuntu-14.04" - image used for creating VMs |
+| | * public_network: "ext-net" - name of public network |
+| | * volume_size: 2 - cinder volume size |
+| | * block_sizes: "4096" - data block size |
+| | * queue_depths: "4" |
+| | * StorPerf_ip: "192.168.200.2" |
+| | * query_interval: 10 - state query interval |
+| | * timeout: 600 - maximum allowed job time |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Storperf |
+| | |
+| | StorPerf is a tool to measure block and object storage |
+| | performance in an NFVI. |
+| | |
+| | StorPerf is delivered as a Docker container from |
+| | https://hub.docker.com/r/opnfv/storperf/tags/. |
+| | |
++--------------+--------------------------------------------------------------+
+|references | Storperf_ |
+| | |
+| | ETSI-NFV-TST001 |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | Test can be configured with different: |
+| | |
+| | * agent_count |
+| | * volume_size |
+| | * block_sizes |
+| | * queue_depths |
+| | * query_interval |
+| | * timeout |
+| | * target=[device or path] |
+| | The path to either an attached storage device |
+| | (/dev/vdb, etc) or a directory path (/opt/storperf) that |
+| | will be used to execute the performance test. In the case |
+| | of a device, the entire device will be used. If not |
+| | specified, the current directory will be used. |
+| | * workload=[workload module] |
+| | If not specified, the default is to run all workloads. The |
+| | workload types are: |
+| | - rs: 100% Read, sequential data |
+| | - ws: 100% Write, sequential data |
+| | - rr: 100% Read, random access |
+| | - wr: 100% Write, random access |
+| | - rw: 70% Read / 30% write, random access |
+| | * nossd: Do not perform SSD style preconditioning. |
+| | * nowarm: Do not perform a warmup prior to |
+| | measurements. |
+| | * report= [job_id] |
+| | Query the status of the supplied job_id and report on |
+| | metrics. If a workload is supplied, will report on only |
+| | that subset. |
+| | |
+| | There are default values for each above-mentioned option. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | If you do not have an Ubuntu 14.04 image in Glance, you will |
+|conditions | need to add one. A key pair for launching agents is also |
+| | required. |
+| | |
+| | Storperf is required to be installed in the environment. |
+| | There are two possible methods for Storperf installation: |
+| | Run container on Jump Host |
+| | Run container in a VM |
+| | |
+| | Running StorPerf on Jump Host |
+| | Requirements: |
+| | - Docker must be installed |
+| | - Jump Host must have access to the OpenStack Controller |
+| | API |
+| | - Jump Host must have internet connectivity for |
+| | downloading docker image |
+| | - Enough floating IPs must be available to match your |
+| | agent count |
+| | |
+| | Running StorPerf in a VM |
+| | Requirements: |
+| | - VM has docker installed |
+| | - VM has OpenStack Controller credentials and can |
+| | communicate with the Controller API |
+| | - VM has internet connectivity for downloading the |
+| | docker image |
+| | - Enough floating IPs must be available to match your |
+| | agent count |
+| | |
+| | No POD specific requirements have been identified. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | The Storperf is installed and Ubuntu 14.04 image is stored |
+| | in glance. TC is invoked and logs are produced and stored. |
+| | |
+| | Result: Logs are stored. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | None. Storage performance results are fetched and stored. |
+| | |
++--------------+--------------------------------------------------------------+