aboutsummaryrefslogtreecommitdiffstats
path: root/docs/userguide/opnfv_yardstick_tc012.rst
diff options
context:
space:
mode:
authorJingLu5 <lvjing5@huawei.com>2016-11-15 11:42:10 +0800
committerJingLu5 <lvjing5@huawei.com>2016-12-05 08:39:24 +0800
commit8272a44abd449661e78b9a9945f9ca9677fe98a8 (patch)
tree88f6a6032c28abe973e1419390ee92775f8c9c5c /docs/userguide/opnfv_yardstick_tc012.rst
parenteeeca4f0cdeff152ec061bb9b40ae305547c027e (diff)
enhance test cases description and provide more info(in progress)
JIRA: YARDSTICK-389 This work is about to improve test case.rst to provide more info Change-Id: If27fe462a43f6344a06e61944b72e912d2a516b7 Signed-off-by: JingLu5 <lvjing5@huawei.com>
Diffstat (limited to 'docs/userguide/opnfv_yardstick_tc012.rst')
-rw-r--r--docs/userguide/opnfv_yardstick_tc012.rst101
1 files changed, 75 insertions, 26 deletions
diff --git a/docs/userguide/opnfv_yardstick_tc012.rst b/docs/userguide/opnfv_yardstick_tc012.rst
index ffce06eb9..b56e829f5 100644
--- a/docs/userguide/opnfv_yardstick_tc012.rst
+++ b/docs/userguide/opnfv_yardstick_tc012.rst
@@ -7,29 +7,60 @@
Yardstick Test Case Description TC012
*************************************
-.. _man-pages: http://manpages.ubuntu.com/manpages/trusty/bw_mem.8.html
+.. _bw_mem: http://manpages.ubuntu.com/manpages/trusty/bw_mem.8.html
+-----------------------------------------------------------------------------+
|Memory Bandwidth |
| |
+--------------+--------------------------------------------------------------+
-|test case id | OPNFV_YARDSTICK_TC012_Memory Bandwidth |
+|test case id | OPNFV_YARDSTICK_TC012_MEMORY BANDWIDTH |
| | |
+--------------+--------------------------------------------------------------+
-|metric | Megabyte per second (MBps) |
+|metric | Memory read/write bandwidth (MBps) |
| | |
+--------------+--------------------------------------------------------------+
-|test purpose | Measure the rate at which data can be read from and written |
-| | to the memory (this includes all levels of memory). |
+|test purpose | The purpose of TC012 is to evaluate the IaaS compute |
+| | performance with regards to memory throughput. |
+| | It measures the rate at which data can be read from and |
+| | written to the memory (this includes all levels of memory). |
+| | |
+| | The purpose is also to be able to spot the trends. |
+| | Test results, graphs and similar shall be stored for |
+| | comparison reasons and product evolution understanding |
+| | between different OPNFV versions and/or configurations. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | LMbench |
+| | |
+| | LMbench is a suite of operating system microbenchmarks. |
+| | This test uses bw_mem tool from that suite including: |
+| | * Cached file read |
+| | * Memory copy (bcopy) |
+| | * Memory read |
+| | * Memory write |
+| | * Pipe |
+| | * TCP |
+| | |
+| | (LMbench is not always part of a Linux distribution, hence |
+| | it needs to be installed. As an example see the |
+| | /yardstick/tools/ directory for how to generate a Linux |
+| | image with LMbench included.) |
+| | |
++--------------+--------------------------------------------------------------+
+|test | LMbench bw_mem benchmark allocates twice the specified |
+|description | amount of memory, zeros it, and then times the copying of |
+| | the first half to the second half. The benchmark is invoked |
+| | in a host VM on a compute blade. Results are reported in |
+| | megabytes moved per second. |
| | |
+--------------+--------------------------------------------------------------+
|configuration | File: opnfv_yardstick_tc012.yaml |
| | |
| | * SLA (optional): 15000 (MBps) min_bw: The minimum amount of |
| | memory bandwidth that is accepted. |
-| | * Size: 10 240 kB - test allocates twice that size (20 480kB)|
-| | zeros it and then measures the time it takes to copy from |
-| | one side to another. |
+| | * Size: 10 240 kB - test allocates twice that size |
+| | (20 480kB) zeros it and then measures the time it takes to |
+| | copy from one side to another. |
| | * Benchmark: rdwr - measures the time to read data into |
| | memory and then write data to the same location. |
| | * Warmup: 0 - the number of iterations to perform before |
@@ -38,20 +69,13 @@ Yardstick Test Case Description TC012
| | * Interval: 1 - there is 1 second delay between each |
| | iteration. |
| | |
-+--------------+--------------------------------------------------------------+
-|test tool | Lmbench |
-| | |
-| | Lmbench is a suite of operating system microbenchmarks. This |
-| | test uses bw_mem tool from that suite. |
-| | Lmbench is not always part of a Linux distribution, hence it |
-| | needs to be installed in the test image. |
-| | |
-+--------------+--------------------------------------------------------------+
-|references | man-pages_ |
-| | |
-| | McVoy, Larry W., and Carl Staelin. "lmbench: Portable Tools |
-| | for Performance Analysis." USENIX annual technical |
-| | conference. 1996. |
+| | SLA is optional. The SLA in this test case serves as an |
+| | example. Considerably higher bandwidth is expected. |
+| | However, to cover most configurations, both baremetal and |
+| | fully virtualized ones, this value should be possible to |
+| | achieve and acceptable for black box testing. |
+| | Many heavy IO applications start to suffer badly if the |
+| | read/write bandwidths are lower than this. |
| | |
+--------------+--------------------------------------------------------------+
|applicability | Test can be configured with different: |
@@ -62,7 +86,19 @@ Yardstick Test Case Description TC012
| | * number of warmup iterations; |
| | * iterations and intervals. |
| | |
-| | There are default values for each above-mentioned option. |
+| | Default values exist. |
+| | |
+| | SLA (optional) : min_bandwidth: The minimun memory bandwidth |
+| | that is accepted. |
+| | |
++--------------+--------------------------------------------------------------+
+|usability | This test case is one of Yardstick's generic test. Thus it |
+| | is runnable on most of the scenarios. |
+| | |
++--------------+--------------------------------------------------------------+
+|references | LMbench bw_mem_ |
+| | |
+| | ETSI-NFV-TST001 |
| | |
+--------------+--------------------------------------------------------------+
|pre-test | The test case image needs to be installed into Glance |
@@ -74,10 +110,23 @@ Yardstick Test Case Description TC012
|test sequence | description and expected result |
| | |
+--------------+--------------------------------------------------------------+
-|step 1 | The host is installed as client. Lmbench's bw_mem tool is |
-| | invoked and logs are produced and stored. |
+|step 1 | A host VM with LMbench installed is booted. |
| | |
-| | Result: logs are stored. |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with the host VM by using ssh. |
+| | "lmbench_bandwidth_benchmark" bash script is copied from |
+| | Jump Host to the host VM via ssh tunnel. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | 'lmbench_bandwidth_benchmark' script is invoked. LMbench's |
+| | bw_mem benchmark starts to measures memory read/write |
+| | bandwidth. Memory read/write bandwidth results are recorded |
+| | and checked against the SLA. Logs are produced and stored. |
+| | |
+| | Result: Logs are stored. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | The host VM is deleted. |
| | |
+--------------+--------------------------------------------------------------+
|test verdict | Test fails if the measured memory bandwidth is below the SLA |