aboutsummaryrefslogtreecommitdiffstats
path: root/docs/userguide/opnfv_yardstick_tc010.rst
diff options
context:
space:
mode:
authorJing Lu <lvjing5@huawei.com>2016-12-28 01:23:09 +0000
committerGerrit Code Review <gerrit@opnfv.org>2016-12-28 01:23:09 +0000
commit827e341ed222a562a87d8f238fc0a6a16cfb1393 (patch)
treec6890816c0f134c6b3551436bae0d92e0319e921 /docs/userguide/opnfv_yardstick_tc010.rst
parent85a8cc0f326e3db55c59185d9c41f5a70b2f560f (diff)
parent8272a44abd449661e78b9a9945f9ca9677fe98a8 (diff)
Merge "enhance test cases description and provide more info(in progress)"
Diffstat (limited to 'docs/userguide/opnfv_yardstick_tc010.rst')
-rw-r--r--docs/userguide/opnfv_yardstick_tc010.rst116
1 files changed, 94 insertions, 22 deletions
diff --git a/docs/userguide/opnfv_yardstick_tc010.rst b/docs/userguide/opnfv_yardstick_tc010.rst
index ab793de76..202307de6 100644
--- a/docs/userguide/opnfv_yardstick_tc010.rst
+++ b/docs/userguide/opnfv_yardstick_tc010.rst
@@ -7,21 +7,71 @@
Yardstick Test Case Description TC010
*************************************
-.. _man-pages: http://manpages.ubuntu.com/manpages/trusty/lat_mem_rd.8.html
+.. _lat_mem_rd: http://manpages.ubuntu.com/manpages/trusty/lat_mem_rd.8.html
+-----------------------------------------------------------------------------+
|Memory Latency |
| |
+--------------+--------------------------------------------------------------+
-|test case id | OPNFV_YARDSTICK_TC010_Memory Latency |
+|test case id | OPNFV_YARDSTICK_TC010_MEMORY LATENCY |
| | |
+--------------+--------------------------------------------------------------+
-|metric | Latency in nanoseconds |
+|metric | Memory read latency (nanoseconds) |
| | |
+--------------+--------------------------------------------------------------+
-|test purpose | Measure the memory read latency for varying memory sizes and |
-| | strides. Whole memory hierarchy is measured including all |
-| | levels of cache. |
+|test purpose | The purpose of TC010 is to evaluate the IaaS compute |
+| | performance with regards to memory read latency. |
+| | It measures the memory read latency for varying memory sizes |
+| | and strides. Whole memory hierarchy is measured. |
+| | |
+| | The purpose is also to be able to spot the trends. |
+| | Test results, graphs and similar shall be stored for |
+| | comparison reasons and product evolution understanding |
+| | between different OPNFV versions and/or configurations. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Lmbench |
+| | |
+| | Lmbench is a suite of operating system microbenchmarks. This |
+| | test uses lat_mem_rd tool from that suite including: |
+| | * Context switching |
+| | * Networking: connection establishment, pipe, TCP, UDP, and |
+| | RPC hot potato |
+| | * File system creates and deletes |
+| | * Process creation |
+| | * Signal handling |
+| | * System call overhead |
+| | * Memory read latency |
+| | |
+| | (LMbench is not always part of a Linux distribution, hence |
+| | it needs to be installed. As an example see the |
+| | /yardstick/tools/ directory for how to generate a Linux |
+| | image with LMbench included.) |
+| | |
++--------------+--------------------------------------------------------------+
+|test | LMbench lat_mem_rd benchmark measures memory read latency |
+|description | for varying memory sizes and strides. |
+| | |
+| | The benchmark runs as two nested loops. The outer loop is |
+| | the stride size. The inner loop is the array size. For each |
+| | array size, the benchmark creates a ring of pointers that |
+| | point backward one stride.Traversing the array is done by: |
+| | |
+| | p = (char **)*p; |
+| | |
+| | in a for loop (the over head of the for loop is not |
+| | significant; the loop is an unrolled loop 100 loads long). |
+| | The size of the array varies from 512 bytes to (typically) |
+| | eight megabytes. For the small sizes, the cache will have an |
+| | effect, and the loads will be much faster. This becomes much |
+| | more apparent when the data is plotted. |
+| | |
+| | Only data accesses are measured; the instruction cache is |
+| | not measured. |
+| | |
+| | The results are reported in nanoseconds per load and have |
+| | been verified accurate to within a few nanoseconds on an SGI |
+| | Indy. |
| | |
+--------------+--------------------------------------------------------------+
|configuration | File: opnfv_yardstick_tc010.yaml |
@@ -33,20 +83,13 @@ Yardstick Test Case Description TC010
| | * Interval: 1 - there is 1 second delay between each |
| | iteration. |
| | |
-+--------------+--------------------------------------------------------------+
-|test tool | Lmbench |
-| | |
-| | Lmbench is a suite of operating system microbenchmarks. This |
-| | test uses lat_mem_rd tool from that suite. |
-| | Lmbench is not always part of a Linux distribution, hence it |
-| | needs to be installed in the test image |
-| | |
-+--------------+--------------------------------------------------------------+
-|references | man-pages_ |
-| | |
-| | McVoy, Larry W.,and Carl Staelin. "lmbench: Portable Tools |
-| | for Performance Analysis." USENIX annual technical |
-| | conference 1996. |
+| | SLA is optional. The SLA in this test case serves as an |
+| | example. Considerably lower read latency is expected. |
+| | However, to cover most configurations, both baremetal and |
+| | fully virtualized ones, this value should be possible to |
+| | achieve and acceptable for black box testing. |
+| | Many heavy IO applications start to suffer badly if the |
+| | read latency is higher than this. |
| | |
+--------------+--------------------------------------------------------------+
|applicability | Test can be configured with different: |
@@ -55,12 +98,21 @@ Yardstick Test Case Description TC010
| | * stop_size; |
| | * iterations and intervals. |
| | |
-| | There are default values for each above-mentioned option. |
+| | Default values exist. |
| | |
| | SLA (optional) : max_latency: The maximum memory latency |
| | that is accepted. |
| | |
+--------------+--------------------------------------------------------------+
+|usability | This test case is one of Yardstick's generic test. Thus it |
+| | is runnable on most of the scenarios. |
+| | |
++--------------+--------------------------------------------------------------+
+|references | LMbench lat_mem_rd_ |
+| | |
+| | ETSI-NFV-TST001 |
+| | |
++--------------+--------------------------------------------------------------+
|pre-test | The test case image needs to be installed into Glance |
|conditions | with Lmbench included in the image. |
| | |
@@ -70,12 +122,32 @@ Yardstick Test Case Description TC010
|test sequence | description and expected result |
| | |
+--------------+--------------------------------------------------------------+
-|step 1 | The host is installed as client. Lmbench's lat_mem_rd tool |
+|step 1 | The host is installed as client. LMbench's lat_mem_rd tool |
| | is invoked and logs are produced and stored. |
| | |
| | Result: logs are stored. |
| | |
+--------------+--------------------------------------------------------------+
+|step 1 | A host VM with LMbench installed is booted. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with the host VM by using ssh. |
+| | 'lmbench_latency_benchmark' bash script is copyied from Jump |
+| | Host to the host VM via the ssh tunnel. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | 'lmbench_latency_benchmark' script is invoked. LMbench's |
+| | lat_mem_rd benchmark starts to measures memory read latency |
+| | for varying memory sizes and strides. Memory read latency |
+| | are recorded and checked against the SLA. Logs are produced |
+| | and stored. |
+| | |
+| | Result: Logs are stored. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | The host VM is deleted. |
+| | |
++--------------+--------------------------------------------------------------+
|test verdict | Test fails if the measured memory latency is above the SLA |
| | value or if there is a test case execution problem. |
| | |