diff options
Diffstat (limited to 'docs/userguide/opnfv_yardstick_tc012.rst')
| -rw-r--r-- | docs/userguide/opnfv_yardstick_tc012.rst | 101 |
1 files changed, 75 insertions, 26 deletions
diff --git a/docs/userguide/opnfv_yardstick_tc012.rst b/docs/userguide/opnfv_yardstick_tc012.rst index ffce06eb9..b56e829f5 100644 --- a/docs/userguide/opnfv_yardstick_tc012.rst +++ b/docs/userguide/opnfv_yardstick_tc012.rst @@ -7,29 +7,60 @@ Yardstick Test Case Description TC012 ************************************* -.. _man-pages: http://manpages.ubuntu.com/manpages/trusty/bw_mem.8.html +.. _bw_mem: http://manpages.ubuntu.com/manpages/trusty/bw_mem.8.html +-----------------------------------------------------------------------------+ |Memory Bandwidth | | | +--------------+--------------------------------------------------------------+ -|test case id | OPNFV_YARDSTICK_TC012_Memory Bandwidth | +|test case id | OPNFV_YARDSTICK_TC012_MEMORY BANDWIDTH | | | | +--------------+--------------------------------------------------------------+ -|metric | Megabyte per second (MBps) | +|metric | Memory read/write bandwidth (MBps) | | | | +--------------+--------------------------------------------------------------+ -|test purpose | Measure the rate at which data can be read from and written | -| | to the memory (this includes all levels of memory). | +|test purpose | The purpose of TC012 is to evaluate the IaaS compute | +| | performance with regards to memory throughput. | +| | It measures the rate at which data can be read from and | +| | written to the memory (this includes all levels of memory). | +| | | +| | The purpose is also to be able to spot the trends. | +| | Test results, graphs and similar shall be stored for | +| | comparison reasons and product evolution understanding | +| | between different OPNFV versions and/or configurations. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | LMbench | +| | | +| | LMbench is a suite of operating system microbenchmarks. | +| | This test uses bw_mem tool from that suite including: | +| | * Cached file read | +| | * Memory copy (bcopy) | +| | * Memory read | +| | * Memory write | +| | * Pipe | +| | * TCP | +| | | +| | (LMbench is not always part of a Linux distribution, hence | +| | it needs to be installed. As an example see the | +| | /yardstick/tools/ directory for how to generate a Linux | +| | image with LMbench included.) | +| | | ++--------------+--------------------------------------------------------------+ +|test | LMbench bw_mem benchmark allocates twice the specified | +|description | amount of memory, zeros it, and then times the copying of | +| | the first half to the second half. The benchmark is invoked | +| | in a host VM on a compute blade. Results are reported in | +| | megabytes moved per second. | | | | +--------------+--------------------------------------------------------------+ |configuration | File: opnfv_yardstick_tc012.yaml | | | | | | * SLA (optional): 15000 (MBps) min_bw: The minimum amount of | | | memory bandwidth that is accepted. | -| | * Size: 10 240 kB - test allocates twice that size (20 480kB)| -| | zeros it and then measures the time it takes to copy from | -| | one side to another. | +| | * Size: 10 240 kB - test allocates twice that size | +| | (20 480kB) zeros it and then measures the time it takes to | +| | copy from one side to another. | | | * Benchmark: rdwr - measures the time to read data into | | | memory and then write data to the same location. | | | * Warmup: 0 - the number of iterations to perform before | @@ -38,20 +69,13 @@ Yardstick Test Case Description TC012 | | * Interval: 1 - there is 1 second delay between each | | | iteration. | | | | -+--------------+--------------------------------------------------------------+ -|test tool | Lmbench | -| | | -| | Lmbench is a suite of operating system microbenchmarks. This | -| | test uses bw_mem tool from that suite. | -| | Lmbench is not always part of a Linux distribution, hence it | -| | needs to be installed in the test image. | -| | | -+--------------+--------------------------------------------------------------+ -|references | man-pages_ | -| | | -| | McVoy, Larry W., and Carl Staelin. "lmbench: Portable Tools | -| | for Performance Analysis." USENIX annual technical | -| | conference. 1996. | +| | SLA is optional. The SLA in this test case serves as an | +| | example. Considerably higher bandwidth is expected. | +| | However, to cover most configurations, both baremetal and | +| | fully virtualized ones, this value should be possible to | +| | achieve and acceptable for black box testing. | +| | Many heavy IO applications start to suffer badly if the | +| | read/write bandwidths are lower than this. | | | | +--------------+--------------------------------------------------------------+ |applicability | Test can be configured with different: | @@ -62,7 +86,19 @@ Yardstick Test Case Description TC012 | | * number of warmup iterations; | | | * iterations and intervals. | | | | -| | There are default values for each above-mentioned option. | +| | Default values exist. | +| | | +| | SLA (optional) : min_bandwidth: The minimun memory bandwidth | +| | that is accepted. | +| | | ++--------------+--------------------------------------------------------------+ +|usability | This test case is one of Yardstick's generic test. Thus it | +| | is runnable on most of the scenarios. | +| | | ++--------------+--------------------------------------------------------------+ +|references | LMbench bw_mem_ | +| | | +| | ETSI-NFV-TST001 | | | | +--------------+--------------------------------------------------------------+ |pre-test | The test case image needs to be installed into Glance | @@ -74,10 +110,23 @@ Yardstick Test Case Description TC012 |test sequence | description and expected result | | | | +--------------+--------------------------------------------------------------+ -|step 1 | The host is installed as client. Lmbench's bw_mem tool is | -| | invoked and logs are produced and stored. | +|step 1 | A host VM with LMbench installed is booted. | | | | -| | Result: logs are stored. | ++--------------+--------------------------------------------------------------+ +|step 2 | Yardstick is connected with the host VM by using ssh. | +| | "lmbench_bandwidth_benchmark" bash script is copied from | +| | Jump Host to the host VM via ssh tunnel. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | 'lmbench_bandwidth_benchmark' script is invoked. LMbench's | +| | bw_mem benchmark starts to measures memory read/write | +| | bandwidth. Memory read/write bandwidth results are recorded | +| | and checked against the SLA. Logs are produced and stored. | +| | | +| | Result: Logs are stored. | +| | | ++--------------+--------------------------------------------------------------+ +|step 4 | The host VM is deleted. | | | | +--------------+--------------------------------------------------------------+ |test verdict | Test fails if the measured memory bandwidth is below the SLA | |