1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
|
.. This work is licensed under a Creative Commons Attribution 4.0 International
.. License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, Ericsson AB and others.
*************************************
Yardstick Test Case Description TC012
*************************************
.. _bw_mem: http://manpages.ubuntu.com/manpages/trusty/bw_mem.8.html
+-----------------------------------------------------------------------------+
|Memory Bandwidth |
| |
+--------------+--------------------------------------------------------------+
|test case id | OPNFV_YARDSTICK_TC012_MEMORY BANDWIDTH |
| | |
+--------------+--------------------------------------------------------------+
|metric | Memory read/write bandwidth (MBps) |
| | |
+--------------+--------------------------------------------------------------+
|test purpose | The purpose of TC012 is to evaluate the IaaS compute |
| | performance with regards to memory throughput. |
| | It measures the rate at which data can be read from and |
| | written to the memory (this includes all levels of memory). |
| | |
| | The purpose is also to be able to spot the trends. |
| | Test results, graphs and similar shall be stored for |
| | comparison reasons and product evolution understanding |
| | between different OPNFV versions and/or configurations. |
| | |
+--------------+--------------------------------------------------------------+
|test tool | LMbench |
| | |
| | LMbench is a suite of operating system microbenchmarks. |
| | This test uses bw_mem tool from that suite including: |
| | |
| | * Cached file read |
| | * Memory copy (bcopy) |
| | * Memory read |
| | * Memory write |
| | * Pipe |
| | * TCP |
| | |
| | (LMbench is not always part of a Linux distribution, hence |
| | it needs to be installed. As an example see the |
| | /yardstick/tools/ directory for how to generate a Linux |
| | image with LMbench included.) |
| | |
+--------------+--------------------------------------------------------------+
|test | LMbench bw_mem benchmark allocates twice the specified |
|description | amount of memory, zeros it, and then times the copying of |
| | the first half to the second half. The benchmark is invoked |
| | in a host VM on a compute blade. Results are reported in |
| | megabytes moved per second. |
| | |
+--------------+--------------------------------------------------------------+
|configuration | File: opnfv_yardstick_tc012.yaml |
| | |
| | * SLA (optional): 15000 (MBps) min_bw: The minimum amount of |
| | memory bandwidth that is accepted. |
| | * Size: 10 240 kB - test allocates twice that size |
| | (20 480kB) zeros it and then measures the time it takes to |
| | copy from one side to another. |
| | * Benchmark: rdwr - measures the time to read data into |
| | memory and then write data to the same location. |
| | * Warmup: 0 - the number of iterations to perform before |
| | taking actual measurements. |
| | * Iterations: 10 - test is run 10 times iteratively. |
| | * Interval: 1 - there is 1 second delay between each |
| | iteration. |
| | |
| | SLA is optional. The SLA in this test case serves as an |
| | example. Considerably higher bandwidth is expected. |
| | However, to cover most configurations, both baremetal and |
| | fully virtualized ones, this value should be possible to |
| | achieve and acceptable for black box testing. |
| | Many heavy IO applications start to suffer badly if the |
| | read/write bandwidths are lower than this. |
| | |
+--------------+--------------------------------------------------------------+
|applicability | Test can be configured with different: |
| | |
| | * memory sizes; |
| | * memory operations (such as rd, wr, rdwr, cp, frd, fwr, |
| | fcp, bzero, bcopy); |
| | * number of warmup iterations; |
| | * iterations and intervals. |
| | |
| | Default values exist. |
| | |
| | SLA (optional) : min_bandwidth: The minimun memory bandwidth |
| | that is accepted. |
| | |
+--------------+--------------------------------------------------------------+
|usability | This test case is one of Yardstick's generic test. Thus it |
| | is runnable on most of the scenarios. |
| | |
+--------------+--------------------------------------------------------------+
|references | LMbench bw_mem_ |
| | |
| | ETSI-NFV-TST001 |
| | |
+--------------+--------------------------------------------------------------+
|pre-test | The test case image needs to be installed into Glance |
|conditions | with Lmbench included in the image. |
| | |
| | No POD specific requirements have been identified. |
| | |
+--------------+--------------------------------------------------------------+
|test sequence | description and expected result |
| | |
+--------------+--------------------------------------------------------------+
|step 1 | A host VM with LMbench installed is booted. |
| | |
+--------------+--------------------------------------------------------------+
|step 2 | Yardstick is connected with the host VM by using ssh. |
| | "lmbench_bandwidth_benchmark" bash script is copied from |
| | Jump Host to the host VM via ssh tunnel. |
| | |
+--------------+--------------------------------------------------------------+
|step 3 | 'lmbench_bandwidth_benchmark' script is invoked. LMbench's |
| | bw_mem benchmark starts to measures memory read/write |
| | bandwidth. Memory read/write bandwidth results are recorded |
| | and checked against the SLA. Logs are produced and stored. |
| | |
| | Result: Logs are stored. |
| | |
+--------------+--------------------------------------------------------------+
|step 4 | The host VM is deleted. |
| | |
+--------------+--------------------------------------------------------------+
|test verdict | Test fails if the measured memory bandwidth is below the SLA |
| | value or if there is a test case execution problem. |
| | |
+--------------+--------------------------------------------------------------+
|
