1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
<.. This work is licensed under a Creative Commons Attribution 4.0 International
.. License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
*************************************
Yardstick Test Case Description TC078
*************************************
.. _spec_cpu2006: https://www.spec.org/cpu2006/
+-----------------------------------------------------------------------------+
|Compute Performance |
| |
+--------------+--------------------------------------------------------------+
|test case id | OPNFV_YARDSTICK_TC078_SPEC CPU 2006 |
| | |
+--------------+--------------------------------------------------------------+
|metric | compute-intensive performance |
| | |
+--------------+--------------------------------------------------------------+
|test purpose | The purpose of TC078 is to evaluate the IaaS compute |
| | performance by using SPEC CPU 2006 benchmark. The SPEC CPU |
| | 2006 benchmark has several different ways to measure |
| | computer performance. One way is to measure how fast the |
| | computer completes a single task; this is called a speed |
| | measurement. Another way is to measure how many tasks |
| | computer can accomplish in a certain amount of time; this is |
| | called a throughput, capacity or rate measurement. |
| | |
+--------------+--------------------------------------------------------------+
|test tool | SPEC CPU 2006 |
| | |
| | The SPEC CPU 2006 benchmark is SPEC's industry-standardized, |
| | CPU-intensive benchmark suite, stressing a system's |
| | processor, memory subsystem and compiler. This benchmark |
| | suite includes the SPECint benchmarks and the SPECfp |
| | benchmarks. The SPECint 2006 benchmark contains 12 different |
| | enchmark tests and the SPECfp 2006 benchmark contains 19 |
| | different benchmark tests. |
| | |
| | SPEC CPU 2006 is not always part of a Linux distribution. |
| | SPEC requires that users purchase a license and agree with |
| | their terms and conditions. For this test case, users must |
| | manually download cpu2006-1.2.iso from the SPEC website and |
| | save it under the yardstick/resources folder (e.g. /home/ |
| | opnfv/repos/yardstick/yardstick/resources/cpu2006-1.2.iso) |
| | SPEC CPU® 2006 benchmark is available for purchase via the |
| | SPEC order form (https://www.spec.org/order.html). |
| | |
+--------------+--------------------------------------------------------------+
|test | This test case uses SPEC CPU 2006 benchmark to measure |
|description | compute-intensive performance of hosts. |
| | |
+--------------+--------------------------------------------------------------+
|configuration | file: spec_cpu.yaml (in the 'samples' directory) |
| | |
| | benchmark_subset is set to int. |
| | |
| | SLA is not available in this test case. |
| | |
+--------------+--------------------------------------------------------------+
|applicability | Test can be configured with different: |
| | |
| | * benchmark_subset - a subset of SPEC CPU2006 benchmarks to |
| | run; |
| | * SPECint_benchmark - a SPECint benchmark to run; |
| | * SPECint_benchmark - a SPECfp benchmark to run; |
| | * output_format - desired report format; |
| | * runspec_config - SPEC CPU2006 config file provided to the |
| | runspec binary; |
| | * runspec_iterations - the number of benchmark iterations |
| | to execute. For a reportable run, must be 3; |
| | * runspec_tune - tuning to use (base, peak, or all). For a |
| | reportable run, must be either base or all. Reportable |
| | runs do base first, then (optionally) peak; |
| | * runspec_size - size of input data to run (test, train, or |
| | ref). Reportable runs ensure that your binaries can |
| | produce correct results with the test and train workloads |
| | |
+--------------+--------------------------------------------------------------+
|usability | This test case is used for executing SPEC CPU 2006 benchmark |
| | physical servers. The SPECint 2006 benchmark takes |
| | approximately 5 hours. |
| | |
+--------------+--------------------------------------------------------------+
|references | spec_cpu2006_ |
| | |
| | ETSI-NFV-TST001 |
| | |
+--------------+--------------------------------------------------------------+
|pre-test | To run and install SPEC CPU2006, the following are required: |
|conditions | * For SPECint2006: Both C99 and C++98 compilers; |
| | * For SPECfp2006: All three of C99, C++98 and Fortran-95 |
| | compilers; |
| | * At least 8GB of disk space availabile on the system. |
| | |
+--------------+--------------------------------------------------------------+
|test sequence | description and expected result |
| | |
+--------------+--------------------------------------------------------------+
|step 1 | cpu2006-1.2.iso has been saved under the yardstick/resources |
| | folder (e.g. /home/opnfv/repos/yardstick/yardstick/resources |
| | /cpu2006-1.2.iso). Additional, to use your custom runspec |
| | config file you can save it under the yardstick/resources/ |
| | files folder and specify the config file name in the |
| | runspec_config parameter. |
| | |
+--------------+--------------------------------------------------------------+
|step 2 | Upload SPEC CPU2006 ISO to the target server and install |
| | SPEC CPU2006 via ansible. |
| | |
+--------------+--------------------------------------------------------------+
|step 3 | Yardstick is connected with the target server by using ssh. |
| | If custom runspec config file is used, this file is copyied |
| | from yardstick to the target server via the ssh tunnel. |
| | |
+--------------+--------------------------------------------------------------+
|step 4 | SPEC CPU2006 benchmark is invoked and SPEC CPU 2006 metrics |
| | are generated. |
| | |
+--------------+--------------------------------------------------------------+
|step 5 | Text, HTML, CSV, PDF, and Configuration file outputs for the |
| | SPEC CPU 2006 metrics are fetch from the server and stored |
| | under /tmp/result folder. |
| | |
+--------------+--------------------------------------------------------------+
|step 6 | uninstall SPEC CPU2006 and remove cpu2006-1.2.iso from the |
| | target server . |
| | |
+--------------+--------------------------------------------------------------+
|test verdict | None. SPEC CPU2006 results are collected and stored. |
| | |
+--------------+--------------------------------------------------------------+
|
