yardstick -

about summary refs log tree commit diff stats
Age	Commit message (Expand)	Author	Files	Lines
2016-12-01	Add swagger support for Rest API	chenjiankun	1	-0/+0
Open Platform for NFV and OPNFV are trademarks of the Open Platform for NFV Project, Inc.
Linux Foundation is a registered trademark of The Linux Foundation. Linux is a registered trademark of Linus Torvalds.
Please see our terms of use, trademark policy, and privacy policy.
'>65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134
.. This work is licensed under a Creative Commons Attribution 4.0 International
.. License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.

*************************************
Yardstick Test Case Description TC078
*************************************

.. _spec_cpu2006: https://www.spec.org/cpu2006/

+-----------------------------------------------------------------------------+
|Compute Performance                                                          |
|                                                                             |
+--------------+--------------------------------------------------------------+
|test case id  | OPNFV_YARDSTICK_TC078_SPEC CPU 2006                          |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|metric        | compute-intensive performance                                |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|test purpose  | The purpose of TC078 is to evaluate the IaaS compute         |
|              | performance by using SPEC CPU 2006 benchmark. The SPEC CPU   |
|              | 2006 benchmark has several different ways to measure         |
|              | computer performance. One way is to measure how fast the     |
|              | computer completes a single task; this is called a speed     |
|              | measurement. Another way is to measure how many tasks        |
|              | computer can accomplish in a certain amount of time; this is |
|              | called a throughput, capacity or rate measurement.           |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|test tool     | SPEC CPU 2006                                                |
|              |                                                              |
|              | The SPEC CPU 2006 benchmark is SPEC's industry-standardized, |
|              | CPU-intensive benchmark suite, stressing a system's          |
|              | processor, memory subsystem and compiler. This benchmark     |
|              | suite includes the SPECint benchmarks and the SPECfp         |
|              | benchmarks. The SPECint 2006 benchmark contains 12 different |
|              | enchmark tests and the SPECfp 2006 benchmark contains 19     |
|              | different benchmark tests.                                   |
|              |                                                              |
|              | SPEC CPU 2006 is not always part of a Linux distribution.    |
|              | SPEC requires that users purchase a license and agree with   |
|              | their terms and conditions. For this test case, users must   |
|              | manually download cpu2006-1.2.iso from the SPEC website and  |
|              | save it under the yardstick/resources folder (e.g. /home/    |
|              | opnfv/repos/yardstick/yardstick/resources/cpu2006-1.2.iso)   |
|              | SPEC CPU® 2006 benchmark is available for purchase via the   |
|              | SPEC order form (https://www.spec.org/order.html).           |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|test          | This test case uses SPEC CPU 2006 benchmark to measure       |
|description   | compute-intensive performance of hosts.                      |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|configuration | file: spec_cpu.yaml (in the 'samples' directory)             |
|              |                                                              |
|              | benchmark_subset is set to int.                              |
|              |                                                              |
|              | SLA is not available in this test case.                      |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|applicability | Test can be configured with different:                       |
|              |                                                              |
|              |  * benchmark_subset - a subset of SPEC CPU2006 benchmarks to |
|              |    run;                                                      |
|              |  * SPECint_benchmark - a SPECint benchmark to run;           |
|              |  * SPECint_benchmark - a SPECfp benchmark to run;            |
|              |  * output_format - desired report format;                    |
|              |  * runspec_config - SPEC CPU2006 config file provided to the |
|              |    runspec binary;                                           |
|              |  * runspec_iterations - the number of benchmark iterations   |
|              |    to execute. For a reportable run, must be 3;              |
|              |  * runspec_tune - tuning to use (base, peak, or all). For a  |
|              |    reportable run, must be either base or all. Reportable    |
|              |    runs do base first, then (optionally) peak;               |
|              |  * runspec_size - size of input data to run (test, train, or |
|              |    ref). Reportable runs ensure that your binaries can       |
|              |    produce correct results with the test and train workloads |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|usability     | This test case is used for executing SPEC CPU 2006 benchmark |
|              | physical servers. The SPECint 2006 benchmark takes           |
|              | approximately 5 hours.                                       |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|references    | spec_cpu2006_                                                |
|              |                                                              |
|              | ETSI-NFV-TST001                                              |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|pre-test      | To run and install SPEC CPU2006, the following are required: |
|conditions    |  * For SPECint2006: Both C99 and C++98 compilers;            |
|              |  * For SPECfp2006: All three of C99, C++98 and Fortran-95    |
|              |    compilers;                                                |
|              |  * At least 8GB of disk space availabile on the system.      |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|test sequence | description and expected result                              |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|step 1        | cpu2006-1.2.iso has been saved under the yardstick/resources |
|              | folder (e.g. /home/opnfv/repos/yardstick/yardstick/resources |
|              | /cpu2006-1.2.iso). Additional, to use your custom runspec    |
|              | config file you can save it under the yardstick/resources/   |
|              | files folder and specify the config file name in the         |
|              | runspec_config parameter.                                    |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|step 2        | Upload SPEC CPU2006 ISO to the target server and install     |
|              | SPEC CPU2006 via ansible.                                    |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|step 3        | Yardstick is connected with the target server by using ssh.  |
|              | If custom runspec config file is used, this file is copyied  |
|              | from yardstick to the target server via the ssh tunnel.      |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|step 4        | SPEC CPU2006 benchmark is invoked and SPEC CPU 2006 metrics  |
|              | are generated.                                               |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|step 5        | Text, HTML, CSV, PDF, and Configuration file outputs for the |
|              | SPEC CPU 2006 metrics are fetch from the server and stored   |
|              | under /tmp/result folder.                                    |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|step 6        | uninstall SPEC CPU2006 and remove cpu2006-1.2.iso from the   |
|              | target server .                                              |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|test verdict  | None. SPEC CPU2006 results are collected and stored.         |
|              |                                                              |
+--------------+--------------------------------------------------------------+