aboutsummaryrefslogtreecommitdiffstats
path: root/docs
diff options
context:
space:
mode:
Diffstat (limited to 'docs')
-rw-r--r--docs/release/release-notes/release-notes.rst66
-rw-r--r--docs/testing/user/userguide/opnfv_yardstick_tc078.rst133
2 files changed, 188 insertions, 11 deletions
diff --git a/docs/release/release-notes/release-notes.rst b/docs/release/release-notes/release-notes.rst
index d89f9ed24..6d55ada86 100644
--- a/docs/release/release-notes/release-notes.rst
+++ b/docs/release/release-notes/release-notes.rst
@@ -38,7 +38,11 @@ Version History
| *Date* | *Version* | *Comment* |
| | | |
+----------------+--------------------+---------------------------------+
-| | 3.0 | Yardstick for Danube release |
+| | 3.1 | Yardstick for Danube release |
+| | | |
+| | | Note: The 3.1 tag is due to git |
+| | | tag issue during Danube 3.0 |
+| | | release |
| | | |
+----------------+--------------------+---------------------------------+
| May 4th, 2017 | 2.0 | Yardstick for Danube release |
@@ -139,19 +143,19 @@ Release Data
| **Project** | Yardstick |
| | |
+--------------------------------------+--------------------------------------+
-| **Repo/tag** | yardstick/Danube.2.0 |
+| **Repo/tag** | yardstick/Danube.3.1 |
| | |
+--------------------------------------+--------------------------------------+
-| **Yardstick Docker image tag** | Danube.2.0 |
+| **Yardstick Docker image tag** | Danube.3.1 |
| | |
+--------------------------------------+--------------------------------------+
| **Release designation** | Danube |
| | |
+--------------------------------------+--------------------------------------+
-| **Release date** | May 4th, 2017 |
+| **Release date** | July 14th, 2017 |
| | |
+--------------------------------------+--------------------------------------+
-| **Purpose of the delivery** | OPNFV Danube release 2.0 |
+| **Purpose of the delivery** | OPNFV Danube release 3.0 |
| | |
+--------------------------------------+--------------------------------------+
@@ -171,7 +175,7 @@ Software Deliverables
---------------------
- - The Yardstick Docker image: https://hub.docker.com/r/opnfv/yardstick (tag: danube.2.0)
+ - The Yardstick Docker image: https://hub.docker.com/r/opnfv/yardstick (tag: danube.3.1)
**Contexts**
@@ -515,7 +519,7 @@ Feature additions
Scenario Matrix
===============
-For Danube 2.0, Yardstick was tested on the following scenarios:
+For Danube 3.0, Yardstick was tested on the following scenarios:
+-------------------------+---------+---------+---------+---------+
| Scenario | Apex | Compass | Fuel | Joid |
@@ -613,10 +617,50 @@ Known Issues/Faults
Corrected Faults
----------------
+Danube.3.1:
+
++----------------------------+------------------------------------------------+
+| **JIRA REFERENCE** | **DESCRIPTION** |
+| | |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-714 | Add yardstick env influxdb/grafana command for |
+| | CentOS |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-655 | Monitor command in tc019 may not show the |
+| | real nova-api service status |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-397 | HA testing framework improvement |
+| | |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-660 | Improve monitor_process pass criteria |
+| | |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-657 | HA monitor_multi bug, |
+| | KeyError: 'max_outage_time' |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-647 | TC025 fault_type value is wrong when using |
+| | baremetal pod scripts |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-659 | Terminate openstack service process using kill |
+| | command in HA test cases |
++----------------------------+------------------------------------------------+
+| JIRA: ARMBAND-275 | Yardstick TC005 fails with |
+| | "Cannot map zero-fill pages" error |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-561 | Bugfix: AttributeError: 'dict' object has no |
+| | attribute 'split' if run sample/ping-hot.yaml |
++----------------------------+------------------------------------------------+
+| JIRA: ARMBAND-268 | ERROR No JSON object could be decoded from |
+| | LMBENCH in TC010 |
++----------------------------+------------------------------------------------+
+| JIRA: YARDSTICK-680 | storperf test case tc074 do not get results |
+| | |
++----------------------------+------------------------------------------------+
+
Danube.2.0:
+----------------------------+------------------------------------------------+
-| **JIRA REFERENCE** | **SLOGAN** |
+| **JIRA REFERENCE** | **DESCRIPTION** |
| | |
+----------------------------+------------------------------------------------+
| JIRA: YARDSTICK-608 | Set work directory in Yardstick container |
@@ -662,7 +706,7 @@ Danube.2.0:
Danube.1.0:
+----------------------------+------------------------------------------------+
-| **JIRA REFERENCE** | **SLOGAN** |
+| **JIRA REFERENCE** | **DESCRIPTION** |
| | |
+----------------------------+------------------------------------------------+
| JIRA: YARDSTICK-599 | Could not load EntryPoint.parse when using |
@@ -673,7 +717,7 @@ Danube.1.0:
+----------------------------+------------------------------------------------+
-Danube 2.0 known restrictions/issues
+Danube 3.1 known restrictions/issues
====================================
+-----------+-----------+----------------------------------------------+
| Installer | Scenario | Issue |
@@ -695,7 +739,7 @@ Open JIRA tickets
=================
+----------------------------+------------------------------------------------+
-| **JIRA REFERENCE** | **SLOGAN** |
+| **JIRA REFERENCE** | **DESCRIPTION** |
| | |
+----------------------------+------------------------------------------------+
| JIRA: YARDSTICK-626 | Fio and Lmbench don't work in Ubuntu-arm64 |
diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc078.rst b/docs/testing/user/userguide/opnfv_yardstick_tc078.rst
new file mode 100644
index 000000000..560d55e07
--- /dev/null
+++ b/docs/testing/user/userguide/opnfv_yardstick_tc078.rst
@@ -0,0 +1,133 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+*************************************
+Yardstick Test Case Description TC078
+*************************************
+
+.. _spec_cpu2006: https://www.spec.org/cpu2006/
+
++-----------------------------------------------------------------------------+
+|Compute Performance |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC078_SPEC CPU 2006 |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | compute-intensive performance |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The purpose of TC078 is to evaluate the IaaS compute |
+| | performance by using SPEC CPU 2006 benchmark. The SPEC CPU |
+| | 2006 benchmark has several different ways to measure |
+| | computer performance. One way is to measure how fast the |
+| | computer completes a single task; this is called a speed |
+| | measurement. Another way is to measure how many tasks |
+| | computer can accomplish in a certain amount of time; this is |
+| | called a throughput, capacity or rate measurement. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | SPEC CPU 2006 |
+| | |
+| | The SPEC CPU 2006 benchmark is SPEC's industry-standardized, |
+| | CPU-intensive benchmark suite, stressing a system's |
+| | processor, memory subsystem and compiler. This benchmark |
+| | suite includes the SPECint benchmarks and the SPECfp |
+| | benchmarks. The SPECint 2006 benchmark contains 12 different |
+| | enchmark tests and the SPECfp 2006 benchmark contains 19 |
+| | different benchmark tests. |
+| | |
+| | SPEC CPU 2006 is not always part of a Linux distribution. |
+| | SPEC requires that users purchase a license and agree with |
+| | their terms and conditions. For this test case, users must |
+| | manually download cpu2006-1.2.iso from the SPEC website and |
+| | save it under the yardstick/resources folder (e.g. /home/ |
+| | opnfv/repos/yardstick/yardstick/resources/cpu2006-1.2.iso) |
+| | SPEC CPU® 2006 benchmark is available for purchase via the |
+| | SPEC order form (https://www.spec.org/order.html). |
+| | |
++--------------+--------------------------------------------------------------+
+|test | This test case uses SPEC CPU 2006 benchmark to measure |
+|description | compute-intensive performance of hosts. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | file: spec_cpu.yaml (in the 'samples' directory) |
+| | |
+| | benchmark_subset is set to int. |
+| | |
+| | SLA is not available in this test case. |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | Test can be configured with different: |
+| | |
+| | * benchmark_subset - a subset of SPEC CPU2006 benchmarks to |
+| | run; |
+| | * SPECint_benchmark - a SPECint benchmark to run; |
+| | * SPECint_benchmark - a SPECfp benchmark to run; |
+| | * output_format - desired report format; |
+| | * runspec_config - SPEC CPU2006 config file provided to the |
+| | runspec binary; |
+| | * runspec_iterations - the number of benchmark iterations |
+| | to execute. For a reportable run, must be 3; |
+| | * runspec_tune - tuning to use (base, peak, or all). For a |
+| | reportable run, must be either base or all. Reportable |
+| | runs do base first, then (optionally) peak; |
+| | * runspec_size - size of input data to run (test, train, or |
+| | ref). Reportable runs ensure that your binaries can |
+| | produce correct results with the test and train workloads |
+| | |
++--------------+--------------------------------------------------------------+
+|usability | This test case is used for executing SPEC CPU 2006 benchmark |
+| | physical servers. The SPECint 2006 benchmark takes |
+| | approximately 5 hours. |
+| | |
++--------------+--------------------------------------------------------------+
+|references | spec_cpu2006_ |
+| | |
+| | ETSI-NFV-TST001 |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | To run and install SPEC CPU2006, the following are required: |
+|conditions | * For SPECint2006: Both C99 and C++98 compilers; |
+| | * For SPECfp2006: All three of C99, C++98 and Fortran-95 |
+| | compilers; |
+| | * At least 8GB of disk space availabile on the system. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | cpu2006-1.2.iso has been saved under the yardstick/resources |
+| | folder (e.g. /home/opnfv/repos/yardstick/yardstick/resources |
+| | /cpu2006-1.2.iso). Additional, to use your custom runspec |
+| | config file you can save it under the yardstick/resources/ |
+| | files folder and specify the config file name in the |
+| | runspec_config parameter. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Upload SPEC CPU2006 ISO to the target server and install |
+| | SPEC CPU2006 via ansible. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | Yardstick is connected with the target server by using ssh. |
+| | If custom runspec config file is used, this file is copyied |
+| | from yardstick to the target server via the ssh tunnel. |
+--------------+---------------------------------------------------------------+
+|step 4 | SPEC CPU2006 benchmark is invoked and SPEC CPU 2006 metrics |
+| | are generated. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 5 | Text, HTML, CSV, PDF, and Configuration file outputs for the |
+| | SPEC CPU 2006 metrics are fetch from the server and stored |
+| | under /tmp/result folder. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 6 | uninstall SPEC CPU2006 and remove cpu2006-1.2.iso from the |
+| | target server . |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | None. SPEC CPU2006 results are collected and stored. |
+| | |
++--------------+--------------------------------------------------------------+