diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/release/release-notes/release-notes.rst | 66 | ||||
| -rw-r--r-- | docs/testing/user/userguide/opnfv_yardstick_tc078.rst | 133 |
2 files changed, 188 insertions, 11 deletions
diff --git a/docs/release/release-notes/release-notes.rst b/docs/release/release-notes/release-notes.rst index d89f9ed24..6d55ada86 100644 --- a/docs/release/release-notes/release-notes.rst +++ b/docs/release/release-notes/release-notes.rst @@ -38,7 +38,11 @@ Version History | *Date* | *Version* | *Comment* | | | | | +----------------+--------------------+---------------------------------+ -| | 3.0 | Yardstick for Danube release | +| | 3.1 | Yardstick for Danube release | +| | | | +| | | Note: The 3.1 tag is due to git | +| | | tag issue during Danube 3.0 | +| | | release | | | | | +----------------+--------------------+---------------------------------+ | May 4th, 2017 | 2.0 | Yardstick for Danube release | @@ -139,19 +143,19 @@ Release Data | **Project** | Yardstick | | | | +--------------------------------------+--------------------------------------+ -| **Repo/tag** | yardstick/Danube.2.0 | +| **Repo/tag** | yardstick/Danube.3.1 | | | | +--------------------------------------+--------------------------------------+ -| **Yardstick Docker image tag** | Danube.2.0 | +| **Yardstick Docker image tag** | Danube.3.1 | | | | +--------------------------------------+--------------------------------------+ | **Release designation** | Danube | | | | +--------------------------------------+--------------------------------------+ -| **Release date** | May 4th, 2017 | +| **Release date** | July 14th, 2017 | | | | +--------------------------------------+--------------------------------------+ -| **Purpose of the delivery** | OPNFV Danube release 2.0 | +| **Purpose of the delivery** | OPNFV Danube release 3.0 | | | | +--------------------------------------+--------------------------------------+ @@ -171,7 +175,7 @@ Software Deliverables --------------------- - - The Yardstick Docker image: https://hub.docker.com/r/opnfv/yardstick (tag: danube.2.0) + - The Yardstick Docker image: https://hub.docker.com/r/opnfv/yardstick (tag: danube.3.1) **Contexts** @@ -515,7 +519,7 @@ Feature additions Scenario Matrix =============== -For Danube 2.0, Yardstick was tested on the following scenarios: +For Danube 3.0, Yardstick was tested on the following scenarios: +-------------------------+---------+---------+---------+---------+ | Scenario | Apex | Compass | Fuel | Joid | @@ -613,10 +617,50 @@ Known Issues/Faults Corrected Faults ---------------- +Danube.3.1: + ++----------------------------+------------------------------------------------+ +| **JIRA REFERENCE** | **DESCRIPTION** | +| | | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-714 | Add yardstick env influxdb/grafana command for | +| | CentOS | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-655 | Monitor command in tc019 may not show the | +| | real nova-api service status | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-397 | HA testing framework improvement | +| | | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-660 | Improve monitor_process pass criteria | +| | | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-657 | HA monitor_multi bug, | +| | KeyError: 'max_outage_time' | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-647 | TC025 fault_type value is wrong when using | +| | baremetal pod scripts | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-659 | Terminate openstack service process using kill | +| | command in HA test cases | ++----------------------------+------------------------------------------------+ +| JIRA: ARMBAND-275 | Yardstick TC005 fails with | +| | "Cannot map zero-fill pages" error | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-561 | Bugfix: AttributeError: 'dict' object has no | +| | attribute 'split' if run sample/ping-hot.yaml | ++----------------------------+------------------------------------------------+ +| JIRA: ARMBAND-268 | ERROR No JSON object could be decoded from | +| | LMBENCH in TC010 | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-680 | storperf test case tc074 do not get results | +| | | ++----------------------------+------------------------------------------------+ + Danube.2.0: +----------------------------+------------------------------------------------+ -| **JIRA REFERENCE** | **SLOGAN** | +| **JIRA REFERENCE** | **DESCRIPTION** | | | | +----------------------------+------------------------------------------------+ | JIRA: YARDSTICK-608 | Set work directory in Yardstick container | @@ -662,7 +706,7 @@ Danube.2.0: Danube.1.0: +----------------------------+------------------------------------------------+ -| **JIRA REFERENCE** | **SLOGAN** | +| **JIRA REFERENCE** | **DESCRIPTION** | | | | +----------------------------+------------------------------------------------+ | JIRA: YARDSTICK-599 | Could not load EntryPoint.parse when using | @@ -673,7 +717,7 @@ Danube.1.0: +----------------------------+------------------------------------------------+ -Danube 2.0 known restrictions/issues +Danube 3.1 known restrictions/issues ==================================== +-----------+-----------+----------------------------------------------+ | Installer | Scenario | Issue | @@ -695,7 +739,7 @@ Open JIRA tickets ================= +----------------------------+------------------------------------------------+ -| **JIRA REFERENCE** | **SLOGAN** | +| **JIRA REFERENCE** | **DESCRIPTION** | | | | +----------------------------+------------------------------------------------+ | JIRA: YARDSTICK-626 | Fio and Lmbench don't work in Ubuntu-arm64 | diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc078.rst b/docs/testing/user/userguide/opnfv_yardstick_tc078.rst new file mode 100644 index 000000000..560d55e07 --- /dev/null +++ b/docs/testing/user/userguide/opnfv_yardstick_tc078.rst @@ -0,0 +1,133 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Technologies Co.,Ltd and others. + +************************************* +Yardstick Test Case Description TC078 +************************************* + +.. _spec_cpu2006: https://www.spec.org/cpu2006/ + ++-----------------------------------------------------------------------------+ +|Compute Performance | +| | ++--------------+--------------------------------------------------------------+ +|test case id | OPNFV_YARDSTICK_TC078_SPEC CPU 2006 | +| | | ++--------------+--------------------------------------------------------------+ +|metric | compute-intensive performance | +| | | ++--------------+--------------------------------------------------------------+ +|test purpose | The purpose of TC078 is to evaluate the IaaS compute | +| | performance by using SPEC CPU 2006 benchmark. The SPEC CPU | +| | 2006 benchmark has several different ways to measure | +| | computer performance. One way is to measure how fast the | +| | computer completes a single task; this is called a speed | +| | measurement. Another way is to measure how many tasks | +| | computer can accomplish in a certain amount of time; this is | +| | called a throughput, capacity or rate measurement. | +| | | ++--------------+--------------------------------------------------------------+ +|test tool | SPEC CPU 2006 | +| | | +| | The SPEC CPU 2006 benchmark is SPEC's industry-standardized, | +| | CPU-intensive benchmark suite, stressing a system's | +| | processor, memory subsystem and compiler. This benchmark | +| | suite includes the SPECint benchmarks and the SPECfp | +| | benchmarks. The SPECint 2006 benchmark contains 12 different | +| | enchmark tests and the SPECfp 2006 benchmark contains 19 | +| | different benchmark tests. | +| | | +| | SPEC CPU 2006 is not always part of a Linux distribution. | +| | SPEC requires that users purchase a license and agree with | +| | their terms and conditions. For this test case, users must | +| | manually download cpu2006-1.2.iso from the SPEC website and | +| | save it under the yardstick/resources folder (e.g. /home/ | +| | opnfv/repos/yardstick/yardstick/resources/cpu2006-1.2.iso) | +| | SPEC CPU® 2006 benchmark is available for purchase via the | +| | SPEC order form (https://www.spec.org/order.html). | +| | | ++--------------+--------------------------------------------------------------+ +|test | This test case uses SPEC CPU 2006 benchmark to measure | +|description | compute-intensive performance of hosts. | +| | | ++--------------+--------------------------------------------------------------+ +|configuration | file: spec_cpu.yaml (in the 'samples' directory) | +| | | +| | benchmark_subset is set to int. | +| | | +| | SLA is not available in this test case. | +| | | ++--------------+--------------------------------------------------------------+ +|applicability | Test can be configured with different: | +| | | +| | * benchmark_subset - a subset of SPEC CPU2006 benchmarks to | +| | run; | +| | * SPECint_benchmark - a SPECint benchmark to run; | +| | * SPECint_benchmark - a SPECfp benchmark to run; | +| | * output_format - desired report format; | +| | * runspec_config - SPEC CPU2006 config file provided to the | +| | runspec binary; | +| | * runspec_iterations - the number of benchmark iterations | +| | to execute. For a reportable run, must be 3; | +| | * runspec_tune - tuning to use (base, peak, or all). For a | +| | reportable run, must be either base or all. Reportable | +| | runs do base first, then (optionally) peak; | +| | * runspec_size - size of input data to run (test, train, or | +| | ref). Reportable runs ensure that your binaries can | +| | produce correct results with the test and train workloads | +| | | ++--------------+--------------------------------------------------------------+ +|usability | This test case is used for executing SPEC CPU 2006 benchmark | +| | physical servers. The SPECint 2006 benchmark takes | +| | approximately 5 hours. | +| | | ++--------------+--------------------------------------------------------------+ +|references | spec_cpu2006_ | +| | | +| | ETSI-NFV-TST001 | +| | | ++--------------+--------------------------------------------------------------+ +|pre-test | To run and install SPEC CPU2006, the following are required: | +|conditions | * For SPECint2006: Both C99 and C++98 compilers; | +| | * For SPECfp2006: All three of C99, C++98 and Fortran-95 | +| | compilers; | +| | * At least 8GB of disk space availabile on the system. | +| | | ++--------------+--------------------------------------------------------------+ +|test sequence | description and expected result | +| | | ++--------------+--------------------------------------------------------------+ +|step 1 | cpu2006-1.2.iso has been saved under the yardstick/resources | +| | folder (e.g. /home/opnfv/repos/yardstick/yardstick/resources | +| | /cpu2006-1.2.iso). Additional, to use your custom runspec | +| | config file you can save it under the yardstick/resources/ | +| | files folder and specify the config file name in the | +| | runspec_config parameter. | +| | | ++--------------+--------------------------------------------------------------+ +|step 2 | Upload SPEC CPU2006 ISO to the target server and install | +| | SPEC CPU2006 via ansible. | +| | | ++--------------+--------------------------------------------------------------+ +|step 3 | Yardstick is connected with the target server by using ssh. | +| | If custom runspec config file is used, this file is copyied | +| | from yardstick to the target server via the ssh tunnel. | +--------------+---------------------------------------------------------------+ +|step 4 | SPEC CPU2006 benchmark is invoked and SPEC CPU 2006 metrics | +| | are generated. | +| | | ++--------------+--------------------------------------------------------------+ +|step 5 | Text, HTML, CSV, PDF, and Configuration file outputs for the | +| | SPEC CPU 2006 metrics are fetch from the server and stored | +| | under /tmp/result folder. | +| | | ++--------------+--------------------------------------------------------------+ +|step 6 | uninstall SPEC CPU2006 and remove cpu2006-1.2.iso from the | +| | target server . | +| | | ++--------------+--------------------------------------------------------------+ +|test verdict | None. SPEC CPU2006 results are collected and stored. | +| | | ++--------------+--------------------------------------------------------------+ |