diff options
| -rw-r--r-- | docs/release/index.rst | 13 | ||||
| -rw-r--r-- | docs/release/release-notes.rst | 413 |
2 files changed, 426 insertions, 0 deletions
diff --git a/docs/release/index.rst b/docs/release/index.rst new file mode 100644 index 000000000..c9cadc539 --- /dev/null +++ b/docs/release/index.rst @@ -0,0 +1,13 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Ericsson AB and others. + +====================== +Yardstick Release Note +====================== + +.. toctree:: + :maxdepth: 2 + + release-notes diff --git a/docs/release/release-notes.rst b/docs/release/release-notes.rst new file mode 100644 index 000000000..23f2f012e --- /dev/null +++ b/docs/release/release-notes.rst @@ -0,0 +1,413 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Ericsson AB and others. + + +============================================ +OPNFV Brahmaputra Release Note for Yardstick +============================================ + +.. toctree:: + :maxdepth: 2 + +.. _Yardstick: https://wiki.opnfv.org/yardstick + +.. _Dashboard: http://130.211.154.108/grafana/dashboard/db/yardstick-main + +.. _NFV-TST001: https://docbox.etsi.org/ISG/NFV/Open/Drafts/TST001_-_Pre-deployment_Validation/ + + +Abstract +======== + +This document compiles the release notes for the OPNFV Brahmaputra release +for Yardstick framework as well as Yardstick_ Project deliverables. + +License +======= + +The *Yardstick framework*, the *Yardstick test cases* and the *ApexLake* +experimental framework are opensource software, licensed under the terms of the +Apache License, Version 2.0. + + +Version History +=============== + ++---------------+--------------------+---------------------------------+ +| *Date* | *Version* | *Comment* | +| | | | ++---------------+--------------------+---------------------------------+ +| Feb 25th,2016 | 1.0 | Brahmaputra release | +| | | | ++---------------+--------------------+---------------------------------+ + + +Important Notes +=============== + +The software delivered in the OPNFV Yardstick_ Project, comprising the +*Yardstick framework*, the *Yardstick test cases* and the experimental +framework *Apex Lake* is a realization of the methodology in ETSI-ISG +NFV-TST001_. + +The *Yardstick* framework is *installer*, *infrastructure* and *application* +independent. + + +Summary +======= + +This Brahmaputra release provides *Yardstick* as a framework for NFVI testing +and OPNFV feature testing, automated in the OPNFV CI pipeline, including: + +* Documentation generated with Sphinx + + * User Guide + + * Code Documentation + + * Release notes (this document) + + * Results + +* Automated Yardstick test suite (daily, weekly) + + * Jenkins Jobs for OPNFV community labs + +* Automated Yardstick test results visualization + + * Dashboard_ using Grafana (user:opnfv/password: opnfv), influxDB used as + backend + +* Yardstick framework source code + +* Yardstick test cases yaml files + +For Brahmaputra release, the *Yardstick framework* is used for the following +testing: + +* OPNFV platform testing - generic test cases to measure the categories: + + * Compute + + * Network + + * Storage + +* Test cases for the following OPNFV Projects: + + * High Availability + + * IPv6 + + * KVM + + * Parser + +The *Yardstick framework* is developed in the OPNFV community, by the +Yardstick_ team. + +.. note:: The test case description template used for the Yardstick test cases + is based on the document ETSI-ISG NFV-TST001_; the results report template + used for the Yardstick results is based on the IEEE Std 829-2008. + + +Release Data +============ + ++--------------------------------------+--------------------------------------+ +| **Project** | Yardstick | +| | | ++--------------------------------------+--------------------------------------+ +| **Repo/tag** | yardstick/brahmaputra.1.0 | +| | | ++--------------------------------------+--------------------------------------+ +| **Yardstick Docker image tag** | brahmaputra.1.0 | +| | | ++--------------------------------------+--------------------------------------+ +| **Release designation** | Brahmaputra | +| | | ++--------------------------------------+--------------------------------------+ +| **Release date** | Feb 25th, 2016 | +| | | ++--------------------------------------+--------------------------------------+ +| **Purpose of the delivery** | OPNFV Brahmaputra release | +| | | ++--------------------------------------+--------------------------------------+ + + +Version Change +-------------- + +Module Version Changes +~~~~~~~~~~~~~~~~~~~~~~ + +This is the first tracked release of Yardstick. It is based on following +upstream versions: + +- OpenStack Liberty + +- OpenDaylight Beryllium + + +Document Version Changes +~~~~~~~~~~~~~~~~~~~~~~~~ + +This is the first tracked version of the Yardstick framework in OPNFV. +It includes the following documentation: + +- Yardstick User Guide + +- Yardstick Code Documentation + +- Yardstick Release Notes for Yardstick + +- Test Results report for Brahmaputra testing with Yardstick + + +Reason for Version +------------------ + +Feature additions +~~~~~~~~~~~~~~~~~ + +This is the first tracked version of OPNFV Yardstick. + + +Corrected Faults +~~~~~~~~~~~~~~~~ + +This is the first tracked version of OPNFV Yardstick. + + +Known Faults +~~~~~~~~~~~~ + + ++----------------------------+------------------------------------------------+ +| **JIRA REFERENCE** | **SLOGAN** | +| | | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-175 | Running test suite, if a test cases running | +| | failed, the test is stopped. | +| | | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-176 | Fix plotter bug since Output format has been | +| | changed. | +| | | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-216 | ArgsAlreadyParsedError: arguments already | +| | parsed: cannot register CLI option. | +| | | ++----------------------------+------------------------------------------------+ + +.. note:: The faults not related to *Yardstick* framework, addressing scenarios + which were not fully verified, are listed in the OPNFV installer's release + notes. + + +Deliverables +------------ + +Software Deliverables +~~~~~~~~~~~~~~~~~~~~~ + +**Yardstick framework source code <brahmaputra.1.0>** + ++--------------------------------------+--------------------------------------+ +| **Project** | Yardstick | +| | | ++--------------------------------------+--------------------------------------+ +| **Repo/tag** | yardstick/brahmaputra.1.0 | +| | | ++--------------------------------------+--------------------------------------+ +| **Yardstick Docker image tag** | brahmaputra.1.0 | +| | | ++--------------------------------------+--------------------------------------+ +| **Release designation** | Brahmaputra | +| | | ++--------------------------------------+--------------------------------------+ +| **Release date** | Feb 25th, 2016 | +| | | ++--------------------------------------+--------------------------------------+ +| **Purpose of the delivery** | OPNFV Brahmaputra release | +| | | ++--------------------------------------+--------------------------------------+ + +**Contexts** + ++---------------------+-------------------------------------------------------+ +| **Context** | **Description** | +| | | ++---------------------+-------------------------------------------------------+ +| *Heat* | Models orchestration using OpenStack Heat | +| | | ++---------------------+-------------------------------------------------------+ +| *Node* | Models Baremetal, Controller, Compute | +| | | ++---------------------+-------------------------------------------------------+ + + +**Runners** + ++---------------------+-------------------------------------------------------+ +| **Runner** | **Description** | +| | | ++---------------------+-------------------------------------------------------+ +| *Arithmetic* | Steps every run arithmetically according to specified | +| | input value | +| | | ++---------------------+-------------------------------------------------------+ +| *Duration* | Runs for a specified period of time | +| | | ++---------------------+-------------------------------------------------------+ +| *Iteration* | Runs for a specified number of iterations | +| | | ++---------------------+-------------------------------------------------------+ +| *Sequence* | Selects input value to a scenario from an input file | +| | and runs all entries sequentially | +| | | ++---------------------+-------------------------------------------------------+ + +**Scenarios** + ++---------------------+-------------------------------------------------------+ +| **Category** | **Delivered** | +| | | ++---------------------+-------------------------------------------------------+ +| *Availability* | Attacker: | +| | | +| | * baremetal, process | +| | | +| | HA tools: | +| | | +| | * check host, openstack, process, service | +| | * kill process | +| | * start/stop service | +| | | +| | Monitor: | +| | | +| | * command, process | +| | | ++---------------------+-------------------------------------------------------+ +| *Compute* | * cpuload | +| | | +| | * cyclictest | +| | | +| | * lmbench | +| | | +| | * perf | +| | | +| | * unixbench | +| | | ++---------------------+-------------------------------------------------------+ +| *Networking* | * iperf3 | +| | | +| | * netperf | +| | | +| | * ping | +| | | +| | * ping6 | +| | | +| | * pktgen | +| | | +| | * sfc | +| | | +| | * sfc with tacker | +| | | +| | * vtc instantion validation | +| | | +| | * vtc instantion validation with noisy neighbors | +| | | +| | * vtc throughput | +| | | +| | * vtc throughput in the presence of noisy neighbors | +| | | ++---------------------+-------------------------------------------------------+ +| *Parser* | Tosca2Heat | +| | | ++---------------------+-------------------------------------------------------+ +| *Storage* | fio | +| | | ++---------------------+-------------------------------------------------------+ + +**API to Other Frameworks** + ++---------------------+-------------------------------------------------------+ +| **Framework** | **Description** | +| | | ++---------------------+-------------------------------------------------------+ +| *ApexLake* | Experimental framework that enables the user to | +| | validate NFVI from the perspective of a VNF. | +| | A virtual Traffic Classifier is utilized as VNF. | +| | Enables experiments with SR-IOV on Compute Node. | +| | | ++---------------------+-------------------------------------------------------+ + +**Test Results Output** + ++-----------------------------+-----------------------------------------------+ +| **Dispatcher** | **Description** | +| | | ++-----------------------------+-----------------------------------------------+ +| file | Log to a file. | +| | | ++-----------------------------+-----------------------------------------------+ +| http | Post data to html. | +| | | ++-----------------------------+-----------------------------------------------+ +| influxdb | Post data to influxdB. | +| | | ++-----------------------------+-----------------------------------------------+ + + +Delivered Test cases +~~~~~~~~~~~~~~~~~~~~ + +* Generic NFVI test cases + + * OPNFV_YARDSTICK_TCOO1 - NW Performance + + * OPNFV_YARDSTICK_TCOO2 - NW Latency + + * OPNFV_YARDSTICK_TCOO5 - Storage Performance + + * OPNFV_YARDSTICK_TCOO8 - Packet Loss Extended Test + + * OPNFV_YARDSTICK_TCOO9 - Packet Loss + + * OPNFV_YARDSTICK_TCO10 - Memory Latency + + * OPNFV_YARDSTICK_TCO11 - Packet Delay Variation Between VMs + + * OPNFV_YARDSTICK_TCO12 - Memory Bandwidth + + * OPNFV_YARDSTICK_TCO14 - Processing Speed + + * OPNFV_YARDSTICK_TCO24 - CPU Load + + * OPNFV_YARDSTICK_TCO37 - Latency, CPU Load, Throughput, Packet Loss + + * OPNFV_YARDSTICK_TCO38 - Latency, CPU Load, Throughput, Packet Loss Extended + Test + + +* Test Cases for OPNFV HA Project: + + * OPNFV_YARDSTICK_TCO19 - HA: Control node Openstack service down + + * OPNFV_YARDSTICK_TC025 - HA: OpenStacK Controller Node abnormally down + +* Test Case for OPNFV IPv6 Project: + + * OPNFV_YARDSTICK_TCO27 - IPv6 connectivity + +* Test Case for OPNFV KVM Project: + + * OPNFV_YARDSTICK_TCO28 - KVM Latency measurements + +* Test Case for OPNFV Parser Project: + + * OPNFV_YARDSTICK_TCO40 - Verify Parser Yang-to-Tosca |