diff options
Diffstat (limited to 'docs/release/release-notes')
| -rw-r--r-- | docs/release/release-notes/release-notes.rst | 226 |
1 files changed, 119 insertions, 107 deletions
diff --git a/docs/release/release-notes/release-notes.rst b/docs/release/release-notes/release-notes.rst index 8df0776df..4631e9307 100644 --- a/docs/release/release-notes/release-notes.rst +++ b/docs/release/release-notes/release-notes.rst @@ -2,7 +2,7 @@ License ======= -OPNFV Colorado release note for Yardstick Docs +OPNFV Danube release note for Yardstick Docs are licensed under a Creative Commons Attribution 4.0 International License. You should have received a copy of the license along with this. If not, see <http://creativecommons.org/licenses/by/4.0/>. @@ -12,7 +12,7 @@ experimental framework are opensource software, licensed under the terms of the Apache License, Version 2.0. ========================================= -OPNFV Colorado Release Note for Yardstick +OPNFV Danube Release Note for Yardstick ========================================= .. toctree:: @@ -38,13 +38,13 @@ Version History | *Date* | *Version* | *Comment* | | | | | +----------------+--------------------+---------------------------------+ -| Dec 5th, 2016 | 3.0 | Yardstick for Colorado release | +| | 3.0 | Yardstick for Danube release | | | | | +----------------+--------------------+---------------------------------+ -| Oct 27th, 2016 | 2.0 | Yardstick for Colorado release | +| | 2.0 | Yardstick for Danube release | | | | | +----------------+--------------------+---------------------------------+ -| Aug 22nd, 2016 | 1.0 | Yardstick for Colorado release | +| Mar 31st, 2017 | 1.0 | Yardstick for Danube release | | | | | +----------------+--------------------+---------------------------------+ @@ -61,17 +61,17 @@ The *Yardstick* framework is *installer*, *infrastructure* and *application* independent. -OPNFV Colorado Release +OPNFV Danube Release ====================== -This Colorado release provides *Yardstick* as a framework for NFVI testing +This Danube release provides *Yardstick* as a framework for NFVI testing and OPNFV feature testing, automated in the OPNFV CI pipeline, including: * Documentation generated with Sphinx * User Guide - * Code Documentation + * Developer Guide * Release notes (this document) @@ -92,7 +92,7 @@ and OPNFV feature testing, automated in the OPNFV CI pipeline, including: * Yardstick pliug-in configration yaml files, plug-in install/remove scripts -For Colorado release, the *Yardstick framework* is used for the following +For Danube release, the *Yardstick framework* is used for the following testing: * OPNFV platform testing - generic test cases to measure the categories: @@ -103,6 +103,10 @@ testing: * Storage +* OPNFV platform network service benchmarking(NSB) + + * NSB + * Test cases for the following OPNFV Projects: * High Availability @@ -135,19 +139,19 @@ Release Data | **Project** | Yardstick | | | | +--------------------------------------+--------------------------------------+ -| **Repo/tag** | yardstick/colorado.3.0 | +| **Repo/tag** | yardstick/Danube.1.0 | | | | +--------------------------------------+--------------------------------------+ -| **Yardstick Docker image tag** | colorado.3.0 | +| **Yardstick Docker image tag** | Danube.1.0 | | | | +--------------------------------------+--------------------------------------+ -| **Release designation** | Colorado | +| **Release designation** | Danube | | | | +--------------------------------------+--------------------------------------+ -| **Release date** | December 5th, 2016 | +| **Release date** | March 31st, 2017 | | | | +--------------------------------------+--------------------------------------+ -| **Purpose of the delivery** | OPNFV Colorado release 3.0 | +| **Purpose of the delivery** | OPNFV Danube release 1.0 | | | | +--------------------------------------+--------------------------------------+ @@ -158,33 +162,33 @@ Deliverables Documents --------- - - User Guide: http://artifacts.opnfv.org/yardstick/colorado/docs/userguide/index.html + - User Guide: http://artifacts.opnfv.org/yardstick/Danube/docs/userguide/index.html - - Test Results: http://artifacts.opnfv.org/yardstick/colorado/docs/results/overview.html + - Test Results: http://artifacts.opnfv.org/yardstick/Danube/docs/results/overview.html Software Deliverables --------------------- -**Yardstick framework source code <colorado.3.0>** +**Yardstick framework source code <Danube.1.0>** +--------------------------------------+--------------------------------------+ | **Project** | Yardstick | | | | +--------------------------------------+--------------------------------------+ -| **Repo/tag** | yardstick/colorado.3.0 | +| **Repo/tag** | yardstick/Danube.1.0 | | | | +--------------------------------------+--------------------------------------+ -| **Yardstick Docker image tag** | colorado.3.0 | +| **Yardstick Docker image tag** | Danube.1.0 | | | | +--------------------------------------+--------------------------------------+ -| **Release designation** | Colorado | +| **Release designation** | Danube | | | | +--------------------------------------+--------------------------------------+ -| **Release date** | December 5th, 2016 | +| **Release date** | March 31st, 2017 | | | | +--------------------------------------+--------------------------------------+ -| **Purpose of the delivery** | OPNFV Colorado release | +| **Purpose of the delivery** | OPNFV Danube release | | | | +--------------------------------------+--------------------------------------+ @@ -201,6 +205,10 @@ Software Deliverables | *Node* | Models Baremetal, Controller, Compute | | | | +---------------------+-------------------------------------------------------+ +| *Standalone* | Models to deploy/undeploy NFVi infrastructure to run | +| | the VNF | +| | | ++---------------------+-------------------------------------------------------+ **Runners** @@ -295,6 +303,8 @@ Software Deliverables | | | | | * netutilization | | | | +| | * nstat | +| | | +---------------------+-------------------------------------------------------+ | *Parser* | Tosca2Heat | | | | @@ -307,6 +317,9 @@ Software Deliverables | *StorPerf* | storperf | | | | +---------------------+-------------------------------------------------------+ +| *NSB* | vPE thoughput test case | +| | | ++---------------------+-------------------------------------------------------+ **API to Other Frameworks** @@ -398,6 +411,8 @@ Delivered Test cases * OPNFV_YARDSTICK_TCO75 - Network Capacity and Scale + * OPNFV_YARDSTICK_TCO76 - Network frame error rate + * Test Cases for OPNFV HA Project: * OPNFV_YARDSTICK_TCO19 - HA: Control node Openstack service down @@ -473,7 +488,7 @@ upstream versions: - ONOS Goldeneye -- OpenStack Mitaka +- OpenStack Newton - OpenDaylight Beryllium @@ -481,77 +496,95 @@ upstream versions: Document Version Changes ------------------------ -This is the second tracked version of the Yardstick framework in OPNFV. +This is the fouth tracked version of the Yardstick framework in OPNFV. It includes the following documentation updates: -- Yardstick User Guide: added yardstick plugin chapter; added Store Other -Project's Test Results in InfluxDB chapter; Refine yardstick instantion chapter. +- Yardstick User Guide: add "network service benchmarking(NSB)" chapter; + add "Yardstick - NSB Testing -Installation" chapter; + Update Yardstick installation chapter; + +- Yardstick Developer Guide - Yardstick Code Documentation: no changes - Yardstick Release Notes for Yardstick: this document -- Test Results report for Colorado testing with Yardstick: updated listed of +- Test Results report for Danube testing with Yardstick: updated listed of verified scenarios and limitations Feature additions ----------------- - - Yardstick plugin - - Yardstick reporting - - StorPerf Integration - +- Yardstick ReST API support +- Network service benchmarking Scenario Matrix =============== -For Colorado 3.0, Yardstick was tested on the following scenarios: +For Danube 1.0, Yardstick was tested on the following scenarios: +-------------------------+---------+---------+---------+---------+ | Scenario | Apex | Compass | Fuel | Joid | +=========================+=========+=========+=========+=========+ -| os-nosdn-nofeature-noha | | | | X | +| os-nosdn-nofeature-noha | | | X | X | +-------------------------+---------+---------+---------+---------+ -| os-nosdn-nofeature-ha | X | | X | X | +| os-nosdn-nofeature-ha | X | X | X | X | +-------------------------+---------+---------+---------+---------+ -| os-odl_l2-nofeature-ha | X | X | X | X | +| os-odl_l2-nofeature-ha | | X | X | X | +-------------------------+---------+---------+---------+---------+ -| os-odl_l2-nofeature-noha| | X | | | +| os-odl_l2-nofeature-noha| | | X | | +-------------------------+---------+---------+---------+---------+ -| os-odl_l3-nofeature-ha | X | | X | | +| os-odl_l3-nofeature-ha | X | X | X | | +-------------------------+---------+---------+---------+---------+ -| os-odl_l3-nofeature-ha | | X | | | +| os-odl_l3-nofeature-noha| | | X | | +-------------------------+---------+---------+---------+---------+ -| os-onos-sfc-ha | X | | X | X | +| os-onos-sfc-ha | | | | | +-------------------------+---------+---------+---------+---------+ -| os-onos-nofeature-ha | X | | X | X | +| os-onos-nofeature-ha | | X | | X | +-------------------------+---------+---------+---------+---------+ -| os-onos-nofeature-noha | | X | | | +| os-onos-nofeature-noha | | | | | +-------------------------+---------+---------+---------+---------+ | os-odl_l2-sfc-ha | | | X | | +-------------------------+---------+---------+---------+---------+ -| os-odl_l2-sfc-noha | X | X | | | +| os-odl_l2-sfc-noha | | | X | | +-------------------------+---------+---------+---------+---------+ | os-odl_l2-bgpvpn-ha | X | | X | | +-------------------------+---------+---------+---------+---------+ -| os-odl_l2-bgpvpn-noha | | X | | | +| os-odl_l2-bgpvpn-noha | | | X | | +-------------------------+---------+---------+---------+---------+ -| os-nosdn-kvm-ha | | | X | | +| os-nosdn-kvm-ha | X | | X | | +-------------------------+---------+---------+---------+---------+ -| os-nosdn-kvm-noha | | X | | | +| os-nosdn-kvm-noha | | | X | | +-------------------------+---------+---------+---------+---------+ | os-nosdn-ovs-ha | | | X | | +-------------------------+---------+---------+---------+---------+ -| os-nosdn-ovs-noha | X | | X | | +| os-nosdn-ovs-noha | | | X | | +-------------------------+---------+---------+---------+---------+ -| os-ocl-nofeature-ha | | | | | +| os-ocl-nofeature-ha | | X | | | +-------------------------+---------+---------+---------+---------+ | os-nosdn-lxd-ha | | | | X | +-------------------------+---------+---------+---------+---------+ | os-nosdn-lxd-noha | | | | X | +-------------------------+---------+---------+---------+---------+ +| os-nosdn-fdio-ha | X | | | | ++-------------------------+---------+---------+---------+---------+ | os-odl_l2-fdio-noha | X | | | | +-------------------------+---------+---------+---------+---------+ +| os-odl-gluon-noha | X | | | | ++-------------------------+---------+---------+---------+---------+ +| os-nosdn-openo-ha | | X | | | ++-------------------------+---------+---------+---------+---------+ +| os-nosdn-kvm_ovs_dpdk | | | X | | +| -noha | | | | | ++-------------------------+---------+---------+---------+---------+ +| os-nosdn-kvm_ovs_dpdk-ha| | | X | | ++-------------------------+---------+---------+---------+---------+ +| os-nosdn-kvm_ovs_dpdk | | | X | | +| _bar-ha | | | | | ++-------------------------+---------+---------+---------+---------+ +| os-nosdn-kvm_ovs_dpdk | | | X | | +| _bar-noha | | | | | ++-------------------------+---------+---------+---------+---------+ Test results @@ -563,10 +596,10 @@ Test results are available in: The reporting pages can be found at: - * apex: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-apex.html - * compass: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-compass.html - * fuel: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-fuel.html - * joid: http://testresults.opnfv.org/reporting/yardstick/release/colorado/index-status-joid.html + * apex: http://testresults.opnfv.org/reporting/yardstick/release/danube/index-status-apex.html + * compass: http://testresults.opnfv.org/reporting/yardstick/release/danube/index-status-compass.html + * fuel: http://testresults.opnfv.org/reporting/yardstick/release/danube/index-status-fuel.html + * joid: http://testresults.opnfv.org/reporting/yardstick/release/danube/index-status-joid.html You can get additional details through test logs on http://artifacts.opnfv.org/. As no search engine is available on the OPNFV artifact web site you must @@ -578,7 +611,7 @@ for the date of the test you are interested in. Known Issues/Faults ------------ - Floating IP not supported in bgpvpn scenario - - Floating IP not supported in apex-os-odl_l3-nofeature-ha scenario + - VM instance cannot get floating IP in compass-os-odl_l2-nofeature-ha scenario .. note:: The faults not related to *Yardstick* framework, addressing scenarios which were not fully verified, are listed in the OPNFV installer's release @@ -588,59 +621,41 @@ Known Issues/Faults Corrected Faults ---------------- -Colorado.3.0: -+----------------------------+------------------------------------------------+ -| **JIRA REFERENCE** | **SLOGAN** | -| | | -+----------------------------+------------------------------------------------+ -| JIRA: YARDSTICK-239 | Define process for working with Yardstick | -| | Grafana dashboard. | -| | | -+----------------------------+------------------------------------------------+ -| JIRA: YARDSTICK-373 | Add os-odl_l2-fdio-ha scenario support. | -| | | -+----------------------------+------------------------------------------------+ - - -Colorado.2.0: +Danube.1.0: +----------------------------+------------------------------------------------+ | **JIRA REFERENCE** | **SLOGAN** | | | | +----------------------------+------------------------------------------------+ -| JIRA: YARDSTICK-325 | Provide raw format yardstick vm image for | -| | nova-lxd scenario. | +| JIRA: YARDSTICK-593 | fix yardstick-image built without test tools | | | | +----------------------------+------------------------------------------------+ -| JIRA: YARDSTICK-358 | tc027 ipv6 test case to de-coupling to the | -| | installers. | -| | | +| JIRA: YARDSTICK-599 | Could not load EntryPoint.parse when using | +| | 'openstack -h' | +----------------------------+------------------------------------------------+ -| JIRA: YARDSTICK-359 | ipv6 testcase disable port-security on | -| | vRouter. | -| | | +| JIRA: YARDSTICK-600 | netperf_bottleneck execute failed because | +| | is_same_heat_context return True | +----------------------------+------------------------------------------------+ -| JIRA: YARDSTICK-363 | ipv6 testcase to support fuel. | -| | | +| JIRA: YARDSTICK-602 | Don't rely on staic ip addresses as they | +| | are dynamic | +----------------------------+------------------------------------------------+ -| JIRA: YARDSTICK-367 | Add d3 graph presentation to yardstick | -| | reporting. | +| JIRA: YARDSTICK-604 | add storperf flavor for TC074 | | | | +----------------------------+------------------------------------------------+ -| JIRA: YARDSTICK-371 | Provide raw format yardstick vm image for | -| | nova-lxd scenario. | +| JIRA: YARDSTICK-605 | pathspec 'master' did not match any file(s) | +| | known to git | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-607 | Error validating value 'cirros-0.3.5' | | | | +----------------------------+------------------------------------------------+ -| JIRA: YARDSTICK-372 | cannot find yardstick-img-dpdk-modify and | -| | yardstick-img-lxd-modify in environment | -| | varibales. | +| JIRA: YARDSTICK-609 | Remove improper license header in swagger docs | | | | +----------------------------+------------------------------------------------+ -Colorado 3.0 known restrictions/issues -================================== +Danube 1.0 known restrictions/issues +==================================== +-----------+-----------+----------------------------------------------+ | Installer | Scenario | Issue | +===========+===========+==============================================+ @@ -649,22 +664,37 @@ Colorado 3.0 known restrictions/issues +-----------+-----------+----------------------------------------------+ | any | odl_l3-* | Some test cases related to using floating IP | | | | addresses fail because of a known ODL bug. | -| | | https://jira.opnfv.org/browse/APEX-112 | +| | | | +-----------+-----------+----------------------------------------------+ +| compass | odl_l2-* | In some test cases, VM instance will failed | +| | | raising network interfaces. | +| | | | ++-----------+-----------+----------------------------------------------+ + Open JIRA tickets ================= ++----------------------------+------------------------------------------------+ +| **JIRA REFERENCE** | **SLOGAN** | +| | | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-603 | Timeout waiting for floating ip | +| | (which actually pingable) | ++----------------------------+------------------------------------------------+ +| JIRA: YARDSTICK-412 | IPv6 test case should add support for newton | +| | | ++----------------------------+------------------------------------------------+ Useful links ============ - wiki project page: https://wiki.opnfv.org/display/yardstick/Yardstick - - wiki Yardstick Colorado release planing page: https://wiki.opnfv.org/display/yardstick/Yardstick+Colorado+Release+Planning + - wiki Yardstick Danube release planing page: https://wiki.opnfv.org/display/yardstick/Yardstick+Danube+Release+Planning - - wiki Yardstick Colorado release jira page: https://wiki.opnfv.org/display/yardstick/Jira+Yardstick-Colorado + - wiki Yardstick Danube release jira page: https://wiki.opnfv.org/display/yardstick/Jira+Yardstick-Danube - Yardstick repo: https://git.opnfv.org/cgit/yardstick @@ -673,21 +703,3 @@ Useful links - Yardstick grafana dashboard: http://testresults.opnfv.org/grafana/ - Yardstick IRC chanel: #opnfv-yardstick - -.. _`YARDSTICK-239` : https://jira.opnfv.org/browse/YARDSTICK-239 - -.. _`YARDSTICK-325` : https://jira.opnfv.org/browse/YARDSTICK-325 - -.. _`YARDSTICK-358` : https://jira.opnfv.org/browse/YARDSTICK-358 - -.. _`YARDSTICK-359` : https://jira.opnfv.org/browse/YARDSTICK-359 - -.. _`YARDSTICK-363` : https://jira.opnfv.org/browse/YARDSTICK-363 - -.. _`YARDSTICK-367` : https://jira.opnfv.org/browse/YARDSTICK-367 - -.. _`YARDSTICK-371` : https://jira.opnfv.org/browse/YARDSTICK-371 - -.. _`YARDSTICK-372` : https://jira.opnfv.org/browse/YARDSTICK-372 - -.. _`YARDSTICK-373` : https://jira.opnfv.org/browse/YARDSTICK-373 |