aboutsummaryrefslogtreecommitdiffstats
path: root/docs
diff options
context:
space:
mode:
Diffstat (limited to 'docs')
-rw-r--r--docs/release/release-notes/release-notes.rst539
-rw-r--r--docs/release/results/euphrates_fraser_comparison.rst602
-rw-r--r--docs/release/results/images/tc002_pod.pngbin0 -> 39106 bytes
-rw-r--r--docs/release/results/images/tc002_pod_fraser.pngbin0 -> 22935 bytes
-rw-r--r--docs/release/results/images/tc002_scenario.pngbin0 -> 44920 bytes
-rw-r--r--docs/release/results/images/tc002_scenario_fraser.pngbin0 -> 25892 bytes
-rw-r--r--docs/release/results/images/tc010_pod.pngbin0 -> 44349 bytes
-rw-r--r--docs/release/results/images/tc010_pod_fraser.pngbin0 -> 37525 bytes
-rw-r--r--docs/release/results/images/tc010_scenario.pngbin0 -> 51251 bytes
-rw-r--r--docs/release/results/images/tc010_scenario_fraser.pngbin0 -> 45190 bytes
-rw-r--r--docs/release/results/images/tc011_pod.pngbin0 -> 43308 bytes
-rw-r--r--docs/release/results/images/tc011_pod_fraser.pngbin0 -> 25593 bytes
-rw-r--r--docs/release/results/images/tc011_scenario.pngbin0 -> 43647 bytes
-rw-r--r--docs/release/results/images/tc011_scenario_fraser.pngbin0 -> 28152 bytes
-rw-r--r--docs/release/results/images/tc012_pod.pngbin0 -> 47996 bytes
-rw-r--r--docs/release/results/images/tc012_pod_fraser.pngbin0 -> 36168 bytes
-rw-r--r--docs/release/results/images/tc012_scenario.pngbin0 -> 51405 bytes
-rw-r--r--docs/release/results/images/tc012_scenario_fraser.pngbin0 -> 41289 bytes
-rw-r--r--docs/release/results/images/tc014_pod.pngbin0 -> 36462 bytes
-rw-r--r--docs/release/results/images/tc014_pod_fraseer.pngbin0 -> 29513 bytes
-rw-r--r--docs/release/results/images/tc014_scenario.pngbin0 -> 42056 bytes
-rw-r--r--docs/release/results/images/tc014_scenario_fraser.pngbin0 -> 36018 bytes
-rw-r--r--docs/release/results/images/tc069_pod.pngbin0 -> 41823 bytes
-rw-r--r--docs/release/results/images/tc069_pod_fraser.pngbin0 -> 36041 bytes
-rw-r--r--docs/release/results/images/tc069_scenario.pngbin0 -> 46728 bytes
-rw-r--r--docs/release/results/images/tc069_scenario_fraser.pngbin0 -> 43834 bytes
-rw-r--r--docs/release/results/images/tc082_pod.pngbin0 -> 28096 bytes
-rw-r--r--docs/release/results/images/tc082_pod_fraser.pngbin0 -> 25078 bytes
-rw-r--r--docs/release/results/images/tc082_scenario.pngbin0 -> 16082 bytes
-rw-r--r--docs/release/results/images/tc083_pod.pngbin0 -> 29533 bytes
-rw-r--r--docs/release/results/images/tc083_pod_fraser.pngbin0 -> 25476 bytes
-rw-r--r--docs/release/results/images/tc083_scenario.pngbin0 -> 16481 bytes
-rw-r--r--docs/release/results/index.rst1
-rw-r--r--docs/release/results/os-nosdn-kvm-ha.rst270
-rw-r--r--docs/release/results/os-nosdn-nofeature-ha.rst492
-rw-r--r--docs/release/results/os-nosdn-nofeature-noha.rst259
-rw-r--r--docs/release/results/os-odl_l2-bgpvpn-ha.rst53
-rw-r--r--docs/release/results/os-odl_l2-nofeature-ha.rst743
-rw-r--r--docs/release/results/os-odl_l2-sfc-ha.rst231
-rw-r--r--docs/release/results/os-onos-nofeature-ha.rst257
-rw-r--r--docs/release/results/os-onos-sfc-ha.rst517
-rw-r--r--docs/release/results/overview.rst76
-rw-r--r--docs/release/results/results.rst34
-rw-r--r--docs/release/results/tc002-network-latency.rst525
-rw-r--r--docs/release/results/tc010-memory-read-latency.rst510
-rw-r--r--docs/release/results/tc011-packet-delay-variation.rst432
-rw-r--r--docs/release/results/tc012-memory-read-write-bandwidth.rst513
-rw-r--r--docs/release/results/tc014-cpu-processing-speed.rst512
-rw-r--r--docs/release/results/tc069-memory-write-bandwidth.rst516
-rw-r--r--docs/release/results/tc082-context-switches-under-load.rst187
-rw-r--r--docs/release/results/tc083-network-throughput-between-vm.rst187
-rwxr-xr-xdocs/testing/developer/devguide/devguide.rst3
-rw-r--r--docs/testing/user/userguide/13-nsb-installation.rst94
-rw-r--r--docs/testing/user/userguide/14-nsb-operation.rst36
-rw-r--r--docs/testing/user/userguide/15-list-of-tcs.rst2
-rw-r--r--docs/testing/user/userguide/code/pod_ixia.yaml31
-rw-r--r--docs/testing/user/userguide/opnfv_yardstick_tc050.rst52
-rw-r--r--docs/testing/user/userguide/opnfv_yardstick_tc088.rst129
-rw-r--r--docs/testing/user/userguide/opnfv_yardstick_tc089.rst129
-rw-r--r--docs/testing/user/userguide/opnfv_yardstick_tc092.rst196
-rw-r--r--docs/testing/user/userguide/opnfv_yardstick_tc093.rst184
61 files changed, 5020 insertions, 3292 deletions
diff --git a/docs/release/release-notes/release-notes.rst b/docs/release/release-notes/release-notes.rst
index 4ebf0eceb..7ea2616e4 100644
--- a/docs/release/release-notes/release-notes.rst
+++ b/docs/release/release-notes/release-notes.rst
@@ -1,7 +1,8 @@
+=======
License
=======
-OPNFV Euphrates release note for Yardstick Docs
+OPNFV Fraser release note for Yardstick Docs
are licensed under a Creative Commons Attribution 4.0 International License.
You should have received a copy of the license along with this.
If not, see <http://creativecommons.org/licenses/by/4.0/>.
@@ -9,8 +10,9 @@ If not, see <http://creativecommons.org/licenses/by/4.0/>.
The *Yardstick framework*, the *Yardstick test cases* are open-source software,
licensed under the terms of the Apache License, Version 2.0.
-OPNFV Euphrates Release Note for Yardstick
-==========================================
+=======================================
+OPNFV Fraser Release Note for Yardstick
+=======================================
.. toctree::
:maxdepth: 2
@@ -23,50 +25,46 @@ OPNFV Euphrates Release Note for Yardstick
Abstract
---------
+========
This document describes the release note of Yardstick project.
Version History
----------------
+===============
+-------------------+-----------+---------------------------------+
| *Date* | *Version* | *Comment* |
| | | |
+-------------------+-----------+---------------------------------+
-| December 15, 2017 | 5.1.0 | Yardstick for Euphrates release |
+| May 25, 2018 | 6.1.0 | Yardstick for Fraser release |
| | | |
+-------------------+-----------+---------------------------------+
-| October 20, 2017 | 5.0.0 | Yardstick for Euphrates release |
+| April 27, 2018 | 6.0.0 | Yardstick for Fraser release |
| | | |
+-------------------+-----------+---------------------------------+
Important Notes
----------------
+===============
The software delivered in the OPNFV Yardstick_ Project, comprising the
-*Yardstick framework*, the *Yardstick test cases* and the experimental
-framework *Apex Lake* is a realization of the methodology in ETSI-ISG
-NFV-TST001_.
+*Yardstick framework*, and the *Yardstick test cases* is a realization of
+the methodology in ETSI-ISG NFV-TST001_.
The *Yardstick* framework is *installer*, *infrastructure* and *application*
independent.
-OPNFV Euphrates Release
------------------------
+OPNFV Fraser Release
+====================
-This Euphrates release provides *Yardstick* as a framework for NFVI testing
+This Fraser release provides *Yardstick* as a framework for NFVI testing
and OPNFV feature testing, automated in the OPNFV CI pipeline, including:
* Documentation generated with Sphinx
* User Guide
-
* Developer Guide
-
* Release notes (this document)
-
* Results
* Automated Yardstick test suite (daily, weekly)
@@ -84,39 +82,29 @@ and OPNFV feature testing, automated in the OPNFV CI pipeline, including:
* Yardstick plug-in configuration yaml files, plug-in install/remove scripts
-For Euphrates release, the *Yardstick framework* is used for the following
+For Fraser release, the *Yardstick framework* is used for the following
testing:
* OPNFV platform testing - generic test cases to measure the categories:
* Compute
-
* Network
-
* Storage
-* OPNFV platform network service benchmarking(NSB)
+* OPNFV platform network service benchmarking (NSB)
* NSB
* Test cases for the following OPNFV Projects:
* Container4NFV
-
* High Availability
-
* IPv6
-
* KVM
-
* Parser
-
* StorPerf
-
* VSperf
- * virtual Traffic Classifier
-
The *Yardstick framework* is developed in the OPNFV community, by the
Yardstick_ team.
@@ -126,49 +114,47 @@ Yardstick_ team.
Release Data
-------------
+============
+--------------------------------+-----------------------+
| **Project** | Yardstick |
| | |
+--------------------------------+-----------------------+
-| **Repo/tag** | yardstick/opnfv-5.1.0 |
+| **Repo/tag** | yardstick/opnfv-6.1.0 |
| | |
+--------------------------------+-----------------------+
-| **Yardstick Docker image tag** | opnfv-5.1.0 |
+| **Yardstick Docker image tag** | opnfv-6.1.0 |
| | |
+--------------------------------+-----------------------+
-| **Release designation** | Euphrates |
+| **Release designation** | Fraser |
| | |
+--------------------------------+-----------------------+
-| **Release date** | December 15, 2017 |
+| **Release date** | May 25, 2018 |
| | |
+--------------------------------+-----------------------+
-| **Purpose of the delivery** | OPNFV Euphrates 5.1.0 |
+| **Purpose of the delivery** | OPNFV Fraser 6.1.0 |
| | |
+--------------------------------+-----------------------+
Deliverables
-------------
+============
Documents
-^^^^^^^^^
+---------
- - User Guide: http://docs.opnfv.org/en/stable-euphrates/submodules/yardstick/docs/testing/user/userguide/index.html
+ - User Guide: http://docs.opnfv.org/en/stable-fraser/submodules/yardstick/docs/testing/user/userguide/index.html
- - Developer Guide: http://docs.opnfv.org/en/stable-euphrates/submodules/yardstick/docs/testing/developer/devguide/index.html
+ - Developer Guide: http://docs.opnfv.org/en/stable-fraser/submodules/yardstick/docs/testing/developer/devguide/index.html
Software Deliverables
-^^^^^^^^^^^^^^^^^^^^^
-
+---------------------
- - The Yardstick Docker image: https://hub.docker.com/r/opnfv/yardstick (tag: opnfv-5.1.0)
+ - The Yardstick Docker image: https://hub.docker.com/r/opnfv/yardstick (tag: opnfv-6.1.0)
-
-New Contexts
-############
+List of Contexts
+^^^^^^^^^^^^^^^^
+--------------+-------------------------------------------+
| **Context** | **Description** |
@@ -188,31 +174,40 @@ New Contexts
+--------------+-------------------------------------------+
-New Runners
-###########
-
-+--------------+-------------------------------------------------------+
-| **Runner** | **Description** |
-| | |
-+--------------+-------------------------------------------------------+
-| *Arithmetic* | Steps every run arithmetically according to specified |
-| | input value |
-| | |
-+--------------+-------------------------------------------------------+
-| *Duration* | Runs for a specified period of time |
-| | |
-+--------------+-------------------------------------------------------+
-| *Iteration* | Runs for a specified number of iterations |
-| | |
-+--------------+-------------------------------------------------------+
-| *Sequence* | Selects input value to a scenario from an input file |
-| | and runs all entries sequentially |
-| | |
-+--------------+-------------------------------------------------------+
-
-
-New Scenarios
-#############
+List of Runners
+^^^^^^^^^^^^^^^
+
+.. note:: Yardstick Fraser 6.0.0 add two new Runners, "Dynamictp" and "Search".
+
++---------------+-------------------------------------------------------+
+| **Runner** | **Description** |
+| | |
++---------------+-------------------------------------------------------+
+| *Arithmetic* | Steps every run arithmetically according to specified |
+| | input value |
+| | |
++---------------+-------------------------------------------------------+
+| *Duration* | Runs for a specified period of time |
+| | |
++---------------+-------------------------------------------------------+
+| *Iteration* | Runs for a specified number of iterations |
+| | |
++---------------+-------------------------------------------------------+
+| *Sequence* | Selects input value to a scenario from an input file |
+| | and runs all entries sequentially |
+| | |
++---------------+-------------------------------------------------------+
+| **Dynamictp** | A runner that searches for the max throughput with |
+| | binary search |
+| | |
++---------------+-------------------------------------------------------+
+| **Search** | A runner that runs a specific time before it returns |
+| | |
++---------------+-------------------------------------------------------+
+
+
+List of Scenarios
+^^^^^^^^^^^^^^^^^
+----------------+-----------------------------------------------------+
| **Category** | **Delivered** |
@@ -234,224 +229,142 @@ New Scenarios
| | |
+----------------+-----------------------------------------------------+
| *Compute* | * cpuload |
-| | |
| | * cyclictest |
-| | |
| | * lmbench |
-| | |
| | * lmbench_cache |
-| | |
| | * perf |
-| | |
| | * unixbench |
-| | |
| | * ramspeed |
-| | |
| | * cachestat |
-| | |
| | * memeoryload |
-| | |
| | * computecapacity |
-| | |
| | * SpecCPU2006 |
| | |
+----------------+-----------------------------------------------------+
| *Networking* | * iperf3 |
-| | |
| | * netperf |
-| | |
| | * netperf_node |
-| | |
| | * ping |
-| | |
| | * ping6 |
-| | |
| | * pktgen |
-| | |
| | * sfc |
-| | |
| | * sfc with tacker |
-| | |
-| | * vtc instantion validation |
-| | |
-| | * vtc instantion validation with noisy neighbors |
-| | |
-| | * vtc throughput |
-| | |
-| | * vtc throughput in the presence of noisy neighbors |
-| | |
| | * networkcapacity |
-| | |
| | * netutilization |
-| | |
| | * nstat |
-| | |
| | * pktgenDPDK |
| | |
+----------------+-----------------------------------------------------+
| *Parser* | Tosca2Heat |
| | |
+----------------+-----------------------------------------------------+
-| *Storage* | fio |
-| | |
-| | bonnie++ |
-| | |
-| | storagecapacity |
+| *Storage* | * fio |
+| | * bonnie++ |
+| | * storagecapacity |
| | |
+----------------+-----------------------------------------------------+
| *StorPerf* | storperf |
| | |
+----------------+-----------------------------------------------------+
-| *NSB* | vPE thoughput test case |
+| *NSB* | vFW thoughput test case |
| | |
+----------------+-----------------------------------------------------+
-
New Test cases
-^^^^^^^^^^^^^^
-
-* Generic NFVI test cases
+--------------
- * OPNFV_YARDSTICK_TCO78 - SPEC CPU 2006
+.. note:: Yardstick Fraser 6.1.0 added two new test cases, "TC092" and "TC093".
- * OPNFV_YARDSTICK_TCO79 - Bonnie++
+* Generic NFVI test cases
-* Kubernetes Test cases
+ * OPNFV_YARDSTICK_TCO84 - SPEC CPU 2006 for VM
- * OPNFV_YARDSTICK_TCO80 - NETWORK LATENCY BETWEEN CONTAINER
+* HA Test cases
- * OPNFV_YARDSTICK_TCO81 - NETWORK LATENCY BETWEEN CONTAINER AND VM
+ * OPNFV_YARDSTICK_TC087 - SDN Controller resilience in non-HA configuration
+ * OPNFV_YARDSTICK_TC090 - Control node Openstack service down - database instance
+ * OPNFV_YARDSTICK_TC091 - Control node Openstack service down - heat-api
+ * OPNFV_YARDSTICK_TC092 - SDN Controller resilience in HA configuration
+ * OPNFV_YARDSTICK_TC093 - SDN Vswitch resilience in non-HA or HA configuration
Version Change
---------------
+==============
Module Version Changes
-^^^^^^^^^^^^^^^^^^^^^^
+----------------------
-This is the fifth tracked release of Yardstick. It is based on following
+This is the sixth tracked release of Yardstick. It is based on following
upstream versions:
-- OpenStack Ocata
-
-- OpenDayLight Nitrogen
-
-- ONOS Junco
+- OpenStack Pike
+- OpenDayLight Oxygen
Document Version Changes
-^^^^^^^^^^^^^^^^^^^^^^^^
+------------------------
-This is the fifth tracked version of the Yardstick framework in OPNFV.
+This is the sixth tracked version of the Yardstick framework in OPNFV.
It includes the following documentation updates:
- Yardstick User Guide: add "network service benchmarking(NSB)" chapter;
add "Yardstick - NSB Testing -Installation" chapter; add "Yardstick API" chapter;
add "Yardstick user interface" chapter; Update Yardstick installation chapter;
-
- Yardstick Developer Guide
-
- Yardstick Release Notes for Yardstick: this document
Feature additions
-^^^^^^^^^^^^^^^^^
-
-- Yardstick RESTful API support
-
-- Network service benchmarking
-
-- Stress testing with Bottlenecks team
-
-- Yardstick framework improvement:
-
- - yardstick report CLI
-
- - Node context support OpenStack configuration via Ansible
-
- - Https support
+-----------------
- - Kubernetes context type
-
-- Yardstick container local GUI
-
-- Python 3 support
+- Plugin-based test cases support Heat context
+- SR-IOV support for the Heat context
+- Support using existing network in Heat context
+- Support running test cases with existing VNFs/without destroying VNF in Heat context
+- Add vFW scale-up template
+- Improvements of unit tests and gating
+- GUI improvement about passing parameters
Scenario Matrix
----------------
-
-For Euphrates 5.0.0, Yardstick was tested on the following scenarios:
-
-+--------------------------+------+---------+------+------+
-| Scenario | Apex | Compass | Fuel | Joid |
-+==========================+======+=========+======+======+
-| os-nosdn-nofeature-noha | | | X | X |
-+--------------------------+------+---------+------+------+
-| os-nosdn-nofeature-ha | X | X | X | X |
-+--------------------------+------+---------+------+------+
-| os-odl_l2-nofeature-ha | | X | X | X |
-+--------------------------+------+---------+------+------+
-| os-odl_l2-nofeature-noha | | | X | |
-+--------------------------+------+---------+------+------+
-| os-odl_l3-nofeature-ha | X | X | X | |
-+--------------------------+------+---------+------+------+
-| os-odl_l3-nofeature-noha | | | X | |
-+--------------------------+------+---------+------+------+
-| os-onos-sfc-ha | | | | |
-+--------------------------+------+---------+------+------+
-| os-onos-nofeature-ha | | X | | X |
-+--------------------------+------+---------+------+------+
-| os-onos-nofeature-noha | | | | |
-+--------------------------+------+---------+------+------+
-| os-odl_l2-sfc-ha | | | X | |
-+--------------------------+------+---------+------+------+
-| os-odl_l2-sfc-noha | | | X | |
-+--------------------------+------+---------+------+------+
-| os-odl_l2-bgpvpn-ha | X | | X | |
-+--------------------------+------+---------+------+------+
-| os-odl_l2-bgpvpn-noha | | | X | |
-+--------------------------+------+---------+------+------+
-| os-nosdn-kvm-ha | X | | X | |
-+--------------------------+------+---------+------+------+
-| os-nosdn-kvm-noha | | | X | |
-+--------------------------+------+---------+------+------+
-| os-nosdn-ovs-ha | | | X | |
-+--------------------------+------+---------+------+------+
-| os-nosdn-ovs-noha | | | X | |
-+--------------------------+------+---------+------+------+
-| os-ocl-nofeature-ha | | X | | |
-+--------------------------+------+---------+------+------+
-| os-nosdn-lxd-ha | | | | X |
-+--------------------------+------+---------+------+------+
-| os-nosdn-lxd-noha | | | | X |
-+--------------------------+------+---------+------+------+
-| os-nosdn-fdio-ha | X | | | |
-+--------------------------+------+---------+------+------+
-| os-odl_l2-fdio-noha | X | | | |
-+--------------------------+------+---------+------+------+
-| os-odl-gluon-noha | X | | | |
-+--------------------------+------+---------+------+------+
-| os-nosdn-openo-ha | | X | | |
-+--------------------------+------+---------+------+------+
-| os-nosdn-kvm_ovs_dpdk | | | X | |
-| -noha | | | | |
-+--------------------------+------+---------+------+------+
-| os-nosdn-kvm_ovs_dpdk-ha | | | X | |
-+--------------------------+------+---------+------+------+
-| os-nosdn-kvm_ovs_dpdk | | | X | |
-| _bar-ha | | | | |
-+--------------------------+------+---------+------+------+
-| os-nosdn-kvm_ovs_dpdk | | | X | |
-| _bar-noha | | | | |
-+--------------------------+------+---------+------+------+
-| opnfv_os-ovn-nofeature- | X | | | |
-| noha_daily | | | | |
-+--------------------------+------+---------+------+------+
+===============
+
+For Fraser 6.0.0, Yardstick was tested on the following scenarios:
+
++-------------------------+------+---------+----------+------+------+-------+
+| Scenario | Apex | Compass | Fuel-arm | Fuel | Joid | Daisy |
++=========================+======+=========+==========+======+======+=======+
+| os-nosdn-nofeature-noha | X | X | | | X | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-nosdn-nofeature-ha | X | X | X | X | X | X |
++-------------------------+------+---------+----------+------+------+-------+
+| os-nosdn-bar-noha | X | X | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-nosdn-bar-ha | X | | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-odl-bgpvpn-ha | X | | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-nosdn-calipso-noha | X | | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-nosdn-kvm-ha | | X | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-odl_l3-nofeature-ha | | X | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-odl-sfc-ha | | X | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-odl-nofeature-ha | | | | X | | X |
++-------------------------+------+---------+----------+------+------+-------+
+| os-nosdn-ovs-ha | | | | X | | |
++-------------------------+------+---------+----------+------+------+-------+
+| k8-nosdn-nofeature-ha | | X | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| k8-nosdn-stor4nfv-noha | | X | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+
Test results
-------------
+============
Test results are available in:
@@ -459,109 +372,129 @@ Test results are available in:
The reporting pages can be found at:
-+---------------+-------------------------------------------------------------------------------------+
-| apex | http://testresults.opnfv.org/reporting/euphrates/yardstick/status-apex.html |
-+---------------+-------------------------------------------------------------------------------------+
-| compass | http://testresults.opnfv.org/reporting/euphrates/yardstick/status-compass.html |
-+---------------+-------------------------------------------------------------------------------------+
-| fuel\@x86 | http://testresults.opnfv.org/reporting/euphrates/yardstick/status-fuel@x86.html |
-+---------------+-------------------------------------------------------------------------------------+
-| fuel\@aarch64 | http://testresults.opnfv.org/reporting/euphrates/yardstick/status-fuel@aarch64.html |
-+---------------+-------------------------------------------------------------------------------------+
-| joid | http://testresults.opnfv.org/reporting/euphrates/yardstick/status-joid.html |
-+---------------+-------------------------------------------------------------------------------------+
++---------------+----------------------------------------------------------------------------------+
+| apex | http://testresults.opnfv.org/reporting/fraser/yardstick/status-apex.html |
++---------------+----------------------------------------------------------------------------------+
+| compass | http://testresults.opnfv.org/reporting/fraser/yardstick/status-compass.html |
++---------------+----------------------------------------------------------------------------------+
+| fuel\@x86 | http://testresults.opnfv.org/reporting/fraser/yardstick/status-fuel@x86.html |
++---------------+----------------------------------------------------------------------------------+
+| fuel\@aarch64 | http://testresults.opnfv.org/reporting/fraser/yardstick/status-fuel@aarch64.html |
++---------------+----------------------------------------------------------------------------------+
+| joid | http://testresults.opnfv.org/reporting/fraser/yardstick/status-joid.html |
++---------------+----------------------------------------------------------------------------------+
Known Issues/Faults
-^^^^^^^^^^^^^^^^^^^
+-------------------
Corrected Faults
-^^^^^^^^^^^^^^^^
+----------------
+
+Fraser 6.1.0:
+
++--------------------+--------------------------------------------------------------------------+
+| **JIRA REFERENCE** | **DESCRIPTION** |
++====================+==========================================================================+
+| YARDSTICK-995 | Test case spec for SDN Virtual Switch resilience |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1097 | Add pod.yaml file for APEX installer |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1122 | Remove unused code in SampleVNF |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1125 | Update samples/test_suite.yaml |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1132 | Document for Euphrates test case results |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1138 | Support Restart Operation |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1142 | start_service script fails to start openvswitch service in centos distro |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1165 | Bugfix: openrc api dump should be safe_dump |
++--------------------+--------------------------------------------------------------------------+
+
+Fraser 6.0.0:
+
++--------------------+--------------------------------------------------------------------------+
+| **JIRA REFERENCE** | **DESCRIPTION** |
++====================+==========================================================================+
+| YARDSTICK-831 | tc053 kill haproxy wrong |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-842 | load image fails when there's cirros image exist |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-857 | tc006 failed due to volume attached to different location "/dev/vdc" |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-874 | Specify supported architecture for Ubuntu backports repository |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-875 | Check if multiverse repository is available in Ubuntu |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-893 | Fix proxy env handling and ansible multinode support |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-899 | Variable local_iface_name is read before it is set |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-900 | Section in "upload_yardstick_image.yml" invalid |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-911 | Remove 'inconsistent-return-statements' from Pylint checks |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-989 | Yardstick real-time influxdb KPI reporting regressions |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-994 | NSB set-up build script for baremetal broken |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-996 | Error in address input format in "_ip_range_action_partial" |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1003 | Prox vnf descriptor cleanup for tg and vnf |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1006 | Ansible destroy script will fail if vm has already been undefined |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1012 | constants: fix pylint warnings for OSError |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1014 | Remove unused args in |
+| | network_services.traffic_profile.ixia_rfc2544.IXIARFC2544Profile |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1016 | Allow vm to access outside world through default gateway |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1019 | For 'qemu-img version 2.10.1' unit 'MB' is not acceptable ansible script |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1021 | NSB: All Sample VNF test cases timeout after 1 hour of execution |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1036 | Prox: Addition of storage of extra counters for Grafana |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1038 | Missing file which is described in the operation_conf.yaml |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1047 | Error in string format in HeatTemplateError message |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1056 | yardstick report command print error when run test case |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1059 | Reduce the log level if TRex client is no connected |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1073 | Error when retrieving "options" section in "scenario" |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1080 | Running Test Case in Latest Yardstick Docker Image shows Error |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1082 | tc043,tc055, tc063, tc075, pass wrong node name in the ci scenario yaml |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1102 | Don't hide exception traceback from Task.start() |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1107 | bad exception traceback print due to atexit_handler |
++--------------------+--------------------------------------------------------------------------+
+| YARDSTICK-1120 | HA test case tc050 should start monitor before attack |
++--------------------+--------------------------------------------------------------------------+
+
+Fraser 6.0.0 known restrictions/issues
+======================================
-Euphrates 5.1.0:
-
-+---------------------+-------------------------------------------------------------------------+
-| **JIRA REFERENCE** | **DESCRIPTION** |
-| | |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-841 | Fix various NSB license issues |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-73 | How To Work with Test Cases |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-500 | VNF testing documentation |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-826 | Allow overriding Heat IP addresses to match traffic generator profile |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-828 | Refactor doc/testing/user/userguide "Yardstick Installation" |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-830 | build_yardstick_image Ansible mount module doesn't work on Ubuntu 14.04 |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-833 | ansible_common transform password into lower case |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-847 | tc006, tc079, tc082 miss grafana dashboard in local deployment |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-849 | kill process do not accurately kill the process like "nova-api" |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-850 | tc023 miss description and tc050-58 wrong description |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-852 | tc078 cpu2006 fails in some situation |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-854 | yardstick docker lack of trex_client |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-867 | testcase tc078 have no data stored or dashboard to show results |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-871 | Remove img_modify_playbook assignation in build_yardstick_image.yml |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-829 | "nsb_setup.sh" doesn't parse the controller IP correctly |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-839 | NSB Prox BM test cases to be fixed for incorporating scale-up |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-840 | NSB Prox test documentation of vPE and LW-AFTR test cases |
-+---------------------+-------------------------------------------------------------------------+
-| JIRA: YARDSTICK-848 | NSB "Prox" : Cleanup duplicated traffic profile |
-+---------------------+-------------------------------------------------------------------------+
-
-
-
-
-Euphrates 5.0.0:
-
-+---------------------+--------------------------------------------+
-| **JIRA REFERENCE** | **DESCRIPTION** |
-| | |
-+---------------------+--------------------------------------------+
-| JIRA: YARDSTICK-599 | Could not load EntryPoint.parse when using |
-| | 'openstack -h' |
-+---------------------+--------------------------------------------+
-| JIRA: YARDSTICK-602 | Don't rely on staic ip addresses as they |
-| | are dynamic |
-+---------------------+--------------------------------------------+
-
-
-Euphratess 5.0.0 known restrictions/issues
-------------------------------------------
+-----------+-----------+----------------------------------------------+
| Installer | Scenario | Issue |
+===========+===========+==============================================+
-| any | \*-bgpvpn | Floating ips not supported. Some Test cases |
-| | | related to floating ips are excluded. |
-+-----------+-----------+----------------------------------------------+
-| any | odl_l3-\* | Some test cases related to using floating IP |
-| | | addresses fail because of a known ODL bug. |
-| | | |
-+-----------+-----------+----------------------------------------------+
-| compass | odl_l2-\* | In some test cases, VM instance will failed |
-| | | raising network interfaces. |
| | | |
+-----------+-----------+----------------------------------------------+
-
Useful links
-------------
+============
- wiki project page: https://wiki.opnfv.org/display/yardstick/Yardstick
- - wiki Yardstick Euphrates release planing page: https://wiki.opnfv.org/display/yardstick/Yardstick+Euphrates+Release+Planning
+ - wiki Yardstick Fraser release planing page: https://wiki.opnfv.org/display/yardstick/Release+Fraser
- Yardstick repo: https://git.opnfv.org/cgit/yardstick
diff --git a/docs/release/results/euphrates_fraser_comparison.rst b/docs/release/results/euphrates_fraser_comparison.rst
new file mode 100644
index 000000000..53dfb994f
--- /dev/null
+++ b/docs/release/results/euphrates_fraser_comparison.rst
@@ -0,0 +1,602 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+
+=======================================================
+Test results analysis for Euphrates and Fraser releases
+=======================================================
+
+TC002
+-----
+
+The round-trip-time (RTT) between 2 VMs on different blades is measured using
+ping.
+
+Most test run measurements result on average between 0.39 and 4.00 ms.
+Compared with Euphrates release, the average RTT result of the same pod experiences
+a slight decline in Fraser release. For example, the average RTT of arm-pod5 is
+1.518 in Ehphrates and 1.714 in Fraser. The average RTT of intel-pod18 is 1.6575
+ms in Ehphrates and 1.856 ms in Fraser.
+
+{
+
+ "huawei-pod2:stable/euphrates": [0.3925],
+
+ "lf-pod2:stable/euphrates": [0.5315],
+
+ "lf-pod1:stable/euphrates": [0.62],
+
+ "huawei-pod2:stable/fraser": [0.677],
+
+ "lf-pod1:stable/fraser": [0.725],
+
+ "flex-pod2:stable/euphrates": [0.795],
+
+ "huawei-pod12:stable/euphrates": [0.87],
+
+ "ericsson-pod1:stable/fraser": [0.9165],
+
+ "huawei-pod12:stable/fraser": [1.0465],
+
+ "lf-pod2:stable/fraser": [1.2325],
+
+ "intel-pod5:stable/euphrates": [1.25],
+
+ "ericsson-virtual3:stable/euphrates": [1.2655],
+
+ "ericsson-pod1:stable/euphrates": [1.372],
+
+ "zte-pod2:stable/fraser": [1.395],
+
+ "arm-pod5:stable/euphrates": [1.518],
+
+ "huawei-virtual4:stable/euphrates": [1.5355],
+
+ "ericsson-virtual4:stable/fraser": [1.582],
+
+ "huawei-virtual3:stable/euphrates": [1.606],
+
+ "intel-pod18:stable/euphrates": [1.6575],
+
+ "huawei-virtual4:stable/fraser": [1.697],
+
+ "huawei-virtual8:stable/euphrates": [1.709],
+
+ "arm-pod5:stable/fraser": [1.714],
+
+ "huawei-virtual3:stable/fraser": [1.716],
+
+ "intel-pod18:stable/fraser": [1.856],
+
+ "huawei-virtual2:stable/euphrates": [1.872],
+
+ "arm-pod6:stable/euphrates": [1.895],
+
+ "huawei-virtual2:stable/fraser": [1.964],
+
+ "huawei-virtual1:stable/fraser": [1.9765],
+
+ "huawei-virtual9:stable/euphrates": [2.0745],
+
+ "arm-pod6:stable/fraser": [2.209],
+
+ "huawei-virtual1:stable/euphrates": [2.495],
+
+ "ericsson-virtual2:stable/euphrates": [2.7895],
+
+ "ericsson-virtual4:stable/euphrates": [3.768],
+
+ "ericsson-virtual1:stable/euphrates": [3.8035],
+
+ "ericsson-virtual3:stable/fraser": [3.9175],
+
+ "ericsson-virtual2:stable/fraser": [4.004]
+
+}
+
+TC010
+-----
+
+The tool we use to measure memory read latency is lmbench, which is a series of
+micro benchmarks intended to measure basic operating system and hardware system
+metrics. Compared with Euphrates release, the memory read latency of the same pod
+also experience a slight decline. Virtual pods seem to have a higher memory read
+latency than physical pods. Compared with X86 pods, the memory read latency of
+arm pods is significant higher.
+
+{
+
+ "ericsson-pod1:stable/euphrates": [5.7785],
+
+ "flex-pod2:stable/euphrates": [5.908],
+
+ "ericsson-virtual1:stable/euphrates": [6.412],
+
+ "intel-pod18:stable/euphrates": [6.5905],
+
+ "intel-pod5:stable/euphrates": [6.6975],
+
+ "ericsson-pod1:stable/fraser": [7.0645],
+
+ "ericsson-virtual4:stable/euphrates": [7.183],
+
+ "intel-pod18:stable/fraser": [7.4465],
+
+ "zte-pod2:stable/fraser": [8.1865],
+
+ "ericsson-virtual2:stable/euphrates": [8.4985],
+
+ "huawei-pod2:stable/euphrates": [8.877],
+
+ "huawei-pod12:stable/euphrates": [9.091],
+
+ "huawei-pod2:stable/fraser": [9.236],
+
+ "huawei-pod12:stable/fraser": [9.615],
+
+ "ericsson-virtual3:stable/euphrates": [9.719],
+
+ "ericsson-virtual2:stable/fraser": [9.8925],
+
+ "huawei-virtual4:stable/euphrates": [10.1195],
+
+ "huawei-virtual3:stable/euphrates": [10.19],
+
+ "huawei-virtual2:stable/fraser": [10.22],
+
+ "huawei-virtual1:stable/euphrates": [10.3045],
+
+ "huawei-virtual9:stable/euphrates": [10.318],
+
+ "ericsson-virtual4:stable/fraser": [10.5465],
+
+ "ericsson-virtual3:stable/fraser": [10.9355],
+
+ "huawei-virtual3:stable/fraser": [10.95],
+
+ "huawei-virtual2:stable/euphrates": [11.274],
+
+ "huawei-virtual4:stable/fraser": [11.557],
+
+ "lf-pod1:stable/euphrates": [15.7025],
+
+ "lf-pod2:stable/euphrates": [15.8495],
+
+ "lf-pod2:stable/fraser": [16.5595],
+
+ "lf-pod1:stable/fraser": [16.8395],
+
+ "arm-pod5:stable/euphrates": [18.092],
+
+ "arm-pod5:stable/fraser": [18.744],
+
+ "huawei-virtual1:stable/fraser": [19.8235],
+
+ "huawei-virtual8:stable/euphrates": [33.999],
+
+ "arm-pod6:stable/euphrates": [41.5605],
+
+ "arm-pod6:stable/fraser": [55.804]
+
+}
+
+TC011
+-----
+
+Iperf3 is a tool for evaluating the packet delay variation between 2 VMs on
+different blades. In general, the packet delay variation of the two releases
+look similar.
+
+{
+
+ "arm-pod6:stable/fraser": [1],
+
+ "ericsson-pod1:stable/fraser": [1],
+
+ "ericsson-virtual2:stable/fraser": [1],
+
+ "ericsson-virtual3:stable/fraser": [1],
+
+ "lf-pod2:stable/fraser": [1],
+
+ "huawei-virtual1:stable/fraser": [2997],
+
+ "huawei-virtual2:stable/euphrates": [2997],
+
+ "flex-pod2:stable/euphrates": [2997.5],
+
+ "huawei-virtual3:stable/euphrates": [2998],
+
+ "huawei-virtual3:stable/fraser": [2999],
+
+ "huawei-virtual9:stable/euphrates": [3000],
+
+ "huawei-virtual8:stable/euphrates": [3001],
+
+ "huawei-virtual4:stable/euphrates": [3002],
+
+ "huawei-virtual4:stable/fraser": [3002],
+
+ "ericsson-virtual3:stable/euphrates": [3006],
+
+ "huawei-virtual1:stable/euphrates": [3007],
+
+ "ericsson-virtual2:stable/euphrates": [3009],
+
+ "intel-pod18:stable/euphrates": [3010],
+
+ "ericsson-virtual4:stable/euphrates": [3017],
+
+ "lf-pod2:stable/euphrates": [3021],
+
+ "arm-pod5:stable/euphrates": [3022],
+
+ "arm-pod6:stable/euphrates": [3022],
+
+ "ericsson-pod1:stable/euphrates": [3022],
+
+ "huawei-pod12:stable/euphrates": [3022],
+
+ "huawei-pod12:stable/fraser": [3022],
+
+ "huawei-pod2:stable/euphrates": [3022],
+
+ "huawei-pod2:stable/fraser": [3022],
+
+ "intel-pod18:stable/fraser": [3022],
+
+ "intel-pod5:stable/euphrates": [3022],
+
+ "lf-pod1:stable/euphrates": [3022],
+
+ "lf-pod1:stable/fraser": [3022],
+
+ "zte-pod2:stable/fraser": [3022],
+
+ "huawei-virtual2:stable/fraser": [3025]
+
+}
+
+TC012
+-----
+
+Lmbench is also used to measure the memory read and write bandwidth.
+Like TC010, compared with Euphrates release, the memory read and write bandwidth
+of the same pod also experience a slight decline. And compared with X86 pods, the memory
+read and write bandwidth of arm pods is significant lower.
+
+{
+
+ "lf-pod1:stable/euphrates": [22912.39],
+
+ "lf-pod2:stable/euphrates": [22637.67],
+
+ "lf-pod1:stable/fraser": [20552.9],
+
+ "flex-pod2:stable/euphrates": [20229.99],
+
+ "lf-pod2:stable/fraser": [20058.925],
+
+ "ericsson-pod1:stable/fraser": [18930.78],
+
+ "intel-pod18:stable/fraser": [18757.545],
+
+ "ericsson-virtual1:stable/euphrates": [17474.965],
+
+ "ericsson-pod1:stable/euphrates": [17127.38],
+
+ "ericsson-virtual4:stable/euphrates": [16219.97],
+
+ "ericsson-virtual2:stable/euphrates": [15652.28],
+
+ "ericsson-virtual3:stable/euphrates": [15551.26],
+
+ "ericsson-virtual4:stable/fraser": [15389.465],
+
+ "ericsson-virtual2:stable/fraser": [15343.79],
+
+ "huawei-pod2:stable/euphrates": [15017.2],
+
+ "huawei-pod2:stable/fraser": [14870.78],
+
+ "huawei-virtual4:stable/euphrates": [14266.34],
+
+ "huawei-virtual1:stable/euphrates": [14233.035],
+
+ "huawei-virtual3:stable/euphrates": [14227.63],
+
+ "zte-pod2:stable/fraser": [14157.99],
+
+ "huawei-pod12:stable/euphrates": [14147.245],
+
+ "huawei-pod12:stable/fraser": [14126.99],
+
+ "intel-pod18:stable/euphrates": [14058.33],
+
+ "huawei-virtual3:stable/fraser": [13929.67],
+
+ "huawei-virtual2:stable/euphrates": [13862.85],
+
+ "huawei-virtual4:stable/fraser": [13847.155],
+
+ "huawei-virtual2:stable/fraser": [13702.92],
+
+ "huawei-virtual1:stable/fraser": [13496.45],
+
+ "intel-pod5:stable/euphrates": [13280.32],
+
+ "ericsson-virtual3:stable/fraser": [12733.19],
+
+ "huawei-virtual9:stable/euphrates": [12559.445],
+
+ "huawei-virtual8:stable/euphrates": [8998.02],
+
+ "arm-pod5:stable/euphrates": [4388.875],
+
+ "arm-pod5:stable/fraser": [4326.11],
+
+ "arm-pod6:stable/euphrates": [4260.2],
+
+ "arm-pod6:stable/fraser": [3809.885]
+
+}
+
+TC014
+-----
+
+The Unixbench is used to evaluate the IaaS processing speed with regards to
+score of single CPU running and parallel running. Below are the single CPU running
+scores. It can be seen that the processing test results vary from scores 715 to 3737.
+In general, the single CPU score of the two releases look similar.
+
+{
+
+ "lf-pod2:stable/fraser": [3737.6],
+
+ "lf-pod2:stable/euphrates": [3723.95],
+
+ "lf-pod1:stable/fraser": [3702.7],
+
+ "lf-pod1:stable/euphrates": [3669],
+
+ "intel-pod5:stable/euphrates": [3388.6],
+
+ "intel-pod18:stable/euphrates": [3298.4],
+
+ "flex-pod2:stable/euphrates": [3208.6],
+
+ "ericsson-pod1:stable/fraser": [3131.6],
+
+ "intel-pod18:stable/fraser": [3098.1],
+
+ "ericsson-virtual1:stable/euphrates": [2988.9],
+
+ "zte-pod2:stable/fraser": [2831.4],
+
+ "ericsson-pod1:stable/euphrates": [2669.1],
+
+ "ericsson-virtual4:stable/euphrates": [2598.5],
+
+ "ericsson-virtual2:stable/fraser": [2559.7],
+
+ "ericsson-virtual3:stable/euphrates": [2553.15],
+
+ "huawei-pod2:stable/euphrates": [2531.2],
+
+ "huawei-pod2:stable/fraser": [2528.9],
+
+ "ericsson-virtual4:stable/fraser": [2527.8],
+
+ "ericsson-virtual2:stable/euphrates": [2526.9],
+
+ "huawei-virtual4:stable/euphrates": [2407.4],
+
+ "huawei-virtual3:stable/fraser": [2379.1],
+
+ "huawei-virtual3:stable/euphrates": [2374.6],
+
+ "huawei-virtual4:stable/fraser": [2362.1],
+
+ "huawei-virtual2:stable/euphrates": [2326.4],
+
+ "huawei-virtual9:stable/euphrates": [2324.95],
+
+ "huawei-virtual1:stable/euphrates": [2302.6],
+
+ "huawei-virtual2:stable/fraser": [2299.3],
+
+ "huawei-pod12:stable/euphrates": [2232.2],
+
+ "huawei-pod12:stable/fraser": [2229],
+
+ "huawei-virtual1:stable/fraser": [2171.3],
+
+ "ericsson-virtual3:stable/fraser": [2104.8],
+
+ "huawei-virtual8:stable/euphrates": [2085.3],
+
+ "arm-pod5:stable/fraser": [1764.2],
+
+ "arm-pod5:stable/euphrates": [1754.4],
+
+ "arm-pod6:stable/euphrates": [716.15],
+
+ "arm-pod6:stable/fraser": [715.4]
+
+}
+
+TC069
+-----
+
+With the block size changing from 1 kb to 512 kb, the memory write bandwidth
+tends to become larger first and then smaller within every run test. Below are
+the scores for 32mb block array.
+
+{
+
+ "intel-pod18:stable/euphrates": [18871.79],
+
+ "intel-pod18:stable/fraser": [16939.24],
+
+ "intel-pod5:stable/euphrates": [16055.79],
+
+ "arm-pod6:stable/euphrates": [13327.02],
+
+ "arm-pod6:stable/fraser": [11895.71],
+
+ "flex-pod2:stable/euphrates": [9384.585],
+
+ "zte-pod2:stable/fraser": [9375.33],
+
+ "ericsson-pod1:stable/euphrates": [9331.535],
+
+ "huawei-pod12:stable/euphrates": [9164.88],
+
+ "ericsson-pod1:stable/fraser": [9140.42],
+
+ "huawei-pod2:stable/euphrates": [9026.52],
+
+ "huawei-pod12:stable/fraser": [8993.37],
+
+ "huawei-virtual9:stable/euphrates": [8825.805],
+
+ "huawei-pod2:stable/fraser": [8794.01],
+
+ "huawei-virtual2:stable/fraser": [7670.21],
+
+ "ericsson-virtual1:stable/euphrates": [7615.97],
+
+ "ericsson-virtual4:stable/euphrates": [7539.23],
+
+ "arm-pod5:stable/fraser": [7479.32],
+
+ "arm-pod5:stable/euphrates": [7403.38],
+
+ "huawei-virtual3:stable/euphrates": [7247.89],
+
+ "ericsson-virtual2:stable/fraser": [7219.21],
+
+ "huawei-virtual2:stable/euphrates": [7205.35],
+
+ "huawei-virtual1:stable/euphrates": [7196.405],
+
+ "ericsson-virtual3:stable/euphrates": [7173.72],
+
+ "huawei-virtual4:stable/euphrates": [7131.47],
+
+ "ericsson-virtual2:stable/euphrates": [7129.08],
+
+ "huawei-virtual4:stable/fraser": [7059.045],
+
+ "huawei-virtual3:stable/fraser": [7023.57],
+
+ "lf-pod1:stable/euphrates": [6928.18],
+
+ "lf-pod2:stable/euphrates": [6875.88],
+
+ "lf-pod2:stable/fraser": [6834.7],
+
+ "lf-pod1:stable/fraser": [6775.27],
+
+ "ericsson-virtual4:stable/fraser": [6522.86],
+
+ "ericsson-virtual3:stable/fraser": [5835.59],
+
+ "huawei-virtual8:stable/euphrates": [5729.705],
+
+ "huawei-virtual1:stable/fraser": [5617.12]
+
+}
+
+TC082
+-----
+
+For this test case, we use perf to measure context-switches under load.
+High context switch rates are not themselves an issue, but they may point the
+way to a more significant problem.
+
+{
+
+ "zte-pod2:stable/fraser": [306.5],
+
+ "huawei-pod12:stable/euphrates": [316],
+
+ "lf-pod2:stable/fraser": [337.5],
+
+ "intel-pod18:stable/euphrates": [340],
+
+ "intel-pod18:stable/fraser": [343.5],
+
+ "intel-pod5:stable/euphrates": [357.5],
+
+ "ericsson-pod1:stable/euphrates": [384],
+
+ "lf-pod2:stable/euphrates": [394.5],
+
+ "huawei-pod12:stable/fraser": [399],
+
+ "lf-pod1:stable/euphrates": [435],
+
+ "lf-pod1:stable/fraser": [454],
+
+ "flex-pod2:stable/euphrates": [476],
+
+ "huawei-pod2:stable/euphrates": [518],
+
+ "huawei-pod2:stable/fraser": [544.5],
+
+ "arm-pod5:stable/euphrates": [869.5],
+
+ "huawei-virtual9:stable/euphrates": [1002],
+
+ "huawei-virtual4:stable/fraser": [1138],
+
+ "huawei-virtual4:stable/euphrates": [1174],
+
+ "huawei-virtual3:stable/euphrates": [1239],
+
+ "ericsson-pod1:stable/fraser": [1305],
+
+ "huawei-virtual2:stable/euphrates": [1430],
+
+ "huawei-virtual3:stable/fraser": [1433],
+
+ "huawei-virtual1:stable/fraser": [1470],
+
+ "huawei-virtual1:stable/euphrates": [1489],
+
+ "arm-pod6:stable/fraser": [1738.5],
+
+ "arm-pod6:stable/euphrates": [1883.5]
+
+}
+
+TC083
+-----
+
+TC083 measures network latency and throughput between VMs using netperf.
+The test results shown below are for UDP throughout.
+
+{
+
+ "lf-pod1:stable/euphrates": [2204.42],
+
+ "lf-pod2:stable/fraser": [1893.39],
+
+ "intel-pod18:stable/euphrates": [1835.55],
+
+ "lf-pod2:stable/euphrates": [1676.705],
+
+ "intel-pod5:stable/euphrates": [1612.555],
+
+ "zte-pod2:stable/fraser": [1543.995],
+
+ "lf-pod1:stable/fraser": [1480.86],
+
+ "intel-pod18:stable/fraser": [1417.015],
+
+ "flex-pod2:stable/euphrates": [1370.23],
+
+ "huawei-pod12:stable/euphrates": [1300.12]
+
+}
diff --git a/docs/release/results/images/tc002_pod.png b/docs/release/results/images/tc002_pod.png
new file mode 100644
index 000000000..7f92c471d
--- /dev/null
+++ b/docs/release/results/images/tc002_pod.png
Binary files differ
diff --git a/docs/release/results/images/tc002_pod_fraser.png b/docs/release/results/images/tc002_pod_fraser.png
new file mode 100644
index 000000000..797dc3136
--- /dev/null
+++ b/docs/release/results/images/tc002_pod_fraser.png
Binary files differ
diff --git a/docs/release/results/images/tc002_scenario.png b/docs/release/results/images/tc002_scenario.png
new file mode 100644
index 000000000..0ea1ecec6
--- /dev/null
+++ b/docs/release/results/images/tc002_scenario.png
Binary files differ
diff --git a/docs/release/results/images/tc002_scenario_fraser.png b/docs/release/results/images/tc002_scenario_fraser.png
new file mode 100644
index 000000000..ff42e6516
--- /dev/null
+++ b/docs/release/results/images/tc002_scenario_fraser.png
Binary files differ
diff --git a/docs/release/results/images/tc010_pod.png b/docs/release/results/images/tc010_pod.png
new file mode 100644
index 000000000..c7c623481
--- /dev/null
+++ b/docs/release/results/images/tc010_pod.png
Binary files differ
diff --git a/docs/release/results/images/tc010_pod_fraser.png b/docs/release/results/images/tc010_pod_fraser.png
new file mode 100644
index 000000000..23367d34a
--- /dev/null
+++ b/docs/release/results/images/tc010_pod_fraser.png
Binary files differ
diff --git a/docs/release/results/images/tc010_scenario.png b/docs/release/results/images/tc010_scenario.png
new file mode 100644
index 000000000..7c53a5fab
--- /dev/null
+++ b/docs/release/results/images/tc010_scenario.png
Binary files differ
diff --git a/docs/release/results/images/tc010_scenario_fraser.png b/docs/release/results/images/tc010_scenario_fraser.png
new file mode 100644
index 000000000..a481a595f
--- /dev/null
+++ b/docs/release/results/images/tc010_scenario_fraser.png
Binary files differ
diff --git a/docs/release/results/images/tc011_pod.png b/docs/release/results/images/tc011_pod.png
new file mode 100644
index 000000000..8fec72f5a
--- /dev/null
+++ b/docs/release/results/images/tc011_pod.png
Binary files differ
diff --git a/docs/release/results/images/tc011_pod_fraser.png b/docs/release/results/images/tc011_pod_fraser.png
new file mode 100644
index 000000000..82dc9c763
--- /dev/null
+++ b/docs/release/results/images/tc011_pod_fraser.png
Binary files differ
diff --git a/docs/release/results/images/tc011_scenario.png b/docs/release/results/images/tc011_scenario.png
new file mode 100644
index 000000000..2d78ea372
--- /dev/null
+++ b/docs/release/results/images/tc011_scenario.png
Binary files differ
diff --git a/docs/release/results/images/tc011_scenario_fraser.png b/docs/release/results/images/tc011_scenario_fraser.png
new file mode 100644
index 000000000..226d0b856
--- /dev/null
+++ b/docs/release/results/images/tc011_scenario_fraser.png
Binary files differ
diff --git a/docs/release/results/images/tc012_pod.png b/docs/release/results/images/tc012_pod.png
new file mode 100644
index 000000000..0f2a00910
--- /dev/null
+++ b/docs/release/results/images/tc012_pod.png
Binary files differ
diff --git a/docs/release/results/images/tc012_pod_fraser.png b/docs/release/results/images/tc012_pod_fraser.png
new file mode 100644
index 000000000..66e79be85
--- /dev/null
+++ b/docs/release/results/images/tc012_pod_fraser.png
Binary files differ
diff --git a/docs/release/results/images/tc012_scenario.png b/docs/release/results/images/tc012_scenario.png
new file mode 100644
index 000000000..16257988d
--- /dev/null
+++ b/docs/release/results/images/tc012_scenario.png
Binary files differ
diff --git a/docs/release/results/images/tc012_scenario_fraser.png b/docs/release/results/images/tc012_scenario_fraser.png
new file mode 100644
index 000000000..4ef44119a
--- /dev/null
+++ b/docs/release/results/images/tc012_scenario_fraser.png
Binary files differ
diff --git a/docs/release/results/images/tc014_pod.png b/docs/release/results/images/tc014_pod.png
new file mode 100644
index 000000000..63aead2e8
--- /dev/null
+++ b/docs/release/results/images/tc014_pod.png
Binary files differ
diff --git a/docs/release/results/images/tc014_pod_fraseer.png b/docs/release/results/images/tc014_pod_fraseer.png
new file mode 100644
index 000000000..697201d76
--- /dev/null
+++ b/docs/release/results/images/tc014_pod_fraseer.png
Binary files differ
diff --git a/docs/release/results/images/tc014_scenario.png b/docs/release/results/images/tc014_scenario.png
new file mode 100644
index 000000000..98f23ba1b
--- /dev/null
+++ b/docs/release/results/images/tc014_scenario.png
Binary files differ
diff --git a/docs/release/results/images/tc014_scenario_fraser.png b/docs/release/results/images/tc014_scenario_fraser.png
new file mode 100644
index 000000000..f7865dcdc
--- /dev/null
+++ b/docs/release/results/images/tc014_scenario_fraser.png
Binary files differ
diff --git a/docs/release/results/images/tc069_pod.png b/docs/release/results/images/tc069_pod.png
new file mode 100644
index 000000000..66b272cb4
--- /dev/null
+++ b/docs/release/results/images/tc069_pod.png
Binary files differ
diff --git a/docs/release/results/images/tc069_pod_fraser.png b/docs/release/results/images/tc069_pod_fraser.png
new file mode 100644
index 000000000..1cba192d7
--- /dev/null
+++ b/docs/release/results/images/tc069_pod_fraser.png
Binary files differ
diff --git a/docs/release/results/images/tc069_scenario.png b/docs/release/results/images/tc069_scenario.png
new file mode 100644
index 000000000..caf12f8d5
--- /dev/null
+++ b/docs/release/results/images/tc069_scenario.png
Binary files differ
diff --git a/docs/release/results/images/tc069_scenario_fraser.png b/docs/release/results/images/tc069_scenario_fraser.png
new file mode 100644
index 000000000..f988b90c6
--- /dev/null
+++ b/docs/release/results/images/tc069_scenario_fraser.png
Binary files differ
diff --git a/docs/release/results/images/tc082_pod.png b/docs/release/results/images/tc082_pod.png
new file mode 100644
index 000000000..89e01666b
--- /dev/null
+++ b/docs/release/results/images/tc082_pod.png
Binary files differ
diff --git a/docs/release/results/images/tc082_pod_fraser.png b/docs/release/results/images/tc082_pod_fraser.png
new file mode 100644
index 000000000..d54ab901a
--- /dev/null
+++ b/docs/release/results/images/tc082_pod_fraser.png
Binary files differ
diff --git a/docs/release/results/images/tc082_scenario.png b/docs/release/results/images/tc082_scenario.png
new file mode 100644
index 000000000..637a739c3
--- /dev/null
+++ b/docs/release/results/images/tc082_scenario.png
Binary files differ
diff --git a/docs/release/results/images/tc083_pod.png b/docs/release/results/images/tc083_pod.png
new file mode 100644
index 000000000..f874191e4
--- /dev/null
+++ b/docs/release/results/images/tc083_pod.png
Binary files differ
diff --git a/docs/release/results/images/tc083_pod_fraser.png b/docs/release/results/images/tc083_pod_fraser.png
new file mode 100644
index 000000000..942cc2074
--- /dev/null
+++ b/docs/release/results/images/tc083_pod_fraser.png
Binary files differ
diff --git a/docs/release/results/images/tc083_scenario.png b/docs/release/results/images/tc083_scenario.png
new file mode 100644
index 000000000..afd80aa02
--- /dev/null
+++ b/docs/release/results/images/tc083_scenario.png
Binary files differ
diff --git a/docs/release/results/index.rst b/docs/release/results/index.rst
index 0560152e0..63445fd1a 100644
--- a/docs/release/results/index.rst
+++ b/docs/release/results/index.rst
@@ -14,3 +14,4 @@ Yardstick test results
.. include:: ./overview.rst
.. include:: ./results.rst
+.. include:: ./euphrates_fraser_comparison.rst
diff --git a/docs/release/results/os-nosdn-kvm-ha.rst b/docs/release/results/os-nosdn-kvm-ha.rst
deleted file mode 100644
index a8a56f80e..000000000
--- a/docs/release/results/os-nosdn-kvm-ha.rst
+++ /dev/null
@@ -1,270 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-
-
-================================
-Test Results for os-nosdn-kvm-ha
-================================
-
-.. toctree::
- :maxdepth: 2
-
-
-fuel
-====
-
-.. _Grafana: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
-.. _POD2: https://wiki.opnfv.org/pharos?&#community_test_labs
-
-Overview of test results
-------------------------
-
-See Grafana_ for viewing test result metrics for each respective test case. It
-is possible to chose which specific scenarios to look at, and then to zoom in
-on the details of each run test scenario as well.
-
-All of the test case results below are based on 4 scenario test
-runs, each run on the Ericsson POD2_ or LF POD2_ between August 24 and 30 in
-2016.
-
-TC002
------
-The round-trip-time (RTT) between 2 VMs on different blades is measured using
-ping. Most test run measurements result on average between 0.44 and 0.75 ms.
-A few runs start with a 0.65 - 0.68 ms RTT spike (This could be because of
-normal ARP handling). One test run has a greater RTT spike of 1.49 ms.
-To be able to draw conclusions more runs should be made. SLA set to 10 ms.
-The SLA value is used as a reference, it has not been defined by OPNFV.
-
-TC005
------
-The IO read bandwidth looks similar between different dates, with an
-average between approx. 92 and 204 MB/s. Within each test run the results
-vary, with a minimum 2 MB/s and maximum 819 MB/s on the totality. Most runs
-have a minimum BW of 3 MB/s (one run at 2 MB/s). The maximum BW varies more in
-absolute numbers between the dates, between 238 and 819 MB/s.
-SLA set to 400 MB/s. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC010
------
-The measurements for memory latency are similar between test dates and result
-in approx. 2.07 ns. The variations within each test run are similar, between
-1.41 and 3.53 ns.
-SLA set to 30 ns. The SLA value is used as a reference, it has not been defined
-by OPNFV.
-
-TC011
------
-Packet delay variation between 2 VMs on different blades is measured using
-Iperf3. The reported packet delay variation varies between 0.0051 and 0.0243 ms,
-with an average delay variation between 0.0081 ms and 0.0195 ms.
-
-TC012
------
-Between test dates, the average measurements for memory bandwidth result in
-approx. 13.6 GB/s. Within each test run the results vary more, with a minimal
-BW of 6.09 GB/s and maximum of 16.47 GB/s on the totality.
-SLA set to 15 GB/s. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC014
------
-The Unixbench processor test run results vary between scores 2316 and 3619,
-one result each date.
-No SLA set.
-
-TC037
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-CPU utilization statistics are collected during UDP flows sent between the VMs
-using pktgen as packet generator tool. The average measurements for CPU
-utilization ratio vary between 1% to 2%. The peak of CPU utilization ratio
-appears around 7%.
-
-TC069
------
-Between test dates, the average measurements for memory bandwidth vary between
-22.6 and 29.1 GB/s. Within each test run the results vary more, with a minimal
-BW of 20.0 GB/s and maximum of 29.5 GB/s on the totality.
-SLA set to 6 GB/s. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-
-TC070
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-Memory utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The average measurements for memory
-utilization vary between 225MB to 246MB. The peak of memory utilization appears
-around 340MB.
-
-TC071
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-Cache utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The average measurements for cache
-utilization vary between 205MB to 212MB.
-
-TC072
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-Network utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. Total number of packets received per
-second was average on 200 kpps and total number of packets transmitted per
-second was average on 600 kpps.
-
-Detailed test results
----------------------
-The scenario was run on Ericsson POD2_ and LF POD2_ with:
-Fuel 9.0
-OpenStack Mitaka
-OpenVirtualSwitch 2.5.90
-OpenDayLight Beryllium
-
-Rationale for decisions
------------------------
-Pass
-
-Tests were successfully executed and metrics collected.
-No SLA was verified. To be decided on in next release of OPNFV.
-
-Conclusions and recommendations
--------------------------------
-The pktgen test configuration has a relatively large base effect on RTT in
-TC037 compared to TC002, where there is no background load at all. Approx.
-15 ms compared to approx. 0.5 ms, which is more than a 3000 percentage
-difference in RTT results.
-Especially RTT and throughput come out with better results than for instance
-the *fuel-os-nosdn-nofeature-ha* scenario does. The reason for this should
-probably be further analyzed and understood. Also of interest could be
-to make further analyzes to find patterns and reasons for lost traffic.
-Also of interest could be to see if there are continuous variations where
-some test cases stand out with better or worse results than the general test
-case.
-
diff --git a/docs/release/results/os-nosdn-nofeature-ha.rst b/docs/release/results/os-nosdn-nofeature-ha.rst
deleted file mode 100644
index 9e52731d5..000000000
--- a/docs/release/results/os-nosdn-nofeature-ha.rst
+++ /dev/null
@@ -1,492 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-
-
-======================================
-Test Results for os-nosdn-nofeature-ha
-======================================
-
-.. toctree::
- :maxdepth: 2
-
-
-apex
-====
-
-.. _Grafana: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
-.. _POD1: https://wiki.opnfv.org/pharos?&#community_test_labs
-
-
-Overview of test results
-------------------------
-
-See Grafana_ for viewing test result metrics for each respective test case. It
-is possible to chose which specific scenarios to look at, and then to zoom in
-on the details of each run test scenario as well.
-
-All of the test case results below are based on 4 scenario test
-runs, each run on the LF POD1_ between August 25 and 28 in
-2016.
-
-TC002
------
-The round-trip-time (RTT) between 2 VMs on different blades is measured using
-ping. Most test run measurements result on average between 0.74 and 1.08 ms.
-A few runs start with a 0.99 - 1.07 ms RTT spike (This could be because of
-normal ARP handling). One test run has a greater RTT spike of 1.35 ms.
-To be able to draw conclusions more runs should be made. SLA set to 10 ms.
-The SLA value is used as a reference, it has not been defined by OPNFV.
-
-TC005
------
-The IO read bandwidth looks similar between different dates, with an
-average between approx. 128 and 136 MB/s. Within each test run the results
-vary, with a minimum 5 MB/s and maximum 446 MB/s on the totality. Most runs
-have a minimum BW of 5 MB/s (one run at 6 MB/s). The maximum BW varies more in
-absolute numbers between the dates, between 416 and 446 MB/s.
-SLA set to 400 MB/s. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC010
------
-The measurements for memory latency are similar between test dates and result
-in approx. 1.09 ns. The variations within each test run are similar, between
-1.0860 and 1.0880 ns.
-SLA set to 30 ns. The SLA value is used as a reference, it has not been defined
-by OPNFV.
-
-TC011
------
-Packet delay variation between 2 VMs on different blades is measured using
-Iperf3. The reported packet delay variation varies between 0.0025 and 0.0148 ms,
-with an average delay variation between 0.0056 ms and 0.0157 ms.
-
-TC012
------
-Between test dates, the average measurements for memory bandwidth result in
-approx. 19.70 GB/s. Within each test run the results vary more, with a minimal
-BW of 18.16 GB/s and maximum of 20.13 GB/s on the totality.
-SLA set to 15 GB/s. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC014
------
-The Unixbench processor test run results vary between scores 3224.4 and 3842.8,
-one result each date. The average score on the total is 3659.5.
-No SLA set.
-
-TC037
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-CPU utilization statistics are collected during UDP flows sent between the VMs
-using pktgen as packet generator tool. The average measurements for CPU
-utilization ratio vary between 1% to 2%. The peak of CPU utilization ratio
-appears around 7%.
-
-TC069
------
-Between test dates, the average measurements for memory bandwidth vary between
-22.6 and 29.1 GB/s. Within each test run the results vary more, with a minimal
-BW of 20.0 GB/s and maximum of 29.5 GB/s on the totality.
-SLA set to 6 GB/s. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC070
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-Memory utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The average measurements for memory
-utilization vary between 225MB to 246MB. The peak of memory utilization appears
-around 340MB.
-
-TC071
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-Cache utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The average measurements for cache
-utilization vary between 205MB to 212MB.
-
-TC072
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-Network utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. Total number of packets received per
-second was average on 200 kpps and total number of packets transmitted per
-second was average on 600 kpps.
-
-Detailed test results
----------------------
-The scenario was run on LF POD1_ with:
-Apex
-OpenStack Mitaka
-OpenVirtualSwitch 2.5.90
-OpenDayLight Beryllium
-
-Rationale for decisions
------------------------
-Pass
-
-Tests were successfully executed and metrics collected.
-No SLA was verified. To be decided on in next release of OPNFV.
-
-
-Joid
-====
-
-.. _Grafana: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
-.. _POD5: https://wiki.opnfv.org/pharos?&#community_test_labs
-
-
-Overview of test results
-------------------------
-
-See Grafana_ for viewing test result metrics for each respective test case. It
-is possible to chose which specific scenarios to look at, and then to zoom in
-on the details of each run test scenario as well.
-
-All of the test case results below are based on 4 scenario test runs, each run
-on the Intel POD5_ between September 11 and 14 in 2016.
-
-TC002
------
-The round-trip-time (RTT) between 2 VMs on different blades is measured using
-ping. Most test run measurements result on average between 1.59 and 1.70 ms.
-Two test runs have reached the same greater RTT spike of 3.06 ms, which are
-1.66 and 1.70 ms average, but only one has the lower RTT of 1.35 ms. The other
-two runs have no similar spike at all. To be able to draw conclusions more runs
-should be made. SLA set to be 10 ms. The SLA value is used as a reference, it
-has not been defined by OPNFV.
-
-TC005
------
-The IO read bandwidth actually refers to the storage throughput and the
-greatest IO read bandwidth of the four runs is 173.3 MB/s. The IO read
-bandwidth of the four runs looks similar on different four days, with an
-average between 32.7 and 60.4 MB/s. One of the runs has a minimum BW of 429
-KM/s and other has a maximum BW of 173.3 MB/s. The SLA of read bandwidth sets
-to be 400 MB/s, which is used as a reference, and it has not been defined by
-OPNFV.
-
-TC010
------
-The tool we use to measure memory read latency is lmbench, which is a series of
-micro benchmarks intended to measure basic operating system and hardware system
-metrics. The memory read latency of the four runs is 1.1 ns on average. The
-variations within each test run are different, some vary from a large range and
-others have a small change. For example, the largest change is on September 14,
-the memory read latency of which is ranging from 1.12 ns to 1.22 ns. However,
-the results on September 12 change very little, which range from 1.14 ns to
-1.17 ns. The SLA sets to be 30 ns. The SLA value is used as a reference, it has
-not been defined by OPNFV.
-
-TC011
------
-Iperf3 is a tool for evaluating the pocket delay variation between 2 VMs on
-different blades. The reported pocket delay variations of the four test runs
-differ from each other. The results on September 13 within the date look
-similar and the values are between 0.0087 and 0.0190 ms, which is 0.0126 ms on
-average. However, on the fourth day, the pocket delay variation has a large
-wide change within the date, which ranges from 0.0032 ms to 0.0121 ms and has
-the minimum average value. The pocket delay variations of other two test runs
-look relatively similar, which are 0.0076 ms and 0.0152 ms on average. The SLA
-value sets to be 10 ms. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC012
------
-Lmbench is also used to measure the memory read and write bandwidth, in which
-we use bw_mem to obtain the results. Among the four test runs, the memory
-bandwidth within the second day almost keep stable, which is 11.58 GB/s on
-average. And the memory bandwidth of the fourth day look similar as that of the
-second day, both of which remain stable. The other two test runs relatively
-change from a large wide range, in which the minimum memory bandwidth is 11.22
-GB/s and the maximum bandwidth is 16.65 GB/s with an average bandwidth of about
-12.20 GB/s. Here SLA set to be 15 GB/s. The SLA value is used as a reference,
-it has not been defined by OPNFV.
-
-TC014
------
-The Unixbench is used to measure processing speed, that is instructions per
-second. It can be seen from the dashboard that the processing test results
-vary from scores 3272 to 3444, and there is only one result one date. The
-overall average score is 3371. No SLA set.
-
-TC037
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The mean packet throughput of the four test runs is 119.85, 128.02, 121.40 and
-126.08 kpps, of which the result of the second is the highest. The RTT results
-of all the test runs keep flat at approx. 37 ms. It is obvious that the PPS
-results are not as consistent as the RTT results.
-
-The No. flows of the four test runs are 240 k on average and the PPS results
-look a little waved since the largest packet throughput is 184 kpps and the
-minimum throughput is 49 K respectively.
-
-There are no errors of packets received in the four runs, but there are still
-lost packets in all the test runs. The RTT values obtained by ping of the four
-runs have the similar average vaue, that is 38 ms, of which the worest RTT is
-93 ms on Sep. 14th.
-
-CPU load of the four test runs have a large change, since the minimum value and
-the peak of CPU load is 0 percent and 51 percent respectively. And the best
-result is obtained on Sep. 14th.
-
-TC069
------
-With the block size changing from 1 kb to 512 kb, the memory write bandwidth
-tends to become larger first and then smaller within every run test, which
-rangs from 22.3 GB/s to 26.8 GB/s and then to 18.5 GB/s on average. Since the
-test id is one, it is that only the INT memory write bandwidth is tested. On
-the whole, when the block size is 8 kb and 16 kb, the memory write bandwidth
-look similar with a minimal BW of 22.5 GB/s and peak value of 28.7 GB/s. SLA
-sets to be 7 GB/s. The SLA value is used as a a reference, it has not been
-defined by OPNFV.
-
-TC070
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The network latency is measured by ping, and the results of the four test runs
-look similar with each other. Within each test run, the maximum RTT can reach
-more than 80 ms and the average RTT is usually approx. 38 ms. On the whole, the
-average RTTs of the four runs keep flat.
-
-Memory utilization is measured by free, which can display amount of free and
-used memory in the system. The largest amount of used memory is 268 MiB on Sep
-14, which also has the largest minimum memory. Besides, the rest three test
-runs have the similar used memory. On the other hand, the free memory of the
-four runs have the same smallest minimum value, that is about 223 MiB, and the
-maximum free memory of three runs have the similar result, that is 337 MiB,
-except that on Sep. 14th, whose maximum free memory is 254 MiB. On the whole,
-all the test runs have similar average free memory.
-
-Network throughput and packet loss can be measured by pktgen, which is a tool
-in the network for generating traffic loads for network experiments. The mean
-network throughput of the four test runs seem quite different, ranging from
-119.85 kpps to 128.02 kpps. The average number of flows in these tests is
-24000, and each run has a minimum number of flows of 2 and a maximum number
-of flows of 1.001 Mil. At the same time, the corresponding packet throughput
-differ between 49.4k and 193.3k with an average packet throughput of approx.
-125k. On the whole, the PPS results seem consistent. Within each test run of
-the four runs, when number of flows becomes larger, the packet throughput seems
-not larger in the meantime.
-
-TC071
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The network latency is measured by ping, and the results of the four test runs
-look similar with each other. Within each test run, the maximum RTT can reach
-more than 94 ms and the average RTT is usually approx. 35 ms. On the whole, the
-average RTTs of the four runs keep flat.
-
-Cache utilization is measured by cachestat, which can display size of cache and
-buffer in the system. Cache utilization statistics are collected during UDP
-flows sent between the VMs using pktgen as packet generator tool.The largest
-cache size is 212 MiB in the four runs, and the smallest cache size is 75 MiB.
-On the whole, the average cache size of the four runs is approx. 208 MiB.
-Meanwhile, the tread of the buffer size looks similar with each other.
-
-Packet throughput can be measured by pktgen, which is a tool in the network for
-generating traffic loads for network experiments. The mean packet throughput of
-the four test runs seem quite different, ranging from 119.85 kpps to 128.02
-kpps. The average number of flows in these tests is 239.7k, and each run has a
-minimum number of flows of 2 and a maximum number of flows of 1.001 Mil. At the
-same time, the corresponding packet throughput differ between 49.4k and 193.3k
-with an average packet throughput of approx. 125k. On the whole, the PPS results
-seem consistent. Within each test run of the four runs, when number of flows
-becomes larger, the packet throughput seems not larger in the meantime.
-
-TC072
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 32 ms. The PPS results are not as consistent as the RTT results.
-
-Network utilization is measured by sar, that is system activity reporter, which
-can display the average statistics for the time since the system was started.
-Network utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The largest total number of packets
-transmitted per second differs from each other, in which the smallest number of
-packets transmitted per second is 6 pps on Sep. 12ed and the largest of that is
-210.8 kpps. Meanwhile, the largest total number of packets received per second
-differs from each other, in which the smallest number of packets received per
-second is 2 pps on Sep. 13rd and the largest of that is 250.2 kpps.
-
-In some test runs when running with less than approx. 90000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. For the other test runs there is however no
-significant change to the PPS throughput when the number of flows are
-increased. In some test runs the PPS is also greater with 1000000 flows
-compared to other test runs where the PPS result is less with only 2 flows.
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally differs a lot per test run.
-
-Detailed test results
----------------------
-The scenario was run on Intel POD5_ with:
-Joid
-OpenStack Mitaka
-OpenVirtualSwitch 2.5.90
-OpenDayLight Beryllium
-
-Rationale for decisions
------------------------
-Pass
-
-Conclusions and recommendations
--------------------------------
-Tests were successfully executed and metrics collected.
-No SLA was verified. To be decided on in next release of OPNFV.
-
-
diff --git a/docs/release/results/os-nosdn-nofeature-noha.rst b/docs/release/results/os-nosdn-nofeature-noha.rst
deleted file mode 100644
index 8b7c184bb..000000000
--- a/docs/release/results/os-nosdn-nofeature-noha.rst
+++ /dev/null
@@ -1,259 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-
-
-========================================
-Test Results for os-nosdn-nofeature-noha
-========================================
-
-.. toctree::
- :maxdepth: 2
-
-
-Joid
-=====
-
-.. _Grafana: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
-.. _POD5: https://wiki.opnfv.org/pharos?&#community_test_labs
-
-Overview of test results
-------------------------
-
-See Grafana_ for viewing test result metrics for each respective test case. It
-is possible to chose which specific scenarios to look at, and then to zoom in
-on the details of each run test scenario as well.
-
-All of the test case results below are based on 4 scenario test runs, each run
-on the Intel POD5_ between September 12 and 15 in 2016.
-
-TC002
------
-The round-trip-time (RTT) between 2 VMs on different blades is measured using
-ping. Most test run measurements result on average between 1.50 and 1.68 ms.
-Only one test run has reached greatest RTT spike of 2.92 ms, which has
-the smallest RTT of 1.06 ms. The other three runs have no similar spike at all,
-the minimum and average RTTs of which are approx. 1.50 ms and 1.68 ms. SLA set to
-be 10 ms. The SLA value is used as a reference, it has not been defined by
-OPNFV.
-
-TC005
------
-The IO read bandwidth actually refers to the storage throughput, which is
-measured by fio and the greatest IO read bandwidth of the four runs is 177.5
-MB/s. The IO read bandwidth of the four runs looks similar on different four
-days, with an average between 46.7 and 62.5 MB/s. One of the runs has a minimum
-BW of 680 KM/s and other has a maximum BW of 177.5 MB/s. The SLA of read
-bandwidth sets to be 400 MB/s, which is used as a reference, and it has not
-been defined by OPNFV.
-
-The results of storage IOPS for the four runs look similar with each other. The
-test runs all have an approx. 1.55 K/s for IO reading with an minimum value of
-less than 60 times per second.
-
-TC010
------
-The tool we use to measure memory read latency is lmbench, which is a series of
-micro benchmarks intended to measure basic operating system and hardware system
-metrics. The memory read latency of the four runs is between 1.134 ns and 1.227
-ns on average. The variations within each test run are quite different, some
-vary from a large range and others have a small change. For example, the
-largest change is on September 15, the memory read latency of which is ranging
-from 1.116 ns to 1.393 ns. However, the results on September 12 change very
-little, which mainly keep flat and range from 1.124 ns to 1.55 ns. The SLA sets
-to be 30 ns. The SLA value is used as a reference, it has not been defined by
-OPNFV.
-
-TC011
------
-Iperf3 is a tool for evaluating the pocket delay variation between 2 VMs on
-different blades. The reported pocket delay variations of the four test runs
-differ from each other. The results on September 13 within the date look
-similar and the values are between 0.0213 and 0.0225 ms, which is 0.0217 ms on
-average. However, on the third day, the packet delay variation has a large
-wide change within the date, which ranges from 0.008 ms to 0.0225 ms and has
-the minimum value. On Sep. 12, the packet delay is quite long, for the value is
-between 0.0236 and 0.0287 ms and it also has the maximum packet delay of 0.0287
-ms. The packet delay of the last test run is 0.0151 ms on average. The SLA
-value sets to be 10 ms. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC012
------
-Lmbench is also used to measure the memory read and write bandwidth, in which
-we use bw_mem to obtain the results. Among the four test runs, the memory
-bandwidth of three test runs almost keep stable within each run, which is
-11.65, 11.57 and 11.64 GB/s on average. However, the memory read and write
-bandwidth on Sep. 14 has a large range, for it ranges from 11.36 GB/s to 16.68
-GB/s. Here SLA set to be 15 GB/s. The SLA value is used as a reference, it has
-not been defined by OPNFV.
-
-TC014
------
-The Unixbench is used to evaluate the IaaS processing speed with regards to
-score of single cpu running and parallel running. It can be seen from the
-dashboard that the processing test results vary from scores 3222 to 3585, and
-there is only one result one date. No SLA set.
-
-TC037
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The mean packet throughput of the four test runs is 124.8, 160.1, 113.8 and
-137.3 kpps, of which the result of the second is the highest. The RTT results
-of all the test runs keep flat at approx. 37 ms. It is obvious that the PPS
-results are not as consistent as the RTT results.
-
-The No. flows of the four test runs are 240 k on average and the PPS results
-look a little waved since the largest packet throughput is 243.1 kpps and the
-minimum throughput is 37.6 kpps respectively.
-
-There are no errors of packets received in the four runs, but there are still
-lost packets in all the test runs. The RTT values obtained by ping of the four
-runs have the similar average vaue, that is between 32 ms and 41 ms, of which
-the worest RTT is 155 ms on Sep. 14th.
-
-CPU load is measured by mpstat, and CPU load of the four test runs seem a
-little similar, since the minimum value and the peak of CPU load is between 0
-percent and 9 percent respectively. And the best result is obtained on Sep.
-15th, with an CPU load of nine percent.
-
-TC069
------
-With the block size changing from 1 kb to 512 kb, the memory write bandwidth
-tends to become larger first and then smaller within every run test, which
-rangs from 22.4 GB/s to 26.5 GB/s and then to 18.6 GB/s on average. Since the
-test id is one, it is that only the INT memory write bandwidth is tested. On
-the whole, when the block size is 8 kb and 16 kb, the memory write bandwidth
-look similar with a minimal BW of 22.5 GB/s and peak value of 28.7 GB/s. And
-then with the block size becoming larger, the memory write bandwidth tends to
-decrease. SLA sets to be 7 GB/s. The SLA value is used as a a reference, it has
-not been defined by OPNFV.
-
-TC070
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The network latency is measured by ping, and the results of three test runs look
-similar with each other, and Within these test runs, the maximum RTT can reach
-95 ms and the average RTT is usually approx. 36 ms. The network latency tested
-on Sep. 14 shows that it has a peak latency of 155 ms. But on the whole, the
-average RTTs of the four runs keep flat.
-
-Memory utilization is measured by free, which can display amount of free and
-used memory in the system. The largest amount of used memory is 270 MiB on Sep
-13, which also has the smallest minimum memory utilization. Besides, the rest
-three test runs have the similar used memory with an average memory usage of
-264 MiB. On the other hand, the free memory of the four runs have the same
-smallest minimum value, that is about 223 MiB, and the maximum free memory of
-three runs have the similar result, that is 226 MiB, except that on Sep. 13th,
-whose maximum free memory is 273 MiB. On the whole, all the test runs have
-similar average free memory.
-
-Network throughput and packet loss can be measured by pktgen, which is a tool
-in the network for generating traffic loads for network experiments. The mean
-network throughput of the four test runs seem quite different, ranging from
-119.85 kpps to 128.02 kpps. The average number of flows in these tests is
-240000, and each run has a minimum number of flows of 2 and a maximum number
-of flows of 1.001 Mil. At the same time, the corresponding packet throughput
-differ between 38k and 243k with an average packet throughput of approx. 134k.
-On the whole, the PPS results seem consistent. Within each test run of the four
-runs, when number of flows becomes larger, the packet throughput seems not
-larger in the meantime.
-
-TC071
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The network latency is measured by ping, and the results of the four test runs
-look similar with each other. Within each test run, the maximum RTT can reach
-79 ms and the average RTT is usually approx. 35 ms. On the whole, the average
-RTTs of the four runs keep flat.
-
-Cache utilization is measured by cachestat, which can display size of cache and
-buffer in the system. Cache utilization statistics are collected during UDP
-flows sent between the VMs using pktgen as packet generator tool.The largest
-cache size is 214 MiB in the four runs, and the smallest cache size is 100 MiB.
-On the whole, the average cache size of the four runs is approx. 210 MiB.
-Meanwhile, the tread of the buffer size looks similar with each other. On the
-other hand, the mean buffer size of the four runs keep flat, since they have a
-minimum value of approx. 7 MiB and a maximum value of 8 MiB, with an average
-value of about 8 MiB.
-
-Packet throughput can be measured by pktgen, which is a tool in the network for
-generating traffic loads for network experiments. The mean packet throughput of
-the four test runs seem quite different, ranging from 113.8 kpps to 124.8 kpps.
-The average number of flows in these tests is 240k, and each run has a minimum
-number of flows of 2 and a maximum number of flows of 1.001 Mil. At the same
-time, the corresponding packet throughput differ between 47.6k and 243.1k with
-an average packet throughput between 113.8k and 160.1k. Within each test run of
-the four runs, when number of flows becomes larger, the packet throughput seems
-not larger in the meantime.
-
-TC072
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs
-between 0 ms and 79 ms with an average leatency of approx. 35 ms. The PPS
-results are not as consistent as the RTT results, for the mean packet
-throughput of the four runs differ from 113.8 kpps to 124.8 kpps.
-
-Network utilization is measured by sar, that is system activity reporter, which
-can display the average statistics for the time since the system was started.
-Network utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The largest total number of packets
-transmitted per second look similar on the first three runs with a minimum
-number of 10 pps and a maximum number of 97 kpps, except the one on Sep. 15th,
-in which the number of packets transmitted per second is 10 pps. Meanwhile, the
-largest total number of packets received per second differs from each other,
-in which the smallest number of packets received per second is 1 pps and the
-largest of that is 276 kpps.
-
-In some test runs when running with less than approx. 90000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. For the other test runs there is however no
-significant change to the PPS throughput when the number of flows are
-increased. In some test runs the PPS is also greater with 1000000 flows
-compared to other test runs where the PPS result is less with only 2 flows.
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally differs a lot per test run.
-
-Detailed test results
----------------------
-The scenario was run on Intel POD5_ with:
-Joid
-OpenStack Mitaka
-OpenVirtualSwitch 2.5.90
-OpenDayLight Beryllium
-
-Rationale for decisions
------------------------
-Pass
-
-Conclusions and recommendations
--------------------------------
-Tests were successfully executed and metrics collected.
-No SLA was verified. To be decided on in next release of OPNFV.
diff --git a/docs/release/results/os-odl_l2-bgpvpn-ha.rst b/docs/release/results/os-odl_l2-bgpvpn-ha.rst
deleted file mode 100644
index 2bd6dc35d..000000000
--- a/docs/release/results/os-odl_l2-bgpvpn-ha.rst
+++ /dev/null
@@ -1,53 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-
-
-====================================
-Test Results for os-odl_l2-bgpvpn-ha
-====================================
-
-.. toctree::
- :maxdepth: 2
-
-
-fuel
-====
-
-.. _Grafana: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
-.. _POD2: https://wiki.opnfv.org/pharos?&#community_test_labs
-
-Overview of test results
-------------------------
-
-See Grafana_ for viewing test result metrics for each respective test case. It
-is possible to chose which specific scenarios to look at, and then to zoom in
-on the details of each run test scenario as well.
-
-All of the test case results below are based on 4 scenario test runs, each run
-on the Ericsson POD2_ between September 7 and 11 in 2016.
-
-TC043
------
-The round-trip-time (RTT) between 2 nodes is measured using
-ping. Most test run measurements result on average between 0.21 and 0.28 ms.
-A few runs start with a 0.32 - 0.35 ms RTT spike (This could be because of
-normal ARP handling). To be able to draw conclusions more runs should be made.
-SLA set to 10 ms. The SLA value is used as a reference, it has not been defined
-by OPNFV.
-
-Detailed test results
----------------------
-The scenario was run on Ericsson POD2_ with:
-Fuel 9.0
-OpenStack Mitaka
-OpenVirtualSwitch 2.5.90
-OpenDayLight Beryllium
-
-Rationale for decisions
------------------------
-Pass
-
-Tests were successfully executed and metrics collected.
-No SLA was verified. To be decided on in next release of OPNFV.
-
diff --git a/docs/release/results/os-odl_l2-nofeature-ha.rst b/docs/release/results/os-odl_l2-nofeature-ha.rst
deleted file mode 100644
index ac0c5bb59..000000000
--- a/docs/release/results/os-odl_l2-nofeature-ha.rst
+++ /dev/null
@@ -1,743 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-
-
-=======================================
-Test Results for os-odl_l2-nofeature-ha
-=======================================
-
-.. toctree::
- :maxdepth: 2
-
-
-apex
-====
-
-.. _Grafana: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
-.. _POD1: https://wiki.opnfv.org/pharos?&#community_test_labs
-
-Overview of test results
-------------------------
-
-See Grafana_ for viewing test result metrics for each respective test case. It
-is possible to chose which specific scenarios to look at, and then to zoom in
-on the details of each run test scenario as well.
-
-All of the test case results below are based on 4 scenario test runs, each run
-on the LF POD1_ between September 14 and 17 in 2016.
-
-TC002
------
-The round-trip-time (RTT) between 2 VMs on different blades is measured using
-ping. Most test run measurements result on average between 0.49 ms and 0.60 ms.
-Only one test run has reached greatest RTT spike of 0.93 ms. Meanwhile, the
-smallest network latency is 0.33 ms, which is obtained on Sep. 14th.
-SLA set to be 10 ms. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC005
------
-The IO read bandwidth actually refers to the storage throughput, which is
-measured by fio and the greatest IO read bandwidth of the four runs is 416
-MB/s. The IO read bandwidth of all four runs looks similar, with an average
-between 128 and 131 MB/s. One of the runs has a minimum BW of 497 KB/s. The SLA
-of read bandwidth sets to be 400 MB/s, which is used as a reference, and it has
-not been defined by OPNFV.
-
-The results of storage IOPS for the four runs look similar with each other. The
-IO read times per second of the four test runs have an average value at 1k per
-second, and meanwhile, the minimum result is only 45 times per second.
-
-TC010
------
-The tool we use to measure memory read latency is lmbench, which is a series of
-micro benchmarks intended to measure basic operating system and hardware system
-metrics. The memory read latency of the four runs is between 1.0859 ns and
-1.0869 ns on average. The variations within each test run are quite different,
-some vary from a large range and others have a small change. For example, the
-largest change is on September 14th, the memory read latency of which is ranging
-from 1.091 ns to 1.086 ns. However.
-The SLA sets to be 30 ns. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC011
------
-Packet delay variation between 2 VMs on different blades is measured using
-Iperf3. On the first two test runs the reported packet delay variation varies between
-0.0037 and 0.0740 ms, with an average delay variation between 0.0096 ms and 0.0321.
-On the second date the delay variation varies between 0.0063 and 0.0096 ms, with
-an average delay variation of 0.0124 - 0.0141 ms.
-
-TC012
------
-Lmbench is also used to measure the memory read and write bandwidth, in which
-we use bw_mem to obtain the results. Among the four test runs, the trend of
-three memory bandwidth almost look similar, which all have a narrow range, and
-the average result is 19.88 GB/s. Here SLA set to be 15 GB/s. The SLA value is
-used as a reference, it has not been defined by OPNFV.
-
-TC014
------
-The Unixbench is used to evaluate the IaaS processing speed with regards to
-score of single cpu running and parallel running. It can be seen from the
-dashboard that the processing test results vary from scores 3754k to 3831k, and
-there is only one result one date. No SLA set.
-
-TC037
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The mean packet throughput of the four test runs is between 307.3 kpps and
-447.1 kpps, of which the result of the third run is the highest. The RTT
-results of all the test runs keep flat at approx. 15 ms. It is obvious that the
-PPS results are not as consistent as the RTT results.
-
-The No. flows of the four test runs are 240 k on average and the PPS results
-look a little waved since the largest packet throughput is 418.1 kpps and the
-minimum throughput is 326.5 kpps respectively.
-
-There are no errors of packets received in the four runs, but there are still
-lost packets in all the test runs. The RTT values obtained by ping of the four
-runs have the similar average vaue, that is approx. 15 ms.
-
-CPU load is measured by mpstat, and CPU load of the four test runs seem a
-little similar, since the minimum value and the peak of CPU load is between 0
-percent and nine percent respectively. And the best result is obtained on Sep.
-1, with an CPU load of nine percent. But on the whole, the CPU load is very
-poor, since the average value is quite small.
-
-TC069
------
-With the block size changing from 1 kb to 512 kb, the memory write bandwidth
-tends to become larger first and then smaller within every run test, which
-rangs from 28.2 GB/s to 29.5 GB/s and then to 29.2 GB/s on average. Since the
-test id is one, it is that only the INT memory write bandwidth is tested. On
-the whole, when the block size is 2 kb or 16 kb, the memory write bandwidth
-look similar with a minimal BW of 25.8 GB/s and peak value of 28.3 GB/s. And
-then with the block size becoming larger, the memory write bandwidth tends to
-decrease. SLA sets to be 7 GB/s. The SLA value is used as a reference, it has
-not been defined by OPNFV.
-
-TC070
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The network latency is measured by ping, and the results of the four test runs
-look similar with each other, and within these test runs, the maximum RTT can
-reach 39 ms and the average RTT is usually approx. 15 ms. The network latency
-tested on Sep. 1 and Sep. 8 have a peak latency of 39 ms. But on the whole,
-the average RTTs of the five runs keep flat and the network latency is
-relatively short.
-
-Memory utilization is measured by free, which can display amount of free and
-used memory in the system. The largest amount of used memory is 267 MiB for the
-four runs. In general, the four test runs have very large memory utilization,
-which can reach 257 MiB on average. On the other hand, for the mean free memory,
-the four test runs have the similar trend with that of the mean used memory.
-In general, the mean free memory change from 233 MiB to 241 MiB.
-
-Packet throughput and packet loss can be measured by pktgen, which is a tool
-in the network for generating traffic loads for network experiments. The mean
-packet throughput of the four test runs seem quite different, ranging from
-305.3 kpps to 447.1 kpps. The average number of flows in these tests is
-240000, and each run has a minimum number of flows of 2 and a maximum number
-of flows of 1.001 Mil. At the same time, the corresponding average packet
-throughput is between 354.4 kpps and 381.8 kpps. In summary, the PPS results
-seem consistent. Within each test run of the four runs, when number of flows
-becomes larger, the packet throughput seems not larger at the same time.
-
-TC071
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The network latency is measured by ping, and the results of the four test runs
-look similar with each other. Within each test run, the maximum RTT is only 42
-ms and the average RTT is usually approx. 15 ms. On the whole, the average
-RTTs of the four runs keep stable and the network latency is relatively small.
-
-Cache utilization is measured by cachestat, which can display size of cache and
-buffer in the system. Cache utilization statistics are collected during UDP
-flows sent between the VMs using pktgen as packet generator tool. The largest
-cache size is 212 MiB, which is same for the four runs, and the smallest cache
-size is 75 MiB. On the whole, the average cache size of the four runs look the
-same and is between 197 MiB and 211 MiB. Meanwhile, the tread of the buffer
-size keep flat, since they have a minimum value of 7 MiB and a maximum value of
-8 MiB, with an average value of about 7.9 MiB.
-
-Packet throughput can be measured by pktgen, which is a tool in the network for
-generating traffic loads for network experiments. The mean packet throughput of
-the four test runs differ from 354.4 kpps to 381.8 kpps. The average number of
-flows in these tests is 240k, and each run has a minimum number of flows of 2
-and a maximum number of flows of 1.001 Mil. At the same time, the corresponding
-packet throughput differ between 305.3 kpps to 447.1 kpps. Within each test run
-of the four runs, when number of flows becomes larger, the packet throughput
-seems not larger in the meantime.
-
-TC072
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs
-between 0 ms and 42 ms with an average leatency of less than 15 ms. The PPS
-results are not as consistent as the RTT results, for the mean packet
-throughput of the four runs differ from 354.4 kpps to 381.8 kpps.
-
-Network utilization is measured by sar, that is system activity reporter, which
-can display the average statistics for the time since the system was started.
-Network utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The largest total number of packets
-transmitted per second look similar for three test runs, whose values change a
-lot from 10 pps to 501 kpps. While results of the rest test run seem the same
-and keep stable with the average number of packets transmitted per second of 10
-pps. However, the total number of packets received per second of the four runs
-look similar, which have a large wide range of 2 pps to 815 kpps.
-
-In some test runs when running with less than approx. 251000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. For the other test runs there is however no
-significant change to the PPS throughput when the number of flows are
-increased. In some test runs the PPS is also greater with 251000 flows
-compared to other test runs where the PPS result is less with only 2 flows.
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally differs a lot per test run.
-
-Detailed test results
----------------------
-The scenario was run on LF POD1_ with:
-Apex
-OpenStack Mitaka
-OpenVirtualSwitch 2.5.90
-OpenDayLight Beryllium
-
-Rationale for decisions
------------------------
-Pass
-
-Conclusions and recommendations
--------------------------------
-Tests were successfully executed and metrics collected.
-No SLA was verified. To be decided on in next release of OPNFV.
-
-
-
-fuel
-====
-
-.. _Grafana: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
-.. _POD2: https://wiki.opnfv.org/pharos?&#community_test_labs
-
-Overview of test results
-------------------------
-
-See Grafana_ for viewing test result metrics for each respective test case. It
-is possible to chose which specific scenarios to look at, and then to zoom in
-on the details of each run test scenario as well.
-
-All of the test case results below are based on 4 scenario test runs, each run
-on the Ericsson POD2_ or LF POD2_ between August 25 and 29 in 2016.
-
-TC002
------
-The round-trip-time (RTT) between 2 VMs on different blades is measured using
-ping. Most test run measurements result on average between 0.5 and 0.6 ms.
-A few runs start with a 1 - 1.5 ms RTT spike (This could be because of normal ARP
-handling). One test run has a greater RTT spike of 1.9 ms, which is the same
-one with the 0.7 ms average. The other runs have no similar spike at all.
-To be able to draw conclusions more runs should be made.
-SLA set to 10 ms. The SLA value is used as a reference, it has not
-been defined by OPNFV.
-
-TC005
------
-The IO read bandwidth looks similar between different dates, with an
-average between approx. 170 and 200 MB/s. Within each test run the results
-vary, with a minimum 2 MB/s and maximum 838 MB/s on the totality. Most runs
-have a minimum BW of 3 MB/s (two runs at 2 MB/s). The maximum BW varies more in
-absolute numbers between the dates, between 617 and 838 MB/s.
-SLA set to 400 MB/s. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC010
------
-The measurements for memory latency are similar between test dates and result
-in approx. 1.2 ns. The variations within each test run are similar, between
-1.215 and 1.219 ns. One exception is February 16, where the average is 1.222
-and varies between 1.22 and 1.28 ns.
-SLA set to 30 ns. The SLA value is used as a reference, it has not been defined
-by OPNFV.
-
-TC011
------
-Packet delay variation between 2 VMs on different blades is measured using
-Iperf3. On the first date the reported packet delay variation varies between
-0.0025 and 0.011 ms, with an average delay variation of 0.0067 ms.
-On the second date the delay variation varies between 0.002 and 0.006 ms, with
-an average delay variation of 0.004 ms.
-
-TC012
------
-Between test dates, the average measurements for memory bandwidth vary between
-17.4 and 17.9 GB/s. Within each test run the results vary more, with a minimal
-BW of 16.4 GB/s and maximum of 18.2 GB/s on the totality.
-SLA set to 15 GB/s. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC014
------
-The Unixbench processor test run results vary between scores 3080 and 3240,
-one result each date. The average score on the total is 3150.
-No SLA set.
-
-TC037
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-CPU utilization statistics are collected during UDP flows sent between the VMs
-using pktgen as packet generator tool. The average measurements for CPU
-utilization ratio vary between 1% to 2%. The peak of CPU utilization ratio
-appears around 7%.
-
-TC069
------
-Between test dates, the average measurements for memory bandwidth vary between
-15.5 and 25.4 GB/s. Within each test run the results vary more, with a minimal
-BW of 9.7 GB/s and maximum of 29.5 GB/s on the totality.
-SLA set to 6 GB/s. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC070
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-Memory utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The average measurements for memory
-utilization vary between 225MB to 246MB. The peak of memory utilization appears
-around 340MB.
-
-TC071
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-Cache utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The average measurements for cache
-utilization vary between 205MB to 212MB.
-
-TC072
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-Network utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. Total number of packets received per
-second was average on 200 kpps and total number of packets transmitted per
-second was average on 600 kpps.
-
-Detailed test results
----------------------
-The scenario was run on Ericsson POD2_ and LF POD2_ with:
-Fuel 9.0
-OpenStack Mitaka
-OpenVirtualSwitch 2.5.90
-OpenDayLight Beryllium
-
-Rationale for decisions
------------------------
-Pass
-
-Tests were successfully executed and metrics collected.
-No SLA was verified. To be decided on in next release of OPNFV.
-
-Conclusions and recommendations
--------------------------------
-The pktgen test configuration has a relatively large base effect on RTT in
-TC037 compared to TC002, where there is no background load at all. Approx.
-15 ms compared to approx. 0.5 ms, which is more than a 3000 percentage
-difference in RTT results.
-Especially RTT and throughput come out with better results than for instance
-the *fuel-os-nosdn-nofeature-ha* scenario does. The reason for this should
-probably be further analyzed and understood. Also of interest could be
-to make further analyzes to find patterns and reasons for lost traffic.
-Also of interest could be to see if there are continuous variations where
-some test cases stand out with better or worse results than the general test
-case.
-
-
-
-Joid
-=====
-
-.. _Grafana: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
-.. _POD6: https://wiki.opnfv.org/pharos?&#community_test_labs
-
-Overview of test results
-------------------------
-
-See Grafana_ for viewing test result metrics for each respective test case. It
-is possible to chose which specific scenarios to look at, and then to zoom in
-on the details of each run test scenario as well.
-
-All of the test case results below are based on 4 scenario test runs, each run
-on the Intel POD6_ between September 1 and 8 in 2016.
-
-TC002
------
-The round-trip-time (RTT) between 2 VMs on different blades is measured using
-ping. Most test run measurements result on average between 1.01 ms and 1.88 ms.
-Only one test run has reached greatest RTT spike of 1.88 ms. Meanwhile, the
-smallest network latency is 1.01 ms, which is obtained on Sep. 1st. In general,
-the average of network latency of the four test runs are between 1.29 ms and
-1.34 ms. SLA set to be 10 ms. The SLA value is used as a reference, it has not
-been defined by OPNFV.
-
-TC005
------
-The IO read bandwidth actually refers to the storage throughput, which is
-measured by fio and the greatest IO read bandwidth of the four runs is 183.65
-MB/s. The IO read bandwidth of the three runs looks similar, with an average
-between 62.9 and 64.3 MB/s, except one on Sep. 1, for its maximum storage
-throughput is only 159.1 MB/s. One of the runs has a minimum BW of 685 KB/s and
-other has a maximum BW of 183.6 MB/s. The SLA of read bandwidth sets to be
-400 MB/s, which is used as a reference, and it has not been defined by OPNFV.
-
-The results of storage IOPS for the four runs look similar with each other. The
-IO read times per second of the four test runs have an average value between
-1.41k per second and 1.64k per second, and meanwhile, the minimum result is
-only 55 times per second.
-
-TC010
------
-The tool we use to measure memory read latency is lmbench, which is a series of
-micro benchmarks intended to measure basic operating system and hardware system
-metrics. The memory read latency of the four runs is between 1.152 ns and 1.179
-ns on average. The variations within each test run are quite different, some
-vary from a large range and others have a small change. For example, the
-largest change is on September 8, the memory read latency of which is ranging
-from 1.120 ns to 1.221 ns. However, the results on September 7 change very
-little. The SLA sets to be 30 ns. The SLA value is used as a reference, it has
-not been defined by OPNFV.
-
-TC011
------
-Iperf3 is a tool for evaluating the packet delay variation between 2 VMs on
-different blades. The reported packet delay variations of the four test runs
-differ from each other. In general, the packet delay of the first two runs look
-similar, for they both stay stable within each run. And the mean packet delay
-of them are 0.0087 ms and 0.0127 ms respectively. Of the four runs, the fourth
-has the worst result, because the packet delay reaches 0.0187 ms. The SLA value
-sets to be 10 ms. The SLA value is used as a reference, it has not been defined
-by OPNFV.
-
-TC012
------
-Lmbench is also used to measure the memory read and write bandwidth, in which
-we use bw_mem to obtain the results. Among the four test runs, the trend of
-three memory bandwidth almost look similar, which all have a narrow range, and
-the average result is 11.78 GB/s. Here SLA set to be 15 GB/s. The SLA value is
-used as a reference, it has not been defined by OPNFV.
-
-TC014
------
-The Unixbench is used to evaluate the IaaS processing speed with regards to
-score of single cpu running and parallel running. It can be seen from the
-dashboard that the processing test results vary from scores 3260k to 3328k, and
-there is only one result one date. No SLA set.
-
-TC037
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The mean packet throughput of the four test runs is between 307.3 kpps and
-447.1 kpps, of which the result of the third run is the highest. The RTT
-results of all the test runs keep flat at approx. 15 ms. It is obvious that the
-PPS results are not as consistent as the RTT results.
-
-The No. flows of the four test runs are 240 k on average and the PPS results
-look a little waved since the largest packet throughput is 418.1 kpps and the
-minimum throughput is 326.5 kpps respectively.
-
-There are no errors of packets received in the four runs, but there are still
-lost packets in all the test runs. The RTT values obtained by ping of the four
-runs have the similar average vaue, that is approx. 15 ms.
-
-CPU load is measured by mpstat, and CPU load of the four test runs seem a
-little similar, since the minimum value and the peak of CPU load is between 0
-percent and nine percent respectively. And the best result is obtained on Sep.
-1, with an CPU load of nine percent. But on the whole, the CPU load is very
-poor, since the average value is quite small.
-
-TC069
------
-With the block size changing from 1 kb to 512 kb, the memory write bandwidth
-tends to become larger first and then smaller within every run test, which
-rangs from 21.9 GB/s to 25.9 GB/s and then to 17.8 GB/s on average. Since the
-test id is one, it is that only the INT memory write bandwidth is tested. On
-the whole, when the block size is 2 kb or 16 kb, the memory write bandwidth
-look similar with a minimal BW of 24.8 GB/s and peak value of 27.8 GB/s. And
-then with the block size becoming larger, the memory write bandwidth tends to
-decrease. SLA sets to be 7 GB/s. The SLA value is used as a reference, it has
-not been defined by OPNFV.
-
-TC070
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The network latency is measured by ping, and the results of the four test runs
-look similar with each other, and within these test runs, the maximum RTT can
-reach 39 ms and the average RTT is usually approx. 15 ms. The network latency
-tested on Sep. 1 and Sep. 8 have a peak latency of 39 ms. But on the whole,
-the average RTTs of the five runs keep flat and the network latency is
-relatively short.
-
-Memory utilization is measured by free, which can display amount of free and
-used memory in the system. The largest amount of used memory is 267 MiB for the
-four runs. In general, the four test runs have very large memory utilization,
-which can reach 257 MiB on average. On the other hand, for the mean free memory,
-the four test runs have the similar trend with that of the mean used memory.
-In general, the mean free memory change from 233 MiB to 241 MiB.
-
-Packet throughput and packet loss can be measured by pktgen, which is a tool
-in the network for generating traffic loads for network experiments. The mean
-packet throughput of the four test runs seem quite different, ranging from
-305.3 kpps to 447.1 kpps. The average number of flows in these tests is
-240000, and each run has a minimum number of flows of 2 and a maximum number
-of flows of 1.001 Mil. At the same time, the corresponding average packet
-throughput is between 354.4 kpps and 381.8 kpps. In summary, the PPS results
-seem consistent. Within each test run of the four runs, when number of flows
-becomes larger, the packet throughput seems not larger at the same time.
-
-TC071
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The network latency is measured by ping, and the results of the four test runs
-look similar with each other. Within each test run, the maximum RTT is only 42
-ms and the average RTT is usually approx. 15 ms. On the whole, the average
-RTTs of the four runs keep stable and the network latency is relatively small.
-
-Cache utilization is measured by cachestat, which can display size of cache and
-buffer in the system. Cache utilization statistics are collected during UDP
-flows sent between the VMs using pktgen as packet generator tool. The largest
-cache size is 212 MiB, which is same for the four runs, and the smallest cache
-size is 75 MiB. On the whole, the average cache size of the four runs look the
-same and is between 197 MiB and 211 MiB. Meanwhile, the tread of the buffer
-size keep flat, since they have a minimum value of 7 MiB and a maximum value of
-8 MiB, with an average value of about 7.9 MiB.
-
-Packet throughput can be measured by pktgen, which is a tool in the network for
-generating traffic loads for network experiments. The mean packet throughput of
-the four test runs differ from 354.4 kpps to 381.8 kpps. The average number of
-flows in these tests is 240k, and each run has a minimum number of flows of 2
-and a maximum number of flows of 1.001 Mil. At the same time, the corresponding
-packet throughput differ between 305.3 kpps to 447.1 kpps. Within each test run
-of the four runs, when number of flows becomes larger, the packet throughput
-seems not larger in the meantime.
-
-TC072
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs
-between 0 ms and 42 ms with an average leatency of less than 15 ms. The PPS
-results are not as consistent as the RTT results, for the mean packet
-throughput of the four runs differ from 354.4 kpps to 381.8 kpps.
-
-Network utilization is measured by sar, that is system activity reporter, which
-can display the average statistics for the time since the system was started.
-Network utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The largest total number of packets
-transmitted per second look similar for three test runs, whose values change a
-lot from 10 pps to 501 kpps. While results of the rest test run seem the same
-and keep stable with the average number of packets transmitted per second of 10
-pps. However, the total number of packets received per second of the four runs
-look similar, which have a large wide range of 2 pps to 815 kpps.
-
-In some test runs when running with less than approx. 251000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. For the other test runs there is however no
-significant change to the PPS throughput when the number of flows are
-increased. In some test runs the PPS is also greater with 251000 flows
-compared to other test runs where the PPS result is less with only 2 flows.
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally differs a lot per test run.
-
-Detailed test results
----------------------
-The scenario was run on Intel POD6_ with:
-Joid
-OpenStack Mitaka
-OpenVirtualSwitch 2.5.90
-OpenDayLight Beryllium
-
-Rationale for decisions
------------------------
-Pass
-
-Conclusions and recommendations
--------------------------------
-Tests were successfully executed and metrics collected.
-No SLA was verified. To be decided on in next release of OPNFV.
-
diff --git a/docs/release/results/os-odl_l2-sfc-ha.rst b/docs/release/results/os-odl_l2-sfc-ha.rst
deleted file mode 100644
index e27562cae..000000000
--- a/docs/release/results/os-odl_l2-sfc-ha.rst
+++ /dev/null
@@ -1,231 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-
-
-==================================
-Test Results for os-odl_l2-sfc-ha
-==================================
-
-.. toctree::
- :maxdepth: 2
-
-
-Fuel
-=====
-
-.. _Grafana: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
-.. _POD2: https://wiki.opnfv.org/pharos?&#community_test_labs
-
-Overview of test results
-------------------------
-
-See Grafana_ for viewing test result metrics for each respective test case. It
-is possible to chose which specific scenarios to look at, and then to zoom in
-on the details of each run test scenario as well.
-
-All of the test case results below are based on 4 scenario test runs, each run
-on the LF POD2_ or Ericsson POD2_ between September 16 and 20 in 2016.
-
-TC002
------
-The round-trip-time (RTT) between 2 VMs on different blades is measured using
-ping. Most test run measurements result on average between 0.32 ms and 1.42 ms.
-Only one test run on Sep. 20 has reached greatest RTT spike of 4.66 ms.
-Meanwhile, the smallest network latency is 0.16 ms, which is obtained on Sep.
-17th. To sum up, the curve of network latency has very small wave, which is
-less than 5 ms. SLA sets to be 10 ms. The SLA value is used as a reference, it
-has not been defined by OPNFV.
-
-TC005
------
-The IO read bandwidth actually refers to the storage throughput, which is
-measured by fio and the greatest IO read bandwidth of the four runs is 734
-MB/s. The IO read bandwidth of the first three runs looks similar, with an
-average of less than 100 KB/s, except one on Sep. 20, whose maximum storage
-throughput can reach 734 MB/s. The SLA of read bandwidth sets to be 400 MB/s,
-which is used as a reference, and it has not been defined by OPNFV.
-
-The results of storage IOPS for the four runs look similar with each other. The
-IO read times per second of the four test runs have an average value between
-1.8k per second and 3.27k per second, and meanwhile, the minimum result is
-only 60 times per second.
-
-TC010
------
-The tool we use to measure memory read latency is lmbench, which is a series of
-micro benchmarks intended to measure basic operating system and hardware system
-metrics. The memory read latency of the four runs is between 1.085 ns and 1.218
-ns on average. The variations within each test run are quite small. For
-Ericsson pod2, the average of memory latency is approx. 1.217 ms. While for LF
-pod2, the average value is about 1.085 ms. It can be seen that the performance
-of LF is better than Ericsson's. The SLA sets to be 30 ns. The SLA value is
-used as a reference, it has not been defined by OPNFV.
-
-TC012
------
-Lmbench is also used to measure the memory read and write bandwidth, in which
-we use bw_mem to obtain the results. The four test runs all have a narrow range
-of change with the average memory and write BW of 18.5 GB/s. Here SLA set to be
-15 GB/s. The SLA value is used as a reference, it has not been defined by OPNFV.
-
-TC014
------
-The Unixbench is used to evaluate the IaaS processing speed with regards to
-score of single cpu running and parallel running. It can be seen from the
-dashboard that the processing test results vary from scores 3209k to 3843k, and
-there is only one result one date. No SLA set.
-
-TC037
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The mean packet throughput of the three test runs is between 439 kpps and
-582 kpps, and the test run on Sep. 17th has the lowest average value of 371
-kpps. The RTT results of all the test runs keep flat at approx. 10 ms. It is
-obvious that the PPS results are not as consistent as the RTT results.
-
-The No. flows of the four test runs are 240 k on average and the PPS results
-look a little waved, since the largest packet throughput is 680 kpps and the
-minimum throughput is 319 kpps respectively.
-
-There are no errors of packets received in the four runs, but there are still
-lost packets in all the test runs. The RTT values obtained by ping of the four
-runs have the similar trend of RTT with the average value of approx. 12 ms.
-
-CPU load is measured by mpstat, and CPU load of the four test runs seem a
-little similar, since the minimum value and the peak of CPU load is between 0
-percent and ten percent respectively. And the best result is obtained on Sep.
-17th, with an CPU load of ten percent. But on the whole, the CPU load is very
-poor, since the average value is quite small.
-
-TC069
------
-With the block size changing from 1 kb to 512 kb, the average memory write
-bandwidth tends to become larger first and then smaller within every run test
-for the two pods, which rangs from 25.1 GB/s to 29.4 GB/s and then to 19.2 GB/s
-on average. Since the test id is one, it is that only the INT memory write
-bandwidth is tested. On the whole, with the block size becoming larger, the
-memory write bandwidth tends to decrease. SLA sets to be 7 GB/s. The SLA value
-is used as a reference, it has not been defined by OPNFV.
-
-TC070
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The network latency is measured by ping, and the results of the four test runs
-look similar with each other, and within these test runs, the maximum RTT can
-reach 27 ms and the average RTT is usually approx. 12 ms. The network latency
-tested on Sep. 27th has a peak latency of 27 ms. But on the whole, the average
-RTTs of the four runs keep flat.
-
-Memory utilization is measured by free, which can display amount of free and
-used memory in the system. The largest amount of used memory is 269 MiB for the
-four runs. In general, the four test runs have very large memory utilization,
-which can reach 251 MiB on average. On the other hand, for the mean free memory,
-the four test runs have the similar trend with that of the mean used memory.
-In general, the mean free memory change from 231 MiB to 248 MiB.
-
-Packet throughput and packet loss can be measured by pktgen, which is a tool
-in the network for generating traffic loads for network experiments. The mean
-packet throughput of the four test runs seem quite different, ranging from
-371 kpps to 582 kpps. The average number of flows in these tests is
-240000, and each run has a minimum number of flows of 2 and a maximum number
-of flows of 1.001 Mil. At the same time, the corresponding average packet
-throughput is between 319 kpps and 680 kpps. In summary, the PPS results
-seem consistent. Within each test run of the four runs, when number of flows
-becomes larger, the packet throughput seems not larger at the same time.
-
-TC071
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The network latency is measured by ping, and the results of the four test runs
-look similar with each other. Within each test run, the maximum RTT is only 24
-ms and the average RTT is usually approx. 12 ms. On the whole, the average
-RTTs of the four runs keep stable and the network latency is relatively small.
-
-Cache utilization is measured by cachestat, which can display size of cache and
-buffer in the system. Cache utilization statistics are collected during UDP
-flows sent between the VMs using pktgen as packet generator tool. The largest
-cache size is 213 MiB, and the smallest cache size is 99 MiB, which is same for
-the four runs. On the whole, the average cache size of the four runs look the
-same and is between 184 MiB and 205 MiB. Meanwhile, the tread of the buffer
-size keep stable, since they have a minimum value of 7 MiB and a maximum value of
-8 MiB.
-
-Packet throughput can be measured by pktgen, which is a tool in the network for
-generating traffic loads for network experiments. The mean packet throughput of
-the four test runs differ from 371 kpps to 582 kpps. The average number of
-flows in these tests is 240k, and each run has a minimum number of flows of 2
-and a maximum number of flows of 1.001 Mil. At the same time, the corresponding
-packet throughput differ between 319 kpps to 680 kpps. Within each test run
-of the four runs, when number of flows becomes larger, the packet throughput
-seems not larger in the meantime.
-
-TC072
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs
-between 0 ms and 24 ms with an average leatency of less than 13 ms. The PPS
-results are not as consistent as the RTT results, for the mean packet
-throughput of the four runs differ from 370 kpps to 582 kpps.
-
-Network utilization is measured by sar, that is system activity reporter, which
-can display the average statistics for the time since the system was started.
-Network utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The largest total number of packets
-transmitted per second look similar for the four test runs, whose values change a
-lot from 10 pps to 697 kpps. However, the total number of packets received per
-second of three runs look similar, which have a large wide range of 2 pps to
-1.497 Mpps, while the results on Sep. 18th and 20th have very small maximum
-number of packets received per second of 817 kpps.
-
-In some test runs when running with less than approx. 251000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. For the other test runs there is however no
-significant change to the PPS throughput when the number of flows are
-increased. In some test runs the PPS is also greater with 251000 flows
-compared to other test runs where the PPS result is less with only 2 flows.
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally differs a lot per test run.
-
-Detailed test results
----------------------
-The scenario was run on Ericsson POD2_ and LF POD2_ with:
-Fuel 9.0
-OpenStack Mitaka
-OpenVirtualSwitch 2.5.90
-OpenDayLight Beryllium
-
-Rationale for decisions
------------------------
-Pass
-
-Conclusions and recommendations
--------------------------------
-Tests were successfully executed and metrics collected.
-No SLA was verified. To be decided on in next release of OPNFV.
diff --git a/docs/release/results/os-onos-nofeature-ha.rst b/docs/release/results/os-onos-nofeature-ha.rst
deleted file mode 100644
index d8b3ace5f..000000000
--- a/docs/release/results/os-onos-nofeature-ha.rst
+++ /dev/null
@@ -1,257 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-
-
-======================================
-Test Results for os-onos-nofeature-ha
-======================================
-
-.. toctree::
- :maxdepth: 2
-
-
-Joid
-=====
-
-.. _Grafana: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
-.. _POD6: https://wiki.opnfv.org/pharos?&#community_test_labs
-
-Overview of test results
-------------------------
-
-See Grafana_ for viewing test result metrics for each respective test case. It
-is possible to chose which specific scenarios to look at, and then to zoom in
-on the details of each run test scenario as well.
-
-All of the test case results below are based on 5 scenario test runs, each run
-on the Intel POD6_ between September 13 and 16 in 2016.
-
-TC002
------
-The round-trip-time (RTT) between 2 VMs on different blades is measured using
-ping. Most test run measurements result on average between 1.50 and 1.68 ms.
-Only one test run has reached greatest RTT spike of 2.62 ms, which has
-the smallest RTT of 1.00 ms. The other four runs have no similar spike at all,
-the minimum and average RTTs of which are approx. 1.06 ms and 1.32 ms. SLA set
-to be 10 ms. The SLA value is used as a reference, it has not been defined by
-OPNFV.
-
-TC005
------
-The IO read bandwidth actually refers to the storage throughput, which is
-measured by fio and the greatest IO read bandwidth of the four runs is 175.4
-MB/s. The IO read bandwidth of the four runs looks similar on different four
-days, with an average between 58.1 and 62.0 MB/s, except one on Sep. 14, for
-its maximum storage throughput is only 133.0 MB/s. One of the runs has a
-minimum BW of 497 KM/s and other has a maximum BW of 177.4 MB/s. The SLA of read
-bandwidth sets to be 400 MB/s, which is used as a reference, and it has not
-been defined by OPNFV.
-
-The results of storage IOPS for the five runs look similar with each other. The
-IO read times per second of the five test runs have an average value between
-1.20 K/s and 1.61 K/s, and meanwhile, the minimum result is only 41 times per
-second.
-
-TC010
------
-The tool we use to measure memory read latency is lmbench, which is a series of
-micro benchmarks intended to measure basic operating system and hardware system
-metrics. The memory read latency of the five runs is between 1.146 ns and 1.172
-ns on average. The variations within each test run are quite different, some
-vary from a large range and others have a small change. For example, the
-largest change is on September 13, the memory read latency of which is ranging
-from 1.152 ns to 1.221 ns. However, the results on September 14 change very
-little. The SLA sets to be 30 ns. The SLA value is used as a reference, it has
-not been defined by OPNFV.
-
-TC011
------
-Iperf3 is a tool for evaluating the packet delay variation between 2 VMs on
-different blades. The reported packet delay variations of the five test runs
-differ from each other. In general, the packet delay of the first two runs look
-similar, for they both stay stable within each run. And the mean packet delay of
-of them are 0.07714 ms and 0.07982 ms respectively. Of the five runs, the third
-has the worst result, because the packet delay reaches 0.08384 ms. The trend of
-therest two runs look the same, for the average packet delay are 0.07808 ms and
-0.07727 ms respectively. The SLA value sets to be 10 ms. The SLA value is used
-as a reference, it has not been defined by OPNFV.
-
-TC012
------
-Lmbench is also used to measure the memory read and write bandwidth, in which
-we use bw_mem to obtain the results. Among the five test runs, the memory
-bandwidth of last three test runs almost keep stable within each run, which is
-11.64, 11.71 and 11.61 GB/s on average. However, the memory read and write
-bandwidth on Sep. 13 has a large range, for it ranges from 6.68 GB/s to 11.73
-GB/s. Here SLA set to be 15 GB/s. The SLA value is used as a reference, it has
-not been defined by OPNFV.
-
-TC014
------
-The Unixbench is used to evaluate the IaaS processing speed with regards to
-score of single cpu running and parallel running. It can be seen from the
-dashboard that the processing test results vary from scores 3208 to 3314, and
-there is only one result one date. No SLA set.
-
-TC037
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The mean packet throughput of the five test runs is between 259.6 kpps and
-318.4 kpps, of which the result of the second run is the highest. The RTT
-results of all the test runs keep flat at approx. 20 ms. It is obvious that the
-PPS results are not as consistent as the RTT results.
-
-The No. flows of the five test runs are 240 k on average and the PPS results
-look a little waved since the largest packet throughput is 398.9 kpps and the
-minimum throughput is 250.6 kpps respectively.
-
-There are no errors of packets received in the five runs, but there are still
-lost packets in all the test runs. The RTT values obtained by ping of the five
-runs have the similar average vaue, that is between 17 ms and 22 ms, of which
-the worest RTT is 53 ms on Sep. 14th.
-
-CPU load is measured by mpstat, and CPU load of the four test runs seem a
-little similar, since the minimum value and the peak of CPU load is between 0
-percent and 10 percent respectively. And the best result is obtained on Sep.
-13rd, with an CPU load of 10 percent.
-
-TC069
------
-With the block size changing from 1 kb to 512 kb, the memory write bandwidth
-tends to become larger first and then smaller within every run test, which
-rangs from 21.6 GB/s to 26.8 GB/s and then to 18.4 GB/s on average. Since the
-test id is one, it is that only the INT memory write bandwidth is tested. On
-the whole, when the block size is 8 kb and 16 kb, the memory write bandwidth
-look similar with a minimal BW of 23.0 GB/s and peak value of 28.6 GB/s. And
-then with the block size becoming larger, the memory write bandwidth tends to
-decrease. SLA sets to be 7 GB/s. The SLA value is used as a a reference, it has
-not been defined by OPNFV.
-
-TC070
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The network latency is measured by ping, and the results of the five test runs
-look similar with each other, and within these test runs, the maximum RTT can
-reach 53 ms and the average RTT is usually approx. 18 ms. The network latency
-tested on Sep. 14 shows that it has a peak latency of 53 ms. But on the whole,
-the average RTTs of the five runs keep flat and the network latency is
-relatively short.
-
-Memory utilization is measured by free, which can display amount of free and
-used memory in the system. The largest amount of used memory is 272 MiB on Sep
-14. In general, the mean used memory of the five test runs have the similar
-trend and the minimum memory used size is approx. 150 MiB, and the average
-used memory size is about 250 MiB. On the other hand, for the mean free memory,
-the five test runs have the similar trend, whose mean free memory change from
-218 MiB to 342 MiB, with an average value of approx. 38 MiB.
-
-Packet throughput and packet loss can be measured by pktgen, which is a tool
-in the network for generating traffic loads for network experiments. The mean
-packet throughput of the five test runs seem quite different, ranging from
-285.29 kpps to 297.76 kpps. The average number of flows in these tests is
-240000, and each run has a minimum number of flows of 2 and a maximum number
-of flows of 1.001 Mil. At the same time, the corresponding packet throughput
-differ between 250.6k and 398.9k with an average packet throughput between
-277.2 K and 318.4 K. In summary, the PPS results seem consistent. Within each
-test run of the five runs, when number of flows becomes larger, the packet
-throughput seems not larger at the same time.
-
-TC071
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The network latency is measured by ping, and the results of the five test runs
-look similar with each other. Within each test run, the maximum RTT is only 49
-ms and the average RTT is usually approx. 20 ms. On the whole, the average
-RTTs of the five runs keep stable and the network latency is relatively short.
-
-Cache utilization is measured by cachestat, which can display size of cache and
-buffer in the system. Cache utilization statistics are collected during UDP
-flows sent between the VMs using pktgen as packet generator tool.The largest
-cache size is 215 MiB in the four runs, and the smallest cache size is 95 MiB.
-On the whole, the average cache size of the five runs change a little and is
-about 200 MiB, except the one on Sep. 14th, the mean cache size is very small,
-which keeps 102 MiB. Meanwhile, the tread of the buffer size keep flat, since
-they have a minimum value of 7 MiB and a maximum value of 8 MiB, with an
-average value of about 7.8 MiB.
-
-Packet throughput can be measured by pktgen, which is a tool in the network for
-generating traffic loads for network experiments. The mean packet throughput of
-the four test runs seem quite different, ranging from 285.29 kpps to 297.76
-kpps. The average number of flows in these tests is 239.7k, and each run has a
-minimum number of flows of 2 and a maximum number of flows of 1.001 Mil. At the
-same time, the corresponding packet throughput differ between 227.3k and 398.9k
-with an average packet throughput between 277.2k and 318.4k. Within each test
-run of the five runs, when number of flows becomes larger, the packet
-throughput seems not larger in the meantime.
-
-TC072
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs
- between 0 ms and 49 ms with an average leatency of less than 22 ms. The PPS
-results are not as consistent as the RTT results, for the mean packet
-throughput of the five runs differ from 250.6 kpps to 398.9 kpps.
-
-Network utilization is measured by sar, that is system activity reporter, which
-can display the average statistics for the time since the system was started.
-Network utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The largest total number of packets
-transmitted per second look similar for four test runs, whose values change a
-lot from 10 pps to 399 kpps, except the one on Sep. 14th, whose total number
-of transmitted per second keep stable, that is 10 pps. Similarly, the total
-number of packets received per second look the same for four runs, except the
-one on Sep. 14th, whose value is only 10 pps.
-
-In some test runs when running with less than approx. 90000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. For the other test runs there is however no
-significant change to the PPS throughput when the number of flows are
-increased. In some test runs the PPS is also greater with 250000 flows
-compared to other test runs where the PPS result is less with only 2 flows.
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally differs a lot per test run.
-
-Detailed test results
----------------------
-The scenario was run on Intel POD6_ with:
-Joid
-OpenStack Mitaka
-Onos Goldeneye
-OpenVirtualSwitch 2.5.90
-OpenDayLight Beryllium
-
-Rationale for decisions
------------------------
-Pass
-
-Conclusions and recommendations
--------------------------------
-Tests were successfully executed and metrics collected.
-No SLA was verified. To be decided on in next release of OPNFV.
diff --git a/docs/release/results/os-onos-sfc-ha.rst b/docs/release/results/os-onos-sfc-ha.rst
deleted file mode 100644
index e52ae3d55..000000000
--- a/docs/release/results/os-onos-sfc-ha.rst
+++ /dev/null
@@ -1,517 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International
-.. License.
-.. http://creativecommons.org/licenses/by/4.0
-
-
-===============================
-Test Results for os-onos-sfc-ha
-===============================
-
-.. toctree::
- :maxdepth: 2
-
-
-fuel
-====
-
-.. _Grafana: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
-.. _POD2: https://wiki.opnfv.org/pharos?&#community_test_labs
-
-Overview of test results
-------------------------
-
-See Grafana_ for viewing test result metrics for each respective test case. It
-is possible to chose which specific scenarios to look at, and then to zoom in
-on the details of each run test scenario as well.
-
-All of the test case results below are based on 4 scenario test runs, each run
-on the Ericsson POD2_ or LF POD2_ between September 5 and 10 in 2016.
-
-TC002
------
-The round-trip-time (RTT) between 2 VMs on different blades is measured using
-ping. Most test run measurements result on average between 0.5 and 0.6 ms.
-A few runs start with a 1 - 1.5 ms RTT spike (This could be because of normal ARP
-handling). One test run has a greater RTT spike of 1.9 ms, which is the same
-one with the 0.7 ms average. The other runs have no similar spike at all.
-To be able to draw conclusions more runs should be made.
-SLA set to 10 ms. The SLA value is used as a reference, it has not
-been defined by OPNFV.
-
-TC005
------
-The IO read bandwidth looks similar between different dates, with an
-average between approx. 170 and 200 MB/s. Within each test run the results
-vary, with a minimum 2 MB/s and maximum 838 MB/s on the totality. Most runs
-have a minimum BW of 3 MB/s (two runs at 2 MB/s). The maximum BW varies more in
-absolute numbers between the dates, between 617 and 838 MB/s.
-SLA set to 400 MB/s. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC010
------
-The measurements for memory latency are similar between test dates and result
-in approx. 1.2 ns. The variations within each test run are similar, between
-1.215 and 1.219 ns. One exception is February 16, where the average is 1.222
-and varies between 1.22 and 1.28 ns.
-SLA set to 30 ns. The SLA value is used as a reference, it has not been defined
-by OPNFV.
-
-TC011
------
-Packet delay variation between 2 VMs on different blades is measured using
-Iperf3. On the first date the reported packet delay variation varies between
-0.0025 and 0.011 ms, with an average delay variation of 0.0067 ms.
-On the second date the delay variation varies between 0.002 and 0.006 ms, with
-an average delay variation of 0.004 ms.
-
-TC012
------
-Between test dates, the average measurements for memory bandwidth vary between
-17.4 and 17.9 GB/s. Within each test run the results vary more, with a minimal
-BW of 16.4 GB/s and maximum of 18.2 GB/s on the totality.
-SLA set to 15 GB/s. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC014
------
-The Unixbench processor test run results vary between scores 3080 and 3240,
-one result each date. The average score on the total is 3150.
-No SLA set.
-
-TC037
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-CPU utilization statistics are collected during UDP flows sent between the VMs
-using pktgen as packet generator tool. The average measurements for CPU
-utilization ratio vary between 1% to 2%. The peak of CPU utilization ratio
-appears around 7%.
-
-TC069
------
-Between test dates, the average measurements for memory bandwidth vary between
-15.5 and 25.4 GB/s. Within each test run the results vary more, with a minimal
-BW of 9.7 GB/s and maximum of 29.5 GB/s on the totality.
-SLA set to 6 GB/s. The SLA value is used as a reference, it has not been
-defined by OPNFV.
-
-TC070
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-Memory utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The average measurements for memory
-utilization vary between 225MB to 246MB. The peak of memory utilization appears
-around 340MB.
-
-TC071
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-Cache utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The average measurements for cache
-utilization vary between 205MB to 212MB.
-
-TC072
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs at
-approx. 15 ms. Some test runs show an increase with many flows, in the range
-towards 16 to 17 ms. One exception standing out is Feb. 15 where the average
-RTT is stable at approx. 13 ms. The PPS results are not as consistent as the
-RTT results.
-In some test runs when running with less than approx. 10000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. Around 20 percent decrease in the worst
-case. For the other test runs there is however no significant change to the PPS
-throughput when the number of flows are increased. In some test runs the PPS
-is also greater with 1000000 flows compared to other test runs where the PPS
-result is less with only 2 flows.
-
-The average PPS throughput in the different runs varies between 414000 and
-452000 PPS. The total amount of packets in each test run is approx. 7500000 to
-8200000 packets. One test run Feb. 15 sticks out with a PPS average of
-558000 and approx. 1100000 packets in total (same as the on mentioned earlier
-for RTT results).
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally range between 100 and 1000 per test run,
-but there are spikes in the range of 10000 lost packets as well, and even
-more in a rare cases.
-
-Network utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. Total number of packets received per
-second was average on 200 kpps and total number of packets transmitted per
-second was average on 600 kpps.
-
-Detailed test results
----------------------
-The scenario was run on Ericsson POD2_ and LF POD2_ with:
-Fuel 9.0
-OpenStack Mitaka
-Onos Goldeneye
-OpenVirtualSwitch 2.5.90
-OpenDayLight Beryllium
-
-Rationale for decisions
------------------------
-Pass
-
-Tests were successfully executed and metrics collected.
-No SLA was verified. To be decided on in next release of OPNFV.
-
-Conclusions and recommendations
--------------------------------
-The pktgen test configuration has a relatively large base effect on RTT in
-TC037 compared to TC002, where there is no background load at all. Approx.
-15 ms compared to approx. 0.5 ms, which is more than a 3000 percentage
-difference in RTT results.
-Especially RTT and throughput come out with better results than for instance
-the *fuel-os-nosdn-nofeature-ha* scenario does. The reason for this should
-probably be further analyzed and understood. Also of interest could be
-to make further analyzes to find patterns and reasons for lost traffic.
-Also of interest could be to see if there are continuous variations where
-some test cases stand out with better or worse results than the general test
-case.
-
-
-Joid
-=====
-
-.. _Grafana: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
-.. _POD6: https://wiki.opnfv.org/pharos?&#community_test_labs
-
-Overview of test results
-------------------------
-
-See Grafana_ for viewing test result metrics for each respective test case. It
-is possible to chose which specific scenarios to look at, and then to zoom in
-on the details of each run test scenario as well.
-
-All of the test case results below are based on 4 scenario test runs, each run
-on the Intel POD6_ between September 8 and 11 in 2016.
-
-TC002
------
-The round-trip-time (RTT) between 2 VMs on different blades is measured using
-ping. Most test run measurements result on average between 1.35 ms and 1.57 ms.
-Only one test run has reached greatest RTT spike of 2.58 ms. Meanwhile, the
-smallest network latency is 1.11 ms, which is obtained on Sep. 11st. In
-general, the average of network latency of the four test runs are between 1.35
-ms and 1.57 ms. SLA set to be 10 ms. The SLA value is used as a reference, it
-has not been defined by OPNFV.
-
-TC005
------
-The IO read bandwidth actually refers to the storage throughput, which is
-measured by fio and the greatest IO read bandwidth of the four runs is 175.4
-MB/s. The IO read bandwidth of the three runs looks similar, with an average
-between 43.7 and 56.3 MB/s, except one on Sep. 8, for its maximum storage
-throughput is only 107.9 MB/s. One of the runs has a minimum BW of 478 KM/s and
-other has a maximum BW of 168.6 MB/s. The SLA of read bandwidth sets to be
-400 MB/s, which is used as a reference, and it has not been defined by OPNFV.
-
-The results of storage IOPS for the four runs look similar with each other. The
-IO read times per second of the four test runs have an average value between
-978 per second and 1.20 K/s, and meanwhile, the minimum result is only 36 times
-per second.
-
-TC010
------
-The tool we use to measure memory read latency is lmbench, which is a series of
-micro benchmarks intended to measure basic operating system and hardware system
-metrics. The memory read latency of the four runs is between 1.164 ns and 1.244
-ns on average. The variations within each test run are quite different, some
-vary from a large range and others have a small change. For example, the
-largest change is on September 10, the memory read latency of which is ranging
-from 1.128 ns to 1.381 ns. However, the results on September 11 change very
-little. The SLA sets to be 30 ns. The SLA value is used as a reference, it has
-not been defined by OPNFV.
-
-TC011
------
-Iperf3 is a tool for evaluating the packet delay variation between 2 VMs on
-different blades. The reported packet delay variations of the four test runs
-differ from each other. In general, the packet delay of two runs look similar,
-for they both stay stable within each run. And the mean packet delay of them
-are 0.0772 ms and 0.0788 ms respectively. Of the four runs, the fourth has the
-worst result, because the packet delay reaches 0.0838 ms. The rest one has a
-large wide range from 0.0666 ms to 0.0798 ms. The SLA value sets to be 10 ms.
-The SLA value is used as a reference, it has not been defined by OPNFV.
-
-TC012
------
-Lmbench is also used to measure the memory read and write bandwidth, in which
-we use bw_mem to obtain the results. Among the four test runs, the trend of the
-memory bandwidth almost look similar, which all have a large wide range, and
-the minimum and maximum results are 9.02 GB/s and 18.14 GB/s. Here SLA set to
-be 15 GB/s. The SLA value is used as a reference, it has not been defined by
-OPNFV.
-
-TC014
------
-The Unixbench is used to evaluate the IaaS processing speed with regards to
-score of single cpu running and parallel running. It can be seen from the
-dashboard that the processing test results vary from scores 3395 to 3475, and
-there is only one result one date. No SLA set.
-
-TC037
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The mean packet throughput of the four test runs is between 362.1 kpps and
-363.5 kpps, of which the result of the third run is the highest. The RTT
-results of all the test runs keep flat at approx. 17 ms. It is obvious that the
-PPS results are not as consistent as the RTT results.
-
-The No. flows of the four test runs are 240 k on average and the PPS results
-look a little waved since the largest packet throughput is 418.1 kpps and the
-minimum throughput is 326.5 kpps respectively.
-
-There are no errors of packets received in the four runs, but there are still
-lost packets in all the test runs. The RTT values obtained by ping of the four
-runs have the similar average vaue, that is approx. 17 ms, of which the worst
-RTT is 39 ms on Sep. 11st.
-
-CPU load is measured by mpstat, and CPU load of the four test runs seem a
-little similar, since the minimum value and the peak of CPU load is between 0
-percent and nine percent respectively. And the best result is obtained on Sep.
-10, with an CPU load of nine percent.
-
-TC069
------
-With the block size changing from 1 kb to 512 kb, the memory write bandwidth
-tends to become larger first and then smaller within every run test, which
-rangs from 25.9 GB/s to 26.6 GB/s and then to 18.1 GB/s on average. Since the
-test id is one, it is that only the INT memory write bandwidth is tested. On
-the whole, when the block size is from 2 kb to 16 kb, the memory write
-bandwidth look similar with a minimal BW of 22.1 GB/s and peak value of 28.6
-GB/s. And then with the block size becoming larger, the memory write bandwidth
-tends to decrease. SLA sets to be 7 GB/s. The SLA value is used as a reference,
-it has not been defined by OPNFV.
-
-TC070
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The network latency is measured by ping, and the results of the four test runs
-look similar with each other, and within these test runs, the maximum RTT can
-reach 39 ms and the average RTT is usually approx. 17 ms. The network latency
-tested on Sep. 11 shows that it has a peak latency of 39 ms. But on the whole,
-the average RTTs of the five runs keep flat and the network latency is
-relatively short.
-
-Memory utilization is measured by free, which can display amount of free and
-used memory in the system. The largest amount of used memory is 270 MiB on the
-first two runs. In general, the mean used memory of two test runs have very
-large memory utilization, which can reach 264 MiB on average. And the other two
-runs have a large wide range of memory usage with the minimum value of 150 MiB
-and the maximum value of 270 MiB. On the other hand, for the mean free memory,
-the four test runs have the similar trend with that of the mean used memory.
-In general, the mean free memory change from 220 MiB to 342 MiB.
-
-Packet throughput and packet loss can be measured by pktgen, which is a tool
-in the network for generating traffic loads for network experiments. The mean
-packet throughput of the four test runs seem quite different, ranging from
-326.5 kpps to 418.1 kpps. The average number of flows in these tests is
-240000, and each run has a minimum number of flows of 2 and a maximum number
-of flows of 1.001 Mil. At the same time, the corresponding packet throughput
-differ between 326.5 kpps and 418.1 kpps with an average packet throughput between
-361.7 kpps and 363.5 kpps. In summary, the PPS results seem consistent. Within each
-test run of the four runs, when number of flows becomes larger, the packet
-throughput seems not larger at the same time.
-
-TC071
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The network latency is measured by ping, and the results of the four test runs
-look similar with each other. Within each test run, the maximum RTT is only 47
-ms and the average RTT is usually approx. 15 ms. On the whole, the average
-RTTs of the four runs keep stable and the network latency is relatively small.
-
-Cache utilization is measured by cachestat, which can display size of cache and
-buffer in the system. Cache utilization statistics are collected during UDP
-flows sent between the VMs using pktgen as packet generator tool. The largest
-cache size is 214 MiB, which is same for the four runs, and the smallest cache
-size is 94 MiB. On the whole, the average cache size of the four runs look the
-same and is between 198 MiB and 207 MiB. Meanwhile, the tread of the buffer
-size keep flat, since they have a minimum value of 7 MiB and a maximum value of
-8 MiB, with an average value of about 7.9 MiB.
-
-Packet throughput can be measured by pktgen, which is a tool in the network for
-generating traffic loads for network experiments. The mean packet throughput of
-the four test runs seem quite the same, which is approx. 363 kpps. The average
-number of flows in these tests is 240k, and each run has a minimum number of
-flows of 2 and a maximum number of flows of 1.001 Mil. At the same time, the
-corresponding packet throughput differ between 327 kpps and 418 kpps with an
-average packet throughput of about 363 kpps. Within each test run of the four
-runs, when number of flows becomes larger, the packet throughput seems not
-larger in the meantime.
-
-TC072
------
-The amount of packets per second (PPS) and round trip times (RTT) between 2 VMs
-on different blades are measured when increasing the amount of UDP flows sent
-between the VMs using pktgen as packet generator tool.
-
-Round trip times and packet throughput between VMs can typically be affected by
-the amount of flows set up and result in higher RTT and less PPS throughput.
-
-The RTT results are similar throughout the different test dates and runs
-between 0 ms and 47 ms with an average leatency of less than 16 ms. The PPS
-results are not as consistent as the RTT results, for the mean packet
-throughput of the four runs differ from 361.7 kpps to 365.0 kpps.
-
-Network utilization is measured by sar, that is system activity reporter, which
-can display the average statistics for the time since the system was started.
-Network utilization statistics are collected during UDP flows sent between the
-VMs using pktgen as packet generator tool. The largest total number of packets
-transmitted per second look similar for two test runs, whose values change a
-lot from 10 pps to 432 kpps. While results of the other test runs seem the same
-and keep stable with the average number of packets transmitted per second of 10
-pps. However, the total number of packets received per second of the four runs
-look similar, which have a large wide range of 2 pps to 657 kpps.
-
-In some test runs when running with less than approx. 250000 flows the PPS
-throughput is normally flatter compared to when running with more flows, after
-which the PPS throughput decreases. For the other test runs there is however no
-significant change to the PPS throughput when the number of flows are
-increased. In some test runs the PPS is also greater with 250000 flows
-compared to other test runs where the PPS result is less with only 2 flows.
-
-There are lost packets reported in most of the test runs. There is no observed
-correlation between the amount of flows and the amount of lost packets.
-The lost amount of packets normally differs a lot per test run.
-
-Detailed test results
----------------------
-The scenario was run on Intel POD6_ with:
-Joid
-OpenStack Mitaka
-Onos Goldeneye
-OpenVirtualSwitch 2.5.90
-OpenDayLight Beryllium
-
-Rationale for decisions
------------------------
-Pass
-
-Conclusions and recommendations
--------------------------------
-Tests were successfully executed and metrics collected.
-No SLA was verified. To be decided on in next release of OPNFV.
-
diff --git a/docs/release/results/overview.rst b/docs/release/results/overview.rst
index b4a050545..9fd74797c 100644
--- a/docs/release/results/overview.rst
+++ b/docs/release/results/overview.rst
@@ -42,55 +42,31 @@ environment, features or test framework.
The list of scenarios supported by each installer can be described as follows:
-+-------------------------+---------+---------+---------+---------+
-| Scenario | Apex | Compass | Fuel | Joid |
-+=========================+=========+=========+=========+=========+
-| os-nosdn-nofeature-noha | | | X | X |
-+-------------------------+---------+---------+---------+---------+
-| os-nosdn-nofeature-ha | X | X | X | X |
-+-------------------------+---------+---------+---------+---------+
-| os-odl_l2-nofeature-ha | X | X | X | X |
-+-------------------------+---------+---------+---------+---------+
-| os-odl_l2-nofeature-noha| | | X | |
-+-------------------------+---------+---------+---------+---------+
-| os-odl_l3-nofeature-ha | X | X | X | |
-+-------------------------+---------+---------+---------+---------+
-| os-odl_l3-nofeature-noha| | | X | |
-+-------------------------+---------+---------+---------+---------+
-| os-onos-sfc-ha | X | X | X | X |
-+-------------------------+---------+---------+---------+---------+
-| os-onos-sfc-noha | | | X | |
-+-------------------------+---------+---------+---------+---------+
-| os-onos-nofeature-ha | X | X | X | X |
-+-------------------------+---------+---------+---------+---------+
-| os-onos-nofeature-noha | | | X | |
-+-------------------------+---------+---------+---------+---------+
-| os-odl_l2-sfc-ha | | | X | |
-+-------------------------+---------+---------+---------+---------+
-| os-odl_l2-sfc-noha | X | X | X | |
-+-------------------------+---------+---------+---------+---------+
-| os-odl_l2-bgpvpn-ha | X | | X | |
-+-------------------------+---------+---------+---------+---------+
-| os-odl_l2-bgpvpn-noha | | X | X | |
-+-------------------------+---------+---------+---------+---------+
-| os-nosdn-kvm-ha | | | X | |
-+-------------------------+---------+---------+---------+---------+
-| os-nosdn-kvm-noha | | X | X | |
-+-------------------------+---------+---------+---------+---------+
-| os-nosdn-ovs-ha | | | X | |
-+-------------------------+---------+---------+---------+---------+
-| os-nosdn-ovs-noha | X | | X | |
-+-------------------------+---------+---------+---------+---------+
-| os-ocl-nofeature-ha | | | | |
-+-------------------------+---------+---------+---------+---------+
-| os-nosdn-lxd-ha | | | | X |
-+-------------------------+---------+---------+---------+---------+
-| os-nosdn-lxd-noha | | | | X |
-+-------------------------+---------+---------+---------+---------+
-| os-odl_l2-fdio-noha | X | | | |
-+-------------------------+---------+---------+---------+---------+
-| os-odl_l2-moon-ha | | X | | |
-+-------------------------+---------+---------+---------+---------+
++-------------------------+------+---------+----------+------+------+-------+
+| Scenario | Apex | Compass | Fuel-arm | Fuel | Joid | Daisy |
++=========================+======+=========+==========+======+======+=======+
+| os-nosdn-nofeature-noha | X | | | | X | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-nosdn-nofeature-ha | X | | X | X | X | X |
++-------------------------+------+---------+----------+------+------+-------+
+| os-nosdn-bar-noha | X | | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-nosdn-bar-ha | X | | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-odl-bgpvpn-ha | X | | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-nosdn-calipso-noha | X | | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-nosdn-kvm-ha | | X | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-odl_l3-nofeature-ha | | X | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-odl-sfc-ha | | X | | | | |
++-------------------------+------+---------+----------+------+------+-------+
+| os-odl-nofeature-ha | | | | X | | X |
++-------------------------+------+---------+----------+------+------+-------+
+| os-nosdn-ovs-ha | | | | X | | |
++-------------------------+------+---------+----------+------+------+-------+
To qualify for release, the scenarios must have deployed and been successfully
tested in four consecutive installations to establish stability of deployment
@@ -103,4 +79,4 @@ References
* IEEE Std 829-2008. "Standard for Software and System Test Documentation".
-* OPNFV Colorado release note for Yardstick.
+* OPNFV Fraser release note for Yardstick.
diff --git a/docs/release/results/results.rst b/docs/release/results/results.rst
index 04c6b9f87..0ed92f867 100644
--- a/docs/release/results/results.rst
+++ b/docs/release/results/results.rst
@@ -2,13 +2,16 @@
.. License.
.. http://creativecommons.org/licenses/by/4.0
-Results listed by scenario
+Results listed by test cases
==========================
-The following sections describe the yardstick results as evaluated for the
-Colorado release scenario validation runs. Each section describes the
-determined state of the specific scenario as deployed in the Colorado
-release process.
+.. _TOM: https://wiki.opnfv.org/display/testing/R+post-processing+of+the+Yardstick+results
+
+
+The following sections describe the yardstick test case results as evaluated
+for the OPNFV Fraser release scenario validation runs. Each section describes
+the determined state of the specific test case as executed in the Fraser release
+process. All test date are analyzed using TOM_ tool.
Scenario Results
================
@@ -16,24 +19,27 @@ Scenario Results
.. _Dashboard: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
.. _Jenkins: https://build.opnfv.org/ci/view/yardstick/
+
The following documents contain results of Yardstick test cases executed on
-OPNFV labs, triggered by OPNFV CI pipeline, documented per scenario.
+OPNFV labs, triggered by OPNFV CI pipeline, documented per test case.
+For hardware details of OPNFV labs, please visit: https://wiki.opnfv.org/display/pharos/Community+Labs
.. toctree::
:maxdepth: 1
- os-nosdn-nofeature-ha.rst
- os-nosdn-nofeature-noha.rst
- os-odl_l2-nofeature-ha.rst
- os-odl_l2-bgpvpn-ha.rst
- os-odl_l2-sfc-ha.rst
- os-nosdn-kvm-ha.rst
- os-onos-nofeature-ha.rst
- os-onos-sfc-ha.rst
+ tc002-network-latency.rst
+ tc010-memory-read-latency.rst
+ tc011-packet-delay-variation.rst
+ tc012-memory-read-write-bandwidth.rst
+ tc014-cpu-processing-speed.rst
+ tc069-memory-write-bandwidth.rst
+ tc082-context-switches-under-load.rst
+ tc083-network-throughput-between-vm.rst
Test results of executed tests are avilable in Dashboard_ and logs in Jenkins_.
+Test results for Fraser release are collected from April 10, 2018 to May 13, 2018.
Feature Test Results
====================
diff --git a/docs/release/results/tc002-network-latency.rst b/docs/release/results/tc002-network-latency.rst
new file mode 100644
index 000000000..064983bec
--- /dev/null
+++ b/docs/release/results/tc002-network-latency.rst
@@ -0,0 +1,525 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+
+
+======================================
+Test results for TC002 network latency
+======================================
+
+.. toctree::
+ :maxdepth: 2
+
+
+Overview of test case
+=====================
+
+TC002 verifies that network latency is within acceptable boundaries when packets travel between hosts located on same or different compute blades.
+Ping packets (ICMP protocol's mandatory ECHO_REQUEST datagram) are sent from host VM to target VM(s) to elicit ICMP ECHO_RESPONSE.
+
+Metric: RTT (Round Trip Time)
+Unit: ms
+
+
+Euphrates release
+-----------------
+
+Test results per scenario and pod (lower is better):
+
+{
+
+ "os-nosdn-ovs_dpdk-ha:huawei-pod2:compass": [0.214],
+
+ "os-odl_l2-moon-ha:huawei-pod2:compass": [0.309],
+
+ "os-nosdn-ovs_dpdk-noha:huawei-virtual3:compass": [0.3145],
+
+ "os-nosdn-ovs-ha:lf-pod2:fuel": [0.3585],
+
+ "os-odl_l3-nofeature-ha:huawei-pod2:compass": [0.3765],
+
+ "os-nosdn-ovs_dpdk-noha:huawei-virtual4:compass": [0.403],
+
+ "os-odl-sfc-ha:huawei-pod2:compass": [0.413],
+
+ "os-nosdn-ovs-ha:ericsson-pod1:fuel": [0.494],
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [0.5715],
+
+ "os-nosdn-nofeature-noha:lf-pod1:apex": [0.5785],
+
+ "os-odl-sfc-ha:lf-pod1:apex": [0.617],
+
+ "os-odl-nofeature-ha:lf-pod1:apex": [0.62],
+
+ "os-nosdn-bar-noha:lf-pod1:apex": [0.632],
+
+ "os-odl-nofeature-noha:lf-pod1:apex": [0.635],
+
+ "os-odl-bgpvpn-ha:lf-pod1:apex": [0.658],
+
+ "os-odl-sfc-noha:lf-pod1:apex": [0.663],
+
+ "os-nosdn-bar-ha:lf-pod1:apex": [0.668],
+
+ "os-ovn-nofeature-noha:lf-pod1:apex": [0.668],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [0.6815],
+
+ "os-nosdn-kvm-ha:huawei-pod2:compass": [0.7005],
+
+ "os-nosdn-bar-ha:huawei-pod2:compass": [0.778],
+
+ "os-nosdn-ovs-noha:ericsson-virtual4:fuel": [0.7825],
+
+ "os-nosdn-ovs-noha:ericsson-virtual2:fuel": [0.7885],
+
+ "os-nosdn-nofeature-ha:flex-pod2:apex": [0.795],
+
+ "os-nosdn-ovs-noha:ericsson-virtual1:fuel": [0.8045],
+
+ "os-nosdn-nofeature-noha:huawei-pod12:joid": [0.8335],
+
+ "os-nosdn-ovs-noha:ericsson-virtual3:fuel": [0.8755],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [0.8855],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-virtual3:compass": [0.8895],
+
+ "os-nosdn-openbaton-ha:huawei-pod12:joid": [0.901],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-virtual4:compass": [0.956],
+
+ "os-nosdn-lxd-noha:intel-pod5:joid": [1.131],
+
+ "os-odl_l2-moon-noha:huawei-virtual4:compass": [1.173],
+
+ "os-odl-sfc-ha:huawei-virtual8:compass": [1.2015],
+
+ "os-odl_l2-moon-noha:huawei-virtual3:compass": [1.204],
+
+ "os-nosdn-lxd-ha:intel-pod5:joid": [1.2245],
+
+ "os-odl-nofeature-ha:lf-pod2:fuel": [1.2285],
+
+ "os-odl-sfc-noha:huawei-virtual4:compass": [1.3055],
+
+ "os-nosdn-nofeature-noha:huawei-virtual4:compass": [1.309],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual4:compass": [1.313],
+
+ "os-nosdn-nofeature-noha:huawei-virtual8:compass": [1.319],
+
+ "os-odl-nofeature-ha:ericsson-pod1:fuel": [1.3425],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual3:compass": [1.3475],
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [1.348],
+
+ "os-nosdn-kvm-noha:huawei-virtual4:compass": [1.432],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual9:compass": [1.442],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [1.4505],
+
+ "os-nosdn-nofeature-ha:arm-pod5:fuel": [1.497],
+
+ "os-odl-sfc-noha:huawei-virtual3:compass": [1.504],
+
+ "os-odl-nofeature-ha:arm-pod5:fuel": [1.519],
+
+ "os-nosdn-nofeature-noha:intel-pod5:joid": [1.5415],
+
+ "os-nosdn-nofeature-noha:huawei-virtual3:compass": [1.5785],
+
+ "os-nosdn-nofeature-ha:intel-pod5:joid": [1.604],
+
+ "os-nosdn-kvm-noha:huawei-virtual3:compass": [1.61],
+
+ "os-nosdn-nofeature-noha:intel-pod18:joid": [1.633],
+
+ "os-nosdn-openbaton-ha:intel-pod18:joid": [1.6485],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual2:compass": [1.7085],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [1.71],
+
+ "os-nosdn-nofeature-ha:huawei-virtual2:compass": [1.7955],
+
+ "os-odl-nofeature-ha:arm-pod6:fuel": [1.838],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual4:compass": [1.88],
+
+ "os-odl_l2-moon-ha:huawei-virtual3:compass": [1.8975],
+
+ "os-nosdn-kvm-noha:huawei-virtual8:compass": [1.923],
+
+ "os-odl_l2-moon-ha:huawei-virtual4:compass": [1.944],
+
+ "os-odl-sfc-ha:huawei-virtual3:compass": [1.968],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual3:compass": [1.986],
+
+ "os-nosdn-bar-ha:huawei-virtual4:compass": [2.0415],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [2.071],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [2.0855],
+
+ "os-odl-sfc-ha:huawei-virtual4:compass": [2.1085],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [2.1135],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual3:fuel": [2.234],
+
+ "os-nosdn-nofeature-ha:huawei-virtual9:compass": [2.294],
+
+ "os-nosdn-kvm-ha:huawei-virtual3:compass": [2.304],
+
+ "os-nosdn-bar-ha:huawei-virtual3:compass": [2.378],
+
+ "os-nosdn-kvm-ha:huawei-virtual4:compass": [2.397],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [2.472],
+
+ "os-nosdn-nofeature-noha:huawei-virtual1:compass": [2.603],
+
+ "os-nosdn-nofeature-noha:huawei-virtual2:compass": [2.635],
+
+ "os-odl-nofeature-noha:ericsson-virtual3:fuel": [2.9055],
+
+ "os-odl-nofeature-noha:ericsson-virtual2:fuel": [3.1295],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual2:fuel": [3.337],
+
+ "os-odl-nofeature-noha:ericsson-virtual4:fuel": [3.634],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual1:fuel": [3.875],
+
+ "os-odl-nofeature-noha:ericsson-virtual1:fuel": [3.9655],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual4:fuel": [3.9795]
+
+}
+
+
+The influence of the scenario
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc002_scenario.png
+ :width: 800px
+ :alt: TC002 influence of scenario
+
+{
+
+ "os-odl_l2-moon-ha": [0.3415],
+
+ "os-nosdn-ovs-ha": [0.3625],
+
+ "os-nosdn-ovs_dpdk-noha": [0.378],
+
+ "os-nosdn-ovs_dpdk-ha": [0.5265],
+
+ "os-nosdn-bar-noha": [0.632],
+
+ "os-odl-bgpvpn-ha": [0.658],
+
+ "os-ovn-nofeature-noha": [0.668],
+
+ "os-odl_l3-nofeature-ha": [0.8545],
+
+ "os-nosdn-ovs-noha": [0.8575],
+
+ "os-nosdn-bar-ha": [0.903],
+
+ "os-odl-sfc-ha": [1.127],
+
+ "os-nosdn-lxd-noha": [1.131],
+
+ "os-nosdn-nofeature-ha": [1.152],
+
+ "os-odl_l2-moon-noha": [1.1825],
+
+ "os-nosdn-lxd-ha": [1.2245],
+
+ "os-odl_l3-nofeature-noha": [1.337],
+
+ "os-odl-nofeature-ha": [1.352],
+
+ "os-odl-sfc-noha": [1.4255],
+
+ "os-nosdn-kvm-noha": [1.5045],
+
+ "os-nosdn-openbaton-ha": [1.5665],
+
+ "os-nosdn-nofeature-noha": [1.729],
+
+ "os-nosdn-kvm-ha": [1.7745],
+
+ "os-odl-nofeature-noha": [3.106]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc002_pod.png
+ :width: 800px
+ :alt: TC002 influence of the POD
+
+{
+
+ "huawei-pod2": [0.3925],
+
+ "lf-pod2": [0.5315],
+
+ "lf-pod1": [0.62],
+
+ "flex-pod2": [0.795],
+
+ "huawei-pod12": [0.87],
+
+ "intel-pod5": [1.25],
+
+ "ericsson-virtual3": [1.2655],
+
+ "ericsson-pod1": [1.372],
+
+ "arm-pod5": [1.518],
+
+ "huawei-virtual4": [1.5355],
+
+ "huawei-virtual3": [1.606],
+
+ "intel-pod18": [1.6575],
+
+ "huawei-virtual8": [1.709],
+
+ "huawei-virtual2": [1.872],
+
+ "arm-pod6": [1.895],
+
+ "huawei-virtual9": [2.0745],
+
+ "huawei-virtual1": [2.495],
+
+ "ericsson-virtual2": [2.7895],
+
+ "ericsson-virtual4": [3.768],
+
+ "ericsson-virtual1": [3.8035]
+
+}
+
+
+Fraser release
+--------------
+
+Test results per scenario and pod (lower is better):
+
+{
+
+ "os-odl_l3-nofeature-ha:huawei-pod2:compass": [0.42],
+
+ "os-odl-sfc-ha:huawei-pod2:compass": [0.557],
+
+ "os-nosdn-ovs-ha:ericsson-pod1:fuel": [0.5765],
+
+ "os-nosdn-kvm-ha:huawei-pod2:compass": [0.582],
+
+ "os-odl-bgpvpn-ha:lf-pod1:apex": [0.678],
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [0.7075],
+
+ "os-nosdn-calipso-noha:lf-pod1:apex": [0.713],
+
+ "os-nosdn-nofeature-noha:lf-pod1:apex": [0.7155],
+
+ "os-nosdn-bar-ha:lf-pod1:apex": [0.732],
+
+ "os-nosdn-bar-noha:lf-pod1:apex": [0.7415],
+
+ "os-odl-nofeature-noha:lf-pod1:apex": [0.7565],
+
+ "os-nosdn-ovs-ha:arm-pod6:fuel": [0.8015],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [0.908],
+
+ "os-odl-nofeature-ha:ericsson-pod1:fuel": [0.9165],
+
+ "os-nosdn-bar-ha:huawei-pod2:compass": [0.969],
+
+ "os-nosdn-ovs-noha:ericsson-virtual2:fuel": [0.9765],
+
+ "os-nosdn-nofeature-noha:huawei-pod12:joid": [1.0245],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [1.0495],
+
+ "os-odl-sfc-noha:huawei-virtual4:compass": [1.1645],
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [1.206],
+
+ "os-odl-sfc-noha:huawei-virtual3:compass": [1.236],
+
+ "os-nosdn-ovs-noha:ericsson-virtual4:fuel": [1.241],
+
+ "os-nosdn-nofeature-ha:zte-pod2:daisy": [1.2805],
+
+ "os-odl-nofeature-ha:lf-pod2:fuel": [1.286],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual3:compass": [1.299],
+
+ "os-odl-sfc-ha:huawei-virtual4:compass": [1.305],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual4:compass": [1.309],
+
+ "os-nosdn-kvm-noha:huawei-virtual4:compass": [1.314],
+
+ "os-nosdn-nofeature-noha:huawei-virtual4:compass": [1.431],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [1.457],
+
+ "os-odl-nofeature-ha:zte-pod2:daisy": [1.517],
+
+ "os-nosdn-kvm-noha:huawei-virtual3:compass": [1.576],
+
+ "os-nosdn-nofeature-noha:huawei-virtual3:compass": [1.592],
+
+ "os-odl-nofeature-ha:arm-pod5:fuel": [1.714],
+
+ "os-nosdn-nofeature-noha:intel-pod18:joid": [1.809],
+
+ "os-nosdn-bar-noha:huawei-virtual4:compass": [1.81],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [1.8505],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [1.8895],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [1.909],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual4:compass": [1.925],
+
+ "os-nosdn-nofeature-noha:huawei-virtual2:compass": [1.964],
+
+ "os-nosdn-openbaton-ha:intel-pod18:joid": [1.9755],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [1.9765],
+
+ "os-nosdn-bar-noha:huawei-virtual3:compass": [1.9915],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual3:compass": [1.9925],
+
+ "os-nosdn-kvm-ha:huawei-virtual4:compass": [2.0265],
+
+ "os-odl-nofeature-ha:arm-pod6:fuel": [2.106],
+
+ "os-odl-sfc-ha:huawei-virtual3:compass": [2.124],
+
+ "os-nosdn-kvm-ha:huawei-virtual3:compass": [2.185],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [2.281],
+
+ "os-nosdn-bar-ha:huawei-virtual4:compass": [2.432],
+
+ "os-odl-nofeature-noha:ericsson-virtual4:fuel": [2.483],
+
+ "os-nosdn-bar-ha:huawei-virtual3:compass": [2.524],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual3:fuel": [3.9175],
+
+ "os-odl-nofeature-noha:ericsson-virtual2:fuel": [4.338],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual2:fuel": [4.641]
+
+}
+
+
+The influence of the scenario
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc002_scenario_fraser.png
+ :width: 800px
+ :alt: TC002 influence of scenario
+
+{
+
+ "os-odl-bgpvpn-ha": [0.678],
+
+ "os-nosdn-calipso-noha": [0.713],
+
+ "os-nosdn-ovs-ha": [0.7245],
+
+ "os-odl_l3-nofeature-ha": [0.7435],
+
+ "os-odl-sfc-ha": [0.796],
+
+ "os-nosdn-kvm-ha": [1.059],
+
+ "os-nosdn-bar-ha": [1.083],
+
+ "os-nosdn-ovs-noha": [1.09],
+
+ "os-odl-sfc-noha": [1.196],
+
+ "os-nosdn-nofeature-noha": [1.26],
+
+ "os-nosdn-nofeature-ha": [1.291],
+
+ "os-odl_l3-nofeature-noha": [1.308],
+
+ "os-nosdn-bar-noha": [1.4125],
+
+ "os-nosdn-kvm-noha": [1.4475],
+
+ "os-odl-nofeature-ha": [1.508],
+
+ "os-odl-nofeature-noha": [1.914],
+
+ "os-nosdn-openbaton-ha": [1.9755]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc002_pod_fraser.png
+ :width: 800px
+ :alt: TC002 influence of the POD
+
+{
+
+ "huawei-pod2": [0.677],
+
+ "lf-pod1": [0.725],
+
+ "ericsson-pod1": [0.9165],
+
+ "huawei-pod12": [1.0465],
+
+ "lf-pod2": [1.2325],
+
+ "zte-pod2": [1.395],
+
+ "ericsson-virtual4": [1.582],
+
+ "huawei-virtual4": [1.697],
+
+ "arm-pod5": [1.714],
+
+ "huawei-virtual3": [1.716],
+
+ "intel-pod18": [1.856],
+
+ "huawei-virtual2": [1.964],
+
+ "huawei-virtual1": [1.9765],
+
+ "arm-pod6": [2.209],
+
+ "ericsson-virtual3": [3.9175],
+
+ "ericsson-virtual2": [4.004]
+
+}
diff --git a/docs/release/results/tc010-memory-read-latency.rst b/docs/release/results/tc010-memory-read-latency.rst
new file mode 100644
index 000000000..81559d647
--- /dev/null
+++ b/docs/release/results/tc010-memory-read-latency.rst
@@ -0,0 +1,510 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+
+
+==========================================
+Test results for TC010 memory read latency
+==========================================
+
+.. toctree::
+ :maxdepth: 2
+
+
+Overview of test case
+=====================
+
+TC010 measures the memory read latency for varying memory sizes and strides.
+The test results shown below are for memory size of 16MB.
+
+Metric: Memory read latency
+Unit: ns
+
+
+Euphrates release
+-----------------
+
+Test results per scenario and pod (lower is better):
+
+{
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [5.3165],
+
+ "os-nosdn-nofeature-ha:flex-pod2:apex": [5.908],
+
+ "os-nosdn-ovs-noha:ericsson-virtual1:fuel": [6.412],
+
+ "os-nosdn-nofeature-noha:intel-pod18:joid": [6.545],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [6.592],
+
+ "os-nosdn-nofeature-noha:intel-pod5:joid": [6.5975],
+
+ "os-nosdn-ovs-ha:ericsson-pod1:fuel": [6.7675],
+
+ "os-odl-nofeature-ha:ericsson-pod1:fuel": [6.7675],
+
+ "os-nosdn-openbaton-ha:intel-pod18:joid": [6.7945],
+
+ "os-nosdn-nofeature-ha:intel-pod5:joid": [6.839],
+
+ "os-nosdn-ovs-noha:ericsson-virtual4:fuel": [6.9695],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual4:fuel": [7.123],
+
+ "os-odl-nofeature-noha:ericsson-virtual4:fuel": [7.289],
+
+ "os-nosdn-ovs-noha:ericsson-virtual2:fuel": [7.4315],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual2:fuel": [7.9],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-pod2:compass": [8.178],
+
+ "os-nosdn-ovs_dpdk-noha:huawei-virtual3:compass": [8.616],
+
+ "os-nosdn-ovs_dpdk-noha:huawei-virtual4:compass": [8.646],
+
+ "os-odl_l3-nofeature-ha:huawei-pod2:compass": [8.8615],
+
+ "os-odl-sfc-ha:huawei-pod2:compass": [8.87],
+
+ "os-nosdn-bar-ha:huawei-pod2:compass": [8.877],
+
+ "os-odl_l2-moon-ha:huawei-pod2:compass": [8.892],
+
+ "os-nosdn-ovs-noha:ericsson-virtual3:fuel": [8.898],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [8.952],
+
+ "os-nosdn-kvm-ha:huawei-pod2:compass": [8.9745],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-virtual3:compass": [9.0375],
+
+ "os-nosdn-openbaton-ha:huawei-pod12:joid": [9.083],
+
+ "os-nosdn-nofeature-noha:huawei-pod12:joid": [9.09],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [9.094],
+
+ "os-odl_l2-moon-noha:huawei-virtual4:compass": [9.293],
+
+ "os-odl_l2-moon-noha:huawei-virtual3:compass": [9.3525],
+
+ "os-odl-sfc-noha:huawei-virtual4:compass": [9.477],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual3:compass": [9.5445],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual4:compass": [9.5575],
+
+ "os-nosdn-nofeature-noha:huawei-virtual4:compass": [9.6435],
+
+ "os-nosdn-nofeature-noha:huawei-virtual1:compass": [9.68],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-virtual4:compass": [9.728],
+
+ "os-nosdn-nofeature-noha:huawei-virtual3:compass": [9.751],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual3:fuel": [9.8645],
+
+ "os-nosdn-kvm-noha:huawei-virtual3:compass": [9.969],
+
+ "os-odl-sfc-noha:huawei-virtual3:compass": [10.029],
+
+ "os-nosdn-kvm-noha:huawei-virtual4:compass": [10.088],
+
+ "os-odl-nofeature-noha:ericsson-virtual2:fuel": [10.2985],
+
+ "os-nosdn-nofeature-ha:huawei-virtual9:compass": [10.318],
+
+ "os-nosdn-nofeature-noha:huawei-virtual2:compass": [10.3215],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [10.617],
+
+ "os-odl-nofeature-noha:ericsson-virtual3:fuel": [10.762],
+
+ "os-nosdn-bar-ha:huawei-virtual3:compass": [10.7715],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [10.866],
+
+ "os-odl-sfc-ha:huawei-virtual3:compass": [10.871],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual3:compass": [11.1605],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [11.227],
+
+ "os-nosdn-bar-ha:huawei-virtual4:compass": [11.348],
+
+ "os-odl-sfc-ha:huawei-virtual4:compass": [11.453],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual2:compass": [11.571],
+
+ "os-odl_l2-moon-ha:huawei-virtual3:compass": [11.5925],
+
+ "os-nosdn-nofeature-ha:huawei-virtual2:compass": [11.689],
+
+ "os-odl_l2-moon-ha:huawei-virtual4:compass": [11.8695],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual4:compass": [12.199],
+
+ "os-nosdn-kvm-ha:huawei-virtual4:compass": [12.433],
+
+ "os-nosdn-kvm-ha:huawei-virtual3:compass": [12.713],
+
+ "os-nosdn-ovs-ha:lf-pod2:fuel": [15.328],
+
+ "os-odl-nofeature-ha:lf-pod1:apex": [15.4265],
+
+ "os-odl-nofeature-noha:lf-pod1:apex": [15.428],
+
+ "os-ovn-nofeature-noha:lf-pod1:apex": [15.545],
+
+ "os-nosdn-nofeature-noha:lf-pod1:apex": [15.55],
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [15.6395],
+
+ "os-odl-sfc-noha:lf-pod1:apex": [15.696],
+
+ "os-odl-sfc-ha:lf-pod1:apex": [15.774],
+
+ "os-nosdn-bar-ha:lf-pod1:apex": [16.6455],
+
+ "os-nosdn-bar-noha:lf-pod1:apex": [16.861],
+
+ "os-odl-nofeature-ha:arm-pod5:fuel": [18.071],
+
+ "os-nosdn-nofeature-ha:arm-pod5:fuel": [18.116],
+
+ "os-odl-nofeature-ha:lf-pod2:fuel": [18.8365],
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [18.927],
+
+ "os-nosdn-nofeature-noha:huawei-virtual8:compass": [29.557],
+
+ "os-odl-sfc-ha:huawei-virtual8:compass": [32.492],
+
+ "os-nosdn-kvm-noha:huawei-virtual8:compass": [37.623],
+
+ "os-odl-nofeature-ha:arm-pod6:fuel": [41.345],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [42.3795],
+
+}
+
+
+The influence of the scenario
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc010_scenario.png
+ :width: 800px
+ :alt: TC010 influence of scenario
+
+{
+
+ "os-nosdn-ovs-noha": [7.9],
+
+ "os-nosdn-ovs_dpdk-noha": [8.641],
+
+ "os-nosdn-ovs_dpdk-ha": [8.6815],
+
+ "os-nosdn-openbaton-ha": [8.882],
+
+ "os-odl_l2-moon-ha": [8.948],
+
+ "os-odl_l3-nofeature-ha": [8.992],
+
+ "os-nosdn-nofeature-ha": [9.118],
+
+ "os-nosdn-nofeature-noha": [9.174],
+
+ "os-odl_l2-moon-noha": [9.312],
+
+ "os-odl_l3-nofeature-noha": [9.5535],
+
+ "os-odl-nofeature-noha": [9.673],
+
+ "os-odl-sfc-noha": [9.8385],
+
+ "os-odl-sfc-ha": [9.98],
+
+ "os-nosdn-kvm-noha": [10.088],
+
+ "os-nosdn-kvm-ha": [11.1705],
+
+ "os-nosdn-bar-ha": [12.1395],
+
+ "os-nosdn-ovs-ha": [15.3195],
+
+ "os-ovn-nofeature-noha": [15.545],
+
+ "os-odl-nofeature-ha": [16.301],
+
+ "os-nosdn-bar-noha": [16.861]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc010_pod.png
+ :width: 800px
+ :alt: TC010 influence of the POD
+
+{
+
+ "ericsson-pod1": [5.7785],
+
+ "flex-pod2": [5.908],
+
+ "ericsson-virtual1": [6.412],
+
+ "intel-pod18": [6.5905],
+
+ "intel-pod5": [6.6975],
+
+ "ericsson-virtual4": [7.183],
+
+ "ericsson-virtual2": [8.4985],
+
+ "huawei-pod2": [8.877],
+
+ "huawei-pod12": [9.091],
+
+ "ericsson-virtual3": [9.719],
+
+ "huawei-virtual4": [10.1195],
+
+ "huawei-virtual3": [10.19],
+
+ "huawei-virtual1": [10.3045],
+
+ "huawei-virtual9": [10.318],
+
+ "huawei-virtual2": [11.274],
+
+ "lf-pod1": [15.7025],
+
+ "lf-pod2": [15.8495],
+
+ "arm-pod5": [18.092],
+
+ "huawei-virtual8": [33.999],
+
+ "arm-pod6": [41.5605]
+
+}
+
+
+Fraser release
+--------------
+
+Test results per scenario and pod (lower is better):
+
+{
+
+ "os-odl-nofeature-ha:ericsson-pod1:fuel": [6.8675],
+
+ "os-nosdn-nofeature-noha:intel-pod18:joid": [6.991],
+
+ "os-nosdn-openbaton-ha:intel-pod18:joid": [7.5535],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [7.571],
+ "os-nosdn-ovs-ha:ericsson-pod1:fuel": [7.635],
+
+ "os-nosdn-nofeature-ha:zte-pod2:daisy": [8.153],
+
+ "os-odl-nofeature-ha:zte-pod2:daisy": [8.1935],
+
+ "os-nosdn-bar-ha:huawei-pod2:compass": [9.1715],
+
+ "os-odl-sfc-ha:huawei-pod2:compass": [9.1875],
+
+ "os-odl_l3-nofeature-ha:huawei-pod2:compass": [9.241],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [9.255],
+
+ "os-nosdn-kvm-ha:huawei-pod2:compass": [9.388],
+
+ "os-nosdn-nofeature-noha:huawei-virtual4:compass": [9.5825],
+
+ "os-nosdn-nofeature-noha:huawei-pod12:joid": [9.5875],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [9.6345],
+
+ "os-odl-sfc-noha:huawei-virtual4:compass": [9.6535],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual2:fuel": [9.743],
+
+ "os-odl-sfc-noha:huawei-virtual3:compass": [9.82],
+
+ "os-odl-nofeature-noha:ericsson-virtual2:fuel": [9.8715],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual3:compass": [9.982],
+
+ "os-nosdn-bar-noha:huawei-virtual4:compass": [10.0195],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual4:compass": [10.1285],
+
+ "os-nosdn-nofeature-noha:huawei-virtual3:compass": [10.1335],
+
+ "os-nosdn-nofeature-noha:huawei-virtual2:compass": [10.22],
+
+ "os-nosdn-bar-noha:huawei-virtual3:compass": [10.2845],
+
+ "os-nosdn-ovs-noha:ericsson-virtual4:fuel": [10.4185],
+
+ "os-nosdn-ovs-noha:ericsson-virtual2:fuel": [10.4555],
+
+ "os-nosdn-kvm-noha:huawei-virtual3:compass": [10.5635],
+
+ "os-nosdn-kvm-noha:huawei-virtual4:compass": [10.6515],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual3:fuel": [10.9355],
+
+ "os-odl-nofeature-noha:ericsson-virtual4:fuel": [11.2015],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual3:compass": [12.984],
+
+ "os-nosdn-bar-ha:huawei-virtual3:compass": [13.306],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [13.721],
+
+ "os-nosdn-bar-ha:huawei-virtual4:compass": [14.133],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [14.158],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual4:compass": [14.375],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [14.396],
+
+ "os-nosdn-kvm-ha:huawei-virtual4:compass": [14.9375],
+
+ "os-odl-sfc-ha:huawei-virtual3:compass": [14.957],
+
+ "os-nosdn-calipso-noha:lf-pod1:apex": [16.3445],
+
+ "os-nosdn-ovs-ha:lf-pod2:fuel": [16.478],
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [16.4895],
+
+ "os-odl-nofeature-noha:lf-pod1:apex": [16.55],
+
+ "os-nosdn-nofeature-noha:lf-pod1:apex": [16.5665],
+
+ "os-odl-sfc-noha:lf-pod1:apex": [16.598],
+
+ "os-ovn-nofeature-noha:lf-pod1:apex": [16.805],
+
+ "os-odl-nofeature-ha:lf-pod1:apex": [16.9095],
+
+ "os-nosdn-bar-ha:lf-pod1:apex": [17.494],
+
+ "os-nosdn-bar-noha:lf-pod1:apex": [17.4995],
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [18.094],
+
+ "os-odl-nofeature-ha:arm-pod5:fuel": [18.744],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [19.8235],
+
+ "os-odl-nofeature-ha:lf-pod2:fuel": [20.758],
+
+ "os-nosdn-kvm-ha:huawei-virtual3:compass": [26.5245],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [55.667],
+
+ "os-odl-nofeature-ha:arm-pod6:fuel": [56.175],
+
+ "os-nosdn-ovs-ha:arm-pod6:fuel": [57.86]
+
+}
+
+
+The influence of the scenario
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc010_scenario_fraser.png
+ :width: 800px
+ :alt: TC010 influence of scenario
+
+{
+
+ "os-nosdn-openbaton-ha": [7.5535],
+
+ "os-odl-nofeature-ha": [8.2535],
+
+ "os-odl-sfc-ha": [9.251],
+
+ "os-nosdn-nofeature-ha": [9.464],
+
+ "os-odl-sfc-noha": [9.8265],
+
+ "os-odl_l3-nofeature-ha": [9.836],
+
+ "os-odl_l3-nofeature-noha": [10.0565],
+
+ "os-nosdn-nofeature-noha": [10.079],
+
+ "os-nosdn-kvm-ha": [10.418],
+
+ "os-nosdn-ovs-noha": [10.43],
+
+ "os-nosdn-kvm-noha": [10.603],
+
+ "os-nosdn-bar-noha": [11.067],
+
+ "os-nosdn-bar-ha": [13.911],
+
+ "os-odl-nofeature-noha": [14.046],
+
+ "os-nosdn-calipso-noha": [16.3445],
+
+ "os-nosdn-ovs-ha": [16.478],
+
+ "os-ovn-nofeature-noha": [16.805]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc010_pod_fraser.png
+ :width: 800px
+ :alt: TC010 influence of the POD
+
+{
+
+ "ericsson-pod1": [7.0645],
+
+ "intel-pod18": [7.4465],
+
+ "zte-pod2": [8.1865],
+
+ "huawei-pod2": [9.236],
+
+ "huawei-pod12": [9.615],
+
+ "ericsson-virtual2": [9.8925],
+
+ "huawei-virtual2": [10.22],
+
+ "ericsson-virtual4": [10.5465],
+
+ "ericsson-virtual3": [10.9355],
+
+ "huawei-virtual3": [10.95],
+
+ "huawei-virtual4": [11.557],
+
+ "lf-pod2": [16.5595],
+
+ "lf-pod1": [16.8395],
+
+ "arm-pod5": [18.744],
+
+ "huawei-virtual1": [19.8235],
+
+ "arm-pod6": [55.804]
+
+}
diff --git a/docs/release/results/tc011-packet-delay-variation.rst b/docs/release/results/tc011-packet-delay-variation.rst
new file mode 100644
index 000000000..f255b50ca
--- /dev/null
+++ b/docs/release/results/tc011-packet-delay-variation.rst
@@ -0,0 +1,432 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+
+
+=============================================
+Test results for TC011 packet delay variation
+=============================================
+
+.. toctree::
+ :maxdepth: 2
+
+
+Overview of test case
+=====================
+
+TC011 measures the packet delay variation sending the packets from one VM to the other.
+
+Metric: packet delay variation (jitter)
+Unit: ms
+
+
+Euphrates release
+-----------------
+
+Test results per scenario and pod (lower is better):
+
+{
+
+ "os-nosdn-kvm-noha:huawei-virtual3:compass": [2996],
+
+ "os-nosdn-nofeature-noha:huawei-virtual2:compass": [2996],
+
+ "os-nosdn-ovs_dpdk-noha:huawei-virtual4:compass": [2996],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual4:compass": [2996],
+
+ "os-nosdn-kvm-ha:huawei-virtual3:compass": [2997],
+
+ "os-nosdn-nofeature-ha:huawei-virtual2:compass": [2997],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-virtual3:compass": [2997],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-virtual4:compass": [2997],
+
+ "os-odl-sfc-ha:huawei-virtual4:compass": [2997],
+
+ "os-nosdn-nofeature-ha:flex-pod2:apex": [2997.5],
+
+ "os-nosdn-bar-ha:huawei-virtual3:compass": [2998],
+
+ "os-odl-sfc-ha:huawei-virtual8:compass": [2998],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [2999],
+
+ "os-odl_l2-moon-ha:huawei-virtual4:compass": [2999.5],
+
+ "os-nosdn-nofeature-ha:huawei-virtual9:compass": [3000],
+
+ "os-nosdn-nofeature-noha:huawei-virtual1:compass": [3001],
+
+ "os-nosdn-bar-ha:huawei-virtual4:compass": [3002],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [3002],
+
+ "os-nosdn-ovs_dpdk-noha:huawei-virtual3:compass": [3002],
+
+ "os-odl-sfc-ha:huawei-virtual3:compass": [3002],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual4:compass": [3003],
+
+ "os-nosdn-openbaton-ha:intel-pod18:joid": [3003.5],
+
+ "os-nosdn-kvm-noha:huawei-virtual4:compass": [3004],
+
+ "os-nosdn-kvm-noha:huawei-virtual8:compass": [3004],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [3004.5],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual3:compass": [3005],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual3:fuel": [3006],
+
+ "os-nosdn-kvm-ha:huawei-virtual4:compass": [3006.5],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual2:fuel": [3009],
+
+ "os-nosdn-nofeature-noha:huawei-virtual3:compass": [3010],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual2:compass": [3010],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual3:compass": [3012],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [3017],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual4:fuel": [3017],
+
+ "os-odl-sfc-noha:huawei-virtual4:compass": [3017],
+
+ "os-nosdn-nofeature-noha:intel-pod18:joid": [3018],
+
+ "os-nosdn-nofeature-ha:intel-pod5:joid": [3020],
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [3021],
+
+ "os-nosdn-bar-ha:huawei-pod2:compass": [3022],
+
+ "os-nosdn-bar-ha:lf-pod1:apex": [3022],
+
+ "os-nosdn-bar-noha:lf-pod1:apex": [3022],
+
+ "os-nosdn-kvm-ha:huawei-pod2:compass": [3022],
+
+ "os-nosdn-nofeature-ha:arm-pod5:fuel": [3022],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [3022],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [3022],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [3022],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [3022],
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [3022],
+
+ "os-nosdn-nofeature-noha:huawei-pod12:joid": [3022],
+
+ "os-nosdn-nofeature-noha:intel-pod5:joid": [3022],
+
+ "os-nosdn-nofeature-noha:lf-pod1:apex": [3022],
+
+ "os-nosdn-openbaton-ha:huawei-pod12:joid": [3022],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-pod2:compass": [3022],
+
+ "os-odl-nofeature-ha:arm-pod5:fuel": [3022],
+
+ "os-odl-sfc-ha:huawei-pod2:compass": [3022],
+
+ "os-odl-sfc-ha:lf-pod1:apex": [3022],
+
+ "os-odl-sfc-noha:huawei-virtual3:compass": [3022],
+
+ "os-odl-sfc-noha:lf-pod1:apex": [3022],
+
+ "os-odl_l2-moon-ha:huawei-pod2:compass": [3022],
+
+ "os-odl_l2-moon-ha:huawei-virtual3:compass": [3022],
+
+ "os-odl_l2-moon-noha:huawei-virtual3:compass": [3022],
+
+ "os-odl_l3-nofeature-ha:huawei-pod2:compass": [3022],
+
+ "os-ovn-nofeature-noha:lf-pod1:apex": [3022],
+
+ "os-nosdn-nofeature-noha:huawei-virtual4:compass": [3023],
+
+ "os-odl_l2-moon-noha:huawei-virtual4:compass": [3023],
+
+ "os-nosdn-nofeature-noha:huawei-virtual8:compass": [3024]
+
+}
+
+
+The influence of the scenario
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc011_scenario.png
+ :width: 800px
+ :alt: TC011 influence of scenario
+
+{
+
+ "os-nosdn-ovs_dpdk-noha": [2996],
+
+ "os-odl_l3-nofeature-noha": [2997],
+
+ "os-nosdn-kvm-noha": [2999],
+
+ "os-nosdn-ovs_dpdk-ha": [3002],
+
+ "os-nosdn-kvm-ha": [3014.5],
+
+ "os-odl-sfc-noha": [3018],
+
+ "os-nosdn-nofeature-noha": [3020],
+
+ "os-nosdn-openbaton-ha": [3020],
+
+ "os-nosdn-bar-ha": [3022],
+
+ "os-nosdn-bar-noha": [3022],
+
+ "os-nosdn-nofeature-ha": [3022],
+
+ "os-odl-nofeature-ha": [3022],
+
+ "os-odl-sfc-ha": [3022],
+
+ "os-odl_l2-moon-ha": [3022],
+
+ "os-odl_l2-moon-noha": [3022],
+
+ "os-odl_l3-nofeature-ha": [3022],
+
+ "os-ovn-nofeature-noha": [3022]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc011_pod.png
+ :width: 800px
+ :alt: TC011 influence of the POD
+
+{
+
+ "huawei-virtual2": [2997],
+
+ "flex-pod2": [2997.5],
+
+ "huawei-virtual3": [2998],
+
+ "huawei-virtual9": [3000],
+
+ "huawei-virtual8": [3001],
+
+ "huawei-virtual4": [3002],
+
+ "ericsson-virtual3": [3006],
+
+ "huawei-virtual1": [3007],
+
+ "ericsson-virtual2": [3009],
+
+ "intel-pod18": [3010],
+
+ "ericsson-virtual4": [3017],
+
+ "lf-pod2": [3021],
+
+ "arm-pod5": [3022],
+
+ "arm-pod6": [3022],
+
+ "ericsson-pod1": [3022],
+
+ "huawei-pod12": [3022],
+
+ "huawei-pod2": [3022],
+
+ "intel-pod5": [3022],
+
+ "lf-pod1": [3022]
+
+}
+
+
+Fraser release
+--------------
+
+Test results per scenario and pod (lower is better):
+
+{
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [1],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [1],
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [1],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual2:fuel": [1],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual3:fuel": [1],
+
+ "os-ovn-nofeature-noha:lf-pod1:apex": [1511.5],
+
+ "os-nosdn-kvm-noha:huawei-virtual3:compass": [2996],
+
+ "os-nosdn-bar-ha:huawei-virtual4:compass": [2997],
+
+ "os-nosdn-bar-noha:huawei-virtual4:compass": [2997],
+
+ "os-nosdn-kvm-ha:huawei-virtual4:compass": [2997],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [2997],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [2997],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [2997],
+
+ "os-odl-sfc-ha:huawei-virtual3:compass": [2997],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual3:compass": [2997],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual3:compass": [3000],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual4:compass": [3003],
+
+ "os-nosdn-bar-noha:huawei-virtual3:compass": [3011],
+
+ "os-nosdn-bar-ha:huawei-virtual3:compass": [3015.5],
+
+ "os-nosdn-kvm-noha:huawei-virtual4:compass": [3019],
+
+ "os-nosdn-nofeature-noha:huawei-virtual4:compass": [3021],
+
+ "os-odl-sfc-ha:huawei-virtual4:compass": [3021],
+
+ "os-nosdn-bar-ha:huawei-pod2:compass": [3022],
+
+ "os-nosdn-bar-ha:lf-pod1:apex": [3022],
+
+ "os-nosdn-bar-noha:lf-pod1:apex": [3022],
+
+ "os-nosdn-calipso-noha:lf-pod1:apex": [3022],
+
+ "os-nosdn-kvm-ha:huawei-pod2:compass": [3022],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [3022],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [3022],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [3022],
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [3022],
+
+ "os-nosdn-nofeature-ha:zte-pod2:daisy": [3022],
+
+ "os-nosdn-nofeature-noha:huawei-pod12:joid": [3022],
+
+ "os-nosdn-nofeature-noha:intel-pod18:joid": [3022],
+
+ "os-nosdn-nofeature-noha:lf-pod1:apex": [3022],
+
+ "os-nosdn-openbaton-ha:intel-pod18:joid": [3022],
+
+ "os-odl-sfc-ha:huawei-pod2:compass": [3022],
+
+ "os-odl_l3-nofeature-ha:huawei-pod2:compass": [3022],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual4:compass": [3022],
+
+ "os-odl-sfc-noha:huawei-virtual4:compass": [3022.5],
+
+ "os-nosdn-nofeature-noha:huawei-virtual3:compass": [3023],
+
+ "os-odl-sfc-noha:huawei-virtual3:compass": [3023],
+
+ "os-nosdn-nofeature-noha:huawei-virtual2:compass": [3025]
+
+}
+
+
+The influence of the scenario
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc011_scenario_fraser.png
+ :width: 800px
+ :alt: TC011 influence of scenario
+
+{
+
+ "os-ovn-nofeature-noha": [1511.5],
+
+ "os-nosdn-kvm-noha": [2997],
+
+ "os-odl-sfc-ha": [3021],
+
+ "os-nosdn-bar-ha": [3022],
+
+ "os-nosdn-bar-noha": [3022],
+
+ "os-nosdn-calipso-noha": [3022],
+
+ "os-nosdn-kvm-ha": [3022],
+
+ "os-nosdn-nofeature-ha": [3022],
+
+ "os-nosdn-nofeature-noha": [3022],
+
+ "os-nosdn-openbaton-ha": [3022],
+
+ "os-odl_l3-nofeature-ha": [3022],
+
+ "os-odl_l3-nofeature-noha": [3022],
+
+ "os-odl-sfc-noha": [3023]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc011_pod_fraser.png
+ :width: 800px
+ :alt: TC011 influence of the POD
+
+{
+
+ "arm-pod6": [1],
+
+ "ericsson-pod1": [1],
+
+ "ericsson-virtual2": [1],
+
+ "ericsson-virtual3": [1],
+
+ "lf-pod2": [1],
+
+ "huawei-virtual1": [2997],
+
+ "huawei-virtual3": [2999],
+
+ "huawei-virtual4": [3002],
+
+ "huawei-pod12": [3022],
+
+ "huawei-pod2": [3022],
+
+ "intel-pod18": [3022],
+
+ "lf-pod1": [3022],
+
+ "zte-pod2": [3022],
+
+ "huawei-virtual2": [3025]
+
+}
diff --git a/docs/release/results/tc012-memory-read-write-bandwidth.rst b/docs/release/results/tc012-memory-read-write-bandwidth.rst
new file mode 100644
index 000000000..71d69cde9
--- /dev/null
+++ b/docs/release/results/tc012-memory-read-write-bandwidth.rst
@@ -0,0 +1,513 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+
+
+==================================================
+Test results for TC012 memory read/write bandwidth
+==================================================
+
+.. toctree::
+ :maxdepth: 2
+
+
+Overview of test case
+=====================
+
+TC012 measures the rate at which data can be read from and written to the memory (this includes all levels of memory).
+In this test case, the bandwidth to read data from memory and then write data to the same memory location are measured.
+
+Metric: memory bandwidth
+Unit: MBps
+
+
+Euphrates release
+-----------------
+
+Test results per scenario and pod (higher is better):
+
+{
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [23126.325],
+
+ "os-odl-nofeature-noha:lf-pod1:apex": [23123.975],
+
+ "os-odl-nofeature-ha:lf-pod1:apex": [23068.965],
+
+ "os-odl-nofeature-ha:lf-pod2:fuel": [22972.46],
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [22912.015],
+
+ "os-nosdn-nofeature-noha:lf-pod1:apex": [22911.35],
+
+ "os-ovn-nofeature-noha:lf-pod1:apex": [22900.93],
+
+ "os-nosdn-bar-ha:lf-pod1:apex": [22767.56],
+
+ "os-nosdn-bar-noha:lf-pod1:apex": [22721.83],
+
+ "os-odl-sfc-noha:lf-pod1:apex": [22511.565],
+
+ "os-nosdn-ovs-ha:lf-pod2:fuel": [22071.235],
+
+ "os-odl-sfc-ha:lf-pod1:apex": [21646.415],
+
+ "os-nosdn-nofeature-ha:flex-pod2:apex": [20229.99],
+
+ "os-nosdn-ovs-noha:ericsson-virtual4:fuel": [17491.18],
+
+ "os-nosdn-ovs-noha:ericsson-virtual1:fuel": [17474.965],
+
+ "os-nosdn-ovs-ha:ericsson-pod1:fuel": [17141.375],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [17134.99],
+
+ "os-odl-nofeature-ha:ericsson-pod1:fuel": [17124.27],
+
+ "os-nosdn-ovs-noha:ericsson-virtual2:fuel": [16599.325],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual4:fuel": [16309.13],
+
+ "os-odl-nofeature-noha:ericsson-virtual4:fuel": [16137.48],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual2:fuel": [15960.76],
+
+ "os-nosdn-ovs-noha:ericsson-virtual3:fuel": [15685.505],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual3:fuel": [15536.65],
+
+ "os-odl-nofeature-noha:ericsson-virtual3:fuel": [15431.795],
+
+ "os-odl-nofeature-noha:ericsson-virtual2:fuel": [15129.27],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-pod2:compass": [15125.51],
+
+ "os-odl_l3-nofeature-ha:huawei-pod2:compass": [15030.65],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [15019.89],
+
+ "os-odl-sfc-ha:huawei-pod2:compass": [15005.11],
+
+ "os-nosdn-bar-ha:huawei-pod2:compass": [14975.645],
+
+ "os-nosdn-kvm-ha:huawei-pod2:compass": [14968.97],
+
+ "os-odl_l2-moon-ha:huawei-pod2:compass": [14968.97],
+
+ "os-nosdn-ovs_dpdk-noha:huawei-virtual4:compass": [14741.425],
+
+ "os-nosdn-ovs_dpdk-noha:huawei-virtual3:compass": [14714.28],
+
+ "os-odl_l2-moon-noha:huawei-virtual4:compass": [14674.38],
+
+ "os-odl_l2-moon-noha:huawei-virtual3:compass": [14664.12],
+
+ "os-odl-sfc-noha:huawei-virtual4:compass": [14587.62],
+
+ "os-nosdn-nofeature-noha:huawei-virtual3:compass": [14539.94],
+
+ "os-nosdn-nofeature-noha:huawei-virtual4:compass": [14534.54],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual3:compass": [14511.925],
+
+ "os-nosdn-nofeature-noha:huawei-virtual1:compass": [14496.875],
+
+ "os-odl_l2-moon-ha:huawei-virtual3:compass": [14378.87],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual4:compass": [14366.69],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [14356.695],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual3:compass": [14341.605],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-virtual3:compass": [14327.78],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-virtual4:compass": [14313.81],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [14284.365],
+
+ "os-nosdn-nofeature-noha:huawei-pod12:joid": [14157.99],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [14144.86],
+
+ "os-nosdn-openbaton-ha:huawei-pod12:joid": [14138.9],
+
+ "os-nosdn-kvm-noha:huawei-virtual3:compass": [14117.7],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [14097.255],
+
+ "os-nosdn-nofeature-noha:huawei-virtual2:compass": [14085.675],
+
+ "os-odl-sfc-noha:huawei-virtual3:compass": [14071.605],
+
+ "os-nosdn-openbaton-ha:intel-pod18:joid": [14059.51],
+
+ "os-odl-sfc-ha:huawei-virtual4:compass": [14057.155],
+
+ "os-odl-sfc-ha:huawei-virtual3:compass": [14051.945],
+
+ "os-nosdn-bar-ha:huawei-virtual3:compass": [14020.74],
+
+ "os-nosdn-kvm-noha:huawei-virtual4:compass": [14017.915],
+
+ "os-nosdn-nofeature-noha:intel-pod18:joid": [13954.27],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual4:compass": [13915.87],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual2:compass": [13874.59],
+
+ "os-nosdn-nofeature-noha:intel-pod5:joid": [13812.215],
+
+ "os-odl_l2-moon-ha:huawei-virtual4:compass": [13777.59],
+
+ "os-nosdn-bar-ha:huawei-virtual4:compass": [13765.36],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [13559.905],
+
+ "os-nosdn-nofeature-ha:huawei-virtual2:compass": [13477.52],
+
+ "os-nosdn-kvm-ha:huawei-virtual3:compass": [13255.17],
+
+ "os-nosdn-nofeature-ha:intel-pod5:joid": [13189.64],
+
+ "os-nosdn-kvm-ha:huawei-virtual4:compass": [12718.545],
+
+ "os-nosdn-nofeature-ha:huawei-virtual9:compass": [12559.445],
+
+ "os-nosdn-nofeature-noha:huawei-virtual8:compass": [12409.66],
+
+ "os-nosdn-kvm-noha:huawei-virtual8:compass": [8832.515],
+
+ "os-odl-sfc-ha:huawei-virtual8:compass": [8823.955],
+
+ "os-odl-nofeature-ha:arm-pod5:fuel": [4398.08],
+
+ "os-nosdn-nofeature-ha:arm-pod5:fuel": [4375.75],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [4260.77],
+
+ "os-odl-nofeature-ha:arm-pod6:fuel": [4259.62]
+
+}
+
+
+The influence of the scenario
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc012_scenario.png
+ :width: 800px
+ :alt: TC012 influence of scenario
+
+{
+
+ "os-ovn-nofeature-noha": [22900.93],
+
+ "os-nosdn-bar-noha": [22721.83],
+
+ "os-nosdn-ovs-ha": [22063.67],
+
+ "os-odl-nofeature-ha": [17146.05],
+
+ "os-odl-nofeature-noha": [16017.41],
+
+ "os-nosdn-ovs-noha": [16005.74],
+
+ "os-nosdn-nofeature-noha": [15290.94],
+
+ "os-nosdn-nofeature-ha": [15038.74],
+
+ "os-nosdn-bar-ha": [14972.975],
+
+ "os-odl_l2-moon-ha": [14956.955],
+
+ "os-odl_l3-nofeature-ha": [14839.21],
+
+ "os-odl-sfc-ha": [14823.48],
+
+ "os-nosdn-ovs_dpdk-ha": [14822.17],
+
+ "os-nosdn-ovs_dpdk-noha": [14725.9],
+
+ "os-odl_l2-moon-noha": [14665.4],
+
+ "os-odl_l3-nofeature-noha": [14483.09],
+
+ "os-odl-sfc-noha": [14373.21],
+
+ "os-nosdn-openbaton-ha": [14135.325],
+
+ "os-nosdn-kvm-noha": [14020.26],
+
+ "os-nosdn-kvm-ha": [13996.02]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc012_pod.png
+ :width: 800px
+ :alt: TC012 influence of the POD
+
+{
+
+ "lf-pod1": [22912.39],
+
+ "lf-pod2": [22637.67],
+
+ "flex-pod2": [20229.99],
+
+ "ericsson-virtual1": [17474.965],
+
+ "ericsson-pod1": [17127.38],
+
+ "ericsson-virtual4": [16219.97],
+
+ "ericsson-virtual2": [15652.28],
+
+ "ericsson-virtual3": [15551.26],
+
+ "huawei-pod2": [15017.2],
+
+ "huawei-virtual4": [14266.34],
+
+ "huawei-virtual1": [14233.035],
+
+ "huawei-virtual3": [14227.63],
+
+ "huawei-pod12": [14147.245],
+
+ "intel-pod18": [14058.33],
+
+ "huawei-virtual2": [13862.85],
+
+ "intel-pod5": [13280.32],
+
+ "huawei-virtual9": [12559.445],
+
+ "huawei-virtual8": [8998.02],
+
+ "arm-pod5": [4388.875],
+
+ "arm-pod6": [4260.2]
+
+}
+
+
+Fraser release
+--------------
+
+Test results per scenario and pod (higher is better):
+
+{
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [21421.795],
+
+ "os-odl-sfc-noha:lf-pod1:apex": [21075],
+
+ "os-odl-sfc-ha:lf-pod1:apex": [21017.44],
+
+ "os-nosdn-bar-noha:lf-pod1:apex": [20991.46],
+
+ "os-nosdn-bar-ha:lf-pod1:apex": [20812.405],
+
+ "os-ovn-nofeature-noha:lf-pod1:apex": [20694.035],
+
+ "os-nosdn-nofeature-noha:lf-pod1:apex": [20672.765],
+
+ "os-odl-nofeature-ha:lf-pod2:fuel": [20269.65],
+
+ "os-nosdn-calipso-noha:lf-pod1:apex": [20186.32],
+
+ "os-odl-nofeature-noha:lf-pod1:apex": [19959.915],
+
+ "os-nosdn-ovs-ha:lf-pod2:fuel": [19719.38],
+
+ "os-odl-nofeature-ha:lf-pod1:apex": [19654.505],
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [19391.145],
+
+ "os-nosdn-nofeature-noha:intel-pod18:joid": [19378.64],
+
+ "os-odl-nofeature-ha:ericsson-pod1:fuel": [19103.43],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [18688.695],
+
+ "os-nosdn-openbaton-ha:intel-pod18:joid": [18557.95],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [17088.61],
+
+ "os-nosdn-ovs-ha:ericsson-pod1:fuel": [17040.78],
+
+ "os-nosdn-ovs-noha:ericsson-virtual2:fuel": [16057.235],
+
+ "os-odl-nofeature-noha:ericsson-virtual4:fuel": [15622.355],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual2:fuel": [15422.235],
+
+ "os-odl-sfc-ha:huawei-pod2:compass": [15403.09],
+
+ "os-odl-nofeature-noha:ericsson-virtual2:fuel": [15141.58],
+
+ "os-nosdn-bar-ha:huawei-pod2:compass": [14922.37],
+
+ "os-odl_l3-nofeature-ha:huawei-pod2:compass": [14864.195],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [14856.295],
+
+ "os-nosdn-kvm-ha:huawei-pod2:compass": [14796.035],
+
+ "os-odl-sfc-noha:huawei-virtual4:compass": [14484.375],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [14441.955],
+
+ "os-odl-sfc-noha:huawei-virtual3:compass": [14373],
+
+ "os-nosdn-nofeature-noha:huawei-virtual4:compass": [14330.44],
+
+ "os-nosdn-ovs-noha:ericsson-virtual4:fuel": [14320.305],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual3:compass": [14253.715],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [14203.655],
+
+ "os-nosdn-nofeature-noha:huawei-virtual3:compass": [14179.93],
+
+ "os-odl-nofeature-ha:zte-pod2:daisy": [14177.135],
+
+ "os-nosdn-nofeature-ha:zte-pod2:daisy": [14150.825],
+
+ "os-nosdn-nofeature-noha:huawei-pod12:joid": [14100.87],
+
+ "os-nosdn-bar-noha:huawei-virtual4:compass": [14033.36],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual4:compass": [13963.73],
+
+ "os-nosdn-kvm-noha:huawei-virtual3:compass": [13874.775],
+
+ "os-nosdn-kvm-noha:huawei-virtual4:compass": [13805.65],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual3:compass": [13754.63],
+
+ "os-nosdn-nofeature-noha:huawei-virtual2:compass": [13702.92],
+
+ "os-nosdn-bar-ha:huawei-virtual3:compass": [13638.115],
+
+ "os-odl-sfc-ha:huawei-virtual3:compass": [13637.83],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual4:compass": [13635.66],
+
+ "os-nosdn-bar-noha:huawei-virtual3:compass": [13635.58],
+
+ "os-nosdn-bar-ha:huawei-virtual4:compass": [13544.95],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [13514.27],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [13496.45],
+
+ "os-odl-sfc-ha:huawei-virtual4:compass": [13475.38],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual3:fuel": [12733.19],
+
+ "os-nosdn-kvm-ha:huawei-virtual4:compass": [12682.805],
+
+ "os-odl-nofeature-ha:arm-pod5:fuel": [4326.11],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [3824.13],
+
+ "os-odl-nofeature-ha:arm-pod6:fuel": [3797.795],
+
+ "os-nosdn-ovs-ha:arm-pod6:fuel": [3749.91]
+
+}
+
+
+The influence of the scenario
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc012_scenario_fraser.png
+ :width: 800px
+ :alt: TC012 influence of scenario
+
+{
+
+ "os-ovn-nofeature-noha": [20694.035],
+
+ "os-nosdn-calipso-noha": [20186.32],
+
+ "os-nosdn-openbaton-ha": [18557.95],
+
+ "os-nosdn-ovs-ha": [17048.17],
+
+ "os-odl-nofeature-noha": [16191.125],
+
+ "os-nosdn-ovs-noha": [15790.32],
+
+ "os-nosdn-bar-ha": [14833.97],
+
+ "os-odl-sfc-ha": [14828.72],
+
+ "os-odl_l3-nofeature-ha": [14801.25],
+
+ "os-nosdn-kvm-ha": [14700.1],
+
+ "os-nosdn-nofeature-ha": [14610.48],
+
+ "os-nosdn-nofeature-noha": [14555.975],
+
+ "os-odl-sfc-noha": [14508.14],
+
+ "os-nosdn-bar-noha": [14395.22],
+
+ "os-odl-nofeature-ha": [14231.245],
+
+ "os-odl_l3-nofeature-noha": [14161.58],
+
+ "os-nosdn-kvm-noha": [13845.685]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc012_pod_fraser.png
+ :width: 800px
+ :alt: TC012 influence of the POD
+
+{
+
+ "lf-pod1": [20552.9],
+
+ "lf-pod2": [20058.925],
+
+ "ericsson-pod1": [18930.78],
+
+ "intel-pod18": [18757.545],
+
+ "ericsson-virtual4": [15389.465],
+
+ "ericsson-virtual2": [15343.79],
+
+ "huawei-pod2": [14870.78],
+
+ "zte-pod2": [14157.99],
+
+ "huawei-pod12": [14126.99],
+
+ "huawei-virtual3": [13929.67],
+
+ "huawei-virtual4": [13847.155],
+
+ "huawei-virtual2": [13702.92],
+
+ "huawei-virtual1": [13496.45],
+
+ "ericsson-virtual3": [12733.19],
+
+ "arm-pod5": [4326.11],
+
+ "arm-pod6": [3809.885]
+
+}
diff --git a/docs/release/results/tc014-cpu-processing-speed.rst b/docs/release/results/tc014-cpu-processing-speed.rst
new file mode 100644
index 000000000..a2eeb6302
--- /dev/null
+++ b/docs/release/results/tc014-cpu-processing-speed.rst
@@ -0,0 +1,512 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+
+
+===========================================
+Test results for TC014 cpu processing speed
+===========================================
+
+.. toctree::
+ :maxdepth: 2
+
+
+Overview of test case
+=====================
+
+TC014 measures score of single cpu running using UnixBench.
+
+Metric: score of single CPU running
+Unit: N/A
+
+
+Euphrates release
+-----------------
+
+Test results per scenario and pod (higher is better):
+
+{
+
+ "os-odl-sfc-noha:lf-pod1:apex": [3735.2],
+
+ "os-nosdn-ovs-ha:lf-pod2:fuel": [3725.5],
+
+ "os-odl-nofeature-ha:lf-pod2:fuel": [3711],
+
+ "os-odl-nofeature-ha:lf-pod1:apex": [3708.4],
+
+ "os-nosdn-nofeature-noha:lf-pod1:apex": [3705.7],
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [3704],
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [3703.2],
+
+ "os-odl-nofeature-noha:lf-pod1:apex": [3702.8],
+
+ "os-odl-sfc-ha:lf-pod1:apex": [3698.7],
+
+ "os-ovn-nofeature-noha:lf-pod1:apex": [3654.8],
+
+ "os-nosdn-bar-ha:lf-pod1:apex": [3635.55],
+
+ "os-nosdn-bar-noha:lf-pod1:apex": [3633.2],
+
+ "os-nosdn-nofeature-noha:intel-pod18:joid": [3450.3],
+
+ "os-nosdn-nofeature-noha:intel-pod5:joid": [3406.4],
+
+ "os-nosdn-nofeature-ha:intel-pod5:joid": [3360.4],
+
+ "os-nosdn-openbaton-ha:intel-pod18:joid": [3340.65],
+
+ "os-nosdn-nofeature-ha:flex-pod2:apex": [3208.6],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [3134.8],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [3056.2],
+
+ "os-nosdn-ovs-noha:ericsson-virtual1:fuel": [2988.9],
+
+ "os-nosdn-ovs-ha:ericsson-pod1:fuel": [2773.7],
+
+ "os-nosdn-ovs-noha:ericsson-virtual4:fuel": [2645.85],
+
+ "os-nosdn-ovs-noha:ericsson-virtual2:fuel": [2625.3],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual4:fuel": [2601.3],
+
+ "os-odl-nofeature-noha:ericsson-virtual4:fuel": [2590.4],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual2:fuel": [2570.2],
+
+ "os-nosdn-ovs-noha:ericsson-virtual3:fuel": [2558.8],
+
+ "os-odl-nofeature-ha:ericsson-pod1:fuel": [2556.5],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual3:fuel": [2554.6],
+
+ "os-odl-nofeature-noha:ericsson-virtual3:fuel": [2536.75],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-pod2:compass": [2533.55],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [2531.85],
+
+ "os-odl-sfc-ha:huawei-pod2:compass": [2531.7],
+
+ "os-odl_l3-nofeature-ha:huawei-pod2:compass": [2531.2],
+
+ "os-odl_l2-moon-ha:huawei-pod2:compass": [2531],
+
+ "os-nosdn-bar-ha:huawei-pod2:compass": [2529.6],
+
+ "os-nosdn-kvm-ha:huawei-pod2:compass": [2520.5],
+
+ "os-odl-nofeature-noha:ericsson-virtual2:fuel": [2481.15],
+
+ "os-nosdn-ovs_dpdk-noha:huawei-virtual4:compass": [2474],
+
+ "os-nosdn-ovs_dpdk-noha:huawei-virtual3:compass": [2472.6],
+
+ "os-odl_l2-moon-noha:huawei-virtual4:compass": [2471],
+
+ "os-odl_l2-moon-noha:huawei-virtual3:compass": [2470.6],
+
+ "os-nosdn-nofeature-noha:huawei-virtual3:compass": [2464.15],
+
+ "os-odl-sfc-noha:huawei-virtual4:compass": [2455.9],
+
+ "os-nosdn-nofeature-noha:huawei-virtual4:compass": [2455.3],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual3:compass": [2446.85],
+
+ "os-odl_l2-moon-ha:huawei-virtual3:compass": [2444.75],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual4:compass": [2430.9],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [2421.3],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-virtual4:compass": [2415.7],
+
+ "os-nosdn-kvm-noha:huawei-virtual3:compass": [2399.4],
+
+ "os-odl-sfc-ha:huawei-virtual3:compass": [2391.85],
+
+ "os-nosdn-kvm-noha:huawei-virtual4:compass": [2391.45],
+
+ "os-nosdn-nofeature-noha:huawei-virtual1:compass": [2380.7],
+
+ "os-odl-sfc-ha:huawei-virtual4:compass": [2379.6],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-virtual3:compass": [2371.9],
+
+ "os-odl-sfc-noha:huawei-virtual3:compass": [2364.6],
+
+ "os-nosdn-bar-ha:huawei-virtual3:compass": [2363.4],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [2362],
+
+ "os-nosdn-kvm-ha:huawei-virtual4:compass": [2358.5],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual3:compass": [2358.45],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual2:compass": [2336],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual4:compass": [2326.6],
+
+ "os-nosdn-nofeature-ha:huawei-virtual9:compass": [2324.95],
+
+ "os-nosdn-nofeature-noha:huawei-virtual8:compass": [2320.2],
+
+ "os-nosdn-bar-ha:huawei-virtual4:compass": [2318.5],
+
+ "os-odl_l2-moon-ha:huawei-virtual4:compass": [2312.8],
+
+ "os-nosdn-nofeature-noha:huawei-virtual2:compass": [2311.7],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [2301.15],
+
+ "os-nosdn-nofeature-ha:huawei-virtual2:compass": [2297.7],
+
+ "os-nosdn-nofeature-noha:huawei-pod12:joid": [2232.8],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [2232.1],
+
+ "os-nosdn-openbaton-ha:huawei-pod12:joid": [2230],
+
+ "os-nosdn-kvm-ha:huawei-virtual3:compass": [2154],
+
+ "os-odl-sfc-ha:huawei-virtual8:compass": [2150.1],
+
+ "os-nosdn-kvm-noha:huawei-virtual8:compass": [2004.3],
+
+ "os-odl-nofeature-ha:arm-pod5:fuel": [1754.5],
+
+ "os-nosdn-nofeature-ha:arm-pod5:fuel": [1754.15],
+
+ "os-odl-nofeature-ha:arm-pod6:fuel": [716.15],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [716.05]
+
+}
+
+
+The influence of the scenario
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc014_scenario.png
+ :width: 800px
+ :alt: TC014 influence of scenario
+
+{
+
+ "os-nosdn-ovs-ha": [3725.5],
+
+ "os-ovn-nofeature-noha": [3654.8],
+
+ "os-nosdn-bar-noha": [3633.2],
+
+ "os-odl-nofeature-ha": [3407.8],
+
+ "os-nosdn-ovs-noha": [2583.2],
+
+ "os-odl-nofeature-noha": [2578.9],
+
+ "os-nosdn-nofeature-noha": [2553.2],
+
+ "os-nosdn-nofeature-ha": [2532.8],
+
+ "os-odl_l2-moon-ha": [2530.5],
+
+ "os-nosdn-bar-ha": [2527],
+
+ "os-odl_l3-nofeature-ha": [2501.5],
+
+ "os-nosdn-ovs_dpdk-noha": [2473.65],
+
+ "os-odl-sfc-ha": [2472.9],
+
+ "os-odl_l2-moon-noha": [2470.8],
+
+ "os-nosdn-ovs_dpdk-ha": [2461.9],
+
+ "os-odl_l3-nofeature-noha": [2442.8],
+
+ "os-nosdn-kvm-noha": [2392.9],
+
+ "os-odl-sfc-noha": [2370.5],
+
+ "os-nosdn-kvm-ha": [2358.5],
+
+ "os-nosdn-openbaton-ha": [2231.8]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc014_pod.png
+ :width: 800px
+ :alt: TC014 influence of the POD
+
+{
+
+ "lf-pod2": [3723.95],
+
+ "lf-pod1": [3669],
+
+ "intel-pod5": [3388.6],
+
+ "intel-pod18": [3298.4],
+
+ "flex-pod2": [3208.6],
+
+ "ericsson-virtual1": [2988.9],
+
+ "ericsson-pod1": [2669.1],
+
+ "ericsson-virtual4": [2598.5],
+
+ "ericsson-virtual3": [2553.15],
+
+ "huawei-pod2": [2531.2],
+
+ "ericsson-virtual2": [2526.9],
+
+ "huawei-virtual4": [2407.4],
+
+ "huawei-virtual3": [2374.6],
+
+ "huawei-virtual2": [2326.4],
+
+ "huawei-virtual9": [2324.95],
+
+ "huawei-virtual1": [2302.6],
+
+ "huawei-pod12": [2232.2],
+
+ "huawei-virtual8": [2085.3],
+
+ "arm-pod5": [1754.4],
+
+ "arm-pod6": [716.15]
+
+}
+
+
+Fraser release
+--------------
+
+Test results per scenario and pod (higher is better):
+
+{
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [3747.3],
+
+ "os-nosdn-calipso-noha:lf-pod1:apex": [3727.2],
+
+ "os-odl-nofeature-ha:lf-pod1:apex": [3726.5],
+
+ "os-ovn-nofeature-noha:lf-pod1:apex": [3723.8],
+
+ "os-odl-nofeature-noha:lf-pod1:apex": [3718.9],
+
+ "os-nosdn-nofeature-noha:lf-pod1:apex": [3717.75],
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [3706.5],
+
+ "os-odl-nofeature-ha:lf-pod2:fuel": [3704.9],
+
+ "os-nosdn-ovs-ha:lf-pod2:fuel": [3687.7],
+
+ "os-nosdn-bar-noha:lf-pod1:apex": [3635.4],
+
+ "os-nosdn-bar-ha:lf-pod1:apex": [3632.55],
+
+ "os-odl-sfc-noha:lf-pod1:apex": [3569],
+
+ "os-nosdn-nofeature-noha:intel-pod18:joid": [3432.1],
+
+ "os-odl-nofeature-ha:ericsson-pod1:fuel": [3133.85],
+
+ "os-nosdn-ovs-ha:ericsson-pod1:fuel": [3079.8],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [3074.75],
+
+ "os-nosdn-openbaton-ha:intel-pod18:joid": [2976.2],
+
+ "os-nosdn-nofeature-ha:zte-pod2:daisy": [2910.95],
+
+ "os-odl-nofeature-ha:zte-pod2:daisy": [2801.1],
+
+ "os-nosdn-ovs-noha:ericsson-virtual2:fuel": [2603],
+
+ "os-odl-nofeature-noha:ericsson-virtual2:fuel": [2559.7],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual2:fuel": [2539.1],
+
+ "os-odl_l3-nofeature-ha:huawei-pod2:compass": [2530.5],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [2529.4],
+
+ "os-odl-sfc-ha:huawei-pod2:compass": [2528.9],
+
+ "os-odl-nofeature-noha:ericsson-virtual4:fuel": [2527.8],
+
+ "os-nosdn-bar-ha:huawei-pod2:compass": [2527.4],
+
+ "os-nosdn-kvm-ha:huawei-pod2:compass": [2517.8],
+
+ "os-nosdn-nofeature-noha:huawei-virtual4:compass": [2472.4],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [2469.1],
+
+ "os-odl-sfc-noha:huawei-virtual3:compass": [2452.05],
+
+ "os-odl-sfc-noha:huawei-virtual4:compass": [2438.7],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual3:compass": [2418.4],
+
+ "os-nosdn-ovs-noha:ericsson-virtual4:fuel": [2404.35],
+
+ "os-nosdn-kvm-noha:huawei-virtual3:compass": [2391],
+
+ "os-nosdn-kvm-noha:huawei-virtual4:compass": [2376.75],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual4:compass": [2376.2],
+
+ "os-nosdn-nofeature-noha:huawei-virtual3:compass": [2359.45],
+
+ "os-nosdn-bar-noha:huawei-virtual4:compass": [2353.3],
+
+ "os-odl-sfc-ha:huawei-virtual3:compass": [2351.9],
+
+ "os-nosdn-bar-ha:huawei-virtual3:compass": [2339.4],
+
+ "os-odl-sfc-ha:huawei-virtual4:compass": [2335.6],
+
+ "os-nosdn-bar-ha:huawei-virtual4:compass": [2328],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual3:compass": [2324.5],
+
+ "os-nosdn-bar-noha:huawei-virtual3:compass": [2317.3],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [2313.95],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual4:compass": [2308.1],
+
+ "os-nosdn-nofeature-noha:huawei-virtual2:compass": [2299.3],
+
+ "os-nosdn-kvm-ha:huawei-virtual4:compass": [2250.4],
+
+ "os-nosdn-nofeature-noha:huawei-pod12:joid": [2229.7],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [2228.8],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [2171.3],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual3:fuel": [2104.8],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [1961.35],
+
+ "os-nosdn-ovs-ha:arm-pod5:fuel": [1764.2],
+
+ "os-odl-nofeature-ha:arm-pod5:fuel": [1730.95],
+
+ "os-nosdn-ovs-ha:arm-pod6:fuel": [715.55],
+
+ "os-odl-nofeature-ha:arm-pod6:fuel": [715.4],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [715.25]
+
+}
+
+
+The influence of the scenario
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc014_scenario_fraser.png
+ :width: 800px
+ :alt: TC014 influence of scenario
+
+{
+
+ "os-nosdn-calipso-noha": [3727.2],
+
+ "os-ovn-nofeature-noha": [3723.8],
+
+ "os-odl-nofeature-noha": [3128.05],
+
+ "os-nosdn-openbaton-ha": [2976.2],
+
+ "os-nosdn-ovs-ha": [2814.5],
+
+ "os-odl-nofeature-ha": [2801.4],
+
+ "os-nosdn-nofeature-ha": [2649.7],
+
+ "os-nosdn-ovs-noha": [2587.3],
+
+ "os-odl_l3-nofeature-ha": [2528.45],
+
+ "os-odl-sfc-ha": [2527.6],
+
+ "os-nosdn-bar-ha": [2526.55],
+
+ "os-nosdn-kvm-ha": [2516.95],
+
+ "os-odl-sfc-noha": [2453.65],
+
+ "os-nosdn-bar-noha": [2447.7],
+
+ "os-nosdn-nofeature-noha": [2443.85],
+
+ "os-odl_l3-nofeature-noha": [2394.3],
+
+ "os-nosdn-kvm-noha": [2379.7]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc014_pod_fraser.png
+ :width: 800px
+ :alt: TC014 influence of the POD
+
+{
+
+ "lf-pod2": [3737.6],
+
+ "lf-pod1": [3702.7],
+
+ "ericsson-pod1": [3131.6],
+
+ "intel-pod18": [3098.1],
+
+ "zte-pod2": [2831.4],
+
+ "ericsson-virtual2": [2559.7],
+
+ "huawei-pod2": [2528.9],
+
+ "ericsson-virtual4": [2527.8],
+
+ "huawei-virtual3": [2379.1],
+
+ "huawei-virtual4": [2362.1],
+
+ "huawei-virtual2": [2299.3],
+
+ "huawei-pod12": [2229],
+
+ "huawei-virtual1": [2171.3],
+
+ "ericsson-virtual3": [2104.8],
+
+ "arm-pod5": [1764.2],
+
+ "arm-pod6": [715.4]
+
+}
diff --git a/docs/release/results/tc069-memory-write-bandwidth.rst b/docs/release/results/tc069-memory-write-bandwidth.rst
new file mode 100644
index 000000000..4cd3be3b0
--- /dev/null
+++ b/docs/release/results/tc069-memory-write-bandwidth.rst
@@ -0,0 +1,516 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+
+
+=============================================
+Test results for TC069 memory write bandwidth
+=============================================
+
+.. toctree::
+ :maxdepth: 2
+
+
+Overview of test case
+=====================
+
+TC069 measures the maximum possible cache and memory performance while reading and writing certain
+blocks of data (starting from 1Kb and further in power of 2) continuously through ALU and FPU
+respectively. Measure different aspects of memory performance via synthetic simulations.
+Each simulation consists of four performances (Copy, Scale, Add, Triad).
+The test results shown below are for writing 32MB integer block size.
+
+Metric: memory write bandwidth
+Unit: MBps
+
+
+Euphrates release
+-----------------
+
+Test results per scenario and pod (higher is better):
+
+{
+
+ "os-nosdn-nofeature-noha:intel-pod18:joid": [20113.395],
+
+ "os-nosdn-openbaton-ha:intel-pod18:joid": [19183.58],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [17851.35],
+
+ "os-nosdn-nofeature-noha:intel-pod5:joid": [16312.37],
+
+ "os-nosdn-nofeature-ha:intel-pod5:joid": [15633.245],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [13332.065],
+
+ "os-odl-nofeature-ha:arm-pod6:fuel": [13327.02],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [9462.74],
+
+ "os-nosdn-nofeature-ha:flex-pod2:apex": [9384.585],
+
+ "os-odl-nofeature-ha:ericsson-pod1:fuel": [9235.98],
+
+ "os-nosdn-nofeature-noha:huawei-pod12:joid": [9213.6],
+
+ "os-nosdn-openbaton-ha:huawei-pod12:joid": [9152.18],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [9079.45],
+
+ "os-odl_l2-moon-ha:huawei-pod2:compass": [9071.13],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [9068.06],
+
+ "os-odl-sfc-ha:huawei-pod2:compass": [9031.24],
+
+ "os-odl_l3-nofeature-ha:huawei-pod2:compass": [9019.53],
+
+ "os-nosdn-bar-ha:huawei-pod2:compass": [8977.3],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-pod2:compass": [8960.635],
+
+ "os-nosdn-nofeature-ha:huawei-virtual9:compass": [8825.805],
+
+ "os-nosdn-kvm-ha:huawei-pod2:compass": [8282.75],
+
+ "os-odl_l2-moon-noha:huawei-virtual4:compass": [8116.33],
+
+ "os-nosdn-ovs-noha:ericsson-virtual4:fuel": [8083.97],
+
+ "os-odl_l2-moon-noha:huawei-virtual3:compass": [8083.52],
+
+ "os-nosdn-nofeature-noha:huawei-virtual3:compass": [7799.145],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual3:compass": [7776.12],
+
+ "os-nosdn-ovs_dpdk-noha:huawei-virtual3:compass": [7680.37],
+
+ "os-nosdn-ovs-noha:ericsson-virtual1:fuel": [7615.97],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual4:fuel": [7612.62],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual2:fuel": [7518.62],
+
+ "os-nosdn-nofeature-noha:huawei-virtual2:compass": [7489.67],
+
+ "os-nosdn-ovs-noha:ericsson-virtual2:fuel": [7478.57],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-virtual4:compass": [7465.82],
+
+ "os-nosdn-kvm-noha:huawei-virtual3:compass": [7443.16],
+
+ "os-odl-nofeature-noha:ericsson-virtual4:fuel": [7442.855],
+
+ "os-nosdn-nofeature-ha:arm-pod5:fuel": [7440.65],
+
+ "os-odl-sfc-noha:huawei-virtual4:compass": [7401.16],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [7389.505],
+
+ "os-odl-nofeature-ha:arm-pod5:fuel": [7385.76],
+
+ "os-nosdn-nofeature-noha:huawei-virtual1:compass": [7382.345],
+
+ "os-odl_l2-moon-ha:huawei-virtual3:compass": [7286.385],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual4:compass": [7272.06],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual4:compass": [7261.73],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual3:fuel": [7253.64],
+
+ "os-odl-sfc-noha:huawei-virtual3:compass": [7247.89],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual2:compass": [7214.01],
+
+ "os-nosdn-ovs_dpdk-ha:huawei-virtual3:compass": [7207.39],
+
+ "os-nosdn-ovs_dpdk-noha:huawei-virtual4:compass": [7205.565],
+
+ "os-nosdn-ovs-noha:ericsson-virtual3:fuel": [7201.005],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [7132.835],
+
+ "os-odl-nofeature-noha:ericsson-virtual3:fuel": [7117.05],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual3:compass": [7064.18],
+
+ "os-odl_l2-moon-ha:huawei-virtual4:compass": [6997.295],
+
+ "os-odl-nofeature-ha:lf-pod1:apex": [6992.21],
+
+ "os-odl-sfc-ha:huawei-virtual4:compass": [6975.63],
+
+ "os-odl-nofeature-noha:lf-pod1:apex": [6972.63],
+
+ "os-nosdn-nofeature-noha:lf-pod1:apex": [6955],
+
+ "os-ovn-nofeature-noha:lf-pod1:apex": [6954.5],
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [6953.35],
+
+ "os-odl-sfc-noha:lf-pod1:apex": [6951.89],
+
+ "os-nosdn-nofeature-ha:huawei-virtual2:compass": [6932.29],
+
+ "os-nosdn-nofeature-noha:huawei-virtual4:compass": [6929.54],
+
+ "os-nosdn-kvm-noha:huawei-virtual4:compass": [6921.6],
+
+ "os-nosdn-ovs-ha:lf-pod2:fuel": [6913.355],
+
+ "os-odl-nofeature-ha:lf-pod2:fuel": [6848.58],
+
+ "os-odl-sfc-ha:lf-pod1:apex": [6818.74],
+
+ "os-nosdn-bar-noha:lf-pod1:apex": [6812.16],
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [6808.18],
+
+ "os-odl-nofeature-noha:ericsson-virtual2:fuel": [6807.565],
+
+ "os-nosdn-bar-ha:lf-pod1:apex": [6774.76],
+
+ "os-nosdn-bar-ha:huawei-virtual4:compass": [6759.4],
+
+ "os-nosdn-nofeature-noha:huawei-virtual8:compass": [6756.9],
+
+ "os-nosdn-bar-ha:huawei-virtual3:compass": [6543.46],
+
+ "os-nosdn-kvm-ha:huawei-virtual3:compass": [6504.34],
+
+ "os-odl-sfc-ha:huawei-virtual3:compass": [6481.005],
+
+ "os-nosdn-kvm-ha:huawei-virtual4:compass": [6461.5],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [6152.375],
+
+ "os-odl-sfc-ha:huawei-virtual8:compass": [5941.7],
+
+ "os-nosdn-kvm-noha:huawei-virtual8:compass": [4564.515]
+
+}
+
+
+The influence of the scenario
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc069_scenario.png
+ :width: 800px
+ :alt: TC069 influence of scenario
+
+{
+
+ "os-nosdn-openbaton-ha": [9187.16],
+
+ "os-odl_l2-moon-ha": [9010.57],
+
+ "os-nosdn-nofeature-ha": [8886.75],
+
+ "os-odl_l3-nofeature-ha": [8779.67],
+
+ "os-odl_l2-moon-noha": [8114.995],
+
+ "os-nosdn-ovs_dpdk-ha": [7864.07],
+
+ "os-odl_l3-nofeature-noha": [7632.11],
+
+ "os-odl-sfc-ha": [7624.67],
+
+ "os-nosdn-nofeature-noha": [7470.66],
+
+ "os-odl-nofeature-ha": [7372.23],
+
+ "os-nosdn-ovs_dpdk-noha": [7311.54],
+
+ "os-odl-sfc-noha": [7300.56],
+
+ "os-nosdn-ovs-noha": [7280.005],
+
+ "os-odl-nofeature-noha": [7162.67],
+
+ "os-nosdn-kvm-ha": [7130.775],
+
+ "os-nosdn-kvm-noha": [7041.13],
+
+ "os-ovn-nofeature-noha": [6954.5],
+
+ "os-nosdn-ovs-ha": [6913.355],
+
+ "os-nosdn-bar-ha": [6829.17],
+
+ "os-nosdn-bar-noha": [6812.16]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc069_pod.png
+ :width: 800px
+ :alt: TC069 influence of the POD
+
+{
+
+ "intel-pod18": [18871.79],
+
+ "intel-pod5": [16055.79],
+
+ "arm-pod6": [13327.02],
+
+ "flex-pod2": [9384.585],
+
+ "ericsson-pod1": [9331.535],
+
+ "huawei-pod12": [9164.88],
+
+ "huawei-pod2": [9026.52],
+
+ "huawei-virtual9": [8825.805],
+
+ "ericsson-virtual1": [7615.97],
+
+ "ericsson-virtual4": [7539.23],
+
+ "arm-pod5": [7403.38],
+
+ "huawei-virtual3": [7247.89],
+
+ "huawei-virtual2": [7205.35],
+
+ "huawei-virtual1": [7196.405],
+
+ "ericsson-virtual3": [7173.72],
+
+ "huawei-virtual4": [7131.47],
+
+ "ericsson-virtual2": [7129.08],
+
+ "lf-pod1": [6928.18],
+
+ "lf-pod2": [6875.88],
+
+ "huawei-virtual8": [5729.705]
+
+}
+
+
+Fraser release
+--------------
+
+Test results per scenario and pod (higher is better):
+
+{
+
+ "os-nosdn-nofeature-noha:intel-pod18:joid": [18382.49],
+
+ "os-nosdn-openbaton-ha:intel-pod18:joid": [16774.52],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [16680.305],
+
+ "os-nosdn-ovs-ha:arm-pod6:fuel": [11925.22],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [11895.71],
+
+ "os-odl-nofeature-ha:arm-pod6:fuel": [11880.7],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [9471.095],
+
+ "os-odl-nofeature-ha:zte-pod2:daisy": [9375.33],
+
+ "os-nosdn-nofeature-ha:zte-pod2:daisy": [9372.95],
+
+ "os-odl-nofeature-ha:ericsson-pod1:fuel": [9174.36],
+
+ "os-nosdn-nofeature-noha:huawei-pod12:joid": [9051.57],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [8894.74],
+
+ "os-odl_l3-nofeature-ha:huawei-pod2:compass": [8857.23],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [8855.8],
+
+ "os-nosdn-bar-ha:huawei-pod2:compass": [8840.94],
+
+ "os-odl-sfc-ha:huawei-pod2:compass": [8826.23],
+
+ "os-nosdn-nofeature-noha:huawei-virtual4:compass": [8039.48],
+
+ "os-nosdn-nofeature-noha:huawei-virtual2:compass": [7670.21],
+
+ "os-nosdn-ovs-ha:arm-pod5:fuel": [7590.9],
+
+ "os-odl-sfc-noha:huawei-virtual4:compass": [7579.625],
+
+ "os-nosdn-bar-noha:huawei-virtual3:compass": [7511.775],
+
+ "os-odl-nofeature-ha:arm-pod5:fuel": [7475.16],
+
+ "os-nosdn-bar-noha:huawei-virtual4:compass": [7435.08],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual2:fuel": [7426.79],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [7362.8],
+
+ "os-nosdn-kvm-noha:huawei-virtual4:compass": [7263.45],
+
+ "os-nosdn-nofeature-noha:huawei-virtual3:compass": [7262.72],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual3:compass": [7241.07],
+
+ "os-odl-nofeature-noha:ericsson-virtual2:fuel": [7219.21],
+
+ "os-nosdn-kvm-noha:huawei-virtual3:compass": [7174.33],
+
+ "os-odl-sfc-noha:huawei-virtual3:compass": [7170.795],
+
+ "os-odl-nofeature-noha:lf-pod1:apex": [7158.335],
+
+ "os-nosdn-kvm-ha:huawei-pod2:compass": [7122.45],
+
+ "os-odl-sfc-ha:huawei-virtual4:compass": [7104.9],
+
+ "os-nosdn-ovs-noha:ericsson-virtual2:fuel": [7044.37],
+
+ "os-nosdn-bar-ha:huawei-virtual3:compass": [7011.075],
+
+ "os-nosdn-ovs-ha:ericsson-pod1:fuel": [6950.28],
+
+ "os-nosdn-ovs-noha:ericsson-virtual4:fuel": [6918.31],
+
+ "os-nosdn-bar-ha:huawei-virtual4:compass": [6903.11],
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [6880.98],
+
+ "os-odl-sfc-ha:lf-pod1:apex": [6863.39],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual3:compass": [6851.54],
+
+ "os-nosdn-nofeature-noha:lf-pod1:apex": [6834.75],
+
+ "os-nosdn-calipso-noha:lf-pod1:apex": [6833.92],
+
+ "os-nosdn-ovs-ha:lf-pod2:fuel": [6814.68],
+
+ "os-ovn-nofeature-noha:lf-pod1:apex": [6809.44],
+
+ "os-odl_l3-nofeature-ha:huawei-virtual4:compass": [6784.48],
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [6737.64],
+
+ "os-nosdn-bar-noha:lf-pod1:apex": [6708.61],
+
+ "os-nosdn-bar-ha:lf-pod1:apex": [6697.2],
+
+ "os-odl-nofeature-ha:lf-pod1:apex": [6626.51],
+
+ "os-odl-sfc-noha:lf-pod1:apex": [6609.57],
+
+ "os-odl-sfc-ha:huawei-virtual3:compass": [6606.87],
+
+ "os-odl_l3-nofeature-noha:huawei-virtual4:compass": [6547.39],
+
+ "os-odl-nofeature-ha:lf-pod2:fuel": [6465.48],
+
+ "os-odl-nofeature-noha:ericsson-virtual4:fuel": [6413],
+
+ "os-nosdn-kvm-ha:huawei-virtual4:compass": [6409.075],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [6128.79],
+
+ "os-nosdn-nofeature-noha:ericsson-virtual3:fuel": [5835.59],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [5617.12]
+
+}
+
+
+The influence of the scenario
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc069_scenario_fraser.png
+ :width: 800px
+ :alt: TC069 influence of scenario
+
+{
+
+ "os-nosdn-openbaton-ha": [16774.52],
+
+ "os-odl-nofeature-ha": [9363.69],
+
+ "os-nosdn-nofeature-ha": [8878.01],
+
+ "os-odl_l3-nofeature-ha": [8748.4],
+
+ "os-odl-sfc-ha": [8708.045],
+
+ "os-nosdn-nofeature-noha": [7426.79],
+
+ "os-nosdn-kvm-noha": [7230.79],
+
+ "os-odl-sfc-noha": [7224.11],
+
+ "os-odl-nofeature-noha": [7187.84],
+
+ "os-nosdn-ovs-noha": [7044.37],
+
+ "os-nosdn-bar-ha": [6947.87],
+
+ "os-odl_l3-nofeature-noha": [6895.96],
+
+ "os-nosdn-kvm-ha": [6890.92],
+
+ "os-nosdn-calipso-noha": [6833.92],
+
+ "os-nosdn-ovs-ha": [6833.495],
+
+ "os-nosdn-bar-noha": [6811.66],
+
+ "os-ovn-nofeature-noha": [6809.44]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc069_pod_fraser.png
+ :width: 800px
+ :alt: TC069 influence of the POD
+
+{
+
+ "intel-pod18": [16939.24],
+
+ "arm-pod6": [11895.71],
+
+ "zte-pod2": [9375.33],
+
+ "ericsson-pod1": [9140.42],
+
+ "huawei-pod12": [8993.37],
+
+ "huawei-pod2": [8794.01],
+
+ "huawei-virtual2": [7670.21],
+
+ "arm-pod5": [7479.32],
+
+ "ericsson-virtual2": [7219.21],
+
+ "huawei-virtual4": [7059.045],
+
+ "huawei-virtual3": [7023.57],
+
+ "lf-pod2": [6834.7],
+
+ "lf-pod1": [6775.27],
+
+ "ericsson-virtual4": [6522.86],
+
+ "ericsson-virtual3": [5835.59],
+
+ "huawei-virtual1": [5617.12]
+
+}
diff --git a/docs/release/results/tc082-context-switches-under-load.rst b/docs/release/results/tc082-context-switches-under-load.rst
new file mode 100644
index 000000000..92bc69907
--- /dev/null
+++ b/docs/release/results/tc082-context-switches-under-load.rst
@@ -0,0 +1,187 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+
+
+==================================================
+Test results for TC082 context switches under load
+==================================================
+
+.. toctree::
+ :maxdepth: 2
+
+
+Overview of test case
+=====================
+
+TC082 measures various software performance events using perf.
+The test results shown below are for context-switches.
+
+Metric: context switches
+Unit: N/A
+
+
+Euphrates release
+-----------------
+
+Test results per scenario and pod (lower is better):
+
+{
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [316],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [340],
+
+ "os-nosdn-nofeature-ha:intel-pod5:joid": [357.5],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [384],
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [394.5],
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [435],
+
+ "os-nosdn-nofeature-ha:flex-pod2:apex": [476],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [518],
+
+ "os-odl-nofeature-ha:arm-pod5:fuel": [863],
+
+ "os-nosdn-nofeature-ha:arm-pod5:fuel": [871],
+
+ "os-nosdn-nofeature-ha:huawei-virtual9:compass": [1002],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [1174],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [1239],
+
+ "os-nosdn-nofeature-ha:huawei-virtual2:compass": [1430],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [1489],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [1883.5]
+
+}
+
+
+The influence of the scenario
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc082_scenario.png
+ :width: 800px
+ :alt: TC082 influence of scenario
+
+{
+
+ "os-nosdn-nofeature-ha": [505],
+
+ "os-odl-nofeature-ha": [863]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc082_pod.png
+ :width: 800px
+ :alt: TC082 influence of the POD
+
+{
+
+ "huawei-pod12": [316],
+
+ "intel-pod18": [340],
+
+ "intel-pod5": [357.5],
+
+ "ericsson-pod1": [384],
+
+ "lf-pod2": [394.5],
+
+ "lf-pod1": [435],
+
+ "flex-pod2": [476],
+
+ "huawei-pod2": [518],
+
+ "arm-pod5": [869.5],
+
+ "huawei-virtual9": [1002],
+
+ "huawei-virtual4": [1174],
+
+ "huawei-virtual3": [1239],
+
+ "huawei-virtual2": [1430],
+
+ "huawei-virtual1": [1489],
+
+ "arm-pod6": [1883.5]
+
+}
+
+
+Fraser release
+--------------
+
+Test results per scenario and pod (lower is better):
+
+{
+
+ "os-nosdn-nofeature-ha:zte-pod2:daisy": [306.5],
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [337.5],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [343.5],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [399],
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [454],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [544.5],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [1138],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [1305],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [1433],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [1470],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [1738.5]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc082_pod_fraser.png
+ :width: 800px
+ :alt: TC082 influence of the POD
+
+{
+
+ "zte-pod2": [306.5],
+
+ "lf-pod2": [337.5],
+
+ "intel-pod18": [343.5],
+
+ "huawei-pod12": [399],
+
+ "lf-pod1": [454],
+
+ "huawei-pod2": [544.5],
+
+ "huawei-virtual4": [1138],
+
+ "ericsson-pod1": [1305],
+
+ "huawei-virtual3": [1433],
+
+ "huawei-virtual1": [1470],
+
+ "arm-pod6": [1738.5]
+
+}
diff --git a/docs/release/results/tc083-network-throughput-between-vm.rst b/docs/release/results/tc083-network-throughput-between-vm.rst
new file mode 100644
index 000000000..0389eaafe
--- /dev/null
+++ b/docs/release/results/tc083-network-throughput-between-vm.rst
@@ -0,0 +1,187 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+
+
+=====================================================
+Test results for TC083 network throughput between VMs
+=====================================================
+
+.. toctree::
+ :maxdepth: 2
+
+
+Overview of test case
+=====================
+
+TC083 measures network latency and throughput between VMs using netperf.
+The test results shown below are for UDP throughout.
+
+Metric: UDP stream throughput
+Unit: 10^6bits/s
+
+
+Euphrates release
+-----------------
+
+Test results per scenario and pod (higher is better):
+
+{
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [2204.42],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [1835.55],
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [1676.705],
+
+ "os-nosdn-nofeature-ha:intel-pod5:joid": [1612.555],
+
+ "os-nosdn-nofeature-ha:flex-pod2:apex": [1370.23],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [1300.12],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [1070.455],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [1004.32],
+
+ "os-nosdn-nofeature-ha:huawei-virtual9:compass": [753.46],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [735.07],
+
+ "os-odl-nofeature-ha:arm-pod5:fuel": [531.63],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [493.985],
+
+ "os-nosdn-nofeature-ha:arm-pod5:fuel": [448.82],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [193.43],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [189.99],
+
+ "os-nosdn-nofeature-ha:huawei-virtual2:compass": [80.15]
+
+}
+
+
+The influence of the scenario
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc083_scenario.png
+ :width: 800px
+ :alt: TC083 influence of scenario
+
+{
+
+ "os-nosdn-nofeature-ha": [1109.12],
+
+ "os-odl-nofeature-ha": [531.63]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc083_pod.png
+ :width: 800px
+ :alt: TC083 influence of the POD
+
+{
+
+ "lf-pod1": [2204.42],
+
+ "intel-pod18": [1835.55],
+
+ "lf-pod2": [1676.705],
+
+ "intel-pod5": [1612.555],
+
+ "flex-pod2": [1370.23],
+
+ "huawei-pod12": [1300.12],
+
+ "huawei-pod2": [1070.455],
+
+ "ericsson-pod1": [1004.32],
+
+ "huawei-virtual9": [753.46],
+
+ "huawei-virtual4": [735.07],
+
+ "huawei-virtual3": [493.985],
+
+ "arm-pod5": [451.38],
+
+ "arm-pod6": [193.43],
+
+ "huawei-virtual1": [189.99],
+
+ "huawei-virtual2": [80.15]
+
+}
+
+
+Fraser release
+--------------
+
+Test results per scenario and pod (higher is better):
+
+{
+
+ "os-nosdn-nofeature-ha:lf-pod2:fuel": [1893.39],
+
+ "os-nosdn-nofeature-ha:zte-pod2:daisy": [1543.995],
+
+ "os-nosdn-nofeature-ha:lf-pod1:apex": [1480.86],
+
+ "os-nosdn-nofeature-ha:intel-pod18:joid": [1417.015],
+
+ "os-nosdn-nofeature-ha:huawei-pod12:joid": [1028.55],
+
+ "os-nosdn-nofeature-ha:huawei-pod2:compass": [1007.65],
+
+ "os-nosdn-nofeature-ha:ericsson-pod1:fuel": [811.795],
+
+ "os-nosdn-nofeature-ha:huawei-virtual4:compass": [552.95],
+
+ "os-nosdn-nofeature-ha:arm-pod6:fuel": [227.655],
+
+ "os-nosdn-nofeature-ha:huawei-virtual1:compass": [216.63],
+
+ "os-nosdn-nofeature-ha:huawei-virtual3:compass": [59.255]
+
+}
+
+
+The influence of the POD
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. image:: images/tc083_pod_fraser.png
+ :width: 800px
+ :alt: TC083 influence of the POD
+
+{
+
+ "lf-pod2": [1893.39],
+
+ "zte-pod2": [1543.995],
+
+ "lf-pod1": [1480.86],
+
+ "intel-pod18": [1417.015],
+
+ "huawei-pod12": [1028.55],
+
+ "huawei-pod2": [1007.65],
+
+ "ericsson-pod1": [811.795],
+
+ "huawei-virtual4": [552.95],
+
+ "arm-pod6": [227.655],
+
+ "huawei-virtual1": [216.63],
+
+ "huawei-virtual3": [59.255]
+
+}
diff --git a/docs/testing/developer/devguide/devguide.rst b/docs/testing/developer/devguide/devguide.rst
index dade49b75..04d5350be 100755
--- a/docs/testing/developer/devguide/devguide.rst
+++ b/docs/testing/developer/devguide/devguide.rst
@@ -432,7 +432,8 @@ Yardstick committers and contributors to review your codes.
:width: 800px
:alt: Gerrit for code review
-You can find Yardstick people info `here <https://wiki.opnfv.org/display/yardstick/People>`_.
+You can find a list Yardstick people `here <https://wiki.opnfv.org/display/yardstick/People>`_,
+or use the ``yardstick-reviewers`` and ``yardstick-committers`` groups in gerrit.
Modify the code under review in Gerrit
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/docs/testing/user/userguide/13-nsb-installation.rst b/docs/testing/user/userguide/13-nsb-installation.rst
index 00f8cfd97..3e0ed0bfb 100644
--- a/docs/testing/user/userguide/13-nsb-installation.rst
+++ b/docs/testing/user/userguide/13-nsb-installation.rst
@@ -135,6 +135,15 @@ Ansible:
ansible_user=root
ansible_pass=root
+.. note::
+
+ SSH access without password needs to be configured for all your nodes defined in
+ ``yardstick-install-inventory.ini`` file.
+ If you want to use password authentication you need to install sshpass
+
+ .. code-block:: console
+
+ sudo -EH apt-get install sshpass
To execute an installation for a Bare-Metal or a Standalone context:
@@ -1049,40 +1058,8 @@ IxLoad
Config ``pod_ixia.yaml``
- .. code-block:: yaml
-
- nodes:
- -
- name: trafficgen_1
- role: IxNet
- ip: 1.2.1.1 #ixia machine ip
- user: user
- password: r00t
- key_filename: /root/.ssh/id_rsa
- tg_config:
- ixchassis: "1.2.1.7" #ixia chassis ip
- tcl_port: "8009" # tcl server port
- lib_path: "/opt/ixia/ixos-api/8.01.0.2/lib/ixTcl1.0"
- root_dir: "/opt/ixia/ixos-api/8.01.0.2/"
- py_bin_path: "/opt/ixia/ixload/8.01.106.3/bin/"
- py_lib_path: "/opt/ixia/ixnetwork/8.01.1029.14/lib/PythonApi"
- dut_result_dir: "/mnt/ixia"
- version: 8.1
- interfaces:
- xe0: # logical name from topology.yaml and vnfd.yaml
- vpci: "2:5" # Card:port
- driver: "none"
- dpdk_port_num: 0
- local_ip: "152.16.100.20"
- netmask: "255.255.0.0"
- local_mac: "00:98:10:64:14:00"
- xe1: # logical name from topology.yaml and vnfd.yaml
- vpci: "2:6" # [(Card, port)]
- driver: "none"
- dpdk_port_num: 1
- local_ip: "152.40.40.20"
- netmask: "255.255.0.0"
- local_mac: "00:98:28:28:14:00"
+ .. literalinclude:: code/pod_ixia.yaml
+ :language: console
for sriov/ovs_dpdk pod files, please refer to above Standalone Virtualization for ovs-dpdk/sriov configuration
@@ -1104,10 +1081,10 @@ IxLoad
IxNetwork
---------
-1. Software needed: ``IxNetworkAPI<ixnetwork verson>Linux64.bin.tgz``
- (Download from ixia support site)
- Install - ``IxNetworkAPI<ixnetwork verson>Linux64.bin.tgz``
-2. Update pod_ixia.yaml file with ixia details.
+IxNetwork testcases use IxNetwork API Python Bindings module, which is
+installed as part of the requirements of the project.
+
+1. Update ``pod_ixia.yaml`` file with ixia details.
.. code-block:: console
@@ -1115,44 +1092,12 @@ IxNetwork
Config pod_ixia.yaml
- .. code-block:: yaml
-
- nodes:
- -
- name: trafficgen_1
- role: IxNet
- ip: 1.2.1.1 #ixia machine ip
- user: user
- password: r00t
- key_filename: /root/.ssh/id_rsa
- tg_config:
- ixchassis: "1.2.1.7" #ixia chassis ip
- tcl_port: "8009" # tcl server port
- lib_path: "/opt/ixia/ixos-api/8.01.0.2/lib/ixTcl1.0"
- root_dir: "/opt/ixia/ixos-api/8.01.0.2/"
- py_bin_path: "/opt/ixia/ixload/8.01.106.3/bin/"
- py_lib_path: "/opt/ixia/ixnetwork/8.01.1029.14/lib/PythonApi"
- dut_result_dir: "/mnt/ixia"
- version: 8.1
- interfaces:
- xe0: # logical name from topology.yaml and vnfd.yaml
- vpci: "2:5" # Card:port
- driver: "none"
- dpdk_port_num: 0
- local_ip: "152.16.100.20"
- netmask: "255.255.0.0"
- local_mac: "00:98:10:64:14:00"
- xe1: # logical name from topology.yaml and vnfd.yaml
- vpci: "2:6" # [(Card, port)]
- driver: "none"
- dpdk_port_num: 1
- local_ip: "152.40.40.20"
- netmask: "255.255.0.0"
- local_mac: "00:98:28:28:14:00"
+ .. literalinclude:: code/pod_ixia.yaml
+ :language: console
for sriov/ovs_dpdk pod files, please refer to above Standalone Virtualization for ovs-dpdk/sriov configuration
-3. Start IxNetwork TCL Server
+2. Start IxNetwork TCL Server
You will also need to configure the IxNetwork machine to start the IXIA
IxNetworkTclServer. This can be started like so:
@@ -1161,6 +1106,5 @@ IxNetwork
``Start->Programs->Ixia->IxNetwork->IxNetwork 7.21.893.14 GA->IxNetworkTclServer``
(or ``IxNetworkApiServer``)
-4. Execute testcase in samplevnf folder e.g.
+3. Execute testcase in samplevnf folder e.g.
``<repo>/samples/vnf_samples/nsut/vfw/tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml``
-
diff --git a/docs/testing/user/userguide/14-nsb-operation.rst b/docs/testing/user/userguide/14-nsb-operation.rst
index 250f0a8e1..851c6528e 100644
--- a/docs/testing/user/userguide/14-nsb-operation.rst
+++ b/docs/testing/user/userguide/14-nsb-operation.rst
@@ -423,3 +423,39 @@ options section.
options:
tg_0:
queues_per_port: 2
+
+
+Standalone configuration
+------------------------
+
+NSB supports certain Standalone deployment configurations.
+Standalone supports provisioning a VM in a standalone visualised environment using kvm/qemu.
+There two types of Standalone contexts available: OVS-DPDK and SRIOV.
+OVS-DPDK uses OVS network with DPDK drivers.
+SRIOV enables network traffic to bypass the software switch layer of the Hyper-V stack.
+
+Standalone with OVS-DPDK
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+SampleVNF image is spawned in a VM on a baremetal server.
+OVS with DPDK is installed on the baremetal server.
+
+.. note:: Ubuntu 17.10 requires DPDK v.17.05 and higher, DPDK v.17.05 requires OVS v.2.8.0.
+
+Default values for OVS-DPDK:
+
+ * queues: 4
+ * lcore_mask: ""
+ * pmd_cpu_mask: "0x6"
+
+Sample test case file
+^^^^^^^^^^^^^^^^^^^^^
+
+ 1. Prepare SampleVNF image and copy it to ``flavor/images``.
+ 2. Prepare context files for TREX and SampleVNF under ``contexts/file``.
+ 3. Add bridge named ``br-int`` to the baremetal where SampleVNF image is deployed.
+ 4. Modify ``networks/phy_port`` accordingly to the baremetal setup.
+ 5. Run test from:
+
+.. literalinclude:: /submodules/yardstick/samples/vnf_samples/nsut/acl/tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml
+ :language: yaml
diff --git a/docs/testing/user/userguide/15-list-of-tcs.rst b/docs/testing/user/userguide/15-list-of-tcs.rst
index 678f0f9a9..37ce819f1 100644
--- a/docs/testing/user/userguide/15-list-of-tcs.rst
+++ b/docs/testing/user/userguide/15-list-of-tcs.rst
@@ -84,6 +84,8 @@ H A
opnfv_yardstick_tc057.rst
opnfv_yardstick_tc058.rst
opnfv_yardstick_tc087.rst
+ opnfv_yardstick_tc092.rst
+ opnfv_yardstick_tc093.rst
IPv6
----
diff --git a/docs/testing/user/userguide/code/pod_ixia.yaml b/docs/testing/user/userguide/code/pod_ixia.yaml
new file mode 100644
index 000000000..4ab56fe4e
--- /dev/null
+++ b/docs/testing/user/userguide/code/pod_ixia.yaml
@@ -0,0 +1,31 @@
+nodes:
+-
+ name: trafficgen_1
+ role: IxNet
+ ip: 1.2.1.1 #ixia machine ip
+ user: user
+ password: r00t
+ key_filename: /root/.ssh/id_rsa
+ tg_config:
+ ixchassis: "1.2.1.7" #ixia chassis ip
+ tcl_port: "8009" # tcl server port
+ lib_path: "/opt/ixia/ixos-api/8.01.0.2/lib/ixTcl1.0"
+ root_dir: "/opt/ixia/ixos-api/8.01.0.2/"
+ py_bin_path: "/opt/ixia/ixload/8.01.106.3/bin/"
+ dut_result_dir: "/mnt/ixia"
+ version: 8.1
+ interfaces:
+ xe0: # logical name from topology.yaml and vnfd.yaml
+ vpci: "2:5" # Card:port
+ driver: "none"
+ dpdk_port_num: 0
+ local_ip: "152.16.100.20"
+ netmask: "255.255.0.0"
+ local_mac: "00:98:10:64:14:00"
+ xe1: # logical name from topology.yaml and vnfd.yaml
+ vpci: "2:6" # [(Card, port)]
+ driver: "none"
+ dpdk_port_num: 1
+ local_ip: "152.40.40.20"
+ netmask: "255.255.0.0"
+ local_mac: "00:98:28:28:14:00"
diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc050.rst b/docs/testing/user/userguide/opnfv_yardstick_tc050.rst
index 8890c9d53..82a491b72 100644
--- a/docs/testing/user/userguide/opnfv_yardstick_tc050.rst
+++ b/docs/testing/user/userguide/opnfv_yardstick_tc050.rst
@@ -34,23 +34,20 @@ Yardstick Test Case Description TC050
| | 2) host: which is the name of a control node being attacked. |
| | 3) interface: the network interface to be turned off. |
| | |
-| | There are four instance of the "close-interface" monitor: |
-| | attacker1(for public netork): |
-| | -fault_type: "close-interface" |
-| | -host: node1 |
-| | -interface: "br-ex" |
-| | attacker2(for management netork): |
-| | -fault_type: "close-interface" |
-| | -host: node1 |
-| | -interface: "br-mgmt" |
-| | attacker3(for storage netork): |
-| | -fault_type: "close-interface" |
-| | -host: node1 |
-| | -interface: "br-storage" |
-| | attacker4(for private netork): |
-| | -fault_type: "close-interface" |
-| | -host: node1 |
-| | -interface: "br-mesh" |
+| | The interface to be closed by the attacker can be set by the |
+| | variable of "{{ interface_name }}" |
+| | |
+| | attackers: |
+| | - |
+| | fault_type: "general-attacker" |
+| | host: {{ attack_host }} |
+| | key: "close-br-public" |
+| | attack_key: "close-interface" |
+| | action_parameter: |
+| | interface: {{ interface_name }} |
+| | rollback_parameter: |
+| | interface: {{ interface_name }} |
+| | |
+--------------+--------------------------------------------------------------+
|monitors | In this test case, the monitor named "openstack-cmd" is |
| | needed. The monitor needs needs two parameters: |
@@ -61,17 +58,17 @@ Yardstick Test Case Description TC050
| | |
| | There are four instance of the "openstack-cmd" monitor: |
| | monitor1: |
-| | -monitor_type: "openstack-cmd" |
-| | -command_name: "nova image-list" |
+| | - monitor_type: "openstack-cmd" |
+| | - command_name: "nova image-list" |
| | monitor2: |
-| | -monitor_type: "openstack-cmd" |
-| | -command_name: "neutron router-list" |
+| | - monitor_type: "openstack-cmd" |
+| | - command_name: "neutron router-list" |
| | monitor3: |
-| | -monitor_type: "openstack-cmd" |
-| | -command_name: "heat stack-list" |
+| | - monitor_type: "openstack-cmd" |
+| | - command_name: "heat stack-list" |
| | monitor4: |
-| | -monitor_type: "openstack-cmd" |
-| | -command_name: "cinder list" |
+| | - monitor_type: "openstack-cmd" |
+| | - command_name: "cinder list" |
+--------------+--------------------------------------------------------------+
|metrics | In this test case, there is one metric: |
| | 1)service_outage_time: which indicates the maximum outage |
@@ -109,9 +106,9 @@ Yardstick Test Case Description TC050
+--------------+--------------------------------------------------------------+
|step 2 | do attacker: connect the host through SSH, and then execute |
| | the turnoff network interface script with param value |
-| | specified by "interface". |
+| | specified by "{{ interface_name }}". |
| | |
-| | Result: Network interfaces will be turned down. |
+| | Result: The specified network interface will be down. |
| | |
+--------------+--------------------------------------------------------------+
|step 3 | stop monitors after a period of time specified by |
@@ -133,3 +130,4 @@ Yardstick Test Case Description TC050
| | execution problem. |
| | |
+--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc088.rst b/docs/testing/user/userguide/opnfv_yardstick_tc088.rst
new file mode 100644
index 000000000..2423a6b31
--- /dev/null
+++ b/docs/testing/user/userguide/opnfv_yardstick_tc088.rst
@@ -0,0 +1,129 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Yin Kanglin and others.
+.. 14_ykl@tongji.edu.cn
+
+*************************************
+Yardstick Test Case Description TC088
+*************************************
+
++-----------------------------------------------------------------------------+
+|Control Node Openstack Service High Availability - Nova Scheduler |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC088: Control node Openstack service down - |
+| | nova scheduler |
++--------------+--------------------------------------------------------------+
+|test purpose | This test case will verify the high availability of the |
+| | compute scheduler service provided by OpenStack (nova- |
+| | scheduler) on control node. |
+| | |
++--------------+--------------------------------------------------------------+
+|test method | This test case kills the processes of nova-scheduler service |
+| | on a selected control node, then checks whether the request |
+| | of the related OpenStack command is OK and the killed |
+| | processes are recovered. |
+| | |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, an attacker called "kill-process" is |
+| | needed. This attacker includes three parameters: |
+| | 1) fault_type: which is used for finding the attacker's |
+| | scripts. It should be always set to "kill-process" in this |
+| | test case. |
+| | 2) process_name: which is the process name of the specified |
+| | OpenStack service. If there are multiple processes use the |
+| | same name on the host, all of them are killed by this |
+| | attacker. |
+| | In this case. This parameter should always set to "nova- |
+| | scheduler". |
+| | 3) host: which is the name of a control node being attacked. |
+| | |
+| | e.g. |
+| | -fault_type: "kill-process" |
+| | -process_name: "nova-scheduler" |
+| | -host: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|monitors | In this test case, one kind of monitor is needed: |
+| | 1. the "process" monitor check whether a process is running |
+| | on a specific node, which needs three parameters: |
+| | 1) monitor_type: which used for finding the monitor class and|
+| | related scripts. It should be always set to "process" |
+| | for this monitor. |
+| | 2) process_name: which is the process name for monitor |
+| | 3) host: which is the name of the node running the process |
+| | |
+| | e.g. |
+| | monitor: |
+| | -monitor_type: "process" |
+| | -process_name: "nova-scheduler" |
+| | -host: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|operations | In this test case, the following operations are needed: |
+| | 1. "nova-create-instance": create a VM instance to check |
+| | whether the nova-scheduler works normally. |
+| | |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there are one metric: |
+| | 1)process_recover_time: which indicates the maximum time |
+| | (seconds) from the process being killed to recovered |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI NFV REL001 |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1) test case file: opnfv_yardstick_tc088.yaml |
+| | -Attackers: see above "attackers" description |
+| | -waiting_time: which is the time (seconds) from the process |
+| | being killed to stopping monitors the monitors |
+| | -Monitors: see above "monitors" description |
+| | -SLA: see above "metrics" description |
+| | |
+| | 2)POD file: pod.yaml |
+| | The POD configuration should record on pod.yaml first. |
+| | the "host" item in this test case will use the node name in |
+| | the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | do attacker: connect the host through SSH, and then execute |
+| | the kill process script with param value specified by |
+| | "process_name" |
+| | |
+| | Result: Process will be killed. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | start monitors: |
+| | each monitor will run with independently process |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | create a new instance to check whether the nova scheduler |
+| | works normally. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | stop the monitor after a period of time specified by |
+| | "waiting_time" |
+| | |
+| | Result: The monitor info will be aggregated. |
+| | |
++--------------+--------------------------------------------------------------+
+|post-action | It is the action when the test cases exist. It will check the|
+| | status of the specified process on the host, and restart the |
+| | process if it is not running for next test cases |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc089.rst b/docs/testing/user/userguide/opnfv_yardstick_tc089.rst
new file mode 100644
index 000000000..0a8b2570b
--- /dev/null
+++ b/docs/testing/user/userguide/opnfv_yardstick_tc089.rst
@@ -0,0 +1,129 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Yin Kanglin and others.
+.. 14_ykl@tongji.edu.cn
+
+*************************************
+Yardstick Test Case Description TC089
+*************************************
+
++-----------------------------------------------------------------------------+
+|Control Node Openstack Service High Availability - Nova Conductor |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC089: Control node Openstack service down - |
+| | nova conductor |
++--------------+--------------------------------------------------------------+
+|test purpose | This test case will verify the high availability of the |
+| | compute database proxy service provided by OpenStack (nova- |
+| | conductor) on control node. |
+| | |
++--------------+--------------------------------------------------------------+
+|test method | This test case kills the processes of nova-conductor service |
+| | on a selected control node, then checks whether the request |
+| | of the related OpenStack command is OK and the killed |
+| | processes are recovered. |
+| | |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, an attacker called "kill-process" is |
+| | needed. This attacker includes three parameters: |
+| | 1) fault_type: which is used for finding the attacker's |
+| | scripts. It should be always set to "kill-process" in this |
+| | test case. |
+| | 2) process_name: which is the process name of the specified |
+| | OpenStack service. If there are multiple processes use the |
+| | same name on the host, all of them are killed by this |
+| | attacker. |
+| | In this case. This parameter should always set to "nova- |
+| | conductor". |
+| | 3) host: which is the name of a control node being attacked. |
+| | |
+| | e.g. |
+| | -fault_type: "kill-process" |
+| | -process_name: "nova-conductor" |
+| | -host: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|monitors | In this test case, one kind of monitor is needed: |
+| | 1. the "process" monitor check whether a process is running |
+| | on a specific node, which needs three parameters: |
+| | 1) monitor_type: which used for finding the monitor class and|
+| | related scripts. It should be always set to "process" |
+| | for this monitor. |
+| | 2) process_name: which is the process name for monitor |
+| | 3) host: which is the name of the node running the process |
+| | |
+| | e.g. |
+| | monitor: |
+| | -monitor_type: "process" |
+| | -process_name: "nova-conductor" |
+| | -host: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|operations | In this test case, the following operations are needed: |
+| | 1. "nova-create-instance": create a VM instance to check |
+| | whether the nova-conductor works normally. |
+| | |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there are one metric: |
+| | 1)process_recover_time: which indicates the maximum time |
+| | (seconds) from the process being killed to recovered |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | ETSI NFV REL001 |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1) test case file: opnfv_yardstick_tc089.yaml |
+| | -Attackers: see above "attackers" description |
+| | -waiting_time: which is the time (seconds) from the process |
+| | being killed to stopping monitors the monitors |
+| | -Monitors: see above "monitors" description |
+| | -SLA: see above "metrics" description |
+| | |
+| | 2)POD file: pod.yaml |
+| | The POD configuration should record on pod.yaml first. |
+| | the "host" item in this test case will use the node name in |
+| | the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | do attacker: connect the host through SSH, and then execute |
+| | the kill process script with param value specified by |
+| | "process_name" |
+| | |
+| | Result: Process will be killed. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | start monitors: |
+| | each monitor will run with independently process |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | create a new instance to check whether the nova conductor |
+| | works normally. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | stop the monitor after a period of time specified by |
+| | "waiting_time" |
+| | |
+| | Result: The monitor info will be aggregated. |
+| | |
++--------------+--------------------------------------------------------------+
+|post-action | It is the action when the test cases exist. It will check the|
+| | status of the specified process on the host, and restart the |
+| | process if it is not running for next test cases |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc092.rst b/docs/testing/user/userguide/opnfv_yardstick_tc092.rst
new file mode 100644
index 000000000..895074a85
--- /dev/null
+++ b/docs/testing/user/userguide/opnfv_yardstick_tc092.rst
@@ -0,0 +1,196 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Ericsson and others.
+
+*************************************
+Yardstick Test Case Description TC092
+*************************************
+
++-----------------------------------------------------------------------------+
+|SDN Controller resilience in HA configuration |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC092: SDN controller resilience and high |
+| | availability HA configuration |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | This test validates SDN controller node high availability by |
+| | verifying there is no impact on the data plane connectivity |
+| | when one SDN controller fails in a HA configuration, |
+| | i.e. all existing configured network services DHCP, ARP, L2, |
+| | L3VPN, Security Groups should continue to operate |
+| | between the existing VMs while one SDN controller instance |
+| | is offline and rebooting. |
+| | |
+| | The test also validates that network service operations such |
+| | as creating a new VM in an existing or new L2 network |
+| | network remain operational while one instance of the |
+| | SDN controller is offline and recovers from the failure. |
+| | |
++--------------+--------------------------------------------------------------+
+|test method | This test case: |
+| | 1. fails one instance of a SDN controller cluster running |
+| | in a HA configuration on the OpenStack controller node |
+| | |
+| | 2. checks if already configured L2 connectivity between |
+| | existing VMs is not impacted |
+| | |
+| | 3. verifies that the system never loses the ability to |
+| | execute virtual network operations, even when the |
+| | failed SDN Controller is still recovering |
+| | |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, an attacker called “kill-process” is |
+| | needed. This attacker includes three parameters: |
+| | 1. ``fault_type``: which is used for finding the attacker's |
+| | scripts. It should be set to 'kill-process' in this test |
+| | |
+| | 2. ``process_name``: should be set to sdn controller |
+| | process |
+| | |
+| | 3. ``host``: which is the name of a control node where |
+| | opendaylight process is running |
+| | |
+| | example: |
+| | - ``fault_type``: “kill-process” |
+| | - ``process_name``: “opendaylight-karaf” (TBD) |
+| | - ``host``: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|monitors | In this test case, the following monitors are needed |
+| | 1. ``ping_same_network_l2``: monitor pinging traffic |
+| | between the VMs in same neutron network |
+| | |
+| | 2. ``ping_external_snat``: monitor ping traffic from VMs to |
+| | external destinations (e.g. google.com) |
+| | |
+| | 3. ``SDN controller process monitor``: a monitor checking |
+| | the state of a specified SDN controller process. It |
+| | measures the recovery time of the given process. |
+| | |
++--------------+--------------------------------------------------------------+
+|operations | In this test case, the following operations are needed: |
+| | 1. "nova-create-instance-in_network": create a VM instance |
+| | in one of the existing neutron network. |
+| | |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there are two metrics: |
+| | 1. process_recover_time: which indicates the maximun |
+| | time (seconds) from the process being killed to |
+| | recovered |
+| | |
+| | 2. packet_drop: measure the packets that have been dropped |
+| | by the monitors using pktgen. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | TBD |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1. test case file: opnfv_yardstick_tc092.yaml |
+| | - Attackers: see above “attackers” discription |
+| | - Monitors: see above “monitors” discription |
+| | - waiting_time: which is the time (seconds) from the |
+| | process being killed to stoping monitors the |
+| | monitors |
+| | - SLA: see above “metrics” discription |
+| | |
+| | 2. POD file: pod.yaml The POD configuration should record |
+| | on pod.yaml first. the “host” item in this test case |
+| | will use the node name in the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | Description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-action | 1. The OpenStack cluster is set up with an SDN controller |
+| | running in a three node cluster configuration. |
+| | |
+| | 2. One or more neutron networks are created with two or |
+| | more VMs attached to each of the neutron networks. |
+| | |
+| | 3. The neutron networks are attached to a neutron router |
+| | which is attached to an external network the towards |
+| | DCGW. |
+| | |
+| | 4. The master node of SDN controller cluster is known. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | Start ip connectivity monitors: |
+| | 1. Check the L2 connectivity between the VMs in the same |
+| | neutron network. |
+| | |
+| | 2. Check the external connectivity of the VMs. |
+| | |
+| | Each monitor runs in an independent process. |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Start attacker: |
+| | SSH to the VIM node and kill the SDN controller process |
+| | determined in step 2. |
+| | |
+| | Result: One SDN controller service will be shut down |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | Restart the SDN controller. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | Create a new VM in the existing Neutron network while the |
+| | SDN controller is offline or still recovering. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 5 | Stop IP connectivity monitors after a period of time |
+| | specified by “waiting_time” |
+| | |
+| | Result: The monitor info will be aggregated |
+| | |
++--------------+--------------------------------------------------------------+
+|step 6 | Verify the IP connectivity monitor result |
+| | |
+| | Result: IP connectivity monitor should not have any packet |
+| | drop failures reported |
+| | |
++--------------+--------------------------------------------------------------+
+|step 7 | Verify process_recover_time, which indicates the maximun |
+| | time (seconds) from the process being killed to recovered, |
+| | is within the SLA. This step blocks until either the |
+| | process has recovered or a timeout occurred. |
+| | |
+| | Result: process_recover_time is within SLA limits, if not, |
+| | test case failed and stopped. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 8 | Start IP connectivity monitors for the new VM: |
+| | 1. Check the L2 connectivity from the existing VMs to the |
+| | new VM in the Neutron network. |
+| | |
+| | 2. Check connectivity from one VM to an external host on |
+| | the Internet to verify SNAT functionality. |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 9 | Stop IP connectivity monitors after a period of time |
+| | specified by “waiting_time” |
+| | |
+| | Result: The monitor info will be aggregated |
+| | |
++--------------+--------------------------------------------------------------+
+|step 10 | Verify the IP connectivity monitor result |
+| | |
+| | Result: IP connectivity monitor should not have any packet |
+| | drop failures reported |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | Fails only if SLA is not passed, or if there is a test case |
+| | execution problem. |
+| | |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc093.rst b/docs/testing/user/userguide/opnfv_yardstick_tc093.rst
new file mode 100644
index 000000000..31fa5d3d3
--- /dev/null
+++ b/docs/testing/user/userguide/opnfv_yardstick_tc093.rst
@@ -0,0 +1,184 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intracom Telecom and others.
+.. mardim@intracom-telecom.com
+
+*************************************
+Yardstick Test Case Description TC093
+*************************************
+
++-----------------------------------------------------------------------------+
+|SDN Vswitch resilience in non-HA or HA configuration |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC093: SDN Vswitch resilience in |
+| | non-HA or HA configuration |
++--------------+--------------------------------------------------------------+
+|test purpose | This test validates that network data plane services are |
+| | resilient in the event of Virtual Switch failure |
+| | in compute nodes. Specifically, the test verifies that |
+| | existing data plane connectivity is not permanently impacted |
+| | i.e. all configured network services such as DHCP, ARP, L2, |
+| | L3 Security Groups continue to operate between the existing |
+| | VMs eventually after the Virtual Switches have finished |
+| | rebooting. |
+| | |
+| | The test also validates that new network service operations |
+| | (creating a new VM in the existing L2/L3 network or in a new |
+| | network, etc.) are operational after the Virtual Switches |
+| | have recovered from a failure. |
+| | |
++--------------+--------------------------------------------------------------+
+|test method | This testcase first checks if the already configured |
+| | DHCP/ARP/L2/L3/SNAT connectivity is proper. After |
+| | it fails and restarts again the VSwitch services which are |
+| | running on both OpenStack compute nodes, and then checks if |
+| | already configured DHCP/ARP/L2/L3/SNAT connectivity is not |
+| | permanently impacted (even if there are some packet |
+| | loss events) between VMs and the system is able to execute |
+| | new virtual network operations once the Vswitch services |
+| | are restarted and have been fully recovered |
+| | |
++--------------+--------------------------------------------------------------+
+|attackers | In this test case, two attackers called “kill-process” are |
+| | needed. These attackers include three parameters: |
+| | 1. fault_type: which is used for finding the attacker's |
+| | scripts. It should be set to 'kill-process' in this test |
+| | |
+| | 2. process_name: should be set to the name of the Vswitch |
+| | process |
+| | |
+| | 3. host: which is the name of the compute node where the |
+| | Vswitch process is running |
+| | |
+| | e.g. -fault_type: "kill-process" |
+| | -process_name: "openvswitch" |
+| | -host: node1 |
+| | |
++--------------+--------------------------------------------------------------+
+|monitors | This test case utilizes two monitors of type "ip-status" |
+| | and one monitor of type "process" to track the following |
+| | conditions: |
+| | 1. "ping_same_network_l2": monitor ICMP traffic between |
+| | VMs in the same Neutron network |
+| | |
+| | 2. "ping_external_snat": monitor ICMP traffic from VMs to |
+| | an external host on the Internet to verify SNAT |
+| | functionality. |
+| | |
+| | 3. "Vswitch process monitor": a monitor checking the |
+| | state of the specified Vswitch process. It measures |
+| | the recovery time of the given process. |
+| | |
+| | Monitors of type "ip-status" use the "ping" utility to |
+| | verify reachability of a given target IP. |
+| | |
++--------------+--------------------------------------------------------------+
+|operations | In this test case, the following operations are needed: |
+| | 1. "nova-create-instance-in_network": create a VM instance |
+| | in one of the existing Neutron network. |
+| | |
++--------------+--------------------------------------------------------------+
+|metrics | In this test case, there are two metrics: |
+| | 1. process_recover_time: which indicates the maximun |
+| | time (seconds) from the process being killed to |
+| | recovered |
+| | |
+| | 2. outage_time: measures the total time in which |
+| | monitors were failing in their tasks (e.g. total time of |
+| | Ping failure) |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | Developed by the project. Please see folder: |
+| | "yardstick/benchmark/scenarios/availability/ha_tools" |
+| | |
++--------------+--------------------------------------------------------------+
+|references | none |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | This test case needs two configuration files: |
+| | 1. test case file: opnfv_yardstick_tc093.yaml |
+| | - Attackers: see above “attackers” description |
+| | - monitor_time: which is the time (seconds) from |
+| | starting to stoping the monitors |
+| | - Monitors: see above “monitors” discription |
+| | - SLA: see above “metrics” description |
+| | |
+| | 2. POD file: pod.yaml The POD configuration should record |
+| | on pod.yaml first. the “host” item in this test case |
+| | will use the node name in the pod.yaml. |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | Description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-action | 1. The Vswitches are set up in both compute nodes. |
+| | |
+| | 2. One or more Neutron networks are created with two or |
+| | more VMs attached to each of the Neutron networks. |
+| | |
+| | 3. The Neutron networks are attached to a Neutron router |
+| | which is attached to an external network towards the |
+| | DCGW. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | Start IP connectivity monitors: |
+| | 1. Check the L2 connectivity between the VMs in the same |
+| | Neutron network. |
+| | |
+| | 2. Check connectivity from one VM to an external host on |
+| | the Internet to verify SNAT functionality. |
+| | |
+| | Result: The monitor info will be collected. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Start attackers: |
+| | SSH connect to the VIM compute nodes and kill the Vswitch |
+| | processes |
+| | |
+| | Result: the SDN Vswitch services will be shutdown |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | Verify the results of the IP connectivity monitors. |
+| | |
+| | Result: The outage_time metric reported by the monitors |
+| | is not greater than the max_outage_time. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | Restart the SDN Vswitch services. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 5 | Create a new VM in the existing Neutron network |
+| | |
++--------------+--------------------------------------------------------------+
+|step 6 | Verify connectivity between VMs as follows: |
+| | 1. Check the L2 connectivity between the previously |
+| | existing VM and the newly created VM on the same |
+| | Neutron network by sending ICMP messages |
+| | |
++--------------+--------------------------------------------------------------+
+|step 7 | Stop IP connectivity monitors after a period of time |
+| | specified by “monitor_time” |
+| | |
+| | Result: The monitor info will be aggregated |
+| | |
++--------------+--------------------------------------------------------------+
+|step 8 | Verify the IP connectivity monitor results |
+| | |
+| | Result: IP connectivity monitor should not have any packet |
+| | drop failures reported |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | This test fails if the SLAs are not met or if there is a |
+| | test case execution problem. The SLAs are define as follows |
+| | for this test: |
+| | * SDN Vswitch recovery |
+| | * process_recover_time <= 30 sec |
+| | |
+| | * no impact on data plane connectivity during SDN |
+| | Vswitch failure and recovery. |
+| | * packet_drop == 0 |
+| | |
++--------------+--------------------------------------------------------------+
+