Doc for Xreview by other test projects

Change-Id: I9976969344c5ac4859b0e79b88157e54ec4198d9
Signed-off-by: JingLu5 <lvjing5@huawei.com>

1 files changed, 78 insertions, 73 deletions

diff --git a/docs/results/overview.rst b/docs/results/overview.rst
index 7f3a34e56..ee0ebe504 100644
--- a/docs/results/overview.rst
+++ b/docs/results/overview.rst
@@ -3,24 +3,10 @@
 .. http://creativecommons.org/licenses/by/4.0
 .. (c) OPNFV, Ericsson AB and others.
 
-=====================
-Yardstick Test Report
-=====================
-
-.. toctree::
-   :maxdepth: 2
-
-Introduction
-============
-
-Document Identifier
--------------------
-
-This document is part of deliverables of the OPNFV release brahmaputra.3.0
-
-Scope
------
+Yardstick test tesult document overview
+=======================================
 
+.. _`Yardstick user guide`: artifacts.opnfv.org/yardstick/docs/userguide/index.html
 .. _Dashboard: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-main
 .. _Jenkins: https://build.opnfv.org/ci/view/yardstick/
 .. _Scenarios: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-scenarios
@@ -28,70 +14,89 @@ Scope
 This document provides an overview of the results of test cases developed by
 the OPNFV Yardstick Project, executed on OPNFV community labs.
 
-OPNFV Continous Integration provides automated build, deploy and testing for
-the software developed in OPNFV. Unless stated, the reported tests are
-automated via Jenkins Jobs.
-
-Test results are visible in the following dashboard:
-
-* *Yardstick* Dashboard_:  uses influx DB to store test results and Grafana for
-   visualization (user: opnfv/ password: opnfv)
-
-
-References
-----------
-
-* IEEE Std 829-2008. "Standard for Software and System Test Documentation".
-
-* OPNFV Brahamputra release note for Yardstick.
-
-
-
-General
-=======
+Yardstick project is described in `Yardstick user guide`_.
 
-Yardstick Test Cases have been executed for scenarios and features defined in
-this OPNFV release.
+Yardstick is run systematically at the end of an OPNFV fresh installation.
+The system under test (SUT) is installed by the installer Apex, Compass, Fuel
+or Joid on Performance Optimized Datacenter (POD); One single installer per
+POD. All the runnable test cases are run sequentially. The installer and the
+POD are considered to evaluate whether the test case can be run or not. That is
+why all the number of test cases may vary from 1 installer to another and from
+1 POD to POD.
 
-The test environments were installed by one of the following: Apex, Compass,
-Fuel or Joid; one single installer per POD.
+OPNFV CI provides automated build, deploy and testing for
+the software developed in OPNFV. Unless stated, the reported tests are
+automated via Jenkins Jobs. Yardsrick test results from OPNFV Continous
+Integration can be found in the following dashboard:
 
-The results of executed tests are available in Dashboard_ and all logs stored
-in Jenkins_.
+* *Yardstick* Dashboard_:  uses influx DB to store Yardstick CI test results and
+   Grafana for visualization (user: opnfv/ password: opnfv)
 
-After one week of measurments, in general, SDN ONOS showed lower latency than
-SDN ODL, which showed lower latency than an environment installed with pure
-OpenStack. Additional time and PODs make this a non-conclusive statement, see
-Scenarios_ for a snapshot and Dashboard_ for complete results.
+The results of executed test cases are available in Dashboard_ and all logs are
+stored in Jenkins_.
 
 It was not possible to execute the entire Yardstick test cases suite on the
 PODs assigned for release verification over a longer period of time, due to
 continuous work on the software components and blocking faults either on
-environment, feature or test framework.
-
-Four consecutive successful runs was defined as criteria for release.
-It is a recommendation to run Yardstick test cases over a longer period
-of time in order to better understand the behavior of the system.
-
+environment, features or test framework.
+
+The list of scenarios supported by each installer can be described as follows:
+
++-------------------------+---------+---------+---------+---------+
+|         Scenario        |  Apex   | Compass |  Fuel   |   Joid  |
++=========================+=========+=========+=========+=========+
+| os-nosdn-nofeature-noha |         |         |         |    X    |
++-------------------------+---------+---------+---------+---------+
+| os-nosdn-nofeature-ha   |    X    |         |    X    |    X    |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l2-nofeature-ha  |    X    |    X    |    X    |    X    |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l2-nofeature-noha|         |    X    |         |         |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l3-nofeature-ha  |    X    |         |    X    |         |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l3-nofeature-ha  |         |    X    |         |         |
++-------------------------+---------+---------+---------+---------+
+| os-onos-sfc-ha          |    X    |         |    X    |    X    |
++-------------------------+---------+---------+---------+---------+
+| os-onos-nofeature-ha    |    X    |         |    X    |    X    |
++-------------------------+---------+---------+---------+---------+
+| os-onos-nofeature-noha  |         |    X    |         |         |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l2-sfc-ha        |         |         |    X    |         |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l2-sfc-noha      |    X    |    X    |         |         |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l2-bgpvpn-ha     |    X    |         |    X    |         |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l2-bgpvpn-noha   |         |    X    |         |         |
++-------------------------+---------+---------+---------+---------+
+| os-nosdn-kvm-ha         |         |         |    X    |         |
++-------------------------+---------+---------+---------+---------+
+| os-nosdn-kvm-noha       |         |    X    |         |         |
++-------------------------+---------+---------+---------+---------+
+| os-nosdn-ovs-ha         |         |         |         |         |
++-------------------------+---------+---------+---------+---------+
+| os-nosdn-ovs-noha       |    X    |    X    |         |         |
++-------------------------+---------+---------+---------+---------+
+| os-ocl-nofeature-ha     |         |         |         |         |
++-------------------------+---------+---------+---------+---------+
+| os-nosdn-lxd-ha         |         |         |         |    X    |
++-------------------------+---------+---------+---------+---------+
+| os-nosdn-lxd-noha       |         |         |         |    X    |
++-------------------------+---------+---------+---------+---------+
+| os-odl_l2-fdio-noha     |    X    |         |         |         |
++-------------------------+---------+---------+---------+---------+
+
+To qualify for release, the scenarios must have deployed and been successfully
+tested in four consecutive installations to establish stability of deployment
+and feature capability. It is a recommendation to run Yardstick test
+cases over a longer period of time in order to better understand the behavior
+of the system under test.
 
+References
+----------
 
-Document change procedures and history
---------------------------------------
+* IEEE Std 829-2008. "Standard for Software and System Test Documentation".
 
-+--------------------------------------+--------------------------------------+
-| **Project**                          | Yardstick                            |
-|                                      |                                      |
-+--------------------------------------+--------------------------------------+
-| **Repo/tag**                         | yardstick/brahmaputra.3.0            |
-|                                      |                                      |
-+--------------------------------------+--------------------------------------+
-| **Release designation**              | Brahmaputra                          |
-|                                      |                                      |
-+--------------------------------------+--------------------------------------+
-| **Release date**                     | Apr 27th, 2016                       |
-|                                      |                                      |
-+--------------------------------------+--------------------------------------+
-| **Purpose of the delivery**          | OPNFV Brahmaputra release test       |
-|                                      | results.                             |
-|                                      |                                      |
-+--------------------------------------+--------------------------------------+
+* OPNFV Colorado release note for Yardstick.