Merge "Migrate to Danube document guide"

8 files changed, 350 insertions, 0 deletions

diff --git a/docs/development/design/apidocs/index.rst b/docs/development/design/apidocs/index.rst
new file mode 100644
index 00000000..241a2680
--- /dev/null
+++ b/docs/development/design/apidocs/index.rst
@@ -0,0 +1,13 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) 2015 Dell Inc.
+.. (c) 2016 ZTE Corp.
+
+################
+QTIP Configguide
+################
+
+.. toctree::
+   :maxdepth: 2
+
+   ./qtip_restful_api.rst
diff --git a/docs/development/design/apidocs/qtip_restful_api.rst b/docs/development/design/apidocs/qtip_restful_api.rst
new file mode 100644
index 00000000..06c01292
--- /dev/null
+++ b/docs/development/design/apidocs/qtip_restful_api.rst
@@ -0,0 +1,10 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) 2015 Dell Inc.
+.. (c) 2016 ZTE Corp.
+
+****************
+Qtip restful api
+****************
+
+You can get all the Qtip restful api by http://restful_api.qtip.openzero.net/api/spec.html.
diff --git a/docs/development/design/compute_QPI.rst b/docs/development/design/compute_QPI.rst
new file mode 100644
index 00000000..2e5aa87c
--- /dev/null
+++ b/docs/development/design/compute_QPI.rst
@@ -0,0 +1,69 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) 2016 ZTE Corp.
+
+
+***********
+Compute QPI
+***********
+
+The compute QPI gives user an overall score for system compute performace.
+
+Summary
+=======
+
+The compute QPI are calibrated a ZTE `E9000 <http://www.zte.com.cn/global/products/cocloud/cloud_computing/cloud_infrastructure/cloud_hw/429552>`_ server as a baseline with score of 2500 points.
+Higher scores are better, with double the score indicating double the performance.
+The compute QPI provides three different kinds of scores:
+
+* Workload Scores
+* Section Scores
+* Compute QPI Scores
+
+Baseline
+========
+
+ZTE E9000 server with an 2 Deca core Intel Xeon CPU processor,128560.0MB Memory.
+
+Workload Scores
+===============
+
+Each time a workload is executed QTIP calculates a score based on the computer's performance
+compared to the baseline performance.
+
+Section Scores
+==============
+
+QTIP uses a number of different tests, or workloads, to measure performance.
+The workloads are divided into five different sections:
+
++-----------------+--------------------------------------------------------------+------------------------------------------+
+| Section         | Detail                                                       | Indication                               |
++=================+==============================================================+==========================================+
+| Integer         | Integer workloads measure the integer instruction performace | All app relies on integer                |
+|                 | of host or vm by performing Dhrystone test.                  | performance                              |
++-----------------+--------------------------------------------------------------+------------------------------------------+
+| Floating point  | Floating point workloads measure the floating pointperfo     | Floating point performance is especially |
+|                 | rmance by performing Whetstone test.                         | important in video games,digital content |
+|                 |                                                              | creation applications.                   |
++-----------------+--------------------------------------------------------------+------------------------------------------+
+| Memory          | Memory workloads measure memory bandwidth by performing      | Software working with cipher large       |
+|                 | RamSpeed test.                                               | amounts data relies on SSL Performace.   |
++-----------------+--------------------------------------------------------------+------------------------------------------+
+| DPI             | DPI workloads measure deep-packet inspection speed by        | Software working with network packet     |
+|                 | performing nDPI test.                                        | anlysis relies on DPI performance.       |
++-----------------+--------------------------------------------------------------+------------------------------------------+
+| SSL             | SSL Performance workloads measure cipher speeds by           | Software working with cipher large       |
+|                 | using the OpenSSL tool.                                      | amounts data relies on SSL Performace    |
++-----------------+--------------------------------------------------------------+------------------------------------------+
+
+A section score is the `geometric mean <https://en.wikipedia.org/wiki/Geometric_mean>`_ of all the workload scores for workloads
+that are part of the section. These scores are useful for determining the performance of
+the computer in a particular area.
+
+Compute QPI Scores
+==================
+
+The compute QPI score is the `weighted arithmetic mean <https://en.wikipedia.org/wiki/Weighted_arithmetic_mean>`_ of the five section scores.
+The compute QPI score provides a way to quickly compare performance across different
+computers and different platforms without getting bogged down in details.
diff --git a/docs/development/design/dashboard.rst b/docs/development/design/dashboard.rst
new file mode 100644
index 00000000..60c4720d
--- /dev/null
+++ b/docs/development/design/dashboard.rst
@@ -0,0 +1,151 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) 2016 ZTE Corp.
+
+
+*********
+Dashboard
+*********
+
+The dashboard gives user an intuitive view of benchmark result.
+
+Purpose
+=======
+
+The basic element to be displayed is QPI a.k.a. QTIP Performance Index. But it
+is also important to show user
+
+#. How is the final score calculated?
+#. Under what condition is the test plan executed?
+#. How many runs of a performance tests have been executed and is there any deviation?
+#. Comparison of benchmark result from different PODs or configuration
+
+Templates
+=========
+
+Different board templates are created to satisfy the above requirements.
+
+Composition
+-----------
+
+QTIP gives a simple score but there must be a complex formula behind it. This
+view explains the composition of the QPI.
+
+Condition
+---------
+
+The condition of a benchmark result includes
+
+* System Under Test
+
+  * Hardware environment
+  * Hypervisor version
+  * Operation System release version
+  * System Configuration
+
+* Test Tools
+
+  * Release version
+  * Configuration
+
+* Test Facility
+
+  * Laboratory
+  * Engineer
+  * Date
+
+Conditions that do NOT have an obvious affect on the test result may be ignored,
+e.g. temperature, power supply.
+
+Stats
+-----
+
+Performance tests are actually measurement of specific metrics. All measurement
+comes with uncertainty. The final result is normally one or a group of metrics
+calculated from many repeats.
+
+For each metric, the stats board shall consist of a diagram of all measured
+values and a box of stats::
+
+  ^                                                  +------------+
+  |                                                  |  count: ?  |
+  |                                                  |average: ?  |
+  |                                                  |    min: ?  |
+  |                   X                              |    max: ?  |
+  | XXXX          XXXX X              XXXXX          |            |
+  |X    XX      XX      XX XXX     XXX     XX        |            |
+  |       XXXXXX          X   XXXXX          XX      |            |
+  |                                                  |            |
+  |                                                  |            |
+  |                                                  |            |
+  |                                                  |            |
+  |                                                  |            |
+  +--------------------------------------------->    +------------+
+
+The type of diagram and selection of stats shall depend on what metric to show.
+
+Comparison
+----------
+
+Comparison can be done between different PODs or different configuration on the
+same PODs.
+
+In a comparison view, the metrics are displayed in the same diagram. And the
+parameters are listed side by side.
+
+Both common parameters and different parameters are listed. Common values are
+merged to the same cell. And user may configure the view to hide common rows.
+
+A draft design is as following::
+
+    ^
+    |
+    |
+    |
+    |           XXXXXXXX
+    |         XXX      XX+-+ XXXXXXXXXX
+    |      XXX          +XXXX         XXXXX
+    +-+XX X         +--+    ++            XXXXXX     +-+
+    | X+-+X   +----+          +-+              +----+X
+    |X    +--+                   +---+         XXXXXX X
+    |                                 +-------+        X
+    |
+    |
+    +----------------------------------------------------->
+
+    +--------------------+----------------+---------------+
+    | different param 1  |                |               |
+    |                    |                |               |
+    +-----------------------------------------------------+
+    | different param 2  |                |               |
+    |                    |                |               |
+    +-------------------------------------+---------------+
+    | common param 1     |                                |
+    |                    |                                |
+    +-------------------------------------+---------------+
+    | different param 3  |                |               |
+    |                    |                |               |
+    +-------------------------------------+---------------+
+    | common param 2     |                                |
+    |                    |                                |
+    +--------------------+--------------------------------+
+                                             +------------+
+                                             | HIDE COMMON|
+                                             +------------+
+
+Time line
+---------
+
+Time line diagram for analysis of time critical performance test::
+
+  +-----------------+-----------+-------------+-------------+-----+
+  |                 |           |             |             |     |
+  +----------------->           |             |             |     |
+  |                 +----------->             |             |     |
+  |                 ? ms        +------------->             |     |
+  |                             ? ms          +------------>+     |
+  |                                           ? ms          ? ms  |
+  |                                                               |
+  +---------------------------------------------------------------+
+
+The time cost between checkpoints shall be displayed in the diagram.
diff --git a/docs/development/design/index.rst b/docs/development/design/index.rst
new file mode 100644
index 00000000..b6dd0c01
--- /dev/null
+++ b/docs/development/design/index.rst
@@ -0,0 +1,15 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) 2016 ZTE Corp.
+
+
+##########################
+QTIP Design Specifications
+##########################
+
+.. toctree::
+   :maxdepth: 2
+
+   dashboard.rst
+   compute_QPI.rst
+   integration_with_yardstick.rst
diff --git a/docs/development/design/integration_with_yardstick.rst b/docs/development/design/integration_with_yardstick.rst
new file mode 100644
index 00000000..a8298d6f
--- /dev/null
+++ b/docs/development/design/integration_with_yardstick.rst
@@ -0,0 +1,92 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) 2016 ZTE Corp.
+
+
+**************************
+Integration with Yardstick
+**************************
+
+Problem description
+===================
+
+For each specified QPI [1]_, QTIP needs to select a suite of test cases and collect
+required test results. Based on these results, Qtip calculates the score.
+
+Proposed change
+===============
+Qtip has a flexible architecture [2]_ to support different mode: standalone and agent.
+It is recommended to use **agent mode** to work with existing test runners. Yardstick will
+act as a runner to generate test result and trigger Qtip agent on the completion of test.
+
+
+Work Items in Yardstick
+-----------------------
+
+1. Create a customized suite in Yardstick
+
+Yardstick not only has many existing suites but also support customized suites. Qtip could
+create a suite named **Qtip-PoC** in Yardstick repo to verify workflow of Qtip agent mode.
+
+2. Launch Qtip in Yardstick
+
+Whether to launch Qtip will be determined by checking the existence of OS environment
+variable *QTIP*. If it exists, Qtip will be launched by using Yardstick CLI
+`yardstick plugin install` [3]_.
+
+3. Yardstick interacts with Qtip
+
+See
+`Yardstick-Qtip+integration <https://wiki.opnfv.org/display/yardstick/Yardstick-Qtip+integration>`_
+for details.
+
+Work Items in Qtip
+------------------
+
+1. Provide an API for Yardstick to post test result and environment info
+
+After completing test execution, Yardstick will post test result and enviroment info with
+JSON format via QTIP API. See
+`Yardstick-Qtip+integration <https://wiki.opnfv.org/display/yardstick/Yardstick-Qtip+integration>`_
+for details.
+
+2. Parse yardstick test result
+
+When Qtip agent receive Yarstick test result and enviroment info, Qtip agent will extract
+metrics which is definded in metric spec configuration file. Based on these metrics, Qtip
+agent will caculate QPI.
+
+3. Provide an API for querying QPI
+
+Qtip will provide an API for querying QPI. See
+`Yardstick-Qtip+integration <https://wiki.opnfv.org/display/yardstick/Yardstick-Qtip+integration>`_
+for details.
+
+Implementation
+==============
+
+Assignee(s)
+-----------
+
+*Primary assignee:*
+  wu.zhihui
+
+*Other contributors*
+  TBD
+
+Testing
+=======
+
+The changes will be covered by new unit test.
+
+Documentation
+=============
+
+TBD
+
+Reference
+=========
+
+.. [1] QTIP performance index
+.. [2] https://wiki.opnfv.org/display/qtip/Architecture
+.. [3] https://wiki.opnfv.org/display/yardstick/How+to+install+a+plug-in+into+Yardstick
diff --git a/docs/development/overview/.gitkeep b/docs/development/overview/.gitkeep
new file mode 100644
index 00000000..e69de29b
--- /dev/null
+++ b/docs/development/overview/.gitkeep
diff --git a/docs/development/requirement/.gitkeep b/docs/development/requirement/.gitkeep
new file mode 100644
index 00000000..e69de29b
--- /dev/null
+++ b/docs/development/requirement/.gitkeep