aboutsummaryrefslogtreecommitdiffstats
path: root/docs/testing/developer/design/dashboard.rst
diff options
context:
space:
mode:
authorYujun Zhang <zhang.yujunz@zte.com.cn>2017-03-03 16:00:58 +0800
committerYujun Zhang <zhang.yujunz@zte.com.cn>2017-03-03 16:06:09 +0800
commitfbf89b04f8ec8eed8ea905757dad4566c34786d8 (patch)
treec710299a499f199dc4650eef0d8080b79cdee6f0 /docs/testing/developer/design/dashboard.rst
parent473deae9a10162f000c49ca49b4e31b28c4bf0d8 (diff)
Restructure docs
- Following the changes as in functest[1] since the guide[2] in wiki is a bit out of date - Some folders are empty for now as a placeholder Now: - release - release notes - testing - developer - design - overview - requirement - user - config guide - user guide - installation - scenarios [1]: https://gerrit.opnfv.org/gerrit/#/c/29151/ [2]: https://wiki.opnfv.org/display/DOC/Documentation+Guide Change-Id: I81b4597536c8bf02a925641d71d0969f6f5537f4 Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
Diffstat (limited to 'docs/testing/developer/design/dashboard.rst')
-rw-r--r--docs/testing/developer/design/dashboard.rst151
1 files changed, 151 insertions, 0 deletions
diff --git a/docs/testing/developer/design/dashboard.rst b/docs/testing/developer/design/dashboard.rst
new file mode 100644
index 00000000..60c4720d
--- /dev/null
+++ b/docs/testing/developer/design/dashboard.rst
@@ -0,0 +1,151 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) 2016 ZTE Corp.
+
+
+*********
+Dashboard
+*********
+
+The dashboard gives user an intuitive view of benchmark result.
+
+Purpose
+=======
+
+The basic element to be displayed is QPI a.k.a. QTIP Performance Index. But it
+is also important to show user
+
+#. How is the final score calculated?
+#. Under what condition is the test plan executed?
+#. How many runs of a performance tests have been executed and is there any deviation?
+#. Comparison of benchmark result from different PODs or configuration
+
+Templates
+=========
+
+Different board templates are created to satisfy the above requirements.
+
+Composition
+-----------
+
+QTIP gives a simple score but there must be a complex formula behind it. This
+view explains the composition of the QPI.
+
+Condition
+---------
+
+The condition of a benchmark result includes
+
+* System Under Test
+
+ * Hardware environment
+ * Hypervisor version
+ * Operation System release version
+ * System Configuration
+
+* Test Tools
+
+ * Release version
+ * Configuration
+
+* Test Facility
+
+ * Laboratory
+ * Engineer
+ * Date
+
+Conditions that do NOT have an obvious affect on the test result may be ignored,
+e.g. temperature, power supply.
+
+Stats
+-----
+
+Performance tests are actually measurement of specific metrics. All measurement
+comes with uncertainty. The final result is normally one or a group of metrics
+calculated from many repeats.
+
+For each metric, the stats board shall consist of a diagram of all measured
+values and a box of stats::
+
+ ^ +------------+
+ | | count: ? |
+ | |average: ? |
+ | | min: ? |
+ | X | max: ? |
+ | XXXX XXXX X XXXXX | |
+ |X XX XX XX XXX XXX XX | |
+ | XXXXXX X XXXXX XX | |
+ | | |
+ | | |
+ | | |
+ | | |
+ | | |
+ +---------------------------------------------> +------------+
+
+The type of diagram and selection of stats shall depend on what metric to show.
+
+Comparison
+----------
+
+Comparison can be done between different PODs or different configuration on the
+same PODs.
+
+In a comparison view, the metrics are displayed in the same diagram. And the
+parameters are listed side by side.
+
+Both common parameters and different parameters are listed. Common values are
+merged to the same cell. And user may configure the view to hide common rows.
+
+A draft design is as following::
+
+ ^
+ |
+ |
+ |
+ | XXXXXXXX
+ | XXX XX+-+ XXXXXXXXXX
+ | XXX +XXXX XXXXX
+ +-+XX X +--+ ++ XXXXXX +-+
+ | X+-+X +----+ +-+ +----+X
+ |X +--+ +---+ XXXXXX X
+ | +-------+ X
+ |
+ |
+ +----------------------------------------------------->
+
+ +--------------------+----------------+---------------+
+ | different param 1 | | |
+ | | | |
+ +-----------------------------------------------------+
+ | different param 2 | | |
+ | | | |
+ +-------------------------------------+---------------+
+ | common param 1 | |
+ | | |
+ +-------------------------------------+---------------+
+ | different param 3 | | |
+ | | | |
+ +-------------------------------------+---------------+
+ | common param 2 | |
+ | | |
+ +--------------------+--------------------------------+
+ +------------+
+ | HIDE COMMON|
+ +------------+
+
+Time line
+---------
+
+Time line diagram for analysis of time critical performance test::
+
+ +-----------------+-----------+-------------+-------------+-----+
+ | | | | | |
+ +-----------------> | | | |
+ | +-----------> | | |
+ | ? ms +-------------> | |
+ | ? ms +------------>+ |
+ | ? ms ? ms |
+ | |
+ +---------------------------------------------------------------+
+
+The time cost between checkpoints shall be displayed in the diagram.