aboutsummaryrefslogtreecommitdiffstats
path: root/docs/designspec/dashboard.rst
blob: ad5520b6203a773e6360b328a5fa6743906c2873 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) 2016 ZTE Corp.


*********
Dashboard
*********

The dashboard gives user an intuitive view of benchmark result.

Purpose
=======

The basic element to be displayed is QPI a.k.a. QTIP Performance Index. But it
is also important to show user

#. How is the final score calculated?
#. Under what condition is the test plan executed?
#. How many runs of a performance tests have been executed and is there any deviation?
#. Comparison of benchmark result from different PODs or configuration

Templates
=========

Different board templates are created to satisfy the above requirements.

Composition
-----------

QTIP gives a simple score but there must be a complex formula behind it. This
view explains the composition of the QPI.

Condition
---------

The condition of a benchmark result includes

* System Under Test

  * Hardware environment
  * Hypervisor version
  * Operation System release version
  * System Configuration

* Test Tools

  * Release version
  * Configuration

* Test Facility

  * Laboratory
  * Engineer
  * Date

Conditions that do NOT have an obvious affect on the test result may be ignored,
e.g. temperature, power supply.

Deviation
---------

Performance tests are usually repeated many times to reduce random disturbance.
This view shall show an overview of deviation among different runs.

Comparison
----------

Comparison can be done between different PODs or different configuration on the
same PODs.