diff options
author | 2017-02-18 23:50:47 +0800 | |
---|---|---|
committer | 2017-02-18 23:50:47 +0800 | |
commit | b9de54adc5963e54303dccf5930f6b20cc445dae (patch) | |
tree | 557ba58f02b53f26dd1b03ab3cb02186fda6a2c0 /docs/development/design/dashboard.rst | |
parent | 93421c6375a450baa7bedb323e36b5efd238a58d (diff) |
Migrate to Danube document guide
https://wiki.opnfv.org/display/DOC
Change-Id: I85535f158022fe0aeb7c888273dc81fbbf4ddeda
Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
Diffstat (limited to 'docs/development/design/dashboard.rst')
-rw-r--r-- | docs/development/design/dashboard.rst | 151 |
1 files changed, 151 insertions, 0 deletions
diff --git a/docs/development/design/dashboard.rst b/docs/development/design/dashboard.rst new file mode 100644 index 00000000..60c4720d --- /dev/null +++ b/docs/development/design/dashboard.rst @@ -0,0 +1,151 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2016 ZTE Corp. + + +********* +Dashboard +********* + +The dashboard gives user an intuitive view of benchmark result. + +Purpose +======= + +The basic element to be displayed is QPI a.k.a. QTIP Performance Index. But it +is also important to show user + +#. How is the final score calculated? +#. Under what condition is the test plan executed? +#. How many runs of a performance tests have been executed and is there any deviation? +#. Comparison of benchmark result from different PODs or configuration + +Templates +========= + +Different board templates are created to satisfy the above requirements. + +Composition +----------- + +QTIP gives a simple score but there must be a complex formula behind it. This +view explains the composition of the QPI. + +Condition +--------- + +The condition of a benchmark result includes + +* System Under Test + + * Hardware environment + * Hypervisor version + * Operation System release version + * System Configuration + +* Test Tools + + * Release version + * Configuration + +* Test Facility + + * Laboratory + * Engineer + * Date + +Conditions that do NOT have an obvious affect on the test result may be ignored, +e.g. temperature, power supply. + +Stats +----- + +Performance tests are actually measurement of specific metrics. All measurement +comes with uncertainty. The final result is normally one or a group of metrics +calculated from many repeats. + +For each metric, the stats board shall consist of a diagram of all measured +values and a box of stats:: + + ^ +------------+ + | | count: ? | + | |average: ? | + | | min: ? | + | X | max: ? | + | XXXX XXXX X XXXXX | | + |X XX XX XX XXX XXX XX | | + | XXXXXX X XXXXX XX | | + | | | + | | | + | | | + | | | + | | | + +---------------------------------------------> +------------+ + +The type of diagram and selection of stats shall depend on what metric to show. + +Comparison +---------- + +Comparison can be done between different PODs or different configuration on the +same PODs. + +In a comparison view, the metrics are displayed in the same diagram. And the +parameters are listed side by side. + +Both common parameters and different parameters are listed. Common values are +merged to the same cell. And user may configure the view to hide common rows. + +A draft design is as following:: + + ^ + | + | + | + | XXXXXXXX + | XXX XX+-+ XXXXXXXXXX + | XXX +XXXX XXXXX + +-+XX X +--+ ++ XXXXXX +-+ + | X+-+X +----+ +-+ +----+X + |X +--+ +---+ XXXXXX X + | +-------+ X + | + | + +-----------------------------------------------------> + + +--------------------+----------------+---------------+ + | different param 1 | | | + | | | | + +-----------------------------------------------------+ + | different param 2 | | | + | | | | + +-------------------------------------+---------------+ + | common param 1 | | + | | | + +-------------------------------------+---------------+ + | different param 3 | | | + | | | | + +-------------------------------------+---------------+ + | common param 2 | | + | | | + +--------------------+--------------------------------+ + +------------+ + | HIDE COMMON| + +------------+ + +Time line +--------- + +Time line diagram for analysis of time critical performance test:: + + +-----------------+-----------+-------------+-------------+-----+ + | | | | | | + +-----------------> | | | | + | +-----------> | | | + | ? ms +-------------> | | + | ? ms +------------>+ | + | ? ms ? ms | + | | + +---------------------------------------------------------------+ + +The time cost between checkpoints shall be displayed in the diagram. |