diff options
Diffstat (limited to 'docs/testing/developer/design/dashboard.rst')
| -rw-r--r-- | docs/testing/developer/design/dashboard.rst | 151 |
1 files changed, 0 insertions, 151 deletions
diff --git a/docs/testing/developer/design/dashboard.rst b/docs/testing/developer/design/dashboard.rst deleted file mode 100644 index 60c4720d..00000000 --- a/docs/testing/developer/design/dashboard.rst +++ /dev/null @@ -1,151 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) 2016 ZTE Corp. - - -********* -Dashboard -********* - -The dashboard gives user an intuitive view of benchmark result. - -Purpose -======= - -The basic element to be displayed is QPI a.k.a. QTIP Performance Index. But it -is also important to show user - -#. How is the final score calculated? -#. Under what condition is the test plan executed? -#. How many runs of a performance tests have been executed and is there any deviation? -#. Comparison of benchmark result from different PODs or configuration - -Templates -========= - -Different board templates are created to satisfy the above requirements. - -Composition ------------ - -QTIP gives a simple score but there must be a complex formula behind it. This -view explains the composition of the QPI. - -Condition ---------- - -The condition of a benchmark result includes - -* System Under Test - - * Hardware environment - * Hypervisor version - * Operation System release version - * System Configuration - -* Test Tools - - * Release version - * Configuration - -* Test Facility - - * Laboratory - * Engineer - * Date - -Conditions that do NOT have an obvious affect on the test result may be ignored, -e.g. temperature, power supply. - -Stats ------ - -Performance tests are actually measurement of specific metrics. All measurement -comes with uncertainty. The final result is normally one or a group of metrics -calculated from many repeats. - -For each metric, the stats board shall consist of a diagram of all measured -values and a box of stats:: - - ^ +------------+ - | | count: ? | - | |average: ? | - | | min: ? | - | X | max: ? | - | XXXX XXXX X XXXXX | | - |X XX XX XX XXX XXX XX | | - | XXXXXX X XXXXX XX | | - | | | - | | | - | | | - | | | - | | | - +---------------------------------------------> +------------+ - -The type of diagram and selection of stats shall depend on what metric to show. - -Comparison ----------- - -Comparison can be done between different PODs or different configuration on the -same PODs. - -In a comparison view, the metrics are displayed in the same diagram. And the -parameters are listed side by side. - -Both common parameters and different parameters are listed. Common values are -merged to the same cell. And user may configure the view to hide common rows. - -A draft design is as following:: - - ^ - | - | - | - | XXXXXXXX - | XXX XX+-+ XXXXXXXXXX - | XXX +XXXX XXXXX - +-+XX X +--+ ++ XXXXXX +-+ - | X+-+X +----+ +-+ +----+X - |X +--+ +---+ XXXXXX X - | +-------+ X - | - | - +-----------------------------------------------------> - - +--------------------+----------------+---------------+ - | different param 1 | | | - | | | | - +-----------------------------------------------------+ - | different param 2 | | | - | | | | - +-------------------------------------+---------------+ - | common param 1 | | - | | | - +-------------------------------------+---------------+ - | different param 3 | | | - | | | | - +-------------------------------------+---------------+ - | common param 2 | | - | | | - +--------------------+--------------------------------+ - +------------+ - | HIDE COMMON| - +------------+ - -Time line ---------- - -Time line diagram for analysis of time critical performance test:: - - +-----------------+-----------+-------------+-------------+-----+ - | | | | | | - +-----------------> | | | | - | +-----------> | | | - | ? ms +-------------> | | - | ? ms +------------>+ | - | ? ms ? ms | - | | - +---------------------------------------------------------------+ - -The time cost between checkpoints shall be displayed in the diagram. |