blob: a8298d6f6f57f6c49e556ebdb5524b365b4ad4ef (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
|
.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) 2016 ZTE Corp.
**************************
Integration with Yardstick
**************************
Problem description
===================
For each specified QPI [1]_, QTIP needs to select a suite of test cases and collect
required test results. Based on these results, Qtip calculates the score.
Proposed change
===============
Qtip has a flexible architecture [2]_ to support different mode: standalone and agent.
It is recommended to use **agent mode** to work with existing test runners. Yardstick will
act as a runner to generate test result and trigger Qtip agent on the completion of test.
Work Items in Yardstick
-----------------------
1. Create a customized suite in Yardstick
Yardstick not only has many existing suites but also support customized suites. Qtip could
create a suite named **Qtip-PoC** in Yardstick repo to verify workflow of Qtip agent mode.
2. Launch Qtip in Yardstick
Whether to launch Qtip will be determined by checking the existence of OS environment
variable *QTIP*. If it exists, Qtip will be launched by using Yardstick CLI
`yardstick plugin install` [3]_.
3. Yardstick interacts with Qtip
See
`Yardstick-Qtip+integration <https://wiki.opnfv.org/display/yardstick/Yardstick-Qtip+integration>`_
for details.
Work Items in Qtip
------------------
1. Provide an API for Yardstick to post test result and environment info
After completing test execution, Yardstick will post test result and enviroment info with
JSON format via QTIP API. See
`Yardstick-Qtip+integration <https://wiki.opnfv.org/display/yardstick/Yardstick-Qtip+integration>`_
for details.
2. Parse yardstick test result
When Qtip agent receive Yarstick test result and enviroment info, Qtip agent will extract
metrics which is definded in metric spec configuration file. Based on these metrics, Qtip
agent will caculate QPI.
3. Provide an API for querying QPI
Qtip will provide an API for querying QPI. See
`Yardstick-Qtip+integration <https://wiki.opnfv.org/display/yardstick/Yardstick-Qtip+integration>`_
for details.
Implementation
==============
Assignee(s)
-----------
*Primary assignee:*
wu.zhihui
*Other contributors*
TBD
Testing
=======
The changes will be covered by new unit test.
Documentation
=============
TBD
Reference
=========
.. [1] QTIP performance index
.. [2] https://wiki.opnfv.org/display/qtip/Architecture
.. [3] https://wiki.opnfv.org/display/yardstick/How+to+install+a+plug-in+into+Yardstick
|
