From 1b0913c31a8df2f2f0de4cd70a889ee5704cbfd1 Mon Sep 17 00:00:00 2001 From: "wu.zhihui" Date: Wed, 28 Dec 2016 17:03:47 +0800 Subject: Yardstick integration design specification Refer to https://etherpad.opnfv.org/p/qtip-yardstick, I write this design specification. JIRA: QTIP-147 Change-Id: I651c6ffd898fa22c237b788edab7e0a43828ae7c Signed-off-by: wu.zhihui --- docs/designspec/index.rst | 1 + docs/designspec/integration_with_yardstick.rst | 92 ++++++++++++++++++++++++++ 2 files changed, 93 insertions(+) create mode 100644 docs/designspec/integration_with_yardstick.rst diff --git a/docs/designspec/index.rst b/docs/designspec/index.rst index d3bed217..b6dd0c01 100644 --- a/docs/designspec/index.rst +++ b/docs/designspec/index.rst @@ -12,3 +12,4 @@ QTIP Design Specifications dashboard.rst compute_QPI.rst + integration_with_yardstick.rst diff --git a/docs/designspec/integration_with_yardstick.rst b/docs/designspec/integration_with_yardstick.rst new file mode 100644 index 00000000..a8298d6f --- /dev/null +++ b/docs/designspec/integration_with_yardstick.rst @@ -0,0 +1,92 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2016 ZTE Corp. + + +************************** +Integration with Yardstick +************************** + +Problem description +=================== + +For each specified QPI [1]_, QTIP needs to select a suite of test cases and collect +required test results. Based on these results, Qtip calculates the score. + +Proposed change +=============== +Qtip has a flexible architecture [2]_ to support different mode: standalone and agent. +It is recommended to use **agent mode** to work with existing test runners. Yardstick will +act as a runner to generate test result and trigger Qtip agent on the completion of test. + + +Work Items in Yardstick +----------------------- + +1. Create a customized suite in Yardstick + +Yardstick not only has many existing suites but also support customized suites. Qtip could +create a suite named **Qtip-PoC** in Yardstick repo to verify workflow of Qtip agent mode. + +2. Launch Qtip in Yardstick + +Whether to launch Qtip will be determined by checking the existence of OS environment +variable *QTIP*. If it exists, Qtip will be launched by using Yardstick CLI +`yardstick plugin install` [3]_. + +3. Yardstick interacts with Qtip + +See +`Yardstick-Qtip+integration `_ +for details. + +Work Items in Qtip +------------------ + +1. Provide an API for Yardstick to post test result and environment info + +After completing test execution, Yardstick will post test result and enviroment info with +JSON format via QTIP API. See +`Yardstick-Qtip+integration `_ +for details. + +2. Parse yardstick test result + +When Qtip agent receive Yarstick test result and enviroment info, Qtip agent will extract +metrics which is definded in metric spec configuration file. Based on these metrics, Qtip +agent will caculate QPI. + +3. Provide an API for querying QPI + +Qtip will provide an API for querying QPI. See +`Yardstick-Qtip+integration `_ +for details. + +Implementation +============== + +Assignee(s) +----------- + +*Primary assignee:* + wu.zhihui + +*Other contributors* + TBD + +Testing +======= + +The changes will be covered by new unit test. + +Documentation +============= + +TBD + +Reference +========= + +.. [1] QTIP performance index +.. [2] https://wiki.opnfv.org/display/qtip/Architecture +.. [3] https://wiki.opnfv.org/display/yardstick/How+to+install+a+plug-in+into+Yardstick -- cgit 1.2.3-korg