From d250e41b8f94e2b1cb7f75ba72d040a2e6cb8967 Mon Sep 17 00:00:00 2001 From: kubi Date: Wed, 23 Mar 2016 12:10:03 +0800 Subject: add software Architecture chapter for user guide this is a part of SWA, referenced with Per Holmgren's patch(9815). Because Brahmaputra 2.0 is coming, Qiliang,Patrick,Mingjiang Lin will work with me together to finish this chapter before doc freeze. Change-Id: I1e6abe55658671fb1fcc66f2d81b5c2d0b364822 Signed-off-by: kubi (cherry picked from commit bf1746e6432e18f17b6d8e890c72c93cd9ac928c) --- docs/userguide/01-introduction.rst | 2 + docs/userguide/architecture.rst | 184 +++++++++++++++++++++++++++++++++++++ docs/userguide/index.rst | 1 + 3 files changed, 187 insertions(+) mode change 100644 => 100755 docs/userguide/01-introduction.rst create mode 100644 docs/userguide/architecture.rst diff --git a/docs/userguide/01-introduction.rst b/docs/userguide/01-introduction.rst old mode 100644 new mode 100755 index 90c112a47..3c6100ae7 --- a/docs/userguide/01-introduction.rst +++ b/docs/userguide/01-introduction.rst @@ -41,6 +41,8 @@ This document consists of the following chapters: * Chapter :doc:`02-methodology` describes the methodology implemented by the Yardstick Project for :term:`NFVI` verification. +* Chapter :doc:`architecture` provides information on the software architecture + of yardstick. * Chapter :doc:`04-vtc-overview` provides information on the :term:`VTC`. * Chapter :doc:`apexlake_installation` provides instructions to install the diff --git a/docs/userguide/architecture.rst b/docs/userguide/architecture.rst new file mode 100644 index 000000000..ccf8d482c --- /dev/null +++ b/docs/userguide/architecture.rst @@ -0,0 +1,184 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2016 Huawei Technologies Co.,Ltd and others + +============ +Architecture +============ + +Abstract +======== +This chapter describes the yardstick framework software architecture. we will introduce it from Use-Case View, +Logical View, Process View and Deployment View. More technical details will be introduced in this chapter. + +Overview +======== + +Architecture overview +--------------------- +Yardstick is mainly written in Python, and test configurations are made +in YAML. Documentation is written in reStructuredText format, i.e. .rst +files. Yardstick is inspired by Rally. Yardstick is intended to run on a +computer with access and credentials to a cloud. The test case is described +in a configuration file given as an argument. + +How it works: the benchmark task configuration file is parsed and converted into +an internal model. The context part of the model is converted into a Heat +template and deployed into a stack. Each scenario is run using a runner, either +serially or in parallel. Each runner runs in its own subprocess executing +commands in a VM using SSH. The output of each scenario is written as json +records to a file or influxdb or http server, we use influxdb as the backend, +the test result will be shown with grafana. + + +Concept +------- +**Benchmark** - assess the relative performance of something + +**Benchmark** configuration file - describes a single test case in yaml format + +**Context** - The set of Cloud resources used by a scenario, such as user +names, image names, affinity rules and network configurations. A context is +converted into a simplified Heat template, which is used to deploy onto the +Openstack environment. + +**Data** - Output produced by running a benchmark, written to a file in json format + +**Runner** - Logic that determines how a test scenario is run and reported, for +example the number of test iterations, input value stepping and test duration. +Predefined runner types exist for re-usage, see `Runner types`_. + +**Scenario** - Type/class of measurement for example Ping, Pktgen, (Iperf, LmBench, ...) + +**SLA** - Relates to what result boundary a test case must meet to pass. For +example a latency limit, amount or ratio of lost packets and so on. Action +based on :term:`SLA` can be configured, either just to log (monitor) or to stop +further testing (assert). The :term:`SLA` criteria is set in the benchmark +configuration file and evaluated by the runner. + + +Runner types +------------ + +There exists several predefined runner types to choose between when designing +a test scenario: + +**Arithmetic:** +Every test run arithmetically steps the specified input value(s) in the +test scenario, adding a value to the previous input value. It is also possible +to combine several input values for the same test case in different +combinations. + +Snippet of an Arithmetic runner configuration: +:: + + + runner: + type: Arithmetic + iterators: + - + name: stride + start: 64 + stop: 128 + step: 64 + +**Duration:** +The test runs for a specific period of time before completed. + +Snippet of a Duration runner configuration: +:: + + + runner: + type: Duration + duration: 30 + +**Sequence:** +The test changes a specified input value to the scenario. The input values +to the sequence are specified in a list in the benchmark configuration file. + +Snippet of a Sequence runner configuration: +:: + + + runner: + type: Sequence + scenario_option_name: packetsize + sequence: + - 100 + - 200 + - 250 + + +**Iteration:** +Tests are run a specified number of times before completed. + +Snippet of an Iteration runner configuration: +:: + + + runner: + type: Iteration + iterations: 2 + + + + +Use-Case View +============= +TBD(kubi) + +Logical View +============ +TBD(Qiliang) + +Process View (Test execution flow) +================================== +TBD(Limingjiang) + +Deployment View +=============== +TBD(Patrick) + + + + + + + + + + + + +Yardstick Directory structure +============================= + +**yardstick/** - Yardstick main directory. + +*ci/* - Used for continuous integration of Yardstick at different PODs and + with support for different installers. + +*docs/* - All documentation is stored here, such as configuration guides, + user guides and Yardstick descriptions. + +*etc/* - Used for test cases requiring specific POD configurations. + +*samples/* - test case samples are stored here, most of all scenario and + feature's samples are shown in this directory. + +*tests/* - Here both Yardstick internal tests (*functional/* and *unit/*) as + well as the test cases run to verify the NFVI (*opnfv/*) are stored. + Also configurations of what to run daily and weekly at the different + PODs is located here. + +*tools/* - Currently contains tools to build image for VMs which are deployed + by Heat. Currently contains how to build the yardstick-trusty-server + image with the different tools that are needed from within the image. + +*vTC/* - Contains the files for running the virtual Traffic Classifier tests. + +*yardstick/* - Contains the internals of Yardstick: Runners, Scenario, Contexts, + CLI parsing, keys, plotting tools, dispatcher and so on. + diff --git a/docs/userguide/index.rst b/docs/userguide/index.rst index 3cad237e2..72a92a69f 100644 --- a/docs/userguide/index.rst +++ b/docs/userguide/index.rst @@ -12,6 +12,7 @@ Yardstick Overview 01-introduction 02-methodology + architecture 04-vtc-overview apexlake_installation apexlake_api -- cgit 1.2.3-korg