diff options
Diffstat (limited to 'docs/userguide/architecture.rst')
| -rw-r--r-- | docs/userguide/architecture.rst | 184 |
1 files changed, 184 insertions, 0 deletions
diff --git a/docs/userguide/architecture.rst b/docs/userguide/architecture.rst new file mode 100644 index 000000000..ccf8d482c --- /dev/null +++ b/docs/userguide/architecture.rst @@ -0,0 +1,184 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2016 Huawei Technologies Co.,Ltd and others + +============ +Architecture +============ + +Abstract +======== +This chapter describes the yardstick framework software architecture. we will introduce it from Use-Case View, +Logical View, Process View and Deployment View. More technical details will be introduced in this chapter. + +Overview +======== + +Architecture overview +--------------------- +Yardstick is mainly written in Python, and test configurations are made +in YAML. Documentation is written in reStructuredText format, i.e. .rst +files. Yardstick is inspired by Rally. Yardstick is intended to run on a +computer with access and credentials to a cloud. The test case is described +in a configuration file given as an argument. + +How it works: the benchmark task configuration file is parsed and converted into +an internal model. The context part of the model is converted into a Heat +template and deployed into a stack. Each scenario is run using a runner, either +serially or in parallel. Each runner runs in its own subprocess executing +commands in a VM using SSH. The output of each scenario is written as json +records to a file or influxdb or http server, we use influxdb as the backend, +the test result will be shown with grafana. + + +Concept +------- +**Benchmark** - assess the relative performance of something + +**Benchmark** configuration file - describes a single test case in yaml format + +**Context** - The set of Cloud resources used by a scenario, such as user +names, image names, affinity rules and network configurations. A context is +converted into a simplified Heat template, which is used to deploy onto the +Openstack environment. + +**Data** - Output produced by running a benchmark, written to a file in json format + +**Runner** - Logic that determines how a test scenario is run and reported, for +example the number of test iterations, input value stepping and test duration. +Predefined runner types exist for re-usage, see `Runner types`_. + +**Scenario** - Type/class of measurement for example Ping, Pktgen, (Iperf, LmBench, ...) + +**SLA** - Relates to what result boundary a test case must meet to pass. For +example a latency limit, amount or ratio of lost packets and so on. Action +based on :term:`SLA` can be configured, either just to log (monitor) or to stop +further testing (assert). The :term:`SLA` criteria is set in the benchmark +configuration file and evaluated by the runner. + + +Runner types +------------ + +There exists several predefined runner types to choose between when designing +a test scenario: + +**Arithmetic:** +Every test run arithmetically steps the specified input value(s) in the +test scenario, adding a value to the previous input value. It is also possible +to combine several input values for the same test case in different +combinations. + +Snippet of an Arithmetic runner configuration: +:: + + + runner: + type: Arithmetic + iterators: + - + name: stride + start: 64 + stop: 128 + step: 64 + +**Duration:** +The test runs for a specific period of time before completed. + +Snippet of a Duration runner configuration: +:: + + + runner: + type: Duration + duration: 30 + +**Sequence:** +The test changes a specified input value to the scenario. The input values +to the sequence are specified in a list in the benchmark configuration file. + +Snippet of a Sequence runner configuration: +:: + + + runner: + type: Sequence + scenario_option_name: packetsize + sequence: + - 100 + - 200 + - 250 + + +**Iteration:** +Tests are run a specified number of times before completed. + +Snippet of an Iteration runner configuration: +:: + + + runner: + type: Iteration + iterations: 2 + + + + +Use-Case View +============= +TBD(kubi) + +Logical View +============ +TBD(Qiliang) + +Process View (Test execution flow) +================================== +TBD(Limingjiang) + +Deployment View +=============== +TBD(Patrick) + + + + + + + + + + + + +Yardstick Directory structure +============================= + +**yardstick/** - Yardstick main directory. + +*ci/* - Used for continuous integration of Yardstick at different PODs and + with support for different installers. + +*docs/* - All documentation is stored here, such as configuration guides, + user guides and Yardstick descriptions. + +*etc/* - Used for test cases requiring specific POD configurations. + +*samples/* - test case samples are stored here, most of all scenario and + feature's samples are shown in this directory. + +*tests/* - Here both Yardstick internal tests (*functional/* and *unit/*) as + well as the test cases run to verify the NFVI (*opnfv/*) are stored. + Also configurations of what to run daily and weekly at the different + PODs is located here. + +*tools/* - Currently contains tools to build image for VMs which are deployed + by Heat. Currently contains how to build the yardstick-trusty-server + image with the different tools that are needed from within the image. + +*vTC/* - Contains the files for running the virtual Traffic Classifier tests. + +*yardstick/* - Contains the internals of Yardstick: Runners, Scenario, Contexts, + CLI parsing, keys, plotting tools, dispatcher and so on. + |