From 754c1b2e6c66de99e1c1faa00fa76256bbf37e99 Mon Sep 17 00:00:00 2001 From: Deepak S Date: Mon, 13 Feb 2017 00:35:33 -0800 Subject: Docs: Adding preliminary documentation for Network Service Benchmarking. This patch adds overview & installation instruction to get started on VNF testing. Change-Id: I73c9616d7154f701f542ff9a49c32a9516902ff4 Signed-off-by: Deepak S --- docs/userguide/01-introduction.rst | 21 +- docs/userguide/07-installation.rst | 401 ---------------------------- docs/userguide/07-nsb-overview.rst | 177 ++++++++++++ docs/userguide/08-nsb_installation.rst | 253 ++++++++++++++++++ docs/userguide/08-yardstick_plugin.rst | 144 ---------- docs/userguide/09-installation.rst | 401 ++++++++++++++++++++++++++++ docs/userguide/09-result-store-InfluxDB.rst | 86 ------ docs/userguide/10-grafana.rst | 119 --------- docs/userguide/10-yardstick_plugin.rst | 144 ++++++++++ docs/userguide/11-list-of-tcs.rst | 129 --------- docs/userguide/11-result-store-InfluxDB.rst | 86 ++++++ docs/userguide/12-grafana.rst | 119 +++++++++ docs/userguide/13-list-of-tcs.rst | 129 +++++++++ docs/userguide/index.rst | 12 +- 14 files changed, 1331 insertions(+), 890 deletions(-) delete mode 100644 docs/userguide/07-installation.rst create mode 100644 docs/userguide/07-nsb-overview.rst create mode 100644 docs/userguide/08-nsb_installation.rst delete mode 100644 docs/userguide/08-yardstick_plugin.rst create mode 100644 docs/userguide/09-installation.rst delete mode 100644 docs/userguide/09-result-store-InfluxDB.rst delete mode 100644 docs/userguide/10-grafana.rst create mode 100644 docs/userguide/10-yardstick_plugin.rst delete mode 100644 docs/userguide/11-list-of-tcs.rst create mode 100644 docs/userguide/11-result-store-InfluxDB.rst create mode 100644 docs/userguide/12-grafana.rst create mode 100644 docs/userguide/13-list-of-tcs.rst (limited to 'docs') diff --git a/docs/userguide/01-introduction.rst b/docs/userguide/01-introduction.rst index 9d9cf0fb5..0e0eea002 100755 --- a/docs/userguide/01-introduction.rst +++ b/docs/userguide/01-introduction.rst @@ -46,18 +46,27 @@ This document consists of the following chapters: * Chapter :doc:`04-vtc-overview` provides information on the :term:`VTC`. * Chapter :doc:`05-apexlake_installation` provides instructions to install the - experimental framework *ApexLake* and chapter :doc:`06-apexlake_api` explains - how this framework is integrated in *Yardstick*. + experimental framework *ApexLake* -* Chapter :doc:`07-installation` provides instructions to install *Yardstick*. +* Chapter :doc:`06-apexlake_api` explains how this framework is integrated in + *Yardstick*. -* Chapter :doc:`08-yardstick_plugin` provides information on how to integrate +* Chapter :doc:`07-nsb-overview` describes the methodology implemented by the + yardstick - Network service benchmarking to test real world usecase for a + given VNF + +* Chapter :doc:`08-nsb_installation` provides instructions to install + *Yardstick - Network service benchmarking testing*. + +* Chapter :doc:`09-installation` provides instructions to install *Yardstick*. + +* Chapter :doc:`10-yardstick_plugin` provides information on how to integrate other OPNFV testing projects into *Yardstick*. -* Chapter :doc:`09-result-store-InfluxDB` provides inforamtion on how to run +* Chapter :doc:`11-result-store-InfluxDB` provides inforamtion on how to run plug-in test cases and store test results into community's InfluxDB. -* Chapter :doc:`10-list-of-tcs` includes a list of available Yardstick test +* Chapter :doc:`12-list-of-tcs` includes a list of available Yardstick test cases. diff --git a/docs/userguide/07-installation.rst b/docs/userguide/07-installation.rst deleted file mode 100644 index 9c2082a27..000000000 --- a/docs/userguide/07-installation.rst +++ /dev/null @@ -1,401 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, Ericsson AB, Huawei Technologies Co.,Ltd and others. - -Yardstick Installation -====================== - -Abstract --------- - -Yardstick supports installation on Ubuntu 14.04 or via a Docker image. The -installation procedure on Ubuntu 14.04 or via the docker image are detailed in -the section below. - -To use Yardstick you should have access to an OpenStack environment, with at -least Nova, Neutron, Glance, Keystone and Heat installed. - -The steps needed to run Yardstick are: - -1. Install Yardstick. -2. Load OpenStack environment variables. -3. Create a Neutron external network. -4. Build Yardstick flavor and a guest image. -5. Load the guest image into the OpenStack environment. -6. Create the test configuration .yaml file. -7. Run the test case. - - -Prerequisites -------------- - -The OPNFV deployment is out of the scope of this document but it can be -found in http://artifacts.opnfv.org/opnfvdocs/colorado/docs/configguide/index.html. -The OPNFV platform is considered as the System Under Test (SUT) in this -document. - -Several prerequisites are needed for Yardstick: - - #. A Jumphost to run Yardstick on - #. A Docker daemon shall be installed on the Jumphost - #. A public/external network created on the SUT - #. Connectivity from the Jumphost to the SUT public/external network - -WARNING: Connectivity from Jumphost is essential and it is of paramount -importance to make sure it is working before even considering to install -and run Yardstick. Make also sure you understand how your networking is -designed to work. - -NOTE: **Jumphost** refers to any server which meets the previous -requirements. Normally it is the same server from where the OPNFV -deployment has been triggered previously. - -NOTE: If your Jumphost is operating behind a company http proxy and/or -Firewall, please consult first the section `Proxy Support`_, towards -the end of this document. The section details some tips/tricks which -*may* be of help in a proxified environment. - - -Installing Yardstick on Ubuntu 14.04 ------------------------------------- - -.. _install-framework: - -You can install Yardstick framework directly on Ubuntu 14.04 or in an Ubuntu -14.04 Docker image. No matter which way you choose to install Yardstick -framework, the following installation steps are identical. - -If you choose to use the Ubuntu 14.04 Docker image, You can pull the Ubuntu -14.04 Docker image from Docker hub: - -:: - - docker pull ubuntu:14.04 - -Installing Yardstick framework -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Download source code and install python dependencies: - -:: - - git clone https://gerrit.opnfv.org/gerrit/yardstick - cd yardstick - ./install.sh - - -Installing Yardstick using Docker ---------------------------------- - -Yardstick has a Docker image, this Docker image (**Yardstick-stable**) -serves as a replacement for installing the Yardstick framework in a virtual -environment (for example as done in :ref:`install-framework`). -It is recommended to use this Docker image to run Yardstick test. - -Pulling the Yardstick Docker image -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. _dockerhub: https://hub.docker.com/r/opnfv/yardstick/ - -Pull the Yardstick Docker image ('opnfv/yardstick') from the public dockerhub -registry under the OPNFV account: [dockerhub_], with the following docker -command:: - - docker pull opnfv/yardstick:stable - -After pulling the Docker image, check that it is available with the -following docker command:: - - [yardsticker@jumphost ~]$ docker images - REPOSITORY TAG IMAGE ID CREATED SIZE - opnfv/yardstick stable a4501714757a 1 day ago 915.4 MB - -Run the Docker image: - -:: - - docker run --privileged=true -it opnfv/yardstick:stable /bin/bash - -In the container the Yardstick repository is located in the /home/opnfv/repos -directory. - - -OpenStack parameters and credentials ------------------------------------- - -Environment variables -^^^^^^^^^^^^^^^^^^^^^ -Before running Yardstick it is necessary to export OpenStack environment variables -from the OpenStack *openrc* file (using the ``source`` command) and export the -external network name ``export EXTERNAL_NETWORK="external-network-name"``, -the default name for the external network is ``net04_ext``. - -Credential environment variables in the *openrc* file have to include at least: - -* OS_AUTH_URL -* OS_USERNAME -* OS_PASSWORD -* OS_TENANT_NAME - -A sample openrc file may look like this: - -* export OS_PASSWORD=console -* export OS_TENANT_NAME=admin -* export OS_AUTH_URL=http://172.16.1.222:35357/v2.0 -* export OS_USERNAME=admin -* export OS_VOLUME_API_VERSION=2 -* export EXTERNAL_NETWORK=net04_ext - - -Yardstick falvor and guest images ---------------------------------- - -Before executing Yardstick test cases, make sure that yardstick guest image and -yardstick flavor are available in OpenStack. -Detailed steps about creating yardstick flavor and building yardstick-trusty-server -image can be found below. - -Yardstick-flavor -^^^^^^^^^^^^^^^^ -Most of the sample test cases in Yardstick are using an OpenStack flavor called -*yardstick-flavor* which deviates from the OpenStack standard m1.tiny flavor by the -disk size - instead of 1GB it has 3GB. Other parameters are the same as in m1.tiny. - -Create yardstick-flavor: - -:: - - nova flavor-create yardstick-flavor 100 512 3 1 - - -.. _guest-image: - -Building a guest image -^^^^^^^^^^^^^^^^^^^^^^ -Most of the sample test cases in Yardstick are using a guest image called -*yardstick-trusty-server* which deviates from an Ubuntu Cloud Server image -containing all the required tools to run test cases supported by Yardstick. -Yardstick has a tool for building this custom image. It is necessary to have -sudo rights to use this tool. - -Also you may need install several additional packages to use this tool, by -follwing the commands below: - -:: - - apt-get update && apt-get install -y \ - qemu-utils \ - kpartx - -This image can be built using the following command while in the directory where -Yardstick is installed (``~/yardstick`` if the framework is installed -by following the commands above): - -:: - - export YARD_IMG_ARCH="amd64" - sudo echo "Defaults env_keep += \"YARD_IMG_ARCH\"" >> /etc/sudoers - sudo ./tools/yardstick-img-modify tools/ubuntu-server-cloudimg-modify.sh - -**Warning:** the script will create files by default in: -``/tmp/workspace/yardstick`` and the files will be owned by root! - -If you are building this guest image in inside a docker container make sure the -container is granted with privilege. - -The created image can be added to OpenStack using the ``glance image-create`` or -via the OpenStack Dashboard. - -Example command: - -:: - - glance --os-image-api-version 1 image-create \ - --name yardstick-image --is-public true \ - --disk-format qcow2 --container-format bare \ - --file /tmp/workspace/yardstick/yardstick-image.img - -Some Yardstick test cases use a Cirros image, you can find one at -http://download.cirros-cloud.net/0.3.3/cirros-0.3.3-x86_64-disk.img - - -Automatic flavor and image creation ------------------------------------ -Yardstick has a script for automatic creating yardstick flavor and building -guest images. This script is mainly used in CI, but you can still use it in -your local environment. - -Example command: - -:: - - export YARD_IMG_ARCH="amd64" - sudo echo "Defaults env_keep += \"YARD_IMG_ARCH\"" >> /etc/sudoers - source $YARDSTICK_REPO_DIR/tests/ci/load_images.sh - - -Yardstick default key pair -^^^^^^^^^^^^^^^^^^^^^^^^^^ -Yardstick uses a SSH key pair to connect to the guest image. This key pair can -be found in the ``resources/files`` directory. To run the ``ping-hot.yaml`` test -sample, this key pair needs to be imported to the OpenStack environment. - - -Examples and verifying the install ----------------------------------- - -It is recommended to verify that Yardstick was installed successfully -by executing some simple commands and test samples. Before executing yardstick -test cases make sure yardstick flavor and building yardstick-trusty-server -image can be found in glance and openrc file is sourced. Below is an example -invocation of yardstick help command and ping.py test sample: -:: - - yardstick –h - yardstick task start samples/ping.yaml - -Each testing tool supported by Yardstick has a sample configuration file. -These configuration files can be found in the **samples** directory. - -Default location for the output is ``/tmp/yardstick.out``. - - -Deploy InfluxDB and Grafana locally ------------------------------------- - -.. pull docker images - -Pull docker images - -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -:: - - docker pull tutum/influxdb - docker pull grafana/grafana - -Run influxdb and config -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Run influxdb -:: - - docker run -d --name influxdb \ - -p 8083:8083 -p 8086:8086 --expose 8090 --expose 8099 \ - tutum/influxdb - docker exec -it influxdb bash - -Config influxdb -:: - - influx - >CREATE USER root WITH PASSWORD 'root' WITH ALL PRIVILEGES - >CREATE DATABASE yardstick; - >use yardstick; - >show MEASUREMENTS; - -Run grafana and config -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Run grafana -:: - - docker run -d --name grafana -p 3000:3000 grafana/grafana - -Config grafana -:: - - http://{YOUR_IP_HERE}:3000 - log on using admin/admin and config database resource to be {YOUR_IP_HERE}:8086 - -.. image:: images/Grafana_config.png - :width: 800px - :alt: Grafana data source configration - -Config yardstick conf -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -cp ./etc/yardstick/yardstick.conf.sample /etc/yardstick/yardstick.conf - -vi /etc/yardstick/yardstick.conf -Config yardstick.conf -:: - - [DEFAULT] - debug = True - dispatcher = influxdb - - [dispatcher_influxdb] - timeout = 5 - target = http://{YOUR_IP_HERE}:8086 - db_name = yardstick - username = root - password = root - -Now you can run yardstick test cases and store the results in influxdb -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - - -Create a test suite for yardstick ------------------------------------- - -A test suite in yardstick is a yaml file which include one or more test cases. -Yardstick is able to support running test suite task, so you can customize you -own test suite and run it in one task. - -"tests/opnfv/test_suites" is where yardstick put ci test-suite. A typical test -suite is like below: - -fuel_test_suite.yaml - -:: - - --- - # Fuel integration test task suite - - schema: "yardstick:suite:0.1" - - name: "fuel_test_suite" - test_cases_dir: "samples/" - test_cases: - - - file_name: ping.yaml - - - file_name: iperf3.yaml - -As you can see, there are two test cases in fuel_test_suite, the syntax is simple -here, you must specify the schema and the name, then you just need to list the -test cases in the tag "test_cases" and also mark their relative directory in the -tag "test_cases_dir". - -Yardstick test suite also support constraints and task args for each test case. -Here is another sample to show this, which is digested from one big test suite. - -os-nosdn-nofeature-ha.yaml - -:: - - --- - - schema: "yardstick:suite:0.1" - - name: "os-nosdn-nofeature-ha" - test_cases_dir: "tests/opnfv/test_cases/" - test_cases: - - - file_name: opnfv_yardstick_tc002.yaml - - - file_name: opnfv_yardstick_tc005.yaml - - - file_name: opnfv_yardstick_tc043.yaml - constraint: - installer: compass - pod: huawei-pod1 - task_args: - huawei-pod1: '{"pod_info": "etc/yardstick/.../pod.yaml", - "host": "node4.LF","target": "node5.LF"}' - -As you can see in test case "opnfv_yardstick_tc043.yaml", there are two tags, "constraint" and -"task_args". "constraint" is where you can specify which installer or pod it can be run in -the ci environment. "task_args" is where you can specify the task arguments for each pod. - -All in all, to create a test suite in yardstick, you just need to create a suite yaml file -and add test cases and constraint or task arguments if necessary. - diff --git a/docs/userguide/07-nsb-overview.rst b/docs/userguide/07-nsb-overview.rst new file mode 100644 index 000000000..19719f1a7 --- /dev/null +++ b/docs/userguide/07-nsb-overview.rst @@ -0,0 +1,177 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2016-2017 Intel Corporation. + +===================================== +Network Services Benchmarking (NSB) +===================================== + +Abstract +======== + +.. _Yardstick: https://wiki.opnfv.org/yardstick + +This chapter provides an overview of the NSB, a contribution to OPNFV +Yardstick_ from Intel. + +Overview +======== + +GOAL: Extend Yardstick to perform real world VNFs and NFVi Characterization and +benchmarking with repeatable and deterministic methods. + +The Network Service Benchmarking (NSB) extends the yardstick framework to do +VNF characterization and benchmarking in three different execution +environments viz., bare metal i.e. native Linux environment, standalone virtual +environment and managed virtualized environment (e.g. Open stack etc.). +It also brings in the capability to interact with external traffic generators +both hardware & software based for triggering and validating the traffic +according to user defined profiles. + +NSB extension includes: + • Generic data models of Network Services, based on ETSI specs + • New Standalone context for VNF testing like SRIOV, OVS, OVS-DPDK etc + • Generic VNF configuration models and metrics implemented with Python + classes + • Traffic generator features and traffic profiles + • L1-L3 state-less traffic profiles + • L4-L7 state-full traffic profiles + • Tunneling protocol / network overlay support + • Test case samples + • Ping + • Trex + • vPE,vCGNAT, vFirewall etc - ipv4 throughput, latency etc + • Traffic generators like Trex, ab/nginx, ixia, iperf etc + • KPIs for a given use case: + • System agent support for collecting NFvi KPI. This includes: + o CPU statistic + o Memory BW + o OVS-DPDK Stats + • Network KPIs – eg, inpackets, outpackets, thoughput, latency etc + • VNF KPIs – packet_in, packet_drop, packet_fwd etc + +Architecture +============ +The Network Service (NS) defines a set of Virtual Network Functions (VNF) +connected together using NFV infrastructure. + +The Yardstick NSB extension can support multiple VNFs created by different +vendors including traffic generators. Every VNF being tested has its +own data model. The Network service defines a VNF modelling on base of performed +network functionality. The part of the data model is a set of the configuration +parameters, number of connection points used and flavor including core and +memory amount. + +The ETSI defines a Network Service as a set of configurable VNFs working in +some NFV Infrastructure connecting each other using Virtual Links available +through Connection Points. The ETSI MANO specification defines a set of +management entities called Network Service Descriptors (NSD) and +VNF Descriptors (VNFD) that define real Network Service. The picture below +makes an example how the real Network Operator use-case can map into ETSI +Network service definition + +Network Service framework performs the necessary test steps. It may involve + o Interacting with traffic generator and providing the inputs on traffic + type / packet structure to generate the required traffic as per the + test case. Traffic profiles will be used for this. + o Executing the commands required for the test procedure and analyses the + command output for confirming whether the command got executed correctly + or not. E.g. As per the test case, run the traffic for the given + time period / wait for the necessary time delay + o Verify the test result. + o Validate the traffic flow from SUT + o Fetch the table / data from SUT and verify the value as per the test case + o Upload the logs from SUT onto the Test Harness server + o Read the KPI’s provided by particular VNF + +Components of Network Service +------------------------------ + +* *Models for Network Service benchmarking*: The Network Service benchmarking + requires the proper modelling approach. The NSB provides models using Python + files and defining of NSDs and VNFDs. + +The benchmark control application being a part of OPNFV yardstick can call +that python models to instantiate and configure the VNFs. Depending on +infrastructure type (bare-metal or fully virtualized) that calls could be +made directly or using MANO system. + +* *Traffic generators in NSB*: Any benchmark application requires a set of + traffic generator and traffic profiles defining the method in which traffic + is generated. + +The Network Service benchmarking model extends the Network Service +definition with a set of Traffic Generators (TG) that are treated +same way as other VNFs being a part of benchmarked network service. +Same as other VNFs the traffic generator are instantiated and terminated. + +Every traffic generator has own configuration defined as a traffic profile and +a set of KPIs supported. The python models for TG is extended by specific calls +to listen and generate traffic. + +* *The stateless TREX traffic generator*: The main traffic generator used as + Network Service stimulus is open source TREX tool. + +The TREX tool can generate any kind of stateless traffic. + +.. code-block:: console + + +--------+ +-------+ +--------+ + | | | | | | + | Trex | ---> | VNF | ---> | Trex | + | | | | | | + +--------+ +-------+ +--------+ + +Supported testcases scenarios: +• Correlated UDP traffic using TREX traffic generator and replay VNF. + o using different IMIX configuration like pure voice, pure video traffic etc + o using different number IP flows like 1 flow, 1K, 16K, 64K, 256K, 1M flows + o Using different number of rules configured like 1 rule, 1K, 10K rules + +For UDP correlated traffic following Key Performance Indicators are collected +for every combination of test case parameters: + • RFC2544 throughput for various loss rate defined (1% is a default) + +Graphical Overview +================== + +NSB Testing with yardstick framework facilitate performance testing of various +VNFs provided. + +.. code-block:: console + +-----------+ + | | +-----------+ + | vPE | ->|TGen Port 0| + | TestCase | | +-----------+ + | | | + +-----------+ +------------------+ +-------+ | + | | -- API --> | VNF | <---> + +-----------+ | Yardstick | +-------+ | + | Test Case | --> | NSB Testing | | + +-----------+ | | | + | | | | + | +------------------+ | + +-----------+ | +-----------+ + | Traffic | ->|TGen Port 1| + | patterns | +-----------+ + +-----------+ + Figure 1: Network Service - 2 server configuration + + +Install +======= + +run the nsb_install.sh with root privileges + +Run +=== + +source ~/.bash_profile +cd /yardstick/cmd +sudo -E ./NSBperf.py --vnf vpe --test tc_baremetal_rfc2544_ipv4_1flow_64B.yaml + +Development Environment +======================= + +Ubuntu 14.04, Ubuntu 16.04 diff --git a/docs/userguide/08-nsb_installation.rst b/docs/userguide/08-nsb_installation.rst new file mode 100644 index 000000000..a390bb7d7 --- /dev/null +++ b/docs/userguide/08-nsb_installation.rst @@ -0,0 +1,253 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2016-2017 Intel Corporation. + +Yardstick - NSB Testing -Installation +===================================== + +Abstract +-------- + +Yardstick supports installation on Ubuntu 14.04 or via a Docker image. The +installation procedure on Ubuntu 14.04 or via the docker image are detailed in +the section below. + +The Network Service Benchmarking (NSB) extends the yardstick framework to do +VNF characterization and benchmarking in three different execution +environments viz., bare metal i.e. native Linux environment, standalone virtual +environment and managed virtualized environment (e.g. Open stack etc.). +It also brings in the capability to interact with external traffic generators +both hardware & software based for triggering and validating the traffic +according to user defined profiles. + +The steps needed to run Yardstick with NSB testing are: + +* Install Yardstick (NSB Testing). +* Setup pod.yaml describing Test topology +* Create the test configuration yaml file. +* Run the test case. + + +Prerequisites +------------- + +Refer chapter 08-instalaltion.rst for more information on yardstick +prerequisites + +Several prerequisites are needed for Yardstick(VNF testing): +* Python Modules: pyzmq, pika. +* flex +* bison +* build-essential +* automake +* libtool +* librabbitmq-dev +* rabbitmq-server +* collectd +* intel-cmt-cat + +Installing Yardstick on Ubuntu 14.04 +------------------------------------ + +.. _install-framework: + +You can install Yardstick framework directly on Ubuntu 14.04 or in an Ubuntu +14.04 Docker image. No matter which way you choose to install Yardstick +framework, the following installation steps are identical. + +If you choose to use the Ubuntu 14.04 Docker image, You can pull the Ubuntu +14.04 Docker image from Docker hub: + +:: + + docker pull ubuntu:14.04 + +Installing Yardstick framework +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Download source code and install python dependencies: + +:: + + git clone https://gerrit.opnfv.org/gerrit/yardstick + cd yardstick + ./nsb_setup.sh + +It will automatically download all the packages needed for NSB Testing setup. + +System Topology: +----------------- + +.. code-block:: console + + +----------+ +----------+ + | | | | + | | (0)----->(0) | Ping/ | + | TG1 | | vPE/ | + | | | 2Trex | + | | (1)<-----(1) | | + +----------+ +----------+ + trafficgen_1 vnf + + +OpenStack parameters and credentials +------------------------------------ + +Environment variables +^^^^^^^^^^^^^^^^^^^^^ +Before running Yardstick (NSB Testing) it is necessary to export traffic +generator libraries. + +:: + source ~/.bash_profile + +Config yardstick conf +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +cp ./etc/yardstick/yardstick.conf.sample /etc/yardstick/yardstick.conf + +vi /etc/yardstick/yardstick.conf + +Config yardstick.conf +:: + + [DEFAULT] + debug = True + dispatcher = influxdb + + [dispatcher_influxdb] + timeout = 5 + target = http://{YOUR_IP_HERE}:8086 + db_name = yardstick + username = root + password = root + + [nsb] + trex_path=/opt/nsb_bin/trex/scripts + bin_path=/opt/nsb_bin + + +Config pod.yaml describing Topology +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Before executing Yardstick test cases, make sure that pod.yaml reflects the +topology and update all the required fields. + +copy /etc/yardstick/nodes/pod.yaml.nsb.example to /etc/yardstick/nodes/pod.yaml + +Config pod.yaml +:: + nodes: + - + name: trafficgen_1 + role: TrafficGen + ip: 1.1.1.1 + user: root + password: r00t + interfaces: + xe0: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.0" + driver: i40e # default kernel driver + dpdk_port_num: 0 + local_ip: "152.16.100.20" + netmask: "255.255.255.0" + local_mac: "00:00:00:00:00:01" + xe1: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.1" + driver: i40e # default kernel driver + dpdk_port_num: 1 + local_ip: "152.16.40.20" + netmask: "255.255.255.0" + local_mac: "00:00.00:00:00:02" + + - + name: vnf + role: vnf + ip: 1.1.1.2 + user: root + password: r00t + host: 1.1.1.2 #BM - host == ip, virtualized env - Host - compute node + interfaces: + xe0: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.0" + driver: i40e # default kernel driver + dpdk_port_num: 0 + local_ip: "152.16.100.19" + netmask: "255.255.255.0" + local_mac: "00:00:00:00:00:03" + + xe1: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.1" + driver: i40e # default kernel driver + dpdk_port_num: 1 + local_ip: "152.16.40.19" + netmask: "255.255.255.0" + local_mac: "00:00:00:00:00:04" + routing_table: + - network: "152.16.100.20" + netmask: "255.255.255.0" + gateway: "152.16.100.20" + if: "xe0" + - network: "152.16.40.20" + netmask: "255.255.255.0" + gateway: "152.16.40.20" + if: "xe1" + nd_route_tbl: + - network: "0064:ff9b:0:0:0:0:9810:6414" + netmask: "112" + gateway: "0064:ff9b:0:0:0:0:9810:6414" + if: "xe0" + - network: "0064:ff9b:0:0:0:0:9810:2814" + netmask: "112" + gateway: "0064:ff9b:0:0:0:0:9810:2814" + if: "xe1" + +Enable yardstick virtual environment +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Before executing yardstick test cases, make sure to activate yardstick +python virtual environment + +:: + source /opt/nsb_bin/yardstick_venv/bin/activate + + +Examples and verifying the install +---------------------------------- + +It is recommended to verify that Yardstick was installed successfully +by executing some simple commands and test samples. Before executing yardstick +test cases make sure yardstick flavor and building yardstick-trusty-server +image can be found in glance and openrc file is sourced. Below is an example +invocation of yardstick help command and ping.py test sample: +:: + + yardstick –h + yardstick task start samples/ping.yaml + +Each testing tool supported by Yardstick has a sample configuration file. +These configuration files can be found in the **samples** directory. + +Default location for the output is ``/tmp/yardstick.out``. + + +Run Yardstick - Network Service Testcases +----------------------------------------- + +NS testing - using NSBperf CLI +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +:: + + source /opt/nsb_setup/yardstick_venv/bin/activate + PYTHONPATH: ". ~/.bash_profile" + cd /yardstick/cmd + Execute command: ./NSPerf.py -h + ./NSBperf.py --vnf --test + eg: ./NSBperf.py --vnf vpe --test tc_baremetal_rfc2544_ipv4_1flow_64B.yaml + +NS testing - using yardstick CLI +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +:: + + source /opt/nsb_setup/yardstick_venv/bin/activate + PYTHONPATH: ". ~/.bash_profile" + Go to test case forlder type we want to execute. + e.g. /samples/vnf_samples/nsut// + run: yardstick --debug task start diff --git a/docs/userguide/08-yardstick_plugin.rst b/docs/userguide/08-yardstick_plugin.rst deleted file mode 100644 index f16dedd02..000000000 --- a/docs/userguide/08-yardstick_plugin.rst +++ /dev/null @@ -1,144 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, Ericsson AB, Huawei Technologies Co.,Ltd and others. - -=================================== -Installing a plug-in into yardstick -=================================== - -Abstract -======== - -Yardstick currently provides a ``plugin`` CLI command to support integration -with other OPNFV testing projects. Below is an example invocation of yardstick -plugin command and Storperf plug-in sample. - - -Installing Storperf into yardstick -================================== - -Storperf is delivered as a Docker container from -https://hub.docker.com/r/opnfv/storperf/tags/. - -There are two possible methods for installation in your environment: - -* Run container on Jump Host -* Run container in a VM - -In this introduction we will install Storperf on Jump Host. - - -Step 0: Environment preparation ->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - -Running Storperf on Jump Host -Requirements: - -* Docker must be installed -* Jump Host must have access to the OpenStack Controller API -* Jump Host must have internet connectivity for downloading docker image -* Enough floating IPs must be available to match your agent count - -Before installing Storperf into yardstick you need to check your openstack -environment and other dependencies: - -1. Make sure docker is installed. -2. Make sure Keystone, Nova, Neutron, Glance, Heat are installed correctly. -3. Make sure Jump Host have access to the OpenStack Controller API. -4. Make sure Jump Host must have internet connectivity for downloading docker image. -5. You need to know where to get basic openstack Keystone authorization info, such as - OS_PASSWORD, OS_TENANT_NAME, OS_AUTH_URL, OS_USERNAME. -6. To run a Storperf container, you need to have OpenStack Controller environment - variables defined and passed to Storperf container. The best way to do this is to - put environment variables in a "storperf_admin-rc" file. The storperf_admin-rc - should include credential environment variables at least: - -* OS_AUTH_URL -* OS_TENANT_ID -* OS_TENANT_NAME -* OS_PROJECT_NAME -* OS_USERNAME -* OS_PASSWORD -* OS_REGION_NAME - -For this storperf_admin-rc file, during environment preparation a "prepare_storperf_admin-rc.sh" -script can be used to generate it. -:: - - #!/bin/bash - AUTH_URL=${OS_AUTH_URL} - USERNAME=${OS_USERNAME:-admin} - PASSWORD=${OS_PASSWORD:-console} - TENANT_NAME=${OS_TENANT_NAME:-admin} - VOLUME_API_VERSION=${OS_VOLUME_API_VERSION:-2} - PROJECT_NAME=${OS_PROJECT_NAME:-$TENANT_NAME} - TENANT_ID=`keystone tenant-get admin|grep 'id'|awk -F '|' '{print $3}'|sed -e 's/^[[:space:]]*//'` - rm -f ~/storperf_admin-rc - touch ~/storperf_admin-rc - echo "OS_AUTH_URL="$AUTH_URL >> ~/storperf_admin-rc - echo "OS_USERNAME="$USERNAME >> ~/storperf_admin-rc - echo "OS_PASSWORD="$PASSWORD >> ~/storperf_admin-rc - echo "OS_TENANT_NAME="$TENANT_NAME >> ~/storperf_admin-rc - echo "OS_VOLUME_API_VERSION="$VOLUME_API_VERSION >> ~/storperf_admin-rc - echo "OS_PROJECT_NAME="$PROJECT_NAME >> ~/storperf_admin-rc - echo "OS_TENANT_ID="$TENANT_ID >> ~/storperf_admin-rc - - -Step 1: Plug-in configuration file preparation -++++++++++++++++++++++++++++++++++++++++++++++ - -To install a plug-in, first you need to prepare a plug-in configuration file in -YAML format and store it in the "plugin" directory. The plugin configration file -work as the input of yardstick "plugin" command. Below is the Storperf plug-in -configuration file sample: -:: - - --- - # StorPerf plugin configuration file - # Used for integration StorPerf into Yardstick as a plugin - schema: "yardstick:plugin:0.1" - plugins: - name: storperf - deployment: - ip: 192.168.23.2 - user: root - password: root - -In the plug-in configuration file, you need to specify the plug-in name and the -plug-in deployment info, including node ip, node login username and password. -Here the Storperf will be installed on IP 192.168.23.2 which is the Jump Host -in my local environment. - -Step 2: Plug-in install/remove scripts preparation -++++++++++++++++++++++++++++++++++++++++++++++++++ - -Under "yardstick/resource/scripts directory", there are two folders: a "install" -folder and a "remove" folder. You need to store the plug-in install/remove script -in these two folders respectively. - -The detailed installation or remove operation should de defined in these two scripts. -The name of both install and remove scripts should match the plugin-in name that you -specified in the plug-in configuration file. -For example, the install and remove scripts for Storperf are both named to "storperf.bash". - - -Step 3: Install and remove Storperf -+++++++++++++++++++++++++++++++++++ - -To install Storperf, simply execute the following command -:: - - # Install Storperf - yardstick plugin install plugin/storperf.yaml - -removing Storperf from yardstick -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To remove Storperf, simply execute the following command -:: - - # Remove Storperf - yardstick plugin remove plugin/storperf.yaml - -What yardstick plugin command does is using the username and password to log into the deployment target and then execute the corresponding install or remove script. diff --git a/docs/userguide/09-installation.rst b/docs/userguide/09-installation.rst new file mode 100644 index 000000000..9c2082a27 --- /dev/null +++ b/docs/userguide/09-installation.rst @@ -0,0 +1,401 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Ericsson AB, Huawei Technologies Co.,Ltd and others. + +Yardstick Installation +====================== + +Abstract +-------- + +Yardstick supports installation on Ubuntu 14.04 or via a Docker image. The +installation procedure on Ubuntu 14.04 or via the docker image are detailed in +the section below. + +To use Yardstick you should have access to an OpenStack environment, with at +least Nova, Neutron, Glance, Keystone and Heat installed. + +The steps needed to run Yardstick are: + +1. Install Yardstick. +2. Load OpenStack environment variables. +3. Create a Neutron external network. +4. Build Yardstick flavor and a guest image. +5. Load the guest image into the OpenStack environment. +6. Create the test configuration .yaml file. +7. Run the test case. + + +Prerequisites +------------- + +The OPNFV deployment is out of the scope of this document but it can be +found in http://artifacts.opnfv.org/opnfvdocs/colorado/docs/configguide/index.html. +The OPNFV platform is considered as the System Under Test (SUT) in this +document. + +Several prerequisites are needed for Yardstick: + + #. A Jumphost to run Yardstick on + #. A Docker daemon shall be installed on the Jumphost + #. A public/external network created on the SUT + #. Connectivity from the Jumphost to the SUT public/external network + +WARNING: Connectivity from Jumphost is essential and it is of paramount +importance to make sure it is working before even considering to install +and run Yardstick. Make also sure you understand how your networking is +designed to work. + +NOTE: **Jumphost** refers to any server which meets the previous +requirements. Normally it is the same server from where the OPNFV +deployment has been triggered previously. + +NOTE: If your Jumphost is operating behind a company http proxy and/or +Firewall, please consult first the section `Proxy Support`_, towards +the end of this document. The section details some tips/tricks which +*may* be of help in a proxified environment. + + +Installing Yardstick on Ubuntu 14.04 +------------------------------------ + +.. _install-framework: + +You can install Yardstick framework directly on Ubuntu 14.04 or in an Ubuntu +14.04 Docker image. No matter which way you choose to install Yardstick +framework, the following installation steps are identical. + +If you choose to use the Ubuntu 14.04 Docker image, You can pull the Ubuntu +14.04 Docker image from Docker hub: + +:: + + docker pull ubuntu:14.04 + +Installing Yardstick framework +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Download source code and install python dependencies: + +:: + + git clone https://gerrit.opnfv.org/gerrit/yardstick + cd yardstick + ./install.sh + + +Installing Yardstick using Docker +--------------------------------- + +Yardstick has a Docker image, this Docker image (**Yardstick-stable**) +serves as a replacement for installing the Yardstick framework in a virtual +environment (for example as done in :ref:`install-framework`). +It is recommended to use this Docker image to run Yardstick test. + +Pulling the Yardstick Docker image +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. _dockerhub: https://hub.docker.com/r/opnfv/yardstick/ + +Pull the Yardstick Docker image ('opnfv/yardstick') from the public dockerhub +registry under the OPNFV account: [dockerhub_], with the following docker +command:: + + docker pull opnfv/yardstick:stable + +After pulling the Docker image, check that it is available with the +following docker command:: + + [yardsticker@jumphost ~]$ docker images + REPOSITORY TAG IMAGE ID CREATED SIZE + opnfv/yardstick stable a4501714757a 1 day ago 915.4 MB + +Run the Docker image: + +:: + + docker run --privileged=true -it opnfv/yardstick:stable /bin/bash + +In the container the Yardstick repository is located in the /home/opnfv/repos +directory. + + +OpenStack parameters and credentials +------------------------------------ + +Environment variables +^^^^^^^^^^^^^^^^^^^^^ +Before running Yardstick it is necessary to export OpenStack environment variables +from the OpenStack *openrc* file (using the ``source`` command) and export the +external network name ``export EXTERNAL_NETWORK="external-network-name"``, +the default name for the external network is ``net04_ext``. + +Credential environment variables in the *openrc* file have to include at least: + +* OS_AUTH_URL +* OS_USERNAME +* OS_PASSWORD +* OS_TENANT_NAME + +A sample openrc file may look like this: + +* export OS_PASSWORD=console +* export OS_TENANT_NAME=admin +* export OS_AUTH_URL=http://172.16.1.222:35357/v2.0 +* export OS_USERNAME=admin +* export OS_VOLUME_API_VERSION=2 +* export EXTERNAL_NETWORK=net04_ext + + +Yardstick falvor and guest images +--------------------------------- + +Before executing Yardstick test cases, make sure that yardstick guest image and +yardstick flavor are available in OpenStack. +Detailed steps about creating yardstick flavor and building yardstick-trusty-server +image can be found below. + +Yardstick-flavor +^^^^^^^^^^^^^^^^ +Most of the sample test cases in Yardstick are using an OpenStack flavor called +*yardstick-flavor* which deviates from the OpenStack standard m1.tiny flavor by the +disk size - instead of 1GB it has 3GB. Other parameters are the same as in m1.tiny. + +Create yardstick-flavor: + +:: + + nova flavor-create yardstick-flavor 100 512 3 1 + + +.. _guest-image: + +Building a guest image +^^^^^^^^^^^^^^^^^^^^^^ +Most of the sample test cases in Yardstick are using a guest image called +*yardstick-trusty-server* which deviates from an Ubuntu Cloud Server image +containing all the required tools to run test cases supported by Yardstick. +Yardstick has a tool for building this custom image. It is necessary to have +sudo rights to use this tool. + +Also you may need install several additional packages to use this tool, by +follwing the commands below: + +:: + + apt-get update && apt-get install -y \ + qemu-utils \ + kpartx + +This image can be built using the following command while in the directory where +Yardstick is installed (``~/yardstick`` if the framework is installed +by following the commands above): + +:: + + export YARD_IMG_ARCH="amd64" + sudo echo "Defaults env_keep += \"YARD_IMG_ARCH\"" >> /etc/sudoers + sudo ./tools/yardstick-img-modify tools/ubuntu-server-cloudimg-modify.sh + +**Warning:** the script will create files by default in: +``/tmp/workspace/yardstick`` and the files will be owned by root! + +If you are building this guest image in inside a docker container make sure the +container is granted with privilege. + +The created image can be added to OpenStack using the ``glance image-create`` or +via the OpenStack Dashboard. + +Example command: + +:: + + glance --os-image-api-version 1 image-create \ + --name yardstick-image --is-public true \ + --disk-format qcow2 --container-format bare \ + --file /tmp/workspace/yardstick/yardstick-image.img + +Some Yardstick test cases use a Cirros image, you can find one at +http://download.cirros-cloud.net/0.3.3/cirros-0.3.3-x86_64-disk.img + + +Automatic flavor and image creation +----------------------------------- +Yardstick has a script for automatic creating yardstick flavor and building +guest images. This script is mainly used in CI, but you can still use it in +your local environment. + +Example command: + +:: + + export YARD_IMG_ARCH="amd64" + sudo echo "Defaults env_keep += \"YARD_IMG_ARCH\"" >> /etc/sudoers + source $YARDSTICK_REPO_DIR/tests/ci/load_images.sh + + +Yardstick default key pair +^^^^^^^^^^^^^^^^^^^^^^^^^^ +Yardstick uses a SSH key pair to connect to the guest image. This key pair can +be found in the ``resources/files`` directory. To run the ``ping-hot.yaml`` test +sample, this key pair needs to be imported to the OpenStack environment. + + +Examples and verifying the install +---------------------------------- + +It is recommended to verify that Yardstick was installed successfully +by executing some simple commands and test samples. Before executing yardstick +test cases make sure yardstick flavor and building yardstick-trusty-server +image can be found in glance and openrc file is sourced. Below is an example +invocation of yardstick help command and ping.py test sample: +:: + + yardstick –h + yardstick task start samples/ping.yaml + +Each testing tool supported by Yardstick has a sample configuration file. +These configuration files can be found in the **samples** directory. + +Default location for the output is ``/tmp/yardstick.out``. + + +Deploy InfluxDB and Grafana locally +------------------------------------ + +.. pull docker images + +Pull docker images + +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +:: + + docker pull tutum/influxdb + docker pull grafana/grafana + +Run influxdb and config +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Run influxdb +:: + + docker run -d --name influxdb \ + -p 8083:8083 -p 8086:8086 --expose 8090 --expose 8099 \ + tutum/influxdb + docker exec -it influxdb bash + +Config influxdb +:: + + influx + >CREATE USER root WITH PASSWORD 'root' WITH ALL PRIVILEGES + >CREATE DATABASE yardstick; + >use yardstick; + >show MEASUREMENTS; + +Run grafana and config +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Run grafana +:: + + docker run -d --name grafana -p 3000:3000 grafana/grafana + +Config grafana +:: + + http://{YOUR_IP_HERE}:3000 + log on using admin/admin and config database resource to be {YOUR_IP_HERE}:8086 + +.. image:: images/Grafana_config.png + :width: 800px + :alt: Grafana data source configration + +Config yardstick conf +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +cp ./etc/yardstick/yardstick.conf.sample /etc/yardstick/yardstick.conf + +vi /etc/yardstick/yardstick.conf +Config yardstick.conf +:: + + [DEFAULT] + debug = True + dispatcher = influxdb + + [dispatcher_influxdb] + timeout = 5 + target = http://{YOUR_IP_HERE}:8086 + db_name = yardstick + username = root + password = root + +Now you can run yardstick test cases and store the results in influxdb +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + + +Create a test suite for yardstick +------------------------------------ + +A test suite in yardstick is a yaml file which include one or more test cases. +Yardstick is able to support running test suite task, so you can customize you +own test suite and run it in one task. + +"tests/opnfv/test_suites" is where yardstick put ci test-suite. A typical test +suite is like below: + +fuel_test_suite.yaml + +:: + + --- + # Fuel integration test task suite + + schema: "yardstick:suite:0.1" + + name: "fuel_test_suite" + test_cases_dir: "samples/" + test_cases: + - + file_name: ping.yaml + - + file_name: iperf3.yaml + +As you can see, there are two test cases in fuel_test_suite, the syntax is simple +here, you must specify the schema and the name, then you just need to list the +test cases in the tag "test_cases" and also mark their relative directory in the +tag "test_cases_dir". + +Yardstick test suite also support constraints and task args for each test case. +Here is another sample to show this, which is digested from one big test suite. + +os-nosdn-nofeature-ha.yaml + +:: + + --- + + schema: "yardstick:suite:0.1" + + name: "os-nosdn-nofeature-ha" + test_cases_dir: "tests/opnfv/test_cases/" + test_cases: + - + file_name: opnfv_yardstick_tc002.yaml + - + file_name: opnfv_yardstick_tc005.yaml + - + file_name: opnfv_yardstick_tc043.yaml + constraint: + installer: compass + pod: huawei-pod1 + task_args: + huawei-pod1: '{"pod_info": "etc/yardstick/.../pod.yaml", + "host": "node4.LF","target": "node5.LF"}' + +As you can see in test case "opnfv_yardstick_tc043.yaml", there are two tags, "constraint" and +"task_args". "constraint" is where you can specify which installer or pod it can be run in +the ci environment. "task_args" is where you can specify the task arguments for each pod. + +All in all, to create a test suite in yardstick, you just need to create a suite yaml file +and add test cases and constraint or task arguments if necessary. + diff --git a/docs/userguide/09-result-store-InfluxDB.rst b/docs/userguide/09-result-store-InfluxDB.rst deleted file mode 100644 index a0bb48a80..000000000 --- a/docs/userguide/09-result-store-InfluxDB.rst +++ /dev/null @@ -1,86 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, 2016 Huawei Technologies Co.,Ltd and others. - -============================================== -Store Other Project's Test Results in InfluxDB -============================================== - -Abstract -======== - -.. _Framework: https://wiki.opnfv.org/download/attachments/6827660/wiki.png?version=1&modificationDate=1470298075000&api=v2 - -This chapter illustrates how to run plug-in test cases and store test results -into community's InfluxDB. The framework is shown in Framework_. - - -.. image:: images/InfluxDB_store.png - :width: 800px - :alt: Store Other Project's Test Results in InfluxDB - -Store Storperf Test Results into Community's InfluxDB -===================================================== - -.. _Influxdb: https://git.opnfv.org/cgit/yardstick/tree/yardstick/dispatcher/influxdb.py -.. _Mingjiang: limingjiang@huawei.com -.. _Visual: https://wiki.opnfv.org/download/attachments/6827660/tc074.PNG?version=1&modificationDate=1470298075000&api=v2 -.. _Login: http://testresults.opnfv.org/grafana/login - -As shown in Framework_, there are two ways to store Storperf test results -into community's InfluxDB: - -1. Yardstick asks Storperf to run the test case. After the test case is - completed, Yardstick reads test results via ReST API from Storperf and - posts test data to the influxDB. - -2. Additionally, Storperf can run tests by itself and post the test result - directly to the InfluxDB. The method for posting data directly to influxDB - will be supported in the future. - -Our plan is to support rest-api in D release so that other testing projects can -call the rest-api to use yardstick dispatcher service to push data to yardstick's -influxdb database. - -For now, influxdb only support line protocol, and the json protocol is deprecated. - -Take ping test case for example, the raw_result is json format like this: -:: - - "benchmark": { - "timestamp": 1470315409.868095, - "errors": "", - "data": { - "rtt": { - "ares": 1.125 - } - }, - "sequence": 1 - }, - "runner_id": 2625 - } - -With the help of "influxdb_line_protocol", the json is transform to like below as a line string: -:: - - 'ping,deploy_scenario=unknown,host=athena.demo,installer=unknown,pod_name=unknown, - runner_id=2625,scenarios=Ping,target=ares.demo,task_id=77755f38-1f6a-4667-a7f3- - 301c99963656,version=unknown rtt.ares=1.125 1470315409868094976' - -So, for data output of json format, you just need to transform json into line format and call -influxdb api to post the data into the database. All this function has been implemented in Influxdb_. -If you need support on this, please contact Mingjiang_. -:: - - curl -i -XPOST 'http://104.197.68.199:8086/write?db=yardstick' -- - data-binary 'ping,deploy_scenario=unknown,host=athena.demo,installer=unknown, ...' - -Grafana will be used for visualizing the collected test data, which is shown in Visual_. Grafana -can be accessed by Login_. - - -.. image:: images/results_visualization.png - :width: 800px - :alt: results visualization - diff --git a/docs/userguide/10-grafana.rst b/docs/userguide/10-grafana.rst deleted file mode 100644 index 416857b71..000000000 --- a/docs/userguide/10-grafana.rst +++ /dev/null @@ -1,119 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) 2016 Huawei Technologies Co.,Ltd and others - -================= -Grafana dashboard -================= - - -Abstract -======== - -This chapter describes the Yardstick grafana dashboard. The Yardstick grafana -dashboard can be found here: http://testresults.opnfv.org/grafana/ - - -.. image:: images/login.png - :width: 800px - :alt: Yardstick grafana dashboard - - -Public access -============= - -Yardstick provids a public account for accessing to the dashboard. The username -and password are both set to ‘opnfv’. - - -Testcase dashboard -================== - -For each test case, there is a dedicated dashboard. Shown here is the dashboard -of TC002. - - -.. image:: images/TC002.png - :width: 800px - :alt:TC002 dashboard - -For each test case dashboard. On the top left, we have a dashboard selection, -you can switch to different test cases using this pull-down menu. - -Underneath, we have a pod and scenario selection. -All the pods and scenarios that have ever published test data to the InfluxDB -will be shown here. - -You can check multiple pods or scenarios. - -For each test case, we have a short description and a link to detailed test -case information in Yardstick user guide. - -Underneath, it is the result presentation section. -You can use the time period selection on the top right corner to zoom in or -zoom out the chart. - - -Administration access -===================== - -For a user with administration rights it is easy to update and save any -dashboard configuration. Saved updates immediately take effect and become live. -This may cause issues like: - -- Changes and updates made to the live configuration in Grafana can compromise - existing Grafana content in an unwanted, unpredicted or incompatible way. - Grafana as such is not version controlled, there exists one single Grafana - configuration per dashboard. -- There is a risk several people can disturb each other when doing updates to - the same Grafana dashboard at the same time. - -Any change made by administrator should be careful. - - -Add a dashboard into yardstick grafana -====================================== - -Due to security concern, users that using the public opnfv account are not able -to edit the yardstick grafana directly.It takes a few more steps for a -non-yardstick user to add a custom dashboard into yardstick grafana. - -There are 6 steps to go. - - -.. image:: images/add.png - :width: 800px - :alt: Add a dashboard into yardstick grafana - - -1. You need to build a local influxdb and grafana, so you can do the work - locally. You can refer to How to deploy InfluxDB and Grafana locally wiki - page about how to do this. - -2. Once step one is done, you can fetch the existing grafana dashboard - configuration file from the yardstick repository and import it to your local - grafana. After import is done, you grafana dashboard will be ready to use - just like the community’s dashboard. - -3. The third step is running some test cases to generate test results and - publishing it to your local influxdb. - -4. Now you have some data to visualize in your dashboard. In the fourth step, - it is time to create your own dashboard. You can either modify an existing - dashboard or try to create a new one from scratch. If you choose to modify - an existing dashboard then in the curtain menu of the existing dashboard do - a "Save As..." into a new dashboard copy instance, and then continue doing - all updates and saves within the dashboard copy. - -5. When finished with all Grafana configuration changes in this temporary - dashboard then chose "export" of the updated dashboard copy into a JSON file - and put it up for review in Gerrit, in file /yardstick/dashboard/Yardstick-TCxxx-yyyyyyyyyyyyy. - For instance a typical default name of the file would be "Yardstick-TC001 Copy-1234567891234". - -6. Once you finish your dashboard, the next step is exporting the configuration - file and propose a patch into Yardstick. Yardstick team will review and - merge it into Yardstick repository. After approved review Yardstick team - will do an "import" of the JSON file and also a "save dashboard" as soon as - possible to replace the old live dashboard configuration. - diff --git a/docs/userguide/10-yardstick_plugin.rst b/docs/userguide/10-yardstick_plugin.rst new file mode 100644 index 000000000..f16dedd02 --- /dev/null +++ b/docs/userguide/10-yardstick_plugin.rst @@ -0,0 +1,144 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Ericsson AB, Huawei Technologies Co.,Ltd and others. + +=================================== +Installing a plug-in into yardstick +=================================== + +Abstract +======== + +Yardstick currently provides a ``plugin`` CLI command to support integration +with other OPNFV testing projects. Below is an example invocation of yardstick +plugin command and Storperf plug-in sample. + + +Installing Storperf into yardstick +================================== + +Storperf is delivered as a Docker container from +https://hub.docker.com/r/opnfv/storperf/tags/. + +There are two possible methods for installation in your environment: + +* Run container on Jump Host +* Run container in a VM + +In this introduction we will install Storperf on Jump Host. + + +Step 0: Environment preparation +>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> + +Running Storperf on Jump Host +Requirements: + +* Docker must be installed +* Jump Host must have access to the OpenStack Controller API +* Jump Host must have internet connectivity for downloading docker image +* Enough floating IPs must be available to match your agent count + +Before installing Storperf into yardstick you need to check your openstack +environment and other dependencies: + +1. Make sure docker is installed. +2. Make sure Keystone, Nova, Neutron, Glance, Heat are installed correctly. +3. Make sure Jump Host have access to the OpenStack Controller API. +4. Make sure Jump Host must have internet connectivity for downloading docker image. +5. You need to know where to get basic openstack Keystone authorization info, such as + OS_PASSWORD, OS_TENANT_NAME, OS_AUTH_URL, OS_USERNAME. +6. To run a Storperf container, you need to have OpenStack Controller environment + variables defined and passed to Storperf container. The best way to do this is to + put environment variables in a "storperf_admin-rc" file. The storperf_admin-rc + should include credential environment variables at least: + +* OS_AUTH_URL +* OS_TENANT_ID +* OS_TENANT_NAME +* OS_PROJECT_NAME +* OS_USERNAME +* OS_PASSWORD +* OS_REGION_NAME + +For this storperf_admin-rc file, during environment preparation a "prepare_storperf_admin-rc.sh" +script can be used to generate it. +:: + + #!/bin/bash + AUTH_URL=${OS_AUTH_URL} + USERNAME=${OS_USERNAME:-admin} + PASSWORD=${OS_PASSWORD:-console} + TENANT_NAME=${OS_TENANT_NAME:-admin} + VOLUME_API_VERSION=${OS_VOLUME_API_VERSION:-2} + PROJECT_NAME=${OS_PROJECT_NAME:-$TENANT_NAME} + TENANT_ID=`keystone tenant-get admin|grep 'id'|awk -F '|' '{print $3}'|sed -e 's/^[[:space:]]*//'` + rm -f ~/storperf_admin-rc + touch ~/storperf_admin-rc + echo "OS_AUTH_URL="$AUTH_URL >> ~/storperf_admin-rc + echo "OS_USERNAME="$USERNAME >> ~/storperf_admin-rc + echo "OS_PASSWORD="$PASSWORD >> ~/storperf_admin-rc + echo "OS_TENANT_NAME="$TENANT_NAME >> ~/storperf_admin-rc + echo "OS_VOLUME_API_VERSION="$VOLUME_API_VERSION >> ~/storperf_admin-rc + echo "OS_PROJECT_NAME="$PROJECT_NAME >> ~/storperf_admin-rc + echo "OS_TENANT_ID="$TENANT_ID >> ~/storperf_admin-rc + + +Step 1: Plug-in configuration file preparation +++++++++++++++++++++++++++++++++++++++++++++++ + +To install a plug-in, first you need to prepare a plug-in configuration file in +YAML format and store it in the "plugin" directory. The plugin configration file +work as the input of yardstick "plugin" command. Below is the Storperf plug-in +configuration file sample: +:: + + --- + # StorPerf plugin configuration file + # Used for integration StorPerf into Yardstick as a plugin + schema: "yardstick:plugin:0.1" + plugins: + name: storperf + deployment: + ip: 192.168.23.2 + user: root + password: root + +In the plug-in configuration file, you need to specify the plug-in name and the +plug-in deployment info, including node ip, node login username and password. +Here the Storperf will be installed on IP 192.168.23.2 which is the Jump Host +in my local environment. + +Step 2: Plug-in install/remove scripts preparation +++++++++++++++++++++++++++++++++++++++++++++++++++ + +Under "yardstick/resource/scripts directory", there are two folders: a "install" +folder and a "remove" folder. You need to store the plug-in install/remove script +in these two folders respectively. + +The detailed installation or remove operation should de defined in these two scripts. +The name of both install and remove scripts should match the plugin-in name that you +specified in the plug-in configuration file. +For example, the install and remove scripts for Storperf are both named to "storperf.bash". + + +Step 3: Install and remove Storperf ++++++++++++++++++++++++++++++++++++ + +To install Storperf, simply execute the following command +:: + + # Install Storperf + yardstick plugin install plugin/storperf.yaml + +removing Storperf from yardstick +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To remove Storperf, simply execute the following command +:: + + # Remove Storperf + yardstick plugin remove plugin/storperf.yaml + +What yardstick plugin command does is using the username and password to log into the deployment target and then execute the corresponding install or remove script. diff --git a/docs/userguide/11-list-of-tcs.rst b/docs/userguide/11-list-of-tcs.rst deleted file mode 100644 index 1b5806cd9..000000000 --- a/docs/userguide/11-list-of-tcs.rst +++ /dev/null @@ -1,129 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, Ericsson AB and others. - -==================== -Yardstick Test Cases -==================== - -Abstract -======== - -This chapter lists available Yardstick test cases. -Yardstick test cases are divided in two main categories: - -* *Generic NFVI Test Cases* - Test Cases developed to realize the methodology -described in :doc:`02-methodology` - -* *OPNFV Feature Test Cases* - Test Cases developed to verify one or more -aspect of a feature delivered by an OPNFV Project, including the test cases -developed for the :term:`VTC`. - -Generic NFVI Test Case Descriptions -=================================== - -.. toctree:: - :maxdepth: 1 - - opnfv_yardstick_tc001.rst - opnfv_yardstick_tc002.rst - opnfv_yardstick_tc004.rst - opnfv_yardstick_tc005.rst - opnfv_yardstick_tc008.rst - opnfv_yardstick_tc009.rst - opnfv_yardstick_tc010.rst - opnfv_yardstick_tc011.rst - opnfv_yardstick_tc012.rst - opnfv_yardstick_tc014.rst - opnfv_yardstick_tc024.rst - opnfv_yardstick_tc037.rst - opnfv_yardstick_tc038.rst - opnfv_yardstick_tc042.rst - opnfv_yardstick_tc043.rst - opnfv_yardstick_tc044.rst - opnfv_yardstick_tc055.rst - opnfv_yardstick_tc061.rst - opnfv_yardstick_tc063.rst - opnfv_yardstick_tc069.rst - opnfv_yardstick_tc070.rst - opnfv_yardstick_tc071.rst - opnfv_yardstick_tc072.rst - opnfv_yardstick_tc073.rst - opnfv_yardstick_tc075.rst - opnfv_yardstick_tc076.rst - -OPNFV Feature Test Cases -======================== - -H A ---- - -.. toctree:: - :maxdepth: 1 - - opnfv_yardstick_tc019.rst - opnfv_yardstick_tc025.rst - opnfv_yardstick_tc045.rst - opnfv_yardstick_tc046.rst - opnfv_yardstick_tc047.rst - opnfv_yardstick_tc048.rst - opnfv_yardstick_tc049.rst - opnfv_yardstick_tc050.rst - opnfv_yardstick_tc051.rst - opnfv_yardstick_tc052.rst - opnfv_yardstick_tc053.rst - opnfv_yardstick_tc054.rst - -IPv6 ----- - -.. toctree:: - :maxdepth: 1 - - opnfv_yardstick_tc027.rst - -KVM ---- - -.. toctree:: - :maxdepth: 1 - - opnfv_yardstick_tc028.rst - -Parser ------- - -.. toctree:: - :maxdepth: 1 - - opnfv_yardstick_tc040.rst - - StorPerf ------------ - -.. toctree:: - :maxdepth: 1 - - opnfv_yardstick_tc074.rst - -virtual Traffic Classifier --------------------------- - -.. toctree:: - :maxdepth: 1 - - opnfv_yardstick_tc006.rst - opnfv_yardstick_tc007.rst - opnfv_yardstick_tc020.rst - opnfv_yardstick_tc021.rst - -Templates -========= - -.. toctree:: - :maxdepth: 1 - - testcase_description_v2_template - Yardstick_task_templates - diff --git a/docs/userguide/11-result-store-InfluxDB.rst b/docs/userguide/11-result-store-InfluxDB.rst new file mode 100644 index 000000000..a0bb48a80 --- /dev/null +++ b/docs/userguide/11-result-store-InfluxDB.rst @@ -0,0 +1,86 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2016 Huawei Technologies Co.,Ltd and others. + +============================================== +Store Other Project's Test Results in InfluxDB +============================================== + +Abstract +======== + +.. _Framework: https://wiki.opnfv.org/download/attachments/6827660/wiki.png?version=1&modificationDate=1470298075000&api=v2 + +This chapter illustrates how to run plug-in test cases and store test results +into community's InfluxDB. The framework is shown in Framework_. + + +.. image:: images/InfluxDB_store.png + :width: 800px + :alt: Store Other Project's Test Results in InfluxDB + +Store Storperf Test Results into Community's InfluxDB +===================================================== + +.. _Influxdb: https://git.opnfv.org/cgit/yardstick/tree/yardstick/dispatcher/influxdb.py +.. _Mingjiang: limingjiang@huawei.com +.. _Visual: https://wiki.opnfv.org/download/attachments/6827660/tc074.PNG?version=1&modificationDate=1470298075000&api=v2 +.. _Login: http://testresults.opnfv.org/grafana/login + +As shown in Framework_, there are two ways to store Storperf test results +into community's InfluxDB: + +1. Yardstick asks Storperf to run the test case. After the test case is + completed, Yardstick reads test results via ReST API from Storperf and + posts test data to the influxDB. + +2. Additionally, Storperf can run tests by itself and post the test result + directly to the InfluxDB. The method for posting data directly to influxDB + will be supported in the future. + +Our plan is to support rest-api in D release so that other testing projects can +call the rest-api to use yardstick dispatcher service to push data to yardstick's +influxdb database. + +For now, influxdb only support line protocol, and the json protocol is deprecated. + +Take ping test case for example, the raw_result is json format like this: +:: + + "benchmark": { + "timestamp": 1470315409.868095, + "errors": "", + "data": { + "rtt": { + "ares": 1.125 + } + }, + "sequence": 1 + }, + "runner_id": 2625 + } + +With the help of "influxdb_line_protocol", the json is transform to like below as a line string: +:: + + 'ping,deploy_scenario=unknown,host=athena.demo,installer=unknown,pod_name=unknown, + runner_id=2625,scenarios=Ping,target=ares.demo,task_id=77755f38-1f6a-4667-a7f3- + 301c99963656,version=unknown rtt.ares=1.125 1470315409868094976' + +So, for data output of json format, you just need to transform json into line format and call +influxdb api to post the data into the database. All this function has been implemented in Influxdb_. +If you need support on this, please contact Mingjiang_. +:: + + curl -i -XPOST 'http://104.197.68.199:8086/write?db=yardstick' -- + data-binary 'ping,deploy_scenario=unknown,host=athena.demo,installer=unknown, ...' + +Grafana will be used for visualizing the collected test data, which is shown in Visual_. Grafana +can be accessed by Login_. + + +.. image:: images/results_visualization.png + :width: 800px + :alt: results visualization + diff --git a/docs/userguide/12-grafana.rst b/docs/userguide/12-grafana.rst new file mode 100644 index 000000000..416857b71 --- /dev/null +++ b/docs/userguide/12-grafana.rst @@ -0,0 +1,119 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2016 Huawei Technologies Co.,Ltd and others + +================= +Grafana dashboard +================= + + +Abstract +======== + +This chapter describes the Yardstick grafana dashboard. The Yardstick grafana +dashboard can be found here: http://testresults.opnfv.org/grafana/ + + +.. image:: images/login.png + :width: 800px + :alt: Yardstick grafana dashboard + + +Public access +============= + +Yardstick provids a public account for accessing to the dashboard. The username +and password are both set to ‘opnfv’. + + +Testcase dashboard +================== + +For each test case, there is a dedicated dashboard. Shown here is the dashboard +of TC002. + + +.. image:: images/TC002.png + :width: 800px + :alt:TC002 dashboard + +For each test case dashboard. On the top left, we have a dashboard selection, +you can switch to different test cases using this pull-down menu. + +Underneath, we have a pod and scenario selection. +All the pods and scenarios that have ever published test data to the InfluxDB +will be shown here. + +You can check multiple pods or scenarios. + +For each test case, we have a short description and a link to detailed test +case information in Yardstick user guide. + +Underneath, it is the result presentation section. +You can use the time period selection on the top right corner to zoom in or +zoom out the chart. + + +Administration access +===================== + +For a user with administration rights it is easy to update and save any +dashboard configuration. Saved updates immediately take effect and become live. +This may cause issues like: + +- Changes and updates made to the live configuration in Grafana can compromise + existing Grafana content in an unwanted, unpredicted or incompatible way. + Grafana as such is not version controlled, there exists one single Grafana + configuration per dashboard. +- There is a risk several people can disturb each other when doing updates to + the same Grafana dashboard at the same time. + +Any change made by administrator should be careful. + + +Add a dashboard into yardstick grafana +====================================== + +Due to security concern, users that using the public opnfv account are not able +to edit the yardstick grafana directly.It takes a few more steps for a +non-yardstick user to add a custom dashboard into yardstick grafana. + +There are 6 steps to go. + + +.. image:: images/add.png + :width: 800px + :alt: Add a dashboard into yardstick grafana + + +1. You need to build a local influxdb and grafana, so you can do the work + locally. You can refer to How to deploy InfluxDB and Grafana locally wiki + page about how to do this. + +2. Once step one is done, you can fetch the existing grafana dashboard + configuration file from the yardstick repository and import it to your local + grafana. After import is done, you grafana dashboard will be ready to use + just like the community’s dashboard. + +3. The third step is running some test cases to generate test results and + publishing it to your local influxdb. + +4. Now you have some data to visualize in your dashboard. In the fourth step, + it is time to create your own dashboard. You can either modify an existing + dashboard or try to create a new one from scratch. If you choose to modify + an existing dashboard then in the curtain menu of the existing dashboard do + a "Save As..." into a new dashboard copy instance, and then continue doing + all updates and saves within the dashboard copy. + +5. When finished with all Grafana configuration changes in this temporary + dashboard then chose "export" of the updated dashboard copy into a JSON file + and put it up for review in Gerrit, in file /yardstick/dashboard/Yardstick-TCxxx-yyyyyyyyyyyyy. + For instance a typical default name of the file would be "Yardstick-TC001 Copy-1234567891234". + +6. Once you finish your dashboard, the next step is exporting the configuration + file and propose a patch into Yardstick. Yardstick team will review and + merge it into Yardstick repository. After approved review Yardstick team + will do an "import" of the JSON file and also a "save dashboard" as soon as + possible to replace the old live dashboard configuration. + diff --git a/docs/userguide/13-list-of-tcs.rst b/docs/userguide/13-list-of-tcs.rst new file mode 100644 index 000000000..1b5806cd9 --- /dev/null +++ b/docs/userguide/13-list-of-tcs.rst @@ -0,0 +1,129 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Ericsson AB and others. + +==================== +Yardstick Test Cases +==================== + +Abstract +======== + +This chapter lists available Yardstick test cases. +Yardstick test cases are divided in two main categories: + +* *Generic NFVI Test Cases* - Test Cases developed to realize the methodology +described in :doc:`02-methodology` + +* *OPNFV Feature Test Cases* - Test Cases developed to verify one or more +aspect of a feature delivered by an OPNFV Project, including the test cases +developed for the :term:`VTC`. + +Generic NFVI Test Case Descriptions +=================================== + +.. toctree:: + :maxdepth: 1 + + opnfv_yardstick_tc001.rst + opnfv_yardstick_tc002.rst + opnfv_yardstick_tc004.rst + opnfv_yardstick_tc005.rst + opnfv_yardstick_tc008.rst + opnfv_yardstick_tc009.rst + opnfv_yardstick_tc010.rst + opnfv_yardstick_tc011.rst + opnfv_yardstick_tc012.rst + opnfv_yardstick_tc014.rst + opnfv_yardstick_tc024.rst + opnfv_yardstick_tc037.rst + opnfv_yardstick_tc038.rst + opnfv_yardstick_tc042.rst + opnfv_yardstick_tc043.rst + opnfv_yardstick_tc044.rst + opnfv_yardstick_tc055.rst + opnfv_yardstick_tc061.rst + opnfv_yardstick_tc063.rst + opnfv_yardstick_tc069.rst + opnfv_yardstick_tc070.rst + opnfv_yardstick_tc071.rst + opnfv_yardstick_tc072.rst + opnfv_yardstick_tc073.rst + opnfv_yardstick_tc075.rst + opnfv_yardstick_tc076.rst + +OPNFV Feature Test Cases +======================== + +H A +--- + +.. toctree:: + :maxdepth: 1 + + opnfv_yardstick_tc019.rst + opnfv_yardstick_tc025.rst + opnfv_yardstick_tc045.rst + opnfv_yardstick_tc046.rst + opnfv_yardstick_tc047.rst + opnfv_yardstick_tc048.rst + opnfv_yardstick_tc049.rst + opnfv_yardstick_tc050.rst + opnfv_yardstick_tc051.rst + opnfv_yardstick_tc052.rst + opnfv_yardstick_tc053.rst + opnfv_yardstick_tc054.rst + +IPv6 +---- + +.. toctree:: + :maxdepth: 1 + + opnfv_yardstick_tc027.rst + +KVM +--- + +.. toctree:: + :maxdepth: 1 + + opnfv_yardstick_tc028.rst + +Parser +------ + +.. toctree:: + :maxdepth: 1 + + opnfv_yardstick_tc040.rst + + StorPerf +----------- + +.. toctree:: + :maxdepth: 1 + + opnfv_yardstick_tc074.rst + +virtual Traffic Classifier +-------------------------- + +.. toctree:: + :maxdepth: 1 + + opnfv_yardstick_tc006.rst + opnfv_yardstick_tc007.rst + opnfv_yardstick_tc020.rst + opnfv_yardstick_tc021.rst + +Templates +========= + +.. toctree:: + :maxdepth: 1 + + testcase_description_v2_template + Yardstick_task_templates + diff --git a/docs/userguide/index.rst b/docs/userguide/index.rst index 60e1340ac..826a9d9bf 100644 --- a/docs/userguide/index.rst +++ b/docs/userguide/index.rst @@ -16,10 +16,12 @@ Yardstick Overview 04-vtc-overview 05-apexlake_installation 06-apexlake_api - 07-installation - 08-yardstick_plugin - 09-result-store-InfluxDB - 10-grafana - 11-list-of-tcs + 07-nsb-overview + 08-nsb_installation + 09-installation + 10-yardstick_plugin + 11-result-store-InfluxDB + 12-grafana + 13-list-of-tcs glossary references -- cgit 1.2.3-korg