From b24dc4e0c48aa7d9aa98c3e06d6d2cd2c6bed1d6 Mon Sep 17 00:00:00 2001 From: Emma Foley Date: Wed, 7 Mar 2018 18:40:28 +0000 Subject: [docs] Move user guide chapters to make room for new chapter. * Increment the numbering on chapters * Fix some of the changes flagged by doc8 Change-Id: I9dbf4a3d33c2b38f7567bc3493466e72a59e1a0c Signed-off-by: Emma Foley (cherry picked from commit 3248682fcc789f95ff8246a8d1f208a6a3abd46d) --- docs/testing/user/userguide/01-introduction.rst | 29 +- docs/testing/user/userguide/03-architecture.rst | 22 +- docs/testing/user/userguide/04-installation.rst | 292 +---- .../testing/user/userguide/05-yardstick_plugin.rst | 157 --- .../user/userguide/06-result-store-InfluxDB.rst | 86 -- .../testing/user/userguide/06-yardstick-plugin.rst | 159 +++ docs/testing/user/userguide/07-grafana.rst | 119 -- .../user/userguide/07-result-store-InfluxDB.rst | 88 ++ docs/testing/user/userguide/08-api.rst | 746 ------------- docs/testing/user/userguide/08-grafana.rst | 121 ++ docs/testing/user/userguide/09-api.rst | 769 +++++++++++++ .../user/userguide/09-yardstick_user_interface.rst | 29 - docs/testing/user/userguide/10-vtc-overview.rst | 128 --- .../user/userguide/10-yardstick-user-interface.rst | 30 + docs/testing/user/userguide/11-nsb-overview.rst | 205 ---- docs/testing/user/userguide/11-vtc-overview.rst | 128 +++ docs/testing/user/userguide/12-nsb-overview.rst | 206 ++++ .../testing/user/userguide/12-nsb_installation.rst | 1170 -------------------- .../testing/user/userguide/13-nsb-installation.rst | 1166 +++++++++++++++++++ docs/testing/user/userguide/13-nsb_operation.rst | 306 ----- docs/testing/user/userguide/14-nsb-operation.rst | 315 ++++++ docs/testing/user/userguide/15-list-of-tcs.rst | 10 +- docs/testing/user/userguide/index.rst | 18 +- 23 files changed, 3074 insertions(+), 3225 deletions(-) delete mode 100644 docs/testing/user/userguide/05-yardstick_plugin.rst delete mode 100644 docs/testing/user/userguide/06-result-store-InfluxDB.rst create mode 100644 docs/testing/user/userguide/06-yardstick-plugin.rst delete mode 100644 docs/testing/user/userguide/07-grafana.rst create mode 100644 docs/testing/user/userguide/07-result-store-InfluxDB.rst delete mode 100644 docs/testing/user/userguide/08-api.rst create mode 100644 docs/testing/user/userguide/08-grafana.rst create mode 100644 docs/testing/user/userguide/09-api.rst delete mode 100644 docs/testing/user/userguide/09-yardstick_user_interface.rst delete mode 100644 docs/testing/user/userguide/10-vtc-overview.rst create mode 100644 docs/testing/user/userguide/10-yardstick-user-interface.rst delete mode 100644 docs/testing/user/userguide/11-nsb-overview.rst create mode 100644 docs/testing/user/userguide/11-vtc-overview.rst create mode 100644 docs/testing/user/userguide/12-nsb-overview.rst delete mode 100644 docs/testing/user/userguide/12-nsb_installation.rst create mode 100644 docs/testing/user/userguide/13-nsb-installation.rst delete mode 100644 docs/testing/user/userguide/13-nsb_operation.rst create mode 100644 docs/testing/user/userguide/14-nsb-operation.rst (limited to 'docs') diff --git a/docs/testing/user/userguide/01-introduction.rst b/docs/testing/user/userguide/01-introduction.rst index c1d5def98..c0c0a0a2b 100755 --- a/docs/testing/user/userguide/01-introduction.rst +++ b/docs/testing/user/userguide/01-introduction.rst @@ -42,43 +42,44 @@ This document consists of the following chapters: * Chapter :doc:`02-methodology` describes the methodology implemented by the *Yardstick* Project for :term:`NFVI` verification. -* Chapter :doc:`03-architecture` provides information on the software architecture - of *Yardstick*. +* Chapter :doc:`03-architecture` provides information on the software + architecture of *Yardstick*. * Chapter :doc:`04-installation` provides instructions to install *Yardstick*. -* Chapter :doc:`05-yardstick_plugin` provides information on how to integrate +* Chapter :doc:`06-yardstick-plugin` provides information on how to integrate other OPNFV testing projects into *Yardstick*. -* Chapter :doc:`06-result-store-InfluxDB` provides inforamtion on how to run +* Chapter :doc:`07-result-store-InfluxDB` provides inforamtion on how to run plug-in test cases and store test results into community's InfluxDB. -* Chapter :doc:`07-grafana` provides inforamtion on *Yardstick* grafana dashboard - and how to add a dashboard into *Yardstick* grafana dashboard. +* Chapter :doc:`08-grafana` provides inforamtion on *Yardstick* grafana + dashboard and how to add a dashboard into *Yardstick* grafana dashboard. -* Chapter :doc:`08-api` provides inforamtion on *Yardstick* ReST API and how to +* Chapter :doc:`09-api` provides inforamtion on *Yardstick* ReST API and how to use *Yardstick* API. -* Chapter :doc:`09-yardstick_user_interface` provides inforamtion on how to use +* Chapter :doc:`10-yardstick-user-interface` provides inforamtion on how to use yardstick report CLI to view the test result in table format and also values pinned on to a graph -* Chapter :doc:`10-vtc-overview` provides information on the :term:`VTC`. +* Chapter :doc:`11-vtc-overview` provides information on the :term:`VTC`. -* Chapter :doc:`13-nsb-overview` describes the methodology implemented by the +* Chapter :doc:`12-nsb-overview` describes the methodology implemented by the Yardstick - Network service benchmarking to test real world usecase for a given VNF. -* Chapter :doc:`14-nsb_installation` provides instructions to install - *Yardstick - Network service benchmarking testing*. +* Chapter :doc:`13-nsb_installation` provides instructions to install + *Yardstick - Network Service Benchmarking (NSB) testing*. + +* Chapter :doc:`14-nsb-operation` provides information on running *NSB* * Chapter :doc:`15-list-of-tcs` includes a list of available *Yardstick* test cases. - Contact Yardstick ================= Feedback? `Contact us`_ -.. _Contact us: opnfv-users@lists.opnfv.org +.. _Contact us: mailto:opnfv-users@lists.opnfv.org&subject="[yardstick]" diff --git a/docs/testing/user/userguide/03-architecture.rst b/docs/testing/user/userguide/03-architecture.rst index 8336b609d..622002ee4 100755 --- a/docs/testing/user/userguide/03-architecture.rst +++ b/docs/testing/user/userguide/03-architecture.rst @@ -9,8 +9,9 @@ Architecture Abstract ======== -This chapter describes the yardstick framework software architecture. we will introduce it from Use-Case View, -Logical View, Process View and Deployment View. More technical details will be introduced in this chapter. +This chapter describes the yardstick framework software architecture. We will +introduce it from Use-Case View, Logical View, Process View and Deployment +View. More technical details will be introduced in this chapter. Overview ======== @@ -23,8 +24,8 @@ files. Yardstick is inspired by Rally. Yardstick is intended to run on a computer with access and credentials to a cloud. The test case is described in a configuration file given as an argument. -How it works: the benchmark task configuration file is parsed and converted into -an internal model. The context part of the model is converted into a Heat +How it works: the benchmark task configuration file is parsed and converted +into an internal model. The context part of the model is converted into a Heat template and deployed into a stack. Each scenario is run using a runner, either serially or in parallel. Each runner runs in its own subprocess executing commands in a VM using SSH. The output of each scenario is written as json @@ -43,13 +44,15 @@ names, image names, affinity rules and network configurations. A context is converted into a simplified Heat template, which is used to deploy onto the Openstack environment. -**Data** - Output produced by running a benchmark, written to a file in json format +**Data** - Output produced by running a benchmark, written to a file in json +format **Runner** - Logic that determines how a test scenario is run and reported, for example the number of test iterations, input value stepping and test duration. Predefined runner types exist for re-usage, see `Runner types`_. -**Scenario** - Type/class of measurement for example Ping, Pktgen, (Iperf, LmBench, ...) +**Scenario** - Type/class of measurement for example Ping, Pktgen, (Iperf, +LmBench, ...) **SLA** - Relates to what result boundary a test case must meet to pass. For example a latency limit, amount or ratio of lost packets and so on. Action @@ -128,8 +131,8 @@ Snippet of an Iteration runner configuration: Use-Case View ============= Yardstick Use-Case View shows two kinds of users. One is the Tester who will -do testing in cloud, the other is the User who is more concerned with test result -and result analyses. +do testing in cloud, the other is the User who is more concerned with test +result and result analyses. For testers, they will run a single test case or test case suite to verify infrastructure compliance or bencnmark their own infrastructure performance. @@ -254,7 +257,8 @@ Yardstick Directory structure *tools/* - Currently contains tools to build image for VMs which are deployed by Heat. Currently contains how to build the yardstick-trusty-server - image with the different tools that are needed from within the image. + image with the different tools that are needed from within the + image. *plugin/* - Plug-in configuration files are stored here. diff --git a/docs/testing/user/userguide/04-installation.rst b/docs/testing/user/userguide/04-installation.rst index cac814667..a4846230e 100644 --- a/docs/testing/user/userguide/04-installation.rst +++ b/docs/testing/user/userguide/04-installation.rst @@ -39,18 +39,18 @@ Several prerequisites are needed for Yardstick: 4. Connectivity from the Jumphost to the SUT public/external network .. note:: *Jumphost* refers to any server which meets the previous -requirements. Normally it is the same server from where the OPNFV -deployment has been triggered. + requirements. Normally it is the same server from where the OPNFV + deployment has been triggered. .. warning:: Connectivity from Jumphost is essential and it is of paramount -importance to make sure it is working before even considering to install -and run Yardstick. Make also sure you understand how your networking is -designed to work. + importance to make sure it is working before even considering to install + and run Yardstick. Make also sure you understand how your networking is + designed to work. .. note:: If your Jumphost is operating behind a company http proxy and/or -Firewall, please first consult `Proxy Support`_ section which is towards the -end of this document. That section details some tips/tricks which *may* be of -help in a proxified environment. + Firewall, please first consult `Proxy Support`_ section which is towards + the end of this document. That section details some tips/tricks which *may* + be of help in a proxified environment. Install Yardstick using Docker (first option) (**recommended**) @@ -85,27 +85,30 @@ Run the Docker image to get a Yardstick container:: docker run -itd --privileged -v /var/run/docker.sock:/var/run/docker.sock \ -p 8888:5000 --name yardstick opnfv/yardstick:stable -.. table:: Description of the parameters used with ``docker run`` command - - ======================= ==================================================== - Parameters Detail - ======================= ==================================================== - -itd -i: interactive, Keep STDIN open even if not - attached - -t: allocate a pseudo-TTY detached mode, in the - background - ======================= ==================================================== - --privileged If you want to build ``yardstick-image`` in - Yardstick container, this parameter is needed - ======================= ==================================================== - -p 8888:5000 Redirect the a host port (8888) to a container port - (5000) - ======================= ==================================================== - -v /var/run/docker.sock If you want to use yardstick env grafana/influxdb to - :/var/run/docker.sock create a grafana/influxdb container out of Yardstick - container - ======================= ==================================================== - --name yardstick The name for this container +Description of the parameters used with ``docker run`` command + + +------------------------+--------------------------------------------------+ + | Parameters | Detail | + +========================+==================================================+ + | -itd | -i: interactive, Keep STDIN open even if not | + | | attached | + | +--------------------------------------------------+ + | | -t: allocate a pseudo-TTY detached mode, in the | + | | background | + +------------------------+--------------------------------------------------+ + | --privileged | If you want to build ``yardstick-image`` in | + | | Yardstick container, this parameter is needed | + +------------------------+--------------------------------------------------+ + | -p 8888:5000 | Redirect the a host port (8888) to a container | + | | port (5000) | + +------------------------+--------------------------------------------------+ + | -v /var/run/docker.sock| If you want to use yardstick env | + | :/var/run/docker.sock | grafana/influxdb to create a grafana/influxdb | + | | container out of Yardstick container | + +------------------------+--------------------------------------------------+ + | --name yardstick | The name for this container | + +------------------------+--------------------------------------------------+ + If the host is restarted ^^^^^^^^^^^^^^^^^^^^^^^^ @@ -135,18 +138,18 @@ automatically:: yardstick env prepare .. note:: Since Euphrates release, the above command will not be able to -automatically configure the ``/etc/yardstick/openstack.creds`` file. So before -running the above command, it is necessary to create the -``/etc/yardstick/openstack.creds`` file and save OpenStack environment -variables into it manually. If you have the openstack credential file saved -outside the Yardstick Docker container, you can do this easily by mapping the -credential file into Yardstick container using:: + automatically configure the ``/etc/yardstick/openstack.creds`` file. So before + running the above command, it is necessary to create the + ``/etc/yardstick/openstack.creds`` file and save OpenStack environment + variables into it manually. If you have the openstack credential file saved + outside the Yardstick Docker container, you can do this easily by mapping the + credential file into Yardstick container using:: - '-v /path/to/credential_file:/etc/yardstick/openstack.creds' + '-v /path/to/credential_file:/etc/yardstick/openstack.creds' -when running the Yardstick container. For details of the required OpenStack -environment variables please refer to section `Export OpenStack environment -variables`_. + when running the Yardstick container. For details of the required OpenStack + environment variables please refer to section `Export OpenStack environment + variables`_. The ``env prepare`` command may take up to 6-8 minutes to finish building yardstick-image and other environment preparation. Meanwhile if you wish to @@ -222,8 +225,8 @@ Yardstick is installed:: sudo -EH tools/yardstick-img-modify tools/ubuntu-server-cloudimg-modify.sh .. warning:: Before building the guest image inside the Yardstick container, -make sure the container is granted with privilege. The script will create files -by default in ``/tmp/workspace/yardstick`` and the files will be owned by root. + make sure the container is granted with privilege. The script will create files + by default in ``/tmp/workspace/yardstick`` and the files will be owned by root. The created image can be added to OpenStack using the OpenStack client or via the OpenStack Dashboard:: @@ -270,7 +273,7 @@ For usage of Yardstick GUI, please watch our demo video at `Yardstick GUI demo`_. .. note:: The Yardstick GUI is still in development, the GUI layout and -features may change. + features may change. Delete the Yardstick container ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -433,7 +436,7 @@ of Yardstick ``help`` command and ``ping.py`` test sample:: yardstick task start samples/ping.yaml .. note:: The above commands could be run in both the Yardstick container and -the Ubuntu directly. + the Ubuntu directly. Each testing tool supported by Yardstick has a sample configuration file. These configuration files can be found in the ``samples`` directory. @@ -468,10 +471,10 @@ Then you can run a test case and visit http://host_ip:1948 (``admin``/``admin``) to see the results. .. note:: Executing ``yardstick env`` command to deploy InfluxDB and Grafana -requires Jumphost's docker API version => 1.24. Run the following command to -check the docker API version on the Jumphost:: + requires Jumphost's docker API version => 1.24. Run the following command to + check the docker API version on the Jumphost:: - docker version + docker version Manual deployment of InfluxDB and Grafana containers @@ -537,200 +540,6 @@ Deploy InfluxDB and Grafana directly in Ubuntu (**Todo**) --------------------------------------------------------- -Yardstick common CLI --------------------- - -List test cases -^^^^^^^^^^^^^^^ - -``yardstick testcase list``: This command line would list all test cases in -Yardstick. It would show like below:: - - +--------------------------------------------------------------------------------------- - | Testcase Name | Description - +--------------------------------------------------------------------------------------- - | opnfv_yardstick_tc001 | Measure network throughput using pktgen - | opnfv_yardstick_tc002 | measure network latency using ping - | opnfv_yardstick_tc005 | Measure Storage IOPS, throughput and latency using fio. - ... - +--------------------------------------------------------------------------------------- - - -Show a test case config file -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Take opnfv_yardstick_tc002 for an example. This test case measure network -latency. You just need to type in ``yardstick testcase show -opnfv_yardstick_tc002``, and the console would show the config yaml of this -test case:: - - --- - - schema: "yardstick:task:0.1" - description: > - Yardstick TC002 config file; - measure network latency using ping; - - {% set image = image or "cirros-0.3.5" %} - - {% set provider = provider or none %} - {% set physical_network = physical_network or 'physnet1' %} - {% set segmentation_id = segmentation_id or none %} - {% set packetsize = packetsize or 100 %} - - scenarios: - {% for i in range(2) %} - - - type: Ping - options: - packetsize: {{packetsize}} - host: athena.demo - target: ares.demo - - runner: - type: Duration - duration: 60 - interval: 10 - - sla: - max_rtt: 10 - action: monitor - {% endfor %} - - context: - name: demo - image: {{image}} - flavor: yardstick-flavor - user: cirros - - placement_groups: - pgrp1: - policy: "availability" - - servers: - athena: - floating_ip: true - placement: "pgrp1" - ares: - placement: "pgrp1" - - networks: - test: - cidr: '10.0.1.0/24' - {% if provider == "vlan" %} - provider: {{provider}} - physical_network: {{physical_network}}å - {% if segmentation_id %} - segmentation_id: {{segmentation_id}} - {% endif %} - {% endif %} - - -Start a task to run yardstick test case -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If you want run a test case, then you need to use ``yardstick task start -`` this command support some parameters as below:: - - +---------------------+--------------------------------------------------+ - | Parameters | Detail | - +=====================+==================================================+ - | -d | show debug log of yardstick running | - | | | - +---------------------+--------------------------------------------------+ - | --task-args | If you want to customize test case parameters, | - | | use "--task-args" to pass the value. The format | - | | is a json string with parameter key-value pair. | - | | | - +---------------------+--------------------------------------------------+ - | --task-args-file | If you want to use yardstick | - | | env prepare command(or | - | | related API) to load the | - +---------------------+--------------------------------------------------+ - | --parse-only | | - | | | - | | | - +---------------------+--------------------------------------------------+ - | --output-file \ | Specify where to output the log. if not pass, | - | OUTPUT_FILE_PATH | the default value is | - | | "/tmp/yardstick/yardstick.log" | - | | | - +---------------------+--------------------------------------------------+ - | --suite \ | run a test suite, TEST_SUITE_PATH specify where | - | TEST_SUITE_PATH | the test suite locates | - | | | - +---------------------+--------------------------------------------------+ - - -Run Yardstick in a local environment ------------------------------------- - -We also have a guide about how to run Yardstick in a local environment. -This work is contributed by Tapio Tallgren. -You can find this guide at `How to run Yardstick in a local environment`_. - - -Create a test suite for Yardstick ------------------------------------- - -A test suite in yardstick is a yaml file which include one or more test cases. -Yardstick is able to support running test suite task, so you can customize your -own test suite and run it in one task. - -``tests/opnfv/test_suites`` is the folder where Yardstick puts CI test suite. -A typical test suite is like below (the ``fuel_test_suite.yaml`` example):: - - --- - # Fuel integration test task suite - - schema: "yardstick:suite:0.1" - - name: "fuel_test_suite" - test_cases_dir: "samples/" - test_cases: - - - file_name: ping.yaml - - - file_name: iperf3.yaml - -As you can see, there are two test cases in the ``fuel_test_suite.yaml``. The -``schema`` and the ``name`` must be specified. The test cases should be listed -via the tag ``test_cases`` and their relative path is also marked via the tag -``test_cases_dir``. - -Yardstick test suite also supports constraints and task args for each test -case. Here is another sample (the ``os-nosdn-nofeature-ha.yaml`` example) to -show this, which is digested from one big test suite:: - - --- - - schema: "yardstick:suite:0.1" - - name: "os-nosdn-nofeature-ha" - test_cases_dir: "tests/opnfv/test_cases/" - test_cases: - - - file_name: opnfv_yardstick_tc002.yaml - - - file_name: opnfv_yardstick_tc005.yaml - - - file_name: opnfv_yardstick_tc043.yaml - constraint: - installer: compass - pod: huawei-pod1 - task_args: - huawei-pod1: '{"pod_info": "etc/yardstick/.../pod.yaml", - "host": "node4.LF","target": "node5.LF"}' - -As you can see in test case ``opnfv_yardstick_tc043.yaml``, there are two -tags, ``constraint`` and ``task_args``. ``constraint`` is to specify which -installer or pod it can be run in the CI environment. ``task_args`` is to -specify the task arguments for each pod. - -All in all, to create a test suite in Yardstick, you just need to create a -yaml file and add test cases, constraint or task arguments if necessary. - - Proxy Support ------------- @@ -790,7 +599,7 @@ stop and delete the container:: sudo docker rm yardstick .. warning:: Be careful, the above ``rm`` command will delete the container -completely. Everything on this container will be lost. + completely. Everything on this container will be lost. Then follow the previous instructions `Prepare the Yardstick container`_ to rebuild the Yardstick container. @@ -804,4 +613,3 @@ References .. _`Cirros 0.3.5`: http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img .. _`Ubuntu 16.04`: https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64-disk1.img .. _`Yardstick GUI demo`: https://www.youtube.com/watch?v=M3qbJDp6QBk -.. _`How to run Yardstick in a local environment`: https://wiki.opnfv.org/display/yardstick/How+to+run+Yardstick+in+a+local+environment diff --git a/docs/testing/user/userguide/05-yardstick_plugin.rst b/docs/testing/user/userguide/05-yardstick_plugin.rst deleted file mode 100644 index 679ce7900..000000000 --- a/docs/testing/user/userguide/05-yardstick_plugin.rst +++ /dev/null @@ -1,157 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, Ericsson AB, Huawei Technologies Co.,Ltd and others. - -=================================== -Installing a plug-in into Yardstick -=================================== - - -Abstract -======== - -Yardstick provides a ``plugin`` CLI command to support integration with other -OPNFV testing projects. Below is an example invocation of Yardstick plugin -command and Storperf plug-in sample. - - -Installing Storperf into Yardstick -================================== - -Storperf is delivered as a Docker container from -https://hub.docker.com/r/opnfv/storperf/tags/. - -There are two possible methods for installation in your environment: - -* Run container on Jump Host -* Run container in a VM - -In this introduction we will install Storperf on Jump Host. - - -Step 0: Environment preparation ->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - -Running Storperf on Jump Host -Requirements: - -* Docker must be installed -* Jump Host must have access to the OpenStack Controller API -* Jump Host must have internet connectivity for downloading docker image -* Enough floating IPs must be available to match your agent count - -Before installing Storperf into yardstick you need to check your openstack -environment and other dependencies: - -1. Make sure docker is installed. -2. Make sure Keystone, Nova, Neutron, Glance, Heat are installed correctly. -3. Make sure Jump Host have access to the OpenStack Controller API. -4. Make sure Jump Host must have internet connectivity for downloading docker image. -5. You need to know where to get basic openstack Keystone authorization info, such as - OS_PASSWORD, OS_PROJECT_NAME, OS_AUTH_URL, OS_USERNAME. -6. To run a Storperf container, you need to have OpenStack Controller environment - variables defined and passed to Storperf container. The best way to do this is to - put environment variables in a "storperf_admin-rc" file. The storperf_admin-rc - should include credential environment variables at least: - -* OS_AUTH_URL -* OS_USERNAME -* OS_PASSWORD -* OS_PROJECT_NAME -* OS_PROJECT_ID -* OS_USER_DOMAIN_ID - -*Yardstick* has a "prepare_storperf_admin-rc.sh" script which can be used to -generate the "storperf_admin-rc" file, this script is located at -test/ci/prepare_storperf_admin-rc.sh - -:: - - #!/bin/bash - # Prepare storperf_admin-rc for StorPerf. - AUTH_URL=${OS_AUTH_URL} - USERNAME=${OS_USERNAME:-admin} - PASSWORD=${OS_PASSWORD:-console} - - # OS_TENANT_NAME is still present to keep backward compatibility with legacy - # deployments, but should be replaced by OS_PROJECT_NAME. - TENANT_NAME=${OS_TENANT_NAME:-admin} - PROJECT_NAME=${OS_PROJECT_NAME:-$TENANT_NAME} - PROJECT_ID=`openstack project show admin|grep '\bid\b' |awk -F '|' '{print $3}'|sed -e 's/^[[:space:]]*//'` - USER_DOMAIN_ID=${OS_USER_DOMAIN_ID:-default} - - rm -f ~/storperf_admin-rc - touch ~/storperf_admin-rc - - echo "OS_AUTH_URL="$AUTH_URL >> ~/storperf_admin-rc - echo "OS_USERNAME="$USERNAME >> ~/storperf_admin-rc - echo "OS_PASSWORD="$PASSWORD >> ~/storperf_admin-rc - echo "OS_PROJECT_NAME="$PROJECT_NAME >> ~/storperf_admin-rc - echo "OS_PROJECT_ID="$PROJECT_ID >> ~/storperf_admin-rc - echo "OS_USER_DOMAIN_ID="$USER_DOMAIN_ID >> ~/storperf_admin-rc - - -The generated "storperf_admin-rc" file will be stored in the root directory. If -you installed *Yardstick* using Docker, this file will be located in the -container. You may need to copy it to the root directory of the Storperf -deployed host. - -Step 1: Plug-in configuration file preparation ->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - -To install a plug-in, first you need to prepare a plug-in configuration file in -YAML format and store it in the "plugin" directory. The plugin configration file -work as the input of yardstick "plugin" command. Below is the Storperf plug-in -configuration file sample: -:: - - --- - # StorPerf plugin configuration file - # Used for integration StorPerf into Yardstick as a plugin - schema: "yardstick:plugin:0.1" - plugins: - name: storperf - deployment: - ip: 192.168.23.2 - user: root - password: root - -In the plug-in configuration file, you need to specify the plug-in name and the -plug-in deployment info, including node ip, node login username and password. -Here the Storperf will be installed on IP 192.168.23.2 which is the Jump Host -in my local environment. - -Step 2: Plug-in install/remove scripts preparation ->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - -In "yardstick/resource/scripts" directory, there are two folders: a "install" -folder and a "remove" folder. You need to store the plug-in install/remove -scripts in these two folders respectively. - -The detailed installation or remove operation should de defined in these two -scripts. The name of both install and remove scripts should match the plugin-in -name that you specified in the plug-in configuration file. - -For example, the install and remove scripts for Storperf are both named to -"storperf.bash". - -Step 3: Install and remove Storperf ->>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - -To install Storperf, simply execute the following command:: - - # Install Storperf - yardstick plugin install plugin/storperf.yaml - -removing Storperf from yardstick -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To remove Storperf, simply execute the following command:: - - # Remove Storperf - yardstick plugin remove plugin/storperf.yaml - -What yardstick plugin command does is using the username and password to log -into the deployment target and then execute the corresponding install or remove -script. diff --git a/docs/testing/user/userguide/06-result-store-InfluxDB.rst b/docs/testing/user/userguide/06-result-store-InfluxDB.rst deleted file mode 100644 index 747927889..000000000 --- a/docs/testing/user/userguide/06-result-store-InfluxDB.rst +++ /dev/null @@ -1,86 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, 2016 Huawei Technologies Co.,Ltd and others. - -============================================== -Store Other Project's Test Results in InfluxDB -============================================== - -Abstract -======== - -.. _Framework: https://wiki.opnfv.org/download/attachments/6827660/wiki.png?version=1&modificationDate=1470298075000&api=v2 - -This chapter illustrates how to run plug-in test cases and store test results -into community's InfluxDB. The framework is shown in Framework_. - - -.. image:: images/InfluxDB_store.png - :width: 800px - :alt: Store Other Project's Test Results in InfluxDB - -Store Storperf Test Results into Community's InfluxDB -===================================================== - -.. _Influxdb: https://git.opnfv.org/cgit/yardstick/tree/yardstick/dispatcher/influxdb.py -.. _Mingjiang: limingjiang@huawei.com -.. _Visual: https://wiki.opnfv.org/download/attachments/6827660/tc074.PNG?version=1&modificationDate=1470298075000&api=v2 -.. _Login: http://testresults.opnfv.org/grafana/login - -As shown in Framework_, there are two ways to store Storperf test results -into community's InfluxDB: - -1. Yardstick executes Storperf test case (TC074), posting test job to Storperf - container via ReST API. After the test job is completed, Yardstick reads - test results via ReST API from Storperf and posts test data to the influxDB. - -2. Additionally, Storperf can run tests by itself and post the test result - directly to the InfluxDB. The method for posting data directly to influxDB - will be supported in the future. - -Our plan is to support rest-api in D release so that other testing projects can -call the rest-api to use yardstick dispatcher service to push data to yardstick's -influxdb database. - -For now, influxdb only support line protocol, and the json protocol is deprecated. - -Take ping test case for example, the raw_result is json format like this: -:: - - "benchmark": { - "timestamp": 1470315409.868095, - "errors": "", - "data": { - "rtt": { - "ares": 1.125 - } - }, - "sequence": 1 - }, - "runner_id": 2625 - } - -With the help of "influxdb_line_protocol", the json is transform to like below as a line string: -:: - - 'ping,deploy_scenario=unknown,host=athena.demo,installer=unknown,pod_name=unknown, - runner_id=2625,scenarios=Ping,target=ares.demo,task_id=77755f38-1f6a-4667-a7f3- - 301c99963656,version=unknown rtt.ares=1.125 1470315409868094976' - -So, for data output of json format, you just need to transform json into line format and call -influxdb api to post the data into the database. All this function has been implemented in Influxdb_. -If you need support on this, please contact Mingjiang_. -:: - - curl -i -XPOST 'http://104.197.68.199:8086/write?db=yardstick' -- - data-binary 'ping,deploy_scenario=unknown,host=athena.demo,installer=unknown, ...' - -Grafana will be used for visualizing the collected test data, which is shown in Visual_. Grafana -can be accessed by Login_. - - -.. image:: images/results_visualization.png - :width: 800px - :alt: results visualization - diff --git a/docs/testing/user/userguide/06-yardstick-plugin.rst b/docs/testing/user/userguide/06-yardstick-plugin.rst new file mode 100644 index 000000000..bc35e239d --- /dev/null +++ b/docs/testing/user/userguide/06-yardstick-plugin.rst @@ -0,0 +1,159 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Ericsson AB, Huawei Technologies Co.,Ltd and others. + +=================================== +Installing a plug-in into Yardstick +=================================== + + +Abstract +======== + +Yardstick provides a ``plugin`` CLI command to support integration with other +OPNFV testing projects. Below is an example invocation of Yardstick plugin +command and Storperf plug-in sample. + + +Installing Storperf into Yardstick +================================== + +Storperf is delivered as a Docker container from +https://hub.docker.com/r/opnfv/storperf/tags/. + +There are two possible methods for installation in your environment: + +* Run container on Jump Host +* Run container in a VM + +In this introduction we will install Storperf on Jump Host. + + +Step 0: Environment preparation +------------------------------- + +Running Storperf on Jump Host +Requirements: + +* Docker must be installed +* Jump Host must have access to the OpenStack Controller API +* Jump Host must have internet connectivity for downloading docker image +* Enough floating IPs must be available to match your agent count + +Before installing Storperf into yardstick you need to check your openstack +environment and other dependencies: + +1. Make sure docker is installed. +2. Make sure Keystone, Nova, Neutron, Glance, Heat are installed correctly. +3. Make sure Jump Host have access to the OpenStack Controller API. +4. Make sure Jump Host must have internet connectivity for downloading docker + image. +5. You need to know where to get basic openstack Keystone authorization info, + such as OS_PASSWORD, OS_PROJECT_NAME, OS_AUTH_URL, OS_USERNAME. +6. To run a Storperf container, you need to have OpenStack Controller + environment variables defined and passed to Storperf container. The best way + to do this is to put environment variables in a "storperf_admin-rc" file. + The storperf_admin-rc should include credential environment variables at + least: + + * OS_AUTH_URL + * OS_USERNAME + * OS_PASSWORD + * OS_PROJECT_NAME + * OS_PROJECT_ID + * OS_USER_DOMAIN_ID + +*Yardstick* has a ``prepare_storperf_admin-rc.sh`` script which can be used to +generate the ``storperf_admin-rc`` file, this script is located at +``test/ci/prepare_storperf_admin-rc.sh`` + +:: + + #!/bin/bash + # Prepare storperf_admin-rc for StorPerf. + AUTH_URL=${OS_AUTH_URL} + USERNAME=${OS_USERNAME:-admin} + PASSWORD=${OS_PASSWORD:-console} + + # OS_TENANT_NAME is still present to keep backward compatibility with legacy + # deployments, but should be replaced by OS_PROJECT_NAME. + TENANT_NAME=${OS_TENANT_NAME:-admin} + PROJECT_NAME=${OS_PROJECT_NAME:-$TENANT_NAME} + PROJECT_ID=`openstack project show admin|grep '\bid\b' |awk -F '|' '{print $3}'|sed -e 's/^[[:space:]]*//'` + USER_DOMAIN_ID=${OS_USER_DOMAIN_ID:-default} + + rm -f ~/storperf_admin-rc + touch ~/storperf_admin-rc + + echo "OS_AUTH_URL="$AUTH_URL >> ~/storperf_admin-rc + echo "OS_USERNAME="$USERNAME >> ~/storperf_admin-rc + echo "OS_PASSWORD="$PASSWORD >> ~/storperf_admin-rc + echo "OS_PROJECT_NAME="$PROJECT_NAME >> ~/storperf_admin-rc + echo "OS_PROJECT_ID="$PROJECT_ID >> ~/storperf_admin-rc + echo "OS_USER_DOMAIN_ID="$USER_DOMAIN_ID >> ~/storperf_admin-rc + + +The generated ``storperf_admin-rc`` file will be stored in the root directory. +If you installed *Yardstick* using Docker, this file will be located in the +container. You may need to copy it to the root directory of the Storperf +deployed host. + +Step 1: Plug-in configuration file preparation +---------------------------------------------- + +To install a plug-in, first you need to prepare a plug-in configuration file in +YAML format and store it in the "plugin" directory. The plugin configration +file work as the input of yardstick "plugin" command. Below is the Storperf +plug-in configuration file sample: +:: + + --- + # StorPerf plugin configuration file + # Used for integration StorPerf into Yardstick as a plugin + schema: "yardstick:plugin:0.1" + plugins: + name: storperf + deployment: + ip: 192.168.23.2 + user: root + password: root + +In the plug-in configuration file, you need to specify the plug-in name and the +plug-in deployment info, including node ip, node login username and password. +Here the Storperf will be installed on IP 192.168.23.2 which is the Jump Host +in my local environment. + +Step 2: Plug-in install/remove scripts preparation +-------------------------------------------------- + +In ``yardstick/resource/scripts`` directory, there are two folders: an +``install`` folder and a ``remove`` folder. You need to store the plug-in +install/remove scripts in these two folders respectively. + +The detailed installation or remove operation should de defined in these two +scripts. The name of both install and remove scripts should match the plugin-in +name that you specified in the plug-in configuration file. + +For example, the install and remove scripts for Storperf are both named +``storperf.bash``. + +Step 3: Install and remove Storperf +----------------------------------- + +To install Storperf, simply execute the following command:: + + # Install Storperf + yardstick plugin install plugin/storperf.yaml + +Removing Storperf from yardstick +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To remove Storperf, simply execute the following command:: + + # Remove Storperf + yardstick plugin remove plugin/storperf.yaml + +What yardstick plugin command does is using the username and password to log +into the deployment target and then execute the corresponding install or remove +script. diff --git a/docs/testing/user/userguide/07-grafana.rst b/docs/testing/user/userguide/07-grafana.rst deleted file mode 100644 index 416857b71..000000000 --- a/docs/testing/user/userguide/07-grafana.rst +++ /dev/null @@ -1,119 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) 2016 Huawei Technologies Co.,Ltd and others - -================= -Grafana dashboard -================= - - -Abstract -======== - -This chapter describes the Yardstick grafana dashboard. The Yardstick grafana -dashboard can be found here: http://testresults.opnfv.org/grafana/ - - -.. image:: images/login.png - :width: 800px - :alt: Yardstick grafana dashboard - - -Public access -============= - -Yardstick provids a public account for accessing to the dashboard. The username -and password are both set to ‘opnfv’. - - -Testcase dashboard -================== - -For each test case, there is a dedicated dashboard. Shown here is the dashboard -of TC002. - - -.. image:: images/TC002.png - :width: 800px - :alt:TC002 dashboard - -For each test case dashboard. On the top left, we have a dashboard selection, -you can switch to different test cases using this pull-down menu. - -Underneath, we have a pod and scenario selection. -All the pods and scenarios that have ever published test data to the InfluxDB -will be shown here. - -You can check multiple pods or scenarios. - -For each test case, we have a short description and a link to detailed test -case information in Yardstick user guide. - -Underneath, it is the result presentation section. -You can use the time period selection on the top right corner to zoom in or -zoom out the chart. - - -Administration access -===================== - -For a user with administration rights it is easy to update and save any -dashboard configuration. Saved updates immediately take effect and become live. -This may cause issues like: - -- Changes and updates made to the live configuration in Grafana can compromise - existing Grafana content in an unwanted, unpredicted or incompatible way. - Grafana as such is not version controlled, there exists one single Grafana - configuration per dashboard. -- There is a risk several people can disturb each other when doing updates to - the same Grafana dashboard at the same time. - -Any change made by administrator should be careful. - - -Add a dashboard into yardstick grafana -====================================== - -Due to security concern, users that using the public opnfv account are not able -to edit the yardstick grafana directly.It takes a few more steps for a -non-yardstick user to add a custom dashboard into yardstick grafana. - -There are 6 steps to go. - - -.. image:: images/add.png - :width: 800px - :alt: Add a dashboard into yardstick grafana - - -1. You need to build a local influxdb and grafana, so you can do the work - locally. You can refer to How to deploy InfluxDB and Grafana locally wiki - page about how to do this. - -2. Once step one is done, you can fetch the existing grafana dashboard - configuration file from the yardstick repository and import it to your local - grafana. After import is done, you grafana dashboard will be ready to use - just like the community’s dashboard. - -3. The third step is running some test cases to generate test results and - publishing it to your local influxdb. - -4. Now you have some data to visualize in your dashboard. In the fourth step, - it is time to create your own dashboard. You can either modify an existing - dashboard or try to create a new one from scratch. If you choose to modify - an existing dashboard then in the curtain menu of the existing dashboard do - a "Save As..." into a new dashboard copy instance, and then continue doing - all updates and saves within the dashboard copy. - -5. When finished with all Grafana configuration changes in this temporary - dashboard then chose "export" of the updated dashboard copy into a JSON file - and put it up for review in Gerrit, in file /yardstick/dashboard/Yardstick-TCxxx-yyyyyyyyyyyyy. - For instance a typical default name of the file would be "Yardstick-TC001 Copy-1234567891234". - -6. Once you finish your dashboard, the next step is exporting the configuration - file and propose a patch into Yardstick. Yardstick team will review and - merge it into Yardstick repository. After approved review Yardstick team - will do an "import" of the JSON file and also a "save dashboard" as soon as - possible to replace the old live dashboard configuration. - diff --git a/docs/testing/user/userguide/07-result-store-InfluxDB.rst b/docs/testing/user/userguide/07-result-store-InfluxDB.rst new file mode 100644 index 000000000..cde931376 --- /dev/null +++ b/docs/testing/user/userguide/07-result-store-InfluxDB.rst @@ -0,0 +1,88 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2016 Huawei Technologies Co.,Ltd and others. + +============================================== +Store Other Project's Test Results in InfluxDB +============================================== + +Abstract +======== + +.. _Framework: https://wiki.opnfv.org/download/attachments/6827660/wiki.png?version=1&modificationDate=1470298075000&api=v2 + +This chapter illustrates how to run plug-in test cases and store test results +into community's InfluxDB. The framework is shown in Framework_. + + +.. image:: images/InfluxDB_store.png + :width: 800px + :alt: Store Other Project's Test Results in InfluxDB + +Store Storperf Test Results into Community's InfluxDB +===================================================== + +.. _Influxdb: https://git.opnfv.org/cgit/yardstick/tree/yardstick/dispatcher/influxdb.py +.. _Mingjiang: mailto:limingjiang@huawei.com +.. _Visual: https://wiki.opnfv.org/download/attachments/6827660/tc074.PNG?version=1&modificationDate=1470298075000&api=v2 +.. _Login: http://testresults.opnfv.org/grafana/login + +As shown in Framework_, there are two ways to store Storperf test results +into community's InfluxDB: + +1. Yardstick executes Storperf test case (TC074), posting test job to Storperf + container via ReST API. After the test job is completed, Yardstick reads + test results via ReST API from Storperf and posts test data to the influxDB. + +2. Additionally, Storperf can run tests by itself and post the test result + directly to the InfluxDB. The method for posting data directly to influxDB + will be supported in the future. + +Our plan is to support rest-api in D release so that other testing projects can +call the rest-api to use yardstick dispatcher service to push data to +Yardstick's InfluxDB database. + +For now, InfluxDB only supports line protocol, and the json protocol is +deprecated. + +Take ping test case for example, the ``raw_result`` is json format like this: +:: + + "benchmark": { + "timestamp": 1470315409.868095, + "errors": "", + "data": { + "rtt": { + "ares": 1.125 + } + }, + "sequence": 1 + }, + "runner_id": 2625 + } + +With the help of "influxdb_line_protocol", the json is transform to like below +as a line string:: + + 'ping,deploy_scenario=unknown,host=athena.demo,installer=unknown,pod_name=unknown, + runner_id=2625,scenarios=Ping,target=ares.demo,task_id=77755f38-1f6a-4667-a7f3- + 301c99963656,version=unknown rtt.ares=1.125 1470315409868094976' + +So, for data output of json format, you just need to transform json into line +format and call influxdb api to post the data into the database. All this +function has been implemented in Influxdb_. If you need support on this, please +contact Mingjiang_. +:: + + curl -i -XPOST 'http://104.197.68.199:8086/write?db=yardstick' -- + data-binary 'ping,deploy_scenario=unknown,host=athena.demo,installer=unknown, ...' + +Grafana will be used for visualizing the collected test data, which is shown in +Visual_. Grafana can be accessed by Login_. + + +.. image:: images/results_visualization.png + :width: 800px + :alt: results visualization + diff --git a/docs/testing/user/userguide/08-api.rst b/docs/testing/user/userguide/08-api.rst deleted file mode 100644 index 2206c2ac8..000000000 --- a/docs/testing/user/userguide/08-api.rst +++ /dev/null @@ -1,746 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, Huawei Technologies Co.,Ltd and others. - -Yardstick Restful API -====================== - - -Abstract --------- - -Yardstick support restful API since Danube. - - -Available API -------------- - -/yardstick/env/action -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to prepare Yardstick test environment. For Euphrates, it supports: - -1. Prepare yardstick test environment, including set external network environment variable, load Yardstick VM images and create flavors; -2. Start an InfluxDB Docker container and config Yardstick output to InfluxDB; -3. Start a Grafana Docker container and config it with the InfluxDB. - -Which API to call will depend on the parameters. - - -Method: POST - - -Prepare Yardstick test environment -Example:: - - { - 'action': 'prepare_env' - } - -This is an asynchronous API. You need to call /yardstick/asynctask API to get the task result. - - -Start and config an InfluxDB docker container -Example:: - - { - 'action': 'create_influxdb' - } - -This is an asynchronous API. You need to call /yardstick/asynctask API to get the task result. - - -Start and config a Grafana docker container -Example:: - - { - 'action': 'create_grafana' - } - -This is an asynchronous API. You need to call /yardstick/asynctask API to get the task result. - - -/yardstick/asynctask -^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to get the status of asynchronous tasks - - -Method: GET - - -Get the status of asynchronous tasks -Example:: - - http://:/yardstick/asynctask?task_id=3f3f5e03-972a-4847-a5f8-154f1b31db8c - -The returned status will be 0(running), 1(finished) and 2(failed). - -NOTE:: - - : The ip of the host where you start your yardstick container - : The outside port of port mapping which set when you start start yardstick container - - -/yardstick/testcases -^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to list all released Yardstick test cases. - - -Method: GET - - -Get a list of released test cases -Example:: - - http://:/yardstick/testcases - - -/yardstick/testcases/release/action -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to run a Yardstick released test case. - - -Method: POST - - -Run a released test case -Example:: - - { - 'action': 'run_test_case', - 'args': { - 'opts': {}, - 'testcase': 'opnfv_yardstick_tc002' - } - } - -This is an asynchronous API. You need to call /yardstick/results to get the result. - - -/yardstick/testcases/samples/action -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to run a Yardstick sample test case. - - -Method: POST - - -Run a sample test case -Example:: - - { - 'action': 'run_test_case', - 'args': { - 'opts': {}, - 'testcase': 'ping' - } - } - -This is an asynchronous API. You need to call /yardstick/results to get the result. - - -/yardstick/testcases//docs -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to the documentation of a certain released test case. - - -Method: GET - - -Get the documentation of a certain test case -Example:: - - http://:/yardstick/taskcases/opnfv_yardstick_tc002/docs - - -/yardstick/testsuites/action -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to run a Yardstick test suite. - - -Method: POST - - -Run a test suite -Example:: - - { - 'action': 'run_test_suite', - 'args': { - 'opts': {}, - 'testsuite': 'opnfv_smoke' - } - } - -This is an asynchronous API. You need to call /yardstick/results to get the result. - - -/yardstick/tasks//log -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to get the real time log of test case execution. - - -Method: GET - - -Get real time of test case execution -Example:: - - http://:/yardstick/tasks/14795be8-f144-4f54-81ce-43f4e3eab33f/log?index=0 - - -/yardstick/results -^^^^^^^^^^^^^^^^^^ - -Description: This API is used to get the test results of tasks. If you call /yardstick/testcases/samples/action API, it will return a task id. You can use the returned task id to get the results by using this API. - - -Method: GET - - -Get test results of one task -Example:: - - http://:/yardstick/results?task_id=3f3f5e03-972a-4847-a5f8-154f1b31db8c - -This API will return a list of test case result - - -/api/v2/yardstick/openrcs -^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API provides functionality of handling OpenStack credential file (openrc). For Euphrates, it supports: - -1. Upload an openrc file for an OpenStack environment; -2. Update an openrc; -3. Get openrc file information; -4. Delete an openrc file. - -Which API to call will depend on the parameters. - - -METHOD: POST - - -Upload an openrc file for an OpenStack environment -Example:: - - { - 'action': 'upload_openrc', - 'args': { - 'file': file, - 'environment_id': environment_id - } - } - - -METHOD: POST - - -Update an openrc file -Example:: - - { - 'action': 'update_openrc', - 'args': { - 'openrc': { - "EXTERNAL_NETWORK": "ext-net", - "OS_AUTH_URL": "http://192.168.23.51:5000/v3", - "OS_IDENTITY_API_VERSION": "3", - "OS_IMAGE_API_VERSION": "2", - "OS_PASSWORD": "console", - "OS_PROJECT_DOMAIN_NAME": "default", - "OS_PROJECT_NAME": "admin", - "OS_USERNAME": "admin", - "OS_USER_DOMAIN_NAME": "default" - }, - 'environment_id': environment_id - } - } - - -/api/v2/yardstick/openrcs/ -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API provides functionality of handling OpenStack credential file (openrc). For Euphrates, it supports: - -1. Get openrc file information; -2. Delete an openrc file. - - -METHOD: GET - -Get openrc file information -Example:: - - http://:/api/v2/yardstick/openrcs/5g6g3e02-155a-4847-a5f8-154f1b31db8c - - -METHOD: DELETE - - -Delete openrc file -Example:: - - http://:/api/v2/yardstick/openrcs/5g6g3e02-155a-4847-a5f8-154f1b31db8c - - -/api/v2/yardstick/pods -^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API provides functionality of handling Yardstick pod file (pod.yaml). For Euphrates, it supports: - -1. Upload a pod file; - -Which API to call will depend on the parameters. - - -METHOD: POST - - -Upload a pod.yaml file -Example:: - - { - 'action': 'upload_pod_file', - 'args': { - 'file': file, - 'environment_id': environment_id - } - } - - -/api/v2/yardstick/pods/ -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API provides functionality of handling Yardstick pod file (pod.yaml). For Euphrates, it supports: - -1. Get pod file information; -2. Delete an openrc file. - -METHOD: GET - -Get pod file information -Example:: - - http://:/api/v2/yardstick/pods/5g6g3e02-155a-4847-a5f8-154f1b31db8c - - -METHOD: DELETE - -Delete openrc file -Example:: - - http://:/api/v2/yardstick/pods/5g6g3e02-155a-4847-a5f8-154f1b31db8c - - -/api/v2/yardstick/images -^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to do some work related to Yardstick VM images. For Euphrates, it supports: - -1. Load Yardstick VM images; - -Which API to call will depend on the parameters. - - -METHOD: POST - - -Load VM images -Example:: - - { - 'action': 'load_image', - 'args': { - 'name': 'yardstick-image' - } - } - - -/api/v2/yardstick/images/ -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to do some work related to Yardstick VM images. For Euphrates, it supports: - -1. Get image's information; -2. Delete images - -METHOD: GET - -Get image information -Example:: - - http://:/api/v2/yardstick/images/5g6g3e02-155a-4847-a5f8-154f1b31db8c - - -METHOD: DELETE - -Delete images -Example:: - - http://:/api/v2/yardstick/images/5g6g3e02-155a-4847-a5f8-154f1b31db8c - - -/api/v2/yardstick/tasks -^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to do some work related to yardstick tasks. For Euphrates, it supports: - -1. Create a Yardstick task; - -Which API to call will depend on the parameters. - - -METHOD: POST - - -Create a Yardstick task -Example:: - - { - 'action': 'create_task', - 'args': { - 'name': 'task1', - 'project_id': project_id - } - } - - -/api/v2/yardstick/tasks/ -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to do some work related to yardstick tasks. For Euphrates, it supports: - -1. Add a environment to a task -2. Add a test case to a task; -3. Add a test suite to a task; -4. run a Yardstick task; -5. Get a tasks' information; -6. Delete a task. - - -METHOD: PUT - -Add a environment to a task - -Example:: - - { - 'action': 'add_environment', - 'args': { - 'environment_id': 'e3cadbbb-0419-4fed-96f1-a232daa0422a' - } - } - - -METHOD: PUT - -Add a test case to a task -Example:: - - { - 'action': 'add_case', - 'args': { - 'case_name': 'opnfv_yardstick_tc002', - 'case_content': case_content - } - } - - - -METHOD: PUT - -Add a test suite to a task -Example:: - - { - 'action': 'add_suite', - 'args': { - 'suite_name': 'opnfv_smoke', - 'suite_content': suite_content - } - } - - -METHOD: PUT - -Run a task - -Example:: - - { - 'action': 'run' - } - - - -METHOD: GET - -Get a task's information -Example:: - - http://:/api/v2/yardstick/tasks/5g6g3e02-155a-4847-a5f8-154f1b31db8c - - -METHOD: DELETE - -Delete a task - -Example:: - http://:/api/v2/yardstick/tasks/5g6g3e02-155a-4847-a5f8-154f1b31db8c - - -/api/v2/yardstick/testcases -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to do some work related to yardstick testcases. For Euphrates, it supports: - -1. Upload a test case; -2. Get all released test cases' information; - -Which API to call will depend on the parameters. - - -METHOD: POST - - -Upload a test case -Example:: - - { - 'action': 'upload_case', - 'args': { - 'file': file - } - } - - -METHOD: GET - - -Get all released test cases' information -Example:: - - http://:/api/v2/yardstick/testcases - - -/api/v2/yardstick/testcases/ -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to do some work related to yardstick testcases. For Euphrates, it supports: - -1. Get certain released test case's information; -2. Delete a test case. - -METHOD: GET - - -Get certain released test case's information -Example:: - - http://:/api/v2/yardstick/testcases/opnfv_yardstick_tc002 - - -METHOD: DELETE - - -Delete a certain test case -Example:: - http://:/api/v2/yardstick/testcases/opnfv_yardstick_tc002 - - -/api/v2/yardstick/testsuites -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to do some work related to yardstick test suites. For Euphrates, it supports: - -1. Create a test suite; -2. Get all test suites; - -Which API to call will depend on the parameters. - - -METHOD: POST - - -Create a test suite -Example:: - - { - 'action': 'create_suite', - 'args': { - 'name': , - 'testcases': [ - 'opnfv_yardstick_tc002' - ] - } - } - - -METHOD: GET - - -Get all test suite -Example:: - - http://:/api/v2/yardstick/testsuites - - -/api/v2/yardstick/testsuites -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to do some work related to yardstick test suites. For Euphrates, it supports: - -1. Get certain test suite's information; -2. Delete a test case. - -METHOD: GET - - -Get certain test suite's information -Example:: - - http://:/api/v2/yardstick/testsuites/ - - -METHOD: DELETE - - -Delete a certain test suite -Example:: - - http://:/api/v2/yardstick/testsuites/ - - -/api/v2/yardstick/projects -^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to do some work related to yardstick test projects. For Euphrates, it supports: - -1. Create a Yardstick project; -2. Get all projects; - -Which API to call will depend on the parameters. - - -METHOD: POST - - -Create a Yardstick project -Example:: - - { - 'action': 'create_project', - 'args': { - 'name': 'project1' - } - } - - -METHOD: GET - - -Get all projects' information -Example:: - - http://:/api/v2/yardstick/projects - - -/api/v2/yardstick/projects -^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to do some work related to yardstick test projects. For Euphrates, it supports: - -1. Get certain project's information; -2. Delete a project. - -METHOD: GET - - -Get certain project's information -Example:: - - http://:/api/v2/yardstick/projects/ - - -METHOD: DELETE - - -Delete a certain project -Example:: - - http://:/api/v2/yardstick/projects/ - - -/api/v2/yardstick/containers -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to do some work related to Docker containers. For Euphrates, it supports: - -1. Create a Grafana Docker container; -2. Create an InfluxDB Docker container; - -Which API to call will depend on the parameters. - - -METHOD: POST - - -Create a Grafana Docker container -Example:: - - { - 'action': 'create_grafana', - 'args': { - 'environment_id': - } - } - - -METHOD: POST - - -Create an InfluxDB Docker container -Example:: - - { - 'action': 'create_influxdb', - 'args': { - 'environment_id': - } - } - - -/api/v2/yardstick/containers/ -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Description: This API is used to do some work related to Docker containers. For Euphrates, it supports: - -1. Get certain container's information; -2. Delete a container. - -METHOD: GET - - -Get certain container's information -Example:: - - http://:/api/v2/yardstick/containers/ - - -METHOD: DELETE - - -Delete a certain container -Example:: - - http://:/api/v2/yardstick/containers/ diff --git a/docs/testing/user/userguide/08-grafana.rst b/docs/testing/user/userguide/08-grafana.rst new file mode 100644 index 000000000..29bc23a08 --- /dev/null +++ b/docs/testing/user/userguide/08-grafana.rst @@ -0,0 +1,121 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2016 Huawei Technologies Co.,Ltd and others + +================= +Grafana dashboard +================= + + +Abstract +======== + +This chapter describes the Yardstick grafana dashboard. The Yardstick grafana +dashboard can be found here: http://testresults.opnfv.org/grafana/ + + +.. image:: images/login.png + :width: 800px + :alt: Yardstick grafana dashboard + + +Public access +============= + +Yardstick provids a public account for accessing to the dashboard. The username +and password are both set to ‘opnfv’. + + +Testcase dashboard +================== + +For each test case, there is a dedicated dashboard. Shown here is the dashboard +of TC002. + + +.. image:: images/TC002.png + :width: 800px + :alt:TC002 dashboard + +For each test case dashboard. On the top left, we have a dashboard selection, +you can switch to different test cases using this pull-down menu. + +Underneath, we have a pod and scenario selection. +All the pods and scenarios that have ever published test data to the InfluxDB +will be shown here. + +You can check multiple pods or scenarios. + +For each test case, we have a short description and a link to detailed test +case information in Yardstick user guide. + +Underneath, it is the result presentation section. +You can use the time period selection on the top right corner to zoom in or +zoom out the chart. + + +Administration access +===================== + +For a user with administration rights it is easy to update and save any +dashboard configuration. Saved updates immediately take effect and become live. +This may cause issues like: + +- Changes and updates made to the live configuration in Grafana can compromise + existing Grafana content in an unwanted, unpredicted or incompatible way. + Grafana as such is not version controlled, there exists one single Grafana + configuration per dashboard. +- There is a risk several people can disturb each other when doing updates to + the same Grafana dashboard at the same time. + +Any change made by administrator should be careful. + + +Add a dashboard into yardstick grafana +====================================== + +Due to security concern, users that using the public opnfv account are not able +to edit the yardstick grafana directly.It takes a few more steps for a +non-yardstick user to add a custom dashboard into yardstick grafana. + +There are 6 steps to go. + + +.. image:: images/add.png + :width: 800px + :alt: Add a dashboard into yardstick grafana + + +1. You need to build a local influxdb and grafana, so you can do the work + locally. You can refer to How to deploy InfluxDB and Grafana locally wiki + page about how to do this. + +2. Once step one is done, you can fetch the existing grafana dashboard + configuration file from the yardstick repository and import it to your local + grafana. After import is done, you grafana dashboard will be ready to use + just like the community’s dashboard. + +3. The third step is running some test cases to generate test results and + publishing it to your local influxdb. + +4. Now you have some data to visualize in your dashboard. In the fourth step, + it is time to create your own dashboard. You can either modify an existing + dashboard or try to create a new one from scratch. If you choose to modify + an existing dashboard then in the curtain menu of the existing dashboard do + a "Save As..." into a new dashboard copy instance, and then continue doing + all updates and saves within the dashboard copy. + +5. When finished with all Grafana configuration changes in this temporary + dashboard then chose "export" of the updated dashboard copy into a JSON file + and put it up for review in Gerrit, in file + ``/yardstick/dashboard/Yardstick-TCxxx-yyyyyyyyyyyyy``. + For instance a typical default name of the file would be + ``Yardstick-TC001 Copy-1234567891234``. + +6. Once you finish your dashboard, the next step is exporting the configuration + file and propose a patch into Yardstick. Yardstick team will review and + merge it into Yardstick repository. After approved review Yardstick team + will do an "import" of the JSON file and also a "save dashboard" as soon as + possible to replace the old live dashboard configuration. + diff --git a/docs/testing/user/userguide/09-api.rst b/docs/testing/user/userguide/09-api.rst new file mode 100644 index 000000000..f0ae3980b --- /dev/null +++ b/docs/testing/user/userguide/09-api.rst @@ -0,0 +1,769 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Huawei Technologies Co.,Ltd and others. + +===================== +Yardstick Restful API +===================== + + +Abstract +======== + +Yardstick support restful API since Danube. + + +Available API +============= + +/yardstick/env/action +--------------------- + +Description: This API is used to prepare Yardstick test environment. +For Euphrates, it supports: + +1. Prepare yardstick test environment, including setting the + ``EXTERNAL_NETWORK`` environment variable, load Yardstick VM images and + create flavors; +2. Start an InfluxDB Docker container and config Yardstick output to InfluxDB; +3. Start a Grafana Docker container and config it with the InfluxDB. + +Which API to call will depend on the parameters. + + +Method: POST + + +Prepare Yardstick test environment +Example:: + + { + 'action': 'prepare_env' + } + +This is an asynchronous API. You need to call ``/yardstick/asynctask`` API to +get the task result. + + +Start and config an InfluxDB docker container +Example:: + + { + 'action': 'create_influxdb' + } + +This is an asynchronous API. You need to call ``/yardstick/asynctask`` API to +get the task result. + + +Start and config a Grafana docker container +Example:: + + { + 'action': 'create_grafana' + } + +This is an asynchronous API. You need to call ``/yardstick/asynctask`` API to +get the task result. + + +/yardstick/asynctask +-------------------- + +Description: This API is used to get the status of asynchronous tasks + + +Method: GET + + +Get the status of asynchronous tasks +Example:: + + http://:/yardstick/asynctask?task_id=3f3f5e03-972a-4847-a5f8-154f1b31db8c + +The returned status will be 0(running), 1(finished) and 2(failed). + +NOTE:: + + : The ip of the host where you start your yardstick container + : The outside port of port mapping which set when you start start yardstick container + + +/yardstick/testcases +-------------------- + +Description: This API is used to list all released Yardstick test cases. + + +Method: GET + + +Get a list of released test cases +Example:: + + http://:/yardstick/testcases + + +/yardstick/testcases/release/action +----------------------------------- + +Description: This API is used to run a Yardstick released test case. + + +Method: POST + + +Run a released test case +Example:: + + { + 'action': 'run_test_case', + 'args': { + 'opts': {}, + 'testcase': 'opnfv_yardstick_tc002' + } + } + +This is an asynchronous API. You need to call ``/yardstick/results`` to get the +result. + + +/yardstick/testcases/samples/action +----------------------------------- + +Description: This API is used to run a Yardstick sample test case. + + +Method: POST + + +Run a sample test case +Example:: + + { + 'action': 'run_test_case', + 'args': { + 'opts': {}, + 'testcase': 'ping' + } + } + +This is an asynchronous API. You need to call ``/yardstick/results`` to get +the result. + + +/yardstick/testcases//docs +----------------------------------------- + +Description: This API is used to the documentation of a certain released test +case. + + +Method: GET + + +Get the documentation of a certain test case +Example:: + + http://:/yardstick/taskcases/opnfv_yardstick_tc002/docs + + +/yardstick/testsuites/action +---------------------------- + +Description: This API is used to run a Yardstick test suite. + + +Method: POST + + +Run a test suite +Example:: + + { + 'action': 'run_test_suite', + 'args': { + 'opts': {}, + 'testsuite': 'opnfv_smoke' + } + } + +This is an asynchronous API. You need to call /yardstick/results to get the +result. + + +/yardstick/tasks//log +------------------------------ + +Description: This API is used to get the real time log of test case execution. + + +Method: GET + + +Get real time of test case execution +Example:: + + http://:/yardstick/tasks/14795be8-f144-4f54-81ce-43f4e3eab33f/log?index=0 + + +/yardstick/results +------------------ + +Description: This API is used to get the test results of tasks. If you call +/yardstick/testcases/samples/action API, it will return a task id. You can use +the returned task id to get the results by using this API. + + +Method: GET + + +Get test results of one task +Example:: + + http://:/yardstick/results?task_id=3f3f5e03-972a-4847-a5f8-154f1b31db8c + +This API will return a list of test case result + + +/api/v2/yardstick/openrcs +------------------------- + +Description: This API provides functionality of handling OpenStack credential +file (openrc). For Euphrates, it supports: + +1. Upload an openrc file for an OpenStack environment; +2. Update an openrc; +3. Get openrc file information; +4. Delete an openrc file. + +Which API to call will depend on the parameters. + + +METHOD: POST + + +Upload an openrc file for an OpenStack environment +Example:: + + { + 'action': 'upload_openrc', + 'args': { + 'file': file, + 'environment_id': environment_id + } + } + + +METHOD: POST + + +Update an openrc file +Example:: + + { + 'action': 'update_openrc', + 'args': { + 'openrc': { + "EXTERNAL_NETWORK": "ext-net", + "OS_AUTH_URL": "http://192.168.23.51:5000/v3", + "OS_IDENTITY_API_VERSION": "3", + "OS_IMAGE_API_VERSION": "2", + "OS_PASSWORD": "console", + "OS_PROJECT_DOMAIN_NAME": "default", + "OS_PROJECT_NAME": "admin", + "OS_USERNAME": "admin", + "OS_USER_DOMAIN_NAME": "default" + }, + 'environment_id': environment_id + } + } + + +/api/v2/yardstick/openrcs/ +------------------------------------- + +Description: This API provides functionality of handling OpenStack credential file (openrc). For Euphrates, it supports: + +1. Get openrc file information; +2. Delete an openrc file. + + +METHOD: GET + +Get openrc file information +Example:: + + http://:/api/v2/yardstick/openrcs/5g6g3e02-155a-4847-a5f8-154f1b31db8c + + +METHOD: DELETE + + +Delete openrc file +Example:: + + http://:/api/v2/yardstick/openrcs/5g6g3e02-155a-4847-a5f8-154f1b31db8c + + +/api/v2/yardstick/pods +---------------------- + +Description: This API provides functionality of handling Yardstick pod file +(pod.yaml). For Euphrates, it supports: + +1. Upload a pod file; + +Which API to call will depend on the parameters. + + +METHOD: POST + + +Upload a pod.yaml file +Example:: + + { + 'action': 'upload_pod_file', + 'args': { + 'file': file, + 'environment_id': environment_id + } + } + + +/api/v2/yardstick/pods/ +------------------------------- + +Description: This API provides functionality of handling Yardstick pod file (pod.yaml). For Euphrates, it supports: + +1. Get pod file information; +2. Delete an openrc file. + +METHOD: GET + +Get pod file information +Example:: + + http://:/api/v2/yardstick/pods/5g6g3e02-155a-4847-a5f8-154f1b31db8c + + +METHOD: DELETE + +Delete openrc file +Example:: + + http://:/api/v2/yardstick/pods/5g6g3e02-155a-4847-a5f8-154f1b31db8c + + +/api/v2/yardstick/images +------------------------ + +Description: This API is used to do some work related to Yardstick VM images. +For Euphrates, it supports: + +1. Load Yardstick VM images; + +Which API to call will depend on the parameters. + + +METHOD: POST + + +Load VM images +Example:: + + { + 'action': 'load_image', + 'args': { + 'name': 'yardstick-image' + } + } + + +/api/v2/yardstick/images/ +----------------------------------- + +Description: This API is used to do some work related to Yardstick VM images. For Euphrates, it supports: + +1. Get image's information; +2. Delete images + +METHOD: GET + +Get image information +Example:: + + http://:/api/v2/yardstick/images/5g6g3e02-155a-4847-a5f8-154f1b31db8c + + +METHOD: DELETE + +Delete images +Example:: + + http://:/api/v2/yardstick/images/5g6g3e02-155a-4847-a5f8-154f1b31db8c + + +/api/v2/yardstick/tasks +----------------------- + +Description: This API is used to do some work related to yardstick tasks. For +Euphrates, it supports: + +1. Create a Yardstick task; + +Which API to call will depend on the parameters. + + +METHOD: POST + + +Create a Yardstick task +Example:: + + { + 'action': 'create_task', + 'args': { + 'name': 'task1', + 'project_id': project_id + } + } + + +/api/v2/yardstick/tasks/ +-------------------------------- + +Description: This API is used to do some work related to yardstick tasks. For Euphrates, it supports: + +1. Add a environment to a task +2. Add a test case to a task; +3. Add a test suite to a task; +4. run a Yardstick task; +5. Get a tasks' information; +6. Delete a task. + + +METHOD: PUT + +Add a environment to a task + +Example:: + + { + 'action': 'add_environment', + 'args': { + 'environment_id': 'e3cadbbb-0419-4fed-96f1-a232daa0422a' + } + } + + +METHOD: PUT + +Add a test case to a task +Example:: + + { + 'action': 'add_case', + 'args': { + 'case_name': 'opnfv_yardstick_tc002', + 'case_content': case_content + } + } + + + +METHOD: PUT + +Add a test suite to a task +Example:: + + { + 'action': 'add_suite', + 'args': { + 'suite_name': 'opnfv_smoke', + 'suite_content': suite_content + } + } + + +METHOD: PUT + +Run a task + +Example:: + + { + 'action': 'run' + } + + + +METHOD: GET + +Get a task's information +Example:: + + http://:/api/v2/yardstick/tasks/5g6g3e02-155a-4847-a5f8-154f1b31db8c + + +METHOD: DELETE + +Delete a task + +Example:: + + http://:/api/v2/yardstick/tasks/5g6g3e02-155a-4847-a5f8-154f1b31db8c + + +/api/v2/yardstick/testcases +--------------------------- + +Description: This API is used to do some work related to Yardstick testcases. +For Euphrates, it supports: + +1. Upload a test case; +2. Get all released test cases' information; + +Which API to call will depend on the parameters. + + +METHOD: POST + + +Upload a test case +Example:: + + { + 'action': 'upload_case', + 'args': { + 'file': file + } + } + + +METHOD: GET + + +Get all released test cases' information +Example:: + + http://:/api/v2/yardstick/testcases + + +/api/v2/yardstick/testcases/ +--------------------------------------- + +Description: This API is used to do some work related to yardstick testcases. For Euphrates, it supports: + +1. Get certain released test case's information; +2. Delete a test case. + +METHOD: GET + + +Get certain released test case's information +Example:: + + http://:/api/v2/yardstick/testcases/opnfv_yardstick_tc002 + + +METHOD: DELETE + + +Delete a certain test case +Example:: + + http://:/api/v2/yardstick/testcases/opnfv_yardstick_tc002 + + +/api/v2/yardstick/testsuites +---------------------------- + +Description: This API is used to do some work related to yardstick test suites. +For Euphrates, it supports: + +1. Create a test suite; +2. Get all test suites; + +Which API to call will depend on the parameters. + + +METHOD: POST + + +Create a test suite +Example:: + + { + 'action': 'create_suite', + 'args': { + 'name': , + 'testcases': [ + 'opnfv_yardstick_tc002' + ] + } + } + + +METHOD: GET + + +Get all test suite +Example:: + + http://:/api/v2/yardstick/testsuites + + +/api/v2/yardstick/testsuites +---------------------------- + +Description: This API is used to do some work related to yardstick test suites. For Euphrates, it supports: + +1. Get certain test suite's information; +2. Delete a test case. + +METHOD: GET + + +Get certain test suite's information +Example:: + + http://:/api/v2/yardstick/testsuites/ + + +METHOD: DELETE + + +Delete a certain test suite +Example:: + + http://:/api/v2/yardstick/testsuites/ + + +/api/v2/yardstick/projects +-------------------------- + +Description: This API is used to do some work related to Yardstick test +projects. For Euphrates, it supports: + +1. Create a Yardstick project; +2. Get all projects; + +Which API to call will depend on the parameters. + + +METHOD: POST + + +Create a Yardstick project +Example:: + + { + 'action': 'create_project', + 'args': { + 'name': 'project1' + } + } + + +METHOD: GET + + +Get all projects' information +Example:: + + http://:/api/v2/yardstick/projects + + +/api/v2/yardstick/projects +-------------------------- + +Description: This API is used to do some work related to yardstick test projects. For Euphrates, it supports: + +1. Get certain project's information; +2. Delete a project. + +METHOD: GET + + +Get certain project's information +Example:: + + http://:/api/v2/yardstick/projects/ + + +METHOD: DELETE + + +Delete a certain project +Example:: + + http://:/api/v2/yardstick/projects/ + + +/api/v2/yardstick/containers +---------------------------- + +Description: This API is used to do some work related to Docker containers. +For Euphrates, it supports: + +1. Create a Grafana Docker container; +2. Create an InfluxDB Docker container; + +Which API to call will depend on the parameters. + + +METHOD: POST + + +Create a Grafana Docker container +Example:: + + { + 'action': 'create_grafana', + 'args': { + 'environment_id': + } + } + + +METHOD: POST + + +Create an InfluxDB Docker container +Example:: + + { + 'action': 'create_influxdb', + 'args': { + 'environment_id': + } + } + + +/api/v2/yardstick/containers/ +------------------------------------------- + +Description: This API is used to do some work related to Docker containers. For Euphrates, it supports: + +1. Get certain container's information; +2. Delete a container. + +METHOD: GET + + +Get certain container's information +Example:: + + http://:/api/v2/yardstick/containers/ + + +METHOD: DELETE + + +Delete a certain container +Example:: + + http://:/api/v2/yardstick/containers/ diff --git a/docs/testing/user/userguide/09-yardstick_user_interface.rst b/docs/testing/user/userguide/09-yardstick_user_interface.rst deleted file mode 100644 index 9058dd46d..000000000 --- a/docs/testing/user/userguide/09-yardstick_user_interface.rst +++ /dev/null @@ -1,29 +0,0 @@ -Yardstick User Interface -======================== - -This interface provides a user to view the test result -in table format and also values pinned on to a graph. - - -Command -------- -:: - - yardstick report generate - - -Description ------------ - -1. When the command is triggered using the task-id and the testcase -name provided the respective values are retrieved from the -database (influxdb in this particular case). - -2. The values are then formatted and then provided to the html -template framed with complete html body using Django Framework. - -3. Then the whole template is written into a html file. - -The graph is framed with Timestamp on x-axis and output values -(differ from testcase to testcase) on y-axis with the help of -"Highcharts". diff --git a/docs/testing/user/userguide/10-vtc-overview.rst b/docs/testing/user/userguide/10-vtc-overview.rst deleted file mode 100644 index 8ed17873d..000000000 --- a/docs/testing/user/userguide/10-vtc-overview.rst +++ /dev/null @@ -1,128 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, National Center of Scientific Research "Demokritos" and others. - -========================== -Virtual Traffic Classifier -========================== - -Abstract -======== - -.. _TNOVA: http://www.t-nova.eu/ -.. _TNOVAresults: http://www.t-nova.eu/results/ -.. _Yardstick: https://wiki.opnfv.org/yardstick - -This chapter provides an overview of the virtual Traffic Classifier, a -contribution to OPNFV Yardstick_ from the EU Project TNOVA_. -Additional documentation is available in TNOVAresults_. - -Overview -======== - -The virtual Traffic Classifier (:term:`VTC`) :term:`VNF`, comprises of a -Virtual Network Function Component (:term:`VNFC`). The :term:`VNFC` contains -both the Traffic Inspection module, and the Traffic forwarding module, needed -to run the :term:`VNF`. The exploitation of Deep Packet Inspection -(:term:`DPI`) methods for traffic classification is built around two basic -assumptions: - -* third parties unaffiliated with either source or recipient are able to -inspect each IP packet’s payload - -* the classifier knows the relevant syntax of each application’s packet -payloads (protocol signatures, data patterns, etc.). - -The proposed :term:`DPI` based approach will only use an indicative, small -number of the initial packets from each flow in order to identify the content -and not inspect each packet. - -In this respect it follows the Packet Based per Flow State (term:`PBFS`). This -method uses a table to track each session based on the 5-tuples (src address, -dest address, src port,dest port, transport protocol) that is maintained for -each flow. - -Concepts -======== - -* *Traffic Inspection*: The process of packet analysis and application -identification of network traffic that passes through the :term:`VTC`. - -* *Traffic Forwarding*: The process of packet forwarding from an incoming -network interface to a pre-defined outgoing network interface. - -* *Traffic Rule Application*: The process of packet tagging, based on a -predefined set of rules. Packet tagging may include e.g. Type of Service -(:term:`ToS`) field modification. - -Architecture -============ - -The Traffic Inspection module is the most computationally intensive component -of the :term:`VNF`. It implements filtering and packet matching algorithms in -order to support the enhanced traffic forwarding capability of the :term:`VNF`. -The component supports a flow table (exploiting hashing algorithms for fast -indexing of flows) and an inspection engine for traffic classification. - -The implementation used for these experiments exploits the nDPI library. -The packet capturing mechanism is implemented using libpcap. When the -:term:`DPI` engine identifies a new flow, the flow register is updated with the -appropriate information and transmitted across the Traffic Forwarding module, -which then applies any required policy updates. - -The Traffic Forwarding moudle is responsible for routing and packet forwarding. -It accepts incoming network traffic, consults the flow table for classification -information for each incoming flow and then applies pre-defined policies -marking e.g. :term:`ToS`/Differentiated Services Code Point (:term:`DSCP`) -multimedia traffic for Quality of Service (:term:`QoS`) enablement on the -forwarded traffic. -It is assumed that the traffic is forwarded using the default policy until it -is identified and new policies are enforced. - -The expected response delay is considered to be negligible, as only a small -number of packets are required to identify each flow. - -Graphical Overview -================== - -.. code-block:: console - - +----------------------------+ - | | - | Virtual Traffic Classifier | - | | - | Analysing/Forwarding | - | ------------> | - | ethA ethB | - | | - +----------------------------+ - | ^ - | | - v | - +----------------------------+ - | | - | Virtual Switch | - | | - +----------------------------+ - -Install -======= - -run the vTC/build.sh with root privileges - -Run -=== - -:: - - sudo ./pfbridge -a eth1 -b eth2 - - -.. note:: Virtual Traffic Classifier is not support in OPNFV Danube release. - - -Development Environment -======================= - -Ubuntu 14.04 Ubuntu 16.04 diff --git a/docs/testing/user/userguide/10-yardstick-user-interface.rst b/docs/testing/user/userguide/10-yardstick-user-interface.rst new file mode 100644 index 000000000..cadec78ef --- /dev/null +++ b/docs/testing/user/userguide/10-yardstick-user-interface.rst @@ -0,0 +1,30 @@ +======================== +Yardstick User Interface +======================== + +This interface provides a user to view the test result +in table format and also values pinned on to a graph. + + +Command +======= +:: + + yardstick report generate + + +Description +=========== + +1. When the command is triggered using the task-id and the testcase +name provided the respective values are retrieved from the +database (influxdb in this particular case). + +2. The values are then formatted and then provided to the html +template framed with complete html body using Django Framework. + +3. Then the whole template is written into a html file. + +The graph is framed with Timestamp on x-axis and output values +(differ from testcase to testcase) on y-axis with the help of +"Highcharts". diff --git a/docs/testing/user/userguide/11-nsb-overview.rst b/docs/testing/user/userguide/11-nsb-overview.rst deleted file mode 100644 index 332dba47d..000000000 --- a/docs/testing/user/userguide/11-nsb-overview.rst +++ /dev/null @@ -1,205 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, 2016-2017 Intel Corporation. - -Network Services Benchmarking (NSB) -=================================== - -Abstract --------- - -.. _Yardstick: https://wiki.opnfv.org/yardstick - -This chapter provides an overview of the NSB, a contribution to OPNFV -Yardstick_ from Intel. - -Overview --------- - -The goal of NSB is to Extend Yardstick to perform real world VNFs and NFVi Characterization and -benchmarking with repeatable and deterministic methods. - -The Network Service Benchmarking (NSB) extends the yardstick framework to do -VNF characterization and benchmarking in three different execution -environments - bare metal i.e. native Linux environment, standalone virtual -environment and managed virtualized environment (e.g. Open stack etc.). -It also brings in the capability to interact with external traffic generators -both hardware & software based for triggering and validating the traffic -according to user defined profiles. - -NSB extension includes: - - - Generic data models of Network Services, based on ETSI spec `ETSI GS NFV-TST 001 `_ - - - New Standalone context for VNF testing like SRIOV, OVS, OVS-DPDK etc - - - Generic VNF configuration models and metrics implemented with Python - classes - - - Traffic generator features and traffic profiles - - - L1-L3 state-less traffic profiles - - - L4-L7 state-full traffic profiles - - - Tunneling protocol / network overlay support - - - Test case samples - - - Ping - - - Trex - - - vPE,vCGNAT, vFirewall etc - ipv4 throughput, latency etc - - - Traffic generators like Trex, ab/nginx, ixia, iperf etc - - - KPIs for a given use case: - - - System agent support for collecting NFVi KPI. This includes: - - - CPU statistic - - - Memory BW - - - OVS-DPDK Stats - - - Network KPIs, e.g., inpackets, outpackets, thoughput, latency etc - - - VNF KPIs, e.g., packet_in, packet_drop, packet_fwd etc - -Architecture ------------- - -The Network Service (NS) defines a set of Virtual Network Functions (VNF) -connected together using NFV infrastructure. - -The Yardstick NSB extension can support multiple VNFs created by different -vendors including traffic generators. Every VNF being tested has its -own data model. The Network service defines a VNF modelling on base of performed -network functionality. The part of the data model is a set of the configuration -parameters, number of connection points used and flavor including core and -memory amount. - -The ETSI defines a Network Service as a set of configurable VNFs working in -some NFV Infrastructure connecting each other using Virtual Links available -through Connection Points. The ETSI MANO specification defines a set of -management entities called Network Service Descriptors (NSD) and -VNF Descriptors (VNFD) that define real Network Service. The picture below -makes an example how the real Network Operator use-case can map into ETSI -Network service definition - -Network Service framework performs the necessary test steps. It may involve - - - Interacting with traffic generator and providing the inputs on traffic - type / packet structure to generate the required traffic as per the - test case. Traffic profiles will be used for this. - - - Executing the commands required for the test procedure and analyses the - command output for confirming whether the command got executed correctly - or not. E.g. As per the test case, run the traffic for the given - time period / wait for the necessary time delay - - - Verify the test result. - - - Validate the traffic flow from SUT - - - Fetch the table / data from SUT and verify the value as per the test case - - - Upload the logs from SUT onto the Test Harness server - - - Read the KPI's provided by particular VNF - -Components of Network Service -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - - * *Models for Network Service benchmarking*: The Network Service benchmarking - requires the proper modelling approach. The NSB provides models using Python - files and defining of NSDs and VNFDs. - - The benchmark control application being a part of OPNFV yardstick can call - that python models to instantiate and configure the VNFs. Depending on - infrastructure type (bare-metal or fully virtualized) that calls could be - made directly or using MANO system. - - * *Traffic generators in NSB*: Any benchmark application requires a set of - traffic generator and traffic profiles defining the method in which traffic - is generated. - - The Network Service benchmarking model extends the Network Service - definition with a set of Traffic Generators (TG) that are treated - same way as other VNFs being a part of benchmarked network service. - Same as other VNFs the traffic generator are instantiated and terminated. - - Every traffic generator has own configuration defined as a traffic profile and - a set of KPIs supported. The python models for TG is extended by specific calls - to listen and generate traffic. - - * *The stateless TREX traffic generator*: The main traffic generator used as - Network Service stimulus is open source TREX tool. - - The TREX tool can generate any kind of stateless traffic. - - .. code-block:: console - - +--------+ +-------+ +--------+ - | | | | | | - | Trex | ---> | VNF | ---> | Trex | - | | | | | | - +--------+ +-------+ +--------+ - - Supported testcases scenarios: - - - Correlated UDP traffic using TREX traffic generator and replay VNF. - - - using different IMIX configuration like pure voice, pure video traffic etc - - - using different number IP flows like 1 flow, 1K, 16K, 64K, 256K, 1M flows - - - Using different number of rules configured like 1 rule, 1K, 10K rules - - For UDP correlated traffic following Key Performance Indicators are collected - for every combination of test case parameters: - - - RFC2544 throughput for various loss rate defined (1% is a default) - -Graphical Overview ------------------- - -NSB Testing with yardstick framework facilitate performance testing of various -VNFs provided. - -.. code-block:: console - - +-----------+ - | | +-----------+ - | vPE | ->|TGen Port 0| - | TestCase | | +-----------+ - | | | - +-----------+ +------------------+ +-------+ | - | | -- API --> | VNF | <---> - +-----------+ | Yardstick | +-------+ | - | Test Case | --> | NSB Testing | | - +-----------+ | | | - | | | | - | +------------------+ | - +-----------+ | +-----------+ - | Traffic | ->|TGen Port 1| - | patterns | +-----------+ - +-----------+ - - Figure 1: Network Service - 2 server configuration - -VNFs supported for chracterization: -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -1. CGNAPT - Carrier Grade Network Address and port Translation -2. vFW - Virtual Firewall -3. vACL - Access Control List -4. Prox - Packet pROcessing eXecution engine: - - VNF can act as Drop, Basic Forwarding (no touch), - L2 Forwarding (change MAC), GRE encap/decap, Load balance based on - packet fields, Symmetric load balancing - - QinQ encap/decap IPv4/IPv6, ARP, QoS, Routing, Unmpls, Policing, ACL -5. UDP_Replay diff --git a/docs/testing/user/userguide/11-vtc-overview.rst b/docs/testing/user/userguide/11-vtc-overview.rst new file mode 100644 index 000000000..47582358c --- /dev/null +++ b/docs/testing/user/userguide/11-vtc-overview.rst @@ -0,0 +1,128 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, National Center of Scientific Research "Demokritos" and others. + +========================== +Virtual Traffic Classifier +========================== + +Abstract +======== + +.. _TNOVA: http://www.t-nova.eu/ +.. _TNOVAresults: http://www.t-nova.eu/results/ +.. _Yardstick: https://wiki.opnfv.org/yardstick + +This chapter provides an overview of the virtual Traffic Classifier, a +contribution to OPNFV Yardstick_ from the EU Project TNOVA_. +Additional documentation is available in TNOVAresults_. + +Overview +======== + +The virtual Traffic Classifier (:term:`VTC`) :term:`VNF`, comprises of a +Virtual Network Function Component (:term:`VNFC`). The :term:`VNFC` contains +both the Traffic Inspection module, and the Traffic forwarding module, needed +to run the :term:`VNF`. The exploitation of Deep Packet Inspection +(:term:`DPI`) methods for traffic classification is built around two basic +assumptions: + +* third parties unaffiliated with either source or recipient are able to + inspect each IP packet's payload + +* the classifier knows the relevant syntax of each application's packet + payloads (protocol signatures, data patterns, etc.). + +The proposed :term:`DPI` based approach will only use an indicative, small +number of the initial packets from each flow in order to identify the content +and not inspect each packet. + +In this respect it follows the Packet Based per Flow State (term:`PBFS`). This +method uses a table to track each session based on the 5-tuples (src address, +dest address, src port,dest port, transport protocol) that is maintained for +each flow. + +Concepts +======== + +* *Traffic Inspection*: The process of packet analysis and application + identification of network traffic that passes through the :term:`VTC`. + +* *Traffic Forwarding*: The process of packet forwarding from an incoming + network interface to a pre-defined outgoing network interface. + +* *Traffic Rule Application*: The process of packet tagging, based on a + predefined set of rules. Packet tagging may include e.g. Type of Service + (:term:`ToS`) field modification. + +Architecture +============ + +The Traffic Inspection module is the most computationally intensive component +of the :term:`VNF`. It implements filtering and packet matching algorithms in +order to support the enhanced traffic forwarding capability of the :term:`VNF`. +The component supports a flow table (exploiting hashing algorithms for fast +indexing of flows) and an inspection engine for traffic classification. + +The implementation used for these experiments exploits the nDPI library. +The packet capturing mechanism is implemented using libpcap. When the +:term:`DPI` engine identifies a new flow, the flow register is updated with the +appropriate information and transmitted across the Traffic Forwarding module, +which then applies any required policy updates. + +The Traffic Forwarding moudle is responsible for routing and packet forwarding. +It accepts incoming network traffic, consults the flow table for classification +information for each incoming flow and then applies pre-defined policies +marking e.g. :term:`ToS`/Differentiated Services Code Point (:term:`DSCP`) +multimedia traffic for Quality of Service (:term:`QoS`) enablement on the +forwarded traffic. +It is assumed that the traffic is forwarded using the default policy until it +is identified and new policies are enforced. + +The expected response delay is considered to be negligible, as only a small +number of packets are required to identify each flow. + +Graphical Overview +================== + +.. code-block:: console + + +----------------------------+ + | | + | Virtual Traffic Classifier | + | | + | Analysing/Forwarding | + | ------------> | + | ethA ethB | + | | + +----------------------------+ + | ^ + | | + v | + +----------------------------+ + | | + | Virtual Switch | + | | + +----------------------------+ + +Install +======= + +run the vTC/build.sh with root privileges + +Run +=== + +:: + + sudo ./pfbridge -a eth1 -b eth2 + + +.. note:: Virtual Traffic Classifier is not support in OPNFV Danube release. + + +Development Environment +======================= + +Ubuntu 14.04 Ubuntu 16.04 diff --git a/docs/testing/user/userguide/12-nsb-overview.rst b/docs/testing/user/userguide/12-nsb-overview.rst new file mode 100644 index 000000000..71a5c1130 --- /dev/null +++ b/docs/testing/user/userguide/12-nsb-overview.rst @@ -0,0 +1,206 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2016-2017 Intel Corporation. + +=================================== +Network Services Benchmarking (NSB) +=================================== + +Abstract +======== + +.. _Yardstick: https://wiki.opnfv.org/yardstick + +This chapter provides an overview of the NSB, a contribution to OPNFV +Yardstick_ from Intel. + +Overview +======== + +The goal of NSB is to Extend Yardstick to perform real world VNFs and NFVi +Characterization and benchmarking with repeatable and deterministic methods. + +The Network Service Benchmarking (NSB) extends the yardstick framework to do +VNF characterization and benchmarking in three different execution +environments - bare metal i.e. native Linux environment, standalone virtual +environment and managed virtualized environment (e.g. Open stack etc.). +It also brings in the capability to interact with external traffic generators +both hardware & software based for triggering and validating the traffic +according to user defined profiles. + +NSB extension includes: + + - Generic data models of Network Services, based on ETSI spec `ETSI GS NFV-TST 001 `_ + + - New Standalone context for VNF testing like SRIOV, OVS, OVS-DPDK etc + + - Generic VNF configuration models and metrics implemented with Python + classes + + - Traffic generator features and traffic profiles + + - L1-L3 state-less traffic profiles + + - L4-L7 state-full traffic profiles + + - Tunneling protocol / network overlay support + + - Test case samples + + - Ping + + - Trex + + - vPE,vCGNAT, vFirewall etc - ipv4 throughput, latency etc + + - Traffic generators like Trex, ab/nginx, ixia, iperf etc + + - KPIs for a given use case: + + - System agent support for collecting NFVi KPI. This includes: + + - CPU statistic + + - Memory BW + + - OVS-DPDK Stats + + - Network KPIs, e.g., inpackets, outpackets, thoughput, latency etc + + - VNF KPIs, e.g., packet_in, packet_drop, packet_fwd etc + +Architecture +============ + +The Network Service (NS) defines a set of Virtual Network Functions (VNF) +connected together using NFV infrastructure. + +The Yardstick NSB extension can support multiple VNFs created by different +vendors including traffic generators. Every VNF being tested has its +own data model. The Network service defines a VNF modelling on base of +performed network functionality. The part of the data model is a set of the +configuration parameters, number of connection points used and flavor including +core and memory amount. + +The ETSI defines a Network Service as a set of configurable VNFs working in +some NFV Infrastructure connecting each other using Virtual Links available +through Connection Points. The ETSI MANO specification defines a set of +management entities called Network Service Descriptors (NSD) and +VNF Descriptors (VNFD) that define real Network Service. The picture below +makes an example how the real Network Operator use-case can map into ETSI +Network service definition + +Network Service framework performs the necessary test steps. It may involve + + - Interacting with traffic generator and providing the inputs on traffic + type / packet structure to generate the required traffic as per the + test case. Traffic profiles will be used for this. + + - Executing the commands required for the test procedure and analyses the + command output for confirming whether the command got executed correctly + or not. E.g. As per the test case, run the traffic for the given + time period / wait for the necessary time delay + + - Verify the test result. + + - Validate the traffic flow from SUT + + - Fetch the table / data from SUT and verify the value as per the test case + + - Upload the logs from SUT onto the Test Harness server + + - Read the KPI's provided by particular VNF + +Components of Network Service +----------------------------- + + * *Models for Network Service benchmarking*: The Network Service benchmarking + requires the proper modelling approach. The NSB provides models using Python + files and defining of NSDs and VNFDs. + + The benchmark control application being a part of OPNFV yardstick can call + that python models to instantiate and configure the VNFs. Depending on + infrastructure type (bare-metal or fully virtualized) that calls could be + made directly or using MANO system. + + * *Traffic generators in NSB*: Any benchmark application requires a set of + traffic generator and traffic profiles defining the method in which traffic + is generated. + + The Network Service benchmarking model extends the Network Service + definition with a set of Traffic Generators (TG) that are treated + same way as other VNFs being a part of benchmarked network service. + Same as other VNFs the traffic generator are instantiated and terminated. + + Every traffic generator has own configuration defined as a traffic profile + and a set of KPIs supported. The python models for TG is extended by + specific calls to listen and generate traffic. + + * *The stateless TREX traffic generator*: The main traffic generator used as + Network Service stimulus is open source TREX tool. + + The TREX tool can generate any kind of stateless traffic. + + .. code-block:: console + + +--------+ +-------+ +--------+ + | | | | | | + | Trex | ---> | VNF | ---> | Trex | + | | | | | | + +--------+ +-------+ +--------+ + + Supported testcases scenarios: + + - Correlated UDP traffic using TREX traffic generator and replay VNF. + + - using different IMIX configuration like pure voice, pure video traffic etc + + - using different number IP flows like 1 flow, 1K, 16K, 64K, 256K, 1M flows + + - Using different number of rules configured like 1 rule, 1K, 10K rules + + For UDP correlated traffic following Key Performance Indicators are collected + for every combination of test case parameters: + + - RFC2544 throughput for various loss rate defined (1% is a default) + +Graphical Overview +================== + +NSB Testing with yardstick framework facilitate performance testing of various +VNFs provided. + +.. code-block:: console + + +-----------+ + | | +-----------+ + | vPE | ->|TGen Port 0| + | TestCase | | +-----------+ + | | | + +-----------+ +------------------+ +-------+ | + | | -- API --> | VNF | <---> + +-----------+ | Yardstick | +-------+ | + | Test Case | --> | NSB Testing | | + +-----------+ | | | + | | | | + | +------------------+ | + +-----------+ | +-----------+ + | Traffic | ->|TGen Port 1| + | patterns | +-----------+ + +-----------+ + + Figure 1: Network Service - 2 server configuration + +VNFs supported for chracterization: +----------------------------------- + +1. CGNAPT - Carrier Grade Network Address and port Translation +2. vFW - Virtual Firewall +3. vACL - Access Control List +4. Prox - Packet pROcessing eXecution engine: + - VNF can act as Drop, Basic Forwarding (no touch), + L2 Forwarding (change MAC), GRE encap/decap, Load balance based on + packet fields, Symmetric load balancing + - QinQ encap/decap IPv4/IPv6, ARP, QoS, Routing, Unmpls, Policing, ACL +5. UDP_Replay diff --git a/docs/testing/user/userguide/12-nsb_installation.rst b/docs/testing/user/userguide/12-nsb_installation.rst deleted file mode 100644 index 5631c6578..000000000 --- a/docs/testing/user/userguide/12-nsb_installation.rst +++ /dev/null @@ -1,1170 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, 2016-2017 Intel Corporation. - -Yardstick - NSB Testing -Installation -===================================== - -Abstract --------- - -The Network Service Benchmarking (NSB) extends the yardstick framework to do -VNF characterization and benchmarking in three different execution -environments viz., bare metal i.e. native Linux environment, standalone virtual -environment and managed virtualized environment (e.g. Open stack etc.). -It also brings in the capability to interact with external traffic generators -both hardware & software based for triggering and validating the traffic -according to user defined profiles. - -The steps needed to run Yardstick with NSB testing are: - -* Install Yardstick (NSB Testing). -* Setup/Reference pod.yaml describing Test topology -* Create/Reference the test configuration yaml file. -* Run the test case. - - -Prerequisites -------------- - -Refer chapter Yardstick Installation for more information on yardstick -prerequisites - -Several prerequisites are needed for Yardstick(VNF testing): - - - Python Modules: pyzmq, pika. - - - flex - - - bison - - - build-essential - - - automake - - - libtool - - - librabbitmq-dev - - - rabbitmq-server - - - collectd - - - intel-cmt-cat - -Hardware & Software Ingredients -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -SUT requirements: - - - +-----------+--------------------+ - | Item | Description | - +-----------+--------------------+ - | Memory | Min 20GB | - +-----------+--------------------+ - | NICs | 2 x 10G | - +-----------+--------------------+ - | OS | Ubuntu 16.04.3 LTS | - +-----------+--------------------+ - | kernel | 4.4.0-34-generic | - +-----------+--------------------+ - | DPDK | 17.02 | - +-----------+--------------------+ - -Boot and BIOS settings: - - - +------------------+---------------------------------------------------+ - | Boot settings | default_hugepagesz=1G hugepagesz=1G hugepages=16 | - | | hugepagesz=2M hugepages=2048 isolcpus=1-11,22-33 | - | | nohz_full=1-11,22-33 rcu_nocbs=1-11,22-33 | - | | iommu=on iommu=pt intel_iommu=on | - | | Note: nohz_full and rcu_nocbs is to disable Linux | - | | kernel interrupts | - +------------------+---------------------------------------------------+ - |BIOS | CPU Power and Performance Policy | - | | CPU C-state Disabled | - | | CPU P-state Disabled | - | | Enhanced Intel® Speedstep® Tech Disabled | - | | Hyper-Threading Technology (If supported) Enabled | - | | Virtualization Techology Enabled | - | | Intel(R) VT for Direct I/O Enabled | - | | Coherency Enabled | - | | Turbo Boost Disabled | - +------------------+---------------------------------------------------+ - - - -Install Yardstick (NSB Testing) -------------------------------- - -Download the source code and install Yardstick from it - -.. code-block:: console - - git clone https://gerrit.opnfv.org/gerrit/yardstick - - cd yardstick - - # Switch to latest stable branch - # git checkout - git checkout stable/euphrates - -Configure the network proxy, either using the environment variables or setting -the global environment file: - -.. code-block:: ini - - cat /etc/environment - http_proxy='http://proxy.company.com:port' - https_proxy='http://proxy.company.com:port' - -.. code-block:: console - - export http_proxy='http://proxy.company.com:port' - export https_proxy='http://proxy.company.com:port' - -The last step is to modify the Yardstick installation inventory, used by -Ansible: - -.. code-block:: ini - - cat ./ansible/yardstick-install-inventory.ini - [jumphost] - localhost ansible_connection=local - - [yardstick-standalone] - yardstick-standalone-node ansible_host=192.168.1.2 - yardstick-standalone-node-2 ansible_host=192.168.1.3 - - # section below is only due backward compatibility. - # it will be removed later - [yardstick:children] - jumphost - - [all:vars] - ansible_user=root - ansible_pass=root - - -To execute an installation for a Bare-Metal or a Standalone context: - -.. code-block:: console - - ./nsb_setup.sh - - -To execute an installation for an OpenStack context: - -.. code-block:: console - - ./nsb_setup.sh - -Above command setup docker with latest yardstick code. To execute - -.. code-block:: console - - docker exec -it yardstick bash - -It will also automatically download all the packages needed for NSB Testing setup. -Refer chapter :doc:`04-installation` for more on docker **Install Yardstick using Docker (recommended)** - -System Topology: ----------------- - -.. code-block:: console - - +----------+ +----------+ - | | | | - | | (0)----->(0) | | - | TG1 | | DUT | - | | | | - | | (1)<-----(1) | | - +----------+ +----------+ - trafficgen_1 vnf - - -Environment parameters and credentials --------------------------------------- - -Config yardstick conf -^^^^^^^^^^^^^^^^^^^^^ - -If user did not run 'yardstick env influxdb' inside the container, which will generate -correct yardstick.conf, then create the config file manually (run inside the container): - - cp ./etc/yardstick/yardstick.conf.sample /etc/yardstick/yardstick.conf - vi /etc/yardstick/yardstick.conf - -Add trex_path, trex_client_lib and bin_path in 'nsb' section. - -:: - - [DEFAULT] - debug = True - dispatcher = file, influxdb - - [dispatcher_influxdb] - timeout = 5 - target = http://{YOUR_IP_HERE}:8086 - db_name = yardstick - username = root - password = root - - [nsb] - trex_path=/opt/nsb_bin/trex/scripts - bin_path=/opt/nsb_bin - trex_client_lib=/opt/nsb_bin/trex_client/stl - -Run Yardstick - Network Service Testcases ------------------------------------------ - - -NS testing - using yardstick CLI -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - - See :doc:`04-installation` - -.. code-block:: console - - - docker exec -it yardstick /bin/bash - source /etc/yardstick/openstack.creds (only for heat TC if nsb_setup.sh was NOT used) - export EXTERNAL_NETWORK="" (only for heat TC) - yardstick --debug task start yardstick/samples/vnf_samples/nsut// - -Network Service Benchmarking - Bare-Metal ------------------------------------------ - -Bare-Metal Config pod.yaml describing Topology -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Bare-Metal 2-Node setup: -######################## -.. code-block:: console - - +----------+ +----------+ - | | | | - | | (0)----->(0) | | - | TG1 | | DUT | - | | | | - | | (n)<-----(n) | | - +----------+ +----------+ - trafficgen_1 vnf - -Bare-Metal 3-Node setup - Correlated Traffic: -############################################# -.. code-block:: console - - +----------+ +----------+ +------------+ - | | | | | | - | | | | | | - | | (0)----->(0) | | | UDP | - | TG1 | | DUT | | Replay | - | | | | | | - | | | |(1)<---->(0)| | - +----------+ +----------+ +------------+ - trafficgen_1 vnf trafficgen_2 - - -Bare-Metal Config pod.yaml -^^^^^^^^^^^^^^^^^^^^^^^^^^ -Before executing Yardstick test cases, make sure that pod.yaml reflects the -topology and update all the required fields.:: - - cp /etc/yardstick/nodes/pod.yaml.nsb.sample /etc/yardstick/nodes/pod.yaml - -.. code-block:: YAML - - nodes: - - - name: trafficgen_1 - role: TrafficGen - ip: 1.1.1.1 - user: root - password: r00t - interfaces: - xe0: # logical name from topology.yaml and vnfd.yaml - vpci: "0000:07:00.0" - driver: i40e # default kernel driver - dpdk_port_num: 0 - local_ip: "152.16.100.20" - netmask: "255.255.255.0" - local_mac: "00:00:00:00:00:01" - xe1: # logical name from topology.yaml and vnfd.yaml - vpci: "0000:07:00.1" - driver: i40e # default kernel driver - dpdk_port_num: 1 - local_ip: "152.16.40.20" - netmask: "255.255.255.0" - local_mac: "00:00.00:00:00:02" - - - - name: vnf - role: vnf - ip: 1.1.1.2 - user: root - password: r00t - host: 1.1.1.2 #BM - host == ip, virtualized env - Host - compute node - interfaces: - xe0: # logical name from topology.yaml and vnfd.yaml - vpci: "0000:07:00.0" - driver: i40e # default kernel driver - dpdk_port_num: 0 - local_ip: "152.16.100.19" - netmask: "255.255.255.0" - local_mac: "00:00:00:00:00:03" - - xe1: # logical name from topology.yaml and vnfd.yaml - vpci: "0000:07:00.1" - driver: i40e # default kernel driver - dpdk_port_num: 1 - local_ip: "152.16.40.19" - netmask: "255.255.255.0" - local_mac: "00:00:00:00:00:04" - routing_table: - - network: "152.16.100.20" - netmask: "255.255.255.0" - gateway: "152.16.100.20" - if: "xe0" - - network: "152.16.40.20" - netmask: "255.255.255.0" - gateway: "152.16.40.20" - if: "xe1" - nd_route_tbl: - - network: "0064:ff9b:0:0:0:0:9810:6414" - netmask: "112" - gateway: "0064:ff9b:0:0:0:0:9810:6414" - if: "xe0" - - network: "0064:ff9b:0:0:0:0:9810:2814" - netmask: "112" - gateway: "0064:ff9b:0:0:0:0:9810:2814" - if: "xe1" - - -Network Service Benchmarking - Standalone Virtualization --------------------------------------------------------- - -SR-IOV: -^^^^^^^ - -SR-IOV Pre-requisites -##################### - -On Host: - a) Create a bridge for VM to connect to external network - - .. code-block:: console - - brctl addbr br-int - brctl addif br-int #This interface is connected to internet - - b) Build guest image for VNF to run. - Most of the sample test cases in Yardstick are using a guest image called - ``yardstick-image`` which deviates from an Ubuntu Cloud Server image - Yardstick has a tool for building this custom image with samplevnf. - It is necessary to have ``sudo`` rights to use this tool. - - Also you may need to install several additional packages to use this tool, by - following the commands below:: - - sudo apt-get update && sudo apt-get install -y qemu-utils kpartx - - This image can be built using the following command in the directory where Yardstick is installed - - .. code-block:: console - - export YARD_IMG_ARCH='amd64' - sudo echo "Defaults env_keep += \'YARD_IMG_ARCH\'" >> /etc/sudoers - - Please use ansible script to generate a cloud image refer to :doc:`04-installation` - - for more details refer to chapter :doc:`04-installation` - - .. note:: VM should be build with static IP and should be accessible from yardstick host. - - -SR-IOV Config pod.yaml describing Topology -########################################## - -SR-IOV 2-Node setup: -#################### -.. code-block:: console - - +--------------------+ - | | - | | - | DUT | - | (VNF) | - | | - +--------------------+ - | VF NIC | | VF NIC | - +--------+ +--------+ - ^ ^ - | | - | | - +----------+ +-------------------------+ - | | | ^ ^ | - | | | | | | - | | (0)<----->(0) | ------ | | - | TG1 | | SUT | | - | | | | | - | | (n)<----->(n) |------------------ | - +----------+ +-------------------------+ - trafficgen_1 host - - - -SR-IOV 3-Node setup - Correlated Traffic -######################################## -.. code-block:: console - - +--------------------+ - | | - | | - | DUT | - | (VNF) | - | | - +--------------------+ - | VF NIC | | VF NIC | - +--------+ +--------+ - ^ ^ - | | - | | - +----------+ +-------------------------+ +--------------+ - | | | ^ ^ | | | - | | | | | | | | - | | (0)<----->(0) | ------ | | | TG2 | - | TG1 | | SUT | | | (UDP Replay) | - | | | | | | | - | | (n)<----->(n) | ------ | (n)<-->(n) | | - +----------+ +-------------------------+ +--------------+ - trafficgen_1 host trafficgen_2 - -Before executing Yardstick test cases, make sure that pod.yaml reflects the -topology and update all the required fields. - -.. code-block:: console - - cp /etc/yardstick/nodes/standalone/trex_bm.yaml.sample /etc/yardstick/nodes/standalone/pod_trex.yaml - cp /etc/yardstick/nodes/standalone/host_sriov.yaml /etc/yardstick/nodes/standalone/host_sriov.yaml - -.. note:: Update all the required fields like ip, user, password, pcis, etc... - -SR-IOV Config pod_trex.yaml -########################### - -.. code-block:: YAML - - nodes: - - - name: trafficgen_1 - role: TrafficGen - ip: 1.1.1.1 - user: root - password: r00t - key_filename: /root/.ssh/id_rsa - interfaces: - xe0: # logical name from topology.yaml and vnfd.yaml - vpci: "0000:07:00.0" - driver: i40e # default kernel driver - dpdk_port_num: 0 - local_ip: "152.16.100.20" - netmask: "255.255.255.0" - local_mac: "00:00:00:00:00:01" - xe1: # logical name from topology.yaml and vnfd.yaml - vpci: "0000:07:00.1" - driver: i40e # default kernel driver - dpdk_port_num: 1 - local_ip: "152.16.40.20" - netmask: "255.255.255.0" - local_mac: "00:00.00:00:00:02" - -SR-IOV Config host_sriov.yaml -############################# - -.. code-block:: YAML - - nodes: - - - name: sriov - role: Sriov - ip: 192.168.100.101 - user: "" - password: "" - -SR-IOV testcase update: ``/samples/vnf_samples/nsut/vfw/tc_sriov_rfc2544_ipv4_1rule_1flow_64B_trex.yaml`` - -Update "contexts" section -""""""""""""""""""""""""" - -.. code-block:: YAML - - contexts: - - name: yardstick - type: Node - file: /etc/yardstick/nodes/standalone/pod_trex.yaml - - type: StandaloneSriov - file: /etc/yardstick/nodes/standalone/host_sriov.yaml - name: yardstick - vm_deploy: True - flavor: - images: "/var/lib/libvirt/images/ubuntu.qcow2" - ram: 4096 - extra_specs: - hw:cpu_sockets: 1 - hw:cpu_cores: 6 - hw:cpu_threads: 2 - user: "" # update VM username - password: "" # update password - servers: - vnf: - network_ports: - mgmt: - cidr: '1.1.1.61/24' # Update VM IP address, if static, / or if dynamic, / - xe0: - - uplink_0 - xe1: - - downlink_0 - networks: - uplink_0: - phy_port: "0000:05:00.0" - vpci: "0000:00:07.0" - cidr: '152.16.100.10/24' - gateway_ip: '152.16.100.20' - downlink_0: - phy_port: "0000:05:00.1" - vpci: "0000:00:08.0" - cidr: '152.16.40.10/24' - gateway_ip: '152.16.100.20' - - - -OVS-DPDK: -^^^^^^^^^ - -OVS-DPDK Pre-requisites -####################### - -On Host: - a) Create a bridge for VM to connect to external network - - .. code-block:: console - - brctl addbr br-int - brctl addif br-int #This interface is connected to internet - - b) Build guest image for VNF to run. - Most of the sample test cases in Yardstick are using a guest image called - ``yardstick-image`` which deviates from an Ubuntu Cloud Server image - Yardstick has a tool for building this custom image with samplevnf. - It is necessary to have ``sudo`` rights to use this tool. - - Also you may need to install several additional packages to use this tool, by - following the commands below:: - - sudo apt-get update && sudo apt-get install -y qemu-utils kpartx - - This image can be built using the following command in the directory where Yardstick is installed:: - - export YARD_IMG_ARCH='amd64' - sudo echo "Defaults env_keep += \'YARD_IMG_ARCH\'" >> /etc/sudoers - sudo tools/yardstick-img-dpdk-modify tools/ubuntu-server-cloudimg-samplevnf-modify.sh - - for more details refer to chapter :doc:`04-installation` - - .. note:: VM should be build with static IP and should be accessible from yardstick host. - - c) OVS & DPDK version. - - OVS 2.7 and DPDK 16.11.1 above version is supported - - d) Setup OVS/DPDK on host. - Please refer to below link on how to setup `OVS-DPDK `_ - - -OVS-DPDK Config pod.yaml describing Topology -############################################ - -OVS-DPDK 2-Node setup: -###################### - - -.. code-block:: console - - +--------------------+ - | | - | | - | DUT | - | (VNF) | - | | - +--------------------+ - | virtio | | virtio | - +--------+ +--------+ - ^ ^ - | | - | | - +--------+ +--------+ - | vHOST0 | | vHOST1 | - +----------+ +-------------------------+ - | | | ^ ^ | - | | | | | | - | | (0)<----->(0) | ------ | | - | TG1 | | SUT | | - | | | (ovs-dpdk) | | - | | (n)<----->(n) |------------------ | - +----------+ +-------------------------+ - trafficgen_1 host - - -OVS-DPDK 3-Node setup - Correlated Traffic -########################################## - -.. code-block:: console - - +--------------------+ - | | - | | - | DUT | - | (VNF) | - | | - +--------------------+ - | virtio | | virtio | - +--------+ +--------+ - ^ ^ - | | - | | - +--------+ +--------+ - | vHOST0 | | vHOST1 | - +----------+ +-------------------------+ +------------+ - | | | ^ ^ | | | - | | | | | | | | - | | (0)<----->(0) | ------ | | | TG2 | - | TG1 | | SUT | | |(UDP Replay)| - | | | (ovs-dpdk) | | | | - | | (n)<----->(n) | ------ |(n)<-->(n)| | - +----------+ +-------------------------+ +------------+ - trafficgen_1 host trafficgen_2 - - -Before executing Yardstick test cases, make sure that pod.yaml reflects the -topology and update all the required fields. - -.. code-block:: console - - cp /etc/yardstick/nodes/standalone/trex_bm.yaml.sample /etc/yardstick/nodes/standalone/pod_trex.yaml - cp /etc/yardstick/nodes/standalone/host_ovs.yaml /etc/yardstick/nodes/standalone/host_ovs.yaml - -.. note:: Update all the required fields like ip, user, password, pcis, etc... - -OVS-DPDK Config pod_trex.yaml -############################# - -.. code-block:: YAML - - nodes: - - - name: trafficgen_1 - role: TrafficGen - ip: 1.1.1.1 - user: root - password: r00t - interfaces: - xe0: # logical name from topology.yaml and vnfd.yaml - vpci: "0000:07:00.0" - driver: i40e # default kernel driver - dpdk_port_num: 0 - local_ip: "152.16.100.20" - netmask: "255.255.255.0" - local_mac: "00:00:00:00:00:01" - xe1: # logical name from topology.yaml and vnfd.yaml - vpci: "0000:07:00.1" - driver: i40e # default kernel driver - dpdk_port_num: 1 - local_ip: "152.16.40.20" - netmask: "255.255.255.0" - local_mac: "00:00.00:00:00:02" - -OVS-DPDK Config host_ovs.yaml -############################# - -.. code-block:: YAML - - nodes: - - - name: ovs_dpdk - role: OvsDpdk - ip: 192.168.100.101 - user: "" - password: "" - -ovs_dpdk testcase update: ``/samples/vnf_samples/nsut/vfw/tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml`` - -Update "contexts" section -""""""""""""""""""""""""" - -.. code-block:: YAML - - contexts: - - name: yardstick - type: Node - file: /etc/yardstick/nodes/standalone/pod_trex.yaml - - type: StandaloneOvsDpdk - name: yardstick - file: /etc/yardstick/nodes/standalone/pod_ovs.yaml - vm_deploy: True - ovs_properties: - version: - ovs: 2.7.0 - dpdk: 16.11.1 - pmd_threads: 2 - ram: - socket_0: 2048 - socket_1: 2048 - queues: 4 - vpath: "/usr/local" - - flavor: - images: "/var/lib/libvirt/images/ubuntu.qcow2" - ram: 4096 - extra_specs: - hw:cpu_sockets: 1 - hw:cpu_cores: 6 - hw:cpu_threads: 2 - user: "" # update VM username - password: "" # update password - servers: - vnf: - network_ports: - mgmt: - cidr: '1.1.1.61/24' # Update VM IP address, if static, / or if dynamic, / - xe0: - - uplink_0 - xe1: - - downlink_0 - networks: - uplink_0: - phy_port: "0000:05:00.0" - vpci: "0000:00:07.0" - cidr: '152.16.100.10/24' - gateway_ip: '152.16.100.20' - downlink_0: - phy_port: "0000:05:00.1" - vpci: "0000:00:08.0" - cidr: '152.16.40.10/24' - gateway_ip: '152.16.100.20' - - -Network Service Benchmarking - OpenStack with SR-IOV support ------------------------------------------------------------- - -This section describes how to run a Sample VNF test case, using Heat context, -with SR-IOV. It also covers how to install OpenStack in Ubuntu 16.04, using -DevStack, with SR-IOV support. - - -Single node OpenStack setup with external TG -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. code-block:: console - - +----------------------------+ - |OpenStack(DevStack) | - | | - | +--------------------+ | - | |sample-VNF VM | | - | | | | - | | DUT | | - | | (VNF) | | - | | | | - | +--------+ +--------+ | - | | VF NIC | | VF NIC | | - | +-----+--+--+----+---+ | - | ^ ^ | - | | | | - +----------+ +---------+----------+-------+ - | | | VF0 VF1 | - | | | ^ ^ | - | | | | SUT | | - | TG | (PF0)<----->(PF0) +---------+ | | - | | | | | - | | (PF1)<----->(PF1) +--------------------+ | - | | | | - +----------+ +----------------------------+ - trafficgen_1 host - - -Host pre-configuration -###################### - -.. warning:: The following configuration requires sudo access to the system. Make - sure that your user have the access. - -Enable the Intel VT-d or AMD-Vi extension in the BIOS. Some system manufacturers -disable this extension by default. - -Activate the Intel VT-d or AMD-Vi extension in the kernel by modifying the GRUB -config file ``/etc/default/grub``. - -For the Intel platform: - -.. code:: bash - - ... - GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on" - ... - -For the AMD platform: - -.. code:: bash - - ... - GRUB_CMDLINE_LINUX_DEFAULT="amd_iommu=on" - ... - -Update the grub configuration file and restart the system: - -.. warning:: The following command will reboot the system. - -.. code:: bash - - sudo update-grub - sudo reboot - -Make sure the extension has been enabled: - -.. code:: bash - - sudo journalctl -b 0 | grep -e IOMMU -e DMAR - - Feb 06 14:50:14 hostname kernel: ACPI: DMAR 0x000000006C406000 0001E0 (v01 INTEL S2600WF 00000001 INTL 20091013) - Feb 06 14:50:14 hostname kernel: DMAR: IOMMU enabled - Feb 06 14:50:14 hostname kernel: DMAR: Host address width 46 - Feb 06 14:50:14 hostname kernel: DMAR: DRHD base: 0x000000d37fc000 flags: 0x0 - Feb 06 14:50:14 hostname kernel: DMAR: dmar0: reg_base_addr d37fc000 ver 1:0 cap 8d2078c106f0466 ecap f020de - Feb 06 14:50:14 hostname kernel: DMAR: DRHD base: 0x000000e0ffc000 flags: 0x0 - Feb 06 14:50:14 hostname kernel: DMAR: dmar1: reg_base_addr e0ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020de - Feb 06 14:50:14 hostname kernel: DMAR: DRHD base: 0x000000ee7fc000 flags: 0x0 - -Setup system proxy (if needed). Add the following configuration into the -``/etc/environment`` file: - -.. note:: The proxy server name/port and IPs should be changed according to - actuall/current proxy configuration in the lab. - -.. code:: bash - - export http_proxy=http://proxy.company.com:port - export https_proxy=http://proxy.company.com:port - export ftp_proxy=http://proxy.company.com:port - export no_proxy=localhost,127.0.0.1,company.com,,,... - export NO_PROXY=localhost,127.0.0.1,company.com,,,... - -Upgrade the system: - -.. code:: bash - - sudo -EH apt-get update - sudo -EH apt-get upgrade - sudo -EH apt-get dist-upgrade - -Install dependencies needed for the DevStack - -.. code:: bash - - sudo -EH apt-get install python - sudo -EH apt-get install python-dev - sudo -EH apt-get install python-pip - -Setup SR-IOV ports on the host: - -.. note:: The ``enp24s0f0``, ``enp24s0f0`` are physical function (PF) interfaces - on a host and ``enp24s0f3`` is a public interface used in OpenStack, so the - interface names should be changed according to the HW environment used for - testing. - -.. code:: bash - - sudo ip link set dev enp24s0f0 up - sudo ip link set dev enp24s0f1 up - sudo ip link set dev enp24s0f3 up - - # Create VFs on PF - echo 2 | sudo tee /sys/class/net/enp24s0f0/device/sriov_numvfs - echo 2 | sudo tee /sys/class/net/enp24s0f1/device/sriov_numvfs - - -DevStack installation -##################### - -Use official `Devstack `_ -documentation to install OpenStack on a host. Please note, that stable -``pike`` branch of devstack repo should be used during the installation. -The required `local.conf`` configuration file are described below. - -DevStack configuration file: - -.. note:: Update the devstack configuration file by replacing angluar brackets - with a short description inside. - -.. note:: Use ``lspci | grep Ether`` & ``lspci -n | grep `` - commands to get device and vendor id of the virtual function (VF). - -.. literalinclude:: code/single-devstack-local.conf - :language: console - -Start the devstack installation on a host. - - -TG host configuration -##################### - -Yardstick automatically install and configure Trex traffic generator on TG -host based on provided POD file (see below). Anyway, it's recommended to check -the compatibility of the installed NIC on the TG server with software Trex using -the manual at https://trex-tgn.cisco.com/trex/doc/trex_manual.html. - - -Run the Sample VNF test case -############################ - -There is an example of Sample VNF test case ready to be executed in an -OpenStack environment with SR-IOV support: ``samples/vnf_samples/nsut/vfw/ -tc_heat_sriov_external_rfc2544_ipv4_1rule_1flow_64B_trex.yaml``. - -Install yardstick using `Install Yardstick (NSB Testing)`_ steps for OpenStack -context. - -Create pod file for TG in the yardstick repo folder located in the yardstick -container: - -.. note:: The ``ip``, ``user``, ``password`` and ``vpci`` fields show be changed - according to HW environment used for the testing. Use ``lshw -c network -businfo`` - command to get the PF PCI address for ``vpci`` field. - -.. literalinclude:: code/single-yardstick-pod.conf - :language: console - -Run the Sample vFW RFC2544 SR-IOV TC (``samples/vnf_samples/nsut/vfw/ -tc_heat_sriov_external_rfc2544_ipv4_1rule_1flow_64B_trex.yaml``) in the heat -context using steps described in `NS testing - using yardstick CLI`_ section. - - -Multi node OpenStack TG and VNF setup (two nodes) -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. code-block:: console - - +----------------------------+ +----------------------------+ - |OpenStack(DevStack) | |OpenStack(DevStack) | - | | | | - | +--------------------+ | | +--------------------+ | - | |sample-VNF VM | | | |sample-VNF VM | | - | | | | | | | | - | | TG | | | | DUT | | - | | trafficgen_1 | | | | (VNF) | | - | | | | | | | | - | +--------+ +--------+ | | +--------+ +--------+ | - | | VF NIC | | VF NIC | | | | VF NIC | | VF NIC | | - | +----+---+--+----+---+ | | +-----+--+--+----+---+ | - | ^ ^ | | ^ ^ | - | | | | | | | | - +--------+-----------+-------+ +---------+----------+-------+ - | VF0 VF1 | | VF0 VF1 | - | ^ ^ | | ^ ^ | - | | SUT2 | | | | SUT1 | | - | | +-------+ (PF0)<----->(PF0) +---------+ | | - | | | | | | - | +-------------------+ (PF1)<----->(PF1) +--------------------+ | - | | | | - +----------------------------+ +----------------------------+ - host2 (compute) host1 (controller) - - -Controller/Compute pre-configuration -#################################### - -Pre-configuration of the controller and compute hosts are the same as -described in `Host pre-configuration`_ section. Follow the steps in the section. - - -DevStack configuration -###################### - -Use official `Devstack `_ -documentation to install OpenStack on a host. Please note, that stable -``pike`` branch of devstack repo should be used during the installation. -The required `local.conf`` configuration file are described below. - -.. note:: Update the devstack configuration files by replacing angluar brackets - with a short description inside. - -.. note:: Use ``lspci | grep Ether`` & ``lspci -n | grep `` - commands to get device and vendor id of the virtual function (VF). - -DevStack configuration file for controller host: - -.. literalinclude:: code/multi-devstack-controller-local.conf - :language: console - -DevStack configuration file for compute host: - -.. literalinclude:: code/multi-devstack-compute-local.conf - :language: console - -Start the devstack installation on the controller and compute hosts. - - -Run the sample vFW TC -##################### - -Install yardstick using `Install Yardstick (NSB Testing)`_ steps for OpenStack -context. - -Run sample vFW RFC2544 SR-IOV TC (``samples/vnf_samples/nsut/vfw/ -tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml``) in the heat -context using steps described in `NS testing - using yardstick CLI`_ section -and the following yardtick command line arguments: - -.. code:: bash - - yardstick -d task start --task-args='{"provider": "sriov"}' \ - samples/vnf_samples/nsut/vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml - - -Enabling other Traffic generator --------------------------------- - -IxLoad: -^^^^^^^ - -1. Software needed: IxLoadAPI ``Linux64.bin.tgz and Linux64.bin.tar.gz`` (Download from ixia support site) - Install - ``Linux64.bin.tgz & Linux64.bin.tar.gz`` - If the installation was not done inside the container, after installing the IXIA client, - check /opt/ixia/ixload//bin/ixloadpython and make sure you can run this cmd - inside the yardstick container. Usually user is required to copy or link /opt/ixia/python//bin/ixiapython - to /usr/bin/ixiapython inside the container. - -2. Update pod_ixia.yaml file with ixia details. - - .. code-block:: console - - cp /etc/yardstick/nodes/pod.yaml.nsb.sample.ixia etc/yardstick/nodes/pod_ixia.yaml - - Config pod_ixia.yaml - - .. code-block:: yaml - - - nodes: - - - name: trafficgen_1 - role: IxNet - ip: 1.2.1.1 #ixia machine ip - user: user - password: r00t - key_filename: /root/.ssh/id_rsa - tg_config: - ixchassis: "1.2.1.7" #ixia chassis ip - tcl_port: "8009" # tcl server port - lib_path: "/opt/ixia/ixos-api/8.01.0.2/lib/ixTcl1.0" - root_dir: "/opt/ixia/ixos-api/8.01.0.2/" - py_bin_path: "/opt/ixia/ixload/8.01.106.3/bin/" - py_lib_path: "/opt/ixia/ixnetwork/8.01.1029.14/lib/PythonApi" - dut_result_dir: "/mnt/ixia" - version: 8.1 - interfaces: - xe0: # logical name from topology.yaml and vnfd.yaml - vpci: "2:5" # Card:port - driver: "none" - dpdk_port_num: 0 - local_ip: "152.16.100.20" - netmask: "255.255.0.0" - local_mac: "00:98:10:64:14:00" - xe1: # logical name from topology.yaml and vnfd.yaml - vpci: "2:6" # [(Card, port)] - driver: "none" - dpdk_port_num: 1 - local_ip: "152.40.40.20" - netmask: "255.255.0.0" - local_mac: "00:98:28:28:14:00" - - for sriov/ovs_dpdk pod files, please refer to above Standalone Virtualization for ovs-dpdk/sriov configuration - -3. Start IxOS TCL Server (Install 'Ixia IxExplorer IxOS ') - You will also need to configure the IxLoad machine to start the IXIA - IxosTclServer. This can be started like so: - - - Connect to the IxLoad machine using RDP - - Go to: - ``Start->Programs->Ixia->IxOS->IxOS 8.01-GA-Patch1->Ixia Tcl Server IxOS 8.01-GA-Patch1`` - or - ``"C:\Program Files (x86)\Ixia\IxOS\8.01-GA-Patch1\ixTclServer.exe"`` - -4. Create a folder "Results" in c:\ and share the folder on the network. - -5. execute testcase in samplevnf folder. - eg ``/samples/vnf_samples/nsut/vfw/tc_baremetal_http_ixload_1b_Requests-65000_Concurrency.yaml`` - -IxNetwork: -^^^^^^^^^^ - -1. Software needed: ``IxNetworkAPILinux64.bin.tgz`` (Download from ixia support site) - Install - ``IxNetworkAPILinux64.bin.tgz`` -2. Update pod_ixia.yaml file with ixia details. - - .. code-block:: console - - cp /etc/yardstick/nodes/pod.yaml.nsb.sample.ixia etc/yardstick/nodes/pod_ixia.yaml - - Config pod_ixia.yaml - - .. code-block:: yaml - - nodes: - - - name: trafficgen_1 - role: IxNet - ip: 1.2.1.1 #ixia machine ip - user: user - password: r00t - key_filename: /root/.ssh/id_rsa - tg_config: - ixchassis: "1.2.1.7" #ixia chassis ip - tcl_port: "8009" # tcl server port - lib_path: "/opt/ixia/ixos-api/8.01.0.2/lib/ixTcl1.0" - root_dir: "/opt/ixia/ixos-api/8.01.0.2/" - py_bin_path: "/opt/ixia/ixload/8.01.106.3/bin/" - py_lib_path: "/opt/ixia/ixnetwork/8.01.1029.14/lib/PythonApi" - dut_result_dir: "/mnt/ixia" - version: 8.1 - interfaces: - xe0: # logical name from topology.yaml and vnfd.yaml - vpci: "2:5" # Card:port - driver: "none" - dpdk_port_num: 0 - local_ip: "152.16.100.20" - netmask: "255.255.0.0" - local_mac: "00:98:10:64:14:00" - xe1: # logical name from topology.yaml and vnfd.yaml - vpci: "2:6" # [(Card, port)] - driver: "none" - dpdk_port_num: 1 - local_ip: "152.40.40.20" - netmask: "255.255.0.0" - local_mac: "00:98:28:28:14:00" - - for sriov/ovs_dpdk pod files, please refer to above Standalone Virtualization for ovs-dpdk/sriov configuration - -3. Start IxNetwork TCL Server - You will also need to configure the IxNetwork machine to start the IXIA - IxNetworkTclServer. This can be started like so: - - - Connect to the IxNetwork machine using RDP - - Go to: ``Start->Programs->Ixia->IxNetwork->IxNetwork 7.21.893.14 GA->IxNetworkTclServer`` (or ``IxNetworkApiServer``) - -4. execute testcase in samplevnf folder. - eg ``/samples/vnf_samples/nsut/vfw/tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml`` - diff --git a/docs/testing/user/userguide/13-nsb-installation.rst b/docs/testing/user/userguide/13-nsb-installation.rst new file mode 100644 index 000000000..00f8cfd97 --- /dev/null +++ b/docs/testing/user/userguide/13-nsb-installation.rst @@ -0,0 +1,1166 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2016-2017 Intel Corporation. + +===================================== +Yardstick - NSB Testing -Installation +===================================== + +Abstract +======== + +The Network Service Benchmarking (NSB) extends the yardstick framework to do +VNF characterization and benchmarking in three different execution +environments viz., bare metal i.e. native Linux environment, standalone virtual +environment and managed virtualized environment (e.g. Open stack etc.). +It also brings in the capability to interact with external traffic generators +both hardware & software based for triggering and validating the traffic +according to user defined profiles. + +The steps needed to run Yardstick with NSB testing are: + +* Install Yardstick (NSB Testing). +* Setup/Reference pod.yaml describing Test topology +* Create/Reference the test configuration yaml file. +* Run the test case. + + +Prerequisites +============= + +Refer chapter Yardstick Installation for more information on yardstick +prerequisites + +Several prerequisites are needed for Yardstick (VNF testing): + + * Python Modules: pyzmq, pika. + * flex + * bison + * build-essential + * automake + * libtool + * librabbitmq-dev + * rabbitmq-server + * collectd + * intel-cmt-cat + +Hardware & Software Ingredients +------------------------------- + +SUT requirements: + + + ======= =================== + Item Description + ======= =================== + Memory Min 20GB + NICs 2 x 10G + OS Ubuntu 16.04.3 LTS + kernel 4.4.0-34-generic + DPDK 17.02 + ======= =================== + +Boot and BIOS settings: + + + ============= ================================================= + Boot settings default_hugepagesz=1G hugepagesz=1G hugepages=16 + hugepagesz=2M hugepages=2048 isolcpus=1-11,22-33 + nohz_full=1-11,22-33 rcu_nocbs=1-11,22-33 + iommu=on iommu=pt intel_iommu=on + Note: nohz_full and rcu_nocbs is to disable Linux + kernel interrupts + BIOS CPU Power and Performance Policy + CPU C-state Disabled + CPU P-state Disabled + Enhanced Intel® Speedstep® Tech Disabl + Hyper-Threading Technology (If supported) Enabled + Virtualization Techology Enabled + Intel(R) VT for Direct I/O Enabled + Coherency Enabled + Turbo Boost Disabled + ============= ================================================= + + + +Install Yardstick (NSB Testing) +=============================== + +Download the source code and install Yardstick from it + +.. code-block:: console + + git clone https://gerrit.opnfv.org/gerrit/yardstick + + cd yardstick + + # Switch to latest stable branch + # git checkout + git checkout stable/euphrates + +Configure the network proxy, either using the environment variables or setting +the global environment file: + +.. code-block:: ini + + cat /etc/environment + http_proxy='http://proxy.company.com:port' + https_proxy='http://proxy.company.com:port' + +.. code-block:: console + + export http_proxy='http://proxy.company.com:port' + export https_proxy='http://proxy.company.com:port' + +The last step is to modify the Yardstick installation inventory, used by +Ansible: + +.. code-block:: ini + + cat ./ansible/yardstick-install-inventory.ini + [jumphost] + localhost ansible_connection=local + + [yardstick-standalone] + yardstick-standalone-node ansible_host=192.168.1.2 + yardstick-standalone-node-2 ansible_host=192.168.1.3 + + # section below is only due backward compatibility. + # it will be removed later + [yardstick:children] + jumphost + + [all:vars] + ansible_user=root + ansible_pass=root + + +To execute an installation for a Bare-Metal or a Standalone context: + +.. code-block:: console + + ./nsb_setup.sh + + +To execute an installation for an OpenStack context: + +.. code-block:: console + + ./nsb_setup.sh + +Above command setup docker with latest yardstick code. To execute + +.. code-block:: console + + docker exec -it yardstick bash + +It will also automatically download all the packages needed for NSB Testing +setup. Refer chapter :doc:`04-installation` for more on docker +**Install Yardstick using Docker (recommended)** + +System Topology: +================ + +.. code-block:: console + + +----------+ +----------+ + | | | | + | | (0)----->(0) | | + | TG1 | | DUT | + | | | | + | | (1)<-----(1) | | + +----------+ +----------+ + trafficgen_1 vnf + + +Environment parameters and credentials +====================================== + +Config yardstick conf +--------------------- + +If user did not run 'yardstick env influxdb' inside the container, which will +generate correct ``yardstick.conf``, then create the config file manually (run +inside the container): +:: + + cp ./etc/yardstick/yardstick.conf.sample /etc/yardstick/yardstick.conf + vi /etc/yardstick/yardstick.conf + +Add trex_path, trex_client_lib and bin_path in 'nsb' section. + +:: + + [DEFAULT] + debug = True + dispatcher = file, influxdb + + [dispatcher_influxdb] + timeout = 5 + target = http://{YOUR_IP_HERE}:8086 + db_name = yardstick + username = root + password = root + + [nsb] + trex_path=/opt/nsb_bin/trex/scripts + bin_path=/opt/nsb_bin + trex_client_lib=/opt/nsb_bin/trex_client/stl + +Run Yardstick - Network Service Testcases +========================================= + + +NS testing - using yardstick CLI +-------------------------------- + + See :doc:`04-installation` + +.. code-block:: console + + + docker exec -it yardstick /bin/bash + source /etc/yardstick/openstack.creds (only for heat TC if nsb_setup.sh was NOT used) + export EXTERNAL_NETWORK="" (only for heat TC) + yardstick --debug task start yardstick/samples/vnf_samples/nsut// + +Network Service Benchmarking - Bare-Metal +========================================= + +Bare-Metal Config pod.yaml describing Topology +---------------------------------------------- + +Bare-Metal 2-Node setup +^^^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: console + + +----------+ +----------+ + | | | | + | | (0)----->(0) | | + | TG1 | | DUT | + | | | | + | | (n)<-----(n) | | + +----------+ +----------+ + trafficgen_1 vnf + +Bare-Metal 3-Node setup - Correlated Traffic +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: console + + +----------+ +----------+ +------------+ + | | | | | | + | | | | | | + | | (0)----->(0) | | | UDP | + | TG1 | | DUT | | Replay | + | | | | | | + | | | |(1)<---->(0)| | + +----------+ +----------+ +------------+ + trafficgen_1 vnf trafficgen_2 + + +Bare-Metal Config pod.yaml +-------------------------- +Before executing Yardstick test cases, make sure that pod.yaml reflects the +topology and update all the required fields.:: + + cp /etc/yardstick/nodes/pod.yaml.nsb.sample /etc/yardstick/nodes/pod.yaml + +.. code-block:: YAML + + nodes: + - + name: trafficgen_1 + role: TrafficGen + ip: 1.1.1.1 + user: root + password: r00t + interfaces: + xe0: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.0" + driver: i40e # default kernel driver + dpdk_port_num: 0 + local_ip: "152.16.100.20" + netmask: "255.255.255.0" + local_mac: "00:00:00:00:00:01" + xe1: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.1" + driver: i40e # default kernel driver + dpdk_port_num: 1 + local_ip: "152.16.40.20" + netmask: "255.255.255.0" + local_mac: "00:00.00:00:00:02" + + - + name: vnf + role: vnf + ip: 1.1.1.2 + user: root + password: r00t + host: 1.1.1.2 #BM - host == ip, virtualized env - Host - compute node + interfaces: + xe0: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.0" + driver: i40e # default kernel driver + dpdk_port_num: 0 + local_ip: "152.16.100.19" + netmask: "255.255.255.0" + local_mac: "00:00:00:00:00:03" + + xe1: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.1" + driver: i40e # default kernel driver + dpdk_port_num: 1 + local_ip: "152.16.40.19" + netmask: "255.255.255.0" + local_mac: "00:00:00:00:00:04" + routing_table: + - network: "152.16.100.20" + netmask: "255.255.255.0" + gateway: "152.16.100.20" + if: "xe0" + - network: "152.16.40.20" + netmask: "255.255.255.0" + gateway: "152.16.40.20" + if: "xe1" + nd_route_tbl: + - network: "0064:ff9b:0:0:0:0:9810:6414" + netmask: "112" + gateway: "0064:ff9b:0:0:0:0:9810:6414" + if: "xe0" + - network: "0064:ff9b:0:0:0:0:9810:2814" + netmask: "112" + gateway: "0064:ff9b:0:0:0:0:9810:2814" + if: "xe1" + + +Network Service Benchmarking - Standalone Virtualization +======================================================== + +SR-IOV +------ + +SR-IOV Pre-requisites +^^^^^^^^^^^^^^^^^^^^^ + +On Host: + a) Create a bridge for VM to connect to external network + + .. code-block:: console + + brctl addbr br-int + brctl addif br-int #This interface is connected to internet + + b) Build guest image for VNF to run. + Most of the sample test cases in Yardstick are using a guest image called + ``yardstick-image`` which deviates from an Ubuntu Cloud Server image + Yardstick has a tool for building this custom image with samplevnf. + It is necessary to have ``sudo`` rights to use this tool. + + Also you may need to install several additional packages to use this tool, by + following the commands below:: + + sudo apt-get update && sudo apt-get install -y qemu-utils kpartx + + This image can be built using the following command in the directory where Yardstick is installed + + .. code-block:: console + + export YARD_IMG_ARCH='amd64' + sudo echo "Defaults env_keep += \'YARD_IMG_ARCH\'" >> /etc/sudoers + + Please use ansible script to generate a cloud image refer to :doc:`04-installation` + + for more details refer to chapter :doc:`04-installation` + + .. note:: VM should be build with static IP and should be accessible from yardstick host. + + +SR-IOV Config pod.yaml describing Topology +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +SR-IOV 2-Node setup: +^^^^^^^^^^^^^^^^^^^^ +.. code-block:: console + + +--------------------+ + | | + | | + | DUT | + | (VNF) | + | | + +--------------------+ + | VF NIC | | VF NIC | + +--------+ +--------+ + ^ ^ + | | + | | + +----------+ +-------------------------+ + | | | ^ ^ | + | | | | | | + | | (0)<----->(0) | ------ | | + | TG1 | | SUT | | + | | | | | + | | (n)<----->(n) |------------------ | + +----------+ +-------------------------+ + trafficgen_1 host + + + +SR-IOV 3-Node setup - Correlated Traffic +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: console + + +--------------------+ + | | + | | + | DUT | + | (VNF) | + | | + +--------------------+ + | VF NIC | | VF NIC | + +--------+ +--------+ + ^ ^ + | | + | | + +----------+ +-------------------------+ +--------------+ + | | | ^ ^ | | | + | | | | | | | | + | | (0)<----->(0) | ------ | | | TG2 | + | TG1 | | SUT | | | (UDP Replay) | + | | | | | | | + | | (n)<----->(n) | ------ | (n)<-->(n) | | + +----------+ +-------------------------+ +--------------+ + trafficgen_1 host trafficgen_2 + +Before executing Yardstick test cases, make sure that pod.yaml reflects the +topology and update all the required fields. + +.. code-block:: console + + cp /etc/yardstick/nodes/standalone/trex_bm.yaml.sample /etc/yardstick/nodes/standalone/pod_trex.yaml + cp /etc/yardstick/nodes/standalone/host_sriov.yaml /etc/yardstick/nodes/standalone/host_sriov.yaml + +.. note:: Update all the required fields like ip, user, password, pcis, etc... + +SR-IOV Config pod_trex.yaml +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: YAML + + nodes: + - + name: trafficgen_1 + role: TrafficGen + ip: 1.1.1.1 + user: root + password: r00t + key_filename: /root/.ssh/id_rsa + interfaces: + xe0: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.0" + driver: i40e # default kernel driver + dpdk_port_num: 0 + local_ip: "152.16.100.20" + netmask: "255.255.255.0" + local_mac: "00:00:00:00:00:01" + xe1: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.1" + driver: i40e # default kernel driver + dpdk_port_num: 1 + local_ip: "152.16.40.20" + netmask: "255.255.255.0" + local_mac: "00:00.00:00:00:02" + +SR-IOV Config host_sriov.yaml +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: YAML + + nodes: + - + name: sriov + role: Sriov + ip: 192.168.100.101 + user: "" + password: "" + +SR-IOV testcase update: +``/samples/vnf_samples/nsut/vfw/tc_sriov_rfc2544_ipv4_1rule_1flow_64B_trex.yaml`` + +Update "contexts" section +""""""""""""""""""""""""" + +.. code-block:: YAML + + contexts: + - name: yardstick + type: Node + file: /etc/yardstick/nodes/standalone/pod_trex.yaml + - type: StandaloneSriov + file: /etc/yardstick/nodes/standalone/host_sriov.yaml + name: yardstick + vm_deploy: True + flavor: + images: "/var/lib/libvirt/images/ubuntu.qcow2" + ram: 4096 + extra_specs: + hw:cpu_sockets: 1 + hw:cpu_cores: 6 + hw:cpu_threads: 2 + user: "" # update VM username + password: "" # update password + servers: + vnf: + network_ports: + mgmt: + cidr: '1.1.1.61/24' # Update VM IP address, if static, / or if dynamic, / + xe0: + - uplink_0 + xe1: + - downlink_0 + networks: + uplink_0: + phy_port: "0000:05:00.0" + vpci: "0000:00:07.0" + cidr: '152.16.100.10/24' + gateway_ip: '152.16.100.20' + downlink_0: + phy_port: "0000:05:00.1" + vpci: "0000:00:08.0" + cidr: '152.16.40.10/24' + gateway_ip: '152.16.100.20' + + + +OVS-DPDK +-------- + +OVS-DPDK Pre-requisites +^^^^^^^^^^^^^^^^^^^^^^^ + +On Host: + a) Create a bridge for VM to connect to external network + + .. code-block:: console + + brctl addbr br-int + brctl addif br-int #This interface is connected to internet + + b) Build guest image for VNF to run. + Most of the sample test cases in Yardstick are using a guest image called + ``yardstick-image`` which deviates from an Ubuntu Cloud Server image + Yardstick has a tool for building this custom image with samplevnf. + It is necessary to have ``sudo`` rights to use this tool. + + Also you may need to install several additional packages to use this tool, by + following the commands below:: + + sudo apt-get update && sudo apt-get install -y qemu-utils kpartx + + This image can be built using the following command in the directory where Yardstick is installed:: + + export YARD_IMG_ARCH='amd64' + sudo echo "Defaults env_keep += \'YARD_IMG_ARCH\'" >> /etc/sudoers + sudo tools/yardstick-img-dpdk-modify tools/ubuntu-server-cloudimg-samplevnf-modify.sh + + for more details refer to chapter :doc:`04-installation` + + .. note:: VM should be build with static IP and should be accessible from yardstick host. + + c) OVS & DPDK version. + - OVS 2.7 and DPDK 16.11.1 above version is supported + + d) Setup OVS/DPDK on host. + Please refer to below link on how to setup `OVS-DPDK `_ + + +OVS-DPDK Config pod.yaml describing Topology +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +OVS-DPDK 2-Node setup +^^^^^^^^^^^^^^^^^^^^^ + + +.. code-block:: console + + +--------------------+ + | | + | | + | DUT | + | (VNF) | + | | + +--------------------+ + | virtio | | virtio | + +--------+ +--------+ + ^ ^ + | | + | | + +--------+ +--------+ + | vHOST0 | | vHOST1 | + +----------+ +-------------------------+ + | | | ^ ^ | + | | | | | | + | | (0)<----->(0) | ------ | | + | TG1 | | SUT | | + | | | (ovs-dpdk) | | + | | (n)<----->(n) |------------------ | + +----------+ +-------------------------+ + trafficgen_1 host + + +OVS-DPDK 3-Node setup - Correlated Traffic +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console + + +--------------------+ + | | + | | + | DUT | + | (VNF) | + | | + +--------------------+ + | virtio | | virtio | + +--------+ +--------+ + ^ ^ + | | + | | + +--------+ +--------+ + | vHOST0 | | vHOST1 | + +----------+ +-------------------------+ +------------+ + | | | ^ ^ | | | + | | | | | | | | + | | (0)<----->(0) | ------ | | | TG2 | + | TG1 | | SUT | | |(UDP Replay)| + | | | (ovs-dpdk) | | | | + | | (n)<----->(n) | ------ |(n)<-->(n)| | + +----------+ +-------------------------+ +------------+ + trafficgen_1 host trafficgen_2 + + +Before executing Yardstick test cases, make sure that pod.yaml reflects the +topology and update all the required fields. + +.. code-block:: console + + cp /etc/yardstick/nodes/standalone/trex_bm.yaml.sample /etc/yardstick/nodes/standalone/pod_trex.yaml + cp /etc/yardstick/nodes/standalone/host_ovs.yaml /etc/yardstick/nodes/standalone/host_ovs.yaml + +.. note:: Update all the required fields like ip, user, password, pcis, etc... + +OVS-DPDK Config pod_trex.yaml +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: YAML + + nodes: + - + name: trafficgen_1 + role: TrafficGen + ip: 1.1.1.1 + user: root + password: r00t + interfaces: + xe0: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.0" + driver: i40e # default kernel driver + dpdk_port_num: 0 + local_ip: "152.16.100.20" + netmask: "255.255.255.0" + local_mac: "00:00:00:00:00:01" + xe1: # logical name from topology.yaml and vnfd.yaml + vpci: "0000:07:00.1" + driver: i40e # default kernel driver + dpdk_port_num: 1 + local_ip: "152.16.40.20" + netmask: "255.255.255.0" + local_mac: "00:00.00:00:00:02" + +OVS-DPDK Config host_ovs.yaml +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: YAML + + nodes: + - + name: ovs_dpdk + role: OvsDpdk + ip: 192.168.100.101 + user: "" + password: "" + +ovs_dpdk testcase update: +``/samples/vnf_samples/nsut/vfw/tc_ovs_rfc2544_ipv4_1rule_1flow_64B_trex.yaml`` + +Update "contexts" section +""""""""""""""""""""""""" + +.. code-block:: YAML + + contexts: + - name: yardstick + type: Node + file: /etc/yardstick/nodes/standalone/pod_trex.yaml + - type: StandaloneOvsDpdk + name: yardstick + file: /etc/yardstick/nodes/standalone/pod_ovs.yaml + vm_deploy: True + ovs_properties: + version: + ovs: 2.7.0 + dpdk: 16.11.1 + pmd_threads: 2 + ram: + socket_0: 2048 + socket_1: 2048 + queues: 4 + vpath: "/usr/local" + + flavor: + images: "/var/lib/libvirt/images/ubuntu.qcow2" + ram: 4096 + extra_specs: + hw:cpu_sockets: 1 + hw:cpu_cores: 6 + hw:cpu_threads: 2 + user: "" # update VM username + password: "" # update password + servers: + vnf: + network_ports: + mgmt: + cidr: '1.1.1.61/24' # Update VM IP address, if static, / or if dynamic, / + xe0: + - uplink_0 + xe1: + - downlink_0 + networks: + uplink_0: + phy_port: "0000:05:00.0" + vpci: "0000:00:07.0" + cidr: '152.16.100.10/24' + gateway_ip: '152.16.100.20' + downlink_0: + phy_port: "0000:05:00.1" + vpci: "0000:00:08.0" + cidr: '152.16.40.10/24' + gateway_ip: '152.16.100.20' + + +Network Service Benchmarking - OpenStack with SR-IOV support +============================================================ + +This section describes how to run a Sample VNF test case, using Heat context, +with SR-IOV. It also covers how to install OpenStack in Ubuntu 16.04, using +DevStack, with SR-IOV support. + + +Single node OpenStack setup with external TG +-------------------------------------------- + +.. code-block:: console + + +----------------------------+ + |OpenStack(DevStack) | + | | + | +--------------------+ | + | |sample-VNF VM | | + | | | | + | | DUT | | + | | (VNF) | | + | | | | + | +--------+ +--------+ | + | | VF NIC | | VF NIC | | + | +-----+--+--+----+---+ | + | ^ ^ | + | | | | + +----------+ +---------+----------+-------+ + | | | VF0 VF1 | + | | | ^ ^ | + | | | | SUT | | + | TG | (PF0)<----->(PF0) +---------+ | | + | | | | | + | | (PF1)<----->(PF1) +--------------------+ | + | | | | + +----------+ +----------------------------+ + trafficgen_1 host + + +Host pre-configuration +^^^^^^^^^^^^^^^^^^^^^^ + +.. warning:: The following configuration requires sudo access to the system. Make + sure that your user have the access. + +Enable the Intel VT-d or AMD-Vi extension in the BIOS. Some system manufacturers +disable this extension by default. + +Activate the Intel VT-d or AMD-Vi extension in the kernel by modifying the GRUB +config file ``/etc/default/grub``. + +For the Intel platform: + +.. code:: bash + + ... + GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on" + ... + +For the AMD platform: + +.. code:: bash + + ... + GRUB_CMDLINE_LINUX_DEFAULT="amd_iommu=on" + ... + +Update the grub configuration file and restart the system: + +.. warning:: The following command will reboot the system. + +.. code:: bash + + sudo update-grub + sudo reboot + +Make sure the extension has been enabled: + +.. code:: bash + + sudo journalctl -b 0 | grep -e IOMMU -e DMAR + + Feb 06 14:50:14 hostname kernel: ACPI: DMAR 0x000000006C406000 0001E0 (v01 INTEL S2600WF 00000001 INTL 20091013) + Feb 06 14:50:14 hostname kernel: DMAR: IOMMU enabled + Feb 06 14:50:14 hostname kernel: DMAR: Host address width 46 + Feb 06 14:50:14 hostname kernel: DMAR: DRHD base: 0x000000d37fc000 flags: 0x0 + Feb 06 14:50:14 hostname kernel: DMAR: dmar0: reg_base_addr d37fc000 ver 1:0 cap 8d2078c106f0466 ecap f020de + Feb 06 14:50:14 hostname kernel: DMAR: DRHD base: 0x000000e0ffc000 flags: 0x0 + Feb 06 14:50:14 hostname kernel: DMAR: dmar1: reg_base_addr e0ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020de + Feb 06 14:50:14 hostname kernel: DMAR: DRHD base: 0x000000ee7fc000 flags: 0x0 + +Setup system proxy (if needed). Add the following configuration into the +``/etc/environment`` file: + +.. note:: The proxy server name/port and IPs should be changed according to + actuall/current proxy configuration in the lab. + +.. code:: bash + + export http_proxy=http://proxy.company.com:port + export https_proxy=http://proxy.company.com:port + export ftp_proxy=http://proxy.company.com:port + export no_proxy=localhost,127.0.0.1,company.com,,,... + export NO_PROXY=localhost,127.0.0.1,company.com,,,... + +Upgrade the system: + +.. code:: bash + + sudo -EH apt-get update + sudo -EH apt-get upgrade + sudo -EH apt-get dist-upgrade + +Install dependencies needed for the DevStack + +.. code:: bash + + sudo -EH apt-get install python + sudo -EH apt-get install python-dev + sudo -EH apt-get install python-pip + +Setup SR-IOV ports on the host: + +.. note:: The ``enp24s0f0``, ``enp24s0f0`` are physical function (PF) interfaces + on a host and ``enp24s0f3`` is a public interface used in OpenStack, so the + interface names should be changed according to the HW environment used for + testing. + +.. code:: bash + + sudo ip link set dev enp24s0f0 up + sudo ip link set dev enp24s0f1 up + sudo ip link set dev enp24s0f3 up + + # Create VFs on PF + echo 2 | sudo tee /sys/class/net/enp24s0f0/device/sriov_numvfs + echo 2 | sudo tee /sys/class/net/enp24s0f1/device/sriov_numvfs + + +DevStack installation +^^^^^^^^^^^^^^^^^^^^^ + +Use official `Devstack `_ +documentation to install OpenStack on a host. Please note, that stable +``pike`` branch of devstack repo should be used during the installation. +The required `local.conf`` configuration file are described below. + +DevStack configuration file: + +.. note:: Update the devstack configuration file by replacing angluar brackets + with a short description inside. + +.. note:: Use ``lspci | grep Ether`` & ``lspci -n | grep `` + commands to get device and vendor id of the virtual function (VF). + +.. literalinclude:: code/single-devstack-local.conf + :language: console + +Start the devstack installation on a host. + + +TG host configuration +^^^^^^^^^^^^^^^^^^^^^ + +Yardstick automatically install and configure Trex traffic generator on TG +host based on provided POD file (see below). Anyway, it's recommended to check +the compatibility of the installed NIC on the TG server with software Trex using +the manual at https://trex-tgn.cisco.com/trex/doc/trex_manual.html. + + +Run the Sample VNF test case +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There is an example of Sample VNF test case ready to be executed in an +OpenStack environment with SR-IOV support: ``samples/vnf_samples/nsut/vfw/ +tc_heat_sriov_external_rfc2544_ipv4_1rule_1flow_64B_trex.yaml``. + +Install yardstick using `Install Yardstick (NSB Testing)`_ steps for OpenStack +context. + +Create pod file for TG in the yardstick repo folder located in the yardstick +container: + +.. note:: The ``ip``, ``user``, ``password`` and ``vpci`` fields show be changed + according to HW environment used for the testing. Use ``lshw -c network -businfo`` + command to get the PF PCI address for ``vpci`` field. + +.. literalinclude:: code/single-yardstick-pod.conf + :language: console + +Run the Sample vFW RFC2544 SR-IOV TC (``samples/vnf_samples/nsut/vfw/ +tc_heat_sriov_external_rfc2544_ipv4_1rule_1flow_64B_trex.yaml``) in the heat +context using steps described in `NS testing - using yardstick CLI`_ section. + + +Multi node OpenStack TG and VNF setup (two nodes) +------------------------------------------------- + +.. code-block:: console + + +----------------------------+ +----------------------------+ + |OpenStack(DevStack) | |OpenStack(DevStack) | + | | | | + | +--------------------+ | | +--------------------+ | + | |sample-VNF VM | | | |sample-VNF VM | | + | | | | | | | | + | | TG | | | | DUT | | + | | trafficgen_1 | | | | (VNF) | | + | | | | | | | | + | +--------+ +--------+ | | +--------+ +--------+ | + | | VF NIC | | VF NIC | | | | VF NIC | | VF NIC | | + | +----+---+--+----+---+ | | +-----+--+--+----+---+ | + | ^ ^ | | ^ ^ | + | | | | | | | | + +--------+-----------+-------+ +---------+----------+-------+ + | VF0 VF1 | | VF0 VF1 | + | ^ ^ | | ^ ^ | + | | SUT2 | | | | SUT1 | | + | | +-------+ (PF0)<----->(PF0) +---------+ | | + | | | | | | + | +-------------------+ (PF1)<----->(PF1) +--------------------+ | + | | | | + +----------------------------+ +----------------------------+ + host2 (compute) host1 (controller) + + +Controller/Compute pre-configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Pre-configuration of the controller and compute hosts are the same as +described in `Host pre-configuration`_ section. Follow the steps in the section. + + +DevStack configuration +^^^^^^^^^^^^^^^^^^^^^^ + +Use official `Devstack `_ +documentation to install OpenStack on a host. Please note, that stable +``pike`` branch of devstack repo should be used during the installation. +The required `local.conf`` configuration file are described below. + +.. note:: Update the devstack configuration files by replacing angluar brackets + with a short description inside. + +.. note:: Use ``lspci | grep Ether`` & ``lspci -n | grep `` + commands to get device and vendor id of the virtual function (VF). + +DevStack configuration file for controller host: + +.. literalinclude:: code/multi-devstack-controller-local.conf + :language: console + +DevStack configuration file for compute host: + +.. literalinclude:: code/multi-devstack-compute-local.conf + :language: console + +Start the devstack installation on the controller and compute hosts. + + +Run the sample vFW TC +^^^^^^^^^^^^^^^^^^^^^ + +Install yardstick using `Install Yardstick (NSB Testing)`_ steps for OpenStack +context. + +Run sample vFW RFC2544 SR-IOV TC (``samples/vnf_samples/nsut/vfw/ +tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml``) in the heat +context using steps described in `NS testing - using yardstick CLI`_ section +and the following yardtick command line arguments: + +.. code:: bash + + yardstick -d task start --task-args='{"provider": "sriov"}' \ + samples/vnf_samples/nsut/vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex.yaml + + +Enabling other Traffic generator +================================ + +IxLoad +^^^^^^ + +1. Software needed: IxLoadAPI ``Linux64.bin.tgz`` and + ``Linux64.bin.tar.gz`` (Download from ixia support site) + Install - ``Linux64.bin.tgz`` and + ``Linux64.bin.tar.gz`` + If the installation was not done inside the container, after installing + the IXIA client, check ``/opt/ixia/ixload//bin/ixloadpython`` and make + sure you can run this cmd inside the yardstick container. Usually user is + required to copy or link ``/opt/ixia/python//bin/ixiapython`` to + ``/usr/bin/ixiapython`` inside the container. + +2. Update ``pod_ixia.yaml`` file with ixia details. + + .. code-block:: console + + cp /etc/yardstick/nodes/pod.yaml.nsb.sample.ixia etc/yardstick/nodes/pod_ixia.yaml + + Config ``pod_ixia.yaml`` + + .. code-block:: yaml + + nodes: + - + name: trafficgen_1 + role: IxNet + ip: 1.2.1.1 #ixia machine ip + user: user + password: r00t + key_filename: /root/.ssh/id_rsa + tg_config: + ixchassis: "1.2.1.7" #ixia chassis ip + tcl_port: "8009" # tcl server port + lib_path: "/opt/ixia/ixos-api/8.01.0.2/lib/ixTcl1.0" + root_dir: "/opt/ixia/ixos-api/8.01.0.2/" + py_bin_path: "/opt/ixia/ixload/8.01.106.3/bin/" + py_lib_path: "/opt/ixia/ixnetwork/8.01.1029.14/lib/PythonApi" + dut_result_dir: "/mnt/ixia" + version: 8.1 + interfaces: + xe0: # logical name from topology.yaml and vnfd.yaml + vpci: "2:5" # Card:port + driver: "none" + dpdk_port_num: 0 + local_ip: "152.16.100.20" + netmask: "255.255.0.0" + local_mac: "00:98:10:64:14:00" + xe1: # logical name from topology.yaml and vnfd.yaml + vpci: "2:6" # [(Card, port)] + driver: "none" + dpdk_port_num: 1 + local_ip: "152.40.40.20" + netmask: "255.255.0.0" + local_mac: "00:98:28:28:14:00" + + for sriov/ovs_dpdk pod files, please refer to above Standalone Virtualization for ovs-dpdk/sriov configuration + +3. Start IxOS TCL Server (Install 'Ixia IxExplorer IxOS ') + You will also need to configure the IxLoad machine to start the IXIA + IxosTclServer. This can be started like so: + + * Connect to the IxLoad machine using RDP + * Go to: + ``Start->Programs->Ixia->IxOS->IxOS 8.01-GA-Patch1->Ixia Tcl Server IxOS 8.01-GA-Patch1`` + or + ``"C:\Program Files (x86)\Ixia\IxOS\8.01-GA-Patch1\ixTclServer.exe"`` + +4. Create a folder ``Results`` in c:\ and share the folder on the network. + +5. Execute testcase in samplevnf folder e.g. + ``/samples/vnf_samples/nsut/vfw/tc_baremetal_http_ixload_1b_Requests-65000_Concurrency.yaml`` + +IxNetwork +--------- + +1. Software needed: ``IxNetworkAPILinux64.bin.tgz`` + (Download from ixia support site) + Install - ``IxNetworkAPILinux64.bin.tgz`` +2. Update pod_ixia.yaml file with ixia details. + + .. code-block:: console + + cp /etc/yardstick/nodes/pod.yaml.nsb.sample.ixia etc/yardstick/nodes/pod_ixia.yaml + + Config pod_ixia.yaml + + .. code-block:: yaml + + nodes: + - + name: trafficgen_1 + role: IxNet + ip: 1.2.1.1 #ixia machine ip + user: user + password: r00t + key_filename: /root/.ssh/id_rsa + tg_config: + ixchassis: "1.2.1.7" #ixia chassis ip + tcl_port: "8009" # tcl server port + lib_path: "/opt/ixia/ixos-api/8.01.0.2/lib/ixTcl1.0" + root_dir: "/opt/ixia/ixos-api/8.01.0.2/" + py_bin_path: "/opt/ixia/ixload/8.01.106.3/bin/" + py_lib_path: "/opt/ixia/ixnetwork/8.01.1029.14/lib/PythonApi" + dut_result_dir: "/mnt/ixia" + version: 8.1 + interfaces: + xe0: # logical name from topology.yaml and vnfd.yaml + vpci: "2:5" # Card:port + driver: "none" + dpdk_port_num: 0 + local_ip: "152.16.100.20" + netmask: "255.255.0.0" + local_mac: "00:98:10:64:14:00" + xe1: # logical name from topology.yaml and vnfd.yaml + vpci: "2:6" # [(Card, port)] + driver: "none" + dpdk_port_num: 1 + local_ip: "152.40.40.20" + netmask: "255.255.0.0" + local_mac: "00:98:28:28:14:00" + + for sriov/ovs_dpdk pod files, please refer to above Standalone Virtualization for ovs-dpdk/sriov configuration + +3. Start IxNetwork TCL Server + You will also need to configure the IxNetwork machine to start the IXIA + IxNetworkTclServer. This can be started like so: + + * Connect to the IxNetwork machine using RDP + * Go to: + ``Start->Programs->Ixia->IxNetwork->IxNetwork 7.21.893.14 GA->IxNetworkTclServer`` + (or ``IxNetworkApiServer``) + +4. Execute testcase in samplevnf folder e.g. + ``/samples/vnf_samples/nsut/vfw/tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_ixia.yaml`` + diff --git a/docs/testing/user/userguide/13-nsb_operation.rst b/docs/testing/user/userguide/13-nsb_operation.rst deleted file mode 100644 index e791b048d..000000000 --- a/docs/testing/user/userguide/13-nsb_operation.rst +++ /dev/null @@ -1,306 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International -.. License. -.. http://creativecommons.org/licenses/by/4.0 -.. (c) OPNFV, 2016-2017 Intel Corporation. - -Yardstick - NSB Testing - Operation -=================================== - -Abstract --------- - -NSB test configuration and OpenStack setup requirements - - -OpenStack Network Configuration -------------------------------- - -NSB requires certain OpenStack deployment configurations. -For optimal VNF characterization using external traffic generators NSB requires -provider/external networks. - - -Provider networks -^^^^^^^^^^^^^^^^^ - -The VNFs require a clear L2 connect to the external network in order to generate -realistic traffic from multiple address ranges and port - -In order to prevent Neutron from filtering traffic we have to disable Neutron Port Security. -We also disable DHCP on the data ports because we are binding the ports to DPDK and do not need -DHCP addresses. We also disable gateways because multiple default gateways can prevent SSH access -to the VNF from the floating IP. We only want a gateway on the mgmt network - -.. code-block:: yaml - - uplink_0: - cidr: '10.1.0.0/24' - gateway_ip: 'null' - port_security_enabled: False - enable_dhcp: 'false' - -Heat Topologies -^^^^^^^^^^^^^^^ - -By default Heat will attach every node to every Neutron network that is created. -For scale-out tests we do not want to attach every node to every network. - -For each node you can specify which ports are on which network using the -network_ports dictionary. - -In this example we have ``TRex xe0 <-> xe0 VNF xe1 <-> xe0 UDP_Replay`` - -.. code-block:: yaml - - vnf_0: - floating_ip: true - placement: "pgrp1" - network_ports: - mgmt: - - mgmt - uplink_0: - - xe0 - downlink_0: - - xe1 - tg_0: - floating_ip: true - placement: "pgrp1" - network_ports: - mgmt: - - mgmt - uplink_0: - - xe0 - # Trex always needs two ports - uplink_1: - - xe1 - tg_1: - floating_ip: true - placement: "pgrp1" - network_ports: - mgmt: - - mgmt - downlink_0: - - xe0 - -Collectd KPIs -------------- - -NSB can collect KPIs from collected. We have support for various plugins enabled by the -Barometer project. - -The default yardstick-samplevnf has collectd installed. This allows for collecting KPIs -from the VNF. - -Collecting KPIs from the NFVi is more complicated and requires manual setup. -We assume that collectd is not installed on the compute nodes. - -To collectd KPIs from the NFVi compute nodes: - - - * install_collectd on the compute nodes - * create pod.yaml for the compute nodes - * enable specific plugins depending on the vswitch and DPDK - - example pod.yaml section for Compute node running collectd. - -.. code-block:: yaml - - - - name: "compute-1" - role: Compute - ip: "10.1.2.3" - user: "root" - ssh_port: "22" - password: "" - collectd: - interval: 5 - plugins: - # for libvirtd stats - virt: {} - intel_pmu: {} - ovs_stats: - # path to OVS socket - ovs_socket_path: /var/run/openvswitch/db.sock - intel_rdt: {} - - - -Scale-Up --------- - -VNFs performance data with scale-up - - * Helps to figure out optimal number of cores specification in the Virtual Machine template creation or VNF - * Helps in comparison between different VNF vendor offerings - * Better the scale-up index, indicates the performance scalability of a particular solution - -Heat -^^^^ - -For VNF scale-up tests we increase the number for VNF worker threads and ports. In the case of VNFs -we also need to increase the number of VCPUs and memory allocated to the VNF. - -An example scale-up Heat testcase is: - -.. literalinclude:: /submodules/yardstick/samples/vnf_samples/nsut/vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale-up.yaml - :language: yaml - -This testcase template requires specifying the number of VCPUs, Memory and Ports. -We set the VCPUs and memory using the ``--task-args`` options - -.. code-block:: console - - yardstick task start --task-args='{"mem": 10480, "vcpus": 4, "ports": 2}' \ - samples/vnf_samples/nsut/vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale-up.yaml - -In order to support ports scale-up, traffic and topology templates need to be used in testcase. - -A example topology template is: - -.. literalinclude:: /submodules/yardstick/samples/vnf_samples/nsut/vfw/vfw-tg-topology-scale-up.yaml - :language: yaml - -This template has ``vports`` as an argument. To pass this argument it needs to -be configured in ``extra_args`` scenario definition. Please note that more -argument can be defined in that section. All of them will be passed to topology -and traffic profile templates - -For example: - -.. code-block:: yaml - - schema: yardstick:task:0.1 - scenarios: - - type: NSPerf - traffic_profile: ../../traffic_profiles/ipv4_throughput-scale-up.yaml - extra_args: - vports: {{ vports }} - topology: vfw-tg-topology-scale-up.yaml - -A example traffic profile template is: - -.. literalinclude:: /submodules/yardstick/samples/vnf_samples/traffic_profiles/ipv4_throughput-scale-up.yaml - :language: yaml - -There is an option to provide predefined config for SampleVNFs. Path to config -file may by specified in ``vnf_config`` scenario section. - -.. code-block:: yaml - - vnf__0: - rules: acl_1rule.yaml - vnf_config: {lb_config: 'SW', file: vfw_vnf_pipeline_cores_4_ports_2_lb_1_sw.conf } - - -Baremetal -^^^^^^^^^ - 1. Follow above traffic generator section to setup. - 2. edit num of threads in ``/samples/vnf_samples/nsut/vfw/tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex_scale_up.yaml`` - - e.g, 6 Threads for given VNF - -.. code-block:: yaml - - - schema: yardstick:task:0.1 - scenarios: - {% for worker_thread in [1, 2 ,3 , 4, 5, 6] %} - - type: NSPerf - traffic_profile: ../../traffic_profiles/ipv4_throughput.yaml - topology: vfw-tg-topology.yaml - nodes: - tg__0: trafficgen_1.yardstick - vnf__0: vnf.yardstick - options: - framesize: - uplink: {64B: 100} - downlink: {64B: 100} - flow: - src_ip: [{'tg__0': 'xe0'}] - dst_ip: [{'tg__0': 'xe1'}] - count: 1 - traffic_type: 4 - rfc2544: - allowed_drop_rate: 0.0001 - 0.0001 - vnf__0: - rules: acl_1rule.yaml - vnf_config: {lb_config: 'HW', lb_count: 1, worker_config: '1C/1T', worker_threads: {{worker_thread}}} - nfvi_enable: True - runner: - type: Iteration - iterations: 10 - interval: 35 - {% endfor %} - context: - type: Node - name: yardstick - nfvi_type: baremetal - file: /etc/yardstick/nodes/pod.yaml - -Scale-Out --------------------- - -VNFs performance data with scale-out - - * Helps in capacity planning to meet the given network node requirements - * Helps in comparison between different VNF vendor offerings - * Better the scale-out index, provides the flexibility in meeting future capacity requirements - - -Standalone -^^^^^^^^^^ - -Scale-out not supported on Baremetal. - -1. Follow above traffic generator section to setup. -2. Generate testcase for standalone virtualization using ansible scripts - - .. code-block:: console - - cd /ansible - trex: standalone_ovs_scale_out_trex_test.yaml or standalone_sriov_scale_out_trex_test.yaml - ixia: standalone_ovs_scale_out_ixia_test.yaml or standalone_sriov_scale_out_ixia_test.yaml - ixia_correlated: standalone_ovs_scale_out_ixia_correlated_test.yaml or standalone_sriov_scale_out_ixia_correlated_test.yaml - - update the ovs_dpdk or sriov above Ansible scripts reflect the setup - -3. run the test - - .. code-block:: console - - /samples/vnf_samples/nsut/tc_sriov_vfw_udp_ixia_correlated_scale_out-1.yaml - /samples/vnf_samples/nsut/tc_sriov_vfw_udp_ixia_correlated_scale_out-2.yaml - -Heat -^^^^ - -There are sample scale-out all-VM Heat tests. These tests only use VMs and don't use external traffic. - -The tests use UDP_Replay and correlated traffic. - -.. code-block:: console - - /samples/vnf_samples/nsut/cgnapt/tc_heat_rfc2544_ipv4_1flow_64B_trex_correlated_scale_4.yaml - -To run the test you need to increase OpenStack CPU, Memory and Port quotas. - - -Traffic Generator tuning ------------------------- - -The TRex traffic generator can be setup to use multiple threads per core, this is for multiqueue testing. - -TRex does not automatically enable multiple threads because we currently cannot detect the number of queues on a device. - -To enable multiple queue set the queues_per_port value in the TG VNF options section. - -.. code-block:: yaml - - scenarios: - - type: NSPerf - nodes: - tg__0: tg_0.yardstick - - options: - tg_0: - queues_per_port: 2 diff --git a/docs/testing/user/userguide/14-nsb-operation.rst b/docs/testing/user/userguide/14-nsb-operation.rst new file mode 100644 index 000000000..2e741822e --- /dev/null +++ b/docs/testing/user/userguide/14-nsb-operation.rst @@ -0,0 +1,315 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, 2016-2017 Intel Corporation. + +Yardstick - NSB Testing - Operation +=================================== + +Abstract +-------- + +NSB test configuration and OpenStack setup requirements + + +OpenStack Network Configuration +------------------------------- + +NSB requires certain OpenStack deployment configurations. +For optimal VNF characterization using external traffic generators NSB requires +provider/external networks. + + +Provider networks +^^^^^^^^^^^^^^^^^ + +The VNFs require a clear L2 connect to the external network in order to +generate realistic traffic from multiple address ranges and ports. + +In order to prevent Neutron from filtering traffic we have to disable Neutron +Port Security. We also disable DHCP on the data ports because we are binding +the ports to DPDK and do not need DHCP addresses. We also disable gateways +because multiple default gateways can prevent SSH access to the VNF from the +floating IP. We only want a gateway on the mgmt network + +.. code-block:: yaml + + uplink_0: + cidr: '10.1.0.0/24' + gateway_ip: 'null' + port_security_enabled: False + enable_dhcp: 'false' + +Heat Topologies +^^^^^^^^^^^^^^^ + +By default Heat will attach every node to every Neutron network that is +created. For scale-out tests we do not want to attach every node to every +network. + +For each node you can specify which ports are on which network using the +network_ports dictionary. + +In this example we have ``TRex xe0 <-> xe0 VNF xe1 <-> xe0 UDP_Replay`` + +.. code-block:: yaml + + vnf_0: + floating_ip: true + placement: "pgrp1" + network_ports: + mgmt: + - mgmt + uplink_0: + - xe0 + downlink_0: + - xe1 + tg_0: + floating_ip: true + placement: "pgrp1" + network_ports: + mgmt: + - mgmt + uplink_0: + - xe0 + # Trex always needs two ports + uplink_1: + - xe1 + tg_1: + floating_ip: true + placement: "pgrp1" + network_ports: + mgmt: + - mgmt + downlink_0: + - xe0 + +Collectd KPIs +------------- + +NSB can collect KPIs from collected. We have support for various plugins +enabled by the Barometer project. + +The default yardstick-samplevnf has collectd installed. This allows for +collecting KPIs from the VNF. + +Collecting KPIs from the NFVi is more complicated and requires manual setup. +We assume that collectd is not installed on the compute nodes. + +To collectd KPIs from the NFVi compute nodes: + + + * install_collectd on the compute nodes + * create pod.yaml for the compute nodes + * enable specific plugins depending on the vswitch and DPDK + + example pod.yaml section for Compute node running collectd. + +.. code-block:: yaml + + - + name: "compute-1" + role: Compute + ip: "10.1.2.3" + user: "root" + ssh_port: "22" + password: "" + collectd: + interval: 5 + plugins: + # for libvirtd stats + virt: {} + intel_pmu: {} + ovs_stats: + # path to OVS socket + ovs_socket_path: /var/run/openvswitch/db.sock + intel_rdt: {} + + + +Scale-Up +-------- + +VNFs performance data with scale-up + + * Helps to figure out optimal number of cores specification in the Virtual + Machine template creation or VNF + * Helps in comparison between different VNF vendor offerings + * Better the scale-up index, indicates the performance scalability of a + particular solution + +Heat +^^^^ +For VNF scale-up tests we increase the number for VNF worker threads. In the +case of VNFs we also need to increase the number of VCPUs and memory allocated +to the VNF. + +An example scale-up Heat testcase is: + +.. literalinclude:: /submodules/yardstick/samples/vnf_samples/nsut/vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale-up.yaml + :language: yaml + +This testcase template requires specifying the number of VCPUs, Memory and Ports. +We set the VCPUs and memory using the ``--task-args`` options + +.. code-block:: console + + yardstick task start --task-args='{"mem": 10480, "vcpus": 4, "ports": 2}' \ + samples/vnf_samples/nsut/vfw/tc_heat_rfc2544_ipv4_1rule_1flow_64B_trex_scale-up.yaml + +In order to support ports scale-up, traffic and topology templates need to be used in testcase. + +A example topology template is: + +.. literalinclude:: /submodules/yardstick/samples/vnf_samples/nsut/vfw/vfw-tg-topology-scale-up.yaml + :language: yaml + +This template has ``vports`` as an argument. To pass this argument it needs to +be configured in ``extra_args`` scenario definition. Please note that more +argument can be defined in that section. All of them will be passed to topology +and traffic profile templates + +For example: + +.. code-block:: yaml + + schema: yardstick:task:0.1 + scenarios: + - type: NSPerf + traffic_profile: ../../traffic_profiles/ipv4_throughput-scale-up.yaml + extra_args: + vports: {{ vports }} + topology: vfw-tg-topology-scale-up.yaml + +A example traffic profile template is: + +.. literalinclude:: /submodules/yardstick/samples/vnf_samples/traffic_profiles/ipv4_throughput-scale-up.yaml + :language: yaml + +There is an option to provide predefined config for SampleVNFs. Path to config +file may by specified in ``vnf_config`` scenario section. + +.. code-block:: yaml + + vnf__0: + rules: acl_1rule.yaml + vnf_config: {lb_config: 'SW', file: vfw_vnf_pipeline_cores_4_ports_2_lb_1_sw.conf } + + +Baremetal +^^^^^^^^^ + 1. Follow above traffic generator section to setup. + 2. Edit num of threads in + ``/samples/vnf_samples/nsut/vfw/tc_baremetal_rfc2544_ipv4_1rule_1flow_64B_trex_scale_up.yaml`` + e.g, 6 Threads for given VNF + +.. code-block:: yaml + + + schema: yardstick:task:0.1 + scenarios: + {% for worker_thread in [1, 2 ,3 , 4, 5, 6] %} + - type: NSPerf + traffic_profile: ../../traffic_profiles/ipv4_throughput.yaml + topology: vfw-tg-topology.yaml + nodes: + tg__0: trafficgen_1.yardstick + vnf__0: vnf.yardstick + options: + framesize: + uplink: {64B: 100} + downlink: {64B: 100} + flow: + src_ip: [{'tg__0': 'xe0'}] + dst_ip: [{'tg__0': 'xe1'}] + count: 1 + traffic_type: 4 + rfc2544: + allowed_drop_rate: 0.0001 - 0.0001 + vnf__0: + rules: acl_1rule.yaml + vnf_config: {lb_config: 'HW', lb_count: 1, worker_config: '1C/1T', worker_threads: {{worker_thread}}} + nfvi_enable: True + runner: + type: Iteration + iterations: 10 + interval: 35 + {% endfor %} + context: + type: Node + name: yardstick + nfvi_type: baremetal + file: /etc/yardstick/nodes/pod.yaml + +Scale-Out +-------------------- + +VNFs performance data with scale-out helps + + * in capacity planning to meet the given network node requirements + * in comparison between different VNF vendor offerings + * better the scale-out index, provides the flexibility in meeting future + capacity requirements + + +Standalone +^^^^^^^^^^ + +Scale-out not supported on Baremetal. + +1. Follow above traffic generator section to setup. +2. Generate testcase for standalone virtualization using ansible scripts + + .. code-block:: console + + cd /ansible + trex: standalone_ovs_scale_out_trex_test.yaml or standalone_sriov_scale_out_trex_test.yaml + ixia: standalone_ovs_scale_out_ixia_test.yaml or standalone_sriov_scale_out_ixia_test.yaml + ixia_correlated: standalone_ovs_scale_out_ixia_correlated_test.yaml or standalone_sriov_scale_out_ixia_correlated_test.yaml + + update the ovs_dpdk or sriov above Ansible scripts reflect the setup + +3. run the test + + .. code-block:: console + + /samples/vnf_samples/nsut/tc_sriov_vfw_udp_ixia_correlated_scale_out-1.yaml + /samples/vnf_samples/nsut/tc_sriov_vfw_udp_ixia_correlated_scale_out-2.yaml + +Heat +^^^^ + +There are sample scale-out all-VM Heat tests. These tests only use VMs and +don't use external traffic. + +The tests use UDP_Replay and correlated traffic. + +.. code-block:: console + + /samples/vnf_samples/nsut/cgnapt/tc_heat_rfc2544_ipv4_1flow_64B_trex_correlated_scale_4.yaml + +To run the test you need to increase OpenStack CPU, Memory and Port quotas. + + +Traffic Generator tuning +------------------------ + +The TRex traffic generator can be setup to use multiple threads per core, this +is for multiqueue testing. + +TRex does not automatically enable multiple threads because we currently cannot +detect the number of queues on a device. + +To enable multiple queue set the ``queues_per_port`` value in the TG VNF +options section. + +.. code-block:: yaml + + scenarios: + - type: NSPerf + nodes: + tg__0: tg_0.yardstick + + options: + tg_0: + queues_per_port: 2 diff --git a/docs/testing/user/userguide/15-list-of-tcs.rst b/docs/testing/user/userguide/15-list-of-tcs.rst index cb99c49cf..678f0f9a9 100644 --- a/docs/testing/user/userguide/15-list-of-tcs.rst +++ b/docs/testing/user/userguide/15-list-of-tcs.rst @@ -14,11 +14,11 @@ This chapter lists available Yardstick test cases. Yardstick test cases are divided in two main categories: * *Generic NFVI Test Cases* - Test Cases developed to realize the methodology -described in :doc:`02-methodology` + described in :doc:`02-methodology` * *OPNFV Feature Test Cases* - Test Cases developed to verify one or more -aspect of a feature delivered by an OPNFV Project, including the test cases -developed for the :term:`VTC`. + aspect of a feature delivered by an OPNFV Project, including the test cases + developed for the :term:`VTC`. Generic NFVI Test Case Descriptions =================================== @@ -109,8 +109,8 @@ Parser opnfv_yardstick_tc040.rst - StorPerf ------------ +StorPerf +-------- .. toctree:: :maxdepth: 1 diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst index 61e157e52..57c6cf594 100644 --- a/docs/testing/user/userguide/index.rst +++ b/docs/testing/user/userguide/index.rst @@ -17,15 +17,15 @@ Yardstick User Guide 02-methodology 03-architecture 04-installation - 05-yardstick_plugin - 06-result-store-InfluxDB - 07-grafana - 08-api - 09-yardstick_user_interface - 10-vtc-overview - 11-nsb-overview - 12-nsb_installation - 13-nsb_operation + 06-yardstick-plugin + 07-result-store-InfluxDB + 08-grafana + 09-api + 10-yardstick-user-interface + 11-vtc-overview + 12-nsb-overview + 13-nsb-installation + 14-nsb-operation 15-list-of-tcs nsb/nsb-list-of-tcs glossary -- cgit 1.2.3-korg