diff options
Diffstat (limited to 'docs/userguide')
| -rwxr-xr-x | docs/userguide/01-introduction.rst | 20 | ||||
| -rw-r--r-- | docs/userguide/02-methodology.rst | 27 | ||||
| -rwxr-xr-x | docs/userguide/03-architecture.rst | 7 | ||||
| -rw-r--r-- | docs/userguide/07-installation.rst | 331 | ||||
| -rw-r--r-- | docs/userguide/08-yardstick_plugin.rst | 8 | ||||
| -rw-r--r-- | docs/userguide/09-result-store-InfluxDB.rst | 4 | ||||
| -rw-r--r-- | docs/userguide/10-grafana.rst | 119 | ||||
| -rw-r--r-- | docs/userguide/11-list-of-tcs.rst (renamed from docs/userguide/10-list-of-tcs.rst) | 20 | ||||
| -rw-r--r-- | docs/userguide/images/TC002.png | bin | 0 -> 106382 bytes | |||
| -rw-r--r-- | docs/userguide/images/add.png | bin | 0 -> 169904 bytes | |||
| -rw-r--r-- | docs/userguide/images/login.png | bin | 0 -> 32761 bytes | |||
| -rw-r--r-- | docs/userguide/index.rst | 3 | ||||
| -rw-r--r-- | docs/userguide/opnfv_yardstick_tc043.rst | 2 | ||||
| -rw-r--r-- | docs/userguide/opnfv_yardstick_tc053.rst | 2 | ||||
| -rw-r--r-- | docs/userguide/opnfv_yardstick_tc073.rst | 2 | ||||
| -rw-r--r-- | docs/userguide/opnfv_yardstick_tc074.rst | 4 | ||||
| -rw-r--r-- | docs/userguide/references.rst | 18 |
17 files changed, 371 insertions, 196 deletions
diff --git a/docs/userguide/01-introduction.rst b/docs/userguide/01-introduction.rst index 3db6ce001..9d9cf0fb5 100755 --- a/docs/userguide/01-introduction.rst +++ b/docs/userguide/01-introduction.rst @@ -40,18 +40,25 @@ This document consists of the following chapters: * Chapter :doc:`02-methodology` describes the methodology implemented by the Yardstick Project for :term:`NFVI` verification. -* Chapter :doc:`architecture` provides information on the software architecture +* Chapter :doc:`03-architecture` provides information on the software architecture of yardstick. + * Chapter :doc:`04-vtc-overview` provides information on the :term:`VTC`. -* Chapter :doc:`apexlake_installation` provides instructions to install the - experimental framework *ApexLake* and chapter :doc:`apexlake_api` explains +* Chapter :doc:`05-apexlake_installation` provides instructions to install the + experimental framework *ApexLake* and chapter :doc:`06-apexlake_api` explains how this framework is integrated in *Yardstick*. -* Chapter :doc:`03-installation` provides instructions to install *Yardstick*. +* Chapter :doc:`07-installation` provides instructions to install *Yardstick*. + +* Chapter :doc:`08-yardstick_plugin` provides information on how to integrate + other OPNFV testing projects into *Yardstick*. -* Chapter :doc:`03-list-of-tcs` includes a list of available Yardstick - test cases. +* Chapter :doc:`09-result-store-InfluxDB` provides inforamtion on how to run + plug-in test cases and store test results into community's InfluxDB. + +* Chapter :doc:`10-list-of-tcs` includes a list of available Yardstick test + cases. Contact Yardstick @@ -60,3 +67,4 @@ Contact Yardstick Feedback? `Contact us`_ .. _Contact us: opnfv-users@lists.opnfv.org + diff --git a/docs/userguide/02-methodology.rst b/docs/userguide/02-methodology.rst index 3fa432a98..34d271095 100644 --- a/docs/userguide/02-methodology.rst +++ b/docs/userguide/02-methodology.rst @@ -59,7 +59,7 @@ The metrics, as defined by ETSI GS NFV-TST001, are shown in :ref:`Table1 <table2_1>`, :ref:`Table2 <table2_2>` and :ref:`Table3 <table2_3>`. -In OPNFV Brahmaputra release, generic test cases covering aspects of the listed +In OPNFV Colorado release, generic test cases covering aspects of the listed metrics are available; further OPNFV releases will provide extended testing of these metrics. The view of available Yardstick test cases cross ETSI definitions in @@ -169,24 +169,27 @@ options). | | | | | +---------+-------------------+----------------+------------------------------+ | Compute | TC003 [1]_ | TC003 [1]_ | TC013 [1]_ | -| | TC004 [1]_ | TC004 [1]_ | TC015 [1]_ | -| | TC014 | TC010 | | -| | TC024 | TC012 | | -| | | | | +| | TC004 | TC004 | TC015 [1]_ | +| | TC010 | TC024 | | +| | TC012 | TC055 | | +| | TC014 | | | +| | TC069 | | | +---------+-------------------+----------------+------------------------------+ -| Network | TC002 | TC001 | TC016 [1]_ | -| | TC011 | TC008 | TC018 [1]_ | -| | | TC009 | | -| | | | | +| Network | TC001 | TC044 | TC016 [1]_ | +| | TC002 | TC073 | TC018 [1]_ | +| | TC009 | TC075 | | +| | TC011 | | | +| | TC042 | | | +| | TC043 | | | +---------+-------------------+----------------+------------------------------+ -| Storage | TC005 | TC005 | TC017 [1]_ | -| | | | | +| Storage | TC005 | TC063 | TC017 [1]_ | +---------+-------------------+----------------+------------------------------+ .. note:: The description in this OPNFV document is intended as a reference for users to understand the scope of the Yardstick Project and the deliverables of the Yardstick framework. For complete description of - the methodology, refer to the ETSI document. + the methodology, please refer to the ETSI document. .. rubric:: Footnotes .. [1] To be included in future deliveries. + diff --git a/docs/userguide/03-architecture.rst b/docs/userguide/03-architecture.rst index 3abb67b7d..ace3117c2 100755 --- a/docs/userguide/03-architecture.rst +++ b/docs/userguide/03-architecture.rst @@ -222,7 +222,7 @@ Deployment View =============== Yardstick deployment view shows how the yardstick tool can be deployed into the underlying platform. Generally, yardstick tool is installed on JumpServer(see -`03-installation` for detail installation steps), and JumpServer is +`07-installation` for detail installation steps), and JumpServer is connected with other control/compute servers by networking. Based on this deployment, yardstick can run the test cases on these hosts, and get the test result for better showing. @@ -256,8 +256,11 @@ Yardstick Directory structure by Heat. Currently contains how to build the yardstick-trusty-server image with the different tools that are needed from within the image. +*plugin/* - Plug-in configuration files are stored here. + *vTC/* - Contains the files for running the virtual Traffic Classifier tests. *yardstick/* - Contains the internals of Yardstick: Runners, Scenario, Contexts, - CLI parsing, keys, plotting tools, dispatcher and so on. + CLI parsing, keys, plotting tools, dispatcher, plugin + install/remove scripts and so on. diff --git a/docs/userguide/07-installation.rst b/docs/userguide/07-installation.rst index 475719c72..9c2082a27 100644 --- a/docs/userguide/07-installation.rst +++ b/docs/userguide/07-installation.rst @@ -9,19 +9,52 @@ Yardstick Installation Abstract -------- -Yardstick currently supports installation on Ubuntu 14.04 or by using a Docker -image. Detailed steps about installing Yardstick using both of these options -can be found below. +Yardstick supports installation on Ubuntu 14.04 or via a Docker image. The +installation procedure on Ubuntu 14.04 or via the docker image are detailed in +the section below. -To use Yardstick you should have access to an OpenStack environment, -with at least Nova, Neutron, Glance, Keystone and Heat installed. +To use Yardstick you should have access to an OpenStack environment, with at +least Nova, Neutron, Glance, Keystone and Heat installed. The steps needed to run Yardstick are: -1. Install Yardstick and create the test configuration .yaml file. -2. Build a guest image and load the image into the OpenStack environment. -3. Create a Neutron external network and load OpenStack environment variables. -4. Run the test case. +1. Install Yardstick. +2. Load OpenStack environment variables. +3. Create a Neutron external network. +4. Build Yardstick flavor and a guest image. +5. Load the guest image into the OpenStack environment. +6. Create the test configuration .yaml file. +7. Run the test case. + + +Prerequisites +------------- + +The OPNFV deployment is out of the scope of this document but it can be +found in http://artifacts.opnfv.org/opnfvdocs/colorado/docs/configguide/index.html. +The OPNFV platform is considered as the System Under Test (SUT) in this +document. + +Several prerequisites are needed for Yardstick: + + #. A Jumphost to run Yardstick on + #. A Docker daemon shall be installed on the Jumphost + #. A public/external network created on the SUT + #. Connectivity from the Jumphost to the SUT public/external network + +WARNING: Connectivity from Jumphost is essential and it is of paramount +importance to make sure it is working before even considering to install +and run Yardstick. Make also sure you understand how your networking is +designed to work. + +NOTE: **Jumphost** refers to any server which meets the previous +requirements. Normally it is the same server from where the OPNFV +deployment has been triggered previously. + +NOTE: If your Jumphost is operating behind a company http proxy and/or +Firewall, please consult first the section `Proxy Support`_, towards +the end of this document. The section details some tips/tricks which +*may* be of help in a proxified environment. Installing Yardstick on Ubuntu 14.04 @@ -29,195 +62,177 @@ Installing Yardstick on Ubuntu 14.04 .. _install-framework: -Installing Yardstick framework -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Install dependencies: -:: +You can install Yardstick framework directly on Ubuntu 14.04 or in an Ubuntu +14.04 Docker image. No matter which way you choose to install Yardstick +framework, the following installation steps are identical. + +If you choose to use the Ubuntu 14.04 Docker image, You can pull the Ubuntu +14.04 Docker image from Docker hub: - sudo apt-get update && sudo apt-get install -y \ - wget \ - git \ - sshpass \ - qemu-utils \ - kpartx \ - libffi-dev \ - libssl-dev \ - python \ - python-dev \ - python-virtualenv \ - libxml2-dev \ - libxslt1-dev \ - python-setuptools - -Create a python virtual environment, source it and update setuptools: :: - virtualenv ~/yardstick_venv - source ~/yardstick_venv/bin/activate - easy_install -U setuptools + docker pull ubuntu:14.04 +Installing Yardstick framework +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Download source code and install python dependencies: + :: git clone https://gerrit.opnfv.org/gerrit/yardstick cd yardstick - python setup.py install + ./install.sh -There is also a YouTube video, showing the above steps: -.. image:: http://img.youtube.com/vi/4S4izNolmR0/0.jpg - :alt: http://www.youtube.com/watch?v=4S4izNolmR0 - :target: http://www.youtube.com/watch?v=4S4izNolmR0 +Installing Yardstick using Docker +--------------------------------- -Installing extra tools -^^^^^^^^^^^^^^^^^^^^^^ -yardstick-plot -"""""""""""""" -Yardstick has an internal plotting tool ``yardstick-plot``, which can be installed -using the following command: -:: +Yardstick has a Docker image, this Docker image (**Yardstick-stable**) +serves as a replacement for installing the Yardstick framework in a virtual +environment (for example as done in :ref:`install-framework`). +It is recommended to use this Docker image to run Yardstick test. - sudo apt-get install -y g++ libfreetype6-dev libpng-dev pkg-config - python setup.py develop easy_install yardstick[plot] +Pulling the Yardstick Docker image +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. _guest-image: +.. _dockerhub: https://hub.docker.com/r/opnfv/yardstick/ -Building a guest image -^^^^^^^^^^^^^^^^^^^^^^ -Yardstick has a tool for building an Ubuntu Cloud Server image containing all -the required tools to run test cases supported by Yardstick. It is necessary to -have sudo rights to use this tool. +Pull the Yardstick Docker image ('opnfv/yardstick') from the public dockerhub +registry under the OPNFV account: [dockerhub_], with the following docker +command:: -Also you may need install several additional packages to use this tool, by -follwing the commands below: -:: + docker pull opnfv/yardstick:stable - apt-get update && apt-get install -y \ - qemu-utils \ - kpartx +After pulling the Docker image, check that it is available with the +following docker command:: + + [yardsticker@jumphost ~]$ docker images + REPOSITORY TAG IMAGE ID CREATED SIZE + opnfv/yardstick stable a4501714757a 1 day ago 915.4 MB + +Run the Docker image: -This image can be built using the following command while in the directory where -Yardstick is installed (``~/yardstick`` if the framework is installed -by following the commands above): :: - sudo ./tools/yardstick-img-modify tools/ubuntu-server-cloudimg-modify.sh + docker run --privileged=true -it opnfv/yardstick:stable /bin/bash -**Warning:** the script will create files by default in: -``/tmp/workspace/yardstick`` and the files will be owned by root! +In the container the Yardstick repository is located in the /home/opnfv/repos +directory. -The created image can be added to OpenStack using the ``glance image-create`` or -via the OpenStack Dashboard. -Example command: -:: +OpenStack parameters and credentials +------------------------------------ - glance --os-image-api-version 1 image-create \ - --name yardstick-trusty-server --is-public true \ - --disk-format qcow2 --container-format bare \ - --file /tmp/workspace/yardstick/yardstick-trusty-server.img +Environment variables +^^^^^^^^^^^^^^^^^^^^^ +Before running Yardstick it is necessary to export OpenStack environment variables +from the OpenStack *openrc* file (using the ``source`` command) and export the +external network name ``export EXTERNAL_NETWORK="external-network-name"``, +the default name for the external network is ``net04_ext``. +Credential environment variables in the *openrc* file have to include at least: -Installing Yardstick using Docker +* OS_AUTH_URL +* OS_USERNAME +* OS_PASSWORD +* OS_TENANT_NAME + +A sample openrc file may look like this: + +* export OS_PASSWORD=console +* export OS_TENANT_NAME=admin +* export OS_AUTH_URL=http://172.16.1.222:35357/v2.0 +* export OS_USERNAME=admin +* export OS_VOLUME_API_VERSION=2 +* export EXTERNAL_NETWORK=net04_ext + + +Yardstick falvor and guest images --------------------------------- -Yardstick has two Docker images, first one (**Yardstick-framework**) serves as a -replacement for installing the Yardstick framework in a virtual environment (for -example as done in :ref:`install-framework`), while the other image is mostly for -CI purposes (**Yardstick-CI**). +Before executing Yardstick test cases, make sure that yardstick guest image and +yardstick flavor are available in OpenStack. +Detailed steps about creating yardstick flavor and building yardstick-trusty-server +image can be found below. -Yardstick-framework image -^^^^^^^^^^^^^^^^^^^^^^^^^ -Download the source code: +Yardstick-flavor +^^^^^^^^^^^^^^^^ +Most of the sample test cases in Yardstick are using an OpenStack flavor called +*yardstick-flavor* which deviates from the OpenStack standard m1.tiny flavor by the +disk size - instead of 1GB it has 3GB. Other parameters are the same as in m1.tiny. + +Create yardstick-flavor: :: - git clone https://gerrit.opnfv.org/gerrit/yardstick + nova flavor-create yardstick-flavor 100 512 3 1 -Build the Docker image and tag it as *yardstick-framework*: -:: +.. _guest-image: - cd yardstick - docker build -t yardstick-framework . +Building a guest image +^^^^^^^^^^^^^^^^^^^^^^ +Most of the sample test cases in Yardstick are using a guest image called +*yardstick-trusty-server* which deviates from an Ubuntu Cloud Server image +containing all the required tools to run test cases supported by Yardstick. +Yardstick has a tool for building this custom image. It is necessary to have +sudo rights to use this tool. -Run the Docker instance: +Also you may need install several additional packages to use this tool, by +follwing the commands below: :: - docker run --name yardstick_instance -i -t yardstick-framework - -To build a guest image for Yardstick, see :ref:`guest-image`. + apt-get update && apt-get install -y \ + qemu-utils \ + kpartx -Yardstick-CI image -^^^^^^^^^^^^^^^^^^ -Pull the Yardstick-CI Docker image from Docker hub: +This image can be built using the following command while in the directory where +Yardstick is installed (``~/yardstick`` if the framework is installed +by following the commands above): :: - docker pull opnfv/yardstick:$DOCKER_TAG + export YARD_IMG_ARCH="amd64" + sudo echo "Defaults env_keep += \"YARD_IMG_ARCH\"" >> /etc/sudoers + sudo ./tools/yardstick-img-modify tools/ubuntu-server-cloudimg-modify.sh -Where ``$DOCKER_TAG`` is latest for master branch, as for the release branches, -this coincides with its release name, such as brahmaputra.1.0. +**Warning:** the script will create files by default in: +``/tmp/workspace/yardstick`` and the files will be owned by root! -Run the Docker image: +If you are building this guest image in inside a docker container make sure the +container is granted with privilege. + +The created image can be added to OpenStack using the ``glance image-create`` or +via the OpenStack Dashboard. + +Example command: :: - docker run \ - --privileged=true \ - --rm \ - -t \ - -e "INSTALLER_TYPE=${INSTALLER_TYPE}" \ - -e "INSTALLER_IP=${INSTALLER_IP}" \ - opnfv/yardstick \ - exec_tests.sh ${YARDSTICK_DB_BACKEND} ${YARDSTICK_SUITE_NAME} - -Where ``${INSTALLER_TYPE}`` can be apex, compass, fuel or joid, ``${INSTALLER_IP}`` -is the installer master node IP address (i.e. 10.20.0.2 is default for fuel). ``${YARDSTICK_DB_BACKEND}`` -is the IP and port number of DB, ``${YARDSTICK_SUITE_NAME}`` is the test suite you want to run. -For more details, please refer to the Jenkins job defined in Releng project, labconfig information -and sshkey are required. See the link -https://git.opnfv.org/cgit/releng/tree/jjb/yardstick/yardstick-ci-jobs.yml. - -Note: exec_tests.sh is used for executing test suite here, furthermore, if someone wants to execute the -test suite manually, it can be used as long as the parameters are configured correct. Another script -called run_tests.sh is used for unittest in Jenkins verify job, in local manaul environment, -it is recommended to run before test suite execuation. - -Basic steps performed by the **Yardstick-CI** container: - -1. clone yardstick and releng repos -2. setup OS credentials (releng scripts) -3. install yardstick and dependencies -4. build yardstick cloud image and upload it to glance -5. upload cirros-0.3.3 cloud image to glance -6. run yardstick test scenarios -7. cleanup + glance --os-image-api-version 1 image-create \ + --name yardstick-image --is-public true \ + --disk-format qcow2 --container-format bare \ + --file /tmp/workspace/yardstick/yardstick-image.img +Some Yardstick test cases use a Cirros image, you can find one at +http://download.cirros-cloud.net/0.3.3/cirros-0.3.3-x86_64-disk.img -OpenStack parameters and credentials ------------------------------------- -Yardstick-flavor -^^^^^^^^^^^^^^^^ -Most of the sample test cases in Yardstick are using an OpenStack flavor called -*yardstick-flavor* which deviates from the OpenStack standard m1.tiny flavor by the -disk size - instead of 1GB it has 3GB. Other parameters are the same as in m1.tiny. +Automatic flavor and image creation +----------------------------------- +Yardstick has a script for automatic creating yardstick flavor and building +guest images. This script is mainly used in CI, but you can still use it in +your local environment. -Environment variables -^^^^^^^^^^^^^^^^^^^^^ -Before running Yardstick it is necessary to export OpenStack environment variables -from the OpenStack *openrc* file (using the ``source`` command) and export the -external network name ``export EXTERNAL_NETWORK="external-network-name"``, -the default name for the external network is ``net04_ext``. +Example command: -Credential environment variables in the *openrc* file have to include at least: +:: + + export YARD_IMG_ARCH="amd64" + sudo echo "Defaults env_keep += \"YARD_IMG_ARCH\"" >> /etc/sudoers + source $YARDSTICK_REPO_DIR/tests/ci/load_images.sh -* OS_AUTH_URL -* OS_USERNAME -* OS_PASSWORD -* OS_TENANT_NAME Yardstick default key pair ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -230,8 +245,10 @@ Examples and verifying the install ---------------------------------- It is recommended to verify that Yardstick was installed successfully -by executing some simple commands and test samples. Below is an example invocation -of yardstick help command and ping.py test sample: +by executing some simple commands and test samples. Before executing yardstick +test cases make sure yardstick flavor and building yardstick-trusty-server +image can be found in glance and openrc file is sourced. Below is an example +invocation of yardstick help command and ping.py test sample: :: yardstick –h @@ -240,18 +257,8 @@ of yardstick help command and ping.py test sample: Each testing tool supported by Yardstick has a sample configuration file. These configuration files can be found in the **samples** directory. -Example invocation of ``yardstick-plot`` tool: -:: - - yardstick-plot -i /tmp/yardstick.out -o /tmp/plots/ - Default location for the output is ``/tmp/yardstick.out``. -More info about the tool can be found by executing: -:: - - yardstick-plot -h - Deploy InfluxDB and Grafana locally ------------------------------------ @@ -259,6 +266,7 @@ Deploy InfluxDB and Grafana locally .. pull docker images Pull docker images + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ :: @@ -299,6 +307,8 @@ Config grafana log on using admin/admin and config database resource to be {YOUR_IP_HERE}:8086 .. image:: images/Grafana_config.png + :width: 800px + :alt: Grafana data source configration Config yardstick conf ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -319,7 +329,7 @@ Config yardstick.conf username = root password = root -Now you can run yardstick test case and store the results in influxdb +Now you can run yardstick test cases and store the results in influxdb ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -350,12 +360,12 @@ fuel_test_suite.yaml - file_name: iperf3.yaml -As you can see, there are two test cases in fuel_test_suite, the syntas is simple +As you can see, there are two test cases in fuel_test_suite, the syntax is simple here, you must specify the schema and the name, then you just need to list the test cases in the tag "test_cases" and also mark their relative directory in the tag "test_cases_dir". -Yardstick test suite also support constraints and task args for each test suite. +Yardstick test suite also support constraints and task args for each test case. Here is another sample to show this, which is digested from one big test suite. os-nosdn-nofeature-ha.yaml @@ -382,9 +392,10 @@ os-nosdn-nofeature-ha.yaml huawei-pod1: '{"pod_info": "etc/yardstick/.../pod.yaml", "host": "node4.LF","target": "node5.LF"}' -As you can see in test case "opnfv_yardstick_tc043.yaml", it has two tags, "constraint" and +As you can see in test case "opnfv_yardstick_tc043.yaml", there are two tags, "constraint" and "task_args". "constraint" is where you can specify which installer or pod it can be run in the ci environment. "task_args" is where you can specify the task arguments for each pod. All in all, to create a test suite in yardstick, you just need to create a suite yaml file and add test cases and constraint or task arguments if necessary. + diff --git a/docs/userguide/08-yardstick_plugin.rst b/docs/userguide/08-yardstick_plugin.rst index e68db650d..f16dedd02 100644 --- a/docs/userguide/08-yardstick_plugin.rst +++ b/docs/userguide/08-yardstick_plugin.rst @@ -48,11 +48,11 @@ environment and other dependencies: 3. Make sure Jump Host have access to the OpenStack Controller API. 4. Make sure Jump Host must have internet connectivity for downloading docker image. 5. You need to know where to get basic openstack Keystone authorization info, such as -OS_PASSWORD, OS_TENANT_NAME, OS_AUTH_URL, OS_USERNAME. + OS_PASSWORD, OS_TENANT_NAME, OS_AUTH_URL, OS_USERNAME. 6. To run a Storperf container, you need to have OpenStack Controller environment -variables defined and passed to Storperf container. The best way to do this is to -put environment variables in a "storperf_admin-rc" file. The storperf_admin-rc -should include credential environment variables at least: + variables defined and passed to Storperf container. The best way to do this is to + put environment variables in a "storperf_admin-rc" file. The storperf_admin-rc + should include credential environment variables at least: * OS_AUTH_URL * OS_TENANT_ID diff --git a/docs/userguide/09-result-store-InfluxDB.rst b/docs/userguide/09-result-store-InfluxDB.rst index 5c49e9f7c..a0bb48a80 100644 --- a/docs/userguide/09-result-store-InfluxDB.rst +++ b/docs/userguide/09-result-store-InfluxDB.rst @@ -17,7 +17,7 @@ into community's InfluxDB. The framework is shown in Framework_. .. image:: images/InfluxDB_store.png - :width: 1200px + :width: 800px :alt: Store Other Project's Test Results in InfluxDB Store Storperf Test Results into Community's InfluxDB @@ -81,6 +81,6 @@ can be accessed by Login_. .. image:: images/results_visualization.png - :width: 1200px + :width: 800px :alt: results visualization diff --git a/docs/userguide/10-grafana.rst b/docs/userguide/10-grafana.rst new file mode 100644 index 000000000..416857b71 --- /dev/null +++ b/docs/userguide/10-grafana.rst @@ -0,0 +1,119 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2016 Huawei Technologies Co.,Ltd and others + +================= +Grafana dashboard +================= + + +Abstract +======== + +This chapter describes the Yardstick grafana dashboard. The Yardstick grafana +dashboard can be found here: http://testresults.opnfv.org/grafana/ + + +.. image:: images/login.png + :width: 800px + :alt: Yardstick grafana dashboard + + +Public access +============= + +Yardstick provids a public account for accessing to the dashboard. The username +and password are both set to ‘opnfv’. + + +Testcase dashboard +================== + +For each test case, there is a dedicated dashboard. Shown here is the dashboard +of TC002. + + +.. image:: images/TC002.png + :width: 800px + :alt:TC002 dashboard + +For each test case dashboard. On the top left, we have a dashboard selection, +you can switch to different test cases using this pull-down menu. + +Underneath, we have a pod and scenario selection. +All the pods and scenarios that have ever published test data to the InfluxDB +will be shown here. + +You can check multiple pods or scenarios. + +For each test case, we have a short description and a link to detailed test +case information in Yardstick user guide. + +Underneath, it is the result presentation section. +You can use the time period selection on the top right corner to zoom in or +zoom out the chart. + + +Administration access +===================== + +For a user with administration rights it is easy to update and save any +dashboard configuration. Saved updates immediately take effect and become live. +This may cause issues like: + +- Changes and updates made to the live configuration in Grafana can compromise + existing Grafana content in an unwanted, unpredicted or incompatible way. + Grafana as such is not version controlled, there exists one single Grafana + configuration per dashboard. +- There is a risk several people can disturb each other when doing updates to + the same Grafana dashboard at the same time. + +Any change made by administrator should be careful. + + +Add a dashboard into yardstick grafana +====================================== + +Due to security concern, users that using the public opnfv account are not able +to edit the yardstick grafana directly.It takes a few more steps for a +non-yardstick user to add a custom dashboard into yardstick grafana. + +There are 6 steps to go. + + +.. image:: images/add.png + :width: 800px + :alt: Add a dashboard into yardstick grafana + + +1. You need to build a local influxdb and grafana, so you can do the work + locally. You can refer to How to deploy InfluxDB and Grafana locally wiki + page about how to do this. + +2. Once step one is done, you can fetch the existing grafana dashboard + configuration file from the yardstick repository and import it to your local + grafana. After import is done, you grafana dashboard will be ready to use + just like the community’s dashboard. + +3. The third step is running some test cases to generate test results and + publishing it to your local influxdb. + +4. Now you have some data to visualize in your dashboard. In the fourth step, + it is time to create your own dashboard. You can either modify an existing + dashboard or try to create a new one from scratch. If you choose to modify + an existing dashboard then in the curtain menu of the existing dashboard do + a "Save As..." into a new dashboard copy instance, and then continue doing + all updates and saves within the dashboard copy. + +5. When finished with all Grafana configuration changes in this temporary + dashboard then chose "export" of the updated dashboard copy into a JSON file + and put it up for review in Gerrit, in file /yardstick/dashboard/Yardstick-TCxxx-yyyyyyyyyyyyy. + For instance a typical default name of the file would be "Yardstick-TC001 Copy-1234567891234". + +6. Once you finish your dashboard, the next step is exporting the configuration + file and propose a patch into Yardstick. Yardstick team will review and + merge it into Yardstick repository. After approved review Yardstick team + will do an "import" of the JSON file and also a "save dashboard" as soon as + possible to replace the old live dashboard configuration. + diff --git a/docs/userguide/10-list-of-tcs.rst b/docs/userguide/11-list-of-tcs.rst index 7e8c85433..8798a8f51 100644 --- a/docs/userguide/10-list-of-tcs.rst +++ b/docs/userguide/11-list-of-tcs.rst @@ -49,6 +49,7 @@ Generic NFVI Test Case Descriptions opnfv_yardstick_tc070.rst opnfv_yardstick_tc071.rst opnfv_yardstick_tc072.rst + opnfv_yardstick_tc073.rst opnfv_yardstick_tc075.rst OPNFV Feature Test Cases @@ -62,6 +63,16 @@ H A opnfv_yardstick_tc019.rst opnfv_yardstick_tc025.rst + opnfv_yardstick_tc045.rst + opnfv_yardstick_tc046.rst + opnfv_yardstick_tc047.rst + opnfv_yardstick_tc048.rst + opnfv_yardstick_tc049.rst + opnfv_yardstick_tc050.rst + opnfv_yardstick_tc051.rst + opnfv_yardstick_tc052.rst + opnfv_yardstick_tc053.rst + opnfv_yardstick_tc054.rst IPv6 ---- @@ -87,6 +98,14 @@ Parser opnfv_yardstick_tc040.rst + StorPerf +----------- + +.. toctree:: + :maxdepth: 1 + + opnfv_yardstick_tc074.rst + virtual Traffic Classifier -------------------------- @@ -106,3 +125,4 @@ Templates testcase_description_v2_template Yardstick_task_templates + diff --git a/docs/userguide/images/TC002.png b/docs/userguide/images/TC002.png Binary files differnew file mode 100644 index 000000000..89154efcc --- /dev/null +++ b/docs/userguide/images/TC002.png diff --git a/docs/userguide/images/add.png b/docs/userguide/images/add.png Binary files differnew file mode 100644 index 000000000..a88a1b146 --- /dev/null +++ b/docs/userguide/images/add.png diff --git a/docs/userguide/images/login.png b/docs/userguide/images/login.png Binary files differnew file mode 100644 index 000000000..045e010e4 --- /dev/null +++ b/docs/userguide/images/login.png diff --git a/docs/userguide/index.rst b/docs/userguide/index.rst index 0aa112a45..60e1340ac 100644 --- a/docs/userguide/index.rst +++ b/docs/userguide/index.rst @@ -19,6 +19,7 @@ Yardstick Overview 07-installation 08-yardstick_plugin 09-result-store-InfluxDB - 10-list-of-tcs + 10-grafana + 11-list-of-tcs glossary references diff --git a/docs/userguide/opnfv_yardstick_tc043.rst b/docs/userguide/opnfv_yardstick_tc043.rst index b6e557d86..59d7c6993 100644 --- a/docs/userguide/opnfv_yardstick_tc043.rst +++ b/docs/userguide/opnfv_yardstick_tc043.rst @@ -13,7 +13,7 @@ Yardstick Test Case Description TC043 |Network Latency Between NFVI Nodes | | | +--------------+--------------------------------------------------------------+ -|test case id | OPNFV_YARDSTICK_TC043_Latency_between_NFVI_nodes_ | +|test case id | OPNFV_YARDSTICK_TC043_Latency_between_NFVI_nodes | | | measurements | | | | +--------------+--------------------------------------------------------------+ diff --git a/docs/userguide/opnfv_yardstick_tc053.rst b/docs/userguide/opnfv_yardstick_tc053.rst index 8808d12d9..3c6bbc628 100644 --- a/docs/userguide/opnfv_yardstick_tc053.rst +++ b/docs/userguide/opnfv_yardstick_tc053.rst @@ -13,7 +13,7 @@ Yardstick Test Case Description TC053 | | +--------------+--------------------------------------------------------------+ |test case id | OPNFV_YARDSTICK_TC053: OpenStack Controller Load Balance | -| | Service High Availability- | +| | Service High Availability | +--------------+--------------------------------------------------------------+ |test purpose | This test case will verify the high availability of the | | | load balance service(current is HAProxy) that supports | diff --git a/docs/userguide/opnfv_yardstick_tc073.rst b/docs/userguide/opnfv_yardstick_tc073.rst index a6499eabb..ad4526405 100644 --- a/docs/userguide/opnfv_yardstick_tc073.rst +++ b/docs/userguide/opnfv_yardstick_tc073.rst @@ -37,7 +37,7 @@ Yardstick Test Case Description TC073 | | For SLA max_mean_latency is set to 100. | | | | +--------------+--------------------------------------------------------------+ -|test tool | netperf | +|test tool | netperf_ | | | Netperf is a software application that provides network | | | bandwidth testing between two hosts on a network. It | | | supports Unix domain sockets, TCP, SCTP, DLPI and UDP via | diff --git a/docs/userguide/opnfv_yardstick_tc074.rst b/docs/userguide/opnfv_yardstick_tc074.rst index c938f5dfd..92cd51439 100644 --- a/docs/userguide/opnfv_yardstick_tc074.rst +++ b/docs/userguide/opnfv_yardstick_tc074.rst @@ -7,7 +7,7 @@ Yardstick Test Case Description TC074 ************************************* -.. Storperf: https://wiki.opnfv.org/display/storperf/Storperf +.. _Storperf: https://wiki.opnfv.org/display/storperf/Storperf +-----------------------------------------------------------------------------+ |Storperf | @@ -44,7 +44,7 @@ Yardstick Test Case Description TC074 | | * timeout: 600 - maximum allowed job time | | | | +--------------+--------------------------------------------------------------+ -|test tool | Storperf | +|test tool | Storperf_ | | | | | | StorPerf is a tool to measure block and object storage | | | performance in an NFVI. | diff --git a/docs/userguide/references.rst b/docs/userguide/references.rst index 551926135..05729ba75 100644 --- a/docs/userguide/references.rst +++ b/docs/userguide/references.rst @@ -15,25 +15,34 @@ OPNFV * Pharos wiki: https://wiki.opnfv.org/pharos * VTC: https://wiki.opnfv.org/vtc * Yardstick CI: https://build.opnfv.org/ci/view/yardstick/ -* Yardstick and ETSI TST001 presentation: https://wiki.opnfv.org/_media/opnfv_summit_-_bridging_opnfv_and_etsi.pdf -* Yardstick Project presentation: https://wiki.opnfv.org/_media/opnfv_summit_-_yardstick_project.pdf +* Yardstick and ETSI TST001 presentation: https://wiki.opnfv.org/display/yardstick/Yardstick?preview=%2F2925202%2F2925205%2Fopnfv_summit_-_bridging_opnfv_and_etsi.pdf +* Yardstick Project presentation: https://wiki.opnfv.org/display/yardstick/Yardstick?preview=%2F2925202%2F2925208%2Fopnfv_summit_-_yardstick_project.pdf * Yardstick wiki: https://wiki.opnfv.org/yardstick References used in Test Cases ============================= +* cachestat: https://github.com/brendangregg/perf-tools/tree/master/fs * cirros-image: https://download.cirros-cloud.net * cyclictest: https://rt.wiki.kernel.org/index.php/Cyclictest * DPDKpktgen: https://github.com/Pktgen/Pktgen-DPDK/ * DPDK supported NICs: http://dpdk.org/doc/nics +* fdisk: http://www.tldp.org/HOWTO/Partition/fdisk_partitioning.html * fio: http://www.bluestop.org/fio/HOWTO.txt +* free: http://manpages.ubuntu.com/manpages/trusty/en/man1/free.1.html * iperf3: https://iperf.fr/ +* iostat: http://linux.die.net/man/1/iostat * Lmbench man-pages: http://manpages.ubuntu.com/manpages/trusty/lat_mem_rd.8.html * Memory bandwidth man-pages: http://manpages.ubuntu.com/manpages/trusty/bw_mem.8.html -* unixbench: https://github.com/kdlucas/byte-unixbench/blob/master/UnixBench * mpstat man-pages: http://manpages.ubuntu.com/manpages/trusty/man1/mpstat.1.html +* netperf: http://www.netperf.org/netperf/training/Netperf.html * pktgen: https://www.kernel.org/doc/Documentation/networking/pktgen.txt +* RAMspeed: http://alasir.com/software/ramspeed/ +* sar: http://linux.die.net/man/1/sar * SR-IOV: https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking +* Storperf: https://wiki.opnfv.org/display/storperf/Storperf +* unixbench: https://github.com/kdlucas/byte-unixbench/blob/master/UnixBench + Research ======== @@ -46,5 +55,6 @@ Standards ========= * ETSI NFV: http://www.etsi.org/technologies-clusters/technologies/nfv -* ETSI GS-NFV TST 001: https://docbox.etsi.org/ISG/NFV/Open/Drafts/TST001_-_Pre-deployment_Validation/ +* ETSI GS-NFV TST 001: http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_NFV-TST001v010101p.pdf * RFC2544: https://www.ietf.org/rfc/rfc2544.txt + |
