diff options
Diffstat (limited to 'docs/userguide')
| -rw-r--r-- | docs/userguide/02-methodology.rst | 20 | ||||
| -rw-r--r-- | docs/userguide/07-installation.rst | 159 | ||||
| -rw-r--r-- | docs/userguide/10-grafana.rst | 119 | ||||
| -rw-r--r-- | docs/userguide/11-list-of-tcs.rst (renamed from docs/userguide/10-list-of-tcs.rst) | 0 | ||||
| -rw-r--r-- | docs/userguide/images/TC002.png | bin | 0 -> 106382 bytes | |||
| -rw-r--r-- | docs/userguide/images/add.png | bin | 0 -> 169904 bytes | |||
| -rw-r--r-- | docs/userguide/images/login.png | bin | 0 -> 32761 bytes | |||
| -rw-r--r-- | docs/userguide/index.rst | 3 | ||||
| -rw-r--r-- | docs/userguide/opnfv_yardstick_tc043.rst | 2 | ||||
| -rw-r--r-- | docs/userguide/opnfv_yardstick_tc073.rst | 2 | ||||
| -rw-r--r-- | docs/userguide/references.rst | 6 |
11 files changed, 255 insertions, 56 deletions
diff --git a/docs/userguide/02-methodology.rst b/docs/userguide/02-methodology.rst index 1849ffd0c..34d271095 100644 --- a/docs/userguide/02-methodology.rst +++ b/docs/userguide/02-methodology.rst @@ -170,17 +170,19 @@ options). +---------+-------------------+----------------+------------------------------+ | Compute | TC003 [1]_ | TC003 [1]_ | TC013 [1]_ | | | TC004 | TC004 | TC015 [1]_ | -| | TC014 | TC010 | | -| | TC024 | TC012 | | -| | TC069 | TC055 | | +| | TC010 | TC024 | | +| | TC012 | TC055 | | +| | TC014 | | | +| | TC069 | | | +---------+-------------------+----------------+------------------------------+ -| Network | TC001 | TC001 | TC016 [1]_ | -| | TC002 | TC008 | TC018 [1]_ | -| | TC011 | TC009 | | -| | TC073 | TC075 | | +| Network | TC001 | TC044 | TC016 [1]_ | +| | TC002 | TC073 | TC018 [1]_ | +| | TC009 | TC075 | | +| | TC011 | | | +| | TC042 | | | +| | TC043 | | | +---------+-------------------+----------------+------------------------------+ -| Storage | TC005 | TC005 | TC017 [1]_ | -| | | TC063 | | +| Storage | TC005 | TC063 | TC017 [1]_ | +---------+-------------------+----------------+------------------------------+ .. note:: The description in this OPNFV document is intended as a reference for diff --git a/docs/userguide/07-installation.rst b/docs/userguide/07-installation.rst index aa45b61af..fc68d3931 100644 --- a/docs/userguide/07-installation.rst +++ b/docs/userguide/07-installation.rst @@ -9,22 +9,52 @@ Yardstick Installation Abstract -------- -Yardstick supports installation on Ubuntu 14.04 or by using a Docker image. -The installation procedure on Ubuntu 14.04 or via the docker image are -detailed in the section below. +Yardstick supports installation on Ubuntu 14.04 or via a Docker image. The +installation procedure on Ubuntu 14.04 or via the docker image are detailed in +the section below. -To use Yardstick you should have access to an OpenStack environment, -with at least Nova, Neutron, Glance, Keystone and Heat installed. +To use Yardstick you should have access to an OpenStack environment, with at +least Nova, Neutron, Glance, Keystone and Heat installed. The steps needed to run Yardstick are: 1. Install Yardstick. -2. Create the test configuration .yaml file. -3. Build a guest image。 -4. Load the image into the OpenStack environment. -5. Create a Neutron external network. -6. Load OpenStack environment variables. -6. Run the test case. +2. Load OpenStack environment variables. +3. Create a Neutron external network. +4. Build Yardstick flavor and a guest image. +5. Load the guest image into the OpenStack environment. +6. Create the test configuration .yaml file. +7. Run the test case. + + +Prerequisites +------------- + +The OPNFV deployment is out of the scope of this document but it can be +found in http://artifacts.opnfv.org/opnfvdocs/colorado/docs/configguide/index.html. +The OPNFV platform is considered as the System Under Test (SUT) in this +document. + +Several prerequisites are needed for Yardstick: + + #. A Jumphost to run Yardstick on + #. A Docker daemon shall be installed on the Jumphost + #. A public/external network created on the SUT + #. Connectivity from the Jumphost to the SUT public/external network + +WARNING: Connectivity from Jumphost is essential and it is of paramount +importance to make sure it is working before even considering to install +and run Yardstick. Make also sure you understand how your networking is +designed to work. + +NOTE: **Jumphost** refers to any server which meets the previous +requirements. Normally it is the same server from where the OPNFV +deployment has been triggered previously. + +NOTE: If your Jumphost is operating behind a company http proxy and/or +Firewall, please consult first the section `Proxy Support`_, towards +the end of this document. The section details some tips/tricks which +*may* be of help in a proxified environment. Installing Yardstick on Ubuntu 14.04 @@ -33,9 +63,9 @@ Installing Yardstick on Ubuntu 14.04 .. _install-framework: You can install Yardstick framework directly on Ubuntu 14.04 or in an Ubuntu -14.04 Docker image. -No matter which way you choose to install Yardstick framework, the following -installation steps are identical. +14.04 Docker image. No matter which way you choose to install Yardstick +framework, the following installation steps are identical. + If you choose to use the Ubuntu 14.04 Docker image, You can pull the Ubuntu 14.04 Docker image from Docker hub: @@ -91,47 +121,42 @@ at: http://www.youtube.com/watch?v=4S4izNolmR0 Installing Yardstick using Docker --------------------------------- -Yardstick iteself has a Docker image, this Docker image (**Yardstick-stable**) +Yardstick has a Docker image, this Docker image (**Yardstick-stable**) serves as a replacement for installing the Yardstick framework in a virtual environment (for example as done in :ref:`install-framework`). It is recommended to use this Docker image to run Yardstick test. -Yardstick-stable image -^^^^^^^^^^^^^^^^^^^^^^ -Pull the Yardstick-stable Docker image from Docker hub: +Pulling the Yardstick Docker image +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -:: +.. _dockerhub: https://hub.docker.com/r/opnfv/yardstick/ + +Pull the Yardstick Docker image ('opnfv/yardstick') from the public dockerhub +registry under the OPNFV account: [dockerhub_], with the following docker +command:: docker pull opnfv/yardstick:stable +After pulling the Docker image, check that it is available with the +following docker command:: + + [yardsticker@jumphost ~]$ docker images + REPOSITORY TAG IMAGE ID CREATED SIZE + opnfv/yardstick stable a4501714757a 1 day ago 915.4 MB + Run the Docker image: :: - docker run --privileged=true -it openfv/yardstick /bin/bash + docker run --privileged=true -it opnfv/yardstick:stable /bin/bash -In the container run yardstick task command to execute a test case. -Before executing Yardstick test case, make sure that yardstick-trusty-server -image and yardstick flavor is available in OpenStack. -Detailed steps about creating yardstick flavor and building yardstick-trusty-server -image can be found below. +In the container the Yardstick repository is located in the /home/opnfv/repos +directory. OpenStack parameters and credentials ------------------------------------ -Yardstick-flavor -^^^^^^^^^^^^^^^^ -Most of the sample test cases in Yardstick are using an OpenStack flavor called -*yardstick-flavor* which deviates from the OpenStack standard m1.tiny flavor by the -disk size - instead of 1GB it has 3GB. Other parameters are the same as in m1.tiny. - -Create yardstick-flavor: - -:: - - nova flavor-create yardstick-flavor 100 512 3 1 - Environment variables ^^^^^^^^^^^^^^^^^^^^^ Before running Yardstick it is necessary to export OpenStack environment variables @@ -146,14 +171,46 @@ Credential environment variables in the *openrc* file have to include at least: * OS_PASSWORD * OS_TENANT_NAME +A sample openrc file may look like this: + +* export OS_PASSWORD=console +* export OS_TENANT_NAME=admin +* export OS_AUTH_URL=http://172.16.1.222:35357/v2.0 +* export OS_USERNAME=admin +* export OS_VOLUME_API_VERSION=2 +* export EXTERNAL_NETWORK=net04_ext + + +Yardstick falvor and guest images +--------------------------------- + +Before executing Yardstick test cases, make sure that yardstick guest image and +yardstick flavor are available in OpenStack. +Detailed steps about creating yardstick flavor and building yardstick-trusty-server +image can be found below. + +Yardstick-flavor +^^^^^^^^^^^^^^^^ +Most of the sample test cases in Yardstick are using an OpenStack flavor called +*yardstick-flavor* which deviates from the OpenStack standard m1.tiny flavor by the +disk size - instead of 1GB it has 3GB. Other parameters are the same as in m1.tiny. + +Create yardstick-flavor: + +:: + + nova flavor-create yardstick-flavor 100 512 3 1 + .. _guest-image: Building a guest image ^^^^^^^^^^^^^^^^^^^^^^ -Yardstick has a tool for building an Ubuntu Cloud Server image containing all -the required tools to run test cases supported by Yardstick. It is necessary to -have sudo rights to use this tool. +Most of the sample test cases in Yardstick are using a guest image called +*yardstick-trusty-server* which deviates from an Ubuntu Cloud Server image +containing all the required tools to run test cases supported by Yardstick. +Yardstick has a tool for building this custom image. It is necessary to have +sudo rights to use this tool. Also you may need install several additional packages to use this tool, by follwing the commands below: @@ -176,8 +233,10 @@ by following the commands above): **Warning:** the script will create files by default in: ``/tmp/workspace/yardstick`` and the files will be owned by root! + If you are building this guest image in inside a docker container make sure the container is granted with privilege. + The created image can be added to OpenStack using the ``glance image-create`` or via the OpenStack Dashboard. @@ -190,6 +249,24 @@ Example command: --disk-format qcow2 --container-format bare \ --file /tmp/workspace/yardstick/yardstick-trusty-server.img +Some Yardstick test cases use a Cirros image, you can find one at +http://download.cirros-cloud.net/0.3.3/cirros-0.3.3-x86_64-disk.img + + +Automatic flavor and image creation +----------------------------------- +Yardstick has a script for automatic creating yardstick flavor and building +guest images. This script is mainly used in CI, but you can still use it in +your local environment. + +Example command: + +:: + + export YARD_IMG_ARCH="amd64" + sudo echo "Defaults env_keep += \"YARD_IMG_ARCH\"" >> /etc/sudoers + source $YARDSTICK_REPO_DIR/tests/ci/load_images.sh + Yardstick default key pair ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -322,7 +399,7 @@ here, you must specify the schema and the name, then you just need to list the test cases in the tag "test_cases" and also mark their relative directory in the tag "test_cases_dir". -Yardstick test suite also support constraints and task args for each test suite. +Yardstick test suite also support constraints and task args for each test case. Here is another sample to show this, which is digested from one big test suite. os-nosdn-nofeature-ha.yaml diff --git a/docs/userguide/10-grafana.rst b/docs/userguide/10-grafana.rst new file mode 100644 index 000000000..416857b71 --- /dev/null +++ b/docs/userguide/10-grafana.rst @@ -0,0 +1,119 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2016 Huawei Technologies Co.,Ltd and others + +================= +Grafana dashboard +================= + + +Abstract +======== + +This chapter describes the Yardstick grafana dashboard. The Yardstick grafana +dashboard can be found here: http://testresults.opnfv.org/grafana/ + + +.. image:: images/login.png + :width: 800px + :alt: Yardstick grafana dashboard + + +Public access +============= + +Yardstick provids a public account for accessing to the dashboard. The username +and password are both set to ‘opnfv’. + + +Testcase dashboard +================== + +For each test case, there is a dedicated dashboard. Shown here is the dashboard +of TC002. + + +.. image:: images/TC002.png + :width: 800px + :alt:TC002 dashboard + +For each test case dashboard. On the top left, we have a dashboard selection, +you can switch to different test cases using this pull-down menu. + +Underneath, we have a pod and scenario selection. +All the pods and scenarios that have ever published test data to the InfluxDB +will be shown here. + +You can check multiple pods or scenarios. + +For each test case, we have a short description and a link to detailed test +case information in Yardstick user guide. + +Underneath, it is the result presentation section. +You can use the time period selection on the top right corner to zoom in or +zoom out the chart. + + +Administration access +===================== + +For a user with administration rights it is easy to update and save any +dashboard configuration. Saved updates immediately take effect and become live. +This may cause issues like: + +- Changes and updates made to the live configuration in Grafana can compromise + existing Grafana content in an unwanted, unpredicted or incompatible way. + Grafana as such is not version controlled, there exists one single Grafana + configuration per dashboard. +- There is a risk several people can disturb each other when doing updates to + the same Grafana dashboard at the same time. + +Any change made by administrator should be careful. + + +Add a dashboard into yardstick grafana +====================================== + +Due to security concern, users that using the public opnfv account are not able +to edit the yardstick grafana directly.It takes a few more steps for a +non-yardstick user to add a custom dashboard into yardstick grafana. + +There are 6 steps to go. + + +.. image:: images/add.png + :width: 800px + :alt: Add a dashboard into yardstick grafana + + +1. You need to build a local influxdb and grafana, so you can do the work + locally. You can refer to How to deploy InfluxDB and Grafana locally wiki + page about how to do this. + +2. Once step one is done, you can fetch the existing grafana dashboard + configuration file from the yardstick repository and import it to your local + grafana. After import is done, you grafana dashboard will be ready to use + just like the community’s dashboard. + +3. The third step is running some test cases to generate test results and + publishing it to your local influxdb. + +4. Now you have some data to visualize in your dashboard. In the fourth step, + it is time to create your own dashboard. You can either modify an existing + dashboard or try to create a new one from scratch. If you choose to modify + an existing dashboard then in the curtain menu of the existing dashboard do + a "Save As..." into a new dashboard copy instance, and then continue doing + all updates and saves within the dashboard copy. + +5. When finished with all Grafana configuration changes in this temporary + dashboard then chose "export" of the updated dashboard copy into a JSON file + and put it up for review in Gerrit, in file /yardstick/dashboard/Yardstick-TCxxx-yyyyyyyyyyyyy. + For instance a typical default name of the file would be "Yardstick-TC001 Copy-1234567891234". + +6. Once you finish your dashboard, the next step is exporting the configuration + file and propose a patch into Yardstick. Yardstick team will review and + merge it into Yardstick repository. After approved review Yardstick team + will do an "import" of the JSON file and also a "save dashboard" as soon as + possible to replace the old live dashboard configuration. + diff --git a/docs/userguide/10-list-of-tcs.rst b/docs/userguide/11-list-of-tcs.rst index 8798a8f51..8798a8f51 100644 --- a/docs/userguide/10-list-of-tcs.rst +++ b/docs/userguide/11-list-of-tcs.rst diff --git a/docs/userguide/images/TC002.png b/docs/userguide/images/TC002.png Binary files differnew file mode 100644 index 000000000..89154efcc --- /dev/null +++ b/docs/userguide/images/TC002.png diff --git a/docs/userguide/images/add.png b/docs/userguide/images/add.png Binary files differnew file mode 100644 index 000000000..a88a1b146 --- /dev/null +++ b/docs/userguide/images/add.png diff --git a/docs/userguide/images/login.png b/docs/userguide/images/login.png Binary files differnew file mode 100644 index 000000000..045e010e4 --- /dev/null +++ b/docs/userguide/images/login.png diff --git a/docs/userguide/index.rst b/docs/userguide/index.rst index 0aa112a45..60e1340ac 100644 --- a/docs/userguide/index.rst +++ b/docs/userguide/index.rst @@ -19,6 +19,7 @@ Yardstick Overview 07-installation 08-yardstick_plugin 09-result-store-InfluxDB - 10-list-of-tcs + 10-grafana + 11-list-of-tcs glossary references diff --git a/docs/userguide/opnfv_yardstick_tc043.rst b/docs/userguide/opnfv_yardstick_tc043.rst index b6e557d86..59d7c6993 100644 --- a/docs/userguide/opnfv_yardstick_tc043.rst +++ b/docs/userguide/opnfv_yardstick_tc043.rst @@ -13,7 +13,7 @@ Yardstick Test Case Description TC043 |Network Latency Between NFVI Nodes | | | +--------------+--------------------------------------------------------------+ -|test case id | OPNFV_YARDSTICK_TC043_Latency_between_NFVI_nodes_ | +|test case id | OPNFV_YARDSTICK_TC043_Latency_between_NFVI_nodes | | | measurements | | | | +--------------+--------------------------------------------------------------+ diff --git a/docs/userguide/opnfv_yardstick_tc073.rst b/docs/userguide/opnfv_yardstick_tc073.rst index a6499eabb..ad4526405 100644 --- a/docs/userguide/opnfv_yardstick_tc073.rst +++ b/docs/userguide/opnfv_yardstick_tc073.rst @@ -37,7 +37,7 @@ Yardstick Test Case Description TC073 | | For SLA max_mean_latency is set to 100. | | | | +--------------+--------------------------------------------------------------+ -|test tool | netperf | +|test tool | netperf_ | | | Netperf is a software application that provides network | | | bandwidth testing between two hosts on a network. It | | | supports Unix domain sockets, TCP, SCTP, DLPI and UDP via | diff --git a/docs/userguide/references.rst b/docs/userguide/references.rst index 7f6a0f0f0..05729ba75 100644 --- a/docs/userguide/references.rst +++ b/docs/userguide/references.rst @@ -15,8 +15,8 @@ OPNFV * Pharos wiki: https://wiki.opnfv.org/pharos * VTC: https://wiki.opnfv.org/vtc * Yardstick CI: https://build.opnfv.org/ci/view/yardstick/ -* Yardstick and ETSI TST001 presentation: https://wiki.opnfv.org/_media/opnfv_summit_-_bridging_opnfv_and_etsi.pdf -* Yardstick Project presentation: https://wiki.opnfv.org/_media/opnfv_summit_-_yardstick_project.pdf +* Yardstick and ETSI TST001 presentation: https://wiki.opnfv.org/display/yardstick/Yardstick?preview=%2F2925202%2F2925205%2Fopnfv_summit_-_bridging_opnfv_and_etsi.pdf +* Yardstick Project presentation: https://wiki.opnfv.org/display/yardstick/Yardstick?preview=%2F2925202%2F2925208%2Fopnfv_summit_-_yardstick_project.pdf * Yardstick wiki: https://wiki.opnfv.org/yardstick References used in Test Cases @@ -55,6 +55,6 @@ Standards ========= * ETSI NFV: http://www.etsi.org/technologies-clusters/technologies/nfv -* ETSI GS-NFV TST 001: https://docbox.etsi.org/ISG/NFV/Open/Drafts/TST001_-_Pre-deployment_Validation/ +* ETSI GS-NFV TST 001: http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_NFV-TST001v010101p.pdf * RFC2544: https://www.ietf.org/rfc/rfc2544.txt |
