diff options
| -rw-r--r-- | INFO | 5 | ||||
| -rw-r--r-- | docs/results/os-onos-nofeature-ha.rst | 2 | ||||
| -rw-r--r-- | docs/results/results.rst | 2 | ||||
| -rw-r--r-- | docs/results/yardstick-opnfv-ha.rst | 2 | ||||
| -rw-r--r-- | docs/userguide/02-methodology.rst | 20 | ||||
| -rw-r--r-- | docs/userguide/07-installation.rst | 157 | ||||
| -rw-r--r-- | docs/userguide/10-grafana.rst | 86 | ||||
| -rw-r--r-- | docs/userguide/11-list-of-tcs.rst (renamed from docs/userguide/10-list-of-tcs.rst) | 0 | ||||
| -rw-r--r-- | docs/userguide/images/TC002.png | bin | 0 -> 106382 bytes | |||
| -rw-r--r-- | docs/userguide/images/add.png | bin | 0 -> 169904 bytes | |||
| -rw-r--r-- | docs/userguide/images/login.png | bin | 0 -> 32761 bytes | |||
| -rw-r--r-- | docs/userguide/index.rst | 3 | ||||
| -rw-r--r-- | docs/userguide/opnfv_yardstick_tc043.rst | 2 | ||||
| -rw-r--r-- | docs/userguide/opnfv_yardstick_tc073.rst | 2 | ||||
| -rw-r--r-- | docs/userguide/references.rst | 6 | ||||
| -rwxr-xr-x | setup.py | 5 | ||||
| -rwxr-xr-x | tools/ubuntu-server-cloudimg-modify.sh | 2 |
17 files changed, 234 insertions, 60 deletions
@@ -17,6 +17,11 @@ wenjing_chu@dell.com liangqi1@huawei.com jean.gaoliang@huawei.com vincenzo.m.riccobene@intel.com +<<<<<<< HEAD +lvjing5@huawei.com +wu.zhihui1@zte.com.cn +14_ykl@tongji.edu.cn +limingjiang@huawei.com Link to TSC approval: http://meetbot.opnfv.org/meetings/ Link to approval of additional submitters: diff --git a/docs/results/os-onos-nofeature-ha.rst b/docs/results/os-onos-nofeature-ha.rst index e5587505f..d8b3ace5f 100644 --- a/docs/results/os-onos-nofeature-ha.rst +++ b/docs/results/os-onos-nofeature-ha.rst @@ -240,7 +240,7 @@ The lost amount of packets normally differs a lot per test run. Detailed test results --------------------- -The scenario was run on Intel POD5_ with: +The scenario was run on Intel POD6_ with: Joid OpenStack Mitaka Onos Goldeneye diff --git a/docs/results/results.rst b/docs/results/results.rst index d7572e463..04c6b9f87 100644 --- a/docs/results/results.rst +++ b/docs/results/results.rst @@ -29,7 +29,7 @@ OPNFV labs, triggered by OPNFV CI pipeline, documented per scenario. os-odl_l2-bgpvpn-ha.rst os-odl_l2-sfc-ha.rst os-nosdn-kvm-ha.rst - os-onos-nofeature-h.rst + os-onos-nofeature-ha.rst os-onos-sfc-ha.rst Test results of executed tests are avilable in Dashboard_ and logs in Jenkins_. diff --git a/docs/results/yardstick-opnfv-ha.rst b/docs/results/yardstick-opnfv-ha.rst index 4ee9de847..ef1617342 100644 --- a/docs/results/yardstick-opnfv-ha.rst +++ b/docs/results/yardstick-opnfv-ha.rst @@ -114,5 +114,5 @@ There are several improvement points for HA test: a) Running test cases in different enveriment deployed by different installers, such as compass4nfv, apex and joid, with different versiones. b) The period of each request is a little long, it needs more accurate test - method. +method. c) More test cases with different faults and different monitors are needed. diff --git a/docs/userguide/02-methodology.rst b/docs/userguide/02-methodology.rst index 1849ffd0c..34d271095 100644 --- a/docs/userguide/02-methodology.rst +++ b/docs/userguide/02-methodology.rst @@ -170,17 +170,19 @@ options). +---------+-------------------+----------------+------------------------------+ | Compute | TC003 [1]_ | TC003 [1]_ | TC013 [1]_ | | | TC004 | TC004 | TC015 [1]_ | -| | TC014 | TC010 | | -| | TC024 | TC012 | | -| | TC069 | TC055 | | +| | TC010 | TC024 | | +| | TC012 | TC055 | | +| | TC014 | | | +| | TC069 | | | +---------+-------------------+----------------+------------------------------+ -| Network | TC001 | TC001 | TC016 [1]_ | -| | TC002 | TC008 | TC018 [1]_ | -| | TC011 | TC009 | | -| | TC073 | TC075 | | +| Network | TC001 | TC044 | TC016 [1]_ | +| | TC002 | TC073 | TC018 [1]_ | +| | TC009 | TC075 | | +| | TC011 | | | +| | TC042 | | | +| | TC043 | | | +---------+-------------------+----------------+------------------------------+ -| Storage | TC005 | TC005 | TC017 [1]_ | -| | | TC063 | | +| Storage | TC005 | TC063 | TC017 [1]_ | +---------+-------------------+----------------+------------------------------+ .. note:: The description in this OPNFV document is intended as a reference for diff --git a/docs/userguide/07-installation.rst b/docs/userguide/07-installation.rst index aa45b61af..d1ab84d65 100644 --- a/docs/userguide/07-installation.rst +++ b/docs/userguide/07-installation.rst @@ -9,22 +9,52 @@ Yardstick Installation Abstract -------- -Yardstick supports installation on Ubuntu 14.04 or by using a Docker image. -The installation procedure on Ubuntu 14.04 or via the docker image are -detailed in the section below. +Yardstick supports installation on Ubuntu 14.04 or via a Docker image. The +installation procedure on Ubuntu 14.04 or via the docker image are detailed in +the section below. -To use Yardstick you should have access to an OpenStack environment, -with at least Nova, Neutron, Glance, Keystone and Heat installed. +To use Yardstick you should have access to an OpenStack environment, with at +least Nova, Neutron, Glance, Keystone and Heat installed. The steps needed to run Yardstick are: 1. Install Yardstick. -2. Create the test configuration .yaml file. -3. Build a guest image。 -4. Load the image into the OpenStack environment. -5. Create a Neutron external network. -6. Load OpenStack environment variables. -6. Run the test case. +2. Load OpenStack environment variables. +3. Create a Neutron external network. +4. Build Yardstick flavor and a guest image. +5. Load the guest image into the OpenStack environment. +6. Create the test configuration .yaml file. +7. Run the test case. + + +Prerequisites +------------- + +The OPNFV deployment is out of the scope of this document but it can be +found in http://artifacts.opnfv.org/opnfvdocs/colorado/docs/configguide/index.html. +The OPNFV platform is considered as the System Under Test (SUT) in this +document. + +Several prerequisites are needed for Yardstick: + + #. A Jumphost to run Yardstick on + #. A Docker daemon shall be installed on the Jumphost + #. A public/external network created on the SUT + #. Connectivity from the Jumphost to the SUT public/external network + +WARNING: Connectivity from Jumphost is essential and it is of paramount +importance to make sure it is working before even considering to install +and run Yardstick. Make also sure you understand how your networking is +designed to work. + +NOTE: **Jumphost** refers to any server which meets the previous +requirements. Normally it is the same server from where the OPNFV +deployment has been triggered previously. + +NOTE: If your Jumphost is operating behind a company http proxy and/or +Firewall, please consult first the section `Proxy Support`_, towards +the end of this document. The section details some tips/tricks which +*may* be of help in a proxified environment. Installing Yardstick on Ubuntu 14.04 @@ -33,9 +63,9 @@ Installing Yardstick on Ubuntu 14.04 .. _install-framework: You can install Yardstick framework directly on Ubuntu 14.04 or in an Ubuntu -14.04 Docker image. -No matter which way you choose to install Yardstick framework, the following -installation steps are identical. +14.04 Docker image. No matter which way you choose to install Yardstick +framework, the following installation steps are identical. + If you choose to use the Ubuntu 14.04 Docker image, You can pull the Ubuntu 14.04 Docker image from Docker hub: @@ -91,47 +121,42 @@ at: http://www.youtube.com/watch?v=4S4izNolmR0 Installing Yardstick using Docker --------------------------------- -Yardstick iteself has a Docker image, this Docker image (**Yardstick-stable**) +Yardstick has a Docker image, this Docker image (**Yardstick-stable**) serves as a replacement for installing the Yardstick framework in a virtual environment (for example as done in :ref:`install-framework`). It is recommended to use this Docker image to run Yardstick test. -Yardstick-stable image -^^^^^^^^^^^^^^^^^^^^^^ -Pull the Yardstick-stable Docker image from Docker hub: +Pulling the Yardstick Docker image +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -:: +.. _dockerhub: https://hub.docker.com/r/opnfv/yardstick/ + +Pull the Yardstick Docker image ('opnfv/yardstick') from the public dockerhub +registry under the OPNFV account: [dockerhub_], with the following docker +command:: docker pull opnfv/yardstick:stable +After pulling the Docker image, check that it is available with the +following docker command:: + + [yardsticker@jumphost ~]$ docker images + REPOSITORY TAG IMAGE ID CREATED SIZE + opnfv/yardstick stable a4501714757a 1 day ago 915.4 MB + Run the Docker image: :: - docker run --privileged=true -it openfv/yardstick /bin/bash + docker run --privileged=true -it opnfv/yardstick:stable /bin/bash -In the container run yardstick task command to execute a test case. -Before executing Yardstick test case, make sure that yardstick-trusty-server -image and yardstick flavor is available in OpenStack. -Detailed steps about creating yardstick flavor and building yardstick-trusty-server -image can be found below. +In the container the Yardstick repository is located in the /home/opnfv/repos +directory. OpenStack parameters and credentials ------------------------------------ -Yardstick-flavor -^^^^^^^^^^^^^^^^ -Most of the sample test cases in Yardstick are using an OpenStack flavor called -*yardstick-flavor* which deviates from the OpenStack standard m1.tiny flavor by the -disk size - instead of 1GB it has 3GB. Other parameters are the same as in m1.tiny. - -Create yardstick-flavor: - -:: - - nova flavor-create yardstick-flavor 100 512 3 1 - Environment variables ^^^^^^^^^^^^^^^^^^^^^ Before running Yardstick it is necessary to export OpenStack environment variables @@ -146,14 +171,46 @@ Credential environment variables in the *openrc* file have to include at least: * OS_PASSWORD * OS_TENANT_NAME +A sample openrc file may look like this: + +* export OS_PASSWORD=console +* export OS_TENANT_NAME=admin +* export OS_AUTH_URL=http://172.16.1.222:35357/v2.0 +* export OS_USERNAME=admin +* export OS_VOLUME_API_VERSION=2 +* export EXTERNAL_NETWORK=net04_ext + + +Yardstick falvor and guest images +--------------------------------- + +Before executing Yardstick test cases, make sure that yardstick guest image and +yardstick flavor are available in OpenStack. +Detailed steps about creating yardstick flavor and building yardstick-trusty-server +image can be found below. + +Yardstick-flavor +^^^^^^^^^^^^^^^^ +Most of the sample test cases in Yardstick are using an OpenStack flavor called +*yardstick-flavor* which deviates from the OpenStack standard m1.tiny flavor by the +disk size - instead of 1GB it has 3GB. Other parameters are the same as in m1.tiny. + +Create yardstick-flavor: + +:: + + nova flavor-create yardstick-flavor 100 512 3 1 + .. _guest-image: Building a guest image ^^^^^^^^^^^^^^^^^^^^^^ -Yardstick has a tool for building an Ubuntu Cloud Server image containing all -the required tools to run test cases supported by Yardstick. It is necessary to -have sudo rights to use this tool. +Most of the sample test cases in Yardstick are using a guest image called +*yardstick-trusty-server* which deviates from an Ubuntu Cloud Server image +containing all the required tools to run test cases supported by Yardstick. +Yardstick has a tool for building this custom image. It is necessary to have +sudo rights to use this tool. Also you may need install several additional packages to use this tool, by follwing the commands below: @@ -176,8 +233,10 @@ by following the commands above): **Warning:** the script will create files by default in: ``/tmp/workspace/yardstick`` and the files will be owned by root! + If you are building this guest image in inside a docker container make sure the container is granted with privilege. + The created image can be added to OpenStack using the ``glance image-create`` or via the OpenStack Dashboard. @@ -190,6 +249,24 @@ Example command: --disk-format qcow2 --container-format bare \ --file /tmp/workspace/yardstick/yardstick-trusty-server.img +Some Yardstick test cases use a Cirros image, you can find one at +http://download.cirros-cloud.net/0.3.3/cirros-0.3.3-x86_64-disk.img + + +Automatic flavor and image creation +----------------------------------- +Yardstick has a script for automatic creating yardstick flavor and building +guest images. This script is mainly used in CI, but you can still use it in +your local environment. + +Example command: + +:: + + export YARD_IMG_ARCH="amd64" + sudo echo "Defaults env_keep += \"YARD_IMG_ARCH\"" >> /etc/sudoers + source $YARDSTICK_REPO_DIR/tests/ci/load_images.sh + Yardstick default key pair ^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/docs/userguide/10-grafana.rst b/docs/userguide/10-grafana.rst new file mode 100644 index 000000000..e871a18da --- /dev/null +++ b/docs/userguide/10-grafana.rst @@ -0,0 +1,86 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International +.. License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2016 Huawei Technologies Co.,Ltd and others + +================= +Grafana dashboard +================= + +Abstract +======== + +This chapter describes the Yardstick grafana dashboard. The Yardstick grafana +dashboard can be found here: http://testresults.opnfv.org/grafana/ + + +.. image:: images/login.png + :width: 800px + :alt: Yardstick grafana dashboard + +Public access +============= + +Yardstick provids a public account for accessing to the dashboard. The username +and password are both set to ‘opnfv’. + +Testcase dashboard +================== + +For each test case, there is a dedicated dashboard. Shown here is the dashboard +of TC002. + + +.. image:: images/TC002.png + :width: 800px + :alt:TC002 dashboard + +For each test case dashboard. On the top left, we have a dashboard selection, +you can switch to different test cases using this pull-down menu. + +Underneath, we have a pod and scenario selection. +All the pods and scenarios that have ever published test data to the Influx DB +will be shown here. + +You can check multiple pods or scenarios. + +For each test case, we have a short description and a link to detailed test case information in Yardstick user guide. + +Underneath, it is the result presentation section. +You can use the time period selection on the top right corner to zoom in or zoom out the chart. + +Add a dashboard into yardstick grafana +====================================== + +Due to security concern, users that using the public opnfv account are not able +to edit the yardstick grafana directly.It takes a few more steps for a +non-yardstick user to add a custom dashboard into yardstick grafana. + +There are 6 steps to go. + + +.. image:: images/add.png + :width: 800px + :alt: Add a dashboard into yardstick grafana + + +First, You need to build a local influxdb and grafana, so you can do the work +locally. You can refer to How to deploy InfluxDB and Grafana locally wiki page +about how to do this. + +Once step one is done, you can fetch the existing grafana dashboard +configuration file from the yardstick repository and import it to your local +grafana. After import is done, you grafana dashboard will be ready to use just +like the community’s dashboard. + +The third step is running some test cases to generate test results and +publishing it to your local influxdb. + +Now you have some data to visualize in your dashboard. In the fourth step, it +is time to create your own dashboard. You can either modify an existing +dashboard or try to create a new one from scratch. + +Either way, once you finish the dashboard, the next step is exporting the +configuration file and propose a patch into Yardstick. Yardstick team will +review and merge it into Yardstick repository. + diff --git a/docs/userguide/10-list-of-tcs.rst b/docs/userguide/11-list-of-tcs.rst index 8798a8f51..8798a8f51 100644 --- a/docs/userguide/10-list-of-tcs.rst +++ b/docs/userguide/11-list-of-tcs.rst diff --git a/docs/userguide/images/TC002.png b/docs/userguide/images/TC002.png Binary files differnew file mode 100644 index 000000000..89154efcc --- /dev/null +++ b/docs/userguide/images/TC002.png diff --git a/docs/userguide/images/add.png b/docs/userguide/images/add.png Binary files differnew file mode 100644 index 000000000..a88a1b146 --- /dev/null +++ b/docs/userguide/images/add.png diff --git a/docs/userguide/images/login.png b/docs/userguide/images/login.png Binary files differnew file mode 100644 index 000000000..045e010e4 --- /dev/null +++ b/docs/userguide/images/login.png diff --git a/docs/userguide/index.rst b/docs/userguide/index.rst index 0aa112a45..60e1340ac 100644 --- a/docs/userguide/index.rst +++ b/docs/userguide/index.rst @@ -19,6 +19,7 @@ Yardstick Overview 07-installation 08-yardstick_plugin 09-result-store-InfluxDB - 10-list-of-tcs + 10-grafana + 11-list-of-tcs glossary references diff --git a/docs/userguide/opnfv_yardstick_tc043.rst b/docs/userguide/opnfv_yardstick_tc043.rst index b6e557d86..59d7c6993 100644 --- a/docs/userguide/opnfv_yardstick_tc043.rst +++ b/docs/userguide/opnfv_yardstick_tc043.rst @@ -13,7 +13,7 @@ Yardstick Test Case Description TC043 |Network Latency Between NFVI Nodes | | | +--------------+--------------------------------------------------------------+ -|test case id | OPNFV_YARDSTICK_TC043_Latency_between_NFVI_nodes_ | +|test case id | OPNFV_YARDSTICK_TC043_Latency_between_NFVI_nodes | | | measurements | | | | +--------------+--------------------------------------------------------------+ diff --git a/docs/userguide/opnfv_yardstick_tc073.rst b/docs/userguide/opnfv_yardstick_tc073.rst index a6499eabb..ad4526405 100644 --- a/docs/userguide/opnfv_yardstick_tc073.rst +++ b/docs/userguide/opnfv_yardstick_tc073.rst @@ -37,7 +37,7 @@ Yardstick Test Case Description TC073 | | For SLA max_mean_latency is set to 100. | | | | +--------------+--------------------------------------------------------------+ -|test tool | netperf | +|test tool | netperf_ | | | Netperf is a software application that provides network | | | bandwidth testing between two hosts on a network. It | | | supports Unix domain sockets, TCP, SCTP, DLPI and UDP via | diff --git a/docs/userguide/references.rst b/docs/userguide/references.rst index 7f6a0f0f0..05729ba75 100644 --- a/docs/userguide/references.rst +++ b/docs/userguide/references.rst @@ -15,8 +15,8 @@ OPNFV * Pharos wiki: https://wiki.opnfv.org/pharos * VTC: https://wiki.opnfv.org/vtc * Yardstick CI: https://build.opnfv.org/ci/view/yardstick/ -* Yardstick and ETSI TST001 presentation: https://wiki.opnfv.org/_media/opnfv_summit_-_bridging_opnfv_and_etsi.pdf -* Yardstick Project presentation: https://wiki.opnfv.org/_media/opnfv_summit_-_yardstick_project.pdf +* Yardstick and ETSI TST001 presentation: https://wiki.opnfv.org/display/yardstick/Yardstick?preview=%2F2925202%2F2925205%2Fopnfv_summit_-_bridging_opnfv_and_etsi.pdf +* Yardstick Project presentation: https://wiki.opnfv.org/display/yardstick/Yardstick?preview=%2F2925202%2F2925208%2Fopnfv_summit_-_yardstick_project.pdf * Yardstick wiki: https://wiki.opnfv.org/yardstick References used in Test Cases @@ -55,6 +55,6 @@ Standards ========= * ETSI NFV: http://www.etsi.org/technologies-clusters/technologies/nfv -* ETSI GS-NFV TST 001: https://docbox.etsi.org/ISG/NFV/Open/Drafts/TST001_-_Pre-deployment_Validation/ +* ETSI GS-NFV TST 001: http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_NFV-TST001v010101p.pdf * RFC2544: https://www.ietf.org/rfc/rfc2544.txt @@ -63,5 +63,8 @@ setup( 'yardstick-plot=yardstick.plot.plotter:main [plot]' ], }, - scripts=['tools/yardstick-img-modify'] + scripts=['tools/yardstick-img-modify', + 'tools/yardstick-img-lxd-modify', + 'tools/yardstick-img-dpdk-modify' + ] ) diff --git a/tools/ubuntu-server-cloudimg-modify.sh b/tools/ubuntu-server-cloudimg-modify.sh index 3ef060847..49c842c97 100755 --- a/tools/ubuntu-server-cloudimg-modify.sh +++ b/tools/ubuntu-server-cloudimg-modify.sh @@ -24,7 +24,7 @@ if [ $# -eq 1 ]; then fi # iperf3 only available for trusty in backports -if [grep -q trusty /etc/apt/sources.list ]; then +if [ grep -q trusty /etc/apt/sources.list ]; then if [ $YARD_IMG_ARCH = "arm64" ]; then echo "deb [arch=arm64] http://ports.ubuntu.com/ trusty-backports main restricted universe multiverse" >> /etc/apt/sources.list else |
