summaryrefslogtreecommitdiffstats
path: root/docs/testing/user/userguide
diff options
context:
space:
mode:
Diffstat (limited to 'docs/testing/user/userguide')
-rw-r--r--docs/testing/user/userguide/04-installation.rst101
-rw-r--r--docs/testing/user/userguide/12-nsb_installation.rst48
-rw-r--r--docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst2
-rw-r--r--docs/testing/user/userguide/nsb/tc_prox_context_lw_aftr_port.rst107
-rw-r--r--docs/testing/user/userguide/nsb/tc_prox_context_vpe_port.rst108
-rw-r--r--docs/testing/user/userguide/opnfv_yardstick_tc081.rst4
-rw-r--r--docs/testing/user/userguide/opnfv_yardstick_tc084.rst140
7 files changed, 498 insertions, 12 deletions
diff --git a/docs/testing/user/userguide/04-installation.rst b/docs/testing/user/userguide/04-installation.rst
index dc528db6c..caebecc09 100644
--- a/docs/testing/user/userguide/04-installation.rst
+++ b/docs/testing/user/userguide/04-installation.rst
@@ -107,6 +107,12 @@ Run the Docker image to get a Yardstick container::
======================= ====================================================
--name yardstick The name for this container
+If the host is restarted
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+The yardstick container must be started if the host is rebooted::
+
+ docker start yardstick
Configure the Yardstick container environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -301,12 +307,6 @@ Prerequisite preparation::
sudo -EH pip install appdirs==1.4.0
sudo -EH pip install virtualenv
-Create a virtual environment::
-
- virtualenv ~/yardstick_venv
- export YARDSTICK_VENV=~/yardstick_venv
- source ~/yardstick_venv/bin/activate
-
Download the source code and install Yardstick from it::
git clone https://gerrit.opnfv.org/gerrit/yardstick
@@ -314,6 +314,10 @@ Download the source code and install Yardstick from it::
cd ~/yardstick
sudo -EH ./install.sh
+If the host is ever restarted, nginx and uwsgi need to be restarted::
+
+ service nginx restart
+ uwsgi -i /etc/yardstick/yardstick.ini
Configure the Yardstick environment (**Todo**)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -331,6 +335,91 @@ For uninstalling Yardstick, just delete the virtual environment::
rm -rf ~/yardstick_venv
+Install Yardstick directly in OpenSUSE
+--------------------------------------
+
+.. _install-framework:
+
+You can install Yardstick framework directly in OpenSUSE.
+
+
+Install Yardstick
+^^^^^^^^^^^^^^^^^
+
+Prerequisite preparation::
+
+ sudo -EH zypper -n install -y gcc \
+ wget \
+ git \
+ sshpass \
+ qemu-tools \
+ kpartx \
+ libffi-devel \
+ libopenssl-devel \
+ python \
+ python-devel \
+ python-virtualenv \
+ libxml2-devel \
+ libxslt-devel \
+ python-setuptools-git
+
+Create a virtual environment::
+
+ virtualenv ~/yardstick_venv
+ export YARDSTICK_VENV=~/yardstick_venv
+ source ~/yardstick_venv/bin/activate
+ sudo -EH easy_install -U setuptools
+
+Download the source code and install Yardstick from it::
+
+ git clone https://gerrit.opnfv.org/gerrit/yardstick
+ export YARDSTICK_REPO_DIR=~/yardstick
+ cd yardstick
+ sudo -EH python setup.py install
+ sudo -EH pip install -r requirements.txt
+
+Install missing python modules::
+
+ sudo -EH pip install pyyaml \
+ oslo_utils \
+ oslo_serialization \
+ oslo_config \
+ paramiko \
+ python.heatclient \
+ python.novaclient \
+ python.glanceclient \
+ python.neutronclient \
+ scp \
+ jinja2
+
+
+Configure the Yardstick environment
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Source the OpenStack environment variables::
+
+ source DEVSTACK_DIRECTORY/openrc
+
+Export the Openstack external network. The default installation of Devstack
+names the external network public::
+
+ export EXTERNAL_NETWORK=public
+ export OS_USERNAME=demo
+
+Change the API version used by Yardstick to v2.0 (the devstack openrc sets it
+to v3)::
+
+ export OS_AUTH_URL=http://PUBLIC_IP_ADDRESS:5000/v2.0
+
+
+Uninstall Yardstick
+^^^^^^^^^^^^^^^^^^^
+
+For unistalling Yardstick, just delete the virtual environment::
+
+ rm -rf ~/yardstick_venv
+
+
Verify the installation
-----------------------
diff --git a/docs/testing/user/userguide/12-nsb_installation.rst b/docs/testing/user/userguide/12-nsb_installation.rst
index 8cc26acd5..a584ca231 100644
--- a/docs/testing/user/userguide/12-nsb_installation.rst
+++ b/docs/testing/user/userguide/12-nsb_installation.rst
@@ -112,12 +112,52 @@ Download the source code and install Yardstick from it
# git checkout <tag or stable branch>
git checkout stable/euphrates
- # For Bare-Metal or Standalone Virtualization
- ./nsb_setup.sh
+Configure the network proxy, either using the environment variables or setting
+the global environment file:
- # For OpenStack
- ./nsb_setup.sh <path to admin-openrc.sh>
+.. code-block:: ini
+ cat /etc/environment
+ http_proxy='http://proxy.company.com:port'
+ https_proxy='http://proxy.company.com:port'
+.. code-block:: console
+ export http_proxy='http://proxy.company.com:port'
+ export https_proxy='http://proxy.company.com:port'
+
+The last step is to modify the Yardstick installation inventory, used by
+Ansible:
+
+.. code-block:: ini
+ cat ./ansible/yardstick-install-inventory.ini
+ [jumphost]
+ localhost ansible_connection=local
+
+ [yardstick-standalone]
+ yardstick-standalone-node ansible_host=192.168.1.2
+ yardstick-standalone-node-2 ansible_host=192.168.1.3
+
+ # section below is only due backward compatibility.
+ # it will be removed later
+ [yardstick:children]
+ jumphost
+
+ [all:vars]
+ ansible_user=root
+ ansible_pass=root
+
+
+To execute an installation for a Bare-Metal or a Standalone context:
+
+.. code-block:: console
+
+ ./nsb_setup.sh
+
+
+To execute an installation for an OpenStack context:
+
+.. code-block:: console
+
+ ./nsb_setup.sh <path to admin-openrc.sh>
Above command setup docker with latest yardstick code. To execute
diff --git a/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst b/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst
index 43aa3d69a..895837283 100644
--- a/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst
+++ b/docs/testing/user/userguide/nsb/nsb-list-of-tcs.rst
@@ -26,3 +26,5 @@ NSB PROX Test Case Descriptions
tc_prox_context_mpls_tagging_port
tc_prox_context_buffering_port
tc_prox_context_load_balancer_port
+ tc_prox_context_vpe_port
+ tc_prox_context_lw_after_port
diff --git a/docs/testing/user/userguide/nsb/tc_prox_context_lw_aftr_port.rst b/docs/testing/user/userguide/nsb/tc_prox_context_lw_aftr_port.rst
new file mode 100644
index 000000000..5a1fada05
--- /dev/null
+++ b/docs/testing/user/userguide/nsb/tc_prox_context_lw_aftr_port.rst
@@ -0,0 +1,107 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2017 Intel Corporation.
+
+************************************************
+Yardstick Test Case Description: NSB PROX LwAFTR
+************************************************
+
++-----------------------------------------------------------------------------+
+|NSB PROX test for NFVI characterization |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_prox_{context}_lw_aftr-{port_num} |
+| | |
+| | * context = baremetal or heat_context; |
+| | * port_num = 4; |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | * Network Throughput; |
+| | * TG Packets Out; |
+| | * TG Packets In; |
+| | * VNF Packets Out; |
+| | * VNF Packets In; |
+| | * Dropped packets; |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The PROX LW_AFTR test will take packets in from one |
+| | port and remove the ipv6 encapsulation and forward them to |
+| | another port. While forwarded packets in other direction |
+| | will be encapsulated in an ipv6 header. |
+| | |
+| | The lw_aftr test cases are implemented to run in baremetal |
+| | and heat context an require 4 port topology to run the |
+| | default configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The LW_AFTR test cases are listed below: |
+| | |
+| | * tc_prox_baremetal_lw_aftr-4.yaml |
+| | * tc_prox_heat_context_lw_aftr-4.yaml |
+| | |
+| | Test duration is set as 300sec for each test. |
+| | The minimum packet size for MPLS test is 68 bytes. This is |
+| | set in the traffic profile and can be configured to use |
+| | higher packet sizes. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | PROX |
+| | PROX is a DPDK application that can simulate VNF workloads |
+| | and can generate traffic and used for NFVI characterization |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | The PROX lwAFTR test cases can be configured with |
+| | different: |
+| | |
+| | * packet sizes; |
+| | * test durations; |
+| | * tolerated loss; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | For Openstack test case image (yardstick-samplevnfs) needs |
+|conditions | to be installed into Glance with Prox and Dpdk included in |
+| | it. |
+| | |
+| | For Baremetal tests cases Prox and Dpdk must be installed in |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | For Baremetal test: The TG and VNF are started on the hosts |
+| | based on the pod file. |
+| | |
+| | For Heat test: Two host VMs are booted, as Traffic generator |
+| | and VNF(LW_AFTR workload) based on the test flavor. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with the TG and VNF by using ssh. |
+| | The test will resolve the topology and instantiate the VNF |
+| | and TG and collect the KPI's/metrics. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | The TG will send packets to the VNF. If the number of |
+| | dropped packets is more than the tolerated loss the line |
+| | rate or throughput is halved. This is done until the dropped |
+| | packets are within an acceptable tolerated loss. |
+| | |
+| | The KPI is the number of packets per second for 86 bytes |
+| | packet size with an accepted minimal packet loss for the |
+| | default configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | In Baremetal test: The test quits the application and unbind |
+| | the dpdk ports. |
+| | |
+| | In Heat test: Two host VMs are deleted on test completion. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will achieve a Throughput with an accepted |
+| | minimal tolerated packet loss. |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/userguide/nsb/tc_prox_context_vpe_port.rst b/docs/testing/user/userguide/nsb/tc_prox_context_vpe_port.rst
new file mode 100644
index 000000000..6827b0525
--- /dev/null
+++ b/docs/testing/user/userguide/nsb/tc_prox_context_vpe_port.rst
@@ -0,0 +1,108 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, 2017 Intel Corporation.
+
+**********************************************
+Yardstick Test Case Description: NSB PROXi VPE
+**********************************************
+
++-----------------------------------------------------------------------------+
+|NSB PROX test for NFVI characterization |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | tc_prox_{context}_vpe-{port_num} |
+| | |
+| | * context = baremetal or heat_context; |
+| | * port_num = 4; |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | * Network Throughput; |
+| | * TG Packets Out; |
+| | * TG Packets In; |
+| | * VNF Packets Out; |
+| | * VNF Packets In; |
+| | * Dropped packets; |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The PROX VPE test handles packet processing, routing, QinQ |
+| | encapsulation, flows, ACL rules, adds/removes MPLS tagging |
+| | and performs QoS before forwarding packet to another port. |
+| | The reverse applies to forwarded packets in the other |
+| | direction. |
+| | |
+| | The VPE test cases are implemented to run in baremetal |
+| | and heat context an require 4 port topology to run the |
+| | default configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | The VPE test cases are listed below: |
+| | |
+| | * tc_prox_baremetal_vpe-4.yaml |
+| | * tc_prox_heat_context_vpe-4.yaml |
+| | |
+| | Test duration is set as 300sec for each test. |
+| | The minimum packet size for VPE test is 68 bytes. This is |
+| | set in the traffic profile and can be configured to use |
+| | higher packet sizes. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | PROX |
+| | PROX is a DPDK application that can simulate VNF workloads |
+| | and can generate traffic and used for NFVI characterization |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | The PROX VPE test cases can be configured with |
+| | different: |
+| | |
+| | * packet sizes; |
+| | * test durations; |
+| | * tolerated loss; |
+| | |
+| | Default values exist. |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | For Openstack test case image (yardstick-samplevnfs) needs |
+|conditions | to be installed into Glance with Prox and Dpdk included in |
+| | it. |
+| | |
+| | For Baremetal tests cases Prox and Dpdk must be installed in |
+| | the hosts where the test is executed. The pod.yaml file must |
+| | have the necessary system and NIC information |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | For Baremetal test: The TG and VNF are started on the hosts |
+| | based on the pod file. |
+| | |
+| | For Heat test: Two host VMs are booted, as Traffic generator |
+| | and VNF(VPE workload) based on the test flavor. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Yardstick is connected with the TG and VNF by using ssh. |
+| | The test will resolve the topology and instantiate the VNF |
+| | and TG and collect the KPI's/metrics. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | The TG will send packets to the VNF. If the number of |
+| | dropped packets is more than the tolerated loss the line |
+| | rate or throughput is halved. This is done until the dropped |
+| | packets are within an acceptable tolerated loss. |
+| | |
+| | The KPI is the number of packets per second for 68 bytes |
+| | packet size with an accepted minimal packet loss for the |
+| | default configuration. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | In Baremetal test: The test quits the application and unbind |
+| | the dpdk ports. |
+| | |
+| | In Heat test: Two host VMs are deleted on test completion. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | The test case will achieve a Throughput with an accepted |
+| | minimal tolerated packet loss. |
++--------------+--------------------------------------------------------------+
+
diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc081.rst b/docs/testing/user/userguide/opnfv_yardstick_tc081.rst
index 90af8a382..793c3fdd5 100644
--- a/docs/testing/user/userguide/opnfv_yardstick_tc081.rst
+++ b/docs/testing/user/userguide/opnfv_yardstick_tc081.rst
@@ -4,7 +4,7 @@
.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
*************************************
-Yardstick Test Case Description TC080
+Yardstick Test Case Description TC081
*************************************
.. _cirros-image: https://download.cirros-cloud.net
@@ -21,7 +21,7 @@ Yardstick Test Case Description TC080
|metric | RTT (Round Trip Time) |
| | |
+--------------+--------------------------------------------------------------+
-|test purpose | The purpose of TC080 is to do a basic verification that |
+|test purpose | The purpose of TC081 is to do a basic verification that |
| | network latency is within acceptable boundaries when packets |
| | travel between a containers and a VM. |
| | |
diff --git a/docs/testing/user/userguide/opnfv_yardstick_tc084.rst b/docs/testing/user/userguide/opnfv_yardstick_tc084.rst
new file mode 100644
index 000000000..2e7b28e25
--- /dev/null
+++ b/docs/testing/user/userguide/opnfv_yardstick_tc084.rst
@@ -0,0 +1,140 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.
+
+*************************************
+Yardstick Test Case Description TC084
+*************************************
+
+.. _spec_cpu_2006: https://www.spec.org/cpu2006/
+
++-----------------------------------------------------------------------------+
+|Compute Performance |
+| |
++--------------+--------------------------------------------------------------+
+|test case id | OPNFV_YARDSTICK_TC084_SPEC CPU 2006 FOR VM |
+| | |
++--------------+--------------------------------------------------------------+
+|metric | compute-intensive performance |
+| | |
++--------------+--------------------------------------------------------------+
+|test purpose | The purpose of TC084 is to evaluate the IaaS compute |
+| | performance by using SPEC CPU 2006 benchmark. The SPEC CPU |
+| | 2006 benchmark has several different ways to measure |
+| | computer performance. One way is to measure how fast the |
+| | computer completes a single task; this is called a speed |
+| | measurement. Another way is to measure how many tasks |
+| | computer can accomplish in a certain amount of time; this is |
+| | called a throughput, capacity or rate measurement. |
+| | |
++--------------+--------------------------------------------------------------+
+|test tool | SPEC CPU 2006 |
+| | |
+| | The SPEC CPU 2006 benchmark is SPEC's industry-standardized, |
+| | CPU-intensive benchmark suite, stressing a system's |
+| | processor, memory subsystem and compiler. This benchmark |
+| | suite includes the SPECint benchmarks and the SPECfp |
+| | benchmarks. The SPECint 2006 benchmark contains 12 different |
+| | benchmark tests and the SPECfp 2006 benchmark contains 19 |
+| | different benchmark tests. |
+| | |
+| | SPEC CPU 2006 is not always part of a Linux distribution. |
+| | SPEC requires that users purchase a license and agree with |
+| | their terms and conditions. For this test case, users must |
+| | manually download cpu2006-1.2.iso from the SPEC website and |
+| | save it under the yardstick/resources folder (e.g. /home/ |
+| | opnfv/repos/yardstick/yardstick/resources/cpu2006-1.2.iso) |
+| | SPEC CPU® 2006 benchmark is available for purchase via the |
+| | SPEC order form (https://www.spec.org/order.html). |
+| | |
++--------------+--------------------------------------------------------------+
+|test | This test case uses SPEC CPU 2006 benchmark to measure |
+|description | compute-intensive performance of VMs. |
+| | |
++--------------+--------------------------------------------------------------+
+|configuration | file: opnfv_yardstick_tc084.yaml |
+| | |
+| | benchmark_subset is set to int. |
+| | |
+| | SLA is not available in this test case. |
+| | |
++--------------+--------------------------------------------------------------+
+|applicability | Test can be configured with different: |
+| | |
+| | * benchmark_subset - a subset of SPEC CPU 2006 benchmarks |
+| | to run; |
+| | * SPECint_benchmark - a SPECint benchmark to run; |
+| | * SPECint_benchmark - a SPECfp benchmark to run; |
+| | * output_format - desired report format; |
+| | * runspec_config - SPEC CPU 2006 config file provided to |
+| | the runspec binary; |
+| | * runspec_iterations - the number of benchmark iterations |
+| | to execute. For a reportable run, must be 3; |
+| | * runspec_tune - tuning to use (base, peak, or all). For a |
+| | reportable run, must be either base or all. Reportable |
+| | runs do base first, then (optionally) peak; |
+| | * runspec_size - size of input data to run (test, train, or |
+| | ref). Reportable runs ensure that your binaries can |
+| | produce correct results with the test and train workloads |
+| | |
++--------------+--------------------------------------------------------------+
+|usability | This test case is used for executing SPEC CPU 2006 benchmark |
+| | on virtual machines. The SPECint 2006 benchmark takes |
+| | approximately 5 hours. (The time may vary due to different |
+| | VM cpu configurations) |
+| | |
++--------------+--------------------------------------------------------------+
+|references | spec_cpu_2006_ |
+| | |
+| | ETSI-NFV-TST001 |
+| | |
++--------------+--------------------------------------------------------------+
+|pre-test | To run and install SPEC CPU 2006, the following are |
+|conditions | required: |
+| | * For SPECint 2006: Both C99 and C++98 compilers are |
+| | installed in VM images; |
+| | * For SPECfp 2006: All three of C99, C++98 and Fortran-95 |
+| | compilers installed in VM images; |
+| | * At least 4GB of disk space availabile on VM. |
+| | |
+| | gcc 4.8.* and g++ 4.8.* version have been tested in Ubuntu |
+| | 14.04, Ubuntu 16.04 and Redhat Enterprise Linux 7.4 image. |
+| | Higher gcc and g++ version may cause compiling error. |
+| | |
+| | For more SPEC CPU 2006 dependencies please visit |
+| | (https://www.spec.org/cpu2006/Docs/techsupport.html) |
+| | |
++--------------+--------------------------------------------------------------+
+|test sequence | description and expected result |
+| | |
++--------------+--------------------------------------------------------------+
+|step 1 | cpu2006-1.2.iso has been saved under the yardstick/resources |
+| | folder (e.g. /home/opnfv/repos/yardstick/yardstick/resources |
+| | /cpu2006-1.2.iso). Additionally, to use your custom runspec |
+| | config file you can save it under the yardstick/resources/ |
+| | files folder and specify the config file name in the |
+| | runspec_config parameter. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 2 | Upload SPEC CPU 2006 ISO to the target VM using scp and |
+| | install SPEC CPU 2006. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 3 | Connect to the target server using SSH. |
+| | If custom runspec config file is used, copy this file from |
+| | yardstick to the target VM via the SSH tunnel. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 4 | SPEC CPU 2006 benchmark is invoked and SPEC CPU 2006 metrics |
+| | are generated. |
+| | |
++--------------+--------------------------------------------------------------+
+|step 5 | Text, HTML, CSV, PDF, and Configuration file outputs for the |
+| | SPEC CPU 2006 metrics are fetched from the VM and stored |
+| | under /tmp/result folder. |
+| | |
++--------------+--------------------------------------------------------------+
+|test verdict | None. SPEC CPU 2006 results are collected and stored. |
+| | |
++--------------+--------------------------------------------------------------+