Moved doc files to testing document structure testing/user ... testing/developer and modified doc index to match dir structure

Change-Id: I4b1a535808a48773505fa7874c61707cd349fced
Signed-off-by: Trevor Cooper <trevor.cooper@intel.com>

9 files changed, 3435 insertions, 0 deletions

diff --git a/docs/testing/user/configguide/LICENSE b/docs/testing/user/configguide/LICENSE
new file mode 100644
index 00000000..7bc572ce
--- /dev/null
+++ b/docs/testing/user/configguide/LICENSE
@@ -0,0 +1,2 @@
+This work is licensed under a Creative Commons Attribution 4.0 International License.
+http://creativecommons.org/licenses/by/4.0
diff --git a/docs/testing/user/configguide/TCLServerProperties.png b/docs/testing/user/configguide/TCLServerProperties.png
new file mode 100644
index 00000000..682de7c5
--- /dev/null
+++ b/docs/testing/user/configguide/TCLServerProperties.png
diff --git a/docs/testing/user/configguide/installation.rst b/docs/testing/user/configguide/installation.rst
new file mode 100644
index 00000000..1965a8f5
--- /dev/null
+++ b/docs/testing/user/configguide/installation.rst
@@ -0,0 +1,310 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation, AT&T and others.
+
+.. _vsperf-installation:
+
+======================
+Installing vswitchperf
+======================
+
+Downloading vswitchperf
+-----------------------
+
+The vswitchperf can be downloaded from its official git repository, which is
+hosted by OPNFV. It is necessary to install a ``git`` at your DUT before downloading
+vswitchperf. Installation of ``git`` is specific to the packaging system used by
+Linux OS installed at DUT.
+
+Example of installation of GIT package and its dependencies:
+
+* in case of OS based on RedHat Linux:
+
+  .. code:: bash
+
+     sudo yum install git
+
+
+* in case of Ubuntu or Debian:
+
+  .. code:: bash
+
+     sudo apt-get install git
+
+After the ``git`` is successfully installed at DUT, then vswitchperf can be downloaded
+as follows:
+
+.. code:: bash
+
+   git clone http://git.opnfv.org/vswitchperf
+
+The last command will create a directory ``vswitchperf`` with a local copy of vswitchperf
+repository.
+
+Supported Operating Systems
+---------------------------
+
+* CentOS 7.3
+* Fedora 24 (kernel 4.8 requires DPDK 16.11 and newer)
+* Fedora 25 (kernel 4.9 requires DPDK 16.11 and newer)
+* openSUSE 42.2
+* RedHat 7.2 Enterprise Linux
+* RedHat 7.3 Enterprise Linux
+* Ubuntu 14.04
+* Ubuntu 16.04
+* Ubuntu 16.10 (kernel 4.8 requires DPDK 16.11 and newer)
+
+Supported vSwitches
+-------------------
+
+The vSwitch must support Open Flow 1.3 or greater.
+
+* Open vSwitch
+* Open vSwitch with DPDK support
+* TestPMD application from DPDK (supports p2p and pvp scenarios)
+
+Supported Hypervisors
+---------------------
+
+* Qemu version 2.3 or greater (version 2.5.0 is recommended)
+
+Supported VNFs
+--------------
+
+In theory, it is possible to use any VNF image, which is compatible
+with supported hypervisor. However such VNF must ensure, that appropriate
+number of network interfaces is configured and that traffic is properly
+forwarded among them. For new vswitchperf users it is recommended to start
+with official vloop-vnf_ image, which is maintained by vswitchperf community.
+
+.. _vloop-vnf:
+
+vloop-vnf
+=========
+
+The official VM image is called vloop-vnf and it is available for free download
+from OPNFV artifactory. This image is based on Linux Ubuntu distribution and it
+supports following applications for traffic forwarding:
+
+* DPDK testpmd
+* Linux Bridge
+* Custom l2fwd module
+
+The vloop-vnf can be downloaded to DUT, for example by ``wget``:
+
+  .. code:: bash
+
+     wget http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20160823.qcow2
+
+**NOTE:** In case that ``wget`` is not installed at your DUT, you could install it at RPM
+based system by ``sudo yum install wget`` or at DEB based system by ``sudo apt-get install
+wget``.
+
+Changelog of vloop-vnf:
+
+  * `vloop-vnf-ubuntu-14.04_20160823`_
+
+    * ethtool installed
+    * only 1 NIC is configured by default to speed up boot with 1 NIC setup
+    * security updates applied
+
+  * `vloop-vnf-ubuntu-14.04_20160804`_
+
+    * Linux kernel 4.4.0 installed
+    * libnuma-dev installed
+    * security updates applied
+
+  * `vloop-vnf-ubuntu-14.04_20160303`_
+
+    * snmpd service is disabled by default to avoid error messages during VM boot
+    * security updates applied
+
+  * `vloop-vnf-ubuntu-14.04_20151216`_
+
+    * version with development tools required for build of DPDK and l2fwd
+
+.. _vsperf-installation-script:
+
+Installation
+------------
+
+The test suite requires Python 3.3 or newer and relies on a number of other
+system and python packages. These need to be installed for the test suite
+to function.
+
+Installation of required packages, preparation of Python 3 virtual
+environment and compilation of OVS, DPDK and QEMU is performed by
+script **systems/build_base_machine.sh**. It should be executed under
+user account, which will be used for vsperf execution.
+
+**NOTE:** Password-less sudo access must be configured for given
+user account before script is executed.
+
+.. code:: bash
+
+    $ cd systems
+    $ ./build_base_machine.sh
+
+**NOTE:** you don't need to go into any of the systems subdirectories,
+simply run the top level **build_base_machine.sh**, your OS will be detected
+automatically.
+
+Script **build_base_machine.sh** will install all the vsperf dependencies
+in terms of system packages, Python 3.x and required Python modules.
+In case of CentOS 7 or RHEL it will install Python 3.3 from an additional
+repository provided by Software Collections (`a link`_). Installation script
+will also use `virtualenv`_ to create a vsperf virtual environment, which is
+isolated from the default Python environment. This environment will reside in a
+directory called **vsperfenv** in $HOME. It will ensure, that system wide Python
+installation is not modified or broken by VSPERF installation. The complete list
+of Python packages installed inside virtualenv can be found at file
+``requirements.txt``, which is located at vswitchperf repository.
+
+**NOTE:** For RHEL 7.3 Enterprise and CentOS 7.3 OVS Vanilla is not
+built from upstream source due to kernel incompatibilities. Please see the
+instructions in the vswitchperf_design document for details on configuring
+OVS Vanilla for binary package usage.
+
+.. _vpp-installation:
+
+VPP installation
+================
+
+Currently vswitchperf installation scripts do not support automatic build
+of VPP. In order to execute tests with VPP, it is required to install it
+manually. Please refer to the official documentation of `fd.io`_ project to
+install VPP from `packages`_ or from the `sources`_.
+
+See details about :ref:`vpp-test`.
+
+.. _fd.io: https://fd.io/
+.. _packages: https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_packages
+.. _sources: https://wiki.fd.io/view/VPP/Build,_install,_and_test_images
+
+Using vswitchperf
+-----------------
+
+You will need to activate the virtual environment every time you start a
+new shell session. Its activation is specific to your OS:
+
+* CentOS 7 and RHEL
+
+  .. code:: bash
+
+     $ scl enable python33 bash
+     $ source $HOME/vsperfenv/bin/activate
+
+* Fedora and Ubuntu
+
+  .. code:: bash
+
+     $ source $HOME/vsperfenv/bin/activate
+
+After the virtual environment is configued, then VSPERF can be used.
+For example:
+
+  .. code:: bash
+
+     (vsperfenv) $ cd vswitchperf
+     (vsperfenv) $ ./vsperf --help
+
+Gotcha
+======
+
+In case you will see following error during environment activation:
+
+.. code:: bash
+
+   $ source $HOME/vsperfenv/bin/activate
+   Badly placed ()'s.
+
+then check what type of shell you are using:
+
+.. code:: bash
+
+   $ echo $SHELL
+   /bin/tcsh
+
+See what scripts are available in $HOME/vsperfenv/bin
+
+.. code:: bash
+
+   $ ls $HOME/vsperfenv/bin/
+   activate          activate.csh      activate.fish     activate_this.py
+
+source the appropriate script
+
+.. code:: bash
+
+   $ source bin/activate.csh
+
+Working Behind a Proxy
+======================
+
+If you're behind a proxy, you'll likely want to configure this before
+running any of the above. For example:
+
+  .. code:: bash
+
+    export http_proxy=proxy.mycompany.com:123
+    export https_proxy=proxy.mycompany.com:123
+
+.. _a link: http://www.softwarecollections.org/en/scls/rhscl/python33/
+.. _virtualenv: https://virtualenv.readthedocs.org/en/latest/
+.. _vloop-vnf-ubuntu-14.04_20160823: http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20160823.qcow2
+.. _vloop-vnf-ubuntu-14.04_20160804: http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20160804.qcow2
+.. _vloop-vnf-ubuntu-14.04_20160303: http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20160303.qcow2
+.. _vloop-vnf-ubuntu-14.04_20151216: http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20151216.qcow2
+
+Hugepage Configuration
+----------------------
+
+Systems running vsperf with either dpdk and/or tests with guests must configure
+hugepage amounts to support running these configurations. It is recommended
+to configure 1GB hugepages as the pagesize.
+
+The amount of hugepages needed depends on your configuration files in vsperf.
+Each guest image requires 2048 MB by default according to the default settings
+in the ``04_vnf.conf`` file.
+
+.. code:: bash
+
+    GUEST_MEMORY = ['2048']
+
+The dpdk startup parameters also require an amount of hugepages depending on
+your configuration in the ``02_vswitch.conf`` file.
+
+.. code:: bash
+
+    DPDK_SOCKET_MEM = ['1024', '0']
+
+**NOTE:** Option ``DPDK_SOCKET_MEM`` is used by all vSwitches with DPDK support.
+It means Open vSwitch, VPP and TestPMD.
+
+VSPerf will verify hugepage amounts are free before executing test
+environments. In case of hugepage amounts not being free, test initialization
+will fail and testing will stop.
+
+**NOTE:** In some instances on a test failure dpdk resources may not
+release hugepages used in dpdk configuration. It is recommended to configure a
+few extra hugepages to prevent a false detection by VSPerf that not enough free
+hugepages are available to execute the test environment. Normally dpdk would use
+previously allocated hugepages upon initialization.
+
+Depending on your OS selection configuration of hugepages may vary. Please refer
+to your OS documentation to set hugepages correctly. It is recommended to set
+the required amount of hugepages to be allocated by default on reboots.
+
+Information on hugepage requirements for dpdk can be found at
+http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html
+
+You can review your hugepage amounts by executing the following command
+
+.. code:: bash
+
+    cat /proc/meminfo | grep Huge
+
+If no hugepages are available vsperf will try to automatically allocate some.
+Allocation is controlled by ``HUGEPAGE_RAM_ALLOCATION`` configuration parameter in
+``02_vswitch.conf`` file. Default is 2GB, resulting in either 2 1GB hugepages
+or 1024 2MB hugepages.
diff --git a/docs/testing/user/configguide/trafficgen.rst b/docs/testing/user/configguide/trafficgen.rst
new file mode 100644
index 00000000..4e42b2be
--- /dev/null
+++ b/docs/testing/user/configguide/trafficgen.rst
@@ -0,0 +1,671 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation, AT&T and others.
+
+.. _trafficgen-installation:
+
+===========================
+'vsperf' Traffic Gen Guide
+===========================
+
+Overview
+--------
+
+VSPERF supports the following traffic generators:
+
+  * Dummy_ (DEFAULT)
+  * Ixia_
+  * `Spirent TestCenter`_
+  * `Xena Networks`_
+  * MoonGen_
+
+To see the list of traffic gens from the cli:
+
+.. code-block:: console
+
+    $ ./vsperf --list-trafficgens
+
+This guide provides the details of how to install
+and configure the various traffic generators.
+
+Background Information
+----------------------
+The traffic default configuration can be found in **conf/03_traffic.conf**,
+and is configured as follows:
+
+.. code-block:: console
+
+    TRAFFIC = {
+        'traffic_type' : 'rfc2544_throughput',
+        'frame_rate' : 100,
+        'bidir' : 'True',  # will be passed as string in title format to tgen
+        'multistream' : 0,
+        'stream_type' : 'L4',
+        'pre_installed_flows' : 'No',           # used by vswitch implementation
+        'flow_type' : 'port',                   # used by vswitch implementation
+
+        'l2': {
+            'framesize': 64,
+            'srcmac': '00:00:00:00:00:00',
+            'dstmac': '00:00:00:00:00:00',
+        },
+        'l3': {
+            'proto': 'udp',
+            'srcip': '1.1.1.1',
+            'dstip': '90.90.90.90',
+        },
+        'l4': {
+            'srcport': 3000,
+            'dstport': 3001,
+        },
+        'vlan': {
+            'enabled': False,
+            'id': 0,
+            'priority': 0,
+            'cfi': 0,
+        },
+    }
+
+The framesize parameter can be overridden from the configuration
+files by adding the following to your custom configuration file
+``10_custom.conf``:
+
+.. code-block:: console
+
+    TRAFFICGEN_PKT_SIZES = (64, 128,)
+
+OR from the commandline:
+
+.. code-block:: console
+
+    $ ./vsperf --test-params "TRAFFICGEN_PKT_SIZES=(x,y)" $TESTNAME
+
+You can also modify the traffic transmission duration and the number
+of tests run by the traffic generator by extending the example
+commandline above to:
+
+.. code-block:: console
+
+    $ ./vsperf --test-params "TRAFFICGEN_PKT_SIZES=(x,y);TRAFFICGEN_DURATION=10;" \
+                             "TRAFFICGEN_RFC2544_TESTS=1" $TESTNAME
+
+.. _trafficgen-dummy:
+
+Dummy
+-----
+
+The Dummy traffic generator can be used to test VSPERF installation or
+to demonstrate VSPERF functionality at DUT without connection
+to a real traffic generator.
+
+You could also use the Dummy generator in case, that your external
+traffic generator is not supported by VSPERF. In such case you could
+use VSPERF to setup your test scenario and then transmit the traffic.
+After the transmission is completed you could specify values for all
+collected metrics and VSPERF will use them to generate final reports.
+
+Setup
+~~~~~
+
+To select the Dummy generator please add the following to your
+custom configuration file ``10_custom.conf``.
+
+.. code-block:: console
+
+     TRAFFICGEN = 'Dummy'
+
+OR run ``vsperf`` with the ``--trafficgen`` argument
+
+.. code-block:: console
+
+    $ ./vsperf --trafficgen Dummy $TESTNAME
+
+Where $TESTNAME is the name of the vsperf test you would like to run.
+This will setup the vSwitch and the VNF (if one is part of your test)
+print the traffic configuration and prompt you to transmit traffic
+when the setup is complete.
+
+.. code-block:: console
+
+    Please send 'continuous' traffic with the following stream config:
+    30mS, 90mpps, multistream False
+    and the following flow config:
+    {
+        "flow_type": "port",
+        "l3": {
+            "srcip": "1.1.1.1",
+            "proto": "tcp",
+            "dstip": "90.90.90.90"
+        },
+        "traffic_type": "rfc2544_continuous",
+        "multistream": 0,
+        "bidir": "True",
+        "vlan": {
+            "cfi": 0,
+            "priority": 0,
+            "id": 0,
+            "enabled": false
+        },
+        "frame_rate": 90,
+        "l2": {
+            "dstport": 3001,
+            "srcport": 3000,
+            "dstmac": "00:00:00:00:00:00",
+            "srcmac": "00:00:00:00:00:00",
+            "framesize": 64
+        }
+    }
+    What was the result for 'frames tx'?
+
+When your traffic generator has completed traffic transmission and provided
+the results please input these at the VSPERF prompt. VSPERF will try
+to verify the input:
+
+.. code-block:: console
+
+    Is '$input_value' correct?
+
+Please answer with y OR n.
+
+VSPERF will ask you to provide a value for every of collected metrics. The list
+of metrics can be found at traffic-type-metrics_.
+Finally vsperf will print out the results for your test and generate the
+appropriate logs and report files.
+
+.. _traffic-type-metrics:
+
+Metrics collected for supported traffic types
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Below you could find a list of metrics collected by VSPERF for each of supported
+traffic types.
+
+RFC2544 Throughput and Continuous:
+
+  * frames tx
+  * frames rx
+  * min latency
+  * max latency
+  * avg latency
+  * frameloss
+
+RFC2544 Back2back:
+
+  * b2b frames
+  * b2b frame loss %
+
+Dummy result pre-configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In case of a Dummy traffic generator it is possible to pre-configure the test
+results. This is useful for creation of demo testcases, which do not require
+a real traffic generator. Such testcase can be run by any user and it will still
+generate all reports and result files.
+
+Result values can be specified within ``TRAFFICGEN_DUMMY_RESULTS`` dictionary,
+where every of collected metrics must be properly defined. Please check the list
+of traffic-type-metrics_.
+
+Dictionary with dummy results can be passed by CLI argument ``--test-params``
+or specified in ``Parameters`` section of testcase definition.
+
+Example of testcase execution with dummy results defined by CLI argument:
+
+.. code-block:: console
+
+    $ ./vsperf back2back --trafficgen Dummy --test-params \
+      "TRAFFICGEN_DUMMY_RESULTS={'b2b frames':'3000','b2b frame loss %':'0.0'}"
+
+Example of testcase definition with pre-configured dummy results:
+
+.. code-block:: python
+
+    {
+        "Name": "back2back",
+        "Traffic Type": "rfc2544_back2back",
+        "Deployment": "p2p",
+        "biDirectional": "True",
+        "Description": "LTD.Throughput.RFC2544.BackToBackFrames",
+        "Parameters" : {
+            'TRAFFICGEN_DUMMY_RESULTS' : {'b2b frames':'3000','b2b frame loss %':'0.0'}
+        },
+    },
+
+**NOTE:** Pre-configured results for the Dummy traffic generator will be used only
+in case, that the Dummy traffic generator is used. Otherwise the option
+``TRAFFICGEN_DUMMY_RESULTS`` will be ignored.
+
+.. _Ixia:
+
+Ixia
+----
+
+VSPERF can use both IxNetwork and IxExplorer TCL servers to control Ixia chassis.
+However usage of IxNetwork TCL server is a preferred option. Following sections
+will describe installation and configuration of IxNetwork components used by VSPERF.
+
+Installation
+~~~~~~~~~~~~
+
+On the system under the test you need to install IxNetworkTclClient$(VER\_NUM)Linux.bin.tgz.
+
+On the IXIA client software system you need to install IxNetwork TCL server. After its
+installation you should configure it as follows:
+
+    1. Find the IxNetwork TCL server app (start -> All Programs -> IXIA ->
+       IxNetwork -> IxNetwork\_$(VER\_NUM) -> IxNetwork TCL Server)
+    2. Right click on IxNetwork TCL Server, select properties - Under shortcut tab in
+       the Target dialogue box make sure there is the argument "-tclport xxxx"
+       where xxxx is your port number (take note of this port number as you will
+       need it for the 10\_custom.conf file).
+
+       .. image:: TCLServerProperties.png
+
+    3. Hit Ok and start the TCL server application
+
+VSPERF configuration
+~~~~~~~~~~~~~~~~~~~~
+
+There are several configuration options specific to the IxNetwork traffic generator
+from IXIA. It is essential to set them correctly, before the VSPERF is executed
+for the first time.
+
+Detailed description of options follows:
+
+ * ``TRAFFICGEN_IXNET_MACHINE`` - IP address of server, where IxNetwork TCL Server is running
+ * ``TRAFFICGEN_IXNET_PORT`` - PORT, where IxNetwork TCL Server is accepting connections from
+   TCL clients
+ * ``TRAFFICGEN_IXNET_USER`` - username, which will be used during communication with IxNetwork
+   TCL Server and IXIA chassis
+ * ``TRAFFICGEN_IXIA_HOST`` - IP address of IXIA traffic generator chassis
+ * ``TRAFFICGEN_IXIA_CARD`` - identification of card with dedicated ports at IXIA chassis
+ * ``TRAFFICGEN_IXIA_PORT1`` - identification of the first dedicated port at ``TRAFFICGEN_IXIA_CARD``
+   at IXIA chassis; VSPERF uses two separated ports for traffic generation. In case of
+   unidirectional traffic, it is essential to correctly connect 1st IXIA port to the 1st NIC
+   at DUT, i.e. to the first PCI handle from ``WHITELIST_NICS`` list. Otherwise traffic may not
+   be able to pass through the vSwitch.
+ * ``TRAFFICGEN_IXIA_PORT2`` - identification of the second dedicated port at ``TRAFFICGEN_IXIA_CARD``
+   at IXIA chassis; VSPERF uses two separated ports for traffic generation. In case of
+   unidirectional traffic, it is essential to correctly connect 2nd IXIA port to the 2nd NIC
+   at DUT, i.e. to the second PCI handle from ``WHITELIST_NICS`` list. Otherwise traffic may not
+   be able to pass through the vSwitch.
+ * ``TRAFFICGEN_IXNET_LIB_PATH`` - path to the DUT specific installation of IxNetwork TCL API
+ * ``TRAFFICGEN_IXNET_TCL_SCRIPT`` - name of the TCL script, which VSPERF will use for
+   communication with IXIA TCL server
+ * ``TRAFFICGEN_IXNET_TESTER_RESULT_DIR`` - folder accessible from IxNetwork TCL server,
+   where test results are stored, e.g. ``c:/ixia_results``; see test-results-share_
+ * ``TRAFFICGEN_IXNET_DUT_RESULT_DIR`` - directory accessible from the DUT, where test
+   results from IxNetwork TCL server are stored, e.g. ``/mnt/ixia_results``; see
+   test-results-share_
+
+.. _test-results-share:
+
+Test results share
+~~~~~~~~~~~~~~~~~~
+
+VSPERF is not able to retrieve test results via TCL API directly. Instead, all test
+results are stored at IxNetwork TCL server. Results are stored at folder defined by
+``TRAFFICGEN_IXNET_TESTER_RESULT_DIR`` configuration parameter. Content of this
+folder must be shared (e.g. via samba protocol) between TCL Server and DUT, where
+VSPERF is executed. VSPERF expects, that test results will be available at directory
+configured by ``TRAFFICGEN_IXNET_DUT_RESULT_DIR`` configuration parameter.
+
+Example of sharing configuration:
+
+ * Create a new folder at IxNetwork TCL server machine, e.g. ``c:\ixia_results``
+ * Modify sharing options of ``ixia_results`` folder to share it with everybody
+ * Create a new directory at DUT, where shared directory with results
+   will be mounted, e.g. ``/mnt/ixia_results``
+ * Update your custom VSPERF configuration file as follows:
+
+   .. code-block:: python
+
+       TRAFFICGEN_IXNET_TESTER_RESULT_DIR = 'c:/ixia_results'
+       TRAFFICGEN_IXNET_DUT_RESULT_DIR = '/mnt/ixia_results'
+
+   **NOTE:** It is essential to use slashes '/' also in path
+   configured by ``TRAFFICGEN_IXNET_TESTER_RESULT_DIR`` parameter.
+ * Install cifs-utils package.
+
+   e.g. at rpm based Linux distribution:
+
+   .. code-block:: console
+
+       yum install cifs-utils
+
+ * Mount shared directory, so VSPERF can access test results.
+
+   e.g. by adding new record into ``/etc/fstab``
+
+   .. code-block:: console
+
+       mount -t cifs //_TCL_SERVER_IP_OR_FQDN_/ixia_results /mnt/ixia_results
+             -o file_mode=0777,dir_mode=0777,nounix
+
+It is recommended to verify, that any new file inserted into ``c:/ixia_results`` folder
+is visible at DUT inside ``/mnt/ixia_results`` directory.
+
+.. _`Spirent TestCenter`:
+
+Spirent Setup
+-------------
+
+Spirent installation files and instructions are available on the
+Spirent support website at:
+
+http://support.spirent.com
+
+Select a version of Spirent TestCenter software to utilize. This example
+will use Spirent TestCenter v4.57 as an example. Substitute the appropriate
+version in place of 'v4.57' in the examples, below.
+
+On the CentOS 7 System
+~~~~~~~~~~~~~~~~~~~~~~
+
+Download and install the following:
+
+Spirent TestCenter Application, v4.57 for 64-bit Linux Client
+
+Spirent Virtual Deployment Service (VDS)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Spirent VDS is required for both TestCenter hardware and virtual
+chassis in the vsperf environment. For installation, select the version
+that matches the Spirent TestCenter Application version. For v4.57,
+the matching VDS version is 1.0.55. Download either the ova (VMware)
+or qcow2 (QEMU) image and create a VM with it. Initialize the VM
+according to Spirent installation instructions.
+
+Using Spirent TestCenter Virtual (STCv)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+STCv is available in both ova (VMware) and qcow2 (QEMU) formats. For
+VMware, download:
+
+Spirent TestCenter Virtual Machine for VMware, v4.57 for Hypervisor - VMware ESX.ESXi
+
+Virtual test port performance is affected by the hypervisor configuration. For
+best practice results in deploying STCv, the following is suggested:
+
+- Create a single VM with two test ports rather than two VMs with one port each
+- Set STCv in DPDK mode
+- Give STCv 2*n + 1 cores, where n = the number of ports. For vsperf, cores = 5.
+- Turning off hyperthreading and pinning these cores will improve performance
+- Give STCv 2 GB of RAM
+
+To get the highest performance and accuracy, Spirent TestCenter hardware is
+recommended. vsperf can run with either stype test ports.
+
+Using STC REST Client
+~~~~~~~~~~~~~~~~~~~~~
+The stcrestclient package provides the stchttp.py ReST API wrapper module.
+This allows simple function calls, nearly identical to those provided by
+StcPython.py, to be used to access TestCenter server sessions via the
+STC ReST API. Basic ReST functionality is provided by the resthttp module,
+and may be used for writing ReST clients independent of STC.
+
+- Project page: <https://github.com/Spirent/py-stcrestclient>
+- Package download: <http://pypi.python.org/pypi/stcrestclient>
+
+To use REST interface, follow the instructions in the Project page to
+install the package. Once installed, the scripts named with 'rest' keyword
+can be used. For example: testcenter-rfc2544-rest.py can be used to run
+RFC 2544 tests using the REST interface.
+
+Configuration:
+~~~~~~~~~~~~~~
+
+1. The Labserver and license server addresses. These parameters applies to
+   all the tests, and are mandatory for all tests.
+
+.. code-block:: console
+
+    TRAFFICGEN_STC_LAB_SERVER_ADDR = " "
+    TRAFFICGEN_STC_LICENSE_SERVER_ADDR = " "
+    TRAFFICGEN_STC_PYTHON2_PATH = " "
+    TRAFFICGEN_STC_TESTCENTER_PATH = " "
+    TRAFFICGEN_STC_TEST_SESSION_NAME = " "
+    TRAFFICGEN_STC_CSV_RESULTS_FILE_PREFIX = " "
+
+2. For RFC2544 tests, the following parameters are mandatory
+
+.. code-block:: console
+
+    TRAFFICGEN_STC_EAST_CHASSIS_ADDR = " "
+    TRAFFICGEN_STC_EAST_SLOT_NUM = " "
+    TRAFFICGEN_STC_EAST_PORT_NUM = " "
+    TRAFFICGEN_STC_EAST_INTF_ADDR = " "
+    TRAFFICGEN_STC_EAST_INTF_GATEWAY_ADDR = " "
+    TRAFFICGEN_STC_WEST_CHASSIS_ADDR = ""
+    TRAFFICGEN_STC_WEST_SLOT_NUM = " "
+    TRAFFICGEN_STC_WEST_PORT_NUM = " "
+    TRAFFICGEN_STC_WEST_INTF_ADDR = " "
+    TRAFFICGEN_STC_WEST_INTF_GATEWAY_ADDR = " "
+    TRAFFICGEN_STC_RFC2544_TPUT_TEST_FILE_NAME
+
+3. RFC2889 tests: Currently, the forwarding, address-caching, and
+   address-learning-rate tests of RFC2889 are supported.
+   The testcenter-rfc2889-rest.py script implements the rfc2889 tests.
+   The configuration for RFC2889 involves test-case definition, and parameter
+   definition, as described below. New results-constants, as shown below, are
+   added to support these tests.
+
+Example of testcase definition for RFC2889 tests:
+
+.. code-block:: python
+
+    {
+        "Name": "phy2phy_forwarding",
+        "Deployment": "p2p",
+        "Description": "LTD.Forwarding.RFC2889.MaxForwardingRate",
+        "Parameters" : {
+            "TRAFFIC" : {
+                "traffic_type" : "rfc2889_forwarding",
+            },
+        },
+    }
+
+For RFC2889 tests, specifying the locations for the monitoring ports is mandatory.
+Necessary parameters are:
+
+.. code-block:: console
+
+    TRAFFICGEN_STC_RFC2889_TEST_FILE_NAME
+    TRAFFICGEN_STC_EAST_CHASSIS_ADDR = " "
+    TRAFFICGEN_STC_EAST_SLOT_NUM = " "
+    TRAFFICGEN_STC_EAST_PORT_NUM = " "
+    TRAFFICGEN_STC_EAST_INTF_ADDR = " "
+    TRAFFICGEN_STC_EAST_INTF_GATEWAY_ADDR = " "
+    TRAFFICGEN_STC_WEST_CHASSIS_ADDR = ""
+    TRAFFICGEN_STC_WEST_SLOT_NUM = " "
+    TRAFFICGEN_STC_WEST_PORT_NUM = " "
+    TRAFFICGEN_STC_WEST_INTF_ADDR = " "
+    TRAFFICGEN_STC_WEST_INTF_GATEWAY_ADDR = " "
+    TRAFFICGEN_STC_VERBOSE = "True"
+    TRAFFICGEN_STC_RFC2889_LOCATIONS="//10.1.1.1/1/1,//10.1.1.1/2/2"
+
+Other Configurations are :
+
+.. code-block:: console
+
+    TRAFFICGEN_STC_RFC2889_MIN_LR = 1488
+    TRAFFICGEN_STC_RFC2889_MAX_LR = 14880
+    TRAFFICGEN_STC_RFC2889_MIN_ADDRS = 1000
+    TRAFFICGEN_STC_RFC2889_MAX_ADDRS = 65536
+    TRAFFICGEN_STC_RFC2889_AC_LR = 1000
+
+The first 2 values are for address-learning test where as other 3 values are
+for the Address caching capacity test. LR: Learning Rate. AC: Address Caching.
+Maximum value for address is 16777216. Whereas, maximum for LR is 4294967295.
+
+Results for RFC2889 Tests: Forwarding tests outputs following values:
+
+.. code-block:: console
+
+    TX_RATE_FPS : "Transmission Rate in Frames/sec"
+    THROUGHPUT_RX_FPS: "Received Throughput Frames/sec"
+    TX_RATE_MBPS : " Transmission rate in MBPS"
+    THROUGHPUT_RX_MBPS: "Received Throughput in MBPS"
+    TX_RATE_PERCENT: "Transmission Rate in Percentage"
+    FRAME_LOSS_PERCENT: "Frame loss in Percentage"
+    FORWARDING_RATE_FPS: " Maximum Forwarding Rate in FPS"
+
+
+Whereas, the address caching test outputs following values,
+
+.. code-block:: console
+
+    CACHING_CAPACITY_ADDRS = 'Number of address it can cache'
+    ADDR_LEARNED_PERCENT = 'Percentage of address successfully learned'
+
+and address learning test outputs just a single value:
+
+.. code-block:: console
+
+    OPTIMAL_LEARNING_RATE_FPS = 'Optimal learning rate in fps'
+
+Note that 'FORWARDING_RATE_FPS', 'CACHING_CAPACITY_ADDRS',
+'ADDR_LEARNED_PERCENT' and 'OPTIMAL_LEARNING_RATE_FPS' are the new
+result-constants added to support RFC2889 tests.
+
+.. _`Xena Networks`:
+
+Xena Networks
+-------------
+
+Installation
+~~~~~~~~~~~~
+
+Xena Networks traffic generator requires specific files and packages to be
+installed. It is assumed the user has access to the Xena2544.exe file which
+must be placed in VSPerf installation location under the tools/pkt_gen/xena
+folder. Contact Xena Networks for the latest version of this file. The user
+can also visit www.xenanetworks/downloads to obtain the file with a valid
+support contract.
+
+**Note** VSPerf has been fully tested with version v2.43 of Xena2544.exe
+
+To execute the Xena2544.exe file under Linux distributions the mono-complete
+package must be installed. To install this package follow the instructions
+below. Further information can be obtained from
+http://www.mono-project.com/docs/getting-started/install/linux/
+
+.. code-block:: console
+
+    rpm --import "http://keyserver.ubuntu.com/pks/lookup?op=get&search=0x3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF"
+    yum-config-manager --add-repo http://download.mono-project.com/repo/centos/
+    yum -y install mono-complete
+
+To prevent gpg errors on future yum installation of packages the mono-project
+repo should be disabled once installed.
+
+.. code-block:: console
+
+    yum-config-manager --disable download.mono-project.com_repo_centos_
+
+Configuration
+~~~~~~~~~~~~~
+
+Connection information for your Xena Chassis must be supplied inside the
+``10_custom.conf`` or ``03_custom.conf`` file. The following parameters must be
+set to allow for proper connections to the chassis.
+
+.. code-block:: console
+
+    TRAFFICGEN_XENA_IP = ''
+    TRAFFICGEN_XENA_PORT1 = ''
+    TRAFFICGEN_XENA_PORT2 = ''
+    TRAFFICGEN_XENA_USER = ''
+    TRAFFICGEN_XENA_PASSWORD = ''
+    TRAFFICGEN_XENA_MODULE1 = ''
+    TRAFFICGEN_XENA_MODULE2 = ''
+
+RFC2544 Throughput Testing
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Xena traffic generator testing for rfc2544 throughput can be modified for
+different behaviors if needed. The default options for the following are
+optimized for best results.
+
+.. code-block:: console
+
+    TRAFFICGEN_XENA_2544_TPUT_INIT_VALUE = '10.0'
+    TRAFFICGEN_XENA_2544_TPUT_MIN_VALUE = '0.1'
+    TRAFFICGEN_XENA_2544_TPUT_MAX_VALUE = '100.0'
+    TRAFFICGEN_XENA_2544_TPUT_VALUE_RESOLUTION = '0.5'
+    TRAFFICGEN_XENA_2544_TPUT_USEPASS_THRESHHOLD = 'false'
+    TRAFFICGEN_XENA_2544_TPUT_PASS_THRESHHOLD = '0.0'
+
+Each value modifies the behavior of rfc 2544 throughput testing. Refer to your
+Xena documentation to understand the behavior changes in modifying these
+values.
+
+Continuous Traffic Testing
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Xena continuous traffic by default does a 3 second learning preemption to allow
+the DUT to receive learning packets before a continuous test is performed. If
+a custom test case requires this learning be disabled, you can disable the option
+or modify the length of the learning by modifying the following settings.
+
+.. code-block:: console
+
+    TRAFFICGEN_XENA_CONT_PORT_LEARNING_ENABLED = False
+    TRAFFICGEN_XENA_CONT_PORT_LEARNING_DURATION = 3
+
+MoonGen
+-------
+
+Installation
+~~~~~~~~~~~~
+
+MoonGen architecture overview and general installation instructions
+can be found here:
+
+https://github.com/emmericp/MoonGen
+
+* Note:  Today, MoonGen with VSPERF only supports 10Gbps line speeds.
+
+For VSPERF use, MoonGen should be cloned from here (as opposed to the
+previously mentioned GitHub):
+
+git clone https://github.com/atheurer/lua-trafficgen
+
+and use the master branch:
+
+git checkout master
+
+VSPERF uses a particular Lua script with the MoonGen project:
+
+trafficgen.lua
+
+Follow MoonGen set up and execution instructions here:
+
+https://github.com/atheurer/lua-trafficgen/blob/master/README.md
+
+Note one will need to set up ssh login to not use passwords between the server
+running MoonGen and the device under test (running the VSPERF test
+infrastructure).  This is because VSPERF on one server uses 'ssh' to
+configure and run MoonGen upon the other server.
+
+One can set up this ssh access by doing the following on both servers:
+
+.. code-block:: console
+
+    ssh-keygen -b 2048 -t rsa
+    ssh-copy-id <other server>
+
+Configuration
+~~~~~~~~~~~~~
+
+Connection information for MoonGen must be supplied inside the
+``10_custom.conf`` or ``03_custom.conf`` file. The following parameters must be
+set to allow for proper connections to the host with MoonGen.
+
+.. code-block:: console
+
+    TRAFFICGEN_MOONGEN_HOST_IP_ADDR = ""
+    TRAFFICGEN_MOONGEN_USER = ""
+    TRAFFICGEN_MOONGEN_BASE_DIR = ""
+    TRAFFICGEN_MOONGEN_PORTS = ""
+    TRAFFICGEN_MOONGEN_LINE_SPEED_GBPS = ""
diff --git a/docs/testing/user/configguide/upgrade.rst b/docs/testing/user/configguide/upgrade.rst
new file mode 100644
index 00000000..cf92572c
--- /dev/null
+++ b/docs/testing/user/configguide/upgrade.rst
@@ -0,0 +1,183 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation, AT&T and others.
+
+=====================
+Upgrading vswitchperf
+=====================
+
+Generic
+-------
+
+In case, that VSPERF is cloned from git repository, then it is easy to
+upgrade it to the newest stable version or to the development version.
+
+You could get a list of stable releases by ``git`` command. It is necessary
+to update local git repository first.
+
+**NOTE:** Git commands must be executed from directory, where VSPERF repository
+was cloned, e.g. ``vswitchperf``.
+
+Update of local git repository:
+
+.. code:: bash
+
+   $ git pull
+
+List of stable releases:
+
+.. code:: bash
+
+   $ git tag
+
+   brahmaputra.1.0
+   colorado.1.0
+   colorado.2.0
+   colorado.3.0
+   danube.1.0
+
+You could select which stable release should be used. For example, select ``danube.1.0``:
+
+.. code:: bash
+
+   $ git checkout danube.1.0
+
+
+Development version of VSPERF can be selected by:
+
+.. code:: bash
+
+   $ git checkout master
+
+Colorado to Danube upgrade notes
+--------------------------------
+
+Obsoleted features
+~~~~~~~~~~~~~~~~~~
+
+Support of vHost Cuse interface has been removed in Danube release. It means,
+that it is not possible to select ``QemuDpdkVhostCuse`` as a VNF anymore. Option
+``QemuDpdkVhostUser`` should be used instead. Please check you configuration files
+and definition of your testcases for any occurrence of:
+
+.. code:: python
+
+   VNF = "QemuDpdkVhostCuse"
+
+or
+
+.. code:: python
+
+   "VNF" : "QemuDpdkVhostCuse"
+
+In case that ``QemuDpdkVhostCuse`` is found, it must be modified to ``QemuDpdkVhostUser``.
+
+**NOTE:** In case that execution of VSPERF is automated by scripts (e.g. for
+CI purposes), then these scripts must be checked and updated too. It means,
+that any occurrence of:
+
+.. code:: bash
+
+   ./vsperf --vnf QemuDpdkVhostCuse
+
+must be updated to:
+
+.. code:: bash
+
+   ./vsperf --vnf QemuDpdkVhostUser
+
+Configuration
+~~~~~~~~~~~~~
+
+Several configuration changes were introduced during Danube release. The most
+important changes are discussed below.
+
+Paths to DPDK, OVS and QEMU
+===========================
+
+VSPERF uses external tools for proper testcase execution. Thus it is important
+to properly configure paths to these tools. In case that tools are installed
+by installation scripts and are located inside ``./src`` directory inside
+VSPERF home, then no changes are needed. On the other hand, if path settings
+was changed by custom configuration file, then it is required to update configuration
+accordingly. Please check your configuration files for following configuration
+options:
+
+.. code:: bash
+
+   OVS_DIR
+   OVS_DIR_VANILLA
+   OVS_DIR_USER
+   OVS_DIR_CUSE
+
+   RTE_SDK_USER
+   RTE_SDK_CUSE
+
+   QEMU_DIR
+   QEMU_DIR_USER
+   QEMU_DIR_CUSE
+   QEMU_BIN
+
+In case that any of these options is defined, then configuration must be updated.
+All paths to the tools are now stored inside ``PATHS`` dictionary. Please
+refer to the :ref:`paths-documentation` and update your configuration where necessary.
+
+Configuration change via CLI
+============================
+
+In previous releases it was possible to modify selected configuration options
+(mostly VNF specific) via command line interface, i.e. by ``--test-params``
+argument. This concept has been generalized in Danube release and it is
+possible to modify any configuration parameter via CLI or via **Parameters**
+section of the testcase definition. Old configuration options were obsoleted
+and it is required to specify configuration parameter name in the same form
+as it is defined inside configuration file, i.e. in uppercase. Please
+refer to the :ref:`overriding-parameters-documentation` for additional details.
+
+**NOTE:** In case that execution of VSPERF is automated by scripts (e.g. for
+CI purposes), then these scripts must be checked and updated too. It means,
+that any occurrence of
+
+.. code:: bash
+
+   guest_loopback
+   vanilla_tgen_port1_ip
+   vanilla_tgen_port1_mac
+   vanilla_tgen_port2_ip
+   vanilla_tgen_port2_mac
+   tunnel_type
+
+shall be changed to the uppercase form and data type of entered values must
+match to data types of original values from configuration files.
+
+In case that ``guest_nic1_name`` or ``guest_nic2_name`` is changed,
+then new dictionary ``GUEST_NICS`` must be modified accordingly.
+Please see :ref:`configuration-of-guest-options` and ``conf/04_vnf.conf`` for additional
+details.
+
+Traffic configuration via CLI
+=============================
+
+In previous releases it was possible to modify selected attributes of generated
+traffic via command line interface. This concept has been enhanced in Danube
+release and it is now possible to modify all traffic specific options via
+CLI or by ``TRAFFIC`` dictionary in configuration file. Detailed description
+is available at :ref:`configuration-of-traffic-dictionary` section of documentation.
+
+Please check your automated scripts for VSPERF execution for following CLI
+parameters and update them according to the documentation:
+
+.. code:: bash
+
+   bidir
+   duration
+   frame_rate
+   iload
+   lossrate
+   multistream
+   pkt_sizes
+   pre-installed_flows
+   rfc2544_tests
+   stream_type
+   traffic_type
+
diff --git a/docs/testing/user/userguide/integration.rst b/docs/testing/user/userguide/integration.rst
new file mode 100644
index 00000000..83b29da6
--- /dev/null
+++ b/docs/testing/user/userguide/integration.rst
@@ -0,0 +1,504 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation, AT&T and others.
+
+.. _integration-tests:
+
+Integration tests
+=================
+
+VSPERF includes a set of integration tests defined in conf/integration.
+These tests can be run by specifying --integration as a parameter to vsperf.
+Current tests in conf/integration include switch functionality and Overlay
+tests.
+
+Tests in the conf/integration can be used to test scaling of different switch
+configurations by adding steps into the test case.
+
+For the overlay tests VSPERF supports VXLAN, GRE and GENEVE tunneling protocols.
+Testing of these protocols is limited to unidirectional traffic and
+P2P (Physical to Physical scenarios).
+
+NOTE: The configuration for overlay tests provided in this guide is for
+unidirectional traffic only.
+
+Executing Integration Tests
+---------------------------
+
+To execute integration tests VSPERF is run with the integration parameter. To
+view the current test list simply execute the following command:
+
+.. code-block:: console
+
+    ./vsperf --integration --list
+
+The standard tests included are defined inside the
+``conf/integration/01_testcases.conf`` file.
+
+Executing Tunnel encapsulation tests
+------------------------------------
+
+The VXLAN OVS DPDK encapsulation tests requires IPs, MAC addresses,
+bridge names and WHITELIST_NICS for DPDK.
+
+NOTE: Only Ixia traffic generators currently support the execution of the tunnel
+encapsulation tests. Support for other traffic generators may come in a future
+release.
+
+Default values are already provided. To customize for your environment, override
+the following variables in you user_settings.py file:
+
+  .. code-block:: python
+
+    # Variables defined in conf/integration/02_vswitch.conf
+    # Tunnel endpoint for Overlay P2P deployment scenario
+    # used for br0
+    VTEP_IP1 = '192.168.0.1/24'
+
+    # Used as remote_ip in adding OVS tunnel port and
+    # to set ARP entry in OVS (e.g. tnl/arp/set br-ext 192.168.240.10 02:00:00:00:00:02
+    VTEP_IP2 = '192.168.240.10'
+
+    # Network to use when adding a route for inner frame data
+    VTEP_IP2_SUBNET = '192.168.240.0/24'
+
+    # Bridge names
+    TUNNEL_INTEGRATION_BRIDGE = 'br0'
+    TUNNEL_EXTERNAL_BRIDGE = 'br-ext'
+
+    # IP of br-ext
+    TUNNEL_EXTERNAL_BRIDGE_IP = '192.168.240.1/24'
+
+    # vxlan|gre|geneve
+    TUNNEL_TYPE = 'vxlan'
+
+    # Variables defined conf/integration/03_traffic.conf
+    # For OP2P deployment scenario
+    TRAFFICGEN_PORT1_MAC = '02:00:00:00:00:01'
+    TRAFFICGEN_PORT2_MAC = '02:00:00:00:00:02'
+    TRAFFICGEN_PORT1_IP = '1.1.1.1'
+    TRAFFICGEN_PORT2_IP = '192.168.240.10'
+
+To run VXLAN encapsulation tests:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --integration \
+             --test-params 'TUNNEL_TYPE=vxlan' overlay_p2p_tput
+
+To run GRE encapsulation tests:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --integration \
+             --test-params 'TUNNEL_TYPE=gre' overlay_p2p_tput
+
+To run GENEVE encapsulation tests:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --integration \
+             --test-params 'TUNNEL_TYPE=geneve' overlay_p2p_tput
+
+To run OVS NATIVE tunnel tests (VXLAN/GRE/GENEVE):
+
+1. Install the OVS kernel modules
+
+  .. code:: console
+
+    cd src/ovs/ovs
+    sudo -E make modules_install
+
+2. Set the following variables:
+
+  .. code-block:: python
+
+    VSWITCH = 'OvsVanilla'
+    # Specify vport_* kernel module to test.
+    PATHS['vswitch']['OvsVanilla']['src']['modules'] = [
+        'vport_vxlan',
+        'vport_gre',
+        'vport_geneve',
+        'datapath/linux/openvswitch.ko',
+    ]
+
+  **NOTE:** In case, that Vanilla OVS is installed from binary package, then
+  please set ``PATHS['vswitch']['OvsVanilla']['bin']['modules']`` instead.
+
+3. Run tests:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --integration \
+             --test-params 'TUNNEL_TYPE=vxlan' overlay_p2p_tput
+
+
+Executing VXLAN decapsulation tests
+------------------------------------
+
+To run VXLAN decapsulation tests:
+
+1. Set the variables used in "Executing Tunnel encapsulation tests"
+
+2. Set dstmac of DUT_NIC2_MAC to the MAC adddress of the 2nd NIC of your DUT
+
+  .. code-block:: python
+
+    DUT_NIC2_MAC = '<DUT NIC2 MAC>'
+
+3. Run test:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --integration overlay_p2p_decap_cont
+
+If you want to use different values for your VXLAN frame, you may set:
+
+  .. code-block:: python
+
+    VXLAN_FRAME_L3 = {'proto': 'udp',
+                      'packetsize': 64,
+                      'srcip': TRAFFICGEN_PORT1_IP,
+                      'dstip': '192.168.240.1',
+                     }
+    VXLAN_FRAME_L4 = {'srcport': 4789,
+                      'dstport': 4789,
+                      'vni': VXLAN_VNI,
+                      'inner_srcmac': '01:02:03:04:05:06',
+                      'inner_dstmac': '06:05:04:03:02:01',
+                      'inner_srcip': '192.168.0.10',
+                      'inner_dstip': '192.168.240.9',
+                      'inner_proto': 'udp',
+                      'inner_srcport': 3000,
+                      'inner_dstport': 3001,
+                     }
+
+
+Executing GRE decapsulation tests
+---------------------------------
+
+To run GRE decapsulation tests:
+
+1. Set the variables used in "Executing Tunnel encapsulation tests"
+
+2. Set dstmac of DUT_NIC2_MAC to the MAC adddress of the 2nd NIC of your DUT
+
+  .. code-block:: python
+
+    DUT_NIC2_MAC = '<DUT NIC2 MAC>'
+
+3. Run test:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --test-params 'TUNNEL_TYPE=gre' \
+             --integration overlay_p2p_decap_cont
+
+
+If you want to use different values for your GRE frame, you may set:
+
+  .. code-block:: python
+
+    GRE_FRAME_L3 = {'proto': 'gre',
+                    'packetsize': 64,
+                    'srcip': TRAFFICGEN_PORT1_IP,
+                    'dstip': '192.168.240.1',
+                   }
+
+    GRE_FRAME_L4 = {'srcport': 0,
+                    'dstport': 0
+                    'inner_srcmac': '01:02:03:04:05:06',
+                    'inner_dstmac': '06:05:04:03:02:01',
+                    'inner_srcip': '192.168.0.10',
+                    'inner_dstip': '192.168.240.9',
+                    'inner_proto': 'udp',
+                    'inner_srcport': 3000,
+                    'inner_dstport': 3001,
+                   }
+
+
+Executing GENEVE decapsulation tests
+------------------------------------
+
+IxNet 7.3X does not have native support of GENEVE protocol. The
+template, GeneveIxNetTemplate.xml_ClearText.xml, should be imported
+into IxNET for this testcase to work.
+
+To import the template do:
+
+1. Run the IxNetwork TCL Server
+2. Click on the Traffic menu
+3. Click on the Traffic actions and click Edit Packet Templates
+4. On the Template editor window, click Import. Select the template
+   located at ``3rd_party/ixia/GeneveIxNetTemplate.xml_ClearText.xml``
+   and click import.
+5. Restart the TCL Server.
+
+To run GENEVE decapsulation tests:
+
+1. Set the variables used in "Executing Tunnel encapsulation tests"
+
+2. Set dstmac of DUT_NIC2_MAC to the MAC adddress of the 2nd NIC of your DUT
+
+  .. code-block:: python
+
+    DUT_NIC2_MAC = '<DUT NIC2 MAC>'
+
+3. Run test:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --test-params 'tunnel_type=geneve' \
+             --integration overlay_p2p_decap_cont
+
+
+If you want to use different values for your GENEVE frame, you may set:
+
+  .. code-block:: python
+
+    GENEVE_FRAME_L3 = {'proto': 'udp',
+                       'packetsize': 64,
+                       'srcip': TRAFFICGEN_PORT1_IP,
+                       'dstip': '192.168.240.1',
+                      }
+
+    GENEVE_FRAME_L4 = {'srcport': 6081,
+                       'dstport': 6081,
+                       'geneve_vni': 0,
+                       'inner_srcmac': '01:02:03:04:05:06',
+                       'inner_dstmac': '06:05:04:03:02:01',
+                       'inner_srcip': '192.168.0.10',
+                       'inner_dstip': '192.168.240.9',
+                       'inner_proto': 'udp',
+                       'inner_srcport': 3000,
+                       'inner_dstport': 3001,
+                      }
+
+
+Executing Native/Vanilla OVS VXLAN decapsulation tests
+------------------------------------------------------
+
+To run VXLAN decapsulation tests:
+
+1. Set the following variables in your user_settings.py file:
+
+  .. code-block:: python
+
+    PATHS['vswitch']['OvsVanilla']['src']['modules'] = [
+        'vport_vxlan',
+        'datapath/linux/openvswitch.ko',
+    ]
+
+    DUT_NIC1_MAC = '<DUT NIC1 MAC ADDRESS>'
+
+    TRAFFICGEN_PORT1_IP = '172.16.1.2'
+    TRAFFICGEN_PORT2_IP = '192.168.1.11'
+
+    VTEP_IP1 = '172.16.1.2/24'
+    VTEP_IP2 = '192.168.1.1'
+    VTEP_IP2_SUBNET = '192.168.1.0/24'
+    TUNNEL_EXTERNAL_BRIDGE_IP = '172.16.1.1/24'
+    TUNNEL_INT_BRIDGE_IP = '192.168.1.1'
+
+    VXLAN_FRAME_L2 = {'srcmac':
+                      '01:02:03:04:05:06',
+                      'dstmac': DUT_NIC1_MAC
+                     }
+
+    VXLAN_FRAME_L3 = {'proto': 'udp',
+                      'packetsize': 64,
+                      'srcip': TRAFFICGEN_PORT1_IP,
+                      'dstip': '172.16.1.1',
+                     }
+
+    VXLAN_FRAME_L4 = {
+                      'srcport': 4789,
+                      'dstport': 4789,
+                      'protocolpad': 'true',
+                      'vni': 99,
+                      'inner_srcmac': '01:02:03:04:05:06',
+                      'inner_dstmac': '06:05:04:03:02:01',
+                      'inner_srcip': '192.168.1.2',
+                      'inner_dstip': TRAFFICGEN_PORT2_IP,
+                      'inner_proto': 'udp',
+                      'inner_srcport': 3000,
+                      'inner_dstport': 3001,
+                     }
+
+  **NOTE:** In case, that Vanilla OVS is installed from binary package, then
+  please set ``PATHS['vswitch']['OvsVanilla']['bin']['modules']`` instead.
+
+2. Run test:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --integration \
+             --test-params 'tunnel_type=vxlan' overlay_p2p_decap_cont
+
+Executing Native/Vanilla OVS GRE decapsulation tests
+----------------------------------------------------
+
+To run GRE decapsulation tests:
+
+1. Set the following variables in your user_settings.py file:
+
+  .. code-block:: python
+
+    PATHS['vswitch']['OvsVanilla']['src']['modules'] = [
+        'vport_gre',
+        'datapath/linux/openvswitch.ko',
+    ]
+
+    DUT_NIC1_MAC = '<DUT NIC1 MAC ADDRESS>'
+
+    TRAFFICGEN_PORT1_IP = '172.16.1.2'
+    TRAFFICGEN_PORT2_IP = '192.168.1.11'
+
+    VTEP_IP1 = '172.16.1.2/24'
+    VTEP_IP2 = '192.168.1.1'
+    VTEP_IP2_SUBNET = '192.168.1.0/24'
+    TUNNEL_EXTERNAL_BRIDGE_IP = '172.16.1.1/24'
+    TUNNEL_INT_BRIDGE_IP = '192.168.1.1'
+
+    GRE_FRAME_L2 = {'srcmac':
+                    '01:02:03:04:05:06',
+                    'dstmac': DUT_NIC1_MAC
+                   }
+
+    GRE_FRAME_L3 = {'proto': 'udp',
+                    'packetsize': 64,
+                    'srcip': TRAFFICGEN_PORT1_IP,
+                    'dstip': '172.16.1.1',
+                   }
+
+    GRE_FRAME_L4 = {
+                    'srcport': 4789,
+                    'dstport': 4789,
+                    'protocolpad': 'true',
+                    'inner_srcmac': '01:02:03:04:05:06',
+                    'inner_dstmac': '06:05:04:03:02:01',
+                    'inner_srcip': '192.168.1.2',
+                    'inner_dstip': TRAFFICGEN_PORT2_IP,
+                    'inner_proto': 'udp',
+                    'inner_srcport': 3000,
+                    'inner_dstport': 3001,
+                   }
+
+  **NOTE:** In case, that Vanilla OVS is installed from binary package, then
+  please set ``PATHS['vswitch']['OvsVanilla']['bin']['modules']`` instead.
+
+2. Run test:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --integration \
+             --test-params 'tunnel_type=gre' overlay_p2p_decap_cont
+
+Executing Native/Vanilla OVS GENEVE decapsulation tests
+-------------------------------------------------------
+
+To run GENEVE decapsulation tests:
+
+1. Set the following variables in your user_settings.py file:
+
+  .. code-block:: python
+
+    PATHS['vswitch']['OvsVanilla']['src']['modules'] = [
+        'vport_geneve',
+        'datapath/linux/openvswitch.ko',
+    ]
+
+    DUT_NIC1_MAC = '<DUT NIC1 MAC ADDRESS>'
+
+    TRAFFICGEN_PORT1_IP = '172.16.1.2'
+    TRAFFICGEN_PORT2_IP = '192.168.1.11'
+
+    VTEP_IP1 = '172.16.1.2/24'
+    VTEP_IP2 = '192.168.1.1'
+    VTEP_IP2_SUBNET = '192.168.1.0/24'
+    TUNNEL_EXTERNAL_BRIDGE_IP = '172.16.1.1/24'
+    TUNNEL_INT_BRIDGE_IP = '192.168.1.1'
+
+    GENEVE_FRAME_L2 = {'srcmac':
+                       '01:02:03:04:05:06',
+                       'dstmac': DUT_NIC1_MAC
+                      }
+
+    GENEVE_FRAME_L3 = {'proto': 'udp',
+                       'packetsize': 64,
+                       'srcip': TRAFFICGEN_PORT1_IP,
+                       'dstip': '172.16.1.1',
+                      }
+
+    GENEVE_FRAME_L4 = {'srcport': 6081,
+                       'dstport': 6081,
+                       'protocolpad': 'true',
+                       'geneve_vni': 0,
+                       'inner_srcmac': '01:02:03:04:05:06',
+                       'inner_dstmac': '06:05:04:03:02:01',
+                       'inner_srcip': '192.168.1.2',
+                       'inner_dstip': TRAFFICGEN_PORT2_IP,
+                       'inner_proto': 'udp',
+                       'inner_srcport': 3000,
+                       'inner_dstport': 3001,
+                      }
+
+  **NOTE:** In case, that Vanilla OVS is installed from binary package, then
+  please set ``PATHS['vswitch']['OvsVanilla']['bin']['modules']`` instead.
+
+2. Run test:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --integration \
+             --test-params 'tunnel_type=geneve' overlay_p2p_decap_cont
+
+
+Executing Tunnel encapsulation+decapsulation tests
+--------------------------------------------------
+
+The OVS DPDK encapsulation_decapsulation tests requires IPs, MAC addresses,
+bridge names and WHITELIST_NICS for DPDK.
+
+The test cases can test the tunneling encap and decap without using any ingress
+overlay traffic as compared to above test cases. To achieve this the OVS is
+configured to perform encap and decap in a series on the same traffic stream as
+given below.
+
+TRAFFIC-IN --> [ENCAP] --> [MOD-PKT] --> [DECAP] --> TRAFFIC-OUT
+
+
+Default values are already provided. To customize for your environment, override
+the following variables in you user_settings.py file:
+
+  .. code-block:: python
+
+    # Variables defined in conf/integration/02_vswitch.conf
+
+    # Bridge names
+    TUNNEL_EXTERNAL_BRIDGE1 = 'br-phy1'
+    TUNNEL_EXTERNAL_BRIDGE2 = 'br-phy2'
+    TUNNEL_MODIFY_BRIDGE1 = 'br-mod1'
+    TUNNEL_MODIFY_BRIDGE2 = 'br-mod2'
+
+    # IP of br-mod1
+    TUNNEL_MODIFY_BRIDGE_IP1 = '10.0.0.1/24'
+
+    # Mac of br-mod1
+    TUNNEL_MODIFY_BRIDGE_MAC1 = '00:00:10:00:00:01'
+
+    # IP of br-mod2
+    TUNNEL_MODIFY_BRIDGE_IP2 = '20.0.0.1/24'
+
+    #Mac of br-mod2
+    TUNNEL_MODIFY_BRIDGE_MAC2 = '00:00:20:00:00:01'
+
+    # vxlan|gre|geneve, Only VXLAN is supported for now.
+    TUNNEL_TYPE = 'vxlan'
+
+To run VXLAN encapsulation+decapsulation tests:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --integration \
+             overlay_p2p_mod_tput
diff --git a/docs/testing/user/userguide/teststeps.rst b/docs/testing/user/userguide/teststeps.rst
new file mode 100644
index 00000000..870c3d80
--- /dev/null
+++ b/docs/testing/user/userguide/teststeps.rst
@@ -0,0 +1,667 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation, AT&T and others.
+
+.. _step-driven-tests:
+
+Step driven tests
+=================
+
+In general, test scenarios are defined by a ``deployment`` used in the particular
+test case definition. The chosen deployment scenario will take care of the vSwitch
+configuration, deployment of VNFs and it can also affect configuration of a traffic
+generator. In order to allow a more flexible way of testcase scripting, VSPERF supports
+a detailed step driven testcase definition. It can be used to configure and
+program vSwitch, deploy and terminate VNFs, execute a traffic generator,
+modify a VSPERF configuration, execute external commands, etc.
+
+Execution of step driven tests is done on a step by step work flow starting
+with step 0 as defined inside the test case. Each step of the test increments
+the step number by one which is indicated in the log.
+
+.. code-block:: console
+
+    (testcases.integration) - Step 0 'vswitch add_vport ['br0']' start
+
+Step driven tests can be used for both performance and integration testing.
+In case of integration test, each step in the test case is validated. If a step
+does not pass validation the test will fail and terminate. The test will continue
+until a failure is detected or all steps pass. A csv report file is generated after
+a test completes with an OK or FAIL result.
+
+In case of performance test, the validation of steps is not performed and
+standard output files with results from traffic generator and underlying OS
+details are generated by vsperf.
+
+Step driven testcases can be used in two different ways:
+
+    # description of full testcase - in this case ``clean`` deployment is used
+      to indicate that vsperf should neither configure vSwitch nor deploy any VNF.
+      Test shall perform all required vSwitch configuration and programming and
+      deploy required number of VNFs.
+
+    # modification of existing deployment - in this case, any of supported
+      deployments can be used to perform initial vSwitch configuration and
+      deployment of VNFs. Additional actions defined by TestSteps can be used
+      to alter vSwitch configuration or deploy additional VNFs. After the last
+      step is processed, the test execution will continue with traffic execution.
+
+Test objects and their functions
+--------------------------------
+
+Every test step can call a function of one of the supported test objects. The list
+of supported objects and their most common functions follows:
+
+    * ``vswitch`` - provides functions for vSwitch configuration
+
+      List of supported functions:
+
+        * ``add_switch br_name`` - creates a new switch (bridge) with given ``br_name``
+        * ``del_switch br_name`` - deletes switch (bridge) with given ``br_name``
+        * ``add_phy_port br_name`` - adds a physical port into bridge specified by ``br_name``
+        * ``add_vport br_name`` - adds a virtual port into bridge specified by ``br_name``
+        * ``del_port br_name port_name`` - removes physical or virtual port specified by
+          ``port_name`` from bridge ``br_name``
+        * ``add_flow br_name flow`` - adds flow specified by ``flow`` dictionary into
+          the bridge ``br_name``; Content of flow dictionary will be passed to the vSwitch.
+          In case of Open vSwitch it will be passed to the ``ovs-ofctl add-flow`` command.
+          Please see Open vSwitch documentation for the list of supported flow parameters.
+        * ``del_flow br_name [flow]`` - deletes flow specified by ``flow`` dictionary from
+          bridge ``br_name``; In case that optional parameter ``flow`` is not specified
+          or set to an empty dictionary ``{}``, then all flows from bridge ``br_name``
+          will be deleted.
+        * ``dump_flows br_name`` - dumps all flows from bridge specified by ``br_name``
+        * ``enable_stp br_name`` - enables Spanning Tree Protocol for bridge ``br_name``
+        * ``disable_stp br_name`` - disables Spanning Tree Protocol for bridge ``br_name``
+        * ``enable_rstp br_name`` - enables Rapid Spanning Tree Protocol for bridge ``br_name``
+        * ``disable_rstp br_name`` - disables Rapid Spanning Tree Protocol for bridge ``br_name``
+
+        Examples:
+
+        .. code-block:: python
+
+            ['vswitch', 'add_switch', 'int_br0']
+
+            ['vswitch', 'del_switch', 'int_br0']
+
+            ['vswitch', 'add_phy_port', 'int_br0']
+
+            ['vswitch', 'del_port', 'int_br0', '#STEP[2][0]']
+
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '1', 'actions': ['output:2'],
+             'idle_timeout': '0'}],
+
+            ['vswitch', 'enable_rstp', 'int_br0']
+
+    * ``vnf[ID]`` - provides functions for deployment and termination of VNFs; Optional
+      alfanumerical ``ID`` is used for VNF identification in case that testcase
+      deploys multiple VNFs.
+
+      List of supported functions:
+
+        * ``start`` - starts a VNF based on VSPERF configuration
+        * ``stop`` - gracefully terminates given VNF
+
+        Examples:
+
+        .. code-block:: python
+
+            ['vnf1', 'start']
+            ['vnf2', 'start']
+            ['vnf2', 'stop']
+            ['vnf1', 'stop']
+
+    * ``trafficgen`` - triggers traffic generation
+
+      List of supported functions:
+
+        * ``send_traffic traffic`` - starts a traffic based on the vsperf configuration
+          and given ``traffic`` dictionary. More details about ``traffic`` dictionary
+          and its possible values are available at :ref:`Traffic Generator Integration Guide
+          <step-5-supported-traffic-types>`
+
+        Examples:
+
+        .. code-block:: python
+
+            ['trafficgen', 'send_traffic', {'traffic_type' : 'rfc2544_throughput'}]
+
+            ['trafficgen', 'send_traffic', {'traffic_type' : 'rfc2544_back2back', 'bidir' : 'True'}]
+
+    * ``settings`` - reads or modifies VSPERF configuration
+
+      List of supported functions:
+
+        * ``getValue param`` - returns value of given ``param``
+        * ``setValue param value`` - sets value of ``param`` to given ``value``
+
+        Examples:
+
+        .. code-block:: python
+
+            ['settings', 'getValue', 'TOOLS']
+
+            ['settings', 'setValue', 'GUEST_USERNAME', ['root']]
+
+    * ``namespace`` - creates or modifies network namespaces
+
+      List of supported functions:
+
+        * ``create_namespace name`` - creates new namespace with given ``name``
+        * ``delete_namespace name`` - deletes namespace specified by its ``name``
+        * ``assign_port_to_namespace port name [port_up]`` - assigns NIC specified by ``port``
+          into given namespace ``name``; If optional parameter ``port_up`` is set to ``True``,
+          then port will be brought up.
+        * ``add_ip_to_namespace_eth port name addr cidr`` - assigns an IP address ``addr``/``cidr``
+          to the NIC specified by ``port`` within namespace ``name``
+        * ``reset_port_to_root port name`` - returns given ``port`` from namespace ``name`` back
+          to the root namespace
+
+        Examples:
+
+        .. code-block:: python
+
+            ['namespace', 'create_namespace', 'testns']
+
+            ['namespace', 'assign_port_to_namespace', 'eth0', 'testns']
+
+    * ``veth`` - manipulates with eth and veth devices
+
+      List of supported functions:
+
+        * ``add_veth_port port peer_port`` - adds a pair of veth ports named ``port`` and
+          ``peer_port``
+        * ``del_veth_port port peer_port`` - deletes a veth port pair specified by ``port``
+          and ``peer_port``
+        * ``bring_up_eth_port eth_port [namespace]`` - brings up ``eth_port`` in (optional)
+          ``namespace``
+
+        Examples:
+
+        .. code-block:: python
+
+            ['veth', 'add_veth_port', 'veth', 'veth1']
+
+            ['veth', 'bring_up_eth_port', 'eth1']
+
+    * ``tools`` - provides a set of helper functions
+
+      List of supported functions:
+
+        * ``Assert condition`` - evaluates given ``condition`` and raises ``AssertionError``
+          in case that condition is not ``True``
+        * ``Eval expression`` - evaluates given expression as a python code and returns
+          its result
+        * ``Exec command [regex]`` - executes a shell command and filters its output by
+          (optional) regular expression
+
+        Examples:
+
+        .. code-block:: python
+
+            ['tools', 'exec', 'numactl -H', 'available: ([0-9]+)']
+            ['tools', 'assert', '#STEP[-1][0]>1']
+
+    * ``wait`` - is used for test case interruption. This object doesn't have
+      any functions. Once reached, vsperf will pause test execution and waits
+      for press of ``Enter key``. It can be used during testcase design
+      for debugging purposes.
+
+      Examples:
+
+      .. code-block:: python
+
+        ['wait']
+
+Test Macros
+-----------
+
+Test profiles can include macros as part of the test step. Each step in the
+profile may return a value such as a port name. Recall macros use #STEP to
+indicate the recalled value inside the return structure. If the method the
+test step calls returns a value it can be later recalled, for example:
+
+.. code-block:: python
+
+    {
+        "Name": "vswitch_add_del_vport",
+        "Deployment": "clean",
+        "Description": "vSwitch - add and delete virtual port",
+        "TestSteps": [
+                ['vswitch', 'add_switch', 'int_br0'],               # STEP 0
+                ['vswitch', 'add_vport', 'int_br0'],                # STEP 1
+                ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'],  # STEP 2
+                ['vswitch', 'del_switch', 'int_br0'],               # STEP 3
+             ]
+    }
+
+This test profile uses the vswitch add_vport method which returns a string
+value of the port added. This is later called by the del_port method using the
+name from step 1.
+
+It is also possible to use negative indexes in step macros. In that case
+``#STEP[-1]`` will refer to the result from previous step, ``#STEP[-2]``
+will refer to result of step called before previous step, etc. It means,
+that you could change ``STEP 2`` from previous example to achieve the same
+functionality:
+
+.. code-block:: python
+
+                ['vswitch', 'del_port', 'int_br0', '#STEP[-1][0]'],  # STEP 2
+
+Also commonly used steps can be created as a separate profile.
+
+.. code-block:: python
+
+    STEP_VSWITCH_PVP_INIT = [
+        ['vswitch', 'add_switch', 'int_br0'],           # STEP 0
+        ['vswitch', 'add_phy_port', 'int_br0'],         # STEP 1
+        ['vswitch', 'add_phy_port', 'int_br0'],         # STEP 2
+        ['vswitch', 'add_vport', 'int_br0'],            # STEP 3
+        ['vswitch', 'add_vport', 'int_br0'],            # STEP 4
+    ]
+
+This profile can then be used inside other testcases
+
+.. code-block:: python
+
+    {
+        "Name": "vswitch_pvp",
+        "Deployment": "clean",
+        "Description": "vSwitch - configure switch and one vnf",
+        "TestSteps": STEP_VSWITCH_PVP_INIT +
+                     [
+                        ['vnf', 'start'],
+                        ['vnf', 'stop'],
+                     ] +
+                     STEP_VSWITCH_PVP_FINIT
+    }
+
+HelloWorld and other basic Testcases
+------------------------------------
+
+The following examples are for demonstration purposes.
+You can run them by copying and pasting into the
+conf/integration/01_testcases.conf file.
+A command-line instruction is shown at the end of each
+example.
+
+HelloWorld
+^^^^^^^^^^
+
+The first example is a HelloWorld testcase.
+It simply creates a bridge with 2 physical ports, then sets up a flow to drop
+incoming packets from the port that was instantiated at the STEP #1.
+There's no interaction with the traffic generator.
+Then the flow, the 2 ports and the bridge are deleted.
+'add_phy_port' method creates a 'dpdk' type interface that will manage the
+physical port. The string value returned is the port name that will be referred
+by 'del_port' later on.
+
+.. code-block:: python
+
+    {
+        "Name": "HelloWorld",
+        "Description": "My first testcase",
+        "Deployment": "clean",
+        "TestSteps": [
+            ['vswitch', 'add_switch', 'int_br0'],   # STEP 0
+            ['vswitch', 'add_phy_port', 'int_br0'], # STEP 1
+            ['vswitch', 'add_phy_port', 'int_br0'], # STEP 2
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \
+                'actions': ['drop'], 'idle_timeout': '0'}],
+            ['vswitch', 'del_flow', 'int_br0'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[2][0]'],
+            ['vswitch', 'del_switch', 'int_br0'],
+        ]
+
+    },
+
+To run HelloWorld test:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --integration HelloWorld
+
+Specify a Flow by the IP address
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The next example shows how to explicitly set up a flow by specifying a
+destination IP address.
+All packets received from the port created at STEP #1 that have a destination
+IP address = 90.90.90.90 will be forwarded to the port created at the STEP #2.
+
+.. code-block:: python
+
+    {
+        "Name": "p2p_rule_l3da",
+        "Description": "Phy2Phy with rule on L3 Dest Addr",
+        "Deployment": "clean",
+        "biDirectional": "False",
+        "TestSteps": [
+            ['vswitch', 'add_switch', 'int_br0'],   # STEP 0
+            ['vswitch', 'add_phy_port', 'int_br0'], # STEP 1
+            ['vswitch', 'add_phy_port', 'int_br0'], # STEP 2
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \
+                'dl_type': '0x0800', 'nw_dst': '90.90.90.90', \
+                'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}],
+            ['trafficgen', 'send_traffic', \
+                {'traffic_type' : 'rfc2544_continuous'}],
+            ['vswitch', 'dump_flows', 'int_br0'],   # STEP 5
+            ['vswitch', 'del_flow', 'int_br0'],     # STEP 7 == del-flows
+            ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[2][0]'],
+            ['vswitch', 'del_switch', 'int_br0'],
+        ]
+    },
+
+To run the test:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --integration p2p_rule_l3da
+
+Multistream feature
+^^^^^^^^^^^^^^^^^^^
+
+The next testcase uses the multistream feature.
+The traffic generator will send packets with different UDP ports.
+That is accomplished by using "Stream Type" and "MultiStream" keywords.
+4 different flows are set to forward all incoming packets.
+
+.. code-block:: python
+
+    {
+        "Name": "multistream_l4",
+        "Description": "Multistream on UDP ports",
+        "Deployment": "clean",
+        "Parameters": {
+            'TRAFFIC' : {
+                "multistream": 4,
+                "stream_type": "L4",
+            },
+        },
+        "TestSteps": [
+            ['vswitch', 'add_switch', 'int_br0'],   # STEP 0
+            ['vswitch', 'add_phy_port', 'int_br0'], # STEP 1
+            ['vswitch', 'add_phy_port', 'int_br0'], # STEP 2
+            # Setup Flows
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \
+                'dl_type': '0x0800', 'nw_proto': '17', 'udp_dst': '0', \
+                'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}],
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \
+                'dl_type': '0x0800', 'nw_proto': '17', 'udp_dst': '1', \
+                'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}],
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \
+                'dl_type': '0x0800', 'nw_proto': '17', 'udp_dst': '2', \
+                'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}],
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \
+                'dl_type': '0x0800', 'nw_proto': '17', 'udp_dst': '3', \
+                'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}],
+            # Send mono-dir traffic
+            ['trafficgen', 'send_traffic', \
+                {'traffic_type' : 'rfc2544_continuous', \
+                'bidir' : 'False'}],
+            # Clean up
+            ['vswitch', 'del_flow', 'int_br0'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[2][0]'],
+            ['vswitch', 'del_switch', 'int_br0'],
+         ]
+    },
+
+To run the test:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --integration multistream_l4
+
+PVP with a VM Replacement
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This example launches a 1st VM in a PVP topology, then the VM is replaced
+by another VM.
+When VNF setup parameter in ./conf/04_vnf.conf is "QemuDpdkVhostUser"
+'add_vport' method creates a 'dpdkvhostuser' type port to connect a VM.
+
+.. code-block:: python
+
+    {
+        "Name": "ex_replace_vm",
+        "Description": "PVP with VM replacement",
+        "Deployment": "clean",
+        "TestSteps": [
+            ['vswitch', 'add_switch', 'int_br0'],       # STEP 0
+            ['vswitch', 'add_phy_port', 'int_br0'],     # STEP 1
+            ['vswitch', 'add_phy_port', 'int_br0'],     # STEP 2
+            ['vswitch', 'add_vport', 'int_br0'],        # STEP 3    vm1
+            ['vswitch', 'add_vport', 'int_br0'],        # STEP 4
+
+            # Setup Flows
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \
+                'actions': ['output:#STEP[3][1]'], 'idle_timeout': '0'}],
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[4][1]', \
+                'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}],
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[2][1]', \
+                'actions': ['output:#STEP[4][1]'], 'idle_timeout': '0'}],
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[3][1]', \
+                'actions': ['output:#STEP[1][1]'], 'idle_timeout': '0'}],
+
+            # Start VM 1
+            ['vnf1', 'start'],
+            # Now we want to replace VM 1 with another VM
+            ['vnf1', 'stop'],
+
+            ['vswitch', 'add_vport', 'int_br0'],        # STEP 11    vm2
+            ['vswitch', 'add_vport', 'int_br0'],        # STEP 12
+            ['vswitch', 'del_flow', 'int_br0'],
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \
+                'actions': ['output:#STEP[11][1]'], 'idle_timeout': '0'}],
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[12][1]', \
+                'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}],
+
+            # Start VM 2
+            ['vnf2', 'start'],
+            ['vnf2', 'stop'],
+            ['vswitch', 'dump_flows', 'int_br0'],
+
+            # Clean up
+            ['vswitch', 'del_flow', 'int_br0'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[2][0]'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[3][0]'],    # vm1
+            ['vswitch', 'del_port', 'int_br0', '#STEP[4][0]'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[11][0]'],   # vm2
+            ['vswitch', 'del_port', 'int_br0', '#STEP[12][0]'],
+            ['vswitch', 'del_switch', 'int_br0'],
+        ]
+    },
+
+To run the test:
+
+  .. code-block:: console
+
+     ./vsperf --conf-file user_settings.py --integration ex_replace_vm
+
+VM with a Linux bridge
+^^^^^^^^^^^^^^^^^^^^^^
+
+This example setups a PVP topology and routes traffic to the VM based on
+the destination IP address. A command-line parameter is used to select a Linux
+bridge as a guest loopback application. It is also possible to select a guest
+loopback application by a configuration option ``GUEST_LOOPBACK``.
+
+.. code-block:: python
+
+    {
+        "Name": "ex_pvp_rule_l3da",
+        "Description": "PVP with flow on L3 Dest Addr",
+        "Deployment": "clean",
+        "TestSteps": [
+            ['vswitch', 'add_switch', 'int_br0'],       # STEP 0
+            ['vswitch', 'add_phy_port', 'int_br0'],     # STEP 1
+            ['vswitch', 'add_phy_port', 'int_br0'],     # STEP 2
+            ['vswitch', 'add_vport', 'int_br0'],        # STEP 3    vm1
+            ['vswitch', 'add_vport', 'int_br0'],        # STEP 4
+            # Setup Flows
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \
+                'dl_type': '0x0800', 'nw_dst': '90.90.90.90', \
+                'actions': ['output:#STEP[3][1]'], 'idle_timeout': '0'}],
+            # Each pkt from the VM is forwarded to the 2nd dpdk port
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[4][1]', \
+                'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}],
+            # Start VMs
+            ['vnf1', 'start'],
+            ['trafficgen', 'send_traffic', \
+                {'traffic_type' : 'rfc2544_continuous', \
+                'bidir' : 'False'}],
+            ['vnf1', 'stop'],
+            # Clean up
+            ['vswitch', 'dump_flows', 'int_br0'],       # STEP 10
+            ['vswitch', 'del_flow', 'int_br0'],         # STEP 11
+            ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[2][0]'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[3][0]'],  # vm1 ports
+            ['vswitch', 'del_port', 'int_br0', '#STEP[4][0]'],
+            ['vswitch', 'del_switch', 'int_br0'],
+        ]
+    },
+
+To run the test:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --test-params \
+            "GUEST_LOOPBACK=['linux_bridge']" --integration ex_pvp_rule_l3da
+
+Forward packets based on UDP port
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This examples launches 2 VMs connected in parallel.
+Incoming packets will be forwarded to one specific VM depending on the
+destination UDP port.
+
+.. code-block:: python
+
+    {
+        "Name": "ex_2pvp_rule_l4dp",
+        "Description": "2 PVP with flows on L4 Dest Port",
+        "Deployment": "clean",
+        "Parameters": {
+            'TRAFFIC' : {
+                "multistream": 2,
+                "stream_type": "L4",
+            },
+        },
+        "TestSteps": [
+            ['vswitch', 'add_switch', 'int_br0'],       # STEP 0
+            ['vswitch', 'add_phy_port', 'int_br0'],     # STEP 1
+            ['vswitch', 'add_phy_port', 'int_br0'],     # STEP 2
+            ['vswitch', 'add_vport', 'int_br0'],        # STEP 3    vm1
+            ['vswitch', 'add_vport', 'int_br0'],        # STEP 4
+            ['vswitch', 'add_vport', 'int_br0'],        # STEP 5    vm2
+            ['vswitch', 'add_vport', 'int_br0'],        # STEP 6
+            # Setup Flows to reply ICMPv6 and similar packets, so to
+            # avoid flooding internal port with their re-transmissions
+            ['vswitch', 'add_flow', 'int_br0', \
+                {'priority': '1', 'dl_src': '00:00:00:00:00:01', \
+                'actions': ['output:#STEP[3][1]'], 'idle_timeout': '0'}],
+            ['vswitch', 'add_flow', 'int_br0', \
+                {'priority': '1', 'dl_src': '00:00:00:00:00:02', \
+                'actions': ['output:#STEP[4][1]'], 'idle_timeout': '0'}],
+            ['vswitch', 'add_flow', 'int_br0', \
+                {'priority': '1', 'dl_src': '00:00:00:00:00:03', \
+                'actions': ['output:#STEP[5][1]'], 'idle_timeout': '0'}],
+            ['vswitch', 'add_flow', 'int_br0', \
+                {'priority': '1', 'dl_src': '00:00:00:00:00:04', \
+                'actions': ['output:#STEP[6][1]'], 'idle_timeout': '0'}],
+            # Forward UDP packets depending on dest port
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \
+                'dl_type': '0x0800', 'nw_proto': '17', 'udp_dst': '0', \
+                'actions': ['output:#STEP[3][1]'], 'idle_timeout': '0'}],
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \
+                'dl_type': '0x0800', 'nw_proto': '17', 'udp_dst': '1', \
+                'actions': ['output:#STEP[5][1]'], 'idle_timeout': '0'}],
+            # Send VM output to phy port #2
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[4][1]', \
+                'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}],
+            ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[6][1]', \
+                'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}],
+            # Start VMs
+            ['vnf1', 'start'],                          # STEP 16
+            ['vnf2', 'start'],                          # STEP 17
+            ['trafficgen', 'send_traffic', \
+                {'traffic_type' : 'rfc2544_continuous', \
+                'bidir' : 'False'}],
+            ['vnf1', 'stop'],
+            ['vnf2', 'stop'],
+            ['vswitch', 'dump_flows', 'int_br0'],
+            # Clean up
+            ['vswitch', 'del_flow', 'int_br0'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[2][0]'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[3][0]'],  # vm1 ports
+            ['vswitch', 'del_port', 'int_br0', '#STEP[4][0]'],
+            ['vswitch', 'del_port', 'int_br0', '#STEP[5][0]'],  # vm2 ports
+            ['vswitch', 'del_port', 'int_br0', '#STEP[6][0]'],
+            ['vswitch', 'del_switch', 'int_br0'],
+        ]
+    },
+
+To run the test:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py --integration ex_2pvp_rule_l4dp
+
+Modification of existing PVVP deployment
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This is an example of modification of a standard deployment scenario with additional TestSteps.
+Standard PVVP scenario is used to configure a vSwitch and to deploy two VNFs connected
+in series. Additional TestSteps will deploy a 3rd VNF and connect it in parallel to
+already configured VNFs. Traffic generator is instructed (by Multistream feature) to send
+two separate traffic streams. One stream will be sent to the standalone VNF and second
+to two chained VNFs.
+
+In case, that test is defined as a performance test, then traffic results will be collected
+and available in both csv and rst report files.
+
+.. code-block:: python
+
+    {
+        "Name": "pvvp_pvp_cont",
+        "Deployment": "pvvp",
+        "Description": "PVVP and PVP in parallel with Continuous Stream",
+        "Parameters" : {
+            "TRAFFIC" : {
+                "traffic_type" : "rfc2544_continuous",
+                "multistream": 2,
+            },
+        },
+        "TestSteps": [
+                        ['vswitch', 'add_vport', 'br0'],
+                        ['vswitch', 'add_vport', 'br0'],
+                        # priority must be higher than default 32768, otherwise flows won't match
+                        ['vswitch', 'add_flow', 'br0',
+                         {'in_port': '1', 'actions': ['output:#STEP[-2][1]'], 'idle_timeout': '0', 'dl_type':'0x0800',
+                                                      'nw_proto':'17', 'tp_dst':'0', 'priority': '33000'}],
+                        ['vswitch', 'add_flow', 'br0',
+                         {'in_port': '2', 'actions': ['output:#STEP[-2][1]'], 'idle_timeout': '0', 'dl_type':'0x0800',
+                                                      'nw_proto':'17', 'tp_dst':'0', 'priority': '33000'}],
+                        ['vswitch', 'add_flow', 'br0', {'in_port': '#STEP[-4][1]', 'actions': ['output:1'],
+                                                        'idle_timeout': '0'}],
+                        ['vswitch', 'add_flow', 'br0', {'in_port': '#STEP[-4][1]', 'actions': ['output:2'],
+                                                        'idle_timeout': '0'}],
+                        ['vswitch', 'dump_flows', 'br0'],
+                        ['vnf1', 'start'],
+                     ]
+    },
+
+To run the test:
+
+  .. code-block:: console
+
+    ./vsperf --conf-file user_settings.py pvvp_pvp_cont
+
diff --git a/docs/testing/user/userguide/testusage.rst b/docs/testing/user/userguide/testusage.rst
new file mode 100644
index 00000000..c6037aaf
--- /dev/null
+++ b/docs/testing/user/userguide/testusage.rst
@@ -0,0 +1,848 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation, AT&T and others.
+
+vSwitchPerf test suites userguide
+---------------------------------
+
+General
+^^^^^^^
+
+VSPERF requires a traffic generators to run tests, automated traffic gen
+support in VSPERF includes:
+
+- IXIA traffic generator (IxNetwork hardware) and a machine that runs the IXIA
+  client software.
+- Spirent traffic generator (TestCenter hardware chassis or TestCenter virtual
+  in a VM) and a VM to run the Spirent Virtual Deployment Service image,
+  formerly known as "Spirent LabServer".
+- Xena Network traffic generator (Xena hardware chassis) that houses the Xena
+  Traffic generator modules.
+- Moongen software traffic generator. Requires a separate machine running
+  moongen to execute packet generation.
+
+If you want to use another traffic generator, please select the :ref:`trafficgen-dummy`
+generator.
+
+VSPERF Installation
+^^^^^^^^^^^^^^^^^^^
+
+To see the supported Operating Systems, vSwitches and system requirements,
+please follow the `installation instructions <vsperf-installation>`.
+
+Traffic Generator Setup
+^^^^^^^^^^^^^^^^^^^^^^^
+
+Follow the `Traffic generator instructions <trafficgen-installation>` to
+install and configure a suitable traffic generator.
+
+Cloning and building src dependencies
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+In order to run VSPERF, you will need to download DPDK and OVS. You can
+do this manually and build them in a preferred location, OR you could
+use vswitchperf/src. The vswitchperf/src directory contains makefiles
+that will allow you to clone and build the libraries that VSPERF depends
+on, such as DPDK and OVS. To clone and build simply:
+
+.. code-block:: console
+
+    $ cd src
+    $ make
+
+VSPERF can be used with stock OVS (without DPDK support). When build
+is finished, the libraries are stored in src_vanilla directory.
+
+The 'make' builds all options in src:
+
+* Vanilla OVS
+* OVS with vhost_user as the guest access method (with DPDK support)
+
+The vhost_user build will reside in src/ovs/
+The Vanilla OVS build will reside in vswitchperf/src_vanilla
+
+To delete a src subdirectory and its contents to allow you to re-clone simply
+use:
+
+.. code-block:: console
+
+     $ make clobber
+
+Configure the ``./conf/10_custom.conf`` file
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The ``10_custom.conf`` file is the configuration file that overrides
+default configurations in all the other configuration files in ``./conf``
+The supplied ``10_custom.conf`` file **MUST** be modified, as it contains
+configuration items for which there are no reasonable default values.
+
+The configuration items that can be added is not limited to the initial
+contents. Any configuration item mentioned in any .conf file in
+``./conf`` directory can be added and that item will be overridden by
+the custom configuration value.
+
+Further details about configuration files evaluation and special behaviour
+of options with ``GUEST_`` prefix could be found at :ref:`design document
+<design-configuration>`.
+
+Using a custom settings file
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If your ``10_custom.conf`` doesn't reside in the ``./conf`` directory
+of if you want to use an alternative configuration file, the file can
+be passed to ``vsperf`` via the ``--conf-file`` argument.
+
+.. code-block:: console
+
+    $ ./vsperf --conf-file <path_to_custom_conf> ...
+
+Note that configuration passed in via the environment (``--load-env``)
+or via another command line argument will override both the default and
+your custom configuration files. This "priority hierarchy" can be
+described like so (1 = max priority):
+
+1. Testcase definition section ``Parameters``
+2. Command line arguments
+3. Environment variables
+4. Configuration file(s)
+
+Further details about configuration files evaluation and special behaviour
+of options with ``GUEST_`` prefix could be found at :ref:`design document
+<design-configuration>`.
+
+.. _overriding-parameters-documentation:
+
+Overriding values defined in configuration files
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The configuration items can be overridden by command line argument
+``--test-params``. In this case, the configuration items and
+their values should be passed in form of ``item=value`` and separated
+by semicolon.
+
+Example:
+
+.. code:: console
+
+    $ ./vsperf --test-params "TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,);" \
+                             "GUEST_LOOPBACK=['testpmd','l2fwd']" pvvp_tput
+
+The second option is to override configuration items by ``Parameters`` section
+of the test case definition. The configuration items can be added into ``Parameters``
+dictionary with their new values. These values will override values defined in
+configuration files or specified by ``--test-params`` command line argument.
+
+Example:
+
+.. code:: python
+
+    "Parameters" : {'TRAFFICGEN_PKT_SIZES' : (128,),
+                    'TRAFFICGEN_DURATION' : 10,
+                    'GUEST_LOOPBACK' : ['testpmd','l2fwd'],
+                   }
+
+**NOTE:** In both cases, configuration item names and their values must be specified
+in the same form as they are defined inside configuration files. Parameter names
+must be specified in uppercase and data types of original and new value must match.
+Python syntax rules related to data types and structures must be followed.
+For example, parameter ``TRAFFICGEN_PKT_SIZES`` above is defined as a tuple
+with a single value ``128``. In this case trailing comma is mandatory, otherwise
+value can be wrongly interpreted as a number instead of a tuple and vsperf
+execution would fail. Please check configuration files for default values and their
+types and use them as a basis for any customized values. In case of any doubt, please
+check official python documentation related to data structures like tuples, lists
+and dictionaries.
+
+**NOTE:** Vsperf execution will terminate with runtime error in case, that unknown
+parameter name is passed via ``--test-params`` CLI argument or defined in ``Parameters``
+section of test case definition. It is also forbidden to redefine a value of
+``TEST_PARAMS`` configuration item via CLI or ``Parameters`` section.
+
+vloop_vnf
+^^^^^^^^^
+
+VSPERF uses a VM image called vloop_vnf for looping traffic in the deployment
+scenarios involving VMs. The image can be downloaded from
+`<http://artifacts.opnfv.org/>`__.
+
+Please see the installation instructions for information on :ref:`vloop-vnf`
+images.
+
+.. _l2fwd-module:
+
+l2fwd Kernel Module
+^^^^^^^^^^^^^^^^^^^
+
+A Kernel Module that provides OSI Layer 2 Ipv4 termination or forwarding with
+support for Destination Network Address Translation (DNAT) for both the MAC and
+IP addresses. l2fwd can be found in <vswitchperf_dir>/src/l2fwd
+
+Executing tests
+^^^^^^^^^^^^^^^
+
+All examples inside these docs assume, that user is inside the VSPERF
+directory. VSPERF can be executed from any directory.
+
+Before running any tests make sure you have root permissions by adding
+the following line to /etc/sudoers:
+
+.. code-block:: console
+
+    username ALL=(ALL)       NOPASSWD: ALL
+
+username in the example above should be replaced with a real username.
+
+To list the available tests:
+
+.. code-block:: console
+
+    $ ./vsperf --list
+
+To run a single test:
+
+.. code-block:: console
+
+    $ ./vsperf $TESTNAME
+
+Where $TESTNAME is the name of the vsperf test you would like to run.
+
+To run a group of tests, for example all tests with a name containing
+'RFC2544':
+
+.. code-block:: console
+
+    $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf --tests="RFC2544"
+
+To run all tests:
+
+.. code-block:: console
+
+    $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf
+
+Some tests allow for configurable parameters, including test duration
+(in seconds) as well as packet sizes (in bytes).
+
+.. code:: bash
+
+    $ ./vsperf --conf-file user_settings.py \
+        --tests RFC2544Tput \
+        --test-params "TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,)"
+
+For all available options, check out the help dialog:
+
+.. code-block:: console
+
+    $ ./vsperf --help
+
+Executing Vanilla OVS tests
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+1. If needed, recompile src for all OVS variants
+
+   .. code-block:: console
+
+       $ cd src
+       $ make distclean
+       $ make
+
+2. Update your ``10_custom.conf`` file to use Vanilla OVS:
+
+   .. code-block:: python
+
+       VSWITCH = 'OvsVanilla'
+
+3. Run test:
+
+   .. code-block:: console
+
+       $ ./vsperf --conf-file=<path_to_custom_conf>
+
+   Please note if you don't want to configure Vanilla OVS through the
+   configuration file, you can pass it as a CLI argument.
+
+   .. code-block:: console
+
+       $ ./vsperf --vswitch OvsVanilla
+
+
+Executing tests with VMs
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+To run tests using vhost-user as guest access method:
+
+1. Set VHOST_METHOD and VNF of your settings file to:
+
+   .. code-block:: python
+
+       VSWITCH = 'OvsDpdkVhost'
+       VNF = 'QemuDpdkVhost'
+
+2. If needed, recompile src for all OVS variants
+
+   .. code-block:: console
+
+       $ cd src
+       $ make distclean
+       $ make
+
+3. Run test:
+
+   .. code-block:: console
+
+       $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf
+
+Executing tests with VMs using Vanilla OVS
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To run tests using Vanilla OVS:
+
+1. Set the following variables:
+
+   .. code-block:: python
+
+       VSWITCH = 'OvsVanilla'
+       VNF = 'QemuVirtioNet'
+
+       VANILLA_TGEN_PORT1_IP = n.n.n.n
+       VANILLA_TGEN_PORT1_MAC = nn:nn:nn:nn:nn:nn
+
+       VANILLA_TGEN_PORT2_IP = n.n.n.n
+       VANILLA_TGEN_PORT2_MAC = nn:nn:nn:nn:nn:nn
+
+       VANILLA_BRIDGE_IP = n.n.n.n
+
+   or use ``--test-params`` option
+
+   .. code-block:: console
+
+       $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf \
+                  --test-params "VANILLA_TGEN_PORT1_IP=n.n.n.n;" \
+                                "VANILLA_TGEN_PORT1_MAC=nn:nn:nn:nn:nn:nn;" \
+                                "VANILLA_TGEN_PORT2_IP=n.n.n.n;" \
+                                "VANILLA_TGEN_PORT2_MAC=nn:nn:nn:nn:nn:nn"
+
+2. If needed, recompile src for all OVS variants
+
+   .. code-block:: console
+
+       $ cd src
+       $ make distclean
+       $ make
+
+3. Run test:
+
+   .. code-block:: console
+
+       $ ./vsperf --conf-file<path_to_custom_conf>/10_custom.conf
+
+.. _vpp-test:
+
+Executing VPP tests
+^^^^^^^^^^^^^^^^^^^
+
+Currently it is not possible to use standard scenario deployments for execution of
+tests with VPP. It means, that deployments ``p2p``, ``pvp``, ``pvvp`` and in general any
+:ref:`pxp-deployment` won't work with VPP. However it is possible to use VPP in
+:ref:`step-driven-tests`. A basic set of VPP testcases covering ``phy2phy``, ``pvp``
+and ``pvvp`` tests are already prepared.
+
+List of performance tests with VPP support follows:
+
+* phy2phy_tput_vpp:              VPP: LTD.Throughput.RFC2544.PacketLossRatio
+* phy2phy_cont_vpp:              VPP: Phy2Phy Continuous Stream
+* phy2phy_back2back_vpp:         VPP: LTD.Throughput.RFC2544.BackToBackFrames
+* pvp_tput_vpp:                  VPP: LTD.Throughput.RFC2544.PacketLossRatio
+* pvp_cont_vpp:                  VPP: PVP Continuous Stream
+* pvp_back2back_vpp:             VPP: LTD.Throughput.RFC2544.BackToBackFrames
+* pvvp_tput_vpp:                 VPP: LTD.Throughput.RFC2544.PacketLossRatio
+* pvvp_cont_vpp:                 VPP: PVP Continuous Stream
+* pvvp_back2back_vpp:            VPP: LTD.Throughput.RFC2544.BackToBackFrames
+
+In order to execute testcases with VPP it is required to:
+
+* install VPP manually, see :ref:`vpp-installation`
+* configure ``WHITELIST_NICS``, with two physical NICs connected to the traffic generator
+* configure traffic generator, see :ref:`trafficgen-installation`
+
+After that it is possible to execute VPP testcases listed above.
+
+For example:
+
+.. code-block:: console
+
+    $ ./vsperf --conf-file=<path_to_custom_conf> phy2phy_tput_vpp
+
+.. _vfio-pci:
+
+Using vfio_pci with DPDK
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To use vfio with DPDK instead of igb_uio add into your custom configuration
+file the following parameter:
+
+.. code-block:: python
+
+    PATHS['dpdk']['src']['modules'] = ['uio', 'vfio-pci']
+
+
+**NOTE:** In case, that DPDK is installed from binary package, then please
+set ``PATHS['dpdk']['bin']['modules']`` instead.
+
+**NOTE:** Please ensure that Intel VT-d is enabled in BIOS.
+
+**NOTE:** Please ensure your boot/grub parameters include
+the following:
+
+.. code-block:: console
+
+    iommu=pt intel_iommu=on
+
+To check that IOMMU is enabled on your platform:
+
+.. code-block:: console
+
+    $ dmesg | grep IOMMU
+    [    0.000000] Intel-IOMMU: enabled
+    [    0.139882] dmar: IOMMU 0: reg_base_addr fbffe000 ver 1:0 cap d2078c106f0466 ecap f020de
+    [    0.139888] dmar: IOMMU 1: reg_base_addr ebffc000 ver 1:0 cap d2078c106f0466 ecap f020de
+    [    0.139893] IOAPIC id 2 under DRHD base  0xfbffe000 IOMMU 0
+    [    0.139894] IOAPIC id 0 under DRHD base  0xebffc000 IOMMU 1
+    [    0.139895] IOAPIC id 1 under DRHD base  0xebffc000 IOMMU 1
+    [    3.335744] IOMMU: dmar0 using Queued invalidation
+    [    3.335746] IOMMU: dmar1 using Queued invalidation
+    ....
+
+.. _SRIOV-support:
+
+Using SRIOV support
+^^^^^^^^^^^^^^^^^^^
+
+To use virtual functions of NIC with SRIOV support, use extended form
+of NIC PCI slot definition:
+
+.. code-block:: python
+
+    WHITELIST_NICS = ['0000:05:00.0|vf0', '0000:05:00.1|vf3']
+
+Where 'vf' is an indication of virtual function usage and following
+number defines a VF to be used. In case that VF usage is detected,
+then vswitchperf will enable SRIOV support for given card and it will
+detect PCI slot numbers of selected VFs.
+
+So in example above, one VF will be configured for NIC '0000:05:00.0'
+and four VFs will be configured for NIC '0000:05:00.1'. Vswitchperf
+will detect PCI addresses of selected VFs and it will use them during
+test execution.
+
+At the end of vswitchperf execution, SRIOV support will be disabled.
+
+SRIOV support is generic and it can be used in different testing scenarios.
+For example:
+
+* vSwitch tests with DPDK or without DPDK support to verify impact
+  of VF usage on vSwitch performance
+* tests without vSwitch, where traffic is forwared directly
+  between VF interfaces by packet forwarder (e.g. testpmd application)
+* tests without vSwitch, where VM accesses VF interfaces directly
+  by PCI-passthrough_ to measure raw VM throughput performance.
+
+.. _PCI-passthrough:
+
+Using QEMU with PCI passthrough support
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Raw virtual machine throughput performance can be measured by execution of PVP
+test with direct access to NICs by PCI passthrough. To execute VM with direct
+access to PCI devices, enable vfio-pci_. In order to use virtual functions,
+SRIOV-support_ must be enabled.
+
+Execution of test with PCI passthrough with vswitch disabled:
+
+.. code-block:: console
+
+    $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf \
+               --vswitch none --vnf QemuPciPassthrough pvp_tput
+
+Any of supported guest-loopback-application_ can be used inside VM with
+PCI passthrough support.
+
+Note: Qemu with PCI passthrough support can be used only with PVP test
+deployment.
+
+.. _guest-loopback-application:
+
+Selection of loopback application for tests with VMs
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To select the loopback applications which will forward packets inside VMs,
+the following parameter should be configured:
+
+.. code-block:: python
+
+     GUEST_LOOPBACK = ['testpmd']
+
+or use ``--test-params`` CLI argument:
+
+.. code-block:: console
+
+        $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf \
+              --test-params "GUEST_LOOPBACK=['testpmd']"
+
+Supported loopback applications are:
+
+.. code-block:: console
+
+     'testpmd'       - testpmd from dpdk will be built and used
+     'l2fwd'         - l2fwd module provided by Huawei will be built and used
+     'linux_bridge'  - linux bridge will be configured
+     'buildin'       - nothing will be configured by vsperf; VM image must
+                       ensure traffic forwarding between its interfaces
+
+Guest loopback application must be configured, otherwise traffic
+will not be forwarded by VM and testcases with VM related deployments
+will fail. Guest loopback application is set to 'testpmd' by default.
+
+**NOTE:** In case that only 1 or more than 2 NICs are configured for VM,
+then 'testpmd' should be used. As it is able to forward traffic between
+multiple VM NIC pairs.
+
+**NOTE:** In case of linux_bridge, all guest NICs are connected to the same
+bridge inside the guest.
+
+Mergable Buffers Options with QEMU
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Mergable buffers can be disabled with VSPerf within QEMU. This option can
+increase performance significantly when not using jumbo frame sized packets.
+By default VSPerf disables mergable buffers. If you wish to enable it you
+can modify the setting in the a custom conf file.
+
+.. code-block:: python
+
+    GUEST_NIC_MERGE_BUFFERS_DISABLE = [False]
+
+Then execute using the custom conf file.
+
+.. code-block:: console
+
+        $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf
+
+Alternatively you can just pass the param during execution.
+
+.. code-block:: console
+
+        $ ./vsperf --test-params "GUEST_NIC_MERGE_BUFFERS_DISABLE=[False]"
+
+
+Selection of dpdk binding driver for tests with VMs
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To select dpdk binding driver, which will specify which driver the vm NICs will
+use for dpdk bind, the following configuration parameter should be configured:
+
+.. code-block:: console
+
+     GUEST_DPDK_BIND_DRIVER = ['igb_uio_from_src']
+
+The supported dpdk guest bind drivers are:
+
+.. code-block:: console
+
+    'uio_pci_generic'	   - Use uio_pci_generic driver
+    'igb_uio_from_src'     - Build and use the igb_uio driver from the dpdk src
+                             files
+    'vfio_no_iommu'        - Use vfio with no iommu option. This requires custom
+                             guest images that support this option. The default
+                             vloop image does not support this driver.
+
+Note: uio_pci_generic does not support sr-iov testcases with guests attached.
+This is because uio_pci_generic only supports legacy interrupts. In case
+uio_pci_generic is selected with the vnf as QemuPciPassthrough it will be
+modified to use igb_uio_from_src instead.
+
+Note: vfio_no_iommu requires kernels equal to or greater than 4.5 and dpdk
+16.04 or greater. Using this option will also taint the kernel.
+
+Please refer to the dpdk documents at http://dpdk.org/doc/guides for more
+information on these drivers.
+
+Multi-Queue Configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+VSPerf currently supports multi-queue with the following limitations:
+
+1.  Requires QEMU 2.5 or greater and any OVS version higher than 2.5. The
+    default upstream package versions installed by VSPerf satisfies this
+    requirement.
+
+2.  Guest image must have ethtool utility installed if using l2fwd or linux
+    bridge inside guest for loopback.
+
+3.  If using OVS versions 2.5.0 or less enable old style multi-queue as shown
+    in the ''02_vswitch.conf'' file.
+
+    .. code-block:: python
+
+        OVS_OLD_STYLE_MQ = True
+
+To enable multi-queue for dpdk modify the ''02_vswitch.conf'' file.
+
+.. code-block:: python
+
+    VSWITCH_DPDK_MULTI_QUEUES = 2
+
+**NOTE:** you should consider using the switch affinity to set a pmd cpu mask
+that can optimize your performance. Consider the numa of the NIC in use if this
+applies by checking /sys/class/net/<eth_name>/device/numa_node and setting an
+appropriate mask to create PMD threads on the same numa node.
+
+When multi-queue is enabled, each dpdk or dpdkvhostuser port that is created
+on the switch will set the option for multiple queues. If old style multi queue
+has been enabled a global option for multi queue will be used instead of the
+port by port option.
+
+To enable multi-queue on the guest modify the ''04_vnf.conf'' file.
+
+.. code-block:: python
+
+    GUEST_NIC_QUEUES = [2]
+
+Enabling multi-queue at the guest will add multiple queues to each NIC port when
+qemu launches the guest.
+
+In case of Vanilla OVS, multi-queue is enabled on the tuntap ports and nic
+queues will be enabled inside the guest with ethtool. Simply enabling the
+multi-queue on the guest is sufficient for Vanilla OVS multi-queue.
+
+Testpmd should be configured to take advantage of multi-queue on the guest if
+using DPDKVhostUser. This can be done by modifying the ''04_vnf.conf'' file.
+
+.. code-block:: python
+
+    GUEST_TESTPMD_PARAMS = ['-l 0,1,2,3,4  -n 4 --socket-mem 512 -- '
+                            '--burst=64 -i --txqflags=0xf00 '
+                            '--nb-cores=4 --rxq=2 --txq=2 '
+                            '--disable-hw-vlan']
+
+**NOTE:** The guest SMP cores must be configured to allow for testpmd to use the
+optimal number of cores to take advantage of the multiple guest queues.
+
+In case of using Vanilla OVS and qemu virtio-net you can increase performance
+by binding vhost-net threads to cpus. This can be done by enabling the affinity
+in the ''04_vnf.conf'' file. This can be done to non multi-queue enabled
+configurations as well as there will be 2 vhost-net threads.
+
+.. code-block:: python
+
+    VSWITCH_VHOST_NET_AFFINITIZATION = True
+
+    VSWITCH_VHOST_CPU_MAP = [4,5,8,11]
+
+**NOTE:** This method of binding would require a custom script in a real
+environment.
+
+**NOTE:** For optimal performance guest SMPs and/or vhost-net threads should be
+on the same numa as the NIC in use if possible/applicable. Testpmd should be
+assigned at least (nb_cores +1) total cores with the cpu mask.
+
+Executing Packet Forwarding tests
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To select the applications which will forward packets,
+the following parameters should be configured:
+
+.. code-block:: python
+
+    VSWITCH = 'none'
+    PKTFWD = 'TestPMD'
+
+or use ``--vswitch`` and ``--fwdapp`` CLI arguments:
+
+.. code-block:: console
+
+    $ ./vsperf phy2phy_cont --conf-file user_settings.py \
+               --vswitch none \
+               --fwdapp TestPMD
+
+Supported Packet Forwarding applications are:
+
+.. code-block:: console
+
+    'testpmd'       - testpmd from dpdk
+
+
+1. Update your ''10_custom.conf'' file to use the appropriate variables
+   for selected Packet Forwarder:
+
+   .. code-block:: python
+
+      # testpmd configuration
+      TESTPMD_ARGS = []
+      # packet forwarding mode supported by testpmd; Please see DPDK documentation
+      # for comprehensive list of modes supported by your version.
+      # e.g. io|mac|mac_retry|macswap|flowgen|rxonly|txonly|csum|icmpecho|...
+      # Note: Option "mac_retry" has been changed to "mac retry" since DPDK v16.07
+      TESTPMD_FWD_MODE = 'csum'
+      # checksum calculation layer: ip|udp|tcp|sctp|outer-ip
+      TESTPMD_CSUM_LAYER = 'ip'
+      # checksum calculation place: hw (hardware) | sw (software)
+      TESTPMD_CSUM_CALC = 'sw'
+      # recognize tunnel headers: on|off
+      TESTPMD_CSUM_PARSE_TUNNEL = 'off'
+
+2. Run test:
+
+   .. code-block:: console
+
+      $ ./vsperf phy2phy_tput --conf-file <path_to_settings_py>
+
+Executing Packet Forwarding tests with one guest
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+TestPMD with DPDK 16.11 or greater can be used to forward packets as a switch to a single guest using TestPMD vdev
+option. To set this configuration the following parameters should be used.
+
+    .. code-block:: python
+
+        VSWITCH = 'none'
+        PKTFWD = 'TestPMD'
+
+or use ``--vswitch`` and ``--fwdapp`` CLI arguments:
+
+    .. code-block:: console
+
+        $ ./vsperf pvp_tput --conf-file user_settings.py \
+                   --vswitch none \
+                   --fwdapp TestPMD
+
+Guest forwarding application only supports TestPMD in this configuration.
+
+    .. code-block:: python
+
+        GUEST_LOOPBACK = ['testpmd']
+
+For optimal performance one cpu per port +1 should be used for TestPMD. Also set additional params for packet forwarding
+application to use the correct number of nb-cores.
+
+    .. code-block:: python
+
+        DPDK_SOCKET_MEM = ['1024', '0']
+        VSWITCHD_DPDK_ARGS = ['-l', '46,44,42,40,38', '-n', '4']
+        TESTPMD_ARGS = ['--nb-cores=4', '--txq=1', '--rxq=1']
+
+For guest TestPMD 3 VCpus should be assigned with the following TestPMD params.
+
+    .. code-block:: python
+
+        GUEST_TESTPMD_PARAMS = ['-l 0,1,2 -n 4 --socket-mem 1024 -- '
+                                '--burst=64 -i --txqflags=0xf00 '
+                                '--disable-hw-vlan --nb-cores=2 --txq=1 --rxq=1']
+
+Execution of TestPMD can be run with the following command line
+
+    .. code-block:: console
+
+        ./vsperf pvp_tput --vswitch=none --fwdapp=TestPMD --conf-file <path_to_settings_py>
+
+**NOTE:** To achieve the best 0% loss numbers with rfc2544 throughput testing, other tunings should be applied to host
+and guest such as tuned profiles and CPU tunings to prevent possible interrupts to worker threads.
+
+VSPERF modes of operation
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+VSPERF can be run in different modes. By default it will configure vSwitch,
+traffic generator and VNF. However it can be used just for configuration
+and execution of traffic generator. Another option is execution of all
+components except traffic generator itself.
+
+Mode of operation is driven by configuration parameter -m or --mode
+
+.. code-block:: console
+
+    -m MODE, --mode MODE  vsperf mode of operation;
+        Values:
+            "normal" - execute vSwitch, VNF and traffic generator
+            "trafficgen" - execute only traffic generator
+            "trafficgen-off" - execute vSwitch and VNF
+            "trafficgen-pause" - execute vSwitch and VNF but wait before traffic transmission
+
+In case, that VSPERF is executed in "trafficgen" mode, then configuration
+of traffic generator can be modified through ``TRAFFIC`` dictionary passed to the
+``--test-params`` option. It is not needed to specify all values of ``TRAFFIC``
+dictionary. It is sufficient to specify only values, which should be changed.
+Detailed description of ``TRAFFIC`` dictionary can be found at
+:ref:`configuration-of-traffic-dictionary`.
+
+Example of execution of VSPERF in "trafficgen" mode:
+
+.. code-block:: console
+
+    $ ./vsperf -m trafficgen --trafficgen IxNet --conf-file vsperf.conf \
+        --test-params "TRAFFIC={'traffic_type':'rfc2544_continuous','bidir':'False','framerate':60}"
+
+Code change verification by pylint
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Every developer participating in VSPERF project should run
+pylint before his python code is submitted for review. Project
+specific configuration for pylint is available at 'pylint.rc'.
+
+Example of manual pylint invocation:
+
+.. code-block:: console
+
+          $ pylint --rcfile ./pylintrc ./vsperf
+
+GOTCHAs:
+^^^^^^^^
+
+Custom image fails to boot
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Using custom VM images may not boot within VSPerf pxp testing because of
+the drive boot and shared type which could be caused by a missing scsi
+driver inside the image. In case of issues you can try changing the drive
+boot type to ide.
+
+.. code-block:: python
+
+    GUEST_BOOT_DRIVE_TYPE = ['ide']
+    GUEST_SHARED_DRIVE_TYPE = ['ide']
+
+OVS with DPDK and QEMU
+~~~~~~~~~~~~~~~~~~~~~~~
+
+If you encounter the following error: "before (last 100 chars):
+'-path=/dev/hugepages,share=on: unable to map backing store for
+hugepages: Cannot allocate memory\r\n\r\n" during qemu initialization,
+check the amount of hugepages on your system:
+
+.. code-block:: console
+
+    $ cat /proc/meminfo | grep HugePages
+
+
+By default the vswitchd is launched with 1Gb of memory, to  change
+this, modify --socket-mem parameter in conf/02_vswitch.conf to allocate
+an appropriate amount of memory:
+
+.. code-block:: python
+
+    DPDK_SOCKET_MEM = ['1024', '0']
+    VSWITCHD_DPDK_ARGS = ['-c', '0x4', '-n', '4']
+    VSWITCHD_DPDK_CONFIG = {
+        'dpdk-init' : 'true',
+        'dpdk-lcore-mask' : '0x4',
+        'dpdk-socket-mem' : '1024,0',
+    }
+
+Note: Option ``VSWITCHD_DPDK_ARGS`` is used for vswitchd, which supports ``--dpdk``
+parameter. In recent vswitchd versions, option ``VSWITCHD_DPDK_CONFIG`` will be
+used to configure vswitchd via ``ovs-vsctl`` calls.
+
+
+More information
+^^^^^^^^^^^^^^^^
+
+For more information and details refer to the rest of vSwitchPerfuser documentation.
+
diff --git a/docs/testing/user/userguide/yardstick.rst b/docs/testing/user/userguide/yardstick.rst
new file mode 100644
index 00000000..b5e5c72d
--- /dev/null
+++ b/docs/testing/user/userguide/yardstick.rst
@@ -0,0 +1,250 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation, AT&T and others.
+
+Execution of vswitchperf testcases by Yardstick
+-----------------------------------------------
+
+General
+^^^^^^^
+
+Yardstick is a generic framework for a test execution, which is used for
+validation of installation of OPNFV platform. In the future, Yardstick will
+support two options of vswitchperf testcase execution:
+
+- plugin mode, which will execute native vswitchperf testcases; Tests will
+  be executed natively by vsperf, and test results will be processed and
+  reported by yardstick.
+- traffic generator mode, which will run vswitchperf in **trafficgen**
+  mode only; Yardstick framework will be used to launch VNFs and to configure
+  flows to ensure, that traffic is properly routed. This mode will allow to
+  test OVS performance in real world scenarios.
+
+In Colorado release only the traffic generator mode is supported.
+
+Yardstick Installation
+^^^^^^^^^^^^^^^^^^^^^^
+
+In order to run Yardstick testcases, you will need to prepare your test
+environment. Please follow the `installation instructions
+<http://artifacts.opnfv.org/yardstick/docs/user_guides_framework/index.html>`__
+to install the yardstick.
+
+Please note, that yardstick uses OpenStack for execution of testcases.
+OpenStack must be installed with Heat and Neutron services. Otherwise
+vswitchperf testcases cannot be executed.
+
+VM image with vswitchperf
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A special VM image is required for execution of vswitchperf specific testcases
+by yardstick. It is possible to use a sample VM image available at OPNFV
+artifactory or to build customized image.
+
+Sample VM image with vswitchperf
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Sample VM image is available at vswitchperf section of OPNFV artifactory
+for free download:
+
+.. code-block:: console
+
+    $ wget http://artifacts.opnfv.org/vswitchperf/vnf/vsperf-yardstick-image.qcow2
+
+This image can be used for execution of sample testcases with dummy traffic
+generator.
+
+**NOTE:** Traffic generators might require an installation of client software.
+This software is not included in the sample image and must be installed by user.
+
+**NOTE:** This image will be updated only in case, that new features related
+to yardstick integration will be added to the vswitchperf.
+
+Preparation of custom VM image
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In general, any Linux distribution supported by vswitchperf can be used as
+a base image for vswitchperf. One of the possibilities is to modify vloop-vnf
+image, which can be downloaded from `<http://artifacts.opnfv.org/vswitchperf.html/>`__
+(see :ref:`vloop-vnf`).
+
+Please follow the :ref:`vsperf-installation` to
+install vswitchperf inside vloop-vnf image. As vswitchperf will be run in
+trafficgen mode, it is possible to skip installation and compilation of OVS,
+QEMU and DPDK to keep image size smaller.
+
+In case, that selected traffic generator requires installation of additional
+client software, please follow appropriate documentation. For example in case
+of IXIA, you would need to install IxOS and IxNetowrk TCL API.
+
+VM image usage
+~~~~~~~~~~~~~~
+
+Image with vswitchperf must be uploaded into the glance service and
+vswitchperf specific flavor configured, e.g.:
+
+.. code-block:: console
+
+    $ glance --os-username admin --os-image-api-version 1 image-create --name \
+      vsperf --is-public true --disk-format qcow2 --container-format bare --file \
+      vsperf-yardstick-image.qcow2
+
+    $ nova --os-username admin flavor-create vsperf-flavor 100 2048 25 1
+
+Testcase execution
+^^^^^^^^^^^^^^^^^^
+
+After installation, yardstick is available as python package within yardstick
+specific virtual environment. It means, that yardstick environment must be
+enabled before the test execution, e.g.:
+
+.. code-block:: console
+
+   source ~/yardstick_venv/bin/activate
+
+
+Next step is configuration of OpenStack environment, e.g. in case of devstack:
+
+.. code-block:: console
+
+   source /opt/openstack/devstack/openrc
+   export EXTERNAL_NETWORK=public
+
+Vswitchperf testcases executable by yardstick are located at vswitchperf
+repository inside ``yardstick/tests`` directory. Example of their download
+and execution follows:
+
+.. code-block:: console
+
+   git clone https://gerrit.opnfv.org/gerrit/vswitchperf
+   cd vswitchperf
+
+   yardstick -d task start yardstick/tests/rfc2544_throughput_dummy.yaml
+
+**NOTE:** Optional argument ``-d`` shows debug output.
+
+Testcase customization
+^^^^^^^^^^^^^^^^^^^^^^
+
+Yardstick testcases are described by YAML files. vswitchperf specific testcases
+are part of the vswitchperf repository and their yaml files can be found at
+``yardstick/tests`` directory. For detailed description of yaml file structure,
+please see yardstick documentation and testcase samples. Only vswitchperf specific
+parts will be discussed here.
+
+Example of yaml file:
+
+.. code-block:: yaml
+
+    ...
+    scenarios:
+    -
+      type: Vsperf
+      options:
+        testname: 'p2p_rfc2544_throughput'
+        trafficgen_port1: 'eth1'
+        trafficgen_port2: 'eth3'
+        external_bridge: 'br-ex'
+        test_params: 'TRAFFICGEN_DURATION=30;TRAFFIC={'traffic_type':'rfc2544_throughput}'
+        conf_file: '~/vsperf-yardstick.conf'
+
+      host: vsperf.demo
+
+      runner:
+        type: Sequence
+        scenario_option_name: frame_size
+        sequence:
+        - 64
+        - 128
+        - 512
+        - 1024
+        - 1518
+      sla:
+        metrics: 'throughput_rx_fps'
+        throughput_rx_fps: 500000
+        action: monitor
+
+    context:
+    ...
+
+Section option
+~~~~~~~~~~~~~~
+
+Section **option** defines details of vswitchperf test scenario. Lot of options
+are identical to the vswitchperf parameters passed through ``--test-params``
+argument. Following options are supported:
+
+- **frame_size** - a packet size for which test should be executed;
+  Multiple packet sizes can be tested by modification of Sequence runner
+  section inside YAML definition. Default: '64'
+- **conf_file** - sets path to the vswitchperf configuration file, which will be
+  uploaded to VM; Default: '~/vsperf-yardstick.conf'
+- **setup_script** - sets path to the setup script, which will be executed
+  during setup and teardown phases
+- **trafficgen_port1** - specifies device name of 1st interface connected to
+  the trafficgen
+- **trafficgen_port2** - specifies device name of 2nd interface connected to
+  the trafficgen
+- **external_bridge** - specifies name of external bridge configured in OVS;
+  Default: 'br-ex'
+- **test_params** - specifies a string with a list of vsperf configuration
+  parameters, which will be passed to the ``--test-params`` CLI argument;
+  Parameters should be stated in the form of ``param=value`` and separated
+  by a semicolon. Configuration of traffic generator is driven by ``TRAFFIC``
+  dictionary, which can be also updated by values defined by ``test_params``.
+  Please check VSPERF documentation for details about available configuration
+  parameters and their data types.
+  In case that both **test_params** and **conf_file** are specified,
+  then values from **test_params** will override values defined
+  in the configuration file.
+
+In case that **trafficgen_port1** and/or **trafficgen_port2** are defined, then
+these interfaces will be inserted into the **external_bridge** of OVS. It is
+expected, that OVS runs at the same node, where the testcase is executed. In case
+of more complex OpenStack installation or a need of additional OVS configuration,
+**setup_script** can be used.
+
+**NOTE** It is essential to specify a configuration for selected traffic generator.
+In case, that standalone testcase is created, then traffic generator can be
+selected and configured directly in YAML file by **test_params**. On the other
+hand, if multiple testcases should be executed with the same traffic generator
+settings, then a customized configuration file should be prepared and its name
+passed by **conf_file** option.
+
+Section runner
+~~~~~~~~~~~~~~
+
+Yardstick supports several `runner types
+<http://artifacts.opnfv.org/yardstick/docs/userguide/architecture.html#runner-types>`__.
+In case of vswitchperf specific TCs, **Sequence** runner type can be used to
+execute the testcase for given list of frame sizes.
+
+
+Section sla
+~~~~~~~~~~~
+
+In case that sla section is not defined, then testcase will be always
+considered as successful. On the other hand, it is possible to define a set of
+test metrics and their minimal values to evaluate test success. Any numeric
+value, reported by vswitchperf inside CSV result file, can be used.
+Multiple metrics can be defined as a coma separated list of items. Minimal
+value must be set separately for each metric.
+
+e.g.:
+
+.. code-block:: yaml
+
+      sla:
+          metrics: 'throughput_rx_fps,throughput_rx_mbps'
+          throughput_rx_fps: 500000
+          throughput_rx_mbps: 1000
+
+In case that any of defined metrics will be lower than defined value, then
+testcase will be marked as failed. Based on ``action`` policy, yardstick
+will either stop test execution (value ``assert``) or it will run next test
+(value ``monitor``).
+
+**NOTE** The throughput SLA (or any other SLA) cannot be set to a meaningful
+value without knowledge of the server and networking environment, possibly
+including prior testing in that environment to establish a baseline SLA level
+under well-understood circumstances.