diff options
Diffstat (limited to 'docs/testing/user')
| -rw-r--r-- | docs/testing/user/configguide/LICENSE | 2 | ||||
| -rw-r--r-- | docs/testing/user/configguide/TCLServerProperties.png | bin | 0 -> 11667 bytes | |||
| -rw-r--r-- | docs/testing/user/configguide/installation.rst | 310 | ||||
| -rw-r--r-- | docs/testing/user/configguide/trafficgen.rst | 671 | ||||
| -rw-r--r-- | docs/testing/user/configguide/upgrade.rst | 183 | ||||
| -rw-r--r-- | docs/testing/user/userguide/integration.rst | 504 | ||||
| -rw-r--r-- | docs/testing/user/userguide/teststeps.rst | 667 | ||||
| -rw-r--r-- | docs/testing/user/userguide/testusage.rst | 848 | ||||
| -rw-r--r-- | docs/testing/user/userguide/yardstick.rst | 250 |
9 files changed, 3435 insertions, 0 deletions
diff --git a/docs/testing/user/configguide/LICENSE b/docs/testing/user/configguide/LICENSE new file mode 100644 index 00000000..7bc572ce --- /dev/null +++ b/docs/testing/user/configguide/LICENSE @@ -0,0 +1,2 @@ +This work is licensed under a Creative Commons Attribution 4.0 International License. +http://creativecommons.org/licenses/by/4.0 diff --git a/docs/testing/user/configguide/TCLServerProperties.png b/docs/testing/user/configguide/TCLServerProperties.png Binary files differnew file mode 100644 index 00000000..682de7c5 --- /dev/null +++ b/docs/testing/user/configguide/TCLServerProperties.png diff --git a/docs/testing/user/configguide/installation.rst b/docs/testing/user/configguide/installation.rst new file mode 100644 index 00000000..1965a8f5 --- /dev/null +++ b/docs/testing/user/configguide/installation.rst @@ -0,0 +1,310 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intel Corporation, AT&T and others. + +.. _vsperf-installation: + +====================== +Installing vswitchperf +====================== + +Downloading vswitchperf +----------------------- + +The vswitchperf can be downloaded from its official git repository, which is +hosted by OPNFV. It is necessary to install a ``git`` at your DUT before downloading +vswitchperf. Installation of ``git`` is specific to the packaging system used by +Linux OS installed at DUT. + +Example of installation of GIT package and its dependencies: + +* in case of OS based on RedHat Linux: + + .. code:: bash + + sudo yum install git + + +* in case of Ubuntu or Debian: + + .. code:: bash + + sudo apt-get install git + +After the ``git`` is successfully installed at DUT, then vswitchperf can be downloaded +as follows: + +.. code:: bash + + git clone http://git.opnfv.org/vswitchperf + +The last command will create a directory ``vswitchperf`` with a local copy of vswitchperf +repository. + +Supported Operating Systems +--------------------------- + +* CentOS 7.3 +* Fedora 24 (kernel 4.8 requires DPDK 16.11 and newer) +* Fedora 25 (kernel 4.9 requires DPDK 16.11 and newer) +* openSUSE 42.2 +* RedHat 7.2 Enterprise Linux +* RedHat 7.3 Enterprise Linux +* Ubuntu 14.04 +* Ubuntu 16.04 +* Ubuntu 16.10 (kernel 4.8 requires DPDK 16.11 and newer) + +Supported vSwitches +------------------- + +The vSwitch must support Open Flow 1.3 or greater. + +* Open vSwitch +* Open vSwitch with DPDK support +* TestPMD application from DPDK (supports p2p and pvp scenarios) + +Supported Hypervisors +--------------------- + +* Qemu version 2.3 or greater (version 2.5.0 is recommended) + +Supported VNFs +-------------- + +In theory, it is possible to use any VNF image, which is compatible +with supported hypervisor. However such VNF must ensure, that appropriate +number of network interfaces is configured and that traffic is properly +forwarded among them. For new vswitchperf users it is recommended to start +with official vloop-vnf_ image, which is maintained by vswitchperf community. + +.. _vloop-vnf: + +vloop-vnf +========= + +The official VM image is called vloop-vnf and it is available for free download +from OPNFV artifactory. This image is based on Linux Ubuntu distribution and it +supports following applications for traffic forwarding: + +* DPDK testpmd +* Linux Bridge +* Custom l2fwd module + +The vloop-vnf can be downloaded to DUT, for example by ``wget``: + + .. code:: bash + + wget http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20160823.qcow2 + +**NOTE:** In case that ``wget`` is not installed at your DUT, you could install it at RPM +based system by ``sudo yum install wget`` or at DEB based system by ``sudo apt-get install +wget``. + +Changelog of vloop-vnf: + + * `vloop-vnf-ubuntu-14.04_20160823`_ + + * ethtool installed + * only 1 NIC is configured by default to speed up boot with 1 NIC setup + * security updates applied + + * `vloop-vnf-ubuntu-14.04_20160804`_ + + * Linux kernel 4.4.0 installed + * libnuma-dev installed + * security updates applied + + * `vloop-vnf-ubuntu-14.04_20160303`_ + + * snmpd service is disabled by default to avoid error messages during VM boot + * security updates applied + + * `vloop-vnf-ubuntu-14.04_20151216`_ + + * version with development tools required for build of DPDK and l2fwd + +.. _vsperf-installation-script: + +Installation +------------ + +The test suite requires Python 3.3 or newer and relies on a number of other +system and python packages. These need to be installed for the test suite +to function. + +Installation of required packages, preparation of Python 3 virtual +environment and compilation of OVS, DPDK and QEMU is performed by +script **systems/build_base_machine.sh**. It should be executed under +user account, which will be used for vsperf execution. + +**NOTE:** Password-less sudo access must be configured for given +user account before script is executed. + +.. code:: bash + + $ cd systems + $ ./build_base_machine.sh + +**NOTE:** you don't need to go into any of the systems subdirectories, +simply run the top level **build_base_machine.sh**, your OS will be detected +automatically. + +Script **build_base_machine.sh** will install all the vsperf dependencies +in terms of system packages, Python 3.x and required Python modules. +In case of CentOS 7 or RHEL it will install Python 3.3 from an additional +repository provided by Software Collections (`a link`_). Installation script +will also use `virtualenv`_ to create a vsperf virtual environment, which is +isolated from the default Python environment. This environment will reside in a +directory called **vsperfenv** in $HOME. It will ensure, that system wide Python +installation is not modified or broken by VSPERF installation. The complete list +of Python packages installed inside virtualenv can be found at file +``requirements.txt``, which is located at vswitchperf repository. + +**NOTE:** For RHEL 7.3 Enterprise and CentOS 7.3 OVS Vanilla is not +built from upstream source due to kernel incompatibilities. Please see the +instructions in the vswitchperf_design document for details on configuring +OVS Vanilla for binary package usage. + +.. _vpp-installation: + +VPP installation +================ + +Currently vswitchperf installation scripts do not support automatic build +of VPP. In order to execute tests with VPP, it is required to install it +manually. Please refer to the official documentation of `fd.io`_ project to +install VPP from `packages`_ or from the `sources`_. + +See details about :ref:`vpp-test`. + +.. _fd.io: https://fd.io/ +.. _packages: https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_packages +.. _sources: https://wiki.fd.io/view/VPP/Build,_install,_and_test_images + +Using vswitchperf +----------------- + +You will need to activate the virtual environment every time you start a +new shell session. Its activation is specific to your OS: + +* CentOS 7 and RHEL + + .. code:: bash + + $ scl enable python33 bash + $ source $HOME/vsperfenv/bin/activate + +* Fedora and Ubuntu + + .. code:: bash + + $ source $HOME/vsperfenv/bin/activate + +After the virtual environment is configued, then VSPERF can be used. +For example: + + .. code:: bash + + (vsperfenv) $ cd vswitchperf + (vsperfenv) $ ./vsperf --help + +Gotcha +====== + +In case you will see following error during environment activation: + +.. code:: bash + + $ source $HOME/vsperfenv/bin/activate + Badly placed ()'s. + +then check what type of shell you are using: + +.. code:: bash + + $ echo $SHELL + /bin/tcsh + +See what scripts are available in $HOME/vsperfenv/bin + +.. code:: bash + + $ ls $HOME/vsperfenv/bin/ + activate activate.csh activate.fish activate_this.py + +source the appropriate script + +.. code:: bash + + $ source bin/activate.csh + +Working Behind a Proxy +====================== + +If you're behind a proxy, you'll likely want to configure this before +running any of the above. For example: + + .. code:: bash + + export http_proxy=proxy.mycompany.com:123 + export https_proxy=proxy.mycompany.com:123 + +.. _a link: http://www.softwarecollections.org/en/scls/rhscl/python33/ +.. _virtualenv: https://virtualenv.readthedocs.org/en/latest/ +.. _vloop-vnf-ubuntu-14.04_20160823: http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20160823.qcow2 +.. _vloop-vnf-ubuntu-14.04_20160804: http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20160804.qcow2 +.. _vloop-vnf-ubuntu-14.04_20160303: http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20160303.qcow2 +.. _vloop-vnf-ubuntu-14.04_20151216: http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-14.04_20151216.qcow2 + +Hugepage Configuration +---------------------- + +Systems running vsperf with either dpdk and/or tests with guests must configure +hugepage amounts to support running these configurations. It is recommended +to configure 1GB hugepages as the pagesize. + +The amount of hugepages needed depends on your configuration files in vsperf. +Each guest image requires 2048 MB by default according to the default settings +in the ``04_vnf.conf`` file. + +.. code:: bash + + GUEST_MEMORY = ['2048'] + +The dpdk startup parameters also require an amount of hugepages depending on +your configuration in the ``02_vswitch.conf`` file. + +.. code:: bash + + DPDK_SOCKET_MEM = ['1024', '0'] + +**NOTE:** Option ``DPDK_SOCKET_MEM`` is used by all vSwitches with DPDK support. +It means Open vSwitch, VPP and TestPMD. + +VSPerf will verify hugepage amounts are free before executing test +environments. In case of hugepage amounts not being free, test initialization +will fail and testing will stop. + +**NOTE:** In some instances on a test failure dpdk resources may not +release hugepages used in dpdk configuration. It is recommended to configure a +few extra hugepages to prevent a false detection by VSPerf that not enough free +hugepages are available to execute the test environment. Normally dpdk would use +previously allocated hugepages upon initialization. + +Depending on your OS selection configuration of hugepages may vary. Please refer +to your OS documentation to set hugepages correctly. It is recommended to set +the required amount of hugepages to be allocated by default on reboots. + +Information on hugepage requirements for dpdk can be found at +http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html + +You can review your hugepage amounts by executing the following command + +.. code:: bash + + cat /proc/meminfo | grep Huge + +If no hugepages are available vsperf will try to automatically allocate some. +Allocation is controlled by ``HUGEPAGE_RAM_ALLOCATION`` configuration parameter in +``02_vswitch.conf`` file. Default is 2GB, resulting in either 2 1GB hugepages +or 1024 2MB hugepages. diff --git a/docs/testing/user/configguide/trafficgen.rst b/docs/testing/user/configguide/trafficgen.rst new file mode 100644 index 00000000..4e42b2be --- /dev/null +++ b/docs/testing/user/configguide/trafficgen.rst @@ -0,0 +1,671 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intel Corporation, AT&T and others. + +.. _trafficgen-installation: + +=========================== +'vsperf' Traffic Gen Guide +=========================== + +Overview +-------- + +VSPERF supports the following traffic generators: + + * Dummy_ (DEFAULT) + * Ixia_ + * `Spirent TestCenter`_ + * `Xena Networks`_ + * MoonGen_ + +To see the list of traffic gens from the cli: + +.. code-block:: console + + $ ./vsperf --list-trafficgens + +This guide provides the details of how to install +and configure the various traffic generators. + +Background Information +---------------------- +The traffic default configuration can be found in **conf/03_traffic.conf**, +and is configured as follows: + +.. code-block:: console + + TRAFFIC = { + 'traffic_type' : 'rfc2544_throughput', + 'frame_rate' : 100, + 'bidir' : 'True', # will be passed as string in title format to tgen + 'multistream' : 0, + 'stream_type' : 'L4', + 'pre_installed_flows' : 'No', # used by vswitch implementation + 'flow_type' : 'port', # used by vswitch implementation + + 'l2': { + 'framesize': 64, + 'srcmac': '00:00:00:00:00:00', + 'dstmac': '00:00:00:00:00:00', + }, + 'l3': { + 'proto': 'udp', + 'srcip': '1.1.1.1', + 'dstip': '90.90.90.90', + }, + 'l4': { + 'srcport': 3000, + 'dstport': 3001, + }, + 'vlan': { + 'enabled': False, + 'id': 0, + 'priority': 0, + 'cfi': 0, + }, + } + +The framesize parameter can be overridden from the configuration +files by adding the following to your custom configuration file +``10_custom.conf``: + +.. code-block:: console + + TRAFFICGEN_PKT_SIZES = (64, 128,) + +OR from the commandline: + +.. code-block:: console + + $ ./vsperf --test-params "TRAFFICGEN_PKT_SIZES=(x,y)" $TESTNAME + +You can also modify the traffic transmission duration and the number +of tests run by the traffic generator by extending the example +commandline above to: + +.. code-block:: console + + $ ./vsperf --test-params "TRAFFICGEN_PKT_SIZES=(x,y);TRAFFICGEN_DURATION=10;" \ + "TRAFFICGEN_RFC2544_TESTS=1" $TESTNAME + +.. _trafficgen-dummy: + +Dummy +----- + +The Dummy traffic generator can be used to test VSPERF installation or +to demonstrate VSPERF functionality at DUT without connection +to a real traffic generator. + +You could also use the Dummy generator in case, that your external +traffic generator is not supported by VSPERF. In such case you could +use VSPERF to setup your test scenario and then transmit the traffic. +After the transmission is completed you could specify values for all +collected metrics and VSPERF will use them to generate final reports. + +Setup +~~~~~ + +To select the Dummy generator please add the following to your +custom configuration file ``10_custom.conf``. + +.. code-block:: console + + TRAFFICGEN = 'Dummy' + +OR run ``vsperf`` with the ``--trafficgen`` argument + +.. code-block:: console + + $ ./vsperf --trafficgen Dummy $TESTNAME + +Where $TESTNAME is the name of the vsperf test you would like to run. +This will setup the vSwitch and the VNF (if one is part of your test) +print the traffic configuration and prompt you to transmit traffic +when the setup is complete. + +.. code-block:: console + + Please send 'continuous' traffic with the following stream config: + 30mS, 90mpps, multistream False + and the following flow config: + { + "flow_type": "port", + "l3": { + "srcip": "1.1.1.1", + "proto": "tcp", + "dstip": "90.90.90.90" + }, + "traffic_type": "rfc2544_continuous", + "multistream": 0, + "bidir": "True", + "vlan": { + "cfi": 0, + "priority": 0, + "id": 0, + "enabled": false + }, + "frame_rate": 90, + "l2": { + "dstport": 3001, + "srcport": 3000, + "dstmac": "00:00:00:00:00:00", + "srcmac": "00:00:00:00:00:00", + "framesize": 64 + } + } + What was the result for 'frames tx'? + +When your traffic generator has completed traffic transmission and provided +the results please input these at the VSPERF prompt. VSPERF will try +to verify the input: + +.. code-block:: console + + Is '$input_value' correct? + +Please answer with y OR n. + +VSPERF will ask you to provide a value for every of collected metrics. The list +of metrics can be found at traffic-type-metrics_. +Finally vsperf will print out the results for your test and generate the +appropriate logs and report files. + +.. _traffic-type-metrics: + +Metrics collected for supported traffic types +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Below you could find a list of metrics collected by VSPERF for each of supported +traffic types. + +RFC2544 Throughput and Continuous: + + * frames tx + * frames rx + * min latency + * max latency + * avg latency + * frameloss + +RFC2544 Back2back: + + * b2b frames + * b2b frame loss % + +Dummy result pre-configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In case of a Dummy traffic generator it is possible to pre-configure the test +results. This is useful for creation of demo testcases, which do not require +a real traffic generator. Such testcase can be run by any user and it will still +generate all reports and result files. + +Result values can be specified within ``TRAFFICGEN_DUMMY_RESULTS`` dictionary, +where every of collected metrics must be properly defined. Please check the list +of traffic-type-metrics_. + +Dictionary with dummy results can be passed by CLI argument ``--test-params`` +or specified in ``Parameters`` section of testcase definition. + +Example of testcase execution with dummy results defined by CLI argument: + +.. code-block:: console + + $ ./vsperf back2back --trafficgen Dummy --test-params \ + "TRAFFICGEN_DUMMY_RESULTS={'b2b frames':'3000','b2b frame loss %':'0.0'}" + +Example of testcase definition with pre-configured dummy results: + +.. code-block:: python + + { + "Name": "back2back", + "Traffic Type": "rfc2544_back2back", + "Deployment": "p2p", + "biDirectional": "True", + "Description": "LTD.Throughput.RFC2544.BackToBackFrames", + "Parameters" : { + 'TRAFFICGEN_DUMMY_RESULTS' : {'b2b frames':'3000','b2b frame loss %':'0.0'} + }, + }, + +**NOTE:** Pre-configured results for the Dummy traffic generator will be used only +in case, that the Dummy traffic generator is used. Otherwise the option +``TRAFFICGEN_DUMMY_RESULTS`` will be ignored. + +.. _Ixia: + +Ixia +---- + +VSPERF can use both IxNetwork and IxExplorer TCL servers to control Ixia chassis. +However usage of IxNetwork TCL server is a preferred option. Following sections +will describe installation and configuration of IxNetwork components used by VSPERF. + +Installation +~~~~~~~~~~~~ + +On the system under the test you need to install IxNetworkTclClient$(VER\_NUM)Linux.bin.tgz. + +On the IXIA client software system you need to install IxNetwork TCL server. After its +installation you should configure it as follows: + + 1. Find the IxNetwork TCL server app (start -> All Programs -> IXIA -> + IxNetwork -> IxNetwork\_$(VER\_NUM) -> IxNetwork TCL Server) + 2. Right click on IxNetwork TCL Server, select properties - Under shortcut tab in + the Target dialogue box make sure there is the argument "-tclport xxxx" + where xxxx is your port number (take note of this port number as you will + need it for the 10\_custom.conf file). + + .. image:: TCLServerProperties.png + + 3. Hit Ok and start the TCL server application + +VSPERF configuration +~~~~~~~~~~~~~~~~~~~~ + +There are several configuration options specific to the IxNetwork traffic generator +from IXIA. It is essential to set them correctly, before the VSPERF is executed +for the first time. + +Detailed description of options follows: + + * ``TRAFFICGEN_IXNET_MACHINE`` - IP address of server, where IxNetwork TCL Server is running + * ``TRAFFICGEN_IXNET_PORT`` - PORT, where IxNetwork TCL Server is accepting connections from + TCL clients + * ``TRAFFICGEN_IXNET_USER`` - username, which will be used during communication with IxNetwork + TCL Server and IXIA chassis + * ``TRAFFICGEN_IXIA_HOST`` - IP address of IXIA traffic generator chassis + * ``TRAFFICGEN_IXIA_CARD`` - identification of card with dedicated ports at IXIA chassis + * ``TRAFFICGEN_IXIA_PORT1`` - identification of the first dedicated port at ``TRAFFICGEN_IXIA_CARD`` + at IXIA chassis; VSPERF uses two separated ports for traffic generation. In case of + unidirectional traffic, it is essential to correctly connect 1st IXIA port to the 1st NIC + at DUT, i.e. to the first PCI handle from ``WHITELIST_NICS`` list. Otherwise traffic may not + be able to pass through the vSwitch. + * ``TRAFFICGEN_IXIA_PORT2`` - identification of the second dedicated port at ``TRAFFICGEN_IXIA_CARD`` + at IXIA chassis; VSPERF uses two separated ports for traffic generation. In case of + unidirectional traffic, it is essential to correctly connect 2nd IXIA port to the 2nd NIC + at DUT, i.e. to the second PCI handle from ``WHITELIST_NICS`` list. Otherwise traffic may not + be able to pass through the vSwitch. + * ``TRAFFICGEN_IXNET_LIB_PATH`` - path to the DUT specific installation of IxNetwork TCL API + * ``TRAFFICGEN_IXNET_TCL_SCRIPT`` - name of the TCL script, which VSPERF will use for + communication with IXIA TCL server + * ``TRAFFICGEN_IXNET_TESTER_RESULT_DIR`` - folder accessible from IxNetwork TCL server, + where test results are stored, e.g. ``c:/ixia_results``; see test-results-share_ + * ``TRAFFICGEN_IXNET_DUT_RESULT_DIR`` - directory accessible from the DUT, where test + results from IxNetwork TCL server are stored, e.g. ``/mnt/ixia_results``; see + test-results-share_ + +.. _test-results-share: + +Test results share +~~~~~~~~~~~~~~~~~~ + +VSPERF is not able to retrieve test results via TCL API directly. Instead, all test +results are stored at IxNetwork TCL server. Results are stored at folder defined by +``TRAFFICGEN_IXNET_TESTER_RESULT_DIR`` configuration parameter. Content of this +folder must be shared (e.g. via samba protocol) between TCL Server and DUT, where +VSPERF is executed. VSPERF expects, that test results will be available at directory +configured by ``TRAFFICGEN_IXNET_DUT_RESULT_DIR`` configuration parameter. + +Example of sharing configuration: + + * Create a new folder at IxNetwork TCL server machine, e.g. ``c:\ixia_results`` + * Modify sharing options of ``ixia_results`` folder to share it with everybody + * Create a new directory at DUT, where shared directory with results + will be mounted, e.g. ``/mnt/ixia_results`` + * Update your custom VSPERF configuration file as follows: + + .. code-block:: python + + TRAFFICGEN_IXNET_TESTER_RESULT_DIR = 'c:/ixia_results' + TRAFFICGEN_IXNET_DUT_RESULT_DIR = '/mnt/ixia_results' + + **NOTE:** It is essential to use slashes '/' also in path + configured by ``TRAFFICGEN_IXNET_TESTER_RESULT_DIR`` parameter. + * Install cifs-utils package. + + e.g. at rpm based Linux distribution: + + .. code-block:: console + + yum install cifs-utils + + * Mount shared directory, so VSPERF can access test results. + + e.g. by adding new record into ``/etc/fstab`` + + .. code-block:: console + + mount -t cifs //_TCL_SERVER_IP_OR_FQDN_/ixia_results /mnt/ixia_results + -o file_mode=0777,dir_mode=0777,nounix + +It is recommended to verify, that any new file inserted into ``c:/ixia_results`` folder +is visible at DUT inside ``/mnt/ixia_results`` directory. + +.. _`Spirent TestCenter`: + +Spirent Setup +------------- + +Spirent installation files and instructions are available on the +Spirent support website at: + +http://support.spirent.com + +Select a version of Spirent TestCenter software to utilize. This example +will use Spirent TestCenter v4.57 as an example. Substitute the appropriate +version in place of 'v4.57' in the examples, below. + +On the CentOS 7 System +~~~~~~~~~~~~~~~~~~~~~~ + +Download and install the following: + +Spirent TestCenter Application, v4.57 for 64-bit Linux Client + +Spirent Virtual Deployment Service (VDS) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Spirent VDS is required for both TestCenter hardware and virtual +chassis in the vsperf environment. For installation, select the version +that matches the Spirent TestCenter Application version. For v4.57, +the matching VDS version is 1.0.55. Download either the ova (VMware) +or qcow2 (QEMU) image and create a VM with it. Initialize the VM +according to Spirent installation instructions. + +Using Spirent TestCenter Virtual (STCv) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +STCv is available in both ova (VMware) and qcow2 (QEMU) formats. For +VMware, download: + +Spirent TestCenter Virtual Machine for VMware, v4.57 for Hypervisor - VMware ESX.ESXi + +Virtual test port performance is affected by the hypervisor configuration. For +best practice results in deploying STCv, the following is suggested: + +- Create a single VM with two test ports rather than two VMs with one port each +- Set STCv in DPDK mode +- Give STCv 2*n + 1 cores, where n = the number of ports. For vsperf, cores = 5. +- Turning off hyperthreading and pinning these cores will improve performance +- Give STCv 2 GB of RAM + +To get the highest performance and accuracy, Spirent TestCenter hardware is +recommended. vsperf can run with either stype test ports. + +Using STC REST Client +~~~~~~~~~~~~~~~~~~~~~ +The stcrestclient package provides the stchttp.py ReST API wrapper module. +This allows simple function calls, nearly identical to those provided by +StcPython.py, to be used to access TestCenter server sessions via the +STC ReST API. Basic ReST functionality is provided by the resthttp module, +and may be used for writing ReST clients independent of STC. + +- Project page: <https://github.com/Spirent/py-stcrestclient> +- Package download: <http://pypi.python.org/pypi/stcrestclient> + +To use REST interface, follow the instructions in the Project page to +install the package. Once installed, the scripts named with 'rest' keyword +can be used. For example: testcenter-rfc2544-rest.py can be used to run +RFC 2544 tests using the REST interface. + +Configuration: +~~~~~~~~~~~~~~ + +1. The Labserver and license server addresses. These parameters applies to + all the tests, and are mandatory for all tests. + +.. code-block:: console + + TRAFFICGEN_STC_LAB_SERVER_ADDR = " " + TRAFFICGEN_STC_LICENSE_SERVER_ADDR = " " + TRAFFICGEN_STC_PYTHON2_PATH = " " + TRAFFICGEN_STC_TESTCENTER_PATH = " " + TRAFFICGEN_STC_TEST_SESSION_NAME = " " + TRAFFICGEN_STC_CSV_RESULTS_FILE_PREFIX = " " + +2. For RFC2544 tests, the following parameters are mandatory + +.. code-block:: console + + TRAFFICGEN_STC_EAST_CHASSIS_ADDR = " " + TRAFFICGEN_STC_EAST_SLOT_NUM = " " + TRAFFICGEN_STC_EAST_PORT_NUM = " " + TRAFFICGEN_STC_EAST_INTF_ADDR = " " + TRAFFICGEN_STC_EAST_INTF_GATEWAY_ADDR = " " + TRAFFICGEN_STC_WEST_CHASSIS_ADDR = "" + TRAFFICGEN_STC_WEST_SLOT_NUM = " " + TRAFFICGEN_STC_WEST_PORT_NUM = " " + TRAFFICGEN_STC_WEST_INTF_ADDR = " " + TRAFFICGEN_STC_WEST_INTF_GATEWAY_ADDR = " " + TRAFFICGEN_STC_RFC2544_TPUT_TEST_FILE_NAME + +3. RFC2889 tests: Currently, the forwarding, address-caching, and + address-learning-rate tests of RFC2889 are supported. + The testcenter-rfc2889-rest.py script implements the rfc2889 tests. + The configuration for RFC2889 involves test-case definition, and parameter + definition, as described below. New results-constants, as shown below, are + added to support these tests. + +Example of testcase definition for RFC2889 tests: + +.. code-block:: python + + { + "Name": "phy2phy_forwarding", + "Deployment": "p2p", + "Description": "LTD.Forwarding.RFC2889.MaxForwardingRate", + "Parameters" : { + "TRAFFIC" : { + "traffic_type" : "rfc2889_forwarding", + }, + }, + } + +For RFC2889 tests, specifying the locations for the monitoring ports is mandatory. +Necessary parameters are: + +.. code-block:: console + + TRAFFICGEN_STC_RFC2889_TEST_FILE_NAME + TRAFFICGEN_STC_EAST_CHASSIS_ADDR = " " + TRAFFICGEN_STC_EAST_SLOT_NUM = " " + TRAFFICGEN_STC_EAST_PORT_NUM = " " + TRAFFICGEN_STC_EAST_INTF_ADDR = " " + TRAFFICGEN_STC_EAST_INTF_GATEWAY_ADDR = " " + TRAFFICGEN_STC_WEST_CHASSIS_ADDR = "" + TRAFFICGEN_STC_WEST_SLOT_NUM = " " + TRAFFICGEN_STC_WEST_PORT_NUM = " " + TRAFFICGEN_STC_WEST_INTF_ADDR = " " + TRAFFICGEN_STC_WEST_INTF_GATEWAY_ADDR = " " + TRAFFICGEN_STC_VERBOSE = "True" + TRAFFICGEN_STC_RFC2889_LOCATIONS="//10.1.1.1/1/1,//10.1.1.1/2/2" + +Other Configurations are : + +.. code-block:: console + + TRAFFICGEN_STC_RFC2889_MIN_LR = 1488 + TRAFFICGEN_STC_RFC2889_MAX_LR = 14880 + TRAFFICGEN_STC_RFC2889_MIN_ADDRS = 1000 + TRAFFICGEN_STC_RFC2889_MAX_ADDRS = 65536 + TRAFFICGEN_STC_RFC2889_AC_LR = 1000 + +The first 2 values are for address-learning test where as other 3 values are +for the Address caching capacity test. LR: Learning Rate. AC: Address Caching. +Maximum value for address is 16777216. Whereas, maximum for LR is 4294967295. + +Results for RFC2889 Tests: Forwarding tests outputs following values: + +.. code-block:: console + + TX_RATE_FPS : "Transmission Rate in Frames/sec" + THROUGHPUT_RX_FPS: "Received Throughput Frames/sec" + TX_RATE_MBPS : " Transmission rate in MBPS" + THROUGHPUT_RX_MBPS: "Received Throughput in MBPS" + TX_RATE_PERCENT: "Transmission Rate in Percentage" + FRAME_LOSS_PERCENT: "Frame loss in Percentage" + FORWARDING_RATE_FPS: " Maximum Forwarding Rate in FPS" + + +Whereas, the address caching test outputs following values, + +.. code-block:: console + + CACHING_CAPACITY_ADDRS = 'Number of address it can cache' + ADDR_LEARNED_PERCENT = 'Percentage of address successfully learned' + +and address learning test outputs just a single value: + +.. code-block:: console + + OPTIMAL_LEARNING_RATE_FPS = 'Optimal learning rate in fps' + +Note that 'FORWARDING_RATE_FPS', 'CACHING_CAPACITY_ADDRS', +'ADDR_LEARNED_PERCENT' and 'OPTIMAL_LEARNING_RATE_FPS' are the new +result-constants added to support RFC2889 tests. + +.. _`Xena Networks`: + +Xena Networks +------------- + +Installation +~~~~~~~~~~~~ + +Xena Networks traffic generator requires specific files and packages to be +installed. It is assumed the user has access to the Xena2544.exe file which +must be placed in VSPerf installation location under the tools/pkt_gen/xena +folder. Contact Xena Networks for the latest version of this file. The user +can also visit www.xenanetworks/downloads to obtain the file with a valid +support contract. + +**Note** VSPerf has been fully tested with version v2.43 of Xena2544.exe + +To execute the Xena2544.exe file under Linux distributions the mono-complete +package must be installed. To install this package follow the instructions +below. Further information can be obtained from +http://www.mono-project.com/docs/getting-started/install/linux/ + +.. code-block:: console + + rpm --import "http://keyserver.ubuntu.com/pks/lookup?op=get&search=0x3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF" + yum-config-manager --add-repo http://download.mono-project.com/repo/centos/ + yum -y install mono-complete + +To prevent gpg errors on future yum installation of packages the mono-project +repo should be disabled once installed. + +.. code-block:: console + + yum-config-manager --disable download.mono-project.com_repo_centos_ + +Configuration +~~~~~~~~~~~~~ + +Connection information for your Xena Chassis must be supplied inside the +``10_custom.conf`` or ``03_custom.conf`` file. The following parameters must be +set to allow for proper connections to the chassis. + +.. code-block:: console + + TRAFFICGEN_XENA_IP = '' + TRAFFICGEN_XENA_PORT1 = '' + TRAFFICGEN_XENA_PORT2 = '' + TRAFFICGEN_XENA_USER = '' + TRAFFICGEN_XENA_PASSWORD = '' + TRAFFICGEN_XENA_MODULE1 = '' + TRAFFICGEN_XENA_MODULE2 = '' + +RFC2544 Throughput Testing +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Xena traffic generator testing for rfc2544 throughput can be modified for +different behaviors if needed. The default options for the following are +optimized for best results. + +.. code-block:: console + + TRAFFICGEN_XENA_2544_TPUT_INIT_VALUE = '10.0' + TRAFFICGEN_XENA_2544_TPUT_MIN_VALUE = '0.1' + TRAFFICGEN_XENA_2544_TPUT_MAX_VALUE = '100.0' + TRAFFICGEN_XENA_2544_TPUT_VALUE_RESOLUTION = '0.5' + TRAFFICGEN_XENA_2544_TPUT_USEPASS_THRESHHOLD = 'false' + TRAFFICGEN_XENA_2544_TPUT_PASS_THRESHHOLD = '0.0' + +Each value modifies the behavior of rfc 2544 throughput testing. Refer to your +Xena documentation to understand the behavior changes in modifying these +values. + +Continuous Traffic Testing +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Xena continuous traffic by default does a 3 second learning preemption to allow +the DUT to receive learning packets before a continuous test is performed. If +a custom test case requires this learning be disabled, you can disable the option +or modify the length of the learning by modifying the following settings. + +.. code-block:: console + + TRAFFICGEN_XENA_CONT_PORT_LEARNING_ENABLED = False + TRAFFICGEN_XENA_CONT_PORT_LEARNING_DURATION = 3 + +MoonGen +------- + +Installation +~~~~~~~~~~~~ + +MoonGen architecture overview and general installation instructions +can be found here: + +https://github.com/emmericp/MoonGen + +* Note: Today, MoonGen with VSPERF only supports 10Gbps line speeds. + +For VSPERF use, MoonGen should be cloned from here (as opposed to the +previously mentioned GitHub): + +git clone https://github.com/atheurer/lua-trafficgen + +and use the master branch: + +git checkout master + +VSPERF uses a particular Lua script with the MoonGen project: + +trafficgen.lua + +Follow MoonGen set up and execution instructions here: + +https://github.com/atheurer/lua-trafficgen/blob/master/README.md + +Note one will need to set up ssh login to not use passwords between the server +running MoonGen and the device under test (running the VSPERF test +infrastructure). This is because VSPERF on one server uses 'ssh' to +configure and run MoonGen upon the other server. + +One can set up this ssh access by doing the following on both servers: + +.. code-block:: console + + ssh-keygen -b 2048 -t rsa + ssh-copy-id <other server> + +Configuration +~~~~~~~~~~~~~ + +Connection information for MoonGen must be supplied inside the +``10_custom.conf`` or ``03_custom.conf`` file. The following parameters must be +set to allow for proper connections to the host with MoonGen. + +.. code-block:: console + + TRAFFICGEN_MOONGEN_HOST_IP_ADDR = "" + TRAFFICGEN_MOONGEN_USER = "" + TRAFFICGEN_MOONGEN_BASE_DIR = "" + TRAFFICGEN_MOONGEN_PORTS = "" + TRAFFICGEN_MOONGEN_LINE_SPEED_GBPS = "" diff --git a/docs/testing/user/configguide/upgrade.rst b/docs/testing/user/configguide/upgrade.rst new file mode 100644 index 00000000..cf92572c --- /dev/null +++ b/docs/testing/user/configguide/upgrade.rst @@ -0,0 +1,183 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intel Corporation, AT&T and others. + +===================== +Upgrading vswitchperf +===================== + +Generic +------- + +In case, that VSPERF is cloned from git repository, then it is easy to +upgrade it to the newest stable version or to the development version. + +You could get a list of stable releases by ``git`` command. It is necessary +to update local git repository first. + +**NOTE:** Git commands must be executed from directory, where VSPERF repository +was cloned, e.g. ``vswitchperf``. + +Update of local git repository: + +.. code:: bash + + $ git pull + +List of stable releases: + +.. code:: bash + + $ git tag + + brahmaputra.1.0 + colorado.1.0 + colorado.2.0 + colorado.3.0 + danube.1.0 + +You could select which stable release should be used. For example, select ``danube.1.0``: + +.. code:: bash + + $ git checkout danube.1.0 + + +Development version of VSPERF can be selected by: + +.. code:: bash + + $ git checkout master + +Colorado to Danube upgrade notes +-------------------------------- + +Obsoleted features +~~~~~~~~~~~~~~~~~~ + +Support of vHost Cuse interface has been removed in Danube release. It means, +that it is not possible to select ``QemuDpdkVhostCuse`` as a VNF anymore. Option +``QemuDpdkVhostUser`` should be used instead. Please check you configuration files +and definition of your testcases for any occurrence of: + +.. code:: python + + VNF = "QemuDpdkVhostCuse" + +or + +.. code:: python + + "VNF" : "QemuDpdkVhostCuse" + +In case that ``QemuDpdkVhostCuse`` is found, it must be modified to ``QemuDpdkVhostUser``. + +**NOTE:** In case that execution of VSPERF is automated by scripts (e.g. for +CI purposes), then these scripts must be checked and updated too. It means, +that any occurrence of: + +.. code:: bash + + ./vsperf --vnf QemuDpdkVhostCuse + +must be updated to: + +.. code:: bash + + ./vsperf --vnf QemuDpdkVhostUser + +Configuration +~~~~~~~~~~~~~ + +Several configuration changes were introduced during Danube release. The most +important changes are discussed below. + +Paths to DPDK, OVS and QEMU +=========================== + +VSPERF uses external tools for proper testcase execution. Thus it is important +to properly configure paths to these tools. In case that tools are installed +by installation scripts and are located inside ``./src`` directory inside +VSPERF home, then no changes are needed. On the other hand, if path settings +was changed by custom configuration file, then it is required to update configuration +accordingly. Please check your configuration files for following configuration +options: + +.. code:: bash + + OVS_DIR + OVS_DIR_VANILLA + OVS_DIR_USER + OVS_DIR_CUSE + + RTE_SDK_USER + RTE_SDK_CUSE + + QEMU_DIR + QEMU_DIR_USER + QEMU_DIR_CUSE + QEMU_BIN + +In case that any of these options is defined, then configuration must be updated. +All paths to the tools are now stored inside ``PATHS`` dictionary. Please +refer to the :ref:`paths-documentation` and update your configuration where necessary. + +Configuration change via CLI +============================ + +In previous releases it was possible to modify selected configuration options +(mostly VNF specific) via command line interface, i.e. by ``--test-params`` +argument. This concept has been generalized in Danube release and it is +possible to modify any configuration parameter via CLI or via **Parameters** +section of the testcase definition. Old configuration options were obsoleted +and it is required to specify configuration parameter name in the same form +as it is defined inside configuration file, i.e. in uppercase. Please +refer to the :ref:`overriding-parameters-documentation` for additional details. + +**NOTE:** In case that execution of VSPERF is automated by scripts (e.g. for +CI purposes), then these scripts must be checked and updated too. It means, +that any occurrence of + +.. code:: bash + + guest_loopback + vanilla_tgen_port1_ip + vanilla_tgen_port1_mac + vanilla_tgen_port2_ip + vanilla_tgen_port2_mac + tunnel_type + +shall be changed to the uppercase form and data type of entered values must +match to data types of original values from configuration files. + +In case that ``guest_nic1_name`` or ``guest_nic2_name`` is changed, +then new dictionary ``GUEST_NICS`` must be modified accordingly. +Please see :ref:`configuration-of-guest-options` and ``conf/04_vnf.conf`` for additional +details. + +Traffic configuration via CLI +============================= + +In previous releases it was possible to modify selected attributes of generated +traffic via command line interface. This concept has been enhanced in Danube +release and it is now possible to modify all traffic specific options via +CLI or by ``TRAFFIC`` dictionary in configuration file. Detailed description +is available at :ref:`configuration-of-traffic-dictionary` section of documentation. + +Please check your automated scripts for VSPERF execution for following CLI +parameters and update them according to the documentation: + +.. code:: bash + + bidir + duration + frame_rate + iload + lossrate + multistream + pkt_sizes + pre-installed_flows + rfc2544_tests + stream_type + traffic_type + diff --git a/docs/testing/user/userguide/integration.rst b/docs/testing/user/userguide/integration.rst new file mode 100644 index 00000000..83b29da6 --- /dev/null +++ b/docs/testing/user/userguide/integration.rst @@ -0,0 +1,504 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intel Corporation, AT&T and others. + +.. _integration-tests: + +Integration tests +================= + +VSPERF includes a set of integration tests defined in conf/integration. +These tests can be run by specifying --integration as a parameter to vsperf. +Current tests in conf/integration include switch functionality and Overlay +tests. + +Tests in the conf/integration can be used to test scaling of different switch +configurations by adding steps into the test case. + +For the overlay tests VSPERF supports VXLAN, GRE and GENEVE tunneling protocols. +Testing of these protocols is limited to unidirectional traffic and +P2P (Physical to Physical scenarios). + +NOTE: The configuration for overlay tests provided in this guide is for +unidirectional traffic only. + +Executing Integration Tests +--------------------------- + +To execute integration tests VSPERF is run with the integration parameter. To +view the current test list simply execute the following command: + +.. code-block:: console + + ./vsperf --integration --list + +The standard tests included are defined inside the +``conf/integration/01_testcases.conf`` file. + +Executing Tunnel encapsulation tests +------------------------------------ + +The VXLAN OVS DPDK encapsulation tests requires IPs, MAC addresses, +bridge names and WHITELIST_NICS for DPDK. + +NOTE: Only Ixia traffic generators currently support the execution of the tunnel +encapsulation tests. Support for other traffic generators may come in a future +release. + +Default values are already provided. To customize for your environment, override +the following variables in you user_settings.py file: + + .. code-block:: python + + # Variables defined in conf/integration/02_vswitch.conf + # Tunnel endpoint for Overlay P2P deployment scenario + # used for br0 + VTEP_IP1 = '192.168.0.1/24' + + # Used as remote_ip in adding OVS tunnel port and + # to set ARP entry in OVS (e.g. tnl/arp/set br-ext 192.168.240.10 02:00:00:00:00:02 + VTEP_IP2 = '192.168.240.10' + + # Network to use when adding a route for inner frame data + VTEP_IP2_SUBNET = '192.168.240.0/24' + + # Bridge names + TUNNEL_INTEGRATION_BRIDGE = 'br0' + TUNNEL_EXTERNAL_BRIDGE = 'br-ext' + + # IP of br-ext + TUNNEL_EXTERNAL_BRIDGE_IP = '192.168.240.1/24' + + # vxlan|gre|geneve + TUNNEL_TYPE = 'vxlan' + + # Variables defined conf/integration/03_traffic.conf + # For OP2P deployment scenario + TRAFFICGEN_PORT1_MAC = '02:00:00:00:00:01' + TRAFFICGEN_PORT2_MAC = '02:00:00:00:00:02' + TRAFFICGEN_PORT1_IP = '1.1.1.1' + TRAFFICGEN_PORT2_IP = '192.168.240.10' + +To run VXLAN encapsulation tests: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --integration \ + --test-params 'TUNNEL_TYPE=vxlan' overlay_p2p_tput + +To run GRE encapsulation tests: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --integration \ + --test-params 'TUNNEL_TYPE=gre' overlay_p2p_tput + +To run GENEVE encapsulation tests: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --integration \ + --test-params 'TUNNEL_TYPE=geneve' overlay_p2p_tput + +To run OVS NATIVE tunnel tests (VXLAN/GRE/GENEVE): + +1. Install the OVS kernel modules + + .. code:: console + + cd src/ovs/ovs + sudo -E make modules_install + +2. Set the following variables: + + .. code-block:: python + + VSWITCH = 'OvsVanilla' + # Specify vport_* kernel module to test. + PATHS['vswitch']['OvsVanilla']['src']['modules'] = [ + 'vport_vxlan', + 'vport_gre', + 'vport_geneve', + 'datapath/linux/openvswitch.ko', + ] + + **NOTE:** In case, that Vanilla OVS is installed from binary package, then + please set ``PATHS['vswitch']['OvsVanilla']['bin']['modules']`` instead. + +3. Run tests: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --integration \ + --test-params 'TUNNEL_TYPE=vxlan' overlay_p2p_tput + + +Executing VXLAN decapsulation tests +------------------------------------ + +To run VXLAN decapsulation tests: + +1. Set the variables used in "Executing Tunnel encapsulation tests" + +2. Set dstmac of DUT_NIC2_MAC to the MAC adddress of the 2nd NIC of your DUT + + .. code-block:: python + + DUT_NIC2_MAC = '<DUT NIC2 MAC>' + +3. Run test: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --integration overlay_p2p_decap_cont + +If you want to use different values for your VXLAN frame, you may set: + + .. code-block:: python + + VXLAN_FRAME_L3 = {'proto': 'udp', + 'packetsize': 64, + 'srcip': TRAFFICGEN_PORT1_IP, + 'dstip': '192.168.240.1', + } + VXLAN_FRAME_L4 = {'srcport': 4789, + 'dstport': 4789, + 'vni': VXLAN_VNI, + 'inner_srcmac': '01:02:03:04:05:06', + 'inner_dstmac': '06:05:04:03:02:01', + 'inner_srcip': '192.168.0.10', + 'inner_dstip': '192.168.240.9', + 'inner_proto': 'udp', + 'inner_srcport': 3000, + 'inner_dstport': 3001, + } + + +Executing GRE decapsulation tests +--------------------------------- + +To run GRE decapsulation tests: + +1. Set the variables used in "Executing Tunnel encapsulation tests" + +2. Set dstmac of DUT_NIC2_MAC to the MAC adddress of the 2nd NIC of your DUT + + .. code-block:: python + + DUT_NIC2_MAC = '<DUT NIC2 MAC>' + +3. Run test: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --test-params 'TUNNEL_TYPE=gre' \ + --integration overlay_p2p_decap_cont + + +If you want to use different values for your GRE frame, you may set: + + .. code-block:: python + + GRE_FRAME_L3 = {'proto': 'gre', + 'packetsize': 64, + 'srcip': TRAFFICGEN_PORT1_IP, + 'dstip': '192.168.240.1', + } + + GRE_FRAME_L4 = {'srcport': 0, + 'dstport': 0 + 'inner_srcmac': '01:02:03:04:05:06', + 'inner_dstmac': '06:05:04:03:02:01', + 'inner_srcip': '192.168.0.10', + 'inner_dstip': '192.168.240.9', + 'inner_proto': 'udp', + 'inner_srcport': 3000, + 'inner_dstport': 3001, + } + + +Executing GENEVE decapsulation tests +------------------------------------ + +IxNet 7.3X does not have native support of GENEVE protocol. The +template, GeneveIxNetTemplate.xml_ClearText.xml, should be imported +into IxNET for this testcase to work. + +To import the template do: + +1. Run the IxNetwork TCL Server +2. Click on the Traffic menu +3. Click on the Traffic actions and click Edit Packet Templates +4. On the Template editor window, click Import. Select the template + located at ``3rd_party/ixia/GeneveIxNetTemplate.xml_ClearText.xml`` + and click import. +5. Restart the TCL Server. + +To run GENEVE decapsulation tests: + +1. Set the variables used in "Executing Tunnel encapsulation tests" + +2. Set dstmac of DUT_NIC2_MAC to the MAC adddress of the 2nd NIC of your DUT + + .. code-block:: python + + DUT_NIC2_MAC = '<DUT NIC2 MAC>' + +3. Run test: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --test-params 'tunnel_type=geneve' \ + --integration overlay_p2p_decap_cont + + +If you want to use different values for your GENEVE frame, you may set: + + .. code-block:: python + + GENEVE_FRAME_L3 = {'proto': 'udp', + 'packetsize': 64, + 'srcip': TRAFFICGEN_PORT1_IP, + 'dstip': '192.168.240.1', + } + + GENEVE_FRAME_L4 = {'srcport': 6081, + 'dstport': 6081, + 'geneve_vni': 0, + 'inner_srcmac': '01:02:03:04:05:06', + 'inner_dstmac': '06:05:04:03:02:01', + 'inner_srcip': '192.168.0.10', + 'inner_dstip': '192.168.240.9', + 'inner_proto': 'udp', + 'inner_srcport': 3000, + 'inner_dstport': 3001, + } + + +Executing Native/Vanilla OVS VXLAN decapsulation tests +------------------------------------------------------ + +To run VXLAN decapsulation tests: + +1. Set the following variables in your user_settings.py file: + + .. code-block:: python + + PATHS['vswitch']['OvsVanilla']['src']['modules'] = [ + 'vport_vxlan', + 'datapath/linux/openvswitch.ko', + ] + + DUT_NIC1_MAC = '<DUT NIC1 MAC ADDRESS>' + + TRAFFICGEN_PORT1_IP = '172.16.1.2' + TRAFFICGEN_PORT2_IP = '192.168.1.11' + + VTEP_IP1 = '172.16.1.2/24' + VTEP_IP2 = '192.168.1.1' + VTEP_IP2_SUBNET = '192.168.1.0/24' + TUNNEL_EXTERNAL_BRIDGE_IP = '172.16.1.1/24' + TUNNEL_INT_BRIDGE_IP = '192.168.1.1' + + VXLAN_FRAME_L2 = {'srcmac': + '01:02:03:04:05:06', + 'dstmac': DUT_NIC1_MAC + } + + VXLAN_FRAME_L3 = {'proto': 'udp', + 'packetsize': 64, + 'srcip': TRAFFICGEN_PORT1_IP, + 'dstip': '172.16.1.1', + } + + VXLAN_FRAME_L4 = { + 'srcport': 4789, + 'dstport': 4789, + 'protocolpad': 'true', + 'vni': 99, + 'inner_srcmac': '01:02:03:04:05:06', + 'inner_dstmac': '06:05:04:03:02:01', + 'inner_srcip': '192.168.1.2', + 'inner_dstip': TRAFFICGEN_PORT2_IP, + 'inner_proto': 'udp', + 'inner_srcport': 3000, + 'inner_dstport': 3001, + } + + **NOTE:** In case, that Vanilla OVS is installed from binary package, then + please set ``PATHS['vswitch']['OvsVanilla']['bin']['modules']`` instead. + +2. Run test: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --integration \ + --test-params 'tunnel_type=vxlan' overlay_p2p_decap_cont + +Executing Native/Vanilla OVS GRE decapsulation tests +---------------------------------------------------- + +To run GRE decapsulation tests: + +1. Set the following variables in your user_settings.py file: + + .. code-block:: python + + PATHS['vswitch']['OvsVanilla']['src']['modules'] = [ + 'vport_gre', + 'datapath/linux/openvswitch.ko', + ] + + DUT_NIC1_MAC = '<DUT NIC1 MAC ADDRESS>' + + TRAFFICGEN_PORT1_IP = '172.16.1.2' + TRAFFICGEN_PORT2_IP = '192.168.1.11' + + VTEP_IP1 = '172.16.1.2/24' + VTEP_IP2 = '192.168.1.1' + VTEP_IP2_SUBNET = '192.168.1.0/24' + TUNNEL_EXTERNAL_BRIDGE_IP = '172.16.1.1/24' + TUNNEL_INT_BRIDGE_IP = '192.168.1.1' + + GRE_FRAME_L2 = {'srcmac': + '01:02:03:04:05:06', + 'dstmac': DUT_NIC1_MAC + } + + GRE_FRAME_L3 = {'proto': 'udp', + 'packetsize': 64, + 'srcip': TRAFFICGEN_PORT1_IP, + 'dstip': '172.16.1.1', + } + + GRE_FRAME_L4 = { + 'srcport': 4789, + 'dstport': 4789, + 'protocolpad': 'true', + 'inner_srcmac': '01:02:03:04:05:06', + 'inner_dstmac': '06:05:04:03:02:01', + 'inner_srcip': '192.168.1.2', + 'inner_dstip': TRAFFICGEN_PORT2_IP, + 'inner_proto': 'udp', + 'inner_srcport': 3000, + 'inner_dstport': 3001, + } + + **NOTE:** In case, that Vanilla OVS is installed from binary package, then + please set ``PATHS['vswitch']['OvsVanilla']['bin']['modules']`` instead. + +2. Run test: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --integration \ + --test-params 'tunnel_type=gre' overlay_p2p_decap_cont + +Executing Native/Vanilla OVS GENEVE decapsulation tests +------------------------------------------------------- + +To run GENEVE decapsulation tests: + +1. Set the following variables in your user_settings.py file: + + .. code-block:: python + + PATHS['vswitch']['OvsVanilla']['src']['modules'] = [ + 'vport_geneve', + 'datapath/linux/openvswitch.ko', + ] + + DUT_NIC1_MAC = '<DUT NIC1 MAC ADDRESS>' + + TRAFFICGEN_PORT1_IP = '172.16.1.2' + TRAFFICGEN_PORT2_IP = '192.168.1.11' + + VTEP_IP1 = '172.16.1.2/24' + VTEP_IP2 = '192.168.1.1' + VTEP_IP2_SUBNET = '192.168.1.0/24' + TUNNEL_EXTERNAL_BRIDGE_IP = '172.16.1.1/24' + TUNNEL_INT_BRIDGE_IP = '192.168.1.1' + + GENEVE_FRAME_L2 = {'srcmac': + '01:02:03:04:05:06', + 'dstmac': DUT_NIC1_MAC + } + + GENEVE_FRAME_L3 = {'proto': 'udp', + 'packetsize': 64, + 'srcip': TRAFFICGEN_PORT1_IP, + 'dstip': '172.16.1.1', + } + + GENEVE_FRAME_L4 = {'srcport': 6081, + 'dstport': 6081, + 'protocolpad': 'true', + 'geneve_vni': 0, + 'inner_srcmac': '01:02:03:04:05:06', + 'inner_dstmac': '06:05:04:03:02:01', + 'inner_srcip': '192.168.1.2', + 'inner_dstip': TRAFFICGEN_PORT2_IP, + 'inner_proto': 'udp', + 'inner_srcport': 3000, + 'inner_dstport': 3001, + } + + **NOTE:** In case, that Vanilla OVS is installed from binary package, then + please set ``PATHS['vswitch']['OvsVanilla']['bin']['modules']`` instead. + +2. Run test: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --integration \ + --test-params 'tunnel_type=geneve' overlay_p2p_decap_cont + + +Executing Tunnel encapsulation+decapsulation tests +-------------------------------------------------- + +The OVS DPDK encapsulation_decapsulation tests requires IPs, MAC addresses, +bridge names and WHITELIST_NICS for DPDK. + +The test cases can test the tunneling encap and decap without using any ingress +overlay traffic as compared to above test cases. To achieve this the OVS is +configured to perform encap and decap in a series on the same traffic stream as +given below. + +TRAFFIC-IN --> [ENCAP] --> [MOD-PKT] --> [DECAP] --> TRAFFIC-OUT + + +Default values are already provided. To customize for your environment, override +the following variables in you user_settings.py file: + + .. code-block:: python + + # Variables defined in conf/integration/02_vswitch.conf + + # Bridge names + TUNNEL_EXTERNAL_BRIDGE1 = 'br-phy1' + TUNNEL_EXTERNAL_BRIDGE2 = 'br-phy2' + TUNNEL_MODIFY_BRIDGE1 = 'br-mod1' + TUNNEL_MODIFY_BRIDGE2 = 'br-mod2' + + # IP of br-mod1 + TUNNEL_MODIFY_BRIDGE_IP1 = '10.0.0.1/24' + + # Mac of br-mod1 + TUNNEL_MODIFY_BRIDGE_MAC1 = '00:00:10:00:00:01' + + # IP of br-mod2 + TUNNEL_MODIFY_BRIDGE_IP2 = '20.0.0.1/24' + + #Mac of br-mod2 + TUNNEL_MODIFY_BRIDGE_MAC2 = '00:00:20:00:00:01' + + # vxlan|gre|geneve, Only VXLAN is supported for now. + TUNNEL_TYPE = 'vxlan' + +To run VXLAN encapsulation+decapsulation tests: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --integration \ + overlay_p2p_mod_tput diff --git a/docs/testing/user/userguide/teststeps.rst b/docs/testing/user/userguide/teststeps.rst new file mode 100644 index 00000000..870c3d80 --- /dev/null +++ b/docs/testing/user/userguide/teststeps.rst @@ -0,0 +1,667 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intel Corporation, AT&T and others. + +.. _step-driven-tests: + +Step driven tests +================= + +In general, test scenarios are defined by a ``deployment`` used in the particular +test case definition. The chosen deployment scenario will take care of the vSwitch +configuration, deployment of VNFs and it can also affect configuration of a traffic +generator. In order to allow a more flexible way of testcase scripting, VSPERF supports +a detailed step driven testcase definition. It can be used to configure and +program vSwitch, deploy and terminate VNFs, execute a traffic generator, +modify a VSPERF configuration, execute external commands, etc. + +Execution of step driven tests is done on a step by step work flow starting +with step 0 as defined inside the test case. Each step of the test increments +the step number by one which is indicated in the log. + +.. code-block:: console + + (testcases.integration) - Step 0 'vswitch add_vport ['br0']' start + +Step driven tests can be used for both performance and integration testing. +In case of integration test, each step in the test case is validated. If a step +does not pass validation the test will fail and terminate. The test will continue +until a failure is detected or all steps pass. A csv report file is generated after +a test completes with an OK or FAIL result. + +In case of performance test, the validation of steps is not performed and +standard output files with results from traffic generator and underlying OS +details are generated by vsperf. + +Step driven testcases can be used in two different ways: + + # description of full testcase - in this case ``clean`` deployment is used + to indicate that vsperf should neither configure vSwitch nor deploy any VNF. + Test shall perform all required vSwitch configuration and programming and + deploy required number of VNFs. + + # modification of existing deployment - in this case, any of supported + deployments can be used to perform initial vSwitch configuration and + deployment of VNFs. Additional actions defined by TestSteps can be used + to alter vSwitch configuration or deploy additional VNFs. After the last + step is processed, the test execution will continue with traffic execution. + +Test objects and their functions +-------------------------------- + +Every test step can call a function of one of the supported test objects. The list +of supported objects and their most common functions follows: + + * ``vswitch`` - provides functions for vSwitch configuration + + List of supported functions: + + * ``add_switch br_name`` - creates a new switch (bridge) with given ``br_name`` + * ``del_switch br_name`` - deletes switch (bridge) with given ``br_name`` + * ``add_phy_port br_name`` - adds a physical port into bridge specified by ``br_name`` + * ``add_vport br_name`` - adds a virtual port into bridge specified by ``br_name`` + * ``del_port br_name port_name`` - removes physical or virtual port specified by + ``port_name`` from bridge ``br_name`` + * ``add_flow br_name flow`` - adds flow specified by ``flow`` dictionary into + the bridge ``br_name``; Content of flow dictionary will be passed to the vSwitch. + In case of Open vSwitch it will be passed to the ``ovs-ofctl add-flow`` command. + Please see Open vSwitch documentation for the list of supported flow parameters. + * ``del_flow br_name [flow]`` - deletes flow specified by ``flow`` dictionary from + bridge ``br_name``; In case that optional parameter ``flow`` is not specified + or set to an empty dictionary ``{}``, then all flows from bridge ``br_name`` + will be deleted. + * ``dump_flows br_name`` - dumps all flows from bridge specified by ``br_name`` + * ``enable_stp br_name`` - enables Spanning Tree Protocol for bridge ``br_name`` + * ``disable_stp br_name`` - disables Spanning Tree Protocol for bridge ``br_name`` + * ``enable_rstp br_name`` - enables Rapid Spanning Tree Protocol for bridge ``br_name`` + * ``disable_rstp br_name`` - disables Rapid Spanning Tree Protocol for bridge ``br_name`` + + Examples: + + .. code-block:: python + + ['vswitch', 'add_switch', 'int_br0'] + + ['vswitch', 'del_switch', 'int_br0'] + + ['vswitch', 'add_phy_port', 'int_br0'] + + ['vswitch', 'del_port', 'int_br0', '#STEP[2][0]'] + + ['vswitch', 'add_flow', 'int_br0', {'in_port': '1', 'actions': ['output:2'], + 'idle_timeout': '0'}], + + ['vswitch', 'enable_rstp', 'int_br0'] + + * ``vnf[ID]`` - provides functions for deployment and termination of VNFs; Optional + alfanumerical ``ID`` is used for VNF identification in case that testcase + deploys multiple VNFs. + + List of supported functions: + + * ``start`` - starts a VNF based on VSPERF configuration + * ``stop`` - gracefully terminates given VNF + + Examples: + + .. code-block:: python + + ['vnf1', 'start'] + ['vnf2', 'start'] + ['vnf2', 'stop'] + ['vnf1', 'stop'] + + * ``trafficgen`` - triggers traffic generation + + List of supported functions: + + * ``send_traffic traffic`` - starts a traffic based on the vsperf configuration + and given ``traffic`` dictionary. More details about ``traffic`` dictionary + and its possible values are available at :ref:`Traffic Generator Integration Guide + <step-5-supported-traffic-types>` + + Examples: + + .. code-block:: python + + ['trafficgen', 'send_traffic', {'traffic_type' : 'rfc2544_throughput'}] + + ['trafficgen', 'send_traffic', {'traffic_type' : 'rfc2544_back2back', 'bidir' : 'True'}] + + * ``settings`` - reads or modifies VSPERF configuration + + List of supported functions: + + * ``getValue param`` - returns value of given ``param`` + * ``setValue param value`` - sets value of ``param`` to given ``value`` + + Examples: + + .. code-block:: python + + ['settings', 'getValue', 'TOOLS'] + + ['settings', 'setValue', 'GUEST_USERNAME', ['root']] + + * ``namespace`` - creates or modifies network namespaces + + List of supported functions: + + * ``create_namespace name`` - creates new namespace with given ``name`` + * ``delete_namespace name`` - deletes namespace specified by its ``name`` + * ``assign_port_to_namespace port name [port_up]`` - assigns NIC specified by ``port`` + into given namespace ``name``; If optional parameter ``port_up`` is set to ``True``, + then port will be brought up. + * ``add_ip_to_namespace_eth port name addr cidr`` - assigns an IP address ``addr``/``cidr`` + to the NIC specified by ``port`` within namespace ``name`` + * ``reset_port_to_root port name`` - returns given ``port`` from namespace ``name`` back + to the root namespace + + Examples: + + .. code-block:: python + + ['namespace', 'create_namespace', 'testns'] + + ['namespace', 'assign_port_to_namespace', 'eth0', 'testns'] + + * ``veth`` - manipulates with eth and veth devices + + List of supported functions: + + * ``add_veth_port port peer_port`` - adds a pair of veth ports named ``port`` and + ``peer_port`` + * ``del_veth_port port peer_port`` - deletes a veth port pair specified by ``port`` + and ``peer_port`` + * ``bring_up_eth_port eth_port [namespace]`` - brings up ``eth_port`` in (optional) + ``namespace`` + + Examples: + + .. code-block:: python + + ['veth', 'add_veth_port', 'veth', 'veth1'] + + ['veth', 'bring_up_eth_port', 'eth1'] + + * ``tools`` - provides a set of helper functions + + List of supported functions: + + * ``Assert condition`` - evaluates given ``condition`` and raises ``AssertionError`` + in case that condition is not ``True`` + * ``Eval expression`` - evaluates given expression as a python code and returns + its result + * ``Exec command [regex]`` - executes a shell command and filters its output by + (optional) regular expression + + Examples: + + .. code-block:: python + + ['tools', 'exec', 'numactl -H', 'available: ([0-9]+)'] + ['tools', 'assert', '#STEP[-1][0]>1'] + + * ``wait`` - is used for test case interruption. This object doesn't have + any functions. Once reached, vsperf will pause test execution and waits + for press of ``Enter key``. It can be used during testcase design + for debugging purposes. + + Examples: + + .. code-block:: python + + ['wait'] + +Test Macros +----------- + +Test profiles can include macros as part of the test step. Each step in the +profile may return a value such as a port name. Recall macros use #STEP to +indicate the recalled value inside the return structure. If the method the +test step calls returns a value it can be later recalled, for example: + +.. code-block:: python + + { + "Name": "vswitch_add_del_vport", + "Deployment": "clean", + "Description": "vSwitch - add and delete virtual port", + "TestSteps": [ + ['vswitch', 'add_switch', 'int_br0'], # STEP 0 + ['vswitch', 'add_vport', 'int_br0'], # STEP 1 + ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'], # STEP 2 + ['vswitch', 'del_switch', 'int_br0'], # STEP 3 + ] + } + +This test profile uses the vswitch add_vport method which returns a string +value of the port added. This is later called by the del_port method using the +name from step 1. + +It is also possible to use negative indexes in step macros. In that case +``#STEP[-1]`` will refer to the result from previous step, ``#STEP[-2]`` +will refer to result of step called before previous step, etc. It means, +that you could change ``STEP 2`` from previous example to achieve the same +functionality: + +.. code-block:: python + + ['vswitch', 'del_port', 'int_br0', '#STEP[-1][0]'], # STEP 2 + +Also commonly used steps can be created as a separate profile. + +.. code-block:: python + + STEP_VSWITCH_PVP_INIT = [ + ['vswitch', 'add_switch', 'int_br0'], # STEP 0 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 1 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 2 + ['vswitch', 'add_vport', 'int_br0'], # STEP 3 + ['vswitch', 'add_vport', 'int_br0'], # STEP 4 + ] + +This profile can then be used inside other testcases + +.. code-block:: python + + { + "Name": "vswitch_pvp", + "Deployment": "clean", + "Description": "vSwitch - configure switch and one vnf", + "TestSteps": STEP_VSWITCH_PVP_INIT + + [ + ['vnf', 'start'], + ['vnf', 'stop'], + ] + + STEP_VSWITCH_PVP_FINIT + } + +HelloWorld and other basic Testcases +------------------------------------ + +The following examples are for demonstration purposes. +You can run them by copying and pasting into the +conf/integration/01_testcases.conf file. +A command-line instruction is shown at the end of each +example. + +HelloWorld +^^^^^^^^^^ + +The first example is a HelloWorld testcase. +It simply creates a bridge with 2 physical ports, then sets up a flow to drop +incoming packets from the port that was instantiated at the STEP #1. +There's no interaction with the traffic generator. +Then the flow, the 2 ports and the bridge are deleted. +'add_phy_port' method creates a 'dpdk' type interface that will manage the +physical port. The string value returned is the port name that will be referred +by 'del_port' later on. + +.. code-block:: python + + { + "Name": "HelloWorld", + "Description": "My first testcase", + "Deployment": "clean", + "TestSteps": [ + ['vswitch', 'add_switch', 'int_br0'], # STEP 0 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 1 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 2 + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \ + 'actions': ['drop'], 'idle_timeout': '0'}], + ['vswitch', 'del_flow', 'int_br0'], + ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'], + ['vswitch', 'del_port', 'int_br0', '#STEP[2][0]'], + ['vswitch', 'del_switch', 'int_br0'], + ] + + }, + +To run HelloWorld test: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --integration HelloWorld + +Specify a Flow by the IP address +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The next example shows how to explicitly set up a flow by specifying a +destination IP address. +All packets received from the port created at STEP #1 that have a destination +IP address = 90.90.90.90 will be forwarded to the port created at the STEP #2. + +.. code-block:: python + + { + "Name": "p2p_rule_l3da", + "Description": "Phy2Phy with rule on L3 Dest Addr", + "Deployment": "clean", + "biDirectional": "False", + "TestSteps": [ + ['vswitch', 'add_switch', 'int_br0'], # STEP 0 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 1 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 2 + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \ + 'dl_type': '0x0800', 'nw_dst': '90.90.90.90', \ + 'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}], + ['trafficgen', 'send_traffic', \ + {'traffic_type' : 'rfc2544_continuous'}], + ['vswitch', 'dump_flows', 'int_br0'], # STEP 5 + ['vswitch', 'del_flow', 'int_br0'], # STEP 7 == del-flows + ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'], + ['vswitch', 'del_port', 'int_br0', '#STEP[2][0]'], + ['vswitch', 'del_switch', 'int_br0'], + ] + }, + +To run the test: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --integration p2p_rule_l3da + +Multistream feature +^^^^^^^^^^^^^^^^^^^ + +The next testcase uses the multistream feature. +The traffic generator will send packets with different UDP ports. +That is accomplished by using "Stream Type" and "MultiStream" keywords. +4 different flows are set to forward all incoming packets. + +.. code-block:: python + + { + "Name": "multistream_l4", + "Description": "Multistream on UDP ports", + "Deployment": "clean", + "Parameters": { + 'TRAFFIC' : { + "multistream": 4, + "stream_type": "L4", + }, + }, + "TestSteps": [ + ['vswitch', 'add_switch', 'int_br0'], # STEP 0 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 1 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 2 + # Setup Flows + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \ + 'dl_type': '0x0800', 'nw_proto': '17', 'udp_dst': '0', \ + 'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}], + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \ + 'dl_type': '0x0800', 'nw_proto': '17', 'udp_dst': '1', \ + 'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}], + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \ + 'dl_type': '0x0800', 'nw_proto': '17', 'udp_dst': '2', \ + 'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}], + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \ + 'dl_type': '0x0800', 'nw_proto': '17', 'udp_dst': '3', \ + 'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}], + # Send mono-dir traffic + ['trafficgen', 'send_traffic', \ + {'traffic_type' : 'rfc2544_continuous', \ + 'bidir' : 'False'}], + # Clean up + ['vswitch', 'del_flow', 'int_br0'], + ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'], + ['vswitch', 'del_port', 'int_br0', '#STEP[2][0]'], + ['vswitch', 'del_switch', 'int_br0'], + ] + }, + +To run the test: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --integration multistream_l4 + +PVP with a VM Replacement +^^^^^^^^^^^^^^^^^^^^^^^^^ + +This example launches a 1st VM in a PVP topology, then the VM is replaced +by another VM. +When VNF setup parameter in ./conf/04_vnf.conf is "QemuDpdkVhostUser" +'add_vport' method creates a 'dpdkvhostuser' type port to connect a VM. + +.. code-block:: python + + { + "Name": "ex_replace_vm", + "Description": "PVP with VM replacement", + "Deployment": "clean", + "TestSteps": [ + ['vswitch', 'add_switch', 'int_br0'], # STEP 0 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 1 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 2 + ['vswitch', 'add_vport', 'int_br0'], # STEP 3 vm1 + ['vswitch', 'add_vport', 'int_br0'], # STEP 4 + + # Setup Flows + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \ + 'actions': ['output:#STEP[3][1]'], 'idle_timeout': '0'}], + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[4][1]', \ + 'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}], + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[2][1]', \ + 'actions': ['output:#STEP[4][1]'], 'idle_timeout': '0'}], + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[3][1]', \ + 'actions': ['output:#STEP[1][1]'], 'idle_timeout': '0'}], + + # Start VM 1 + ['vnf1', 'start'], + # Now we want to replace VM 1 with another VM + ['vnf1', 'stop'], + + ['vswitch', 'add_vport', 'int_br0'], # STEP 11 vm2 + ['vswitch', 'add_vport', 'int_br0'], # STEP 12 + ['vswitch', 'del_flow', 'int_br0'], + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \ + 'actions': ['output:#STEP[11][1]'], 'idle_timeout': '0'}], + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[12][1]', \ + 'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}], + + # Start VM 2 + ['vnf2', 'start'], + ['vnf2', 'stop'], + ['vswitch', 'dump_flows', 'int_br0'], + + # Clean up + ['vswitch', 'del_flow', 'int_br0'], + ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'], + ['vswitch', 'del_port', 'int_br0', '#STEP[2][0]'], + ['vswitch', 'del_port', 'int_br0', '#STEP[3][0]'], # vm1 + ['vswitch', 'del_port', 'int_br0', '#STEP[4][0]'], + ['vswitch', 'del_port', 'int_br0', '#STEP[11][0]'], # vm2 + ['vswitch', 'del_port', 'int_br0', '#STEP[12][0]'], + ['vswitch', 'del_switch', 'int_br0'], + ] + }, + +To run the test: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --integration ex_replace_vm + +VM with a Linux bridge +^^^^^^^^^^^^^^^^^^^^^^ + +This example setups a PVP topology and routes traffic to the VM based on +the destination IP address. A command-line parameter is used to select a Linux +bridge as a guest loopback application. It is also possible to select a guest +loopback application by a configuration option ``GUEST_LOOPBACK``. + +.. code-block:: python + + { + "Name": "ex_pvp_rule_l3da", + "Description": "PVP with flow on L3 Dest Addr", + "Deployment": "clean", + "TestSteps": [ + ['vswitch', 'add_switch', 'int_br0'], # STEP 0 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 1 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 2 + ['vswitch', 'add_vport', 'int_br0'], # STEP 3 vm1 + ['vswitch', 'add_vport', 'int_br0'], # STEP 4 + # Setup Flows + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \ + 'dl_type': '0x0800', 'nw_dst': '90.90.90.90', \ + 'actions': ['output:#STEP[3][1]'], 'idle_timeout': '0'}], + # Each pkt from the VM is forwarded to the 2nd dpdk port + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[4][1]', \ + 'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}], + # Start VMs + ['vnf1', 'start'], + ['trafficgen', 'send_traffic', \ + {'traffic_type' : 'rfc2544_continuous', \ + 'bidir' : 'False'}], + ['vnf1', 'stop'], + # Clean up + ['vswitch', 'dump_flows', 'int_br0'], # STEP 10 + ['vswitch', 'del_flow', 'int_br0'], # STEP 11 + ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'], + ['vswitch', 'del_port', 'int_br0', '#STEP[2][0]'], + ['vswitch', 'del_port', 'int_br0', '#STEP[3][0]'], # vm1 ports + ['vswitch', 'del_port', 'int_br0', '#STEP[4][0]'], + ['vswitch', 'del_switch', 'int_br0'], + ] + }, + +To run the test: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --test-params \ + "GUEST_LOOPBACK=['linux_bridge']" --integration ex_pvp_rule_l3da + +Forward packets based on UDP port +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This examples launches 2 VMs connected in parallel. +Incoming packets will be forwarded to one specific VM depending on the +destination UDP port. + +.. code-block:: python + + { + "Name": "ex_2pvp_rule_l4dp", + "Description": "2 PVP with flows on L4 Dest Port", + "Deployment": "clean", + "Parameters": { + 'TRAFFIC' : { + "multistream": 2, + "stream_type": "L4", + }, + }, + "TestSteps": [ + ['vswitch', 'add_switch', 'int_br0'], # STEP 0 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 1 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 2 + ['vswitch', 'add_vport', 'int_br0'], # STEP 3 vm1 + ['vswitch', 'add_vport', 'int_br0'], # STEP 4 + ['vswitch', 'add_vport', 'int_br0'], # STEP 5 vm2 + ['vswitch', 'add_vport', 'int_br0'], # STEP 6 + # Setup Flows to reply ICMPv6 and similar packets, so to + # avoid flooding internal port with their re-transmissions + ['vswitch', 'add_flow', 'int_br0', \ + {'priority': '1', 'dl_src': '00:00:00:00:00:01', \ + 'actions': ['output:#STEP[3][1]'], 'idle_timeout': '0'}], + ['vswitch', 'add_flow', 'int_br0', \ + {'priority': '1', 'dl_src': '00:00:00:00:00:02', \ + 'actions': ['output:#STEP[4][1]'], 'idle_timeout': '0'}], + ['vswitch', 'add_flow', 'int_br0', \ + {'priority': '1', 'dl_src': '00:00:00:00:00:03', \ + 'actions': ['output:#STEP[5][1]'], 'idle_timeout': '0'}], + ['vswitch', 'add_flow', 'int_br0', \ + {'priority': '1', 'dl_src': '00:00:00:00:00:04', \ + 'actions': ['output:#STEP[6][1]'], 'idle_timeout': '0'}], + # Forward UDP packets depending on dest port + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \ + 'dl_type': '0x0800', 'nw_proto': '17', 'udp_dst': '0', \ + 'actions': ['output:#STEP[3][1]'], 'idle_timeout': '0'}], + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[1][1]', \ + 'dl_type': '0x0800', 'nw_proto': '17', 'udp_dst': '1', \ + 'actions': ['output:#STEP[5][1]'], 'idle_timeout': '0'}], + # Send VM output to phy port #2 + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[4][1]', \ + 'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}], + ['vswitch', 'add_flow', 'int_br0', {'in_port': '#STEP[6][1]', \ + 'actions': ['output:#STEP[2][1]'], 'idle_timeout': '0'}], + # Start VMs + ['vnf1', 'start'], # STEP 16 + ['vnf2', 'start'], # STEP 17 + ['trafficgen', 'send_traffic', \ + {'traffic_type' : 'rfc2544_continuous', \ + 'bidir' : 'False'}], + ['vnf1', 'stop'], + ['vnf2', 'stop'], + ['vswitch', 'dump_flows', 'int_br0'], + # Clean up + ['vswitch', 'del_flow', 'int_br0'], + ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'], + ['vswitch', 'del_port', 'int_br0', '#STEP[2][0]'], + ['vswitch', 'del_port', 'int_br0', '#STEP[3][0]'], # vm1 ports + ['vswitch', 'del_port', 'int_br0', '#STEP[4][0]'], + ['vswitch', 'del_port', 'int_br0', '#STEP[5][0]'], # vm2 ports + ['vswitch', 'del_port', 'int_br0', '#STEP[6][0]'], + ['vswitch', 'del_switch', 'int_br0'], + ] + }, + +To run the test: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py --integration ex_2pvp_rule_l4dp + +Modification of existing PVVP deployment +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This is an example of modification of a standard deployment scenario with additional TestSteps. +Standard PVVP scenario is used to configure a vSwitch and to deploy two VNFs connected +in series. Additional TestSteps will deploy a 3rd VNF and connect it in parallel to +already configured VNFs. Traffic generator is instructed (by Multistream feature) to send +two separate traffic streams. One stream will be sent to the standalone VNF and second +to two chained VNFs. + +In case, that test is defined as a performance test, then traffic results will be collected +and available in both csv and rst report files. + +.. code-block:: python + + { + "Name": "pvvp_pvp_cont", + "Deployment": "pvvp", + "Description": "PVVP and PVP in parallel with Continuous Stream", + "Parameters" : { + "TRAFFIC" : { + "traffic_type" : "rfc2544_continuous", + "multistream": 2, + }, + }, + "TestSteps": [ + ['vswitch', 'add_vport', 'br0'], + ['vswitch', 'add_vport', 'br0'], + # priority must be higher than default 32768, otherwise flows won't match + ['vswitch', 'add_flow', 'br0', + {'in_port': '1', 'actions': ['output:#STEP[-2][1]'], 'idle_timeout': '0', 'dl_type':'0x0800', + 'nw_proto':'17', 'tp_dst':'0', 'priority': '33000'}], + ['vswitch', 'add_flow', 'br0', + {'in_port': '2', 'actions': ['output:#STEP[-2][1]'], 'idle_timeout': '0', 'dl_type':'0x0800', + 'nw_proto':'17', 'tp_dst':'0', 'priority': '33000'}], + ['vswitch', 'add_flow', 'br0', {'in_port': '#STEP[-4][1]', 'actions': ['output:1'], + 'idle_timeout': '0'}], + ['vswitch', 'add_flow', 'br0', {'in_port': '#STEP[-4][1]', 'actions': ['output:2'], + 'idle_timeout': '0'}], + ['vswitch', 'dump_flows', 'br0'], + ['vnf1', 'start'], + ] + }, + +To run the test: + + .. code-block:: console + + ./vsperf --conf-file user_settings.py pvvp_pvp_cont + diff --git a/docs/testing/user/userguide/testusage.rst b/docs/testing/user/userguide/testusage.rst new file mode 100644 index 00000000..c6037aaf --- /dev/null +++ b/docs/testing/user/userguide/testusage.rst @@ -0,0 +1,848 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intel Corporation, AT&T and others. + +vSwitchPerf test suites userguide +--------------------------------- + +General +^^^^^^^ + +VSPERF requires a traffic generators to run tests, automated traffic gen +support in VSPERF includes: + +- IXIA traffic generator (IxNetwork hardware) and a machine that runs the IXIA + client software. +- Spirent traffic generator (TestCenter hardware chassis or TestCenter virtual + in a VM) and a VM to run the Spirent Virtual Deployment Service image, + formerly known as "Spirent LabServer". +- Xena Network traffic generator (Xena hardware chassis) that houses the Xena + Traffic generator modules. +- Moongen software traffic generator. Requires a separate machine running + moongen to execute packet generation. + +If you want to use another traffic generator, please select the :ref:`trafficgen-dummy` +generator. + +VSPERF Installation +^^^^^^^^^^^^^^^^^^^ + +To see the supported Operating Systems, vSwitches and system requirements, +please follow the `installation instructions <vsperf-installation>`. + +Traffic Generator Setup +^^^^^^^^^^^^^^^^^^^^^^^ + +Follow the `Traffic generator instructions <trafficgen-installation>` to +install and configure a suitable traffic generator. + +Cloning and building src dependencies +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In order to run VSPERF, you will need to download DPDK and OVS. You can +do this manually and build them in a preferred location, OR you could +use vswitchperf/src. The vswitchperf/src directory contains makefiles +that will allow you to clone and build the libraries that VSPERF depends +on, such as DPDK and OVS. To clone and build simply: + +.. code-block:: console + + $ cd src + $ make + +VSPERF can be used with stock OVS (without DPDK support). When build +is finished, the libraries are stored in src_vanilla directory. + +The 'make' builds all options in src: + +* Vanilla OVS +* OVS with vhost_user as the guest access method (with DPDK support) + +The vhost_user build will reside in src/ovs/ +The Vanilla OVS build will reside in vswitchperf/src_vanilla + +To delete a src subdirectory and its contents to allow you to re-clone simply +use: + +.. code-block:: console + + $ make clobber + +Configure the ``./conf/10_custom.conf`` file +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The ``10_custom.conf`` file is the configuration file that overrides +default configurations in all the other configuration files in ``./conf`` +The supplied ``10_custom.conf`` file **MUST** be modified, as it contains +configuration items for which there are no reasonable default values. + +The configuration items that can be added is not limited to the initial +contents. Any configuration item mentioned in any .conf file in +``./conf`` directory can be added and that item will be overridden by +the custom configuration value. + +Further details about configuration files evaluation and special behaviour +of options with ``GUEST_`` prefix could be found at :ref:`design document +<design-configuration>`. + +Using a custom settings file +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If your ``10_custom.conf`` doesn't reside in the ``./conf`` directory +of if you want to use an alternative configuration file, the file can +be passed to ``vsperf`` via the ``--conf-file`` argument. + +.. code-block:: console + + $ ./vsperf --conf-file <path_to_custom_conf> ... + +Note that configuration passed in via the environment (``--load-env``) +or via another command line argument will override both the default and +your custom configuration files. This "priority hierarchy" can be +described like so (1 = max priority): + +1. Testcase definition section ``Parameters`` +2. Command line arguments +3. Environment variables +4. Configuration file(s) + +Further details about configuration files evaluation and special behaviour +of options with ``GUEST_`` prefix could be found at :ref:`design document +<design-configuration>`. + +.. _overriding-parameters-documentation: + +Overriding values defined in configuration files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The configuration items can be overridden by command line argument +``--test-params``. In this case, the configuration items and +their values should be passed in form of ``item=value`` and separated +by semicolon. + +Example: + +.. code:: console + + $ ./vsperf --test-params "TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,);" \ + "GUEST_LOOPBACK=['testpmd','l2fwd']" pvvp_tput + +The second option is to override configuration items by ``Parameters`` section +of the test case definition. The configuration items can be added into ``Parameters`` +dictionary with their new values. These values will override values defined in +configuration files or specified by ``--test-params`` command line argument. + +Example: + +.. code:: python + + "Parameters" : {'TRAFFICGEN_PKT_SIZES' : (128,), + 'TRAFFICGEN_DURATION' : 10, + 'GUEST_LOOPBACK' : ['testpmd','l2fwd'], + } + +**NOTE:** In both cases, configuration item names and their values must be specified +in the same form as they are defined inside configuration files. Parameter names +must be specified in uppercase and data types of original and new value must match. +Python syntax rules related to data types and structures must be followed. +For example, parameter ``TRAFFICGEN_PKT_SIZES`` above is defined as a tuple +with a single value ``128``. In this case trailing comma is mandatory, otherwise +value can be wrongly interpreted as a number instead of a tuple and vsperf +execution would fail. Please check configuration files for default values and their +types and use them as a basis for any customized values. In case of any doubt, please +check official python documentation related to data structures like tuples, lists +and dictionaries. + +**NOTE:** Vsperf execution will terminate with runtime error in case, that unknown +parameter name is passed via ``--test-params`` CLI argument or defined in ``Parameters`` +section of test case definition. It is also forbidden to redefine a value of +``TEST_PARAMS`` configuration item via CLI or ``Parameters`` section. + +vloop_vnf +^^^^^^^^^ + +VSPERF uses a VM image called vloop_vnf for looping traffic in the deployment +scenarios involving VMs. The image can be downloaded from +`<http://artifacts.opnfv.org/>`__. + +Please see the installation instructions for information on :ref:`vloop-vnf` +images. + +.. _l2fwd-module: + +l2fwd Kernel Module +^^^^^^^^^^^^^^^^^^^ + +A Kernel Module that provides OSI Layer 2 Ipv4 termination or forwarding with +support for Destination Network Address Translation (DNAT) for both the MAC and +IP addresses. l2fwd can be found in <vswitchperf_dir>/src/l2fwd + +Executing tests +^^^^^^^^^^^^^^^ + +All examples inside these docs assume, that user is inside the VSPERF +directory. VSPERF can be executed from any directory. + +Before running any tests make sure you have root permissions by adding +the following line to /etc/sudoers: + +.. code-block:: console + + username ALL=(ALL) NOPASSWD: ALL + +username in the example above should be replaced with a real username. + +To list the available tests: + +.. code-block:: console + + $ ./vsperf --list + +To run a single test: + +.. code-block:: console + + $ ./vsperf $TESTNAME + +Where $TESTNAME is the name of the vsperf test you would like to run. + +To run a group of tests, for example all tests with a name containing +'RFC2544': + +.. code-block:: console + + $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf --tests="RFC2544" + +To run all tests: + +.. code-block:: console + + $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf + +Some tests allow for configurable parameters, including test duration +(in seconds) as well as packet sizes (in bytes). + +.. code:: bash + + $ ./vsperf --conf-file user_settings.py \ + --tests RFC2544Tput \ + --test-params "TRAFFICGEN_DURATION=10;TRAFFICGEN_PKT_SIZES=(128,)" + +For all available options, check out the help dialog: + +.. code-block:: console + + $ ./vsperf --help + +Executing Vanilla OVS tests +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +1. If needed, recompile src for all OVS variants + + .. code-block:: console + + $ cd src + $ make distclean + $ make + +2. Update your ``10_custom.conf`` file to use Vanilla OVS: + + .. code-block:: python + + VSWITCH = 'OvsVanilla' + +3. Run test: + + .. code-block:: console + + $ ./vsperf --conf-file=<path_to_custom_conf> + + Please note if you don't want to configure Vanilla OVS through the + configuration file, you can pass it as a CLI argument. + + .. code-block:: console + + $ ./vsperf --vswitch OvsVanilla + + +Executing tests with VMs +^^^^^^^^^^^^^^^^^^^^^^^^ + +To run tests using vhost-user as guest access method: + +1. Set VHOST_METHOD and VNF of your settings file to: + + .. code-block:: python + + VSWITCH = 'OvsDpdkVhost' + VNF = 'QemuDpdkVhost' + +2. If needed, recompile src for all OVS variants + + .. code-block:: console + + $ cd src + $ make distclean + $ make + +3. Run test: + + .. code-block:: console + + $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf + +Executing tests with VMs using Vanilla OVS +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To run tests using Vanilla OVS: + +1. Set the following variables: + + .. code-block:: python + + VSWITCH = 'OvsVanilla' + VNF = 'QemuVirtioNet' + + VANILLA_TGEN_PORT1_IP = n.n.n.n + VANILLA_TGEN_PORT1_MAC = nn:nn:nn:nn:nn:nn + + VANILLA_TGEN_PORT2_IP = n.n.n.n + VANILLA_TGEN_PORT2_MAC = nn:nn:nn:nn:nn:nn + + VANILLA_BRIDGE_IP = n.n.n.n + + or use ``--test-params`` option + + .. code-block:: console + + $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf \ + --test-params "VANILLA_TGEN_PORT1_IP=n.n.n.n;" \ + "VANILLA_TGEN_PORT1_MAC=nn:nn:nn:nn:nn:nn;" \ + "VANILLA_TGEN_PORT2_IP=n.n.n.n;" \ + "VANILLA_TGEN_PORT2_MAC=nn:nn:nn:nn:nn:nn" + +2. If needed, recompile src for all OVS variants + + .. code-block:: console + + $ cd src + $ make distclean + $ make + +3. Run test: + + .. code-block:: console + + $ ./vsperf --conf-file<path_to_custom_conf>/10_custom.conf + +.. _vpp-test: + +Executing VPP tests +^^^^^^^^^^^^^^^^^^^ + +Currently it is not possible to use standard scenario deployments for execution of +tests with VPP. It means, that deployments ``p2p``, ``pvp``, ``pvvp`` and in general any +:ref:`pxp-deployment` won't work with VPP. However it is possible to use VPP in +:ref:`step-driven-tests`. A basic set of VPP testcases covering ``phy2phy``, ``pvp`` +and ``pvvp`` tests are already prepared. + +List of performance tests with VPP support follows: + +* phy2phy_tput_vpp: VPP: LTD.Throughput.RFC2544.PacketLossRatio +* phy2phy_cont_vpp: VPP: Phy2Phy Continuous Stream +* phy2phy_back2back_vpp: VPP: LTD.Throughput.RFC2544.BackToBackFrames +* pvp_tput_vpp: VPP: LTD.Throughput.RFC2544.PacketLossRatio +* pvp_cont_vpp: VPP: PVP Continuous Stream +* pvp_back2back_vpp: VPP: LTD.Throughput.RFC2544.BackToBackFrames +* pvvp_tput_vpp: VPP: LTD.Throughput.RFC2544.PacketLossRatio +* pvvp_cont_vpp: VPP: PVP Continuous Stream +* pvvp_back2back_vpp: VPP: LTD.Throughput.RFC2544.BackToBackFrames + +In order to execute testcases with VPP it is required to: + +* install VPP manually, see :ref:`vpp-installation` +* configure ``WHITELIST_NICS``, with two physical NICs connected to the traffic generator +* configure traffic generator, see :ref:`trafficgen-installation` + +After that it is possible to execute VPP testcases listed above. + +For example: + +.. code-block:: console + + $ ./vsperf --conf-file=<path_to_custom_conf> phy2phy_tput_vpp + +.. _vfio-pci: + +Using vfio_pci with DPDK +^^^^^^^^^^^^^^^^^^^^^^^^^ + +To use vfio with DPDK instead of igb_uio add into your custom configuration +file the following parameter: + +.. code-block:: python + + PATHS['dpdk']['src']['modules'] = ['uio', 'vfio-pci'] + + +**NOTE:** In case, that DPDK is installed from binary package, then please +set ``PATHS['dpdk']['bin']['modules']`` instead. + +**NOTE:** Please ensure that Intel VT-d is enabled in BIOS. + +**NOTE:** Please ensure your boot/grub parameters include +the following: + +.. code-block:: console + + iommu=pt intel_iommu=on + +To check that IOMMU is enabled on your platform: + +.. code-block:: console + + $ dmesg | grep IOMMU + [ 0.000000] Intel-IOMMU: enabled + [ 0.139882] dmar: IOMMU 0: reg_base_addr fbffe000 ver 1:0 cap d2078c106f0466 ecap f020de + [ 0.139888] dmar: IOMMU 1: reg_base_addr ebffc000 ver 1:0 cap d2078c106f0466 ecap f020de + [ 0.139893] IOAPIC id 2 under DRHD base 0xfbffe000 IOMMU 0 + [ 0.139894] IOAPIC id 0 under DRHD base 0xebffc000 IOMMU 1 + [ 0.139895] IOAPIC id 1 under DRHD base 0xebffc000 IOMMU 1 + [ 3.335744] IOMMU: dmar0 using Queued invalidation + [ 3.335746] IOMMU: dmar1 using Queued invalidation + .... + +.. _SRIOV-support: + +Using SRIOV support +^^^^^^^^^^^^^^^^^^^ + +To use virtual functions of NIC with SRIOV support, use extended form +of NIC PCI slot definition: + +.. code-block:: python + + WHITELIST_NICS = ['0000:05:00.0|vf0', '0000:05:00.1|vf3'] + +Where 'vf' is an indication of virtual function usage and following +number defines a VF to be used. In case that VF usage is detected, +then vswitchperf will enable SRIOV support for given card and it will +detect PCI slot numbers of selected VFs. + +So in example above, one VF will be configured for NIC '0000:05:00.0' +and four VFs will be configured for NIC '0000:05:00.1'. Vswitchperf +will detect PCI addresses of selected VFs and it will use them during +test execution. + +At the end of vswitchperf execution, SRIOV support will be disabled. + +SRIOV support is generic and it can be used in different testing scenarios. +For example: + +* vSwitch tests with DPDK or without DPDK support to verify impact + of VF usage on vSwitch performance +* tests without vSwitch, where traffic is forwared directly + between VF interfaces by packet forwarder (e.g. testpmd application) +* tests without vSwitch, where VM accesses VF interfaces directly + by PCI-passthrough_ to measure raw VM throughput performance. + +.. _PCI-passthrough: + +Using QEMU with PCI passthrough support +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Raw virtual machine throughput performance can be measured by execution of PVP +test with direct access to NICs by PCI passthrough. To execute VM with direct +access to PCI devices, enable vfio-pci_. In order to use virtual functions, +SRIOV-support_ must be enabled. + +Execution of test with PCI passthrough with vswitch disabled: + +.. code-block:: console + + $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf \ + --vswitch none --vnf QemuPciPassthrough pvp_tput + +Any of supported guest-loopback-application_ can be used inside VM with +PCI passthrough support. + +Note: Qemu with PCI passthrough support can be used only with PVP test +deployment. + +.. _guest-loopback-application: + +Selection of loopback application for tests with VMs +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To select the loopback applications which will forward packets inside VMs, +the following parameter should be configured: + +.. code-block:: python + + GUEST_LOOPBACK = ['testpmd'] + +or use ``--test-params`` CLI argument: + +.. code-block:: console + + $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf \ + --test-params "GUEST_LOOPBACK=['testpmd']" + +Supported loopback applications are: + +.. code-block:: console + + 'testpmd' - testpmd from dpdk will be built and used + 'l2fwd' - l2fwd module provided by Huawei will be built and used + 'linux_bridge' - linux bridge will be configured + 'buildin' - nothing will be configured by vsperf; VM image must + ensure traffic forwarding between its interfaces + +Guest loopback application must be configured, otherwise traffic +will not be forwarded by VM and testcases with VM related deployments +will fail. Guest loopback application is set to 'testpmd' by default. + +**NOTE:** In case that only 1 or more than 2 NICs are configured for VM, +then 'testpmd' should be used. As it is able to forward traffic between +multiple VM NIC pairs. + +**NOTE:** In case of linux_bridge, all guest NICs are connected to the same +bridge inside the guest. + +Mergable Buffers Options with QEMU +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Mergable buffers can be disabled with VSPerf within QEMU. This option can +increase performance significantly when not using jumbo frame sized packets. +By default VSPerf disables mergable buffers. If you wish to enable it you +can modify the setting in the a custom conf file. + +.. code-block:: python + + GUEST_NIC_MERGE_BUFFERS_DISABLE = [False] + +Then execute using the custom conf file. + +.. code-block:: console + + $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf + +Alternatively you can just pass the param during execution. + +.. code-block:: console + + $ ./vsperf --test-params "GUEST_NIC_MERGE_BUFFERS_DISABLE=[False]" + + +Selection of dpdk binding driver for tests with VMs +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To select dpdk binding driver, which will specify which driver the vm NICs will +use for dpdk bind, the following configuration parameter should be configured: + +.. code-block:: console + + GUEST_DPDK_BIND_DRIVER = ['igb_uio_from_src'] + +The supported dpdk guest bind drivers are: + +.. code-block:: console + + 'uio_pci_generic' - Use uio_pci_generic driver + 'igb_uio_from_src' - Build and use the igb_uio driver from the dpdk src + files + 'vfio_no_iommu' - Use vfio with no iommu option. This requires custom + guest images that support this option. The default + vloop image does not support this driver. + +Note: uio_pci_generic does not support sr-iov testcases with guests attached. +This is because uio_pci_generic only supports legacy interrupts. In case +uio_pci_generic is selected with the vnf as QemuPciPassthrough it will be +modified to use igb_uio_from_src instead. + +Note: vfio_no_iommu requires kernels equal to or greater than 4.5 and dpdk +16.04 or greater. Using this option will also taint the kernel. + +Please refer to the dpdk documents at http://dpdk.org/doc/guides for more +information on these drivers. + +Multi-Queue Configuration +^^^^^^^^^^^^^^^^^^^^^^^^^ + +VSPerf currently supports multi-queue with the following limitations: + +1. Requires QEMU 2.5 or greater and any OVS version higher than 2.5. The + default upstream package versions installed by VSPerf satisfies this + requirement. + +2. Guest image must have ethtool utility installed if using l2fwd or linux + bridge inside guest for loopback. + +3. If using OVS versions 2.5.0 or less enable old style multi-queue as shown + in the ''02_vswitch.conf'' file. + + .. code-block:: python + + OVS_OLD_STYLE_MQ = True + +To enable multi-queue for dpdk modify the ''02_vswitch.conf'' file. + +.. code-block:: python + + VSWITCH_DPDK_MULTI_QUEUES = 2 + +**NOTE:** you should consider using the switch affinity to set a pmd cpu mask +that can optimize your performance. Consider the numa of the NIC in use if this +applies by checking /sys/class/net/<eth_name>/device/numa_node and setting an +appropriate mask to create PMD threads on the same numa node. + +When multi-queue is enabled, each dpdk or dpdkvhostuser port that is created +on the switch will set the option for multiple queues. If old style multi queue +has been enabled a global option for multi queue will be used instead of the +port by port option. + +To enable multi-queue on the guest modify the ''04_vnf.conf'' file. + +.. code-block:: python + + GUEST_NIC_QUEUES = [2] + +Enabling multi-queue at the guest will add multiple queues to each NIC port when +qemu launches the guest. + +In case of Vanilla OVS, multi-queue is enabled on the tuntap ports and nic +queues will be enabled inside the guest with ethtool. Simply enabling the +multi-queue on the guest is sufficient for Vanilla OVS multi-queue. + +Testpmd should be configured to take advantage of multi-queue on the guest if +using DPDKVhostUser. This can be done by modifying the ''04_vnf.conf'' file. + +.. code-block:: python + + GUEST_TESTPMD_PARAMS = ['-l 0,1,2,3,4 -n 4 --socket-mem 512 -- ' + '--burst=64 -i --txqflags=0xf00 ' + '--nb-cores=4 --rxq=2 --txq=2 ' + '--disable-hw-vlan'] + +**NOTE:** The guest SMP cores must be configured to allow for testpmd to use the +optimal number of cores to take advantage of the multiple guest queues. + +In case of using Vanilla OVS and qemu virtio-net you can increase performance +by binding vhost-net threads to cpus. This can be done by enabling the affinity +in the ''04_vnf.conf'' file. This can be done to non multi-queue enabled +configurations as well as there will be 2 vhost-net threads. + +.. code-block:: python + + VSWITCH_VHOST_NET_AFFINITIZATION = True + + VSWITCH_VHOST_CPU_MAP = [4,5,8,11] + +**NOTE:** This method of binding would require a custom script in a real +environment. + +**NOTE:** For optimal performance guest SMPs and/or vhost-net threads should be +on the same numa as the NIC in use if possible/applicable. Testpmd should be +assigned at least (nb_cores +1) total cores with the cpu mask. + +Executing Packet Forwarding tests +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +To select the applications which will forward packets, +the following parameters should be configured: + +.. code-block:: python + + VSWITCH = 'none' + PKTFWD = 'TestPMD' + +or use ``--vswitch`` and ``--fwdapp`` CLI arguments: + +.. code-block:: console + + $ ./vsperf phy2phy_cont --conf-file user_settings.py \ + --vswitch none \ + --fwdapp TestPMD + +Supported Packet Forwarding applications are: + +.. code-block:: console + + 'testpmd' - testpmd from dpdk + + +1. Update your ''10_custom.conf'' file to use the appropriate variables + for selected Packet Forwarder: + + .. code-block:: python + + # testpmd configuration + TESTPMD_ARGS = [] + # packet forwarding mode supported by testpmd; Please see DPDK documentation + # for comprehensive list of modes supported by your version. + # e.g. io|mac|mac_retry|macswap|flowgen|rxonly|txonly|csum|icmpecho|... + # Note: Option "mac_retry" has been changed to "mac retry" since DPDK v16.07 + TESTPMD_FWD_MODE = 'csum' + # checksum calculation layer: ip|udp|tcp|sctp|outer-ip + TESTPMD_CSUM_LAYER = 'ip' + # checksum calculation place: hw (hardware) | sw (software) + TESTPMD_CSUM_CALC = 'sw' + # recognize tunnel headers: on|off + TESTPMD_CSUM_PARSE_TUNNEL = 'off' + +2. Run test: + + .. code-block:: console + + $ ./vsperf phy2phy_tput --conf-file <path_to_settings_py> + +Executing Packet Forwarding tests with one guest +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +TestPMD with DPDK 16.11 or greater can be used to forward packets as a switch to a single guest using TestPMD vdev +option. To set this configuration the following parameters should be used. + + .. code-block:: python + + VSWITCH = 'none' + PKTFWD = 'TestPMD' + +or use ``--vswitch`` and ``--fwdapp`` CLI arguments: + + .. code-block:: console + + $ ./vsperf pvp_tput --conf-file user_settings.py \ + --vswitch none \ + --fwdapp TestPMD + +Guest forwarding application only supports TestPMD in this configuration. + + .. code-block:: python + + GUEST_LOOPBACK = ['testpmd'] + +For optimal performance one cpu per port +1 should be used for TestPMD. Also set additional params for packet forwarding +application to use the correct number of nb-cores. + + .. code-block:: python + + DPDK_SOCKET_MEM = ['1024', '0'] + VSWITCHD_DPDK_ARGS = ['-l', '46,44,42,40,38', '-n', '4'] + TESTPMD_ARGS = ['--nb-cores=4', '--txq=1', '--rxq=1'] + +For guest TestPMD 3 VCpus should be assigned with the following TestPMD params. + + .. code-block:: python + + GUEST_TESTPMD_PARAMS = ['-l 0,1,2 -n 4 --socket-mem 1024 -- ' + '--burst=64 -i --txqflags=0xf00 ' + '--disable-hw-vlan --nb-cores=2 --txq=1 --rxq=1'] + +Execution of TestPMD can be run with the following command line + + .. code-block:: console + + ./vsperf pvp_tput --vswitch=none --fwdapp=TestPMD --conf-file <path_to_settings_py> + +**NOTE:** To achieve the best 0% loss numbers with rfc2544 throughput testing, other tunings should be applied to host +and guest such as tuned profiles and CPU tunings to prevent possible interrupts to worker threads. + +VSPERF modes of operation +^^^^^^^^^^^^^^^^^^^^^^^^^ + +VSPERF can be run in different modes. By default it will configure vSwitch, +traffic generator and VNF. However it can be used just for configuration +and execution of traffic generator. Another option is execution of all +components except traffic generator itself. + +Mode of operation is driven by configuration parameter -m or --mode + +.. code-block:: console + + -m MODE, --mode MODE vsperf mode of operation; + Values: + "normal" - execute vSwitch, VNF and traffic generator + "trafficgen" - execute only traffic generator + "trafficgen-off" - execute vSwitch and VNF + "trafficgen-pause" - execute vSwitch and VNF but wait before traffic transmission + +In case, that VSPERF is executed in "trafficgen" mode, then configuration +of traffic generator can be modified through ``TRAFFIC`` dictionary passed to the +``--test-params`` option. It is not needed to specify all values of ``TRAFFIC`` +dictionary. It is sufficient to specify only values, which should be changed. +Detailed description of ``TRAFFIC`` dictionary can be found at +:ref:`configuration-of-traffic-dictionary`. + +Example of execution of VSPERF in "trafficgen" mode: + +.. code-block:: console + + $ ./vsperf -m trafficgen --trafficgen IxNet --conf-file vsperf.conf \ + --test-params "TRAFFIC={'traffic_type':'rfc2544_continuous','bidir':'False','framerate':60}" + +Code change verification by pylint +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Every developer participating in VSPERF project should run +pylint before his python code is submitted for review. Project +specific configuration for pylint is available at 'pylint.rc'. + +Example of manual pylint invocation: + +.. code-block:: console + + $ pylint --rcfile ./pylintrc ./vsperf + +GOTCHAs: +^^^^^^^^ + +Custom image fails to boot +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Using custom VM images may not boot within VSPerf pxp testing because of +the drive boot and shared type which could be caused by a missing scsi +driver inside the image. In case of issues you can try changing the drive +boot type to ide. + +.. code-block:: python + + GUEST_BOOT_DRIVE_TYPE = ['ide'] + GUEST_SHARED_DRIVE_TYPE = ['ide'] + +OVS with DPDK and QEMU +~~~~~~~~~~~~~~~~~~~~~~~ + +If you encounter the following error: "before (last 100 chars): +'-path=/dev/hugepages,share=on: unable to map backing store for +hugepages: Cannot allocate memory\r\n\r\n" during qemu initialization, +check the amount of hugepages on your system: + +.. code-block:: console + + $ cat /proc/meminfo | grep HugePages + + +By default the vswitchd is launched with 1Gb of memory, to change +this, modify --socket-mem parameter in conf/02_vswitch.conf to allocate +an appropriate amount of memory: + +.. code-block:: python + + DPDK_SOCKET_MEM = ['1024', '0'] + VSWITCHD_DPDK_ARGS = ['-c', '0x4', '-n', '4'] + VSWITCHD_DPDK_CONFIG = { + 'dpdk-init' : 'true', + 'dpdk-lcore-mask' : '0x4', + 'dpdk-socket-mem' : '1024,0', + } + +Note: Option ``VSWITCHD_DPDK_ARGS`` is used for vswitchd, which supports ``--dpdk`` +parameter. In recent vswitchd versions, option ``VSWITCHD_DPDK_CONFIG`` will be +used to configure vswitchd via ``ovs-vsctl`` calls. + + +More information +^^^^^^^^^^^^^^^^ + +For more information and details refer to the rest of vSwitchPerfuser documentation. + diff --git a/docs/testing/user/userguide/yardstick.rst b/docs/testing/user/userguide/yardstick.rst new file mode 100644 index 00000000..b5e5c72d --- /dev/null +++ b/docs/testing/user/userguide/yardstick.rst @@ -0,0 +1,250 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intel Corporation, AT&T and others. + +Execution of vswitchperf testcases by Yardstick +----------------------------------------------- + +General +^^^^^^^ + +Yardstick is a generic framework for a test execution, which is used for +validation of installation of OPNFV platform. In the future, Yardstick will +support two options of vswitchperf testcase execution: + +- plugin mode, which will execute native vswitchperf testcases; Tests will + be executed natively by vsperf, and test results will be processed and + reported by yardstick. +- traffic generator mode, which will run vswitchperf in **trafficgen** + mode only; Yardstick framework will be used to launch VNFs and to configure + flows to ensure, that traffic is properly routed. This mode will allow to + test OVS performance in real world scenarios. + +In Colorado release only the traffic generator mode is supported. + +Yardstick Installation +^^^^^^^^^^^^^^^^^^^^^^ + +In order to run Yardstick testcases, you will need to prepare your test +environment. Please follow the `installation instructions +<http://artifacts.opnfv.org/yardstick/docs/user_guides_framework/index.html>`__ +to install the yardstick. + +Please note, that yardstick uses OpenStack for execution of testcases. +OpenStack must be installed with Heat and Neutron services. Otherwise +vswitchperf testcases cannot be executed. + +VM image with vswitchperf +^^^^^^^^^^^^^^^^^^^^^^^^^ + +A special VM image is required for execution of vswitchperf specific testcases +by yardstick. It is possible to use a sample VM image available at OPNFV +artifactory or to build customized image. + +Sample VM image with vswitchperf +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Sample VM image is available at vswitchperf section of OPNFV artifactory +for free download: + +.. code-block:: console + + $ wget http://artifacts.opnfv.org/vswitchperf/vnf/vsperf-yardstick-image.qcow2 + +This image can be used for execution of sample testcases with dummy traffic +generator. + +**NOTE:** Traffic generators might require an installation of client software. +This software is not included in the sample image and must be installed by user. + +**NOTE:** This image will be updated only in case, that new features related +to yardstick integration will be added to the vswitchperf. + +Preparation of custom VM image +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In general, any Linux distribution supported by vswitchperf can be used as +a base image for vswitchperf. One of the possibilities is to modify vloop-vnf +image, which can be downloaded from `<http://artifacts.opnfv.org/vswitchperf.html/>`__ +(see :ref:`vloop-vnf`). + +Please follow the :ref:`vsperf-installation` to +install vswitchperf inside vloop-vnf image. As vswitchperf will be run in +trafficgen mode, it is possible to skip installation and compilation of OVS, +QEMU and DPDK to keep image size smaller. + +In case, that selected traffic generator requires installation of additional +client software, please follow appropriate documentation. For example in case +of IXIA, you would need to install IxOS and IxNetowrk TCL API. + +VM image usage +~~~~~~~~~~~~~~ + +Image with vswitchperf must be uploaded into the glance service and +vswitchperf specific flavor configured, e.g.: + +.. code-block:: console + + $ glance --os-username admin --os-image-api-version 1 image-create --name \ + vsperf --is-public true --disk-format qcow2 --container-format bare --file \ + vsperf-yardstick-image.qcow2 + + $ nova --os-username admin flavor-create vsperf-flavor 100 2048 25 1 + +Testcase execution +^^^^^^^^^^^^^^^^^^ + +After installation, yardstick is available as python package within yardstick +specific virtual environment. It means, that yardstick environment must be +enabled before the test execution, e.g.: + +.. code-block:: console + + source ~/yardstick_venv/bin/activate + + +Next step is configuration of OpenStack environment, e.g. in case of devstack: + +.. code-block:: console + + source /opt/openstack/devstack/openrc + export EXTERNAL_NETWORK=public + +Vswitchperf testcases executable by yardstick are located at vswitchperf +repository inside ``yardstick/tests`` directory. Example of their download +and execution follows: + +.. code-block:: console + + git clone https://gerrit.opnfv.org/gerrit/vswitchperf + cd vswitchperf + + yardstick -d task start yardstick/tests/rfc2544_throughput_dummy.yaml + +**NOTE:** Optional argument ``-d`` shows debug output. + +Testcase customization +^^^^^^^^^^^^^^^^^^^^^^ + +Yardstick testcases are described by YAML files. vswitchperf specific testcases +are part of the vswitchperf repository and their yaml files can be found at +``yardstick/tests`` directory. For detailed description of yaml file structure, +please see yardstick documentation and testcase samples. Only vswitchperf specific +parts will be discussed here. + +Example of yaml file: + +.. code-block:: yaml + + ... + scenarios: + - + type: Vsperf + options: + testname: 'p2p_rfc2544_throughput' + trafficgen_port1: 'eth1' + trafficgen_port2: 'eth3' + external_bridge: 'br-ex' + test_params: 'TRAFFICGEN_DURATION=30;TRAFFIC={'traffic_type':'rfc2544_throughput}' + conf_file: '~/vsperf-yardstick.conf' + + host: vsperf.demo + + runner: + type: Sequence + scenario_option_name: frame_size + sequence: + - 64 + - 128 + - 512 + - 1024 + - 1518 + sla: + metrics: 'throughput_rx_fps' + throughput_rx_fps: 500000 + action: monitor + + context: + ... + +Section option +~~~~~~~~~~~~~~ + +Section **option** defines details of vswitchperf test scenario. Lot of options +are identical to the vswitchperf parameters passed through ``--test-params`` +argument. Following options are supported: + +- **frame_size** - a packet size for which test should be executed; + Multiple packet sizes can be tested by modification of Sequence runner + section inside YAML definition. Default: '64' +- **conf_file** - sets path to the vswitchperf configuration file, which will be + uploaded to VM; Default: '~/vsperf-yardstick.conf' +- **setup_script** - sets path to the setup script, which will be executed + during setup and teardown phases +- **trafficgen_port1** - specifies device name of 1st interface connected to + the trafficgen +- **trafficgen_port2** - specifies device name of 2nd interface connected to + the trafficgen +- **external_bridge** - specifies name of external bridge configured in OVS; + Default: 'br-ex' +- **test_params** - specifies a string with a list of vsperf configuration + parameters, which will be passed to the ``--test-params`` CLI argument; + Parameters should be stated in the form of ``param=value`` and separated + by a semicolon. Configuration of traffic generator is driven by ``TRAFFIC`` + dictionary, which can be also updated by values defined by ``test_params``. + Please check VSPERF documentation for details about available configuration + parameters and their data types. + In case that both **test_params** and **conf_file** are specified, + then values from **test_params** will override values defined + in the configuration file. + +In case that **trafficgen_port1** and/or **trafficgen_port2** are defined, then +these interfaces will be inserted into the **external_bridge** of OVS. It is +expected, that OVS runs at the same node, where the testcase is executed. In case +of more complex OpenStack installation or a need of additional OVS configuration, +**setup_script** can be used. + +**NOTE** It is essential to specify a configuration for selected traffic generator. +In case, that standalone testcase is created, then traffic generator can be +selected and configured directly in YAML file by **test_params**. On the other +hand, if multiple testcases should be executed with the same traffic generator +settings, then a customized configuration file should be prepared and its name +passed by **conf_file** option. + +Section runner +~~~~~~~~~~~~~~ + +Yardstick supports several `runner types +<http://artifacts.opnfv.org/yardstick/docs/userguide/architecture.html#runner-types>`__. +In case of vswitchperf specific TCs, **Sequence** runner type can be used to +execute the testcase for given list of frame sizes. + + +Section sla +~~~~~~~~~~~ + +In case that sla section is not defined, then testcase will be always +considered as successful. On the other hand, it is possible to define a set of +test metrics and their minimal values to evaluate test success. Any numeric +value, reported by vswitchperf inside CSV result file, can be used. +Multiple metrics can be defined as a coma separated list of items. Minimal +value must be set separately for each metric. + +e.g.: + +.. code-block:: yaml + + sla: + metrics: 'throughput_rx_fps,throughput_rx_mbps' + throughput_rx_fps: 500000 + throughput_rx_mbps: 1000 + +In case that any of defined metrics will be lower than defined value, then +testcase will be marked as failed. Based on ``action`` policy, yardstick +will either stop test execution (value ``assert``) or it will run next test +(value ``monitor``). + +**NOTE** The throughput SLA (or any other SLA) cannot be set to a meaningful +value without knowledge of the server and networking environment, possibly +including prior testing in that environment to establish a baseline SLA level +under well-understood circumstances. |
