diff options
Diffstat (limited to 'docs/userguide')
| -rw-r--r-- | docs/userguide/index.rst | 1 | ||||
| -rwxr-xr-x | docs/userguide/integration.rst | 95 | ||||
| -rwxr-xr-x | docs/userguide/testusage.rst | 138 | ||||
| -rwxr-xr-x | docs/userguide/yardstick.rst | 223 |
4 files changed, 448 insertions, 9 deletions
diff --git a/docs/userguide/index.rst b/docs/userguide/index.rst index 591a1211..1a796dbf 100644 --- a/docs/userguide/index.rst +++ b/docs/userguide/index.rst @@ -12,3 +12,4 @@ VSPERF User Guide testusage.rst integration.rst + yardstick.rst diff --git a/docs/userguide/integration.rst b/docs/userguide/integration.rst index 27bf2cd0..eccd0c76 100755 --- a/docs/userguide/integration.rst +++ b/docs/userguide/integration.rst @@ -7,13 +7,101 @@ Integration tests VSPERF includes a set of integration tests defined in conf/integration. These tests can be run by specifying --integration as a parameter to vsperf. -Current tests in conf/integration are Overlay tests. +Current tests in conf/integration include switch functionality and Overlay +tests. -VSPERF supports VXLAN, GRE and GENEVE tunneling protocols. +Tests in the conf/integration can be used to test scaling of different switch +configurations by adding steps into the test case. + +For the overlay tests VSPERF supports VXLAN, GRE and GENEVE tunneling protocols. Testing of these protocols is limited to unidirectional traffic and P2P (Physical to Physical scenarios). -NOTE: The configuration for overlay tests provided in this guide is for unidirectional traffic only. +NOTE: The configuration for overlay tests provided in this guide is for +unidirectional traffic only. + +Executing Integration Tests +--------------------------- + +To execute integration tests VSPERF is run with the integration parameter. To +view the current test list simply execute the following command: + +.. code-block:: console + + ./vsperf --integration --list + +The standard tests included are defined inside the +``conf/integration/01_testcases.conf`` file. + +Test Steps +---------- + +Execution of integration tests are done on a step by step work flow starting +with step 0 as defined inside the test case. Each step of the test increments +the step number by one which is indicated in the log. + +.. code-block:: console + + (testcases.integration) - Step 1 - 'vswitch add_switch ['int_br1']' ... OK + +Each step in the test case is validated. If a step does not pass validation the +test will fail and terminate. The test will continue until a failure is detected +or all steps pass. A csv report file is generated after a test completes with an +OK or FAIL result. + +Test Macros +----------- + +Test profiles can include macros as part of the test step. Each step in the +profile may return a value such as a port name. Recall macros use #STEP to +indicate the recalled value inside the return structure. If the method the +test step calls returns a value it can be later recalled, for example: + +.. code-block:: python + + { + "Name": "vswitch_add_del_vport", + "Deployment": "clean", + "Description": "vSwitch - add and delete virtual port", + "TestSteps": [ + ['vswitch', 'add_switch', 'int_br0'], # STEP 0 + ['vswitch', 'add_vport', 'int_br0'], # STEP 1 + ['vswitch', 'del_port', 'int_br0', '#STEP[1][0]'], # STEP 2 + ['vswitch', 'del_switch', 'int_br0'], # STEP 3 + ] + } + +This test profile uses the the vswitch add_vport method which returns a string +value of the port added. This is later called by the del_port method using the +name from step 1. + +Also commonly used steps can be created as a separate profile. + +.. code-block:: python + + STEP_VSWITCH_PVP_INIT = [ + ['vswitch', 'add_switch', 'int_br0'], # STEP 0 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 1 + ['vswitch', 'add_phy_port', 'int_br0'], # STEP 2 + ['vswitch', 'add_vport', 'int_br0'], # STEP 3 + ['vswitch', 'add_vport', 'int_br0'], # STEP 4 + ] + +This profile can then be used inside other testcases + +.. code-block:: python + + { + "Name": "vswitch_pvp", + "Deployment": "clean", + "Description": "vSwitch - configure switch and one vnf", + "TestSteps": STEP_VSWITCH_PVP_INIT + + [ + ['vnf', 'start'], + ['vnf', 'stop'], + ] + + STEP_VSWITCH_PVP_FINIT + } Executing Tunnel encapsulation tests ------------------------------------ @@ -90,7 +178,6 @@ To run OVS NATIVE tunnel tests (VXLAN/GRE/GENEVE): .. code-block:: python VSWITCH = 'OvsVanilla' - VSWITCH_VANILLA_PHY_PORT_NAMES = ['nic1name', 'nic2name'] # Specify vport_* kernel module to test. VSWITCH_VANILLA_KERNEL_MODULES = ['vport_vxlan', 'vport_gre', diff --git a/docs/userguide/testusage.rst b/docs/userguide/testusage.rst index c20651b5..d807590d 100755 --- a/docs/userguide/testusage.rst +++ b/docs/userguide/testusage.rst @@ -174,7 +174,7 @@ Some tests allow for configurable parameters, including test duration $ ./vsperf --conf-file user_settings.py --tests RFC2544Tput - --test-param "duration=10;pkt_sizes=128" + --test-params "duration=10;pkt_sizes=128" For all available options, check out the help dialog: @@ -199,7 +199,6 @@ for Vanilla OVS: .. code-block:: console VSWITCH = 'OvsVanilla' - VSWITCH_VANILLA_PHY_PORT_NAMES = ['$PORT1', '$PORT2'] Where $PORT1 and $PORT2 are the Linux interfaces you'd like to bind to the vswitch. @@ -291,7 +290,7 @@ To run tests using Vanilla OVS: or use --test-param $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf - --test-param "vanilla_tgen_tx_ip=n.n.n.n; + --test-params "vanilla_tgen_tx_ip=n.n.n.n; vanilla_tgen_tx_mac=nn:nn:nn:nn:nn:nn" @@ -309,6 +308,8 @@ To run tests using Vanilla OVS: $ ./vsperf --conf-file<path_to_custom_conf>/10_custom.conf +.. _vfio-pci: + Using vfio_pci with DPDK ^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -346,6 +347,65 @@ To check that IOMMU is enabled on your platform: [ 3.335746] IOMMU: dmar1 using Queued invalidation .... +.. _SRIOV-support: + +Using SRIOV support +^^^^^^^^^^^^^^^^^^^ + +To use virtual functions of NIC with SRIOV support, use extended form +of NIC PCI slot definition: + +.. code-block:: python + + WHITELIST_NICS = ['0000:05:00.0|vf0', '0000:05:00.1|vf3'] + +Where 'vf' is an indication of virtual function usage and following +number defines a VF to be used. In case that VF usage is detected, +then vswitchperf will enable SRIOV support for given card and it will +detect PCI slot numbers of selected VFs. + +So in example above, one VF will be configured for NIC '0000:05:00.0' +and four VFs will be configured for NIC '0000:05:00.1'. Vswitchperf +will detect PCI addresses of selected VFs and it will use them during +test execution. + +At the end of vswitchperf execution, SRIOV support will be disabled. + +SRIOV support is generic and it can be used in different testing scenarios. +For example: + +* vSwitch tests with DPDK or without DPDK support to verify impact + of VF usage on vSwitch performance +* tests without vSwitch, where traffic is forwared directly + between VF interfaces by packet forwarder (e.g. testpmd application) +* tests without vSwitch, where VM accesses VF interfaces directly + by PCI-passthrough_ to measure raw VM throughput performance. + +.. _PCI-passthrough: + +Using QEMU with PCI passthrough support +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Raw virtual machine throughput performance can be measured by execution of PVP +test with direct access to NICs by PCI passthrough. To execute VM with direct +access to PCI devices, enable vfio-pci_. In order to use virtual functions, +SRIOV-support_ must be enabled. + +Execution of test with PCI passthrough with vswitch disabled: + +.. code-block:: console + + $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf + --vswtich none --vnf QemuPciPassthrough pvp_tput + +Any of supported guest-loopback-application_ can be used inside VM with +PCI passthrough support. + +Note: Qemu with PCI passthrough support can be used only with PVP test +deployment. + +.. _guest-loopback-application: + Selection of loopback application for PVP and PVVP tests ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -361,7 +421,7 @@ or use --test-param .. code-block:: console $ ./vsperf --conf-file=<path_to_custom_conf>/10_custom.conf - --test-param "guest_loopback=testpmd" + --test-params "guest_loopback=testpmd" Supported loopback applications are: @@ -377,6 +437,64 @@ Guest loopback application must be configured, otherwise traffic will not be forwarded by VM and testcases with PVP and PVVP deployments will fail. Guest loopback application is set to 'testpmd' by default. +Multi-Queue Configuration +^^^^^^^^^^^^^^^^^^^^^^^^^ + +VSPerf currently supports multi-queue with the following limitations: + + 1. Execution of pvp/pvvp tests require testpmd as the loopback if multi-queue + is enabled at the guest. + + 2. Requires QemuDpdkVhostUser as the vnf. + + 3. Requires switch to be set to OvsDpdkVhost. + + 4. Requires QEMU 2.5 or greater and any OVS version higher than 2.5. The + default upstream package versions installed by VSPerf satisfy this + requirement. + +To enable multi-queue modify the ''02_vswitch.conf'' file to enable multi-queue +on the switch. + + .. code-block:: console + + VSWITCH_MULTI_QUEUES = 2 + +**NOTE:** you should consider using the switch affinity to set a pmd cpu mask +that can optimize your performance. Consider the numa of the NIC in use if this +applies by checking /sys/class/net/<eth_name>/device/numa_node and setting an +appropriate mask to create PMD threads on the same numa node. + +When multi-queue is enabled, each dpdk or dpdkvhostuser port that is created +on the switch will set the option for multiple queues. + +To enable multi-queue on the guest modify the ''04_vnf.conf'' file. + + .. code-block:: console + + GUEST_NIC_QUEUES = 2 + +Enabling multi-queue at the guest will add multiple queues to each NIC port when +qemu launches the guest. + +Testpmd should be configured to take advantage of multi-queue on the guest. This +can be done by modifying the ''04_vnf.conf'' file. + + .. code-block:: console + + GUEST_TESTPMD_CPU_MASK = '-l 0,1,2,3,4' + + GUEST_TESTPMD_NB_CORES = 4 + GUEST_TESTPMD_TXQ = 2 + GUEST_TESTPMD_RXQ = 2 + +**NOTE:** The guest SMP cores must be configured to allow for testpmd to use the +optimal number of cores to take advantage of the multiple guest queues. + +**NOTE:** For optimal performance guest SMPs should be on the same numa as the +NIC in use if possible/applicable. Testpmd should be assigned at least +(nb_cores +1) total cores with the cpu mask. + Executing Packet Forwarding tests ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -443,7 +561,7 @@ Mode of operation is driven by configuration parameter -m or --mode "trafficgen-pause" - execute vSwitch and VNF but wait before traffic transmission In case, that VSPERF is executed in "trafficgen" mode, then configuration -of traffic generator should be configured through --test-param option. +of traffic generator should be configured through --test-params option. Supported CLI options useful for traffic generator configuration are: .. code-block:: console @@ -509,6 +627,16 @@ an appropriate amount of memory: .. code-block:: console VSWITCHD_DPDK_ARGS = ['-c', '0x4', '-n', '4', '--socket-mem 1024,0'] + VSWITCHD_DPDK_CONFIG = { + 'dpdk-init' : 'true', + 'dpdk-lcore-mask' : '0x4', + 'dpdk-socket-mem' : '1024,0', + } + +Note: Option VSWITCHD_DPDK_ARGS is used for vswitchd, which supports --dpdk +parameter. In recent vswitchd versions, option VSWITCHD_DPDK_CONFIG will be +used to configure vswitchd via ovs-vsctl calls. + More information ^^^^^^^^^^^^^^^^ diff --git a/docs/userguide/yardstick.rst b/docs/userguide/yardstick.rst new file mode 100755 index 00000000..7f09668d --- /dev/null +++ b/docs/userguide/yardstick.rst @@ -0,0 +1,223 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intel Corporation, AT&T and others. + +Execution of vswitchperf testcases by Yardstick +----------------------------------------------- + +General +^^^^^^^ + +Yardstick is a generic framework for a test execution, which is used for +validation of installation of OPNFV platform. In the future, Yardstick will +support two options of vswitchperf testcase execution: + +- plugin mode, which will execute native vswitchperf testcases; Tests will + be executed the same way as today, but test results will be processed and + reported by yardstick. +- traffic generator mode, which will run vswitchperf in **trafficgen** + mode only; Yardstick framework will be used to launch VNFs and to configure + flows to ensure, that traffic is properly routed. This mode will allow to + test OVS performance in real world scenarios. + +In Colorado release only the traffic generator mode is supported. + +Yardstick Installation +^^^^^^^^^^^^^^^^^^^^^^ + +In order to run Yardstick testcases, you will need to prepare your test +environment. Please follow the `installation instructions +<http://artifacts.opnfv.org/yardstick/brahmaputra/docs/user_guides_framework/index.html>`__ +to install the yardstick. + +Please note, that yardstick uses OpenStack for execution of testcases. +OpenStack must be installed with Heat and Neutron services. Otherwise +vswitchperf testcases cannot be executed. + +Vswitchperf VM image preparation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In general, any Linux distribution supported by vswitchperf can be used as +a base image for vswitchperf. One of the possibilities is to modify vloop-vnf +image, which can be downloaded from `<http://artifacts.opnfv.org/>`__. + +.. code-block:: console + + $ wget http://artifacts.opnfv.org/vswitchperf/vloop-vnf-ubuntu-14.04_20151216.qcow2 + +Please follow the `installation instructions +<http://artifacts.opnfv.org/vswitchperf/docs/configguide/installation.html>`__ to +install vswitchperf inside vloop-vnf image. As vswitchperf will be run in +trafficgen mode, it is possible to skip installation and compilation of OVS, +QEMU and DPDK to keep image size smaller. + +In case, that selected traffic generator requires installation of additional +client software, please follow appropriate documentation. For example in case +of IXIA, you would need to install IxOS and IxNetowrk TCL API. + +Final image with vswitchperf must be uploaded into the glance service and +vswitchperf specific flavor configured, e.g.: + +.. code-block:: console + + $ glance --os-username admin --os-image-api-version 1 image-create --name + vsperf --is-public true --disk-format qcow2 --container-format bare --file + image.qcow2 + + $ nova --os-username admin flavor-create vsperf-flavor 100 2048 25 1 + +Testcase customization +^^^^^^^^^^^^^^^^^^^^^^ + +Yardstick testcases are described by YAML files. vswitchperf specific testcases +are part of the vswitchperf repository and their yaml files can be found at +``yardstick/tests`` directory. For detailed description of yaml file sctructure, +please see yardstick documentation and testcase samples. Only vswitchperf specific +parts will be discussed here. + +Example of yaml file: + +.. code-block:: yaml + + ... + scenarios: + - + type: Vsperf + options: + testname: 'rfc2544_p2p_tput' + traffic_type: 'rfc2544' + pkt_sizes: '64' + bidirectional: 'True' + iload: 100 + duration: 30 + trafficgen_port1: 'eth1' + trafficgen_port2: 'eth3' + external_bridge: 'br-ex' + conf-file: '~/vsperf-yardstick.conf' + + host: vsperf.demo + + runner: + type: Sequence + scenario_option_name: pkt_sizes + sequence: + - 64 + - 128 + - 512 + - 1024 + - 1518 + sla: + metrics: 'throughput_rx_fps' + throughput_rx_fps: 500000 + action: monitor + + context: + ... + +Section option +~~~~~~~~~~~~~~ + +Section **option** defines details of vswitchperf test scenario. Lot of options +are identical to the vswitchperf parameters passed through ``--test-params`` +argument. Following options are supported: + +- **traffic_type** - specifies the type of traffic executed by traffic generator; + valid values are "rfc2544", "continuous" and "back2back"; Default: 'rfc2544' +- **pkt_sizes** - a packet size for which test should be executed; + Multiple packet sizes can be tested by modification of Sequence runner + section inside YAML definition. Default: '64' +- **duration** - sets duration for which traffic will be generated; Default: 30 +- **bidirectional** - specifies if traffic will be uni (False) or bi-directional + (True); Default: False +- **iload** - specifies frame rate; Default: 100 +- **rfc2544_trials** - specifies the number of trials performed for each packet + size +- **multistream** - specifies the number of simulated streams; Default: 0 (i.e. + multistream feature is disabled) +- **stream_type** - specifies network layer used for multistream simulation + the valid values are "L4", "L3" and "L2"; Default: 'L4' +- **conf-file** - sets path to the vswitchperf configuration file, which will be + uploaded to VM; Default: '~/vsperf-yardstick.conf' +- **setup-script** - sets path to the setup script, which will be executed + during setup and teardown phases +- **trafficgen_port1** - specifies device name of 1st interface connected to + the trafficgen +- **trafficgen_port2** - specifies device name of 2nd interface connected to + the trafficgen +- **external_bridge** - specifies name of external bridge configured in OVS; + Default: 'br-ex' + +In case that **trafficgen_port1** and/or **trafficgen_port2** are defined, then +these interfaces will be inserted into the **external_bridge** of OVS. It is +expected, that OVS runs at the same node, where the testcase is executed. In case +of more complex OpenStack installation or a need of additional OVS configuration, +**setup-script** can be used. + +Note: It is essential to prepare customized configuration file for the vsperf +and to specify its name by **conf-file** option. Config file must specify, which +traffic generator will be used and configure traffic generator specific options. + +Section runner +~~~~~~~~~~~~~~ + +Yardstick supports several `runner types +<http://artifacts.opnfv.org/yardstick/brahmaputra/docs/userguide/architecture.html#runner-types>`__. +In case of vswitchperf specific TCs, **Sequence** runner type can be used to +execute the testcase for given list of packet sizes. + + +Section sla +~~~~~~~~~~~ + +In case that sla section is not defined, then testcase will be always +considered as successful. On the other hand, it is possible to define a set of +test metrics and their minimal values to evaluate test success. Any numeric +value, reported by vswitchperf inside CSV result file, can be used. +Multiple metrics can be defined as a coma separated list of items. Minimal +value must be set separately for each metric. + +e.g.: + +.. code-block:: yaml + + sla: + metrics: 'throughput_rx_fps,throughput_rx_mbps' + throughput_rx_fps: 500000 + throughput_rx_mbps: 1000 + +In case that any of defined metrics will be lower than defined value, then +testcase will be marked as failed. Based on ``action`` policy, yardstick +will either stop test execution (value ``assert``) or it will run next test +(value ``monitor``). + +Testcase execution +^^^^^^^^^^^^^^^^^^ + +After installation, yardstick is available as python package within yardstick +specific virtual environment. It means, that before test execution yardstick +environment must be enabled, e.g.: + +.. code-block:: console + + source ~/yardstick_venv/bin/activate + + +Next step is configuration of OpenStack environment, e.g. in case of devstack: + +.. code-block:: console + + source /opt/openstack/devstack/openrc + export EXTERNAL_NETWORK=public + +Vswitchperf testcases executable by yardstick are located at vswitchperf +repository inside ``yardstick/tests`` directory. Example of their download +and execution follows: + +.. code-block:: console + + git clone https://gerrit.opnfv.org/gerrit/vswitchperf + cd vswitchperf + + yardstick -d task start yardstick/tests/p2p_cont.yaml + +Note: Optional argument ``-d`` shows debug output. |