summaryrefslogtreecommitdiffstats
path: root/docs/testing
diff options
context:
space:
mode:
Diffstat (limited to 'docs/testing')
-rw-r--r--docs/testing/developer/devguide/design/vswitchperf_design.rst24
-rw-r--r--docs/testing/user/configguide/installation.rst27
-rw-r--r--docs/testing/user/configguide/trafficgen.rst43
-rw-r--r--docs/testing/user/userguide/index.rst1
-rw-r--r--docs/testing/user/userguide/teststeps.rst19
-rw-r--r--docs/testing/user/userguide/trafficcapture.rst297
6 files changed, 400 insertions, 11 deletions
diff --git a/docs/testing/developer/devguide/design/vswitchperf_design.rst b/docs/testing/developer/devguide/design/vswitchperf_design.rst
index 33051493..96ffcf62 100644
--- a/docs/testing/developer/devguide/design/vswitchperf_design.rst
+++ b/docs/testing/developer/devguide/design/vswitchperf_design.rst
@@ -415,6 +415,30 @@ Detailed description of ``TRAFFIC`` dictionary items follows:
congestion (DEI header field).
Data type: int (NOTE: must fit to 1 bit)
Default value: 0
+ 'capture' - A dictionary with traffic capture configuration.
+ NOTE: It is supported only by T-Rex traffic generator.
+ 'enabled' - Specifies if traffic should be captured
+ Data type: bool
+ Default value: False
+ 'tx_ports' - A list of ports, where frames transmitted towards DUT will
+ be captured. Ports have numbers 0 and 1. TX packet capture
+ is disabled if list of ports is empty.
+ Data type: list
+ Default value: [0]
+ 'rx_ports' - A list of ports, where frames received from DUT will
+ be captured. Ports have numbers 0 and 1. RX packet capture
+ is disabled if list of ports is empty.
+ Data type: list
+ Default value: [1]
+ 'count' - A number of frames to be captured. The same count value
+ is applied to both TX and RX captures.
+ Data type: int
+ Default value: 1
+ 'filter' - An expression used to filter TX and RX packets. It uses the same
+ syntax as pcap library. See pcap-filter man page for additional
+ details.
+ Data type: str
+ Default value: ''
.. _configuration-of-guest-options:
diff --git a/docs/testing/user/configguide/installation.rst b/docs/testing/user/configguide/installation.rst
index 8bad4efd..51588007 100644
--- a/docs/testing/user/configguide/installation.rst
+++ b/docs/testing/user/configguide/installation.rst
@@ -49,6 +49,8 @@ Supported Operating Systems
* Fedora 25 (kernel 4.9 requires DPDK 16.11 and newer)
* openSUSE 42.2
* openSUSE 42.3
+* openSUSE Tumbleweed
+* SLES 15
* RedHat 7.2 Enterprise Linux
* RedHat 7.3 Enterprise Linux
* Ubuntu 14.04
@@ -134,13 +136,19 @@ The test suite requires Python 3.3 or newer and relies on a number of other
system and python packages. These need to be installed for the test suite
to function.
+Updated kernel and certain development packages are required by DPDK,
+OVS (especially Vanilla OVS) and QEMU. It is necessary to check if the
+versions of these packages are not being **held-back** and if the
+DNF/APT/YUM configuration does not prevent their modification, by
+enforcing settings such as **"exclude-kernel"**.
+
Installation of required packages, preparation of Python 3 virtual
environment and compilation of OVS, DPDK and QEMU is performed by
-script **systems/build_base_machine.sh**. It should be executed under
+script **systems/build_base_machine.sh**. It should be executed under the
user account, which will be used for vsperf execution.
**NOTE:** Password-less sudo access must be configured for given
-user account before script is executed.
+user account before the script is executed.
.. code:: bash
@@ -154,13 +162,14 @@ automatically.
Script **build_base_machine.sh** will install all the vsperf dependencies
in terms of system packages, Python 3.x and required Python modules.
In case of CentOS 7 or RHEL it will install Python 3.3 from an additional
-repository provided by Software Collections (`a link`_). Installation script
+repository provided by Software Collections (`a link`_). The installation script
will also use `virtualenv`_ to create a vsperf virtual environment, which is
-isolated from the default Python environment. This environment will reside in a
-directory called **vsperfenv** in $HOME. It will ensure, that system wide Python
-installation is not modified or broken by VSPERF installation. The complete list
-of Python packages installed inside virtualenv can be found at file
-``requirements.txt``, which is located at vswitchperf repository.
+isolated from the default Python environment, using the Python3 package located
+in **/usr/bin/python3**. This environment will reside in a directory called
+**vsperfenv** in $HOME. It will ensure, that system wide Python installation
+ is not modified or broken by VSPERF installation. The complete list of Python
+packages installed inside virtualenv can be found in the file
+``requirements.txt``, which is located at the vswitchperf repository.
**NOTE:** For RHEL 7.3 Enterprise and CentOS 7.3 OVS Vanilla is not
built from upstream source due to kernel incompatibilities. Please see the
@@ -193,7 +202,7 @@ new shell session. Its activation is specific to your OS:
.. code:: bash
- $ scl enable python33 bash
+ $ scl enable rh-python34 bash
$ source $HOME/vsperfenv/bin/activate
* Fedora and Ubuntu
diff --git a/docs/testing/user/configguide/trafficgen.rst b/docs/testing/user/configguide/trafficgen.rst
index 4b9eec6e..33824486 100644
--- a/docs/testing/user/configguide/trafficgen.rst
+++ b/docs/testing/user/configguide/trafficgen.rst
@@ -44,7 +44,8 @@ and is configured as follows:
'stream_type' : 'L4',
'pre_installed_flows' : 'No', # used by vswitch implementation
'flow_type' : 'port', # used by vswitch implementation
-
+ 'flow_control' : False, # supported only by IxNet
+ 'learning_frames' : True, # supported only by IxNet
'l2': {
'framesize': 64,
'srcmac': '00:00:00:00:00:00',
@@ -67,6 +68,13 @@ and is configured as follows:
'priority': 0,
'cfi': 0,
},
+ 'capture': {
+ 'enabled': False,
+ 'tx_ports' : [0],
+ 'rx_ports' : [1],
+ 'count': 1,
+ 'filter': '',
+ },
}
The framesize parameter can be overridden from the configuration
@@ -783,6 +791,10 @@ It is neccesary for proper connection between Trex server and VSPERF.
cd trex-core/scripts/
./t-rex-64 -i
+**NOTE:** Please check your firewall settings at both DUT and T-Rex server.
+Firewall must allow a connection from DUT (VSPERF) to the T-Rex server running
+at TCP port 4501.
+
For additional information about Trex stateless mode see Trex stateless documentation:
https://trex-tgn.cisco.com/trex/doc/trex_stateless.html
@@ -826,6 +838,15 @@ Default value of this parameter is defined in conf/03_traffic.conf as follows:
TRAFFICGEN_TREX_RFC2544_TPUT_THRESHOLD = ''
+T-Rex can have learning packets enabled. For certain tests it may be beneficial
+to send some packets before starting test traffic to allow switch learning to take
+place. This can be adjusted with the following configurations:
+
+.. code-block:: console
+
+ TRAFFICGEN_TREX_LEARNING_MODE=True
+ TRAFFICGEN_TREX_LEARNING_DURATION=5
+
SR-IOV and Multistream layer 2
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
T-Rex by default only accepts packets on the receive side if the destination mac matches the
@@ -840,3 +861,23 @@ modified. Enable Promiscuous mode when doing multistream at layer 2 testing with
.. code-block:: console
TRAFFICGEN_TREX_PROMISCUOUS=True
+
+RFC2544 Validation
+~~~~~~~~~~~~~~~~~~
+
+T-Rex can perform a verification run for a longer duration once the binary search of the
+RFC2544 trials have completed. This duration should be at least 60 seconds. This is similar
+to other traffic generator functionality where a more sustained time can be attempted to
+verify longer runs from the result of the search. This can be configured with the following
+params
+
+.. code-block:: console
+
+ TRAFFICGEN_TREX_VERIFICATION_MODE = False
+ TRAFFICGEN_TREX_VERIFICATION_DURATION = 60
+ TRAFFICGEN_TREX_MAXIMUM_VERIFICATION_TRIALS = 10
+
+The duration and maximum number of attempted verification trials can be set to change the
+behavior of this step. If the verification step fails, it will resume the binary search
+with new values where the maximum output will be the last attempted frame rate minus the
+current set thresh hold.
diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst
index 64d91657..350fbe54 100644
--- a/docs/testing/user/userguide/index.rst
+++ b/docs/testing/user/userguide/index.rst
@@ -16,5 +16,6 @@ VSPERF Test Guide
./testusage.rst
./teststeps.rst
./integration.rst
+ ./trafficcapture.rst
./yardstick.rst
./testlist.rst
diff --git a/docs/testing/user/userguide/teststeps.rst b/docs/testing/user/userguide/teststeps.rst
index 8be67310..08c95311 100644
--- a/docs/testing/user/userguide/teststeps.rst
+++ b/docs/testing/user/userguide/teststeps.rst
@@ -131,6 +131,21 @@ of supported objects and their most common functions follows:
['vnf2', 'stop'],
['vnf1', 'stop'],
+ * ``VNF[ID]`` - provides access to VNFs deployed automatically by testcase deployment
+ scenario. For Example ``pvvp`` deployment automatically starts two VNFs before any
+ TestStep is executed. It is possible to access these VNFs by VNF0 and VNF1 labels.
+
+ List of supported functions is identical to ``vnf[ID]`` option above except functions
+ ``start`` and ``stop``.
+
+ Examples:
+
+ .. code-block:: python
+
+ ['VNF0', 'execute_and_wait', 'ifconfig eth2 5.5.5.1/24 up'],
+ ['VNF1', 'execute_and_wait', 'ifconfig eth2 5.5.5.2/24 up', 120, 'root.*#'],
+ ['VNF2', 'execute_and_wait', 'ping -c1 5.5.5.1'],
+
* ``trafficgen`` - triggers traffic generation
List of supported functions:
@@ -256,7 +271,9 @@ of supported objects and their most common functions follows:
in case that condition is not ``True``
* ``Eval expression`` - evaluates given expression as a python code and returns
its result
- * ``Exec_Shell command`` - executes a shell command
+ * ``Exec_Shell command`` - executes a shell command and wait until it finishes
+ * ``Exec_Shell_Background command`` - executes a shell command at background;
+ Command will be automatically terminated at the end of testcase execution.
* ``Exec_Python code`` - executes a python code
diff --git a/docs/testing/user/userguide/trafficcapture.rst b/docs/testing/user/userguide/trafficcapture.rst
new file mode 100644
index 00000000..fa09bfed
--- /dev/null
+++ b/docs/testing/user/userguide/trafficcapture.rst
@@ -0,0 +1,297 @@
+Traffic Capture
+---------------
+
+Tha ability to capture traffic at multiple points of the system is crucial to
+many of the functional tests. It allows the verification of functionality for
+both the vSwitch and the NICs using hardware acceleration for packet
+manipulation and modification.
+
+There are three different methods of traffic capture supported by VSPERF.
+Detailed descriptions of these methods as well as their pros and cons can be
+found in the following chapters.
+
+Traffic Capture inside of a VM
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This method uses the standard PVP scenario, in which vSwitch first processes
+and modifies the packet before forwarding it to the VM. Inside of the VM we
+capture the traffic using **tcpdump** or a similiar technique. The capture
+information is the used to verify the expected modifications to the packet done
+by vSwitch.
+
+.. code-block:: console
+
+ _
+ +--------------------------------------------------+ |
+ | | |
+ | +------------------------------------------+ | |
+ | | Traffic capture and Packet Forwarding | | |
+ | +------------------------------------------+ | |
+ | ^ : | |
+ | | | | | Guest
+ | : v | |
+ | +---------------+ +---------------+ | |
+ | | logical port 0| | logical port 1| | |
+ +---+---------------+----------+---------------+---+ _|
+ ^ :
+ | |
+ : v _
+ +---+---------------+----------+---------------+---+ |
+ | | logical port 0| | logical port 1| | |
+ | +---------------+ +---------------+ | |
+ | ^ : | |
+ | | | | | Host
+ | : v | |
+ | +--------------+ +--------------+ | |
+ | | phy port | vSwitch | phy port | | |
+ +---+--------------+------------+--------------+---+ _|
+ ^ :
+ | |
+ : v
+ +--------------------------------------------------+
+ | |
+ | traffic generator |
+ | |
+ +--------------------------------------------------+
+
+PROS:
+
+- supports testing with all traffic generators
+- easy to use and implement into test
+- allows testing hardware offloading on the ingress side
+
+CONS:
+
+- does not allow testing hardware offloading on the egress side
+
+An example of Traffic Capture in VM test:
+
+.. code-block:: python
+
+ # Capture Example 1 - Traffic capture inside VM (PVP scenario)
+ # This TestCase will modify VLAN ID set by the traffic generator to the new value.
+ # Correct VLAN ID settings is verified by inspection of captured frames.
+ {
+ Name: capture_pvp_modify_vid,
+ Deployment: pvp,
+ Description: Test and verify VLAN ID modification by Open vSwitch,
+ Parameters : {
+ VSWITCH : OvsDpdkVhost, # works also for Vanilla OVS
+ TRAFFICGEN_DURATION : 5,
+ TRAFFIC : {
+ traffic_type : rfc2544_continuous,
+ frame_rate : 100,
+ 'vlan': {
+ 'enabled': True,
+ 'id': 8,
+ 'priority': 1,
+ 'cfi': 0,
+ },
+ },
+ GUEST_LOOPBACK : ['linux_bridge'],
+ },
+ TestSteps: [
+ # replace original flows with vlan ID modification
+ ['!vswitch', 'add_flow', 'br0', {'in_port': '1', 'actions': ['mod_vlan_vid:4','output:3']}],
+ ['!vswitch', 'add_flow', 'br0', {'in_port': '2', 'actions': ['mod_vlan_vid:4','output:4']}],
+ ['vswitch', 'dump_flows', 'br0'],
+ # verify that received frames have modified vlan ID
+ ['VNF0', 'execute_and_wait', 'tcpdump -i eth0 -c 5 -w dump.pcap vlan 4 &'],
+ ['trafficgen', 'send_traffic',{}],
+ ['!VNF0', 'execute_and_wait', 'tcpdump -qer dump.pcap vlan 4 2>/dev/null | wc -l','|^(\d+)$'],
+ ['tools', 'assert', '#STEP[-1][0] == 5'],
+ ],
+ },
+
+Traffic Capture for testing NICs with HW offloading/acceleration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The NIC with hardware acceleration/offloading is inserted as an additional card
+into the server. Two ports on this card are then connected together using
+a patch cable as shown in the diagram. Only a single port of the tested NIC is
+setup with DPDK acceleration, while the other is handled by the Linux Ip stack
+allowing for traffic capture. The two NICs are then connected by vSwitch so the
+original card can forward the processed packets to the traffic generator. The
+ports handled by Linux IP stack allow for capturing packets, which are then
+analyzed for changes done by both the vSwitch and the NIC with hardware
+acceleration.
+
+.. code-block:: console
+
+ _
+ +------------------------------------------------+ |
+ | | |
+ | +----------------------------------------+ | |
+ | | vSwitch | | |
+ | | +----------------------------------+ | | |
+ | | | | | | |
+ | | | +------------------+ | | | |
+ | | | | | v | | |
+ | +----------------------------------------+ | | Device under Test
+ | ^ | ^ | | |
+ | | | | | | |
+ | | v | v | |
+ | +--------------+ +--------------+ | |
+ | | | | NIC w HW acc | | |
+ | | phy ports | | phy ports | | |
+ +---+--------------+----------+--------------+---+ _|
+ ^ : ^ :
+ | | | |
+ | | +-------+
+ : v Patch Cable
+ +------------------------------------------------+
+ | |
+ | traffic generator |
+ | |
+ +------------------------------------------------+
+
+PROS:
+
+- allows testing hardware offloading on both the ingress and egress side
+- supports testing with all traffic generators
+- relatively easy to use and implement into tests
+
+CONS:
+
+- a more complex setup with two cards
+- if the tested card only has one port, an additional card is needed
+
+An example of Traffic Capture for testing NICs with HW offloading test:
+
+.. code-block:: python
+
+ # Capture Example 2 - Setup with 2 NICs, where traffic is captured after it is
+ # processed by NIC under the test (2nd NIC). See documentation for further details.
+ # This TestCase will strip VLAN headers from traffic sent by the traffic generator.
+ # The removal of VLAN headers is verified by inspection of captured frames.
+ #
+ # NOTE: This setup expects a DUT with two NICs with two ports each. First NIC is
+ # connected to the traffic generator (standard VSPERF setup). Ports of a second NIC
+ # are interconnected by a patch cable. PCI addresses of all four ports have to be
+ # properly configured in the WHITELIST_NICS parameter.
+ {
+ Name: capture_p2p2p_strip_vlan_ovs,
+ Deployment: clean,
+ Description: P2P Continuous Stream,
+ Parameters : {
+ _CAPTURE_P2P2P_OVS_ACTION : 'strip_vlan',
+ TRAFFIC : {
+ bidir : False,
+ traffic_type : rfc2544_continuous,
+ frame_rate : 100,
+ 'l2': {
+ 'srcmac': ca:fe:00:00:00:00,
+ 'dstmac': 00:00:00:00:00:01
+ },
+ 'vlan': {
+ 'enabled': True,
+ 'id': 8,
+ 'priority': 1,
+ 'cfi': 0,
+ },
+ },
+ # suppress DPDK configuration, so physical interfaces are not bound to DPDK driver
+ 'WHITELIST_NICS' : [],
+ 'NICS' : [],
+ },
+ TestSteps: _CAPTURE_P2P2P_SETUP + [
+ # capture traffic after processing by NIC under the test (after possible egress HW offloading)
+ ['tools', 'exec_shell_background', 'tcpdump -i [2][device] -c 5 -w capture.pcap '
+ 'ether src [l2][srcmac]'],
+ ['trafficgen', 'send_traffic', {}],
+ ['vswitch', 'dump_flows', 'br0'],
+ ['vswitch', 'dump_flows', 'br1'],
+ # there must be 5 captured frames...
+ ['tools', 'exec_shell', 'tcpdump -r capture.pcap | wc -l', '|^(\d+)$'],
+ ['tools', 'assert', '#STEP[-1][0] == 5'],
+ # ...but no vlan headers
+ ['tools', 'exec_shell', 'tcpdump -r capture.pcap vlan | wc -l', '|^(\d+)$'],
+ ['tools', 'assert', '#STEP[-1][0] == 0'],
+ ],
+ },
+
+
+Traffic Capture on the Traffic Generator
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Using the functionality of the Traffic generator makes it possible to configure
+Traffic Capture on both it's ports. With Traffic Capture enabled, VSPERF
+instructs the Traffic Generator to automatically export captured data into
+a pcap file. The captured packets are then sent to VSPERF for analysis and
+verification, monitoring any changes done by both vSwitch and the NICs.
+
+Vsperf currently only supports this functionality with the **T-Rex** generator.
+
+.. code-block:: console
+
+ _
+ +--------------------------------------------------+ |
+ | | |
+ | +--------------------------+ | |
+ | | | | |
+ | | v | | Host
+ | +--------------+ +--------------+ | |
+ | | phy port | vSwitch | phy port | | |
+ +---+--------------+------------+--------------+---+ _|
+ ^ :
+ | |
+ : v
+ +--------------------------------------------------+
+ | |
+ | traffic generator |
+ | |
+ +--------------------------------------------------+
+
+PROS:
+
+- allows testing hardware offloading on both the ingress and egress side
+- does not require an additional NIC
+
+CONS:
+
+- currently only supported by **T-Rex** traffic generator
+
+An example Traffic Capture on the Traffic Generator test:
+
+.. code-block:: python
+
+
+ # Capture Example 3 - Traffic capture by traffic generator.
+ # This TestCase uses OVS flow to add VLAN tag with given ID into every
+ # frame send by traffic generator. Correct frame modificaiton is verified by
+ # inspection of packet capture received by T-Rex.
+ {
+ Name: capture_p2p_add_vlan_ovs_trex,
+ Deployment: clean,
+ Description: OVS: Test VLAN tag modification and verify it by traffic capture,
+ vSwitch : OvsDpdkVhost, # works also for Vanilla OVS
+ Parameters : {
+ TRAFFICGEN : Trex,
+ TRAFFICGEN_DURATION : 5,
+ TRAFFIC : {
+ traffic_type : rfc2544_continuous,
+ frame_rate : 100,
+ # enable capture of five RX frames
+ 'capture': {
+ 'enabled': True,
+ 'tx_ports' : [],
+ 'rx_ports' : [1],
+ 'count' : 5,
+ },
+ },
+ },
+ TestSteps : STEP_VSWITCH_P2P_INIT + [
+ # replace standard L2 flows by flows, which will add VLAN tag with ID 3
+ ['!vswitch', 'add_flow', 'int_br0', {'in_port': '1', 'actions': ['mod_vlan_vid:3','output:2']}],
+ ['!vswitch', 'add_flow', 'int_br0', {'in_port': '2', 'actions': ['mod_vlan_vid:3','output:1']}],
+ ['vswitch', 'dump_flows', 'int_br0'],
+ ['trafficgen', 'send_traffic', {}],
+ ['trafficgen', 'get_results'],
+ # verify that captured frames have vlan tag with ID 3
+ ['tools', 'exec_shell', 'tcpdump -qer /#STEP[-1][0][capture_rx] vlan 3 '
+ '2>/dev/null | wc -l', '|^(\d+)$'],
+ # number of received frames with expected VLAN id must match the number of captured frames
+ ['tools', 'assert', '#STEP[-1][0] == 5'],
+ ] + STEP_VSWITCH_P2P_FINIT,
+ },
+