diff options
Diffstat (limited to 'docs/testing')
| -rw-r--r-- | docs/testing/developer/devguide/design/vswitchperf_design.rst | 24 | ||||
| -rw-r--r-- | docs/testing/user/configguide/installation.rst | 27 | ||||
| -rw-r--r-- | docs/testing/user/configguide/trafficgen.rst | 43 | ||||
| -rw-r--r-- | docs/testing/user/userguide/index.rst | 1 | ||||
| -rw-r--r-- | docs/testing/user/userguide/teststeps.rst | 19 | ||||
| -rw-r--r-- | docs/testing/user/userguide/trafficcapture.rst | 297 |
6 files changed, 400 insertions, 11 deletions
diff --git a/docs/testing/developer/devguide/design/vswitchperf_design.rst b/docs/testing/developer/devguide/design/vswitchperf_design.rst index 33051493..96ffcf62 100644 --- a/docs/testing/developer/devguide/design/vswitchperf_design.rst +++ b/docs/testing/developer/devguide/design/vswitchperf_design.rst @@ -415,6 +415,30 @@ Detailed description of ``TRAFFIC`` dictionary items follows: congestion (DEI header field). Data type: int (NOTE: must fit to 1 bit) Default value: 0 + 'capture' - A dictionary with traffic capture configuration. + NOTE: It is supported only by T-Rex traffic generator. + 'enabled' - Specifies if traffic should be captured + Data type: bool + Default value: False + 'tx_ports' - A list of ports, where frames transmitted towards DUT will + be captured. Ports have numbers 0 and 1. TX packet capture + is disabled if list of ports is empty. + Data type: list + Default value: [0] + 'rx_ports' - A list of ports, where frames received from DUT will + be captured. Ports have numbers 0 and 1. RX packet capture + is disabled if list of ports is empty. + Data type: list + Default value: [1] + 'count' - A number of frames to be captured. The same count value + is applied to both TX and RX captures. + Data type: int + Default value: 1 + 'filter' - An expression used to filter TX and RX packets. It uses the same + syntax as pcap library. See pcap-filter man page for additional + details. + Data type: str + Default value: '' .. _configuration-of-guest-options: diff --git a/docs/testing/user/configguide/installation.rst b/docs/testing/user/configguide/installation.rst index 8bad4efd..51588007 100644 --- a/docs/testing/user/configguide/installation.rst +++ b/docs/testing/user/configguide/installation.rst @@ -49,6 +49,8 @@ Supported Operating Systems * Fedora 25 (kernel 4.9 requires DPDK 16.11 and newer) * openSUSE 42.2 * openSUSE 42.3 +* openSUSE Tumbleweed +* SLES 15 * RedHat 7.2 Enterprise Linux * RedHat 7.3 Enterprise Linux * Ubuntu 14.04 @@ -134,13 +136,19 @@ The test suite requires Python 3.3 or newer and relies on a number of other system and python packages. These need to be installed for the test suite to function. +Updated kernel and certain development packages are required by DPDK, +OVS (especially Vanilla OVS) and QEMU. It is necessary to check if the +versions of these packages are not being **held-back** and if the +DNF/APT/YUM configuration does not prevent their modification, by +enforcing settings such as **"exclude-kernel"**. + Installation of required packages, preparation of Python 3 virtual environment and compilation of OVS, DPDK and QEMU is performed by -script **systems/build_base_machine.sh**. It should be executed under +script **systems/build_base_machine.sh**. It should be executed under the user account, which will be used for vsperf execution. **NOTE:** Password-less sudo access must be configured for given -user account before script is executed. +user account before the script is executed. .. code:: bash @@ -154,13 +162,14 @@ automatically. Script **build_base_machine.sh** will install all the vsperf dependencies in terms of system packages, Python 3.x and required Python modules. In case of CentOS 7 or RHEL it will install Python 3.3 from an additional -repository provided by Software Collections (`a link`_). Installation script +repository provided by Software Collections (`a link`_). The installation script will also use `virtualenv`_ to create a vsperf virtual environment, which is -isolated from the default Python environment. This environment will reside in a -directory called **vsperfenv** in $HOME. It will ensure, that system wide Python -installation is not modified or broken by VSPERF installation. The complete list -of Python packages installed inside virtualenv can be found at file -``requirements.txt``, which is located at vswitchperf repository. +isolated from the default Python environment, using the Python3 package located +in **/usr/bin/python3**. This environment will reside in a directory called +**vsperfenv** in $HOME. It will ensure, that system wide Python installation + is not modified or broken by VSPERF installation. The complete list of Python +packages installed inside virtualenv can be found in the file +``requirements.txt``, which is located at the vswitchperf repository. **NOTE:** For RHEL 7.3 Enterprise and CentOS 7.3 OVS Vanilla is not built from upstream source due to kernel incompatibilities. Please see the @@ -193,7 +202,7 @@ new shell session. Its activation is specific to your OS: .. code:: bash - $ scl enable python33 bash + $ scl enable rh-python34 bash $ source $HOME/vsperfenv/bin/activate * Fedora and Ubuntu diff --git a/docs/testing/user/configguide/trafficgen.rst b/docs/testing/user/configguide/trafficgen.rst index 4b9eec6e..33824486 100644 --- a/docs/testing/user/configguide/trafficgen.rst +++ b/docs/testing/user/configguide/trafficgen.rst @@ -44,7 +44,8 @@ and is configured as follows: 'stream_type' : 'L4', 'pre_installed_flows' : 'No', # used by vswitch implementation 'flow_type' : 'port', # used by vswitch implementation - + 'flow_control' : False, # supported only by IxNet + 'learning_frames' : True, # supported only by IxNet 'l2': { 'framesize': 64, 'srcmac': '00:00:00:00:00:00', @@ -67,6 +68,13 @@ and is configured as follows: 'priority': 0, 'cfi': 0, }, + 'capture': { + 'enabled': False, + 'tx_ports' : [0], + 'rx_ports' : [1], + 'count': 1, + 'filter': '', + }, } The framesize parameter can be overridden from the configuration @@ -783,6 +791,10 @@ It is neccesary for proper connection between Trex server and VSPERF. cd trex-core/scripts/ ./t-rex-64 -i +**NOTE:** Please check your firewall settings at both DUT and T-Rex server. +Firewall must allow a connection from DUT (VSPERF) to the T-Rex server running +at TCP port 4501. + For additional information about Trex stateless mode see Trex stateless documentation: https://trex-tgn.cisco.com/trex/doc/trex_stateless.html @@ -826,6 +838,15 @@ Default value of this parameter is defined in conf/03_traffic.conf as follows: TRAFFICGEN_TREX_RFC2544_TPUT_THRESHOLD = '' +T-Rex can have learning packets enabled. For certain tests it may be beneficial +to send some packets before starting test traffic to allow switch learning to take +place. This can be adjusted with the following configurations: + +.. code-block:: console + + TRAFFICGEN_TREX_LEARNING_MODE=True + TRAFFICGEN_TREX_LEARNING_DURATION=5 + SR-IOV and Multistream layer 2 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ T-Rex by default only accepts packets on the receive side if the destination mac matches the @@ -840,3 +861,23 @@ modified. Enable Promiscuous mode when doing multistream at layer 2 testing with .. code-block:: console TRAFFICGEN_TREX_PROMISCUOUS=True + +RFC2544 Validation +~~~~~~~~~~~~~~~~~~ + +T-Rex can perform a verification run for a longer duration once the binary search of the +RFC2544 trials have completed. This duration should be at least 60 seconds. This is similar +to other traffic generator functionality where a more sustained time can be attempted to +verify longer runs from the result of the search. This can be configured with the following +params + +.. code-block:: console + + TRAFFICGEN_TREX_VERIFICATION_MODE = False + TRAFFICGEN_TREX_VERIFICATION_DURATION = 60 + TRAFFICGEN_TREX_MAXIMUM_VERIFICATION_TRIALS = 10 + +The duration and maximum number of attempted verification trials can be set to change the +behavior of this step. If the verification step fails, it will resume the binary search +with new values where the maximum output will be the last attempted frame rate minus the +current set thresh hold. diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst index 64d91657..350fbe54 100644 --- a/docs/testing/user/userguide/index.rst +++ b/docs/testing/user/userguide/index.rst @@ -16,5 +16,6 @@ VSPERF Test Guide ./testusage.rst ./teststeps.rst ./integration.rst + ./trafficcapture.rst ./yardstick.rst ./testlist.rst diff --git a/docs/testing/user/userguide/teststeps.rst b/docs/testing/user/userguide/teststeps.rst index 8be67310..08c95311 100644 --- a/docs/testing/user/userguide/teststeps.rst +++ b/docs/testing/user/userguide/teststeps.rst @@ -131,6 +131,21 @@ of supported objects and their most common functions follows: ['vnf2', 'stop'], ['vnf1', 'stop'], + * ``VNF[ID]`` - provides access to VNFs deployed automatically by testcase deployment + scenario. For Example ``pvvp`` deployment automatically starts two VNFs before any + TestStep is executed. It is possible to access these VNFs by VNF0 and VNF1 labels. + + List of supported functions is identical to ``vnf[ID]`` option above except functions + ``start`` and ``stop``. + + Examples: + + .. code-block:: python + + ['VNF0', 'execute_and_wait', 'ifconfig eth2 5.5.5.1/24 up'], + ['VNF1', 'execute_and_wait', 'ifconfig eth2 5.5.5.2/24 up', 120, 'root.*#'], + ['VNF2', 'execute_and_wait', 'ping -c1 5.5.5.1'], + * ``trafficgen`` - triggers traffic generation List of supported functions: @@ -256,7 +271,9 @@ of supported objects and their most common functions follows: in case that condition is not ``True`` * ``Eval expression`` - evaluates given expression as a python code and returns its result - * ``Exec_Shell command`` - executes a shell command + * ``Exec_Shell command`` - executes a shell command and wait until it finishes + * ``Exec_Shell_Background command`` - executes a shell command at background; + Command will be automatically terminated at the end of testcase execution. * ``Exec_Python code`` - executes a python code diff --git a/docs/testing/user/userguide/trafficcapture.rst b/docs/testing/user/userguide/trafficcapture.rst new file mode 100644 index 00000000..fa09bfed --- /dev/null +++ b/docs/testing/user/userguide/trafficcapture.rst @@ -0,0 +1,297 @@ +Traffic Capture +--------------- + +Tha ability to capture traffic at multiple points of the system is crucial to +many of the functional tests. It allows the verification of functionality for +both the vSwitch and the NICs using hardware acceleration for packet +manipulation and modification. + +There are three different methods of traffic capture supported by VSPERF. +Detailed descriptions of these methods as well as their pros and cons can be +found in the following chapters. + +Traffic Capture inside of a VM +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This method uses the standard PVP scenario, in which vSwitch first processes +and modifies the packet before forwarding it to the VM. Inside of the VM we +capture the traffic using **tcpdump** or a similiar technique. The capture +information is the used to verify the expected modifications to the packet done +by vSwitch. + +.. code-block:: console + + _ + +--------------------------------------------------+ | + | | | + | +------------------------------------------+ | | + | | Traffic capture and Packet Forwarding | | | + | +------------------------------------------+ | | + | ^ : | | + | | | | | Guest + | : v | | + | +---------------+ +---------------+ | | + | | logical port 0| | logical port 1| | | + +---+---------------+----------+---------------+---+ _| + ^ : + | | + : v _ + +---+---------------+----------+---------------+---+ | + | | logical port 0| | logical port 1| | | + | +---------------+ +---------------+ | | + | ^ : | | + | | | | | Host + | : v | | + | +--------------+ +--------------+ | | + | | phy port | vSwitch | phy port | | | + +---+--------------+------------+--------------+---+ _| + ^ : + | | + : v + +--------------------------------------------------+ + | | + | traffic generator | + | | + +--------------------------------------------------+ + +PROS: + +- supports testing with all traffic generators +- easy to use and implement into test +- allows testing hardware offloading on the ingress side + +CONS: + +- does not allow testing hardware offloading on the egress side + +An example of Traffic Capture in VM test: + +.. code-block:: python + + # Capture Example 1 - Traffic capture inside VM (PVP scenario) + # This TestCase will modify VLAN ID set by the traffic generator to the new value. + # Correct VLAN ID settings is verified by inspection of captured frames. + { + Name: capture_pvp_modify_vid, + Deployment: pvp, + Description: Test and verify VLAN ID modification by Open vSwitch, + Parameters : { + VSWITCH : OvsDpdkVhost, # works also for Vanilla OVS + TRAFFICGEN_DURATION : 5, + TRAFFIC : { + traffic_type : rfc2544_continuous, + frame_rate : 100, + 'vlan': { + 'enabled': True, + 'id': 8, + 'priority': 1, + 'cfi': 0, + }, + }, + GUEST_LOOPBACK : ['linux_bridge'], + }, + TestSteps: [ + # replace original flows with vlan ID modification + ['!vswitch', 'add_flow', 'br0', {'in_port': '1', 'actions': ['mod_vlan_vid:4','output:3']}], + ['!vswitch', 'add_flow', 'br0', {'in_port': '2', 'actions': ['mod_vlan_vid:4','output:4']}], + ['vswitch', 'dump_flows', 'br0'], + # verify that received frames have modified vlan ID + ['VNF0', 'execute_and_wait', 'tcpdump -i eth0 -c 5 -w dump.pcap vlan 4 &'], + ['trafficgen', 'send_traffic',{}], + ['!VNF0', 'execute_and_wait', 'tcpdump -qer dump.pcap vlan 4 2>/dev/null | wc -l','|^(\d+)$'], + ['tools', 'assert', '#STEP[-1][0] == 5'], + ], + }, + +Traffic Capture for testing NICs with HW offloading/acceleration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The NIC with hardware acceleration/offloading is inserted as an additional card +into the server. Two ports on this card are then connected together using +a patch cable as shown in the diagram. Only a single port of the tested NIC is +setup with DPDK acceleration, while the other is handled by the Linux Ip stack +allowing for traffic capture. The two NICs are then connected by vSwitch so the +original card can forward the processed packets to the traffic generator. The +ports handled by Linux IP stack allow for capturing packets, which are then +analyzed for changes done by both the vSwitch and the NIC with hardware +acceleration. + +.. code-block:: console + + _ + +------------------------------------------------+ | + | | | + | +----------------------------------------+ | | + | | vSwitch | | | + | | +----------------------------------+ | | | + | | | | | | | + | | | +------------------+ | | | | + | | | | | v | | | + | +----------------------------------------+ | | Device under Test + | ^ | ^ | | | + | | | | | | | + | | v | v | | + | +--------------+ +--------------+ | | + | | | | NIC w HW acc | | | + | | phy ports | | phy ports | | | + +---+--------------+----------+--------------+---+ _| + ^ : ^ : + | | | | + | | +-------+ + : v Patch Cable + +------------------------------------------------+ + | | + | traffic generator | + | | + +------------------------------------------------+ + +PROS: + +- allows testing hardware offloading on both the ingress and egress side +- supports testing with all traffic generators +- relatively easy to use and implement into tests + +CONS: + +- a more complex setup with two cards +- if the tested card only has one port, an additional card is needed + +An example of Traffic Capture for testing NICs with HW offloading test: + +.. code-block:: python + + # Capture Example 2 - Setup with 2 NICs, where traffic is captured after it is + # processed by NIC under the test (2nd NIC). See documentation for further details. + # This TestCase will strip VLAN headers from traffic sent by the traffic generator. + # The removal of VLAN headers is verified by inspection of captured frames. + # + # NOTE: This setup expects a DUT with two NICs with two ports each. First NIC is + # connected to the traffic generator (standard VSPERF setup). Ports of a second NIC + # are interconnected by a patch cable. PCI addresses of all four ports have to be + # properly configured in the WHITELIST_NICS parameter. + { + Name: capture_p2p2p_strip_vlan_ovs, + Deployment: clean, + Description: P2P Continuous Stream, + Parameters : { + _CAPTURE_P2P2P_OVS_ACTION : 'strip_vlan', + TRAFFIC : { + bidir : False, + traffic_type : rfc2544_continuous, + frame_rate : 100, + 'l2': { + 'srcmac': ca:fe:00:00:00:00, + 'dstmac': 00:00:00:00:00:01 + }, + 'vlan': { + 'enabled': True, + 'id': 8, + 'priority': 1, + 'cfi': 0, + }, + }, + # suppress DPDK configuration, so physical interfaces are not bound to DPDK driver + 'WHITELIST_NICS' : [], + 'NICS' : [], + }, + TestSteps: _CAPTURE_P2P2P_SETUP + [ + # capture traffic after processing by NIC under the test (after possible egress HW offloading) + ['tools', 'exec_shell_background', 'tcpdump -i [2][device] -c 5 -w capture.pcap ' + 'ether src [l2][srcmac]'], + ['trafficgen', 'send_traffic', {}], + ['vswitch', 'dump_flows', 'br0'], + ['vswitch', 'dump_flows', 'br1'], + # there must be 5 captured frames... + ['tools', 'exec_shell', 'tcpdump -r capture.pcap | wc -l', '|^(\d+)$'], + ['tools', 'assert', '#STEP[-1][0] == 5'], + # ...but no vlan headers + ['tools', 'exec_shell', 'tcpdump -r capture.pcap vlan | wc -l', '|^(\d+)$'], + ['tools', 'assert', '#STEP[-1][0] == 0'], + ], + }, + + +Traffic Capture on the Traffic Generator +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Using the functionality of the Traffic generator makes it possible to configure +Traffic Capture on both it's ports. With Traffic Capture enabled, VSPERF +instructs the Traffic Generator to automatically export captured data into +a pcap file. The captured packets are then sent to VSPERF for analysis and +verification, monitoring any changes done by both vSwitch and the NICs. + +Vsperf currently only supports this functionality with the **T-Rex** generator. + +.. code-block:: console + + _ + +--------------------------------------------------+ | + | | | + | +--------------------------+ | | + | | | | | + | | v | | Host + | +--------------+ +--------------+ | | + | | phy port | vSwitch | phy port | | | + +---+--------------+------------+--------------+---+ _| + ^ : + | | + : v + +--------------------------------------------------+ + | | + | traffic generator | + | | + +--------------------------------------------------+ + +PROS: + +- allows testing hardware offloading on both the ingress and egress side +- does not require an additional NIC + +CONS: + +- currently only supported by **T-Rex** traffic generator + +An example Traffic Capture on the Traffic Generator test: + +.. code-block:: python + + + # Capture Example 3 - Traffic capture by traffic generator. + # This TestCase uses OVS flow to add VLAN tag with given ID into every + # frame send by traffic generator. Correct frame modificaiton is verified by + # inspection of packet capture received by T-Rex. + { + Name: capture_p2p_add_vlan_ovs_trex, + Deployment: clean, + Description: OVS: Test VLAN tag modification and verify it by traffic capture, + vSwitch : OvsDpdkVhost, # works also for Vanilla OVS + Parameters : { + TRAFFICGEN : Trex, + TRAFFICGEN_DURATION : 5, + TRAFFIC : { + traffic_type : rfc2544_continuous, + frame_rate : 100, + # enable capture of five RX frames + 'capture': { + 'enabled': True, + 'tx_ports' : [], + 'rx_ports' : [1], + 'count' : 5, + }, + }, + }, + TestSteps : STEP_VSWITCH_P2P_INIT + [ + # replace standard L2 flows by flows, which will add VLAN tag with ID 3 + ['!vswitch', 'add_flow', 'int_br0', {'in_port': '1', 'actions': ['mod_vlan_vid:3','output:2']}], + ['!vswitch', 'add_flow', 'int_br0', {'in_port': '2', 'actions': ['mod_vlan_vid:3','output:1']}], + ['vswitch', 'dump_flows', 'int_br0'], + ['trafficgen', 'send_traffic', {}], + ['trafficgen', 'get_results'], + # verify that captured frames have vlan tag with ID 3 + ['tools', 'exec_shell', 'tcpdump -qer /#STEP[-1][0][capture_rx] vlan 3 ' + '2>/dev/null | wc -l', '|^(\d+)$'], + # number of received frames with expected VLAN id must match the number of captured frames + ['tools', 'assert', '#STEP[-1][0] == 5'], + ] + STEP_VSWITCH_P2P_FINIT, + }, + |