diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/configguide/trafficgen.rst | 196 | ||||
| -rw-r--r-- | docs/design/trafficgen_integration_guide.rst | 20 | ||||
| -rwxr-xr-x | docs/design/vswitchperf_design.rst | 124 | ||||
| -rw-r--r-- | docs/release/NEWS.rst | 55 | ||||
| -rw-r--r-- | docs/requirements/vswitchperf_ltd.rst | 48 | ||||
| -rwxr-xr-x | docs/userguide/integration.rst | 10 | ||||
| -rw-r--r-- | docs/userguide/teststeps.rst | 26 | ||||
| -rwxr-xr-x | docs/userguide/testusage.rst | 85 | ||||
| -rwxr-xr-x | docs/userguide/yardstick.rst | 29 |
9 files changed, 445 insertions, 148 deletions
diff --git a/docs/configguide/trafficgen.rst b/docs/configguide/trafficgen.rst index 6d75a56f..5190bc8e 100644 --- a/docs/configguide/trafficgen.rst +++ b/docs/configguide/trafficgen.rst @@ -7,15 +7,15 @@ =========================== Overview ---------------------- +-------- + VSPERF supports the following traffic generators: - * Dummy (DEFAULT): Allows you to use your own external - traffic generator. - * IXIA (IxNet and IxOS) - * Spirent TestCenter - * Xena Networks - * MoonGen + * Dummy_ (DEFAULT) + * Ixia_ + * `Spirent TestCenter`_ + * `Xena Networks`_ + * MoonGen_ To see the list of traffic gens from the cli: @@ -28,20 +28,27 @@ and configure the various traffic generators. Background Information ---------------------- -The traffic default configuration can be found in -tools/pkt_gen/trafficgen/trafficgenhelper.py, and is configured as -follows: +The traffic default configuration can be found in **conf/03_traffic.conf**, +and is configured as follows: .. code-block:: console - TRAFFIC_DEFAULTS = { + TRAFFIC = { + 'traffic_type' : 'rfc2544_throughput', + 'frame_rate' : 100, + 'bidir' : 'True', # will be passed as string in title format to tgen + 'multistream' : 0, + 'stream_type' : 'L4', + 'pre_installed_flows' : 'No', # used by vswitch implementation + 'flow_type' : 'port', # used by vswitch implementation + 'l2': { 'framesize': 64, 'srcmac': '00:00:00:00:00:00', 'dstmac': '00:00:00:00:00:00', }, 'l3': { - 'proto': 'tcp', + 'proto': 'udp', 'srcip': '1.1.1.1', 'dstip': '90.90.90.90', }, @@ -80,12 +87,25 @@ commandline above to: $ ./vsperf --test-params "TRAFFICGEN_PKT_SIZES=(x,y);TRAFFICGEN_DURATION=10;" \ "TRAFFICGEN_RFC2544_TESTS=1" $TESTNAME -Dummy Setup ------------- +Dummy +----- + +The Dummy traffic generator can be used to test VSPERF installation or +to demonstrate VSPERF functionality at DUT without connection +to a real traffic generator. + +You could also use the Dummy generator in case, that your external +traffic generator is not supported by VSPERF. In such case you could +use VSPERF to setup your test scenario and then transmit the traffic. +After the transmission is completed you could specify values for all +collected metrics and VSPERF will use them to generate final reports. + +Setup +~~~~~ + To select the Dummy generator please add the following to your custom configuration file ``10_custom.conf``. - .. code-block:: console TRAFFICGEN = 'Dummy' @@ -133,8 +153,8 @@ when the setup is complete. } What was the result for 'frames tx'? -When your traffic gen has completed traffic transmission and provided -the results please input these at the vsperf prompt. vsperf will try +When your traffic generator has completed traffic transmission and provided +the results please input these at the VSPERF prompt. VSPERF will try to verify the input: .. code-block:: console @@ -143,72 +163,134 @@ to verify the input: Please answer with y OR n. -VPSERF will ask you for: - * Result for 'frames tx' - * Result for 'frames rx' - * Result for 'min latency' - * Result for 'max latency' - * Result for 'avg latency' - +VSPERF will ask you to provide a value for every of collected metrics. The list +of metrics can be found at traffic-type-metrics_. Finally vsperf will print out the results for your test and generate the -appropriate logs and csv files. +appropriate logs and report files. +.. _traffic-type-metrics: -IXIA Setup ----------- +Metrics collected for supported traffic types +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -On the CentOS 7 system -~~~~~~~~~~~~~~~~~~~~~~ +Below you could find a list of metrics collected by VSPERF for each of supported +traffic types. + +RFC2544 Throughput and Continuous: + + * frames tx + * frames rx + * min latency + * max latency + * avg latency + * frameloss + +RFC2544 Back2back: + + * b2b frames + * b2b frame loss % + +Dummy result pre-configuration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In case of a Dummy traffic generator it is possible to pre-configure the test +results. This is useful for creation of demo testcases, which do not require +a real traffic generator. Such testcase can be run by any user and it will still +generate all reports and result files. + +Result values can be specified within ``TRAFFICGEN_DUMMY_RESULTS`` dictionary, +where every of collected metrics must be properly defined. Please check the list +of traffic-type-metrics_. + +Dictionary with dummy results can be passed by CLI argument ``--test-params`` +or specified in ``Parameters`` section of testcase definition. -You need to install IxNetworkTclClient$(VER\_NUM)Linux.bin.tgz. +Example of testcase execution with dummy results defined by CLI argument: -On the IXIA client software system -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. code-block:: console + + $ ./vsperf back2back --trafficgen Dummy --test-params \ + "TRAFFICGEN_DUMMY_RESULTS={'b2b frames':'3000','b2b frame loss %':'0.0'}" + +Example of testcase definition with pre-configured dummy results: -Find the IxNetwork TCL server app (start -> All Programs -> IXIA -> -IxNetwork -> IxNetwork\_$(VER\_NUM) -> IxNetwork TCL Server) +.. code-block:: python -Right click on IxNetwork TCL Server, select properties - Under shortcut tab in -the Target dialogue box make sure there is the argument "-tclport xxxx" -where xxxx is your port number (take note of this port number as you will -need it for the 10\_custom.conf file). + { + "Name": "back2back", + "Traffic Type": "rfc2544_back2back", + "Deployment": "p2p", + "biDirectional": "True", + "Description": "LTD.Throughput.RFC2544.BackToBackFrames", + "Parameters" : { + 'TRAFFICGEN_DUMMY_RESULTS' : {'b2b frames':'3000','b2b frame loss %':'0.0'} + }, + }, -.. image:: TCLServerProperties.png +**NOTE:** Pre-configured results for the Dummy traffic generator will be used only +in case, that the Dummy traffic generator is used. Otherwise the option +``TRAFFICGEN_DUMMY_RESULTS`` will be ignored. -Hit Ok and start the TCL server application +.. _Ixia: + +Ixia +---- + +VSPERF can use both IxNetwork and IxExplorer TCL servers to control Ixia chassis. +However usage of IxNetwork TCL server is a preferred option. Following sections +will describe installation and configuration of IxNetwork components used by VSPERF. + +Installation +~~~~~~~~~~~~ + +On the system under the test you need to install IxNetworkTclClient$(VER\_NUM)Linux.bin.tgz. + +On the IXIA client software system you need to install IxNetwork TCL server. After its +installation you should configure it as follows: + + 1. Find the IxNetwork TCL server app (start -> All Programs -> IXIA -> + IxNetwork -> IxNetwork\_$(VER\_NUM) -> IxNetwork TCL Server) + 2. Right click on IxNetwork TCL Server, select properties - Under shortcut tab in + the Target dialogue box make sure there is the argument "-tclport xxxx" + where xxxx is your port number (take note of this port number as you will + need it for the 10\_custom.conf file). + + .. image:: TCLServerProperties.png + + 3. Hit Ok and start the TCL server application VSPERF configuration ~~~~~~~~~~~~~~~~~~~~ -There are several configuration options specific to the IxNetworks traffic generator +There are several configuration options specific to the IxNetwork traffic generator from IXIA. It is essential to set them correctly, before the VSPERF is executed for the first time. Detailed description of options follows: - * TRAFFICGEN_IXNET_MACHINE - IP address of server, where IxNetwork TCL Server is running - * TRAFFICGEN_IXNET_PORT - PORT, where IxNetwork TCL Server is accepting connections from + * ``TRAFFICGEN_IXNET_MACHINE`` - IP address of server, where IxNetwork TCL Server is running + * ``TRAFFICGEN_IXNET_PORT`` - PORT, where IxNetwork TCL Server is accepting connections from TCL clients - * TRAFFICGEN_IXNET_USER - username, which will be used during communication with IxNetwork + * ``TRAFFICGEN_IXNET_USER`` - username, which will be used during communication with IxNetwork TCL Server and IXIA chassis - * TRAFFICGEN_IXIA_HOST - IP address of IXIA traffic generator chassis - * TRAFFICGEN_IXIA_CARD - identification of card with dedicated ports at IXIA chassis - * TRAFFICGEN_IXIA_PORT1 - identification of the first dedicated port at TRAFFICGEN_IXIA_CARD + * ``TRAFFICGEN_IXIA_HOST`` - IP address of IXIA traffic generator chassis + * ``TRAFFICGEN_IXIA_CARD`` - identification of card with dedicated ports at IXIA chassis + * ``TRAFFICGEN_IXIA_PORT1`` - identification of the first dedicated port at ``TRAFFICGEN_IXIA_CARD`` at IXIA chassis; VSPERF uses two separated ports for traffic generation. In case of unidirectional traffic, it is essential to correctly connect 1st IXIA port to the 1st NIC - at DUT, i.e. to the first PCI handle from WHITELIST_NICS list. Otherwise traffic may not + at DUT, i.e. to the first PCI handle from ``WHITELIST_NICS`` list. Otherwise traffic may not be able to pass through the vSwitch. - * TRAFFICGEN_IXIA_PORT2 - identification of the second dedicated port at TRAFFICGEN_IXIA_CARD + * ``TRAFFICGEN_IXIA_PORT2`` - identification of the second dedicated port at ``TRAFFICGEN_IXIA_CARD`` at IXIA chassis; VSPERF uses two separated ports for traffic generation. In case of unidirectional traffic, it is essential to correctly connect 2nd IXIA port to the 2nd NIC - at DUT, i.e. to the second PCI handle from WHITELIST_NICS list. Otherwise traffic may not + at DUT, i.e. to the second PCI handle from ``WHITELIST_NICS`` list. Otherwise traffic may not be able to pass through the vSwitch. - * TRAFFICGEN_IXNET_LIB_PATH - path to the DUT specific installation of IxNetwork TCL API - * TRAFFICGEN_IXNET_TCL_SCRIPT - name of the TCL script, which VSPERF will use for + * ``TRAFFICGEN_IXNET_LIB_PATH`` - path to the DUT specific installation of IxNetwork TCL API + * ``TRAFFICGEN_IXNET_TCL_SCRIPT`` - name of the TCL script, which VSPERF will use for communication with IXIA TCL server - * TRAFFICGEN_IXNET_TESTER_RESULT_DIR - folder accessible from IxNetwork TCL server, + * ``TRAFFICGEN_IXNET_TESTER_RESULT_DIR`` - folder accessible from IxNetwork TCL server, where test results are stored, e.g. ``c:/ixia_results``; see test-results-share_ - * TRAFFICGEN_IXNET_DUT_RESULT_DIR - directory accessible from the DUT, where test + * ``TRAFFICGEN_IXNET_DUT_RESULT_DIR`` - directory accessible from the DUT, where test results from IxNetwork TCL server are stored, e.g. ``/mnt/ixia_results``; see test-results-share_ @@ -237,7 +319,7 @@ Example of sharing configuration: TRAFFICGEN_IXNET_TESTER_RESULT_DIR = 'c:/ixia_results' TRAFFICGEN_IXNET_DUT_RESULT_DIR = '/mnt/ixia_results' - Note: It is essential to use slashes '/' also in path + **NOTE:** It is essential to use slashes '/' also in path configured by ``TRAFFICGEN_IXNET_TESTER_RESULT_DIR`` parameter. * Install cifs-utils package. @@ -259,6 +341,8 @@ Example of sharing configuration: It is recommended to verify, that any new file inserted into ``c:/ixia_results`` folder is visible at DUT inside ``/mnt/ixia_results`` directory. +.. _`Spirent TestCenter`: + Spirent Setup ------------- @@ -360,6 +444,8 @@ The mandatory configurations are enlisted below. TRAFFICGEN_STC_RFC2889_TEST_FILE_NAME = " " TRAFFICGEN_STC_RFC2889_LOCATIONS= " " +.. _`Xena Networks`: + Xena Networks ------------- diff --git a/docs/design/trafficgen_integration_guide.rst b/docs/design/trafficgen_integration_guide.rst index 9bb0e825..1457052b 100644 --- a/docs/design/trafficgen_integration_guide.rst +++ b/docs/design/trafficgen_integration_guide.rst @@ -81,29 +81,28 @@ Step 3 - configuration All configuration values, required for correct traffic generator function, are passed from VSPERF to the traffic generator in a dictionary. Default values shared among -all traffic generators are defined in **tools/pkt_gen/trafficgen/trafficgenhelper.py** -as **TRAFFIC_DEFAULTS** dictionary. Default values are loaded by **ITrafficGenerator** -interface class automatically, so it is not needed to load them explicitly. In case -that there are any traffic generator specific default values, then they should -be set within class specific **__init__** function. +all traffic generators are defined in **conf/03_traffic.conf** within **TRAFFIC** +dictionary. Default values are loaded by **ITrafficGenerator** interface class +automatically, so it is not needed to load them explicitly. In case that there are +any traffic generator specific default values, then they should be set within class +specific **__init__** function. VSPERF passes test specific configuration within **traffic** dictionary to every start and send function. So implementation of these functions must ensure, that default values are updated with the testcase specific values. Proper merge -of values is assured by call of **merge_spec** function from **trafficgenhelper** -module. +of values is assured by call of **merge_spec** function from **conf** module. Example of **merge_spec** usage in **tools/pkt_gen/sample_tg/sample_tg.py** module: .. code-block:: python - from tools.pkt_gen.trafficgen.trafficgenhelper import merge_spec + from conf import merge_spec def start_rfc2544_throughput(self, traffic=None, duration=30): self._params = {} self._params['traffic'] = self.traffic_defaults.copy() if traffic: - self._params['traffic'] = trafficgen.merge_spec( + self._params['traffic'] = merge_spec( self._params['traffic'], traffic) @@ -199,8 +198,7 @@ functions: e.g. **rfc2544_throughput**, **rfc2544_continuous** or **rfc2544_back2back**. * param **frame_rate**: Defines desired percentage of frame - rate used during continuous stream tests. It can be set by test - parameter iLoad or by CLI parameter iload. + rate used during continuous stream tests. * param **bidir**: Specifies if generated traffic will be full-duplex (true) or half-duplex (false). * param **multistream**: Defines number of flows simulated by traffic diff --git a/docs/design/vswitchperf_design.rst b/docs/design/vswitchperf_design.rst index 4f33a99f..96b97631 100755 --- a/docs/design/vswitchperf_design.rst +++ b/docs/design/vswitchperf_design.rst @@ -263,6 +263,130 @@ a section in the ``conf\10_custom.conf`` file that can be used. .. _VSPERF installation scripts: http://artifacts.opnfv.org/vswitchperf/docs/configguide/installation.html#other-requirements +Configuration of TRAFFIC dictionary +----------------------------------- + +TRAFFIC dictionary is used for configuration of traffic generator. Default values +can be found in configuration file ``conf/03_traffic.conf``. These default values +can be modified by (first option has the highest priorty): + + 1. ``Parameters`` section of testcase defintion + 2. command line options specified by ``--test-params`` argument + 3. custom configuration file + +It is to note, that in case of option 1 and 2, it is possible to specify only +values, which should be changed. In case of custom configuration file, it is +required to specify whole ``TRAFFIC`` dictionary with its all values or explicitly +call and update() method of ``TRAFFIC`` dictionary. + +Detailed description of ``TRAFFIC`` dictionary items follows: + +.. code-block:: console + + 'traffic_type' - One of the supported traffic types. + E.g. rfc2544_throughput, rfc2544_back2back + or rfc2544_continuous + Data type: str + Default value: "rfc2544_throughput". + 'bidir' - Specifies if generated traffic will be full-duplex (True) + or half-duplex (False) + Data type: str + Supported values: "True", "False" + Default value: "False". + 'frame_rate' - Defines desired percentage of frame rate used during + continuous stream tests. + Data type: int + Default value: 100. + 'multistream' - Defines number of flows simulated by traffic generator. + Value 0 disables multistream feature + Data type: int + Supported values: 0-65535 + Default value: 0. + 'stream_type' - Stream type is an extension of the "multistream" feature. + If multistream is disabled, then stream type will be + ignored. Stream type defines ISO OSI network layer used + for simulation of multiple streams. + Data type: str + Supported values: + "L2" - iteration of destination MAC address + "L3" - iteration of destination IP address + "L4" - iteration of destination port + of selected transport protocol + Default value: "L4". + 'pre_installed_flows' + - Pre-installed flows is an extension of the multistream" + feature. If multistream is disabled, then pre-installed + flows will be ignored. It defines if stream specific flows + will be inserted into OVS or not. + Data type: str + Supported values: + "Yes" - flows will be inserted into OVS + "No" - flows won't be inserted into OVS + Default value: "No". + 'flow_type' - Defines flows complexity. + Data type: str + Supported values: + "port" - flow is defined by ingress ports + "IP" - flow is defined by ingress ports + and src and dst IP addresses + Default value: "port" + 'l2' - A dictionary with l2 network layer details. Supported + values are: + 'srcmac' - Specifies source MAC address filled by traffic generator. + NOTE: It can be modified by vsperf in some scenarios. + Data type: str + Default value: "00:00:00:00:00:00". + 'dstmac' - Specifies destination MAC address filled by traffic generator. + NOTE: It can be modified by vsperf in some scenarios. + Data type: str + Default value: "00:00:00:00:00:00". + 'framesize' - Specifies default frame size. This value should not be + changed directly. It will be overridden during testcase + execution by values specified by list TRAFFICGEN_PKT_SIZES. + Data type: int + Default value: 64 + 'l3' - A dictionary with l3 network layer details. Supported + values are: + 'srcip' - Specifies source MAC address filled by traffic generator. + NOTE: It can be modified by vsperf in some scenarios. + Data type: str + Default value: "1.1.1.1". + 'dstip' - Specifies destination MAC address filled by traffic generator. + NOTE: It can be modified by vsperf in some scenarios. + Data type: str + Default value: "90.90.90.90". + 'proto' - Specifies deflaut protocol type. + Please check particular traffic generator implementation + for supported protocol types. + Data type: str + Default value: "udp". + 'l4' - A dictionary with l4 network layer details. Supported + values are: + 'srcport' - Specifies source port of selected transport protocol. + NOTE: It can be modified by vsperf in some scenarios. + Data type: int + Default value: 3000 + 'dstport' - Specifies destination port of selected transport protocol. + NOTE: It can be modified by vsperf in some scenarios. + Data type: int + Default value: 3001 + 'vlan' - A dictionary with vlan encapsulation details. Supported + values are: + 'enabled' - Specifies if vlan encapsulation should be enabled or + disabled. + Data type: bool + Default value: False + 'id' - Specifies vlan id. + Data type: int (NOTE: must fit to 12 bits) + Default value: 0 + 'priority' - Specifies a vlan priority (PCP header field). + Data type: int (NOTE: must fit to 3 bits) + Default value: 0 + 'cfi' - Specifies if frames can or cannot be dropped during + congestion (DEI header field). + Data type: int (NOTE: must fit to 1 bit) + Default value: 0 + Configuration of GUEST options ------------------------------ diff --git a/docs/release/NEWS.rst b/docs/release/NEWS.rst index e1a9fa3b..a31e2738 100644 --- a/docs/release/NEWS.rst +++ b/docs/release/NEWS.rst @@ -2,14 +2,56 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) OPNFV, Intel Corporation, AT&T and others. -OPNFV D Release -=============== -* Remove support for vhost cuse -* Add Vanilla OVS Multi-queue with non testpmd options -* Add support for Multi-queue with OVS 2.5.0 or less +OPNFV Danube Release +==================== + +* Support for testpmd as a vswitch for PVP scenario with vHost User +* Traffic type naming harmonized with RFC2544 +* Support for step driven performance testcases +* Scripts with licenses not compatible with Apache 2.0 were isolated + in 3rd_party directory +* Several bugfixes, CI script and documentation updates +* Installation scripts: + + * Support for Ubuntu 16.04 LTS and 16.10 + * Support for RHEL7.3 + * Support for CentOS7.3 + +* Traffic Generators: + + * Spirent Testcenter: Support for RFC2889 tests + * Xena: bugfixes and improvements of RFC2544 continuous accuracy + * MoonGen: bugfixes, code clean up and update of usage instructions + * Dummy: Support for preconfigured test results + * Ixia: bugfixes + +* Integration tests: + + * New tests for multi VM scenarios + * New test for numa vHost awareness feature + +* Configuration changes: + + * Support for OVS, DPDK or QEMU installed from binary packages + * Support for modification of any configuration parameter or traffic + detail via CLI option --test-params or via "Parameters" section + of testcase definition + +* Guest specific: + + * Support for multi VM scenarios with VM connected in serial or in parallel + * Support for VM with 1, 2, 4, 6... network interfaces + * Support for driver binding option + * Support for flexible testpmd configuration + * Support for configurable merge-buffers + * Support for configurable drive options + * Support for multi-queue with non testpmd options by Vanilla OVS + * Support for multi-queue with OVS 2.5.0 or less + * Remove support for vHost Cuse OPNFV Colorado Release ====================== + * Support for DPDK v16.07 * Support for yardstick testing framework * Support for stp/rstp configuration @@ -23,10 +65,12 @@ OPNFV Colorado Release * Support for Red Hat Enterprise Linux * Support for mode of operation (trafficgen, trafficgen-off) * Support for Integration tests for OVS with DPDK including: + * Physical ports. * Virtual ports (vhost user and vhost cuse). * Flow addition and removal tests. * Overlay (VXLAN, GRE and NVGRE) encapsulation and decapsulation tests. + * Supporting configuration of OVS with DPDK through the OVS DB as well as the legacy commandline arguments. * Support for VM loopback (SR-IOV) benchmarking. @@ -35,6 +79,7 @@ OPNFV Colorado Release OPNFV Brahmaputra Release ========================= + Supports both OVS and OVS with DPDK. Available tests: diff --git a/docs/requirements/vswitchperf_ltd.rst b/docs/requirements/vswitchperf_ltd.rst index 0d297880..69497c5c 100644 --- a/docs/requirements/vswitchperf_ltd.rst +++ b/docs/requirements/vswitchperf_ltd.rst @@ -145,8 +145,8 @@ Test ID: LTD.Throughput.RFC2544.PacketLossRatio Note: Other values can be tested if required by the user. The selected frame sizes are those previously defined under `Default - Test Parameters <#DefaultParams>`__. The test can also be used to - determine the average latency of the traffic. + Test Parameters <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__. + The test can also be used to determine the average latency of the traffic. Under the `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__ test methodology, the test duration will @@ -197,8 +197,8 @@ Test ID: LTD.Throughput.RFC2544.PacketLossRatioFrameModification Note: Other values can be tested if required by the user. The selected frame sizes are those previously defined under `Default - Test Parameters <#DefaultParams>`__. The test can also be used to - determine the average latency of the traffic. + Test Parameters <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__. + The test can also be used to determine the average latency of the traffic. Under the `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__ test methodology, the test duration will @@ -268,7 +268,7 @@ Test ID: LTD.Throughput.RFC2544.Profile and severe. The selected frame sizes are those previously defined under `Default - Test Parameters <#DefaultParams>`__. + Test Parameters <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__. The offered traffic rate is described as a percentage delta with respect to the DUT's RFC 2544 Throughput as determined by @@ -319,7 +319,8 @@ Test ID: LTD.Throughput.RFC2544.SystemRecoveryTime The aim of this test is to determine the length of time it takes the DUT to recover from an overload condition for a constant load (fixed length frames at a fixed interval time). The selected frame sizes are those - previously defined under `Default Test Parameters <#DefaultParams>`__, + previously defined under `Default Test Parameters + <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__, traffic should be sent to the DUT under normal conditions. During the duration of the test and while the traffic flows are passing though the DUT, at least one situation leading to an overload condition for the DUT @@ -371,7 +372,9 @@ Test ID: LTD.Throughput.RFC2544.BackToBackFrames The aim of this test is to characterize the ability of the DUT to process back-to-back frames. For each frame size previously defined - under `Default Test Parameters <#DefaultParams>`__, a burst of traffic + under `Default Test Parameters + <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__, + a burst of traffic is sent to the DUT with the minimum inter-frame gap between each frame. If the number of received frames equals the number of frames that were transmitted, the burst size should be increased and traffic is sent to @@ -525,7 +528,8 @@ Test ID: LTD.Throughput.RFC6201.ResetTime Both reset methods SHOULD be exercised. For each frame size previously defined under `Default Test - Parameters <#DefaultParams>`__, traffic should be sent to the DUT under + Parameters <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__, + traffic should be sent to the DUT under normal conditions. During the duration of the test and while the traffic flows are passing through the DUT, the DUT should be reset and the Reset time measured. The Reset time is the total time that a device is @@ -610,7 +614,8 @@ Test ID: LTD.Throughput.RFC2889.MaxForwardingRate is varied between the throughput and the Maximum Offered Load for fixed length frames at a fixed time interval. The selected frame sizes are those previously defined under `Default Test - Parameters <#DefaultParams>`__. The throughput is the maximum offered + Parameters <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__. + The throughput is the maximum offered load with 0% frame loss (measured by the prerequisite test), and the Maximum Offered Load (as defined by `RFC2285 <https://www.rfc-editor.org/rfc/rfc2285.txt>`__) is *"the highest @@ -744,7 +749,8 @@ Test ID: LTD.Throughput.RFC2889.BroadcastFrameForwarding The aim of this test is to determine the maximum forwarding rate of the DUT when forwarding broadcast traffic. For each frame previously defined - under `Default Test Parameters <#DefaultParams>`__, the traffic should + under `Default Test Parameters <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__, + the traffic should be set up as broadcast traffic. The traffic throughput of the DUT should be measured. @@ -912,7 +918,7 @@ Test ID: LTD.Throughput.Overlay.Network.<tech>.RFC2544.PacketLossRatio - Switch the packet to the correct port The selected frame sizes are those previously defined under `Default - Test Parameters <#DefaultParams>`__. + Test Parameters <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__. Thus, each test comprises an overlay technology, a network function, and a packet size *with* overlay network overhead included @@ -1035,8 +1041,8 @@ Test ID: LTD.Throughput.RFC2544.MatchAction.PacketLossRatio The default loss percentages to be tested are: - X = 0% - X = 10^-7% Other values can be tested if required by the user. The selected - frame sizes are those previously defined under Default Test - Parameters. + frame sizes are those previously defined under `Default Test Parameters + <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__. The test can also be used to determine the average latency of the traffic when a match action is applied to packets in a flow. Under @@ -1202,8 +1208,8 @@ Test ID: LTD.Scalability.Flows.RFC2544.0PacketLoss before passing traffic. For each frame size previously defined under `Default Test - Parameters <#DefaultParams>`__ and for each of the following number of - flows: + Parameters <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__ + and for each of the following number of flows: - 1,000 - 2,000 @@ -1331,8 +1337,8 @@ Test ID: LTD.Scalability.VNF.RFC2544.PacketLossRatio approach used needs to be documented as part of the test report. The selected frame sizes are those previously defined under `Default - Test Parameters <#DefaultParams>`__. The test can also be used to - determine the average latency of the traffic. + Test Parameters <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__. + The test can also be used to determine the average latency of the traffic. Under the `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__ test methodology, the test duration will @@ -1392,7 +1398,7 @@ Test ID: LTD.Scalability.VNF.RFC2544.PacketLossProfile be tested is 3. The selected frame sizes are those previously defined under `Default - Test Parameters <#DefaultParams>`__. + Test Parameters <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__. The offered traffic rate is described as a percentage delta with respect to the DUT's RFC 2544 Throughput as determined by @@ -1473,7 +1479,8 @@ Test ID: LTD.Activation.RFC2889.AddressCachingCapacity MAC learning. The aim of this test is to determine the address caching capacity of the DUT for a constant load (fixed length frames at a fixed interval time). The selected frame sizes are those previously defined - under `Default Test Parameters <#DefaultParams>`__. + under `Default Test Parameters + <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__. In order to run this test the aging time, that is the maximum time the DUT will keep a learned address in its flow table, and a set of initial @@ -1526,7 +1533,8 @@ Test ID: LTD.Activation.RFC2889.AddressLearningRate MAC learning. The aim of this test is to determine the rate of address learning of the DUT for a constant load (fixed length frames at a fixed interval time). The selected frame sizes are those previously defined - under `Default Test Parameters <#DefaultParams>`__, traffic should be + under `Default Test Parameters <http://artifacts.opnfv.org/vswitchperf/docs/index.html#default-test-parameters>`__, + traffic should be sent with each IPv4/IPv6 address incremented by one. The rate at which the DUT learns a new address should be measured. The maximum caching capacity from LTD.Memory.RFC2889.AddressCachingCapacity should be taken diff --git a/docs/userguide/integration.rst b/docs/userguide/integration.rst index 003e8adb..60ed9245 100755 --- a/docs/userguide/integration.rst +++ b/docs/userguide/integration.rst @@ -82,21 +82,21 @@ To run VXLAN encapsulation tests: .. code-block:: console ./vsperf --conf-file user_settings.py --integration \ - --test-params 'tunnel_type=vxlan' overlay_p2p_tput + --test-params 'TUNNEL_TYPE=vxlan' overlay_p2p_tput To run GRE encapsulation tests: .. code-block:: console ./vsperf --conf-file user_settings.py --integration \ - --test-params 'tunnel_type=gre' overlay_p2p_tput + --test-params 'TUNNEL_TYPE=gre' overlay_p2p_tput To run GENEVE encapsulation tests: .. code-block:: console ./vsperf --conf-file user_settings.py --integration \ - --test-params 'tunnel_type=geneve' overlay_p2p_tput + --test-params 'TUNNEL_TYPE=geneve' overlay_p2p_tput To run OVS NATIVE tunnel tests (VXLAN/GRE/GENEVE): @@ -128,7 +128,7 @@ To run OVS NATIVE tunnel tests (VXLAN/GRE/GENEVE): .. code-block:: console ./vsperf --conf-file user_settings.py --integration \ - --test-params 'tunnel_type=vxlan' overlay_p2p_tput + --test-params 'TUNNEL_TYPE=vxlan' overlay_p2p_tput Executing VXLAN decapsulation tests @@ -189,7 +189,7 @@ To run GRE decapsulation tests: .. code-block:: console - ./vsperf --conf-file user_settings.py --test-params 'tunnel_type=gre' \ + ./vsperf --conf-file user_settings.py --test-params 'TUNNEL_TYPE=gre' \ --integration overlay_p2p_decap_cont diff --git a/docs/userguide/teststeps.rst b/docs/userguide/teststeps.rst index 5e2d9570..5029f538 100644 --- a/docs/userguide/teststeps.rst +++ b/docs/userguide/teststeps.rst @@ -374,8 +374,12 @@ That is accomplished by using "Stream Type" and "MultiStream" keywords. "Name": "multistream_l4", "Description": "Multistream on UDP ports", "Deployment": "clean", - "Stream Type": "L4", - "MultiStream": 4, + "Parameters": { + 'TRAFFIC' : { + "multistream": 4, + "stream_type": "L4", + }, + }, "TestSteps": [ ['vswitch', 'add_switch', 'int_br0'], # STEP 0 ['vswitch', 'add_phy_port', 'int_br0'], # STEP 1 @@ -542,8 +546,12 @@ destination UDP port. "Name": "ex_2pvp_rule_l4dp", "Description": "2 PVP with flows on L4 Dest Port", "Deployment": "clean", - "Stream Type": "L4", # loop UDP ports - "MultiStream": 2, + "Parameters": { + 'TRAFFIC' : { + "multistream": 2, + "stream_type": "L4", + }, + }, "TestSteps": [ ['vswitch', 'add_switch', 'int_br0'], # STEP 0 ['vswitch', 'add_phy_port', 'int_br0'], # STEP 1 @@ -622,12 +630,14 @@ and available in both csv and rst report files. { "Name": "pvvp_pvp_cont", - "Traffic Type": "continuous", "Deployment": "pvvp", "Description": "PVVP and PVP in parallel with Continuous Stream", - "biDirectional": "True", - "iLoad": "100", - "MultiStream": "2", + "Parameters" : { + "TRAFFIC" : { + "traffic_type" : "rfc2544_continuous", + "multistream": 2, + }, + }, "TestSteps": [ ['vswitch', 'add_vport', 'br0'], ['vswitch', 'add_vport', 'br0'], diff --git a/docs/userguide/testusage.rst b/docs/userguide/testusage.rst index adbe603b..721fd635 100755 --- a/docs/userguide/testusage.rst +++ b/docs/userguide/testusage.rst @@ -638,7 +638,7 @@ or use ``--vswitch`` and ``--fwdapp`` CLI arguments: .. code-block:: console - $ ./vsperf --conf-file user_settings.py \ + $ ./vsperf phy2phy_cont --conf-file user_settings.py \ --vswitch none \ --fwdapp TestPMD @@ -672,7 +672,57 @@ Supported Packet Forwarding applications are: .. code-block:: console - $ ./vsperf --conf-file <path_to_settings_py> + $ ./vsperf phy2phy_tput --conf-file <path_to_settings_py> + +Executing Packet Forwarding tests with one guest +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +TestPMD with DPDK 16.11 or greater can be used to forward packets as a switch to a single guest using TestPMD vdev +option. To set this configuration the following parameters should be used. + + .. code-block:: python + + VSWITCH = 'none' + PKTFWD = 'TestPMD' + +or use ``--vswitch`` and ``--fwdapp`` CLI arguments: + + .. code-block:: console + + $ ./vsperf pvp_tput --conf-file user_settings.py \ + --vswitch none \ + --fwdapp TestPMD + +Guest forwarding application only supports TestPMD in this configuration. + + .. code-block:: python + + GUEST_LOOPBACK = ['testpmd'] + +For optimal performance one cpu per port +1 should be used for TestPMD. Also set additional params for packet forwarding +application to use the correct number of nb-cores. + + .. code-block:: python + + VSWITCHD_DPDK_ARGS = ['-l', '46,44,42,40,38', '-n', '4', '--socket-mem 1024,0'] + TESTPMD_ARGS = ['--nb-cores=4', '--txq=1', '--rxq=1'] + +For guest TestPMD 3 VCpus should be assigned with the following TestPMD params. + + .. code-block:: python + + GUEST_TESTPMD_PARAMS = ['-l 0,1,2 -n 4 --socket-mem 1024 -- ' + '--burst=64 -i --txqflags=0xf00 ' + '--disable-hw-vlan --nb-cores=2 --txq=1 --rxq=1'] + +Execution of TestPMD can be run with the following command line + + .. code-block:: console + + ./vsperf pvp_tput --vswitch=none --fwdapp=TestPMD --conf-file <path_to_settings_py> + +**NOTE:** To achieve the best 0% loss numbers with rfc2544 throughput testing, other tunings should be applied to host +and guest such as tuned profiles and CPU tunings to prevent possible interrupts to worker threads. VSPERF modes of operation ^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -694,36 +744,19 @@ Mode of operation is driven by configuration parameter -m or --mode "trafficgen-pause" - execute vSwitch and VNF but wait before traffic transmission In case, that VSPERF is executed in "trafficgen" mode, then configuration -of traffic generator should be configured through ``--test-params`` option. -Supported CLI options useful for traffic generator configuration are: - -.. code-block:: console - - 'traffic_type' - One of the supported traffic types. E.g. - rfc2544_throughput, - rfc2544_back2back or rfc2544_continuous - Default value is "rfc2544_throughput". - 'bidirectional' - Specifies if generated traffic will be full-duplex (true) - or half-duplex (false) - Default value is "false". - 'iload' - Defines desired percentage of frame rate used during - continuous stream tests. - Default value is 100. - 'multistream' - Defines number of flows simulated by traffic generator. - Value 0 disables MultiStream feature - Default value is 0. - 'stream_type' - Stream Type is an extension of the "MultiStream" feature. - If MultiStream is disabled, then Stream Type will be - ignored. Stream Type defines ISO OSI network layer used - for simulation of multiple streams. - Default value is "L4". +of traffic generator can be modified through ``TRAFFIC`` dictionary passed to the +``--test-params`` option. It is not needed to specify all values of ``TRAFFIC`` +dictionary. It is sufficient to specify only values, which should be changed. +Detailed description of ``TRAFFIC`` dictionary can be found at +`Configuration of TRAFFIC dictionary +<http://artifacts.opnfv.org/vswitchperf/docs/index.html#configuration-of-traffic-dictionary>`__ Example of execution of VSPERF in "trafficgen" mode: .. code-block:: console $ ./vsperf -m trafficgen --trafficgen IxNet --conf-file vsperf.conf \ - --test-params "traffic_type=rfc2544_continuous;bidirectional=True;iload=60" + --test-params "TRAFFIC={'traffic_type':'rfc2544_continuous','bidir':'False','framerate':60}" Code change verification by pylint ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/docs/userguide/yardstick.rst b/docs/userguide/yardstick.rst index 985a25ad..c906df8e 100755 --- a/docs/userguide/yardstick.rst +++ b/docs/userguide/yardstick.rst @@ -146,14 +146,10 @@ Example of yaml file: type: Vsperf options: testname: 'p2p_rfc2544_throughput' - traffic_type: 'rfc2544_throughput' - frame_size '64' - bidirectional: 'True' - iload: 100 trafficgen_port1: 'eth1' trafficgen_port2: 'eth3' external_bridge: 'br-ex' - test_params: 'TRAFFICGEN_DURATION=30;' + test_params: 'TRAFFICGEN_DURATION=30;TRAFFIC={'traffic_type':'rfc2544_throughput}' conf_file: '~/vsperf-yardstick.conf' host: vsperf.demo @@ -182,19 +178,9 @@ Section **option** defines details of vswitchperf test scenario. Lot of options are identical to the vswitchperf parameters passed through ``--test-params`` argument. Following options are supported: -- **traffic_type** - specifies the type of traffic executed by traffic generator; - Valid values are ``rfc2544_throughput``, ``rfc2544_continuous`` and ``rfc2544_back2back``. - Default: ``rfc2544_throughput`` - **frame_size** - a packet size for which test should be executed; Multiple packet sizes can be tested by modification of Sequence runner section inside YAML definition. Default: '64' -- **bidirectional** - specifies if traffic will be uni (False) or bi-directional - (True); Default: False -- **iload** - specifies frame rate; Default: 100 -- **multistream** - specifies the number of simulated streams; Default: 0 (i.e. - multistream feature is disabled) -- **stream_type** - specifies network layer used for multistream simulation - the valid values are "L4", "L3" and "L2"; Default: 'L4' - **conf_file** - sets path to the vswitchperf configuration file, which will be uploaded to VM; Default: '~/vsperf-yardstick.conf' - **setup_script** - sets path to the setup script, which will be executed @@ -208,8 +194,10 @@ argument. Following options are supported: - **test_params** - specifies a string with a list of vsperf configuration parameters, which will be passed to the ``--test-params`` CLI argument; Parameters should be stated in the form of ``param=value`` and separated - by a semicolon. Please check VSPERF documentation for details about - available configuration parameters and their data types. + by a semicolon. Configuration of traffic generator is driven by ``TRAFFIC`` + dictionary, which can be also updated by values defined by ``test_params``. + Please check VSPERF documentation for details about available configuration + parameters and their data types. In case that both **test_params** and **conf_file** are specified, then values from **test_params** will override values defined in the configuration file. @@ -220,7 +208,7 @@ expected, that OVS runs at the same node, where the testcase is executed. In cas of more complex OpenStack installation or a need of additional OVS configuration, **setup_script** can be used. -Note: It is essential to specify a configuration for selected traffic generator. +**NOTE** It is essential to specify a configuration for selected traffic generator. In case, that standalone testcase is created, then traffic generator can be selected and configured directly in YAML file by **test_params**. On the other hand, if multiple testcases should be executed with the same traffic generator @@ -259,3 +247,8 @@ In case that any of defined metrics will be lower than defined value, then testcase will be marked as failed. Based on ``action`` policy, yardstick will either stop test execution (value ``assert``) or it will run next test (value ``monitor``). + +**NOTE** The throughput SLA (or any other SLA) cannot be set to a meaningful +value without knowledge of the server and networking environment, possibly +including prior testing in that environment to establish a baseline SLA level +under well-understood circumstances. |