diff options
Diffstat (limited to 'docs/testing')
| -rw-r--r-- | docs/testing/developer/devguide/design/trafficgen_integration_guide.rst | 17 | ||||
| -rw-r--r-- | docs/testing/developer/devguide/design/vswitchperf_design.rst | 40 | ||||
| -rw-r--r-- | docs/testing/user/configguide/index.rst | 1 | ||||
| -rw-r--r-- | docs/testing/user/configguide/tools.rst | 177 | ||||
| -rw-r--r-- | docs/testing/user/configguide/trafficgen.rst | 99 | ||||
| -rw-r--r-- | docs/testing/user/userguide/testusage.rst | 6 |
6 files changed, 333 insertions, 7 deletions
diff --git a/docs/testing/developer/devguide/design/trafficgen_integration_guide.rst b/docs/testing/developer/devguide/design/trafficgen_integration_guide.rst index c88b80ed..671c7fd8 100644 --- a/docs/testing/developer/devguide/design/trafficgen_integration_guide.rst +++ b/docs/testing/developer/devguide/design/trafficgen_integration_guide.rst @@ -199,13 +199,20 @@ functions: Note: There are parameters specific to testing of tunnelling protocols, which are discussed in detail at :ref:`integration-tests` userguide. + Note: A detailed description of the ``TRAFFIC`` dictionary can be found at + :ref:`configuration-of-traffic-dictionary`. + * param **traffic_type**: One of the supported traffic types, - e.g. **rfc2544_throughput**, **rfc2544_continuous** - or **rfc2544_back2back**. - * param **frame_rate**: Defines desired percentage of frame - rate used during continuous stream tests. + e.g. **rfc2544_throughput**, **rfc2544_continuous**, + **rfc2544_back2back** or **burst**. * param **bidir**: Specifies if generated traffic will be full-duplex (true) or half-duplex (false). + * param **frame_rate**: Defines desired percentage of frame + rate used during continuous stream tests. + * param **burst_size**: Defines a number of frames in the single burst, + which is sent by burst traffic type. Burst size is applied for each + direction, i.e. the total number of tx frames will be 2*burst_size + in case of bidirectional traffic. * param **multistream**: Defines number of flows simulated by traffic generator. Value 0 disables MultiStream feature. * param **stream_type**: Stream Type defines ISO OSI network layer @@ -224,6 +231,8 @@ functions: **dstport** and l4 on/off switch **enabled**. * param **vlan**: A dictionary with vlan specific parameters, e.g. **priority**, **cfi**, **id** and vlan on/off switch **enabled**. + * param **scapy**: A dictionary with definition of the frame content for both traffic + directions. The frame content is defined by a SCAPY notation. * param **tests**: Number of times the test is executed. * param **duration**: Duration of continuous test or per iteration duration diff --git a/docs/testing/developer/devguide/design/vswitchperf_design.rst b/docs/testing/developer/devguide/design/vswitchperf_design.rst index 96ffcf62..7fbde886 100644 --- a/docs/testing/developer/devguide/design/vswitchperf_design.rst +++ b/docs/testing/developer/devguide/design/vswitchperf_design.rst @@ -291,8 +291,8 @@ Detailed description of ``TRAFFIC`` dictionary items follows: .. code-block:: console 'traffic_type' - One of the supported traffic types. - E.g. rfc2544_throughput, rfc2544_back2back - or rfc2544_continuous + E.g. rfc2544_throughput, rfc2544_back2back, + rfc2544_continuous or burst Data type: str Default value: "rfc2544_throughput". 'bidir' - Specifies if generated traffic will be full-duplex (True) @@ -304,6 +304,12 @@ Detailed description of ``TRAFFIC`` dictionary items follows: continuous stream tests. Data type: int Default value: 100. + 'burst_size' - Defines a number of frames in the single burst, which is sent + by burst traffic type. Burst size is applied for each direction, + i.e. the total number of tx frames will be 2*burst_size in case of + bidirectional traffic. + Data type: int + Default value: 100. 'multistream' - Defines number of flows simulated by traffic generator. Value 0 disables multistream feature Data type: int @@ -439,6 +445,34 @@ Detailed description of ``TRAFFIC`` dictionary items follows: details. Data type: str Default value: '' + 'scapy' - A dictionary with definition of a frame content for both traffic + directions. The frame content is defined by a SCAPY notation. + NOTE: It is supported only by the T-Rex traffic generator. + Following keywords can be used to refer to the related parts of + the TRAFFIC dictionary: + Ether_src - refers to TRAFFIC['l2']['srcmac'] + Ether_dst - refers to TRAFFIC['l2']['dstmac'] + IP_proto - refers to TRAFFIC['l3']['proto'] + IP_PROTO - refers to upper case version of TRAFFIC['l3']['proto'] + IP_src - refers to TRAFFIC['l3']['srcip'] + IP_dst - refers to TRAFFIC['l3']['dstip'] + IP_PROTO_sport - refers to TRAFFIC['l4']['srcport'] + IP_PROTO_dport - refers to TRAFFIC['l4']['dstport'] + Dot1Q_prio - refers to TRAFFIC['vlan']['priority'] + Dot1Q_id - refers to TRAFFIC['vlan']['cfi'] + Dot1Q_vlan - refers to TRAFFIC['vlan']['id'] + '0' - A string with the frame definition for the 1st direction. + Data type: str + Default value: 'Ether(src={Ether_src}, dst={Ether_dst})/' + 'Dot1Q(prio={Dot1Q_prio}, id={Dot1Q_id}, vlan={Dot1Q_vlan})/' + 'IP(proto={IP_proto}, src={IP_src}, dst={IP_dst})/' + '{IP_PROTO}(sport={IP_PROTO_sport}, dport={IP_PROTO_dport})' + '1' - A string with the frame definition for the 2nd direction. + Data type: str + Default value: 'Ether(src={Ether_dst}, dst={Ether_src})/' + 'Dot1Q(prio={Dot1Q_prio}, id={Dot1Q_id}, vlan={Dot1Q_vlan})/' + 'IP(proto={IP_proto}, src={IP_dst}, dst={IP_src})/' + '{IP_PROTO}(sport={IP_PROTO_dport}, dport={IP_PROTO_sport})', .. _configuration-of-guest-options: @@ -786,7 +820,7 @@ ITrafficGenerator connect() disconnect() - send_burst_traffic(traffic, numpkts, time, framerate) + send_burst_traffic(traffic, time) send_cont_traffic(traffic, time, framerate) start_cont_traffic(traffic, time, framerate) diff --git a/docs/testing/user/configguide/index.rst b/docs/testing/user/configguide/index.rst index 83908a97..75a2082d 100644 --- a/docs/testing/user/configguide/index.rst +++ b/docs/testing/user/configguide/index.rst @@ -48,6 +48,7 @@ VSPERF Install and Configuration ./installation.rst ./upgrade.rst ./trafficgen.rst + ./tools.rst ================= VSPERF Test Guide diff --git a/docs/testing/user/configguide/tools.rst b/docs/testing/user/configguide/tools.rst new file mode 100644 index 00000000..907e86d2 --- /dev/null +++ b/docs/testing/user/configguide/tools.rst @@ -0,0 +1,177 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) OPNFV, Intel Corporation, Spirent, AT&T and others. + +.. _additional-tools-configuration: + +============================================= +'vsperf' Additional Tools Configuration Guide +============================================= + +Overview +-------- + +VSPERF supports the following categories additional tools: + + * `Infrastructure Metrics Collectors`_ + * `Load Generators`_ + * `L3 Cache Management`_ + +Under each category, there are one or more tools supported by VSPERF. +This guide provides the details of how to install (if required) +and configure the above mentioned tools. + +.. _`Infrastructure Metrics Collectors`: + +Infrastructure Metrics Collection +--------------------------------- + +VSPERF supports following two tools for collecting and reporting the metrics: + +* pidstat +* collectd + +*pidstat* is a command in linux systems, which is used for monitoring individual +tasks currently being managed by Linux kernel. In VSPERF this command is used to +monitor *ovs-vswitchd*, *ovsdb-server* and *kvm* processes. + +*collectd* is linux application that collects, stores and transfers various system +metrics. For every category of metrics, there is a separate plugin in collectd. For +example, CPU plugin and Interface plugin provides all the cpu metrics and interface +metrics, respectively. CPU metrics may include user-time, system-time, etc., whereas +interface metrics may include received-packets, dropped-packets, etc. + +Installation +^^^^^^^^^^^^ + +No installation is required for *pidstat*, whereas, collectd has to be installed +separately. For installation of collectd, we recommend to follow the process described +in *OPNFV-Barometer* project, which can be found here `Barometer-Euphrates <http://docs.opnfv.org/en/stable-euphrates/submodules/barometer/docs/release/userguide/feature.userguide.html#building-all-barometer-upstreamed-plugins-from-scratch>`_ or the most +recent release. + +VSPERF assumes that collectd is installed and configured to send metrics over localhost. +The metrics sent should be for the following categories: CPU, Processes, Interface, +OVS, DPDK, Intel-RDT. + +Configuration +^^^^^^^^^^^^^ + +The configuration file for the collectors can be found in **conf/05_collector.conf**. +*pidstat* specific configuration includes: + +* ``PIDSTAT_MONITOR`` - processes to be monitored by pidstat +* ``PIDSTAT_OPTIONS`` - options which will be passed to pidstat command +* ``PIDSTAT_SAMPLE_INTERVAL`` - sampling interval used by pidstat to collect statistics +* ``LOG_FILE_PIDSTAT`` - prefix of pidstat's log file + +The *collectd* configuration option includes: + +* ``COLLECTD_IP`` - IP address where collectd is running +* ``COLLECTD_PORT`` - Port number over which collectd is sending the metrics +* ``COLLECTD_SECURITY_LEVEL`` - Security level for receiving metrics +* ``COLLECTD_AUTH_FILE`` - Authentication file for receiving metrics +* ``LOG_FILE_COLLECTD`` - Prefix for collectd's log file. +* ``COLLECTD_CPU_KEYS`` - Interesting metrics from CPU +* ``COLLECTD_PROCESSES_KEYS`` - Interesting metrics from processes +* ``COLLECTD_INTERFACE_KEYS`` - Interesting metrics from interface +* ``COLLECTD_OVSSTAT_KEYS`` - Interesting metrics from OVS +* ``COLLECTD_DPDKSTAT_KEYS`` - Interesting metrics from DPDK. +* ``COLLECTD_INTELRDT_KEYS`` - Interesting metrics from Intel-RDT +* ``COLLECTD_INTERFACE_XKEYS`` - Metrics to exclude from Interface +* ``COLLECTD_INTELRDT_XKEYS`` - Metrics to exclude from Intel-RDT + + +.. _`Load Generators`: + + +Load Generation +--------------- + +In VSPERF, load generation refers to creating background cpu and memory loads to +study the impact of these loads on system under test. There are two options to +create loads in VSPERF. These options are used for different use-cases. The options are: + +* stress or stress-ng +* Stressor-VMs + +*stress and stress-ng* are linux tools to stress the system in various ways. +It can stress different subsystems such as CPU and memory. *stress-ng* is the +improvised version of *stress*. StressorVMs are custom build virtual-machines +for the noisy-neighbor use-cases. + +Installation +^^^^^^^^^^^^ + +stress and stress-ng can be installed through standard linux installation process. +Information about stress-ng, including the steps for installing can be found +here: `stress-ng <https://github.com/ColinIanKing/stress-ng>`_ + +There are two options for StressorVMs - one is VMs based on stress-ng and second +is VM based on Spirent's cloudstress. VMs based on stress-ng can be found in this +`link <https://github.com/opensource-tnbt/stressng-images>`_ . Spirent's cloudstress +based VM can be downloaded from this `site <https://github.com/spirent/cloudstress>`_ + +These stressorVMs are of OSV based VMs, which are very small in size. Download +these VMs and place it in appropriate location, and this location will used in +the configuration - as mentioned below. + +Configuration +^^^^^^^^^^^^^ + +The configuration file for loadgens can be found in **conf/07_loadgen.conf**. +There are no specific configurations for stress and stress-ng commands based +load-generation. However, for StressorVMs, following configurations apply: + +* ``NN_COUNT`` - Number of stressor VMs required. +* ``NN_MEMORY`` - Comma separated memory configuration for each VM +* ``NN_SMP`` - Comma separated configuration for each VM +* ``NN_IMAGE`` - Comma separated list of Paths for each VM image +* ``NN_SHARED_DRIVE_TYPE`` - Comma separated list of shaed drive type for each VM +* ``NN_BOOT_DRIVE_TYPE`` - Comma separated list of boot drive type for each VM +* ``NN_CORE_BINDING`` - Comma separated lists of list specifying the cores associated with each VM. +* ``NN_NICS_NR`` - Comma seprated list of number of NICS for each VM +* ``NN_BASE_VNC_PORT`` - Base VNC port Index. +* ``NN_LOG_FILE`` - Name of the log file + +.. _`L3 Cache Management`: + +Last Level Cache Management +--------------------------- + +VSPERF support last-level cache management using Intel's RDT tool(s) - the +relavant ones are `Intel CAT-CMT <https://github.com/intel/intel-cmt-cat>`_ and +`Intel RMD <https://github.com/intel/rmd>`_. RMD is a linux daemon that runs on +individual hosts, and provides a REST API for control/orchestration layer to +request LLC for the VMs/Containers/Applications. RDT receives resource policy +form orchestration layer - in this case, from VSPERF - and enforce it on the host. +It achieves this enforcement via kernel interfaces such as resctrlfs and libpqos. +The resource here refer to the last-level cache. User can configure policies to +define how much of cache a CPU can get. The policy configuration is described below. + +Installation +^^^^^^^^^^^^ + +For installation of RMD tool, please install CAT-CMT first and then install RMD. +The details of installation can be found here: `Intel CAT-CMT <https://github.com/intel/intel-cmt-cat>`_ +and `Intel RMD <https://github.com/intel/rmd>`_ + +Configuration +^^^^^^^^^^^^^ + +The configuration file for cache management can be found in **conf/08_llcmanagement.conf**. + +VSPERF provides following configuration options, for user to define and enforce policies via RMD. + +* ``LLC_ALLOCATION`` - Enable or Disable LLC management. +* ``RMD_PORT`` - RMD port (port number on which API server is listening) +* ``RMD_SERVER_IP`` - IP address where RMD is running. Currently only localhost. +* ``RMD_API_VERSION`` - RMD version. Currently it is 'v1' +* ``POLICY_TYPE`` - Specify how the policy is defined - either COS or CUSTOM +* ``VSWITCH_COS`` - Class of service (CoS for Vswitch. CoS can be gold, silver-bf or bronze-shared. +* ``VNF_COS`` - Class of service for VNF +* ``PMD_COS`` - Class of service for PMD +* ``NOISEVM_COS`` - Class of service of Noisy VM. +* ``VSWITCH_CA`` - [min-cache-value, maxi-cache-value] for vswitch +* ``VNF_CA`` - [min-cache-value, max-cache-value] for VNF +* ``PMD_CA`` - [min-cache-value, max-cache-value] for PMD +* ``NOISEVM_CA`` - [min-cache-value, max-cache-value] for Noisy VM diff --git a/docs/testing/user/configguide/trafficgen.rst b/docs/testing/user/configguide/trafficgen.rst index 52b1b4a5..f9e2db11 100644 --- a/docs/testing/user/configguide/trafficgen.rst +++ b/docs/testing/user/configguide/trafficgen.rst @@ -39,6 +39,7 @@ and is configured as follows: TRAFFIC = { 'traffic_type' : 'rfc2544_throughput', 'frame_rate' : 100, + 'burst_size' : 100, 'bidir' : 'True', # will be passed as string in title format to tgen 'multistream' : 0, 'stream_type' : 'L4', @@ -75,8 +76,22 @@ and is configured as follows: 'count': 1, 'filter': '', }, + 'scapy': { + 'enabled': False, + '0' : 'Ether(src={Ether_src}, dst={Ether_dst})/' + 'Dot1Q(prio={Dot1Q_prio}, id={Dot1Q_id}, vlan={Dot1Q_vlan})/' + 'IP(proto={IP_proto}, src={IP_src}, dst={IP_dst})/' + '{IP_PROTO}(sport={IP_PROTO_sport}, dport={IP_PROTO_dport})', + '1' : 'Ether(src={Ether_dst}, dst={Ether_src})/' + 'Dot1Q(prio={Dot1Q_prio}, id={Dot1Q_id}, vlan={Dot1Q_vlan})/' + 'IP(proto={IP_proto}, src={IP_dst}, dst={IP_src})/' + '{IP_PROTO}(sport={IP_PROTO_dport}, dport={IP_PROTO_sport})', + } } +A detailed description of the ``TRAFFIC`` dictionary can be found at +:ref:`configuration-of-traffic-dictionary`. + The framesize parameter can be overridden from the configuration files by adding the following to your custom configuration file ``10_custom.conf``: @@ -857,6 +872,21 @@ place. This can be adjusted with the following configurations: TRAFFICGEN_TREX_LEARNING_MODE=True TRAFFICGEN_TREX_LEARNING_DURATION=5 +Latency measurements have impact on T-Rex performance. Thus vswitchperf uses a separate +latency stream for each direction with limited speed. This workaround is used for RFC2544 +**Throughput** and **Continuous** traffic types. In case of **Burst** traffic type, +the latency statistics are measured for all frames in the burst. Collection of latency +statistics is driven by configuration option ``TRAFFICGEN_TREX_LATENCY_PPS`` as follows: + + * value ``0`` - disables latency measurements + * non zero integer value - enables latency measurements; In case of Throughput + and Continuous traffic types, it specifies a speed of latency specific stream + in PPS. In case of burst traffic type, it enables latency measurements for all frames. + +.. code-block:: console + + TRAFFICGEN_TREX_LATENCY_PPS = 1000 + SR-IOV and Multistream layer 2 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ T-Rex by default only accepts packets on the receive side if the destination mac matches the @@ -907,3 +937,72 @@ The duration and maximum number of attempted verification trials can be set to c behavior of this step. If the verification step fails, it will resume the binary search with new values where the maximum output will be the last attempted frame rate minus the current set thresh hold. + +Scapy frame definition +~~~~~~~~~~~~~~~~~~~~~~ + +It is possible to use a SCAPY frame definition to generate various network protocols +by the **T-Rex** traffic generator. In case that particular network protocol layer +is disabled by the TRAFFIC dictionary (e.g. TRAFFIC['vlan']['enabled'] = False), +then disabled layer will be removed from the scapy format definition by VSPERF. + +The scapy frame definition can refer to values defined by the TRAFFIC dictionary +by following keywords. These keywords are used in next examples. + +* ``Ether_src`` - refers to ``TRAFFIC['l2']['srcmac']`` +* ``Ether_dst`` - refers to ``TRAFFIC['l2']['dstmac']`` +* ``IP_proto`` - refers to ``TRAFFIC['l3']['proto']`` +* ``IP_PROTO`` - refers to upper case version of ``TRAFFIC['l3']['proto']`` +* ``IP_src`` - refers to ``TRAFFIC['l3']['srcip']`` +* ``IP_dst`` - refers to ``TRAFFIC['l3']['dstip']`` +* ``IP_PROTO_sport`` - refers to ``TRAFFIC['l4']['srcport']`` +* ``IP_PROTO_dport`` - refers to ``TRAFFIC['l4']['dstport']`` +* ``Dot1Q_prio`` - refers to ``TRAFFIC['vlan']['priority']`` +* ``Dot1Q_id`` - refers to ``TRAFFIC['vlan']['cfi']`` +* ``Dot1Q_vlan`` - refers to ``TRAFFIC['vlan']['id']`` + +In following examples of SCAPY frame definition only relevant parts of TRAFFIC +dictionary are shown. The rest of the TRAFFIC dictionary is set to default values +as they are defined in ``conf/03_traffic.conf``. + +Please check official documentation of SCAPY project for details about SCAPY frame +definition and supported network layers at: http://www.secdev.org/projects/scapy + +#. Generate ICMP frames: + + .. code-block:: console + + 'scapy': { + 'enabled': True, + '0' : 'Ether(src={Ether_src}, dst={Ether_dst})/IP(proto={IP_proto}, src={IP_src}, dst={IP_dst})/ICMP()', + '1' : 'Ether(src={Ether_dst}, dst={Ether_src})/IP(proto={IP_proto}, src={IP_dst}, dst={IP_src})/ICMP()', + } + +#. Generate IPv6 ICMP Echo Request + + .. code-block:: console + + 'l3' : { + 'srcip': 'feed::01', + 'dstip': 'feed::02', + }, + 'scapy': { + 'enabled': True, + '0' : 'Ether(src={Ether_src}, dst={Ether_dst})/IPv6(src={IP_src}, dst={IP_dst})/ICMPv6EchoRequest()', + '1' : 'Ether(src={Ether_dst}, dst={Ether_src})/IPv6(src={IP_dst}, dst={IP_src})/ICMPv6EchoRequest()', + } + +#. Generate SCTP frames: + + Example uses default SCAPY frame definition, which can reflect ``TRAFFIC['l3']['proto']`` settings. The same + approach can be used to generate other protocols, e.g. TCP. + + .. code-block:: console + + 'l3' : { + 'proto' : 'sctp', + }, + 'scapy': { + 'enabled': True, + } + diff --git a/docs/testing/user/userguide/testusage.rst b/docs/testing/user/userguide/testusage.rst index f679566e..9b331869 100644 --- a/docs/testing/user/userguide/testusage.rst +++ b/docs/testing/user/userguide/testusage.rst @@ -216,6 +216,12 @@ A Kernel Module that provides OSI Layer 2 Ipv4 termination or forwarding with support for Destination Network Address Translation (DNAT) for both the MAC and IP addresses. l2fwd can be found in <vswitchperf_dir>/src/l2fwd +Additional Tools Setup +^^^^^^^^^^^^^^^^^^^^^^ + +Follow the `Additional tools instructions <additional-tools-configuration>` to +install and configure additional tools such as collectors and loadgens. + Executing tests ^^^^^^^^^^^^^^^ |