summaryrefslogtreecommitdiffstats
path: root/docs/requirements/vswitchperf_ltd.rst
diff options
context:
space:
mode:
authorMartin Klozik <martinx.klozik@intel.com>2016-09-06 08:08:34 +0100
committerMartin Klozik <martinx.klozik@intel.com>2016-09-06 08:50:21 +0100
commit3a86248dff1834267900423891c9b2829803fd8a (patch)
tree64c6e3f1da3ed19c560e644386ceb857e520328f /docs/requirements/vswitchperf_ltd.rst
parent99e50b9aa86d3bd92d23a32ff34b5f80b9b3e4da (diff)
docs: Fix links and formatting
Documentation was updated as follows: * links were updated to refer to documentation specific for Colorado release * section decorators were fixed in LTD * links to testcase and deployment descriptions were fixed * duplicate labels removed from LTP&LTD * fixed typos in formatting and text * links to scalability and stress tests fixed after their renaming JIRA: VSPERF-349 Change-Id: I8b1d7a0220d637a684e70a788dd88250c646124e Signed-off-by: Martin Klozik <martinx.klozik@intel.com> Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com> Reviewed-by: Al Morton <acmorton@att.com> Reviewed-by: Christian Trautman <ctrautma@redhat.com> Reviewed-by: Bill Michalowski <bmichalo@redhat.com>
Diffstat (limited to 'docs/requirements/vswitchperf_ltd.rst')
-rw-r--r--docs/requirements/vswitchperf_ltd.rst127
1 files changed, 83 insertions, 44 deletions
diff --git a/docs/requirements/vswitchperf_ltd.rst b/docs/requirements/vswitchperf_ltd.rst
index 23dcd3cf..0d297880 100644
--- a/docs/requirements/vswitchperf_ltd.rst
+++ b/docs/requirements/vswitchperf_ltd.rst
@@ -8,24 +8,25 @@ VSPERF LEVEL TEST DESIGN (LTD)
.. 3.1
+============
Introduction
-===============
+============
The intention of this Level Test Design (LTD) document is to specify the set of
tests to carry out in order to objectively measure the current characteristics
of a virtual switch in the Network Function Virtualization Infrastructure
(NFVI) as well as the test pass criteria. The detailed test cases will be
-defined in details-of-LTD_, preceded by the doc-id_ and the scope_.
+defined in details-of-LTD_, preceded by the doc-id-of-LTD_ and the scope-of-LTD_.
This document is currently in draft form.
.. 3.1.1
-.. _doc-id:
+.. _doc-id-of-LTD:
Document identifier
-=========================
+===================
The document id will be used to uniquely
identify versions of the LTD. The format for the document id will be:
@@ -36,10 +37,10 @@ OPNFV\_vswitchperf\_LTD\_Brahmaputra\_REVIEWED.
.. 3.1.2
-.. _scope:
+.. _scope-of-LTD:
Scope
-==========
+=====
The main purpose of this project is to specify a suite of
performance tests in order to objectively measure the current packet
@@ -58,7 +59,7 @@ release.
.. 3.1.3
References
-===============
+==========
* `RFC 1242 Benchmarking Terminology for Network Interconnection
Devices <http://www.ietf.org/rfc/rfc1242.txt>`__
@@ -81,19 +82,19 @@ References
.. _details-of-LTD:
-===================================
+================================
Details of the Level Test Design
-===================================
+================================
-This section describes the features to be tested (FeaturesToBeTested_), and
-identifies the sets of test cases or scenarios (TestIdentification_).
+This section describes the features to be tested (FeaturesToBeTested-of-LTD_), and
+identifies the sets of test cases or scenarios (TestIdentification-of-LTD_).
.. 3.2.1
-.. _FeaturesToBeTested:
+.. _FeaturesToBeTested-of-LTD:
Features to be tested
-==========================
+=====================
Characterizing virtual switches (i.e. Device Under Test (DUT) in this document)
includes measuring the following performance metrics:
@@ -109,15 +110,16 @@ includes measuring the following performance metrics:
.. 3.2.2
-.. _TestIdentification:
+.. _TestIdentification-of-LTD:
Test identification
-=========================
+===================
-.. 3.2.3.1
+.. 3.2.2.1
Throughput tests
-----------------------
+----------------
+
The following tests aim to determine the maximum forwarding rate that
can be achieved with a virtual switch. The list is not exhaustive but
should indicate the type of tests that should be required. It is
@@ -126,7 +128,8 @@ expected that more will be added.
.. 3.2.2.1.1
Test ID: LTD.Throughput.RFC2544.PacketLossRatio
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC 2544 X% packet loss ratio Throughput and Latency Test
**Prerequisite Test**: N/A
@@ -176,7 +179,8 @@ Test ID: LTD.Throughput.RFC2544.PacketLossRatio
.. 3.2.2.1.2
Test ID: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC 2544 X% packet loss Throughput and Latency Test with
packet modification
@@ -247,6 +251,7 @@ Test ID: LTD.Throughput.RFC2544.PacketLossRatioFrameModification
Test ID: LTD.Throughput.RFC2544.Profile
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC 2544 Throughput and Latency Profile
**Prerequisite Test**: N/A
@@ -301,7 +306,8 @@ Test ID: LTD.Throughput.RFC2544.Profile
.. 3.2.2.1.4
Test ID: LTD.Throughput.RFC2544.SystemRecoveryTime
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC 2544 System Recovery Time Test
**Prerequisite Test** LTD.Throughput.RFC2544.PacketLossRatio
@@ -353,7 +359,8 @@ Test ID: LTD.Throughput.RFC2544.SystemRecoveryTime
.. 3.2.2.1.5
Test ID: LTD.Throughput.RFC2544.BackToBackFrames
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC2544 Back To Back Frames Test
**Prerequisite Test**: N
@@ -396,7 +403,8 @@ Test ID: LTD.Throughput.RFC2544.BackToBackFrames
.. 3.2.2.1.6
Test ID: LTD.Throughput.RFC2889.MaxForwardingRateSoak
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC 2889 X% packet loss Max Forwarding Rate Soak Test
**Prerequisite Test** LTD.Throughput.RFC2544.PacketLossRatio
@@ -438,7 +446,8 @@ Test ID: LTD.Throughput.RFC2889.MaxForwardingRateSoak
.. 3.2.2.1.7
Test ID: LTD.Throughput.RFC2889.MaxForwardingRateSoakFrameModification
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC 2889 Max Forwarding Rate Soak Test with Frame Modification
**Prerequisite Test**:
@@ -494,7 +503,8 @@ Test ID: LTD.Throughput.RFC2889.MaxForwardingRateSoakFrameModification
.. 3.2.2.1.8
Test ID: LTD.Throughput.RFC6201.ResetTime
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC 6201 Reset Time Test
**Prerequisite Test**: N/A
@@ -553,6 +563,7 @@ Test ID: LTD.Throughput.RFC6201.ResetTime
* Transmitting test frame despite the physical link status.
whereas the Timestamp method SHOULD be used if the test tool supports:
+
* Timestamping each frame.
* Monitoring received frame's timestamp.
* Transmitting frames only if the physical link status is up.
@@ -585,7 +596,8 @@ Test ID: LTD.Throughput.RFC6201.ResetTime
.. 3.2.2.1.9
Test ID: LTD.Throughput.RFC2889.MaxForwardingRate
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC2889 Forwarding Rate Test
**Prerequisite Test**: LTD.Throughput.RFC2544.PacketLossRatio
@@ -647,7 +659,8 @@ Test ID: LTD.Throughput.RFC2889.MaxForwardingRate
.. 3.2.2.1.10
Test ID: LTD.Throughput.RFC2889.ForwardPressure
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC2889 Forward Pressure Test
**Prerequisite Test**: LTD.Throughput.RFC2889.MaxForwardingRate
@@ -682,7 +695,8 @@ Test ID: LTD.Throughput.RFC2889.ForwardPressure
.. 3.2.2.1.11
Test ID: LTD.Throughput.RFC2889.ErrorFramesFiltering
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC2889 Error Frames Filtering Test
**Prerequisite Test**: N/A
@@ -718,7 +732,8 @@ Test ID: LTD.Throughput.RFC2889.ErrorFramesFiltering
.. 3.2.2.1.12
Test ID: LTD.Throughput.RFC2889.BroadcastFrameForwarding
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC2889 Broadcast Frame Forwarding Test
**Prerequisite Test**: N
@@ -763,7 +778,8 @@ Test ID: LTD.Throughput.RFC2889.BroadcastFrameForwarding
.. 3.2.2.1.13
Test ID: LTD.Throughput.RFC2544.WorstN-BestN
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: Modified RFC 2544 X% packet loss ratio Throughput and Latency Test
**Prerequisite Test**: N/A
@@ -856,6 +872,7 @@ Test ID: LTD.Throughput.RFC2544.WorstN-BestN
Test ID: LTD.Throughput.Overlay.Network.<tech>.RFC2544.PacketLossRatio
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: <tech> Overlay Network RFC 2544 X% packet loss ratio Throughput and Latency Test
@@ -935,6 +952,7 @@ Test ID: LTD.Throughput.Overlay.Network.<tech>.RFC2544.PacketLossRatio
Test ID: LTD.Throughput.RFC2544.MatchAction.PacketLossRatio
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC 2544 X% packet loss ratio match action Throughput and Latency Test
**Prerequisite Test**: LTD.Throughput.RFC2544.PacketLossRatio
@@ -948,6 +966,7 @@ Test ID: LTD.Throughput.RFC2544.MatchAction.PacketLossRatio
a constant load (fixed length frames at a fixed interval time).
Each test case requires:
+
* selection of a specific match action(s),
* specifying a percentage of total traffic that is elligible
for the match action,
@@ -965,6 +984,7 @@ Test ID: LTD.Throughput.RFC2544.MatchAction.PacketLossRatio
or packet that matches a set of flow classification parameters
(typically frame/packet header fields). A match action may or may
not modify a packet/frame. Match actions include [1]:
+
* output : outputs a packet to a particular port.
* normal: Subjects the packet to traditional L2/L3 processing
(MAC learning).
@@ -973,18 +993,19 @@ Test ID: LTD.Throughput.RFC2544.MatchAction.PacketLossRatio
on which flooding is disabled.
* all: Outputs the packet on all switch physical ports other
than the port on which it was received.
- * local: Outputs the packet on the ``local port,'' which
+ * local: Outputs the packet on the ``local port``, which
corresponds to the network device that has the same name as
the bridge.
* in_port: Outputs the packet on the port from which it was
received.
* Controller: Sends the packet and its metadata to the
- OpenFlow controller as a ``packet in'' message.
+ OpenFlow controller as a ``packet in`` message.
* enqueue: Enqueues the packet on the specified queue
within port.
* drop: discard the packet.
Modifications include [1]:
+
* mod vlan: covered by LTD.Throughput.RFC2544.PacketLossRatioFrameModification
* mod_dl_src: Sets the source Ethernet address.
* mod_dl_dst: Sets the destination Ethernet address.
@@ -1006,6 +1027,7 @@ Test ID: LTD.Throughput.RFC2544.MatchAction.PacketLossRatio
During this test, the DUT must perform the following operations on
the traffic flow:
+
* Perform packet parsing on the DUT’s ingress port.
* Perform any relevant address look-ups on the DUT’s ingress
ports.
@@ -1034,6 +1056,7 @@ Test ID: LTD.Throughput.RFC2544.MatchAction.PacketLossRatio
**Metrics Collected:**
The following are the metrics collected for this test:
+
* The RFC 2544 Throughput in Frames Per Second (FPS) and Mbps
of the DUT for each frame size with X% packet loss.
* The average latency of the traffic flow when passing through
@@ -1057,7 +1080,8 @@ Test ID: LTD.Throughput.RFC2544.MatchAction.PacketLossRatio
.. 3.2.2.2
Packet Latency tests
----------------------------
+--------------------
+
These tests will measure the store and forward latency as well as the packet
delay variation for various packet types through the virtual switch. The
following list is not exhaustive but should indicate the type of tests
@@ -1066,7 +1090,8 @@ that should be required. It is expected that more will be added.
.. 3.2.2.2.1
Test ID: LTD.PacketLatency.InitialPacketProcessingLatency
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: Initial Packet Processing Latency
**Prerequisite Test**: N/A
@@ -1115,7 +1140,8 @@ Test ID: LTD.PacketLatency.InitialPacketProcessingLatency
.. 3.2.2.2.2
Test ID: LTD.PacketDelayVariation.RFC3393.Soak
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: Packet Delay Variation Soak Test
**Prerequisite Tests**: LTD.Throughput.RFC2544.PacketLossRatio (0% Packet Loss)
@@ -1148,7 +1174,8 @@ Test ID: LTD.PacketDelayVariation.RFC3393.Soak
.. 3.2.2.3
Scalability tests
-------------------------
+-----------------
+
The general aim of these tests is to understand the impact of large flow
table size and flow lookups on throughput. The following list is not
exhaustive but should indicate the type of tests that should be required.
@@ -1157,7 +1184,8 @@ It is expected that more will be added.
.. 3.2.2.3.1
Test ID: LTD.Scalability.Flows.RFC2544.0PacketLoss
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC 2544 0% loss Flow Scalability throughput test
**Prerequisite Test**: LTD.Throughput.RFC2544.PacketLossRatio, IF the
@@ -1212,7 +1240,8 @@ Test ID: LTD.Scalability.Flows.RFC2544.0PacketLoss
.. 3.2.2.3.2
Test ID: LTD.MemoryBandwidth.RFC2544.0PacketLoss.Scalability
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC 2544 0% loss Memory Bandwidth Scalability test
**Prerequisite Tests**: LTD.Throughput.RFC2544.PacketLossRatio, IF the
@@ -1262,6 +1291,7 @@ Test ID: LTD.MemoryBandwidth.RFC2544.0PacketLoss.Scalability
Test ID: LTD.Scalability.VNF.RFC2544.PacketLossRatio
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: VNF Scalability RFC 2544 X% packet loss ratio Throughput and
Latency Test
@@ -1342,7 +1372,8 @@ Test ID: LTD.Scalability.VNF.RFC2544.PacketLossRatio
.. 3.2.2.3.4
Test ID: LTD.Scalability.VNF.RFC2544.PacketLossProfile
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: VNF Scalability RFC 2544 Throughput and Latency Profile
**Prerequisite Test**: N/A
@@ -1421,6 +1452,7 @@ Test ID: LTD.Scalability.VNF.RFC2544.PacketLossProfile
Activation tests
----------------
+
The general aim of these tests is to understand the capacity of the
and speed with which the vswitch can accommodate new flows.
@@ -1428,6 +1460,7 @@ and speed with which the vswitch can accommodate new flows.
Test ID: LTD.Activation.RFC2889.AddressCachingCapacity
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC2889 Address Caching Capacity Test
**Prerequisite Test**: N/A
@@ -1479,7 +1512,8 @@ Test ID: LTD.Activation.RFC2889.AddressCachingCapacity
.. 3.2.2.4.2
Test ID: LTD.Activation.RFC2889.AddressLearningRate
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC2889 Address Learning Rate Test
**Prerequisite Test**: LTD.Memory.RFC2889.AddressCachingCapacity
@@ -1516,7 +1550,8 @@ Test ID: LTD.Activation.RFC2889.AddressLearningRate
.. 3.2.2.5
Coupling between control path and datapath Tests
--------------------------------------------------------
+------------------------------------------------
+
The following tests aim to determine how tightly coupled the datapath
and the control path are within a virtual switch. The following list
is not exhaustive but should indicate the type of tests that should be
@@ -1525,7 +1560,8 @@ required. It is expected that more will be added.
.. 3.2.2.5.1
Test ID: LTD.CPDPCouplingFlowAddition
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: Control Path and Datapath Coupling
**Prerequisite Test**:
@@ -1573,7 +1609,8 @@ Test ID: LTD.CPDPCouplingFlowAddition
.. 3.2.2.6
CPU and memory consumption
----------------------------------
+--------------------------
+
The following tests will profile a virtual switch's CPU and memory
utilization under various loads and circumstances. The following
list is not exhaustive but should indicate the type of tests that
@@ -1583,6 +1620,7 @@ should be required. It is expected that more will be added.
Test ID: LTD.Stress.RFC2544.0PacketLoss
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
**Title**: RFC 2544 0% Loss CPU OR Memory Stress Test
**Prerequisite Test**:
@@ -1620,7 +1658,8 @@ Test ID: LTD.Stress.RFC2544.0PacketLoss
.. 3.2.2.7
Summary List of Tests
-----------------------------
+---------------------
+
1. Throughput tests
- Test ID: LTD.Throughput.RFC2544.PacketLossRatio
@@ -1650,7 +1689,7 @@ Summary List of Tests
- LTD.Scalability.VNF.RFC2544.PacketLossProfile
- LTD.Scalability.VNF.RFC2544.PacketLossRatio
-4. Acivation tests
+4. Activation tests
- Test ID: LTD.Activation.RFC2889.AddressCachingCapacity
- Test ID: LTD.Activation.RFC2889.AddressLearningRate