diff options
Diffstat (limited to 'test_spec')
| -rwxr-xr-x | test_spec/ietf_summary/draft-vsperf-bmwg-vswitch-opnfv-00.xml | 554 | ||||
| -rw-r--r-- | test_spec/vswitchperf_ltd.md | 66 |
2 files changed, 588 insertions, 32 deletions
diff --git a/test_spec/ietf_summary/draft-vsperf-bmwg-vswitch-opnfv-00.xml b/test_spec/ietf_summary/draft-vsperf-bmwg-vswitch-opnfv-00.xml new file mode 100755 index 00000000..e4057410 --- /dev/null +++ b/test_spec/ietf_summary/draft-vsperf-bmwg-vswitch-opnfv-00.xml @@ -0,0 +1,554 @@ +<?xml version="1.0" encoding="US-ASCII"?> +<!DOCTYPE rfc SYSTEM "rfc2629.dtd"> +<?rfc toc="yes"?> +<?rfc tocompact="yes"?> +<?rfc tocdepth="3"?> +<?rfc tocindent="yes"?> +<?rfc symrefs="yes"?> +<?rfc sortrefs="yes"?> +<?rfc comments="yes"?> +<?rfc inline="yes"?> +<?rfc compact="yes"?> +<?rfc subcompact="no"?> +<rfc category="info" docName="draft-vsperf-bmwg-vswitch-opnfv-00" + ipr="trust200902"> + <front> + <title abbrev="Benchmarking vSwitches">Benchmarking Virtual Switches in + OPNFV</title> + + <author fullname="Maryam Tahhan" initials="M." surname="Tahhan"> + <organization>Intel</organization> + + <address> + <postal> + <street/> + + <city/> + + <region/> + + <code/> + + <country/> + </postal> + + <phone/> + + <facsimile/> + + <email/> + + <uri/> + </address> + </author> + + <author fullname="Billy O Mahony" initials="B." surname="O Mahony"> + <organization>Intel</organization> + + <address> + <postal> + <street/> + + <city/> + + <region/> + + <code/> + + <country/> + </postal> + + <phone/> + + <facsimile/> + + <email/> + + <uri/> + </address> + </author> + + <author fullname="Al Morton" initials="A." surname="Morton"> + <organization>AT&T Labs</organization> + + <address> + <postal> + <street>200 Laurel Avenue South</street> + + <city>Middletown,</city> + + <region>NJ</region> + + <code>07748</code> + + <country>USA</country> + </postal> + + <phone>+1 732 420 1571</phone> + + <facsimile>+1 732 368 1192</facsimile> + + <email>acmorton@att.com</email> + + <uri>http://home.comcast.net/~acmacm/</uri> + </address> + </author> + + <date day="6" month="June" year="2015"/> + + <abstract> + <t>This memo describes the progress of the Open Platform for NFV (OPNFV) + project on virtual switch performance "VSWITCHPERF". This project + intends to build on the current and completed work of the Benchmarking + Methodology Working Group in IETF, by referencing existing literature. + The Benchmarking Methodology Working Group has traditionally conducted + laboratory characterization of dedicated physical implementations of + internetworking functions. Therefore, this memo begins to describes the + additional considerations when virtual switches are implemented in + general-purpose hardware. The expanded tests and benchmarks are also + influenced by the OPNFV mission to support virtualization of the "telco" + infrastructure.</t> + </abstract> + + <note title="Requirements Language"> + <t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", + "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this + document are to be interpreted as described in <xref + target="RFC2119">RFC 2119</xref>.</t> + + <t/> + </note> + </front> + + <middle> + <section title="Introduction"> + <t>Benchmarking Methodology Working Group (BMWG) has traditionally + conducted laboratory characterization of dedicated physical + implementations of internetworking functions. The Black-box Benchmarks + of Throughput, Latency, Forwarding Rates and others have served our + industry for many years. Now, Network Function Virtualization (NFV) has + the goal to transform how internetwork functions are implemented, and + therefore has garnered much attention.</t> + + <t>This memo describes the progress of the Open Platform for NFV (OPNFV) + project on virtual switch performance characterization, "VSWITCHPERF". + This project intends to build on the current and completed work of the + Benchmarking Methodology Working Group in IETF, by referencing existing + literature. For example, currently the most referenced RFC is <xref + target="RFC2544"/> (which depens on <xref target="RFC1242"/>) and + foundation of the benchmarking work in OPNFV is common and strong.</t> + + <t>See + https://wiki.opnfv.org/characterize_vswitch_performance_for_telco_nfv_use_cases + for more background, and the OPNFV website for general information: + https://www.opnfv.org/</t> + + <t>The authors note that OPNFV distinguishes itself from other open + source compute and networking projects through its emphasis on existing + "telco" services as opposed to cloud-computing. There are many ways in + which telco requirements have different emhasis on performance + dimensions when compared to cloud computing: support for and transfer of + isochronous media streams is one example.</t> + + <t>Note also that the move to NFV Infrastructure has resulted in many + new benchmarking initiatives across the industry, and the authors are + currently doing their best to maintain alignment with many other + projects, and this Internet Draft is evidence of the efforts.</t> + </section> + + <section title="Scope"> + <t>The primary purpose and scope of the memo is to inform BMWG of + work-in-progress that builds on the body of extensive literature and + experience. Additionally, once the initial information conveyed here is + received, this memo may be expanded to include more detail and + commentary from both BMWG and OPNFV communities, under BMWG's chartered + work to characterize the NFV Infrastructure (a virtual switch is an + important aspect of that infrastructure).</t> + </section> + + <section title="Benchmarking Considerations"> + <t>This section highlights some specific considerations (from <xref + target="ID.ietf-bmwg-virtual-net"/>)related to Benchmarks for virtual + switches.</t> + + <section title="Comparison with Physical Network Functions"> + <t>To compare the performance of virtual designs and implementations + with their physical counterparts, identical benchmarks are needed. + BMWG has developed specifications for many network functions this memo + re-uses existing benchmarks through references, and expands them + during development of new methods. A key configuration aspect is the + number of parallel cores required to achieve comparable performance + with a given physical device, or whether some limit of scale was + reached before the cores could achieve the comparable level. </t> + + <t>It's unlikely that the virtual switch will be the only application + running on the SUT, so CPU utilization, Cache utilization, and Memory + footprint should also be recorded for the virtual implementations of + internetworking functions.</t> + </section> + + <section title="Continued Emphasis on Black-Box Benchmarks"> + <t>External observations remain essential as the basis for Benchmarks. + Internal observations with fixed specification and interpretation will + be provided in parallel to assist the development of operations + procedures when the technology is deployed. </t> + </section> + </section> + + <section title="VSWITCHPERF Specification Summary"> + <t>The overall specification in preparation is referred to as a Level + Test Design (LTD) document, which will contain a suite of performace + tests.</t> + + <t>As one might expect, the most fundamental internetworking + characteristics of Throughput and Latency remain important when the + switch is virtualized, and these benchmarks figure prominently in the + specification.</t> + + <t>When considering characteristics important to "telco" network + functions, we must begin to consider additional performance metrics. In + this case, the project specifications have referenced metrics from the + IETF IP Performance Metrics (IPPM) literature. This means that the <xref + target="RFC2544"/> test of Latency is replaced by measurement of a + metric derived from IPPM's <xref target="RFC2679"/>, where a set of + statistical summaries will be provided (mean, max, min, etc.). Further + metrics planned to be benchmarked include packet delay variation as + defined by <xref target="RFC5481"/> , reordering, burst behaviour, DUT + availability, DUT capacity and packet loss in long term testing at + Throughput level, where some low-level of background loss may be present + and characterized.</t> + + <t>Tests have been (or will be) designed to collect the metrics + below:</t> + + <t><list style="symbols"> + <t>Throughput Tests to measure the maximum forwarding rate (in + frames per second or fps) and bit rate (in Mbps) for a constant load + (as defined by RFC1242) without traffic loss.</t> + + <t>Packet and Frame Delay Distribution Tests to measure average, min + and max packet and frame delay for constant loads.</t> + + <t>Packet Delay Tests to understand latency distribution for + different packet sizes and over an extended test run to uncover + outliers.</t> + + <t>Scalability Tests to understand how the virtual switch performs + as the number of flows, active ports, complexity of the forwarding + logic’s configuration… it has to deal with + increases.</t> + + <t>Stream Performance Tests (TCP, UDP) to measure bulk data transfer + performance, i.e. how fast systems can send and receive data through + the switch.</t> + + <t>Request/Response Performance Tests (TCP, UDP) the measure the + transaction rate through the switch.</t> + + <t>Control Path and Datapath Coupling Tests, to understand how + closely coupled the datapath and the control path are as well as the + effect of this coupling on the performance of the DUT (example: + delay of the initial packet of a flow).</t> + + <t>Noisy Neighbour Tests, to understand the effects of resource + sharing on the performance of a virtual switch.</t> + + <t>CPU and Memory Consumption Tests to understand the virtual + switch’s footprint on the system, usually conducted as + auxilliary measurements with benchmarks above. They include: CPU + utilization, Cache utilization and Memory footprint.</t> + </list></t> + + <t>The felixability of deployemnt of a virtual switch within a network + means that the BMWG IETF existing literature needs to be used to + characterize the performance of a switch in various deployment + scenarios. The deployment scenarios under consideration include:</t> + + <t><figure> + <preamble>Physical port to virtual switch to physical + port</preamble> + + <artwork><![CDATA[ __ + +--------------------------------------------------+ | + | +--------------------+ | | + | | | | | + | | v | | Host + | +--------------+ +--------------+ | | + | | phy port | vSwitch | phy port | | | + +---+--------------+------------+--------------+---+ __| + ^ : + | | + : v + +--------------------------------------------------+ + | | + | traffic generator | + | | + +--------------------------------------------------+]]></artwork> + </figure></t> + + <t><figure> + <preamble>Physical port to virtual switch to VNF to virtual switch + to physical port</preamble> + + <artwork><![CDATA[ __ + +---------------------------------------------------+ | + | | | + | +-------------------------------------------+ | | + | | Application | | | + | +-------------------------------------------+ | | + | ^ : | | + | | | | | Guest + | : v | | + | +---------------+ +---------------+ | | + | | logical port 0| | logical port 1| | | + +---+---------------+-----------+---------------+---+ __| + ^ : + | | + : v __ + +---+---------------+----------+---------------+---+ | + | | logical port 0| | logical port 1| | | + | +---------------+ +---------------+ | | + | ^ : | | + | | | | | Host + | : v | | + | +--------------+ +--------------+ | | + | | phy port | vSwitch | phy port | | | + +---+--------------+------------+--------------+---+ __| + ^ : + | | + : v + +--------------------------------------------------+ + | | + | traffic generator | + | | + +--------------------------------------------------+]]></artwork> + </figure><figure> + <preamble>Physical port to virtual switch to VNF to virtual switch + to VNF to virtual switch to physical port</preamble> + + <artwork><![CDATA[ __ + +----------------------+ +----------------------+ | + | Guest 1 | | Guest 2 | | + | +---------------+ | | +---------------+ | | + | | Application | | | | Application | | | + | +---------------+ | | +---------------+ | | + | ^ | | | ^ | | | + | | v | | | v | | Guests + | +---------------+ | | +---------------+ | | + | | logical ports | | | | logical ports | | | + | | 0 1 | | | | 0 1 | | | + +---+---------------+--+ +---+---------------+--+__| + ^ : ^ : + | | | | + : v : v _ + +---+---------------+---------+---------------+--+ | + | | 0 1 | | 3 4 | | | + | | logical ports | | logical ports | | | + | +---------------+ +---------------+ | | + | ^ | ^ | | | Host + | | L-----------------+ v | | + | +--------------+ +--------------+ | | + | | phy ports | vSwitch | phy ports | | | + +---+--------------+----------+--------------+---+_| + ^ : ^ : + | | | | + : v : v + +--------------------------------------------------+ + | | + | traffic generator | + | | + +--------------------------------------------------+]]></artwork> + </figure><figure> + <preamble>Physical port to virtual switch to VNF</preamble> + + <artwork><![CDATA[ __ + +---------------------------------------------------+ | + | | | + | +-------------------------------------------+ | | + | | Application | | | + | +-------------------------------------------+ | | + | ^ | | + | | | | Guest + | : | | + | +---------------+ | | + | | logical port 0| | | + +---+---------------+-------------------------------+ __| + ^ + | + : __ + +---+---------------+------------------------------+ | + | | logical port 0| | | + | +---------------+ | | + | ^ | | + | | | | Host + | : | | + | +--------------+ | | + | | phy port | vSwitch | | + +---+--------------+------------ -------------- ---+ __| + ^ + | + : + +--------------------------------------------------+ + | | + | traffic generator | + | | + +--------------------------------------------------+]]></artwork> + </figure><figure> + <preamble>VNF to virtual switch to physical port</preamble> + + <artwork><![CDATA[ __ + +---------------------------------------------------+ | + | | | + | +-------------------------------------------+ | | + | | Application | | | + | +-------------------------------------------+ | | + | : | | + | | | | Guest + | v | | + | +---------------+ | | + | | logical port | | | + +-------------------------------+---------------+---+ __| + : + | + v __ + +------------------------------+---------------+---+ | + | | logical port | | | + | +---------------+ | | + | : | | + | | | | Host + | v | | + | +--------------+ | | + | vSwitch | phy port | | | + +-------------------------------+--------------+---+ __| + : + | + v + +--------------------------------------------------+ + | | + | traffic generator | + | | + +--------------------------------------------------+]]></artwork> + </figure><figure> + <preamble> VNF to virtual switch to VNF</preamble> + + <artwork><![CDATA[ __ + +----------------------+ +----------------------+ | + | Guest 1 | | Guest 2 | | + | +---------------+ | | +---------------+ | | + | | Application | | | | Application | | | + | +---------------+ | | +---------------+ | | + | | | | ^ | | + | v | | | | | Guests + | +---------------+ | | +---------------+ | | + | | logical ports | | | | logical ports | | | + | | 0 | | | | 0 | | | + +---+---------------+--+ +---+---------------+--+__| + : ^ + | | + v : _ + +---+---------------+---------+---------------+--+ | + | | 1 | | 1 | | | + | | logical ports | | logical ports | | | + | +---------------+ +---------------+ | | + | | ^ | | Host + | L-----------------+ | | + | | | + | vSwitch | | + +------------------------------------------------+_| + +]]></artwork> + </figure></t> + </section> + + <section title="Security Considerations"> + <t>Benchmarking activities as described in this memo are limited to + technology characterization of a Device Under Test/System Under Test + (DUT/SUT) using controlled stimuli in a laboratory environment, with + dedicated address space and the constraints specified in the sections + above.</t> + + <t>The benchmarking network topology will be an independent test setup + and MUST NOT be connected to devices that may forward the test traffic + into a production network, or misroute traffic to the test management + network.</t> + + <t>Further, benchmarking is performed on a "black-box" basis, relying + solely on measurements observable external to the DUT/SUT.</t> + + <t>Special capabilities SHOULD NOT exist in the DUT/SUT specifically for + benchmarking purposes. Any implications for network security arising + from the DUT/SUT SHOULD be identical in the lab and in production + networks.</t> + </section> + + <section anchor="IANA" title="IANA Considerations"> + <t>No IANA Action is requested at this time.</t> + </section> + + <section title="Acknowledgements"> + <t>The authors acknowledge</t> + </section> + </middle> + + <back> + <references title="Normative References"> + <?rfc ?> + + <?rfc include="reference.RFC.2119"?> + + <?rfc include="reference.RFC.2330"?> + + <?rfc include='reference.RFC.2544'?> + + <?rfc include="reference.RFC.2679"?> + + <?rfc include='reference.RFC.2680'?> + + <?rfc include='reference.RFC.3393'?> + + <?rfc include='reference.RFC.3432'?> + + <?rfc include='reference.RFC.2681'?> + + <?rfc include='reference.RFC.5905'?> + + <?rfc include='reference.RFC.4689'?> + + <?rfc include='reference.RFC.4737'?> + + <?rfc include='reference.RFC.5357'?> + + <reference anchor="NFV.PER001"> + <front> + <title>Network Function Virtualization: Performance and Portability + Best Practices</title> + + <author fullname="ETSI NFV" initials="" surname=""> + <organization/> + </author> + + <date month="June" year="2014"/> + </front> + + <seriesInfo name="Group Specification" + value="ETSI GS NFV-PER 001 V1.1.1 (2014-06)"/> + + <format type="PDF"/> + </reference> + </references> + + <references title="Informative References"> + <?rfc include='reference.RFC.1242'?> + + <?rfc include='reference.RFC.5481'?> + + <?rfc include='reference.RFC.6049'?> + + <?rfc include='reference.RFC.6248'?> + + <?rfc include='reference.RFC.6390'?> + + <?rfc include='reference.ID.ietf-bmwg-virtual-net'?> + </references> + </back> +</rfc> diff --git a/test_spec/vswitchperf_ltd.md b/test_spec/vswitchperf_ltd.md index 3090e6ad..7fbd2cc2 100644 --- a/test_spec/vswitchperf_ltd.md +++ b/test_spec/vswitchperf_ltd.md @@ -166,38 +166,40 @@ The following represents possible deployments which can help to determine the pe - Physical port → virtual switch → VNF → virtual switch → VNF → virtual switch → physical port. <pre><code> - __ - +---------------------------------------------------+ +---------------------------------------------------+ | - | Guest 1 | | Guest 2 | | - | +-------------------------------------------+ | | +-------------------------------------------+ | | - | | Application | | | | Application | | | - | +-------------------------------------------+ | | +-------------------------------------------+ | | - | ^ : | | ^ : | | - | | | | | | | | | Guest - | : v | | : v | | - | +---------------+ +---------------+ | | +---------------+ +---------------+ | | - | | logical port 0| | logical port 1| | | | logical port 0| | logical port 1| | | - +---+---------------+-----------+---------------+---+ +---+---------------+-----------+---------------+---+__| - ^ : ^ : - | | | | - : v : v __ - +---+---------------+----------+---------------+------------+---------------+-----------+---------------+---+ | - | | port 0 | | port 1 | | port 2 | | port 3 | | | - | +---------------+ +---------------+ +---------------+ +---------------+ | | - | ^ : ^ : | | - | | | | | | | Host - | : +--------------------+ v | | - | +--------------+ +--------------+ | | - | | phy port | vswitch | phy port | | | - +---+--------------+--------------------------------------------------------------------+--------------+----+__| - ^ : - | | - : v - +-----------------------------------------------------------------------------------------------------------+ - | | - | traffic generator | - | | - +-----------------------------------------------------------------------------------------------------------+ + + __ + +----------------------+ +----------------------+ | + | Guest 1 | | Guest 2 | | + | +---------------+ | | +---------------+ | | + | | Application | | | | Application | | | + | +---------------+ | | +---------------+ | | + | ^ | | | ^ | | | + | | v | | | v | | Guests + | +---------------+ | | +---------------+ | | + | | logical ports | | | | logical ports | | | + | | 0 1 | | | | 0 1 | | | + +---+---------------+--+ +---+---------------+--+__| + ^ : ^ : + | | | | + : v : v _ + +---+---------------+---------+---------------+--+ | + | | 0 1 | | 3 4 | | | + | | logical ports | | logical ports | | | + | +---------------+ +---------------+ | | + | ^ | ^ | | | Host + | | L-----------------+ v | | + | +--------------+ +--------------+ | | + | | phy ports | vSwitch | phy ports | | | + +---+--------------+----------+--------------+---+_| + ^ : ^ : + | | | | + : v : v + +--------------------------------------------------+ + | | + | traffic generator | + | | + +--------------------------------------------------+ + </code></pre> - Physical port → virtual switch → VNF. |