From 3f51477c1ede2eef73b032ff326b532a1e7f638f Mon Sep 17 00:00:00 2001 From: Al Morton Date: Wed, 14 Oct 2015 20:49:08 -0400 Subject: test_spec: IETF Draft: Update to reflect dev since July and IETF-93 comments Among the Updates needed are: - Several New tests created since July, add to Matrix section. - Re-named tests: fix in Matrix section. - Link to Summaries on Wiki. - Link to Test Setup Figures on Wiki. - Add some general points on New Test Equipment Challenges/Metrics. - Mention future test specs, and IFA003 as one source. JIRA: VSPERF-120 Change-Id: I981b0b6301a208d20220f67784b4591aa3f51338 Signed-off-by: Al Morton Reviewed-by: Maryam Tahhan Reviewed-by: Billy O'Mahony Reviewed-by: Gene Snider --- .../draft-vsperf-bmwg-vswitch-opnfv-00.xml | 109 ++++++++++++++++----- 1 file changed, 83 insertions(+), 26 deletions(-) mode change 100644 => 100755 docs/to-be-reorganized/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-00.xml (limited to 'docs/to-be-reorganized/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-00.xml') diff --git a/docs/to-be-reorganized/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-00.xml b/docs/to-be-reorganized/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-00.xml old mode 100644 new mode 100755 index d8351957..b5f7f833 --- a/docs/to-be-reorganized/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-00.xml +++ b/docs/to-be-reorganized/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-00.xml @@ -10,7 +10,7 @@ - Benchmarking Virtual Switches in @@ -94,7 +94,7 @@ </address> </author> - <date day="3" month="July" year="2015"/> + <date day="14" month="October" year="2015"/> <abstract> <t>This memo describes the progress of the Open Platform for NFV (OPNFV) @@ -134,8 +134,8 @@ project on virtual switch performance characterization, "VSWITCHPERF". This project intends to build on the current and completed work of the Benchmarking Methodology Working Group in IETF, by referencing existing - literature. For example, currently the most referenced RFC is <xref - target="RFC2544"/> (which depends on <xref target="RFC1242"/>) and + literature. For example, currently the most often referenced RFC is + <xref target="RFC2544"/> (which depends on <xref target="RFC1242"/>) and foundation of the benchmarking work in OPNFV is common and strong.</t> <t>See @@ -303,7 +303,7 @@ the parameters the vSwitch uses to match for a flow. The recommended flow classification parameters for any vSwitch performance tests are: the input port, the source IP address, the destination IP address and - the ethernet protocol type field. It is essential to increase the flow + the Ethernet protocol type field. It is essential to increase the flow timeout time on a vSwitch before conducting any performance tests that do not measure the flow setup time. Normally the first packet of a particular stream will install the flow in the virtual switch which @@ -423,7 +423,11 @@ Statement</t> </list></t> - <t/> + <t>Some of the above/newer RFCs are being applied in benchmarking for + the first time, and represent a development challenge for test equipment + developers. Fortunately, many members of the testing system community + have engaged on the VSPERF project, including an open source test + system.</t> <t>In addition to this, the LTD also re-uses the terminology defined by:</t> @@ -505,6 +509,10 @@ <t>Noisy Neighbour Tests, to understand the effects of resource sharing on the performance of a virtual switch.</t> + + <t>Tests derived from examination of ETSI NFV Draft GS IFA003 + requirements <xref target="IFA003"/> on characterization of + acceleration technologies applied to vswitches.</t> </list>The flexibility of deployment of a virtual switch within a network means that the BMWG IETF existing literature needs to be used to characterize the performance of a switch in various deployment @@ -702,6 +710,9 @@ | vSwitch | | +------------------------------------------------+_|]]></artwork> </figure></t> + + <t>A set of Deployment Scenario figures is available on the VSPERF Test + Methodology Wiki page <xref target="TestTopo"/>. </t> </section> <section title="3x3 Matrix Coverage"> @@ -709,24 +720,26 @@ "3x3" matrix (introduced in <xref target="I-D.ietf-bmwg-virtual-net"/>). Because the LTD specification ID names are quite long, this section is organized into lists for each occupied cell of the matrix (not all are - occupied, also the matrix has grown to 3x4 to accommodate scale - metrics).</t> + occupied, also the matrix has grown to 3x4 to accommodate scale metrics + when displaying the coverage of many metrics/benchmarks).</t> <t>The tests listed below assess the activation of paths in the data plane, rather than the control plane.</t> - <t>(Editor's Note: a complete list of tests is available here: - https://wiki.opnfv.org/wiki/vswitchperf_test_spec_review )</t> + <t>A complete list of tests with short summaries is available on the + VSPERF "LTD Test Spec Overview" Wiki page <xref target="LTDoverV"/>.</t> <section title="Speed of Activation"> <t><list style="symbols"> - <t>Throughput.RFC2889.AddressLearningRate</t> - - <t>Throughput.RFC2889.AddressCachingCapacity</t> + <t>Activation.RFC2889.AddressLearningRate</t> <t>PacketLatency.InitialPacketProcessingLatency</t> + </list></t> + </section> - <t/> + <section title="Accuracy of Activation section"> + <t><list style="symbols"> + <t>CPDP.Coupling.Flow.Addition</t> </list></t> </section> @@ -740,9 +753,7 @@ <section title="Scale of Activation"> <t><list style="symbols"> - <t>Throughput.RFC2889.AddressCachingCapacity</t> - - <t/> + <t>Activation.RFC2889.AddressCachingCapacity</t> </list></t> </section> @@ -750,17 +761,17 @@ <t><list style="symbols"> <t>Throughput.RFC2544.PacketLossRate</t> + <t>CPU.RFC2544.0PacketLoss</t> + <t>Throughput.RFC2544.PacketLossRateFrameModification</t> <t>Throughput.RFC2544.BackToBackFrames</t> - <t>Throughput.RFC2889.ForwardingRate</t> + <t>Throughput.RFC2889.MaxForwardingRate</t> <t>Throughput.RFC2889.ForwardPressure</t> <t>Throughput.RFC2889.BroadcastFrameForwarding</t> - - <t>RFC2889 Broadcast Frame Latency test</t> </list></t> </section> @@ -768,17 +779,25 @@ <t><list style="symbols"> <t>Throughput.RFC2889.ErrorFramesFiltering</t> - <t/> + <t>Throughput.RFC2544.Profile</t> </list></t> </section> <section title="Reliability of Operation"> <t><list style="symbols"> - <t>Throughput.RFC2544.Soak</t> + <t>Throughput.RFC2889.Soak</t> - <t>Throughput.RFC2544.SoakFrameModification</t> + <t>Throughput.RFC2889.SoakFrameModification</t> - <t/> + <t>PacketDelayVariation.RFC3393.Soak</t> + </list></t> + </section> + + <section title="Scalability of Operation"> + <t><list style="symbols"> + <t>Scalability.RFC2544.0PacketLoss</t> + + <t>MemoryBandwidth.RFC2544.0PacketLoss.Scalability</t> </list></t> </section> @@ -790,11 +809,11 @@ | | | | | | |------------------------------------------------------------------------| | | | | | | -| Activation | X | | X | X | +| Activation | X | X | X | X | | | | | | | |------------------------------------------------------------------------| | | | | | | -| Operation | X | X | X | | +| Operation | X | X | X | X | | | | | | | |------------------------------------------------------------------------| | | | | | | @@ -902,6 +921,44 @@ <?rfc include='reference.RFC.6390'?> <?rfc include='reference.I-D.ietf-bmwg-virtual-net'?> + + <reference anchor="TestTopo"> + <front> + <title>Test Topologies + https://wiki.opnfv.org/vsperf/test_methodology + + + + + + + + + + + + LTD Test Spec Overview + https://wiki.opnfv.org/wiki/vswitchperf_test_spec_review + + + + + + + + + + + + https://docbox.etsi.org/ISG/NFV/Open/Drafts/IFA003_Acceleration_-_vSwitch_Spec/ + + + + + + + + -- cgit 1.2.3-korg