IETF Draft: Address IETF-94 BMWG meeting Comments

Several Comments from the IETF-94 meeting, many are captured in the meeting notes.
Section 6 of the meeting notes gives the comments:
https://www.ietf.org/proceedings/94/minutes/minutes-94-bmwg

JIRA: VSPERF-246

Change-Id: Ibe77b67789024ac48cce00147ce94cf2270d8106
Signed-off-by: Al Morton <acmorton@att.com>
Reviewed-by: Maryam Tahhan <maryam.tahhan@intel.com>
Reviewed-by: Martin Klozik <martinx.klozik@intel.com>
Reviewed-by: Christian Trautman <ctrautma@redhat.com>

1 files changed, 75 insertions, 23 deletions

diff --git a/docs/requirements/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-01.xml b/docs/requirements/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-01.xml
index b5f7f833..a9405a77 100755
--- a/docs/requirements/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-01.xml
+++ b/docs/requirements/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-01.xml
@@ -10,7 +10,7 @@
 <?rfc inline="yes"?>
 <?rfc compact="yes"?>
 <?rfc subcompact="no"?>
-<rfc category="info" docName="draft-vsperf-bmwg-vswitch-opnfv-01"
+<rfc category="info" docName="draft-vsperf-bmwg-vswitch-opnfv-02"
      ipr="trust200902">
   <front>
     <title abbrev="Benchmarking vSwitches">Benchmarking Virtual Switches in
@@ -94,7 +94,7 @@
       </address>
     </author>
 
-    <date day="14" month="October" year="2015"/>
+    <date day="20" month="March" year="2016"/>
 
     <abstract>
       <t>This memo describes the progress of the Open Platform for NFV (OPNFV)
@@ -130,13 +130,15 @@
       the goal to transform how internetwork functions are implemented, and
       therefore has garnered much attention.</t>
 
-      <t>This memo describes the progress of the Open Platform for NFV (OPNFV)
-      project on virtual switch performance characterization, "VSWITCHPERF".
-      This project intends to build on the current and completed work of the
-      Benchmarking Methodology Working Group in IETF, by referencing existing
-      literature. For example, currently the most often referenced RFC is
-      <xref target="RFC2544"/> (which depends on <xref target="RFC1242"/>) and
-      foundation of the benchmarking work in OPNFV is common and strong.</t>
+      <t>This memo summarizes the progress of the Open Platform for NFV
+      (OPNFV) project on virtual switch performance characterization,
+      "VSWITCHPERF", through the Brahmaputra (second) release <xref
+      target="BrahRel"/>. This project intends to build on the current and
+      completed work of the Benchmarking Methodology Working Group in IETF, by
+      referencing existing literature. For example, currently the most often
+      referenced RFC is <xref target="RFC2544"/> (which depends on <xref
+      target="RFC1242"/>) and foundation of the benchmarking work in OPNFV is
+      common and strong.</t>
 
       <t>See
       https://wiki.opnfv.org/characterize_vswitch_performance_for_telco_nfv_use_cases
@@ -151,19 +153,25 @@
       isochronous media streams is one example.</t>
 
       <t>Note also that the move to NFV Infrastructure has resulted in many
-      new benchmarking initiatives across the industry, and the authors are
+      new benchmarking initiatives across the industry. The authors are
       currently doing their best to maintain alignment with many other
-      projects, and this Internet Draft is evidence of the efforts.</t>
+      projects, and this Internet Draft is one part of the efforts. We
+      acknowledge the early work in <xref
+      target="I-D.huang-bmwg-virtual-network-performance"/>, and useful
+      discussion with the authors.</t>
     </section>
 
     <section title="Scope">
-      <t>The primary purpose and scope of the memo is to inform BMWG of
-      work-in-progress that builds on the body of extensive literature and
-      experience. Additionally, once the initial information conveyed here is
-      received, this memo may be expanded to include more detail and
-      commentary from both BMWG and OPNFV communities, under BMWG's chartered
-      work to characterize the NFV Infrastructure (a virtual switch is an
-      important aspect of that infrastructure).</t>
+      <t>The primary purpose and scope of the memo is to inform the industry
+      of work-in-progress that builds on the body of extensive BMWG literature
+      and experience, and describe the extensions needed for benchmarking
+      virtual switches. Inital feedback indicates that many of these
+      extensions may be applicable beyond the current scope (to hardware
+      switches in the NFV Infrastructure and to virtual routers, for example).
+      Additionally, this memo serves as a vehicle to include more detail and
+      commentary from BMWG and other Open Source communities, under BMWG's
+      chartered work to characterize the NFV Infrastructure (a virtual switch
+      is an important aspect of that infrastructure).</t>
     </section>
 
     <section title="Benchmarking Considerations">
@@ -474,7 +482,7 @@
       <t><list style="symbols">
           <t>Throughput Tests to measure the maximum forwarding rate (in
           frames per second or fps) and bit rate (in Mbps) for a constant load
-          (as defined by RFC1242) without traffic loss.</t>
+          (as defined by <xref target="RFC1242"/>) without traffic loss.</t>
 
           <t>Packet and Frame Delay Distribution Tests to measure average, min
           and max packet and frame delay for constant loads.</t>
@@ -501,6 +509,17 @@
           switch&rsquo;s footprint on the system, usually conducted as
           auxiliary measurements with benchmarks above. They include: CPU
           utilization, Cache utilization and Memory footprint.</t>
+
+          <t>The so-called "Soak" tests, where the selected test is conducted
+          over a long period of time (with an ideal duration of 24 hours, and
+          at least 6 hours). The purpose of soak tests is to capture transient
+          changes in performance which may occur due to infrequent processes
+          or the low probability coincidence of two or more processes. The
+          performance must be evaluated periodically during continuous
+          testing, and this results in use of <xref target="RFC2889"/> Frame
+          Rate metrics instead of <xref target="RFC2544"/> Throughput (which
+          requires stopping traffic to allow time for all traffic to exit
+          internal queues).</t>
         </list></t>
 
       <t>Future/planned test specs include:<list style="symbols">
@@ -712,7 +731,7 @@
         </figure></t>
 
       <t>A set of Deployment Scenario figures is available on the VSPERF Test
-      Methodology Wiki page <xref target="TestTopo"/>. </t>
+      Methodology Wiki page <xref target="TestTopo"/>.</t>
     </section>
 
     <section title="3x3 Matrix Coverage">
@@ -721,7 +740,8 @@
       Because the LTD specification ID names are quite long, this section is
       organized into lists for each occupied cell of the matrix (not all are
       occupied, also the matrix has grown to 3x4 to accommodate scale metrics
-      when displaying the coverage of many metrics/benchmarks).</t>
+      when displaying the coverage of many metrics/benchmarks). The current
+      version of the LTD specification is available <xref target="LTD"/>.</t>
 
       <t>The tests listed below assess the activation of paths in the data
       plane, rather than the control plane.</t>
@@ -850,7 +870,9 @@
     </section>
 
     <section title="Acknowledgements">
-      <t>The authors acknowledge</t>
+      <t>The authors appreciate and acknowledge comments from Scott Bradner,
+      Marius Georgescu, Ramki Krishnan, and Doug Montgomery, and others for
+      their reviews.</t>
     </section>
   </middle>
 
@@ -860,6 +882,8 @@
 
       <?rfc include="reference.RFC.2119"?>
 
+      <?rfc ?>
+
       <?rfc include="reference.RFC.2330"?>
 
       <?rfc include='reference.RFC.2544'?>
@@ -922,6 +946,8 @@
 
       <?rfc include='reference.I-D.ietf-bmwg-virtual-net'?>
 
+      <?rfc include='reference.I-D.huang-bmwg-virtual-network-performance'?>
+
       <reference anchor="TestTopo">
         <front>
           <title>Test Topologies
@@ -938,7 +964,33 @@
       <reference anchor="LTDoverV">
         <front>
           <title>LTD Test Spec Overview
-          https://wiki.opnfv.org/wiki/vswitchperf_test_spec_review </title>
+          https://wiki.opnfv.org/wiki/vswitchperf_test_spec_review</title>
+
+          <author>
+            <organization/>
+          </author>
+
+          <date/>
+        </front>
+      </reference>
+
+      <reference anchor="LTD">
+        <front>
+          <title>LTD Test Specification
+          http://artifacts.opnfv.org/vswitchperf/docs/requirements/index.html</title>
+
+          <author>
+            <organization/>
+          </author>
+
+          <date/>
+        </front>
+      </reference>
+
+      <reference anchor="BrahRel">
+        <front>
+          <title>Brahmaputra, Second OPNFV Release
+          https://www.opnfv.org/brahmaputra</title>
 
           <author>
             <organization/>