diff options
Diffstat (limited to 'docs/requirements')
| -rwxr-xr-x | docs/requirements/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-02.txt | 42 | ||||
| -rw-r--r-- | docs/requirements/vswitchperf_ltd.rst | 114 |
2 files changed, 120 insertions, 36 deletions
diff --git a/docs/requirements/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-02.txt b/docs/requirements/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-02.txt index 317a68ab..0f5be592 100755 --- a/docs/requirements/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-02.txt +++ b/docs/requirements/ietf_draft/draft-vsperf-bmwg-vswitch-opnfv-02.txt @@ -54,7 +54,7 @@ Status of This Memo Tahhan, et al. Expires September 22, 2016 [Page 1] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -110,7 +110,7 @@ Table of Contents Tahhan, et al. Expires September 22, 2016 [Page 2] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -166,7 +166,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 3] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -222,7 +222,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 4] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -278,7 +278,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 5] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -334,7 +334,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 6] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -390,7 +390,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 7] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -446,7 +446,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 8] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -502,7 +502,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 9] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -558,7 +558,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 10] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -614,7 +614,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 11] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -670,7 +670,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 12] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -726,7 +726,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 13] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -782,7 +782,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 14] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -838,7 +838,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 15] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -894,7 +894,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 16] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -950,7 +950,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 17] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -1006,7 +1006,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 18] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -1062,7 +1062,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 19] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -1118,7 +1118,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 20] - + Internet-Draft Benchmarking vSwitches March 2016 @@ -1174,7 +1174,7 @@ Internet-Draft Benchmarking vSwitches March 2016 Tahhan, et al. Expires September 22, 2016 [Page 21] - + Internet-Draft Benchmarking vSwitches March 2016 diff --git a/docs/requirements/vswitchperf_ltd.rst b/docs/requirements/vswitchperf_ltd.rst index 1827fe6d..6b882290 100644 --- a/docs/requirements/vswitchperf_ltd.rst +++ b/docs/requirements/vswitchperf_ltd.rst @@ -1285,11 +1285,11 @@ Test ID: LTD.Throughput.RFC2544.Profile to the DUT's RFC 2544 Throughput as determined by LTD.Throughput.RFC2544.PacketLoss Ratio (0% Packet Loss case). A delta of 0% is equivalent to an offered traffic rate equal to the RFC 2544 - Throughput; A delta of +50% indicates an offered rate half-way - between the Throughput and line-rate, whereas a delta of - -50% indicates an offered rate of half the maximum rate. Therefore the - range of the delta figure is natuarlly bounded at -100% (zero offered - traffic) and +100% (traffic offered at line rate). + Maximum Throughput; A delta of +50% indicates an offered rate half-way + between the Maximum RFC2544 Throughput and line-rate, whereas a delta of + -50% indicates an offered rate of half the RFC 2544 Maximum Throughput. + Therefore the range of the delta figure is natuarlly bounded at -100% + (zero offered traffic) and +100% (traffic offered at line rate). The following deltas to the maximum forwarding rate should be applied: @@ -1861,12 +1861,93 @@ Test ID: LTD.Throughput.RFC2544.WorstN-BestN `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__). - Following may also be collected as part of this test, to determine the vSwitch's performance footprint on the system: + - CPU core utilization. - CPU cache utilization. - Memory footprint. - System bus (QPI, PCI, ...) utilization. - CPU cycles consumed per packet. +.. 3.2.3.1.14 + +Test ID: LTD.Throughput.Overlay.Network.<tech>.RFC2544.PacketLossRatio +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + **Title**: <tech> Overlay Network RFC 2544 X% packet loss ratio Throughput and Latency Test + + + NOTE: Throughout this test, four interchangeable overlay technologies are covered by the + same test description. They are: VXLAN, GRE, NVGRE and GENEVE. + + **Prerequisite Test**: N/A + + **Priority**: + + **Description**: + This test evaluates standard switch performance benchmarks for the scenario where an + Overlay Network is deployed for all paths through the vSwitch. Overlay Technologies covered + (replacing <tech> in the test name) include: + + - VXLAN + - GRE + - NVGRE + - GENEVE + + Performance will be assessed for each of the following overlay network functions: + + - Encapsulation only + - De-encapsulation only + - Both Encapsulation and De-encapsulation + + For each native packet, the DUT must perform the following operations: + + - Examine the packet and classify its correct overlay net (tunnel) assignment + - Encapsulate the packet + - Switch the packet to the correct port + + For each encapsulated packet, the DUT must perform the following operations: + + - Examine the packet and classify its correct native network assignment + - De-encapsulate the packet, if required + - Switch the packet to the correct port + + The selected frame sizes are those previously defined under `Default + Test Parameters <#DefaultParams>`__. + + Thus, each test comprises an overlay technology, a network function, + and a packet size *with* overlay network overhead included + (but see also the discussion at + https://etherpad.opnfv.org/p/vSwitchTestsDrafts ). + + The test can also be used to determine the average latency of the traffic. + + Under the `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__ + test methodology, the test duration will + include a number of trials; each trial should run for a minimum period + of 60 seconds. A binary search methodology must be applied for each + trial to obtain the final result for Throughput. + + **Expected Result**: At the end of each trial, the presence or absence + of loss determines the modification of offered load for the next trial, + converging on a maximum rate, or + `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__ Throughput with X% + loss (where the value of X is typically equal to zero). + The Throughput load is re-used in related + `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__ tests and other + tests. + + **Metrics Collected**: + The following are the metrics collected for this test: + + - The maximum Throughput in Frames Per Second (FPS) and Mbps of + the DUT for each frame size with X% packet loss. + - The average latency of the traffic flow when passing through the DUT + and VNFs (if testing for latency, note that this average is different from the + test specified in Section 26.3 of + `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__). + - CPU and memory utilization may also be collected as part of this + test, to determine the vSwitch's performance footprint on the system. + + .. 3.2.3.2 Packet Latency tests @@ -1969,9 +2050,9 @@ It is expected that more will be added. .. 3.2.3.3.1 -Test ID: LTD.Scalability.RFC2544.0PacketLoss +Test ID: LTD.Scalability.Flows.RFC2544.0PacketLoss ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - **Title**: RFC 2544 0% loss Scalability throughput test + **Title**: RFC 2544 0% loss Flow Scalability throughput test **Prerequisite Test**: LTD.Throughput.RFC2544.PacketLossRatio, IF the delta Throughput between the single-flow RFC2544 test and this test with @@ -2219,16 +2300,16 @@ Test ID: LTD.Scalability.VNF.RFC2544.PacketLossProfile The following are the metrics collected for this test: - The forwarding rate in Frames Per Second (FPS) and Mbps of the DUT - for each delta to the maximum forwarding rate and for each frame - size. + for each delta to the maximum forwarding rate and for each frame + size. - The average latency for each delta to the maximum forwarding rate and - for each frame size. + for each frame size. - CPU and memory utilization may also be collected as part of this - test, to determine the vSwitch's performance footprint on the system. + test, to determine the vSwitch's performance footprint on the system. - Any failures experienced (for example if the vSwitch crashes, stops processing packets, restarts or becomes unresponsive to commands) - when the offered load is above Maximum Throughput MUST be recorded - and reported with the results. + when the offered load is above Maximum Throughput MUST be recorded + and reported with the results. .. 3.2.3.4 @@ -2395,7 +2476,7 @@ should be required. It is expected that more will be added. .. 3.2.3.6.1 Test ID: LTD.Stress.RFC2544.0PacketLoss -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ **Title**: RFC 2544 0% Loss CPU OR Memory Stress Test **Prerequisite Test**: @@ -2429,6 +2510,7 @@ Test ID: LTD.Stress.RFC2544.0PacketLoss **Note:** Stress in the test ID can be replaced with the name of the component being stressed, when reporting the results: LTD.CPU.RFC2544.0PacketLoss or LTD.Memory.RFC2544.0PacketLoss + .. 3.2.3.7 Summary List of Tests @@ -2447,6 +2529,8 @@ Summary List of Tests - Test ID: LTD.Throughput.RFC2889.ForwardPressure - Test ID: LTD.Throughput.RFC2889.ErrorFramesFiltering - Test ID: LTD.Throughput.RFC2889.BroadcastFrameForwarding + - Test ID: LTD.Throughput.RFC2544.WorstN-BestN + - Test ID: LTD.Throughput.Overlay.Network.<tech>.RFC2544.PacketLossRatio 2. Packet Latency tests @@ -2455,7 +2539,7 @@ Summary List of Tests 3. Scalability tests - - Test ID: LTD.Scalability.RFC2544.0PacketLoss + - Test ID: LTD.Scalability.Flows.RFC2544.0PacketLoss - Test ID: LTD.MemoryBandwidth.RFC2544.0PacketLoss.Scalability - LTD.Scalability.VNF.RFC2544.PacketLossProfile - LTD.Scalability.VNF.RFC2544.PacketLossRatio |