diff options
| -rw-r--r-- | docs/requirements/vswitchperf_ltd.rst | 91 |
1 files changed, 91 insertions, 0 deletions
diff --git a/docs/requirements/vswitchperf_ltd.rst b/docs/requirements/vswitchperf_ltd.rst index 4a08bdd6..77396121 100644 --- a/docs/requirements/vswitchperf_ltd.rst +++ b/docs/requirements/vswitchperf_ltd.rst @@ -1759,6 +1759,97 @@ Test ID: LTD.Throughput.RFC2889.BroadcastFrameForwarding four test ports are required. One of the ports is connected to the test device, so it can send broadcast frames and listen for miss-routed frames. +.. 3.2.3.1.13 + +Test ID: LTD.Throughput.RFC2544.WorstN-BestN +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + **Title**: Modified RFC 2544 X% packet loss ratio Throughput and Latency Test + + **Prerequisite Test**: N/A + + **Priority**: + + **Description**: + + This test determines the DUT's maximum forwarding rate with X% traffic + loss for a constant load (fixed length frames at a fixed interval time). + The default loss percentages to be tested are: X = 0%, X = 10^-7% + + Modified RFC 2544 throughput benchmarking methodology aims to quantify + the throughput measurement variations observed during standard RFC 2544 + benchmarking measurements of virtual switches and VNFs. The RFC2544 + binary search algorithm is modified to use more samples per test trial + to drive the binary search and yield statistically more meaningful + results. This keeps the heart of the RFC2544 methodology, still relying + on the binary search of throughput at specified loss tolerance, while + providing more useful information about the range of results seen in + testing. Instead of using a single traffic trial per iteration step, + each traffic trial is repeated N times and the success/failure of the + iteration step is based on these N traffic trials. Two types of revised + tests are defined - *Worst-of-N* and *Best-of-N*. + + **Worst-of-N** + + *Worst-of-N* indicates the lowest expected maximum throughput for ( + packet size, loss tolerance) when repeating the test. + + 1. Repeat the same test run N times at a set packet rate, record each + result. + 2. Take the WORST result (highest packet loss) out of N result samples, + called the Worst-of-N sample. + 3. If Worst-of-N sample has loss less than the set loss tolerance, then + the step is successful - increase the test traffic rate. + 4. If Worst-of-N sample has loss greater than the set loss tolerance + then the step failed - decrease the test traffic rate. + 5. Go to step 1. + + **Best-of-N** + + *Best-of-N* indicates the highest expected maximum throughput for ( + packet size, loss tolerance) when repeating the test. + + 1. Repeat the same traffic run N times at a set packet rate, record + each result. + 2. Take the BEST result (least packet loss) out of N result samples, + called the Best-of-N sample. + 3. If Best-of-N sample has loss less than the set loss tolerance, then + the step is successful - increase the test traffic rate. + 4. If Best-of-N sample has loss greater than the set loss tolerance, + then the step failed - decrease the test traffic rate. + 5. Go to step 1. + + Performing both Worst-of-N and Best-of-N benchmark tests yields lower + and upper bounds of expected maximum throughput under the operating + conditions, giving a very good indication to the user of the + deterministic performance range for the tested setup. + + **Expected Result**: At the end of each trial series, the presence or + absence of loss determines the modification of offered load for the + next trial series, converging on a maximum rate, or + `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__ Throughput + with X% loss. + The Throughput load is re-used in related + `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__ tests and other + tests. + + **Metrics Collected**: + + The following are the metrics collected for this test: + + - The maximum forwarding rate in Frames Per Second (FPS) and Mbps of + the DUT for each frame size with X% packet loss. + - The average latency of the traffic flow when passing through the DUT + (if testing for latency, note that this average is different from the + test specified in Section 26.3 of + `RFC2544 <https://www.rfc-editor.org/rfc/rfc2544.txt>`__). + - Following may also be collected as part of this test, to determine + the vSwitch's performance footprint on the system: + - CPU core utilization. + - CPU cache utilization. + - Memory footprint. + - System bus (QPI, PCI, ...) utilization. + - CPU cycles consumed per packet. + .. 3.2.3.2 Packet Latency tests |