diff options
Diffstat (limited to 'docs/testing')
| -rw-r--r-- | docs/testing/developer/devguide/requirements/vswitchperf_ltd.rst | 108 | ||||
| -rw-r--r-- | docs/testing/user/configguide/trafficgen.rst | 25 |
2 files changed, 114 insertions, 19 deletions
diff --git a/docs/testing/developer/devguide/requirements/vswitchperf_ltd.rst b/docs/testing/developer/devguide/requirements/vswitchperf_ltd.rst index e1372520..c703ff40 100644 --- a/docs/testing/developer/devguide/requirements/vswitchperf_ltd.rst +++ b/docs/testing/developer/devguide/requirements/vswitchperf_ltd.rst @@ -413,7 +413,21 @@ Test ID: LTD.Throughput.RFC2889.MaxForwardingRateSoak **Title**: RFC 2889 X% packet loss Max Forwarding Rate Soak Test - **Prerequisite Test** LTD.Throughput.RFC2544.PacketLossRatio + **Prerequisite Tests**: + + LTD.Throughput.RFC2544.PacketLossRatio will determine the offered load and + frame size for which the maximum theoretical throughput of the interface + has not been achieved. As described in RFC 2544 section 24, the final + determination of the benchmark SHOULD be conducted using a full length + trial, and for this purpose the duration is 5 minutes with zero loss ratio. + + It is also essential to verify that the Traffic Generator has sufficient + stability to conduct Soak tests. Therefore, a prerequisite is to perform + this test with the DUT removed and replaced with a cross-over cable (or + other equivalent very low overhead method such as a loopback in a HW switch), + so that the traffic generator (and any other network involved) can be tested + over the Soak period. Note that this test may be challenging for software- + based traffic generators. **Priority**: @@ -422,12 +436,19 @@ Test ID: LTD.Throughput.RFC2889.MaxForwardingRateSoak The aim of this test is to understand the Max Forwarding Rate stability over an extended test duration in order to uncover any outliers. To allow for an extended test duration, the test should ideally run for 24 hours - or, if this is not possible, for at least 6 hours. For this test, each frame - size must be sent at the highest Throughput rate with X% packet loss, as - determined in the prerequisite test. The default loss percentages to be - tested are: - X = 0% - X = 10^-7% + or if this is not possible, for at least 6 hours. - Note: Other values can be tested if required by the user. + For this test, one frame size must be sent at the highest frame rate with + X% packet loss ratio, as determined in the prerequisite test (a short trial). + The loss ratio shall be measured and recorded every 5 minutes during the test + (it may be sufficient to collect lost frame counts and divide by the number + of frames sent in 5 minutes to see if a threshold has been crossed, + and accept some small inaccuracy in the threshold evaluation, not the result). + The default loss ratio is X = 0% and loss ratio > 10^-7% is the default + threshold to terminate the test early (or inform the test operator of + the failure status). + + Note: Other values of X and loss threshold can be tested if required by the user. **Expected Result**: @@ -441,13 +462,13 @@ Test ID: LTD.Throughput.RFC2889.MaxForwardingRateSoak and reporting any time intervals with packet loss. The `RFC2889 <https://www.rfc-editor.org/rfc/rfc2289.txt>`__ Forwarding Rate shall be measured in each interval. - An interval of 60s is suggested. + An interval of 300s is suggested. - CPU and memory utilization may also be collected as part of this test, to determine the vSwitch's performance footprint on the system. - The `RFC5481 <https://www.rfc-editor.org/rfc/rfc5481.txt>`__ PDV form of delay variation on the traffic flow, - using the 99th percentile. + using the 99th percentile, may also be collected. .. 3.2.2.1.7 @@ -457,7 +478,22 @@ Test ID: LTD.Throughput.RFC2889.MaxForwardingRateSoakFrameModification **Title**: RFC 2889 Max Forwarding Rate Soak Test with Frame Modification **Prerequisite Test**: + LTD.Throughput.RFC2544.PacketLossRatioFrameModification (0% Packet Loss) + will determine the offered load and + frame size for which the maximum theoretical throughput of the interface + has not been achieved. As described in RFC 2544 section 24, the final + determination of the benchmark SHOULD be conducted using a full length + trial, and for this purpose the duration is 5 minutes with zero loss ratio. + + It is also essential to verify that the Traffic Generator has sufficient + stability to conduct Soak tests. Therefore, a prerequisite is to perform + this test with the DUT removed and replaced with a cross-over cable (or + other equivalent very low overhead method such as a loopback in a HW switch), + so that the traffic generator (and any other network involved) can be tested + over the Soak period. Note that this test may be challenging for software- + based traffic generators. + **Priority**: @@ -466,9 +502,19 @@ Test ID: LTD.Throughput.RFC2889.MaxForwardingRateSoakFrameModification The aim of this test is to understand the Max Forwarding Rate stability over an extended test duration in order to uncover any outliers. To allow for an extended test duration, the test should ideally run for 24 hours or, if - this is not possible, for at least 6 hour. For this test, each frame - size must be sent at the highest Throughput rate with 0% packet loss, as - determined in the prerequisite test. + this is not possible, for at least 6 hours. + + For this test, one frame size must be sent at the highest frame rate with + X% packet loss ratio, as determined in the prerequisite test (a short trial). + The loss ratio shall be measured and recorded every 5 minutes during the test + (it may be sufficient to collect lost frame counts and divide by the number + of frames sent in 5 minutes to see if a threshold has been crossed, + and accept some small inaccuracy in the threshold evaluation, not the result). + The default loss ratio is X = 0% and loss ratio > 10^-7% is the default + threshold to terminate the test early (or inform the test operator of + the failure status). + + Note: Other values of X and loss threshold can be tested if required by the user. During this test, the DUT must perform the following operations on the traffic flow: @@ -498,13 +544,13 @@ Test ID: LTD.Throughput.RFC2889.MaxForwardingRateSoakFrameModification and reporting any time intervals with packet loss. The `RFC2889 <https://www.rfc-editor.org/rfc/rfc2289.txt>`__ Forwarding Rate shall be measured in each interval. - An interval of 60s is suggested. + An interval of 300s is suggested. - CPU and memory utilization may also be collected as part of this test, to determine the vSwitch's performance footprint on the system. - The `RFC5481 <https://www.rfc-editor.org/rfc/rfc5481.txt>`__ PDV form of delay variation on the traffic flow, using the 99th - percentile. + percentile, may also be collected. .. 3.2.2.1.8 @@ -1150,7 +1196,22 @@ Test ID: LTD.PacketDelayVariation.RFC3393.Soak **Title**: Packet Delay Variation Soak Test - **Prerequisite Tests**: LTD.Throughput.RFC2544.PacketLossRatio (0% Packet Loss) + **Prerequisite Tests**: + + LTD.Throughput.RFC2544.PacketLossRatio will determine the offered load and + frame size for which the maximum theoretical throughput of the interface + has not been achieved. As described in RFC 2544 section 24, the final + determination of the benchmark SHOULD be conducted using a full length + trial, and for this purpose the duration is 5 minutes with zero loss ratio. + + It is also essential to verify that the Traffic Generator has sufficient + stability to conduct Soak tests. Therefore, a prerequisite is to perform + this test with the DUT removed and replaced with a cross-over cable (or + other equivalent very low overhead method such as a loopback in a HW switch), + so that the traffic generator (and any other network involved) can be tested + over the Soak period. Note that this test may be challenging for software- + based traffic generators. + **Priority**: @@ -1160,9 +1221,20 @@ Test ID: LTD.PacketDelayVariation.RFC3393.Soak variation for different frame sizes over an extended test duration and to determine if there are any outliers. To allow for an extended test duration, the test should ideally run for 24 hours or, if this is not - possible, for at least 6 hour. For this test, each frame size must be - sent at the highest possible throughput with 0% packet loss, as - determined in the prerequisite test. + possible, for at least 6 hours. + + For this test, one frame size must be sent at the highest frame rate with + X% packet loss ratio, as determined in the prerequisite test (a short trial). + The loss ratio shall be measured and recorded every 5 minutes during the test + (it may be sufficient to collect lost frame counts and divide by the number + of frames sent in 5 minutes to see if a threshold has been crossed, + and accept some small inaccuracy in the threshold evaluation, not the result). + The default loss ratio is X = 0% and loss ratio > 10^-7% is the default + threshold to terminate the test early (or inform the test operator of + the failure status). + + Note: Other values of X and loss threshold can be tested if required by the user. + **Expected Result**: @@ -1173,7 +1245,7 @@ Test ID: LTD.PacketDelayVariation.RFC3393.Soak - The packet delay variation value for traffic passing through the DUT. - The `RFC5481 <https://www.rfc-editor.org/rfc/rfc5481.txt>`__ PDV form of delay variation on the traffic flow, - using the 99th percentile, for each 60s interval during the test. + using the 99th percentile, for each 300s interval during the test. - CPU and memory utilization may also be collected as part of this test, to determine the vSwitch's performance footprint on the system. diff --git a/docs/testing/user/configguide/trafficgen.rst b/docs/testing/user/configguide/trafficgen.rst index 33824486..4909c55a 100644 --- a/docs/testing/user/configguide/trafficgen.rst +++ b/docs/testing/user/configguide/trafficgen.rst @@ -577,7 +577,7 @@ http://www.mono-project.com/docs/getting-started/install/linux/ rpm --import "http://keyserver.ubuntu.com/pks/lookup?op=get&search=0x3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF" yum-config-manager --add-repo http://download.mono-project.com/repo/centos/ - yum -y install mono-complete + yum -y install mono-complete-5.8.0.127-0.xamarin.3.epel7.x86_64 To prevent gpg errors on future yum installation of packages the mono-project repo should be disabled once installed. @@ -795,6 +795,13 @@ It is neccesary for proper connection between Trex server and VSPERF. Firewall must allow a connection from DUT (VSPERF) to the T-Rex server running at TCP port 4501. +**NOTE:** For high speed cards it may be advantageous to start T-Rex with more transmit queues/cores. + +.. code-block:: console + + cd trex-cores/scripts/ + ./t-rex-64 -i -c 10 + For additional information about Trex stateless mode see Trex stateless documentation: https://trex-tgn.cisco.com/trex/doc/trex_stateless.html @@ -862,6 +869,22 @@ modified. Enable Promiscuous mode when doing multistream at layer 2 testing with TRAFFICGEN_TREX_PROMISCUOUS=True +Card Bandwidth Options +~~~~~~~~~~~~~~~~~~~~~~ + +T-Rex API will attempt to retrieve the highest possible speed from the card using internal +calls to port information. If you are using two separate cards then it will take the lowest +of the two cards as the max speed. If necessary you can try to force the API to use a +specific maximum speed per port. The below configurations can be adjusted to enable this. + +.. code-block:: console + + TRAFFICGEN_TREX_FORCE_PORT_SPEED = True + TRAFFICGEN_TREX_PORT_SPEED = 40000 # 40 gig + +**Note::** Setting higher than possible speeds will result in unpredictable behavior when running +tests such as duration inaccuracy and/or complete test failure. + RFC2544 Validation ~~~~~~~~~~~~~~~~~~ |