aboutsummaryrefslogtreecommitdiffstats
path: root/samples/README
blob: 9d778fde6428d707c46ebe25644eee3cf9e6c4fe (plain) 
1
2
3
4
5

.. This work is licensed under a Creative Commons Attribution 4.0 International
.. License.
.. http://creativecommons.org/licenses/by/4.0

This directory contains benchmarking samples using the yardstick framework.
round-color: #fff0f0 } /* Literal.String */ .highlight .na { color: #336699 } /* Name.Attribute */ .highlight .nb { color: #003388 } /* Name.Builtin */ .highlight .nc { color: #bb0066; font-weight: bold } /* Name.Class */ .highlight .no { color: #003366; font-weight: bold } /* Name.Constant */ .highlight .nd { color: #555555 } /* Name.Decorator */ .highlight .ne { color: #bb0066; font-weight: bold } /* Name.Exception */ .highlight .nf { color: #0066bb; font-weight: bold } /* Name.Function */ .highlight .nl { color: #336699; font-style: italic } /* Name.Label */ .highlight .nn { color: #bb0066; font-weight: bold } /* Name.Namespace */ .highlight .py { color: #336699; font-weight: bold } /* Name.Property */ .highlight .nt { color: #bb0066; font-weight: bold } /* Name.Tag */ .highlight .nv { color: #336699 } /* Name.Variable */ .highlight .ow { color: #008800 } /* Operator.Word */ .highlight .w { color: #bbbbbb } /* Text.Whitespace */ .highlight .mb { color: #0000DD; font-weight: bold } /* Literal.Number.Bin */ .highlight .mf { color: #0000DD; font-weight: bold } /* Literal.Number.Float */ .highlight .mh { color: #0000DD; font-weight: bold } /* Literal.Number.Hex */ .highlight .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */ .highlight .mo { color: #0000DD; font-weight: bold } /* Literal.Number.Oct */ .highlight .sa { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Affix */ .highlight .sb { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Backtick */ .highlight .sc { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Char */ .highlight .dl { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Delimiter */ .highlight .sd { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Doc */ .highlight .s2 { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Double */ .highlight .se { color: #0044dd; background-color: #fff0f0 } /* Literal.String.Escape */ .highlight .sh { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Heredoc */ .highlight .si { color: #3333bb; background-color: #fff0f0 } /* Literal.String.Interpol */ .highlight .sx { color: #22bb22; background-color: #f0fff0 } /* Literal.String.Other */ .highlight .sr { color: #008800; background-color: #fff0ff } /* Literal.String.Regex */ .highlight .s1 { color: #dd2200; background-color: #fff0f0 } /* Literal.String.Single */ .highlight .ss { color: #aa6600; background-color: #fff0f0 } /* Literal.String.Symbol */ .highlight .bp { color: #003388 } /* Name.Builtin.Pseudo */ .highlight .fm { color: #0066bb; font-weight: bold } /* Name.Function.Magic */ .highlight .vc { color: #336699 } /* Name.Variable.Class */ .highlight .vg { color: #dd7700 } /* Name.Variable.Global */ .highlight .vi { color: #3333bb } /* Name.Variable.Instance */ .highlight .vm { color: #336699 } /* Name.Variable.Magic */ .highlight .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */ } 
.. This work is licensed under a Creative Commons Attribution 4.0 International
.. License.
.. http://creativecommons.org/licenses/by/4.0
.. (c) OPNFV, Huawei Technologies Co.,Ltd and others.

*************************************
Yardstick Test Case Description TC011
*************************************

.. _iperf3: https://iperf.fr/

+-----------------------------------------------------------------------------+
|Packet delay variation between VMs                                           |
|                                                                             |
+--------------+--------------------------------------------------------------+
|test case id  | OPNFV_YARDSTICK_TC011_PACKET DELAY VARIATION BETWEEN VMs     |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|metric        | jitter: packet delay variation (ms)                          |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|test purpose  | The purpose of TC011 is to evaluate the IaaS network         |
|              | performance with regards to network jitter (packet delay     |
|              | variation).                                                  |
|              | It measures the packet delay variation sending the packets   |
|              | from one VM to the other.                                    |
|              |                                                              |
|              | The purpose is also to be able to spot the trends.           |
|              | Test results, graphs and similar shall be stored for         |
|              | comparison reasons and product evolution understanding       |
|              | between different OPNFV versions and/or configurations.      |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|test tool     | iperf3                                                       |
|              |                                                              |
|              | iPerf3 is a tool for active measurements of the maximum      |
|              | achievable bandwidth on IP networks. It supports tuning of   |
|              | various parameters related to timing, buffers and protocols. |
|              | The UDP protocols can be used to measure jitter delay.       |
|              |                                                              |
|              | (iperf3 is not always part of a Linux distribution, hence it |
|              | needs to be installed. It is part of the Yardstick Docker    |
|              | image. As an example see the /yardstick/tools/ directory for |
|              | how to generate a Linux image with pktgen included.)         |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|test          | iperf3 test is invoked between a host VM and a target VM.    |
|description   |                                                              |
|              | Jitter calculations are continuously computed by the server, |
|              | as specified by RTP in RFC 1889. The client records a 64 bit |
|              | second/microsecond timestamp in the packet. The server       |
|              | computes the relative transit time as (server's receive time |
|              | - client's send time). The client's and server's clocks do   |
|              | not need to be synchronized; any difference is subtracted    |
|              | outin the jitter calculation. Jitter is the smoothed mean of |
|              | differences between consecutive transit times.               |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|configuration | File: opnfv_yardstick_tc011.yaml                             |
|              |                                                              |
|              | * options:                                                   |
|              |   protocol: udp # The protocol used by iperf3 tools          |
|              |   bandwidth: 20m # It will send the given number of packets  |
|              |                    without pausing                           |
|              | * runner:                                                    |
|              |   duration: 30 # Total test duration 30 seconds.             |
|              |                                                              |
|              | * SLA (optional):                                            |
|              |   jitter: 10 (ms) # The maximum amount of jitter that is     |
|              |     accepted.                                                |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|applicability | Test can be configured with different:                       |
|              |                                                              |
|              | * bandwidth: Test case can be configured with different      |
|              |              bandwidth.                                      |
|              |                                                              |
|              | * duration: The test duration can be configured.             |
|              |                                                              |
|              | * jitter: SLA is optional. The SLA in this test case         |
|              |           serves as an example.                              |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|usability     | This test case is one of Yardstick's generic test. Thus it   |
|              | is runnable on most of the scenarios.                        |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|references    | iperf3_                                                      |
|              |                                                              |
|              | ETSI-NFV-TST001                                              |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|pre-test      | The test case image needs to be installed into Glance        |
|conditions    | with iperf3 included in the image.                           |
|              |                                                              |
|              | No POD specific requirements have been identified.           |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|test sequence | description and expected result                              |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|step 1        | Two host VMs with iperf3 installed are booted, as server and |
|              | client.                                                      |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|step 2        | Yardstick is connected with the host VM by using ssh.        |
|              | A iperf3 server is started on the server VM via the ssh      |
|              | tunnel.                                                      |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|step 3        | iperf3 benchmark is invoked. Jitter is calculated and check  |
|              | against the SLA. Logs are produced and stored.               |
|              |                                                              |
|              | Result: Logs are stored.                                     |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|step 4        | The host VMs are deleted.                                    |
|              |                                                              |
+--------------+--------------------------------------------------------------+
|test verdict  | Test should not PASS if any jitter is above the optional SLA |
|              | value, or if there is a test case execution problem.         |
|              |                                                              |
+--------------+--------------------------------------------------------------+