framework: Add reworked framework to repo

This commit adds the vSwitch Integration Test Framework whose
design, based off TOIT, is outlined in the HLD previously made
availiable to the community for review.

The design of this framework allows developers to add different
implementations of components, specifically vSwitches, Traffic
Generators, Metrics Collectors and VNFs, easily.

The goal of this design is that all testcases should run regardless of
what is "under the hood".

This commit adds support for running the framework for a phy to phy
RFC2544 testcase only. More testcases will be added by the community.

vSwitches supported at this time:
* Intel DPDK (r) accelerated OpenvSwitch

Traffic Generators supported at this time:
* IxNet - IxNetwork Implementation
* Ixia - IxExplorer Implementation
* Dummy - Manual Implementation

Metrics Collectors supported at this time:
* Linux Metrics

No VNFs are supported at this time but the framework outlines how they
should be integrated and provides APIs for them to adhere to.

JIRA: VSPERF-27
Change-Id: I312e1a1199487ffee8f824be06cd97d4f793eee0
Signed-off-by: Stephen Finucane <Stephen.Finucane@intel.com>
Signed-off-by: Meghan Halton <Meghan.Halton@intel.com>
Signed-off-by: Christopher Nolan <Christopher.Nolan@intel.com>
Signed-off-by: Maryam Tahhan <Maryam.Tahhan@intel.com>
Signed-off-by: Ciara Loftus <Ciara.Loftus@intel.com>
Signed-off-by: Mark Kavanagh <Mark.B.Kavanagh@intel.com>
Signed-off-by: Cian Ferriter <Cian.Ferriter@intel.com>
Signed-off-by: Timo Puha <TimoX.Puha@intel.com>
Signed-off-by: Billy O'Mahony <billy.o.mahony@intel.com>
Signed-off-by: Michal Weglicki <MichalX.Weglicki@intel.com>
Signed-off-by: Rory Sexton <Rory.Sexton@intel.com>
Signed-off-by: Ian Stokes <Ian.Stokes@intel.com>
Signed-off-by: Kevin Traynor <Kevin.Traynor@intel.com>
Signed-off-by: Dino Simeon Madarang <dino.simeonx.madarang@intel.com>
Reviewed-by: Eugene Snider <Eugene.Snider@huawei.com>
Reviewed-by: Aihua Li <aihua.li@huawei.com>

3 files changed, 324 insertions, 0 deletions

diff --git a/tools/pkt_gen/trafficgen/__init__.py b/tools/pkt_gen/trafficgen/__init__.py
new file mode 100755
index 00000000..2a3b9bd3
--- /dev/null
+++ b/tools/pkt_gen/trafficgen/__init__.py
@@ -0,0 +1,19 @@
+# Copyright 2015 Intel Corporation.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Trafficgen interface and helpers.
+"""
+
+from tools.pkt_gen.trafficgen.trafficgen import *
+from tools.pkt_gen.trafficgen.trafficgenhelper import *
diff --git a/tools/pkt_gen/trafficgen/trafficgen.py b/tools/pkt_gen/trafficgen/trafficgen.py
new file mode 100755
index 00000000..13af6b81
--- /dev/null
+++ b/tools/pkt_gen/trafficgen/trafficgen.py
@@ -0,0 +1,221 @@
+# Copyright 2015 Intel Corporation.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Abstract "traffic generator" model.
+
+This is an abstract class for traffic generators.
+"""
+
+#TODO update Back2Back method description when Result implementation will
+#be ready.
+
+from tools.pkt_gen.trafficgen.trafficgenhelper import TRAFFIC_DEFAULTS
+
+class ITrafficGenerator(object):
+    """Model of a traffic generator device.
+    """
+    _traffic_defaults = TRAFFIC_DEFAULTS.copy()
+
+    @property
+    def traffic_defaults(self):
+        """Default traffic values.
+
+        These can be expected to be constant across traffic generators,
+        so no setter is provided. Changes to the structure or contents
+        will likely break traffic generator implementations or tests
+        respectively.
+        """
+        return self._traffic_defaults
+
+    def __enter__(self):
+        """Connect to the traffic generator.
+
+        Provide a context manager interface to the traffic generators.
+        This simply calls the :func:`connect` function.
+        """
+        return self.connect()
+
+    def __exit__(self, type_, value, traceback):
+        """Disconnect from the traffic generator.
+
+        Provide a context manager interface to the traffic generators.
+        This simply calls the :func:`disconnect` function.
+        """
+        self.disconnect()
+
+    def connect(self):
+        """Connect to the traffic generator.
+
+        This is an optional function, designed for traffic generators
+        which must be "connected to" (i.e. via SSH or an API) before
+        they can be used. If not required, simply do nothing here.
+
+        Where implemented, this function should raise an exception on
+        failure.
+
+        :returns: None
+        """
+        raise NotImplementedError('Please call an implementation.')
+
+    def disconnect(self):
+        """Disconnect from the traffic generator.
+
+        As with :func:`connect`, this function is optional.
+
+        Where implemented, this function should raise an exception on
+        failure.
+
+        :returns: None
+        """
+        raise NotImplementedError('Please call an implementation.')
+
+    def send_burst_traffic(self, traffic=None, numpkts=100,
+                           time=20, framerate=100):
+        """Send a burst of traffic.
+
+        Send a ``numpkts`` packets of traffic, using ``traffic``
+        configuration, with a timeout of ``time``.
+
+        Attributes:
+        :param traffic: Detailed "traffic" spec, i.e. IP address, VLAN tags
+        :param numpkts: Number of packets to send
+        :param framerate: Expected framerate
+        :param time: Time to wait to receive packets
+
+        :returns: dictionary of strings with following data:
+            - List of Tx Frames,
+            - List of Rx Frames,
+            - List of Tx Bytes,
+            - List of List of Rx Bytes,
+            - Payload Errors and Sequence Errors.
+        """
+        raise NotImplementedError('Please call an implementation.')
+
+    def send_cont_traffic(self, traffic=None, time=20, framerate=0,
+                          multistream=False):
+        """Send a continuous flow of traffic.
+
+        Send packets at ``framerate``, using ``traffic`` configuration,
+        until timeout ``time`` occurs.
+
+        :param traffic: Detailed "traffic" spec, i.e. IP address, VLAN tags
+        :param time: Time to wait to receive packets (secs)
+        :param framerate: Expected framerate
+        :param multistream: Enable multistream output by overriding the
+                        UDP port number in ``traffic`` with values
+                        from 1 to 64,000
+        :returns: dictionary of strings with following data:
+            - Tx Throughput (fps),
+            - Rx Throughput (fps),
+            - Tx Throughput (mbps),
+            - Rx Throughput (mbps),
+            - Tx Throughput (% linerate),
+            - Rx Throughput (% linerate),
+            - Min Latency (ns),
+            - Max Latency (ns),
+            - Avg Latency (ns)
+        """
+        raise NotImplementedError('Please call an implementation.')
+
+    def start_cont_traffic(self, traffic=None, time=20, framerate=0,
+                           multistream=False):
+        """Non-blocking version of 'send_cont_traffic'.
+
+        Start transmission and immediately return. Do not wait for
+        results.
+        """
+        raise NotImplementedError('Please call an implementation.')
+
+    def stop_cont_traffic(self):
+        """Stop continuous transmission and return results.
+        """
+        raise NotImplementedError('Please call an implementation.')
+
+    def send_rfc2544_throughput(self, traffic=None, trials=3, duration=20,
+                                lossrate=0.0, multistream=False):
+        """Send traffic per RFC2544 throughput test specifications.
+
+        Send packets at a variable rate, using ``traffic``
+        configuration, until minimum rate at which no packet loss is
+        detected is found.
+
+        :param traffic: Detailed "traffic" spec, i.e. IP address, VLAN tags
+        :param trials: Number of trials to execute
+        :param duration: Per iteration duration
+        :param lossrate: Acceptable lossrate percentage
+        :param multistream: Enable multistream output by overriding the
+                        UDP port number in ``traffic`` with values
+                        from 1 to 64,000
+        :returns: dictionary of strings with following data:
+            - Tx Throughput (fps),
+            - Rx Throughput (fps),
+            - Tx Throughput (mbps),
+            - Rx Throughput (mbps),
+            - Tx Throughput (% linerate),
+            - Rx Throughput (% linerate),
+            - Min Latency (ns),
+            - Max Latency (ns),
+            - Avg Latency (ns)
+        """
+        raise NotImplementedError('Please call an implementation.')
+
+    def start_rfc2544_throughput(self, traffic=None, trials=3, duration=20,
+                                 lossrate=0.0, multistream=False):
+        """Non-blocking version of 'send_rfc2544_throughput'.
+
+        Start transmission and immediately return. Do not wait for
+        results.
+        """
+        raise NotImplementedError('Please call an implementation.')
+
+    def wait_rfc2544_throughput(self):
+        """Wait for and return results of RFC2544 test.
+        """
+        raise NotImplementedError('Please call an implementation.')
+
+    def send_rfc2544_back2back(self, traffic=None, trials=1, duration=20,
+                               lossrate=0.0, multistream=False):
+        """Send traffic per RFC2544 back2back test specifications.
+
+        Send packets at a fixed rate, using ``traffic``
+        configuration, until minimum time at which no packet loss is
+        detected is found.
+
+        :param traffic: Detailed "traffic" spec, i.e. IP address, VLAN
+            tags
+        :param trials: Number of trials to execute
+        :param duration: Per iteration duration
+        :param lossrate: Acceptable loss percentage
+        :param multistream: Enable multistream output by overriding the
+            UDP port number in ``traffic`` with values from 1 to 64,000
+
+        :returns: Named tuple of Rx Throughput (fps), Rx Throughput (mbps),
+            Tx Rate (% linerate), Rx Rate (% linerate), Tx Count (frames),
+            Back to Back Count (frames), Frame Loss (frames), Frame Loss (%)
+        :rtype: :class:`Back2BackResult`
+        """
+        raise NotImplementedError('Please call an implementation.')
+
+    def start_rfc2544_back2back(self, traffic=None, trials=1, duration=20,
+                                lossrate=0.0, multistream=False):
+        """Non-blocking version of 'send_rfc2544_back2back'.
+
+        Start transmission and immediately return. Do not wait for
+        results.
+        """
+        raise NotImplementedError('Please call an implementation.')
+
+    def wait_rfc2544_back2back(self):
+        """Wait and set results of RFC2544 test.
+        """
+        raise NotImplementedError('Please call an implementation.')
diff --git a/tools/pkt_gen/trafficgen/trafficgenhelper.py b/tools/pkt_gen/trafficgen/trafficgenhelper.py
new file mode 100644
index 00000000..2cd2d2b1
--- /dev/null
+++ b/tools/pkt_gen/trafficgen/trafficgenhelper.py
@@ -0,0 +1,84 @@
+# Copyright 2015 Intel Corporation.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Helper methods collection.
+
+Collection of helper methods used by traffic generators
+implementation.
+"""
+
+from collections import namedtuple
+
+CMD_PREFIX = 'gencmd : '
+TRAFFIC_DEFAULTS = {
+    'l2': {
+        'framesize': 64,
+        'srcmac': '00:00:00:00:00:00',
+        'dstmac': '00:00:00:00:00:00',
+        'srcport': 3000,
+        'dstport': 3001,
+    },
+    'l3': {
+        'proto': 'tcp',
+        'srcip': '1.1.1.1',
+        'dstip': '90.90.90.90',
+    },
+    'vlan': {
+        'enabled': False,
+        'id': 0,
+        'priority': 0,
+        'cfi': 0,
+    },
+}
+
+#TODO remove namedtuples and implement results through IResult interface found
+#in core/results
+
+BurstResult = namedtuple(
+    'BurstResult',
+    'frames_tx frames_rx bytes_tx bytes_rx payload_err seq_err')
+Back2BackResult = namedtuple(
+    'Back2BackResult',
+    'rx_fps rx_mbps tx_percent rx_percent tx_count b2b_frames '
+    'frame_loss_frames frame_loss_percent')
+
+
+def merge_spec(orig, new):
+    """Merges ``new`` dict with ``orig`` dict, and return orig.
+
+    This takes into account nested dictionaries. Example:
+
+        >>> old = {'foo': 1, 'bar': {'foo': 2, 'bar': 3}}
+        >>> new = {'foo': 6, 'bar': {'foo': 7}}
+        >>> merge_spec(old, new)
+        {'foo': 3, 'bar': {'foo': 7, 'bar': 3}}
+
+    You'll notice that ``bar.bar`` is not removed. This is the desired result.
+    """
+    for key in orig:
+        if key not in new:
+            continue
+
+        # Not allowing derived dictionary types for now
+        # pylint: disable=unidiomatic-typecheck
+        if type(orig[key]) == dict:
+            orig[key] = merge_spec(orig[key], new[key])
+        else:
+            orig[key] = new[key]
+
+    for key in new:
+        if key not in orig:
+            orig[key] = new[key]
+
+    return orig
+