Implementation of opnfv-yardstick-tc021 in Yardstick

Includes
 - Yardstick Scenario for integration with ApexLake
 - Yardstick Task .yaml file
 - Documentation

JIRA: YARDSTICK-219

Change-Id: I0554d0a211392902207ba9ceccf0b98dc3c2cdf1
Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>

4 files changed, 319 insertions, 0 deletions

diff --git a/docs/configguide/yardstick_testcases/opnfv_yardstick_tc021.rst b/docs/configguide/yardstick_testcases/opnfv_yardstick_tc021.rst
new file mode 100644
index 000000000..1094e4f29
--- /dev/null
+++ b/docs/configguide/yardstick_testcases/opnfv_yardstick_tc021.rst
@@ -0,0 +1,148 @@
+.. _DPDKpktgen: https://github.com/Pktgen/Pktgen-DPDK/
+.. _rfc2544: https://www.ietf.org/rfc/rfc2544.txt
+
+*************************************
+Yardstick Test Case Description TC021
+*************************************
++-----------------------------------------------------------------------------+
+|Network Performance                                                          |
++==============+==============================================================+
+|test case id  | OPNFV_YARDSTICK_TC0021_Virtual Traffic Classifier            |
+|              | Instantiation Test in Presence of Noisy Neighbours           |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|metric        | Failure                                                      |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|test purpose  | To verify that a newly instantiated vTC is 'alive' and       |
+|              | functional and its instantiation is correctly supported by   |
+|              | the infrastructure in the presence of noisy neighbours.      |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|configuration | file: opnfv_yardstick_tc021.yaml                             |
+|              |                                                              |
+|              | vnic_type: type of VNIC to be used.                          |
+|              |      Allowed values are:                                     |
+|              |           - normal: for default OvS port configuration       |
+|              |           - direct: for SR-IOV port configuration            |
+|              |      Default value: None                                     |
+|              |                                                              |
+|              | vtc_flavor: OpenStack flavor to be used for the vTC          |
+|              |      Default available values are: m1.small, m1.medium,      |
+|              |      and m1.large, but the user can create his/her own       |
+|              |      flavor and give it as input                             |
+|              |      Default value: None                                     |
+|              |                                                              |
+|              | num_of_neighbours: Number of noisy neighbours (VMs) to be    |
+|              |     instantiated during the experiment.                      |
+|              |     Allowed values: range (1, 10)                            |
+|              |                                                              |
+|              | amount_of_ram: RAM to be used by each neighbor.              |
+|              |     Allowed values: ['250M', '1G', '2G', '3G', '4G', '5G',   |
+|              |                      '6G', '7G', '8G', '9G', '10G']          |
+|              |     Deault value: 256M                                       |
+|              |                                                              |
+|              | number_of_cores: Number of noisy neighbours (VMs) to be      |
+|              |     instantiated during the experiment.                      |
+|              |     Allowed values: range (1, 10)                            |
+|              |     Default value: 1                                         |
+|              |                                                              |
+|              | vlan_sender: vlan tag of the network on which the vTC will   |
+|              |      receive traffic (VLAN Network 1).                       |
+|              |      Allowed values: range (1, 4096)                         |
+|              |                                                              |
+|              | vlan_receiver: vlan tag of the network on which the vTC      |
+|              |      will send traffic back to the packet generator          |
+|              |      (VLAN Network 2).                                       |
+|              |      Allowed values: range (1, 4096)                         |
+|              |                                                              |
+|              | default_net_name: neutron name of the defaul network that    |
+|              |      is used for access to the internet from the vTC         |
+|              |      (vNIC 1).                                               |
+|              |                                                              |
+|              | default_subnet_name: subnet name for vNIC1                   |
+|              |      (information available through Neutron).                |
+|              |                                                              |
+|              | vlan_net_1_name: Neutron Name for VLAN Network 1             |
+|              |      (information available through Neutron).                |
+|              |                                                              |
+|              | vlan_subnet_1_name: Subnet Neutron name for VLAN Network 1   |
+|              |      (information available through Neutron).                |
+|              |                                                              |
+|              | vlan_net_2_name: Neutron Name for VLAN Network 2             |
+|              |      (information available through Neutron).                |
+|              |                                                              |
+|              | vlan_subnet_2_name: Subnet Neutron name for VLAN Network 2   |
+|              |      (information available through Neutron).                |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|test tool     | DPDK pktgen                                                  |
+|              |                                                              |
+|              | DPDK Pktgen is not part of a Linux distribution,             |
+|              | hence it needs to be installed by the user.                  |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|references    | DPDK Pktgen: DPDK Pktgen: DPDKpktgen_                        |
+|              | ETSI-NFV-TST001                                              |
+|              | RFC 2544: rfc2544_                                           |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|applicability | Test can be configured with different flavors, vNIC type     |
+|              | and packet sizes. Default values exist as specified above.   |
+|              | The vNIC type and flavor MUST be specified by the user.      |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|pre-test      | The vTC has been successfully instantiated and configured.   |
+|              | The user has correctly assigned the values to the deployment |
+|              |  configuration parameters.                                   |
+|              |                                                              |
+|              | - Multicast traffic MUST be enabled on the network.          |
+|              |      The Data network switches need to be configured in      |
+|              |      order to manage multicast traffic.                      |
+|              |      Installation and configuration of smcroute is required  |
+|              |      before to run the test case.                            |
+|              |      (For further instructions please refer to the ApexLake  |
+|              |      documentation).                                         |
+|              | - In the case of SR-IOV vNICs use, SR-IOV compatible NICs    |
+|              |      must be used on the compute node.                       |
+|              | - Yarsdtick needs to be installed on a host connected to the |
+|              |      data network and the host must have 2 DPDK-compatible   |
+|              |      NICs. Proper configuration of DPDK and DPDK pktgen is   |
+|              |      required before to run the test case.                   |
+|              |      (For further instructions please refer to the ApexLake  |
+|              |      documentation).                                         |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|test sequence | Description and expected results                             |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|step 1        | The noisy neighbours are deployed as required by the user.   |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|step 2        | The vTC is deployed, according to the configuration provided |
+|              | by the user.                                                 |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|step 3        | The vTC is correctly deployed and configured as necessary.   |
+|              | The initialization script has been correctly executed and    |
+|              | the vTC is ready to receive and process the traffic.         |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|step 4        | Test case is executed with the selected parameters:          |
+|              | - vTC flavor                                                 |
+|              | - vNIC type                                                  |
+|              | A constant rate traffic is sent to the vTC for 10 seconds.   |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|step 5        | The vTC instance tags all the packets and sends them back to |
+|              | the packet generator for 10 seconds.                         |
+|              |                                                              |
+|              | The framework checks if the packet generator receives back   |
+|              | all the packets with the correct tag from the vTC.           |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
+|test verdict  |  The vTC is deemed to be successfully instantiated if all    |
+|              |  packets are sent back with the right tag as requested,      |
+|              |  else it is deemed DoA (Dead on arrival)                     |
+|              |                                                              |
++--------------+--------------------------------------------------------------+
diff --git a/tests/opnfv/test_cases/opnfv_yardstick_tc021.yaml b/tests/opnfv/test_cases/opnfv_yardstick_tc021.yaml
new file mode 100644
index 000000000..c62ce2a32
--- /dev/null
+++ b/tests/opnfv/test_cases/opnfv_yardstick_tc021.yaml
@@ -0,0 +1,28 @@
+---
+schema: "yardstick:task:0.1"
+
+scenarios:
+-
+  type: vtc_instantiation_validation_noisy
+
+  options:
+    vlan_sender: 1007
+    vlan_receiver: 1006
+    default_net_name: monitoring
+    default_subnet_name: monitoring_subnet
+    vlan_net_1_name: inbound_traffic_network
+    vlan_subnet_1_name: inbound_traffic_subnet
+    vlan_net_2_name: inbound_traffic_network
+    vlan_subnet_2_name: inbound_traffic_subnet
+    vnic_type: direct # [normal (OvS), direct (SR-IOV)]
+    vtc_flavor: m1.large
+    num_of_neighbours: 2
+    amount_of_ram: 1G
+    number_of_cores: 2
+
+  runner:
+    type: Iteration
+    iterations: 1
+
+context:
+  type: Dummy
diff --git a/tests/unit/benchmark/scenarios/networking/test_vtc_instantiation_validation_noisy.py b/tests/unit/benchmark/scenarios/networking/test_vtc_instantiation_validation_noisy.py
new file mode 100644
index 000000000..e0a46241c
--- /dev/null
+++ b/tests/unit/benchmark/scenarios/networking/test_vtc_instantiation_validation_noisy.py
@@ -0,0 +1,51 @@
+#!/usr/bin/env python
+
+##############################################################################
+# Copyright (c) 2015 Ericsson AB and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+
+# Unittest for yardstick.benchmark.scenarios.networking.iperf3.Iperf
+
+import mock
+import unittest
+
+from yardstick.benchmark.scenarios.networking import vtc_instantiation_validation_noisy
+
+
+class VtcInstantiationValidationiNoisyTestCase(unittest.TestCase):
+
+    def setUp(self):
+        scenario = dict()
+        scenario['options'] = dict()
+        scenario['options']['default_net_name'] = ''
+        scenario['options']['default_subnet_name'] = ''
+        scenario['options']['vlan_net_1_name'] = ''
+        scenario['options']['vlan_subnet_1_name'] = ''
+        scenario['options']['vlan_net_2_name'] = ''
+        scenario['options']['vlan_subnet_2_name'] = ''
+        scenario['options']['vnic_type'] = ''
+        scenario['options']['vtc_flavor'] = ''
+        scenario['options']['packet_size'] = ''
+        scenario['options']['vlan_sender'] = ''
+        scenario['options']['vlan_receiver'] = ''
+        scenario['options']['num_of_neighbours'] = '1'
+        scenario['options']['amount_of_ram'] = '1G'
+        scenario['options']['number_of_cores'] = '1'
+
+        self.vt = vtc_instantiation_validation_noisy.VtcInstantiationValidationNoisy(scenario, '')
+
+    def test_run_for_success(self):
+        result = {}
+        self.vt.run(result)
+
+
+def main():
+    unittest.main()
+
+if __name__ == '__main__':
+    main()
diff --git a/yardstick/benchmark/scenarios/networking/vtc_instantiation_validation_noisy.py b/yardstick/benchmark/scenarios/networking/vtc_instantiation_validation_noisy.py
new file mode 100644
index 000000000..4834a5fc7
--- /dev/null
+++ b/yardstick/benchmark/scenarios/networking/vtc_instantiation_validation_noisy.py
@@ -0,0 +1,92 @@
+#############################################################################
+# Copyright (c) 2015 Ericsson AB and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+
+import logging
+import os
+
+from yardstick.benchmark.scenarios import base
+import experimental_framework.api as api
+
+LOG = logging.getLogger(__name__)
+
+
+class VtcInstantiationValidationNoisy(base.Scenario):
+    """Execute Instantiation Validation TC on the vTC
+    """
+    __scenario_type__ = "vtc_instantiation_validation_noisy"
+
+    def __init__(self, scenario_cfg, context_cfg):
+        self.scenario_cfg = scenario_cfg
+        self.context_cfg = context_cfg
+        self.options = None
+        self.setup_done = False
+
+    def setup(self):
+        '''scenario setup'''
+
+        self.options = self.scenario_cfg['options']
+        self.setup_done = True
+
+    def run(self, result):
+        """execute test"""
+
+        if not self.setup_done:
+            self.setup()
+
+        heat_template = 'vTC.yaml'
+        iterations = 1
+
+        openstack_credentials = {
+            'ip_controller': '0.0.0.0',
+            'heat_url': '***',
+            'auth_uri': os.environ.get('OS_AUTH_URL'),
+            'user': os.environ.get('OS_USERNAME'),
+            'password': os.environ.get('OS_PASSWORD'),
+            'project': os.environ.get('OS_TENANT_NAME')
+        }
+        heat_template_parameters = {
+            'default_net': self.options['default_net_name'],
+            'default_subnet': self.options['default_subnet_name'],
+            'source_net': self.options['vlan_net_1_name'],
+            'source_subnet': self.options['vlan_subnet_1_name'],
+            'destination_net': self.options['vlan_net_2_name'],
+            'destination_subnet': self.options['vlan_subnet_2_name']
+        }
+        deployment_configuration = {
+            'vnic_type': [self.options['vnic_type']],
+            'vtc_flavor': [self.options['vtc_flavor']]
+        }
+
+        test_case = dict()
+        test_case['name'] = 'instantiation_validation_noisy_neighbors_' \
+                            'benchmark.' \
+                            'InstantiationValidationNoisyNeighborsBenchmark'
+        test_case['params'] = dict()
+        test_case['params']['throughput'] = '1'
+        test_case['params']['vlan_sender'] = str(self.options['vlan_sender'])
+        test_case['params']['vlan_receiver'] = \
+            str(self.options['vlan_receiver'])
+        test_case['params']['num_of_neighbours'] = \
+            str(self.options['num_of_neighbours'])
+        test_case['params']['amount_of_ram'] = \
+            str(self.options['amount_of_ram'])
+        test_case['params']['number_of_cores'] = \
+            str(self.options['number_of_cores'])
+
+        try:
+            result = api.FrameworkApi.execute_framework(
+                [test_case],
+                iterations,
+                heat_template,
+                heat_template_parameters,
+                deployment_configuration,
+                openstack_credentials)
+        except Exception as e:
+            LOG.info('Exception: {}'.format(e.message))
+        LOG.info('Got output: {}'.format(result))