Add support for multiple physnets for sr-iov

Update config plugin config
Add readable error message on exception

Change-Id: Ie22de349582abc34d7e62570118022463f835d64
Signed-off-by: ahothan <ahothan@cisco.com>

7 files changed, 132 insertions, 14 deletions

diff --git a/docs/testing/index.rst b/docs/testing/index.rst
index 0b55d48..b795e6b 100644
--- a/docs/testing/index.rst
+++ b/docs/testing/index.rst
@@ -4,7 +4,7 @@
 .. (c) Cisco Systems, Inc
 
 .. toctree::
-   :maxdepth: 1
+   :maxdepth: 2
 
    developer/devguide/index
    user/configguide/index
diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst
index a7eb1e9..fa9d7d0 100644
--- a/docs/testing/user/userguide/index.rst
+++ b/docs/testing/user/userguide/index.rst
@@ -23,6 +23,7 @@ Table of Content
    installation
    examples
    advanced
+   sriov
    server
    faq
 
diff --git a/docs/testing/user/userguide/sriov.rst b/docs/testing/user/userguide/sriov.rst
new file mode 100644
index 0000000..c63a6d7
--- /dev/null
+++ b/docs/testing/user/userguide/sriov.rst
@@ -0,0 +1,60 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. SPDX-License-Identifier: CC-BY-4.0
+.. (c) Cisco Systems, Inc
+
+
+Testing SR-IOV
+==============
+
+NFVbench supports SR-IOV with the PVP and PVVP packet flows. SR-IOV support is not applicable for external chains since the networks have to be setup externally (and can themselves be pre-set to use SR-IOV or not).
+
+Pre-requisites
+--------------
+To test SR-IOV you need to have compute nodes configured to support one or more SR-IOV interfaces (also knows as PF or physical function) and you need OpenStack to be configured to support SR-IOV.
+You will also need to know:
+- the name of the physical networks associated to your SR-IOV interfaces (this is a configuration in Nova compute)
+- the VLAN range that can be used on the switch ports that are wired to the SR-IOV ports. Such switch ports are normally configured in trunk mode with a range of VLAN ids enabled on that port
+
+For example, in the case of 2 SR-IOV ports per compute node, 2 physical networks are generally configured in OpenStack with a distinct name. The VLAN range to use is is also allocated and reserved by the network administrator and in coordination with the corresponding top of rack switch port configuration.
+
+
+Configuration
+-------------
+To enable SR-IOV test, you will need to provide the following configuration options to NFVbench (in the configuration file).
+This example instructs NFVbench to create the left and right networks of a PVP packet flow to run on 2 SRIOV ports named "phys_sriov0" and "phys_sriov1" using resp. segmentation_id 2000 and 2001:
+
+.. code-block:: bash
+    
+    internal_networks:
+       left:
+           segmentation_id: 2000
+           physical_network: phys_sriov0
+       right:
+           segmentation_id: 2001
+           physical_network: phys_sriov1
+
+The segmentation ID fields must be different.
+In the case of PVVP, the middle network also needs to be provisioned properly.
+The same physical network can also be shared by the virtual networks but with different segmentation IDs.
+
+
+NIC NUMA socket placement and flavors
+-------------------------------------
+If the 2 selected ports reside on NICs that are on different NUMA sockets, you will need to explicitly tell Nova to use 2 numa nodes in the flavor used for the VMs in order to satisfy the filters, for example:
+
+.. code-block:: bash
+
+    flavor:
+      # Number of vCPUs for the flavor
+      vcpus: 2
+      # Memory for the flavor in MB
+      ram: 8192
+      # Size of local disk in GB
+      disk: 0
+      extra_specs:
+          "hw:cpu_policy": dedicated
+          "hw:mem_page_size": large
+          "hw:numa_nodes": 2
+
+Failure to do so might cause the VM creation to fail with the Nova error "Instance creation error: Insufficient compute resources: Requested instance NUMA topology together with requested PCI devices cannot fit the given host NUMA topology."
+
diff --git a/nfvbench/cfg.default.yaml b/nfvbench/cfg.default.yaml
index 8de983f..8766d53 100644
--- a/nfvbench/cfg.default.yaml
+++ b/nfvbench/cfg.default.yaml
@@ -207,30 +207,53 @@ generic_poll_sec: 2
 # name of the loop VM
 loop_vm_name: 'nfvbench-loop-vm'
 
-# Default names, subnets and CIDRs for internal networks used by the script.
+# Default names, subnets and CIDRs for PVP/PVVP networks
 # If a network with given name already exists it will be reused.
+# - PVP only uses left and right
+# - PVVP uses left, middle and right
+# - for EXT chains, this structure is not relevant - refer to external_networks
 # Otherwise a new internal network will be created with that name, subnet and CIDR.
+# 
+# segmentation_id can be set to enforce a specific VLAN id - by default (empty) the VLAN id 
+#                 will be assigned by Neutron.
+#                 Must be unique for each network
+# physical_network can be set to pick a specific phsyical network - by default (empty) the
+#                   default physical network will be picked
+# In the case of SR-IOV, both physical_network and segmentation ID must be provided
+# For example to setup PVP using 2 different SR-IOV ports, you must put the appropriate physnet
+# names under left.physical_network and right.physical_network.
+# Example of override configuration to force PVP to run on 2 SRIOV ports (phys_sriov0 and phys_sriov1)
+# using VLAN ID 2000 and 2001:
+# internal_networks:
+#    left:
+#        segmentation_id: 2000
+#        physical_network: phys_sriov0
+#    right:
+#        segmentation_id: 2001
+#        physical_network: phys_sriov1
+
 internal_networks:
-    # Required only when segmentation_id specified
-    physical_network:
     left:
         name: 'nfvbench-net0'
         subnet: 'nfvbench-subnet0'
         cidr: '192.168.1.0/24'
         network_type: 'vlan'
         segmentation_id:
+        physical_network:
     right:
         name: 'nfvbench-net1'
         subnet: 'nfvbench-subnet1'
         cidr: '192.168.2.0/24'
         network_type: 'vlan'
         segmentation_id:
+        physical_network:
     middle:
         name: 'nfvbench-net2'
         subnet: 'nfvbench-subnet2'
         cidr: '192.168.3.0/24'
         network_type: 'vlan'
         segmentation_id:
+        physical_network:
 
 # EXT chain only. Names of edge networks which will be used to send traffic via traffic generator.
 external_networks:
diff --git a/nfvbench/chain_clients.py b/nfvbench/chain_clients.py
index 4be050f..dfd6ff2 100644
--- a/nfvbench/chain_clients.py
+++ b/nfvbench/chain_clients.py
@@ -78,11 +78,12 @@ class BasicStageClient(object):
         networks = self.neutron.list_networks(name=network_name)
         return networks['networks'][0] if networks['networks'] else None
 
-    def _create_net(self, name, subnet, cidr, network_type=None, segmentation_id=None):
+    def _create_net(self, name, subnet, cidr, network_type=None, segmentation_id=None, physical_network=None):
         network = self._lookup_network(name)
         if network:
-            phys_net = self.config.internal_networks.physical_network
-            if segmentation_id is not None and phys_net is not None:
+            # a network of same name already exists, we need to verify it has the same
+            # characteristics
+            if segmentation_id:
                 if network['provider:segmentation_id'] != segmentation_id:
                     raise StageClientException("Mismatch of 'segmentation_id' for reused "
                                                "network '{net}'. Network has id '{seg_id1}', "
@@ -91,13 +92,14 @@ class BasicStageClient(object):
                                                        seg_id1=network['provider:segmentation_id'],
                                                        seg_id2=segmentation_id))
 
-                if network['provider:physical_network'] != phys_net:
+            if physical_network:
+                if network['provider:physical_network'] != physical_network:
                     raise StageClientException("Mismatch of 'physical_network' for reused "
                                                "network '{net}'. Network has '{phys1}', "
                                                "configuration requires '{phys2}'."
                                                .format(net=name,
                                                        phys1=network['provider:physical_network'],
-                                                       phys2=phys_net))
+                                                       phys2=physical_network))
 
             LOG.info('Reusing existing network: ' + name)
             network['is_reuse'] = True
@@ -112,10 +114,10 @@ class BasicStageClient(object):
 
         if network_type:
             body['network']['provider:network_type'] = network_type
-            phys_net = self.config.internal_networks.physical_network
-            if segmentation_id is not None and phys_net is not None:
+            if segmentation_id:
                 body['network']['provider:segmentation_id'] = segmentation_id
-                body['network']['provider:physical_network'] = phys_net
+            if physical_network:
+                body['network']['provider:physical_network'] = physical_network
 
         network = self.neutron.create_network(body)['network']
         body = {
diff --git a/nfvbench/config_plugin.py b/nfvbench/config_plugin.py
index ed6b3c6..417402a 100644
--- a/nfvbench/config_plugin.py
+++ b/nfvbench/config_plugin.py
@@ -36,6 +36,14 @@ class ConfigPluginBase(object):
     def get_config(self):
         """Returns updated default configuration file."""
 
+    def set_config(self, config):
+        """This method is called when the config has changed after this instance was initialized.
+
+        This is needed in teh frequent case where the main config is changed in a copy and to
+        prevent this instance to keep pointing to the old copy of the config
+        """
+        self.config = config
+
     @abc.abstractmethod
     def get_openstack_spec(self):
         """Returns OpenStack specs for host."""
diff --git a/nfvbench/nfvbench.py b/nfvbench/nfvbench.py
index 0dcf2f1..bf39a44 100644
--- a/nfvbench/nfvbench.py
+++ b/nfvbench/nfvbench.py
@@ -32,6 +32,7 @@ from nfvbenchd import WebSocketIoServer
 import os
 import pbr.version
 from pkg_resources import resource_string
+from specs import ChainType
 from specs import Specs
 from summarizer import NFVBenchSummarizer
 import sys
@@ -394,6 +395,13 @@ def override_custom_traffic(config, frame_sizes, unidir):
         "profile": traffic_profile_name
     }
 
+def check_physnet(name, netattrs):
+    if not netattrs.physical_network:
+        raise Exception("SRIOV requires physical_network to be specified for the {n} network"
+                            .format(n=name))
+    if not netattrs.segmentation_id:
+        raise Exception("SRIOV requires segmentation_id to be specified for the {n} network"
+                            .format(n=name))
 
 def main():
     try:
@@ -434,10 +442,11 @@ def main():
             # override default config options with start config at path parsed from CLI
             # check if it is an inline yaml/json config or a file name
             if os.path.isfile(opts.config):
-                print opts.config
+                LOG.info('Loading configuration file: ' + opts.config)
                 config = config_load(opts.config, config)
                 config.name = os.path.basename(opts.config)
             else:
+                LOG.info('Loading configuration string: ' + opts.config)
                 config = config_loads(opts.config, config)
 
         # traffic profile override options
@@ -445,12 +454,26 @@ def main():
 
         # copy over cli options that are used in config
         config.generator_profile = opts.generator_profile
+        if opts.sriov:
+            config.sriov = True
 
         # show running config in json format
         if opts.show_config:
             print json.dumps(config, sort_keys=True, indent=4)
             sys.exit(0)
 
+        if config.sriov and config.service_chain != ChainType.EXT:
+            # if sriov is requested (does not apply to ext chains)
+            # make sure the physnet names are specified
+            check_physnet("left", config.internal_networks.left)
+            check_physnet("right", config.internal_networks.right)
+            if config.service_chain == ChainType.PVVP:
+                check_physnet("middle", config.internal_networks.middle)
+
+        # update the config in the config plugin as it might have changed
+        # in a copy of the dict (config plugin still holds the original dict)
+        config_plugin.set_config(config)
+
         nfvbench = NFVBench(config, openstack_spec, config_plugin, factory)
 
         if opts.server:
@@ -480,11 +503,12 @@ def main():
                 if 'result' in result and result['status']:
                     nfvbench.save(result['result'])
                     nfvbench.print_summary(result['result'])
-    except Exception:
+    except Exception as exc:
         LOG.error({
             'status': NFVBench.STATUS_ERROR,
             'error_message': traceback.format_exc()
         })
+        print str(exc)
         sys.exit(1)
 
 if __name__ == '__main__':