diff options
20 files changed, 783 insertions, 399 deletions
diff --git a/docs/scenarios/os-nosdn-fdio-ha/FDS-nosdn-overview.png b/docs/scenarios/os-nosdn-fdio-ha/FDS-nosdn-overview.png Binary files differnew file mode 100755 index 0000000..0692374 --- /dev/null +++ b/docs/scenarios/os-nosdn-fdio-ha/FDS-nosdn-overview.png diff --git a/docs/scenarios/os-nosdn-fdio-ha/index.rst b/docs/scenarios/os-nosdn-fdio-ha/index.rst new file mode 100644 index 0000000..f944346 --- /dev/null +++ b/docs/scenarios/os-nosdn-fdio-ha/index.rst @@ -0,0 +1,18 @@ +.. OPNFV - Open Platform for Network Function Virtualization +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + + +********************************************************************** +Fast Data Stacks Scenario: os-nosdn-fdio-ha Overview and Description +********************************************************************** + +Scenario: "OpenStack - FD.io" (os-nosdn-fdio-ha) +is a scenario developed as part of the FastDataStacks +OPNFV project. + +.. toctree:: + :numbered: + :maxdepth: 2 + + scenario.description.rst diff --git a/docs/scenarios/os-nosdn-fdio-ha/scenario.description.rst b/docs/scenarios/os-nosdn-fdio-ha/scenario.description.rst new file mode 100644 index 0000000..86eadb2 --- /dev/null +++ b/docs/scenarios/os-nosdn-fdio-ha/scenario.description.rst @@ -0,0 +1,173 @@ +.. OPNFV - Open Platform for Network Function Virtualization +.. This work is licensed under a Creative Commons Attribution 4.0 +.. International License. +.. http://creativecommons.org/licenses/by/4.0 + +Scenario: "OpenStack - FD.io" +============================= + +Scenario: os-nosdn-fdio-ha + +"os-nosdn-fdio-ha" is a scenario developed as part of the FastDataStacks +OPNFV project. The main components of the "os-nosdn-fdio-ha" scenario +are: + + - APEX (TripleO) installer (please also see APEX installer documentation) + - Openstack (in HA configuration) + - FD.io/VPP virtual forwarder for tenant networking + - etcd, which is the VPP ML2 mechanism driver's distributed key-value store, in clustered mode + +Introduction +============ + +NFV and virtualized high performance applications, such as video processing, +require a "fast data stack" solution that provides both carrier grade +forwarding performance, scalability and open extensibility. + +A key component of any NFV solution is the virtual forwarder, which needs to be +a feature rich, high performance, highly scale virtual switch-router. It needs +to leverage hardware accelerators when available and run in user space. In +addition, it should be modular and easily extensible. The Vector Packet +Processor (VPP) supplied by the FD.io project meets these needs, in that +it offers a highly scalable, high performance and easily extensible +software forwarder that entirely runs in user space. + +The "Openstack - FD.io/VPP" scenario provides the capability to realize a set +of use-cases relevant to the deployment of NFV nodes instantiated by means of +an Openstack orchestration system on FD.io/VPP enabled compute nodes. + +A deployment of the "os-nosdn-fdio-ha" scenario consists of 6 or more +servers: + + * 1 Jumphost hosting the APEX installer - running the Undercloud + * 3 Controlhosts, which run the Overcloud and Openstack services as well as the VPP ML2 etcd cluster + * 2 or more Computehosts + +.. image:: FDS-nosdn-overview.png + +Tenant networking leverages FD.io/VPP. Open VSwitch (OVS) is used for all other +connectivity, in particular the connectivity to public networking / the +Internet (i.e. br-ext) is performed via OVS as in any standard OpenStack +deployment. A VPP management agent is used to setup and manage layer 2 +networking for the scenario. Neutron ML2 plugin is configured to use +the VPP ML2 networking mechanism driver. Tenant networking can either leverage +VLANs or plain interfaces (flat networks). Layer 3 connectivity for a tenant +network is provided centrally via qrouter on the control node. As in a +standard OpenStack deployment, the Layer3 agent configures the qrouter and +associated rulesets for security (security groups) and NAT (floating IPs). +Public IP network connectivity for a tenant network is provided by +interconnecting the VPP-based bridge domain representing the tenant network to +qrouter using a tap interface. + +The setup is depicted below: + +Features of the scenario +------------------------ + +Main features of the "os-nosdn-fdio-ha" scenario: + + * Automated installation using the APEX installer + * Fast and scalable tenant networking using FD.io/VPP as forwarder + * Layer 2 networking using VLANs, managed and controlled + through the VPP ML2 plugin + * Layer 3 connectivitiy for tenant networks supplied centrally + on the Control node through standard OpenStack mechanisms. + All layer 3 features apply, including floating IPs (i.e. NAT) + and security groups + * DHCP server for tenant instances provided using the standard + OpenStack dnsmasq server + * OpenStack high availability + * etcd (VPP ML2 mechanism driver's distributed key-value store) high availability + +Networking in this scenario using VPP +------------------------------------- + +The os-nosdn-fdio-ha scenario combines components from two key open +source projects: OpenStack and Fast Data (FD.io). In order to make Fast Data +(FD.io) networking available to this scenario, an ML2 mechanism driver and a +light-weight control plane agent for VPP forwarder has been created. For +details see also https://git.openstack.org/cgit/openstack/networking-vpp/ + +Networking-vpp provides a Neutron ML2 mechanism driver to bring the advantages +of VPP to OpenStack deployments.It uses an etcd cluster on the control node to +keep track of the compute nodes, agent state and port bindings/unbindings. + +It's been written to be as simple and readable as possible, which means it's +naive; the aim was not to write the most efficient mechanism driver ever from +right out of the gate, but to write something simple and understandable and see +how well it works and what needs to be changed. + +As a general rule, everything was implemented in the simplest way, for two +reasons: one is that one sees working results the quickes, and the other is +that it's much easier to replace a simple system with a more complex one than +it is to change a complex one. The current design will change, but the one +that's there at the moment is small and easy to read, even if it makes you pull +faces when you read it. + +Scenario Configuration +====================== + +To enable the "os-nosdn-fdio-ha" scenario check the appropriate settings +in the APEX configuration files. Those are typically found in /etc/opnfv-apex. + +Use the file "os-nosdn-fdio-ha.yaml":: + + global_params: + ha_enabled: true + + deploy_options: + sdn_controller: false + sdn_l3: false + tacker: true + congress: true + sfc: false + vpn: false + vpp: true + dataplane: fdio + performance: + Controller: + vpp: + uio-driver: uio_pci_generic + Compute: + kernel: + hugepagesz: 2M + hugepages: 2048 + intel_iommu: 'on' + iommu: pt + isolcpus: 1,2 + vpp: + uio-driver: uio_pci_generic + + +Validated deployment environments +================================= + +The "os-nosdn-fdio-ha" scenario has been deployed and tested +on the following sets of hardware: + * Linux Foundation lab (Chassis: Cisco UCS-B-5108 blade server, + NICs: 8 external / 32 internal 10GE ports, + RAM: 32G (4 x 8GB DDR4-2133-MHz RDIMM/PC4-17000/single rank/x4/1.2v), + CPU: 3.50 GHz E5-2637 v3/135W 4C/15MB Cache/DDR4 2133MHz + Disk: 1.2 TB 6G SAS 10K rpm SFF HDD) see also + https://wiki.opnfv.org/display/pharos/Lflab+Hosting + * Cisco internal development labs (UCS-B and UCS-C) + + +Limitations, Issues and Workarounds +=================================== + +For specific information on limitations and issues, please refer to the APEX +installer release notes. + +References +========== + + + * FastDataStacks OPNFV project wiki: https://wiki.opnfv.org/display/fds + * Fast Data (FD.io): https://fd.io/ + * FD.io Vector Packet Processor (VPP): https://wiki.fd.io/view/VPP + * ML2 VPP mechanisms driver: https://git.openstack.org/cgit/openstack/networking-vpp/ + * OPNFV Danube release - more information: http://www.opnfv.org/danube + * Networking-vpp launchpad (ticket tracker) https://launchpad.net/networking-vpp + * Networking-vpp Wiki: https://wiki.openstack.org/wiki/Networking-vpp/ + * APEX (TripleO based) installer: https://wiki.opnfv.org/display/apex/Apex diff --git a/docs/scenarios/os-odl_l2-fdio-ha/FDS-basic-components.jpg b/docs/scenarios/os-odl_l2-fdio-ha/FDS-basic-components.jpg Binary files differnew file mode 100755 index 0000000..e92851f --- /dev/null +++ b/docs/scenarios/os-odl_l2-fdio-ha/FDS-basic-components.jpg diff --git a/docs/scenarios/os-odl_l2-fdio-ha/FDS-odl_l2-ha-overview.png b/docs/scenarios/os-odl_l2-fdio-ha/FDS-odl_l2-ha-overview.png Binary files differnew file mode 100755 index 0000000..78526da --- /dev/null +++ b/docs/scenarios/os-odl_l2-fdio-ha/FDS-odl_l2-ha-overview.png diff --git a/docs/scenarios/os-odl_l2-fdio-ha/FDS-simple-callflow.png b/docs/scenarios/os-odl_l2-fdio-ha/FDS-simple-callflow.png Binary files differnew file mode 100755 index 0000000..04546aa --- /dev/null +++ b/docs/scenarios/os-odl_l2-fdio-ha/FDS-simple-callflow.png diff --git a/docs/scenarios/os-odl_l2-fdio-ha/scenario.description.rst b/docs/scenarios/os-odl_l2-fdio-ha/scenario.description.rst index d8787c6..a81e8ed 100755 --- a/docs/scenarios/os-odl_l2-fdio-ha/scenario.description.rst +++ b/docs/scenarios/os-odl_l2-fdio-ha/scenario.description.rst @@ -13,7 +13,7 @@ FastDataStacks OPNFV project. The main components of the - APEX (TripleO) installer (please also see APEX installer documentation) - Openstack (in HA configuration) - - OpenDaylight controller (non-clustered) controlling layer 2 networking + - OpenDaylight controller in clustered mode controlling layer 2 networking - FD.io/VPP virtual forwarder for tenant networking Introduction @@ -39,8 +39,8 @@ NFV infrastructure are reflect different business In order to meet the desired qualities of an NFV infrastructure, the -following components were chosen for the "Openstack - OpenDaylight - - FD.io/VPP" scenario: +following components were chosen for the "Openstack - OpenDaylight - FD.io" +scenario: * FD.io Vector Packet Processor (VPP) - a highly scalable, high performance, extensible virtual forwarder * OpenDaylight Controller - an extensible controller platform which @@ -48,9 +48,9 @@ following components were chosen for the "Openstack - OpenDaylight constructs, supports a diverse set of network devices (virtual and physical) via the "group based policy (GBP)" component, and can be clustered to achieve a highly available - deployment. + deployment - as done in this scenario. -The "Openstack - OpenDaylight - FD.io/VPP" scenario provides the capability to +The "Openstack - OpenDaylight - FD.io" scenario provides the capability to realize a set of use-cases relevant to the deployment of NFV nodes instantiated by means of an Openstack orchestration system on FD.io/VPP enabled compute nodes. The role of the Opendaylight network controller in this integration is @@ -70,11 +70,12 @@ A deployment of the "apex-os-odl_l2-fdio-ha" scenario consists of 4 or more servers: * 1 Jumphost hosting the APEX installer - running the Undercloud - * 3 Controlhosts, which runs the Overcloud as well as OpenDaylight - as a network controller (OpenDaylight only runs on one Controlhost) + * 3 Controlhosts, which run the Overcloud as well as OpenDaylight + as a network controller. OpenDaylight is deployed in clustered + mode and runs on all 3 control nodes. * 2 or more Computehosts -.. image:: FDS-odl_l2-overview.png +.. image:: FDS-odl_l2-ha-overview.png Tenant networking leverages FD.io/VPP. Open VSwitch (OVS) is used for all other connectivity, in particular the connectivity to public networking / the @@ -113,6 +114,7 @@ Main features of the "apex-os-odl_l2-fdio-ha" scenario: All layer 3 features apply, including floating IPs (i.e. NAT) and security groups. * Manual and automatic (via DHCP) addressing on tenant networks + * OpenDaylight controller high availability (clustering) * OpenStack high availability Scenario components and composition @@ -128,88 +130,39 @@ Vector Packet Processor (VPP). Here's a more detailed list of the individual software components involved: -**Openstack Neutron ML2 ODL Plugin**: Handles Neutron data base synchronization -and interaction with the southbound Openstack controller using HTTP. - -**OpenDaylight Neutron Nothbound & Neutron MD-SAL Entry Store**: Presents a -Neutron (v2) extended HTTP API servlet for interaction with Openstack Neutron. -It validates and stores the received Neutron data in the MD-SAL data store -against the Neutron yang model driven. - -**OpenDaylight Neutron Mapper**: The Neutron Mapper listens to Neutron data -change events and is responsible for using Neutron data in creating Group Based -Policy Data objects, e.g. GBP End-Points, Flood-Domains. A GBP End Point -represents a specific NFV/VM port and its identity as derived from a Neutron -Port. The mapped data is stored using the GBP End Point yang model and an -association between the GBP End-Point and its Neutron object is maintained in -the Neutron-GBP map. - -**OpenDaylight Group Based Policy (GBP) Entities store**: Stores for the GBP -data artifacts against the GBP YANG schemas. - -**Neutron Group Based Policy Map store**: Stores the bi-lateral relation -between an End-Point and its corresponding Neutron object. Neutron-GBP map; -keyed by Neutron object type, port, and Neutron UUID, gives the GBP End-Point, -Flood domain respectively. GBP-Neutron map keyed by GBP object type, end-point. - -**Neutron VPP Renderer Mapper**: The Neutron VPP Renderer Mapper listens to -Neutron Store data change events, as well as being able to access directly the -store, and is responsible for converting Neutron data specifically required to -render a VPP node configuration with a given End Point, e.g. the virtual host -interface name assigned to a vhostuser socket.. The mapped data is stored in -the VPP info data store. - -**VPP Info Store**: Stores VPP specific information regarding End-Points, Flood -domains with VLAN, etc. - -**GBP Renderer Manager**: The GBP Renderer Manager is the central point for -dispatching of data to specific device renderers. It uses the information -derived from the GBP end-point and its topology entries to dispatch the task of -configuration to a specific device renderer by writing a renderer policy -configuration into the registered renderer's policy store. The renderer manager -also monitors, by being a data change listener on the VPP Renderer Policy -States, for any errors in the application of a rendered configuration. - -**Renderer Policy Config Store**: The store's schema serves as the API between -the Renderer Manager and specific Renderers like the VPP Renderer. The store -uses a a YANG modeled schema to represent all end-point and associated GBP -policy data. - -**Topology Entries Store**: The yang model based MD-SAL topology store serves -two fundamental roles: 1. It maintains a topological representation of the GBP -End Points, in the context of customer networks. 2. It maintains an association -of each (VPP) compute node's physical interfaces to their neutron provider -network (e.g. The association between an ethernet interface and a Neutron -provider network). - -**VPP Renderer**: The VPP Renderer registers an instance for VPP nodes with the -Renderer Manager by means of inserting operational data into the Renderer -Policy config store. It acts as a listener on the Renderer Policy consumes via -the GBP Policy API data + the specific VPP End Point data, to drive the -configuration of VPP devices using NETCONF Services. -More specifically, the renderer generates: - - * vhost user port configuration that corresponds to the VM port configuration - * VPP bridge instances corresponding to the GBP flood domain - * port or traffic filtering configuration, in accordance with the GBP policy. - -The VPP Renderer also interacts with the Virtual Bridge Domain Service, by -means of the VBD store, in order to establish connectivity between VPP nodes in -a bridge domain. For this it uses the VPP device name, and the flood domain -data derived from the VPP Info and End-Point data respectively. For the -executed configuration operations it updates state in the Renderer policy state -store. - -**Virtual Bridge Domain (VBD) Store and Manager**: The virtual bridge domain -manager is responsible for configuring the VxLAN overlay tunnel infrastructure -to arrive at a desired bridged topology between multiple (VPP) compute nodes. -VDB configures VXLAN tunnels always into a full-mesh with split-horizon group -forwarding applied on any domain facing tunnel interface (i.e. forwarding -behavior will be that used for VPLS). - -**NETCONF Mount Point Service & Connector**: Collectively referred to as -Netconf Services, provide the NETCONF interface for accessing VPP configuration -and operational data stores that are represented as NETCONF mounts. +**Openstack Neutron ML2 OpenDaylight Plugin**: Handles Neutron data base +synchronization and interaction with the southbound controller using a REST +interface. + +**ODL GBP Neutron Mapper**: Maps neutron elements like networks, subnets, +security groups, etc. to GBP entities: Creates policy and configuration for +tenants (endpoints, resolved policies, forwarding rules). + +**ODL GBP Neutron VPP Mapper**: Maps Neutron ports to VPP endpoints in GBP. + +**ODL GBP Location Manager**: Provides real location for endpoints (i.e. Which +physical node an endpoint is connected to). + +**GBP Renderer Manager**: Creates configuration for Renderers (like e.g. +VPP-Renderer or OVS-Renderer). The GBP Renderer Manager is the central point +for dispatching of data to specific device renderers. It uses the information +derived from the GBP end-point and its topology entries to dispatch the task +of configuration to a specific device renderer by writing a renderer policy +configuration into the registered renderer's policy store. The renderer +manager also monitors, by being a data change listener on the VPP Renderer +Policy States, for any errors in the application of a rendered configuration. + +**GBP VPP Renderer Interface Manager**: Listens to VPP endpoints in the +Config DataStore and configures associated interfaces on VPP via HoneyComb. + +**GBP VPP Renderer Renderer Policy Manager**: Manages the creation of +bridge domains using VBD and assigns interfaces to bridge domains. + +**Virtual Bridge Domain Manager (VBD)**: Creates bridge domains (i.e. in case +of VXLAN creates full mesh of VXLAN tunnels, configures split horizon on +tunnel endpoints etc.). VDB configures VXLAN tunnels always into a full-mesh +with split-horizon group forwarding applied on any domain facing tunnel +interface (i.e. forwarding behavior will be that used for VPLS). **Virtual Packet Processor (VPP) and Honeycomb server**: The VPP is the accelerated data plane forwarding engine relying on vhost user interfaces @@ -217,20 +170,45 @@ towards Virtual Machines created by the Nova Agent. The Honeycomb NETCONF configuration server is responsible for driving the configuration of the VPP, and collecting the operational data. -**Rendered Policy State Store**: Stores data regarding the execution of -operations performed by a given renderer. - **Nova Agent**: The Nova Agent, a sub-component of the overall Openstack architecture, is responsible for interacting with the compute node's host Libvirt API to drive the life-cycle of Virtual Machines. It, along with the compute node software, are assumed to be capable of supporting vhost user interfaces. -The picture below show a basic end to end call flow for creating a Neutron -vhostuser port on VPP using a GBP renderer. It showcases how the different -component described above interact. +The picture below shows the key components. + +.. image:: FDS-basic-components.jpg + +To provide a better understanding how the above mentioned components interact +with each other, the following diagram shows how the example of creating a +vhost-user port on VPP through Openstack Neutron: + +To create or update a port, Neutron will send a request to ODL Neutron +Northbound which contains the UUID, along with the host-id as "vpp" and +vif-type as "vhost-user". The GBP Neutron mapper turns the "Neutron speak" of +"ports" into the generic connectivity model that GroupBasedPolicy uses. +Neutron "ports" become generic "GBP Endpoints" which can be consumed by the +GBP Renderer Manager. The GBP Renderer Manager resolves the policy for the +endpoint, i.e. it determines which communication relationships apply to the +specific endpoint, and hands the resolution to a device specific renderer, +which is the VPP renderer in the given case here. VPP renderer turns the +generic policy into VPP specific configuration. Note that in case the policy +would need to be applied to a different device, e.g. an OpenVSwitch (OVS), +then an "OVS Renderer" would be used. VPP Renderer and the topology manager +("Virtual Bridge Domain" manager - i.e. VBD) cooperate to create the actual +network configuration. VPP Renderer configures the interfaces to the virtual +machines (VM), i.e. the vhost-user interface in the given case here and +attaches them to a bridge domain on VPP. VBD handles the setup of connectivity +between bridge domains on individual VPPs, i.e. it maintains the VXLAN tunnels +in the given case here. Both VPP Renderer as well as VBD communicate with the +device through Netconf/YANG. All compute and control nodes run an instance of +VPP and the VPP-configuration agent "Honeycomb". Honeycomb serves as a +Netconf/YANG server, receives the configuration commands from VBD and VPP +Renderer and drives VPP configuration using VPP's local Java APIs. + +.. image:: FDS-simple-callflow.png -.. image:: FDS-basic-callflow.jpg Scenario Configuration ====================== @@ -240,7 +218,11 @@ settings in the APEX configuration files. Those are typically found in /etc/opnfv-apex. File "deploy_settings.yaml" choose opendaylight as controller with version -"boron" and enable vpp as forwarder. "hugepages" need to set to a +"carbon" and enable vpp as forwarder. Also make sure that you set +"ha_enabled" to "true" in the global_params section. "ha_enabled" is the +only real difference from a configuration file perspective between the +scenario with high availability when compared to the ODL-L2 scenario +without high-availability support. "hugepages" need to set to a sufficiently large value for VPP to work. The default value for VPP is 1024, but this only allows for a few VMs to be started. If feasible, choose a significantly larger number on the compute nodes:: @@ -251,7 +233,7 @@ choose a significantly larger number on the compute nodes:: deploy_options: sdn_controller: opendaylight sdn_l3: false - odl_version: boron + odl_version: carbon tacker: true congress: true sfc: false @@ -265,12 +247,22 @@ choose a significantly larger number on the compute nodes:: hugepagesz: 2M intel_iommu: 'on' iommu: pt + isolcpus: 1,2 + vpp: + main-core: 1 + corelist-workers: 2 + uio-driver: uio_pci_generic Compute: kernel: hugepagesz: 2M hugepages: 2048 intel_iommu: 'on' iommu: pt + isolcpus: 1,2 + vpp: + main-core: 1 + corelist-workers: 2 + uio-driver: uio_pci_generic Validated deployment environments @@ -290,9 +282,10 @@ on the following sets of hardware: Limitations, Issues and Workarounds =================================== -There are no known issues. Note that only OpenStack is deployed in -HA mode. OpenDaylight clustering is expected to be added in a future -revision of this scenario. +For specific information on limitations and issues, please refer to the APEX +installer release notes. Note that this high availability scenario +deploys OpenStack in HA mode *and* OpenDaylight in cluster mode. + References ========== @@ -302,5 +295,5 @@ References * Fast Data (FD.io): https://fd.io/ * FD.io Vector Packet Processor (VPP): https://wiki.fd.io/view/VPP * OpenDaylight Controller: https://www.opendaylight.org/ - * OPNFV Colorado release - more information: http://www.opnfv.org/colorado + * OPNFV Danube release - more information: http://www.opnfv.org/danube diff --git a/docs/scenarios/os-odl_l2-fdio-noha/FDS-basic-components.jpg b/docs/scenarios/os-odl_l2-fdio-noha/FDS-basic-components.jpg Binary files differnew file mode 100755 index 0000000..e92851f --- /dev/null +++ b/docs/scenarios/os-odl_l2-fdio-noha/FDS-basic-components.jpg diff --git a/docs/scenarios/os-odl_l2-fdio-noha/FDS-simple-callflow.png b/docs/scenarios/os-odl_l2-fdio-noha/FDS-simple-callflow.png Binary files differnew file mode 100755 index 0000000..04546aa --- /dev/null +++ b/docs/scenarios/os-odl_l2-fdio-noha/FDS-simple-callflow.png diff --git a/docs/scenarios/os-odl_l2-fdio-noha/scenario.description.rst b/docs/scenarios/os-odl_l2-fdio-noha/scenario.description.rst index 076e37c..b392070 100755 --- a/docs/scenarios/os-odl_l2-fdio-noha/scenario.description.rst +++ b/docs/scenarios/os-odl_l2-fdio-noha/scenario.description.rst @@ -39,8 +39,8 @@ NFV infrastructure are reflect different business In order to meet the desired qualities of an NFV infrastructure, the -following components were chosen for the "Openstack - OpenDaylight - - FD.io/VPP" scenario: +following components were chosen for the "Openstack - OpenDaylight - FD.io" +scenario: * FD.io Vector Packet Processor (VPP) - a highly scalable, high performance, extensible virtual forwarder * OpenDaylight Controller - an extensible controller platform which @@ -50,7 +50,7 @@ following components were chosen for the "Openstack - OpenDaylight component, and can be clustered to achieve a highly available deployment. -The "Openstack - OpenDaylight - FD.io/VPP" scenario provides the capability to +The "Openstack - OpenDaylight - FD.io" scenario provides the capability to realize a set of use-cases relevant to the deployment of NFV nodes instantiated by means of an Openstack orchestration system on FD.io/VPP enabled compute nodes. The role of the Opendaylight network controller in this integration is @@ -76,8 +76,8 @@ servers: .. image:: FDS-odl_l2-overview.png -Tenant networking leverages FD.io/VPP. Open VSwitch (OVS) is used for all other -connectivity, in particular the connectivity to public networking / the +Tenant networking leverages FD.io/VPP. Open VSwitch (OVS) is used for all +other connectivity, in particular the connectivity to public networking / the Internet (i.e. br-ext) is performed via OVS as in any standard OpenStack deployment. The OpenDaylight network controller is used to setup and manage layer 2 networking for the scenario. Tenant networking can either leverage @@ -113,94 +113,45 @@ The apex-os-odl_l2-fdio-noha scenario combines components from three key open source projects: OpenStack, OpenDaylight, and Fast Data (FD.io). The key components that realize the apex-os-odl_l2-fdio-noha scenario and which differ from a regular, OVS-based scenario, are the OpenStack ML2 OpenDaylight plugin, -OpenDaylight Neutron Northbound, OpenDaylight Group Based Policy, OpenDaylight -Virtual Bridge Domain Manager, FD.io Honeycomb management agent and FD.io -Vector Packet Processor (VPP). +OpenDaylight Neutron Northbound, OpenDaylight (ODL) Group Based Policy (GBP), +OpenDaylight Virtual Bridge Domain Manager (VBD), FD.io Honeycomb management +agent and FD.io Vector Packet Processor (VPP). Here's a more detailed list of the individual software components involved: -**Openstack Neutron ML2 ODL Plugin**: Handles Neutron data base synchronization -and interaction with the southbound Openstack controller using HTTP. - -**OpenDaylight Neutron Nothbound & Neutron MD-SAL Entry Store**: Presents a -Neutron (v2) extended HTTP API servlet for interaction with Openstack Neutron. -It validates and stores the received Neutron data in the MD-SAL data store -against the Neutron yang model driven. - -**OpenDaylight Neutron Mapper**: The Neutron Mapper listens to Neutron data -change events and is responsible for using Neutron data in creating Group Based -Policy Data objects, e.g. GBP End-Points, Flood-Domains. A GBP End Point -represents a specific NFV/VM port and its identity as derived from a Neutron -Port. The mapped data is stored using the GBP End Point yang model and an -association between the GBP End-Point and its Neutron object is maintained in -the Neutron-GBP map. - -**OpenDaylight Group Based Policy (GBP) Entities store**: Stores for the GBP -data artifacts against the GBP YANG schemas. - -**Neutron Group Based Policy Map store**: Stores the bi-lateral relation -between an End-Point and its corresponding Neutron object. Neutron-GBP map; -keyed by Neutron object type, port, and Neutron UUID, gives the GBP End-Point, -Flood domain respectively. GBP-Neutron map keyed by GBP object type, end-point. - -**Neutron VPP Renderer Mapper**: The Neutron VPP Renderer Mapper listens to -Neutron Store data change events, as well as being able to access directly the -store, and is responsible for converting Neutron data specifically required to -render a VPP node configuration with a given End Point, e.g. the virtual host -interface name assigned to a vhostuser socket.. The mapped data is stored in -the VPP info data store. - -**VPP Info Store**: Stores VPP specific information regarding End-Points, Flood -domains with VLAN, etc. - -**GBP Renderer Manager**: The GBP Renderer Manager is the central point for -dispatching of data to specific device renderers. It uses the information -derived from the GBP end-point and its topology entries to dispatch the task of -configuration to a specific device renderer by writing a renderer policy -configuration into the registered renderer's policy store. The renderer manager -also monitors, by being a data change listener on the VPP Renderer Policy -States, for any errors in the application of a rendered configuration. - -**Renderer Policy Config Store**: The store's schema serves as the API between -the Renderer Manager and specific Renderers like the VPP Renderer. The store -uses a a YANG modeled schema to represent all end-point and associated GBP -policy data. - -**Topology Entries Store**: The yang model based MD-SAL topology store serves -two fundamental roles: 1. It maintains a topological representation of the GBP -End Points, in the context of customer networks. 2. It maintains an association -of each (VPP) compute node's physical interfaces to their neutron provider -network (e.g. The association between an ethernet interface and a Neutron -provider network). - -**VPP Renderer**: The VPP Renderer registers an instance for VPP nodes with the -Renderer Manager by means of inserting operational data into the Renderer -Policy config store. It acts as a listener on the Renderer Policy consumes via -the GBP Policy API data + the specific VPP End Point data, to drive the -configuration of VPP devices using NETCONF Services. -More specifically, the renderer generates: - - * vhost user port configuration that corresponds to the VM port configuration - * VPP bridge instances corresponding to the GBP flood domain - * port or traffic filtering configuration, in accordance with the GBP policy. - -The VPP Renderer also interacts with the Virtual Bridge Domain Service, by -means of the VBD store, in order to establish connectivity between VPP nodes in -a bridge domain. For this it uses the VPP device name, and the flood domain -data derived from the VPP Info and End-Point data respectively. For the -executed configuration operations it updates state in the Renderer policy state -store. - -**Virtual Bridge Domain (VBD) Store and Manager**: The virtual bridge domain -manager is responsible for configuring the VxLAN overlay tunnel infrastructure -to arrive at a desired bridged topology between multiple (VPP) compute nodes. -VDB configures VXLAN tunnels always into a full-mesh with split-horizon group -forwarding applied on any domain facing tunnel interface (i.e. forwarding -behavior will be that used for VPLS). - -**NETCONF Mount Point Service & Connector**: Collectively referred to as -Netconf Services, provide the NETCONF interface for accessing VPP configuration -and operational data stores that are represented as NETCONF mounts. +**Openstack Neutron ML2 OpenDaylight Plugin**: Handles Neutron data base +synchronization and interaction with the southbound controller using a REST +interface. + +**ODL GBP Neutron Mapper**: Maps neutron elements like networks, subnets, +security groups, etc. to GBP entities: Creates policy and configuration for +tenants (endpoints, resolved policies, forwarding rules). + +**ODL GBP Neutron VPP Mapper**: Maps Neutron ports to VPP endpoints in GBP. + +**ODL GBP Location Manager**: Provides real location for endpoints (i.e. Which +physical node an endpoint is connected to). + +**GBP Renderer Manager**: Creates configuration for Renderers (like e.g. +VPP-Renderer or OVS-Renderer). The GBP Renderer Manager is the central point +for dispatching of data to specific device renderers. It uses the information +derived from the GBP end-point and its topology entries to dispatch the task +of configuration to a specific device renderer by writing a renderer policy +configuration into the registered renderer's policy store. The renderer +manager also monitors, by being a data change listener on the VPP Renderer +Policy States, for any errors in the application of a rendered configuration. + +**GBP VPP Renderer Interface Manager**: Listens to VPP endpoints in the +Config DataStore and configures associated interfaces on VPP via HoneyComb. + +**GBP VPP Renderer Renderer Policy Manager**: Manages the creation of +bridge domains using VBD and assigns interfaces to bridge domains. + +**Virtual Bridge Domain Manager (VBD)**: Creates bridge domains (i.e. in case +of VXLAN creates full mesh of VXLAN tunnels, configures split horizon on +tunnel endpoints etc.). VDB configures VXLAN tunnels always into a full-mesh +with split-horizon group forwarding applied on any domain facing tunnel +interface (i.e. forwarding behavior will be that used for VPLS). **Virtual Packet Processor (VPP) and Honeycomb server**: The VPP is the accelerated data plane forwarding engine relying on vhost user interfaces @@ -208,20 +159,45 @@ towards Virtual Machines created by the Nova Agent. The Honeycomb NETCONF configuration server is responsible for driving the configuration of the VPP, and collecting the operational data. -**Rendered Policy State Store**: Stores data regarding the execution of -operations performed by a given renderer. - **Nova Agent**: The Nova Agent, a sub-component of the overall Openstack architecture, is responsible for interacting with the compute node's host Libvirt API to drive the life-cycle of Virtual Machines. It, along with the compute node software, are assumed to be capable of supporting vhost user interfaces. -The picture below show a basic end to end call flow for creating a Neutron -vhostuser port on VPP using a GBP renderer. It showcases how the different -component described above interact. +The picture below shows the key components. + +.. image:: FDS-basic-components.jpg + +To provide a better understanding how the above mentioned components interact +with each other, the following diagram shows how the example of creating a +vhost-user port on VPP through Openstack Neutron: + +To create or update a port, Neutron will send a request to ODL Neutron +Northbound which contains the UUID, along with the host-id as "vpp" and +vif-type as "vhost-user". The GBP Neutron mapper turns the "Neutron speak" of +"ports" into the generic connectivity model that GroupBasedPolicy uses. +Neutron "ports" become generic "GBP Endpoints" which can be consumed by the +GBP Renderer Manager. The GBP Renderer Manager resolves the policy for the +endpoint, i.e. it determines which communication relationships apply to the +specific endpoint, and hands the resolution to a device specific renderer, +which is the VPP renderer in the given case here. VPP renderer turns the +generic policy into VPP specific configuration. Note that in case the policy +would need to be applied to a different device, e.g. an OpenVSwitch (OVS), +then an "OVS Renderer" would be used. VPP Renderer and the topology manager +("Virtual Bridge Domain" manager - i.e. VBD) cooperate to create the actual +network configuration. VPP Renderer configures the interfaces to the virtual +machines (VM), i.e. the vhost-user interface in the given case here and +attaches them to a bridge domain on VPP. VBD handles the setup of connectivity +between bridge domains on individual VPPs, i.e. it maintains the VXLAN tunnels +in the given case here. Both VPP Renderer as well as VBD communicate with the +device through Netconf/YANG. All compute and control nodes run an instance of +VPP and the VPP-configuration agent "Honeycomb". Honeycomb serves as a +Netconf/YANG server, receives the configuration commands from VBD and VPP +Renderer and drives VPP configuration using VPP's local Java APIs. + +.. image:: FDS-simple-callflow.png -.. image:: FDS-basic-callflow.jpg Scenario Configuration ====================== @@ -231,10 +207,10 @@ settings in the APEX configuration files. Those are typically found in /etc/opnfv-apex. File "deploy_settings.yaml" choose opendaylight as controller with version -"boron" and enable vpp as forwarder. "hugepages" need to set to a -sufficiently large value for VPP to work. The default value for VPP is -1024, but this only allows for a few VMs to be started. If feasible, -choose a significantly larger number on the compute nodes:: +"carbon" and enable vpp as forwarder. "hugepages" need to set to a +sufficiently large value for VPP to work. The default value for VPP is 1024, +but this only allows for a few VMs to be started. If feasible, choose a +significantly larger number on the compute nodes:: global_params: ha_enabled: false @@ -242,7 +218,7 @@ choose a significantly larger number on the compute nodes:: deploy_options: sdn_controller: opendaylight sdn_l3: false - odl_version: boron + odl_version: carbon tacker: true congress: true sfc: false @@ -256,14 +232,22 @@ choose a significantly larger number on the compute nodes:: hugepagesz: 2M intel_iommu: 'on' iommu: pt + isolcpus: 1,2 + vpp: + main-core: 1 + corelist-workers: 2 + uio-driver: uio_pci_generic Compute: - nova: - libvirtpin: 1 kernel: hugepagesz: 2M hugepages: 2048 intel_iommu: 'on' iommu: pt + isolcpus: 1,2 + vpp: + main-core: 1 + corelist-workers: 2 + uio-driver: uio_pci_generic Validated deployment environments @@ -283,7 +267,7 @@ on the following sets of hardware: Limitations, Issues and Workarounds =================================== -There are no known issues. +For specific information on limitations and issues, please refer to the APEX installer release notes. References ========== @@ -293,5 +277,5 @@ References * Fast Data (FD.io): https://fd.io/ * FD.io Vector Packet Processor (VPP): https://wiki.fd.io/view/VPP * OpenDaylight Controller: https://www.opendaylight.org/ - * OPNFV Colorado release - more information: http://www.opnfv.org/colorado + * OPNFV Carbon release - more information: http://www.opnfv.org/carbon diff --git a/docs/scenarios/os-odl_l3-fdio-noha/FDS-L3-noha-sample-setup.png b/docs/scenarios/os-odl_l3-fdio-noha/FDS-L3-noha-sample-setup.png Binary files differnew file mode 100755 index 0000000..27c8335 --- /dev/null +++ b/docs/scenarios/os-odl_l3-fdio-noha/FDS-L3-noha-sample-setup.png diff --git a/docs/scenarios/os-odl_l3-fdio-noha/FDS-basic-components.jpg b/docs/scenarios/os-odl_l3-fdio-noha/FDS-basic-components.jpg Binary files differnew file mode 100755 index 0000000..e92851f --- /dev/null +++ b/docs/scenarios/os-odl_l3-fdio-noha/FDS-basic-components.jpg diff --git a/docs/scenarios/os-odl_l3-fdio-noha/FDS-odl_l3-noha-overview.png b/docs/scenarios/os-odl_l3-fdio-noha/FDS-odl_l3-noha-overview.png Binary files differnew file mode 100755 index 0000000..1193ea4 --- /dev/null +++ b/docs/scenarios/os-odl_l3-fdio-noha/FDS-odl_l3-noha-overview.png diff --git a/docs/scenarios/os-odl_l3-fdio-noha/FDS-simple-callflow.png b/docs/scenarios/os-odl_l3-fdio-noha/FDS-simple-callflow.png Binary files differnew file mode 100755 index 0000000..04546aa --- /dev/null +++ b/docs/scenarios/os-odl_l3-fdio-noha/FDS-simple-callflow.png diff --git a/docs/scenarios/os-odl_l3-fdio-noha/scenario.description.rst b/docs/scenarios/os-odl_l3-fdio-noha/scenario.description.rst index bf32eb6..6bbf12b 100755 --- a/docs/scenarios/os-odl_l3-fdio-noha/scenario.description.rst +++ b/docs/scenarios/os-odl_l3-fdio-noha/scenario.description.rst @@ -40,8 +40,8 @@ NFV infrastructure are reflect different business In order to meet the desired qualities of an NFV infrastructure, the -following components were chosen for the "Openstack - OpenDaylight - - FD.io/VPP" scenario: +following components were chosen for the "Openstack - OpenDaylight - FD.io" +scenario: * FD.io Vector Packet Processor (VPP) - a highly scalable, high performance, extensible virtual forwarder * OpenDaylight Controller - an extensible controller platform which @@ -52,7 +52,7 @@ following components were chosen for the "Openstack - OpenDaylight deployment. -The "Openstack - OpenDaylight - FD.io/VPP" scenario provides the capability to +The "Openstack - OpenDaylight - FD.io" scenario provides the capability to realize a set of use-cases relevant to the deployment of NFV nodes instantiated by means of an Openstack orchestration system on FD.io/VPP enabled compute nodes. The role of the Opendaylight network controller in this integration is @@ -74,22 +74,30 @@ servers: * 1 Jumphost hosting the APEX installer - running the Undercloud * 1 Controlhost, which runs the Overcloud as well as OpenDaylight as a network controller - * 2 or more Computehosts - -.. image:: FDS-odl_l3-overview.png - -Tenant and public networking leverages FD.io/VPP. VPP binds to both, the tenant -networking interface as well as to the public networking interface on the -compute and control nodes. The OpenDaylight network controller is used to setup -and manage layer 2 and layer 3 networking for the scenario - with Group Based -Policy (GBP) being the key component. Tenant networking can either leverage -VXLAN (in which case a full mesh of VXLAN tunnels is created) or VLANs. Layer 3 -connectivity is provided by using VPP as a "distributed virtual router". - -The picture below gives an example for distributed routing using VRFs between -tenant networks. - -.. image:: FDS-L3-DVR-example.png + * 2 or more Computehosts. One of the Computehosts also serves as + layer 3 gateway for the tenant networks. + +.. image:: FDS-odl_l3-noha-overview.png + +Tenant and public networking leverages FD.io/VPP. On one of the compute nodes, +VPP binds to both, the tenant networking interface as well as to the public +networking interface. This means that VPP is used for communication within +a tenant network, between tenant networks, as well as between a tenant network +and the Internet. Note that this setup slightly differs from the usual +centralized L3 setup with qrouter on the control node. This setup was chosen +to limit the configuration changes for the introduction of FD.io/VPP. The +OpenDaylight network controller is used to setup and manage layer 2 and +layer 3 networking for the scenario - with Group Based Policy (GBP) being the +key component. Tenant networking can either leverage VXLAN (in which case a +full mesh of VXLAN tunnels is created) or VLANs. + +The picture below shows an example setup with two compute and one control +node. Note that the external network is connected via compute node 0 through +VPP. VPP provides all layer 3 services which are provided in a "vanilla" +OpenStack deployment, including SNAT and DNAT, as well as north-south +and east-west traffic filtering for security purposes ("security groups"). + +.. image:: FDS-L3-noha-sample-setup.png Features of the scenario ------------------------ @@ -100,7 +108,7 @@ Main features of the "apex-os-odl_l3-fdio-noha" scenario: * Fast and scalable tenant networking using FD.io/VPP as forwarder * Layer 2 networking using VLANs or VXLAN, managed and controlled through OpenDaylight - * Layer 3 connectivitiy for tenant networks supplied in a distributed way + * Layer 3 connectivitiy for tenant networks supplied through FD.io/VPP. Layer 3 features, including security groups as well as floating IP addresses (i.e. NAT) are implemented by the FD.io/VPP forwarder * Manual and automatic (via DHCP) addressing on tenant networks @@ -122,86 +130,39 @@ and the Layer 3 scenario "apex-os-odl_l3-fdio-noha" share the same components. Here's a more detailed list of the individual software components involved: -**Openstack Neutron ML2 ODL Plugin**: Handles Neutron data base synchronization -and interaction with the southbound Openstack controller using HTTP. - -**OpenDaylight Neutron Nothbound & Neutron MD-SAL Entry Store**: Presents a -Neutron (v2) extended HTTP API servlet for interaction with Openstack Neutron. -It validates and stores the received Neutron data in the MD-SAL data store -against the Neutron yang model driven. - -**OpenDaylight Neutron Mapper**: The Neutron Mapper listens to Neutron data -change events and is responsible for using Neutron data in creating Group Based -Policy Data objects, e.g. GBP End-Points, Flood-Domains. A GBP End Point -represents a specific NFV/VM port and its identity as derived from a Neutron -Port. The mapped data is stored using the GBP End Point yang model and an -association between the GBP End-Point and its Neutron object is maintained in -the Neutron-GBP map. - -**OpenDaylight Group Based Policy (GBP) Entities store**: Stores for the GBP -data artifacts against the GBP YANG schemas. - -**Neutron Group Based Policy Map store**: Stores the bi-lateral relation -between an End-Point and its corresponding Neutron object. Neutron-GBP map; -keyed by Neutron object type, port, and Neutron UUID, gives the GBP End-Point, -Flood domain respectively. GBP-Neutron map keyed by GBP object type, end-point. - -**Neutron VPP Renderer Mapper**: The Neutron VPP Renderer Mapper listens to -Neutron Store data change events, as well as being able to access directly the -store, and is responsible for converting Neutron data specifically required to -render a VPP node configuration with a given End Point, e.g. the virtual host -interface name assigned to a vhostuser socket.. The mapped data is stored in -the VPP info data store. - -**VPP Info Store**: Stores VPP specific information regarding End-Points, Flood -domains with VLAN, etc. - -**GBP Renderer Manager**: The GBP Renderer Manager is the central point for -dispatching of data to specific device renderers. It uses the information -derived from the GBP end-point and its topology entries to dispatch the task of -configuration to a specific device renderer by writing a renderer policy -configuration into the registered renderer's policy store. The renderer manager -also monitors, by being a data change listener on the VPP Renderer Policy -States, for any errors in the application of a rendered configuration. - -**Renderer Policy Config Store**: The store's schema serves as the API between -the Renderer Manager and specific Renderers like the VPP Renderer. The store -uses a a YANG modeled schema to represent all end-point and associated GBP -policy data. - -**Topology Entries Store**: The yang model based MD-SAL topology store serves -two fundamental roles: 1. It maintains a topological representation of the GBP -End Points, in the context of customer networks. 2. It maintains an association -of each (VPP) compute node's physical interfaces to their neutron provider -network (e.g. The association between an ethernet interface and a Neutron -provider network). - -**VPP Renderer**: The VPP Renderer registers an instance for VPP nodes with the -Renderer Manager by means of inserting operational data into the Renderer -Policy config store. It acts as a listener on the Renderer Policy consumes via -the GBP Policy API data + the specific VPP End Point data, to drive the -configuration of VPP devices using NETCONF Services. -More specifically, the renderer generates: - - * vhost user port configuration that corresponds to the VM port configuration - * VPP bridge instances corresponding to the GBP flood domain - * port or traffic filtering configuration, in accordance with the GBP policy. - -The VPP Renderer also interacts with the Virtual Bridge Domain Service, by -means of the VBD store, in order to establish connectivity between VPP nodes in -a bridge domain. For this it uses the VPP device name, and the flood domain -data derived from the VPP Info and End-Point data respectively. For the -executed configuration operations it updates state in the Renderer policy state -store. - -**Virtual Bridge Domain (VBD) Store and Manager**: The virtual bridge domain -manager is responsible for configuring the VxLAN overlay tunnel infrastructure -to arrive at a desired bridged topology between multiple (VPP) compute nodes. -VDB configures VXLAN tunnels always into a full-mesh with split-horizon group forwarding applied on any domain facing tunnel interface (i.e. forwarding behavior will be that used for VPLS). - -**NETCONF Mount Point Service & Connector**: Collectively referred to as -Netconf Services, provide the NETCONF interface for accessing VPP configuration -and operational data stores that are represented as NETCONF mounts. +**Openstack Neutron ML2 OpenDaylight Plugin**: Handles Neutron data base +synchronization and interaction with the southbound controller using a REST +interface. + +**ODL GBP Neutron Mapper**: Maps neutron elements like networks, subnets, +security groups, etc. to GBP entities: Creates policy and configuration for +tenants (endpoints, resolved policies, forwarding rules). + +**ODL GBP Neutron VPP Mapper**: Maps Neutron ports to VPP endpoints in GBP. + +**ODL GBP Location Manager**: Provides real location for endpoints (i.e. Which +physical node an endpoint is connected to). + +**GBP Renderer Manager**: Creates configuration for Renderers (like e.g. +VPP-Renderer or OVS-Renderer). The GBP Renderer Manager is the central point +for dispatching of data to specific device renderers. It uses the information +derived from the GBP end-point and its topology entries to dispatch the task +of configuration to a specific device renderer by writing a renderer policy +configuration into the registered renderer's policy store. The renderer +manager also monitors, by being a data change listener on the VPP Renderer +Policy States, for any errors in the application of a rendered configuration. + +**GBP VPP Renderer Interface Manager**: Listens to VPP endpoints in the +Config DataStore and configures associated interfaces on VPP via HoneyComb. + +**GBP VPP Renderer Renderer Policy Manager**: Manages the creation of +bridge domains using VBD and assigns interfaces to bridge domains. + +**Virtual Bridge Domain Manager (VBD)**: Creates bridge domains (i.e. in case +of VXLAN creates full mesh of VXLAN tunnels, configures split horizon on +tunnel endpoints etc.). VDB configures VXLAN tunnels always into a full-mesh +with split-horizon group forwarding applied on any domain facing tunnel +interface (i.e. forwarding behavior will be that used for VPLS). **Virtual Packet Processor (VPP) and Honeycomb server**: The VPP is the accelerated data plane forwarding engine relying on vhost user interfaces @@ -209,20 +170,44 @@ towards Virtual Machines created by the Nova Agent. The Honeycomb NETCONF configuration server is responsible for driving the configuration of the VPP, and collecting the operational data. -**Rendered Policy State Store**: Stores data regarding the execution of -operations performed by a given renderer. - **Nova Agent**: The Nova Agent, a sub-component of the overall Openstack architecture, is responsible for interacting with the compute node's host Libvirt API to drive the life-cycle of Virtual Machines. It, along with the compute node software, are assumed to be capable of supporting vhost user interfaces. -The picture below show a basic end to end call flow for creating a Neutron -vhostuser port on VPP using a GBP renderer. It showcases how the different -component described above interact. - -.. image:: FDS-basic-callflow.jpg +The picture below shows the key components. + +.. image:: FDS-basic-components.jpg + +To provide a better understanding how the above mentioned components interact +with each other, the following diagram shows how the example of creating a +vhost-user port on VPP through Openstack Neutron: + +To create or update a port, Neutron will send a request to ODL Neutron +Northbound which contains the UUID, along with the host-id as "vpp" and +vif-type as "vhost-user". The GBP Neutron mapper turns the "Neutron speak" of +"ports" into the generic connectivity model that GroupBasedPolicy uses. +Neutron "ports" become generic "GBP Endpoints" which can be consumed by the +GBP Renderer Manager. The GBP Renderer Manager resolves the policy for the +endpoint, i.e. it determines which communication relationships apply to the +specific endpoint, and hands the resolution to a device specific renderer, +which is the VPP renderer in the given case here. VPP renderer turns the +generic policy into VPP specific configuration. Note that in case the policy +would need to be applied to a different device, e.g. an OpenVSwitch (OVS), +then an "OVS Renderer" would be used. VPP Renderer and the topology manager +("Virtual Bridge Domain" manager - i.e. VBD) cooperate to create the actual +network configuration. VPP Renderer configures the interfaces to the virtual +machines (VM), i.e. the vhost-user interface in the given case here and +attaches them to a bridge domain on VPP. VBD handles the setup of connectivity +between bridge domains on individual VPPs, i.e. it maintains the VXLAN tunnels +in the given case here. Both VPP Renderer as well as VBD communicate with the +device through Netconf/YANG. All compute and control nodes run an instance of +VPP and the VPP-configuration agent "Honeycomb". Honeycomb serves as a +Netconf/YANG server, receives the configuration commands from VBD and VPP +Renderer and drives VPP configuration using VPP's local Java APIs. + +.. image:: FDS-simple-callflow.png Scenario Configuration ====================== @@ -232,25 +217,51 @@ settings in the APEX configuration files. Those are typically found in /etc/opnfv-apex. File "deploy_settings.yaml": Choose Opendaylight as controller with version -"boron" and enable vpp as forwarder:: - - global_params: - ha_enabled: false +"carbon" and enable vpp as forwarder. "odl_routing_node" chooses the node +which is used as the layer 3 gateway for the tenant networks. +"odl_routing_node" is an optional parameter. If omitted, the VPP on the first +compute node will serve as layer 3 gateway:: deploy_options: sdn_controller: opendaylight sdn_l3: true - odl_version: boron - tacker: false - congress: false + odl_version: carbon + odl_routing_node: overcloud-novacompute-0 + tacker: true + congress: true sfc: false vpn: false vpp: true + dataplane: fdio + performance: + Controller: + kernel: + hugepages: 1024 + hugepagesz: 2M + intel_iommu: 'on' + iommu: pt + isolcpus: 1,2 + vpp: + main-core: 1 + corelist-workers: 2 + uio-driver: uio_pci_generic + Compute: + kernel: + hugepagesz: 2M + hugepages: 2048 + intel_iommu: 'on' + iommu: pt + isolcpus: 1,2 + vpp: + main-core: 1 + corelist-workers: 2 + uio-driver: uio_pci_generic Limitations, Issues and Workarounds =================================== -There are no known issues. +For specific information on limitations and issues, please refer to the APEX +installer release notes. References ========== @@ -260,4 +271,4 @@ References * Fast Data (FD.io): https://fd.io/ * FD.io Vector Packet Processor (VPP): https://wiki.fd.io/view/VPP * OpenDaylight Controller: https://www.opendaylight.org/ - * OPNFV Colorado release - more information: http://www.opnfv.org/colorado + * OPNFV Danube release - more information: http://www.opnfv.org/danube diff --git a/testing/robot/data/test_data.py b/testing/robot/data/test_data.py index d78ce4f..97aaf2b 100644 --- a/testing/robot/data/test_data.py +++ b/testing/robot/data/test_data.py @@ -12,6 +12,8 @@ import uuid run_uuid = str(uuid.uuid4()) network_name = 'fds_smoke_network_' + run_uuid subnet_name = 'fds_smoke_subnet_' + run_uuid +sg_client = 'client' +sg_server = 'server' vm1_name = 'fds_smoke_vm1_' + run_uuid vm1_address = '192.168.10.5' vm2_name = 'fds_smoke_vm2_' + run_uuid @@ -20,8 +22,8 @@ port1_name = 'fds_smoke_port1_' + run_uuid port2_name = 'fds_smoke_port2_' + run_uuid subnet_cidr = '192.168.10.0/24' vm_flavor = 'm1.small' -vm_image = 'cirros-0.3.3' -userdata1 = "#!/bin/sh\n\nsudo ip a add {}/24 dev eth0\n".format(vm1_address) +vm_image = 'cirros-0.3.4' +userdata1 = "#!/bin/sh\n\nsudo ip a add {}/24 dev eth0\n while true; do echo curl_passed | nc -l -p 80; done\n".format(vm1_address) userdata2 = "#!/bin/sh\n\nsudo ip a add {}/24 dev eth0\nwhile true; do\n ping -c 1 {} 2>&1 >/dev/null\n " \ "RES=$?\n if [ \"Z$RES\" = \"Z0\" ] ; then\n echo 'ping PASSED'\n break\n else\n echo " \ - "'ping FAILED'\n fi\n sleep 1\ndone\n".format(vm2_address, vm1_address) + "'ping FAILED'\n fi\n sleep 1\ndone\n\nwhile true; do curl {} --retry-delay 1 -m 1; sleep 3; done\n".format(vm2_address, vm1_address, vm1_address) diff --git a/testing/robot/lib/FDSLibrary.py b/testing/robot/lib/FDSLibrary.py index 0cb43ee..32c18eb 100644 --- a/testing/robot/lib/FDSLibrary.py +++ b/testing/robot/lib/FDSLibrary.py @@ -7,35 +7,58 @@ # http://www.apache.org/licenses/LICENSE-2.0 ############################################################################## +from keystoneauth1 import loading +from keystoneauth1 import session +from glanceclient import client as glance from neutronclient.v2_0 import client as neutron from novaclient import client as nova from novaclient.exceptions import NotFound +from robot.api import logger import time import datetime import os import subprocess + class FDSLibrary(): def __init__(self): + auth_obj = loading.get_plugin_loader('password').load_from_options(auth_url=os.getenv('OS_AUTH_URL'), + username=os.getenv('OS_USERNAME'), + password=os.getenv('OS_PASSWORD'), + project_id=os.getenv('OS_PROJECT_ID')) + logger.debug("Initializing glance client.") + self.glance_client = glance.Client('2', session=session.Session(auth=auth_obj)) + logger.debug("Initializing neutron client.") self.neutron_client = neutron.Client(username=os.getenv('OS_USERNAME'), password=os.getenv('OS_PASSWORD'), tenant_name=os.getenv('OS_TENANT_NAME'), auth_url=os.getenv('OS_AUTH_URL')) - - self.nova_client = nova.Client('2', - os.getenv('OS_USERNAME'), - os.getenv('OS_PASSWORD'), - os.getenv('OS_TENANT_NAME'), - os.getenv('OS_AUTH_URL')) + logger.debug("Initializing nova client.") + self.nova_client = nova.Client('2', session=session.Session(auth=auth_obj)) def check_flavor_exists(self, flavor): flavor_list_names = [x.name for x in self.nova_client.flavors.list()] return flavor in flavor_list_names + def create_flavor(self, name, ram, vcpus="1", disk="0"): + response = self.nova_client.flavors.create(name, ram, vcpus, disk) + return response + def check_image_exists(self, image): - image_list_names = [x.name for x in self.nova_client.images.list()] + image_list_names = [x.name for x in self.glance_client.images.list()] return image in image_list_names + def create_image(self, image_name, file_path, disk="qcow2", + container="bare", public="public", property="hw_mem_page_size=large"): + image = self.glance_client.images.create(name=image_name, + visibility=public, + disk_format=disk, + container_format=container, + property=property) + with open(file_path) as image_data: + self.glance_client.images.upload(image.id, image_data) + return image.id + def create_network(self, name): body = {'network': {'name': name}} response = self.neutron_client.create_network(body=body) @@ -101,11 +124,33 @@ class FDSLibrary(): time.sleep(5) return False - def create_security_group(self): - pass + def create_security_group(self, name): + body = {'security_group': { + 'name': name + }} + response = self.neutron_client.create_security_group(body=body) + return response - def create_security_rule(self): - pass + def create_security_rule(self, sg_id, dir, eth, desc=None, proto=None, port_min=None, port_max=None, r_sg_id=None, r_prefix=None): + body = {'security_group_rule': { + 'security_group_id': sg_id, + 'ethertype': eth, + 'direction': dir + }} + if desc is not None: + body['security_group_rule']['description'] = desc + if proto is not None: + body['security_group_rule']['protocol'] = proto + if port_min is not None: + body['security_group_rule']['port_range_min'] = port_min + if port_max is not None: + body['security_group_rule']['port_range_max'] = port_max + if r_sg_id is not None: + body['security_group_rule']['remote_group_id'] = r_sg_id + if r_prefix is not None: + body['security_group_rule']['remote_ip_prefix'] = r_prefix + response = self.neutron_client.create_security_group_rule(body=body) + return response def poll_server(self, vm_id, status, timeout=300): try: @@ -144,6 +189,14 @@ class FDSLibrary(): response = self.neutron_client.delete_network(net_id) return response + def delete_security_group(self, sg_id): + response = self.neutron_client.delete_security_group(sg_id) + return response + + def delete_security_rule(self, rule_id): + response = self.neutron_client.delete_security_group_rule(rule_id) + return response + def ping_vm(self, ip_address): try: output = subprocess.check_output(['ping', '-c', '4', ip_address]) diff --git a/testing/robot/lib/Keywords.robot b/testing/robot/lib/Keywords.robot new file mode 100644 index 0000000..36136a1 --- /dev/null +++ b/testing/robot/lib/Keywords.robot @@ -0,0 +1,109 @@ +############################################################################## +# Copyright (c) 2016 Juraj Linkes (Cisco) and others. +# +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +*** Settings *** +Library OperatingSystem +Library FDSLibrary.py +Variables ../data/test_data.py + +*** Keywords *** + +Ensure Flavor + ${result} = Check Flavor Exists ${vm_flavor} + Return From Keyword If '${result}' == 'True' + Create Flavor ${vm_flavor} ram=768 + ${result} = Check Flavor Exists ${vm_flavor} + Should be True ${result} + +Ensure Image + ${result} = Check Image Exists ${vm_image} + Return From Keyword If '${result}' == 'True' + Create Image ${vm_image} /home/opnfv/functest/data/cirros-0.3.4-x86_64-disk.img + ${result} = Check Image Exists ${vm_image} + Should be True ${result} + +Create tenant network + &{response} = create network ${network_name} + log many &{response} + Set Suite Variable ${network_id} ${response.network['id']} + log ${network_id} + +Create subnet without dhcp + &{response} = create subnet ${subnet_name} ${network_id} ${subnet_cidr} dhcp=False + log many &{response} + Set Suite Variable ${subnet_id} ${response.subnet['id']} + log ${subnet_id} + +Create subnet with dhcp + &{response} = create subnet ${subnet_name} ${network_id} ${subnet_cidr} dhcp=True + log many &{response} + Set Suite Variable ${subnet_id} ${response.subnet['id']} + log ${subnet_id} + +Create security group no default rules + [Arguments] ${name} + &{response} = create security group ${name} + log many &{response} + : FOR ${rule} IN @{response.security_group['security_group_rules']} + \ log ${rule} + \ log ${rule['id']} + \ delete security rule ${rule['id']} + [Return] ${response.security_group['id']} + +Create security group rules + #def create_security_rule(self, sg_id, dir, eth, desc=None, proto=None, port_min=None, port_max=None, r_sg_id=None, r_prefix=None): + &{response} = create security rule ${sg_client} ingress ipv4 + log many &{response} + &{response} = create security rule ${sg_client} egress' ipv4 + log many &{response} + &{response} = create security rule ${sg_server} egress ipv4 + log many &{response} + &{response} = create security rule ${sg_server} ingress ipv4 icmp + log many &{response} + +Create port with ip + [Arguments] ${port_name} ${ip_address} + &{response} = create port ${port_name} ${network_id} ${subnet_id} ${ip_address} + log many &{response} + log ${response.port['id']} + [Return] ${response.port['id']} + +Create vm + [Arguments] ${vm_name} ${port_ids} ${security_groups}=${None} ${userdata}=${None} + Log Many ${vm_name} ${vm_image} ${vm_flavor} ${port_ids} ${userdata} + ${response} = create server ${vm_name} ${vm_image} ${vm_flavor} ${port_ids} ${security_groups} + ... ${userdata} + log many ${response} + log ${response.id} + [Return] ${response.id} + +Check vm console + [Arguments] ${vm_id} ${string} + ${response} = check server console ${vm_id} ${string} + [Return] ${response} + +Poll vm + [Arguments] ${id} ${state} + poll server ${id} ${state} + +Delete vm + [Arguments] ${id} + ${response} = delete server ${id} + log ${response} + Poll vm ${id} ${None} + +Delete ports + [Arguments] ${id} + ${response} = delete port ${id} + log ${response} + +Delete network + [Arguments] ${id} + ${response} = delete net ${id} + log ${response} diff --git a/testing/robot/sec_groups_and_l2-smoke.robot b/testing/robot/sec_groups_and_l2-smoke.robot new file mode 100644 index 0000000..17c5a42 --- /dev/null +++ b/testing/robot/sec_groups_and_l2-smoke.robot @@ -0,0 +1,97 @@ +############################################################################## +# Copyright (c) 2017 Tomas Cechvala (Cisco) and others. +# +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +*** Settings *** +Library OperatingSystem +Library lib/FDSLibrary.py +Variables data/test_data.py +Resource lib/Keywords.robot +Suite Setup Setup Suite +Suite Teardown Teardown Suite + +*** Variables *** + +*** Test Cases *** + +Create network for VMs + Create tenant network + +Create subnet with dhcp for VMs + Create subnet with dhcp + +Create sec groups + ${result} = Create security group no default rules ${sg_server} + Set Suite Variable ${SEC_GR_SERVER} ${result} + ${result} = Create security group no default rules ${sg_client} + Set Suite Variable ${SEC_GR_CLIENT} ${result} + +Create sec rules + Wait Until Keyword Succeeds 3x 3s create security rule ${SEC_GR_CLIENT} egress ipv4 + Wait Until Keyword Succeeds 3x 3s create security rule ${SEC_GR_CLIENT} ingress ipv4 + Wait Until Keyword Succeeds 3x 3s create security rule ${SEC_GR_SERVER} egress ipv4 + Wait Until Keyword Succeeds 3x 3s create security rule ${SEC_GR_SERVER} ingress ipv4 proto=icmp + +Create port for VM1 + ${result} = Create port with ip ${port1_name} ${vm1_address} + Set Suite Variable ${port1_id} ${result} + +Create port for VM2 + ${result} = Create port with ip ${port2_name} ${vm2_address} + Set Suite Variable ${port2_id} ${result} + +Create VM1 + ${port_ids} = Create List ${port1_id} + ${result} = Create vm ${vm1_name} ${port_ids} userdata=${userdata1} + Set Suite Variable ${vm1_id} ${result} + +Wait for VM1 to be active + Should Be True $vm1_id is not $None + Poll vm ${vm1_id} active + +Create VM2 + ${port_ids} = Create List ${port2_id} + ${result} = Create vm ${vm2_name} ${port_ids} userdata=${userdata2} + Set Suite Variable ${vm2_id} ${result} + +Wait for VM2 to be active + Should Be True $vm2_id is not $None + Poll vm ${vm2_id} active + +Check VM2 userdata + ${result} = Check vm console ${vm2_id} PASSED + Should Be True ${result} + +Modify policy + Wait Until Keyword Succeeds 3x 3s create security rule ${SEC_GR_SERVER} ingress ipv4 proto=tcp port_min=80 port_max=80 + +Check VM2 userdata again + ${result} = Check vm console ${vm2_id} curl_passed + Should Be True ${result} + +*** Keywords *** +Setup Suite + Set Suite Variable ${network_id} ${None} + Set Suite Variable ${subnet_id} ${None} + Set Suite Variable ${port1_id} ${None} + Set Suite Variable ${port2_id} ${None} + Set Suite Variable ${vm1_id} ${None} + Set Suite Variable ${vm2_id} ${None} + Set Suite Variable ${SEC_GR_SERVER} ${None} + Set Suite Variable ${SEC_GR_CLIENT} ${None} + Ensure Image + Ensure Flavor + +Teardown Suite + Run Keyword If $vm1_id is not $None Delete vm ${vm1_id} + Run Keyword If $vm2_id is not $None Delete vm ${vm2_id} + Run Keyword If $port1_id is not $None Delete ports ${port1_id} + Run Keyword If $port2_id is not $None Delete ports ${port2_id} + Run Keyword If $network_id is not $None Delete network ${network_id} + Run Keyword If $SEC_GR_SERVER is not $None delete security group ${SEC_GR_SERVER} + Run Keyword If $SEC_GR_CLIENT is not $None delete security group ${SEC_GR_CLIENT} diff --git a/testing/robot/smoke.robot b/testing/robot/smoke.robot index fddac5d..d6f8fe6 100644 --- a/testing/robot/smoke.robot +++ b/testing/robot/smoke.robot @@ -10,6 +10,7 @@ *** Settings *** Library OperatingSystem Library lib/FDSLibrary.py +Library lib/Keywords.robot Variables data/test_data.py Suite Setup Setup Suite Suite Teardown Teardown Suite @@ -61,12 +62,8 @@ Setup Suite Set Suite Variable ${port2_id} ${None} Set Suite Variable ${vm1_id} ${None} Set Suite Variable ${vm2_id} ${None} - ${result} = Check Flavor Exists ${vm_flavor} - Log ${vm_flavor} - Should be True ${result} - ${result} = Check Image Exists ${vm_image} - Log ${vm_image} - Should be True ${result} + Ensure Image + Ensure Flavor Teardown Suite Run Keyword If $vm1_id is not $None Delete vm ${vm1_id} @@ -74,56 +71,3 @@ Teardown Suite Run Keyword If $port1_id is not $None Delete ports ${port1_id} Run Keyword If $port2_id is not $None Delete ports ${port2_id} Run Keyword If $network_id is not $None Delete network ${network_id} - -Create tenant network - &{response} = create network ${network_name} - log many &{response} - Set Suite Variable ${network_id} ${response.network['id']} - log ${network_id} - -Create subnet without dhcp - &{response} = create subnet ${subnet_name} ${network_id} ${subnet_cidr} dhcp=False - log many &{response} - Set Suite Variable ${subnet_id} ${response.subnet['id']} - log ${subnet_id} - -Create port with ip - [Arguments] ${port_name} ${ip_address} - &{response} = create port ${port_name} ${network_id} ${subnet_id} ${ip_address} - log many &{response} - log ${response.port['id']} - [Return] ${response.port['id']} - -Create vm - [Arguments] ${vm_name} ${port_ids} ${security_groups}=${None} ${userdata}=${None} - Log Many ${vm_name} ${vm_image} ${vm_flavor} ${port_ids} ${userdata} - ${response} = create server ${vm_name} ${vm_image} ${vm_flavor} ${port_ids} ${security_groups} - ... ${userdata} - log many ${response} - log ${response.id} - [Return] ${response.id} - -Check vm console - [Arguments] ${vm_id} ${string} - ${response} = check server console ${vm_id} ${string} - [Return] ${response} - -Poll vm - [Arguments] ${id} ${state} - poll server ${id} ${state} - -Delete vm - [Arguments] ${id} - ${response} = delete server ${id} - log ${response} - Poll vm ${id} ${None} - -Delete ports - [Arguments] ${id} - ${response} = delete port ${id} - log ${response} - -Delete network - [Arguments] ${id} - ${response} = delete net ${id} - log ${response} |
