diff options
| -rw-r--r-- | docs/scenario-lifecycle/create-sdf.png | bin | 0 -> 402973 bytes | |||
| -rw-r--r-- | docs/scenario-lifecycle/creating-scenarios.rst | 60 | ||||
| -rw-r--r-- | docs/scenario-lifecycle/current-status.rst | 51 | ||||
| -rw-r--r-- | docs/scenario-lifecycle/index.rst | 2 | ||||
| -rw-r--r-- | docs/scenario-lifecycle/pdf-and-sdf.png | bin | 0 -> 189523 bytes | |||
| -rw-r--r-- | docs/scenario-lifecycle/scenario-tree-danube.png | bin | 0 -> 80299 bytes | |||
| -rw-r--r-- | docs/scenario-lifecycle/workflows.rst | 70 | ||||
| -rw-r--r-- | scenarios/examples/sdf-fdio-example.yaml | 133 | ||||
| -rw-r--r-- | scenarios/templates/sdf-template.yaml | 294 |
9 files changed, 570 insertions, 40 deletions
diff --git a/docs/scenario-lifecycle/create-sdf.png b/docs/scenario-lifecycle/create-sdf.png Binary files differnew file mode 100644 index 0000000..c8a44ba --- /dev/null +++ b/docs/scenario-lifecycle/create-sdf.png diff --git a/docs/scenario-lifecycle/creating-scenarios.rst b/docs/scenario-lifecycle/creating-scenarios.rst index f445a00..dbff9c1 100644 --- a/docs/scenario-lifecycle/creating-scenarios.rst +++ b/docs/scenario-lifecycle/creating-scenarios.rst @@ -6,40 +6,47 @@ Creating Scenarios -------------------- -Purpose -^^^^^^^^^^^ +General +^^^^^^^^^ A new scenario needs to be created, when a new combination of upstream components or features shall be supported, that cannot be provided with the existing scenarios in parallel to their existing features. Typically new scenarios are created as children of existing scenarios. +They start as specific scenario and as they mature, they either merge back +their features to the parent or promote to a generic scenario. -* In some cases an upstream implementation can be replaced by a different solution. - The most obvious example here is the SDN controller. In the first OPNFV release, - only ODL was supported. Later ONOS and OpenContrail were added, thus creating - new scenarios. +Scenario Owners +^^^^^^^^^^^^^^^^ - In most cases, only one of the SDN controllers is needed, thus OPNFV will support - the different SDN controllers by different scenarios. This support will be long- - term, so there will be multiple generic scenarios for these options. +Each scenario must have an "owner". Scenario owners have the following responsibilities: -* Another usecase is feature incompatibilities. For instance, OVS and FD.io - cannot be combined today. Therefore we need different scenarios for them. - If it is expected that such an incompatibility is not solved for longer time, - there can be even separate generic scenarios for these options. +* The scenario owner is responsible for the contents and usage of the scenario. +* He shall define the contents for the scenario deployment: -The overlap between scenarios should only be allowed where they add components -that cannot be integrated in a single deployment. + * The components and their versions that need to be deployed + * Options for the deployment of such components, e.g. settings, optional features, .. + * Which installers to use + * Deployment options (HA, NOHA, hardware types, ..) -If scenario A completely covers scenario B, support of scenario B will be -only provided as long as isolation of development risks is necessary. -However, there might be cases where somebody wants to use scenario B -still as a parent for specific scenarios. +* He shall define the usage of the scenario in the development process: -This is especially the case for generic scenarios, since they need more CI and testing -resources. Therefore a gating process will be introduced for generic scenarios. + * Initiate integration to CI + * Define which testcases to run + * Applies that the scenario joins a release +* The owner maintains the Scenario Descriptor File (SDF) +* Drives for the scenario be supported by more installers + +The scenario owner of a specific scenario typically comes from the feature project +that develops the features introduced by the scenario. + +The scenario owner of a generic scenario will need to drive more integration tasks than +feature development. Thus he typically will come from a project with a broader scope +than a single feature, e.g. a testing project. +The scenario owner of a generic scenario needs to cover issues of all installers, so +only in exceptional cases he will come from an installer project. Creating Generic Scenarios ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -80,16 +87,17 @@ support might be low priority during a final release preparation, e.g. after a M a release, when he expects the scenario merge to other scenarios, and he expects the features may be made available in generic scenarios. A scenario can join a release at the MS0 after its creation. - It should join a release latest on the next MS0 6 month after its - creation (that is it should skip only one release) and merge to its parent - maximum 2 releases later. - .. Editors note: "2 releases" is rather strict maybe refine? * The PTL should explain the infrastructure requirements and clarify that sufficient resources are available for the scenario. * The PTL shall assign a scenario owner. * The scenario owner shall maintain the scenario descriptor file according to the template. -* The scenario owner shall initiate the scenario be integrated in CI or releases. +* The scenario owner shall drive the necessary discussions with installers and testing + teams to get their support. +* In case the scenario needs new keywords in the SDF, the scenario owner shall discuss + those with the installer teams and CI. +* The scenario owner shall initiate the scenario be integrated in CI and + participate in releases. * When the scenario joins a release this needs to be done in time for the relevant milestones. diff --git a/docs/scenario-lifecycle/current-status.rst b/docs/scenario-lifecycle/current-status.rst index 08a3776..c8da13a 100644 --- a/docs/scenario-lifecycle/current-status.rst +++ b/docs/scenario-lifecycle/current-status.rst @@ -6,7 +6,8 @@ Current Status --------------- -tdb: this chapter will summarize the scenario analysis. +This chapter summarizes the scenario analysis to provide some background. +It also defines the way to introduce the scenario processes. Arno ^^^^^^^^ @@ -18,31 +19,55 @@ that could be deployed in two ways, by the two installers available in Arno. Brahmaputra ^^^^^^^^^^^^^^^^ -tbd +In Brahmaputra, we added options for SDN (ONOS, OCL) and some optional +features (sfc, sdnvpn, kvm, l3 enabled ODL). +Thus we had 9 scenarios, some of them to be deployed with 2 installers, +that planned to participate in the release. Not all of them succeeded. Colorado ^^^^^^^^^^^^ -tbd +In Colorado more components and features were added to a total of 17 +combinations of components and features. Some were supported by one +of the four installers, others by multiple installers. In addition HA +and NOHA options were defined. +This lead to 28 combinations that planned to participate. Danube ^^^^^^^^^^ -tbd: Analysis of the 58 scenarios -The analysis can be found in the slides at -https://wiki.opnfv.org/display/INF/Scenario+Consolidation -and will be explain with some text here. -The text will also use the diagrams from the slides, e.g. -show a scenario tree: +In Danube the number of combinations of components and features increased +to 24, but since installer support increased and more scenarios planned +to provide HA and NOHA options, the number of combinations was 54. -.. figure:: scenario-tree.png +In addition to that some scenarios were defined later in during development +and some scenarios worked on ARM support. -and an idea about possible generic scenarios: +This created the need to better understand relationships and +incompatibilities of the scenarios to drive for a manageable process +for scenarios. -.. figure:: scenario-tree+idea.png +As a result the relationship between the scenarios can be +visualized by a scenario tree. -as well as possible ways to reach this. +.. figure:: scenario-tree-danube.png +The process for generic and specific scenarios is not in place for the +Danube release yet. But the different branches of the scenario tree +provide the candidates to define generic scenario during the timeframe +of the next release. + +Euphrates +^^^^^^^^^^ + +tbd: statistics on Euphrates Scenarios + +During Euphrates timeframe, dynamic POD allocation is introduced in CI. +This is a prerequisite to make use of the SDF in the CI pipeline. +Therefore in this timeframe, scenario processes are introduced only in +a documentation way and as support for release management. + +Also the definition of generic scenarios can be done. diff --git a/docs/scenario-lifecycle/index.rst b/docs/scenario-lifecycle/index.rst index 36dd92a..c1a9a52 100644 --- a/docs/scenario-lifecycle/index.rst +++ b/docs/scenario-lifecycle/index.rst @@ -21,4 +21,4 @@ Contents: mano-scenarios.rst current-status.rst scenario-descriptor-files.rst - + workflows.rst diff --git a/docs/scenario-lifecycle/pdf-and-sdf.png b/docs/scenario-lifecycle/pdf-and-sdf.png Binary files differnew file mode 100644 index 0000000..729c5a4 --- /dev/null +++ b/docs/scenario-lifecycle/pdf-and-sdf.png diff --git a/docs/scenario-lifecycle/scenario-tree-danube.png b/docs/scenario-lifecycle/scenario-tree-danube.png Binary files differnew file mode 100644 index 0000000..54c111e --- /dev/null +++ b/docs/scenario-lifecycle/scenario-tree-danube.png diff --git a/docs/scenario-lifecycle/workflows.rst b/docs/scenario-lifecycle/workflows.rst new file mode 100644 index 0000000..c07b0f7 --- /dev/null +++ b/docs/scenario-lifecycle/workflows.rst @@ -0,0 +1,70 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. (c) 2017 OPNFV Ulrich Kleber (Huawei) + + +Workflows +---------- + +Summary +^^^^^^^^ + +The general principle can be summarized by the following diagram: + +.. figure:: pdf-and-sdf.png + +Workflows for Scenario Owners +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The scenario owner creates the descriptor file based on the template. + +.. figure:: create-sdf.png + +Create new scenario from scratch ++++++++++++++++++++++++++++++++++++++++++++++++++ + +This workflow will be exceptional. +Most scenarios can easier start as children of an existing scenario; +thus the author (scenario owner) can derive the SDF from the parent. +But scenarios introducing new technologies affecting the whole architecture, +e.g.containers, or higher level scenarios (e.g.MANO and Multisite which +reference existing scenarios) can start without a parent. + +The following steps need to be done: + + #. (Project team) Define set of components that need to be deployed + #. (Project) Find installers that can deploy the components + #. (Project&installer&CI) Agree on new keywords in SDF (e.g. component, feature name) + #. (Project) Assign owner + #. (Owner) Edit SDF, submit to octopus repo + #. (Owner) register scenario to participate in release as appropriate + #. (Owner&CI-team) Adapt jenkins triggers, so new scenario can be scheduled in valid installer/POD/Options combination(s). + #. (Installer-team) test deployment of components + #. (Project-team) Define test cases; register in test db + +Create child scenario by adding feature to existing scenario ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +Add additional installer to a specific scenario +++++++++++++++++++++++++++++++++++++++++++++++++ + +Add additional hardware or availability option to a scenario +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +Merge child scenario back to parent +++++++++++++++++++++++++++++++++++++ + +Promote specific scenario to generic scenario +++++++++++++++++++++++++++++++++++++++++++++++ + +Introduce SDF for existing Danube/Euphrates scenarios +++++++++++++++++++++++++++++++++++++++++++++++++++++++ + + +Workflows for Installers +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Workflows for CI Tools +^^^^^^^^^^^^^^^^^^^^^^^^^^ + + diff --git a/scenarios/examples/sdf-fdio-example.yaml b/scenarios/examples/sdf-fdio-example.yaml new file mode 100644 index 0000000..cbac4fb --- /dev/null +++ b/scenarios/examples/sdf-fdio-example.yaml @@ -0,0 +1,133 @@ +############################################################################## +# Copyright (c) 2017 Huawei others. +# ulrich.kleber@huawei.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +############################################################################## +# Description: +# This an example for a specific scenario. +# It is derived from: +# apex/config/deploy/os-odl_l2-fdio-ha +############################################################################## + +############################################################################## +# scenario meta-data +scenario-metadata: + name: odl-fdio-devops + title: fdio odl basic for devops + generic-scenario: false + version: 1.0.0 + creation-date: 2017-03-16 + # This scenario introduces fd.io with odl and a basic feature set. + # It is derived from parent odl_l2 nofeature. In a next step, odl_l2 and + # old_l3 functionality shall be merged and provide sfc as well as other + # features. + # This scenario will use newer versions of ODL and other upstream components + # than used in Euphrates. It is planned to release it or DevOps use more + # often than regular OPNFV release cycle. + opnfv-release: colorado + opnfv-version: 3.1.0 # the first opnfv version, the scenario was introduced + owner: Frank Brockners, frank.brockners@cisco.com + # Add additional contact persons e.g. from installers or major components + +############################################################################## + +############################################################################## +# components +components: + - sdn-controller: + component-type: opendaylight + release: carbon + version: ">6.0.1" + features: + - odl_l2 + - vpp + - storage: + component-type: ceph + #$$$$ Should we add num-replicas 3 here? + + - cloud-controller: + type: openstack + release: ocata + modules: + - nova + - cinder + - dashboard + - glance + - heat + - neutron + - tacker + - congress + - dataplane: + type: fdio + release: xx + version: 9.9.9 + features: + - performance: + controller-nodes: + kernel: + hugepages: 1024 # decimal number + hugepagesz: 2M # values like 2M, 1G + intel_iommu: 'on' + iommu: pt + isolcpus: 1,2 + vpp: + main-core: 1 + corelist-workers: 2 + uio-driver: uio_pci_generic + compute-nodes: + kernel: + hugepagesz: 2M + hugepages: 2048 + intel_iommu: 'on' + iommu: pt + isolcpus: 1,2 + vpp: + main-core: 1 + corelist-workers: 2 + uio-driver: uio_pci_generic +############################################################################## + +############################################################################## +# deployment options +deployment-options: + deployment-types: # only intel baremetal is supported + - baremetal: + architecture: x86_64 + availability: + - ha: # We support only HA + nodes: + - name: host1 + roles: + - openstack-controller # need to add fd.io? + - odl + - name: host2 + roles: + - openstack-controller + - odl + - name: host3 + roles: + - openstack-controller + - odl + - name: host4 # need to add fd.io? + roles: + - openstack-compute + - name: host5 + roles: + - openstack-compute + deployment-tools: + - apex: + cpu: intel + pod: baremetal + availability: ha + # fuel support shall be added soon +############################################################################## + +############################################################################## +# Prerequisites +# No other prerequisites +############################################################################## diff --git a/scenarios/templates/sdf-template.yaml b/scenarios/templates/sdf-template.yaml new file mode 100644 index 0000000..81265e8 --- /dev/null +++ b/scenarios/templates/sdf-template.yaml @@ -0,0 +1,294 @@ +############################################################################## +# Copyright (c) 2017 Huawei and others. +# ulrich.kleber@huawei.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +############################################################################## +# Description: +# This is the template for all scenario descriptor files (sdf) +# Every OPNFV scenario is described in an sdf.yaml located in the +# scenarios folder in Octopus repo. +# The sdf is provided by the scenario owner and consumed by CI, deployment +# tools and test frameworks. +# +# Main sections: +# +# metadata (owner, history, description) +# list of components (names, versions, submodules) +# deployment options (HA/NOHA, hardware&virtualization, installers, configurations) +# other prerequisites (e.g. memory requirement more than pharos spec) +# +# More details can be found in the scenario lifecycle document. +############################################################################## + +############################################################################## +# scenario meta-data # Metadata describing this sdf.yaml file and the scenario + # history and purpose. +scenario-metadata: + name: SDF-Template # mandatory +# This is a free name. +# For Generic scenarios, the main distiguishing components can be included in +# the name. The name will be approved by TSC when the generic scenario is +# established. Examples: OS-ODL-OVSNSH, OS-ONOS, OS-ODL-FDIO, +# OS-OVSBasic, OS-FDIOBasic, ... +# For specific scenarios, the name should characterize the main +# feature that is implemented here. Examples: Bgpvpn, Netvirt-gbp-vpp, +# Dpdk-bar, Onos-sfc, ... +# Final rules for naming will be set by the lifecycle document and TSC approval. + title: "SDF template" # mandatory +# descriptive text title maximum 10-12 words telling the main purpose + generic-scenario: false # optional, default = false + version: 1.0.6 # mandatory +# version number of the sdf, three digits separated with dots + creation-date: 2017-05-09 # mandatory +# creation of this sdf file version + # Please add a clear description of the purpose of the scenario, including + # the main benefits and major features (mandatory). + # If applicable, the parent scenario should be mentioned. + opnfv-release: euphrates # mandatory +# the first opnfv release, the scenario was introduced + opnfv-version: + - begins: 5.1.0 # mandatory +# the first opnfv version, the scenario was released with + - ends: 7.3.0 # optional +# the last opnfv version that supports this scenario. Typically the features +# of the scenario should have been merged to generic scenarios then + owner: Ulrich Kleber, ulrich.kleber@huawei.com # mandatory +# author of this file and thus owner of the scenario +# Add additional contact persons e.g. from installers or major components + +############################################################################## + +############################################################################## +# components +# All components/submodules/features in the list shall be deployed +components: # mandatory section +# In this section all components are listed together with their version. +# For some components in addtion submodules or optional features can be listed. + - sdn-controller: # optional, default = no sdn controller +# most important categories here are: sdn-controller, cloud-controller, +# storage, virtual-switching, dataplane, nfvo, vnfm, but new categories +# can be introduced any time. +# Every component to be deployed should be listed with such a section. +# If the component has submodules or optional features, they also need +# to be listed. + type: opendaylight # mandatory, other options e.g.onos, ocl, ovn + release: boron # either release or version or both must be given +# upstream version, human readable release name + version: 5.0.1 +# exact semantic version including patch level +# Normally installers will not be able to pick exact semantic versions, but +# if the scenario requires specific versions, this can be checked offline. +# Following syntax variants can be allowed as well: +# version: 5.*.* +# version: ">5.0.3" + features: # optional +# additional feature configurations as recognized by the installers. + - odl_l2 + - sfc + - bgpvpn + + - storage: # optional + type: ceph + + - cloud-controller: # seems to me mandatory + type: openstack # other option could be kubernetes + release: ocata # either release or version or both must be given + version: 15.0.0 +# An OPNFV version can go only with one openstack version. Typically installers +# cannot pick different openstack version, but this can be checked offline. + modules: # optional +# Installers have a basic set of modules that are deployed by default. Those +# can be listed optional. Scenario owners can list additional optional modules +# with their name as on https://wiki.openstack.org/software/project-navigator, +# but with lower case and dashes, Examples: tacker, kuryr, horizon, vitrage, +# chef-openstack, openstack-charms, etc. +# Independent of big tent, modules can be used if installers support their deployment. + - nova + - cinder + - horizon + - glance + - heat + - neutron: + features: # In some cases features need to be listed specifically + - bgpvpn # listing service plugins as neutron features makes +# it easier for installers, they don't need to take this information from the features +# field in the sdn-controller. + - aodh + - tacker + - congress + + - virtual-switching: # optional +# showing with this example how to include a separate configuration file for this + type: ovs + release: xx # either release or version or both must be given + version: 2.6.1 + features: # optional + - vsperf: + file: scenarios/ovs-config.yaml +# this file then should contain additional configurations to use like +# hugepage-size, iommu, ... +# it will be treated like a C #include statement. +# Please note, the template cannot show all possible options here. There will be +# more in an additional document. +# Correct usage of the keywords/options will be enforced by a schema. +# Also note that some component related deployment information will be in the options +# section. +############################################################################## + +############################################################################## +# deployment options +# In this section, CI will select one of the listed options and needs to pass +# the information to the installer via a parameter or environment. +deployment-options: # mandatory + deployment-types: # at least one section must be provided + - baremetal: + architecture: x86_64 + features: # optional + - dpdk + - other +#$$$$ Discussion open whether we need features here after adding them in roles section + - baremetal: + architecture: arm64 + - virtual: +#$$$$ Discussion open whether this is necessary here. + architecture: x86_64 + features: + - xyz + +#$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ +# Discussion open how to specify the distribution of components on nodes. +# Three proposals: +# 1. specify availability options ha, noha by placing functions on nodes +# 2. specify roles like compute-node, controller-node and only their number, +# thus avoid coupling with hostnames and more flexible mapping to different +# sizes of PODs. +# 3. Leave it to installers and just specify whether ha or noha are supported + +#$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ +# Proposal 1: + availability: # mandatory +# here the configuration for a HA and NONHA deployment is described. +# It is similar to what compass has in host section (minus the POD info), +# or fuel in the dea-override-config or dha-override-config + - ha: # minimum one of ha or noha must be specified + nodes: # a description like this is mandatory + - name: [host1, host2, host3] # avoid to list the same multiple times + roles: + - openstack-controller # took this from compass. Is it sufficient? + - odl + - ceph-adm + - ceph-mon + - name: [host4, host 5] + roles: + - openstack-compute + - ceph-osd + - noha: + hosts: + - name: host1 + roles: + - openstack-controller + - odl + - ceph-adm + - ceph-mon + - name: host2 + roles: + - openstack-compute + - ceph-osd + - name: host3 + roles: + - openstack-compute + - ceph-osd + +#$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ +# Proposal 2: + roles: # mandatory + - controller-node: + components: # list all components that are deployed here. + - openstack: control + - opendaylight + - ceph: [ceph-adm, ceph-mon] + - ovs + - compute-node: + components: + - openstack: compute + - ceph: ceph-osd + - ovs + hardware-features: + - dpdk + - jump-host: # some scenarios, e.g. MANO might deploy components here + + role-distribution: # mandatory + - ha: + controller-node: 3 + compute-node: 2 + jump-host: 1 + - noha: + controller-node: 1 + compute-node: 4 + jump-host: 1 + +#$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ +# Proposal 3: +# no specification of nodes/roles here. ha, noha are defined by installers + + deployment-tools: # mandatory +# In the section for each deployment tool, the combinations of the +# first three options have to be listed. CI can pick any of the sections. + - fuel: # at least one section + cpu: intel # optional, default = intel + pod: baremetal + availability: ha + - fuel: + cpu: intel + pod: virtual + availability: ha + - fuel: + cpu: intel + pod: virtual + availability: noha + - fuel: + cpu: arm + pod: baremetal + availability: noha + - compass: + cpu: intel + pod: baremetal + availability: ha + - joid: + cpu: intel + pod: baremetal + availability: ha + +############################################################################## + +############################################################################## +# Prerequisites +# This section will list additional prerequisites. Currently there is only +# one case where a scenario has additional prerequisites to the Pharos spec. +# Open-O deployment requires 64GB of memory while Pharos spec requires 32GB. +# In general it should be preferred to issue such requirements to pharos +# using the pharos change request process, but in some cases in might be +# better to specify additional prerequisites. +# Another use case for these prerequisites will be usage of specilized +# hardware, e.g. for acceleration. This needs further study. + +prerequisites: # The section can be empty or omitted. + - controller-node: # Prerequisites might be different + RAM: 128GB # optional, just to give examples + cpu: dual-core + features: # optional, see example below + - compute-node: + RAM: 128GB + cpu: dual-core + features: + - dpdk + - jumphost: # Prerequisites can be given also for jumphost + RAM: 128GB + cpu: dual-core + +############################################################################## |
