summaryrefslogtreecommitdiffstats
path: root/requirements/resource_management
diff options
context:
space:
mode:
Diffstat (limited to 'requirements/resource_management')
-rw-r--r--requirements/resource_management/01-intro.rst37
-rw-r--r--requirements/resource_management/02-usecase.rst97
-rwxr-xr-xrequirements/resource_management/03-arch.rst246
-rw-r--r--requirements/resource_management/04-gap.rst99
-rwxr-xr-xrequirements/resource_management/05-impl.rst534
-rw-r--r--requirements/resource_management/06-summary.rst24
-rw-r--r--requirements/resource_management/07-schemas.rst195
-rw-r--r--requirements/resource_management/Makefile18
-rwxr-xr-xrequirements/resource_management/conf.py21
-rw-r--r--requirements/resource_management/glossary.rst63
-rwxr-xr-xrequirements/resource_management/images/figure1.pngbin0 -> 60391 bytes
-rwxr-xr-xrequirements/resource_management/images/figure2.pngbin0 -> 149390 bytes
-rwxr-xr-xrequirements/resource_management/images/figure3.pngbin0 -> 38065 bytes
-rwxr-xr-xrequirements/resource_management/images/figure4.pngbin0 -> 218419 bytes
-rwxr-xr-xrequirements/resource_management/images/figure5.pngbin0 -> 38402 bytes
-rwxr-xr-xrequirements/resource_management/images/figure6.pngbin0 -> 401171 bytes
-rw-r--r--requirements/resource_management/index.rst73
17 files changed, 1407 insertions, 0 deletions
diff --git a/requirements/resource_management/01-intro.rst b/requirements/resource_management/01-intro.rst
new file mode 100644
index 0000000..fdceaf1
--- /dev/null
+++ b/requirements/resource_management/01-intro.rst
@@ -0,0 +1,37 @@
+
+============
+Introduction
+============
+
+Resource reservation is a basic function for the operation of a virtualized
+telecom network. In resource reservation, VIM reserves resources for a certain
+period as requested by the NFVO. A resource reservation will have a start time
+which could be into the future. Therefore, the reserved resources shall be
+available for the NFVO requested purpose (e.g. for a VNF) at the start time for
+the duration asked by NFVO. Resources include all three resource types in an
+NFVI i.e. compute, storage and network.
+
+Besides, NFVO requires abstracted NFVI resource capacity information in order
+to take decisions on VNF placement and other operations related to the virtual
+resources. VIM is required to inform the NFVO of NFVI resource state
+information for this purpose. Promise project aims at delivering the detailed
+requirements on these two features defined in ETSI NFV MAN GS [5], the list of
+gaps in upstream projects, potential implementation architecture and plan, and
+the VIM northbound interface specification for resource reservation and
+capacity management.
+
+Problem description
+===================
+
+OpenStack, a prominent candidate for the VIM, cannot reserve resources for
+future use. OpenStack requires immediate instantiation of Virtual Machines
+(VMs) in order to occupy resources intended to be reserved. Blazar can reserve
+compute resources for future by keeping the VMs in shelved mode. However, such
+reserved resources can also be used for scaling out rather than new VM
+instantiation. Blazar does not support network and storage resource reservation
+yet.
+
+Besides, OpenStack does not provide a northbound interface through which it can
+notify an upper layer management entity e.g. NFVO about capacity changes in its
+NFVI, periodically or in an event driven way. Capacity management is a feature
+defined in ETSI NFV MAN GS [5] and is required in network operation.
diff --git a/requirements/resource_management/02-usecase.rst b/requirements/resource_management/02-usecase.rst
new file mode 100644
index 0000000..b1a1cfa
--- /dev/null
+++ b/requirements/resource_management/02-usecase.rst
@@ -0,0 +1,97 @@
+=======================
+Use cases and scenarios
+=======================
+
+Resource reservation is a basic feature in any virtualization-based network
+operation. In order to perform such resource reservation from NFVO to VIM, NFVI
+capacity information is also necessary at the NFVO side. Below, four use cases
+to show typical requirements and solutions for capacity management and resource
+reservation is presented.
+
+#. Resource capacity management
+#. Resource reservation for immediate use
+#. Resource reservation for future use
+#. Co-existence of reservations and allocation requests without reservation
+
+Resource capacity management
+============================
+
+NFVO takes the first decision on in which NFVI it would instantiate a VNF.
+Along with NFVIs resource attributes (e.g. availability of hardware
+accelerators, particular CPU architectures etc.), NFVO needs to know available
+capacity of an NFVI in order to make an informed decision on selecting
+a particular NFVI. Such capacity information shall be in a coarser granularity
+than the respective VIM, as VIM maintains capacity information of its NFVI
+in fine details. However a very coarse granularity, like simply the number of
+available virtual CPU cores, may not be sufficient. In order to allow the NFVO
+to make well founded allocation decisions, an appropriate level to expose the
+available capacity may be per flavor. Capacity information may be required for
+the complete NFVI, or per partition or availability zone, or other
+granularities. Therefore, VIM requires to inform the NFVO about available
+capacity information regarding its NFVI at a pre-determined abstraction, either
+by a query-response, or in an event-based, or in a periodical way.
+
+Resource reservation for immediate use
+======================================
+
+Reservation is inherently for the future. Even if some reserved resources are
+to be consumed instantly, there is a network latency between the issuance of a
+resource reservation request from the NFVO, a response from the VIM, and actual
+allocation of the requested resources to a VNF/VNFM. Within such latency,
+resource capacity in the NFVI in question could change, e.g., due to failure,
+allocation to a different request. Therefore, the response from a VIM to the
+NFVO to a resource reservation request for immediate use should have a validity
+period which shows until when this VIM can hold the requested resources. During
+this time, the NFVO should proceed to allocation if it wishes to consume the
+reserved requested. If allocation is not performed within the validity period,
+the response from VIM for a particular resource reservation request becomes
+invalid and VIM is not liable to provide those resources to NFVO/VNFM anymore.
+Reservations requests for immediate use do not have a start time but may have
+an end time.
+
+Resource reservation for future use
+===================================
+
+Network operators may want to reserve extra resources for future use. Such
+necessity could arise from predicted congestion in telecom nodes e.g. due to
+local traffic spikes for concerts, natural disasters etc. In such a case, the
+NFVO, while sending a resource reservation request to the VIM, shall include a
+start time (and an end time if necessary). The start time indicates at what
+time the reserved resource shall be available to a designated consumer e.g. a
+VNF/VNFM. Here, the requirement is that the reserved resources shall be
+available when the start time arrives. After the start time has arrived, the
+reserved resources are allocated to the designated consumer(s). An explicit
+allocation request is needed. How actually these requested resources are held
+by the VIM for the period in between the arrival of the resource reservation
+request and the actual allocation is outside the scope of this requirement
+project.
+
+Co-existence of reservations and allocation requests without reservation
+========================================================================
+
+In a real environment VIM will have to handle allocation requests without any
+time reference, i.e. time-unbound, together with time-bound reservations and
+allocation requests with an explicitly indicated end-time. A granted
+reservation for the future will effectively reduce the available capacity for
+any new time-unbound allocation request. The consequence is that reservations,
+even those far in the future, may result in denial of service for new
+allocation requests.
+
+To alleviate this problem several approaches can be taken. They imply an
+implicit or explicit priority scheme:
+
+* Allocation requests without reservation and which are time-unbound will be
+ granted resources in a best-effort way: if there is instant capacity, but the
+ resources may be later withdrawn due to the start time of a previously
+ granted reservation
+* Both allocation requests and reservation requests contain a priority which
+ may be related to SLAs and contractual conditions between the tenant and the
+ NFVI provider. Interactions may look like:
+
+ * A reservation request for future use may cancel another, not yet started,
+ reservation with lower priority
+ * An allocation request without reservations and time-unbound may be granted
+ resources and prevent a future reservation with lower priority from getting
+ resources at start time
+ * A reservation request may result in terminating resources allocated to a
+ request with no reservation, if the latter has lower priority
diff --git a/requirements/resource_management/03-arch.rst b/requirements/resource_management/03-arch.rst
new file mode 100755
index 0000000..23bbac7
--- /dev/null
+++ b/requirements/resource_management/03-arch.rst
@@ -0,0 +1,246 @@
+============================================
+High level architecture and general features
+============================================
+
+Architecture Overview
+=====================
+
+.. figure:: images/figure1.png
+ :width: 90%
+
+ Resource Reservation Architecture
+
+Figure 1 shows the high level architecture for the resource reservation use
+cases. Reserved resources are guaranteed for a given user/client for the period
+expressed by start and end time. User/client represents the requestor and the
+consequent consumer of the reserved resources and correspond to the NFVO or
+VNFM in ETSI NFV terminology.
+
+Note: in this document only reservation requests from NFVO are considered.
+
+General Features
+================
+
+This section provides a list of features that need to be developed in the
+Promise project.
+
+* Resource capacity management
+
+ * Discovery of available resource capacity in resource providers
+ * Monitoring of available resource capacity in resource providers
+ * Update available resource capacity as a result of new or expired
+ reservations, addition/removal of resources. Note: this is a VIM internal
+ function, not an operation in the VIM northbound interface.
+
+* Resource reservation
+
+ * Set start time and end time for allocation
+ * Increase/decrease reserved resource's capacity
+ * Update resource reservations, e.g. add/remove reserved resources
+ * Terminate an allocated resource due to the end time of a reservation
+
+* VIM northbound interfaces
+
+ * Receive/Reply resource reservation requests
+ * Receive/Reply resource capacity management requests
+ * Receive/Reply resource allocation requests for reserved resources when
+ start time arrives
+ * Subscribe/Notify resource reservation event
+
+ * Notify reservation error or process completion prior to reservation start
+ * Notify remaining time until termination of a resource due to the end time
+ of a reservation
+ * Notify termination of a resource due to the end time of a reservation
+
+ * Receive/Reply queries on available resource capacity
+ * Subscribe/Notify changes in available resource capacity
+
+High level northbound interface specification
+=============================================
+
+Resource Capacity Management
+----------------------------
+
+.. figure:: images/figure2.png
+ :width: 90%
+
+ Resource capacity management message flow: notification of capacity change
+
+Figure 2 shows a high level flow for a use case of resource capacity
+management. In this example, the VIM notifies the NFVO of capacity change after
+having received an event regarding a change in capacity (e.g. a fault
+notification) from the NFVI. The NFVO can also retrieve detailed capacity
+information using the Query Capacity Request interface operation.
+
+.. figure:: images/figure3.png
+ :width: 90%
+
+ Resource capacity management message flow: query of capacity density
+
+Figure 3 shows a high level flow for another use case of resource capacity
+management. In this example, the NFVO queries the VIM about the actual capacity
+to instantiate a certain resource according to a certain template, for example
+a VM according to a certain flavor. In this case the VIM responds with the
+number of VMs that could be instantiated according to that flavor with the
+currently available capacity.
+
+Resource Reservation
+--------------------
+
+.. figure:: images/figure4.png
+ :width: 90%
+
+ Resource reservation flow
+
+Figure 4 shows a high level flow for a use case of resource reservation. The
+main steps are:
+
+* The NFVO sends a resource reservation request to the VIM using the Create
+ Resource Reservation Request interface operation.
+* The NFVO gets a reservation identifier reservation associated with this
+ request in the reply message
+* Using the reservation identifier reservation, the NFVO can
+ query/update/terminate a resource reservation using the corresponding
+ interface operations
+* The NFVO is notified that the resource reservation is terminated due to the
+ end time of the reservation
+
+
+Information elements
+====================
+
+Resource Capacity Management
+----------------------------
+
+Notify Capacity Change Event
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The notification change message shall include the following information
+elements:
+
+============================ ========== =====================================
+Name Type Description
+============================ ========== =====================================
+Notification Identifier Identifier issued by the VIM for the
+ capacity change event notification
+Zone Identifier Identifier of the zone where capacity
+ has changed
+Used/Reserved/Total Capacity List Used, reserved and total capacity
+ information regarding the resource
+ items subscribed for notification for
+ which capacity change event occurred
+============================ ========== =====================================
+
+Query Resource Capacity Request
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The capacity management query request message shall include the following
+information elements:
+
+========== ========== ======================================================
+Name Type Description
+========== ========== ======================================================
+Zone Identifier Identifier of the zone where capacity is requested
+Attributes List Attributes of resource items to be notified regarding
+ capacity change events
+Resources List Identifiers of existing resource items to be queried
+ regarding capacity info (such as images, flavors,
+ virtual containers, networks, physical machines, etc.)
+========== ========== ======================================================
+
+The capacity management query request message may also include the following
+information element:
+
+====== ========== ==========================================================
+Name Type Description
+====== ========== ==========================================================
+Flavor Identifier Identifier that is passed in the request to obtain
+ information of the number of virtual resources that can be
+ instantiated according to this flavour with the available
+ capacity
+====== ========== ==========================================================
+
+Query Resource Capacity Reply
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The capacity management query reply message shall include the following
+information elements:
+
+============================ ========== =====================================
+Name Type Description
+============================ ========== =====================================
+Zone Identifier Identifier of the zone where capacity
+ is requested
+Used/Reserved/Total Capacity List Used, reserved and total capacity
+ information regarding each of the
+ resource items requested to check for
+ capacity
+============================ ========== =====================================
+
+The detailed specification of the northbound interface for Capacity Management
+in provided in section 5.1.1.
+
+Resource Reservation
+--------------------
+
+Create Resource Reservation Request
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The create resource reservation request message shall include the following
+information elements:
+
+========== ========== =========================================================
+Name Type Description
+========== ========== =========================================================
+Start Timestamp Start time for consumption of the reserved resources
+End Timestamp End time for consumption of the reserved resources
+Expiry Timestamp If not all reserved resources are allocated between start
+ time and expiry, the VIM shall release the corresponding
+ resources
+Amount Number Amount of the resources per resource item type (i.e.
+ compute/network/storage) that need to be reserved
+Zone Identifier The zone where the resources need(s) to be reserved
+Attributes List Attributes of the resources to be reserved such as DPDK
+ support, hypervisor, network link bandwidth, affinity
+ rules, etc.
+Resources List Identifiers of existing resource items to be reserved
+ (such as images, flavors, virtual containers, networks,
+ physical machines, etc.)
+========== ========== =========================================================
+
+Create Resource Reservation Reply
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The create resource reservation reply message shall include the following
+information elements:
+
+=========== ========== =======================================================
+Name Type Description
+=========== ========== =======================================================
+Reservation Identifier Identification of the reservation instance. It can be
+ used by a consumer to modify the reservation later, and
+ to request the allocation of the reserved resources.
+Message Text Output message that provides additional information
+ about the create resource reservation request (e.g. may
+ be a simple ACK if the request is being background
+ processed by the VIM)
+=========== ========== =======================================================
+
+Notify Reservation Event
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+The notification reservation event message shall include the following
+information elements:
+
+============ ========== =====================================================
+Name Type Description
+============ ========== =====================================================
+Reservation Identifier Identification of the reservation instance triggering
+ the event
+Notification Identifier Identification of the resource event notification
+ issued by the VIM
+Message Text Message describing the event
+============ ========== =====================================================
+
+The detailed specification of the northbound interface for Resource Reservation
+is provided in section 5.1.2.
diff --git a/requirements/resource_management/04-gap.rst b/requirements/resource_management/04-gap.rst
new file mode 100644
index 0000000..908f27f
--- /dev/null
+++ b/requirements/resource_management/04-gap.rst
@@ -0,0 +1,99 @@
+=================================
+Gap analysis in upstream projects
+=================================
+
+This section provides a list of gaps in upstream projects for realizing
+resource reservation and management. The gap analysis work focuses on the
+current OpenStack Blazar project [2]_ in this first release.
+
+OpenStack Blazar
+================
+
+Resource reservation for future use
+-----------------------------------
+
+* Category: Blazar
+* Type: 'missing' (lack of functionality)
+* Description:
+
+ * To-be: To reserve a whole set of compute/storage/network resources in the
+ future
+ * As-is: Blazar currently can do only compute resource reservation by using
+ "Shelved VM"
+
+* Related blueprints:
+
+ * https://blueprints.launchpad.net/blazar/+spec/basic-volume-plugin
+ * https://blueprints.launchpad.net/blazar/+spec/basic-network-plugin
+ * It was planned in Blazar to implement volume and network/fixed ip
+ reservations
+
+Resource reservation update
+---------------------------
+
+* Category: Blazar
+* Type: 'missing' (lack of functionality)
+* Description:
+
+ * To-be: Have the possibility of adding/removing resources to an existing
+ reservation, e..g in case of NFVI failure
+ * As-is: Currently in Blazar, a reservation can only be modified in terms of
+ start/end time
+
+* Related blueprints: N/A
+
+Give me an offer
+----------------
+
+* Category: Blazar
+* Type: 'missing' (lack of functionality)
+* Description:
+
+ * To-be: To have the possibility of giving a quotation to a requesting user
+ and an expiration time. Reserved resources shall be released if they are
+ not claimed before this expiration time.
+ * As-is: Blazar can already send notification e.g. to inform a given user
+ that a reservation is about to expire
+
+* Related blueprints: N/A
+
+StormStack StormForge
+---------------------
+
+Stormify
+^^^^^^^^
+* Stormify enables rapid web applications construction
+* Based on Emberjs style Data stores
+* Developed on Nodejs using coffeescript/javascript
+* Auto RESTful API generation based on Data Models
+* Development starts with defining Data Models
+* Code hosted at github : http://github.com/stormstack/stormify
+
+StormForge
+^^^^^^^^^^
+* Data Model driven management of Resource Providers
+* Based on Stormify Framework and implemented as per the OPNFV Promise
+ requirements
+* Data Models are auto generated and RESTFul API code from YANG schema
+* Currently planned key services include Resource Capacity Management Service
+ and Resource Reservation Service
+* List of YANG schemas for Promise project is attached in the Appendix
+* Code hosted at github: http://github.com/stormstack/stormforge
+
+Resource Discovery
+^^^^^^^^^^^^^^^^^^
+* Category: StormForge
+* Type: 'planning' (lack of functionality)
+* Description
+
+ * To-be: To be able to discover resources in real time from OpenStack
+ components. Planning to add OpenStack Project to interface with promise for
+ real time updates on capacity or any failures
+ * As-is: Currently, resource capacity is learnt using NB APIs related to
+ quota
+
+* Related Blueprints: N/A
+
+
+.. [2] OpenStack Blazar Project, [Online]. Available at
+ https://wiki.openstack.org/wiki/Blazar
diff --git a/requirements/resource_management/05-impl.rst b/requirements/resource_management/05-impl.rst
new file mode 100755
index 0000000..7bbaa0c
--- /dev/null
+++ b/requirements/resource_management/05-impl.rst
@@ -0,0 +1,534 @@
+Detailed architecture and message flows
+=======================================
+
+Detailed northbound interface specification
+-------------------------------------------
+
+.. Note::
+ Once the output of the work from ETSI NFV IFA has been made publicly
+ available, the UML diagrams and REST/JSON examples in this section will be
+ extended
+
+Resource Capacity Management
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Subscribe Capacity Change Event
+_______________________________
+
+**SubscribeRequest (Consumer -> VIM)**
+
+.. uml::
+
+ @startuml
+ class SubscribeRequest {
+ + zone [0..N]: Identifier
+ + attributes [0..1]: String
+ + resourceItems [0..1]: String
+ + thresholds [0..N]: String
+ + notificationId [0..1]: Identifier
+ }
+ @enduml
+
+Subscription from Consumer to VIM to be notified about capacity changes.
+Input Parameters:
+
+* Zone [0..N]: Identification of the zone(s) to notify regarding capacity
+ change events
+* Attributes [0..1]: Attributes of resource items to be notified regarding
+ capacity change events
+* ResourceItems [0..1]: Identifiers of existing resource items to be notified
+ regarding capacity change events (such as images, flavors, virtual
+ containers, networks, physical machines, etc.)
+* Thresholds [0..N]: Lower/Upper limits for triggering change event for
+ used/reserved/total capacity change for specified resource items
+* NotificationId [0..1]: Identification of existing capacity change event
+ notification issued by the VIM. When specified. The previously defined
+ conditions for change event notifications wil be re-used and notification
+ sent to the additional requestor.
+
+Application/json::
+
+ {
+ "zone": ["opnfv-JP8", "opnfv-JP9"],
+ "resourceitems": "numvcinstances"
+ }
+
+**SubscribeReply (VIM -> Consumer)**
+
+.. uml::
+
+ @startuml
+ class SubscribeReply {
+ + subscriptionId [1]: Identifier
+ + created [1]: DateTime
+ + message [0..1]: String
+ }
+ @enduml
+
+Reply Parameters:
+
+* subscriptionId (Identifier): Identification of the capacity change event
+ notification issued by the VIM.
+* created (DateTime): Timestamp when subscription has been created
+* m essage [0..1] (String): Output message that provides additional information
+ about the subscribe request
+
+Application/json::
+
+ {
+ "created": "2015-03-23T00:00:01Z",
+ "notificationId": "abcdef-ghijkl-123456789"
+ }
+
+Query Resource Capacity
+_______________________
+
+**QueryRequest (NFVO -> VIM)**
+
+.. uml::
+
+ @startuml
+ class QueryCapacityRequest {
+ + capacityQueryFilter [0..1]: CapacityQueryFilterClass
+ }
+
+ class CapacityQueryFilter {
+ + zone [0..1]: Identifier
+ + resourceItems [0..1]: String
+ + flavorID [0..1]: Identifier
+ + timePeriod [0..1]: DateTime
+ }
+
+ QueryCapacityRequest "1" *- "0..1" CapacityQueryFilter : ""
+ @enduml
+
+.. -*
+
+Request to find out about used, reserved and total capacity.
+A CapacityQueryFilter can be used to narrow down the capacity details returned
+in the response message.
+
+Input Parameters:
+
+* capacityQueryFilter (CapacityQueryFilterClass): Optional information to
+ narrow down the QueryCapacityRequest, for example to limit the query to given
+ resource items, or a given resource zone. The capacityQueryFilter can also
+ include a FlavorId or template ID. In this case, the QueryCapacity is a
+ request to obtain information of the number of virtual resources that can be
+ instantiated according to this flavor with the actual available capacity.
+ A timePeriod can be specified to narrow down the query to a certain period of time.
+
+Application/json::
+
+ {
+ "capacityqueryfilter": {
+ "resourceitems": "numvcinstances,virtualmemorysize",
+ "zone": "opnfv-JP7"
+ }
+ }
+
+**QueryReply (VIM -> NFVO)**
+
+.. uml::
+
+ @startuml
+ class QueryCapacityReply {
+ + capacityInformation [0..N]: CapacityInformationClass
+ + zone [0..1]: Identifier
+ + lastUpdate [0..1]: DateTime
+ + message [0..1]: String
+ }
+
+ QueryCapacityReply "1" *- "0..N" CapacityInformationClass : ""
+ @enduml
+
+.. -*
+
+Reply Parameters:
+
+* capacityInformation [0..N] (CapacityInformationClass): Capacity information
+ matching the CapacityQueryFilter specified in the QueryCapacityRequest
+* Zone [0..1] (Identifier): Identification of the resource zone
+* lastUpdate [0..1] (DateTime): Timestamp of the capacity last update
+* message [0..1] (String): Output message that provides additional information
+ about the query capacity request
+
+Application/json::
+
+ {
+ "capacityInformation": {
+ "numvcinstances": {
+ "used": 5,
+ "reserved": 1,
+ "total": 10
+ },
+ "virtualmemorysize": {
+ "used": 4,
+ "reserved": 6,
+ "total": 16
+ }
+ },
+ "zone":"opnfv-JP7",
+ "lastUpdate":"2015-03-23T00:00:00Z"
+ }
+
+Notify Capacity Change Event
+____________________________
+
+**CapacityChangeNotification (VIM -> Consumer)**
+
+.. uml::
+
+ @startuml
+ class CapacityChangeNotification {
+ + capacityInformation [0..1]: CapacityInformationClass
+ + zone [0..1]: Identifier
+ + notificationTime [1]: DateTime
+ + notificationId [1]: Identifier
+ }
+
+ CapacityChangeNotification "1" *- "0..1" CapacityInformationClass : ""
+ @enduml
+
+.. -*
+
+Notification about capacity changes
+
+Notify Parameters:
+
+* capacityInformation [0..1] (CapacityInformationClass): Capacity information
+ matching a given subscription request defined by the Consumer
+* zone [0..1] (Identifier): Identification of the resource zone
+* notificationTime [1] (DateTime): Timestamp when the capacity change is
+ detected
+* notificationId [1]: Identification of the capacity change event notification
+ issued by the VIM.
+
+Application/json::
+
+ {
+ "capacity": {
+ "numvcinstances": {
+ "used": 16,
+ "reserved": 2,
+ "total": 20
+ }
+ },
+ "zone": "opnfv-JP8",
+ "notificationTime":"2015-03-23T12:00:05Z",
+ "notificationId":"abcdef-ghijkl-123456789"
+ }
+
+Resource Reservation
+^^^^^^^^^^^^^^^^^^^^
+
+Create Resource Reservation
+___________________________
+
+**CreateResourceReservationRequest (NFVO -> VIM)**
+
+.. uml::
+
+ @startuml
+ class CreateResourceReservationRequest {
+ + start [0..1]: DateTime
+ + end [0..1]: DateTime
+ + expiry [0..1]: DateTime
+ + virtualizationContainerReservation [0..N]: VirtualizationContainerReservationClass
+ + computePoolReservation [0..1]: ComputePoolReservationClass
+ + storagePoolReservation [0..1]: StoragePoolReservationClass
+ + networkReservation [0..1]: NetworkReservationClass
+ + zone [0..1]: Identifier
+ }
+
+ class VirtualizationContainerReservationClass {
+ + containerId [1]: Identifier
+ + flavor [0..1]: FlavorClass
+ }
+
+ CreateResourceReservationRequest "1" *- "0..N" VirtualizationContainerReservationClass : ""
+ VirtualizationContainerReservationClass "1" *-- "0..1" FlavorClass
+ CreateResourceReservationRequest "1" *-- "0..1" ComputePoolReservationClass
+ CreateResourceReservationRequest "1" *-- "0..1" StoragePoolReservationClass
+ CreateResourceReservationRequest "1" *-- "0..1" NetworkReservationClass
+ @enduml
+
+.. -*
+
+**CreateResourceReservationReply (VIM -> NFVO)**
+
+.. uml::
+
+ @startuml
+ class CreateResourceReservationReply {
+ + reservationId [1]: Identifier
+ + virtualizationContainerReserved [0..N]: VirtualizationContainerReservedClass
+ + computePoolReserved [0..1]: ComputePoolReservedClass
+ + storagePoolReserved [0..1]: StoragePoolReservedClass
+ + networkReserved [0..1]: NetworkReservedClass
+ + reservationStatus [1]: String
+ + startTime [0..1]: Time
+ + endTime [0..1]: Time
+ + message [0..1]: String
+ }
+
+ class VirtualizationContainerReservedClass {
+ + containerId [1]: Identifier
+ + flavor [0..1]: FlavorClass
+ }
+
+ CreateResourceReservationReply "1" *- "0..N" VirtualizationContainerReservedClass : ""
+ VirtualizationContainerReservedClass "1" *-- "0..1" FlavorClass
+ CreateResourceReservationReply "1" *-- "0..1" ComputePoolReservedClass
+ CreateResourceReservationReply "1" *-- "0..1" StoragePoolReservedClass
+ CreateResourceReservationReply "1" *-- "0..1" NetworkReservedClass
+ @enduml
+
+.. -*
+
+Subscribe / Notify Reservation Event
+____________________________________
+
+**SubscribeRequest (Consumer -> VIM)**
+
+.. uml::
+
+ @startuml
+ class SubscribeRequest {
+ + reservationId [1]: Identifier
+ + eventType [0..1]: String
+ }
+ @enduml
+
+**SubscribeReply (VIM -> Consumer)**
+
+.. uml::
+
+ @startuml
+ class SubscribeReply {
+ + notificationId [1]: Identifier
+ + created [1]: DateTime
+ + message [0..1]: String
+ }
+ @enduml
+
+**NotifyReservationEvent (VIM -> Consumer)**
+
+.. uml::
+
+ @startuml
+ class ReservationEventNotification {
+ + notificationId [1]: Identifier
+ + reservationId [1]: Identifier
+ + notificationTime [1]: DateTime
+ + vimId [1]: Identifier
+ + eventType [1]: String
+ + eventDetails [1]: String
+ + message [0..1]: String
+ }
+ @enduml
+
+Query Resource Reservation
+__________________________
+
+**QueryResourceReservationRequest (Consumer -> VIM)**
+
+.. uml::
+
+ @startuml
+ class QueryResourceReservationRequest {
+ + reservationQueryFilter [0..1]: ReservationQueryFilterClass
+ }
+
+ QueryResourceReservationRequest "1" *- "0..1" ReservationQueryFilterClass : ""
+ @enduml
+
+.. -*
+
+**QueryResourceReservationReply (VIM -> Consumer)**
+
+.. uml::
+
+ @startuml
+ class CreateResourceReservationReply {
+ + reservationId [1]: Identifier
+ + virtualizationContainerReserved [0..N]: VirtualizationContainerReservedClass
+ + computePoolReserved [0..1]: ComputePoolReservedClass
+ + storagePoolReserved [0..1]: StoragePoolReservedClass
+ + networkReserved [0..1]: NetworkReservedClass
+ + reservationStatus [1]: String
+ + message [0..1]: String
+ }
+
+ class VirtualizationContainerReservedClass {
+ + containerId [1]: Identifier
+ + flavor [0..1]: FlavorClass
+ }
+
+ CreateResourceReservationReply "1" *- "0..N" VirtualizationContainerReservedClass : ""
+ VirtualizationContainerReservedClass "1" *-- "0..1" FlavorClass
+ CreateResourceReservationReply "1" *-- "0..1" ComputePoolReservedClass
+ CreateResourceReservationReply "1" *-- "0..1" StoragePoolReservedClass
+ CreateResourceReservationReply "1" *-- "0..1" NetworkReservedClass
+ @enduml
+
+.. -*
+
+Update Resource Reservation
+___________________________
+
+**UpdateResourceReservationRequest (NFVO ->VIM)**
+
+.. uml::
+
+ @startuml
+ class UpdateResourceReservationRequest {
+ + reservationId [1]: Identifier
+ + start [0..1]: DateTime
+ + end [0..1]: DateTime
+ + expiry [0..1]: DateTime
+ + virtualizationContainerReservation [0..N]: VirtualizationContainerReservationClass
+ + computePoolReservation [0..1]: ComputePoolReservationClass
+ + storagePoolReservation [0..1]: StoragePoolReservationClass
+ + networkReservation [0..1]: NetworkReservationClass
+ + zone [0..1]: Identifier
+ }
+
+ class VirtualizationContainerReservationClass {
+ + containerId [1]: Identifier
+ + flavor [0..1]: FlavorClass
+ }
+
+ UpdateResourceReservationRequest "1" *- "0..N" VirtualizationContainerReservationClass : ""
+ VirtualizationContainerReservationClass "1" *-- "0..1" FlavorClass
+ UpdateResourceReservationRequest "1" *-- "0..1" ComputePoolReservationClass
+ UpdateResourceReservationRequest "1" *-- "0..1" StoragePoolReservationClass
+ UpdateResourceReservationRequest "1" *-- "0..1" NetworkReservationClass
+ @enduml
+
+.. -*
+
+**UpdateResourceReservationReply (VIM -> NFVO)**
+
+.. uml::
+
+ @startuml
+ class UpdateResourceReservationReply {
+ + reservationId [1]: Identifier
+ + virtualizationContainerReserved [0..N]: VirtualizationContainerReservedClass
+ + computePoolReserved [0..1]: ComputePoolReservedClass
+ + storagePoolReserved [0..1]: StoragePoolReservedClass
+ + networkReserved [0..1]: NetworkReservedClass
+ + reservationStatus [1]: String
+ + message [0..1]: String
+ }
+
+ class VirtualizationContainerReservedClass {
+ + containerId [1]: Identifier
+ + flavor [0..1]: FlavorClass
+ }
+
+ UpdateResourceReservationReply "1" *- "0..N" VirtualizationContainerReservedClass : ""
+ VirtualizationContainerReservedClass "1" *-- "0..1" FlavorClass
+ UpdateResourceReservationReply "1" *-- "0..1" ComputePoolReservedClass
+ UpdateResourceReservationReply "1" *-- "0..1" StoragePoolReservedClass
+ UpdateResourceReservationReply "1" *-- "0..1" NetworkReservedClass
+ @enduml
+
+.. -*
+
+Release Resource Reservation
+____________________________
+
+**ReleaseResourceReservationRequest (NFVO -> VIM)**
+
+.. uml::
+
+ @startuml
+ class ReleaseResourceReservationRequest {
+ + reservationId [1]: Identifier
+ }
+ @enduml
+
+**ReleaseResourceReservationReply (VIM -> NFVO)**
+
+.. uml::
+
+ @startuml
+ class ReleaseResourceReservationReply {
+ + reservationId [1]: Identifier
+ + message [0..1]: String
+ }
+ @enduml
+
+
+Detailed Message Flows
+----------------------
+
+Resource Capacity Management
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. figure:: images/figure5.png
+ :width: 90%
+
+ Capacity Management Scenario
+
+Figure 5 shows a detailed message flow between the consumers and the
+functionalblocks inside the VIM and has the following steps:
+
+Step 1: The consumer subscribes to capacity change notifications
+
+Step 2: The Capacity Manager monitors the capacity information for the various
+types of resources by querying the various Controllers (e.g. Nova, Neutron,
+Cinder), either periodically or on demand and updates capacity information in
+the Capacity Map
+
+Step 3: Capacity changes are notified to the consumer
+
+Step 4: The consumer queries the Capacity Manager to retrieve capacity detailed
+information
+
+Resource Reservation
+^^^^^^^^^^^^^^^^^^^^
+
+.. figure:: images/figure6.png
+ :width: 90%
+
+ Resource Reservation for Future Use Scenario
+
+Figure 6 shows a detailed message flow between the consumers and the functional
+blocks inside the VIM and has the following steps:
+
+Step 1: The consumer creates a resource reservation request for future use by
+setting a start and end time for the allocation
+
+Step 2: The consumer gets an immediate reply with a reservation status message
+"reservationStatus" and an identifier to be used with this reservation instance
+"reservationID"
+
+Step 3: The consumer subscribes to reservation notification events
+
+Step 4: The Resource Reservation Manager checks the feasibility of the
+reservation request by consulting the Capacity Manager
+
+Step 5: The Resource Reservation Manager reserves the resources and stores the
+list of reservations IDs generated by the Controllers (e.g. Nova, Neutron,
+Cinder) in the Reservation Map
+
+Step 6: Once the reservation process is completed, the VIM sends a notification
+message to the consumer with information on the reserved resources
+
+Step 7: When start time arrives, the consumer creates a resource allocation
+request.
+
+Step 8: The consumer gets an immediate reply with an allocation status message
+"allocationStatus".
+
+Step 9: The consumer subscribes to allocation notification events
+
+Step 10: The Resource Allocation Manager allocates the reserved resources. If
+not all reserved resources are allocated before expiry, the reserved resources
+are released and a notification is sent to the consumer
+
+Step 11: Once the allocation process is completed, the VIM sends a notification
+message to the consumer with information on the allocated resources
diff --git a/requirements/resource_management/06-summary.rst b/requirements/resource_management/06-summary.rst
new file mode 100644
index 0000000..381ab87
--- /dev/null
+++ b/requirements/resource_management/06-summary.rst
@@ -0,0 +1,24 @@
+======================
+Summary and conclusion
+======================
+
+Resource Reservation and Resource Capacity Management are features to be
+supported by the VIM and exposed to the consumer via the VIM NBI. These
+features have been specified by ETSI NFV.
+
+This document has described several use cases and corresponding high level
+flows where Resource Reservation and Capacity Management are of great benefit
+for the consumer of the virtualised resource management interface: the NFVO or
+the VNFM. The use cases include:
+
+* Notification of changes in capacity in the NFVI
+* Query of available resource capacity
+* Reservation of a resource or set of resources for immediate use
+* Reservation of a resource or set of resources for future use
+
+The Promise project has performed a gap analysis in order to fulfil the
+required functionality. Based on the gap analysis an implementation plan and
+way forward has been proposed, including a possible design architecture and
+high level information model. Immediate next steps of this project is to
+deliver a working Proof-of-Concepts (PoC) and engage upstream communities to
+fill out the gaps identified by Promise.
diff --git a/requirements/resource_management/07-schemas.rst b/requirements/resource_management/07-schemas.rst
new file mode 100644
index 0000000..3887cc7
--- /dev/null
+++ b/requirements/resource_management/07-schemas.rst
@@ -0,0 +1,195 @@
+Promise YANG Schemas based on StormForge
+----------------------------------------
+
+Promise Schema
+^^^^^^^^^^^^^^
+
+.. code::
+
+ module opnfv-promise {
+ namespace "urn:opnfv:vim:promise";
+ prefix prom;
+
+ import opnfv-promise-models { prefix opm; }
+ import complex-types { prefix ct; }
+
+ description
+ "OPNFV Promise Resource Reservation/Allocation controller module";
+
+ revision 2015-04-16 {
+ description
+ "Initial revision.";
+ }
+
+ // MAIN CONTAINER
+
+ container promise {
+ ct:instance-list reservations {
+ description
+ "Aggregate collection of all registered ResourceReservation
+ instances";
+ ct:instance-type opm:ResourceReservation;
+ }
+ }
+
+ rpc list-reservations;
+ rpc create-reservation;
+ rpc cancel-reservation;
+
+ notification reservation-event;
+ notification capacity-event;
+ notification allocation-event;
+ }
+
+OPNFV Promise YANG Schema
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. code::
+
+ module opnfv-promise-models {
+ prefix opm;
+
+ import storm-common-models { prefix scm; }
+ import complex-types { prefix ct; }
+
+ feature resource-reservation;
+
+ ct:complex-type ResourceReservation {
+ ct:extends scm:ResourceElement;
+
+ description
+ "Contains the capacities of various resource services being reserved
+ along with any resource elements needed to be available at
+ the time of allocation(s).";
+
+ reference "OPNFV-PROMISE, Section 3.4.1";
+
+ leaf start { type ct:date-and-time; }
+ leaf end { type ct:date-and-time; }
+ leaf expiry {
+ description
+ "Duration in seconds from start when unallocated reserved resources
+ will be released back into the pool";
+ type number; units "seconds";
+ }
+ leaf zone {
+ type instance-identifier { ct:instance-type scm:AvailabilityZone; }
+ }
+ container capacity {
+ uses scm:compute-capacity;
+ uses scm:networking-capcity;
+ uses scm:storage-capacity;
+ }
+ leaf-list resources {
+ description
+ "Reference to a collection of existing resource elements required by
+ this reservation. It can contain any instance derived from
+ ResourceElement, such as ServerInstances or even other
+ ResourceReservations. If the ResourceReservation request is
+ accepted, the ResourceElement(s) listed here will be placed
+ into 'protected' mode as to prevent accidental delete.";
+ type instance-identifier {
+ ct:instance-type scm:ResourceElement;
+ }
+ // following 'must' statement applies to each element
+ must "boolean(/provider/elements/*[@id=id])" {
+ error-message
+ "One or more of the ResourceElement(s) does not exist in the
+ provider to be reserved";
+ }
+ }
+
+ leaf provider {
+ if-feature multi-provider;
+ config false;
+
+ description
+ "Reference to a specified existing provider from which this
+ reservation will be drawn if used in the context of multi-provider
+ environment.";
+ type instance-identifier {
+ ct:instance-type scm:ResourceProvider;
+ require-instance true;
+ }
+ }
+
+ container remaining {
+ config false;
+ description
+ "Provides visibility into total remaining capacity for this
+ reservation based on allocations that took effect utilizing
+ this reservation ID as a reference.";
+
+ uses scm:compute-capacity;
+ uses scm:networking-capcity;
+ uses scm:storage-capacity;
+ }
+
+ leaf-list allocations {
+ config false;
+ description
+ "Reference to a collection of consumed allocations referencing
+ this reservation.";
+ type instance-identifier {
+ ct:instance-type ResourceAllocation;
+ }
+ }
+
+ }
+
+ ct:complex-type ResourceAllocation {
+ ct:extends scm:ResourceElement;
+
+ description
+ "Contains a list of resources to be allocated with optional reference
+ to an existing reservation.
+
+ If reservation is specified but this request is received prior
+ to reservation start timestamp, then it will be rejected unless
+ 'allocate-on-start' is set to true. 'allocate-on-start' allows
+ the allocation to be auto-initiated and scheduled to run in the
+ future.
+
+ The 'priority' state indicates the classification for dealing
+ with resource starvation scenarios. Lower priority allocations
+ will be forcefully terminated to allow for higher priority
+ allocations to be fulfilled.
+
+ Allocations without reference to an existing reservation will
+ receive the lowest priority.";
+
+ reference "OPNFV-PROMISE, Section 3.4.3";
+
+ leaf reservation {
+ description "Reference to an existing reservation identifier";
+
+ type instance-identifier {
+ ct:instance-type ResourceReservation;
+ require-instance true;
+ }
+ }
+
+ leaf allocate-on-start {
+ description
+ "If 'allocate-on-start' is set to true, the 'planned' allocations will
+ take effect automatically at the reservation 'start' date/time.";
+ type boolean; default false;
+ }
+
+ ct:instance-list resources {
+ description "Contains list of new ResourceElements that will be
+ allocated";
+ ct:instance-type scm:ResourceElement;
+ }
+
+ leaf priority {
+ description
+ "Reflects current priority level of the allocation according to
+ classification rules";
+ type number;
+ config false;
+ }
+ }
+ }
+
+.. -*
diff --git a/requirements/resource_management/Makefile b/requirements/resource_management/Makefile
new file mode 100644
index 0000000..dbf00e2
--- /dev/null
+++ b/requirements/resource_management/Makefile
@@ -0,0 +1,18 @@
+BUILDDIR = build
+
+.PHONY: clean html pdf all
+
+all: html pdf
+
+clean:
+ rm -rf $(BUILDDIR)/* plantuml.jar
+
+html: plantuml.jar
+ sphinx-build -b html -d $(BUILDDIR)/doctrees . $(BUILDDIR)/html
+
+pdf: plantuml.jar
+ sphinx-build -b latex -d $(BUILDDIR)/doctrees . $(BUILDDIR)/latex
+ $(MAKE) -C $(BUILDDIR)/latex all-pdf
+
+plantuml.jar:
+ wget 'http://downloads.sourceforge.net/project/plantuml/plantuml.jar'
diff --git a/requirements/resource_management/conf.py b/requirements/resource_management/conf.py
new file mode 100755
index 0000000..93c3e16
--- /dev/null
+++ b/requirements/resource_management/conf.py
@@ -0,0 +1,21 @@
+import datetime
+
+extensions = ['sphinxcontrib.plantuml']
+plantuml = ['java', '-jar', 'plantuml.jar']
+
+source_suffix = '.rst'
+master_doc = 'index'
+pygments_style = 'sphinx'
+html_use_index = False
+
+pdf_documents = [('index', u'Promise', u'Promise Project', u'OPNFV')]
+plantuml_latex_output_format = 'eps'
+pdf_fit_mode = "shrink"
+pdf_stylesheets = ['sphinx','kerning','a4']
+latex_elements = {'printindex': ''}
+latex_appendices = ['07-schemas']
+
+project = u'Promise: Resource Management'
+copyright = u'%s, OPNFV' % datetime.date.today().year
+version = u'0.0.1'
+release = u'0.0.1'
diff --git a/requirements/resource_management/glossary.rst b/requirements/resource_management/glossary.rst
new file mode 100644
index 0000000..c1997f2
--- /dev/null
+++ b/requirements/resource_management/glossary.rst
@@ -0,0 +1,63 @@
+**Definition of terms**
+
+Different SDOs and communities use different terminology related to
+NFV/Cloud/SDN. This list tries to define an OPNFV terminology,
+mapping/translating the OPNFV terms to terminology used in other contexts.
+
+.. glossary::
+
+ NFV
+ Network Function Virtualization
+
+ NFVI
+ Network Function Virtualization Infrastructure; totality of all hardware
+ and software components which build up the environment in which VNFs are
+ deployed.
+
+ VNF
+ Virtualized Network Function. Implementation of an Network Function that
+ can be deployed on a Network Function Virtualization Infrastructure (NFVI).
+
+ VIM
+ Virtualized Infrastructure Manager; functional block that is responsible
+ for controlling and managing the NFVI compute, storage and network
+ resources, usually within one operator's Infrastructure Domain, e.g. NFVI
+ Point of Presence (NFVI-PoP).
+
+ NFVO
+ Network Functions Virtualization Orchestrator; functional block that
+ manages the Network Service (NS) lifecycle and coordinates the management
+ of NS lifecycle, VNF lifecycle (supported by the VNFM) and NFVI resources
+ (supported by the VIM) to ensure an optimized allocation of the necessary
+ resources and connectivity.
+
+ VNFM
+ Virtualized Network Function Manager; functional block that is responsible
+ for the lifecycle management of VNF.
+
+ Consumer
+ User-side Manager; consumer of the interfaces produced by the VIM; VNFM,
+ NFVO, or Orchestrator in ETSI NFV [7] terminology.
+
+ Administrator
+ Administrator of the system, e.g. OAM in Telco context.
+
+ Virtual Machine (VM)
+ Virtualized computation environment that behaves very much like a physical
+ computer/server.
+
+ Virtual Storage
+ Virtualized non-volatile storage allocated to a VM.
+
+ Virtual network
+ Virtual network routes information among the network interfaces of VM
+ instances and physical network interfaces, providing the necessary
+ connectivity.
+
+ Physical resource
+ Actual resources in NFVI; not visible to Consumer.
+
+ Virtual resource
+ A Virtual Machine (VM), a virtual network, or virtualized storage; Offered
+ resources to "Consumer" as result of infrastructure virtualization; visible
+ to Consumer.
diff --git a/requirements/resource_management/images/figure1.png b/requirements/resource_management/images/figure1.png
new file mode 100755
index 0000000..ef78c00
--- /dev/null
+++ b/requirements/resource_management/images/figure1.png
Binary files differ
diff --git a/requirements/resource_management/images/figure2.png b/requirements/resource_management/images/figure2.png
new file mode 100755
index 0000000..dd002a2
--- /dev/null
+++ b/requirements/resource_management/images/figure2.png
Binary files differ
diff --git a/requirements/resource_management/images/figure3.png b/requirements/resource_management/images/figure3.png
new file mode 100755
index 0000000..20e51c7
--- /dev/null
+++ b/requirements/resource_management/images/figure3.png
Binary files differ
diff --git a/requirements/resource_management/images/figure4.png b/requirements/resource_management/images/figure4.png
new file mode 100755
index 0000000..e0c96a1
--- /dev/null
+++ b/requirements/resource_management/images/figure4.png
Binary files differ
diff --git a/requirements/resource_management/images/figure5.png b/requirements/resource_management/images/figure5.png
new file mode 100755
index 0000000..4e161f4
--- /dev/null
+++ b/requirements/resource_management/images/figure5.png
Binary files differ
diff --git a/requirements/resource_management/images/figure6.png b/requirements/resource_management/images/figure6.png
new file mode 100755
index 0000000..8aeec23
--- /dev/null
+++ b/requirements/resource_management/images/figure6.png
Binary files differ
diff --git a/requirements/resource_management/index.rst b/requirements/resource_management/index.rst
new file mode 100644
index 0000000..855bd28
--- /dev/null
+++ b/requirements/resource_management/index.rst
@@ -0,0 +1,73 @@
+..
+ This work is licensed under a Creative Commons Attribution 3.0 Unported
+ License.
+
+ http://creativecommons.org/licenses/by/3.0/legalcode
+
+.. title::
+ Promise
+
+
+****************************
+Promise: Resource Management
+****************************
+
+:Project: Promise, https://wiki.opnfv.org/promise
+:Editors: Ashiq Khan (NTT DOCOMO), Bertrand Souville (NTT DOCOMO)
+:Authors: Ravi Chunduru (ClearPath Networks), Peter Lee (ClearPath Networks),
+ Gerald Kunzmann (NTT DOCOMO), Ryota Mibu (NEC),
+ Carlos Goncalves (NEC), Arturo Martin De Nicolas (Ericsson)
+:Project creation date: 2014-12-04
+:Submission date: 2015-04-XX
+
+:Abstract: Promise is an OPNFV requirement project. Its objective is to realize
+ ETSI NFV defined resource reservation and NFVI capacity features
+ within the scope of OPNFV. Promise provides the details of the
+ requirements on resource reservation, NFVI capacity management at
+ VIM, specification of the northbound interfaces from VIM relevant to
+ these features, and implementation plan to realize these features in
+ OPNFV.
+
+.. raw:: latex
+
+ \newpage
+
+.. include:: glossary.rst
+
+.. toctree::
+ :maxdepth: 4
+ :numbered:
+
+ 01-intro.rst
+ 02-usecase.rst
+ 03-arch.rst
+ 04-gap.rst
+ 05-impl.rst
+ 06-summary.rst
+
+.. raw:: latex
+
+ \newpage
+
+.. rubric:: References and bibliography
+
+[1] OPNFV, "Promise" requirement project, [Online]. Available at
+https://wiki.opnfv.org/doctor
+
+[2] OpenStack Blazar Project, [Online]. Available at
+https://wiki.openstack.org/wiki/Blazar
+
+[3] Stormforge Project, [Online]. Available at
+https://github.com/stormstack/stormforge
+
+[4] Stormify Project, [Online]. Available at
+https://github.com/stormstack/stormify
+
+[5] ETSI GS NFV MAN 001, "Network Function Virtualisation (NFV); Management
+and Orchestration"
+
+[6] ETSI GS NFV 003, "Network Function Virtualisation (NFV); Terminology
+for Main Concepts in NFV"
+
+[7] ETSI NFV, [Online]. Available at
+http://www.etsi.org/technologies-clusters/technologies/nfv