summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorChaoyi Huang <joehuang@huawei.com>2015-11-06 00:18:53 +0000
committerGerrit Code Review <gerrit@172.30.200.206>2015-11-06 00:18:53 +0000
commit1a148540057b9bcbc32865217319021ba09ae07b (patch)
tree300cc96b7644f387aead8d1fbc673c87a73e7aa0
parent817cb001b0ec1e30fd1db359cd71c00db59deba7 (diff)
parente1d89eb88b3d13d5e1383bac25bb74d90a9f8775 (diff)
Merge "JIRA:MULTISITE-8 -VNF GR Requirement"
-rw-r--r--multisite-vnf-gr-requirement.rst235
1 files changed, 235 insertions, 0 deletions
diff --git a/multisite-vnf-gr-requirement.rst b/multisite-vnf-gr-requirement.rst
new file mode 100644
index 0000000..a3755c2
--- /dev/null
+++ b/multisite-vnf-gr-requirement.rst
@@ -0,0 +1,235 @@
+This work is licensed under a Creative Commons Attribution 3.0 Unported License.
+http://creativecommons.org/licenses/by/3.0/legalcode
+
+
+=======================================
+ Multisite VNF Geo site redundancy
+=======================================
+
+Glossary
+========
+
+
+There are serveral concept required to be understood first
+ **Volume Snapshot**
+
+ **Volume Backup**
+
+ **Volume Replication**
+
+ **VM Snapshot**
+
+Please refer to reference section for these concept and comparison.
+
+
+Problem description
+===================
+
+Abstract
+------------
+
+a VNF (telecom application) should, be able to restore in another site for
+catastrophic failures happened.
+
+Description
+------------
+GR is to deal with more catastrophic failures (flood, earthquake, propagating
+software fault), and that loss of calls, or even temporary loss of service,
+is acceptable. It is also seems more common to accept/expect manual /
+administrator intervene into drive the process, not least because you don’t want
+to trigger the transfer by mistake.
+
+In terms of coordination/replication or backup/restore between geographic sites,
+discussion often (but not always) seems to focus on limited application level
+data/config replication, as opposed to replication backup/restore between of
+cloud infrastructure between different sites.
+
+And finally, the lack of a requirement to do fast media transfer (without
+resignalling) generally removes the need for special networking behavior, with
+slower DNS-style redirection being acceptable.
+
+This use case is more concerns about cloud infrastructure level capability to
+support VNF geo site redundancy
+
+Requirement and candidate solutions analysis
+===========================
+
+For VNF to be restored from the backup site for catastrophic failures,
+the VNF's bootable volume and data volumes must be restorable.
+
+There are three ways of restorable boot and data volumes. Choosing the right one
+largely depends on the underlying characteristics and requirements of a VNF.
+
+1. Nova Quiesce + Cinder Consistency volume snapshot+ Cinder backup
+ - GR software get the attached volumes for the VMs in a VNF from Nova
+ - GR software add these attached volumes to the consistency group in Cinder
+ (NOTE: make sure that the block storage driver supports CG technology)
+ - GR software call Nova API Quiesce to freeze VM and flush buffer
+ - GR software make cgsnapshots of these volumes in Cinder
+ - GR software call Nova API Unquiece to unfreeze VM. (NOTE: Because storage
+ often provide fast snapshot, so the duration between quiece and unquiece is
+ a short interval)
+ - GR software create volumes from the cgsnapshots in Cinder
+ - GR software create backup (incremental) for these volumes to backup storage
+ ( swift or ceph, or.. ) in Cinder if this site failed,
+ - GR software restore these backup volumes to Cinder in the backup site.
+ - GR software boot vm from bootable volume rom Cinder in the backup site and
+ attach the data volumes.
+
+Pros: 1) atomic quiesce / unquiesce api from Nova, make transactional snapshot
+ of a group of VMs is possible, for example, quiesce VM1, quiesce VM2,
+ quiesce VM3, snapshot VM1's volumes, snapshot VM2's volumes, snapshot
+ VM3's volumes, unquiesce VM3, unquiesce VM2, unquiesce VM1. For some
+ telecom application, the order is very important for a group of VMs
+ with strong relationship.
+ 2) leverage the Cinder consistency group functionality.
+
+Cons: Need Nova to expose the quiesce / unquiesce, fortunately it's alreay there
+in Nova-compute, just to add API layer to expose the functionality. NOTE: It's up
+to the GR policy and VNF character. Some VNF may afford short unavailable for GR
+purpose, and some other may use the standby of the VNF or member of the cluster to
+do disaster recovery replication to not interfere the service provided by the VNF.
+For these VNFs which can't be quieced/unquiece should use the option3 (VNF aware)
+to do the backup/replication.
+
+Requirement to OpenStack: Nova needs to expose quiesce / unquiesce api,
+which is lack in Nova now.
+
+Example characteristics and requirements of a VNF:
+ - VNF requires full data consistency during backup/restore process -
+ entire data should be replicated.
+ - VNF's data changes infrequently, which results in less number of volume
+ snapshots during a given time interval (hour, day, etc.);
+ - VNF is not highly dynamic, e.g. the number of scaling (in/out) operations is
+ small.
+ - VNF is not geo-redundant, does not aware of available cloud replication
+ mechanisms, has no built-in logic for replication: doesn't pre-select the
+ minimum replication data required for restarting the VNF in a different site.
+ (NOTE: The VNF who can perform such data cherry picking should consider case 3)
+
+2. Nova Snapshot + Glance Image + Cinder Snapshot + Cinder Backup
+ - GR software create VM snapshot in Nova
+ - Nova quiece the VM internally
+ (NOTE: The upper level application or GR software should take care of avoiding
+ infra level outage induced VNF outage)
+ - Nova create image in Glance
+ - Nova create a snapshot of the VM, including volumes
+ - If the VM is volume backed VM, then create volume snapshot in Cinder
+ - No image uploaded to glance, but add the snapshot in the meta data of the image
+ in Glance
+ - GR software to get the snapshot information from the Glance
+ - GR software create volumes from these snapshots
+ - GR software create backup (incremental) for these volumes to backup storage
+ ( swift or ceph, or.. ) in Cinder if this site failed,
+ - GR software restore these backup volumes to Cinder in the backup site.
+ - GR software boot vm from bootable volume from Cinder in the backup site and attach
+ the data volumes.
+
+Pros: 1) Automatically quiesce/unquiesce, and snapshot of volumes of one VM.
+
+Cons: 1) Impossible to form a transactional group of VMs backup. for example,
+ quiesce VM1, quiesce VM2, quiesce VM3, snapshot VM1, snapshot VM2,
+ snapshot VM3, unquiesce VM3, unquiesce VM2, unquiesce VM1. This is quite
+ important in telecom application in some scenario
+ 2) not leverage the Cinder consistency group.
+ 3) One more service Glance involved in the backup. Not only to manage the
+ increased snapshot in Cinder, but also need to manage the regarding
+ temporary image in Glance.
+
+Requirement to OpenStack: None.
+
+Example: It's suitable for single VM backup/restore, for example, for the small
+scale configuration database virtual machine which is running in active/standby
+model. There is very rare use case for application that only one VM need to be
+taken snapshot for back up.
+
+3. Selective Replication of Persistent Data
+ - GR software creates datastore (Block/Cinder, Object/Swift, App Custom storage)
+ with replication enabled at the relevant scope, for use to selectively
+ backup/replicate desire data to GR backup site
+ - Cinder : Various work underway to provide async replication of cinder
+ volumes for disaster recovery use, including this presentation from
+ Vancouver http://www.slideshare.net/SeanCohen/dude-wheres-my-volume-open-stack-summit-vancouver-2015
+ - Swift : Range of options of using native Swift replicas (at expense of
+ tighter coupling) to replication using backend plugins or volume replication
+ - Custom : A wide range of OpenSource technologies including Cassandra
+ and Ceph, with fully application level solutions also possible
+ - GR software get the reference of storage in the remote site storage
+ - If primary site failed,
+ - GR software managing recovery in backup site gets references to relevant
+ storage and passes to new software instances
+ - Software attaches (or has attached) replicated storage, in the case of
+ volumes promoting to writable.
+
+Pros: 1) Replication will be done in the storage level automatically, no need to
+ create backup regularly, for example, daily.
+ 2) Application selection of limited amount of data to replicate reduces
+ risk of replicating failed state and generates less overhear.
+ 3) Type of replication and model (active/backup, active/active, etc) can
+ be tailored to application needs
+
+Cons: 1) Applications need to be designed with support in mind, including both
+ selection of data to be replicated and consideration of consistency
+ 2) "Standard" support in Openstack for Disaster Recovery currently fairly
+ limited, though active work in this area.
+
+Requirement to OpenStack: save the real ref to volume admin_metadata after it has
+been managed by the driver https://review.openstack.org/#/c/182150/.
+
+Prototype
+-----------
+ None.
+
+Proposed solution
+-----------
+
+ requirements perspective we could recommend all three options for different
+ sceanrio, that it is an operator choice.
+ Options 1 & 2 seem to be more about replicating/backing up any VNF, whereas
+ option 3 is about proving a service to a replication aware application. It should
+ be noted that HA requirement is not a priority here, HA for VNF project will handle
+ the specific HA requirement. It should also be noted that it's up to specific
+ application how to do HA (out of scope here).
+ For the 3rd option, the app should know which volume has replication
+ capability, and write regarding data to this volume, and guarantee
+ consistency by the app itself. Option 3 is preferrable in HA scenario.
+
+
+Gaps
+====
+ 1) Nova to expose quiesce / unquiesce API:
+ https://blueprints.launchpad.net/nova/+spec/expose-quiesce-unquiesce-api
+ 2) save the real ref to volume admin_metadata in Cinder:
+ https://review.openstack.org/#/c/182150/
+
+
+**NAME-THE-MODULE issues:**
+
+* Nova
+
+Affected By
+-----------
+ OPNFV multisite cloud.
+
+References
+==========
+
+ Cinder snapshot ( no material/BP about snapshot itself availble from web )
+ http://docs.openstack.org/cli-reference/content/cinderclient_commands.html
+
+
+ Cinder volume backup
+ https://blueprints.launchpad.net/cinder/+spec/volume-backups
+
+ Cinder incremtal backup
+ https://blueprints.launchpad.net/cinder/+spec/incremental-backup
+
+ Cinder volume replication
+ https://blueprints.launchpad.net/cinder/+spec/volume-replication
+
+ Create VM snapshot with volume backed ( not found better matrial to explain
+ the volume backed VM snapshot, only code tells )
+ https://bugs.launchpad.net/nova/+bug/1322195
+
+ Cinder consistency group
+ https://github.com/openstack/cinder-specs/blob/master/specs/juno/consistency-groups.rst