Align folder structure with Doctor. Split docs into sections.

JIRA:COPPER-1

Change-Id: Ic006eb2780be671b96545865950b831cffbed86d

Signed-off-by: blsaws <bs3131@att.com>

9 files changed, 547 insertions, 0 deletions

diff --git a/design_docs/architecture.rst b/design_docs/architecture.rst
new file mode 100644
index 0000000..bd91ffb
--- /dev/null
+++ b/design_docs/architecture.rst
@@ -0,0 +1,62 @@
+Architecture
+============
+
+Architectural Concept
+---------------------
+The following example diagram illustrates a "relationship diagram" type view of an NFVI platform, in which the roles of components focused on policy management, services, and infrastructure are shown. This view illustrates that a large-scale deployment of NFVI may leverage multiple components of the same "type" (e.g. SDN Controller), which fulfill specific purposes for which they are optimized. For example, a global SDN controller and cloud orchestrator can act as directed by a service orchestrator in the provisioning of VNFs per intent, while various components at a local and global level handle policy-related events directly and/or feed them back through a closed-loop policy design that responds as needed, directly or through the service orchestrator.
+ 
+.. image:: ./images/policy_architecture.png
+   :width: 700 px
+   :alt: policy_architecture.png
+   :align: center
+
+(source of the diagram above: https://wiki.opnfv.org/_media/copper/copper_work_items/policy_architecture.pptx)
+
+Architectural Aspects
+---------------------
+  * Policies are reflected in two high-level goals
+  
+    * Ensure resource requirements of VNFs are applied per VNF designer, service, and tenant intent
+    * Ensure that generic policies are not violated, e.g. *networks connected to VMs must either be public or owned by the VM owner*
+
+  * Policies are distributed through two main means
+  
+    * As part of VNF packages, customized if needed by Service Design tools, expressing intent of the VNF designer and service provider, and possibly customized or supplemented by service orchestrators per the intent of specific tenants
+    * As generic policies provisioned into VIMs (SDN controllers and cloud orchestrators), expressing intent of the service provider re what states/events need to be policy-governed independently of specific VNFs
+
+  * Policies are applied locally and in closed-loop systems per the capabilities of the local policy enforcer and the impact of the related state/event conditions
+
+    * VIMs should be able to execute most policies locally
+    * VIMs may need to pass policy-related state/events to a closed-loop system, where those events are relevant to other components in the architecture (e.g. service orchestrator), or some additional data/arbitration is needed to resolve the state/event condition
+
+  * Policies are localized as they are distributed/delegated
+
+    * High-level policies (e.g. expressing “intent”) can be translated into VNF package elements or generic policies, perhaps using distinct syntaxes
+    * Delegated policy syntaxes are likely VIM-specific, e.g. Datalog (Congress), YANG (ODL-based SDNC), or other schemas specific to other SDNCs (Contrail, ONOS)
+
+  * Closed-loop policy and VNF-lifecycle event handling are //somewhat// distinct
+ 
+    * Closed-loop policy is mostly about resolving conditions that can't be handled locally, but as above in some cases the conditions may be of relevance and either delivered directly or forwarded to service orchestrators
+    * VNF-lifecycle events that can't be handled by the VIM locally are delivered directly to the service orchestrator
+
+  * Some events/analytics need to be collected into a more "open-loop" system which can enable other actions, e.g.
+  
+    * audits and manual interventions
+    * machine-learning focused optimizations of policies (largely a future objective)
+    
+Issues to be investigated as part of establishing an overall cohesive/adaptive policy architecture:
+
+  * For the various components which may fulfill a specific purpose, what capabilities (e.g. APIs) do they have/need to
+  
+    * handle events locally
+    * enable closed-loop policy handling components to subscribe/optimize policy-related events that are of interest
+	
+  * For global controllers and cloud orchestrators
+  
+    * How do they support correlation of events impacting resources in different scopes (network and cloud)
+    * What event/response flows apply to various policy use cases
+	
+  * What specific policy use cases can/should fall into each overall class
+  
+    * locally handled by NFVI components
+    * handled by a closed-loop policy system, either VNF/service-specific or VNF-independent
diff --git a/design_docs/conf.py b/design_docs/conf.py
new file mode 100644
index 0000000..648a226
--- /dev/null
+++ b/design_docs/conf.py
@@ -0,0 +1,285 @@
+# -*- coding: utf-8 -*-
+#
+# Copper documentation build configuration file, created by
+# sphinx-quickstart on Tue Jun  9 19:12:31 2015.
+#
+# This file is execfile()d with the current directory set to its
+# containing dir.
+#
+# Note that not all possible configuration values are present in this
+# autogenerated file.
+#
+# All configuration values have a default; values that are commented out
+# serve to show the default.
+
+import sys
+import os
+import shlex
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#sys.path.insert(0, os.path.abspath('.'))
+
+# -- General configuration ------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#needs_sphinx = '1.0'
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = []
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+# source_suffix = ['.rst', '.md']
+source_suffix = '.rst'
+
+# The encoding of source files.
+#source_encoding = 'utf-8-sig'
+
+# The master toctree document.
+master_doc = 'index'
+
+# General information about the project.
+project = u'Copper'
+copyright = u'2015, OPNFV'
+author = u'OPNFV'
+
+# The version info for the project you're documenting, acts as replacement for
+# |version| and |release|, also used in various other places throughout the
+# built documents.
+#
+# The short X.Y version.
+version = '0'
+# The full version, including alpha/beta/rc tags.
+release = '1'
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# There are two options for replacing |today|: either, you set today to some
+# non-false value, then it is used:
+#today = ''
+# Else, today_fmt is used as the format for a strftime call.
+#today_fmt = '%B %d, %Y'
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+exclude_patterns = ['_build']
+
+# The reST default role (used for this markup: `text`) to use for all
+# documents.
+#default_role = None
+
+# If true, '()' will be appended to :func: etc. cross-reference text.
+#add_function_parentheses = True
+
+# If true, the current module name will be prepended to all description
+# unit titles (such as .. function::).
+#add_module_names = True
+
+# If true, sectionauthor and moduleauthor directives will be shown in the
+# output. They are ignored by default.
+#show_authors = False
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'sphinx'
+
+# A list of ignored prefixes for module index sorting.
+#modindex_common_prefix = []
+
+# If true, keep warnings as "system message" paragraphs in the built documents.
+#keep_warnings = False
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = False
+
+
+# -- Options for HTML output ----------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+html_theme = 'alabaster'
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further.  For a list of options available for each theme, see the
+# documentation.
+#html_theme_options = {}
+
+# Add any paths that contain custom themes here, relative to this directory.
+#html_theme_path = []
+
+# The name for this set of Sphinx documents.  If None, it defaults to
+# "<project> v<release> documentation".
+#html_title = None
+
+# A shorter title for the navigation bar.  Default is the same as html_title.
+#html_short_title = None
+
+# The name of an image file (relative to this directory) to place at the top
+# of the sidebar.
+#html_logo = None
+
+# The name of an image file (within the static path) to use as favicon of the
+# docs.  This file should be a Windows icon file (.ico) being 16x16 or 32x32
+# pixels large.
+#html_favicon = None
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+# Add any extra paths that contain custom files (such as robots.txt or
+# .htaccess) here, relative to this directory. These files are copied
+# directly to the root of the documentation.
+#html_extra_path = []
+
+# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
+# using the given strftime format.
+#html_last_updated_fmt = '%b %d, %Y'
+
+# If true, SmartyPants will be used to convert quotes and dashes to
+# typographically correct entities.
+#html_use_smartypants = True
+
+# Custom sidebar templates, maps document names to template names.
+#html_sidebars = {}
+
+# Additional templates that should be rendered to pages, maps page names to
+# template names.
+#html_additional_pages = {}
+
+# If false, no module index is generated.
+#html_domain_indices = True
+
+# If false, no index is generated.
+#html_use_index = True
+
+# If true, the index is split into individual pages for each letter.
+#html_split_index = False
+
+# If true, links to the reST sources are added to the pages.
+#html_show_sourcelink = True
+
+# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
+#html_show_sphinx = True
+
+# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
+#html_show_copyright = True
+
+# If true, an OpenSearch description file will be output, and all pages will
+# contain a <link> tag referring to it.  The value of this option must be the
+# base URL from which the finished HTML is served.
+#html_use_opensearch = ''
+
+# This is the file name suffix for HTML files (e.g. ".xhtml").
+#html_file_suffix = None
+
+# Language to be used for generating the HTML full-text search index.
+# Sphinx supports the following languages:
+#   'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja'
+#   'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr'
+#html_search_language = 'en'
+
+# A dictionary with options for the search language support, empty by default.
+# Now only 'ja' uses this config value
+#html_search_options = {'type': 'default'}
+
+# The name of a javascript file (relative to the configuration directory) that
+# implements a search results scorer. If empty, the default will be used.
+#html_search_scorer = 'scorer.js'
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'Copperdoc'
+
+# -- Options for LaTeX output ---------------------------------------------
+
+latex_elements = {
+# The paper size ('letterpaper' or 'a4paper').
+#'papersize': 'letterpaper',
+
+# The font size ('10pt', '11pt' or '12pt').
+#'pointsize': '10pt',
+
+# Additional stuff for the LaTeX preamble.
+#'preamble': '',
+
+# Latex figure (float) alignment
+#'figure_align': 'htbp',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+#  author, documentclass [howto, manual, or own class]).
+latex_documents = [
+  (master_doc, 'copper.tex', u'Copper Documentation',
+   u'OPNFV', 'manual'),
+]
+
+# The name of an image file (relative to this directory) to place at the top of
+# the title page.
+#latex_logo = None
+
+# For "manual" documents, if this is true, then toplevel headings are parts,
+# not chapters.
+#latex_use_parts = False
+
+# If true, show page references after internal links.
+#latex_show_pagerefs = False
+
+# If true, show URL addresses after external links.
+#latex_show_urls = False
+
+# Documents to append as an appendix to all manuals.
+#latex_appendices = []
+
+# If false, no module index is generated.
+#latex_domain_indices = True
+
+
+# -- Options for manual page output ---------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+    (master_doc, 'Copper', u'Copper Documentation',
+     [author], 1)
+]
+
+# If true, show URL addresses after external links.
+#man_show_urls = False
+
+
+# -- Options for Texinfo output -------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+#  dir menu entry, description, category)
+texinfo_documents = [
+  (master_doc, 'Copper', u'Copper Documentation',
+   author, 'OPNFV', 'One line description of project.',
+   'Miscellaneous'),
+]
+
+# Documents to append as an appendix to all manuals.
+#texinfo_appendices = []
+
+# If false, no module index is generated.
+#texinfo_domain_indices = True
+
+# How to display URL addresses: 'footnote', 'no', or 'inline'.
+#texinfo_show_urls = 'footnote'
+
+# If true, do not generate a @detailmenu in the "Top" node's menu.
+#texinfo_no_detailmenu = False
+
diff --git a/design_docs/definitions.rst b/design_docs/definitions.rst
new file mode 100644
index 0000000..26671f7
--- /dev/null
+++ b/design_docs/definitions.rst
@@ -0,0 +1,14 @@
+Definitions
+===========
+.. list-table:: Definitions
+   :widths: 15 85
+   :header-rows: 1
+
+   * - Term
+     - Meaning
+
+   * - State
+     - Information that can be used to convey or imply the state of something, e.g. an application, resource, entity, etc. This can include data held inside OPNFV components, "events" that have occurred (e.g. "policy violation"), etc.
+
+   * - Event
+     - An item of significance to the policy engine, for which the engine has become aware through some method of discovery e.g. polling or notification.
diff --git a/design_docs/images/policy_architecture.png b/design_docs/images/policy_architecture.png
new file mode 100644
index 0000000..7a5fb05
--- /dev/null
+++ b/design_docs/images/policy_architecture.png
diff --git a/design_docs/images/policy_architecture.pptx b/design_docs/images/policy_architecture.pptx
new file mode 100644
index 0000000..d88b175
--- /dev/null
+++ b/design_docs/images/policy_architecture.pptx
diff --git a/design_docs/index.rst b/design_docs/index.rst
new file mode 100644
index 0000000..1f251a8
--- /dev/null
+++ b/design_docs/index.rst
@@ -0,0 +1,27 @@
+.. Copper documentation master file, created by
+   sphinx-quickstart on Tue Jun  9 19:12:31 2015.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+OPNFV Copper Project
+====================
+
+Contents:
+
+.. toctree::
+   :numbered: 
+   :maxdepth: 4
+   
+   introduction.rst
+   definitions.rst
+   usecases.rst
+   architecture.rst
+   requirements.rst
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
+
diff --git a/design_docs/introduction.rst b/design_docs/introduction.rst
new file mode 100644
index 0000000..b9d9722
--- /dev/null
+++ b/design_docs/introduction.rst
@@ -0,0 +1,12 @@
+Introduction
+============
+
+..
+ This work is licensed under a Creative Commons Attribution 3.0 Unported License.
+
+ http://creativecommons.org/licenses/by/3.0/legalcode
+ 
+.. NOTE::
+   This is the working documentation for the Copper project.
+
+The `OPNFV Copper <https://wiki.opnfv.org/copper>`_ project aims to help ensure that virtualized infrastructure deployments comply with goals of the VNF designer/user, e.g. re affinity and partitioning (e.g. per regulation, control/user plane separation, cost…). This is a "requirements" project with initial goal to assess "off the shelf" basic OPNFV platform support for policy management, using existing open source projects such as OpenStack Congress and OpenDaylight Group-Based Policy (GBP). The project will assess what policy-related features are currently supported through research into the related projects in OpenStack and ODL, and testing of integrated vanilla distributions of those and other dependent open source projects in the OPNFV’s NFVI platform scope. 
\ No newline at end of file
diff --git a/design_docs/requirements.rst b/design_docs/requirements.rst
new file mode 100644
index 0000000..b45f77d
--- /dev/null
+++ b/design_docs/requirements.rst
@@ -0,0 +1,52 @@
+Requirements
+============
+General requirements for a policy architecture are below, with an assessment of the current state of support for these across major OPNFV components (1=poor, 5=excellent).
+
+  1. Polled monitoring: Exposure of state via request-response APIs.
+  2. Notifications: Exposure of state via pub-sub APIs.
+  3. Realtime/near-realtime notifications: Notifications that occur in actual or near realtime.
+  4. Delegated policy: CRUD operations on policies that are distributed to specific components for local handling, including one/more of monitoring, violation reporting, and enforcement.
+  5. Violation reporting: Reporting of conditions that represent a policy violation.
+  6. Reactive enforcement: Enforcement actions taken in response to policy violation events.
+  7. Proactive enforcement: Enforcement actions taken in advance of policy violation events, e.g. blocking actions that could result in a policy violation.
+  8. Compliance auditing: Periodic auditing of state against policies.
+  
+.. list-table:: Table 1: Assessment of NFVI VIM Support for General Requirements
+   :widths: 10 40 40
+   :header-rows: 1
+
+   * - #
+     - OpenStack
+     - OpenDaylight
+     
+   * - 1
+     - 
+     - 
+
+   * - 2
+     - 
+     - 
+
+   * - 3
+     - 
+     - 
+
+   * - 4
+     - 
+     - 
+
+   * - 5
+     - 
+     - 
+
+   * - 6
+     - 
+     - 
+
+   * - 7
+     - 
+     - 
+
+   * - 8
+     - 
+     - 
diff --git a/design_docs/usecases.rst b/design_docs/usecases.rst
new file mode 100644
index 0000000..33f9d78
--- /dev/null
+++ b/design_docs/usecases.rst
@@ -0,0 +1,95 @@
+Use Cases
+=========
+
+NFVI Self-Service Constraints
+-----------------------------
+
+As an NFVI provider, I need to ensure that my self-service tenants are not able to configure their VNFs in ways that would impact other tenants or the reliability, security, etc of the NFVI.
+
+Example: Network Access Control
+...............................
+
+Networks connected to VMs must be public, or owned by someone in the VM owner's group.
+
+This use case captures the intent of the following sub-use-cases:
+
+  * Link Mirroring: As a troubleshooter, I need to mirror traffic from physical or virtual network ports so that I can investigate trouble reports.
+  * Link Mirroring: As a NFVaaS tenant, I need to be able to mirror traffic on my virtual network ports so that I can investigate trouble reports.
+  * Unauthorized Link Mirroring Prevention: As a NFVaaS tenant, I need to be able to prevent other tenants from mirroring traffic on my virtual network ports so that I can protect the privacy of my service users.
+  * Link Mirroring Delegation: As a NFVaaS tenant, I need to be able to allow my NFVaaS SP customer support to mirror traffic on my virtual network ports so that they can assist in investigating trouble reports.
+
+As implemented through OpenStack Congress: 
+
+.. code:: 
+
+   error :- 
+   nova:vm(vm), 
+   neutron:network(network), 
+   nova:network(vm, network), 
+   neutron:private(network), 
+   nova:owner(vm, vm-own), 
+   neutron:owner(network, net-own), 
+   -same-group(vm-own, net-own)
+		 
+   same-group(user1, user2) :- 
+   ldap:group(user1, g), 
+   ldap:group(user2, g)
+   
+
+Storage Access Control
+......................
+
+Storage resources connected to VMs must be owned by someone in the VM owner's group.
+
+As implemented through OpenStack Congress: 
+
+.. code:: 
+
+  error :- 
+  nova:vm(vm), 
+  cinder:volumes(volume), 
+  nova:volume(vm, volume), 
+  nova:owner(vm, vm-own), 
+  neutron:owner(volume, vol-own), 
+  -same-group(vm-own, vol-own)
+ 
+  same-group(user1, user2) :- 
+  ldap:group(user1, g), 
+  ldap:group(user2, g)
+
+Resource Reclamation
+--------------------
+
+As a service provider or tenant, I need to be informed of VMs that are under-utilized so that I can reclaim the VI resources. (example from `RuleYourCloud blog <http://ruleyourcloud.com/2015/03/12/scaling-up-congress.html>`_) 
+
+As implemented through OpenStack Congress: 
+
+.. code:: 
+
+  reclaim_server(vm) :-
+  ceilometer:stats("cpu_util",vm, avg_cpu),
+  lessthan(avg_cpu, 1)
+
+  error(user_id, email, vm_name) :-
+  reclaim_server(vm),
+  nova:servers(vm, vm_name, user_id),
+  keystone:users(user_id, email)
+
+Workload Placement
+------------------
+
+Affinity
+........
+
+Ensures that the VM instance is launched "with affinity to" specific resources, e.g. within a compute or storage cluster. This is analogous to the affinity rules in `VMWare vSphere DRS <https://pubs.vmware.com/vsphere-50/topic/com.vmware.vsphere.resmgmt.doc_50/GUID-FF28F29C-8B67-4EFF-A2EF-63B3537E6934.html>`_. Examples include: "Same Host Filter", i.e. place on the same compute node as a given set of instances, e.g. as defined in a scheduler hint list.
+
+Anti-Affinity
+.............
+
+Ensures that the VM instance is launched "with anti-affinity to" specific resources, e.g. outside a compute or storage cluster, or geographic location. This filter is analogous to the anti-affinity rules in vSphere DRS. Examples include: "Different Host Filter", i.e. ensures that the VM instance is launched on a different compute node from a given set of instances, as defined in a scheduler hint list.
+
+Configuration Auditing
+----------------------
+
+As a service provider or tenant, I need to periodically verify that resource configuration requirements have not been violated, as a backup means to proactive or reactive policy enforcement.
+