A simple method to separate configuration for base fuel, plugins, PODs

and deployment/test scenarios

READY TO MERGE!

Replaces: https://gerrit.opnfv.org/gerrit/#/c/3995/

Abstract
--------
This deployment framework relies on a configuration structure,
providing base installer configuration, per POD specific configuration,
plugin configuration, and deployment scenario configuration.
- The base installer configuration resembles the least common denominator
  of all HW/POD environment and deployment scenarios (These configurations
  are normally carried by the the installer projects in this case (fuel@OPNFV).
- Per POD specific configuration specifies POD unique parameters, the POD
  parameter possible to alter is governed by the Fuel@OPNFV project.
- Plugin configuration - providing configuration of a specific plugin.
  these configurations maintain there own namespace and are normally maintained
  by collaborative projects building Fuel@OPNFV plugins
- Deployment scenario configuration - provides a high level, POD/HW environment
  independent scenario configuration for a specific deployment. It defines what
  features/plugins that shall be deployed - as well needed overrides of the
  plugin config as well as the base installer-, POD/HW environment-
  configurations. Objects allowed to override
  is governed by the Fuel@OPNFV project.

Executing a deployment
----------------------
deploy.sh must be executed locally at the target lab/pod/jumpserver
A lab configuration structure must be provided - see the section below.
It is straight forward to execute a deployment task - as an example:
sudo deploy.sh -b file:///home/jenkins/config -l ericsson-1 -p pod-2
-s os_odl-l2_no-ha -i file:///home/jenkins/MyIso.iso

-b and -i arguments should be expressed in URI style. The resources can thus be
local or remote.

Feedback
--------
Please give feed-back before I'm going to far on a wrong tangent

Implemented scenarios so far:
-----------------------------
- os_ha
- os_no-ha
- os_odl-l3_ha
- os_odl-l3_no-ha
- os_odl-l2_ha
- os_odl-l2_no-ha
- os_onos_ha
- os_onos_no-ha
- os_kvm_ha
- os_kvm_no-ha
- os_ovs_ha
- os_ovs_no-ha
- os_kvm_ovs_ha
- os_kvm_ovs_no-ha

VERIFIED
READY TO MERGE

JIRA: FUEL-35

Change-Id: I94a9b477d8ed4ee8057c16d8f20fe543f7ecc20d
Signed-off-by: Jonas Bjurel <jonas.bjurel@ericsson.com>

1 files changed, 86 insertions, 91 deletions

diff --git a/ci/README b/ci/README
index aab823bc0..46698a739 100644
--- a/ci/README
+++ b/ci/README
@@ -8,99 +8,94 @@
 # http://www.apache.org/licenses/LICENSE-2.0
 ##############################################################################
 
-The genesis/fuel/ci directory holds all Fuel@OPNFV abstractions for the OPNFV
-community release and continous integration pipeline.
+Abstract
+========
+The fuel/ci directory holds all Fuel@OPNFV programatic abstractions for
+the OPNFV community release and continous integration pipeline.
 There are two Fuel@OPNF autonomous scripts fo this, complying to the OPNFV CI pipeline guideline:
  - build.sh
  - deploy.sh
 
-For more info on usage:
+USAGE
+=====
+For usage information of the CI/CD scripts, please run:
 ./build.sh -h
-sudo ./deploy.sh -h
-python deploy.py -h
-
-usage: python deploy.py [-h] [-nf] [-nh] [-fo] [-co] [-c] [-iso [ISO_FILE]]
-                        [-dea [DEA_FILE]] [-dha [DHA_FILE]] [-s STORAGE_DIR]
-                        [-b PXE_BRIDGE] [-p FUEL_PLUGINS_DIR]
-
-optional arguments:
-  -h, --help           show this help message and exit
-  -nf                  Do not install Fuel Master (and Node VMs when using
-                       libvirt)
-  -nh                  Don't run health check after deployment
-  -fo                  Install Fuel Master only (and Node VMs when using
-                       libvirt)
-  -co                  Cleanup VMs and Virtual Networks according to what is
-                       defined in DHA
-  -c                   Cleanup after deploy
-  -iso [ISO_FILE]      ISO File [default: OPNFV.iso]
-  -dea [DEA_FILE]      Deployment Environment Adapter: dea.yaml
-  -dha [DHA_FILE]      Deployment Hardware Adapter: dha.yaml
-  -s STORAGE_DIR       Storage Directory [default: images]
-  -b PXE_BRIDGE        Linux Bridge for booting up the Fuel Master VM
-                       [default: pxebr]
-  -p FUEL_PLUGINS_DIR  Fuel Plugins directory
-
-
-
-* EXAMPLES:
-
-- Install Fuel Master and deploy OPNFV Cloud from scratch on Hardware Environment:
-
-    sudo ./deploy.sh -iso ~/ISO/opnfv.iso -dea ~/CONF/hardware/dea.yaml -dha ~/CONF/hardware/dha.yaml -s /mnt/images -b pxebr
-
-
-- Install Fuel Master and deploy OPNFV Cloud from scratch on Virtual Environment:
-
-    sudo ./deploy.sh -iso ~/ISO/opnfv.iso -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml -s /mnt/images
-
-
-- Deploy OPNFV Cloud on an already active Environment where Fuel Master VM is running so no need to install Fuel again:
-
-    sudo ./deploy.sh -nf -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml
-
-    => with plugin installation
-    sudo ./deploy.sh -nf -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml
-
-    => with cleanup after deployment is finished
-    sudo ./deploy.sh -nf -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml -c
-
-    => no healthcheck after deployment is completed
-    sudo ./deploy.sh -nf -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml -nh
-
-
-- Install Fuel Master only (and Node VMs when using virtual environment):
-
-    => for virtual environment:
-    sudo ./deploy.sh -iso ~/ISO/opnfv.iso -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml -s /mnt/images
-
-    => for hardware environment:
-    sudo ./deploy.sh -iso ~/ISO/opnfv.iso -dea ~/CONF/hardware/dea.yaml -dha ~/CONF/hardware/dha.yaml -s /mnt/images -b pxebr
-
-
-- Cleanup a running OPNFV environment:
-
-    sudo ./deploy.sh -co -dha ~/CONF/virtual/dha.yaml
-
-
-* WARNINGS:
-
-=>  If optional argument -s <storage_dir> is not specified, Autodeployment will use
-"<current_working_dir>/images" as default, and it will create it, if it hasn't been created before
-
-=>  If optional argument -b <pxe_bridge> is not specified, Autodeployment will use "pxebr" as default,
-if the bridge does not exist, the application will terminate with an error message
-
-=>  If argument -iso [ISO_FILE] is not specified, Autodeployment will use "<current_working_dir>/OPNFV.iso"
-as default, if the iso file does not exist, the application will terminate with an error message
-
-=>  If argument -dea [DEA_FILE] is not specified, Autodeployment will use "<current_working_dir>/dea.yaml"
-as default, if DEA file does not exist, the application will terminate with an error message
-
-=>  If argument -dha [DHA_FILE] is not specified, Autodeployment will use "<current_working_dir>/dha.yaml"
-as default, if DHA file does not exist, the application will terminate with an error message
-
-=> Optional argument -b PXE_BRIDGE is not required for Autodeployment in virtual environment,
-   even if it is specified it will not be used at all because virtual environment is using a different virtual network setup
-
-=> If optional argument -p FUEL_PLUGINS_DIR is not specified, no external plugins will be installed in Fuel
\ No newline at end of file
+./deploy.sh -h
+
+Details on the CI/CD deployment framework
+=========================================
+
+Overview and purpose
+--------------------
+The CI/CD deployment script relies on a configuration structure, providing base
+installer configuration (part of fuel repo: deploy/config), per POD specific
+configuration (part of a separate classified POD configuration repo: securedlab
+, plugin configuration (part of fuel repo: deploy/config/plugins), and
+deployment scenario configuration (part of fuel repo: deploy/scenario).
+
+- The base installer configuration resembles the least common denominator of all
+  HW/POD environment and deployment scenarios (These configurations are
+  normally carried by the the installer projects in this case (fuel@OPNFV).
+- Per POD specific configuration specifies POD unique parameters, the POD
+  parameter possible to alter is governed by the Fuel@OPNFV project.
+- Plugin configuration - providing configuration of a specific plugin.
+  these configurations maintain there own namespace and are normally maintained
+  by collaborative projects building Fuel@OPNFV plugins
+- Deployment scenario configuration - provides a high level, POD/HW environment
+  independent scenario configuration for a specifiv deployment. It defines what
+  features/plugins that shall be deployed - as well needed overrides of the base
+  installer-, POD/HW environment- configurations. Objects allowed to override
+  is governed by the Fuel@OPNFV project.
+
+Executing a deployment
+----------------------
+deploy.sh must be executed locally at the target lab/pod/jumpserver
+A configuration structure must be provided - see the section below.
+It is straight forward to execute a deployment task - as an example:
+sudo deploy.sh -b file:///home/jenkins/config -l LF-1 -p POD-2 -s
+os_odl-l2_ha -i file:///home/jenkins/MyIso.iso
+
+-b and -i arguments should be expressed in URI style (eg: file://...
+or http://.... The resources can thus be local or remote.
+
+Configuration repository structure
+----------------------------------
+The CI deployment engine relies on a configuration directory/file structure
+pointed to by the -b option described above.
+Normally this points to the secure classified OPNFV securedlab repo to which
+only jenkins and andmins have access to, but you may point to any local or
+remote strcture fullfilling the diectory/file structure below.
+The reason that this configuration structure needs to be secure/hidden
+is that there are security sensitive information in the various configuration
+files.
+A local stripped version of this configuration structure with virtual
+deployment configurations also exist under build/config/.
+Following configuration directory and file structure should be adheare to:
+
+TOP
+!
++---- labs
+       !
+       +---- lab-name-1
+       !        !
+       !        +---- pod-name-1
+       !        !        !
+       !        !        +---- fuel
+       !        !               !
+       !        !               +---- config
+       !        !                       !
+       !        !                       +---- dea-pod-override.yaml
+       !        !                       !
+       !        !                       +---- dha.yaml
+       !        !
+       !        +---- pod-name-2
+       !                 !
+       !
+       +---- lab-name-2
+       !        !
+
+
+Creating a deployment scenario
+------------------------------
+Please find deploy/scenario/README for instructions on how to create a new
+deployment scenario.
\ No newline at end of file