summaryrefslogtreecommitdiffstats
path: root/ci
diff options
context:
space:
mode:
authorJonas Bjurel <jonas.bjurel@ericsson.com>2015-12-29 22:16:57 +0100
committerJonas Bjurel <jonas.bjurel@ericsson.com>2016-01-08 13:36:56 +0100
commit7fd2619e0df370fa832eeff00790bcaa52dc4ffc (patch)
tree6a6e6b22b43c0d4eda404265817a9f90a335e62e /ci
parentc14790e44122781b96b1b287f901c36645d7df7b (diff)
A simple method to separate configuration for base fuel, plugins, PODs
and deployment/test scenarios READY TO MERGE! Replaces: https://gerrit.opnfv.org/gerrit/#/c/3995/ Abstract -------- This deployment framework relies on a configuration structure, providing base installer configuration, per POD specific configuration, plugin configuration, and deployment scenario configuration. - The base installer configuration resembles the least common denominator of all HW/POD environment and deployment scenarios (These configurations are normally carried by the the installer projects in this case (fuel@OPNFV). - Per POD specific configuration specifies POD unique parameters, the POD parameter possible to alter is governed by the Fuel@OPNFV project. - Plugin configuration - providing configuration of a specific plugin. these configurations maintain there own namespace and are normally maintained by collaborative projects building Fuel@OPNFV plugins - Deployment scenario configuration - provides a high level, POD/HW environment independent scenario configuration for a specific deployment. It defines what features/plugins that shall be deployed - as well needed overrides of the plugin config as well as the base installer-, POD/HW environment- configurations. Objects allowed to override is governed by the Fuel@OPNFV project. Executing a deployment ---------------------- deploy.sh must be executed locally at the target lab/pod/jumpserver A lab configuration structure must be provided - see the section below. It is straight forward to execute a deployment task - as an example: sudo deploy.sh -b file:///home/jenkins/config -l ericsson-1 -p pod-2 -s os_odl-l2_no-ha -i file:///home/jenkins/MyIso.iso -b and -i arguments should be expressed in URI style. The resources can thus be local or remote. Feedback -------- Please give feed-back before I'm going to far on a wrong tangent Implemented scenarios so far: ----------------------------- - os_ha - os_no-ha - os_odl-l3_ha - os_odl-l3_no-ha - os_odl-l2_ha - os_odl-l2_no-ha - os_onos_ha - os_onos_no-ha - os_kvm_ha - os_kvm_no-ha - os_ovs_ha - os_ovs_no-ha - os_kvm_ovs_ha - os_kvm_ovs_no-ha VERIFIED READY TO MERGE JIRA: FUEL-35 Change-Id: I94a9b477d8ed4ee8057c16d8f20fe543f7ecc20d Signed-off-by: Jonas Bjurel <jonas.bjurel@ericsson.com>
Diffstat (limited to 'ci')
-rw-r--r--ci/README177
-rwxr-xr-xci/deploy.sh184
2 files changed, 264 insertions, 97 deletions
diff --git a/ci/README b/ci/README
index aab823bc0..46698a739 100644
--- a/ci/README
+++ b/ci/README
@@ -8,99 +8,94 @@
# http://www.apache.org/licenses/LICENSE-2.0
##############################################################################
-The genesis/fuel/ci directory holds all Fuel@OPNFV abstractions for the OPNFV
-community release and continous integration pipeline.
+Abstract
+========
+The fuel/ci directory holds all Fuel@OPNFV programatic abstractions for
+the OPNFV community release and continous integration pipeline.
There are two Fuel@OPNF autonomous scripts fo this, complying to the OPNFV CI pipeline guideline:
- build.sh
- deploy.sh
-For more info on usage:
+USAGE
+=====
+For usage information of the CI/CD scripts, please run:
./build.sh -h
-sudo ./deploy.sh -h
-python deploy.py -h
-
-usage: python deploy.py [-h] [-nf] [-nh] [-fo] [-co] [-c] [-iso [ISO_FILE]]
- [-dea [DEA_FILE]] [-dha [DHA_FILE]] [-s STORAGE_DIR]
- [-b PXE_BRIDGE] [-p FUEL_PLUGINS_DIR]
-
-optional arguments:
- -h, --help show this help message and exit
- -nf Do not install Fuel Master (and Node VMs when using
- libvirt)
- -nh Don't run health check after deployment
- -fo Install Fuel Master only (and Node VMs when using
- libvirt)
- -co Cleanup VMs and Virtual Networks according to what is
- defined in DHA
- -c Cleanup after deploy
- -iso [ISO_FILE] ISO File [default: OPNFV.iso]
- -dea [DEA_FILE] Deployment Environment Adapter: dea.yaml
- -dha [DHA_FILE] Deployment Hardware Adapter: dha.yaml
- -s STORAGE_DIR Storage Directory [default: images]
- -b PXE_BRIDGE Linux Bridge for booting up the Fuel Master VM
- [default: pxebr]
- -p FUEL_PLUGINS_DIR Fuel Plugins directory
-
-
-
-* EXAMPLES:
-
-- Install Fuel Master and deploy OPNFV Cloud from scratch on Hardware Environment:
-
- sudo ./deploy.sh -iso ~/ISO/opnfv.iso -dea ~/CONF/hardware/dea.yaml -dha ~/CONF/hardware/dha.yaml -s /mnt/images -b pxebr
-
-
-- Install Fuel Master and deploy OPNFV Cloud from scratch on Virtual Environment:
-
- sudo ./deploy.sh -iso ~/ISO/opnfv.iso -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml -s /mnt/images
-
-
-- Deploy OPNFV Cloud on an already active Environment where Fuel Master VM is running so no need to install Fuel again:
-
- sudo ./deploy.sh -nf -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml
-
- => with plugin installation
- sudo ./deploy.sh -nf -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml
-
- => with cleanup after deployment is finished
- sudo ./deploy.sh -nf -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml -c
-
- => no healthcheck after deployment is completed
- sudo ./deploy.sh -nf -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml -nh
-
-
-- Install Fuel Master only (and Node VMs when using virtual environment):
-
- => for virtual environment:
- sudo ./deploy.sh -iso ~/ISO/opnfv.iso -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml -s /mnt/images
-
- => for hardware environment:
- sudo ./deploy.sh -iso ~/ISO/opnfv.iso -dea ~/CONF/hardware/dea.yaml -dha ~/CONF/hardware/dha.yaml -s /mnt/images -b pxebr
-
-
-- Cleanup a running OPNFV environment:
-
- sudo ./deploy.sh -co -dha ~/CONF/virtual/dha.yaml
-
-
-* WARNINGS:
-
-=> If optional argument -s <storage_dir> is not specified, Autodeployment will use
-"<current_working_dir>/images" as default, and it will create it, if it hasn't been created before
-
-=> If optional argument -b <pxe_bridge> is not specified, Autodeployment will use "pxebr" as default,
-if the bridge does not exist, the application will terminate with an error message
-
-=> If argument -iso [ISO_FILE] is not specified, Autodeployment will use "<current_working_dir>/OPNFV.iso"
-as default, if the iso file does not exist, the application will terminate with an error message
-
-=> If argument -dea [DEA_FILE] is not specified, Autodeployment will use "<current_working_dir>/dea.yaml"
-as default, if DEA file does not exist, the application will terminate with an error message
-
-=> If argument -dha [DHA_FILE] is not specified, Autodeployment will use "<current_working_dir>/dha.yaml"
-as default, if DHA file does not exist, the application will terminate with an error message
-
-=> Optional argument -b PXE_BRIDGE is not required for Autodeployment in virtual environment,
- even if it is specified it will not be used at all because virtual environment is using a different virtual network setup
-
-=> If optional argument -p FUEL_PLUGINS_DIR is not specified, no external plugins will be installed in Fuel \ No newline at end of file
+./deploy.sh -h
+
+Details on the CI/CD deployment framework
+=========================================
+
+Overview and purpose
+--------------------
+The CI/CD deployment script relies on a configuration structure, providing base
+installer configuration (part of fuel repo: deploy/config), per POD specific
+configuration (part of a separate classified POD configuration repo: securedlab
+, plugin configuration (part of fuel repo: deploy/config/plugins), and
+deployment scenario configuration (part of fuel repo: deploy/scenario).
+
+- The base installer configuration resembles the least common denominator of all
+ HW/POD environment and deployment scenarios (These configurations are
+ normally carried by the the installer projects in this case (fuel@OPNFV).
+- Per POD specific configuration specifies POD unique parameters, the POD
+ parameter possible to alter is governed by the Fuel@OPNFV project.
+- Plugin configuration - providing configuration of a specific plugin.
+ these configurations maintain there own namespace and are normally maintained
+ by collaborative projects building Fuel@OPNFV plugins
+- Deployment scenario configuration - provides a high level, POD/HW environment
+ independent scenario configuration for a specifiv deployment. It defines what
+ features/plugins that shall be deployed - as well needed overrides of the base
+ installer-, POD/HW environment- configurations. Objects allowed to override
+ is governed by the Fuel@OPNFV project.
+
+Executing a deployment
+----------------------
+deploy.sh must be executed locally at the target lab/pod/jumpserver
+A configuration structure must be provided - see the section below.
+It is straight forward to execute a deployment task - as an example:
+sudo deploy.sh -b file:///home/jenkins/config -l LF-1 -p POD-2 -s
+os_odl-l2_ha -i file:///home/jenkins/MyIso.iso
+
+-b and -i arguments should be expressed in URI style (eg: file://...
+or http://.... The resources can thus be local or remote.
+
+Configuration repository structure
+----------------------------------
+The CI deployment engine relies on a configuration directory/file structure
+pointed to by the -b option described above.
+Normally this points to the secure classified OPNFV securedlab repo to which
+only jenkins and andmins have access to, but you may point to any local or
+remote strcture fullfilling the diectory/file structure below.
+The reason that this configuration structure needs to be secure/hidden
+is that there are security sensitive information in the various configuration
+files.
+A local stripped version of this configuration structure with virtual
+deployment configurations also exist under build/config/.
+Following configuration directory and file structure should be adheare to:
+
+TOP
+!
++---- labs
+ !
+ +---- lab-name-1
+ ! !
+ ! +---- pod-name-1
+ ! ! !
+ ! ! +---- fuel
+ ! ! !
+ ! ! +---- config
+ ! ! !
+ ! ! +---- dea-pod-override.yaml
+ ! ! !
+ ! ! +---- dha.yaml
+ ! !
+ ! +---- pod-name-2
+ ! !
+ !
+ +---- lab-name-2
+ ! !
+
+
+Creating a deployment scenario
+------------------------------
+Please find deploy/scenario/README for instructions on how to create a new
+deployment scenario. \ No newline at end of file
diff --git a/ci/deploy.sh b/ci/deploy.sh
index d5b70d0d6..d1f411fa5 100755
--- a/ci/deploy.sh
+++ b/ci/deploy.sh
@@ -1,8 +1,180 @@
#!/bin/bash
+##############################################################################
+# Copyright (c) 2015 Ericsson AB and others.
+# jonas.bjurel@ericsson.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+
+############################################################################
+# BEGIN of Exit handlers
+#
+do_exit () {
+ clean
+ echo "Exiting ..."
+}
+#
+# End of Exit handlers
+############################################################################
+
+############################################################################
+# BEGIN of usage description
+#
+usage ()
+{
+cat << EOF
+xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+`basename $0`: Deploys the Fuel@OPNFV stack
+
+usage: `basename $0` -b base-uri -l lab-name -p pod-name -i iso
+ -s deployment-scenario [-S optional Deploy-scenario path URI]
+ [-R optional local relen repo (containing deployment Scenarios]
+
+OPTIONS:
+ -b Base-uri for the stack-configuration structure
+ -l Lab-name
+ -p Pod-name
+ -s Deploy-scenario short-name/base-file-name
+ -i iso url
+
+Description:
+Deploys the Fuel@OPNFV stack on the indicated lab resource
+
+This script provides the Fuel@OPNFV deployment abstraction
+It depends on the OPNFV official configuration directory/file structure
+and provides a fairly simple mechanism to execute a deployment.
+Input parameters to the build script is:
+-b Base URI to the configuration directory (needs to be provided in a URI
+ style, it can be a local resource: file:// or a remote resource http(s)://)
+-l Lab name as defined in the configuration directory, e.g. lf
+-p POD name as defined in the configuration directory, e.g. pod-1
+-s Deployment-scenario, this points to a deployment/test scenario file as
+ defined in the configuration directory:
+ e.g fuel-ocl-heat-ceilometer_scenario_0.0.1.yaml
+ or a deployment short-name as defined by scenario.yaml in the deployment
+ scenario path.
+-i .iso image to be deployed (needs to be provided in a URI
+ style, it can be a local resource: file:// or a remote resource http(s)://)
+
+NOTE: Root priviledges are needed for this script to run
+
+Examples:
+sudo `basename $0` -b file:///home/jenkins/lab-config -l lf -p pod1 -s ha_odl-l3_heat_ceilometer -i file:///home/jenkins/myiso.iso
+EOF
+}
+
+#
+# END of usage description
+############################################################################
+
+############################################################################
+# BEGIN of deployment clean-up
+#
+clean() {
+ echo "Cleaning up deploy tmp directories"
+ rm -rf ${SCRIPT_PATH}/config
+ rm -rf ${SCRIPT_PATH}/ISO
+ rm -rf ${SCRIPT_PATH}/releng
+}
+#
+# END of deployment clean-up
+############################################################################
+
+############################################################################
+# BEGIN of shorthand variables for internal use
+#
+SCRIPT_PATH=$(readlink -f $(dirname ${BASH_SOURCE[0]}))
+DEPLOY_DIR=$(cd ${SCRIPT_PATH}/../deploy; pwd)
+RELENG_REPO=https://jonasbjurel@gerrit.opnfv.org/gerrit/releng
+#
+# END of variables to customize
+############################################################################
+
+############################################################################
+# BEGIN of main
+#
+while getopts "b:l:p:s:i:h" OPTION
+do
+ case $OPTION in
+ b)
+ BASE_CONFIG_URI=${OPTARG}
+ ;;
+ l)
+ TARGET_LAB=${OPTARG}
+ ;;
+ p)
+ TARGET_POD=${OPTARG}
+ ;;
+ s)
+ DEPLOY_SCENARIO=${OPTARG}
+ ;;
+ i)
+ ISO=${OPTARG}
+ ;;
+ h)
+ usage
+ exit 0
+ ;;
+ *)
+ echo "${OPTION} is not a valid argument"
+ echo "Arguments not according to new argument style"
+ echo "Trying old-style compatibility mode"
+ pushd ${DEPLOY_DIR} > /dev/null
+ python deploy.py "$@"
+ popd > /dev/null
+ exit 0
+ ;;
+ esac
+done
+
+if [[ $EUID -ne 0 ]]; then
+ echo "This script must be run as root" 1>&2
+ exit 1
+fi
+
+if [ -z $BASE_CONFIG_URI ] || [ -z $TARGET_LAB ] || \
+ [ -z $TARGET_POD ] || [ -z $DEPLOY_SCENARIO ] || \
+ [ -z $ISO ]; then
+ echo "Arguments not according to new argument style"
+ echo "Trying old-style compatibility mode"
+ pushd ${DEPLOY_DIR} > /dev/null
+ python deploy.py "$@"
+ popd > /dev/null
+ exit 0
+fi
+
+# Enable the automatic exit trap
set -o errexit
-topdir=$(dirname $(readlink -f $BASH_SOURCE))
-deploydir=$(cd ${topdir}/../deploy; pwd)
-pushd ${deploydir} > /dev/null
-echo -e "python deploy.py $@\n"
-python deploy.py $@
-popd > /dev/null \ No newline at end of file
+trap do_exit SIGINT SIGTERM EXIT
+
+clean
+
+pushd ${DEPLOY_DIR} > /dev/null
+# Prepare the deploy config files based on lab/pod information, deployment
+# scenario, etc.
+
+echo "python deploy-config.py -dha ${BASE_CONFIG_URI}/labs/${TARGET_LAB}/${TARGET_POD}/fuel/config/dha.yaml -deab ${DEPLOY_DIR}/config/dea_base.yaml -deao ${BASE_CONFIG_URI}/labs/${TARGET_LAB}/${TARGET_POD}/fuel/config/dea-pod-override.yaml -scenario-base-uri ${DEPLOY_DIR}/scenario -scenario ${DEPLOY_SCENARIO} -plugins ${DEPLOY_DIR}/config/plugins -output ${SCRIPT_PATH}/config"
+
+python deploy-config.py -dha ${BASE_CONFIG_URI}/labs/${TARGET_LAB}/${TARGET_POD}/fuel/config/dha.yaml -deab file://${DEPLOY_DIR}/config/dea_base.yaml -deao ${BASE_CONFIG_URI}/labs/${TARGET_LAB}/${TARGET_POD}/fuel/config/dea-pod-override.yaml -scenario-base-uri file://${DEPLOY_DIR}/scenario -scenario ${DEPLOY_SCENARIO} -plugins file://${DEPLOY_DIR}/config/plugins -output ${SCRIPT_PATH}/config
+
+# Download iso if it doesn't already exists locally
+if [[ $ISO == file://* ]]; then
+ ISO=${ISO#file://}
+else
+ mkdir -p ${SCRIPT_PATH}/ISO
+ curl -o ${SCRIPT_PATH}/ISO/image.iso $ISO
+ ISO=${SCRIPT_PATH}/ISO/image.iso
+fi
+# Start deployment
+echo "python deploy.py -dea ${SCRIPT_PATH}/config/dea.yaml -dha ${SCRIPT_PATH}/config/dha.yaml -iso $ISO"
+python deploy.py -dea ${SCRIPT_PATH}/config/dea.yaml -dha ${SCRIPT_PATH}/config/dha.yaml -iso $ISO
+popd > /dev/null
+
+# TBD: Upload the test-section of the scenario yaml file to the fuel master:
+# var/www/test.yaml
+
+#
+# END of main
+############################################################################