diff options
Diffstat (limited to 'ci')
| -rw-r--r-- | ci/README | 177 | ||||
| -rwxr-xr-x | ci/deploy.sh | 240 |
2 files changed, 319 insertions, 98 deletions
@@ -8,99 +8,94 @@ # http://www.apache.org/licenses/LICENSE-2.0 ############################################################################## -The genesis/fuel/ci directory holds all Fuel@OPNFV abstractions for the OPNFV -community release and continous integration pipeline. +Abstract +======== +The fuel/ci directory holds all Fuel@OPNFV programatic abstractions for +the OPNFV community release and continous integration pipeline. There are two Fuel@OPNF autonomous scripts fo this, complying to the OPNFV CI pipeline guideline: - build.sh - deploy.sh -For more info on usage: +USAGE +===== +For usage information of the CI/CD scripts, please run: ./build.sh -h -sudo ./deploy.sh -h -python deploy.py -h - -usage: python deploy.py [-h] [-nf] [-nh] [-fo] [-co] [-c] [-iso [ISO_FILE]] - [-dea [DEA_FILE]] [-dha [DHA_FILE]] [-s STORAGE_DIR] - [-b PXE_BRIDGE] [-p FUEL_PLUGINS_DIR] - -optional arguments: - -h, --help show this help message and exit - -nf Do not install Fuel Master (and Node VMs when using - libvirt) - -nh Don't run health check after deployment - -fo Install Fuel Master only (and Node VMs when using - libvirt) - -co Cleanup VMs and Virtual Networks according to what is - defined in DHA - -c Cleanup after deploy - -iso [ISO_FILE] ISO File [default: OPNFV.iso] - -dea [DEA_FILE] Deployment Environment Adapter: dea.yaml - -dha [DHA_FILE] Deployment Hardware Adapter: dha.yaml - -s STORAGE_DIR Storage Directory [default: images] - -b PXE_BRIDGE Linux Bridge for booting up the Fuel Master VM - [default: pxebr] - -p FUEL_PLUGINS_DIR Fuel Plugins directory - - - -* EXAMPLES: - -- Install Fuel Master and deploy OPNFV Cloud from scratch on Hardware Environment: - - sudo ./deploy.sh -iso ~/ISO/opnfv.iso -dea ~/CONF/hardware/dea.yaml -dha ~/CONF/hardware/dha.yaml -s /mnt/images -b pxebr - - -- Install Fuel Master and deploy OPNFV Cloud from scratch on Virtual Environment: - - sudo ./deploy.sh -iso ~/ISO/opnfv.iso -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml -s /mnt/images - - -- Deploy OPNFV Cloud on an already active Environment where Fuel Master VM is running so no need to install Fuel again: - - sudo ./deploy.sh -nf -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml - - => with plugin installation - sudo ./deploy.sh -nf -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml - - => with cleanup after deployment is finished - sudo ./deploy.sh -nf -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml -c - - => no healthcheck after deployment is completed - sudo ./deploy.sh -nf -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml -nh - - -- Install Fuel Master only (and Node VMs when using virtual environment): - - => for virtual environment: - sudo ./deploy.sh -iso ~/ISO/opnfv.iso -dea ~/CONF/virtual/dea.yaml -dha ~/CONF/virtual/dha.yaml -s /mnt/images - - => for hardware environment: - sudo ./deploy.sh -iso ~/ISO/opnfv.iso -dea ~/CONF/hardware/dea.yaml -dha ~/CONF/hardware/dha.yaml -s /mnt/images -b pxebr - - -- Cleanup a running OPNFV environment: - - sudo ./deploy.sh -co -dha ~/CONF/virtual/dha.yaml - - -* WARNINGS: - -=> If optional argument -s <storage_dir> is not specified, Autodeployment will use -"<current_working_dir>/images" as default, and it will create it, if it hasn't been created before - -=> If optional argument -b <pxe_bridge> is not specified, Autodeployment will use "pxebr" as default, -if the bridge does not exist, the application will terminate with an error message - -=> If argument -iso [ISO_FILE] is not specified, Autodeployment will use "<current_working_dir>/OPNFV.iso" -as default, if the iso file does not exist, the application will terminate with an error message - -=> If argument -dea [DEA_FILE] is not specified, Autodeployment will use "<current_working_dir>/dea.yaml" -as default, if DEA file does not exist, the application will terminate with an error message - -=> If argument -dha [DHA_FILE] is not specified, Autodeployment will use "<current_working_dir>/dha.yaml" -as default, if DHA file does not exist, the application will terminate with an error message - -=> Optional argument -b PXE_BRIDGE is not required for Autodeployment in virtual environment, - even if it is specified it will not be used at all because virtual environment is using a different virtual network setup - -=> If optional argument -p FUEL_PLUGINS_DIR is not specified, no external plugins will be installed in Fuel
\ No newline at end of file +./deploy.sh -h + +Details on the CI/CD deployment framework +========================================= + +Overview and purpose +-------------------- +The CI/CD deployment script relies on a configuration structure, providing base +installer configuration (part of fuel repo: deploy/config), per POD specific +configuration (part of a separate classified POD configuration repo: securedlab +, plugin configuration (part of fuel repo: deploy/config/plugins), and +deployment scenario configuration (part of fuel repo: deploy/scenario). + +- The base installer configuration resembles the least common denominator of all + HW/POD environment and deployment scenarios (These configurations are + normally carried by the the installer projects in this case (fuel@OPNFV). +- Per POD specific configuration specifies POD unique parameters, the POD + parameter possible to alter is governed by the Fuel@OPNFV project. +- Plugin configuration - providing configuration of a specific plugin. + these configurations maintain there own namespace and are normally maintained + by collaborative projects building Fuel@OPNFV plugins +- Deployment scenario configuration - provides a high level, POD/HW environment + independent scenario configuration for a specifiv deployment. It defines what + features/plugins that shall be deployed - as well needed overrides of the base + installer-, POD/HW environment- configurations. Objects allowed to override + is governed by the Fuel@OPNFV project. + +Executing a deployment +---------------------- +deploy.sh must be executed locally at the target lab/pod/jumpserver +A configuration structure must be provided - see the section below. +It is straight forward to execute a deployment task - as an example: +sudo deploy.sh -b file:///home/jenkins/config -l LF-1 -p POD-2 -s +os_odl-l2_ha -i file:///home/jenkins/MyIso.iso + +-b and -i arguments should be expressed in URI style (eg: file://... +or http://.... The resources can thus be local or remote. + +Configuration repository structure +---------------------------------- +The CI deployment engine relies on a configuration directory/file structure +pointed to by the -b option described above. +Normally this points to the secure classified OPNFV securedlab repo to which +only jenkins and andmins have access to, but you may point to any local or +remote strcture fullfilling the diectory/file structure below. +The reason that this configuration structure needs to be secure/hidden +is that there are security sensitive information in the various configuration +files. +A local stripped version of this configuration structure with virtual +deployment configurations also exist under build/config/. +Following configuration directory and file structure should be adheare to: + +TOP +! ++---- labs + ! + +---- lab-name-1 + ! ! + ! +---- pod-name-1 + ! ! ! + ! ! +---- fuel + ! ! ! + ! ! +---- config + ! ! ! + ! ! +---- dea-pod-override.yaml + ! ! ! + ! ! +---- dha.yaml + ! ! + ! +---- pod-name-2 + ! ! + ! + +---- lab-name-2 + ! ! + + +Creating a deployment scenario +------------------------------ +Please find deploy/scenario/README for instructions on how to create a new +deployment scenario.
\ No newline at end of file diff --git a/ci/deploy.sh b/ci/deploy.sh index d5b70d0d6..2304d7296 100755 --- a/ci/deploy.sh +++ b/ci/deploy.sh @@ -1,8 +1,234 @@ #!/bin/bash -set -o errexit -topdir=$(dirname $(readlink -f $BASH_SOURCE)) -deploydir=$(cd ${topdir}/../deploy; pwd) -pushd ${deploydir} > /dev/null -echo -e "python deploy.py $@\n" -python deploy.py $@ -popd > /dev/null
\ No newline at end of file +set -e +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +############################################################################ +# BEGIN of Exit handlers +# +do_exit () { + clean + echo "Exiting ..." +} +# +# End of Exit handlers +############################################################################ + +############################################################################ +# BEGIN of usage description +# +usage () +{ +cat << EOF +xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx +`basename $0`: Deploys the Fuel@OPNFV stack + +usage: `basename $0` -b base-uri [-B PXE Bridge] [-f] [-F] [-H] -l lab-name -p pod-name -s deploy-scenario [-S image-dir] -i iso + -s deployment-scenario [-S optional Deploy-scenario path URI] + [-R optional local relen repo (containing deployment Scenarios] + +OPTIONS: + -b Base-uri for the stack-configuration structure + -B PXE Bridge for booting of Fuel master + -d Dry-run + -f Deploy on existing Fuel master + -F Do only create a Fuel master + -H No health check + -l Lab-name + -p Pod-name + -s Deploy-scenario short-name/base-file-name + -S Storage dir for VM images + -i iso url + +Description: +Deploys the Fuel@OPNFV stack on the indicated lab resource + +This script provides the Fuel@OPNFV deployment abstraction +It depends on the OPNFV official configuration directory/file structure +and provides a fairly simple mechanism to execute a deployment. +Input parameters to the build script is: +-b Base URI to the configuration directory (needs to be provided in a URI + style, it can be a local resource: file:// or a remote resource http(s)://) +-B PXE Bridge for booting of Fuel master, default is pxebr +-d Dry-run - Produces deploy config files (config/dea.yaml and + config/dha.yaml), but does not execute deploy +-f Deploy on existing Fuel master +-F Do only create a Fuel master +-H Do not run fuel built in health-check after successfull deployment +-l Lab name as defined in the configuration directory, e.g. lf +-p POD name as defined in the configuration directory, e.g. pod-1 +-s Deployment-scenario, this points to a deployment/test scenario file as + defined in the configuration directory: + e.g fuel-ocl-heat-ceilometer_scenario_0.0.1.yaml + or a deployment short-name as defined by scenario.yaml in the deployment + scenario path. +-S Storage dir for VM images, default is fuel/deploy/images +-i .iso image to be deployed (needs to be provided in a URI + style, it can be a local resource: file:// or a remote resource http(s)://) + +NOTE: Root priviledges are needed for this script to run + + +Examples: +sudo `basename $0` -b file:///home/jenkins/lab-config -l lf -p pod1 -s ha_odl-l3_heat_ceilometer -i file:///home/jenkins/myiso.iso +EOF +} + +# +# END of usage description +############################################################################ + +############################################################################ +# BEGIN of deployment clean-up +# +clean() { + echo "Cleaning up deploy tmp directories" + rm -rf ${SCRIPT_PATH}/ISO +} +# +# END of deployment clean-up +############################################################################ + +############################################################################ +# BEGIN of shorthand variables for internal use +# +SCRIPT_PATH=$(readlink -f $(dirname ${BASH_SOURCE[0]})) +DEPLOY_DIR=$(cd ${SCRIPT_PATH}/../deploy; pwd) +PXE_BRIDGE='' +NO_HEALTH_CHECK='' +USE_EXISTING_FUEL='' +FUEL_CREATION_ONLY='' +STORAGE_DIR='' +DRY_RUN=0 +# +# END of variables to customize +############################################################################ + +############################################################################ +# BEGIN of main +# +while getopts "b:B:dfFHl:p:s:S:i:h" OPTION +do + case $OPTION in + b) + BASE_CONFIG_URI=${OPTARG} + if [[ ! $BASE_CONFIG_URI == file://* ]] && \ + [[ ! $BASE_CONFIG_URI == http://* ]] && \ + [[ ! $BASE_CONFIG_URI == https://* ]] && \ + [[ ! $BASE_CONFIG_URI == ftp://* ]]; then + echo "-b $BASE_CONFIG_URI - Not given in URI style" + usage + exit 1 + fi + ;; + B) + PXE_BRIDGE="-b ${OPTARG}" + ;; + d) + DRY_RUN=1 + ;; + f) + USE_EXISTING_FUEL='-nf' + ;; + F) + FUEL_CREATION_ONLY='-fo' + ;; + H) + NO_HEALTH_CHECK='-nh' + ;; + l) + TARGET_LAB=${OPTARG} + ;; + p) + TARGET_POD=${OPTARG} + ;; + s) + DEPLOY_SCENARIO=${OPTARG} + ;; + S) + STORAGE_DIR="-s ${OPTARG}" + ;; + i) + ISO=${OPTARG} + if [[ ! $ISO == file://* ]] && \ + [[ ! $ISO == http://* ]] && \ + [[ ! $ISO == https://* ]] && \ + [[ ! $ISO == ftp://* ]]; then + echo "-i $ISO - Not given in URI style" + usage + exit 1 + fi + + ;; + h) + usage + exit 0 + ;; + *) + echo "${OPTION} is not a valid argument" + echo "Arguments not according to new argument style" + echo "Trying old-style compatibility mode" + pushd ${DEPLOY_DIR} > /dev/null + python deploy.py "$@" + popd > /dev/null + exit 0 + ;; + esac +done + +if [[ $EUID -ne 0 ]]; then + echo "This script must be run as root" 1>&2 + exit 1 +fi + +if [ -z $BASE_CONFIG_URI ] || [ -z $TARGET_LAB ] || \ + [ -z $TARGET_POD ] || [ -z $DEPLOY_SCENARIO ] || \ + [ -z $ISO ]; then + echo "Arguments not according to new argument style" + echo "Trying old-style compatibility mode" + pushd ${DEPLOY_DIR} > /dev/null + python deploy.py "$@" + popd > /dev/null + exit 0 +fi + +# Enable the automatic exit trap +trap do_exit SIGINT SIGTERM EXIT + +# Set no restrictive umask so that Jenkins can removeeee any residuals +umask 0000 + +clean + +pushd ${DEPLOY_DIR} > /dev/null +# Prepare the deploy config files based on lab/pod information, deployment +# scenario, etc. + +echo "python deploy-config.py -dha ${BASE_CONFIG_URI}/labs/${TARGET_LAB}/${TARGET_POD}/fuel/config/dha.yaml -deab file://${DEPLOY_DIR}/config/dea_base.yaml -deao ${BASE_CONFIG_URI}/labs/${TARGET_LAB}/${TARGET_POD}/fuel/config/dea-pod-override.yaml -scenario-base-uri file://${DEPLOY_DIR}/scenario -scenario ${DEPLOY_SCENARIO} -plugins file://${DEPLOY_DIR}/config/plugins -output ${SCRIPT_PATH}/config" + +python deploy-config.py -dha ${BASE_CONFIG_URI}/labs/${TARGET_LAB}/${TARGET_POD}/fuel/config/dha.yaml -deab file://${DEPLOY_DIR}/config/dea_base.yaml -deao ${BASE_CONFIG_URI}/labs/${TARGET_LAB}/${TARGET_POD}/fuel/config/dea-pod-override.yaml -scenario-base-uri file://${DEPLOY_DIR}/scenario -scenario ${DEPLOY_SCENARIO} -plugins file://${DEPLOY_DIR}/config/plugins -output ${SCRIPT_PATH}/config + +if [ $DRY_RUN -eq 0 ]; then + # Download iso if it doesn't already exists locally + if [[ $ISO == file://* ]]; then + ISO=${ISO#file://} + else + mkdir -p ${SCRIPT_PATH}/ISO + curl -o ${SCRIPT_PATH}/ISO/image.iso $ISO + ISO=${SCRIPT_PATH}/ISO/image.iso + fi + # Start deployment + echo "python deploy.py -s $STORAGE_DIR -b $PXE_BRIDGE $USE_EXISTING_FUEL $FUEL_CREATION_ONLY $NO_HEALTH_CHECK -dea ${SCRIPT_PATH}/config/dea.yaml -dha ${SCRIPT_PATH}/config/dha.yaml -iso $ISO" + python deploy.py $STORAGE_DIR $PXE_BRIDGE $USE_EXISTING_FUEL $FUEL_CREATION_ONLY $NO_HEALTH_CHECK -dea ${SCRIPT_PATH}/config/dea.yaml -dha ${SCRIPT_PATH}/config/dha.yaml -iso $ISO +fi +popd > /dev/null + +# +# END of main +############################################################################ |