summaryrefslogtreecommitdiffstats
path: root/ci
diff options
context:
space:
mode:
authorAlexandru Avadanii <Alexandru.Avadanii@enea.com>2017-09-05 13:25:08 +0000
committerGerrit Code Review <gerrit@opnfv.org>2017-09-05 13:25:08 +0000
commita1a413ad65c31ebf5dc42924f7ed04ab02a04872 (patch)
treebfe49085fe03cc6578ca9cd00ea82802bb57d1be /ci
parent14d7bf43d3790a0a5fb69c9eff0e93b9fd63c5ba (diff)
parent1b89628e4571a65245a743e4a85d38438a119b3d (diff)
Merge "build, deploy: Remove obsolete Fuel@Openstack code"
Diffstat (limited to 'ci')
-rw-r--r--ci/README39
-rwxr-xr-xci/build.sh269
-rwxr-xr-xci/clean_cache.sh38
3 files changed, 19 insertions, 327 deletions
diff --git a/ci/README b/ci/README
index 46698a739..898442559 100644
--- a/ci/README
+++ b/ci/README
@@ -1,5 +1,5 @@
##############################################################################
-# Copyright (c) 2015 Ericsson AB and others.
+# Copyright (c) 2017 Ericsson AB, Mirantis Inc., Enea Software AB and others.
# stefan.k.berg@ericsson.com
# jonas.bjurel@ericsson.com
# All rights reserved. This program and the accompanying materials
@@ -12,14 +12,13 @@ Abstract
========
The fuel/ci directory holds all Fuel@OPNFV programatic abstractions for
the OPNFV community release and continous integration pipeline.
-There are two Fuel@OPNF autonomous scripts fo this, complying to the OPNFV CI pipeline guideline:
- - build.sh
+There is now only one Fuel@OPNFV autonomous script for this, complying to the
+OPNFV CI pipeline guideline:
- deploy.sh
USAGE
=====
For usage information of the CI/CD scripts, please run:
-./build.sh -h
./deploy.sh -h
Details on the CI/CD deployment framework
@@ -28,35 +27,31 @@ Details on the CI/CD deployment framework
Overview and purpose
--------------------
The CI/CD deployment script relies on a configuration structure, providing base
-installer configuration (part of fuel repo: deploy/config), per POD specific
+installer configuration (part of fuel repo: mcp/config), per POD specific
configuration (part of a separate classified POD configuration repo: securedlab
-, plugin configuration (part of fuel repo: deploy/config/plugins), and
-deployment scenario configuration (part of fuel repo: deploy/scenario).
+and deployment scenario configuration (part of fuel repo: mcp/config/scenario).
- The base installer configuration resembles the least common denominator of all
- HW/POD environment and deployment scenarios (These configurations are
- normally carried by the the installer projects in this case (fuel@OPNFV).
+ HW/POD environment and deployment scenarios. These configurations are
+ normally carried by the the installer projects in this case (Fuel@OPNFV).
- Per POD specific configuration specifies POD unique parameters, the POD
parameter possible to alter is governed by the Fuel@OPNFV project.
-- Plugin configuration - providing configuration of a specific plugin.
- these configurations maintain there own namespace and are normally maintained
- by collaborative projects building Fuel@OPNFV plugins
- Deployment scenario configuration - provides a high level, POD/HW environment
independent scenario configuration for a specifiv deployment. It defines what
- features/plugins that shall be deployed - as well needed overrides of the base
- installer-, POD/HW environment- configurations. Objects allowed to override
- is governed by the Fuel@OPNFV project.
+ features shall be deployed - as well needed overrides of the base
+ installer, POD/HW environment configurations. Objects allowed to override
+ are governed by the Fuel@OPNFV project.
Executing a deployment
----------------------
deploy.sh must be executed locally at the target lab/pod/jumpserver
A configuration structure must be provided - see the section below.
It is straight forward to execute a deployment task - as an example:
-sudo deploy.sh -b file:///home/jenkins/config -l LF-1 -p POD-2 -s
-os_odl-l2_ha -i file:///home/jenkins/MyIso.iso
+$ sudo deploy.sh -b file:///home/jenkins/config \
+ -l lf -p pod2 -s os-nosdn-nofeature-ha
-b and -i arguments should be expressed in URI style (eg: file://...
-or http://.... The resources can thus be local or remote.
+or http://...). The resources can thus be local or remote.
Configuration repository structure
----------------------------------
@@ -68,9 +63,13 @@ remote strcture fullfilling the diectory/file structure below.
The reason that this configuration structure needs to be secure/hidden
is that there are security sensitive information in the various configuration
files.
+
+FIXME: Below information is out of date and should be refreshed after PDF
+support is fully implemented.
+
A local stripped version of this configuration structure with virtual
deployment configurations also exist under build/config/.
-Following configuration directory and file structure should be adheare to:
+Following configuration directory and file structure should adheare to:
TOP
!
@@ -98,4 +97,4 @@ TOP
Creating a deployment scenario
------------------------------
Please find deploy/scenario/README for instructions on how to create a new
-deployment scenario. \ No newline at end of file
+deployment scenario.
diff --git a/ci/build.sh b/ci/build.sh
deleted file mode 100755
index 82049cdcd..000000000
--- a/ci/build.sh
+++ /dev/null
@@ -1,269 +0,0 @@
-#!/bin/bash
-##############################################################################
-# Copyright (c) 2015 Ericsson AB and others.
-# stefan.k.berg@ericsson.com
-# jonas.bjurel@ericsson.com
-# All rights reserved. This program and the accompanying materials
-# are made available under the terms of the Apache License, Version 2.0
-# which accompanies this distribution, and is available at
-# http://www.apache.org/licenses/LICENSE-2.0
-##############################################################################
-
-############################################################################
-# BEGIN of usage description
-#
-usage ()
-{
-cat | more << EOF
-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-`basename $0`: Builds the Fuel@OPNFV stack
-
-usage: `basename $0` [-s spec-file] [-c cache-URI] [-l log-file] [-f flags]
- [output-directory]
-
-OPTIONS:
- -s spec-file (NOTE! DEPRECATED!)
- define the build-spec file, default ../build/config.mk. The script only
- verifies that the spec-file exists.
- -c cache base URI
- specifies the base URI to a build cache to be used/updated, supported
- methods are http://, ftp:// and file://
- -l log-file
- specifies the output log-file (stdout and stderr), if not specified
- logs are output to console as normal
- -v
- version tag to be applied to the build result
- -r
- alternative remote access method script/program. curl is default.
- -f flag[...]
- build flags:
- s: Do nothing, succeed
- f: Do nothing, fail
- D: Debug mode
- P: Clear the local cache before building. This flag is only
- valid if the "-c cache-URI" options has been specified and
- and the method in the cache-URI is file:// (local cache).
-
- -h help, prints this help text
-
- output-directory, specifies the directory for the output artifacts
- (.iso file). If no output-directory is specified, the current path
- when calling the script is used.
-
-
-Description:
-
-build.sh builds the opnfv .iso artifact.
-To reduce build time it uses build caches on a local or remote location. A
-cache is rebuilt and uploaded if either of the below conditions are met:
-1) The P(opulate) flag is set and the -c cache-base-URI is provided and set
- to the method file:// , if -c is
- not provided the cache will stay local.
-2) If a cache is invalidated by the make system - the exact logic is encoded
- in the cache.mk of the different parts of the build.
-3) A valid cache does not exist on the specified -c cache-base-URI.
-
-A cache has a blob (binary data) and a meta file in the format of:
- <SHA1>.blob
- <SHA1>.meta
-
-Logging is by default to console, but can be directed elsewhere with the -l
-option in which case both stdout and stderr is redirected to that destination.
-
-Built in unit testing of components is enabled by adding the t(est) flag.
-
-Return codes:
- - 0 Success!
- - 1-99 Unspecified build error
- - 100-199 Build system internal error (not build it self)
- - 101 Build system instance busy
- - 200 Build failure
-
-Examples:
- build -c http://opnfv.org/artifactory/fuel/cache \
- -d ~/jenkins/genesis/fuel/ci/output -f ti
-
-NOTE: At current the build scope is set to the git root of the repository, -d
- destination locations outside that scope will not work!
-EOF
-}
-#
-# END of usage description
-############################################################################
-
-############################################################################
-# BEGIN of function error_exit
-
-error_exit() {
- echo "$@" >&2
- exit 1
-}
-
-#
-# END of function error_exit
-############################################################################
-
-
-############################################################################
-# BEGIN of shorthand variables for internal use
-#
-SCRIPT_DIR=$(readlink -f $(dirname ${BASH_SOURCE[0]}))
-BUILD_BASE=$(readlink -e ${SCRIPT_DIR}/../build/)
-RESULT_DIR="${BUILD_BASE}/release"
-BUILD_SPEC="${BUILD_BASE}/config.mk"
-LOCAL_CACHE_ARCH_NAME="${LOCAL_CACHE_ARCH_NAME:-fuel-cache}"
-
-#
-# END of variables to customize
-############################################################################
-
-############################################################################
-# BEGIN of script assigned default variables
-#
-export CACHEBASE="file://$HOME/cache"
-export CACHETRANSPORT="curl --silent"
-CLEAR_CACHE=0
-MAKE_ARGS=""
-
-#
-# END of script assigned variables
-############################################################################
-
-build() {
- echo "CI build parameters:"
- echo "SCRIPT_DIR = $SCRIPT_DIR"
- echo "BUILD_BASE = $BUILD_BASE"
- echo "RESULT_DIR = $RESULT_DIR"
- echo "BUILD_SPEC = $BUILD_SPEC"
- echo "LOCAL_CACHE_ARCH_NAME = $LOCAL_CACHE_ARCH_NAME"
- echo "CLEAR_CACHE = $CLEAR_CACHE"
- echo "DEBUG = $DEBUG"
- echo "OUTPUT_DIR = $OUTPUT_DIR"
- echo "BUILD_LOG = $BUILD_LOG"
- echo "MAKE_ARGS = $MAKE_ARGS"
- echo "CACHEBASE = $CACHEBASE"
- echo "CACHETRANSPORT = $CACHETRANSPORT"
-
-
- if [ "$CLEAR_CACHE" -eq 1 ]; then
- echo $CACHEBASE | grep -q '^file://' $CACHE_BASE
- if [ $? -ne 0 ]; then
- error_exit "Can't clear a non-local cache!"
- else
- CACHEDIR=$(echo $CACHEBASE | sed 's;file://;;')
- echo "Clearing local cache at $CACHEDIR..."
- rm -rvf $CACHEDIR/*
- fi
- fi
-
- echo make ${MAKE_ARGS} cache
-
- cd ${BUILD_BASE}
- if make ${MAKE_ARGS} cache; then
- echo "Copying build result into $OUTPUT_DIR"
- sort ${BUILD_BASE}/gitinfo*.txt > ${OUTPUT_DIR}/gitinfo.txt
- cp ${RESULT_DIR}/*.iso ${OUTPUT_DIR}
- cp ${RESULT_DIR}/*.iso.txt ${OUTPUT_DIR}
- else
- error_exit "Build failed"
- fi
-}
-
-############################################################################
-# BEGIN of main
-#
-while getopts "s:c:l:v:f:r:f:h" OPTION
-do
- case $OPTION in
- s)
- BUILD_SPEC=${OPTARG}
- if [ ! -f ${BUILD_SPEC} ]; then
- echo "spec file does not exist: $BUILD_SPEC - exiting ...."
- exit 100
- fi
- ;;
- c)
- # This value is used by cache.sh
- export CACHEBASE=${OPTARG}
- ;;
- l)
- BUILD_LOG=$(readlink -f ${OPTARG})
- ;;
- v)
- MAKE_ARGS+="REVSTATE=${OPTARG}"
- ;;
- r)
- # This value is used by cache.sh
- export CACHETRANSPORT=${OPTARG}
- ;;
- h)
- usage
- rc=0
- exit $rc
- ;;
- f)
- BUILD_FLAGS=${OPTARG}
- for ((i=0; i<${#BUILD_FLAGS};i++)); do
- case ${BUILD_FLAGS:$i:1} in
- s)
- exit 0
- ;;
-
- f)
- exit 1
- ;;
-
- P)
- CLEAR_CACHE=1
- ;;
-
- D)
- DEBUG=1
- ;;
-
- *)
- error_exit "${BUILD_FLAGS:$i:1} is not a valid build flag - exiting ...."
- ;;
- esac
- done
- ;;
-
- *)
- echo "${OPTION} is not a valid argument"
- rc=100
- exit $rc
- ;;
- esac
-done
-
-# Get output directory
-shift $[$OPTIND - 1]
-case $# in
- 0)
- # No directory on command line
- OUTPUT_DIR=$(pwd)
- ;;
- 1)
- # Directory on command line
- OUTPUT_DIR=$(readlink -f $1)
- ;;
- *)
- error_exit "Too many arguments"
- ;;
-esac
-mkdir -p $OUTPUT_DIR || error_exit "Could not access output directory $OUTPUT_DIR"
-
-
-if [ -n "${BUILD_LOG}" ]; then
- touch ${BUILD_LOG} || error_exit "Could not write to log file ${BUILD_LOG}"
- build 2>&1 | tee ${BUILD_LOG}
-else
- build
-fi
-
-rc=$?
-exit $rc
-
-#
-# END of main
-############################################################################
diff --git a/ci/clean_cache.sh b/ci/clean_cache.sh
deleted file mode 100755
index 177fe821b..000000000
--- a/ci/clean_cache.sh
+++ /dev/null
@@ -1,38 +0,0 @@
-#!/bin/bash
-##############################################################################
-# Copyright (c) 2016 Ericsson AB and others.
-# stefan.k.berg@ericsson.com
-# All rights reserved. This program and the accompanying materials
-# are made available under the terms of the Apache License, Version 2.0
-# which accompanies this distribution, and is available at
-# http://www.apache.org/licenses/LICENSE-2.0
-##############################################################################
-
-# Clean the build cache according to its expiry date. Invoke with the cache
-# directory as the first argument.
-
-if [ -z "$1" ]; then
- echo "No cache directory specified, exiting..."
- exit 1
-else
- CACHEDIR=$1
- echo "Operating on cache $CACHEDIR"
-fi
-
-NOW=$(date '+%s')
-
-cd $CACHEDIR
-echo "Step 1, cleaning orphaned meta and blob files"
-ls *.meta *.blob | sed 's/\..*//' | sort | uniq -u | xargs -n 1 -I {} sh -c "rm -vf {}.*"
-echo "Step 2, cleaning expired files"
-for cache in $(ls -1 *.meta | sed 's/\..*//')
-do
- blob=${cache}.blob
- meta=${cache}.meta
- expiry=$(grep Expires: $meta | sed 's/Expires: *//')
- if [ $expiry -le $NOW ]; then
- echo "$cache expired $(date -d "@$expiry"), removing..."
- rm -f $blob $meta
- fi
-done
-