build: ISO refactor, use docker, enable cache

Previously, Armband Makefile used to call Fuel@OPNFV Makefile
target <iso>, instead of <all>, which resulted in bypassing
using Docker for ISO building completely.

Switch to <all> Makefile target, hence bringing back Docker
as a build container.

This change justifies moving make variables to a separate spec
file, armband.mk.

While at it, enable caching of builds.
This requires Jenkins bash scripts to be updated for the ISO
build job, done in [1].

JIRA: ARMBAND-56

[1] https://gerrit.opnfv.org/gerrit/#/c/18453/

CHANGES:
- all: release (was all: build)
- Common make targets now depend on submodules-init;
- clean-build and clean-docker make targets have been replaced by:
  make clean, make deepclean;

FIXME: These issues should be fixed later:
- (optional) trim local repo paths from gitinfo files
- only account for armband submodules in cache fingerprints;
- identify size mismatch culprit

Change-Id: I7d73409be44983fa21d4c09d2e07b87a33231e03
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>

8 files changed, 487 insertions, 7 deletions

diff --git a/patches/opnfv-fuel/0036-repo-mirror-Allow-multi-arch-local-mirrors.patch b/patches/opnfv-fuel/0036-repo-mirror-Allow-multi-arch-local-mirrors.patch
index d48e9f07..45ebb4e9 100644
--- a/patches/opnfv-fuel/0036-repo-mirror-Allow-multi-arch-local-mirrors.patch
+++ b/patches/opnfv-fuel/0036-repo-mirror-Allow-multi-arch-local-mirrors.patch
@@ -13,20 +13,23 @@ independent (which is true so far).
 
 [ Alexandru Avadanii ]
 Reworked for applying on top of fuel@OPNFV.
+Pass UBUNTU_ARCH to Docker containers.
 
 Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com>
 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
 ---
  build/Makefile                            |  1 +
- build/repo-multi-arch-local-mirrors.patch | 45 +++++++++++++++++++++++++++++++
- 2 files changed, 46 insertions(+)
+ build/docker/Dockerfile                   |  2 +-
+ build/docker/runcontext                   |  8 ++++-
+ build/repo-multi-arch-local-mirrors.patch | 59 +++++++++++++++++++++++++++++++
+ 4 files changed, 68 insertions(+), 2 deletions(-)
  create mode 100644 build/repo-multi-arch-local-mirrors.patch
 
 diff --git a/build/Makefile b/build/Makefile
-index 2d33aa4..0490bab 100644
+index 95b1487..829a231 100644
 --- a/build/Makefile
 +++ b/build/Makefile
-@@ -139,6 +139,7 @@ $(ISOCACHE):
+@@ -135,6 +135,7 @@ $(ISOCACHE):
  	cd /tmp/fuel-main && git am $(TOPDIR)/bootstrap_admin_node.sh.patch
  	cd /tmp/fuel-main && git am $(TOPDIR)/isolinux.cfg.patch
  	cd /tmp/fuel-main/build/repos/fuel-nailgun && git am $(TOPDIR)/Mark-Intel-82599-10-Gigabit-NIC-as-DPDK-capable.patch
@@ -34,12 +37,44 @@ index 2d33aa4..0490bab 100644
  	# Repeat build up to three times
  	sudo -E ./fuel_build_loop
  	cp /tmp/fuel-main/build/artifacts/fuel*.iso .
+diff --git a/build/docker/Dockerfile b/build/docker/Dockerfile
+index 1bb56b7..b38ea4c 100644
+--- a/build/docker/Dockerfile
++++ b/build/docker/Dockerfile
+@@ -30,7 +30,7 @@ RUN echo "Defaults env_keep += \"ftp_proxy http_proxy https_proxy no_proxy RSYNC
+ # Keeping PWD is needed to build as root
+ RUN echo "Defaults env_keep += \"PWD\"" > /etc/sudoers.d/keep-pwd
+ # Keeping variables for ISO build
+-RUN echo "Defaults env_keep += \"MIRROR_UBUNTU MIRROR_UBUNTU_ROOT MIRROR_MOS_UBUNTU MIRROR_MOS_UBUNTU_ROOT MIRROR_FUEL LATEST_TARGET_UBUNTU\"" > /etc/sudoers.d/keep-mos
++RUN echo "Defaults env_keep += \"MIRROR_UBUNTU MIRROR_UBUNTU_ROOT MIRROR_MOS_UBUNTU MIRROR_MOS_UBUNTU_ROOT MIRROR_FUEL LATEST_TARGET_UBUNTU UBUNTU_ARCH\"" > /etc/sudoers.d/keep-mos
+ RUN chmod 0440 /etc/sudoers.d/open-sudo
+ RUN chmod 0440 /etc/sudoers.d/keep-proxies
+ RUN chmod 0440 /etc/sudoers.d/keep-pwd
+diff --git a/build/docker/runcontext b/build/docker/runcontext
+index 9f07776..daad663 100755
+--- a/build/docker/runcontext
++++ b/build/docker/runcontext
+@@ -111,7 +111,13 @@ if [ -n "$CACHEBASE" ]; then
+     fi
+ fi
+ 
+-RUN_CONTEXT_OPT="--cidfile $CID_FILE --privileged=true --rm -e HOME=$HOME -e CACHEDEBUG -e CACHETRANSPORT -e CACHEMAXAGE -e CACHEBASE -e BUILD_FUEL_PLUGINS -e MIRROR_UBUNTU -e MIRROR_UBUNTU_ROOT -e MIRROR_MOS_UBUNTU -e MIRROR_MOS_UBUNTU_ROOT -e MIRROR_FUEL -e LATEST_TARGET_UBUNTU -u $USER_ID:$GROUP_ID -w $PWD -v $GITROOT:$GITROOT -v /sys/fs/cgroup:/sys/fs/cgroup:ro $CACHEMOUNT"
++RUN_CONTEXT_OPT="--cidfile $CID_FILE --privileged=true --rm \
++    -e HOME=$HOME -e CACHEDEBUG -e CACHETRANSPORT -e CACHEMAXAGE -e CACHEBASE \
++    -e BUILD_FUEL_PLUGINS -e MIRROR_UBUNTU -e MIRROR_UBUNTU_ROOT \
++    -e MIRROR_MOS_UBUNTU -e MIRROR_MOS_UBUNTU_ROOT -e MIRROR_FUEL \
++    -e LATEST_TARGET_UBUNTU -e UBUNTU_ARCH \
++    -u $USER_ID:$GROUP_ID -w $PWD \
++    -v $GITROOT:$GITROOT -v /sys/fs/cgroup:/sys/fs/cgroup:ro $CACHEMOUNT"
+ 
+ # Passing "debug" puts up an interactive bash shell
+ if [ "$1" == "debug" ]; then
 diff --git a/build/repo-multi-arch-local-mirrors.patch b/build/repo-multi-arch-local-mirrors.patch
 new file mode 100644
-index 0000000..c3ff26a
+index 0000000..d404287
 --- /dev/null
 +++ b/build/repo-multi-arch-local-mirrors.patch
-@@ -0,0 +1,45 @@
+@@ -0,0 +1,59 @@
 +From: Stanislaw Kardach <stanislaw.kardach@cavium.com>
 +Date: Thu, 25 Feb 2016 13:31:19 +0100
 +Subject: repo mirror: Allow multi-arch local mirrors
@@ -57,7 +92,8 @@ index 0000000..c3ff26a
 +---
 + mirror/ubuntu/module.mk | 2 +-
 + sandbox.mk              | 2 +-
-+ 2 files changed, 2 insertions(+), 2 deletions(-)
++ config.mk               | 2 +-
++ 3 files changed, 3 insertions(+), 3 deletions(-)
 +
 +diff --git a/mirror/ubuntu/module.mk b/mirror/ubuntu/module.mk
 +index 7a9466e..fe1ada2 100644
@@ -85,3 +121,16 @@ index 0000000..c3ff26a
 + if [ -e $(SANDBOX_UBUNTU)/etc/resolv.conf ]; then sudo cp -a $(SANDBOX_UBUNTU)/etc/resolv.conf $(SANDBOX_UBUNTU)/etc/resolv.conf.orig; fi
 + sudo cp /etc/resolv.conf $(SANDBOX_UBUNTU)/etc/resolv.conf
 + if [ -e $(SANDBOX_UBUNTU)/etc/hosts ]; then sudo cp -a $(SANDBOX_UBUNTU)/etc/hosts $(SANDBOX_UBUNTU)/etc/hosts.orig; fi
++diff --git a/config.mk b/config.mk
++index 74ee039..45a3b30 100644
++--- a/config.mk
+++++ b/config.mk
++@@ -49,7 +49,7 @@ UBUNTU_MINOR:=04
++ UBUNTU_RELEASE_NUMBER:=$(UBUNTU_MAJOR).$(UBUNTU_MINOR)
++ UBUNTU_KERNEL_FLAVOR?=lts-trusty
++ UBUNTU_NETBOOT_FLAVOR?=netboot
++-UBUNTU_ARCH:=amd64
+++UBUNTU_ARCH?=amd64
++ UBUNTU_IMAGE_RELEASE:=$(UBUNTU_MAJOR)$(UBUNTU_MINOR)
++ SEPARATE_IMAGES?=/boot,ext2 /,ext4
++
diff --git a/patches/opnfv-fuel/0041-build-Allow-PLUGINS-override.patch b/patches/opnfv-fuel/0041-build-Allow-PLUGINS-override.patch
new file mode 100644
index 00000000..acbbc11c
--- /dev/null
+++ b/patches/opnfv-fuel/0041-build-Allow-PLUGINS-override.patch
@@ -0,0 +1,27 @@
+From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+Date: Wed, 10 Aug 2016 16:43:39 +0200
+Subject: [PATCH] build: Allow PLUGINS override
+
+Instead of using BUILD_FUEL_PLUGINS, which was intended for manual
+overrides of the Fule@OPNFV plugin list, allow PLUGINS to be
+preset to a custom list, which we will provide via Armband
+Makefile spec.
+
+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+---
+ build/f_isoroot/Makefile | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/build/f_isoroot/Makefile b/build/f_isoroot/Makefile
+index 4df7bcb..448f415 100644
+--- a/build/f_isoroot/Makefile
++++ b/build/f_isoroot/Makefile
+@@ -11,7 +11,7 @@
+ # Add Fuel plugin build targets here
+ # Plugins disabled for the Fuel Mitaka uplift. Please re-enable your plugin as you have
+ # verified it!
+-PLUGINS = f_odlpluginbuild f_onosfwpluginbuild f_vsperfpluginbuild f_ovs-nsh-dpdk-pluginbuild f_yardstick-pluginbuild f_kvm-pluginbuild f_bgpvpn-pluginbuild f_collectd-ceilometer-pluginbuild
++PLUGINS ?= f_odlpluginbuild f_onosfwpluginbuild f_vsperfpluginbuild f_ovs-nsh-dpdk-pluginbuild f_yardstick-pluginbuild f_kvm-pluginbuild f_bgpvpn-pluginbuild f_collectd-ceilometer-pluginbuild
+ #PLUGINS = f_odlpluginbuild f_onosfwpluginbuild f_ovsnfv-dpdk-pluginbuild f_vsperfpluginbuild f_ovs-nsh-dpdk-pluginbuild f_bgpvpn-pluginbuild
+ export PLUGINS
+ #NON_8-0_REBASED_PLUGINS = f_bgpvpn-pluginbuild
diff --git a/patches/opnfv-fuel/0042-build-Allow-MIRROR_MOS_UBUNTU-override.patch b/patches/opnfv-fuel/0042-build-Allow-MIRROR_MOS_UBUNTU-override.patch
new file mode 100644
index 00000000..37e4d078
--- /dev/null
+++ b/patches/opnfv-fuel/0042-build-Allow-MIRROR_MOS_UBUNTU-override.patch
@@ -0,0 +1,22 @@
+From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+Date: Tue, 16 Aug 2016 18:45:10 +0200
+Subject: [PATCH] build: Allow MIRROR_MOS_UBUNTU override
+
+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+---
+ build/Makefile | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/build/Makefile b/build/Makefile
+index 4a4bb2e..6e18ddd 100644
+--- a/build/Makefile
++++ b/build/Makefile
+@@ -60,7 +60,7 @@ export MIRROR_UBUNTU_ROOT := $(shell echo -n '/' ; echo "$(MIRROR_UBUNTU_URL)" |
+ 
+ export LATEST_MIRROR_ID_URL := http://$(shell ./select_closest_fuel_mirror.py)
+ 
+-export MIRROR_MOS_UBUNTU := $(shell echo "$(LATEST_MIRROR_ID_URL)" | cut -d'/' -f3)
++export MIRROR_MOS_UBUNTU ?= $(shell echo "$(LATEST_MIRROR_ID_URL)" | cut -d'/' -f3)
+ export LATEST_TARGET_UBUNTU := $(shell curl -sSf "$(MIRROR_MOS_UBUNTU)/mos-repos/ubuntu/$(MOSVERSION).target.txt" | head -1)
+ export MIRROR_MOS_UBUNTU_ROOT := /mos-repos/ubuntu/$(LATEST_TARGET_UBUNTU)
+ 
diff --git a/patches/opnfv-fuel/0043-build-Add-armband.mk-config.patch b/patches/opnfv-fuel/0043-build-Add-armband.mk-config.patch
new file mode 100644
index 00000000..4a312580
--- /dev/null
+++ b/patches/opnfv-fuel/0043-build-Add-armband.mk-config.patch
@@ -0,0 +1,144 @@
+From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+Date: Thu, 11 Aug 2016 15:34:32 +0200
+Subject: [PATCH] build: Add armband.mk config, env var ARMBAND_BASE
+
+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+---
+ build/Makefile          | 10 ++++++++++
+ build/armband.mk        | 53 +++++++++++++++++++++++++++++++++++++++++++++++++
+ build/docker/Dockerfile |  2 +-
+ build/docker/runcontext |  6 +++++-
+ 4 files changed, 69 insertions(+), 2 deletions(-)
+ create mode 100644 build/armband.mk
+
+diff --git a/build/Makefile b/build/Makefile
+index 4454c35..377ecc6 100644
+--- a/build/Makefile
++++ b/build/Makefile
+@@ -14,6 +14,9 @@ SHELL = /bin/bash
+ # settings.
+ -include environment.mk
+ 
++# Override Armband specific information & git commit references
++-include armband.mk
++
+ ############################################################################
+ # BEGIN of variables to customize
+ #
+@@ -129,6 +132,9 @@ $(ISOCACHE):
+ 	sudo docker info
+ 	cd /tmp/fuel-main && make repos
+ 	$(REPOINFO) -r /tmp/fuel-main > gitinfo_fuel.txt
++	@if test -n $(ARMBAND_BASE); then \
++		$(REPOINFO) -r $(ARMBAND_BASE) >> gitinfo_fuel.txt; \
++	fi
+ 	# OPNFV patches at Fuel build time
+ 	# Need to be commited in order for them to be considered by the Fuel
+ 	# build system
+@@ -228,6 +234,10 @@ debug:
+ 	sha1sum *.patch >> .cachedata
+ 	sha1sum fuel_build_loop >> .cachedata
+ 	sha1sum config.mk >> .cachedata
++	@if test -n $(ARMBAND_BASE); then \
++		sha1sum armband.mk >> .cachedata; \
++		$(REPOINFO) $(ARMBAND_BASE) >> .cachedata; \
++	fi
+ 	sha1sum Makefile >> .cachedata
+ 	$(CACHETOOL) getbiweek >> .cachedata
+ 	cat .cachedata | $(CACHETOOL) getid > .cacheid
+diff --git a/build/armband.mk b/build/armband.mk
+new file mode 100644
+index 0000000..1e772f0
+--- /dev/null
++++ b/build/armband.mk
+@@ -0,0 +1,53 @@
++##############################################################################
++# Copyright (c) 2016 Enea AB and others.
++# Alexandru.Avadanii@enea.com
++# All rights reserved. This program and the accompanying materials
++# are made available under the terms of the Apache License, Version 2.0
++# which accompanies this distribution, and is available at
++# http://www.apache.org/licenses/LICENSE-2.0
++##############################################################################
++
++# Only configure Armband stuff when ARMBAND_BASE is set
++ifdef ARMBAND_BASE
++    # Armband plugins, supported archs & specific info
++    export PLUGINS           := f_odlpluginbuild f_bgpvpn-pluginbuild
++    export UBUNTU_ARCH       := amd64 arm64
++    export PRODNO            := OPNFV_A_FUEL
++    export MIRROR_MOS_UBUNTU := linux.enea.com
++    export EXTRA_RPM_REPOS   := armband,http://linux.enea.com/mos-repos/centos/mos9.0-centos7/armband/x86_64,10
++
++    # Armband git submodules for Fuel/OPNFV components
++    export ASTUTE_REPO   := ${ARMBAND_BASE}/upstream/fuel-astute
++    export ASTUTE_COMMIT := HEAD
++
++    export NAILGUN_REPO   := ${ARMBAND_BASE}/upstream/fuel-web
++    export NAILGUN_COMMIT := HEAD
++
++    export FUEL_AGENT_REPO   := ${ARMBAND_BASE}/upstream/fuel-agent
++    export FUEL_AGENT_COMMIT := HEAD
++
++    export FUEL_NAILGUN_AGENT_REPO   := ${ARMBAND_BASE}/upstream/fuel-nailgun-agent
++    export FUEL_NAILGUN_AGENT_COMMIT := HEAD
++
++    export FUEL_MIRROR_REPO   := ${ARMBAND_BASE}/upstream/fuel-mirror
++    export FUEL_MIRROR_COMMIT := HEAD
++
++    export FUELLIB_REPO   := ${ARMBAND_BASE}/upstream/fuel-library
++    export FUELLIB_COMMIT := HEAD
++
++    export ODL_REPO   := ${ARMBAND_BASE}/upstream/fuel-plugin-opendaylight
++    export ODL_BRANCH := armband-workbench
++    export ODL_CHANGE := HEAD
++    export OPNFV_QUAGGE_PACKAGING_REPO := https://github.com/alexandruavadanii/opnfv-quagga-packaging
++
++    export OVS_NSH_DPDK_REPO   := ${ARMBAND_BASE}/upstream/fuel-plugin-ovs
++    export OVS_NSH_DPDK_BRANCH := HEAD
++
++    export VSPERF_REPO   := ${ARMBAND_BASE}/upstream/vswitchperf
++    export VSPERF_BRANCH := armband-workbench
++    export VSPERF_CHANGE := HEAD
++
++    export YARDSTICK_REPO   := ${ARMBAND_BASE}/upstream/yardstick
++    export YARDSTICK_BRANCH := armband-workbench
++    export YARDSTICK_CHANGE := HEAD
++endif
+diff --git a/build/docker/Dockerfile b/build/docker/Dockerfile
+index b38ea4c..624f233 100644
+--- a/build/docker/Dockerfile
++++ b/build/docker/Dockerfile
+@@ -30,7 +30,7 @@ RUN echo "Defaults env_keep += \"ftp_proxy http_proxy https_proxy no_proxy RSYNC
+ # Keeping PWD is needed to build as root
+ RUN echo "Defaults env_keep += \"PWD\"" > /etc/sudoers.d/keep-pwd
+ # Keeping variables for ISO build
+-RUN echo "Defaults env_keep += \"MIRROR_UBUNTU MIRROR_UBUNTU_ROOT MIRROR_MOS_UBUNTU MIRROR_MOS_UBUNTU_ROOT MIRROR_FUEL LATEST_TARGET_UBUNTU UBUNTU_ARCH\"" > /etc/sudoers.d/keep-mos
++RUN echo "Defaults env_keep += \"MIRROR_UBUNTU MIRROR_UBUNTU_ROOT MIRROR_MOS_UBUNTU MIRROR_MOS_UBUNTU_ROOT MIRROR_FUEL LATEST_TARGET_UBUNTU UBUNTU_ARCH ARMBAND_BASE\"" > /etc/sudoers.d/keep-mos
+ RUN chmod 0440 /etc/sudoers.d/open-sudo
+ RUN chmod 0440 /etc/sudoers.d/keep-proxies
+ RUN chmod 0440 /etc/sudoers.d/keep-pwd
+diff --git a/build/docker/runcontext b/build/docker/runcontext
+index daad663..e4874df 100755
+--- a/build/docker/runcontext
++++ b/build/docker/runcontext
+@@ -42,6 +42,10 @@ GITROOT=`git rev-parse --show-toplevel`
+ CID_FILE=`mktemp -u -t runcontext.XXXXXXXXXX`
+ CONTEXT_DIR=`mktemp -d ${GITROOT}/.docker_contextXXXXXX`
+ 
++if [[ $ARMBAND_BASE ]]; then
++    GITROOT=$ARMBAND_BASE
++fi
++
+ # If RSYNC_CONNECT_PROG is used, we need to copy all of
+ # the SSH structure, should one of the keys need to be
+ # used.
+@@ -115,7 +119,7 @@ RUN_CONTEXT_OPT="--cidfile $CID_FILE --privileged=true --rm \
+     -e HOME=$HOME -e CACHEDEBUG -e CACHETRANSPORT -e CACHEMAXAGE -e CACHEBASE \
+     -e BUILD_FUEL_PLUGINS -e MIRROR_UBUNTU -e MIRROR_UBUNTU_ROOT \
+     -e MIRROR_MOS_UBUNTU -e MIRROR_MOS_UBUNTU_ROOT -e MIRROR_FUEL \
+-    -e LATEST_TARGET_UBUNTU -e UBUNTU_ARCH \
++    -e LATEST_TARGET_UBUNTU -e UBUNTU_ARCH -e ARMBAND_BASE \
+     -u $USER_ID:$GROUP_ID -w $PWD \
+     -v $GITROOT:$GITROOT -v /sys/fs/cgroup:/sys/fs/cgroup:ro $CACHEMOUNT"
+ 
diff --git a/patches/opnfv-fuel/0044-build-cache-Allow-LOCAL_CACHE_ARCH_NAME-override.patch b/patches/opnfv-fuel/0044-build-cache-Allow-LOCAL_CACHE_ARCH_NAME-override.patch
new file mode 100644
index 00000000..e477421d
--- /dev/null
+++ b/patches/opnfv-fuel/0044-build-cache-Allow-LOCAL_CACHE_ARCH_NAME-override.patch
@@ -0,0 +1,22 @@
+From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+Date: Thu, 18 Aug 2016 14:39:08 +0200
+Subject: [PATCH] build: cache: Allow LOCAL_CACHE_ARCH_NAME override
+
+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+---
+ ci/build.sh | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/ci/build.sh b/ci/build.sh
+index 62abba1..82049cd 100755
+--- a/ci/build.sh
++++ b/ci/build.sh
+@@ -111,7 +111,7 @@ SCRIPT_DIR=$(readlink -f $(dirname ${BASH_SOURCE[0]}))
+ BUILD_BASE=$(readlink -e ${SCRIPT_DIR}/../build/)
+ RESULT_DIR="${BUILD_BASE}/release"
+ BUILD_SPEC="${BUILD_BASE}/config.mk"
+-LOCAL_CACHE_ARCH_NAME="fuel-cache"
++LOCAL_CACHE_ARCH_NAME="${LOCAL_CACHE_ARCH_NAME:-fuel-cache}"
+ 
+ #
+ # END of variables to customize
diff --git a/patches/opnfv-fuel/0045-build-cache-Consider-UBUNTU_ARCH-in-.cacheid.patch b/patches/opnfv-fuel/0045-build-cache-Consider-UBUNTU_ARCH-in-.cacheid.patch
new file mode 100644
index 00000000..43d5bcc5
--- /dev/null
+++ b/patches/opnfv-fuel/0045-build-cache-Consider-UBUNTU_ARCH-in-.cacheid.patch
@@ -0,0 +1,144 @@
+From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+Date: Wed, 17 Aug 2016 20:36:51 +0200
+Subject: [PATCH] build: cache: Consider UBUNTU_ARCH in .cacheid
+
+Usually UBUNTU_ARCH is not changed inside the same repository,
+but we should consider this case anyway for future merging
+Armband and Fuel@OPNFV repositories/build systems.
+
+This applies to all Fuel plugins and the local repo.
+
+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+---
+ build/f_isoroot/f_bgpvpn-pluginbuild/Makefile              | 1 +
+ build/f_isoroot/f_collectd-ceilometer-pluginbuild/Makefile | 1 +
+ build/f_isoroot/f_kvm-pluginbuild/Makefile                 | 1 +
+ build/f_isoroot/f_odlpluginbuild/Makefile                  | 1 +
+ build/f_isoroot/f_onosfwpluginbuild/Makefile               | 1 +
+ build/f_isoroot/f_ovs-nsh-dpdk-pluginbuild/Makefile        | 1 +
+ build/f_isoroot/f_ovsnfv-dpdk-pluginbuild/Makefile         | 1 +
+ build/f_isoroot/f_repobuild/Makefile                       | 1 +
+ build/f_isoroot/f_vsperfpluginbuild/Makefile               | 1 +
+ build/f_isoroot/f_yardstick-pluginbuild/Makefile           | 1 +
+ 10 files changed, 10 insertions(+)
+
+diff --git a/build/f_isoroot/f_bgpvpn-pluginbuild/Makefile b/build/f_isoroot/f_bgpvpn-pluginbuild/Makefile
+index ed3be29..8ce28fc 100644
+--- a/build/f_isoroot/f_bgpvpn-pluginbuild/Makefile
++++ b/build/f_isoroot/f_bgpvpn-pluginbuild/Makefile
+@@ -67,6 +67,7 @@ release:.bgpvpnbuild
+ 	fi
+ 	sha1sum Makefile >> .cachedata
+ 	sha1sum config.mk >> .cachedata
++	echo -n $(UBUNTU_ARCH) | sha1sum | awk {'print $$1'} >> .cachedata
+ 	cat .cachedata | $(CACHETOOL) getid > .cacheid
+ 
+ # Clean local data related to caching - called prior to ordinary build
+diff --git a/build/f_isoroot/f_collectd-ceilometer-pluginbuild/Makefile b/build/f_isoroot/f_collectd-ceilometer-pluginbuild/Makefile
+index 9ab922d..17bc36d 100644
+--- a/build/f_isoroot/f_collectd-ceilometer-pluginbuild/Makefile
++++ b/build/f_isoroot/f_collectd-ceilometer-pluginbuild/Makefile
+@@ -67,6 +67,7 @@ release:.fastpathmetrics
+ 	fi
+ 	@sha1sum Makefile | awk {'print $$1'} >> .cachedata
+ 	@sha1sum config.mk | awk {'print $$1'} >> .cachedata
++	@echo -n $(UBUNTU_ARCH) | sha1sum | awk {'print $$1'} >> .cachedata
+ 	@cat .cachedata | $(CACHETOOL) getid > .cacheid
+ 
+ # Clean local data related to caching - called prior to ordinary build
+diff --git a/build/f_isoroot/f_kvm-pluginbuild/Makefile b/build/f_isoroot/f_kvm-pluginbuild/Makefile
+index ba2fd0b..2454b5e 100644
+--- a/build/f_isoroot/f_kvm-pluginbuild/Makefile
++++ b/build/f_isoroot/f_kvm-pluginbuild/Makefile
+@@ -70,6 +70,7 @@ release:.kvmbuild
+ 	fi
+ 	@sha1sum Makefile | awk {'print $$1'} >> .cachedata
+ 	@sha1sum config.mk | awk {'print $$1'} >> .cachedata
++	@echo -n $(UBUNTU_ARCH) | sha1sum | awk {'print $$1'} >> .cachedata
+ 	@cat .cachedata | $(CACHETOOL) getid > .cacheid
+ 
+ # Clean local data related to caching - called prior to ordinary build
+diff --git a/build/f_isoroot/f_odlpluginbuild/Makefile b/build/f_isoroot/f_odlpluginbuild/Makefile
+index e2cbf7d..6c717c7 100644
+--- a/build/f_isoroot/f_odlpluginbuild/Makefile
++++ b/build/f_isoroot/f_odlpluginbuild/Makefile
+@@ -58,6 +58,7 @@ release:.odlbuild
+ 	fi
+ 	@sha1sum Makefile | awk {'print $$1'} >> .cachedata
+ 	@sha1sum config.mk | awk {'print $$1'} >> .cachedata
++	@echo -n $(UBUNTU_ARCH) | sha1sum | awk {'print $$1'} >> .cachedata
+ 	cat .cachedata | $(CACHETOOL) getid > .cacheid
+ 
+ # Clean local data related to caching - called prior to ordinary build
+diff --git a/build/f_isoroot/f_onosfwpluginbuild/Makefile b/build/f_isoroot/f_onosfwpluginbuild/Makefile
+index a9dcb22..02ba029 100644
+--- a/build/f_isoroot/f_onosfwpluginbuild/Makefile
++++ b/build/f_isoroot/f_onosfwpluginbuild/Makefile
+@@ -66,6 +66,7 @@ release:.onosbuild
+ 	fi
+ 	@sha1sum Makefile | awk {'print $$1'} >> .cachedata
+ 	@sha1sum config.mk | awk {'print $$1'} >> .cachedata
++	@echo -n $(UBUNTU_ARCH) | sha1sum | awk {'print $$1'} >> .cachedata
+ 	@cat .cachedata | $(CACHETOOL) getid > .cacheid
+ 
+ # Clean local data related to caching - called prior to ordinary build
+diff --git a/build/f_isoroot/f_ovs-nsh-dpdk-pluginbuild/Makefile b/build/f_isoroot/f_ovs-nsh-dpdk-pluginbuild/Makefile
+index ecf586a..2d0d202 100644
+--- a/build/f_isoroot/f_ovs-nsh-dpdk-pluginbuild/Makefile
++++ b/build/f_isoroot/f_ovs-nsh-dpdk-pluginbuild/Makefile
+@@ -67,6 +67,7 @@ release:.ovsbuild
+ 	fi
+ 	@sha1sum Makefile | awk {'print $$1'} >> .cachedata
+ 	@sha1sum config.mk | awk {'print $$1'} >> .cachedata
++	@echo -n $(UBUNTU_ARCH) | sha1sum | awk {'print $$1'} >> .cachedata
+ 	@cat .cachedata | $(CACHETOOL) getid > .cacheid
+ 
+ # Clean local data related to caching - called prior to ordinary build
+diff --git a/build/f_isoroot/f_ovsnfv-dpdk-pluginbuild/Makefile b/build/f_isoroot/f_ovsnfv-dpdk-pluginbuild/Makefile
+index 9e432d1..9455389 100644
+--- a/build/f_isoroot/f_ovsnfv-dpdk-pluginbuild/Makefile
++++ b/build/f_isoroot/f_ovsnfv-dpdk-pluginbuild/Makefile
+@@ -67,6 +67,7 @@ release:.ovsnfvbuild
+ 	fi
+ 	@sha1sum Makefile | awk {'print $$1'} >> .cachedata
+ 	@sha1sum config.mk | awk {'print $$1'} >> .cachedata
++	@echo -n $(UBUNTU_ARCH) | sha1sum | awk {'print $$1'} >> .cachedata
+ 	@cat .cachedata | $(CACHETOOL) getid > .cacheid
+ 
+ # Clean local data related to caching - called prior to ordinary build
+diff --git a/build/f_isoroot/f_repobuild/Makefile b/build/f_isoroot/f_repobuild/Makefile
+index 6fef8ff..f6ebc2c 100644
+--- a/build/f_isoroot/f_repobuild/Makefile
++++ b/build/f_isoroot/f_repobuild/Makefile
+@@ -73,6 +73,7 @@ release:nailgun
+ 	sha1sum Makefile >> .cachedata
+ 	sha1sum config.mk >> .cachedata
+ 	$(CACHETOOL) packages >> .cachedata
++	echo -n $(UBUNTU_ARCH) | sha1sum | awk {'print $$1'} >> .cachedata
+ 	cat .cachedata | $(CACHETOOL) getid > .cacheid
+ 
+ # Clean local data related to caching - called prior to ordinary build
+diff --git a/build/f_isoroot/f_vsperfpluginbuild/Makefile b/build/f_isoroot/f_vsperfpluginbuild/Makefile
+index 9bd236a..73de611 100644
+--- a/build/f_isoroot/f_vsperfpluginbuild/Makefile
++++ b/build/f_isoroot/f_vsperfpluginbuild/Makefile
+@@ -68,6 +68,7 @@ release:.vsperfbuild
+ 	fi
+ 	@sha1sum Makefile | awk {'print $$1'} >> .cachedata
+ 	@sha1sum config.mk | awk {'print $$1'} >> .cachedata
++	@echo -n $(UBUNTU_ARCH) | sha1sum | awk {'print $$1'} >> .cachedata
+ 	@cat .cachedata | $(CACHETOOL) getid > .cacheid
+ 
+ # Clean local data related to caching - called prior to ordinary build
+diff --git a/build/f_isoroot/f_yardstick-pluginbuild/Makefile b/build/f_isoroot/f_yardstick-pluginbuild/Makefile
+index 47723d5..32de3b3 100644
+--- a/build/f_isoroot/f_yardstick-pluginbuild/Makefile
++++ b/build/f_isoroot/f_yardstick-pluginbuild/Makefile
+@@ -68,6 +68,7 @@ release:.yardstickbuild
+ 	fi
+ 	@sha1sum Makefile | awk {'print $$1'} >> .cachedata
+ 	@sha1sum config.mk | awk {'print $$1'} >> .cachedata
++	@echo -n $(UBUNTU_ARCH) | sha1sum | awk {'print $$1'} >> .cachedata
+ 	@cat .cachedata | $(CACHETOOL) getid > .cacheid
+ 
+ # Clean local data related to caching - called prior to ordinary build
diff --git a/patches/opnfv-fuel/0046-build-Use-OPNFV_GIT_SHA-for-ISO-preparer-ID.patch b/patches/opnfv-fuel/0046-build-Use-OPNFV_GIT_SHA-for-ISO-preparer-ID.patch
new file mode 100644
index 00000000..e6d4dc21
--- /dev/null
+++ b/patches/opnfv-fuel/0046-build-Use-OPNFV_GIT_SHA-for-ISO-preparer-ID.patch
@@ -0,0 +1,48 @@
+From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+Date: Wed, 17 Aug 2016 21:56:22 +0200
+Subject: [PATCH] build: Use OPNFV_GIT_SHA for ISO preparer ID
+
+isoinfo -i lists the following information for Fuel@OPNFV ISO:
+"Data preparer id: 86aafaf5454a846c417848bb94f264c4420160f3"
+where the SHA hash is Fuel git repo HEAD SHA.
+
+However, Armband overrides OPNFV_GIT_SHA to the Armband git repo
+commit hash, so the ISO metadata should also reflect this.
+
+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+---
+ build/docker/runcontext  | 2 +-
+ build/install/install.sh | 6 +++++-
+ 2 files changed, 6 insertions(+), 2 deletions(-)
+
+diff --git a/build/docker/runcontext b/build/docker/runcontext
+index e4874df..07b0dc1 100755
+--- a/build/docker/runcontext
++++ b/build/docker/runcontext
+@@ -119,7 +119,7 @@ RUN_CONTEXT_OPT="--cidfile $CID_FILE --privileged=true --rm \
+     -e HOME=$HOME -e CACHEDEBUG -e CACHETRANSPORT -e CACHEMAXAGE -e CACHEBASE \
+     -e BUILD_FUEL_PLUGINS -e MIRROR_UBUNTU -e MIRROR_UBUNTU_ROOT \
+     -e MIRROR_MOS_UBUNTU -e MIRROR_MOS_UBUNTU_ROOT -e MIRROR_FUEL \
+-    -e LATEST_TARGET_UBUNTU -e UBUNTU_ARCH -e ARMBAND_BASE \
++    -e LATEST_TARGET_UBUNTU -e UBUNTU_ARCH -e ARMBAND_BASE -e OPNFV_GIT_SHA \
+     -u $USER_ID:$GROUP_ID -w $PWD \
+     -v $GITROOT:$GITROOT -v /sys/fs/cgroup:/sys/fs/cgroup:ro $CACHEMOUNT"
+ 
+diff --git a/build/install/install.sh b/build/install/install.sh
+index f6308fb..9e003bb 100755
+--- a/build/install/install.sh
++++ b/build/install/install.sh
+@@ -210,8 +210,12 @@ make_iso_image() {
+     find . -name TRANS.TBL -exec rm {} \;
+     rm -rf rr_moved
+ 
++    if [[ -z "$OPNFV_GIT_SHA" ]]; then
++        OPNFV_GIT_SHA=$(git rev-parse --verify HEAD)
++    fi
++
+     mkisofs --quiet -r -V "$VOLUMEID" -publisher "$PUBLISHER" \
+-        -p `git rev-parse --verify HEAD` -J -R -b isolinux/isolinux.bin \
++        -p "$OPNFV_GIT_SHA" -J -R -b isolinux/isolinux.bin \
+         -no-emul-boot \
+         -boot-load-size 4 -boot-info-table \
+         --hide-rr-moved \
diff --git a/patches/opnfv-fuel/0047-build-ubuntu-mirror-blacklist-ftp.acc.umu.se.patch b/patches/opnfv-fuel/0047-build-ubuntu-mirror-blacklist-ftp.acc.umu.se.patch
new file mode 100644
index 00000000..e6e6c2b7
--- /dev/null
+++ b/patches/opnfv-fuel/0047-build-ubuntu-mirror-blacklist-ftp.acc.umu.se.patch
@@ -0,0 +1,24 @@
+From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+Date: Thu, 18 Aug 2016 19:21:34 +0200
+Subject: [PATCH] build: ubuntu mirror: blacklist ftp.acc.umu.se
+
+Another SE mirror acting up lately ...
+
+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+---
+ build/f_isoroot/f_repobuild/select_ubuntu_repo.sh | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/build/f_isoroot/f_repobuild/select_ubuntu_repo.sh b/build/f_isoroot/f_repobuild/select_ubuntu_repo.sh
+index 4bca6bd..67219cb 100755
+--- a/build/f_isoroot/f_repobuild/select_ubuntu_repo.sh
++++ b/build/f_isoroot/f_repobuild/select_ubuntu_repo.sh
+@@ -21,7 +21,7 @@ RSYNC="rsync -4 --contimeout 5 --no-motd --list-only"
+ # Some Ubuntu mirrors seem less reliable for this type of mirroring -
+ # as they are discoved they can be added to the blacklist below in order
+ # for them not to be considered.
+-BLACKLIST="mirrors.se.eu.kernel.org mirror.its.dal.ca"
++BLACKLIST="mirrors.se.eu.kernel.org ftp.acc.umu.se mirror.its.dal.ca"
+ 
+ return_url=0
+