Uplift Armband to Fuel Newton

[ Dan Andresan ]
- puppet: database: Fix Percona XtraBackup sync

[ Alexandru Avadanii ]
- disable all plugins;
- re-enable remote tracking;
- remove "Revert: Point to specific snapshot ...";
- patch context adjustments;
- obsolete "kernel-bump", linux-image-lts-xenial is now the default;
- network-checker iface state check now fixed upstream [1];
- fuel-nailgun-agent hugepage size should also check /proc/meminfo;
- fuel-nailgun-agent CPU detection for AArch64;
- nova AArch64 hugepage support is now upstream;
- obsolete Cirros Test VM direct kernel boot (by switch to AAVMF):
  * f-l/0001-upload_cirros-Add-direct-kernel-boot-support.patch
  * f-w/0001-direct-kernel-boot-for-cirros.patch
- rework m1.micro RAM size patch after puppet manifest split upstream;
- re-enable arch-agnostic plugins which were rebased in Fuel@OPNFV:
  * f_yardstick-pluginbuild
  * f_congress-pluginbuild
- do NOT retire MySQL SST provider patch series (nack: ARMBAND-186),
  rebase (and keep for now) MySQL SST provider patches, as trying to
  use xtrabackup-v2 revelead a regression since Colorado.3.0, and
  these patches simplify troubleshooting a lot;
- AArch64: nova: libvirt: Use host-model cpu (ARMBAND-193);
- AArch64: nova: libvirt: Use pointer_model instead of use_usb_tablet;
- m1.micro RAM size insufficient for TestVM with AAVMF (s/128/256/)
- switch Cirros TestVM to AAVMF from direct kernel boot;
- backport nova libvirt driver fix for deleting instances booted with
  AAVMF firmware from [2];

TODO (later):
- Include ISO build time fixes for cirros_testvm in Armband package;

TODO (ODL, later):
- test & revise leveldb patching;
- bring back Qugga patching for arm64;
- configure systemd service to automatically respawn;

[1] https://review.openstack.org/#/c/417373/
[2] https://review.openstack.org/#/c/357190/

JIRA: ARMBAND-29
JIRA: ARMBAND-32
JIRA: ARMBAND-63
JIRA: ARMBAND-88
JIRA: ARMBAND-116
JIRA: ARMBAND-118
JIRA: ARMBAND-186
JIRA: ARMBAND-193
JIRA: ARMBAND-194
JIRA: ARMBAND-195
JIRA: ARMBAND-196
JIRA: ARMBAND-197

Change-Id: Ia99022e364e61245d109cabab9d0ed7157b4d2f5
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
Signed-off-by: Dan Andresan <dan.andresan@enea.com>

8 files changed, 35 insertions, 995 deletions

diff --git a/patches/opnfv-fuel/upstream-backports/0001-deploy.sh-Remove-check-for-root-rm-umask-0000.patch b/patches/opnfv-fuel/upstream-backports/0001-deploy.sh-Remove-check-for-root-rm-umask-0000.patch
index cd30166a..66515f65 100644
--- a/patches/opnfv-fuel/upstream-backports/0001-deploy.sh-Remove-check-for-root-rm-umask-0000.patch
+++ b/patches/opnfv-fuel/upstream-backports/0001-deploy.sh-Remove-check-for-root-rm-umask-0000.patch
@@ -52,18 +52,6 @@ index 8411714..081806c 100755
  clean
 
  pushd ${DEPLOY_DIR} > /dev/null
-diff --git a/deploy/deploy-config.py b/deploy/deploy-config.py
-index 2a09aa3..02debe2 100644
---- a/deploy/deploy-config.py
-+++ b/deploy/deploy-config.py
-@@ -41,7 +41,6 @@ from common import (
-     check_file_exists,
-     create_dir_if_not_exists,
-     delete,
--    check_if_root,
-     ArgParser,
- )
-
 diff --git a/deploy/deploy.py b/deploy/deploy.py
 index fe213e5..08702d2 100755
 --- a/deploy/deploy.py
diff --git a/patches/opnfv-fuel/upstream-backports/0002-Backport-dpkg-checkbuilddeps-to-mk-build-deps.patch b/patches/opnfv-fuel/upstream-backports/0002-Backport-dpkg-checkbuilddeps-to-mk-build-deps.patch
deleted file mode 100644
index eb81942e..00000000
--- a/patches/opnfv-fuel/upstream-backports/0002-Backport-dpkg-checkbuilddeps-to-mk-build-deps.patch
+++ /dev/null
@@ -1,58 +0,0 @@
-From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
-Date: Wed, 6 Jul 2016 17:03:49 +0200
-Subject: [PATCH] Backport: dpkg-checkbuilddeps to mk-build-deps.
-
-Backported from [1].
-
-[1] https://review.openstack.org/#/c/325210/
-
-Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
----
- ...ce-dpkg-checkbuilddeps-with-mk-build-deps.patch | 38 ++++++++++++++++++++++
- 1 file changed, 38 insertions(+)
- create mode 100644 build/f_repos/patch/fuel-main/0005-replace-dpkg-checkbuilddeps-with-mk-build-deps.patch
-
-diff --git a/build/f_repos/patch/fuel-main/0005-replace-dpkg-checkbuilddeps-with-mk-build-deps.patch b/build/f_repos/patch/fuel-main/0005-replace-dpkg-checkbuilddeps-with-mk-build-deps.patch
-new file mode 100644
-index 0000000..896f3cf
---- /dev/null
-+++ b/build/f_repos/patch/fuel-main/0005-replace-dpkg-checkbuilddeps-with-mk-build-deps.patch
-@@ -0,0 +1,38 @@
-+From: Sergii Golovatiuk <sgolovatiuk@mirantis.com>
-+Date: Fri, 3 Jun 2016 12:26:19 +0200
-+Subject: [PATCH] Replace dpkg-checkbuilddeps with mk-build-deps
-+
-+dpkg-checkbuilddeps doesn't check dependencies inside chroot as in many
-+cases host system might not have required packages. This change replaces
-+dpkg-checkbuilddeps with mk-build-deps in chroot
-+
-+Implements: mos-xenial
-+Change-Id: I27a65893c1bd33e6d82c11bf3d08423562dbe4d2
-+Signed-off-by: Sergii Golovatiuk <sgolovatiuk@mirantis.com>
-+---
-+ packages/deb/module.mk | 5 ++---
-+ 1 file changed, 2 insertions(+), 3 deletions(-)
-+
-+diff --git a/packages/deb/module.mk b/packages/deb/module.mk
-+index 6c93441..c0ba737 100644
-+--- a/packages/deb/module.mk
-++++ b/packages/deb/module.mk
-+@@ -10,7 +10,7 @@ clean-deb:
-+ 	done
-+ 	sudo rm -rf $(BUILD_DIR)/packages/deb
-+
-+-$(BUILD_DIR)/packages/deb/buildd.tar.gz: SANDBOX_DEB_PKGS:=wget bzip2 apt-utils build-essential python-setuptools python-pbr devscripts debhelper fakeroot
-++$(BUILD_DIR)/packages/deb/buildd.tar.gz: SANDBOX_DEB_PKGS:=wget bzip2 apt-utils build-essential fakeroot devscripts equivs debhelper python-setuptools python-pbr
-+ $(BUILD_DIR)/packages/deb/buildd.tar.gz: SANDBOX_UBUNTU:=$(BUILD_DIR)/packages/deb/chroot
-+ $(BUILD_DIR)/packages/deb/buildd.tar.gz: export SANDBOX_UBUNTU_UP:=$(SANDBOX_UBUNTU_UP)
-+ $(BUILD_DIR)/packages/deb/buildd.tar.gz: export SANDBOX_UBUNTU_DOWN:=$(SANDBOX_UBUNTU_DOWN)
-+@@ -47,8 +47,7 @@ $(BUILD_DIR)/packages/deb/$1.done: $(BUILD_DIR)/repos/repos.done
-+ 		sudo -E dch -c $$(SANDBOX_UBUNTU)/tmp/$1/debian/changelog -D $(UBUNTU_RELEASE) -b --force-distribution \
-+ 		-v $(PACKAGE_VERSION)-`awk -F'=' '/DEBRELEASE/ {print $$$$2}' $(BUILD_DIR)/packages/sources/$1/version` \
-+ 		"`awk -F'=' '/DEBMSG/ {print $$$$2}' $(BUILD_DIR)/packages/sources/$1/version`"
-+-	dpkg-checkbuilddeps $(BUILD_DIR)/repos/$1/debian/control 2>&1 | sed 's/^dpkg-checkbuilddeps: Unmet build dependencies: //g' | sed 's/([^()]*)//g;s/|//g' | sudo tee $$(SANDBOX_UBUNTU)/tmp/$1.installdeps
-+-	sudo chroot $$(SANDBOX_UBUNTU) /bin/sh -c "cat /tmp/$1.installdeps | xargs --no-run-if-empty env LC_ALL=C DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true apt-get -y install"
-++	sudo chroot $$(SANDBOX_UBUNTU) /bin/sh -c "mk-build-deps --install --remove --tool 'apt-get --yes --no-remove --no-install-recommends' /tmp/$1/debian/control"
-+ 	sudo chroot $$(SANDBOX_UBUNTU) /bin/sh -c "cd /tmp/$1 ; DEB_BUILD_OPTIONS=nocheck debuild -us -uc -b -d"
-+ 	cp $$(SANDBOX_UBUNTU)/tmp/*.deb $(BUILD_DIR)/packages/deb/packages
-+ 	sudo sh -c "$$$${SANDBOX_UBUNTU_DOWN}"
diff --git a/patches/opnfv-fuel/upstream-backports/0002-build-docker-Use-older-ffi-1.9.14.patch b/patches/opnfv-fuel/upstream-backports/0002-build-docker-Use-older-ffi-1.9.14.patch
new file mode 100644
index 00000000..ccc37cb3
--- /dev/null
+++ b/patches/opnfv-fuel/upstream-backports/0002-build-docker-Use-older-ffi-1.9.14.patch
@@ -0,0 +1,32 @@
+From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+Date: Fri, 13 Jan 2017 18:40:08 +0100
+Subject: [PATCH] build/docker: Use older ffi (1.9.14)
+
+`gem install fbm` tries to install the ffi gem too, which has a
+problem in newer versions (>= 1.9.15) - see [1]:
+"AbstractMemory.c:349:48: error: expected ')' before 'PRIsVALUE'"
+
+To bypass the newly introduced issues, use the old version.
+
+[1] https://jira.opnfv.org/browse/ARMBAND-191
+
+JIRA: ARMBAND-191
+
+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+---
+ build/docker/Dockerfile | 2 ++
+ 1 file changed, 2 insertions(+)
+
+diff --git a/build/docker/Dockerfile b/build/docker/Dockerfile
+index 04c4d2b..54aca35 100644
+--- a/build/docker/Dockerfile
++++ b/build/docker/Dockerfile
+@@ -22,6 +22,8 @@ RUN apt-get install -y software-properties-common python-software-properties \
+     ca-certificates sudo apt-utils lsb-release dosfstools debmirror p7zip-full \
+     build-essential ruby-dev rubygems-integration python-pip git rpm createrepo dpkg-dev
+
++# FIXME(armband): Newer ffi is broken - https://jira.opnfv.org/browse/ARMBAND-191
++RUN gem install ffi -v 1.9.14
+ RUN gem install fpm
+
+ # Temporary: fpb needs to be built from sources
diff --git a/patches/opnfv-fuel/upstream-backports/0003-f_repobuild-Use-packetary-to-build-partial-mirror.patch b/patches/opnfv-fuel/upstream-backports/0003-f_repobuild-Use-packetary-to-build-partial-mirror.patch
deleted file mode 100644
index e7036e16..00000000
--- a/patches/opnfv-fuel/upstream-backports/0003-f_repobuild-Use-packetary-to-build-partial-mirror.patch
+++ /dev/null
@@ -1,825 +0,0 @@
-From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
-Date: Wed, 6 Jul 2016 17:03:49 +0200
-Subject: [PATCH] f_repobuild: Use packetary to build partial mirror
-
-Packetary [1] will replace fuel-mirror in upstream Fuel.
-fuel-mirror is using packetary under the hood already, so the
-transition should not be too hard.
-
-Adapt OPNFV Ubuntu partial mirror build:
-- obsolete opnfv-config.yaml (packetary uses CLI args for this);
-- new opnfv_config.yaml will hold old ubuntu.yaml data, split at
-  build runtime into:
-  * mos_<arch>_mirror.yaml - MOS mirror cfg, only for deps analysis;
-  * ubuntu_<arch>_mirror.yaml - Ubuntu mirror cfg, for pkgs download;
-  * ubuntu_<arch>_packages.yaml - Ubuntu packages to download;
-
-While the consumers of our mirror still use `debootstrap`, which only
-supports the "main" repo component, make sure the "main" package set
-is complete, in the scope of:
-  * ubuntu_<arch>_mirror_main.yaml - Ubuntu [main only] mirror;
-  * ubuntu_<arch>_packages_main.yaml - Ubuntu packages for [main];
-
-New workflow (executed for EACH architecture):
-1. Collect bootstrap package deps from <fuel_bootstrap_cli.yaml>;
-2. Collect all fixture release pkgs from fuel-web's <openstack.yaml>;
-3. Add OPNFV extra pkg deps from "opnfv_config.yaml"
-   (based on old fuel-mirror)
-
-   Currently, this is based on old fuel-mirror's
-   <fuel-mirror/contrib/fuel_mirror/data/ubuntu.yaml>, holding a
-   hardcoded list of package deps; plus eventual extra OPNFV packages,
-   like additional kernels, bootloaders etc.
-   Since we now inspect MOS repos for deps with packetary, this old
-   list should be easy to throw away as soon as we confirm each item
-   is directly required by another, already existing package in the
-   mirror. For now, keep the old list as safety net for missing deps.
-
-   NOTE: Dependencies up to and including this group should also
-   be satisfiable from Ubuntu "main" repo component only.
-
-4. Append plugin dependencies to the package list;
-
-5. Run `packetary unresolved` for MOS repos, gathering MOS deps that
-   should be fetched from Ubuntu. The new packages will be appended
-   to the predefined list from old "ubuntu.yaml";
-
-6. Run `packetary clone` to download all required pkgs for the
-   partial Ubuntu mirror.
-
-7. IF repo component merging is disabled, run `packetary clone` only
-   on Ubuntu[main] repo components to download any (still) missing
-   dependencies for `debootstrap`, which only looks at this component,
-   and not the whole mirror.
-
-8. IF repo component merging is enabled, run first `dpkg-scanpackages`
-   to filter out old duplicate versions of packages, then run
-   `packetary create` to merge all repo comps into a single "main".
-
-v1 -> v2:
-- multiarch support (activated via UBUNTU_ARCH envvar);
-- fixed debootstrap missing deps in "main" component of Ubuntu mirror;
-- factored out some hardcodes into variables/functions;
-- add .gitignore file;
-- move generated config files to "opnfv_config" subdir;
-- added arm64 Ubuntu repo URL (ports.ubuntu.com is separated from the
-  main Ubuntu mirror, archive.ubuntu.com, which only holds x86 pkgs);
-
-v3 -> v4:
-- introduce env var "MIRROR_UBUNTU_MERGE" to control local mirror
-  repo-component merging into a single "main";
-- enable mirror repo-component merging by default, edit
-  fuel_bootstrap_cli.yaml accordingly;
-
-v4 -> v6:
-- edit dea_base to use only main mirror;
-- fix duplicated logs;
-- remove `-q` flag for `create`;
-
-v6 -> v7:
-- rebase for Newton fuel_bootstrap_cli.yaml (s/trusty/xenial/);
-- include OPNFV plugin deps in local Ubuntu mirror;
-- add `vgabios` dependency, required for arm64 guest firmware;
-- remove all fuel-mirror references;
-- merge old fuel-mirror pkglist to OPNFV pkglist;
-- update packetary commit to include upstream bugfixes [2, 3];
-
-NOTE: Without filtering out old package verisons using
-`dpkg-scanpackages`, bootstrap build cannot solve all deps.
-
-FIXME: Packetary solves missing dependecies by also accepting
-different packages that provide the same package name (e.g. "ifupdown"
-is satisfied by "netscript" package).
-
-FIXME: Repo component merging is sort of slow, since packetary insists
-on copying the source files to the destination dir even if they are
-pointing to the same repo.
-
-FIXME: Packetary `create` uses a different directory scheme for the
-created mirror than the upstream Ubuntu, e.g.:
-[p] mirrors/ubuntu/pool/main/l/lvm2_2.02.98-6ubuntu2_amd64.deb
-[U] mirrors/ubuntu/pool/main/l/lvm2/lvm2_2.02.98-6ubuntu2_amd64.deb
-
-This disencourages creating the "merged" mirror in the same location
-as the source partial mirror.
-
-[1] https://wiki.openstack.org/wiki/Packetary
-[2] https://review.openstack.org/#/c/392936/
-[3] https://review.openstack.org/#/c/392937/
-
-JIRA: FUEL-218
-JIRA: FUEL-223
-JIRA: ARMBAND-169
-JIRA: ARMBAND-185
-
-Change-Id: If2ee86f348b7683c83bfaf686baba4f1b8f555f0
-Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
----
- build/config.mk                                    |   4 +
- build/f_isoroot/f_repobuild/.gitignore             |   4 +
- build/f_isoroot/f_repobuild/Makefile               |  37 ++-
- build/f_isoroot/f_repobuild/config.mk              |  23 ++
- .../f_isoroot/f_repobuild/fuel_bootstrap_cli.yaml  |  22 +-
- build/f_isoroot/f_repobuild/opnfv_config.yaml      | 160 ++++++++++++
- build/f_isoroot/f_repobuild/opnfv_mirror_conf.py   |  57 -----
- build/f_isoroot/f_repobuild/opnfv_mirror_ubuntu.py | 277 +++++++++++++++++++++
- deploy/config/dea_base.yaml                        |  16 +-
- 9 files changed, 492 insertions(+), 108 deletions(-)
- create mode 100644 build/f_isoroot/f_repobuild/.gitignore
- create mode 100644 build/f_isoroot/f_repobuild/config.mk
- create mode 100644 build/f_isoroot/f_repobuild/opnfv_config.yaml
- delete mode 100755 build/f_isoroot/f_repobuild/opnfv_mirror_conf.py
- create mode 100755 build/f_isoroot/f_repobuild/opnfv_mirror_ubuntu.py
-
-diff --git a/build/config.mk b/build/config.mk
-index 56317cb..5448c5e 100644
---- a/build/config.mk
-+++ b/build/config.mk
-@@ -17,4 +17,9 @@ FUEL_MAIN_TAG = master
- FUEL_TRACK_REMOTES =
-+
-+# List of space-separated Ubuntu architectures supported with current build
-+# Format: same as `dpkg-architecture -qDEB_HOST_ARCH`
-+# NOTE: Currently only amd64 is supported by Fuel@OPNFV. Armband adds arm64.
-+export UBUNTU_ARCH ?= amd64
-
- ##############################################################################
- # Fuel components pinning / remote tracking; use submodules from f_repos
-diff --git a/build/f_isoroot/f_repobuild/.gitignore b/build/f_isoroot/f_repobuild/.gitignore
-new file mode 100644
-index 0000000..09baca8
---- /dev/null
-+++ b/build/f_isoroot/f_repobuild/.gitignore
-@@ -0,0 +1,4 @@
-+packetary
-+fuel-web
-+nailgun
-+opnfv_config
-diff --git a/build/f_isoroot/f_repobuild/Makefile b/build/f_isoroot/f_repobuild/Makefile
-index 8beb882..9784edf 100644
---- a/build/f_isoroot/f_repobuild/Makefile
-+++ b/build/f_isoroot/f_repobuild/Makefile
-@@ -1,7 +1,8 @@
- ##############################################################################
--# Copyright (c) 2015 Ericsson AB and others.
-+# Copyright (c) 2015,2016 Ericsson AB, Enea AB and others.
- # stefan.k.berg@ericsson.com
- # jonas.bjurel@ericsson.com
-+# Alexandru.Avadanii@enea.com
- # All rights reserved. This program and the accompanying materials
- # are made available under the terms of the Apache License, Version 2.0
- # which accompanies this distribution, and is available at
-@@ -10,41 +11,37 @@
-
- SHELL := /bin/bash
- TOP := $(shell pwd)
--TMP_ROOT_DIR := $(shell echo "$(MIRROR_UBUNTU_ROOT)" | cut -d "/" -f2)
-
- include ../../config.mk
-+include config.mk
-
- export MOS_VERSION
--export OPENSTACK_VERSION
-+export MIRROR_UBUNTU_OPNFV_PATH:=$(TOP)/nailgun/mirrors/ubuntu
-
- .PHONY: all
- all: nailgun
-
- nailgun:
--	sudo apt-get install -y git libxml2-dev libxslt-dev python-dev  python-pip libz-dev libyaml-dev createrepo python-yaml
--	rm -Rf nailgun
--	sudo mkdir -p /var/www/nailgun
--	ln -sf ${F_SUBMOD_DIR}/fuel-mirror fuel-mirror
--	sudo pip install -U -r ./fuel-mirror/requirements.txt
--	sudo pip install ./fuel-mirror
--	sudo pip install ./fuel-mirror/contrib/fuel_mirror
--	./opnfv_mirror_conf.py
--	sudo fuel-mirror --debug --config ./opnfv-config.yaml create --group ubuntu --pattern=ubuntu
--	sudo chmod -R 755 /var/www/nailgun
--	cp -Rp /var/www/nailgun .
--	# In the end we want to have ubuntu repository in mirrors/ubuntu directory
--	-if [ "$(MIRROR_UBUNTU_ROOT)" != "/ubuntu/" ]; then \
--	  mkdir -p nailgun/mirrors/ubuntu;\
--	  mv nailgun/mirrors$(MIRROR_UBUNTU_ROOT)* nailgun/mirrors/ubuntu;\
--	  [ "$(MIRROR_UBUNTU_ROOT)" != "/" ] && rm -rf nailgun/mirrors/$(TMP_ROOT_DIR);\
-+	sudo apt-get install -y createrepo git libxml2-dev libxslt1-dev \
-+		python-dev zlib1g-dev
-+	rm -Rf nailgun packetary opnfv_config && mkdir opnfv_config
-+	# We will analyze fuel-web's fixture files for package lists
-+	ln -sf ${F_SUBMOD_DIR}/fuel-web fuel-web
-+	git clone --quiet $(PACKETARY_REPO)
-+	if [ -n $(PACKETARY_COMMIT) ]; then \
-+		git -C packetary checkout $(PACKETARY_COMMIT); \
- 	fi
-+	sudo pip install -U -r ./packetary/requirements.txt
-+	sudo pip install ./packetary
-+	# Handle config and mirror build in one place
-+	./opnfv_mirror_ubuntu.py
- 	# Store artifact in cache straight away if caching is enabled
- 	# (no .cacheid will be present unless this is a cached build)
- 	test -f .cacheid && $(MAKE) -f Makefile put-cache || exit 0
-
- .PHONY: clean
- clean:
--	@rm -rf ../release/opnfv/nailgun nailgun fuel-mirror opnfv-config.yaml ubuntu.yaml
-+	@rm -rf ../release/opnfv/nailgun nailgun packetary fuel-mirror opnfv_config
-
- .PHONY: release
- release:nailgun
-diff --git a/build/f_isoroot/f_repobuild/config.mk b/build/f_isoroot/f_repobuild/config.mk
-new file mode 100644
-index 0000000..9852093
---- /dev/null
-+++ b/build/f_isoroot/f_repobuild/config.mk
-@@ -0,0 +1,23 @@
-+##############################################################################
-+# Copyright (c) 2016 Ericsson AB, Enea AB and others.
-+# stefan.k.berg@ericsson.com
-+# jonas.bjurel@ericsson.com
-+# Alexandru.Avadanii@enea.com
-+# All rights reserved. This program and the accompanying materials
-+# are made available under the terms of the Apache License, Version 2.0
-+# which accompanies this distribution, and is available at
-+# http://www.apache.org/licenses/LICENSE-2.0
-+##############################################################################
-+
-+# Use a recent master commit, since tags/branches are not yet mature
-+export PACKETARY_REPO?=https://github.com/openstack/packetary
-+export PACKETARY_COMMIT?=3021c001561b4baef352bf0b881d064ac687cc20
-+
-+# arm64 Ubuntu mirror is separated from archive.ubuntu.com
-+export MIRROR_UBUNTU_URL_arm64=http://ports.ubuntu.com/ubuntu-ports/
-+export MIRROR_UBUNTU_ROOT_arm64=ubuntu-ports
-+
-+# Merge all local mirror repo components/section into single "main"
-+# NOTE: When changing this, make sure to also update all consumer config, like:
-+# - fuel_bootstrap_cli.yaml
-+export MIRROR_UBUNTU_MERGE=true
-diff --git a/build/f_isoroot/f_repobuild/fuel_bootstrap_cli.yaml b/build/f_isoroot/f_repobuild/fuel_bootstrap_cli.yaml
-index fcf4257..7cbea29 100644
---- a/build/f_isoroot/f_repobuild/fuel_bootstrap_cli.yaml
-+++ b/build/f_isoroot/f_repobuild/fuel_bootstrap_cli.yaml
-@@ -47,22 +47,10 @@
-   https_proxy: ""
-   repos:
-     - name: ubuntu
--      section: "main universe multiverse"
-+      section: "main"
-       uri: "http://127.0.0.1:8080/mirrors/ubuntu"
-       priority:
-       suite: trusty
--      type: deb
--    - name: ubuntu-updates
--      section: "main universe multiverse"
--      uri: "http://127.0.0.1:8080/mirrors/ubuntu"
--      priority:
--      suite: trusty-updates
--      type: deb
--    - name: ubuntu-security
--      section: "main universe multiverse"
--      uri: "http://127.0.0.1:8080/mirrors/ubuntu"
--      priority:
--      suite: trusty-security
-       type: deb
-     - name: mos
-       section: "main restricted"
-diff --git a/build/f_isoroot/f_repobuild/opnfv_config.yaml b/build/f_isoroot/f_repobuild/opnfv_config.yaml
-new file mode 100644
-index 0000000..bb78747
---- /dev/null
-+++ b/build/f_isoroot/f_repobuild/opnfv_config.yaml
-@@ -0,0 +1,160 @@
-+##############################################################################
-+# Copyright (c) 2016 Enea AB and others.
-+# Alexandru.Avadanii@enea.com
-+# All rights reserved. This program and the accompanying materials
-+# are made available under the terms of the Apache License, Version 2.0
-+# which accompanies this distribution, and is available at
-+# http://www.apache.org/licenses/LICENSE-2.0
-+##############################################################################
-+
-+# REPO definitions
-+
-+# Based on old fuel-mirror data [1], adapted for OPNFV and packetary.
-+# NOTE: 'uri' field will be added by opnfv_mirror_ubuntu.py from ENV.
-+# [1] https://github.com/openstack/fuel-mirror/blob/stable/mitaka/
-+#     contrib/fuel_mirror/data/ubuntu.yaml
-+
-+# Main is a required parameter which defines what repository will be used
-+# for images creation and that mirror should contain all packages for minimal
-+# system creation.
-+groups:
-+  ubuntu:
-+    - name: "ubuntu"
-+      main: true
-+      suite: "trusty"
-+      section:
-+        - "main"
-+        - "multiverse"
-+        - "restricted"
-+        - "universe"
-+      type: "deb"
-+      priority: null
-+
-+    - name: "ubuntu-updates"
-+      suite: "trusty-updates"
-+      section:
-+        - "main"
-+        - "multiverse"
-+        - "restricted"
-+        - "universe"
-+      type: "deb"
-+      priority: null
-+
-+    - name: "ubuntu-security"
-+      suite: "trusty-security"
-+      section:
-+        - "main"
-+        - "multiverse"
-+        - "restricted"
-+        - "universe"
-+      type: "deb"
-+      priority: null
-+
-+  mos:
-+    - name: "mos"
-+      suite: "mos$mos_version"
-+      section:
-+        - "main"
-+        - "restricted"
-+      type: "deb"
-+      priority: 1000
-+
-+    - name: "mos-updates"
-+      suite: "mos$mos_version-updates"
-+      section:
-+        - "main"
-+        - "restricted"
-+      type: "deb"
-+      priority: 1000
-+
-+    - name: "mos-security"
-+      suite: "mos$mos_version-security"
-+      section:
-+        - "main"
-+        - "restricted"
-+      type: "deb"
-+      priority: 1000
-+
-+    - name: "mos-holdback"
-+      suite: "mos$mos_version-holdback"
-+      section:
-+        - "main"
-+        - "restricted"
-+      type: "deb"
-+      priority: 1000
-+
-+# PACKAGES
-+
-+# This section lists packages that should be present in the local Ubuntu mirror,
-+# but are not direct dependencies of any other packages from  MOS or Ubuntu.
-+# e.g.: additional kernels, bootloaders etc.
-+
-+packages:
-+  - name: "vgabios"
-+# Packages are required to build bootstrap images for a system.
-+# The mirror should contiain such packages in addition to local mirror.
-+  - name: "acpi-support"
-+  - name: "anacron"
-+  - name: "aptitude"
-+  - name: "atop"
-+  - name: "acct"
-+  - name: "bash-completion"
-+  - name: "bc"
-+  - name: "build-essential"
-+  - name: "cloud-init"
-+  - name: "conntrackd"
-+  - name: "cpu-checker"
-+  - name: "cpufrequtils"
-+  - name: "debconf-utils"
-+  - name: "devscripts"
-+  - name: "fping"
-+  - name: "git"
-+  - name: "grub-pc"
-+  - name: "htop"
-+  - name: "hwloc"
-+  - name: "ifenslave"
-+  - name: "iperf"
-+  - name: "iptables-persistent"
-+  - name: "irqbalance"
-+  - name: "language-pack-en"
-+  - name: "libapache2-mod-fastcgi"
-+  - name: "libnss3-tools"
-+  - name: "linux-firmware-nonfree"
-+  - name: "linux-headers-generic-lts-trusty"
-+  - name: "linux-headers-generic-lts-xenial"
-+  - name: "linux-image-generic-lts-trusty"
-+  - name: "linux-image-generic-lts-xenial"
-+  - name: "live-boot"
-+  - name: "livecd-rootfs"
-+  - name: "mc"
-+  - name: "memcached"
-+  - name: "monit"
-+  - name: "msmtp-mta"
-+  - name: "multipath-tools"
-+  - name: "multipath-tools-boot"
-+  - name: "nginx"
-+  - name: "ntp"
-+  - name: "openssh-server"
-+  - name: "percona-toolkit"
-+  - name: "percona-xtrabackup"
-+  - name: "pm-utils"
-+  - name: "puppet"
-+  - name: "python-lesscpy"
-+  - name: "python-pip"
-+  - name: "rsyslog-gnutls"
-+  - name: "rsyslog-relp"
-+  - name: "screen"
-+  - name: "squashfs-tools"
-+  - name: "swift-plugin-s3"
-+  - name: "sysfsutils"
-+  - name: "sysstat"
-+  - name: "telnet"
-+  - name: "tmux"
-+  - name: "traceroute"
-+  - name: "ubuntu-standard"
-+  - name: "vim"
-+  - name: "virt-what"
-+  - name: "xinetd"
-+  - name: "xmlstarlet"
-+  - name: "tftpd-hpa"
-+  - name: "syslinux"
-diff --git a/build/f_isoroot/f_repobuild/opnfv_mirror_conf.py b/build/f_isoroot/f_repobuild/opnfv_mirror_conf.py
-deleted file mode 100755
-index 1c7eb59..0000000
---- a/build/f_isoroot/f_repobuild/opnfv_mirror_conf.py
-+++ /dev/null
-@@ -1,57 +0,0 @@
--#!/usr/bin/env python
--##############################################################################
--# Copyright (c) 2015 Ericsson AB and others.
--# mskalski@mirantis.com
--# All rights reserved. This program and the accompanying materials
--# are made available under the terms of the Apache License, Version 2.0
--# which accompanies this distribution, and is available at
--# http://www.apache.org/licenses/LICENSE-2.0
--##############################################################################
--import os
--import yaml
--
--current_snapshot =  os.environ["LATEST_TARGET_UBUNTU"]
--mos_version = os.environ['MOS_VERSION']
--openstack_version = os.environ['OPENSTACK_VERSION']
--mos_ubuntu = os.environ['MIRROR_MOS_UBUNTU']
--mos_ubuntu_root = os.environ['MIRROR_MOS_UBUNTU_ROOT']
--mirror_ubuntu = os.environ['MIRROR_UBUNTU_URL']
--if os.environ.get('BUILD_FUEL_PLUGINS'):
--  plugins = os.environ['BUILD_FUEL_PLUGINS']
--else:
--  plugins = os.environ['PLUGINS']
--
--
--configuration_file = open('fuel-mirror/contrib/fuel_mirror/etc/config.yaml').read()
--conf = yaml.load(configuration_file)
--conf['pattern_dir'] = '.'
--conf['openstack_version'] = openstack_version
--conf['mos_version'] = mos_version
--
--with open('opnfv-config.yaml', 'w') as outfile:
--  outfile.write( yaml.dump(conf, default_flow_style=False) )
--
--pattern_file = open('fuel-mirror/contrib/fuel_mirror/data/ubuntu.yaml').read()
--pattern = yaml.load(pattern_file)
--pattern['mos_baseurl'] = "http://{}{}".format(mos_ubuntu, mos_ubuntu_root)
--pattern['ubuntu_baseurl'] = mirror_ubuntu
--for group in pattern['groups']['mos']:
--  group['uri'] = pattern['mos_baseurl']
--for group in pattern['groups']['ubuntu']:
--  group['uri'] = pattern['ubuntu_baseurl']
--
--for plugin in plugins.split():
--  path = "../{}/packages.yaml".format(plugin)
--  if os.path.isfile(path):
--    f = open(path).read()
--    plugin_yaml = yaml.load(f)
--    plugin_set = set(plugin_yaml['packages'])
--    main_set = set(pattern['packages'])
--    new_packages = plugin_set - main_set
--    print "Plugin {} require new packages: {}".format(plugin, ', '.join(new_packages))
--    pattern['packages'] = pattern['packages'] + list(new_packages)
--
--pattern['requirements']['ubuntu'] = pattern['packages']
--
--with open('ubuntu.yaml', 'w') as outfile:
--  outfile.write( yaml.safe_dump(pattern, default_flow_style=False) )
-diff --git a/build/f_isoroot/f_repobuild/opnfv_mirror_ubuntu.py b/build/f_isoroot/f_repobuild/opnfv_mirror_ubuntu.py
-new file mode 100755
-index 0000000..5b59182
---- /dev/null
-+++ b/build/f_isoroot/f_repobuild/opnfv_mirror_ubuntu.py
-@@ -0,0 +1,277 @@
-+#!/usr/bin/env python
-+##############################################################################
-+# Copyright (c) 2015,2016 Ericsson AB, Mirantis Inc., Enea AB and others.
-+# mskalski@mirantis.com
-+# Alexandru.Avadanii@enea.com
-+# All rights reserved. This program and the accompanying materials
-+# are made available under the terms of the Apache License, Version 2.0
-+# which accompanies this distribution, and is available at
-+# http://www.apache.org/licenses/LICENSE-2.0
-+##############################################################################
-+
-+##############################################################################
-+# Build multiarch partial local Ubuntu mirror
-+##############################################################################
-+# Design quirks / workarounds:
-+# 1. Fuel-agent uses `debootstrap` to build bootstrap and target chroots from
-+#    the local mirror; which only uses the "main" component from the first
-+#    repository, i.e. does not include "updates"/"security".
-+#    In order to fullfill all debootstrap dependencies in "main" repo, we will
-+#    do an extra packetary run using a reduced scope:
-+#    - only "main" component of the first mirror;
-+#    - reduced package dependency list (without MOS/OPNFV plugin deps).
-+# 2. If repo structure is not mandatory to be in sync with official mirrors,
-+#    we can mitigate the issue by "merging" all repo-components into a single
-+#    "main".
-+##############################################################################
-+# Mirror build steps (for EACH architecture in UBUNTU_ARCH):
-+# 1. Collect bootstrap package deps from <fuel_bootstrap_cli.yaml>;
-+# 2. Collect all fixture release packages from fuel-web's <openstack.yaml>;
-+# 3. Parse new "opnfv_config.yaml" list of packages (from old fuel-mirror);
-+# 4. Inherit enviroment variable(s) for mirror URLs, paths etc.
-+#    - Allow arch-specific overrides for each env var;
-+# 5. Mirror config is defined based on common config + OPNFV overrides;
-+#    - Convert old configuration format to packetary style where needed;
-+# 6. Package lists are defined based on common config + OPNFV deps;
-+#    - Keep track of "main" packages separately, required by debootstrap;
-+# 7. Clone/update all mirror components;
-+# 8. IF mirror merging is disabled OR workaround for ifupdown (see below):
-+#    - Clone/update "main" mirror component (fix missing debootstrap deps);
-+# 9. IF mirror merging is enabled:
-+#    - Use `dpkg-scanpackages` to filter out old versions of duplicate pkgs;
-+#    - Run `packetary create` on the set of downloaded packages, merging
-+#      them on the fly into a single-component mirror;
-+##############################################################################
-+
-+import copy
-+import os
-+import shutil
-+import sys
-+import yaml
-+from contextlib import contextmanager
-+from cStringIO import StringIO
-+from packetary.cli.app import main
-+
-+@contextmanager
-+def captureStdOut(output):
-+    stdout = sys.stdout
-+    sys.stdout = output
-+    yield
-+    sys.stdout = stdout
-+
-+# FIXME: Find a better approach for eliminating duplicate logs than this
-+def force_logger_reload():
-+  for mod in sys.modules.keys():
-+    if mod.startswith('logging'):
-+      try:
-+        reload(sys.modules[mod])
-+      except:
-+        pass
-+
-+# Determine missing package dependecies for a mirror defition
-+def get_unres_pkgs(arch, cfg_mirror):
-+  unresolved_pkgs = list()
-+  packetary_output = StringIO()
-+  with captureStdOut(packetary_output):
-+    main('unresolved -a {0} -r {1} -c name version --sep ;'
-+      .format(_ARCHITECTURES[arch], cfg_mirror).split(' '))
-+  for dep_pkg in packetary_output.getvalue().splitlines():
-+    if dep_pkg.startswith('#'):
-+      continue
-+    dep = dep_pkg.split(';')
-+    unresolved_pkgs += [ {'name': dep[0], 'version': dep[1]} ]
-+  force_logger_reload()
-+  return unresolved_pkgs
-+
-+# Package list conversion from `old fuel-mirror` to `packetary` style
-+def from_legacy_pkglist(legacy_pkglist):
-+  pkglist = list()
-+  for pkg in legacy_pkglist:
-+    pkglist += [ {'name': pkg} ]
-+  return pkglist
-+
-+def to_legacy_pkglist(pkglist):
-+  legacy_pkglist = list()
-+  for pkg in pkglist:
-+    legacy_pkglist.append(pkg['name'])
-+  return legacy_pkglist
-+
-+def legacy_diff(base_pkglist, new_pkglist, requester, arch):
-+  diff_set = set(new_pkglist)
-+  if base_pkglist:
-+    diff_set -= set(base_pkglist)
-+  if diff_set:
-+    print(' * {0} requires new packages for arch [{1}]: {2}'
-+          .format(requester, arch, ', '.join(diff_set)))
-+  return list(diff_set)
-+
-+# Create single-component local repo (one arch per call)
-+def do_local_repo(arch, cfg_repo, cfg_packages_paths):
-+  # Packetary does not use a global config file, so pass old settings here.
-+  main('create -t deb -a {0} --repository {1} --package-files {2}'
-+    ' --ignore-errors-num 2 --retries-num 3 --threads-num 10'
-+    .format(_ARCHITECTURES[arch], cfg_repo, cfg_packages_paths).split(' '))
-+  force_logger_reload()
-+
-+# Clone partial local mirror (one arch per call)
-+def do_partial_mirror(arch, cfg_mirror, cfg_packages):
-+  # Note: '-d .' is ignored, as each mirror defines its own path.
-+  main('clone -t deb -a {0} -r {1} -R {2} -d .'
-+    ' --ignore-errors-num 2 --retries-num 3 --threads-num 10'
-+    .format(_ARCHITECTURES[arch], cfg_mirror, cfg_packages).split(' '))
-+  force_logger_reload()
-+
-+# Write configuration (yaml) file (package list / mirror defition)
-+def write_cfg_file(cfg_mirror, data):
-+  with open(cfg_mirror, 'w') as outfile:
-+    outfile.write( yaml.safe_dump(data, default_flow_style=False) )
-+
-+# Allow arch-specific overrides of env vars
-+def get_env(env_var, arch=None):
-+  if arch:
-+    env_var_arch = '{0}_{1}'.format(env_var, arch)
-+    if os.environ.get(env_var_arch):
-+      return os.environ[env_var_arch]
-+  if os.environ.get(env_var):
-+    return os.environ[env_var]
-+  return None
-+
-+# Architecture name mapping (dpkg:packetary) for packetary CLI invocation
-+_ARCHITECTURES = {
-+    "i386": "i386",
-+    "amd64": "x86_64",
-+    "arm64": "aarch64",
-+}
-+
-+# Arch-indepedent configuration (old fuel-mirror + OPNFV extra packages)
-+cfg_dir = 'opnfv_config'
-+cfg_p_opnfv = 'opnfv_config.yaml'
-+mos_version = get_env('MOS_VERSION')
-+ubuntu_arch = get_env('UBUNTU_ARCH')
-+mirror_ubuntu_path = get_env('MIRROR_UBUNTU_OPNFV_PATH')
-+mirror_ubuntu_tmp_path = '{0}.tmp'.format(mirror_ubuntu_path)
-+mirror_ubuntu_merge = get_env('MIRROR_UBUNTU_MERGE')
-+cfg_mm_ubuntu = '{0}/ubuntu_mirror_local.yaml'.format(cfg_dir)
-+fuel_bootstrap_cli_file = open('fuel_bootstrap_cli.yaml').read()
-+fuel_bootstrap_cli = yaml.load(fuel_bootstrap_cli_file)
-+fixture_file = open('fuel-web/nailgun/nailgun/fixtures/openstack.yaml').read()
-+fixture = yaml.load(fixture_file)
-+opnfv_cfg_yaml = open(cfg_p_opnfv).read()
-+opnfv_cfg = yaml.load(opnfv_cfg_yaml)
-+
-+# FIXME: Packetary solves missing dependecies by also accepting
-+# different packages that provide the same name (e.g. "ifupdown" dependency
-+# is satisfied by "netscript" package from "universe" repo-component).
-+# Work around this by resolving all deps in "main" repo-component,
-+# then scan and keep only latest debs for the whole <merged> repo.
-+mirror_ubuntu_resolve_main_deps = True
-+
-+# Create local partial mirror using packetary, one arch at a time
-+for arch in ubuntu_arch.split(' '):
-+  # Mirror / Package env vars, arch-overrideable
-+  mos_ubuntu = get_env('MIRROR_MOS_UBUNTU', arch)
-+  mos_ubuntu_root = get_env('MIRROR_MOS_UBUNTU_ROOT', arch)
-+  mirror_ubuntu = get_env('MIRROR_UBUNTU_URL', arch)
-+  plugins = get_env('BUILD_FUEL_PLUGINS', arch)
-+  if plugins is None:
-+    plugins = get_env('PLUGINS', arch)
-+
-+  # Mirror / Package list configuration files (arch-specific)
-+  cfg_m_mos = '{0}/mos_{1}_mirror.yaml'.format(cfg_dir, arch)
-+  cfg_m_ubuntu = '{0}/ubuntu_{1}_mirror.yaml'.format(cfg_dir, arch)
-+  cfg_p_ubuntu = '{0}/ubuntu_{1}_packages.yaml'.format(cfg_dir, arch)
-+  cfg_m_ubuntu_main = '{0}/ubuntu_{1}_mirror_main.yaml'.format(cfg_dir, arch)
-+  cfg_p_ubuntu_main = '{0}/ubuntu_{1}_packages_main.yaml'.format(cfg_dir, arch)
-+
-+  # Mirror config fork before customizing (arch-specific)
-+  arch_group_mos = 'mos_{0}'.format(arch)
-+  arch_group_ubuntu = 'ubuntu_{0}'.format(arch)
-+  arch_packages = 'packages_{0}'.format(arch)
-+  opnfv_cfg['groups'][arch_group_mos] = copy.deepcopy(opnfv_cfg['groups']['mos'])
-+  opnfv_cfg['groups'][arch_group_ubuntu] = copy.deepcopy(opnfv_cfg['groups']['ubuntu'])
-+  opnfv_cfg[arch_packages] = opnfv_cfg['packages']
-+
-+  # Mirror config update & conversion to packetary input
-+  group_main_ubuntu = dict()
-+  for group in opnfv_cfg['groups'][arch_group_mos]:
-+    group['uri'] = "http://{}{}".format(mos_ubuntu, mos_ubuntu_root)
-+    group['suite'] = group['suite'].replace('$mos_version', mos_version)
-+  for group in opnfv_cfg['groups'][arch_group_ubuntu]:
-+    group['uri'] = mirror_ubuntu
-+    # FIXME: At `create`, packetary insists on copying all pkgs to dest dir,
-+    # so configure it for another dir, which will replace the orig at the end.
-+    group['path'] = mirror_ubuntu_tmp_path
-+    if not group_main_ubuntu and 'main' in group:
-+      group_main_ubuntu = [ copy.deepcopy(group) ]
-+      group_main_ubuntu[0]['section'] = [ 'main' ]
-+
-+  # Mirror config dump: MOS (for dep resolution), Ubuntu, Ubuntu[main]
-+  write_cfg_file(cfg_m_mos, opnfv_cfg['groups'][arch_group_mos])
-+  write_cfg_file(cfg_m_ubuntu, opnfv_cfg['groups'][arch_group_ubuntu])
-+  if mirror_ubuntu_resolve_main_deps or mirror_ubuntu_merge is None:
-+    write_cfg_file(cfg_m_ubuntu_main, group_main_ubuntu)
-+  if mirror_ubuntu_merge is not None:
-+    # FIXME: For multiarch, only one dump would be enough
-+    group_main_ubuntu[0]['origin'] = 'Ubuntu'
-+    group_main_ubuntu[0]['path'] = mirror_ubuntu_path
-+    group_main_ubuntu[0]['uri'] = mirror_ubuntu_path
-+    write_cfg_file(cfg_mm_ubuntu, group_main_ubuntu[0])
-+
-+  # Collect package dependencies from:
-+  ## 1. fuel_bootstrap_cli.yaml (bootstrap image additional packages)
-+  legacy_unresolved = legacy_diff(None, fuel_bootstrap_cli['packages'] + [
-+      fuel_bootstrap_cli['kernel_flavor'],
-+      fuel_bootstrap_cli['kernel_flavor'].replace('image', 'headers')],
-+    'Bootstrap', arch)
-+  ## 2. openstack.yaml fixture definition (default target image packages)
-+  for release in fixture:
-+    editable = release['fields']['attributes_metadata']['editable']
-+    if 'provision' in editable and 'packages' in editable['provision']:
-+      release_pkgs = editable['provision']['packages']['value'].split()
-+      legacy_unresolved += legacy_diff(legacy_unresolved,
-+        release_pkgs, 'Release {0}'.format(release['fields']['name']), arch)
-+  ## 3. OPNFV additional packages (includes old fuel-mirror ubuntu.yaml pkgs)
-+  unresolved = dict()
-+  unresolved['mandatory'] = 'exact'
-+  unresolved['packages'] = from_legacy_pkglist(legacy_unresolved)
-+  if 'packages' in opnfv_cfg:
-+    legacy_diff(legacy_unresolved, to_legacy_pkglist(opnfv_cfg['packages']),
-+      'OPNFV config', arch)
-+    unresolved['packages'] += opnfv_cfg['packages']
-+
-+  # OPNFV plugins dependency resolution
-+  for plugin in plugins.split():
-+    path = "../{}/packages.yaml".format(plugin)
-+    if os.path.isfile(path):
-+      f = open(path).read()
-+      plugin_yaml = yaml.load(f)
-+      new_packages = legacy_diff(to_legacy_pkglist(unresolved['packages']),
-+        plugin_yaml['packages'], 'Plugin {0}'.format(plugin), arch)
-+      unresolved['packages'] += from_legacy_pkglist(new_packages)
-+
-+  # Package list (reduced, i.e. no MOS deps, but with OPNFV plugin deps)
-+  # FIXME: This helps work around packetary solving main deps from universe
-+  if mirror_ubuntu_resolve_main_deps or mirror_ubuntu_merge is None:
-+    write_cfg_file(cfg_p_ubuntu_main, unresolved)
-+
-+  # Mirror package list (full, including MOS/OPNFV plugin deps)
-+  unresolved['packages'] += get_unres_pkgs(arch, cfg_m_mos)
-+  write_cfg_file(cfg_p_ubuntu, unresolved)
-+  do_partial_mirror(arch, cfg_m_ubuntu, cfg_p_ubuntu)
-+  if mirror_ubuntu_resolve_main_deps or mirror_ubuntu_merge is None:
-+    # Ubuntu[main] must be evaluated after Ubuntu
-+    do_partial_mirror(arch, cfg_m_ubuntu_main, cfg_p_ubuntu_main)
-+
-+if mirror_ubuntu_merge is None:
-+  shutil.move(mirror_ubuntu_tmp_path, mirror_ubuntu_path)
-+else:
-+  # Construct single-component mirror from all components
-+  for arch in ubuntu_arch.split(' '):
-+    cfg_pp_ubuntu = '{0}/ubuntu_{1}_packages_paths.yaml'.format(cfg_dir, arch)
-+    # FIXME: We need scanpackages to omit older DEBs
-+    # Inspired from http://askubuntu.com/questions/198474/
-+    os.system('dpkg-scanpackages -a {0} {1} 2>/dev/null | '
-+      'grep -e "^Filename:" | sed "s|Filename: |- file://|g" > {2}'
-+        .format(arch, mirror_ubuntu_tmp_path, cfg_pp_ubuntu))
-+    do_local_repo(arch, cfg_mm_ubuntu, cfg_pp_ubuntu)
-+  shutil.rmtree(mirror_ubuntu_tmp_path)
-diff --git a/deploy/config/dea_base.yaml b/deploy/config/dea_base.yaml
-index ce81994..2520cf5 100644
---- a/deploy/config/dea_base.yaml
-+++ b/deploy/config/dea_base.yaml
-@@ -622,22 +622,10 @@ settings:
-         value:
-         - name: ubuntu
-           priority: null
--          section: main universe multiverse
-+          section: main
-           suite: trusty
-           type: deb
--          uri: http://10.20.0.2:8080/mirrors/ubuntu/
--        - name: ubuntu-updates
--          priority: null
--          section: main universe multiverse
--          suite: trusty-updates
--          type: deb
--          uri: http://10.20.0.2:8080/mirrors/ubuntu/
--        - name: ubuntu-security
--          priority: null
--          section: main universe multiverse
--          suite: trusty-security
--          type: deb
-           uri: http://10.20.0.2:8080/mirrors/ubuntu/
-         - name: mos
-           priority: 1050
-           section: main restricted
diff --git a/patches/opnfv-fuel/upstream-backports/0004-Fuel-rework-for-multiple-libvirt.patch b/patches/opnfv-fuel/upstream-backports/0004-Fuel-rework-for-multiple-libvirt.patch
index 271e0fa0..e08026e6 100644
--- a/patches/opnfv-fuel/upstream-backports/0004-Fuel-rework-for-multiple-libvirt.patch
+++ b/patches/opnfv-fuel/upstream-backports/0004-Fuel-rework-for-multiple-libvirt.patch
@@ -736,7 +736,7 @@ index 3dd5f84..d738154 100644
 @@ -41,17 +41,42 @@ dea-override-config:
    - id: 1
      interfaces: interfaces_1
-     role: mongo,controller
+     role: mongo,controller,congress
 +    transformations: transformations_1
    - id: 2
      interfaces: interfaces_1
diff --git a/patches/opnfv-fuel/upstream-backports/0005-CI-deploy-cache-Store-and-reuse-deploy-artifacts.patch b/patches/opnfv-fuel/upstream-backports/0005-CI-deploy-cache-Store-and-reuse-deploy-artifacts.patch
index 2315c5ad..48fd31db 100644
--- a/patches/opnfv-fuel/upstream-backports/0005-CI-deploy-cache-Store-and-reuse-deploy-artifacts.patch
+++ b/patches/opnfv-fuel/upstream-backports/0005-CI-deploy-cache-Store-and-reuse-deploy-artifacts.patch
@@ -63,9 +63,9 @@ index 0000000..80cd0f4
 +--- a/iso/bootstrap_admin_node.sh
 ++++ b/iso/bootstrap_admin_node.sh
 +@@ -61,6 +61,8 @@ wget \
-+
 + ASTUTE_YAML='/etc/fuel/astute.yaml'
 + BOOTSTRAP_NODE_CONFIG="/etc/fuel/bootstrap_admin_node.conf"
++ CUSTOM_REPOS="/root/default_deb_repos.yaml"
 ++OPNFV_CACHE_PATH="/var/cache/opnfv/bootstraps"
 ++OPNFV_CACHE_TAR="opnfv-bootstraps-cache.tar"
 + bs_build_log='/var/log/fuel-bootstrap-image-build.log'
@@ -179,8 +179,7 @@ diff --git a/deploy/cloud/deployment.py b/deploy/cloud/deployment.py
 index 5dd0263..3db4c0d 100644
 --- a/deploy/cloud/deployment.py
 +++ b/deploy/cloud/deployment.py
-@@ -24,6 +24,8 @@ from common import (
-     delete,
+@@ -24,5 +24,7 @@ from common import (
  )
 
 +from deploy_cache import DeployCache
diff --git a/patches/opnfv-fuel/upstream-backports/0006-Restart-sshd-service-instead-of-reload.patch b/patches/opnfv-fuel/upstream-backports/0006-Restart-sshd-service-instead-of-reload.patch
deleted file mode 100644
index 6e876e85..00000000
--- a/patches/opnfv-fuel/upstream-backports/0006-Restart-sshd-service-instead-of-reload.patch
+++ /dev/null
@@ -1,26 +0,0 @@
-From: Michael Polenchuk <mpolenchuk@mirantis.com>
-Date: Tue, 13 Dec 2016 17:42:23 +0400
-Subject: [PATCH] Restart sshd service instead of reload
-
-In the middle of fuel master node bootstrap the reload of sshd service
-accidentally is losing main process pid, so restart it.
-
-JIRA: FUEL-237
-Change-Id: I89023177e71cb6da4b9d6beff5083d345cf66db8
-Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
----
- build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch | 2 +-
- 1 file changed, 1 insertion(+), 1 deletion(-)
-
-diff --git a/build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch b/build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch
-index 820e387..d01ce8b 100644
---- a/build/f_repos/patch/fuel-main/0001-OPNFV-Additions-to-bootstrap_admin_node.sh.patch
-+++ b/build/f_repos/patch/fuel-main/0001-OPNFV-Additions-to-bootstrap_admin_node.sh.patch
-@@ -41,6 +41,6 @@ index dbcac5f..1826390 100755
- -systemctl enable sshd
- -systemctl start sshd
--+systemctl reload sshd
-++systemctl restart sshd
- 
-  # Enable iptables
-  systemctl enable iptables.service
diff --git a/patches/opnfv-fuel/upstream-backports/0007-build-select_ubuntu_repo-break-on-err.patch b/patches/opnfv-fuel/upstream-backports/0007-build-select_ubuntu_repo-break-on-err.patch
deleted file mode 100644
index fdf7be87..00000000
--- a/patches/opnfv-fuel/upstream-backports/0007-build-select_ubuntu_repo-break-on-err.patch
+++ /dev/null
@@ -1,70 +0,0 @@
-From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
-Date: Tue, 20 Dec 2016 00:49:47 +0100
-Subject: [PATCH] build: select_ubuntu_repo: break on err
-
-Stop ISO build process if no sane Ubuntu mirrors are available,
-instead of continuing with an empty URL, which is bound to fail
-later.
-
-While at it, skip re-assesing the output of select_ubuntu_repo.sh if
-the URL var already is set.
-
-Also, stderr and stdout were both captured in UBUNTU_MIRROR_URL,
-so disable debug by default, and only print debug info in case no
-valid mirror is found.
-
-v2 -> v3:
-- allow "One hour behind" mirror state to be considered, since
-  we already accept "Six hours behind" mirrors;
-
-JIRA: FUEL-240
-
-Change-Id: I791942da234304528ff5cae1891415602e1a62b2
-Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
----
- build/Makefile                                    | 10 +++++++---
- build/f_isoroot/f_repobuild/select_ubuntu_repo.sh |  4 ++--
- 2 files changed, 9 insertions(+), 5 deletions(-)
-
-diff --git a/build/Makefile b/build/Makefile
-index c8fc793..d4b981a 100644
---- a/build/Makefile
-+++ b/build/Makefile
-@@ -51,9 +51,13 @@ export REPOINFO  := $(BUILD_BASE)/repo_info.sh
-
- # Use snapshots
- # Use nearby repositories
--export MIRROR_UBUNTU_URL := $(shell ./f_isoroot/f_repobuild/select_ubuntu_repo.sh --url)
--export MIRROR_UBUNTU := $(shell echo "$(MIRROR_UBUNTU_URL)" | cut -d'/' -f3 )
--export MIRROR_UBUNTU_ROOT := $(shell echo -n '/' ; echo "$(MIRROR_UBUNTU_URL)" | cut -d'/' -f4-)
-+export MIRROR_UBUNTU_URL ?= $(shell ./f_isoroot/f_repobuild/select_ubuntu_repo.sh)
-+export MIRROR_UBUNTU ?= $(shell echo "$(MIRROR_UBUNTU_URL)" | cut -d'/' -f3 )
-+export MIRROR_UBUNTU_ROOT ?= $(shell echo -n '/' ; echo "$(MIRROR_UBUNTU_URL)" | cut -d'/' -f4-)
-+ifeq (,$(MIRROR_UBUNTU_URL))
-+$(warning $(shell ./f_isoroot/f_repobuild/select_ubuntu_repo.sh -d))
-+$(error No sane Ubuntu mirror available)
-+endif
-
- export LATEST_MIRROR_ID_URL := http://$(shell ./select_closest_fuel_mirror.py)
-
-diff --git a/build/f_isoroot/f_repobuild/select_ubuntu_repo.sh b/build/f_isoroot/f_repobuild/select_ubuntu_repo.sh
-index 24bd422..5618eb7 100755
---- a/build/f_isoroot/f_repobuild/select_ubuntu_repo.sh
-+++ b/build/f_isoroot/f_repobuild/select_ubuntu_repo.sh
-@@ -69,14 +69,14 @@ if [ "$1" == "-d" ]; then
- fi
-
- # Hardcode for testing purposes
--DEBUG=1
-+# DEBUG=1
-
- TMPFILE=$(mktemp /tmp/mirrorsXXXXX)A
- trap cleanup exit
-
- # Generate a list of mirrors considered as "up"
- curl -s  https://launchpad.net/ubuntu/+archivemirrors | \
--    grep -P -B8 "statusUP|statusSIX" | \
-+    grep -P -B8 "statusUP|statusONE|statusSIX" | \
-     grep -o -P "(f|ht)tp.*\""  | \
-     sed 's/"$//' | sort | uniq > $TMPFILE
-