aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorAlexandru Avadanii <Alexandru.Avadanii@enea.com>2016-09-24 17:27:16 +0200
committerAlexandru Avadanii <Alexandru.Avadanii@enea.com>2016-10-05 21:44:58 +0200
commit6de5bac371844b70b96b9bd75b62b8d06e5f51fc (patch)
tree7105db6f62d236c91130d47283842aa4a6f16156
parent49f066e78e52fde9ce0d467a838211270f518591 (diff)
build: Use git submodules for Fuel git repos
This change reworks support for cloning (fetching) and patching all Fuel components (fuel-library ... fuel-ui, see [1] for full list), both outside the ISO build process (for development purposes, like testing OPNFV patches apply cleanly to each component), and during the ISO build. The implementation relies heavily on git submodules and patches, so backporting pending upstream gerrit changes, as well as adding new out-of-upstream-tree patches, should be trivial. Also, since without tracking remotes the repos rarely change, leverage the in-place OPNFV build caching mechanism to drastically reduce the number of git clones during each build. The mechanism is detailed more in-depth in the JIRA ticket [3], and it's merely a rehash of the submodule handling in Armband, implemented initially by Stanislaw Kardach <stanislaw.kardach@cavium.com>. Pros (+ new features): - OPNFV patches for Fuel components can be applied (imported) or developed (exported) outside of the ISO build process; - git repo caching reduces the number of external pulls; - reuse the same fuel-* componets in ALL targets e.g. fuel-mirror used to employ 2 slightly different git trees; - one step closer to Fuel/Armband merge (lots of steps left though); Cons: - adds a lot complexity to solve a non-problem (handling the repos is fine in the current form, for the current goals); However, patching <fuel-astute> or <fuel-agent> seems to be quite complicated with the current mechanism, as env vars override our locally patched repos for these components). The proposed change should eliminate this issue completely. CHANGE: Aligned divergent fuel-mirror in ISO vs local repo build by applying the 302 redirect fix ("Fixed handling http redirects") [2] on top of 9.0.1 tag in fuel-mirror repo, and using that for both. This obsoleted the followking mk file: - build/f_isoroot/f_repobuild/config.mk CHANGE: Removed unused make target `get-fuel-repo` from build/config.mk. FIXME: Remote tracking is currently deactivated for all branches. Change this when remote HEAD should be tracked instead, e.g. during Newton rebase. [1] https://github.com/openstack/fuel-main/blob/stable/mitaka/repos.mk#L32 [2] https://review.openstack.org/#/c/334882/ [3] https://jira.opnfv.org/browse/FUEL-200 TODO: Extend build/f_repos/README.md with info about: - branch changes; - tag bumps; - patching now supports subdirs; v5 -> v6: * Added support for nesting patches in subdirs, to be used for feature groups and related changes across submodules. E.g. Upcoming multiarch EFI support could go in: - .../f_repos/patch/fuel-agent/multiarch-efi/...; - .../f_repos/patch/fuel-astute/multiarch-efi/...; While Armband patches could go in: - .../f_repos/patch/fuel-agent/armband/...; - .../f_repos/patch/fuel-astute/armband/...; etc. v6 -> v13: * fix wrong tag used for VERSION dump; * fix wrong tag used for patches-export; * move OPNFV_TAG to main config.mk and use it in FUEL_*_COMMIT; * only run `patches-import` when patches changed / not yet applied; * only run `clean` for stale `.cacheid` (avoid re-applying patches); * allow git submodule path/name to be different (FIXED); * put-cache should only run when artifact is not already cached; * `git am`: use 3-way merge so already applied patches are gently skipped (useful for upstreaming patches from Armband); * Properly support nested patch dirs; * Colorize the output a little; JIRA: FUEL-200 Change-Id: I9dbdd9b3022896d4497d21be8dacc859730db489 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
-rw-r--r--.gitignore1
-rw-r--r--.gitmodules70
-rw-r--r--build/Makefile39
-rw-r--r--build/config.mk75
-rw-r--r--build/f_isoroot/f_repobuild/Makefile7
-rw-r--r--build/f_isoroot/f_repobuild/config.mk13
-rw-r--r--build/f_repos/Makefile165
-rw-r--r--build/f_repos/README.md94
-rw-r--r--build/f_repos/patch/fuel-library/0001-Reduce-ceilometer-memory-usage.patch (renamed from build/patch-repos/build/repos/fuel-library/0001-Reduce-ceilometer-memory-usage.patch)8
-rw-r--r--build/f_repos/patch/fuel-library/0002-Disable-token-revoke-to-increase-keystone-performanc.patch (renamed from build/patch-repos/build/repos/fuel-library/0001-Disable-token-revoke-to-increase-keystone-performanc.patch)12
-rw-r--r--build/f_repos/patch/fuel-main/0001-OPNFV-Additions-to-bootstrap_admin_node.sh.patch (renamed from build/patch-repos/0010-bootstrap_admin_node.sh.patch)13
-rw-r--r--build/f_repos/patch/fuel-main/0002-OPNFV-showmenu-yes-in-isolinux.cfg.patch (renamed from build/patch-repos/0020-isolinux.cfg.patch)5
-rw-r--r--build/f_repos/patch/fuel-main/0003-repo-mirror-Allow-multi-arch-local-mirrors.patch (renamed from build/patch-repos/0030-repo-multi-arch-local-mirrors.patch)30
-rw-r--r--build/f_repos/patch/fuel-main/0004-xorriso-fails-to-add-files-with-path-longer-then-240.patch (renamed from build/patch-repos/0040-xorriso-fails-to-add-files-with-path-longer-then-240.patch)0
-rw-r--r--build/f_repos/patch/fuel-mirror/0001-Fixed-handling-http-redirects.patch87
-rw-r--r--build/f_repos/patch/fuel-web/0001-Mark-Intel-82599-10-Gigabit-NIC-as-DPDK-capable.patch (renamed from build/patch-repos/build/repos/fuel-nailgun/0010-Mark-Intel-82599-10-Gigabit-NIC-as-DPDK-capable.patch)9
m---------build/f_repos/sub/fuel-agent0
m---------build/f_repos/sub/fuel-astute0
m---------build/f_repos/sub/fuel-library0
m---------build/f_repos/sub/fuel-main0
m---------build/f_repos/sub/fuel-menu0
m---------build/f_repos/sub/fuel-mirror0
m---------build/f_repos/sub/fuel-nailgun-agent0
m---------build/f_repos/sub/fuel-ostf0
m---------build/f_repos/sub/fuel-ui0
m---------build/f_repos/sub/fuel-upgrade0
m---------build/f_repos/sub/fuel-web0
m---------build/f_repos/sub/network-checker0
m---------build/f_repos/sub/python-fuelclient0
m---------build/f_repos/sub/shotgun0
30 files changed, 526 insertions, 102 deletions
diff --git a/.gitignore b/.gitignore
index 9b4b3afa3..588243653 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,4 +1,5 @@
*.pyc
+.cache*
.project
.pydevproject
ci/config/
diff --git a/.gitmodules b/.gitmodules
new file mode 100644
index 000000000..84b15ee45
--- /dev/null
+++ b/.gitmodules
@@ -0,0 +1,70 @@
+[submodule "fuel-main"]
+ path = build/f_repos/sub/fuel-main
+ url = https://github.com/openstack/fuel-main.git
+ branch = stable/mitaka
+ ignore = all
+[submodule "fuel-library"]
+ path = build/f_repos/sub/fuel-library
+ url = https://github.com/openstack/fuel-library.git
+ branch = stable/mitaka
+ ignore = all
+[submodule "fuel-web"]
+ path = build/f_repos/sub/fuel-web
+ url = https://github.com/openstack/fuel-web.git
+ branch = stable/mitaka
+ ignore = all
+[submodule "python-fuelclient"]
+ path = build/f_repos/sub/python-fuelclient
+ url = https://github.com/openstack/python-fuelclient.git
+ branch = stable/mitaka
+ ignore = all
+[submodule "fuel-agent"]
+ path = build/f_repos/sub/fuel-agent
+ url = https://github.com/openstack/fuel-agent.git
+ branch = stable/mitaka
+ ignore = all
+[submodule "fuel-nailgun-agent"]
+ path = build/f_repos/sub/fuel-nailgun-agent
+ url = https://github.com/openstack/fuel-nailgun-agent.git
+ branch = stable/mitaka
+ ignore = all
+[submodule "fuel-astute"]
+ path = build/f_repos/sub/fuel-astute
+ url = https://github.com/openstack/fuel-astute.git
+ branch = stable/mitaka
+ ignore = all
+[submodule "fuel-ostf"]
+ path = build/f_repos/sub/fuel-ostf
+ url = https://github.com/openstack/fuel-ostf.git
+ branch = stable/mitaka
+ ignore = all
+[submodule "fuel-mirror"]
+ path = build/f_repos/sub/fuel-mirror
+ url = https://github.com/openstack/fuel-mirror.git
+ branch = stable/mitaka
+ ignore = all
+[submodule "fuel-menu"]
+ path = build/f_repos/sub/fuel-menu
+ url = https://github.com/openstack/fuel-menu.git
+ branch = stable/mitaka
+ ignore = all
+[submodule "shotgun"]
+ path = build/f_repos/sub/shotgun
+ url = https://github.com/openstack/shotgun.git
+ branch = stable/mitaka
+ ignore = all
+[submodule "network-checker"]
+ path = build/f_repos/sub/network-checker
+ url = https://github.com/openstack/network-checker.git
+ branch = stable/mitaka
+ ignore = all
+[submodule "fuel-upgrade"]
+ path = build/f_repos/sub/fuel-upgrade
+ url = https://github.com/openstack/fuel-upgrade.git
+ branch = stable/mitaka
+ ignore = all
+[submodule "fuel-ui"]
+ path = build/f_repos/sub/fuel-ui
+ url = https://github.com/openstack/fuel-ui.git
+ branch = stable/mitaka
+ ignore = all
diff --git a/build/Makefile b/build/Makefile
index 59d9096c6..56acb40fe 100644
--- a/build/Makefile
+++ b/build/Makefile
@@ -80,10 +80,7 @@ ORIGDIR := $(TOPDIR)/origiso
# END of variables to customize
#############################################################################
-# Fuel-main destination path and fuel-* submodule patching, for full list check:
-# https://github.com/openstack/fuel-main/blob/stable/mitaka/repos.mk#L32-L44
FUEL_MAIN_DIR := /tmp/fuel-main
-FUEL_PATCHES := $(shell find $(BUILD_BASE)/patch-repos -name '*.patch' | sort)
SUBCLEAN = $(addsuffix .clean,$(SUBDIRS))
@@ -111,9 +108,12 @@ include cache.mk
$(ISOCACHE):
# Clone Fuel to non-persistent location and build
sudo rm -rf $(FUEL_MAIN_DIR)
+ $(MAKE) -C f_repos -f Makefile release
git clone $(FUEL_MAIN_REPO) $(FUEL_MAIN_DIR)
- git -C $(FUEL_MAIN_DIR) checkout $(FUEL_MAIN_TAG)
- @echo "fuel" `git -C $(FUEL_MAIN_DIR) rev-parse HEAD` >> $(VERSION_FILE)
+ # Save upstream ref, checkout OPNFV tag (upstream ref + patches)
+ @echo "fuel" `git -C $(FUEL_MAIN_DIR) \
+ rev-parse $(F_OPNFV_TAG)-root` >> $(VERSION_FILE)
+ git -C $(FUEL_MAIN_DIR) checkout $(F_OPNFV_TAG)
# Remove Docker optimizations, otherwise multistrap will fail during
# Fuel build.
sudo rm -f /etc/apt/apt.conf.d/docker*
@@ -134,14 +134,7 @@ $(ISOCACHE):
sudo docker info
# fuel-main Makefiles do not like `make -C`
cd $(FUEL_MAIN_DIR) && make repos
- $(REPOINFO) -r $(FUEL_MAIN_DIR) > gitinfo_fuel.txt
- # OPNFV patches at Fuel build time
- # Need to be commited in order for them to be considered by the Fuel
- # build system
- $(foreach patch,$(FUEL_PATCHES),git \
- -C $(subst $(BUILD_BASE)/patch-repos,$(FUEL_MAIN_DIR),$(dir $(patch))) \
- am --whitespace=nowarn --committer-date-is-author-date $(patch) || \
- (echo 'Error: Failed patching Fuel repos!' ; exit 1);)
+ cp f_repos/.cachefuelinfo gitinfo_fuel.txt
# Repeat build up to three times
sudo -E ./fuel_build_loop
@@ -172,6 +165,7 @@ patch-packages:
.PHONY: clean $(SUBCLEAN)
clean: $(SUBCLEAN)
+ $(MAKE) -C f_repos -f Makefile clean
$(MAKE) -C patch-packages -f Makefile clean
@rm -f *.iso
@rm -Rf release
@@ -182,7 +176,8 @@ clean: $(SUBCLEAN)
.PHONY: deepclean
deepclean: clean clean-cache
- make -C docker clean
+ $(MAKE) -C f_repos -f Makefile deepclean
+ $(MAKE) -C docker clean
docker rmi opnfv.org/ubuntu-builder:14.04 &>/dev/null || exit 0
docker rmi opnfv.org/ubuntu-builder:latest &>/dev/null || exit 0
docker rmi ubuntu:14.04 &>/dev/null || exit 0
@@ -224,24 +219,26 @@ debug:
#############################################################################
# Create a unique hash to be used for getting and putting cache, based on:
-# - The commit ID of the full Fuel repo structre
+# - The commit ID of the full Fuel repo structure
# - The contents of all local Fuel patches
+# - Makefile, config and sensitive build scripts fingerprints
+# NOTE: Patching is deterministic, so we can fingerprint (submodule root commit
+# info + OPNFV patches) at once, after patches-import.
+# NOTE: When git submodule remote tracking is active, `git submodule status`
+# will point to the latest commits (remote/branch/HEAD) + OPNFV patches.
.cacheid:
- git clone $(FUEL_MAIN_REPO) $(FUEL_MAIN_DIR)
- git -C $(FUEL_MAIN_DIR) checkout $(FUEL_MAIN_TAG)
- make -C $(FUEL_MAIN_DIR) repos
- $(REPOINFO) -r $(FUEL_MAIN_DIR) > .cachedata
- $(foreach patch,$(FUEL_PATCHES),sha1sum $(patch) >> .cachedata;)
+ $(MAKE) -C f_repos -f Makefile get-cache release
+ git submodule status | cut -c2-41 > .cachedata
sha1sum fuel_build_loop >> .cachedata
sha1sum config.mk >> .cachedata
sha1sum Makefile >> .cachedata
$(CACHETOOL) getbiweek >> .cachedata
cat .cachedata | $(CACHETOOL) getid > .cacheid
- # Not removing fuel-main as it is re-used in build
# Clean local data related to caching - called prior to ordinary build
.PHONY: clean-cache
clean-cache: $(SUBCLEANCACHE)
+ $(MAKE) -C f_repos -f Makefile clean-cache
rm -f .cachedata .cacheid
# Try to download cache - called prior to ordinary build
diff --git a/build/config.mk b/build/config.mk
index f78043e18..ab30518b5 100644
--- a/build/config.mk
+++ b/build/config.mk
@@ -1,5 +1,5 @@
##############################################################################
-# Copyright (c) 2015 Ericsson AB and others.
+# Copyright (c) 2015,2016 Ericsson AB and others.
# stefan.k.berg@ericsson.com
# jonas.bjurel@ericsson.com
# All rights reserved. This program and the accompanying materials
@@ -8,34 +8,59 @@
# http://www.apache.org/licenses/LICENSE-2.0
##############################################################################
-FUEL_MAIN_REPO := https://github.com/openstack/fuel-main
-FUEL_MAIN_TAG := 9.0.1
-MOS_VERSION = 9.0
+# fuel-main tag checked out from upstream `fuel-main` repo before patching it
+FUEL_MAIN_TAG = 9.0.1
+MOS_VERSION = 9.0
OPENSTACK_VERSION = mitaka-9.0
-# Pinning down exact Fuel repo versions for Fuel 9.0.1
-export FUELLIB_COMMIT?=e283b62750d9e26355981b3ad3be7c880944ae0f
-export NAILGUN_COMMIT?=e2b85bafb68c348f25cb7cceda81edc668ba2e64
-export PYTHON_FUELCLIENT_COMMIT?=67d8c693a670d27c239d5d175f3ea2a0512c498c
-export FUEL_AGENT_COMMIT?=7ffbf39caf5845bd82b8ce20a7766cf24aa803fb
-export FUEL_NAILGUN_AGENT_COMMIT?=46fa0db0f8944f9e67699d281d462678aaf4db26
-export ASTUTE_COMMIT?=390b257240d49cc5e94ed5c4fcd940b5f2f6ec64
-export OSTF_COMMIT?=f09c98ff7cc71ee612b2450f68a19f2f9c64345a
-export FUEL_MIRROR_COMMIT?=d1ef06b530ce2149230953bb3810a88ecaff870c
-export FUELMENU_COMMIT?=0ed9e206ed1c6271121d3acf52a6bf757411286b
-export SHOTGUN_COMMIT?=781a8cfa0b6eb290e730429fe2792f2b6f5e0c11
-export NETWORKCHECKER_COMMIT?=fcb47dd095a76288aacf924de574e39709e1f3ca
-export FUELUPGRADE_COMMIT?=c1c4bac6a467145ac4fac73e4a7dd2b00380ecfb
-export FUEL_UI_COMMIT?=90de7ef4477230cb7335453ed26ed4306ca6f04f
-
-# for the patches applying purposes
+# FIXME(alav): Disable remote tracking for now, stick to submodule commits
+FUEL_TRACK_REMOTES =
+
+##############################################################################
+# Fuel components pinning / remote tracking; use submodules from f_repos
+##############################################################################
+
+# git submodule & patch locations for Fuel components
+F_GIT_ROOT := $(shell git rev-parse --show-toplevel)
+F_SUBMOD_DIR := ${F_GIT_ROOT}/build/f_repos/sub
+F_PATCH_DIR := ${F_GIT_ROOT}/build/f_repos/patch
+F_OPNFV_TAG := ${FUEL_MAIN_TAG}-opnfv
+
+# fuel-main repo location used by main Makefile ISO building, use submodule
+FUEL_MAIN_REPO := ${F_SUBMOD_DIR}/fuel-main
+
+export FUELLIB_REPO?=${F_SUBMOD_DIR}/fuel-library
+export NAILGUN_REPO?=${F_SUBMOD_DIR}/fuel-web
+export PYTHON_FUELCLIENT_REPO?=${F_SUBMOD_DIR}/python-fuelclient
+export FUEL_AGENT_REPO?=${F_SUBMOD_DIR}/fuel-agent
+export FUEL_NAILGUN_AGENT_REPO?=${F_SUBMOD_DIR}/fuel-nailgun-agent
+export ASTUTE_REPO?=${F_SUBMOD_DIR}/fuel-astute
+export OSTF_REPO?=${F_SUBMOD_DIR}/fuel-ostf
+export FUEL_MIRROR_REPO?=${F_SUBMOD_DIR}/fuel-mirror
+export FUELMENU_REPO?=${F_SUBMOD_DIR}/fuel-menu
+export SHOTGUN_REPO?=${F_SUBMOD_DIR}/shotgun
+export NETWORKCHECKER_REPO?=${F_SUBMOD_DIR}/network-checker
+export FUELUPGRADE_REPO?=${F_SUBMOD_DIR}/fuel-upgrade
+export FUEL_UI_REPO?=${F_SUBMOD_DIR}/fuel-ui
+
+# OPNFV tags are automatically applied by `make -C f_repos patches-import`
+export FUELLIB_COMMIT?=${F_OPNFV_TAG}
+export NAILGUN_COMMIT?=${F_OPNFV_TAG}
+export PYTHON_FUELCLIENT_COMMIT?=${F_OPNFV_TAG}
+export FUEL_AGENT_COMMIT?=${F_OPNFV_TAG}
+export FUEL_NAILGUN_AGENT_COMMIT?=${F_OPNFV_TAG}
+export ASTUTE_COMMIT?=${F_OPNFV_TAG}
+export OSTF_COMMIT?=${F_OPNFV_TAG}
+export FUEL_MIRROR_COMMIT?=${F_OPNFV_TAG}
+export FUELMENU_COMMIT?=${F_OPNFV_TAG}
+export SHOTGUN_COMMIT?=${F_OPNFV_TAG}
+export NETWORKCHECKER_COMMIT?=${F_OPNFV_TAG}
+export FUELUPGRADE_COMMIT?=${F_OPNFV_TAG}
+export FUEL_UI_COMMIT?=${F_OPNFV_TAG}
+
+# for the patches applying purposes (empty git config in docker build container)
export GIT_COMMITTER_NAME?=Fuel OPNFV
export GIT_COMMITTER_EMAIL?=fuel@opnfv.org
DOCKER_REPO := http://get.docker.com/builds/Linux/x86_64
DOCKER_TAG := docker-latest
-
-.PHONY: get-fuel-repo
-get-fuel-repo:
- @echo $(FUEL_MAIN_REPO) $(FUEL_MAIN_TAG)
-
diff --git a/build/f_isoroot/f_repobuild/Makefile b/build/f_isoroot/f_repobuild/Makefile
index 7c6b16304..ad1a934fa 100644
--- a/build/f_isoroot/f_repobuild/Makefile
+++ b/build/f_isoroot/f_repobuild/Makefile
@@ -13,7 +13,6 @@ TOP := $(shell pwd)
TMP_ROOT_DIR := $(shell echo "$(MIRROR_UBUNTU_ROOT)" | cut -d "/" -f2)
include ../../config.mk
-include config.mk
export MOS_VERSION
export OPENSTACK_VERSION
@@ -25,8 +24,7 @@ nailgun:
sudo apt-get install -y git libxml2-dev libxslt-dev python-dev python-pip libz-dev libyaml-dev createrepo python-yaml
rm -Rf nailgun
sudo mkdir -p /var/www/nailgun
- git clone $(FUEL_MIRROR_REPO)
- cd fuel-mirror && git checkout -q $(FUEL_MIRROR_COMMIT)
+ ln -sf ${F_SUBMOD_DIR}/fuel-mirror fuel-mirror
sudo pip install -U -r ./fuel-mirror/requirements.txt
sudo pip install ./fuel-mirror
sudo pip install ./fuel-mirror/contrib/fuel_mirror
@@ -34,7 +32,7 @@ nailgun:
sudo fuel-mirror --debug --config ./opnfv-config.yaml create --group ubuntu --pattern=ubuntu
sudo chmod -R 755 /var/www/nailgun
cp -Rp /var/www/nailgun .
- # On the end we want to have ubuntu repository in mirrors/ubuntu directory
+ # In the end we want to have ubuntu repository in mirrors/ubuntu directory
-if [ "$(MIRROR_UBUNTU_ROOT)" != "/ubuntu/" ]; then \
mkdir -p nailgun/mirrors/ubuntu;\
mv nailgun/mirrors$(MIRROR_UBUNTU_ROOT)* nailgun/mirrors/ubuntu;\
@@ -65,7 +63,6 @@ release:nailgun
.cacheid:
date +"Repocache %G%V" > .cachedata
sha1sum Makefile >> .cachedata
- sha1sum config.mk >> .cachedata
$(CACHETOOL) packages >> .cachedata
echo -n $(UBUNTU_ARCH) | sha1sum | awk {'print $$1'} >> .cachedata
cat .cachedata | $(CACHETOOL) getid > .cacheid
diff --git a/build/f_isoroot/f_repobuild/config.mk b/build/f_isoroot/f_repobuild/config.mk
deleted file mode 100644
index d81732849..000000000
--- a/build/f_isoroot/f_repobuild/config.mk
+++ /dev/null
@@ -1,13 +0,0 @@
-##############################################################################
-# Copyright (c) 2016 Ericsson AB and others.
-# mskalski@mirantis.com
-# All rights reserved. This program and the accompanying materials
-# are made available under the terms of the Apache License, Version 2.0
-# which accompanies this distribution, and is available at
-# http://www.apache.org/licenses/LICENSE-2.0
-##############################################################################
-
-FUEL_MIRROR_REPO?=https://github.com/openstack/fuel-mirror
-# Point to the commit where 302 redirects are handled
-# https://bugs.launchpad.net/fuel/mitaka/+bug/1593674
-FUEL_MIRROR_COMMIT?=192a3d9f8f993afb12c5108dd9339c6688c23e11
diff --git a/build/f_repos/Makefile b/build/f_repos/Makefile
new file mode 100644
index 000000000..66abf2d71
--- /dev/null
+++ b/build/f_repos/Makefile
@@ -0,0 +1,165 @@
+##############################################################################
+# Copyright (c) 2015,2016 Ericsson AB, Enea AB and others.
+# stefan.k.berg@ericsson.com
+# jonas.bjurel@ericsson.com
+# Alexandru.Avadanii@enea.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+
+SHELL = /bin/bash
+REPOINFO = $(shell readlink -e ../repo_info.sh)
+FREPODIR = $(shell pwd)
+FPATCHES = $(shell find ${F_PATCH_DIR} -name '*.patch')
+
+include ../config.mk
+
+# NOTE: Mechanism overview is presented in ./README.md.
+
+# Submodule consistent states:
+# - NOT initialized (submodule trees are not populated at all);
+# - initialized, bound to saved commits;
+# - initialized, tracking remote origin (only for FUEL_TRACK_REMOTES);
+# - patched (local patches are applied);
+
+# In order to keep things sort of separate, we should only pass up (to main
+# Makefile) the fully-pactched repos, and gather any fingerprinting info here.
+
+# Fuel@OPNFV relies on upstream git repos (one per component) in 1 of 2 ways:
+# - pinned down to tag objects (e.g. "9.0.1")
+# - tracking upstream remote HEAD on a stable or master branch
+# FIXME(alav): Should we support mixed cases? (e.g. pin down only fuel-main)
+
+# To enable remote tracking, set the following var to any non-empty string.
+# Leaving this var empty will bind each git submodule to its saved commit.
+# NOTE: For non-stable branches, unless overriden, always track remotes
+FUEL_TRACK_REMOTES ?= $(shell git symbolic-ref --short HEAD | grep -v stable)
+
+.PHONY: all
+all: release
+
+##############################################################################
+# git submodule operations - to be used stand-alone or from parent Makefile
+##############################################################################
+
+# Fetch & update git submodules, checkout remote HEAD or saved commit
+# Also gather fingerprints for parent gitinfo_fuel.txt.
+.PHONY: sub
+sub: .cachefuelinfo
+
+.cachefuelinfo:
+ @if [ -n "${FUEL_TRACK_REMOTES}" ]; then \
+ git submodule update --init --remote; \
+ else \
+ git submodule update --init; \
+ fi
+ @rm -f $@
+ @git submodule -q foreach '${REPOINFO} . >> ${FREPODIR}/$@'
+
+# Generate patches from submodules
+.PHONY: patches-export
+patches-export: sub
+ @git submodule -q foreach ' \
+ SUB_DIR=${F_PATCH_DIR}/$$name; \
+ git tag | awk "!/root/ && /${F_OPNFV_TAG}-fuel/" | while read F_TAG; do \
+ SUB_FEATURE=`dirname $${F_TAG#${F_OPNFV_TAG}-fuel/}`; \
+ echo "`tput setaf 2`-- exporting $$name ($$F_TAG)`tput sgr0`"; \
+ mkdir -p $$SUB_DIR/$${SUB_FEATURE} && \
+ git format-patch --no-signature --ignore-space-at-eol \
+ -o $$SUB_DIR/$$SUB_FEATURE -N $$F_TAG-root..$$F_TAG; \
+ done'
+ @sed -i -e '1d' -e 's/[[:space:]]*$$//' ${FPATCHES}
+
+# Apply patches from patch/* to respective submodules
+# We rely on `make sub` and/or `make clean` to checkout correct base
+.PHONY: patches-import
+patches-import: sub .cachepatched
+
+.cachepatched: ${FPATCHES}
+ @$(MAKE) clean
+ @git submodule -q foreach ' \
+ SUB_DIR=${F_PATCH_DIR}/$$name; mkdir -p $$SUB_DIR && \
+ git tag ${F_OPNFV_TAG}-root && \
+ git checkout -q -b opnfv-fuel && \
+ find $$SUB_DIR -type d | sort | while read p_dir; do \
+ SUB_PATCHES=$$(ls $$p_dir/*.patch 2>/dev/null); \
+ if [ -n "$$SUB_PATCHES" ]; then \
+ SUB_FEATURE=$${p_dir#$$SUB_DIR}; \
+ SUB_TAG=${F_OPNFV_TAG}-fuel$$SUB_FEATURE/patch; \
+ echo "`tput setaf 2`-- patching $$name ($$SUB_TAG)`tput sgr0`";\
+ git tag $$SUB_TAG-root && git am -3 --whitespace=nowarn \
+ --committer-date-is-author-date $$SUB_PATCHES && \
+ git tag $$SUB_TAG || exit 1; \
+ fi \
+ done && \
+ git tag ${F_OPNFV_TAG}'
+ @touch $@
+
+# Clean any changes made to submodules, checkout upstream Fuel root commit
+.PHONY: clean
+clean:
+ @git submodule -q foreach ' \
+ git am -q --abort > /dev/null 2>&1; \
+ git checkout -q ${F_OPNFV_TAG}-root > /dev/null 2>&1; \
+ git branch -q -D opnfv-fuel > /dev/null 2>&1; \
+ git tag | grep ${F_OPNFV_TAG} | xargs git tag -d > /dev/null 2>&1; \
+ git reset -q --hard HEAD; \
+ git clean -xdff'
+ @rm -f .cachepatched
+
+.PHONY: deepclean
+deepclean: clean clean-cache
+ @git submodule deinit -f .
+ @rm -f .cache*
+
+.PHONY: release
+release: sub
+ # Store artifact in cache straight away if caching is enabled
+ # (no .cacheid will be present unless this is a cached build)
+ test -f .cacheid && $(MAKE) -f Makefile put-cache || exit 0
+ # NOTE: Patches are not included in cache
+ $(MAKE) -f Makefile patches-import
+
+##############################################################################
+# Cache operations - only used when building through ci/build.sh
+##############################################################################
+
+# NOTE: Current method of collecting submodule refs requires submodules to be
+# NOT initialized <OR> NO patches applied.
+# NOTE: Querying `git submodule status` from parent will show the patched tree.
+# Create a unique hash to be used for getting and putting cache, based on:
+# - git submodule SHAs, collected with `git submodule status`
+# - The contents of this Makefile
+.cacheid:
+ @$(MAKE) clean
+ sha1sum Makefile > .cachedata
+ git submodule status | cut -c2-41 >> .cachedata
+ cat .cachedata | $(CACHETOOL) getid > .cacheid
+
+# Clean local data related to caching - called prior to ordinary build
+.PHONY: clean-cache
+clean-cache: clean
+ rm -f .cachedata .cacheid
+
+# Try to download cache - called prior to ordinary build
+.PHONY: get-cache
+get-cache: .cacheid
+ @if $(CACHETOOL) check $(shell cat .cacheid); then \
+ $(MAKE) clean && \
+ $(CACHETOOL) get $(shell cat .cacheid) | \
+ tar xf - -C ${F_GIT_ROOT}/.git; \
+ else \
+ echo "No cache item found for $(shell cat .cacheid)" ;\
+ exit 0;\
+ fi
+
+# Store cache if not already stored - called after ordinary build
+.PHONY: put-cache
+put-cache: .cacheid
+ @if ! $(CACHETOOL) check $(shell cat .cacheid); then \
+ $(MAKE) clean && \
+ tar cf - -C ${F_GIT_ROOT}/.git modules | \
+ $(CACHETOOL) put $(shell cat .cacheid); \
+ fi
diff --git a/build/f_repos/README.md b/build/f_repos/README.md
new file mode 100644
index 000000000..6cc825e05
--- /dev/null
+++ b/build/f_repos/README.md
@@ -0,0 +1,94 @@
+Fuel@OPNFV submodule fetching and patching
+==========================================
+
+This directory holds submodule fetching/patching scripts, intended for
+working with upstream Fuel components (fuel-library, ... , fuel-ui) in
+developing/applying OPNFV patches (backports, custom fixes etc.).
+
+The scripts should be friendly to the following 2 use-cases:
+ - development work: easily cloning, binding repos to specific commits,
+ remote tracking, patch development etc.;
+ - to provide parent build scripts an easy method of tracking upstream
+ references and applying OPNFV patches on top;
+
+Also, we need to support at least the following modes of operations:
+ - submodule bind - each submodule patches will be based on the commit ID
+ saved in the .gitmodules config file;
+ - remote tracking - each submodule will sync with the upstream remote
+ and patches will be applied on top of <sub_remote>/<sub_branch>/HEAD;
+
+Workflow (development)
+----------------------
+The standard development workflow should look as follows:
+
+1. Decide whether remote tracking should be active or not:
+ NOTE: Setting the following var to any non-empty str enables remote track.
+ NOTE: Leaving unset will enable remote track for anything but stable branch.
+
+ $ export FUEL_TRACK_REMOTES=""
+
+2. All Fuel sub-projects are registered as submodules. To initialize them, call:
+ If remote tracking is active, upstream remote is queried and latest remote
+ branch HEAD is fetched. Otherwise, checkout commit IDs from .gitmodules.
+
+ $ make sub
+
+3. Apply patches from `patches/<sub-project>/*` to respective submodules via:
+
+ $ make patches-import
+
+ This will result in creation of:
+ - a tag called `${FUEL_MAIN_TAG}-opnfv-root` at the same commit as Fuel@OPNFV
+ upstream reference (bound to git submodule OR tracking remote HEAD);
+ - a new branch `opnfv-fuel` which will hold all the OPNFV patches,
+ each patch is applied on this new branch with `git-am`;
+ - a tag called `${FUEL_MAIN_TAG}-opnfv` at `opnfv-fuel/HEAD`;
+
+4. Modify sub-projects for whatever you need.
+ Commit your changes when you want them taken into account in the build.
+
+5. Re-create patches via:
+
+ $ make patches-export
+
+ Each commit on `opnfv-fuel` branch of each subproject will be
+ exported to `patches/subproject/` via `git format-patch`.
+
+ NOTE: Only commit (-f) submodules when you need to bump upstream ref.
+ NOTE: DO NOT commit patched submodules!
+
+6. Clean workbench branches and tags with:
+
+ $ make clean
+
+7. De-initialize submodules and force a clean clone with:
+
+ $ make deepclean
+
+Workflow (ISO build)
+--------------------
+Parent build scripts require this mechanism to do some fingerprinting,
+so here is the intended flow for all artifacts to be generated right:
+
+1. (Optional) Cached submodules might be fetched from build cache.
+
+2. Submodules are updated
+ We also dump each submodule's git info using repo_info.sh, since
+ we want to collect git refs before patching (i.e. upstream refs).
+
+3. Make target `release` is built
+ This will make sure the modules are in a clean state, put them in cache,
+ then apply the patches.
+
+4. fuel-main's `${FUEL_MAIN_TAG}-opnfv-root` tag is used to determine VERSION info
+ It will accommodate both bound tags and remote tracking references.
+
+Sub-project maintenance
+-----------------------
+1. Adding a new submodule
+ If you need to add another subproject, you can do it with `git submodule`.
+ Make sure that you specify branch (with `-b`), short name (with `--name`)
+ and point it to `upstream/*` directory, i.e.:
+
+ $ git submodule -b stable/mitaka add --name fuel-web \
+ https://github.com/openstack/fuel-web.git upstream/fuel-web
diff --git a/build/patch-repos/build/repos/fuel-library/0001-Reduce-ceilometer-memory-usage.patch b/build/f_repos/patch/fuel-library/0001-Reduce-ceilometer-memory-usage.patch
index f7fe06884..e51e70b64 100644
--- a/build/patch-repos/build/repos/fuel-library/0001-Reduce-ceilometer-memory-usage.patch
+++ b/build/f_repos/patch/fuel-library/0001-Reduce-ceilometer-memory-usage.patch
@@ -1,11 +1,10 @@
-From 38315ee3945a5444de13918eb29e7771eb9927c7 Mon Sep 17 00:00:00 2001
From: Michael Polenchuk <mpolenchuk@mirantis.com>
Date: Thu, 8 Sep 2016 19:25:14 +0300
-Subject: [PATCH 1/1] Reduce ceilometer memory usage
+Subject: [PATCH] Reduce ceilometer memory usage
Change-Id: I8b1a97bd710c859a3543d1aed8226313f35f4f10
---
- .../manifests/ceilometer/controller.pp | 7 +++++++
+ .../puppet/openstack_tasks/manifests/ceilometer/controller.pp | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/deployment/puppet/openstack_tasks/manifests/ceilometer/controller.pp b/deployment/puppet/openstack_tasks/manifests/ceilometer/controller.pp
@@ -25,6 +24,3 @@ index 2c63ff0..1dda109 100644
+ }
}
}
---
-1.7.9.5
-
diff --git a/build/patch-repos/build/repos/fuel-library/0001-Disable-token-revoke-to-increase-keystone-performanc.patch b/build/f_repos/patch/fuel-library/0002-Disable-token-revoke-to-increase-keystone-performanc.patch
index 4d5be493b..d4baa9c1e 100644
--- a/build/patch-repos/build/repos/fuel-library/0001-Disable-token-revoke-to-increase-keystone-performanc.patch
+++ b/build/f_repos/patch/fuel-library/0002-Disable-token-revoke-to-increase-keystone-performanc.patch
@@ -1,4 +1,3 @@
-From ba542dde34743b276547b7ca3c6591feb9000877 Mon Sep 17 00:00:00 2001
From: iberezovskiy <iberezovskiy@mirantis.com>
Date: Mon, 19 Sep 2016 12:35:05 +0300
Subject: [PATCH] Disable token revoke to increase keystone performance
@@ -6,8 +5,8 @@ Subject: [PATCH] Disable token revoke to increase keystone performance
Change-Id: I4df816369093908ad1eac372f38c92155efbe8e0
Closes-bug: #1625077
---
- .../openstack_tasks/manifests/keystone/keystone.pp | 2 ++
- tests/noop/spec/hosts/keystone/keystone_spec.rb | 5 +++++
+ deployment/puppet/openstack_tasks/manifests/keystone/keystone.pp | 2 ++
+ tests/noop/spec/hosts/keystone/keystone_spec.rb | 5 +++++
2 files changed, 7 insertions(+)
diff --git a/deployment/puppet/openstack_tasks/manifests/keystone/keystone.pp b/deployment/puppet/openstack_tasks/manifests/keystone/keystone.pp
@@ -21,7 +20,7 @@ index 3162287..ba9d7df 100644
+ # Set revoke_by_id to false according to LP #1625077
+ revoke_by_id => false,
}
-
+
Package<| title == 'keystone'|> ~> Service<| title == 'keystone'|>
diff --git a/tests/noop/spec/hosts/keystone/keystone_spec.rb b/tests/noop/spec/hosts/keystone/keystone_spec.rb
index b29e691..3c0f847 100644
@@ -30,7 +29,7 @@ index b29e691..3c0f847 100644
@@ -176,6 +176,11 @@ describe manifest do
should contain_class('keystone').with('sync_db' => primary_controller)
end
-
+
+ it 'should declare keystone class with revoke_by_id set to false' do
+ # Set revoke_by_id to false according to LP #1625077
+ should contain_class('keystone').with('revoke_by_id' => false)
@@ -39,6 +38,3 @@ index b29e691..3c0f847 100644
it 'should configure keystone with paramters' do
should contain_keystone_config('token/caching').with(:value => 'false')
should contain_keystone_config('cache/enabled').with(:value => 'true')
---
-1.7.9.5
-
diff --git a/build/patch-repos/0010-bootstrap_admin_node.sh.patch b/build/f_repos/patch/fuel-main/0001-OPNFV-Additions-to-bootstrap_admin_node.sh.patch
index d3769adfb..446d0b6bd 100644
--- a/build/patch-repos/0010-bootstrap_admin_node.sh.patch
+++ b/build/f_repos/patch/fuel-main/0001-OPNFV-Additions-to-bootstrap_admin_node.sh.patch
@@ -1,10 +1,13 @@
From: Fuel OPNFV <fuel@opnfv.org>
Date: Mon, 13 Jun 2016 22:23:57 +0200
-Subject: OPNFV: Additions to bootstrap_admin_node.sh
+Subject: [PATCH] OPNFV: Additions to bootstrap_admin_node.sh
---
+ iso/bootstrap_admin_node.sh | 39 +++++++++++++++++++++++++++++++--------
+ 1 file changed, 31 insertions(+), 8 deletions(-)
+
diff --git a/iso/bootstrap_admin_node.sh b/iso/bootstrap_admin_node.sh
-index 3197c91..e035145 100755
+index 3197c91..db3123d 100755
--- a/iso/bootstrap_admin_node.sh
+++ b/iso/bootstrap_admin_node.sh
@@ -339,8 +339,22 @@ fuelmenu --save-only --iface=$ADMIN_INTERFACE || fail
@@ -44,7 +47,9 @@ index 3197c91..e035145 100755
if [ ! -f "${ASTUTE_YAML}" ]; then
echo ${fuelmenu_fail_message}
-@@ -377,7 +392,5 @@ if [ ! -f /etc/fuel_build_id ]; then
+@@ -377,9 +392,7 @@ if [ ! -f /etc/fuel_build_id ]; then
+ ln -s ${wwwdir}/${OPENSTACK_VERSION}/ubuntu ${wwwdir}/ubuntu
+ fi
-# Enable sshd
-systemctl enable sshd
@@ -70,7 +75,7 @@ index 3197c91..e035145 100755
if [ "`get_bootstrap_skip`" = "False" ]; then
build_ubuntu_bootstrap bs_status || true
else
-@@ -571,8 +571,8 @@
+@@ -540,8 +563,8 @@ else
fi
#Check if repo is accessible
diff --git a/build/patch-repos/0020-isolinux.cfg.patch b/build/f_repos/patch/fuel-main/0002-OPNFV-showmenu-yes-in-isolinux.cfg.patch
index f09be2a7f..dbff8c7d4 100644
--- a/build/patch-repos/0020-isolinux.cfg.patch
+++ b/build/f_repos/patch/fuel-main/0002-OPNFV-showmenu-yes-in-isolinux.cfg.patch
@@ -1,8 +1,11 @@
From: Fuel OPNFV <fuel@opnfv.org>
Date: Mon, 13 Jun 2016 22:23:57 +0200
-Subject: OPNFV: showmenu=yes in isolinux.cfg
+Subject: [PATCH] OPNFV: showmenu=yes in isolinux.cfg
---
+ iso/isolinux/isolinux.cfg | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
diff --git a/iso/isolinux/isolinux.cfg b/iso/isolinux/isolinux.cfg
index c6b1ed9..77a4b18 100644
--- a/iso/isolinux/isolinux.cfg
diff --git a/build/patch-repos/0030-repo-multi-arch-local-mirrors.patch b/build/f_repos/patch/fuel-main/0003-repo-mirror-Allow-multi-arch-local-mirrors.patch
index d40428743..9469643c3 100644
--- a/build/patch-repos/0030-repo-multi-arch-local-mirrors.patch
+++ b/build/f_repos/patch/fuel-main/0003-repo-mirror-Allow-multi-arch-local-mirrors.patch
@@ -1,6 +1,6 @@
From: Stanislaw Kardach <stanislaw.kardach@cavium.com>
Date: Thu, 25 Feb 2016 13:31:19 +0100
-Subject: repo mirror: Allow multi-arch local mirrors
+Subject: [PATCH] repo mirror: Allow multi-arch local mirrors
This patch allows specifying multiple architectures via UBUNTU_ARCH in
form of a list of space separated architectures. The first architecture
@@ -13,11 +13,24 @@ independent (which is true so far).
Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com>
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
---
+ config.mk | 2 +-
mirror/ubuntu/module.mk | 2 +-
sandbox.mk | 2 +-
- config.mk | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
+diff --git a/config.mk b/config.mk
+index 74ee039..45a3b30 100644
+--- a/config.mk
++++ b/config.mk
+@@ -49,7 +49,7 @@ UBUNTU_MINOR:=04
+ UBUNTU_RELEASE_NUMBER:=$(UBUNTU_MAJOR).$(UBUNTU_MINOR)
+ UBUNTU_KERNEL_FLAVOR?=lts-trusty
+ UBUNTU_NETBOOT_FLAVOR?=netboot
+-UBUNTU_ARCH:=amd64
++UBUNTU_ARCH?=amd64
+ UBUNTU_IMAGE_RELEASE:=$(UBUNTU_MAJOR)$(UBUNTU_MINOR)
+ SEPARATE_IMAGES?=/boot,ext2 /,ext4
+
diff --git a/mirror/ubuntu/module.mk b/mirror/ubuntu/module.mk
index 7a9466e..fe1ada2 100644
--- a/mirror/ubuntu/module.mk
@@ -44,16 +57,3 @@ index 4bc3962..5ffddbd 100644
if [ -e $(SANDBOX_UBUNTU)/etc/resolv.conf ]; then sudo cp -a $(SANDBOX_UBUNTU)/etc/resolv.conf $(SANDBOX_UBUNTU)/etc/resolv.conf.orig; fi
sudo cp /etc/resolv.conf $(SANDBOX_UBUNTU)/etc/resolv.conf
if [ -e $(SANDBOX_UBUNTU)/etc/hosts ]; then sudo cp -a $(SANDBOX_UBUNTU)/etc/hosts $(SANDBOX_UBUNTU)/etc/hosts.orig; fi
-diff --git a/config.mk b/config.mk
-index 74ee039..45a3b30 100644
---- a/config.mk
-+++ b/config.mk
-@@ -49,7 +49,7 @@ UBUNTU_MINOR:=04
- UBUNTU_RELEASE_NUMBER:=$(UBUNTU_MAJOR).$(UBUNTU_MINOR)
- UBUNTU_KERNEL_FLAVOR?=lts-trusty
- UBUNTU_NETBOOT_FLAVOR?=netboot
--UBUNTU_ARCH:=amd64
-+UBUNTU_ARCH?=amd64
- UBUNTU_IMAGE_RELEASE:=$(UBUNTU_MAJOR)$(UBUNTU_MINOR)
- SEPARATE_IMAGES?=/boot,ext2 /,ext4
-
diff --git a/build/patch-repos/0040-xorriso-fails-to-add-files-with-path-longer-then-240.patch b/build/f_repos/patch/fuel-main/0004-xorriso-fails-to-add-files-with-path-longer-then-240.patch
index 734523040..734523040 100644
--- a/build/patch-repos/0040-xorriso-fails-to-add-files-with-path-longer-then-240.patch
+++ b/build/f_repos/patch/fuel-main/0004-xorriso-fails-to-add-files-with-path-longer-then-240.patch
diff --git a/build/f_repos/patch/fuel-mirror/0001-Fixed-handling-http-redirects.patch b/build/f_repos/patch/fuel-mirror/0001-Fixed-handling-http-redirects.patch
new file mode 100644
index 000000000..b82be1128
--- /dev/null
+++ b/build/f_repos/patch/fuel-mirror/0001-Fixed-handling-http-redirects.patch
@@ -0,0 +1,87 @@
+From: Bulat Gaifullin <bgaifullin@mirantis.com>
+Date: Fri, 24 Jun 2016 15:52:41 +0200
+Subject: [PATCH] Fixed handling http redirects
+
+Override method 'redirect_request' for patching
+new request, that has been created on handling
+http redirect.
+
+Change-Id: I40db406e2377bebec1113639b91a0b5262e2e9ad
+Closes-Bug: 1593674
+(cherry picked from commit 192a3d9f8f993afb12c5108dd9339c6688c23e11)
+---
+ packetary/library/connections.py | 21 ++++++++++++++++++++-
+ packetary/tests/test_connections.py | 19 +++++++++++++++++++
+ 2 files changed, 39 insertions(+), 1 deletion(-)
+
+diff --git a/packetary/library/connections.py b/packetary/library/connections.py
+index 36a7a84..49b6c9b 100644
+--- a/packetary/library/connections.py
++++ b/packetary/library/connections.py
+@@ -93,9 +93,23 @@ class ResumableResponse(StreamWrapper):
+ self.stream = response.stream
+
+
+-class RetryHandler(urllib.BaseHandler):
++class RetryHandler(urllib.HTTPRedirectHandler):
+ """urllib Handler to add ability for retrying on server errors."""
+
++ def redirect_request(self, req, fp, code, msg, headers, newurl):
++ new_req = urllib.HTTPRedirectHandler.redirect_request(
++ self, req, fp, code, msg, headers, newurl
++ )
++ if new_req is not None:
++ # We use class assignment for casting new request to type
++ # RetryableRequest
++ new_req.__class__ = RetryableRequest
++ new_req.retries_left = req.retries_left
++ new_req.offset = req.offset
++ new_req.start_time = req.start_time
++ new_req.retry_interval = req.retry_interval
++ return new_req
++
+ @staticmethod
+ def http_request(request):
+ """Initialises http request.
+@@ -118,6 +132,11 @@ class RetryHandler(urllib.BaseHandler):
+ :return: ResumableResponse if success otherwise same response
+ """
+ code, msg = response.getcode(), response.msg
++
++ if 300 <= code < 400:
++ # the redirect group, pass to next handler as is
++ return response
++
+ # the server should response partial content if range is specified
+ if request.offset > 0 and code != 206:
+ raise RangeError(msg)
+diff --git a/packetary/tests/test_connections.py b/packetary/tests/test_connections.py
+index a2621c8..c80b03d 100644
+--- a/packetary/tests/test_connections.py
++++ b/packetary/tests/test_connections.py
+@@ -268,6 +268,25 @@ class TestRetryHandler(base.TestCase):
+ self.handler.http_response(request, response_mock)
+ self.handler.parent.open.assert_called_once_with(request)
+
++ @mock.patch(
++ 'packetary.library.connections.urllib.'
++ 'HTTPRedirectHandler.redirect_request'
++ )
++ def test_redirect_request(self, redirect_mock, _):
++ redirect_mock.return_value = connections.urllib.Request(
++ 'http://localhost/'
++ )
++ req = mock.MagicMock(retries_left=10, retry_interval=5, offset=100)
++ new_req = self.handler.redirect_request(req, -1, 301, "", {}, "")
++ self.assertIsInstance(new_req, connections.RetryableRequest)
++ self.assertEqual(req.retries_left, new_req.retries_left)
++ self.assertEqual(req.retry_interval, new_req.retry_interval)
++ self.assertEqual(req.offset, new_req.offset)
++ redirect_mock.return_value = None
++ self.assertIsNone(
++ self.handler.redirect_request(req, -1, 301, "", {}, "")
++ )
++
+
+ class TestResumeableResponse(base.TestCase):
+ def setUp(self):
diff --git a/build/patch-repos/build/repos/fuel-nailgun/0010-Mark-Intel-82599-10-Gigabit-NIC-as-DPDK-capable.patch b/build/f_repos/patch/fuel-web/0001-Mark-Intel-82599-10-Gigabit-NIC-as-DPDK-capable.patch
index d486f2354..6fa1e436c 100644
--- a/build/patch-repos/build/repos/fuel-nailgun/0010-Mark-Intel-82599-10-Gigabit-NIC-as-DPDK-capable.patch
+++ b/build/f_repos/patch/fuel-web/0001-Mark-Intel-82599-10-Gigabit-NIC-as-DPDK-capable.patch
@@ -12,11 +12,12 @@ diff --git a/nailgun/nailgun/fixtures/openstack.yaml b/nailgun/nailgun/fixtures/
index 74fa509..d08a4fc 100644
--- a/nailgun/nailgun/fixtures/openstack.yaml
+++ b/nailgun/nailgun/fixtures/openstack.yaml
-@@ -2215,3 +2215,3 @@
+@@ -2213,7 +2213,7 @@
+ "15ad:07b0", "8086:15a5", "1137:0043", "1137:0071", "14e4:168a",
+ "14e4:16a9", "14e4:164f", "14e4:168e", "14e4:16af", "14e4:163d",
"14e4:163f", "14e4:168d", "14e4:16a1", "14e4:16a2", "14e4:16ad",
- "14e4:16ae", "14e4:163e", "14e4:16a4"
+ "14e4:16ae", "14e4:163e", "14e4:16a4", "8086:10f8"
]
---
-1.9.1
-
+ - pk: 3
+ extend: *ubuntu_release
diff --git a/build/f_repos/sub/fuel-agent b/build/f_repos/sub/fuel-agent
new file mode 160000
+Subproject 7ffbf39caf5845bd82b8ce20a7766cf24aa803f
diff --git a/build/f_repos/sub/fuel-astute b/build/f_repos/sub/fuel-astute
new file mode 160000
+Subproject 390b257240d49cc5e94ed5c4fcd940b5f2f6ec6
diff --git a/build/f_repos/sub/fuel-library b/build/f_repos/sub/fuel-library
new file mode 160000
+Subproject e283b62750d9e26355981b3ad3be7c880944ae0
diff --git a/build/f_repos/sub/fuel-main b/build/f_repos/sub/fuel-main
new file mode 160000
+Subproject d6a22557d132c592b18c6bac90f5f4b8d1aa3ad
diff --git a/build/f_repos/sub/fuel-menu b/build/f_repos/sub/fuel-menu
new file mode 160000
+Subproject 0ed9e206ed1c6271121d3acf52a6bf757411286
diff --git a/build/f_repos/sub/fuel-mirror b/build/f_repos/sub/fuel-mirror
new file mode 160000
+Subproject d1ef06b530ce2149230953bb3810a88ecaff870
diff --git a/build/f_repos/sub/fuel-nailgun-agent b/build/f_repos/sub/fuel-nailgun-agent
new file mode 160000
+Subproject 46fa0db0f8944f9e67699d281d462678aaf4db2
diff --git a/build/f_repos/sub/fuel-ostf b/build/f_repos/sub/fuel-ostf
new file mode 160000
+Subproject f09c98ff7cc71ee612b2450f68a19f2f9c64345
diff --git a/build/f_repos/sub/fuel-ui b/build/f_repos/sub/fuel-ui
new file mode 160000
+Subproject 90de7ef4477230cb7335453ed26ed4306ca6f04
diff --git a/build/f_repos/sub/fuel-upgrade b/build/f_repos/sub/fuel-upgrade
new file mode 160000
+Subproject c1c4bac6a467145ac4fac73e4a7dd2b00380ecf
diff --git a/build/f_repos/sub/fuel-web b/build/f_repos/sub/fuel-web
new file mode 160000
+Subproject e2b85bafb68c348f25cb7cceda81edc668ba2e6
diff --git a/build/f_repos/sub/network-checker b/build/f_repos/sub/network-checker
new file mode 160000
+Subproject fcb47dd095a76288aacf924de574e39709e1f3c
diff --git a/build/f_repos/sub/python-fuelclient b/build/f_repos/sub/python-fuelclient
new file mode 160000
+Subproject 67d8c693a670d27c239d5d175f3ea2a0512c498
diff --git a/build/f_repos/sub/shotgun b/build/f_repos/sub/shotgun
new file mode 160000
+Subproject 781a8cfa0b6eb290e730429fe2792f2b6f5e0c1