Support for building Fuel behind a http proxy

The build system is now able to work also behind a traditional web
proxy setup if the http_proxy, https_proxy and (if needed) no_proxy
environment variables has been set prior to invoking make.

This is a joint work by Gillian Dunne <gillian.dunne@intel.com> and
Stefan Berg <stefan.k.berg@ericsson.com>.

Verification so far has been with a mock setup, placing the build
machine behind a Squid proxy and blocking outgoing traffic not going
through the proxy by firewall rules. The following environment
variables was set in the host for these tests:

RSYNC_PROXY=10.0.0.1:8888
http_proxy=http://10.0.0.1:8888
https_proxy=http://10.0.0.1:8888
no_proxy=localhost,127.0.0.1,.consultron.com,.sock

*** IMPORTANT NOTE ABOUT THE HOST PROXY SETTINGS ***

The build system will make use the following proxy environment
variables:

http_proxy:
https_proxy
no_proxy
RSYNC_PROXY
RSYNC_CONNECT_PROG

During the build phase, a local Ubuntu package repository is fetched
from upstream in order to be added to the OPNFV Fuel ISO and for parts
of this process rsync is used.

This will require that either RSYNC_PROXY is set according to the
format "<proxy host>:<proxy port>" and that the proxy indicated indeed
allows rsync traffic *or* that RSYNC_CONNECT_PROG is set to use an
alternative transport. For a detailed explanation of these settings,
see the rsync manual page.

*** IMPORTANT NOTE ABOUT THE HOST DOCKER DAEMON SETTINGS ***

The Docker daemon on the host must be configured to use the http proxy
for it to be able to pull the base Ubuntu 14.04 image from the Docker
registry before invoking make! In Ubuntu this is done by adding a line
like:

export http_proxy="http://10.0.0.1:8888/"

to /etc/default/docker and restarting the Docker daemon.

Change-Id: Ieed2269af295d90a4b33d834f723889bdf9c7dc6
Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>

1 files changed, 35 insertions, 5 deletions

diff --git a/fuel/build/Makefile b/fuel/build/Makefile
index 767b930d7..80ef8ea51 100644
--- a/fuel/build/Makefile
+++ b/fuel/build/Makefile
@@ -9,6 +9,11 @@
 ##############################################################################
 
 SHELL = /bin/bash
+
+# This file will be created if needed by runcontext to contain proxy
+# settings.
+-include environment.mk
+
 ############################################################################
 # BEGIN of variables to customize
 #
@@ -77,6 +82,8 @@ $(ISOCACHE):
 	cd /tmp/fuel-main && patch -p0 < $(TOPDIR)/fuel-main_3.patch
 	# Patch for adding dosfstools, as Fuel 6.1 is running mkfs.vfat
 	cd /tmp/fuel-main && patch -p0 < $(TOPDIR)/fuel-main_5.patch
+	# Patch for changing the second layer Docker --bip address
+	cd /tmp/fuel-main && patch -p0 < $(TOPDIR)/fuel-main_6.patch
 	# Remove Docker optimizations, otherwise multistrap will fail during
 	# Fuel build.
 	sudo rm -f /etc/apt/apt.conf.d/docker*
@@ -85,8 +92,8 @@ $(ISOCACHE):
 	cd /tmp/fuel-main && make repos
 	$(REPOINFO) -r /tmp/fuel-main > gitinfo_fuel.txt
 	#
-	cd /tmp/fuel-main && make iso
-	mv /tmp/fuel-main/build/artifacts/fuel*.iso .
+	cd /tmp/fuel-main && sudo make iso
+	cp /tmp/fuel-main/build/artifacts/fuel*.iso .
 
 .PHONY: mount-origiso umount-origiso
 mount-origiso: $(ISOCACHE)
@@ -114,16 +121,38 @@ clean:  $(SUBCLEAN)
 	@rm -f *.iso
 	@rm -Rf release
 	@rm -Rf newiso
+	@rm -Rf .versions
 	@rm -f $(NEWISO)
 	@rm -f $(BUILD_BASE)/gitinfo_*.txt
 
+.PHONY: deepclean
+deepclean: clean clean-cache
+	make -C docker clean
+	docker rmi opnfv.org/ubuntu-builder:14.04 &>/dev/null || exit 0
+	docker rmi opnfv.org/ubuntu-builder:latest &>/dev/null || exit 0
+	docker rmi ubuntu:14.04 &>/dev/null || exit 0
+	@if docker images | grep -q "ubuntu *14.04"; then \
+		echo "Error: ubuntu:14.04 still present!"; \
+		exit 1; \
+	fi
+	@if docker images | grep -q "opnfv.org/ubuntu-builder"; then \
+		echo "Error: opnfv.org/ubuntu-builder still present!"; \
+		exit 1; \
+	fi
+
 $(SUBCLEAN): %.clean:
 	$(MAKE) -C $* -f Makefile clean
 
+.PHONY: setup-env
+setup-env:
+	@if [ -f environment.mk ]; then \
+		sudo bash -c "cat environment.mk >> /etc/environment"; \
+	fi
+
 # Todo: Make things smarter - we shouldn't need to clean everything
 # betwen make invocations.
 .PHONY: iso
-iso:	$(ISOCACHE) $(SUBDIRS) patch-packages
+iso:	setup-env $(ISOCACHE) $(SUBDIRS) patch-packages
 	$(REPOINFO) . > gitinfo_main.txt
 	install/install.sh iso $(ISOCACHE) $(NEWISO) $(PRODNO) $(REVSTATE)
 	@printf "\n\nProduct ISO is $(NEWISO)\n\n"
@@ -132,7 +161,8 @@ iso:	$(ISOCACHE) $(SUBDIRS) patch-packages
 .PHONY: debug
 debug:
 	@docker version >/dev/null 2>&1 || (echo 'No Docker installation available'; exit 1)
-	docker/runcontext $(DOCKERIMG) bash
+	@make -C docker
+	docker/runcontext $(DOCKERIMG) debug
 
 #############################################################################
 # Cache operations - only used when building through ci/build.sh
@@ -152,7 +182,7 @@ debug:
 
 # Clean local data related to caching - called prior to ordinary build
 .PHONY: clean-cache
-clean-cache:
+clean-cache: $(SUBCLEANCACHE)
 	rm -f .cachedata .cacheid
 
 # Try to download cache - called prior to ordinary build