diff options
| author |   Jonas Bjurel <jonas.bjurel@ericsson.com> | 2015-03-02 22:46:45 +0100 |
|---|---|---|
| committer | Jonas Bjurel <jonas.bjurel@ericsson.com> | 2015-03-09 23:00:13 +0100 |
| commit | 6cb518c68eb136ad448e96716ef8b1348440949f (patch) | |
| tree | 757551a8efdc4e85e19fc1b0f27555825580e5c9 /fuel/build | |
| parent | 1e106bbaca2780c10bd02b4e5fa3702d36acdf39 (diff) | |
Starting point for ODL Integration
***CAN BE MERGED***
- ODL build and installation (puppet manifests) including needed dependencies
- Modified in accordance with Randy Levensalors comments
- Added chmod 4755 /bin/fusermount due to that lately some systems dont install fuseiso
withe SUID set for fusermount. Investing ongoing on why and why not consistent.
- Rebased with master
- ODL disabled in top Make (enable it by uncommenting the odl dirs
- Removed white spaces
- Hopefully fixed an issue with the docker ubuntu repos, added --no-cache=true to the fuel/docker/Make file.
TODO:
- Refactor collecting ODL and Java dependencies to one common place
- Refactoring top Makefile
- Adding CI hooks for build and deploy
- Finishing integration plumbing wit Neutron/ML2 and OVS/OVSDB
Change-Id: I13dd4d3a72d099316dfaa9f7cd9140781be2397b
Signed-off-by: Jonas Bjurel <jonas.bjurel@ericsson.com>
Diffstat (limited to 'fuel/build')
133 files changed, 7030 insertions, 0 deletions
diff --git a/fuel/build/DOC/BUILD/README.architecture b/fuel/build/DOC/BUILD/README.architecture new file mode 100644 index 0000000..4629763 --- /dev/null +++ b/fuel/build/DOC/BUILD/README.architecture @@ -0,0 +1,10 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + diff --git a/fuel/build/DOC/BUILD/README.build b/fuel/build/DOC/BUILD/README.build new file mode 100644 index 0000000..519ee6a --- /dev/null +++ b/fuel/build/DOC/BUILD/README.build @@ -0,0 +1,97 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +How to setup and use the OPNFV/BGS/FUEL build system +==================================================== + +1. Build system requirements +---------------------------- +1.1 Host HW/VM requirements +--------------------------- +- An x86_64 host (real iron or VM) with Ubuntu 14.04 LTS installed +- ~30 GB available disc +- 4 GB RAM + +1.2 Host SW dependencies +------------------------ +The host should run Ubuntu 14.04 +On the host, the following packages must be installed: +- docker - see https://docs.docker.com/installation/ubuntulinux/ for installation notes +for Ubuntu 14.04. Note: only use the Ubuntu stock distro of Docker (docker.io) +- git (simply available through apt-get install git) +- make (simply available through apt-get install make) + +2 Setting up the Docker build container +---------------------------------------- +When having installed Docker, add yourself to the docker group: +usermod -a -G docker <userid> + +Also make sure to define rellevant dns servers part of the global dns chain in +in your /etc/default/docker config file, eg. +DOCKER_OPTS=" --dns=8.8.8.8 --dns=8.8.8.4" + +Then restart docker with "service docker.io restart".: + +3. Setting up OPNFV Gerrit in order to being able to clone the code +------------------------------------------------------------------- +- Start by creating a SSH key, if you don't already have one, create one with ssh-keygen + +- Add your generated public key in OPNFV Gerrit -https://gerrit.opnfv.org/ + (this requires a linuxfoundation account, create one if you do not already have one) + +- Select "SSH Public Keys" to the left and then "Add Key" and paste your public key in. + +4. Clone the OPNFV code git repository +-------------------------------------- + So now we're coming to the fun part! Let's clone the code repository: + git clone ssh://<Linux foundation user>@gerrit.opnfv.org:29418/genesis + + Change the directory to "fuel-build" + cd fuel-build/ + +5. Set up (build) your build environment +---------------------------------------- +Select the versions of the components you want to build by editing the +fuel-build/config-spec file. Note if you want to build with OpenDaylight +SDN controller you need to uncomment the lines starting with odl-main and +java-main + +6. Build your fuel .iso image +----------------------------- +That's it, time to build! +make [all] + +This will: +- If not allready existing, initialize the docker build environment +- If not already done, build OpenDaylight from upstream (as defined by fuel-build/ + config-spec) +- If not already done, build fuel from upstream (as defined by fuel-build/config-spec) +- Build the defined additions to fuel (as defined by the structure of this framework) +- Apply changes and patches to fuel (as defined by the structure of this framework) +- Reconstruct a fuel .iso image + +If the build is successful, you will find the generated ISO file in the release +subdirectory! + +NOTE: the build of the baseline for Fuel and odl are cached, if build results reffering +to the same versions as specified in fuel-build/config spec has been built before, these +packages will not be rebuilt. The cache is cleared by "make clean" + +7. Install your stack +--------------------- +Please see DOC/INSTALL/README + +NOTES and TODO: +--------------- +The build system will for now partly run as sudo +BUT NOTE, NONE OF THE MAKEFILES OR SCRIPTS SHALL BE EXECUTED WITH +SUDO/ROOT PRIVILEDGES! +TODO: Change the scripts so that no root priviledges will be needed + diff --git a/fuel/build/DOC/BUILD/README.examples b/fuel/build/DOC/BUILD/README.examples new file mode 100644 index 0000000..4629763 --- /dev/null +++ b/fuel/build/DOC/BUILD/README.examples @@ -0,0 +1,10 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + diff --git a/fuel/build/DOC/INSTALL/README b/fuel/build/DOC/INSTALL/README new file mode 100644 index 0000000..456e26d --- /dev/null +++ b/fuel/build/DOC/INSTALL/README @@ -0,0 +1,154 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + + +This README file describes how to install the OPNFV BGS Stack +============================================================= +1.1 Install Fuel Server +----------------------- +1. Mount the built .iso file (release/opnfv-<version>.iso) as a bootdevice to the fuel server. + +2. Reboot the fuel server. + The system now boots from the ISO image. + +3. Change the grub boot params + When Press Tab to edit . is shown press Tab. + Change showmenu=no to showmenu=yes. + Change netmask=255.255.255.0 to netmask=255.255.0.0. + Press Enter. + +4 Wait until screen Fuel setup . is shown. + Note: This takes about 30 minutes. + +5. Select PXE Setup and change the following fields to appropriate values (example below): + Static Pool Start 10.20.0.3 + Static Pool End 10.20.0.254 + DHCP Pool Start 10.20.128.3 + DHCP Pool End 10.20.128.254 + +6. Select DNS & Hostname and change the following fields to appropriate values: + Hostname <CEE Region name>-fuel + Domain <Domain Name> + Search Domain <Search Domain Name> + Hostname to test DNS <Hostname to test DNS> + +7. Select Time Sync and change the following fields to appropriate values: + NTP Server 1 <Customer NTP server 1> + NTP Server 2 <Customer NTP server 2> + NTP Server 3<Customer NTP server 3> + Note: This step is only to pass the network test, the actual ntp parameters will be set with the pre_deply script. + +8. Start the installation. + Select Quit Setup and press Save and Quit. + Installation starts, wait until a screen with logon credentials is shown. + Note: This takes about 15 minutes. + +10. Enable PXE Booting and put it first in the bios boot order menu. + +11. Reboot the controller- and compute blades. + +12. Connect to Fuel wit a browser towards port 8000 + +13. Create and name a new OpenStack environment, to be installed. + +14. Select Icehouse on Ubuntu . in field OpenStack Release. + +15. Select deployment mode. + Select the Multi-node with HA. + +16. Select compute node mode. + Select KVM as hypervisor. + +17. Select network mode. + Select Neutron with VLAN segmentation. + +18. Select Storage Backends. + Select Ceph for Cinder and default for glance. + + +19. Select additional services. + Check option Install Celiometer (OpenStack Telemetry). + +20. Create the new environment. + +21. Wait for availability nodes. + Wait until all nodes are displayed in top right in <total number of server> TOTAL NODES and <total number of servers> UNALLOCATED NODES. + +22. Open the created environment. + +23. Open the networks tab. + +24. Update the public configuration. + Change the following fields to appropriate values: + IP Range Start to <Public IP Address start> + IP Range End to <Public IP Address end> + CIDR to <CIDR for Public IP Addresses> + Gateway to <Gateway for Public IP Addresses> + Check VLAN tagging. + Set appropriate VLAN id. + +25. Update the management configuration. + Set CIDR to 172.16.255.128/25. + Check VLAN tagging. + Set appropriate VLAN id. + +26. Update the storage configuration. + Set CIDR to the appropriate value + Check VLAN tagging. + Set appropriate VLAN id. + +27. Update the Neutron L2 configuration. + Set VLAN ID range. + +28. Update the Neutron L3 configuration. + Set Internal network CIDR to an appropriate value + Set Internal network gateway to an appropriate value + Set Floating IP ranges. + Set DNS Servers + +29. Save Settings. + +30. Click "verify network" to check the network set-up + +31. Update storage configuration. + +32. Open the nodes tab. + +33. Assign roles. + Check Controller and Telemetry . MongoDB. + Check three nodes to be installed as Controller in pane Assign Role. + Click Apply Changes. + Check Compute. + Check nodes to be installed as Compute nodes in pane Assign Role. + Click Apply Changes. + +34. Configure interfaces. + Check Select All to select all nodes with Control, Telemetry . MongoDB and Compute node roles. + Click Configure Interfaces. + Screen Configure interfaces on number of <number of nodes> nodes is shown. + Assign interfaces (bonded) for mgmt-, admin-, private-, public- and storage networks + +35. Run the pre-deploy script. + Log on as root to the Fuel node. + Print Fuel environment Id (fuel env) + #> id | status | name | mode | release_id | changes <id>| new | <CEE Region name>| ha_compact | 2 | <ite specific information> + +36. Run the pre-deployment script (/opt/opnfv/pre-deploy.sh <id>) + As prompted for-, set the DNS servers to go into /etc/resolv.conf. + As prompted for-, set any Hosts file additions for controllers and compute nodes. You will be prompted for name, FQDN and IP for each entry. Press return when prompted for a name when you have completed your input. + As prompted for-, set NTP upstream configuration for controllers. You will be prompted for a NTP server each entry. Press return when prompted for a NTP server when you have completed your input. + +37. Deploy the environment. + In the Fuel GUI, click Deploy Changes. + +38 Verify the OpenStack Environment + Open the Health Check tab . + Select Select All. + Click Run tests. diff --git a/fuel/build/Makefile b/fuel/build/Makefile new file mode 100644 index 0000000..8e90fcc --- /dev/null +++ b/fuel/build/Makefile @@ -0,0 +1,140 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +############################################################################ +# BEGIN of variables to customize +# +SHELL = /bin/bash + +include config.mk + +export BUILD_BASE = $(shell pwd) +export DEB_DEST := $(BUILD_BASE)/release/packages/ubuntu/pool/main +export PUPPET_DEST := $(BUILD_BASE)/release/puppet/modules + +ISOSRC = file:$(shell pwd)/fuel-6.0.1.iso +ISOCACHE = $(shell pwd)/$(shell basename $(ISOSRC)) +PRODNO = "OPNFV_BGS" +REVSTATE = "P0000" +NEWISO = $(shell pwd)/release/opnfv-${REVSTATE}.iso +DOCKERIMG = opnfv.org/ubuntu-builder:14.04 +# Note! Invoke with "make REVSTATE=RXXXX iso" to make release build! +# Invoke with ICOCACHE=/full/path/to/iso if cached ISO is in non-standard location. +# +# END of variables to customize +############################################################################# +export TOPDIR := $(shell pwd) +ORIGDIR := $(TOPDIR)/origiso + +SUBDIRS := f_isoroot +SUBDIRS += f_opnfv_puppet +SUBDIRS += f_osnaily +SUBDIRS += f_l23network +SUBDIRS += f_resolvconf +SUBDIRS += f_ntp +#SUBDIRS += f_gif +#SUBDIRS += f_libnss +#SUBDIRS += f_libpcs +#SUBDIRS += f_libxt +#SUBDIRS += f_comm_java +#SUBDIRS += f_ca_cert +#SUBDIRS += f_atk_wrap +#SUBDIRS += f_jre_head +#SUBDIRS += f_jre +#SUBDIRS += f_java +#SUBDIRS += f_odl + + + +# f_example is only an example of how to generate a .deb package and +# should not be enabled in official builds. +#SUBDIRS += f_example + +SUBCLEAN = $(addsuffix .clean,$(SUBDIRS)) + +.PHONY: all +all: + @docker version >/dev/null 2>&1 || (echo 'No Docker installation available'; exit 1) + @make -C docker + @docker/runcontext $(DOCKERIMG) $(MAKE) $(MAKEFLAGS) iso + +$(ISOCACHE): + # Clone Fuel to non-persistent location and build + cd /tmp && git clone $(FUEL_MAIN_REPO) + cd /tmp/fuel-main && git checkout $(FUEL_MAIN_TAG) + # Setup cgroups for docker-in-docker + sudo /root/enable_dockerx2 + # Patch to fix race condition when doing "Docker-in-Docker" build + cd /tmp/fuel-main && patch -p1 < $(TOPDIR)/fuel-main_1.patch + # Patch to make the sandbox chroot in Fuel succeed with package + # installation in a Docker build + cd /tmp/fuel-main && patch -p1 < $(TOPDIR)/fuel-main_2.patch + # Remove Docker optimizations, otherwise multistrap will fail during + # Fuel build. + sudo rm -f /etc/apt/apt.conf.d/docker* + # + cd /tmp/fuel-main && ./prepare-build-env.sh + cd /tmp/fuel-main && make iso + mv /tmp/fuel-main/build/artifacts/fuel*.iso . + +.PHONY: mount-origiso umount-origiso +mount-origiso: $(ISOCACHE) + @echo "Mounting original ISO in $(ORIGDIR)" + @mkdir -p $(ORIGDIR) + @fuseiso $(ISOCACHE) $(ORIGDIR) + +umount-origiso: + @echo "Unmounting original ISO from $(ORIGDIR)" + @fusermount -u $(ORIGDIR) + @rmdir $(ORIGDIR) + +.PHONY: $(SUBDIRS) +$(SUBDIRS): + @mkdir -p release/packages/ubuntu/pool/main release/puppet/modules release/isoroot + $(MAKE) -C $@ -f Makefile release + +.PHONY: patch-packages +patch-packages: + ORIGISO=$(ISOCACHE) REVSTATE=$(REVSTATE) $(MAKE) -C $@ -f Makefile release + +.PHONY: prepare +prepare: + $(MAKE) -C opendaylight -f Makefile setup + +.PHONY: odl +odl: + $(MAKE) -C opendaylight -f Makefile + +.PHONY: build-clean $(SUBCLEAN) +build-clean: $(SUBCLEAN) + $(MAKE) -C patch-packages -f Makefile clean + @rm -Rf release + @rm -Rf newiso + @rm -f $(NEWISO) + +.PHONY: clean $(SUBCLEAN) +clean: prepare $(SUBCLEAN) + $(MAKE) -C patch-packages -f Makefile clean + $(MAKE) -C opendaylight -f Makefile clean + @rm -f *.iso + @rm -Rf release + @rm -Rf newiso + @rm -f f_odl + @rm -f $(NEWISO) + +$(SUBCLEAN): %.clean: + $(MAKE) -C $* -f Makefile clean + +# Todo: Make things smarter - we shouldn't need to clean everything +# betwen make invocations. +.PHONY: iso +iso: prepare build-clean odl $(ISOCACHE) $(SUBDIRS) patch-packages + install/install.sh iso $(ISOCACHE) $(NEWISO) $(PRODNO) $(REVSTATE) + @printf "\n\nProduct ISO is $(NEWISO)\n\n" diff --git a/fuel/build/Makefile~ b/fuel/build/Makefile~ new file mode 100644 index 0000000..7f16a28 --- /dev/null +++ b/fuel/build/Makefile~ @@ -0,0 +1,140 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +############################################################################ +# BEGIN of variables to customize +# +SHELL = /bin/bash + +include config.mk + +export BUILD_BASE = $(shell pwd) +export DEB_DEST := $(BUILD_BASE)/release/packages/ubuntu/pool/main +export PUPPET_DEST := $(BUILD_BASE)/release/puppet/modules + +ISOSRC = file:$(shell pwd)/fuel-6.0.1.iso +ISOCACHE = $(shell pwd)/$(shell basename $(ISOSRC)) +PRODNO = "OPNFV_BGS" +REVSTATE = "P0000" +NEWISO = $(shell pwd)/release/opnfv-${REVSTATE}.iso +DOCKERIMG = opnfv.org/ubuntu-builder:14.04 +# Note! Invoke with "make REVSTATE=RXXXX iso" to make release build! +# Invoke with ICOCACHE=/full/path/to/iso if cached ISO is in non-standard location. +# +# END of variables to customize +############################################################################# +export TOPDIR := $(shell pwd) +ORIGDIR := $(TOPDIR)/origiso + +SUBDIRS := f_isoroot +SUBDIRS += f_opnfv_puppet +SUBDIRS += f_osnaily +SUBDIRS += f_l23network +SUBDIRS += f_resolvconf +SUBDIRS += f_ntp +SUBDIRS += f_gif +SUBDIRS += f_libnss +SUBDIRS += f_libpcs +SUBDIRS += f_libxt +SUBDIRS += f_comm_java +SUBDIRS += f_ca_cert +SUBDIRS += f_atk_wrap +SUBDIRS += f_jre_head +SUBDIRS += f_jre +SUBDIRS += f_java +SUBDIRS += f_odl + + + +# f_example is only an example of how to generate a .deb package and +# should not be enabled in official builds. +#SUBDIRS += f_example + +SUBCLEAN = $(addsuffix .clean,$(SUBDIRS)) + +.PHONY: all +all: + @docker version >/dev/null 2>&1 || (echo 'No Docker installation available'; exit 1) + @make -C docker + @docker/runcontext $(DOCKERIMG) $(MAKE) $(MAKEFLAGS) iso + +$(ISOCACHE): + # Clone Fuel to non-persistent location and build + cd /tmp && git clone $(FUEL_MAIN_REPO) + cd /tmp/fuel-main && git checkout $(FUEL_MAIN_TAG) + # Setup cgroups for docker-in-docker + sudo /root/enable_dockerx2 + # Patch to fix race condition when doing "Docker-in-Docker" build + cd /tmp/fuel-main && patch -p1 < $(TOPDIR)/fuel-main_1.patch + # Patch to make the sandbox chroot in Fuel succeed with package + # installation in a Docker build + cd /tmp/fuel-main && patch -p1 < $(TOPDIR)/fuel-main_2.patch + # Remove Docker optimizations, otherwise multistrap will fail during + # Fuel build. + sudo rm -f /etc/apt/apt.conf.d/docker* + # + cd /tmp/fuel-main && ./prepare-build-env.sh + cd /tmp/fuel-main && make iso + mv /tmp/fuel-main/build/artifacts/fuel*.iso . + +.PHONY: mount-origiso umount-origiso +mount-origiso: $(ISOCACHE) + @echo "Mounting original ISO in $(ORIGDIR)" + @mkdir -p $(ORIGDIR) + @fuseiso $(ISOCACHE) $(ORIGDIR) + +umount-origiso: + @echo "Unmounting original ISO from $(ORIGDIR)" + @fusermount -u $(ORIGDIR) + @rmdir $(ORIGDIR) + +.PHONY: $(SUBDIRS) +$(SUBDIRS): + @mkdir -p release/packages/ubuntu/pool/main release/puppet/modules release/isoroot + $(MAKE) -C $@ -f Makefile release + +.PHONY: patch-packages +patch-packages: + ORIGISO=$(ISOCACHE) REVSTATE=$(REVSTATE) $(MAKE) -C $@ -f Makefile release + +.PHONY: prepare +prepare: + $(MAKE) -C opendaylight -f Makefile setup + +.PHONY: odl +odl: + $(MAKE) -C opendaylight -f Makefile + +.PHONY: build-clean $(SUBCLEAN) +build-clean: $(SUBCLEAN) + $(MAKE) -C patch-packages -f Makefile clean + @rm -Rf release + @rm -Rf newiso + @rm -f $(NEWISO) + +.PHONY: clean $(SUBCLEAN) +clean: prepare $(SUBCLEAN) + $(MAKE) -C patch-packages -f Makefile clean + $(MAKE) -C opendaylight -f Makefile clean + @rm -f *.iso + @rm -Rf release + @rm -Rf newiso + @rm -f f_odl + @rm -f $(NEWISO) + +$(SUBCLEAN): %.clean: + $(MAKE) -C $* -f Makefile clean + +# Todo: Make things smarter - we shouldn't need to clean everything +# betwen make invocations. +.PHONY: iso +iso: prepare build-clean odl $(ISOCACHE) $(SUBDIRS) patch-packages + install/install.sh iso $(ISOCACHE) $(NEWISO) $(PRODNO) $(REVSTATE) + @printf "\n\nProduct ISO is $(NEWISO)\n\n" diff --git a/fuel/build/README b/fuel/build/README new file mode 100644 index 0000000..9692f34 --- /dev/null +++ b/fuel/build/README @@ -0,0 +1,23 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +This purpose of this framework is to: + +1) Build fuel baseline from upstream OpenStack Fuel repos (builds in a docker container to isolate dependencies from host + and full re-producability) +2) Apply arbitrary changes to the fuel baseline, consistent and non volatile: + - Add arbitrary packages with puppet modules for install and config. to be applied on controllers as well as computes. + - Change any stack-, or other pack config. using puppet + - Apply patches to the baseline. + - Etc. +3) Re factor/rebuild the .iso image for deployment (also builds in a container, for the same reason as mentioned above) +4) Through a pre-deployment script, setting config's not part of the pristine fuel build can be achieved + +For detailed instructions on how to add content, configuration, build and deply - please see: DOC/ diff --git a/fuel/build/RELEASENOTES b/fuel/build/RELEASENOTES new file mode 100644 index 0000000..76b4208 --- /dev/null +++ b/fuel/build/RELEASENOTES @@ -0,0 +1,32 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +This patch introduces the following features and bug fixes: +----------------------------------------------------------- +1) The OpenDaylight SDN server - unplumbed (config.mk needs to be changed + in order to enable it, see: fuel-build/DOC/BUILD/README.build) +2) cofig-spec - a way to configure the stack component versions + to be built - see the fuel-build/config.mk file +3) A simplified Docker build startegy, pulling a pre-fabricated + docker base image from the Internet + +NOTE: Before you start building - you need to purge any previous +"opnfv.org/ubuntu-builder" docker image: "sudo docker rmi -f opnfv.org/ubuntu-builder +Double check that you dont have any remaining "opnfv.org/ubuntu-builder" images by +"docker images". +A new docker image will automatically be rebuilt in the build process. + +Limitations and known issues: +----------------------------- +1) OpenDaylight is for now just sitting in the controllers, not plumbed to anything + northbound or southbound. +2) OpenDaylight is currently only building with Oracle Java JDK, that is Oracle Java + is used in the build process while OpenJava is used in the runtime environment. + Read the licencing obligations in /fuel-build/LICENCE before you start building. diff --git a/fuel/build/TODO b/fuel/build/TODO new file mode 100644 index 0000000..897a7e8 --- /dev/null +++ b/fuel/build/TODO @@ -0,0 +1,10 @@ +############################################ +# Contributors/version/date +# jonas.bjurel@ericsson.com 0.1 2015.01.28 +############################################ +Following items needs to be done to achieve an OPNFV/BGS R1 Stack: +1) Rebase to Fuel 6.0/6.1 - DONE +2) Add support for CentOS 6.5 - REMAINING +3) Add build-support and manifests for Open daylight Ubuntu and CentOS + - Initial build and deployment of ODL/Helium - DONE + - Configuration and plumbing - IN PROGRESS
\ No newline at end of file diff --git a/fuel/build/add_opnfv_packages b/fuel/build/add_opnfv_packages new file mode 100644 index 0000000..44af92e --- /dev/null +++ b/fuel/build/add_opnfv_packages @@ -0,0 +1,23 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +# Format of this file: +# +#TEXT>old package>url for replacement package +# +# Note! No spaces! +# The TEXT can be anything it is just for information display. +# Iff the packages is new, i.e. not replacing an existing package +# the old package should be specified as NONE. +# Note! For new packages a puppet manifest must actually install it. +# Easiest is to just add it to add_packages.pp. (The same might +# of course be true for an exising package if it isn't already +# is being installed). +# diff --git a/fuel/build/apply_patches b/fuel/build/apply_patches new file mode 100644 index 0000000..6925ff6 --- /dev/null +++ b/fuel/build/apply_patches @@ -0,0 +1,18 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +# Format of this file: +# +#TR>old package>url for replacement package +# +# Note! No spaces! +# Example: +#FIX>foo_0.0.1.deb>https://example.org/foo_0.0.2.deb + diff --git a/fuel/build/config.mk b/fuel/build/config.mk new file mode 100644 index 0000000..da760df --- /dev/null +++ b/fuel/build/config.mk @@ -0,0 +1,25 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +ODL_MAIN_REPO := https://git.opendaylight.org/gerrit/p/controller.git +ODL_MAIN_TAG := release/helium + +FUEL_MAIN_REPO := https://github.com/stackforge/fuel-main +FUEL_MAIN_TAG = stable/6.0 + +JAVA_MAIN_REPO := https://mirrors.kernel.org/ubuntu/pool/universe/o/openjdk-8 +ATK_MAIN_REPO := https://mirrors.kernel.org/ubuntu/pool/main/j/java-atk-wrapper +GIF_MAIN_REPO := https://mirrors.kernel.org/ubuntu/pool/main/g/giflib +CERT_MAIN_REPO := https://mirrors.kernel.org/ubuntu/pool/main/c/ca-certificates +COMM_MAIN_REPO := https://mirrors.kernel.org/ubuntu/pool/main/j/java-common +LIBPCS_MAIN_REPO := https://mirrors.kernel.org/ubuntu/pool/main/p/pcsc-lite +LIBXT_MAIN_REPO := https://mirrors.kernel.org/ubuntu/pool/main/libx/libxtst +DEFJRE_MAIN_REPO := https://mirrors.kernel.org/ubuntu/pool/main/j/java-common +LIBNSS_MAIN_REPO := https://mirrors.kernel.org/ubuntu/pool/main/n/nss diff --git a/fuel/build/docker/Makefile b/fuel/build/docker/Makefile new file mode 100644 index 0000000..c2853c8 --- /dev/null +++ b/fuel/build/docker/Makefile @@ -0,0 +1,23 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +FILES = $(wildcard ubuntu-builder/*) runcontext + +.PHONY: all +all: .docker + +.docker: $(FILES) + /usr/bin/docker build --rm=true --no-cache=true -t opnfv.org/ubuntu-builder:14.04 ubuntu-builder + /usr/bin/docker tag -f opnfv.org/ubuntu-builder:14.04 opnfv.org/ubuntu-builder + touch .docker + +.PHONY: clean +clean: + rm .docker diff --git a/fuel/build/docker/README b/fuel/build/docker/README new file mode 100644 index 0000000..e5ccdfa --- /dev/null +++ b/fuel/build/docker/README @@ -0,0 +1,25 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +The makefile on this level is used for *creating* the Docker image +used for building in Ubuntu 12.04 context. + +There are two requirements before running make on this level: + +1. You need to have Docker installed on your system + +2. If in a corporate network, make sure to have a valid DNS config in + your /etc/default/docker, e.g: + + DOCKER_OPTS=" --dns=8.8.8.8 --dns=8.8.8.4" + +The top level makefile will refer to the "runcontext" file in order to +run the build process as the current user with the current working +directory available to the container. diff --git a/fuel/build/docker/runcontext b/fuel/build/docker/runcontext new file mode 100755 index 0000000..a22fb29 --- /dev/null +++ b/fuel/build/docker/runcontext @@ -0,0 +1,26 @@ +#!/bin/bash +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## +# +context=$1 +shift +GID=`id -g` +USER=`whoami` +res=`docker build -q --rm - <<EOF +FROM $context +RUN date || date +RUN /root/setcontext $USER $UID $GID $HOME +EOF` +GITROOT=`git rev-parse --show-toplevel` +CID=`echo $res | sed 's/.* //'` +docker run --privileged=true --rm -i -e HOME=$HOME -u $USER -w $PWD -v ${HOME}/.ssh:${HOME}/.ssh -v $GITROOT:$GITROOT $CID "$@" +rc=$? +docker rmi $CID > /dev/null +exit $rc diff --git a/fuel/build/docker/ubuntu-builder/Dockerfile b/fuel/build/docker/ubuntu-builder/Dockerfile new file mode 100644 index 0000000..b4e1b4e --- /dev/null +++ b/fuel/build/docker/ubuntu-builder/Dockerfile @@ -0,0 +1,31 @@ +#!/bin/bash +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +FROM ubuntu:14.04 +RUN apt-get update +RUN apt-get install -y software-properties-common python-software-properties \ + make python-setuptools python-all dpkg-dev debhelper \ + fuseiso git genisoimage bind9-host wget curl lintian tmux lxc iptables \ + ca-certificates sudo apt-utils lsb-release + +RUN echo "ALL ALL=NOPASSWD: ALL" > /etc/sudoers.d/open-sudo +RUN chmod 0440 /etc/sudoers.d/open-sudo +RUN chmod 4755 /bin/fusermount + +ADD ./setcontext /root/setcontext +RUN chmod +x /root/setcontext +ADD ./enable_dockerx2 /root/enable_dockerx2 +RUN chmod +x /root/enable_dockerx2 +ADD ./install.sh /root/install.sh +RUN chmod +x /root/install.sh +RUN /root/install.sh + +VOLUME /var/lib/docker diff --git a/fuel/build/docker/ubuntu-builder/enable_dockerx2 b/fuel/build/docker/ubuntu-builder/enable_dockerx2 new file mode 100644 index 0000000..1511c70 --- /dev/null +++ b/fuel/build/docker/ubuntu-builder/enable_dockerx2 @@ -0,0 +1,20 @@ +#!/bin/sh +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## +# +# Needed to expose underlying cgroups to container +echo "running x2" > /x2 +mount -n -t tmpfs -o uid=0,gid=0,mode=0755 cgroup /sys/fs/cgroup +mount -t securityfs none /sys/kernel/security + +for mnt in $(cut -d: -f2 /proc/1/cgroup); do + mkdir /sys/fs/cgroup/$mnt + mount -n -t cgroup -o $mnt cgroup /sys/fs/cgroup/$mnt +done diff --git a/fuel/build/docker/ubuntu-builder/install.sh b/fuel/build/docker/ubuntu-builder/install.sh new file mode 100755 index 0000000..df1af72 --- /dev/null +++ b/fuel/build/docker/ubuntu-builder/install.sh @@ -0,0 +1,25 @@ +#!/bin/bash +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +#Install Oracle Java 7 jdk +echo "Installing JAVA 7" +apt-get update +add-apt-repository ppa:webupd8team/java -y +apt-get update +echo debconf shared/accepted-oracle-license-v1-1 select true | debconf-set-selections +apt-get install oracle-java7-installer -y + +#Install Maven 3 +echo deb http://ppa.launchpad.net/natecarlson/maven3/ubuntu precise main >> /etc/apt/sources.list +echo deb-src http://ppa.launchpad.net/natecarlson/maven3/ubuntu precise main >> /etc/apt/sources.list +apt-get update || exit 1 +sudo apt-get install -y --force-yes maven3 || exit 1 +ln -s /usr/share/maven3/bin/mvn /usr/bin/mvn diff --git a/fuel/build/docker/ubuntu-builder/setcontext b/fuel/build/docker/ubuntu-builder/setcontext new file mode 100755 index 0000000..bc28994 --- /dev/null +++ b/fuel/build/docker/ubuntu-builder/setcontext @@ -0,0 +1,20 @@ +#!/bin/bash +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## +# +SETUSER=$1 +SETUID=$2 +SETGID=$3 +SETHOME=$4 + +getent group $SETUSER || /usr/sbin/groupadd --gid $SETGID $SETUSER +getent passwd $SETUSER || /usr/sbin/adduser --system --uid=$SETUID --gid=$SETGID --home $SETHOME --shell /bin/bash $SETUSER +/usr/sbin/usermod -a -G fuse $SETUSER +exit 0 diff --git a/fuel/build/f_atk_wrap/Makefile b/fuel/build/f_atk_wrap/Makefile new file mode 100644 index 0000000..99010eb --- /dev/null +++ b/fuel/build/f_atk_wrap/Makefile @@ -0,0 +1,42 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# daniel.smith@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) +include ../config.mk + +PACKAGE := libatk-wrapper-java +VERSION := 0.30.4-4_all +DEB_NAME := $(PACKAGE)_$(VERSION).deb + +.PHONY: all +all: release/pool/main/$(DEB_NAME) + +release/pool/main/$(DEB_NAME): +ifeq ($(ATK_MAIN_REPO),) + @echo "No config-spec target for Java, nothing to build" +else + @mkdir -p release/pool/main && \ + cd release/pool/main && \ + wget -N $(ATK_MAIN_REPO)/$(DEB_NAME) + @lintian release/pool/main/$(DEB_NAME) +endif + +.PHONY: clean +clean: + @rm -rf release + @rm -f $(DEB_DEST)/$(DEB_NAME) + + +.PHONY: release +release:release/pool/main/$(DEB_NAME) +ifneq ($(ATK_MAIN_REPO),) + @cp release/pool/main/$(DEB_NAME) $(DEB_DEST) + @cp -Rvp puppet/modules/* $(PUPPET_DEST) +endif + diff --git a/fuel/build/f_atk_wrap/puppet/modules/opnfv/manifests/atk_wrap.pp b/fuel/build/f_atk_wrap/puppet/modules/opnfv/manifests/atk_wrap.pp new file mode 100644 index 0000000..144651e --- /dev/null +++ b/fuel/build/f_atk_wrap/puppet/modules/opnfv/manifests/atk_wrap.pp @@ -0,0 +1,12 @@ +class opnfv::atk_wrap { + if $::osfamily == 'Debian' { + case $::fuel_settings['role'] { + /controller/: { + package { 'libatk-wrapper-java': + ensure => installed, + } + } + } + } +} + diff --git a/fuel/build/f_ca_cert/Makefile b/fuel/build/f_ca_cert/Makefile new file mode 100644 index 0000000..83069cf --- /dev/null +++ b/fuel/build/f_ca_cert/Makefile @@ -0,0 +1,42 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) +include ../config.mk + +PACKAGE := ca-certificates +VERSION := 20130906ubuntu2_all +DEB_NAME := $(PACKAGE)_$(VERSION).deb + +.PHONY: all +all: release/pool/main/$(DEB_NAME) + +release/pool/main/$(DEB_NAME): +ifeq ($(CERT_MAIN_REPO),) + @echo "No config-spec target for Java, nothing to build" +else + @mkdir -p release/pool/main && \ + cd release/pool/main && \ + wget -N $(CERT_MAIN_REPO)/$(DEB_NAME) + @lintian release/pool/main/$(DEB_NAME) +endif + +.PHONY: clean +clean: + @rm -rf release + @rm -f $(DEB_DEST)/$(DEB_NAME) + + +.PHONY: release +release:release/pool/main/$(DEB_NAME) +ifneq ($(CERT_MAIN_REPO),) + @cp release/pool/main/$(DEB_NAME) $(DEB_DEST) + @cp -Rvp puppet/modules/* $(PUPPET_DEST) +endif diff --git a/fuel/build/f_ca_cert/puppet/modules/opnfv/manifests/ca_cert.pp b/fuel/build/f_ca_cert/puppet/modules/opnfv/manifests/ca_cert.pp new file mode 100644 index 0000000..69b5d23 --- /dev/null +++ b/fuel/build/f_ca_cert/puppet/modules/opnfv/manifests/ca_cert.pp @@ -0,0 +1,11 @@ +class opnfv::ca_cert { + if $::osfamily == 'Debian' { + case $::fuel_settings['role'] { + /controller/: { + package { 'ca-certificates': + ensure => installed, + } + } + } + } +} diff --git a/fuel/build/f_comm_java/Makefile b/fuel/build/f_comm_java/Makefile new file mode 100644 index 0000000..a0cac38 --- /dev/null +++ b/fuel/build/f_comm_java/Makefile @@ -0,0 +1,42 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) +include ../config.mk + +PACKAGE := java-common +VERSION := 0.51_all +DEB_NAME := $(PACKAGE)_$(VERSION).deb + +.PHONY: all +all: release/pool/main/$(DEB_NAME) + +release/pool/main/$(DEB_NAME): +ifeq ($(COMM_MAIN_REPO),) + @echo "No config-spec target for Java, nothing to build" +else + @mkdir -p release/pool/main && \ + cd release/pool/main && \ + wget -N $(COMM_MAIN_REPO)/$(DEB_NAME) + @lintian release/pool/main/$(DEB_NAME) +endif + +.PHONY: clean +clean: + @rm -rf release + @rm -f $(DEB_DEST)/$(DEB_NAME) + + +.PHONY: release +release:release/pool/main/$(DEB_NAME) +ifneq ($(COMM_MAIN_REPO),) + @cp release/pool/main/$(DEB_NAME) $(DEB_DEST) + @cp -Rvp puppet/modules/* $(PUPPET_DEST) +endif diff --git a/fuel/build/f_comm_java/puppet/modules/opnfv/manifests/comm_java.pp b/fuel/build/f_comm_java/puppet/modules/opnfv/manifests/comm_java.pp new file mode 100644 index 0000000..dd7ddc5 --- /dev/null +++ b/fuel/build/f_comm_java/puppet/modules/opnfv/manifests/comm_java.pp @@ -0,0 +1,11 @@ +class opnfv::comm_java { + if $::osfamily == 'Debian' { + case $::fuel_settings['role'] { + /controller/: { + package { 'java-common': + ensure => installed, + } + } + } + } +} diff --git a/fuel/build/f_defjre/Makefile b/fuel/build/f_defjre/Makefile new file mode 100644 index 0000000..5b295c1 --- /dev/null +++ b/fuel/build/f_defjre/Makefile @@ -0,0 +1,42 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) +include ../config.mk + +PACKAGE := default-jre +VERSION := 1.7-51_amd64 +DEB_NAME := $(PACKAGE)_$(VERSION).deb + +.PHONY: all +all: release/pool/main/$(DEB_NAME) + +release/pool/main/$(DEB_NAME): +ifeq ($(DEFJRE_MAIN_REPO),) + @echo "No config-spec target for Java, nothing to build" +else + @mkdir -p release/pool/main && \ + cd release/pool/main && \ + wget -N $(DEFJRE_MAIN_REPO)/$(DEB_NAME) + @lintian release/pool/main/$(DEB_NAME) +endif + +.PHONY: clean +clean: + @rm -rf release + @rm -f $(DEB_DEST)/$(DEB_NAME) + + +.PHONY: release +release:release/pool/main/$(DEB_NAME) +ifneq ($(DEFJRE_MAIN_REPO),) + @cp release/pool/main/$(DEB_NAME) $(DEB_DEST) + @cp -Rvp puppet/modules/* $(PUPPET_DEST) +endif diff --git a/fuel/build/f_defjre/puppet/modules/opnfv/manifests/defjre.pp b/fuel/build/f_defjre/puppet/modules/opnfv/manifests/defjre.pp new file mode 100644 index 0000000..605453b --- /dev/null +++ b/fuel/build/f_defjre/puppet/modules/opnfv/manifests/defjre.pp @@ -0,0 +1,11 @@ +class opnfv::defjre { + if $::osfamily == 'Debian' { + case $::fuel_settings['role'] { + /controller/: { + package { 'default-jre': + ensure => installed, + } + } + } + } +} diff --git a/fuel/build/f_example_control_bond/Makefile b/fuel/build/f_example_control_bond/Makefile new file mode 100644 index 0000000..c311a2d --- /dev/null +++ b/fuel/build/f_example_control_bond/Makefile @@ -0,0 +1,24 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) + +.PHONY: all +all: + +.PHONY: clean +clean: + @rm -rf tmp + @rm -rf release + + +.PHONY: release +release: + @cp -Rvp puppet/modules/* $(PUPPET_DEST) diff --git a/fuel/build/f_example_control_bond/README b/fuel/build/f_example_control_bond/README new file mode 100644 index 0000000..4449b2f --- /dev/null +++ b/fuel/build/f_example_control_bond/README @@ -0,0 +1,56 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +This is an example of a modification of the bridge and bonding of interfaces. +Rationale: Fuel currently don't support bonding of the "Admin" network, which +becomes a problem if you want to co-locate the single "Admin" and the redundant +bonded "Management" network on one and the same physical network in order to minimize +number of NICs, switches and cables. +This example show how you can modify the bridge and bonding settings and scripts to +achieve co-location. Of course the FUEL GUI becomes shorted in this respect, I.e. +the "admin" and "management" network cant be dragged across the NICs in the GUI. + +The f_example_control_bond has the following structure: +. ++--------+----------+-----------+ + | | | + puppet/ Makefile README + | (this file) + | + | + modules/ + | + | + | + opnfv/ + | + +--------------+ + | | + files/ manifests/ + | | + | | + | | + control-bond control-bond.pp + +Makefile: +Invoked by the git root Makefile, copies the files up to the release build +directory. + +control-bond: +The control-bond file is the .init file orchestrating the interface-, bridge- +and bond actions required for enabling the control bond + +control-bond.pp: +The control-bond.pp is the pupet manifest orchestrating the placement and +configuration of the control-bond shell script file. + +To activate this example feature, add "SUBDIRS += f_example_control_bond" in the git root Makefile. + + diff --git a/fuel/build/f_example_control_bond/puppet/modules/opnfv/files/control-bond b/fuel/build/f_example_control_bond/puppet/modules/opnfv/files/control-bond new file mode 100644 index 0000000..bf2930d --- /dev/null +++ b/fuel/build/f_example_control_bond/puppet/modules/opnfv/files/control-bond @@ -0,0 +1,90 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +#!/bin/sh +### BEGIN INIT INFO +# Provides: control-bond +# Required-Start: $remote_fs $all +# Required-Stop: +# Default-Start: 2 3 4 5 +# Default-Stop: 0 1 6 +# Short-Description: Bonds eth0 and eth1 and changes bridge configuration. +### END INIT INFO + + +PATH=/sbin:/usr/sbin:/bin:/usr/bin +VSCTL=/usr/bin/ovs-vsctl +APPCTL=/usr/bin/ovs-appctl + + +add_control_bond() { + logger "Starting addition of control-bond" + $VSCTL --may-exist br-fw-admin + $VSCTL --may-exist br-mgmt + $VSCTL add-bond br-fw-admin bond-control eth0 eth1 + + $VSCTL set port br-mgmt tag=66 + $APPCTL bond/set-active-slave bond-control eth0 + + $VSCTL add-port br-fw-admin admin-to-mgmt + $VSCTL add-port br-mgmt mgmt-to-admin + $VSCTL set interface admin-to-mgmt type=patch + $VSCTL set interface mgmt-to-admin type=patch + + $VSCTL set interface admin-to-mgmt options:peer=mgmt-to-admin + $VSCTL set interface mgmt-to-admin options:peer=admin-to-mgmt + + $VSCTL set port admin-to-mgmt trunk=66 + $VSCTL set port mgmt-to-admin trunk=66 + logger "Finished addition of control-bond" +} + +status_control_bond() { + if [ ! -f $VSCTL ]; then + return 1 + else + $VSCTL show | grep -q "admin-to-mgmt" + return $? + fi +} + +case "$1" in + start) + status_control_bond + if [ $? -eq 0 ]; then + exit 0 + else + add_control_bond + fi + ;; + restart|reload|force-reload) + echo "Error: argument '$1' not supported" >&2 + exit 3 + ;; + status) + status_control_bond + if [ $? -eq 0 ]; then + echo "The control-bond is enabled" + exit 0 + else + echo "The control-bond is disabled" + exit 1 + fi + + ;; + stop) + echo "Stop is not supported" + exit 0 + ;; + *) + echo "Usage: $0 start|status" >&2 + exit 3 + ;; +esac diff --git a/fuel/build/f_example_control_bond/puppet/modules/opnfv/manifests/control-bond.pp b/fuel/build/f_example_control_bond/puppet/modules/opnfv/manifests/control-bond.pp new file mode 100644 index 0000000..22fe781 --- /dev/null +++ b/fuel/build/f_example_control_bond/puppet/modules/opnfv/manifests/control-bond.pp @@ -0,0 +1,56 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +# Class: opnfv::control-bond +# +# Bridge control and management networks together using OVS. +# +# + +class opnfv::control-bond { + notify { '*** In Opnfv::control-bond-start ***': } + + file { "/etc/init.d/control-bond": + source => "puppet:///modules/opnfv/control-bond", + owner => 'root', + group => 'root', + mode => '0755', + notify => Service["control-bond"] + } + + + service { "control-bond": + ensure => running, + require => [ File["/etc/init.d/control-bond"], Service["openvswitch-service"] ], + } + + # Only start scripts - we don't want to bring down + # bridge during shutdown + + file { "/etc/rc2.d/S18control-bond": + ensure => 'link', + target => '/etc/init.d/control-bond', + } + + file { "/etc/rc3.d/S18control-bond": + ensure => 'link', + target => '/etc/init.d/control-bond', + } + + file { "/etc/rc4.d/S18control-bond": + ensure => 'link', + target => '/etc/init.d/control-bond', + } + + file { "/etc/rc5.d/S18control-bond": + ensure => 'link', + target => '/etc/init.d/control-bond', + } +} diff --git a/fuel/build/f_example_packadd/Makefile b/fuel/build/f_example_packadd/Makefile new file mode 100644 index 0000000..eadc83f --- /dev/null +++ b/fuel/build/f_example_packadd/Makefile @@ -0,0 +1,38 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) +PACKAGE := foobar +VERSION := 1.0-1 +DEB_NAME := $(PACKAGE)_$(VERSION).deb + +.PHONY: all +all: release/pool/main/$(DEB_NAME) + +release/pool/main/$(DEB_NAME): + @mkdir -p tmp/src + @mkdir -p release/pool/main + @cp -r $(DEB_NAME) tmp/src + @gzip -9 tmp/src/$(DEB_NAME)/usr/share/doc/$(PACKAGE)/changelog.Debian + @fakeroot dpkg-deb --build tmp/src/$(DEB_NAME) + @lintian tmp/src/$(DEB_NAME) + @cp tmp/src/$(DEB_NAME) release/pool/main + +.PHONY: clean +clean: + @rm -rf tmp + @rm -rf release + @rm -f ../release/packages/ubuntu/pool/main/$(DEB_NAME) + + +.PHONY: release +release:release/pool/main/$(DEB_NAME) + @cp release/pool/main/$(DEB_NAME) $(DEB_DEST) + @cp -Rvp puppet/modules/* $(PUPPET_DEST) diff --git a/fuel/build/f_example_packadd/README b/fuel/build/f_example_packadd/README new file mode 100644 index 0000000..55fa7c1 --- /dev/null +++ b/fuel/build/f_example_packadd/README @@ -0,0 +1,47 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +This is an example of a an addition of an arbitrary debian package: foobar +It adds the debian package foobar_1.0-1 to the fuel .iso image and deploys it +to the controllers and the computes. + +The f_example_packadd has the following structure: +. ++--------+----------+-----------+------------+ + | | | | + puppet/ Makefile README foobar_1.0-1/ + | (this file) (deb pack) + | + | + modules/ + | + | + | + opnfv/ + | + | + | + manifests/ + | + | + | + foobar.pp + +Makefile: +Invoked by the git root Makefile, based on the foobar_1.0-1 deb pack artifacts, and the pupet manifests, it builds a deb package +and adds it together with the manifest to a fuel build artifact directory, such that it eventually gets built into the new fuel .iso + +foobar_1.0-1: +Contains arbitrary debian package artifacts + +foobar.pp: +Controls the installation and configuration of foobar + +To activate this example feature, add "SUBDIRS += f_example_packadd" in the git root Makefile. diff --git a/fuel/build/f_example_packadd/foobar_1.0-1/DEBIAN/conffiles b/fuel/build/f_example_packadd/foobar_1.0-1/DEBIAN/conffiles new file mode 100644 index 0000000..bad6a0e --- /dev/null +++ b/fuel/build/f_example_packadd/foobar_1.0-1/DEBIAN/conffiles @@ -0,0 +1 @@ +/etc/foobar/foobar.conf diff --git a/fuel/build/f_example_packadd/foobar_1.0-1/DEBIAN/control b/fuel/build/f_example_packadd/foobar_1.0-1/DEBIAN/control new file mode 100644 index 0000000..f3f0eb6 --- /dev/null +++ b/fuel/build/f_example_packadd/foobar_1.0-1/DEBIAN/control @@ -0,0 +1,10 @@ +Package: foobar +Version: 1.0-1 +Section: utils +Priority: optional +Architecture: amd64 +Depends: python2.7 +Maintainer: Main Tainer <maintainer@somwhere.org> +Description: foo bar daemon + This is a daemon for the Foobar service. + Foo is bar! diff --git a/fuel/build/f_example_packadd/foobar_1.0-1/DEBIAN/postinst b/fuel/build/f_example_packadd/foobar_1.0-1/DEBIAN/postinst new file mode 100755 index 0000000..f48f260 --- /dev/null +++ b/fuel/build/f_example_packadd/foobar_1.0-1/DEBIAN/postinst @@ -0,0 +1,2 @@ +#!/bin/bash -e +date diff --git a/fuel/build/f_example_packadd/foobar_1.0-1/etc/foobar/foobar.conf b/fuel/build/f_example_packadd/foobar_1.0-1/etc/foobar/foobar.conf new file mode 100644 index 0000000..e0eee5c --- /dev/null +++ b/fuel/build/f_example_packadd/foobar_1.0-1/etc/foobar/foobar.conf @@ -0,0 +1,2 @@ +# Foobar config file +foo=bar diff --git a/fuel/build/f_example_packadd/foobar_1.0-1/usr/bin/foobar b/fuel/build/f_example_packadd/foobar_1.0-1/usr/bin/foobar new file mode 100755 index 0000000..c9651a4 --- /dev/null +++ b/fuel/build/f_example_packadd/foobar_1.0-1/usr/bin/foobar @@ -0,0 +1,2 @@ +#!/bin/bash +echo Hello from the foobar package diff --git a/fuel/build/f_example_packadd/foobar_1.0-1/usr/share/doc/foobar/changelog.Debian b/fuel/build/f_example_packadd/foobar_1.0-1/usr/share/doc/foobar/changelog.Debian new file mode 100644 index 0000000..fe776f3 --- /dev/null +++ b/fuel/build/f_example_packadd/foobar_1.0-1/usr/share/doc/foobar/changelog.Debian @@ -0,0 +1,5 @@ +foobar (1.0-1) precise-proposed; urgency=low + + * Genesis + + -- Main Tainer <maintainer@somewhere.org> Tue, 23 Sep 2014 11:13:27 +0200 diff --git a/fuel/build/f_example_packadd/foobar_1.0-1/usr/share/doc/foobar/copyright b/fuel/build/f_example_packadd/foobar_1.0-1/usr/share/doc/foobar/copyright new file mode 100644 index 0000000..4279ce4 --- /dev/null +++ b/fuel/build/f_example_packadd/foobar_1.0-1/usr/share/doc/foobar/copyright @@ -0,0 +1,18 @@ +foobar - a test daemon +Copyright (C) 2014 Main Tainer + +This program is free software: you can redistribute it and/or modify +it under the terms of the GNU General Public License as published by +the Free Software Foundation, either version 3 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License with +your Debian GNU system, in /usr/share/common-licenses/GPL, or with the +Debian GNU gnupg source package as the file COPYING. If not, see +<http://www.gnu.org/licenses/> or write to the Free Software Foundation, +Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA. diff --git a/fuel/build/f_example_packadd/puppet/modules/opnfv/manifests/foobar.pp b/fuel/build/f_example_packadd/puppet/modules/opnfv/manifests/foobar.pp new file mode 100644 index 0000000..c6219ba --- /dev/null +++ b/fuel/build/f_example_packadd/puppet/modules/opnfv/manifests/foobar.pp @@ -0,0 +1,7 @@ +class opnfv::foobar { + if $::osfamily == 'Debian' { + package { 'foobar': + ensure => installed, + } + } +} diff --git a/fuel/build/f_gif/Makefile b/fuel/build/f_gif/Makefile new file mode 100644 index 0000000..7108242 --- /dev/null +++ b/fuel/build/f_gif/Makefile @@ -0,0 +1,42 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) +include ../config.mk + +PACKAGE := libgif4 +VERSION := 4.1.6-11_amd64 +DEB_NAME := $(PACKAGE)_$(VERSION).deb + +.PHONY: all +all: release/pool/main/$(DEB_NAME) + +release/pool/main/$(DEB_NAME): +ifeq ($(GIF_MAIN_REPO),) + @echo "No config-spec target for Java, nothing to build" +else + @mkdir -p release/pool/main && \ + cd release/pool/main && \ + wget -N $(GIF_MAIN_REPO)/$(DEB_NAME) + @lintian release/pool/main/$(DEB_NAME) +endif + +.PHONY: clean +clean: + @rm -rf release + @rm -f $(DEB_DEST)/$(DEB_NAME) + + +.PHONY: release +release:release/pool/main/$(DEB_NAME) +ifneq ($(GIF_MAIN_REPO),) + @cp release/pool/main/$(DEB_NAME) $(DEB_DEST) + @cp -Rvp puppet/modules/* $(PUPPET_DEST) +endif diff --git a/fuel/build/f_gif/puppet/modules/opnfv/manifests/gif.pp b/fuel/build/f_gif/puppet/modules/opnfv/manifests/gif.pp new file mode 100644 index 0000000..d51cbe7 --- /dev/null +++ b/fuel/build/f_gif/puppet/modules/opnfv/manifests/gif.pp @@ -0,0 +1,11 @@ +class opnfv::gif { + if $::osfamily == 'Debian' { + case $::fuel_settings['role'] { + /controller/: { + package { 'libgif4': + ensure => installed, + } + } + } + } +} diff --git a/fuel/build/f_isoroot/Makefile b/fuel/build/f_isoroot/Makefile new file mode 100644 index 0000000..2fdbbae --- /dev/null +++ b/fuel/build/f_isoroot/Makefile @@ -0,0 +1,31 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +SUBDIRS = f_predeployment f_kscfg f_bootstrap +SUBCLEAN = $(addsuffix .clean,$(SUBDIRS)) + +.PHONY: all +all: $(SUBDIRS) + +.PHONY: $(SUBDIRS) +$(SUBDIRS): + @mkdir -p release + $(MAKE) -C $@ -f Makefile release + +.PHONY: clean $(SUBCLEAN) +clean: $(SUBCLEAN) + @rm -Rf release + +$(SUBCLEAN): %.clean: + $(MAKE) -C $* -f Makefile clean + +.PHONY: release +release:clean $(SUBDIRS) + @cp -Rvp release/* ../release/isoroot diff --git a/fuel/build/f_isoroot/README b/fuel/build/f_isoroot/README new file mode 100644 index 0000000..eb54c08 --- /dev/null +++ b/fuel/build/f_isoroot/README @@ -0,0 +1,16 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +This is to (a) replace/patch existing Fuel ISO files as well +as adding new information directly into the ISO structure + +Please separate changes into "patches" and new content into +"additions". Both directories will be anchored at the ISO file +root. diff --git a/fuel/build/f_isoroot/f_bootstrap/Makefile b/fuel/build/f_isoroot/f_bootstrap/Makefile new file mode 100644 index 0000000..7404319 --- /dev/null +++ b/fuel/build/f_isoroot/f_bootstrap/Makefile @@ -0,0 +1,29 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) + +.PHONY: all +all: + @mkdir -p release/opnfv/bootstrap/pre.d + @mkdir -p release/opnfv/bootstrap/post.d + @mkdir -p release/usr/local/sbin + @cp pre-scripts/* release/opnfv/bootstrap/pre.d + @cp post-scripts/* release/opnfv/bootstrap/post.d + @cp bootstrap_admin_node.sh release + @cp bootstrap_admin_node.sh.orig release + +.PHONY: clean +clean: + @rm -rf release + +.PHONY: release +release:all + @cp -Rvp release/* ../release diff --git a/fuel/build/f_isoroot/f_bootstrap/README b/fuel/build/f_isoroot/f_bootstrap/README new file mode 100644 index 0000000..5da954c --- /dev/null +++ b/fuel/build/f_isoroot/f_bootstrap/README @@ -0,0 +1,24 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +This function modifies the adds hooks to the beginning and end of +/usr/local/sbin/bootstrap_admin_nodes.sh, which is run as the +last post step from ks.cfg + +This enables other functions to place scripts into two directories +run either at the beginning or the end of the bootstrap procedure: + + /opt/opnfv/bootstrap/pre.d + /opt/opnfv/bootstrap/post.d + +These will be run in lexical order at pre or post. + +CI note: Using pre.d, it would be possible to "inject" a pre-defined +astute.yaml for Fuel. diff --git a/fuel/build/f_isoroot/f_bootstrap/bootstrap_admin_node.sh b/fuel/build/f_isoroot/f_bootstrap/bootstrap_admin_node.sh new file mode 100755 index 0000000..348ce3c --- /dev/null +++ b/fuel/build/f_isoroot/f_bootstrap/bootstrap_admin_node.sh @@ -0,0 +1,105 @@ +#!/bin/bash + +function countdown() { + local i + sleep 1 + for ((i=$1-1; i>=1; i--)); do + printf '\b\b%02d' "$i" + sleep 1 + done +} + +function fail() { + echo "ERROR: Fuel node deployment FAILED! Check /var/log/puppet/bootstrap_admin_node.log for details" 1>&2 + exit 1 +} +# LANG variable is a workaround for puppet-3.4.2 bug. See LP#1312758 for details +export LANG=en_US.UTF8 +showmenu="no" +if [ -f /root/.showfuelmenu ]; then + . /root/.showfuelmenu +fi + +echo -n "Applying default Fuel settings..." +fuelmenu --save-only --iface=eth0 +echo "Done!" + +### OPNFV addition BEGIN +shopt -s nullglob +for script in /opt/opnfv/bootstrap/pre.d/*.sh +do + echo "Pre script: $script" >> /root/pre.log 2>&1 + $script >> /root/pre.log 2>&1 +done +shopt -u nullglob +### OPNFV addition END + +if [[ "$showmenu" == "yes" || "$showmenu" == "YES" ]]; then + fuelmenu + else + #Give user 15 seconds to enter fuelmenu or else continue + echo + echo -n "Press a key to enter Fuel Setup (or press ESC to skip)... 15" + countdown 15 & pid=$! + if ! read -s -n 1 -t 15 key; then + echo -e "\nSkipping Fuel Setup..." + else + { kill "$pid"; wait $!; } 2>/dev/null + case "$key" in + $'\e') echo "Skipping Fuel Setup.." + echo -n "Applying default Fuel setings..." + fuelmenu --save-only --iface=eth0 + echo "Done!" + ;; + *) echo -e "\nEntering Fuel Setup..." + fuelmenu + ;; + esac + fi +fi +#Reread /etc/sysconfig/network to inform puppet of changes +. /etc/sysconfig/network +hostname "$HOSTNAME" + +### docker stuff +images_dir="/var/www/nailgun/docker/images" + +# extract docker images +mkdir -p $images_dir $sources_dir +rm -f $images_dir/*tar +pushd $images_dir &>/dev/null + +echo "Extracting and loading docker images. (This may take a while)" +lrzip -d -o fuel-images.tar fuel-images.tar.lrz && tar -xf fuel-images.tar && rm -f fuel-images.tar +popd &>/dev/null +service docker start + +# load docker images +for image in $images_dir/*tar ; do + echo "Loading docker image ${image}..." + docker load -i "$image" + # clean up extracted image + rm -f "$image" +done + +# apply puppet +puppet apply --detailed-exitcodes -d -v /etc/puppet/modules/nailgun/examples/host-only.pp +if [ $? -ge 4 ];then + fail +fi +rmdir /var/log/remote && ln -s /var/log/docker-logs/remote /var/log/remote + +dockerctl check || fail +bash /etc/rc.local + +### OPNFV addition BEGIN +shopt -s nullglob +for script in /opt/opnfv/bootstrap/post.d/*.sh +do + echo "Post script: $script" >> /root/post.log 2>&1 + $script >> /root/post.log 2>&1 +done +shopt -u nullglob +### OPNFV addition END + +echo "Fuel node deployment complete!" diff --git a/fuel/build/f_isoroot/f_bootstrap/bootstrap_admin_node.sh.orig b/fuel/build/f_isoroot/f_bootstrap/bootstrap_admin_node.sh.orig new file mode 100755 index 0000000..7b6e6bd --- /dev/null +++ b/fuel/build/f_isoroot/f_bootstrap/bootstrap_admin_node.sh.orig @@ -0,0 +1,84 @@ +#!/bin/bash + +function countdown() { + local i + sleep 1 + for ((i=$1-1; i>=1; i--)); do + printf '\b\b%02d' "$i" + sleep 1 + done +} + +function fail() { + echo "ERROR: Fuel node deployment FAILED! Check /var/log/puppet/bootstrap_admin_node.log for details" 1>&2 + exit 1 +} +# LANG variable is a workaround for puppet-3.4.2 bug. See LP#1312758 for details +export LANG=en_US.UTF8 +showmenu="no" +if [ -f /root/.showfuelmenu ]; then + . /root/.showfuelmenu +fi + +echo -n "Applying default Fuel settings..." +fuelmenu --save-only --iface=eth0 +echo "Done!" + +if [[ "$showmenu" == "yes" || "$showmenu" == "YES" ]]; then + fuelmenu + else + #Give user 15 seconds to enter fuelmenu or else continue + echo + echo -n "Press a key to enter Fuel Setup (or press ESC to skip)... 15" + countdown 15 & pid=$! + if ! read -s -n 1 -t 15 key; then + echo -e "\nSkipping Fuel Setup..." + else + { kill "$pid"; wait $!; } 2>/dev/null + case "$key" in + $'\e') echo "Skipping Fuel Setup.." + echo -n "Applying default Fuel setings..." + fuelmenu --save-only --iface=eth0 + echo "Done!" + ;; + *) echo -e "\nEntering Fuel Setup..." + fuelmenu + ;; + esac + fi +fi +#Reread /etc/sysconfig/network to inform puppet of changes +. /etc/sysconfig/network +hostname "$HOSTNAME" + +### docker stuff +images_dir="/var/www/nailgun/docker/images" + +# extract docker images +mkdir -p $images_dir $sources_dir +rm -f $images_dir/*tar +pushd $images_dir &>/dev/null + +echo "Extracting and loading docker images. (This may take a while)" +lrzip -d -o fuel-images.tar fuel-images.tar.lrz && tar -xf fuel-images.tar && rm -f fuel-images.tar +popd &>/dev/null +service docker start + +# load docker images +for image in $images_dir/*tar ; do + echo "Loading docker image ${image}..." + docker load -i "$image" + # clean up extracted image + rm -f "$image" +done + +# apply puppet +puppet apply --detailed-exitcodes -d -v /etc/puppet/modules/nailgun/examples/host-only.pp +if [ $? -ge 4 ];then + fail +fi +rmdir /var/log/remote && ln -s /var/log/docker-logs/remote /var/log/remote + +dockerctl check || fail +bash /etc/rc.local +echo "Fuel node deployment complete!" diff --git a/fuel/build/f_isoroot/f_bootstrap/post-scripts/00_post_example.sh b/fuel/build/f_isoroot/f_bootstrap/post-scripts/00_post_example.sh new file mode 100755 index 0000000..7ac8965 --- /dev/null +++ b/fuel/build/f_isoroot/f_bootstrap/post-scripts/00_post_example.sh @@ -0,0 +1,4 @@ +#/bin/sh +date +echo "This is an example file run at post-bootstrap." +exit 0 diff --git a/fuel/build/f_isoroot/f_bootstrap/post-scripts/01_fix_iommubug.sh b/fuel/build/f_isoroot/f_bootstrap/post-scripts/01_fix_iommubug.sh new file mode 100755 index 0000000..79aa31a --- /dev/null +++ b/fuel/build/f_isoroot/f_bootstrap/post-scripts/01_fix_iommubug.sh @@ -0,0 +1,9 @@ +#/bin/sh +echo "Setting intel_iommu=off in bootstrap profile - a fix for the Dell systems" +echo "Old settings" +dockerctl shell cobbler cobbler profile report --name bootstrap +echo "Modifying" +dockerctl shell cobbler cobbler profile edit --name bootstrap --kopts "intel_iommu=off" --in-place +echo "New settings" +dockerctl shell cobbler cobbler profile report --name bootstrap + diff --git a/fuel/build/f_isoroot/f_bootstrap/post-scripts/02_fix_console_speed.sh b/fuel/build/f_isoroot/f_bootstrap/post-scripts/02_fix_console_speed.sh new file mode 100755 index 0000000..bf7591b --- /dev/null +++ b/fuel/build/f_isoroot/f_bootstrap/post-scripts/02_fix_console_speed.sh @@ -0,0 +1,15 @@ +#/bin/sh +echo "Changing console speed to 115200 (std is 9600) on bootstrap" +echo "Old settings" +dockerctl shell cobbler cobbler profile report --name bootstrap +echo "Modifying" +dockerctl shell cobbler cobbler profile edit --name bootstrap --kopts "console=tty0 console=ttyS0,115200" --in-place +echo "New settings" +dockerctl shell cobbler cobbler profile report --name bootstrap +echo "Setting console speed to 115200 on ubuntu_1204_x86_64 (std is no serial console)" +echo "Old settings" +dockerctl shell cobbler cobbler profile report --name ubuntu_1204_x86_64 +echo "Modifying" +dockerctl shell cobbler cobbler profile edit --name ubuntu_1204_x86_64 --kopts "console=tty0 console=ttyS0,115200" --in-place +echo "New settings" +dockerctl shell cobbler cobbler profile report --name ubuntu_1204_x86_64 diff --git a/fuel/build/f_isoroot/f_bootstrap/pre-scripts/00_pre_example.sh b/fuel/build/f_isoroot/f_bootstrap/pre-scripts/00_pre_example.sh new file mode 100755 index 0000000..ac427bf --- /dev/null +++ b/fuel/build/f_isoroot/f_bootstrap/pre-scripts/00_pre_example.sh @@ -0,0 +1,4 @@ +#/bin/sh +date +echo "This is an example file run at pre-bootstrap." +exit 0 diff --git a/fuel/build/f_isoroot/f_kscfg/Makefile b/fuel/build/f_isoroot/f_kscfg/Makefile new file mode 100644 index 0000000..38305fd --- /dev/null +++ b/fuel/build/f_isoroot/f_kscfg/Makefile @@ -0,0 +1,25 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) + +.PHONY: all +all: + @mkdir -p release + @cp ks.cfg release + @cp ks.cfg.orig release + +.PHONY: clean +clean: + @rm -rf release + +.PHONY: release +release: all + @cp -Rvp release/* ../release diff --git a/fuel/build/f_isoroot/f_kscfg/README b/fuel/build/f_isoroot/f_kscfg/README new file mode 100644 index 0000000..c85efde --- /dev/null +++ b/fuel/build/f_isoroot/f_kscfg/README @@ -0,0 +1,18 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +This function modifies the ks.cfg of the Fuel master to +make copy whatever is in the top level opnfv directory +of the ISO (populated by, for example, f_configfrontend) to +/opt/opnfv. In the end this is where the installation +parts will evolve. + +This is also the place to force the installation to overwrite +the disk (set forceformat="yes" instead of "no"). diff --git a/fuel/build/f_isoroot/f_kscfg/ks.cfg b/fuel/build/f_isoroot/f_kscfg/ks.cfg new file mode 100755 index 0000000..932e9ac --- /dev/null +++ b/fuel/build/f_isoroot/f_kscfg/ks.cfg @@ -0,0 +1,572 @@ +install +text +%include /tmp/source.ks +reboot --eject +lang en_US.UTF-8 +keyboard us +rootpw r00tme +timezone --utc Etc/UTC +firewall --disabled +selinux --disabled +# ignore unsupported hardware warning +unsupported_hardware +# SKIP CONFIGURING X +skipx +# NEVER ever place zerombr here, it breaks automated installation +%include /tmp/bootloader.ks +%include /tmp/partition.ks + +# PREINSTALL SECTION +# HERE ARE COMMANDS THAT WILL BE LAUNCHED BEFORE +# INSTALLATION PROCESS ITSELF +%pre +#!/bin/sh + +# hard drives +drives="" +removable_drives="" +for drv in `ls -1 /sys/block | grep "sd\|hd\|vd\|cciss"`; do + if (grep -q 0 /sys/block/${drv}/removable); then + drives="${drives} ${drv}" + else + removable_drives="${removable_drives} ${drv}" + fi +done +default_drive=`echo ${drives} ${removable_drives} | awk '{print $1}'` + +installdrive="undefined" +forceformat="no" +for I in `cat /proc/cmdline`; do case "$I" in *=*) eval $I;; esac ; done + +set ${drives} ${removable_drives} +numdrives=`echo $#` + +tgtdrive="${installdrive}" + +function confirm_format { + check_drive="$1" + local confirm_format="no" + + if [[ "$forceformat" == "yes" ]] ; then + return 0 + fi + + if parted -s /dev/$check_drive print &>/dev/null ; then + echo + echo "$check_drive drive contains partition table:" + parted -s /dev/$check_drive print + echo + read -p "Are you sure you want to erase ALL data on disk $check_drive? (y/N)" confirm_format + if [[ "$confirm_format" == "y" ]] || [[ "$confirm_format" == "Y" ]] || [[ "$forceformat" == "yes" ]]; then + return 0 + else + return 1 + fi + else + return 0 + fi +} + +format_confirmed="no" + +if [ $numdrives -lt 1 ]; then + exec < /dev/tty3 > /dev/tty3 2>&1 + chvt 3 + clear + echo + echo '********************************************************************' + echo '* E R R O R *' + echo '* *' + echo '* There is no suitable media available for installation. *' + echo '* Please attach a drive and try again. *' + echo '* *' + echo '********************************************************************' + echo + read -p "Press Enter to shut down the system: " _ + poweroff +fi + +if [ ${numdrives} -gt 1 ] || [ `echo ${drives} | wc -w` -eq 0 ] ; then + exec < /dev/tty3 > /dev/tty3 2>&1 + chvt 3 + while [ "${tgtdrive}" = "undefined" ]; do + clear + echo + echo '********************************************************************************' + echo '* W A R N I N G *' + echo '* *' + echo '* Which of the detected hard drives do you want to be used as *' + echo '* the installation target? *' + echo '* *' + echo '********************************************************************************' + echo + echo "Possible choices" + echo "Persistent drives: ${drives}" + echo "Removable drives: ${removable_drives}" + echo + if [ `echo ${drives} | wc -w` -eq 1 ] ; then + read -t 30 -p "Choose hard drive: " tgtdrive || tgtdrive=$default_drive + else + read -p "Choose hard drive: " tgtdrive + fi + match="no" + for drive in ${drives[@]} ${removable_drives[@]}; do + if [[ "$drive" == "$tgtdrive" ]] && match="yes" ; then + if confirm_format $tgtdrive ; then + format_confirmed="yes" + break + else + tgtdrive="undefined" + read -p "You may select another disk. Press Enter to continue." _ + fi + fi + done + if [[ "$match" == "no" ]]; then + tgtdrive="undefined" + read -p "Invalid choice. Press Enter to continue." _ + fi + done + clear + chvt 1 +else + tgtdrive=`echo ${drives} | sed -e "s/^\s*//" -e "s/\s*$//"` +fi + +if [ "$format_confirmed" != "yes" ] ; then + exec < /dev/tty3 > /dev/tty3 2>&1 + chvt 3 + if ! confirm_format $tgtdrive ; then + clear + echo + echo '********************************************************************' + echo '* E R R O R *' + echo '* *' + echo '* Disk $tgtdrive contains active partition(s). *' + echo '* Installation cannot continue without confirmation. *' + echo '* *' + echo '********************************************************************' + echo + read -p "Press Enter to restart: " _ + reboot + fi + chvt 1 +fi + +# verify tgtdrive is at least 30GB +tgtdrivesize=$(( $(cat "/sys/class/block/${tgtdrive}/size") / 2 / 1024 )) +if [ $tgtdrivesize -lt 30720 ]; then + exec < /dev/tty3 > /dev/tty3 2>&1 + chvt 3 + clear + echo + echo '********************************************************************' + echo '* E R R O R *' + echo '* *' + echo '* Your disk is under 30GB in size. Installation cannot continue. *' + echo '* Restart installation with a larger disk. *' + echo '* *' + echo '********************************************************************' + echo + read -p "Press Enter to restart: " _ + reboot +fi + +# paths in /dev have "/" instead of "!" for cciss devices +tgtdrive=$(echo $tgtdrive | sed -e 's/!/\//') + +# source +if test -e /dev/disk/by-uuid/will_be_substituted_with_actual_uuid; then + echo "harddrive --partition=UUID=will_be_substituted_with_actual_uuid --dir=/" > /tmp/source.ks +else + echo "cdrom" > /tmp/source.ks +fi + +vgremove -ff os +dd if=/dev/zero of=/dev/${tgtdrive} bs=10M count=10 +sleep 3 +hdparm -z /dev/${tgtdrive} +parted -s /dev/${tgtdrive} mklabel gpt +parted -a none -s /dev/${tgtdrive} unit MiB mkpart primary 0 24 +parted -s /dev/${tgtdrive} set 1 bios_grub on +parted -a none -s /dev/${tgtdrive} unit MiB mkpart primary fat16 24 224 +parted -s /dev/${tgtdrive} set 2 boot on +parted -a none -s /dev/${tgtdrive} unit MiB mkpart primary 224 424 +sleep 3 +hdparm -z /dev/${tgtdrive} + +# partition + +# This adds support for the p seperator required for cciss devices +if echo ${tgtdrive} | grep -q -e cciss ; then + bootdev=${tgtdrive}p +else + bootdev=${tgtdrive} +fi +echo > /tmp/partition.ks +echo "partition /boot --onpart=/dev/${bootdev}3" >> /tmp/partition.ks +echo "partition pv.001 --ondisk=${tgtdrive} --size=30000 --grow" >> /tmp/partition.ks +echo "volgroup os pv.001" >> /tmp/partition.ks +echo "logvol swap --vgname=os --recommended --name=swap" >> /tmp/partition.ks +echo "logvol / --vgname=os --size=10000 --name=root --fstype=ext4" >> /tmp/partition.ks +echo "logvol /var --vgname=os --size=1 --grow --name=var --fstype=xfs" >> /tmp/partition.ks + +# bootloader +echo "bootloader --location=mbr --driveorder=${tgtdrive} --append=' biosdevname=0 crashkernel=none'" > /tmp/bootloader.ks + +# Anaconda can not install grub 0.97 on disks which are >4T. +# The reason is that grub does not support such large geometries +# and it simply thinks that the cylinder number has negative value. +# Here we just set geometry manually so that grub thinks that disk +# size is equal to 1G. +# 130 cylinders * (16065 * 512 = 8225280 bytes) = 1G +echo "%post --nochroot --log=/mnt/sysimage/root/anaconda-post-partition.log" > /tmp/post_partition.ks +echo "echo \"device (hd0) /dev/${tgtdrive}\" >> /tmp/grub.script" >> /tmp/post_partition.ks +echo "echo \"geometry (hd0) 130 255 63\" >> /tmp/grub.script" >> /tmp/post_partition.ks +echo "echo \"root (hd0,2)\" >> /tmp/grub.script" >> /tmp/post_partition.ks +echo "echo \"install /grub/stage1 (hd0) /grub/stage2 p /grub/grub.conf\" >> /tmp/grub.script" >> /tmp/post_partition.ks +echo "echo quit >> /tmp/grub.script" >> /tmp/post_partition.ks +echo "cat /tmp/grub.script | chroot /mnt/sysimage /sbin/grub --no-floppy --batch" >> /tmp/post_partition.ks + +%end + +%packages --nobase --excludedocs +@Core +authconfig +bind-utils +cronie +crontabs +curl +daemonize +dhcp +docker-io +fuelmenu +gdisk +lrzip +lsof +man +mlocate +nmap-ncat +ntp +openssh-clients +policycoreutils +rsync +ruby21-puppet +ruby21-rubygem-netaddr +ruby21-rubygem-openstack +selinux-policy-targeted +strace +subscription-manager +sysstat +system-config-firewall-base +tcpdump +telnet +vim-enhanced +virt-what +wget +yum + +%include /tmp/post_partition.ks + +# POSTINSTALL SECTION +# HERE ARE COMMANDS THAT WILL BE LAUNCHED JUST AFTER +# INSTALLATION ITSELF COMPLETED +%post +echo -e "modprobe nf_conntrack_ipv4\nmodprobe nf_conntrack_ipv6\nmodprobe nf_conntrack_tftp\nmodprobe nf_nat_tftp" >> /etc/rc.modules +chmod +x /etc/rc.modules +echo -e "net.nf_conntrack_max=1048576" >> /etc/sysctl.conf +mkdir -p /var/log/coredump +echo -e "kernel.core_pattern=/var/log/coredump/core.%e.%p.%h.%t" >> /etc/sysctl.conf +chmod 777 /var/log/coredump +echo -e "* soft core unlimited\n* hard core unlimited" >> /etc/security/limits.conf + +# Mount installation media in chroot +%post --nochroot --log=/mnt/sysimage/root/anaconda-post-before-chroot.log +#!/bin/sh + +SOURCE="/mnt/sysimage/tmp/source" + +for I in `cat /proc/cmdline`; do case "$I" in *=*) eval $I;; esac ; done + +mkdir -p "${SOURCE}" + +case "${repo}" in + nfs:*) + nfs_url="${repo#nfs:}" + mount -t nfs "${nfs_url}" "${SOURCE}" + ;; + *) + if [ -d "/mnt/source" ]; then + mount -o bind "/mnt/source" "${SOURCE}" + fi + ;; +esac + +%post --log=/root/anaconda-post-after-chroot.log +#!/bin/bash + +set -x + +function save_cfg { + scrFile="/etc/sysconfig/network-scripts/ifcfg-$device" + search="domain $domain\nsearch $domain" + sed -i -e 's#^\(HOSTNAME=\).*$#\1'"$hostname"'#' /etc/sysconfig/network + grep -q "^\s*$ip\s+$hostname" /etc/hosts || echo "$ip $hostname" >> /etc/hosts + echo "${search}\nnameserver 127.0.0.1" > /etc/resolv.conf + [ $dns1 ] && echo -e "${search}\nnameserver $dns1" > /etc/resolv.conf + [ $dns1 ] && echo -e "${search}\nnameserver $dns1" > /etc/dnsmasq.upstream + [ $dns2 ] && echo "nameserver $dns2" >> /etc/resolv.conf + [ $dns2 ] && echo "nameserver $dns2" >> /etc/dnsmasq.upstream + + echo DEVICE=$device > $scrFile + echo ONBOOT=yes >> $scrFile + echo NM_CONTROLLED=no >> $scrFile + echo HWADDR=$hwaddr >> $scrFile + echo USERCTL=no >> $scrFile + echo PEERDNS=no >> $scrFile + if [ $ip ]; then + echo BOOTPROTO=static >> $scrFile + echo IPADDR=$ip >> $scrFile + echo NETMASK=$netmask >> $scrFile + else + echo BOOTPROTO=dhcp >> $scrFile + fi + scrDHCPFile="/etc/sysconfig/network-scripts/ifcfg-$dhcp_interface" + #Ignore gateway and set up DHCP if it is used, otherwise apply it + if [ $dhcp_interface ] && [ "$dhcp_interface" != "$device" ]; then + echo "DEVICE=$dhcp_interface" > $scrDHCPFile + echo "BOOTPROTO=dhcp" >> $scrDHCPFile + echo "ONBOOT=yes" >> $scrDHCPFile + echo "USERCTL=no" >> $scrDHCPFile + else + echo GATEWAY=$gw >> /etc/sysconfig/network + fi +} + +# Default FQDN +hostname="nailgun.mirantis.com" + +for I in `cat /proc/cmdline`; do case "$I" in *=*) eval $I;; esac ; done +hostname=$hostname +domain=${hostname#*.} +ip=$ip +netmask=$netmask +gw=$gw +device="eth0" +hwaddr=`ifconfig $device | grep -i hwaddr | sed -e 's#^.*hwaddr[[:space:]]*##I'` +dhcp_interface=$dhcp_interface +save_cfg + +# Mounting installation source +SOURCE=/tmp/source +FS=/tmp/fs + +echo +mkdir -p ${SOURCE} +mkdir -p ${FS} + +if test -e /dev/disk/by-uuid/will_be_substituted_with_actual_uuid; then + mount /dev/disk/by-uuid/will_be_substituted_with_actual_uuid ${FS} + mount -o loop ${FS}/nailgun.iso ${SOURCE} +fi + +OPENSTACK_VERSION=`cat ${SOURCE}/openstack_version` + +# ---------------------- +# UNPACKING REPOSITORIES +# ---------------------- + +wwwdir="/var/www/nailgun" +repodir="${wwwdir}/${OPENSTACK_VERSION}" + +# Copying Centos files +mkdir -p ${repodir}/centos/x86_64 +cp -r ${SOURCE}/images ${repodir}/centos/x86_64 +cp -r ${SOURCE}/isolinux ${repodir}/centos/x86_64 +cp -r ${SOURCE}/repodata ${repodir}/centos/x86_64 +cp -r ${SOURCE}/Packages ${repodir}/centos/x86_64 +cp ${SOURCE}/.treeinfo ${repodir}/centos/x86_64 + +# Copying Ubuntu files +mkdir -p ${repodir}/ubuntu/x86_64/images +cp -r ${SOURCE}/ubuntu/conf ${repodir}/ubuntu/x86_64 +cp -r ${SOURCE}/ubuntu/db ${repodir}/ubuntu/x86_64 +cp -r ${SOURCE}/ubuntu/dists ${repodir}/ubuntu/x86_64 +cp -r ${SOURCE}/ubuntu/indices ${repodir}/ubuntu/x86_64 +cp -r ${SOURCE}/ubuntu/pool ${repodir}/ubuntu/x86_64 +cp -r ${SOURCE}/ubuntu/installer-amd64/current/images/netboot/ubuntu-installer/amd64/linux ${repodir}/ubuntu/x86_64/images +cp -r ${SOURCE}/ubuntu/installer-amd64/current/images/netboot/ubuntu-installer/amd64/initrd.gz ${repodir}/ubuntu/x86_64/images + +# make links for backward compatibility +ln -s ${repodir}/centos ${wwwdir}/centos +ln -s ${repodir}/ubuntu ${wwwdir}/ubuntu + +# Copying bootstrap image +mkdir -p ${wwwdir}/bootstrap +cp -r ${SOURCE}/bootstrap/initramfs.img ${wwwdir}/bootstrap +cp -r ${SOURCE}/bootstrap/linux ${wwwdir}/bootstrap + +# Copying target images +cp -r ${SOURCE}/targetimages ${wwwdir} + +mkdir -p /root/.ssh +chmod 700 /root/.ssh +cp ${SOURCE}/bootstrap/bootstrap.rsa /root/.ssh +chmod 600 /root/.ssh/bootstrap.rsa + +# -------------------------- +# UNPACKING PUPPET MANIFESTS +# -------------------------- + +# create folders +mkdir -p /etc/puppet/${OPENSTACK_VERSION}/manifests/ +mkdir -p /etc/puppet/${OPENSTACK_VERSION}/modules/ +rm -rf /etc/puppet/modules/ + +# TODO(ikalnitsky): investigate why we need this +cp ${SOURCE}/puppet-slave.tgz ${wwwdir}/ + +# place modules and manifests +tar zxf ${SOURCE}/puppet-slave.tgz -C /etc/puppet/${OPENSTACK_VERSION}/modules +cp /etc/puppet/${OPENSTACK_VERSION}/modules/osnailyfacter/examples/site.pp /etc/puppet/${OPENSTACK_VERSION}/manifests/site.pp +cp ${SOURCE}/centos-versions.yaml ${SOURCE}/ubuntu-versions.yaml /etc/puppet/${OPENSTACK_VERSION}/manifests/ + +# make links for backward compatibility +pushd /etc/puppet +ln -s ${OPENSTACK_VERSION}/manifests/ /etc/puppet/manifests +ln -s ${OPENSTACK_VERSION}/modules/ /etc/puppet/modules +popd + +cp ${SOURCE}/send2syslog.py /bin/send2syslog.py +mkdir -p /var/lib/hiera +touch /var/lib/hiera/common.yaml /etc/puppet/hiera.yaml + +# Deploy docker images and ctl tools if we built ISO with docker containers support +[ -d "${SOURCE}/docker" ] && cp -r ${SOURCE}/docker ${wwwdir}/docker + +# Prepare local repository specification +rm /etc/yum.repos.d/CentOS*.repo +cat > /etc/yum.repos.d/nailgun.repo << EOF +[nailgun] +name=Nailgun Local Repo +baseurl=file:/var/www/nailgun/${OPENSTACK_VERSION}/centos/x86_64 +gpgcheck=0 +EOF + +# Disable subscription-manager plugins +sed -i 's/^enabled.*/enabled=0/' /etc/yum/pluginconf.d/product-id.conf || : +sed -i 's/^enabled.*/enabled=0/' /etc/yum/pluginconf.d/subscription-manager.conf || : + +# Disable GSSAPI in ssh server config +sed -i -e "/^\s*GSSAPICleanupCredentials yes/d" -e "/^\s*GSSAPIAuthentication yes/d" /etc/ssh/sshd_config + +# Copying bootstrap_admin_node.sh, chmod it and +# adding /etc/init/bootstrap_admin_node.conf +cp ${SOURCE}/bootstrap_admin_node.sh /usr/local/sbin/bootstrap_admin_node.sh +chmod 0777 /usr/local/sbin/bootstrap_admin_node.sh +cp ${SOURCE}/bootstrap_admin_node.conf /etc/init/bootstrap_admin_node.conf +echo "ENABLED=1" > /etc/sysconfig/bootstrap_admin_node + +# Copying version.yaml file. It contains COMMIT_SHA of last commit. +RELEASE=$(awk '/release/{gsub(/"/, "");print $2}' ${SOURCE}/version.yaml) +mkdir -p /etc/nailgun /etc/fuel/${RELEASE} /etc/fuel/release_versions +cp ${SOURCE}/version.yaml /etc/nailgun/version.yaml +cp ${SOURCE}/version.yaml /etc/fuel/${RELEASE}/version.yaml +ln -s /etc/fuel/${RELEASE}/version.yaml /etc/fuel/version.yaml +cp ${SOURCE}/version.yaml /etc/fuel/release_versions/`cat ${SOURCE}/openstack_version`.yaml + +# Generete Fuel UUID +uuidgen > /etc/fuel/fuel-uuid + +###Run fuel menu +[ -z "$showmenu" ] && showmenu="no" +cat > /root/.showfuelmenu << EOF +#Set to yes to run Fuel Setup +#Set to no to accept default settings +showmenu=$showmenu +EOF + +# Prepare custom /etc/issue logon banner and script for changing IP in it +cat > /etc/issue << EOF +######################################### +# Welcome to the Fuel server # +######################################### +Server is running on \m platform + +Fuel UI is available on: +http://:8000 + +Default administrator login: root +Default administrator password: r00tme + +Default Fuel UI login: admin +Default Fuel UI password: admin + +Please change root password on first login. + +EOF + + +cat >> '/etc/rc.local' << EOF +first=yes +for ip in \$(ip -o -4 addr | grep "eth." | awk '{print \$4 }' | cut -d/ -f1); do +if [ "\$first" = "yes" ]; then + ipstr="Fuel UI is available on: http://\$ip:8000" + first=no +else + ipstr=\$(printf "%s\n%25s%s" "\$ipstr" " " "http://\$ip:8000") +fi +done +tmpissue=\$(mktemp) +while read -r line; do + if [[ "\$line" =~ "Fuel UI is available on" ]]; then + echo -e "\$ipstr" >> \$tmpissue + elif [[ "\$line" =~ :8000$ ]]; then + : + else + echo -e "\$line" >> \$tmpissue + fi +done < /etc/issue +mv "\$tmpissue" /etc/issue + +EOF + +######### OPNFV addition BEGIN ############ +# Copy data into /opt/opnfv +# TODO: This ought to be a package instead! +mkdir -p /opt/opnfv +cp -r ${SOURCE}/opnfv /opt +######### OPNFV addition END ############ + +# Unmounting source +umount -f ${SOURCE} +rm -rf ${SOURCE} + +umount -f ${FS} || true +rm -rf ${FS} + +# Enabling/configuring NTPD and ntpdate services +echo "server 127.127.1.0" >> /etc/ntp.conf +echo "fudge 127.127.1.0 stratum 10" >> /etc/ntp.conf +echo "tos orphan 7" >> /etc/ntp.conf + +# Do not show error message on ntpdate failure. Customers should not be confused +# if admin node does not have access to the internet time servers. +sed -i /etc/rc.d/init.d/ntpdate -e 's/\([ $RETVAL -eq 0 ] && success || \)failure/\1success/' + +# Disabling splash +sed -i --follow-symlinks -e '/^\skernel/ s/rhgb//' /etc/grub.conf +sed -i --follow-symlinks -e '/^\skernel/ s/quiet//' /etc/grub.conf + +# Disabling console clearing +sed -i 's/getty/getty --noclear/' /etc/init/tty.conf + +# Disabling starting first console from start-ttys service +sed -i --follow-symlinks -e 's/ACTIVE_CONSOLES=.*/ACTIVE_CONSOLES=\/dev\/tty\[2-6\]/' /etc/sysconfig/init + +# Copying default bash settings to the root directory +cp -f /etc/skel/.bash* /root/ + +# Blacklist i2c_piix4 module for VirtualBox so it does not create kernel errors +[[ $(virt-what) = "virtualbox" ]] && echo "blacklist i2c_piix4" > /etc/modprobe.d/blacklist-i2c-piix4.conf + +%end diff --git a/fuel/build/f_isoroot/f_kscfg/ks.cfg.orig b/fuel/build/f_isoroot/f_kscfg/ks.cfg.orig new file mode 100644 index 0000000..44f9f42 --- /dev/null +++ b/fuel/build/f_isoroot/f_kscfg/ks.cfg.orig @@ -0,0 +1,566 @@ +install +text +%include /tmp/source.ks +reboot --eject +lang en_US.UTF-8 +keyboard us +rootpw r00tme +timezone --utc Etc/UTC +firewall --disabled +selinux --disabled +# ignore unsupported hardware warning +unsupported_hardware +# SKIP CONFIGURING X +skipx +# NEVER ever place zerombr here, it breaks automated installation +%include /tmp/bootloader.ks +%include /tmp/partition.ks + +# PREINSTALL SECTION +# HERE ARE COMMANDS THAT WILL BE LAUNCHED BEFORE +# INSTALLATION PROCESS ITSELF +%pre +#!/bin/sh + +# hard drives +drives="" +removable_drives="" +for drv in `ls -1 /sys/block | grep "sd\|hd\|vd\|cciss"`; do + if (grep -q 0 /sys/block/${drv}/removable); then + drives="${drives} ${drv}" + else + removable_drives="${removable_drives} ${drv}" + fi +done +default_drive=`echo ${drives} ${removable_drives} | awk '{print $1}'` + +installdrive="undefined" +forceformat="no" +for I in `cat /proc/cmdline`; do case "$I" in *=*) eval $I;; esac ; done + +set ${drives} ${removable_drives} +numdrives=`echo $#` + +tgtdrive="${installdrive}" + +function confirm_format { + check_drive="$1" + local confirm_format="no" + + if [[ "$forceformat" == "yes" ]] ; then + return 0 + fi + + if parted -s /dev/$check_drive print &>/dev/null ; then + echo + echo "$check_drive drive contains partition table:" + parted -s /dev/$check_drive print + echo + read -p "Are you sure you want to erase ALL data on disk $check_drive? (y/N)" confirm_format + if [[ "$confirm_format" == "y" ]] || [[ "$confirm_format" == "Y" ]] || [[ "$forceformat" == "yes" ]]; then + return 0 + else + return 1 + fi + else + return 0 + fi +} + +format_confirmed="no" + +if [ $numdrives -lt 1 ]; then + exec < /dev/tty3 > /dev/tty3 2>&1 + chvt 3 + clear + echo + echo '********************************************************************' + echo '* E R R O R *' + echo '* *' + echo '* There is no suitable media available for installation. *' + echo '* Please attach a drive and try again. *' + echo '* *' + echo '********************************************************************' + echo + read -p "Press Enter to shut down the system: " _ + poweroff +fi + +if [ ${numdrives} -gt 1 ] || [ `echo ${drives} | wc -w` -eq 0 ] ; then + exec < /dev/tty3 > /dev/tty3 2>&1 + chvt 3 + while [ "${tgtdrive}" = "undefined" ]; do + clear + echo + echo '********************************************************************************' + echo '* W A R N I N G *' + echo '* *' + echo '* Which of the detected hard drives do you want to be used as *' + echo '* the installation target? *' + echo '* *' + echo '********************************************************************************' + echo + echo "Possible choices" + echo "Persistent drives: ${drives}" + echo "Removable drives: ${removable_drives}" + echo + if [ `echo ${drives} | wc -w` -eq 1 ] ; then + read -t 30 -p "Choose hard drive: " tgtdrive || tgtdrive=$default_drive + else + read -p "Choose hard drive: " tgtdrive + fi + match="no" + for drive in ${drives[@]} ${removable_drives[@]}; do + if [[ "$drive" == "$tgtdrive" ]] && match="yes" ; then + if confirm_format $tgtdrive ; then + format_confirmed="yes" + break + else + tgtdrive="undefined" + read -p "You may select another disk. Press Enter to continue." _ + fi + fi + done + if [[ "$match" == "no" ]]; then + tgtdrive="undefined" + read -p "Invalid choice. Press Enter to continue." _ + fi + done + clear + chvt 1 +else + tgtdrive=`echo ${drives} | sed -e "s/^\s*//" -e "s/\s*$//"` +fi + +if [ "$format_confirmed" != "yes" ] ; then + exec < /dev/tty3 > /dev/tty3 2>&1 + chvt 3 + if ! confirm_format $tgtdrive ; then + clear + echo + echo '********************************************************************' + echo '* E R R O R *' + echo '* *' + echo '* Disk $tgtdrive contains active partition(s). *' + echo '* Installation cannot continue without confirmation. *' + echo '* *' + echo '********************************************************************' + echo + read -p "Press Enter to restart: " _ + reboot + fi + chvt 1 +fi + +# verify tgtdrive is at least 30GB +tgtdrivesize=$(( $(cat "/sys/class/block/${tgtdrive}/size") / 2 / 1024 )) +if [ $tgtdrivesize -lt 30720 ]; then + exec < /dev/tty3 > /dev/tty3 2>&1 + chvt 3 + clear + echo + echo '********************************************************************' + echo '* E R R O R *' + echo '* *' + echo '* Your disk is under 30GB in size. Installation cannot continue. *' + echo '* Restart installation with a larger disk. *' + echo '* *' + echo '********************************************************************' + echo + read -p "Press Enter to restart: " _ + reboot +fi + +# paths in /dev have "/" instead of "!" for cciss devices +tgtdrive=$(echo $tgtdrive | sed -e 's/!/\//') + +# source +if test -e /dev/disk/by-uuid/will_be_substituted_with_actual_uuid; then + echo "harddrive --partition=UUID=will_be_substituted_with_actual_uuid --dir=/" > /tmp/source.ks +else + echo "cdrom" > /tmp/source.ks +fi + +vgremove -ff os +dd if=/dev/zero of=/dev/${tgtdrive} bs=10M count=10 +sleep 3 +hdparm -z /dev/${tgtdrive} +parted -s /dev/${tgtdrive} mklabel gpt +parted -a none -s /dev/${tgtdrive} unit MiB mkpart primary 0 24 +parted -s /dev/${tgtdrive} set 1 bios_grub on +parted -a none -s /dev/${tgtdrive} unit MiB mkpart primary fat16 24 224 +parted -s /dev/${tgtdrive} set 2 boot on +parted -a none -s /dev/${tgtdrive} unit MiB mkpart primary 224 424 +sleep 3 +hdparm -z /dev/${tgtdrive} + +# partition + +# This adds support for the p seperator required for cciss devices +if echo ${tgtdrive} | grep -q -e cciss ; then + bootdev=${tgtdrive}p +else + bootdev=${tgtdrive} +fi +echo > /tmp/partition.ks +echo "partition /boot --onpart=/dev/${bootdev}3" >> /tmp/partition.ks +echo "partition pv.001 --ondisk=${tgtdrive} --size=30000 --grow" >> /tmp/partition.ks +echo "volgroup os pv.001" >> /tmp/partition.ks +echo "logvol swap --vgname=os --recommended --name=swap" >> /tmp/partition.ks +echo "logvol / --vgname=os --size=10000 --name=root --fstype=ext4" >> /tmp/partition.ks +echo "logvol /var --vgname=os --size=1 --grow --name=var --fstype=xfs" >> /tmp/partition.ks + +# bootloader +echo "bootloader --location=mbr --driveorder=${tgtdrive} --append=' biosdevname=0 crashkernel=none'" > /tmp/bootloader.ks + +# Anaconda can not install grub 0.97 on disks which are >4T. +# The reason is that grub does not support such large geometries +# and it simply thinks that the cylinder number has negative value. +# Here we just set geometry manually so that grub thinks that disk +# size is equal to 1G. +# 130 cylinders * (16065 * 512 = 8225280 bytes) = 1G +echo "%post --nochroot --log=/mnt/sysimage/root/anaconda-post-partition.log" > /tmp/post_partition.ks +echo "echo \"device (hd0) /dev/${tgtdrive}\" >> /tmp/grub.script" >> /tmp/post_partition.ks +echo "echo \"geometry (hd0) 130 255 63\" >> /tmp/grub.script" >> /tmp/post_partition.ks +echo "echo \"root (hd0,2)\" >> /tmp/grub.script" >> /tmp/post_partition.ks +echo "echo \"install /grub/stage1 (hd0) /grub/stage2 p /grub/grub.conf\" >> /tmp/grub.script" >> /tmp/post_partition.ks +echo "echo quit >> /tmp/grub.script" >> /tmp/post_partition.ks +echo "cat /tmp/grub.script | chroot /mnt/sysimage /sbin/grub --no-floppy --batch" >> /tmp/post_partition.ks + +%end + +%packages --nobase --excludedocs +@Core +authconfig +bind-utils +cronie +crontabs +curl +daemonize +dhcp +docker-io +fuelmenu +gdisk +lrzip +lsof +man +mlocate +nmap-ncat +ntp +openssh-clients +policycoreutils +rsync +ruby21-puppet +ruby21-rubygem-netaddr +ruby21-rubygem-openstack +selinux-policy-targeted +strace +subscription-manager +sysstat +system-config-firewall-base +tcpdump +telnet +vim-enhanced +virt-what +wget +yum + +%include /tmp/post_partition.ks + +# POSTINSTALL SECTION +# HERE ARE COMMANDS THAT WILL BE LAUNCHED JUST AFTER +# INSTALLATION ITSELF COMPLETED +%post +echo -e "modprobe nf_conntrack_ipv4\nmodprobe nf_conntrack_ipv6\nmodprobe nf_conntrack_tftp\nmodprobe nf_nat_tftp" >> /etc/rc.modules +chmod +x /etc/rc.modules +echo -e "net.nf_conntrack_max=1048576" >> /etc/sysctl.conf +mkdir -p /var/log/coredump +echo -e "kernel.core_pattern=/var/log/coredump/core.%e.%p.%h.%t" >> /etc/sysctl.conf +chmod 777 /var/log/coredump +echo -e "* soft core unlimited\n* hard core unlimited" >> /etc/security/limits.conf + +# Mount installation media in chroot +%post --nochroot --log=/mnt/sysimage/root/anaconda-post-before-chroot.log +#!/bin/sh + +SOURCE="/mnt/sysimage/tmp/source" + +for I in `cat /proc/cmdline`; do case "$I" in *=*) eval $I;; esac ; done + +mkdir -p "${SOURCE}" + +case "${repo}" in + nfs:*) + nfs_url="${repo#nfs:}" + mount -t nfs "${nfs_url}" "${SOURCE}" + ;; + *) + if [ -d "/mnt/source" ]; then + mount -o bind "/mnt/source" "${SOURCE}" + fi + ;; +esac + +%post --log=/root/anaconda-post-after-chroot.log +#!/bin/bash + +set -x + +function save_cfg { + scrFile="/etc/sysconfig/network-scripts/ifcfg-$device" + search="domain $domain\nsearch $domain" + sed -i -e 's#^\(HOSTNAME=\).*$#\1'"$hostname"'#' /etc/sysconfig/network + grep -q "^\s*$ip\s+$hostname" /etc/hosts || echo "$ip $hostname" >> /etc/hosts + echo "${search}\nnameserver 127.0.0.1" > /etc/resolv.conf + [ $dns1 ] && echo -e "${search}\nnameserver $dns1" > /etc/resolv.conf + [ $dns1 ] && echo -e "${search}\nnameserver $dns1" > /etc/dnsmasq.upstream + [ $dns2 ] && echo "nameserver $dns2" >> /etc/resolv.conf + [ $dns2 ] && echo "nameserver $dns2" >> /etc/dnsmasq.upstream + + echo DEVICE=$device > $scrFile + echo ONBOOT=yes >> $scrFile + echo NM_CONTROLLED=no >> $scrFile + echo HWADDR=$hwaddr >> $scrFile + echo USERCTL=no >> $scrFile + echo PEERDNS=no >> $scrFile + if [ $ip ]; then + echo BOOTPROTO=static >> $scrFile + echo IPADDR=$ip >> $scrFile + echo NETMASK=$netmask >> $scrFile + else + echo BOOTPROTO=dhcp >> $scrFile + fi + scrDHCPFile="/etc/sysconfig/network-scripts/ifcfg-$dhcp_interface" + #Ignore gateway and set up DHCP if it is used, otherwise apply it + if [ $dhcp_interface ] && [ "$dhcp_interface" != "$device" ]; then + echo "DEVICE=$dhcp_interface" > $scrDHCPFile + echo "BOOTPROTO=dhcp" >> $scrDHCPFile + echo "ONBOOT=yes" >> $scrDHCPFile + echo "USERCTL=no" >> $scrDHCPFile + else + echo GATEWAY=$gw >> /etc/sysconfig/network + fi +} + +# Default FQDN +hostname="nailgun.mirantis.com" + +for I in `cat /proc/cmdline`; do case "$I" in *=*) eval $I;; esac ; done +hostname=$hostname +domain=${hostname#*.} +ip=$ip +netmask=$netmask +gw=$gw +device="eth0" +hwaddr=`ifconfig $device | grep -i hwaddr | sed -e 's#^.*hwaddr[[:space:]]*##I'` +dhcp_interface=$dhcp_interface +save_cfg + +# Mounting installation source +SOURCE=/tmp/source +FS=/tmp/fs + +echo +mkdir -p ${SOURCE} +mkdir -p ${FS} + +if test -e /dev/disk/by-uuid/will_be_substituted_with_actual_uuid; then + mount /dev/disk/by-uuid/will_be_substituted_with_actual_uuid ${FS} + mount -o loop ${FS}/nailgun.iso ${SOURCE} +fi + +OPENSTACK_VERSION=`cat ${SOURCE}/openstack_version` + +# ---------------------- +# UNPACKING REPOSITORIES +# ---------------------- + +wwwdir="/var/www/nailgun" +repodir="${wwwdir}/${OPENSTACK_VERSION}" + +# Copying Centos files +mkdir -p ${repodir}/centos/x86_64 +cp -r ${SOURCE}/images ${repodir}/centos/x86_64 +cp -r ${SOURCE}/isolinux ${repodir}/centos/x86_64 +cp -r ${SOURCE}/repodata ${repodir}/centos/x86_64 +cp -r ${SOURCE}/Packages ${repodir}/centos/x86_64 +cp ${SOURCE}/.treeinfo ${repodir}/centos/x86_64 + +# Copying Ubuntu files +mkdir -p ${repodir}/ubuntu/x86_64/images +cp -r ${SOURCE}/ubuntu/conf ${repodir}/ubuntu/x86_64 +cp -r ${SOURCE}/ubuntu/db ${repodir}/ubuntu/x86_64 +cp -r ${SOURCE}/ubuntu/dists ${repodir}/ubuntu/x86_64 +cp -r ${SOURCE}/ubuntu/indices ${repodir}/ubuntu/x86_64 +cp -r ${SOURCE}/ubuntu/pool ${repodir}/ubuntu/x86_64 +cp -r ${SOURCE}/ubuntu/installer-amd64/current/images/netboot/ubuntu-installer/amd64/linux ${repodir}/ubuntu/x86_64/images +cp -r ${SOURCE}/ubuntu/installer-amd64/current/images/netboot/ubuntu-installer/amd64/initrd.gz ${repodir}/ubuntu/x86_64/images + +# make links for backward compatibility +ln -s ${repodir}/centos ${wwwdir}/centos +ln -s ${repodir}/ubuntu ${wwwdir}/ubuntu + +# Copying bootstrap image +mkdir -p ${wwwdir}/bootstrap +cp -r ${SOURCE}/bootstrap/initramfs.img ${wwwdir}/bootstrap +cp -r ${SOURCE}/bootstrap/linux ${wwwdir}/bootstrap + +# Copying target images +cp -r ${SOURCE}/targetimages ${wwwdir} + +mkdir -p /root/.ssh +chmod 700 /root/.ssh +cp ${SOURCE}/bootstrap/bootstrap.rsa /root/.ssh +chmod 600 /root/.ssh/bootstrap.rsa + +# -------------------------- +# UNPACKING PUPPET MANIFESTS +# -------------------------- + +# create folders +mkdir -p /etc/puppet/${OPENSTACK_VERSION}/manifests/ +mkdir -p /etc/puppet/${OPENSTACK_VERSION}/modules/ +rm -rf /etc/puppet/modules/ + +# TODO(ikalnitsky): investigate why we need this +cp ${SOURCE}/puppet-slave.tgz ${wwwdir}/ + +# place modules and manifests +tar zxf ${SOURCE}/puppet-slave.tgz -C /etc/puppet/${OPENSTACK_VERSION}/modules +cp /etc/puppet/${OPENSTACK_VERSION}/modules/osnailyfacter/examples/site.pp /etc/puppet/${OPENSTACK_VERSION}/manifests/site.pp +cp ${SOURCE}/centos-versions.yaml ${SOURCE}/ubuntu-versions.yaml /etc/puppet/${OPENSTACK_VERSION}/manifests/ + +# make links for backward compatibility +pushd /etc/puppet +ln -s ${OPENSTACK_VERSION}/manifests/ /etc/puppet/manifests +ln -s ${OPENSTACK_VERSION}/modules/ /etc/puppet/modules +popd + +cp ${SOURCE}/send2syslog.py /bin/send2syslog.py +mkdir -p /var/lib/hiera +touch /var/lib/hiera/common.yaml /etc/puppet/hiera.yaml + +# Deploy docker images and ctl tools if we built ISO with docker containers support +[ -d "${SOURCE}/docker" ] && cp -r ${SOURCE}/docker ${wwwdir}/docker + +# Prepare local repository specification +rm /etc/yum.repos.d/CentOS*.repo +cat > /etc/yum.repos.d/nailgun.repo << EOF +[nailgun] +name=Nailgun Local Repo +baseurl=file:/var/www/nailgun/${OPENSTACK_VERSION}/centos/x86_64 +gpgcheck=0 +EOF + +# Disable subscription-manager plugins +sed -i 's/^enabled.*/enabled=0/' /etc/yum/pluginconf.d/product-id.conf || : +sed -i 's/^enabled.*/enabled=0/' /etc/yum/pluginconf.d/subscription-manager.conf || : + +# Disable GSSAPI in ssh server config +sed -i -e "/^\s*GSSAPICleanupCredentials yes/d" -e "/^\s*GSSAPIAuthentication yes/d" /etc/ssh/sshd_config + +# Copying bootstrap_admin_node.sh, chmod it and +# adding /etc/init/bootstrap_admin_node.conf +cp ${SOURCE}/bootstrap_admin_node.sh /usr/local/sbin/bootstrap_admin_node.sh +chmod 0777 /usr/local/sbin/bootstrap_admin_node.sh +cp ${SOURCE}/bootstrap_admin_node.conf /etc/init/bootstrap_admin_node.conf +echo "ENABLED=1" > /etc/sysconfig/bootstrap_admin_node + +# Copying version.yaml file. It contains COMMIT_SHA of last commit. +RELEASE=$(awk '/release/{gsub(/"/, "");print $2}' ${SOURCE}/version.yaml) +mkdir -p /etc/nailgun /etc/fuel/${RELEASE} /etc/fuel/release_versions +cp ${SOURCE}/version.yaml /etc/nailgun/version.yaml +cp ${SOURCE}/version.yaml /etc/fuel/${RELEASE}/version.yaml +ln -s /etc/fuel/${RELEASE}/version.yaml /etc/fuel/version.yaml +cp ${SOURCE}/version.yaml /etc/fuel/release_versions/`cat ${SOURCE}/openstack_version`.yaml + +# Generete Fuel UUID +uuidgen > /etc/fuel/fuel-uuid + +###Run fuel menu +[ -z "$showmenu" ] && showmenu="no" +cat > /root/.showfuelmenu << EOF +#Set to yes to run Fuel Setup +#Set to no to accept default settings +showmenu=$showmenu +EOF + +# Prepare custom /etc/issue logon banner and script for changing IP in it +cat > /etc/issue << EOF +######################################### +# Welcome to the Fuel server # +######################################### +Server is running on \m platform + +Fuel UI is available on: +http://:8000 + +Default administrator login: root +Default administrator password: r00tme + +Default Fuel UI login: admin +Default Fuel UI password: admin + +Please change root password on first login. + +EOF + + +cat >> '/etc/rc.local' << EOF +first=yes +for ip in \$(ip -o -4 addr | grep "eth." | awk '{print \$4 }' | cut -d/ -f1); do +if [ "\$first" = "yes" ]; then + ipstr="Fuel UI is available on: http://\$ip:8000" + first=no +else + ipstr=\$(printf "%s\n%25s%s" "\$ipstr" " " "http://\$ip:8000") +fi +done +tmpissue=\$(mktemp) +while read -r line; do + if [[ "\$line" =~ "Fuel UI is available on" ]]; then + echo -e "\$ipstr" >> \$tmpissue + elif [[ "\$line" =~ :8000$ ]]; then + : + else + echo -e "\$line" >> \$tmpissue + fi +done < /etc/issue +mv "\$tmpissue" /etc/issue + +EOF + + +# Unmounting source +umount -f ${SOURCE} +rm -rf ${SOURCE} + +umount -f ${FS} || true +rm -rf ${FS} + +# Enabling/configuring NTPD and ntpdate services +echo "server 127.127.1.0" >> /etc/ntp.conf +echo "fudge 127.127.1.0 stratum 10" >> /etc/ntp.conf +echo "tos orphan 7" >> /etc/ntp.conf + +# Do not show error message on ntpdate failure. Customers should not be confused +# if admin node does not have access to the internet time servers. +sed -i /etc/rc.d/init.d/ntpdate -e 's/\([ $RETVAL -eq 0 ] && success || \)failure/\1success/' + +# Disabling splash +sed -i --follow-symlinks -e '/^\skernel/ s/rhgb//' /etc/grub.conf +sed -i --follow-symlinks -e '/^\skernel/ s/quiet//' /etc/grub.conf + +# Disabling console clearing +sed -i 's/getty/getty --noclear/' /etc/init/tty.conf + +# Disabling starting first console from start-ttys service +sed -i --follow-symlinks -e 's/ACTIVE_CONSOLES=.*/ACTIVE_CONSOLES=\/dev\/tty\[2-6\]/' /etc/sysconfig/init + +# Copying default bash settings to the root directory +cp -f /etc/skel/.bash* /root/ + +# Blacklist i2c_piix4 module for VirtualBox so it does not create kernel errors +[[ $(virt-what) = "virtualbox" ]] && echo "blacklist i2c_piix4" > /etc/modprobe.d/blacklist-i2c-piix4.conf + +%end
\ No newline at end of file diff --git a/fuel/build/f_isoroot/f_predeployment/Makefile b/fuel/build/f_isoroot/f_predeployment/Makefile new file mode 100644 index 0000000..a5252df --- /dev/null +++ b/fuel/build/f_isoroot/f_predeployment/Makefile @@ -0,0 +1,28 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) + +.PHONY: all +all: + @mkdir -p release/opnfv + @cp pre-deploy.sh release/opnfv + @cp sysinfo.sh release/opnfv + @cp transform_yaml.py release/opnfv + @chmod 755 release/opnfv/* + +.PHONY: clean +clean: + @rm -rf release + + +.PHONY: release +release:clean all + @cp -Rvp release/* ../release diff --git a/fuel/build/f_isoroot/f_predeployment/README b/fuel/build/f_isoroot/f_predeployment/README new file mode 100644 index 0000000..3eef9f2 --- /dev/null +++ b/fuel/build/f_isoroot/f_predeployment/README @@ -0,0 +1,18 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +This is the start of the interactive frontend that will add OPNFV configuration into +the astute.yaml of the nodes. Currently just a test setup - prepare an installation +up to the point of "deploy changes", but run "./pre-deploy.sh <envid> fragment.yaml" +before actually hitting deploy, which will make sure to add the example fragment to +the nodes. + +Note that the only part of the fragment.yaml that actually is acted on is the hosts +part at this time. diff --git a/fuel/build/f_isoroot/f_predeployment/pre-deploy.sh b/fuel/build/f_isoroot/f_predeployment/pre-deploy.sh new file mode 100755 index 0000000..c5c6c42 --- /dev/null +++ b/fuel/build/f_isoroot/f_predeployment/pre-deploy.sh @@ -0,0 +1,401 @@ +#!/bin/bash -e +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + + +error_exit () { + echo "$@" >&2 + exit 1 +} + +get_env() { + local env_id=${1:-""} + + if [ -z $env_id ]; then + local n_envs=$(fuel env --list | grep -v -E "^id|^--|^ *$" | wc -l) + if [ $n_envs -ne 1 ]; then + echo "Usage: $0 [<env-id>]" >&2 + error_exit "If only a single environment is present it can be left" \ + "out. Otherwise the environment must be selected" + fi + env_id=$(fuel env --list | grep -v -E "^id|^--" | awk '{print $1}') + else + if ! fuel --env $env_id environment 2>/dev/null grep -v -E "^id|^--" | \ + grep -q ^$env_id; then + error_exit "No such environment ID: $env_id" + fi + fi + echo $env_id +} + +get_node_uid () { + cat $1 | grep "^uid: " | sed "s/^uid: '//" | sed "s/'$//" +} + +get_node_role () { + cat $1 | grep "^role: " | sed "s/^role: //" +} + +get_next_cic () { + file=$1 + + last=`cat $file | sed 's/.*://' | grep "cic-" | sed 's/cic\-.*sl//' | sort -n | tail -1` + if [ -z "$last" ]; then + next=1 + else + next=$[$last + 2] + fi + echo $next +} + +get_next_compute () { + file=$1 + + last=`cat $file | sed 's/.*://' | grep "cmp-" | sed 's/cmp\-.*sl//' | sort -n | tail -1` + if [ -z "$last" ]; then + next=7 + else + next=$[$last + 2] + fi + echo $next +} + +modify_hostnames () { + env=$1 + file=$2 + for line in `cat $file` + do + old=`echo $line | sed 's/:.*//'` + new=`echo $line | sed 's/.*://'` + echo "Applying: $old -> $new" + + for dfile in deployment_$env/*.yaml + do + sed -i "s/$old/$new/g" $dfile + done + + for pfile in provisioning_$env/*.yaml + do + sed -i "s/$old/$new/g" $pfile + done + done +} + +setup_hostnames () { + ENV=$1 + cd ${CONFIGDIR} + touch hostnames.$ENV + + for dfile in deployment_$ENV/*.yaml + do + uid=`get_node_uid $dfile` + hostname=`grep "^node-$uid:" hostnames.$ENV | sed 's/.*://'` + if [ -z $hostname ]; then + + pfile=provisioning_$ENV/node-$uid.yaml + role=`get_node_role $dfile` + + case $role in + primary-controller) + hostname="cic-pod0-sh0-sl`get_next_cic hostnames.$ENV`" + ;; + controller) + hostname="cic-pod0-sh0-sl`get_next_cic hostnames.$ENV`" + ;; + compute) + hostname="cmp-pod0-sh0-sl`get_next_compute hostnames.$ENV`" + ;; + *) + echo "Unknown node type for UID $uid" + exit 1 + ;; + esac + + echo "node-$uid:$hostname" >> hostnames.$ENV + else + echo "Already got hostname $hostname for node-$uid" + + fi + done + + rm -f hostnames.$ENV.old + mv hostnames.$ENV hostnames.$ENV.old + sort hostnames.$ENV.old | uniq > hostnames.$ENV + modify_hostnames $ENV hostnames.$ENV +} + + + +get_provisioning_info () { + ENV=$1 + mkdir -p ${CONFIGDIR} + cd ${CONFIGDIR} + rm -Rf provisioning_$ENV + echo "Getting provisioning info..." + fuel --env $ENV provisioning --default + if [ $? -ne 0 ]; then + echo "Error: Could not get provisioning info for env $ENV">&2 + exit 1 + fi +} + +get_deployment_info () { + ENV=$1 + mkdir -p ${CONFIGDIR} + cd ${CONFIGDIR} + rm -Rf deployment_$ENV + echo "Getting deployment info..." + fuel --env $ENV deployment --default + if [ $? -ne 0 ]; then + echo "Error: Could not get deployment info for env $ENV">&2 + exit 1 + fi +} + +transform_yaml () { + ENV=$1 + cd ${CONFIGDIR} + for dfile in deployment_$ENV/*.yaml + do + /opt/opnfv/transform_yaml.py $dfile + done +} + +commit_changes () { + ENV=$1 + cd ${CONFIGDIR} + + fuel --env $ENV deployment --upload + fuel --env $ENV provisioning --upload +} + +add_yaml_fragment () { + ENV=$1 + FRAGMENT=${CONFIGDIR}/fragment.yaml.$ENV + + cd ${CONFIGDIR} + for dfile in deployment_$ENV/*.yaml + do + cnt=`grep "^opnfv:" $dfile | wc -l ` + if [ $cnt -eq 0 ]; then + echo "Adding fragment to $dfile" + cat $FRAGMENT >> $dfile + else + echo "Already have fragment in $dfile" + fi + done +} + + +ip_valid() { + IP_ADDRESS="$1" + # Check if the format looks right_ + echo "$IP_ADDRESS" | egrep -qE '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}' || return 1 + #check that each octect is less than or equal to 255: + echo $IP_ADDRESS | awk -F'.' '$1 <=255 && $2 <= 255 && $3 <=255 && $4 <= 255 {print "Y" } ' | grep -q Y || return 1 + return 0 +} + + +generate_ntp_entry() { + FILE=$1 + read -p "NTP server:" NTP_SERVER + if [ -z "$NTP_SERVER" ]; then + return 1 + elif confirm_yes "Are you sure you want to add this entry (y/n): "; then + echo "Confirmed" + echo " server $NTP_SERVER" >> $FILE + fi +} + +generate_hostfile_entry() { + FILE=$1 + read -p "Name:" HOST_NAME + if [ -z "$HOST_NAME" ]; then + return 1 + else + read -p "FQDN:" HOST_FQDN + read -p "IP: " HOST_IP + while ! ip_valid "$HOST_IP" + do + echo "This is not a valid IP! Try again." + read -p "IP: " HOST_IP + done + fi + if confirm_yes "Are you sure you want to add this entry (y/n): "; then + echo "Confirmed" + echo " - name: $HOST_NAME" >> $FILE + echo " address: $HOST_IP" >> $FILE + echo " fqdn: $HOST_FQDN" >> $FILE + else + echo "Not confirmed" + fi + return 0 +} + +generate_dns_entry() { + FILE=$1 + PROMPT=$2 + read -p "${PROMPT}:" DNS_IP + if [ -z "$DNS_IP" ]; then + return 1 + else + while ! ip_valid "$DNS_IP" + do + echo "This is not a valid IP! Try again." + read -p "${PROMPT}: " DNS_IP + done + fi + if confirm_yes "Are you sure you want to add this entry (y/n): "; then + echo "Confirmed" + echo " - $DNS_IP" >> $FILE + else + echo "Not confirmed" + fi + return 0 +} + +confirm_yes() { + prompt=$1 + while true + do + read -p "$prompt" YESNO + case $YESNO in + [Yy]) + return 0 + ;; + [Nn]) + return 1 + ;; + esac + done +} + +generate_yaml_fragment() { + ENV=$1 + FRAGMENT=${CONFIGDIR}/fragment.yaml.$ENV + + if [ -f $FRAGMENT ]; then + echo "Manual configuration already performed, reusing previous data from $FRAGMENT." + echo "Press return to continue or ^C to stop." + read ans + return + fi + + echo "opnfv:" > ${FRAGMENT} + + clear + echo -e "\n\nPre-deployment configuration\n\n" + + echo -e "\n\nIPs for the DNS servers to go into /etc/resolv.conf. You will be" + echo -e "prompted for one IP at the time. Press return on an empty line" + echo -e "to complete your input. If no DNS server is specified, the IP of" + echo -e "the Fuel master will be used instead.\n" + + DNSCICYAML=${CONFIGDIR}/cicdns.yaml.$ENV + rm -f $DNSCICYAML + + echo -e "\n\n" + + while generate_dns_entry $DNSCICYAML "IP for CIC name servers" + do + : + done + + if [ -f $DNSCICYAML ]; then + echo " dns:" >> $FRAGMENT + echo " controller:" >> $FRAGMENT + cat $DNSCICYAML >> $FRAGMENT + fi + + + DNSCMPYAML=${CONFIGDIR}/cmpdns.yaml.$ENV + rm -f $DNSCMPYAML + + echo -e "\n\n" + + while generate_dns_entry $DNSCMPYAML "IP for compute node name servers" + do + : + done + + + if [ -f $DNSCMPYAML ]; then + if [ ! -f $DNSCICYAML ]; then + echo " dns:" >> $FRAGMENT + fi + echo " compute:" >> $FRAGMENT + cat $DNSCMPYAML >> $FRAGMENT + fi + + echo -e "\n\nHosts file additions for controllers and compute nodes. You will be" + echo -e "prompted for name, FQDN and IP for each entry. Press return when prompted" + echo -e "for a name when you have completed your input.\n" + + + HOSTYAML=${CONFIGDIR}/hosts.yaml.$ENV + rm -f $HOSTYAML + while generate_hostfile_entry $HOSTYAML + do + : + done + + if [ -f $HOSTYAML ]; then + echo " hosts:" >> $FRAGMENT + cat $HOSTYAML >> $FRAGMENT + fi + + echo -e "\n\nNTP upstream configuration for controllers.You will be" + echo -e "prompted for a NTP server each entry. Press return when prompted" + echo -e "for a NTP serverwhen you have completed your input.\n" + + + NTPYAML=${CONFIGDIR}/ntp.yaml.$ENV + rm -f $NTPYAML + while generate_ntp_entry $NTPYAML + do + : + done + + if [ -f $NTPYAML ]; then + echo " ntp:" >> $FRAGMENT + echo " controller: |" >> $FRAGMENT + cat $NTPYAML >> $FRAGMENT + + echo " compute: |" >> $FRAGMENT + for ctl in `find $CONFIGDIR/deployment_$ENV -name '*controller*.yaml'` + do + fqdn=`grep "^fqdn:" $ctl | sed 's/fqdn: *//'` + echo " server $fqdn" >> $FRAGMENT + done + fi + + # If nothing added make sure we get an empty opnfv hash + # instead of a NULL hash. + if [ $(wc -l $FRAGMENT | awk '{print $1}') -le 1 ]; then + echo "opnfv: {}" >$FRAGMENT + fi +} + +ENV=$(get_env "$@") + +CONFIGDIR="/var/lib/opnfv" +mkdir -p $CONFIGDIR + +get_deployment_info $ENV +# Uncomment the below to enable the control_bond example +#transform_yaml $ENV +get_provisioning_info $ENV +generate_yaml_fragment $ENV +# The feature to change hostnames from node-<n> to cmp- or cic- is disabled. +# To turn it on, uncomment the following line. +#setup_hostnames $ENV +add_yaml_fragment $ENV +commit_changes $ENV diff --git a/fuel/build/f_isoroot/f_predeployment/sysinfo.sh b/fuel/build/f_isoroot/f_predeployment/sysinfo.sh new file mode 100755 index 0000000..e99cac0 --- /dev/null +++ b/fuel/build/f_isoroot/f_predeployment/sysinfo.sh @@ -0,0 +1,12 @@ +#!/bin/sh +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +dockerctl shell cobbler cobbler system list | grep -v default | xargs -n 1 host | sort | sed 's/\..* /\t/' diff --git a/fuel/build/f_isoroot/f_predeployment/transform_yaml.py b/fuel/build/f_isoroot/f_predeployment/transform_yaml.py new file mode 100755 index 0000000..14eec4c --- /dev/null +++ b/fuel/build/f_isoroot/f_predeployment/transform_yaml.py @@ -0,0 +1,68 @@ +#!/usr/bin/python +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +# Remove control and management network transformations from file. +# Only to be used together with f_control_bond_example (enable in +# pre-deploy.sh) + +import yaml +import re +import sys +import os + +if len(sys.argv) != 2: + sys.stderr.write("Usage: "+sys.argv[0]+" <filename>\n") + sys.exit(1) + +filename = sys.argv[1] +if not os.path.exists(filename): + sys.stderr.write("ERROR: The file "+filename+" could not be opened\n") + sys.exit(1) + +ignore_values = [ "eth0", "eth1", "br-mgmt", "br-fw-admin" ] + +infile = open(filename, 'r') +doc = yaml.load(infile) +infile.close() + +out={} + +for scheme in doc: + if scheme == "network_scheme": + mytransformation = {} + for operation in doc[scheme]: + if operation == "transformations": + # We need the base bridges for l23network to be happy, + # remove everything else. + mytrans = [ { "action": "add-br", "name": "br-mgmt" }, + { "action": "add-br", "name": "br-fw-admin" } ] + for trans in doc[scheme][operation]: + delete = 0 + for ignore in ignore_values: + matchObj = re.search(ignore,str(trans)) + if matchObj: + delete = 1 + if delete == 0: + mytrans.append(trans) + else: + pass + #print "Deleted", trans + + mytransformation[operation] = mytrans + else: + mytransformation[operation] = doc[scheme][operation] + out[scheme] = mytransformation + else: + out[scheme] = doc[scheme] + +outfile = open(filename, 'w') +outfile.write(yaml.dump(out, default_flow_style=False)) +outfile.close() diff --git a/fuel/build/f_java/Makefile b/fuel/build/f_java/Makefile new file mode 100644 index 0000000..759081f --- /dev/null +++ b/fuel/build/f_java/Makefile @@ -0,0 +1,42 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) +include ../config.mk + +PACKAGE := openjdk-8-jdk +VERSION := 8u40~b09-1_amd64 +DEB_NAME := $(PACKAGE)_$(VERSION).deb + +.PHONY: all +all: release/pool/main/$(DEB_NAME) + +release/pool/main/$(DEB_NAME): +ifeq ($(JAVA_MAIN_REPO),) + @echo "No config-spec target for Java, nothing to build" +else + @mkdir -p release/pool/main && \ + cd release/pool/main && \ + wget -N $(JAVA_MAIN_REPO)/$(DEB_NAME) + @lintian release/pool/main/$(DEB_NAME) +endif + +.PHONY: clean +clean: + @rm -rf release + @rm -f $(DEB_DEST)/$(DEB_NAME) + + +.PHONY: release +release:release/pool/main/$(DEB_NAME) +ifneq ($(JAVA_MAIN_REPO),) + @cp release/pool/main/$(DEB_NAME) $(DEB_DEST) + @cp -Rvp puppet/modules/* $(PUPPET_DEST) +endif diff --git a/fuel/build/f_java/README b/fuel/build/f_java/README new file mode 100644 index 0000000..776d47a --- /dev/null +++ b/fuel/build/f_java/README @@ -0,0 +1,44 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +This directory adds the openjdk-8-jdk Java package and related puppet +deployment manifest such that it gets built into the .iso image an deployed +on the stack controller cluster. + +The f_java has the following structure: +. ++--------+----------+-----------+ + | | | + puppet/ Makefile README + | (this file) + | + | + modules/ + | + | + | + opnfv/ + | + | + | + manifests/ + | + | + | + java.pp + +Makefile: +Invoked by the git root Makefile, it pulls the openjdk-8-jdk deb package +from a propper mirror and pushes i together with the manifests to a fuel +build artifact directory, such that it eventually gets built into the new +fuel .iso + +java.pp: +Controls the installation and configuration of java diff --git a/fuel/build/f_java/puppet/modules/opnfv/manifests/java.pp b/fuel/build/f_java/puppet/modules/opnfv/manifests/java.pp new file mode 100644 index 0000000..26e7c39 --- /dev/null +++ b/fuel/build/f_java/puppet/modules/opnfv/manifests/java.pp @@ -0,0 +1,11 @@ +class opnfv::java { + if $::osfamily == 'Debian' { + case $::fuel_settings['role'] { + /controller/: { + package { 'openjdk-8-jdk': + ensure => installed, + } + } + } + } +} diff --git a/fuel/build/f_java/testing/README b/fuel/build/f_java/testing/README new file mode 100644 index 0000000..c0ef7d6 --- /dev/null +++ b/fuel/build/f_java/testing/README @@ -0,0 +1,12 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +In order to test the functionality without performing a full deployment, run "puppet apply" on the +fake_init.pp which will call only the opnfv::java class. diff --git a/fuel/build/f_java/testing/fake_init.pp b/fuel/build/f_java/testing/fake_init.pp new file mode 100644 index 0000000..237b0c1 --- /dev/null +++ b/fuel/build/f_java/testing/fake_init.pp @@ -0,0 +1,13 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +$fuel_settings = parseyaml($astute_settings_yaml) + +include opnfv::java diff --git a/fuel/build/f_jre/Makefile b/fuel/build/f_jre/Makefile new file mode 100644 index 0000000..43f5e76 --- /dev/null +++ b/fuel/build/f_jre/Makefile @@ -0,0 +1,41 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# daniel.smith@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) +include ../config.mk + +PACKAGE := openjdk-8-jre +VERSION := 8u40~b09-1_amd64 +DEB_NAME := $(PACKAGE)_$(VERSION).deb + +.PHONY: all +all: release/pool/main/$(DEB_NAME) + +release/pool/main/$(DEB_NAME): +ifeq ($(JAVA_MAIN_REPO),) + @echo "No config-spec target for Java, nothing to build" +else + @mkdir -p release/pool/main && \ + cd release/pool/main && \ + wget -N $(JAVA_MAIN_REPO)/$(DEB_NAME) +endif + +.PHONY: clean +clean: + @rm -rf release + @rm -f $(DEB_DEST)/$(DEB_NAME) + + +.PHONY: release +release:release/pool/main/$(DEB_NAME) +ifneq ($(JAVA_MAIN_REPO),) + @cp release/pool/main/$(DEB_NAME) $(DEB_DEST) + @cp -Rvp puppet/modules/* $(PUPPET_DEST) +endif + diff --git a/fuel/build/f_jre/README b/fuel/build/f_jre/README new file mode 100644 index 0000000..24e4cdd --- /dev/null +++ b/fuel/build/f_jre/README @@ -0,0 +1,44 @@ + +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# daniel.smith@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +This directory adds the openjdk-8-jre Java package and related puppet +deployment manifest such that it gets built into the .iso image an deployed +on the stack controller cluster. + +The f_jre has the following structure: +. ++--------+----------+-----------+ + | | | + puppet/ Makefile README + | (this file) + | + | + modules/ + | + | + | + opnfv/ + | + | + | + manifests/ + | + | + | + jre.pp + +Makefile: +Invoked by the git root Makefile, it pulls the openjdk-8-jre deb package +from a propper mirror and pushes i together with the manifests to a fuel +build artifact directory, such that it eventually gets built into the new +fuel .iso + +jre.pp: +Controls the installation and configuration of java diff --git a/fuel/build/f_jre/puppet/modules/opnfv/manifests/jre.pp b/fuel/build/f_jre/puppet/modules/opnfv/manifests/jre.pp new file mode 100644 index 0000000..3c342d2 --- /dev/null +++ b/fuel/build/f_jre/puppet/modules/opnfv/manifests/jre.pp @@ -0,0 +1,14 @@ +class opnfv::jre { + if $::osfamily == 'Debian' { + + + case $::fuel_settings['role'] { + /controller/: { + package { 'openjdk-8-jre': + ensure => installed, + } + } + } + } +} + diff --git a/fuel/build/f_jre/testing/README b/fuel/build/f_jre/testing/README new file mode 100644 index 0000000..028230d --- /dev/null +++ b/fuel/build/f_jre/testing/README @@ -0,0 +1,13 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +In order to test the functionality without performing a full deployment, run "puppet apply" on the +fake_init.pp which will call only the opnfv::java class. + diff --git a/fuel/build/f_jre/testing/fake_init.pp b/fuel/build/f_jre/testing/fake_init.pp new file mode 100644 index 0000000..83c7adb --- /dev/null +++ b/fuel/build/f_jre/testing/fake_init.pp @@ -0,0 +1,13 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# daniel.smith@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +$fuel_settings = parseyaml($astute_settings_yaml) + +include opnfv::jre + diff --git a/fuel/build/f_jre_head/Makefile b/fuel/build/f_jre_head/Makefile new file mode 100644 index 0000000..2efc81a --- /dev/null +++ b/fuel/build/f_jre_head/Makefile @@ -0,0 +1,43 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# daniel.smith@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) +include ../config.mk + +PACKAGE := openjdk-8-jre-headless +VERSION := 8u40~b09-1_amd64 +DEB_NAME := $(PACKAGE)_$(VERSION).deb + + +.PHONY: all +all: release/pool/main/$(DEB_NAME) + +release/pool/main/$(DEB_NAME): +ifeq ($(JAVA_MAIN_REPO),) + @echo "No config-spec target for Java, nothing to build" +else + @mkdir -p release/pool/main && \ + cd release/pool/main && \ + wget -N $(JAVA_MAIN_REPO)/$(DEB_NAME) + @lintian release/pool/main/$(DEB_NAME) +endif + +.PHONY: clean +clean: + @rm -rf release + @rm -f $(DEB_DEST)/$(DEB_NAME) + + +.PHONY: release +release:release/pool/main/$(DEB_NAME) +ifneq ($(JAVA_MAIN_REPO),) + @cp release/pool/main/$(DEB_NAME) $(DEB_DEST) + @cp -Rvp puppet/modules/* $(PUPPET_DEST) +endif + diff --git a/fuel/build/f_jre_head/puppet/modules/opnfv/manifests/jre_head.pp b/fuel/build/f_jre_head/puppet/modules/opnfv/manifests/jre_head.pp new file mode 100644 index 0000000..44b649b --- /dev/null +++ b/fuel/build/f_jre_head/puppet/modules/opnfv/manifests/jre_head.pp @@ -0,0 +1,15 @@ +class opnfv::jre_head { + if $::osfamily == 'Debian' { + + case $::fuel_settings['role'] { + /controller/: { + + package { 'openjdk-8-jre-headless' : + ensure => installed, + } + + } + } + } +} + diff --git a/fuel/build/f_l23network/Makefile b/fuel/build/f_l23network/Makefile new file mode 100644 index 0000000..c311a2d --- /dev/null +++ b/fuel/build/f_l23network/Makefile @@ -0,0 +1,24 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) + +.PHONY: all +all: + +.PHONY: clean +clean: + @rm -rf tmp + @rm -rf release + + +.PHONY: release +release: + @cp -Rvp puppet/modules/* $(PUPPET_DEST) diff --git a/fuel/build/f_l23network/README b/fuel/build/f_l23network/README new file mode 100644 index 0000000..9aa4718 --- /dev/null +++ b/fuel/build/f_l23network/README @@ -0,0 +1,35 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +Addition to add entries to /etc/hosts through Astute. + +The astute.yaml file should contain entries as those below to have them picked up during deployment: + +opnfv: + hosts: + - name: test1 + address: 192.168.100.100 + fqdn: test1.opnfv.org + - name: test2 + address: 192.168.100.101 + fqdn: test2.opnfv.org + - name: test3 + address: 192.168.100.102 + fqdn: test3.opnfv.org + +The suggested method for adding this information is to prepare for deployment with the Fuel GUI or CLI, +but before actually deploying: + +1. Download the current deployment for all hosts: fuel --env 1 deployment --default +2. Iterate through the hosts in "deployment_1" and add hosts configuration in the above format to their + respective yaml file. +3. Upload the modifed deployment information: fuel --env 1 deployment --upload + +After deploying, the additions will be included in /etc/astute.yaml of each host. diff --git a/fuel/build/f_l23network/puppet/modules/l23network/lib/puppet/parser/functions/extras_to_hosts.rb b/fuel/build/f_l23network/puppet/modules/l23network/lib/puppet/parser/functions/extras_to_hosts.rb new file mode 100644 index 0000000..33bfad8 --- /dev/null +++ b/fuel/build/f_l23network/puppet/modules/l23network/lib/puppet/parser/functions/extras_to_hosts.rb @@ -0,0 +1,21 @@ +# +# array_or_string_to_array.rb +# + +module Puppet::Parser::Functions + newfunction(:extras_to_hosts, :type => :rvalue, :doc => <<-EOS + convert extras array passed from Astute into + hash for puppet `host` create_resources call + EOS + ) do |args| + hosts=Hash.new + extras=args[0] + extras.each do |extras| + hosts[extras['name']]={:ip=>extras['address'],:host_aliases=>[extras['fqdn']]} + notice("Generating extras host entry #{extras['name']} #{extras['address']} #{extras['fqdn']}") + end + return hosts + end +end + +# vim: set ts=2 sw=2 et : diff --git a/fuel/build/f_l23network/puppet/modules/l23network/manifests/hosts_file.pp b/fuel/build/f_l23network/puppet/modules/l23network/manifests/hosts_file.pp new file mode 100644 index 0000000..05cff8d --- /dev/null +++ b/fuel/build/f_l23network/puppet/modules/l23network/manifests/hosts_file.pp @@ -0,0 +1,18 @@ +class l23network::hosts_file ( + $nodes, + $extras=[], + $hosts_file = "/etc/hosts" +) { + + # OPNFV addition: Add additional lines in /etc/hosts through Astute additions + + $host_resources = nodes_to_hosts($nodes) + $extras_host_resources = extras_to_hosts($extras) + Host { + ensure => present, + target => $hosts_file + } + + create_resources(host, $host_resources) + create_resources(host, $extras_host_resources) +} diff --git a/fuel/build/f_l23network/puppet/modules/l23network/manifests/hosts_file.pp.orig b/fuel/build/f_l23network/puppet/modules/l23network/manifests/hosts_file.pp.orig new file mode 100644 index 0000000..2295e3f --- /dev/null +++ b/fuel/build/f_l23network/puppet/modules/l23network/manifests/hosts_file.pp.orig @@ -0,0 +1,16 @@ +class l23network::hosts_file ( + $nodes, + $hosts_file = "/etc/hosts" +) { + + #Move original hosts file + + $host_resources = nodes_to_hosts($nodes) + + Host { + ensure => present, + target => $hosts_file + } + + create_resources(host, $host_resources) +} diff --git a/fuel/build/f_l23network/testing/README b/fuel/build/f_l23network/testing/README new file mode 100644 index 0000000..b68eddf --- /dev/null +++ b/fuel/build/f_l23network/testing/README @@ -0,0 +1,12 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +In order to test the functionality without performing a full deployment, run "puppet apply" on the fake_init.pp +which will call only the l23network::hosts_file class. diff --git a/fuel/build/f_l23network/testing/fake_init.pp b/fuel/build/f_l23network/testing/fake_init.pp new file mode 100644 index 0000000..bc6b163 --- /dev/null +++ b/fuel/build/f_l23network/testing/fake_init.pp @@ -0,0 +1,13 @@ +$fuel_settings = parseyaml($astute_settings_yaml) + +if $::fuel_settings['nodes'] { + $nodes_hash = $::fuel_settings['nodes'] + $extras_hash = $::fuel_settings['opnfv']['hosts'] + + class {'l23network::hosts_file': + nodes => $nodes_hash, + extras => $extras_hash + } + + include l23network::hosts_file +} diff --git a/fuel/build/f_libnss/Makefile b/fuel/build/f_libnss/Makefile new file mode 100644 index 0000000..e17519a --- /dev/null +++ b/fuel/build/f_libnss/Makefile @@ -0,0 +1,42 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) +include ../config.mk + +PACKAGE := libnss3 +VERSION := 3.17.4-0ubuntu0.14.04.1_amd64 +DEB_NAME := $(PACKAGE)_$(VERSION).deb + +.PHONY: all +all: release/pool/main/$(DEB_NAME) + +release/pool/main/$(DEB_NAME): +ifeq ($(LIBNSS_MAIN_REPO),) + @echo "No config-spec target for Java, nothing to build" +else + @mkdir -p release/pool/main && \ + cd release/pool/main && \ + wget -N $(LIBNSS_MAIN_REPO)/$(DEB_NAME) + @lintian release/pool/main/$(DEB_NAME) +endif + +.PHONY: clean +clean: + @rm -rf release + @rm -f $(DEB_DEST)/$(DEB_NAME) + + +.PHONY: release +release:release/pool/main/$(DEB_NAME) +ifneq ($(LIBNSS_MAIN_REPO),) + @cp release/pool/main/$(DEB_NAME) $(DEB_DEST) + @cp -Rvp puppet/modules/* $(PUPPET_DEST) +endif diff --git a/fuel/build/f_libnss/README b/fuel/build/f_libnss/README new file mode 100644 index 0000000..776d47a --- /dev/null +++ b/fuel/build/f_libnss/README @@ -0,0 +1,44 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +This directory adds the openjdk-8-jdk Java package and related puppet +deployment manifest such that it gets built into the .iso image an deployed +on the stack controller cluster. + +The f_java has the following structure: +. ++--------+----------+-----------+ + | | | + puppet/ Makefile README + | (this file) + | + | + modules/ + | + | + | + opnfv/ + | + | + | + manifests/ + | + | + | + java.pp + +Makefile: +Invoked by the git root Makefile, it pulls the openjdk-8-jdk deb package +from a propper mirror and pushes i together with the manifests to a fuel +build artifact directory, such that it eventually gets built into the new +fuel .iso + +java.pp: +Controls the installation and configuration of java diff --git a/fuel/build/f_libnss/puppet/modules/opnfv/manifests/libnss.pp b/fuel/build/f_libnss/puppet/modules/opnfv/manifests/libnss.pp new file mode 100644 index 0000000..17cfb8d --- /dev/null +++ b/fuel/build/f_libnss/puppet/modules/opnfv/manifests/libnss.pp @@ -0,0 +1,11 @@ +class opnfv::libnss { + if $::osfamily == 'Debian' { + case $::fuel_settings['role'] { + /controller/: { + package { 'libnss3': + ensure => installed, + } + } + } + } +} diff --git a/fuel/build/f_libnss/testing/README b/fuel/build/f_libnss/testing/README new file mode 100644 index 0000000..c0ef7d6 --- /dev/null +++ b/fuel/build/f_libnss/testing/README @@ -0,0 +1,12 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +In order to test the functionality without performing a full deployment, run "puppet apply" on the +fake_init.pp which will call only the opnfv::java class. diff --git a/fuel/build/f_libnss/testing/fake_init.pp b/fuel/build/f_libnss/testing/fake_init.pp new file mode 100644 index 0000000..237b0c1 --- /dev/null +++ b/fuel/build/f_libnss/testing/fake_init.pp @@ -0,0 +1,13 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +$fuel_settings = parseyaml($astute_settings_yaml) + +include opnfv::java diff --git a/fuel/build/f_libpcs/Makefile b/fuel/build/f_libpcs/Makefile new file mode 100644 index 0000000..3e480db --- /dev/null +++ b/fuel/build/f_libpcs/Makefile @@ -0,0 +1,42 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) +include ../config.mk + +PACKAGE := libpcsclite1 +VERSION := 1.8.10-1ubuntu1_amd64 +DEB_NAME := $(PACKAGE)_$(VERSION).deb + +.PHONY: all +all: release/pool/main/$(DEB_NAME) + +release/pool/main/$(DEB_NAME): +ifeq ($(LIBPCS_MAIN_REPO),) + @echo "No config-spec target for Java, nothing to build" +else + @mkdir -p release/pool/main && \ + cd release/pool/main && \ + wget -N $(LIBPCS_MAIN_REPO)/$(DEB_NAME) + @lintian release/pool/main/$(DEB_NAME) +endif + +.PHONY: clean +clean: + @rm -rf release + @rm -f $(DEB_DEST)/$(DEB_NAME) + + +.PHONY: release +release:release/pool/main/$(DEB_NAME) +ifneq ($(LIBPCS_MAIN_REPO),) + @cp release/pool/main/$(DEB_NAME) $(DEB_DEST) + @cp -Rvp puppet/modules/* $(PUPPET_DEST) +endif diff --git a/fuel/build/f_libpcs/puppet/modules/opnfv/manifests/libpcs.pp b/fuel/build/f_libpcs/puppet/modules/opnfv/manifests/libpcs.pp new file mode 100644 index 0000000..1f26247 --- /dev/null +++ b/fuel/build/f_libpcs/puppet/modules/opnfv/manifests/libpcs.pp @@ -0,0 +1,11 @@ +class opnfv::libpcs { + if $::osfamily == 'Debian' { + case $::fuel_settings['role'] { + /controller/: { + package { 'libpcsclite1': + ensure => installed, + } + } + } + } +} diff --git a/fuel/build/f_libxt/Makefile b/fuel/build/f_libxt/Makefile new file mode 100644 index 0000000..bb5c2aa --- /dev/null +++ b/fuel/build/f_libxt/Makefile @@ -0,0 +1,42 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) +include ../config.mk + +PACKAGE := libxtst6 +VERSION := 1.2.2-1_amd64 +DEB_NAME := $(PACKAGE)_$(VERSION).deb + +.PHONY: all +all: release/pool/main/$(DEB_NAME) + +release/pool/main/$(DEB_NAME): +ifeq ($(LIBXT_MAIN_REPO),) + @echo "No config-spec target for Java, nothing to build" +else + @mkdir -p release/pool/main && \ + cd release/pool/main && \ + wget -N $(LIBXT_MAIN_REPO)/$(DEB_NAME) + @lintian release/pool/main/$(DEB_NAME) +endif + +.PHONY: clean +clean: + @rm -rf release + @rm -f $(DEB_DEST)/$(DEB_NAME) + + +.PHONY: release +release:release/pool/main/$(DEB_NAME) +ifneq ($(LIBXT_MAIN_REPO),) + @cp release/pool/main/$(DEB_NAME) $(DEB_DEST) + @cp -Rvp puppet/modules/* $(PUPPET_DEST) +endif diff --git a/fuel/build/f_libxt/puppet/modules/opnfv/manifests/libxt.pp b/fuel/build/f_libxt/puppet/modules/opnfv/manifests/libxt.pp new file mode 100644 index 0000000..a862778 --- /dev/null +++ b/fuel/build/f_libxt/puppet/modules/opnfv/manifests/libxt.pp @@ -0,0 +1,11 @@ +class opnfv::libxt { + if $::osfamily == 'Debian' { + case $::fuel_settings['role'] { + /controller/: { + package { 'libxtst6': + ensure => installed, + } + } + } + } +} diff --git a/fuel/build/f_ntp/Makefile b/fuel/build/f_ntp/Makefile new file mode 100644 index 0000000..c311a2d --- /dev/null +++ b/fuel/build/f_ntp/Makefile @@ -0,0 +1,24 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) + +.PHONY: all +all: + +.PHONY: clean +clean: + @rm -rf tmp + @rm -rf release + + +.PHONY: release +release: + @cp -Rvp puppet/modules/* $(PUPPET_DEST) diff --git a/fuel/build/f_ntp/README b/fuel/build/f_ntp/README new file mode 100644 index 0000000..2bade72 --- /dev/null +++ b/fuel/build/f_ntp/README @@ -0,0 +1,33 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +Addition to add ntp.conf separately for compute hosts and controller hosts through Astute. + +The astute.yaml file should contain entries as those below to have them picked up during deployment: + +opnfv: + ntp: + controller: | + line 1 + line 2 + compute: | + line 1 + line 2 + +The suggested method for adding this information is to prepare for deployment with the Fuel GUI or CLI, +but before actually deploying: + +1. Download the current deployment for all hosts: fuel --env 1 deployment --default +2. Iterate through the hosts in "deployment_1" and add hosts configuration in the above format to their + respective yaml file. +3. Upload the modifed deployment information: fuel --env 1 deployment --upload + +After deploying, the additions will be included in /etc/astute.yaml of each host. + diff --git a/fuel/build/f_ntp/puppet/modules/opnfv/manifests/ntp.pp b/fuel/build/f_ntp/puppet/modules/opnfv/manifests/ntp.pp new file mode 100644 index 0000000..8180e3d --- /dev/null +++ b/fuel/build/f_ntp/puppet/modules/opnfv/manifests/ntp.pp @@ -0,0 +1,71 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +# Class: Ntp +# +# Add Ntp content passed through astute.yaml into ntp.conf depending on the role +# +# Suitable yaml content: +# <begin> +# opnfv: +# ntp: +# controller: | +# line 1 +# line 2 +# compute: | +# line 1 +# line 2 +# <end> +# +# +# + +class opnfv::ntp( + $file='/etc/ntp.conf' +) { + + if $::fuel_settings['role'] { + if ($::fuel_settings['opnfv'] and + $::fuel_settings['opnfv']['ntp']) { + case $::fuel_settings['role'] { + /controller/: { + if $::fuel_settings['opnfv']['ntp']['controller'] { + $template = 'opnfv/ntp.conf.controller.erb' + $file_content = $::fuel_settings['opnfv']['ntp']['controller'] + } + } + /compute/: { + if $::fuel_settings['opnfv']['ntp']['compute'] { + $template = 'opnfv/ntp.conf.compute.erb' + $file_content = $::fuel_settings['opnfv']['ntp']['compute'] + } + } + } + } + } + + if $file_content { + package { 'ntp': + ensure => installed, + } + + file { $file: + content => template($template), + notify => Service['ntp'], + } + + service { 'ntp': + ensure => running, + enable => true, + require => [ Package['ntp'], File[$file]] + } + } +} + diff --git a/fuel/build/f_ntp/puppet/modules/opnfv/templates/ntp.conf.compute.erb b/fuel/build/f_ntp/puppet/modules/opnfv/templates/ntp.conf.compute.erb new file mode 100644 index 0000000..37ecfd7 --- /dev/null +++ b/fuel/build/f_ntp/puppet/modules/opnfv/templates/ntp.conf.compute.erb @@ -0,0 +1,21 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +tinker panic 0 +driftfile /var/lib/ntp/ntp.drift +statistics loopstats peerstats clockstats +filegen loopstats file loopstats type day enable +filegen peerstats file peerstats type day enable +filegen clockstats file clockstats type day enable +restrict -4 default kod notrap nomodify nopeer noquery +restrict -6 default kod notrap nomodify nopeer noquery +restrict 127.0.0.1 +restrict ::1 +<%= @file_content %> diff --git a/fuel/build/f_ntp/puppet/modules/opnfv/templates/ntp.conf.controller.erb b/fuel/build/f_ntp/puppet/modules/opnfv/templates/ntp.conf.controller.erb new file mode 100644 index 0000000..37ecfd7 --- /dev/null +++ b/fuel/build/f_ntp/puppet/modules/opnfv/templates/ntp.conf.controller.erb @@ -0,0 +1,21 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +tinker panic 0 +driftfile /var/lib/ntp/ntp.drift +statistics loopstats peerstats clockstats +filegen loopstats file loopstats type day enable +filegen peerstats file peerstats type day enable +filegen clockstats file clockstats type day enable +restrict -4 default kod notrap nomodify nopeer noquery +restrict -6 default kod notrap nomodify nopeer noquery +restrict 127.0.0.1 +restrict ::1 +<%= @file_content %> diff --git a/fuel/build/f_ntp/testing/README b/fuel/build/f_ntp/testing/README new file mode 100644 index 0000000..6d80b0a --- /dev/null +++ b/fuel/build/f_ntp/testing/README @@ -0,0 +1,12 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +In order to test the functionality without performing a full deployment, run "puppet apply" on the +fake_init.pp which will call only the opnfv::ntp class. diff --git a/fuel/build/f_ntp/testing/fake_init.pp b/fuel/build/f_ntp/testing/fake_init.pp new file mode 100644 index 0000000..b9af218 --- /dev/null +++ b/fuel/build/f_ntp/testing/fake_init.pp @@ -0,0 +1,13 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +$fuel_settings = parseyaml($astute_settings_yaml) + +include opnfv::ntp diff --git a/fuel/build/f_opnfv_puppet/Makefile b/fuel/build/f_opnfv_puppet/Makefile new file mode 100644 index 0000000..c311a2d --- /dev/null +++ b/fuel/build/f_opnfv_puppet/Makefile @@ -0,0 +1,24 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) + +.PHONY: all +all: + +.PHONY: clean +clean: + @rm -rf tmp + @rm -rf release + + +.PHONY: release +release: + @cp -Rvp puppet/modules/* $(PUPPET_DEST) diff --git a/fuel/build/f_opnfv_puppet/README b/fuel/build/f_opnfv_puppet/README new file mode 100644 index 0000000..35bea5a --- /dev/null +++ b/fuel/build/f_opnfv_puppet/README @@ -0,0 +1,12 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +This is the top level "OPNFV" Puppet class which (hopefully) only will be used to include +an appropriate set of sub-classes which themselves will be self-contained. diff --git a/fuel/build/f_opnfv_puppet/puppet/modules/opnfv/manifests/add_packages.pp b/fuel/build/f_opnfv_puppet/puppet/modules/opnfv/manifests/add_packages.pp new file mode 100644 index 0000000..ccb3939 --- /dev/null +++ b/fuel/build/f_opnfv_puppet/puppet/modules/opnfv/manifests/add_packages.pp @@ -0,0 +1,9 @@ +# Class: opnfv::add_packages +# +# Ensure added packages are installed: +# + +class opnfv::add_packages { + if $::osfamily == 'Debian' { + } +} diff --git a/fuel/build/f_opnfv_puppet/puppet/modules/opnfv/manifests/init.pp b/fuel/build/f_opnfv_puppet/puppet/modules/opnfv/manifests/init.pp new file mode 100644 index 0000000..d48956b --- /dev/null +++ b/fuel/build/f_opnfv_puppet/puppet/modules/opnfv/manifests/init.pp @@ -0,0 +1,39 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +# == Class: opnfv +# +# This class is used to perform OPNFV inclusions and settings on top of +# the vanilla Fuel installation. +# +# Currently all logic is self contained, i.e. it is sufficient to +# "include opnfv" from site.pp. + +class opnfv { + # Configure resolv.conf if parameters passed through astute + include opnfv::resolver + # Setup OPNFV style NTP config + include opnfv::ntp + # Make sure all added packages are installed + include opnfv::add_packages + # Setup OpenDaylight + include opnfv::odl + include opnfv::java + include opnfv::jre + include opnfv::jre_head + include opnfv::atk_wrap + include opnfv::gif + include opnfv::ca_cert + include opnfv::comm_java + include opnfv::libpcs + include opnfv::libxt + include opnfv::libnss + include opnfv::defjre +} diff --git a/fuel/build/f_opnfv_puppet/puppet/modules/opnfv/manifests/opncheck.pp b/fuel/build/f_opnfv_puppet/puppet/modules/opnfv/manifests/opncheck.pp new file mode 100644 index 0000000..0822f02 --- /dev/null +++ b/fuel/build/f_opnfv_puppet/puppet/modules/opnfv/manifests/opncheck.pp @@ -0,0 +1,21 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +# Class: opnfv::opncheck +# +# Make sure that /opt/opnfv/pre-deploy.sh has been run by +# verifying there is an "opnfv:" level in the astute.yaml. + +class opnfv::opncheck() +{ + unless $::fuel_settings['opnfv'] { + fail("Error: You have not run /opt/opnfv/pre-deploy.sh on the Fuel master prior to deploying!") + } +} diff --git a/fuel/build/f_osnaily/Makefile b/fuel/build/f_osnaily/Makefile new file mode 100644 index 0000000..c311a2d --- /dev/null +++ b/fuel/build/f_osnaily/Makefile @@ -0,0 +1,24 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) + +.PHONY: all +all: + +.PHONY: clean +clean: + @rm -rf tmp + @rm -rf release + + +.PHONY: release +release: + @cp -Rvp puppet/modules/* $(PUPPET_DEST) diff --git a/fuel/build/f_osnaily/puppet/modules/osnailyfacter/examples/site.pp b/fuel/build/f_osnaily/puppet/modules/osnailyfacter/examples/site.pp new file mode 100644 index 0000000..48c4005 --- /dev/null +++ b/fuel/build/f_osnaily/puppet/modules/osnailyfacter/examples/site.pp @@ -0,0 +1,379 @@ +$fuel_settings = parseyaml($astute_settings_yaml) + +$openstack_version = { + 'keystone' => 'installed', + 'glance' => 'installed', + 'horizon' => 'installed', + 'nova' => 'installed', + 'novncproxy' => 'installed', + 'cinder' => 'installed', +} + +tag("${::fuel_settings['deployment_id']}::${::fuel_settings['environment']}") + +#Stages configuration +stage {'zero': } -> +stage {'opncheck': } -> +stage {'first': } -> +stage {'openstack-custom-repo': } -> +stage {'netconfig': } -> +stage {'corosync_setup': } -> +stage {'openstack-firewall': } -> Stage['main'] + +class begin_deployment () +{ + $role = $::fuel_settings['role'] + notify { "***** Beginning deployment of node ${::hostname} with role $role *****": } +} + +class {'begin_deployment': stage => 'zero' } + +stage {'glance-image': + require => Stage['main'], +} + +if $::fuel_settings['nodes'] { + $nodes_hash = $::fuel_settings['nodes'] +# OPNFV addition to add to hosts file + if ($::fuel_settings['opnfv'] and + $::fuel_settings['opnfv']['hosts']) { + $extras_hash = $::fuel_settings['opnfv']['hosts'] + } else { + $extras_hash = undef + } + + $dns_nameservers=$::fuel_settings['dns_nameservers'] + $node = filter_nodes($nodes_hash,'name',$::hostname) + if empty($node) { + fail("Node $::hostname is not defined in the hash structure") + } + + $default_gateway = $node[0]['default_gateway'] + + $base_syslog_hash = $::fuel_settings['base_syslog'] + $syslog_hash = $::fuel_settings['syslog'] + + $disable_offload = $::fuel_settings['disable_offload'] + if $disable_offload { + L23network::L3::Ifconfig<||> { + ethtool => { + 'K' => ['gso off', 'gro off'], + } + } + } + + $use_neutron = $::fuel_settings['quantum'] + + if (!empty(filter_nodes($::fuel_settings['nodes'], 'role', 'ceph-osd')) or + $::fuel_settings['storage']['volumes_ceph'] or + $::fuel_settings['storage']['images_ceph'] or + $::fuel_settings['storage']['objects_ceph'] + ) { + $use_ceph = true + } else { + $use_ceph = false + } + + + if $use_neutron { + prepare_network_config($::fuel_settings['network_scheme']) + # + $internal_int = get_network_role_property('management', 'interface') + $internal_address = get_network_role_property('management', 'ipaddr') + $internal_netmask = get_network_role_property('management', 'netmask') + # + $public_int = get_network_role_property('ex', 'interface') + if $public_int { + $public_address = get_network_role_property('ex', 'ipaddr') + $public_netmask = get_network_role_property('ex', 'netmask') + + # TODO(Xarses): remove this after completing merge of + # multiple-cluster-networks + L23network::L3::Ifconfig<| title == $public_int |> { + default_gateway => true + } + } else { + # TODO(Xarses): remove this after completing merge of + # multiple-cluster-networks + $fw_admin_int = get_network_role_property('fw-admin', 'interface') + L23network::L3::Ifconfig<| title == $fw_admin_int |> { + default_gateway => true + } + } + # + $storage_address = get_network_role_property('storage', 'ipaddr') + $storage_netmask = get_network_role_property('storage', 'netmask') + } else { + $internal_address = $node[0]['internal_address'] + $internal_netmask = $node[0]['internal_netmask'] + $public_address = $node[0]['public_address'] + $public_netmask = $node[0]['public_netmask'] + $storage_address = $node[0]['storage_address'] + $storage_netmask = $node[0]['storage_netmask'] + $public_br = $node[0]['public_br'] + $internal_br = $node[0]['internal_br'] + $public_int = $::fuel_settings['public_interface'] + $internal_int = $::fuel_settings['management_interface'] + + # TODO(Xarses): remove this after completing merge of + # multiple-cluster-networks + L23network::L3::Ifconfig<| title == $public_int |> { + default_gateway => true + } + + } +} + +if ($::fuel_settings['neutron_mellanox']) { + $mellanox_mode = $::fuel_settings['neutron_mellanox']['plugin'] +} else { + $mellanox_mode = 'disabled' +} + +# This parameter specifies the verbosity level of log messages +# in openstack components config. +# Debug would have set DEBUG level and ignore verbose settings, if any. +# Verbose would have set INFO level messages +# In case of non debug and non verbose - WARNING, default level would have set. +$verbose = true +$debug = $::fuel_settings['debug'] + +### Storage Settings ### +# Determine if any ceph parts have been asked for. +# This will ensure that monitors are set up on controllers, even if no +# ceph-osd roles during deployment + + +### Syslog ### +#TODO(bogdando) move logging options to astute.yaml +# Enable error messages reporting to rsyslog. Rsyslog must be installed in this case. +$use_syslog = $::fuel_settings['use_syslog'] ? { default=>true } +# Syslog facilities for main openstack services +# should vary (reserved usage) +# local1 is reserved for openstack-dashboard +$syslog_log_facility_glance = 'LOG_LOCAL2' +$syslog_log_facility_cinder = 'LOG_LOCAL3' +$syslog_log_facility_neutron = 'LOG_LOCAL4' +$syslog_log_facility_nova = 'LOG_LOCAL6' +$syslog_log_facility_keystone = 'LOG_LOCAL7' +# could be the same +# local0 is free for use +$syslog_log_facility_murano = 'LOG_LOCAL0' +$syslog_log_facility_heat = 'LOG_LOCAL0' +$syslog_log_facility_sahara = 'LOG_LOCAL0' +$syslog_log_facility_ceilometer = 'LOG_LOCAL0' +$syslog_log_facility_ceph = 'LOG_LOCAL0' + +### Monit ### +# Monit for compute nodes. +# If enabled, will install monit and configure its watchdogs to track +# nova-compute/api/network (and openvswitch service, if neutron enabled) +# at compute nodes. +# TODO(bogdando) set to true once monit package shipped with Fuel ISO +$use_monit = false + +$nova_rate_limits = { + 'POST' => 100000, + 'POST_SERVERS' => 100000, + 'PUT' => 1000, 'GET' => 100000, + 'DELETE' => 100000 +} +$cinder_rate_limits = { + 'POST' => 100000, + 'POST_SERVERS' => 100000, + 'PUT' => 100000, 'GET' => 100000, + 'DELETE' => 100000 +} + +### +class advanced_node_netconfig { + $sdn = generate_network_config() + notify {"SDN: ${sdn}": } +} + +case $::operatingsystem { + 'redhat' : { + $queue_provider = 'qpid' + $custom_mysql_setup_class = 'pacemaker_mysql' + } + default: { + $queue_provider='rabbitmq' + $custom_mysql_setup_class='galera' + } +} + +class os_common { + # OPNFV check if pre_deploy.sh has been run, otherwise fail + class {'opnfv::opncheck': stage => 'opncheck' } + if ($::fuel_settings['neutron_mellanox']) { + if ($::mellanox_mode != 'disabled') { + class { 'mellanox_openstack::ofed_recompile' : + stage => 'zero', + } + } + if ($::fuel_settings['storage']['iser']) { + class { 'mellanox_openstack::iser_rename': + stage => 'zero', + storage_parent => $::fuel_settings['neutron_mellanox']['storage_parent'], + iser_interface_name => $::fuel_settings['neutron_mellanox']['iser_interface_name'], + } + Class['mellanox_openstack::ofed_recompile'] -> Class['mellanox_openstack::iser_rename'] + } + } + class {"l23network::hosts_file": stage => 'netconfig', nodes => $nodes_hash, extras => $extras_hash } + class {'l23network': use_ovs=>$use_neutron, stage=> 'netconfig'} + if $use_neutron { + class {'advanced_node_netconfig': stage => 'netconfig' } + } else { + class {'osnailyfacter::network_setup': stage => 'netconfig'} + } + + if ($::osfamily == 'RedHat') { + package {'irqbalance': ensure => present} -> service {'irqbalance': ensure => running } + } + + class { 'openstack::firewall': + stage => 'openstack-firewall', + nova_vnc_ip_range => $::fuel_settings['management_network_range'], + } + + $base_syslog_rserver = { + 'remote_type' => 'tcp', + 'server' => $base_syslog_hash['syslog_server'], + 'port' => $base_syslog_hash['syslog_port'] + } + +### TCP connections keepalives and failover related parameters ### + # configure TCP keepalive for host OS. + # Send 3 probes each 8 seconds, if the connection was idle + # for a 30 seconds. Consider it dead, if there was no responces + # during the check time frame, i.e. 30+3*8=54 seconds overall. + # (note: overall check time frame should be lower then + # nova_report_interval). + class { 'openstack::keepalive' : + stage => 'netconfig', + tcpka_time => '30', + tcpka_probes => '8', + tcpka_intvl => '3', + tcp_retries2 => '5', + } + + # setting kernel reserved ports + # defaults are 49000,35357,41055,58882 + class { 'openstack::reserved_ports': + stage => 'netconfig', + } + + # setting service down time and report interval + # to 60 and 180 for Nova respectively to allow kernel + # to kill dead connections + # (see zendesk #1158 as well) + $nova_report_interval = '60' + $nova_service_down_time = '180' + + $syslog_rserver = { + 'remote_type' => $syslog_hash['syslog_transport'], + 'server' => $syslog_hash['syslog_server'], + 'port' => $syslog_hash['syslog_port'], + } + if $syslog_hash['syslog_server'] != "" and $syslog_hash['syslog_port'] != "" and $syslog_hash['syslog_transport'] != "" { + $rservers = [$base_syslog_rserver, $syslog_rserver] + } else { + $rservers = [$base_syslog_rserver] + } + + if $use_syslog { + class { "::openstack::logging": + stage => 'first', + role => 'client', + show_timezone => true, + # log both locally include auth, and remote + log_remote => true, + log_local => true, + log_auth_local => true, + # keep four weekly log rotations, force rotate if 300M size have exceeded + rotation => 'weekly', + keep => '4', + # should be > 30M + limitsize => '300M', + # remote servers to send logs to + rservers => $rservers, + # should be true, if client is running at virtual node + virtual => str2bool($::is_virtual), + # Rabbit doesn't support syslog directly + rabbit_log_level => 'NOTICE', + debug => $debug, + } + } + + class { 'osnailyfacter::atop': + stage => 'first', + } + + #case $role { + # /controller/: { $hostgroup = 'controller' } + # /swift-proxy/: { $hostgroup = 'swift-proxy' } + # /storage/:{ $hostgroup = 'swift-storage' } + # /compute/: { $hostgroup = 'compute' } + # /cinder/: { $hostgroup = 'cinder' } + # default: { $hostgroup = 'generic' } + #} + + # if $nagios != 'false' { + # class {'nagios': + # proj_name => $proj_name, + # services => [ + # 'host-alive','nova-novncproxy','keystone', 'nova-scheduler', + # 'nova-consoleauth', 'nova-cert', 'haproxy', 'nova-api', 'glance-api', + # 'glance-registry','horizon', 'rabbitmq', 'mysql', + # ], + # whitelist => ['127.0.0.1', $nagios_master], + # hostgroup => $hostgroup , + # } + # } + + # Workaround for fuel bug with firewall + firewall {'003 remote rabbitmq ': + sport => [ 4369, 5672, 15672, 41055, 55672, 61613 ], + source => $::fuel_settings['master_ip'], + proto => 'tcp', + action => 'accept', + require => Class['openstack::firewall'], + } + + firewall {'004 remote puppet ': + sport => [ 8140 ], + source => $master_ip, + proto => 'tcp', + action => 'accept', + require => Class['openstack::firewall'], + } + + class { 'puppet::pull' : + modules_source => $::fuel_settings['puppet_modules_source'], + manifests_source => $::fuel_settings['puppet_manifests_source'], + } +} # OS_COMMON ENDS + + + +node default { + case $::fuel_settings['deployment_mode'] { + "singlenode": { + include "osnailyfacter::cluster_simple" + class {'os_common':} + class {'opnfv':} + } + "multinode": { + include "osnailyfacter::cluster_simple" + class {'os_common':} + class {'opnfv':} + } + /^(ha|ha_compact)$/: { + include "osnailyfacter::cluster_ha" + class {'os_common':} + class {'opnfv':} + } + "rpmcache": { include osnailyfacter::rpmcache } + } +} diff --git a/fuel/build/f_osnaily/puppet/modules/osnailyfacter/examples/site.pp.orig b/fuel/build/f_osnaily/puppet/modules/osnailyfacter/examples/site.pp.orig new file mode 100644 index 0000000..940beda --- /dev/null +++ b/fuel/build/f_osnaily/puppet/modules/osnailyfacter/examples/site.pp.orig @@ -0,0 +1,366 @@ +$fuel_settings = parseyaml($astute_settings_yaml) + +$openstack_version = { + 'keystone' => 'installed', + 'glance' => 'installed', + 'horizon' => 'installed', + 'nova' => 'installed', + 'novncproxy' => 'installed', + 'cinder' => 'installed', +} + +tag("${::fuel_settings['deployment_id']}::${::fuel_settings['environment']}") + +#Stages configuration +stage {'zero': } -> +stage {'first': } -> +stage {'openstack-custom-repo': } -> +stage {'netconfig': } -> +stage {'corosync_setup': } -> +stage {'openstack-firewall': } -> Stage['main'] + +class begin_deployment () +{ + $role = $::fuel_settings['role'] + notify { "***** Beginning deployment of node ${::hostname} with role $role *****": } +} + +class {'begin_deployment': stage => 'zero' } + +stage {'glance-image': + require => Stage['main'], +} + +if $::fuel_settings['nodes'] { + $nodes_hash = $::fuel_settings['nodes'] + $dns_nameservers=$::fuel_settings['dns_nameservers'] + $node = filter_nodes($nodes_hash,'name',$::hostname) + if empty($node) { + fail("Node $::hostname is not defined in the hash structure") + } + + $default_gateway = $node[0]['default_gateway'] + + $base_syslog_hash = $::fuel_settings['base_syslog'] + $syslog_hash = $::fuel_settings['syslog'] + + $disable_offload = $::fuel_settings['disable_offload'] + if $disable_offload { + L23network::L3::Ifconfig<||> { + ethtool => { + 'K' => ['gso off', 'gro off'], + } + } + } + + $use_neutron = $::fuel_settings['quantum'] + + if (!empty(filter_nodes($::fuel_settings['nodes'], 'role', 'ceph-osd')) or + $::fuel_settings['storage']['volumes_ceph'] or + $::fuel_settings['storage']['images_ceph'] or + $::fuel_settings['storage']['objects_ceph'] + ) { + $use_ceph = true + } else { + $use_ceph = false + } + + + if $use_neutron { + prepare_network_config($::fuel_settings['network_scheme']) + # + $internal_int = get_network_role_property('management', 'interface') + $internal_address = get_network_role_property('management', 'ipaddr') + $internal_netmask = get_network_role_property('management', 'netmask') + # + $public_int = get_network_role_property('ex', 'interface') + if $public_int { + $public_address = get_network_role_property('ex', 'ipaddr') + $public_netmask = get_network_role_property('ex', 'netmask') + + # TODO(Xarses): remove this after completing merge of + # multiple-cluster-networks + L23network::L3::Ifconfig<| title == $public_int |> { + default_gateway => true + } + } else { + # TODO(Xarses): remove this after completing merge of + # multiple-cluster-networks + $fw_admin_int = get_network_role_property('fw-admin', 'interface') + L23network::L3::Ifconfig<| title == $fw_admin_int |> { + default_gateway => true + } + } + # + $storage_address = get_network_role_property('storage', 'ipaddr') + $storage_netmask = get_network_role_property('storage', 'netmask') + } else { + $internal_address = $node[0]['internal_address'] + $internal_netmask = $node[0]['internal_netmask'] + $public_address = $node[0]['public_address'] + $public_netmask = $node[0]['public_netmask'] + $storage_address = $node[0]['storage_address'] + $storage_netmask = $node[0]['storage_netmask'] + $public_br = $node[0]['public_br'] + $internal_br = $node[0]['internal_br'] + $public_int = $::fuel_settings['public_interface'] + $internal_int = $::fuel_settings['management_interface'] + + # TODO(Xarses): remove this after completing merge of + # multiple-cluster-networks + L23network::L3::Ifconfig<| title == $public_int |> { + default_gateway => true + } + + } +} + +if ($::fuel_settings['neutron_mellanox']) { + $mellanox_mode = $::fuel_settings['neutron_mellanox']['plugin'] +} else { + $mellanox_mode = 'disabled' +} + +# This parameter specifies the verbosity level of log messages +# in openstack components config. +# Debug would have set DEBUG level and ignore verbose settings, if any. +# Verbose would have set INFO level messages +# In case of non debug and non verbose - WARNING, default level would have set. +$verbose = true +$debug = $::fuel_settings['debug'] + +### Storage Settings ### +# Determine if any ceph parts have been asked for. +# This will ensure that monitors are set up on controllers, even if no +# ceph-osd roles during deployment + + +### Syslog ### +#TODO(bogdando) move logging options to astute.yaml +# Enable error messages reporting to rsyslog. Rsyslog must be installed in this case. +$use_syslog = $::fuel_settings['use_syslog'] ? { default=>true } +# Syslog facilities for main openstack services +# should vary (reserved usage) +# local1 is reserved for openstack-dashboard +$syslog_log_facility_glance = 'LOG_LOCAL2' +$syslog_log_facility_cinder = 'LOG_LOCAL3' +$syslog_log_facility_neutron = 'LOG_LOCAL4' +$syslog_log_facility_nova = 'LOG_LOCAL6' +$syslog_log_facility_keystone = 'LOG_LOCAL7' +# could be the same +# local0 is free for use +$syslog_log_facility_murano = 'LOG_LOCAL0' +$syslog_log_facility_heat = 'LOG_LOCAL0' +$syslog_log_facility_sahara = 'LOG_LOCAL0' +$syslog_log_facility_ceilometer = 'LOG_LOCAL0' +$syslog_log_facility_ceph = 'LOG_LOCAL0' + +### Monit ### +# Monit for compute nodes. +# If enabled, will install monit and configure its watchdogs to track +# nova-compute/api/network (and openvswitch service, if neutron enabled) +# at compute nodes. +# TODO(bogdando) set to true once monit package shipped with Fuel ISO +$use_monit = false + +$nova_rate_limits = { + 'POST' => 100000, + 'POST_SERVERS' => 100000, + 'PUT' => 1000, 'GET' => 100000, + 'DELETE' => 100000 +} +$cinder_rate_limits = { + 'POST' => 100000, + 'POST_SERVERS' => 100000, + 'PUT' => 100000, 'GET' => 100000, + 'DELETE' => 100000 +} + +### +class advanced_node_netconfig { + $sdn = generate_network_config() + notify {"SDN: ${sdn}": } +} + +case $::operatingsystem { + 'redhat' : { + $queue_provider = 'qpid' + $custom_mysql_setup_class = 'pacemaker_mysql' + } + default: { + $queue_provider='rabbitmq' + $custom_mysql_setup_class='galera' + } +} + +class os_common { + if ($::fuel_settings['neutron_mellanox']) { + if ($::mellanox_mode != 'disabled') { + class { 'mellanox_openstack::ofed_recompile' : + stage => 'zero', + } + } + if ($::fuel_settings['storage']['iser']) { + class { 'mellanox_openstack::iser_rename': + stage => 'zero', + storage_parent => $::fuel_settings['neutron_mellanox']['storage_parent'], + iser_interface_name => $::fuel_settings['neutron_mellanox']['iser_interface_name'], + } + Class['mellanox_openstack::ofed_recompile'] -> Class['mellanox_openstack::iser_rename'] + } + } + + class {"l23network::hosts_file": stage => 'netconfig', nodes => $nodes_hash } + class {'l23network': use_ovs=>$use_neutron, stage=> 'netconfig'} + if $use_neutron { + class {'advanced_node_netconfig': stage => 'netconfig' } + } else { + class {'osnailyfacter::network_setup': stage => 'netconfig'} + } + + if ($::osfamily == 'RedHat') { + package {'irqbalance': ensure => present} -> service {'irqbalance': ensure => running } + } + + class { 'openstack::firewall': + stage => 'openstack-firewall', + nova_vnc_ip_range => $::fuel_settings['management_network_range'], + } + + $base_syslog_rserver = { + 'remote_type' => 'tcp', + 'server' => $base_syslog_hash['syslog_server'], + 'port' => $base_syslog_hash['syslog_port'] + } + +### TCP connections keepalives and failover related parameters ### + # configure TCP keepalive for host OS. + # Send 3 probes each 8 seconds, if the connection was idle + # for a 30 seconds. Consider it dead, if there was no responces + # during the check time frame, i.e. 30+3*8=54 seconds overall. + # (note: overall check time frame should be lower then + # nova_report_interval). + class { 'openstack::keepalive' : + stage => 'netconfig', + tcpka_time => '30', + tcpka_probes => '8', + tcpka_intvl => '3', + tcp_retries2 => '5', + } + + # setting kernel reserved ports + # defaults are 49000,35357,41055,58882 + class { 'openstack::reserved_ports': + stage => 'netconfig', + } + + # setting service down time and report interval + # to 60 and 180 for Nova respectively to allow kernel + # to kill dead connections + # (see zendesk #1158 as well) + $nova_report_interval = '60' + $nova_service_down_time = '180' + + $syslog_rserver = { + 'remote_type' => $syslog_hash['syslog_transport'], + 'server' => $syslog_hash['syslog_server'], + 'port' => $syslog_hash['syslog_port'], + } + if $syslog_hash['syslog_server'] != "" and $syslog_hash['syslog_port'] != "" and $syslog_hash['syslog_transport'] != "" { + $rservers = [$base_syslog_rserver, $syslog_rserver] + } else { + $rservers = [$base_syslog_rserver] + } + + if $use_syslog { + class { "::openstack::logging": + stage => 'first', + role => 'client', + show_timezone => true, + # log both locally include auth, and remote + log_remote => true, + log_local => true, + log_auth_local => true, + # keep four weekly log rotations, force rotate if 300M size have exceeded + rotation => 'weekly', + keep => '4', + # should be > 30M + limitsize => '300M', + # remote servers to send logs to + rservers => $rservers, + # should be true, if client is running at virtual node + virtual => str2bool($::is_virtual), + # Rabbit doesn't support syslog directly + rabbit_log_level => 'NOTICE', + debug => $debug, + } + } + + class { 'osnailyfacter::atop': + stage => 'first', + } + + #case $role { + # /controller/: { $hostgroup = 'controller' } + # /swift-proxy/: { $hostgroup = 'swift-proxy' } + # /storage/:{ $hostgroup = 'swift-storage' } + # /compute/: { $hostgroup = 'compute' } + # /cinder/: { $hostgroup = 'cinder' } + # default: { $hostgroup = 'generic' } + #} + + # if $nagios != 'false' { + # class {'nagios': + # proj_name => $proj_name, + # services => [ + # 'host-alive','nova-novncproxy','keystone', 'nova-scheduler', + # 'nova-consoleauth', 'nova-cert', 'haproxy', 'nova-api', 'glance-api', + # 'glance-registry','horizon', 'rabbitmq', 'mysql', + # ], + # whitelist => ['127.0.0.1', $nagios_master], + # hostgroup => $hostgroup , + # } + # } + + # Workaround for fuel bug with firewall + firewall {'003 remote rabbitmq ': + sport => [ 4369, 5672, 15672, 41055, 55672, 61613 ], + source => $::fuel_settings['master_ip'], + proto => 'tcp', + action => 'accept', + require => Class['openstack::firewall'], + } + + firewall {'004 remote puppet ': + sport => [ 8140 ], + source => $master_ip, + proto => 'tcp', + action => 'accept', + require => Class['openstack::firewall'], + } + + class { 'puppet::pull' : + modules_source => $::fuel_settings['puppet_modules_source'], + manifests_source => $::fuel_settings['puppet_manifests_source'], + } +} # OS_COMMON ENDS + + + +node default { + case $::fuel_settings['deployment_mode'] { + "singlenode": { + include "osnailyfacter::cluster_simple" + class {'os_common':} + } + "multinode": { + include "osnailyfacter::cluster_simple" + class {'os_common':} + } + /^(ha|ha_compact)$/: { + include "osnailyfacter::cluster_ha" + class {'os_common':} + } + "rpmcache": { include osnailyfacter::rpmcache } + } +} diff --git a/fuel/build/f_resolvconf/Makefile b/fuel/build/f_resolvconf/Makefile new file mode 100644 index 0000000..c311a2d --- /dev/null +++ b/fuel/build/f_resolvconf/Makefile @@ -0,0 +1,24 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) + +.PHONY: all +all: + +.PHONY: clean +clean: + @rm -rf tmp + @rm -rf release + + +.PHONY: release +release: + @cp -Rvp puppet/modules/* $(PUPPET_DEST) diff --git a/fuel/build/f_resolvconf/README b/fuel/build/f_resolvconf/README new file mode 100644 index 0000000..5ff570f --- /dev/null +++ b/fuel/build/f_resolvconf/README @@ -0,0 +1,36 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +Addition to generate resolv.conf separately for compute hosts and controller +hosts through Astute. + +The astute.yaml file should contain entries as those below to have them picked +up during deployment: + +opnfv: + dns: + compute: + - 100.100.100.2 + - 100.100.100.3 + controller: + - 100.100.100.102 + - 100.100.100.104 + +The suggested method for adding this information is to prepare for deployment +with the Fuel GUI or CLI, but before actually deploying: + +1. Download the current deployment for all hosts: fuel --env 1 deployment --default +2. Iterate through the hosts in "deployment_1" and add hosts configuration in + the above format to their respective yaml file. +3. Upload the modifed deployment information: fuel --env 1 deployment --upload + +After deploying, the additions will be included in /etc/astute.yaml of each +host. + diff --git a/fuel/build/f_resolvconf/puppet/modules/opnfv/manifests/resolver.pp b/fuel/build/f_resolvconf/puppet/modules/opnfv/manifests/resolver.pp new file mode 100644 index 0000000..be4e67d --- /dev/null +++ b/fuel/build/f_resolvconf/puppet/modules/opnfv/manifests/resolver.pp @@ -0,0 +1,63 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +# Class: opnfv::resolver +# +# Add resolver content passed through astute.yaml into resolv.conf +# depending on the role +# +# Suitable yaml content: +# <begin> +# opnfv: +# dns: +# compute: +# - 100.100.100.2 +# - 100.100.100.3 +# controller: +# - 100.100.100.102 +# - 100.100.100.104 +# <end> +# +# +# + +class opnfv::resolver() +{ + if $::fuel_settings['role'] { + if $::fuel_settings['role'] == 'primary-controller' { + $role = 'controller' + } else { + $role = $::fuel_settings['role'] + } + + if ($::fuel_settings['opnfv'] + and $::fuel_settings['opnfv']['dns'] + and $::fuel_settings['opnfv']['dns'][$role]) { + $nameservers=$::fuel_settings['opnfv']['dns'][$role] + + file { '/etc/resolv.conf': + owner => root, + group => root, + mode => '0644', + content => template('opnfv/resolv.conf.erb'), + } +# /etc/resolv.conf is re-generated at each boot by resolvconf, so we +# need to store there as well. + file { '/etc/resolvconf/resolv.conf.d/head': + owner => root, + group => root, + mode => '0644', + content => template('opnfv/resolv.conf.erb'), + } + } + } +} + + diff --git a/fuel/build/f_resolvconf/puppet/modules/opnfv/templates/resolv.conf.erb b/fuel/build/f_resolvconf/puppet/modules/opnfv/templates/resolv.conf.erb new file mode 100644 index 0000000..7a29dca --- /dev/null +++ b/fuel/build/f_resolvconf/puppet/modules/opnfv/templates/resolv.conf.erb @@ -0,0 +1,15 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8) +# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN +# Modified by OPNFV. +<% @nameservers.each do |ns| %>nameserver <%= ns %> +<% end -%> diff --git a/fuel/build/f_resolvconf/testing/README b/fuel/build/f_resolvconf/testing/README new file mode 100644 index 0000000..6846a8d --- /dev/null +++ b/fuel/build/f_resolvconf/testing/README @@ -0,0 +1,13 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +In order to test the functionality without performing a full deployment, +run "puppet apply" on the fake_init.pp which will call only the +opnfv::resolvconf class. diff --git a/fuel/build/f_resolvconf/testing/fake_init.pp b/fuel/build/f_resolvconf/testing/fake_init.pp new file mode 100644 index 0000000..496dcd2 --- /dev/null +++ b/fuel/build/f_resolvconf/testing/fake_init.pp @@ -0,0 +1,13 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +$fuel_settings = parseyaml($astute_settings_yaml) + +include opnfv::resolvconf diff --git a/fuel/build/fuel-main_1.patch b/fuel/build/fuel-main_1.patch new file mode 100644 index 0000000..24b25b2 --- /dev/null +++ b/fuel/build/fuel-main_1.patch @@ -0,0 +1,104 @@ +diff --git a/docker/astute/Dockerfile b/docker/astute/Dockerfile +index 55f617a..cd05f19 100644 +--- a/docker/astute/Dockerfile ++++ b/docker/astute/Dockerfile +@@ -5,7 +5,7 @@ + FROM fuel/centos + MAINTAINER Matthew Mosesohn mmosesohn@mirantis.com + +-RUN rm -rf /etc/yum.repos.d/*; echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo; yum clean all; yum --quiet install -y ruby21-nailgun-mcagents sysstat ++RUN sleep 15; rm -rf /etc/yum.repos.d/*; echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo; yum clean all; yum --quiet install -y ruby21-nailgun-mcagents sysstat + + ADD etc /etc + ADD start.sh /usr/local/bin/start.sh +diff --git a/docker/cobbler/Dockerfile b/docker/cobbler/Dockerfile +index 0c80abd..3a3d966 100644 +--- a/docker/cobbler/Dockerfile ++++ b/docker/cobbler/Dockerfile +@@ -5,7 +5,7 @@ + FROM fuel/centos + MAINTAINER Matthew Mosesohn mmosesohn@mirantis.com + +-RUN rm -rf /etc/yum.repos.d/*;echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo; yum clean all; yum --quiet install -y httpd cobbler dnsmasq xinetd tftp-server; ln -s /etc/dnsmasq.conf /etc/cobbler.dnsmasq.conf ++RUN sleep 15; rm -rf /etc/yum.repos.d/*;echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo; yum clean all; yum --quiet install -y httpd cobbler dnsmasq xinetd tftp-server; ln -s /etc/dnsmasq.conf /etc/cobbler.dnsmasq.conf + + ADD etc /etc + ADD start.sh /usr/local/bin/start.sh +diff --git a/docker/mcollective/Dockerfile b/docker/mcollective/Dockerfile +index e70e87d..d6554b7 100644 +--- a/docker/mcollective/Dockerfile ++++ b/docker/mcollective/Dockerfile +@@ -4,7 +4,7 @@ MAINTAINER Aleksandr Didenko adidenko@mirantis.com + + WORKDIR /root + +-RUN rm -rf /etc/yum.repos.d/*; echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo; yum clean all; yum --quiet install -y sudo ruby21-mcollective ++RUN sleep 15; rm -rf /etc/yum.repos.d/*; echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo; yum clean all; yum --quiet install -y sudo ruby21-mcollective + + ADD etc /etc + ADD start.sh /usr/local/bin/start.sh +diff --git a/docker/ostf/Dockerfile b/docker/ostf/Dockerfile +index 43f911e..8da9108 100644 +--- a/docker/ostf/Dockerfile ++++ b/docker/ostf/Dockerfile +@@ -5,7 +5,7 @@ + FROM fuel/centos + MAINTAINER Matthew Mosesohn mmosesohn@mirantis.com + +-RUN rm -rf /etc/yum.repos.d/*;echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo;yum clean all; yum --quiet install -y python-fuelclient supervisor postgresql-libs ++RUN sleep 15; rm -rf /etc/yum.repos.d/*;echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo;yum clean all; yum --quiet install -y python-fuelclient supervisor postgresql-libs + + ADD etc /etc + ADD start.sh /usr/local/bin/start.sh +diff --git a/docker/postgres/Dockerfile b/docker/postgres/Dockerfile +index b2930db..63cc4c2 100644 +--- a/docker/postgres/Dockerfile ++++ b/docker/postgres/Dockerfile +@@ -3,7 +3,7 @@ FROM fuel/centos + + MAINTAINER Aleksandr Didenko adidenko@mirantis.com + +-RUN rm -rf /etc/yum.repos.d/*; echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo;yum clean all;yum --quiet install -y sudo ++RUN sleep 15; rm -rf /etc/yum.repos.d/*; echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo;yum clean all;yum --quiet install -y sudo + + ADD etc /etc + ADD start.sh /usr/local/bin/start.sh +diff --git a/docker/rabbitmq/Dockerfile b/docker/rabbitmq/Dockerfile +index 201648f..4f3b67c 100644 +--- a/docker/rabbitmq/Dockerfile ++++ b/docker/rabbitmq/Dockerfile +@@ -3,7 +3,7 @@ + FROM fuel/centos + MAINTAINER Aleksandr Didenko adidenko@mirantis.com + +-RUN rm -rf /etc/yum.repos.d/*; echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo; yum clean all; yum --quiet install -y rabbitmq-server ++RUN sleep 15; rm -rf /etc/yum.repos.d/*; echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo; yum clean all; yum --quiet install -y rabbitmq-server + + ADD etc /etc + ADD start.sh /usr/local/bin/start.sh +diff --git a/docker/rsync/Dockerfile b/docker/rsync/Dockerfile +index ef737bd..b6eefd1 100644 +--- a/docker/rsync/Dockerfile ++++ b/docker/rsync/Dockerfile +@@ -5,7 +5,7 @@ + FROM fuel/centos + MAINTAINER Matthew Mosesohn mmosesohn@mirantis.com + +-RUN rm -rf /etc/yum.repos.d/*; echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo;yum clean all;yum --quiet install -y ruby21-puppet xinetd rsync logrotate ++RUN sleep 15; rm -rf /etc/yum.repos.d/*; echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo;yum clean all;yum --quiet install -y ruby21-puppet xinetd rsync logrotate + + ADD etc /etc + ADD start.sh /usr/local/bin/start.sh +diff --git a/docker/rsyslog/Dockerfile b/docker/rsyslog/Dockerfile +index 5efd623..8721b39 100644 +--- a/docker/rsyslog/Dockerfile ++++ b/docker/rsyslog/Dockerfile +@@ -2,7 +2,7 @@ FROM fuel/centos + + MAINTAINER Aleksandr Didenko adidenko@mirantis.com + +-RUN rm -rf /etc/yum.repos.d/*; echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo; yum clean all;yum --quiet install -y anacron rsyslog ++RUN sleep 15; rm -rf /etc/yum.repos.d/*; echo -e "[nailgun]\nname=Nailgun Local Repo\nbaseurl=http://$(route -n | awk '/^0.0.0.0/ { print $2 }'):_PORT_/os/x86_64/\ngpgcheck=0" > /etc/yum.repos.d/nailgun.repo; yum clean all;yum --quiet install -y anacron rsyslog + + ADD etc /etc + ADD start.sh /usr/local/bin/start.sh diff --git a/fuel/build/fuel-main_2.patch b/fuel/build/fuel-main_2.patch new file mode 100644 index 0000000..72588cb --- /dev/null +++ b/fuel/build/fuel-main_2.patch @@ -0,0 +1,18 @@ +*** fuel-main/sandbox.mk.orig 2015-02-13 12:12:55.362989171 +0100 +--- fuel-main/sandbox.mk 2015-02-13 14:50:39.103017653 +0100 +*************** +*** 71,77 **** +--- 71,83 ---- + echo "Updating apt package database" + sudo chroot $(SANDBOX_UBUNTU) apt-get update + echo "Installing additional packages: $(SANDBOX_DEB_PKGS)" ++ test -e $(SANDBOX_UBUNTU)/sbin/start.orig || mv $(SANDBOX_UBUNTU)/sbin/start $(SANDBOX_UBUNTU)/sbin/start.orig ++ echo "#!/bin/sh" > $(SANDBOX_UBUNTU)/sbin/start ++ echo "exit 0" >> $(SANDBOX_UBUNTU)/sbin/start ++ chmod 755 $(SANDBOX_UBUNTU)/sbin/start + test -n "$(SANDBOX_DEB_PKGS)" && sudo chroot $(SANDBOX_UBUNTU) apt-get install --yes $(SANDBOX_DEB_PKGS) ++ test -e $(SANDBOX_UBUNTU)/sbin/start.orig && (cp $(SANDBOX_UBUNTU)/sbin/start.orig $(SANDBOX_UBUNTU)/sbin/start; \ ++ rm $(SANDBOX_UBUNTU)/sbin/start.orig) + echo "SANDBOX_UBUNTU_UP: done" + endef + diff --git a/fuel/build/install/apt-ftparchive-deb.conf b/fuel/build/install/apt-ftparchive-deb.conf new file mode 100644 index 0000000..1101ac9 --- /dev/null +++ b/fuel/build/install/apt-ftparchive-deb.conf @@ -0,0 +1,35 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +Dir { + ArchiveDir "."; +}; + +TreeDefault { + Directory "pool"; +}; + +BinDirectory "pool/main" { + Packages "dists/precise/main/binary-amd64/Packages"; + BinOverride "./indices/override.precise.main"; + ExtraOverride "./indices/override.precise.extra.main"; +}; + +Default { + Packages { + Extensions ".deb"; + Compress ". gzip"; + }; +}; + +Contents { + Compress "gzip"; +}; + diff --git a/fuel/build/install/apt-ftparchive-release.conf b/fuel/build/install/apt-ftparchive-release.conf new file mode 100644 index 0000000..0252882 --- /dev/null +++ b/fuel/build/install/apt-ftparchive-release.conf @@ -0,0 +1,18 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +APT::FTPArchive::Release::Origin "Ubuntu"; +APT::FTPArchive::Release::Label "Ubuntu"; +APT::FTPArchive::Release::Suite "precise"; +APT::FTPArchive::Release::Version "12.04"; +APT::FTPArchive::Release::Codename "precise"; +APT::FTPArchive::Release::Architectures "amd64"; +APT::FTPArchive::Release::Components "main"; +APT::FTPArchive::Release::Description "Ubuntu Precise 12.04 LTS"; diff --git a/fuel/build/install/install.sh b/fuel/build/install/install.sh new file mode 100755 index 0000000..823bcd9 --- /dev/null +++ b/fuel/build/install/install.sh @@ -0,0 +1,447 @@ +#!/bin/bash -e +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + + +my_exit() { + rc=$? + + cd /tmp + + if [ -n "$TMP_HOSTMOUNT" ]; then + if [ -d "$TMP_HOSTMOUNT" ]; then + fusermount -u $TMP_HOSTMOUNT + rmdir $TMP_HOSTMOUNT + fi + fi + + if [ -d "$TMP_OLDISO" ]; then + fusermount -u $TMP_OLDISO + rmdir $TMP_OLDISO + fi + + if [ -f "$CONF" ]; then + rm $CONF + fi + + if [ -d "$TMP_ISOPUPPET" ]; then + rm -Rf $TMP_ISOPUPPET + fi +} + +get_deb_name() { + ar p $1 control.tar.gz | tar xzO ./control | grep "^Package:.* " | sed 's/.* //' + if [ $PIPESTATUS -ne 0 ]; then + echo "Error in get_deb_name($1)" + exit 1 + fi +} + +get_deb_rev() { + ar p $1 control.tar.gz | tar xzO ./control | grep "^Version:.* " | sed 's/.* //' + if [ $PIPESTATUS -ne 0 ]; then + echo "Error in get_deb_rev($1)" + exit 1 + fi +} + + +# Added logic for ".orig" files: +# 1. Is there an .orig file for the new file? +# 2. If the file is not present in base ISO -> Error! +# 3. If the file is changed i base ISO -> Error! (need manual rebase) +# 4. If there is no .orig file, but file is present in base ISO: Error! +verify_orig_files() { + OLDDIR=$1 + NEWDIR=$2 + + pushd $NEWDIR >/dev/null + for f in $(find * -type f -name '*.orig') + do + origfile=$NEWDIR/$f + oldfile=$OLDDIR/$(echo $f | sed 's/.orig$//') + newfile=$NEWDIR/$(echo $f | sed 's/.orig$//') + + origfile=${newfile}.orig + # If no corresponding base file, error + if [ ! -f $oldfile ]; then + printf "\n\n\n\n" + echo "Error: found ${newfile} but no" + echo "Error: ${oldfile}" + echo "Error: Manual rebase is needed!" + printf "\n\n\n\n" + exit 1 + fi + + # If orig file differs from base file, error + if ! diff -q $origfile $oldfile > /dev/null; then + printf "\n\n\n\n" + echo "Error: $origfile differs from" + echo "Error: $oldfile!" + echo "Error: Manual rebase is needed!" + printf "\n\n\n\n" + exit 1 + fi + + done + + + # Check that there we're not overwriting oldfiles without having a ".orig" copy + for f in $(find * -type f ! -name '*.orig') + do + newfile=$NEWDIR/$(echo $f | sed 's/.orig$//') + origfile=${newfile}.orig + oldfile=$OLDDIR/$f + if [ -f $oldfile ]; then + if [ ! -f $origfile ]; then + printf "\n\n\n\n" + echo "Error: Will overwrite $oldfile, but there is no" + echo "Error: $origfile!" + echo "Error: You need to create the `basename $origfile`!" + printf "\n\n\n\n" + exit 1 + fi + fi + done + + + popd >/dev/null +} + +prep_make_live() { + DEST=$TMP_HOSTMOUNT + REPO=$DEST/var/www/nailgun/ubuntu/fuelweb/x86_64 + echo "Live install" + ssh-copy-id root@$FUELHOST + sshfs root@1${FUELHOST}:/ $TMP_HOSTMOUNT + + if [ -f $REPO/dists/precise/main/binary-amd64/Packages.backup ]; then + echo "Error - found backup file for Packages!" + exit 1 + fi + + if [ -f $REPO/dists/precise/main/binary-amd64/Packages.gz.backup ]; then + echo "Error - found backup file for Packages.gz!" + exit 1 + fi + + if [ -f $REPO/dists/precise/Release.backup ]; then + echo "Error - found backup file for Release!" + exit 1 + fi + + if [ -d $DEST/etc/puppet.backup ]; then + echo "Error - found backup file for Puppet!" + exit 1 + fi + + cp $REPO/dists/precise/main/binary-amd64/Packages $REPO/dists/precise/main/binary-amd64/Packages.backup + cp $REPO/dists/precise/main/binary-amd64/Packages.gz $REPO/dists/precise/main/binary-amd64/Packages.gz.backup + cp $REPO/dists/precise/Release $REPO/dists/precise/Release.backup + cp -Rvp $DEST/etc/puppet $DEST/etc/puppet.backup +} + +post_make_live() { + echo "Installing into Puppet:" + cd $TOP/release/puppet/modules + for dir in * + do + echo " $dir" + cp -Rp $dir $DEST/etc/puppet/modules + done +} + +make_live() { + prep_make_live + copy_packages + post_make_live +} + + +prep_make_iso() { + DEST=$TOP/newiso + REPO=$DEST/ubuntu + echo "Making ISO..." + echo "Unpack of old ISO..." + if [ -d newiso ]; then + chmod -R 755 newiso + rm -rf newiso + fi + mkdir newiso + fusermount -u $TMP_OLDISO 2>/dev/null || cat /dev/null + fuseiso -p $ORIGISO $TMP_OLDISO + sleep 1 + cd $TMP_OLDISO + find . | cpio -pd $TOP/newiso + cd .. + fusermount -u $TMP_OLDISO + rm -Rf $TMP_OLDISO + chmod -R 755 $TOP/newiso +} + +make_iso_image() { + echo "Making ISO..." + cd $DEST + find . -name TRANS.TBL -exec rm {} \; + rm -rf rr_moved + + mkisofs --quiet -r -V "$VOLUMEID" -publisher "$PUBLISHER" \ + -p `git rev-parse --verify HEAD` -J -R -b isolinux/isolinux.bin \ + -no-emul-boot \ + -boot-load-size 4 -boot-info-table \ + --hide-rr-moved \ + -x "lost+found" -o $NEWISO . + + isoinfo -d -i $NEWISO +} + +# iso_copy_puppet: Create a new puppet-slave.tgz for the iso +iso_copy_puppet() { + echo "Installing into Puppet..." + mkdir -p $TMP_ISOPUPPET/release/puppet + cd $TMP_ISOPUPPET/release/puppet + tar xzf $DEST/puppet-slave.tgz + cd $TOP/release/puppet/modules + + verify_orig_files $TMP_ISOPUPPET/release/puppet $TOP/release/puppet/modules + # Remove all .orig files before copying as they now have been verfied + find $TOP/release/puppet/modules -type f -name '*.orig' -exec rm {} \; + + for dir in $TOP/release/puppet/modules/* + do + echo " $dir" + cp -Rp $dir $TMP_ISOPUPPET/release/puppet + done + cd $TMP_ISOPUPPET/release/puppet + + + tar czf $DEST/puppet-slave.tgz . + cd $TOP + rm -Rf $TMP_ISOPUPPET +} + +# iso_modify_image: Add/patch files in the ISO root +iso_modify_image () { + # TODO: Add logic for ".orig" files (hey! make a function!), which would look + # something like: + # 1. Is there an .orig file? + # 2. If the file is not present in origiso -> Error exit + # 3. If the file is changed in origiso -> Error exit (need manual rebase) + # 4. Otherwise continue, but don't copy orig file (or maybe we should?) + # ... and corresponding reverse logic: + # 1. If there is no .orig file, but file is present in origiso -> Error exit + echo "Modify ISO files (wild copy)..." + + verify_orig_files $DEST $TOP/release/isoroot + # Remove all .orig files before copying as they now have been verfied + find $TOP/release/isoroot -type f -name '*.orig' -exec rm {} \; + + cd $TOP/release/isoroot + cp -Rvp . $DEST +} + +make_iso() { + prep_make_iso + copy_packages + iso_copy_puppet + iso_modify_image + make_iso_image +} + +copy_packages() { + echo "Copying Debian packages..." + cd $TOP/release/packages/ubuntu/pool/main + for deb in `ls -1 | grep '\.deb$'` + do + echo " $deb" + cp $deb $REPO/pool/main + done + + echo "Running Fuel package patch file" + pushd $REPO/pool/main > /dev/null + + for line in `cat $TOP/apply_patches | grep -v "^#" | grep -v "^$"`; do + echo "Line is $line" + ref=`echo $line | cut -d '>' -f 1` + origpkg=`echo $line| cut -d '>' -f 2` + url=`echo $line | cut -d '>' -f 3` + + if [ -z "$origpkg" ]; then + echo "Error: No origpkg for patching" + exit 1 + fi + + if [ -z "$url" ]; then + echo "Error: No url for patching" + exit 1 + fi + + if [ -z "$ref" ]; then + echo "Error: No reference text for patching" + exit 1 + fi + + echo "CM: Patching Fuel package for $ref" | tee -a $REPORTFILE + echo "CM: Replacing package $origpkg with $url" | tee -a $REPORTFILE + oldrev=`get_deb_rev $origpkg` + rm $origpkg + wget --quiet $url + topkg=`basename $url` + echo "CM: MD5 of new package:" | tee -a $REPORTFILE + md5sum $topkg | tee -a $REPORTFILE + + patchname=`get_deb_name $topkg` + patchrev=`get_deb_rev $topkg` + echo "Correcting dependencies towards $patchname rev $patchrev - old rev $oldrev" | tee -a $REPORTFILE + $TOP/patch-packages/tools/correct_deps $patchname $oldrev $patchrev | tee -a $REPORTFILE + if [ $PIPESTATUS -ne 0 ]; then + exit 1 + fi + done + + printf "Done running Fuel patch file\n\n" + + echo "Running add packages file" + for line in `cat $TOP/add_opnfv_packages | grep -v "^#" | grep -v "^$"`; do + echo "Line is $line" + ref=`echo $line | cut -d '>' -f 1` + origpkg=`echo $line| cut -d '>' -f 2` + url=`echo $line | cut -d '>' -f 3` + + if [ -z "$origpkg" ]; then + echo "Error: No origpkg for patching" + exit 1 + fi + + if [ -z "$url" ]; then + echo "Error: No url for patching" + exit 1 + fi + + if [ -z "$ref" ]; then + echo "Error: No reference text for patching" + exit 1 + fi + + if [ "$origpkg" != "NONE" ]; then + echo "CM: Patching added package for $ref" | tee -a $REPORTFILE + echo "CM: Replacing package $origpkg with $url" | tee -a $REPORTFILE + oldrev=`get_deb_rev $origpkg` + rm $origpkg + else + echo "CM: Adding previoulsy uninstalled package for $ref" tee -a $REPORTFILE + fi + wget --quiet $url + topkg=`basename $url` + echo "CM: MD5 of new package:" | tee -a $REPORTFILE + md5sum $topkg | tee -a $REPORTFILE + if [ "$origpkg" != "NONE" ]; then + patchname=`get_deb_name $topkg` + patchrev=`get_deb_rev $topkg` + echo "Correcting dependencies towards $patchname rev $patchrev - old rev $oldrev" | tee -a $REPORTFILE + $TOP/patch-packages/tools/correct_deps $patchname $oldrev $patchrev | tee -a $REPORTFILE + if [ $PIPESTATUS -ne 0 ]; then + exit 1 + fi + fi + done + printf "Done running add packages file\n\n" + + popd > /dev/null + + if [ -f $TOP/patch-packages/release/patch-replacements ]; then + echo "Applying package patches" | tee -a $REPORTFILE + pushd $REPO/pool/main > /dev/null + printf "\n\n" | tee -a $REPORTFILE + for line in `cat $TOP/patch-packages/release/patch-replacements` + do + frompkg=`echo $line | cut -d ">" -f 1` + topkg=`echo $line | cut -d ">" -f 2` + echo "CM: Applying patch to $frompkg" | tee -a $REPORTFILE + echo "CM: New package rev after patch: $topkg" | tee -a $REPORTFILE + + if [ ! -f $frompkg ]; then + echo "Error: Can't find $frompkg in repo" + exit 1 + else + oldrev=`get_deb_rev $frompkg` + echo "Removing $frompkg from repo" + rm $frompkg + fi + + if [ ! -f $TOP/patch-packages/release/packages/$topkg ]; then + echo "Error: Can't find $topkg in patch release" + exit 1 + else + echo "Adding $topkg to repo" + cp $TOP/patch-packages/release/packages/$topkg . + fi + + patchname=`get_deb_name $topkg` + patchrev=`get_deb_rev $topkg` + echo "Correcting dependencies towards $patchname rev $patchrev - old rev $oldrev" | tee -a $REPORTFILE + $TOP/patch-packages/tools/correct_deps $patchname $oldrev $patchrev | tee -a $REPORTFILE + if [ $PIPESTATUS -ne 0 ]; then + exit 1 + fi + done + popd > /dev/null + fi + + echo "Generating metadata..." + pushd $REPO > /dev/null + + # The below methods are from 15B + APT_REL_CONF="$TOP/install/apt-ftparchive-release.conf" + APT_DEB_CONF="$TOP/install/apt-ftparchive-deb.conf" + + apt-ftparchive -c "${APT_REL_CONF}" generate "${APT_DEB_CONF}" + + # Fuel also needs this index file + cat dists/precise/main/binary-amd64/Packages | \ + awk '/^Package:/{pkg=$2} + /^Version:/{print pkg ": \"" $2 "\""}' > ubuntu-versions.yaml + cp ubuntu-versions.yaml $DEST + + apt-ftparchive -c "${APT_REL_CONF}" release dists/precise/ > dists/precise/Release + gzip -9cf dists/precise/Release > dists/precise/Release.gz + + popd > /dev/null + +} + + +############################################################################# + +trap my_exit EXIT + +CONF=`mktemp /tmp/XXXXXXX` +MODE=$1 +TOP=`pwd` + +if [ $MODE = "iso" ]; then + PUBLISHER="OPNFV" + TMP_OLDISO=`mktemp -d /tmp/XXXXXXX` + TMP_ISOPUPPET=`mktemp -d /tmp/XXXXXXX` + ORIGISO=$2 + NEWISO=$3 + VOLUMEID="$4 $5" + REPORTFILE="${NEWISO}.txt" + if [ ! -f $ORIGISO ]; then + echo "Can't find original MOS 5.1 iso at $ORIGISO" + rm $CONF + exit 1 + fi + + make_iso +else + echo "Unknown mode: $MODE" + exit 1 +fi diff --git a/fuel/build/install/uninstall.sh b/fuel/build/install/uninstall.sh new file mode 100755 index 0000000..36b8884 --- /dev/null +++ b/fuel/build/install/uninstall.sh @@ -0,0 +1,79 @@ +#!/bin/bash -e +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + + +my_exit() { + cd /tmp + if [ -d "$MOUNT" ]; then + fusermount -u $MOUNT + rmdir $MOUNT + fi +} +trap my_exit EXIT + +echo "Live uninstall is currently disabled as it is not tested" +exit 1 + +TOP=`pwd` +MOUNT=`mktemp -d /tmp/XXXXXXX` +ssh-copy-id root@10.20.0.2 +sshfs root@10.20.0.2:/ $MOUNT + +DEST=$MOUNT +REPO=$DEST/var/www/nailgun/ubuntu/fuelweb/x86_64 + +cd $REPO +if [ ! -f $REPO/dists/precise/main/binary-amd64/Packages.backup ]; then + echo "Error - didn't find backup file for Packages!" + exit 1 +fi + +if [ ! -f $REPO/dists/precise/main/binary-amd64/Packages.gz.backup ]; then + echo "Error - didn't find backup file for Packages.gz!" + exit 1 +fi + +if [ ! -f $REPO/dists/precise/Release.backup ]; then + echo "Error - didn't find backup file for Release!" + exit 1 +fi + +if [ ! -f $DEST/etc/puppet/manifests/site.pp.backup ]; then + echo "Error - didn't find backup file for site.pp!" + exit 1 +fi + +echo "Removing Debian packages:" +cd $TOP/release/pool/main +for deb in *.deb +do + echo " $deb" + rm -Rf $REPO/pool/main/$deb +done +cd $REPO + +echo "Removing Puppet modules:" +cd $TOP/puppet/modules +for dir in * +do + echo " $dir" + rm -Rf $DEST/etc/puppet/modules/$dir +done +cd $REPO + +echo "Restoring backups of datafiles" + +rm -f $REPO/dists/precise/main/binary-amd64/Packages $REPO/dists/precise/main/binary-amd64/Packages.gz +rm -f $REPO/dists/precise/Release $DEST/etc/puppet/manifests/site.pp +mv $REPO/dists/precise/main/binary-amd64/Packages.backup $REPO/dists/precise/main/binary-amd64/Packages +mv $REPO/dists/precise/main/binary-amd64/Packages.gz.backup $REPO/dists/precise/main/binary-amd64/Packages.gz +mv $REPO/dists/precise/Release.backup $REPO/dists/precise/Release +mv $DEST/etc/puppet/manifests/site.pp.backup $DEST/etc/puppet/manifests/site.pp diff --git a/fuel/build/opendaylight/Makefile b/fuel/build/opendaylight/Makefile new file mode 100644 index 0000000..9c3d22d --- /dev/null +++ b/fuel/build/opendaylight/Makefile @@ -0,0 +1,63 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +############################################################################ +# BEGIN of variables to customize +# +SHELL = /bin/bash +include ../config.mk + +BUILD_DIR := $(shell pwd) +GIT_DIR := /tmp +export CONFIG_SPEC_SCRIPT +export MAVEN_OPTS = -Xmx1024m -XX:MaxPermSize=512m +MAINTAINER = "Main Tainer <main.tainer@example.org>" +ODL_SHORT_NAME = odl +ODL_VERSION = 0.1-1 +DEPEND = openjdk-8-jdk +TARGET_BUILD_PATH="/tmp/controller/opendaylight/distribution/opendaylight-karaf/target/" +MAVEN_SPEC = $(BUILD_DIR)/odl_maven/settings.xml + +# +# END of variables to customize +############################################################################# + +.PHONY: all +all: odl + +.PHONY: setup +setup: + @if [ ! -d "$(TOPDIR)/f_odl" ]; then ln -s "$(realpath $(BUILD_DIR))/f_odl" "$(realpath $(TOPDIR))/f_odl"; fi + +.PHONY: odl +odl: +ifeq ($(ODL_MAIN_REPO),) + @echo "No config-spec target for ODL, nothing to build" +else + +ifeq ($(shell if [ -e .odl-build.log ];then cat .odl-build.log; fi;),$(ODL_MAIN_TAG)) + @cd /tmp && git clone $(ODL_MAIN_REPO) && cd /tmp/controller && git checkout $(ODL_MAIN_TAG) + @echo "ODL is up to date" +else + @if [ ! -d "/tmp/controller" ]; then cd /tmp && git clone $(ODL_MAIN_REPO); fi; + @cd /tmp/controller && git checkout $(ODL_MAIN_TAG) && mvn -Dmaven.test.skip=true -gs $(MAVEN_SPEC) clean install; + @./make-odl-deb.sh -N $(ODL_SHORT_NAME)_`cd /tmp/controller; git rev-parse --short HEAD` -n $(ODL_SHORT_NAME) -v "$(ODL_VERSION)" -t "$(ODL_MAIN_TAG)" -m $(MAINTAINER) -d $(DEPEND) -p $(TARGET_BUILD_PATH) + @echo $(ODL_MAIN_TAG) > .odl-build.log +endif +endif + +.PHONY: clean $(SUBCLEAN) +clean: $(SUBCLEAN) + @rm -Rf /tmp/controller + @rm -f .odl-build.log + @./make-odl-deb.sh -C + +.PHONY: release +release: diff --git a/fuel/build/opendaylight/README b/fuel/build/opendaylight/README new file mode 100644 index 0000000..7aa392e --- /dev/null +++ b/fuel/build/opendaylight/README @@ -0,0 +1,52 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +This directory builds the OpenDaylight debian package by cloning the +opendaylight.org repo, building the odl tag specified in +"fuel-build/config-spec" and constructing a debian package source tree under +"f_odl", which automatically is linked into "fuel_build/." for further build processing. + +The opendaylight has the following structure: +. ++--------+-------------+-----------+ +| | | | +| Makefile make-odl-deb.sh README +| (this file) +| ++----------+----------+ + | | + odl_maven/ f_odl/ + +Makefile: +Invoked by the git root Makefile, it builds the clones the odl repo from +odl, checkout the tag/branch indicated in "fuelbuild/config-spec", builds +odl, and calls "make-odl-deb.sh" to create a debian package source tree. + +make-odl-deb.sh: +Creates the odl debian package source tree in "f_odl" from the odl build +results. + +odl_maven/: +Contains needed control files for maven OpenDaylight build + +f_odl/: +Contains buildscripts and the generated odl debian package source tree produced +by the odl build (make-odl-deb.sh) which is later used by the root build system. + +NOTE on the controller/ git repo clone: +The git controller repo clone Contains all artifacts from the odl build, it only +exists in /tmp inside the build docker container and is not visible anywhere on +the build host + +Note on build caching: +The latest build results are cached, and will not be rebuilt unless the fuel-build/ +config-spec is changed in respect to odl version or if make clean is applied. +./.odl-build.log and ./.odl-build.history are used to keep adequate bookmaking to +track caching and needed cleanout. diff --git a/fuel/build/opendaylight/f_odl/Makefile b/fuel/build/opendaylight/f_odl/Makefile new file mode 100644 index 0000000..8ac5108 --- /dev/null +++ b/fuel/build/opendaylight/f_odl/Makefile @@ -0,0 +1,45 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +include $(BUILD_BASE)/config.mk +ODL_NAME_SHORT := odl +PACKAGE := odl_$(shell cd /tmp/controller; git rev-parse --short HEAD) +VERSION := 0.1-1 +DEB_NAME := $(PACKAGE)_$(VERSION) + +.PHONY: all +all: release/pool/main/$(DEB_NAME).deb + +release/pool/main/$(DEB_NAME).deb: +ifeq ($(ODL_MAIN_REPO),) + @echo "No config-spec target for ODL, nothing to build" +else + @mkdir -p tmp/src + @mkdir -p release/pool/main + @cp -rp $(DEB_NAME) tmp/src + @gzip -f9 tmp/src/$(DEB_NAME)/usr/share/doc/$(ODL_NAME_SHORT)/changelog.Debian + @fakeroot dpkg-deb --build tmp/src/$(DEB_NAME) + @lintian tmp/src/$(DEB_NAME).deb + @cp tmp/src/$(DEB_NAME).deb release/pool/main +endif + +.PHONY: clean +clean: + @rm -rf tmp + @rm -rf release + @rm -f $(DEB_DEST)/$(DEB_NAME).deb + + +.PHONY: release +release:release/pool/main/$(DEB_NAME).deb +ifneq ($(ODL_MAIN_REPO),) + @cp release/pool/main/$(DEB_NAME).deb $(DEB_DEST) + @cp -Rvp puppet/modules/* $(PUPPET_DEST) +endif diff --git a/fuel/build/opendaylight/f_odl/README b/fuel/build/opendaylight/f_odl/README new file mode 100644 index 0000000..077962d --- /dev/null +++ b/fuel/build/opendaylight/f_odl/README @@ -0,0 +1,49 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +This directory adds the OpenDaylight (odl) package and related puppet +deployment manifest such that it gets built into the .iso image an deployed +on the stack controller cluster. + +The f_odl has the following structure: +. ++--------+----------+-----------+------------+ + | | | | + puppet/ Makefile README odl_<change_id> + | (this file) /<version> + | | + | odl deb pkg src + modules/ + | + | + | + opnfv/ + | + | + | + manifests/ + | + | + | + odl.pp + +Makefile: +Invoked by the git root Makefile, it builds the odl debian package from the +debian pkg source directory (inside this directory) and pushes it together +with the manifests to the fuel build source artifact directory, such that it +eventually gets built into the new fuel .iso + +odl.pp: +Controls the installation and configuration of odl + +odl deb pkg src: +Is the debian package source directory tree including all needed odl artifacts +and debian pakage meta data. This debian source package tree is built from +fuel-build/opendaylight and doesnt exist before it has bee built. diff --git a/fuel/build/opendaylight/f_odl/puppet/modules/opnfv/manifests/odl.pp b/fuel/build/opendaylight/f_odl/puppet/modules/opnfv/manifests/odl.pp new file mode 100644 index 0000000..6165646 --- /dev/null +++ b/fuel/build/opendaylight/f_odl/puppet/modules/opnfv/manifests/odl.pp @@ -0,0 +1,13 @@ +class opnfv::odl { + if $::osfamily == 'Debian' { + + + case $::fuel_settings['role'] { + /controller/: { + package { 'odl': + ensure => installed, + } + } + } + } +} diff --git a/fuel/build/opendaylight/f_odl/testing/README b/fuel/build/opendaylight/f_odl/testing/README new file mode 100644 index 0000000..2ef4976 --- /dev/null +++ b/fuel/build/opendaylight/f_odl/testing/README @@ -0,0 +1,12 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +In order to test the functionality without performing a full deployment, run "puppet apply" on the +fake_init.pp which will call only the opnfv::odl class. diff --git a/fuel/build/opendaylight/f_odl/testing/fake_init.pp b/fuel/build/opendaylight/f_odl/testing/fake_init.pp new file mode 100644 index 0000000..0600d2e --- /dev/null +++ b/fuel/build/opendaylight/f_odl/testing/fake_init.pp @@ -0,0 +1,13 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +$fuel_settings = parseyaml($astute_settings_yaml) + +include opnfv::odl diff --git a/fuel/build/opendaylight/make-odl-deb.sh b/fuel/build/opendaylight/make-odl-deb.sh new file mode 100755 index 0000000..24974dc --- /dev/null +++ b/fuel/build/opendaylight/make-odl-deb.sh @@ -0,0 +1,314 @@ +#!/bin/bash +set -e +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +############################################################################## +# Default variable declarations + +COMMAND= +PACKAGE_NAME= +PACKAGE_SHORT_NAME= +PACKAGE_VERSION= +TARGET_BUILD_PATH= +DEPENDENCIES= +MAINTAINER= +ARCH="amd64" +BUILD_HISTORY=".odl-build-history" + +############################################################################## +# subroutine: usage +# Description: Prints out usage of this script + +usage () +{ +cat <<EOF +usage: $0 options + +$0 creates a ${PACKAGE_NAME} Debian package + +OPTIONS: + -n Package shoer name + -N Package name + -v Version + -t Tag + -p Target build path, the path where the built tar ball is to be fetched + -m Maintainer + -d Package dependencies + -h Prints this message + -C Clean + +E.g.: $0 -n my/deb/src/dest/path -N my-package -v 1.0-1 -t myTag -p path/to/the/source -m "Main Tainer <main.tainer.exampe.org> -d myJavaDependence +EOF +} + +############################################################################## +# subroutine: clean +# Description: Cleans up all artifacts from earlier builds + +clean () +{ +if [ -e $BUILD_HISTORY ]; then + while read line + do + rm -rf $line + done < $BUILD_HISTORY + rm ${BUILD_HISTORY} + exit 0 +fi +} + +############################################################################## +# make-DEBIAN_control +# Description: constructs the Debian pack control file + +make-DEBIAN_control () +{ +cat <<EOF +Package: $PACKAGE_SHORT_NAME +Version: $PACKAGE_VERSION +Section: base +Priority: optional +Architecture: $ARCH +Depends: $DEPENDENCIES +Maintainer: $MAINTAINER +Description: OpenDaylight deamon + This is a daemon for the opendaylight/odl controller service. +EOF +} + +############################################################################## +# subroutine: make-DEBIAN_conffiles +# Description: Constructs the Debian package config files assignment + +make-DEBIAN_conffiles () +{ +cat <<EOF +/etc/odl/etc/all.policy +/etc/odl/etc/config.properties +/etc/odl/etc/custom.properties +/etc/odl/etc/distribution.info +/etc/odl/etc/equinox-debug.properties +/etc/odl/etc/java.util.logging.properties +/etc/odl/etc/jmx.acl.cfg +/etc/odl/etc/jmx.acl.java.lang.Memory.cfg +/etc/odl/etc/jmx.acl.org.apache.karaf.bundle.cfg +/etc/odl/etc/jmx.acl.org.apache.karaf.config.cfg +/etc/odl/etc/jmx.acl.org.apache.karaf.security.jmx.cfg +/etc/odl/etc/jmx.acl.osgi.compendium.cm.cfg +/etc/odl/etc/jre.properties +/etc/odl/etc/keys.properties +/etc/odl/etc/org.apache.felix.fileinstall-deploy.cfg +/etc/odl/etc/org.apache.karaf.command.acl.bundle.cfg +/etc/odl/etc/org.apache.karaf.command.acl.config.cfg +/etc/odl/etc/org.apache.karaf.command.acl.feature.cfg +/etc/odl/etc/org.apache.karaf.command.acl.jaas.cfg +/etc/odl/etc/org.apache.karaf.command.acl.kar.cfg +/etc/odl/etc/org.apache.karaf.command.acl.shell.cfg +/etc/odl/etc/org.apache.karaf.command.acl.system.cfg +/etc/odl/etc/org.apache.karaf.features.cfg +/etc/odl/etc/org.apache.karaf.features.obr.cfg +/etc/odl/etc/org.apache.karaf.features.repos.cfg +/etc/odl/etc/org.apache.karaf.jaas.cfg +/etc/odl/etc/org.apache.karaf.kar.cfg +/etc/odl/etc/org.apache.karaf.log.cfg +/etc/odl/etc/org.apache.karaf.management.cfg +/etc/odl/etc/org.apache.karaf.shell.cfg +/etc/odl/etc/org.ops4j.pax.logging.cfg +/etc/odl/etc/org.ops4j.pax.url.mvn.cfg +/etc/odl/etc/regions-config.xml +/etc/odl/etc/shell.init.script +/etc/odl/etc/startup.properties +/etc/odl/etc/system.properties +/etc/odl/etc/users.properties +/etc/odl/configuration/context.xml +/etc/odl/configuration/logback.xml +/etc/odl/configuration/tomcat-logging.properties +/etc/odl/configuration/tomcat-server.xml +EOF +} + +############################################################################## +# subroutine: make-DEBIAN_postinst +# Description: Constructs the Debian package post installation script + +make-DEBIAN_postinst () +{ +cat <<EOF +#!/bin/bash -e +ln -s /etc/${PACKAGE_SHORT_NAME}/* ${TARGET_INSTALL_PATH} +echo "OpenDaylight $TAG version $PACKAGE_VERSION has been installed" +EOF +} + +############################################################################## +# subroutine: make-DEBIAN_bin +# Description: Constructs the bin script (normally under /usr/bin) + +make-DEBIAN_bin () +{ +cat <<EOF +#!/bin/bash -e +${TARGET_INSTALL_PATH}bin/karaf $@ +EOF +} + +############################################################################## +# subroutine: make-DEBIAN_copyright +# Description: Constructs the copyright text (normally under /usr/share/doc...) + +make-DEBIAN_copyright () +{ +cat <<EOF +OpenDaylight - an open source SDN controller +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +EOF +} + +############################################################################## +# subroutine: make-DEBIAN_changelog +# Description: Constructs the changelog text (normally under /usr/share/doc...) + +make-DEBIAN_changelog () +{ +cat <<EOF +$PACKAGE_SHORT_NAME ($PACKAGE_VERSION) precise-proposed; urgency=low + + * Derived from $PACKAGE_NAME $PACHAGE_VERSION + + -- $MAINTAINER $(date) +EOF +} + +############################################################################## +# MAIN + +while getopts "N:n:v:d:Chm:t:p:" OPTION +do + case $OPTION in + h) + usage + exit 0 + ;; + + N) + PACKAGE_NAME=$OPTARG + COMMAND+="-N ${PACKAGE_NAME} " + ;; + + n) + PACKAGE_SHORT_NAME=$OPTARG + COMMAND+="-n ${PACKAGE_SHORT_NAME} " + ;; + + v) + PACKAGE_VERSION=$OPTARG + COMMAND+="-v ${PACKAGE_VERSION} " + ;; + + p) + TARGET_BUILD_PATH=$OPTARG + COMMAND+="-p ${TARGET_BUILD_PATH} " + ;; + + t) + TAG=$OPTARG + COMMAND+="-t ${TAG} " + ;; + + m) + MAINTAINER=$OPTARG + COMMAND+="-m ${MAINTAINER} " + ;; + + d) + DEPENDENCIES=$OPTARG + COMMAND+="-d ${DEPENDENCIES} " + ;; + + A) + ARCH=$OPTARG + COMMAND+="-A ${ARCH} " + ;; + + C) + COMMAND+="-C " + clean + exit 0 + ;; + esac +done + +# Constructing script variables +DEB_PACK_BASE_PATH="f_${PACKAGE_SHORT_NAME}/${PACKAGE_NAME}_${PACKAGE_VERSION}/" +echo ${DEB_PACK_BASE_PATH} >> "$BUILD_HISTORY" +TARGET_INSTALL_PATH="/usr/share/java/${PACKAGE_SHORT_NAME}/" +DEB_PACK_CONTENT_PATH="${DEB_PACK_BASE_PATH}usr/share/java/${PACKAGE_SHORT_NAME}/" +DEB_PACK_CONFIG_PATH="${DEB_PACK_BASE_PATH}etc/${PACKAGE_SHORT_NAME}" +TARGET_TAR=$(ls ${TARGET_BUILD_PATH}*.tar.gz) +TARGET_TAR="${TARGET_TAR##*/}" +TAR_PATH="${TARGET_TAR%.*}" +TAR_PATH="${TAR_PATH%.*}" +if [ -e $DEB_PACK_BASE_PATH ]; then + rm -R $DEB_PACK_BASE_PATH +fi + +# Create Deb pack content and configuration +mkdir -p ${DEB_PACK_CONTENT_PATH} +cp ${TARGET_BUILD_PATH}${TARGET_TAR} ${DEB_PACK_CONTENT_PATH} +tar -xzf ${DEB_PACK_CONTENT_PATH}${TARGET_TAR} -C ${DEB_PACK_CONTENT_PATH} +rm ${DEB_PACK_CONTENT_PATH}${TARGET_TAR} +mv ${DEB_PACK_CONTENT_PATH}${TAR_PATH}/* ${DEB_PACK_CONTENT_PATH}. +rm -R ${DEB_PACK_CONTENT_PATH}${TAR_PATH} + +# Crate and populate Deb pack config target +mkdir -p ${DEB_PACK_CONFIG_PATH}/etc +mv ${DEB_PACK_CONTENT_PATH}etc/* ${DEB_PACK_CONFIG_PATH}/etc/ +rm -R ${DEB_PACK_CONTENT_PATH}etc +mkdir -p ${DEB_PACK_CONFIG_PATH}/configuration +mv ${DEB_PACK_CONTENT_PATH}configuration/* ${DEB_PACK_CONFIG_PATH}/configuration/ +rm -R ${DEB_PACK_CONTENT_PATH}configuration + +# Set package permisions +find ${DEB_PACK_CONTENT_PATH} -type d -print -exec chmod 755 {} \; +find ${DEB_PACK_CONFIG_PATH}/etc/ -type f -print -exec chmod 644 {} \; +find ${DEB_PACK_CONFIG_PATH}/etc/ -type d -print -exec chmod 755 {} \; + +# Create package usr/bin odl script +mkdir "${DEB_PACK_BASE_PATH}usr/bin" +chmod 755 "${DEB_PACK_BASE_PATH}usr/bin" +make-DEBIAN_bin > "${DEB_PACK_BASE_PATH}usr/bin/odl" +chmod 755 "${DEB_PACK_BASE_PATH}usr/bin/odl" + +# Create Deb pack install meta-data +mkdir "${DEB_PACK_BASE_PATH}DEBIAN" +make-DEBIAN_control > "${DEB_PACK_BASE_PATH}DEBIAN/control" +make-DEBIAN_conffiles > "${DEB_PACK_BASE_PATH}DEBIAN/conffiles" +mkdir -p "${DEB_PACK_BASE_PATH}usr/share/doc/${PACKAGE_SHORT_NAME}" +make-DEBIAN_copyright > "${DEB_PACK_BASE_PATH}usr/share/doc/${PACKAGE_SHORT_NAME}/copyright" +make-DEBIAN_changelog > "${DEB_PACK_BASE_PATH}usr/share/doc/${PACKAGE_SHORT_NAME}/changelog.Debian" + +# Create Deb pack post install symlinks and usr/bin scripts +make-DEBIAN_postinst > "${DEB_PACK_BASE_PATH}DEBIAN/postinst" +chmod 755 "${DEB_PACK_BASE_PATH}DEBIAN/postinst" +mkdir -p "${DEB_PACK_BASE_PATH}usr/bin" diff --git a/fuel/build/opendaylight/odl_maven/settings.xml b/fuel/build/opendaylight/odl_maven/settings.xml new file mode 100644 index 0000000..35a4442 --- /dev/null +++ b/fuel/build/opendaylight/odl_maven/settings.xml @@ -0,0 +1,46 @@ +<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"> + + <profiles> + <profile> + <id>opendaylight-release</id> + <repositories> + <repository> + <releases> + <enabled>true</enabled> + <updatePolicy>never</updatePolicy> + </releases> + <snapshots> + <enabled>false</enabled> + </snapshots> + <id>opendaylight-mirror</id> + <name>opendaylight-mirror</name> + <url>http://nexus.opendaylight.org/content/groups/public/</url> + </repository> + </repositories> + </profile> + + <profile> + <id>opendaylight-snapshots</id> + <repositories> + <repository> + <releases> + <enabled>false</enabled> + </releases> + <snapshots> + <enabled>true</enabled> + </snapshots> + <id>opendaylight-snapshot</id> + <name>opendaylight-snapshot</name> + <url>http://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/</url> + </repository> + </repositories> + </profile> + </profiles> + + <activeProfiles> + <activeProfile>opendaylight-release</activeProfile> + <activeProfile>opendaylight-snapshots</activeProfile> + </activeProfiles> +</settings> diff --git a/fuel/build/patch-packages/Makefile b/fuel/build/patch-packages/Makefile new file mode 100644 index 0000000..9eb22fe --- /dev/null +++ b/fuel/build/patch-packages/Makefile @@ -0,0 +1,26 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +SUBDIRS := python-nova +SUBCLEAN = $(addsuffix .clean,$(SUBDIRS)) + +.PHONY: $(SUBDIRS) $(SUBCLEAN) clean +$(SUBDIRS): + @mkdir -p release/packages + ORIGISO=$(ORIGISO) REVSTATE=$(REVSTATE) $(MAKE) -C $@ -f Makefile release + +release: $(SUBDIRS) + @echo $(ORIGISO) + +clean: $(SUBCLEAN) + @rm -Rf release + +$(SUBCLEAN): %.clean: + $(MAKE) -C $* -f Makefile clean diff --git a/fuel/build/patch-packages/python-nova/Makefile b/fuel/build/patch-packages/python-nova/Makefile new file mode 100644 index 0000000..fbf5fb4 --- /dev/null +++ b/fuel/build/patch-packages/python-nova/Makefile @@ -0,0 +1,29 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) + +.PHONY: all +all: + +.PHONY: clean +clean: + @rm -rf package + @rm -rf *.deb + @rm -rf patch-replacements + @rm -rf .package + +.PHONY: release +release: + ../tools/deb_unpack python-nova_*.deb $(ORIGISO) + patch -s -p0 < nova-console.patch + ../tools/deb_pack $(REVSTATE) + @cp *.deb ../release/packages + @cat patch-replacements >> ../release/patch-replacements diff --git a/fuel/build/patch-packages/python-nova/nova-console.patch b/fuel/build/patch-packages/python-nova/nova-console.patch new file mode 100644 index 0000000..d5b3853 --- /dev/null +++ b/fuel/build/patch-packages/python-nova/nova-console.patch @@ -0,0 +1,28 @@ +*** package/usr/share/pyshared/nova/virt/libvirt/driver.py.orig 2015-02-16 14:19:53.203052869 +0100 +--- package/usr/share/pyshared/nova/virt/libvirt/driver.py 2015-02-16 14:22:53.065017671 +0100 +*************** +*** 3959,3968 **** + # client app is connected. Thus we can't get away + # with a single type=pty console. Instead we have + # to configure two separate consoles. +! consolelog = vconfig.LibvirtConfigGuestSerial() +! consolelog.type = "file" +! consolelog.source_path = self._get_console_log_path(instance) +! guest.add_device(consolelog) + + consolepty = vconfig.LibvirtConfigGuestSerial() + else: +--- 3959,3971 ---- + # client app is connected. Thus we can't get away + # with a single type=pty console. Instead we have + # to configure two separate consoles. +! # Disabled by OPNFV BEGIN +! # consolelog = vconfig.LibvirtConfigGuestSerial() +! # consolelog.type = "file" +! # consolelog.source_path = self._get_console_log_path(instance) +! # guest.add_device(consolelog) +! pass +! # Disabled by OPNFV END + + consolepty = vconfig.LibvirtConfigGuestSerial() + else: diff --git a/fuel/build/patch-packages/tools/correct_deps b/fuel/build/patch-packages/tools/correct_deps new file mode 100755 index 0000000..cfb7d53 --- /dev/null +++ b/fuel/build/patch-packages/tools/correct_deps @@ -0,0 +1,78 @@ +#!/bin/bash +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + + +patch_package () { + deb=$1 + pkgdep=$2 + newrev=$3 + + + tmpdir=`mktemp -d /tmp/patchXXXXX` + + cp $deb $tmpdir + pushd $tmpdir > /dev/null + + mkdir -p repack + dpkg -x $deb repack + + mkdir -p repack/DEBIAN + dpkg -e $deb repack/DEBIAN + + + pushd repack/DEBIAN > /dev/null + + echo "Before: `cat control | grep '^Depends:'`" + sed -i "s/$pkgdep (\([^ ]*\) [^)]*)/$pkgdep (\1 $newrev)/" control + echo "After: `cat control | grep '^Depends:'`" + popd > /dev/null + + fakeroot dpkg-deb --build repack + + popd > /dev/null + + cp $tmpdir/repack.deb $deb + rm -Rf $tmpdir +} + +# Name of package for which to check dependencies to +PKGDEP=$1 +# The old revision of the package in question +OLDREV=$2 +# The new revision of the package in question +NEWREV=$3 + +if [ -z "$PKGDEP" ]; then + echo "No package dependency name" + exit 1 +fi + +if [ -z "$OLDREV" ]; then + echo "No old rev" + exit 1 +fi + +if [ -z "$NEWREV" ]; then + echo "No new rev" + exit 1 +fi + + +for deb in *.deb +do + ar p $deb control.tar.gz | tar xzO ./control | grep -q "^Depends:.* ${PKGDEP} ([^ ]* ${OLDREV})" + if [ $? -eq 0 ]; then + name=`ar p $deb control.tar.gz | tar xzO ./control | grep "^Package:.* " | sed 's/.* //'` + echo "**** Changing dependencies line in $deb ($name) ****" + patch_package $deb $PKGDEP $NEWREV + fi +done + diff --git a/fuel/build/patch-packages/tools/deb_pack b/fuel/build/patch-packages/tools/deb_pack new file mode 100755 index 0000000..f07a15d --- /dev/null +++ b/fuel/build/patch-packages/tools/deb_pack @@ -0,0 +1,63 @@ +#!/bin/bash -e +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + + +my_exit() { + + if [ -d "$ISOROOT" ]; then + fusermount -u $ISOROOT + fi +} + +ME=$(basename $0) + +trap my_exit EXIT + +REV=$1 +if [ -z "$REV" ]; then + echo "$ME: Error - no revision info provided" + exit 1 +fi + +if [ ! -d package ]; then + echo "$ME: Error - package directory does not exist" + exit 1 +fi + +echo "Stepping revision number to $REV" + + +revision_step () +{ + REV=$1 + + sed -i "s/^\(Version:.*$\)/\1-eri$REV/" package/DEBIAN/control +} + +md5sum_gen () +{ + pushd package + find * -type f | grep -v "^DEBIAN/" | xargs md5sum | sort > DEBIAN/md5sums + popd +} + +revision_step $REV +md5sum_gen +fakeroot dpkg-deb --build package +PKGNAME=`grep "^Package: " package/DEBIAN/control | awk '{ print $2 }'` +PKGREV=`grep "^Version: " package/DEBIAN/control | awk '{ print $2 }'` +mv package.deb ${PKGNAME}_${PKGREV}.deb + +ORIGPKG=`cat .package` + +# Add patch into file read by install.sh +echo "$ORIGPKG>${PKGNAME}_${PKGREV}.deb" > patch-replacements + diff --git a/fuel/build/patch-packages/tools/deb_unpack b/fuel/build/patch-packages/tools/deb_unpack new file mode 100755 index 0000000..aaa60b7 --- /dev/null +++ b/fuel/build/patch-packages/tools/deb_unpack @@ -0,0 +1,58 @@ +#!/bin/bash -e +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + + +my_exit() { + + if [ -d "$ISOROOT" ]; then + fusermount -u $ISOROOT + fi +} + +ME=$(basename $0) + +trap my_exit EXIT + + +if [ -z "$1" ]; then + echo "$ME: Error - No package specified" + exit 1 +fi + +if [ -z "$2" ]; then + echo "$ME: Error - No ISO path specified" + exit 1 +fi + +DEB=$1 +ORIGISO=$2 +DEST=package + +if [ -e $DEST -o -d $DEST ]; then + echo "$ME: Error - $DEST already exists" + exit 1 +fi + +ISOROOT=`mktemp -d /tmp/XXXXXXX` +fuseiso -p $ORIGISO $ISOROOT +sleep 1 + +if [ ! -f $ISOROOT/ubuntu/pool/main/$DEB ];then + echo "Could not find package $DEB in source ISO!" +fi + +mkdir -p $DEST +dpkg -x $ISOROOT/ubuntu/pool/main/$DEB $DEST + +mkdir -p $DEST/DEBIAN +dpkg -e $ISOROOT/ubuntu/pool/main/$DEB $DEST/DEBIAN + +echo $DEB > .package
\ No newline at end of file diff --git a/fuel/build/patch-packages/tr_example/Makefile b/fuel/build/patch-packages/tr_example/Makefile new file mode 100644 index 0000000..b7cab35 --- /dev/null +++ b/fuel/build/patch-packages/tr_example/Makefile @@ -0,0 +1,30 @@ +############################################################################## +# Copyright (c) 2015 Ericsson AB and others. +# stefan.k.berg@ericsson.com +# jonas.bjurel@ericsson.com +# All rights reserved. This program and the accompanying materials +# are made available under the terms of the Apache License, Version 2.0 +# which accompanies this distribution, and is available at +# http://www.apache.org/licenses/LICENSE-2.0 +############################################################################## + +TOP := $(shell pwd) + +.PHONY: all +all: + +.PHONY: clean +clean: + @rm -rf package + @rm -rf *.deb + @rm -rf patch-replacements + @rm -rf .package + +.PHONY: release +release: + ../tools/deb_unpack python-oslo.messaging_1.3.0-fuel5.1~mira4_all.deb $(ORIGISO) + @mkdir -p package/etc + @echo "Hello, world" > package/etc/hello.txt + ../tools/deb_pack $(REVSTATE) + @cp *.deb ../release/packages + @cat patch-replacements >> ../release/patch-replacements |