summaryrefslogtreecommitdiffstats
path: root/build/Makefile
AgeCommit message (Collapse)AuthorFilesLines
2016-10-17Point to specific snapshot of mos repositoriesMichal Skalski1-1/+3
Fuel 9.1 was released, it is a first time when new version of Fuel is distributed through the same repositories as previous one, point to specific snaphost of mos ubuntu repo to prevent mixing packages. Change-Id: I64bb52657ce476b0c1fdc8e858231ad751ba12b1 Signed-off-by: Michal Skalski <mskalski@mirantis.com>
2016-10-08build: Use OPNFV_GIT_SHA for ISO preparer IDAlexandru Avadanii1-2/+2
isoinfo -i lists the following information for Fuel@OPNFV ISO: "Data preparer id: 86aafaf5454a846c417848bb94f264c4420160f3" where the SHA hash is Fuel git repo HEAD SHA. For Armband's build system, using only the Fuel commit ID is not enough to fully describe the state of the source code, as patches are also applied to other fuel modules (as git submodules). Instead, a pointer to a valid Armband commit ID should be used. However, Armband overrides OPNFV_GIT_SHA to the Armband git repo commit hash, so the ISO metadata should also reflect this. While at it, allow product name to be overriden, to signal the commit ID should be looked up inside the Armband repository. Change-Id: I33ad490f1afe28c1d439dda40e39cee1955e0ac2 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2016-10-05build: Use git submodules for Fuel git reposAlexandru Avadanii1-21/+18
This change reworks support for cloning (fetching) and patching all Fuel components (fuel-library ... fuel-ui, see [1] for full list), both outside the ISO build process (for development purposes, like testing OPNFV patches apply cleanly to each component), and during the ISO build. The implementation relies heavily on git submodules and patches, so backporting pending upstream gerrit changes, as well as adding new out-of-upstream-tree patches, should be trivial. Also, since without tracking remotes the repos rarely change, leverage the in-place OPNFV build caching mechanism to drastically reduce the number of git clones during each build. The mechanism is detailed more in-depth in the JIRA ticket [3], and it's merely a rehash of the submodule handling in Armband, implemented initially by Stanislaw Kardach <stanislaw.kardach@cavium.com>. Pros (+ new features): - OPNFV patches for Fuel components can be applied (imported) or developed (exported) outside of the ISO build process; - git repo caching reduces the number of external pulls; - reuse the same fuel-* componets in ALL targets e.g. fuel-mirror used to employ 2 slightly different git trees; - one step closer to Fuel/Armband merge (lots of steps left though); Cons: - adds a lot complexity to solve a non-problem (handling the repos is fine in the current form, for the current goals); However, patching <fuel-astute> or <fuel-agent> seems to be quite complicated with the current mechanism, as env vars override our locally patched repos for these components). The proposed change should eliminate this issue completely. CHANGE: Aligned divergent fuel-mirror in ISO vs local repo build by applying the 302 redirect fix ("Fixed handling http redirects") [2] on top of 9.0.1 tag in fuel-mirror repo, and using that for both. This obsoleted the followking mk file: - build/f_isoroot/f_repobuild/config.mk CHANGE: Removed unused make target `get-fuel-repo` from build/config.mk. FIXME: Remote tracking is currently deactivated for all branches. Change this when remote HEAD should be tracked instead, e.g. during Newton rebase. [1] https://github.com/openstack/fuel-main/blob/stable/mitaka/repos.mk#L32 [2] https://review.openstack.org/#/c/334882/ [3] https://jira.opnfv.org/browse/FUEL-200 TODO: Extend build/f_repos/README.md with info about: - branch changes; - tag bumps; - patching now supports subdirs; v5 -> v6: * Added support for nesting patches in subdirs, to be used for feature groups and related changes across submodules. E.g. Upcoming multiarch EFI support could go in: - .../f_repos/patch/fuel-agent/multiarch-efi/...; - .../f_repos/patch/fuel-astute/multiarch-efi/...; While Armband patches could go in: - .../f_repos/patch/fuel-agent/armband/...; - .../f_repos/patch/fuel-astute/armband/...; etc. v6 -> v13: * fix wrong tag used for VERSION dump; * fix wrong tag used for patches-export; * move OPNFV_TAG to main config.mk and use it in FUEL_*_COMMIT; * only run `patches-import` when patches changed / not yet applied; * only run `clean` for stale `.cacheid` (avoid re-applying patches); * allow git submodule path/name to be different (FIXED); * put-cache should only run when artifact is not already cached; * `git am`: use 3-way merge so already applied patches are gently skipped (useful for upstreaming patches from Armband); * Properly support nested patch dirs; * Colorize the output a little; JIRA: FUEL-200 Change-Id: I9dbdd9b3022896d4497d21be8dacc859730db489 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2016-08-31build: Move patches to <patch-repos> dirAlexandru Avadanii1-17/+24
Decouple main Makefile and repository patches by moving all Fuel component patches to a subdir, <patch-repo>. Proposed structure of <patch-repos> dir: - patch-repos/*.patch are intended for <fuel-main> git repo; - patch-repos/build/repo/<fuel-module>/*.patch are intended for <fuel-module>, e.g. "fuel-nailgun"; Full list of Fuel submodules supported by this mechanism at [1]. While at it, replace all occurences of </tmp/fuel-main> with <FUEL_MAIN_DIR> in main Makefile and rewrite a few git/make calls to use the "-C" arg. NOTE: `git -C` is available starting with git 1.8.5. [1] https://github.com/openstack/fuel-main/blob/stable/mitaka/repos.mk #L32-L44 Change-Id: I5e4adc7285054777c3898592ed48482ccd2b706f Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2016-08-31Merge changes from topic 'multiarch-mirrors'Jonas Bjurel1-0/+1
* changes: build: cache: Consider UBUNTU_ARCH in .cacheid repo mirror: Allow multi-arch local mirrors
2016-08-24build: Allow MIRROR_MOS_UBUNTU overrideAlexandru Avadanii1-1/+1
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2016-08-24repo mirror: Allow multi-arch local mirrorsAlexandru Avadanii1-0/+1
Original patch from Stan, signed by me for upstreaming. This patch allows specifying multiple architectures via UBUNTU_ARCH in form of a list of space separated architectures. The first architecture in the list is considered primary and will be used for building all the deb packages by fuel-main. Additional architectures are added to allow targets of other architectures to use the mirror. NOTE: this imposes a requirement that all packages built are arch independent (which is true so far). [ Alexandru Avadanii ] Reworked for applying on top of Fuel@OPNFV. Pass UBUNTU_ARCH to Docker containers. Change-Id: Ibc4ae03223545383813ac58ec34d44c1f97b4694 Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2016-08-03Allow to use Intel NICs for DPDK on Ericsson POD2Michal Skalski1-5/+3
JIRA: FUEL-161 Change-Id: I009ba7e9587f3c47d8e609c96b0666394887929e Signed-off-by: Michal Skalski <mskalski@mirantis.com>
2016-08-02build: fuel-main: Decouple patches, Makefile.Alexandru Avadanii1-8/+5
Tested in armband, with very small adaptions. Use `git apply` and/or `git am` in favor of `patch`. Change patches from old context format to unified diff. This change allows us to decouple Makefile and patches. This means we no longer have to manually stage touched files before committing them to git. Possible improvements: - move patches to separate dir and add ordering info (prefix); - merge patches (currently we have one per touched file); Remove obsolete (unused?) patch: - fuel-main_docker_version.patch Change-Id: I5b621370993b259779c813b47105f632948e6da7 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2016-07-21Pinning down exact Fuel 9.0.1 repo versionsMichal Skalski1-2/+0
stable/mitaka branch is open for new commits for Fuel 9.1. To not be impacted by new upstream changes point to specific commits which were used for Fuel 9.0 General Availabilit (9.0.1). JIRA: https://jira.opnfv.org/browse/FUEL-153 verify: no-cache Change-Id: I7fd8d2f11e4be6f47e39924913c48bfe7d4f2eb3 Signed-off-by: Michal Skalski <mskalski@mirantis.com>
2016-06-14Build local mirrors for stable/mitakaMichal Skalski1-0/+4
Change-Id: I3128652aeb87cb2cfaa91ded1b8d9ebeeb70d33b Signed-off-by: Michal Skalski <mskalski@mirantis.com>
2016-06-08Do not create files when check connectivityMichal Skalski1-1/+1
Change-Id: If0ab25ee843f9892510da3d3c8d1f10a54440545 Signed-off-by: Michal Skalski <mskalski@mirantis.com>
2016-05-30Build system uplift to Fuel 9.0/MitakaStefan K. Berg1-12/+27
Switching to Fuel 9.0/Mitaka for the build system. Overhaul of the patching mechanism. As bootstrap_admin_node.sh has been transitioned into an RPM (fuel-support), the lazy designer found it more simple to patch that script during the Fuel build phase than at the OPNFV ISO generation. The patch mechanism has been changed to a normal context diff instead of the orig/modified file tuples previously used. Hopefully this will require fewer manual rebases (may the fuzz be with us!). Also the ks.cfg patching has transitioned to a context based ordinary patch for the same reasons, but this is as before taking place during OPNFV ISO generation. Patch naming made more descriptive. The reaping mechanism has been slightly modified due to a change in the naming of the node files when these are generated by the Fuel CLI. IMPORTANT 1: The package cache mechanism is currently disabled, it is only possible to install Fuel with a direct internet connection. This will be fixed in a later change set! IMPORTANT 2: All plugins has been disabled! As you have re-certified your plugin with Fuel 9.0, please re-enable it in build/Makefile! Change-Id: Ia918d16a74b68f89d178e06befe6e8a7a9367bf9 Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
2016-05-11Adjust kernel numbers comparisonMichal Skalski1-3/+2
With current solution kernel 4.2.0 is treated as earlier version, final comparison is: 42 < 319. This change will compare coresponding numbers of installed kernel with required version. Change-Id: Iac3aed0eb21ac8ad3138c2169299ca3b43e6e663 Signed-off-by: Michal Skalski <mskalski@mirantis.com>
2016-04-22Correction due to changed cpio behaviorStefan K. Berg1-0/+2
As part of a security fix for cpio, its behavior has changed in terms of how symlinks are handled. This is affecting the Fuel build process, for which this commit is a fix (reverting cpio back to its original behavior by adding the argument "--extract-over-symlinks" in those instances where this is needed). For details, see https://jira.opnfv.org/browse/FUEL-125 Change-Id: I455b11a16ad52d8dd09165165447e4c4a661b45a Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
2016-02-22Fix network connection check issueRuijing Guo1-1/+1
Change-Id: Id42af08e9fe76db7f9d7dbae3b8b5bf60b079b14 Signed-off-by: Ruijing Guo <ruijing.guo@intel.com>
2016-02-07We need to preserve environment for sudo buildStefan K. Berg1-1/+1
In order to preserve environment variables (esp commit ids) for the Fuel build, the environment needs to be kept when the build loop is run. Change-Id: Ia044ea92853147280db0686ac857beda52ab50dc Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
2016-02-04Added environment sanity checks for build.Jonas Bjurel1-0/+6
Based on a discussion I and Stefan had today, better to check early on than crash hours in to the build with vague information of the root-cause. Added checks: - Linus distro - must be Ubuntu - Kernel version - must be 3.19 or later - Docker network sanity check Change-Id: Ie6758064ade4c58388368106d9bc556730d2bcec Signed-off-by: Jonas Bjurel <jonas.bjurel@ericsson.com>
2016-01-29Create local mirrors for offline installationMichal Skalski1-1/+1
- switch to fuel-mirror [1] - plugin developers can create packages.yaml file with names of required packages - use custom configuration for fuel-bootstrap which always point to local repositories - delay process of bootstrap image creation until mirrors will be ready [1] https://github.com/openstack/fuel-mirror Change-Id: Ie4a47c8cc13d9ded8c6b5fe48cdcbfeb1f6277a4 Signed-off-by: Michal Skalski <mskalski@mirantis.com>
2016-01-27Merge "Revert "Fix build failure in fuel base & qemu""Stefan K. Berg1-1/+0
2016-01-24Cache as we go instead of in the endStefan K. Berg1-4/+3
In order to speed up the build process when building with the cache enabled, cache immediately following the build of a sub artifact instead of at the end. Should the build fail, we don't need to rebuild those parts that already were successful in the last build. Change-Id: I23b59f7ba4a04272aea855cf451793158fe12241 Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
2016-01-24Break early in build if docker-in-docker failsStefan K. Berg1-0/+2
Change-Id: Ic5d8779f938133a958627170bd379198bec36342 Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
2016-01-22Select closest Fuel mirror as measured by pingStefan K. Berg1-1/+1
Select the (hopefully) geographically closest Fuel mirror to speed up the build time. Change-Id: I586d90ae58a9c22b1cf60df1bdb907c0e97486fa Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
2016-01-22Merge "Store Fuel ISO in build cache as it is built"Stefan K. Berg1-0/+4
2016-01-22Restore ISO file name to make CI functional againStefan K. Berg1-2/+2
An overzealous designer :) renamed the ISO output file, which broke CI. This will put everything right again. Change-Id: I25dca7ecfaa679872ee3d670b5df56333b3321c4 Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
2016-01-22Store Fuel ISO in build cache as it is builtStefan K. Berg1-0/+4
Right not the build cache is populated after a full (successful) build. As the Fuel ISO takes some time to build, this saves quite some time in the cases where the ISO is built successfully but a following plugin build fails. Change-Id: Ic6be1f5b5d00b9436eb0f82911cf28b449d0a156 Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
2016-01-21Use mirrors snapshotsMichal Skalski1-0/+18
Add logic to fetch information about last suitable snapshot which can be used to download ubuntu packages. This is for example use to build community ISO [1]. This should help in case of snapshot switch [2]. [1] https://github.com/fuel-infra/jenkins-jobs/blob/master/servers/fuel-ci/8.0/builders/community.all.sh#L36-L40 [2] https://bugs.launchpad.net/fuel/+bug/1493390/comments/9 Change-Id: Id5a8d41278561473ea4844631bfefa0049e64dbc Signed-off-by: Michal Skalski <mskalski@mirantis.com>
2016-01-21Turn on fuelmenu by default, turn off biosdevnameStefan K. Berg1-6/+7
Added experimental Fuel build loop, rebuilding up to ten times while keeping the package cache, in the hope that all packages will be able to download. Added snapshot support into this script as well. Changed naming of OPNFV iso label to OPNFV_FUEL_<id>, made changes in ks.cfg and isolinux.cfg to look for this name at install. Change-Id: Ic28a3af769cfc90416d7d988afd6f07eadcef4f1 Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
2016-01-19Revert "Fix build failure in fuel base & qemu"Stefan K. Berg1-1/+0
This reverts commit 0a35e64026402a672af343427bf591142babb6be.
2016-01-18Correction of cache handling for ISO buildStefan K. Berg1-3/+5
The cache id was not generated considering config.mk nor Makefile. Change-Id: I29ae0d60b89d4653cb38cffe83f666c5056daf63 Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
2016-01-17Merge "Fuel 8 uplift"Stefan K. Berg1-5/+4
2016-01-17Fuel 8 upliftStefan K. Berg1-5/+4
Introducing Fuel 8 with associated changes: Improvements in usage of cgroups - they are now bind mounted from the host which solved a Docker-in-Docker problem. The cache tool (cache.mk) has a new argument "getbiweek" which is used by the top Makefile when building the Fuel ISO. This makes sure that there's a fresh build every second week even if the cache is not purged automatically based on its meta data. This is something we should add everywhere (in a separate change set). And obviously, Fuel is now building from stable/8.0 upstream. NOTE! f_repobuild has been disabled for now, it throws an error when run. Will be fixed separately. Change-Id: I7e2ee737fd3bedc75f58539fbc3c5fd1fa08cfa1 Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
2016-01-13Fix build failure in fuel base & qemuRuijing Guo1-0/+1
Change-Id: I4a6cc047011547402641619a5683af3d24c057ee Signed-off-by: Ruijing Guo <ruijing.guo@intel.com>
2015-12-18Uplift of Docker versionsStefan K. Berg1-0/+4
The Docker version 1.5.0 is no longer allowed to acces the Docker remote image repository! Switching to version 1.7.1 instead. Change-Id: Id71cab4b26a02e98a3bb51f31caa394bcf25b7d9 Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
2015-12-03Support for selecting which plugins to buildStefan K. Berg1-0/+6
In order to cut the build time for unofficial builds, this change supports setting the environment variable "BUILD_FUEL_PLUGINS" prior to building. Only the plugin targets from fuel/build/f_isoroot/Makefile that are specified will then be built. In order to completely disable the building of plugins, the environment variable is set to " ". When using this functionality, the resulting iso file will be prepended with the prefix "unofficial-" to clearly indiciate that this is not a full build. Change-Id: Ib0a914ed6577f54f31b056a752691316302081dc Signed-off-by: Stefan K. Berg <stefan.k.berg@ericsson.com>
2015-11-27Restructcture of the directory layoutJonas Bjurel1-0/+197
Restructure of the directory layout due to move of Fuel into it's own repo JIRA: FUEL-85 Change-Id: I3647e1992a508f29dce06a5d6c790725c527f6f5 Signed-off-by: Jonas Bjurel <jonas.bjurel@ericsson.com>