build: Rework patch mechanism for Fuel submodules

While refactoring the patching mechanism, take care of:
- Sync submodule handling with Fuel@OPFNV;
- build: Investigate/prepare for moving patches to Fuel@OPNFV;
- build: Investigate divergent fuel-mirror;
- ISO build: cacheid for Fuel comps should not depend on
  Armband git commit;

CHANGE:
 Rename/shuffle patches while grouping them in "features",
 preparing for upstreaming them to Fuel@OPNFV and beyond.

CHANGE:
 Allow linking patches for better representing the dependency
 between one patch and different features.
 e.g. 0001-Add-arch-to-nailgun-release-and-target-image.patch:
   - part of `multiarch-fuel`, because it extends Fuel;
   - part of `direct-kernel-boot`, as arch is required for that;
   - part of `cross-bootstrap`, target image is arch-dependent;
 NOTE: Patch links are not staged to Fuel@OPNFV, they only serve
 as markers that a specific patch is part of a feature.

CHANGE:
 Kill all Fuel component submodules, now handled in Fuel@OPNFV:
  - fuel-agent
  - fuel-astute
  - fuel-library
  - fuel-mirror
  - fuel-nailgun-agent
  - fuel-web

CHANGE:
 Move armband-fuel-config.mk to armband git root.

FIXME: m1.micro-Increase-profile-RAM-size-to-128MB.patch is NOT
 part of `cross-bootstrap` feature, but patch context says so ...

FIXME: 0001-Add-arm64-deb-repositories-setup.patch is broken at
 `make patches-export` by removing spaces at EOL.

v2 -> v3:
 * Phony patch support (links to show a patch belongs to a feature);
 * Updated README.md
v3 -> v7:
 * Re-export Fuel submodules & plugins patches (update patch context);
 * Update Cavium mail addresses (s/caviumnetworks.com/cavium.com/);
 * Ignore submodule changes;
 * Add armband git repo info to gitinfo_fuel.txt at build time;

Implements: ARMBAND-136
Closes-bug: ARMBAND-95
Closes-bug: ARMBAND-93
Closes-bug: ARMBAND-92

Change-Id: I1a236d9f43b2e6dca22055911f696b43c22b5973
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>

2 files changed, 319 insertions, 0 deletions

diff --git a/patches/fuel-library/multiarch-efi/0001-Add-arm64-templates-to-cobbler-and-nailgun.patch b/patches/fuel-library/multiarch-efi/0001-Add-arm64-templates-to-cobbler-and-nailgun.patch
new file mode 100644
index 00000000..262bd994
--- /dev/null
+++ b/patches/fuel-library/multiarch-efi/0001-Add-arm64-templates-to-cobbler-and-nailgun.patch
@@ -0,0 +1,284 @@
+From: Stanislaw Kardach <stanislaw.kardach@cavium.com>
+Date: Sun, 21 Feb 2016 16:29:52 +0100
+Subject: [PATCH] Add arm64 templates to cobbler and nailgun
+
+Add grub template import to cobbler server manifest
+
+Add arm64 templates to cobbler and nailgun
+
+[Matei Valeanu] Add grub template to cobbler for MAC-files
+Limitation: grub2 needs to be installed directly on the
+root (/) of a device or partition, as the MAC-files will
+search for /grub/grub.cfg on the available partitions
+
+Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com>
+Signed-off-by: Matei Valeanu <Matei.Valeanu@enea.com>
+---
+ deployment/puppet/cobbler/manifests/server.pp      | 36 ++++++++++++++++++++++
+ .../puppet/cobbler/templates/dnsmasq.template.erb  |  2 ++
+ .../cobbler/templates/efidefault.template.erb      |  5 +++
+ .../cobbler/templates/grublocal.template.erb       |  2 ++
+ .../cobbler/templates/grubprofile.template.erb     | 12 ++++++++
+ .../cobbler/templates/grubsystem.template.erb      | 16 ++++++++++
+ .../templates/preseed/ubuntu-1404.preseed.erb      |  2 ++
+ deployment/puppet/fuel/manifests/auxiliaryrepos.pp | 12 +++++---
+ deployment/puppet/fuel/manifests/cobbler.pp        | 36 +++++++++++++++++-----
+ .../puppet/fuel/manifests/cobbler/preseed.pp       | 13 ++++++++
+ 10 files changed, 125 insertions(+), 11 deletions(-)
+ create mode 100644 deployment/puppet/cobbler/templates/efidefault.template.erb
+ create mode 100644 deployment/puppet/cobbler/templates/grublocal.template.erb
+ create mode 100644 deployment/puppet/cobbler/templates/grubprofile.template.erb
+ create mode 100644 deployment/puppet/cobbler/templates/grubsystem.template.erb
+ create mode 100644 deployment/puppet/fuel/manifests/cobbler/preseed.pp
+
+diff --git a/deployment/puppet/cobbler/manifests/server.pp b/deployment/puppet/cobbler/manifests/server.pp
+index 891dff7..52936d5 100644
+--- a/deployment/puppet/cobbler/manifests/server.pp
++++ b/deployment/puppet/cobbler/manifests/server.pp
+@@ -297,6 +297,42 @@ class cobbler::server (
+     notify  => [Service[$cobbler_service], Exec['cobbler_sync']],
+   }
+
++  file { '/etc/cobbler/pxe/efidefault.template':
++    content => template('cobbler/efidefault.template.erb'),
++    owner   => 'root',
++    group   => 'root',
++    mode    => '0644',
++    require => Package[$cobbler::packages::cobbler_package],
++    notify  => [Service[$cobbler_service], Exec['cobbler_sync']],
++  }
++
++  file { '/etc/cobbler/pxe/grubprofile.template':
++    content => template('cobbler/grubprofile.template.erb'),
++    owner   => 'root',
++    group   => 'root',
++    mode    => '0644',
++    require => Package[$cobbler::packages::cobbler_package],
++    notify  => [Service[$cobbler_service], Exec['cobbler_sync']],
++  }
++
++  file { '/etc/cobbler/pxe/grubsystem.template':
++    content => template('cobbler/grubsystem.template.erb'),
++    owner   => 'root',
++    group   => 'root',
++    mode    => '0644',
++    require => Package[$cobbler::packages::cobbler_package],
++    notify  => [Service[$cobbler_service], Exec['cobbler_sync']],
++  }
++
++  file { '/etc/cobbler/pxe/grublocal.template':
++    content => template('cobbler/grublocal.template.erb'),
++    owner   => 'root',
++    group   => 'root',
++    mode    => '0644',
++    require => Package[$cobbler::packages::cobbler_package],
++    notify  => [Service[$cobbler_service], Exec['cobbler_sync']],
++  }
++
+   exec { '/var/lib/tftpboot/chain.c32':
+     command => 'cp /usr/share/syslinux/chain.c32 /var/lib/tftpboot/chain.c32',
+     unless  => 'test -e /var/lib/tftpboot/chain.c32',
+diff --git a/deployment/puppet/cobbler/templates/dnsmasq.template.erb b/deployment/puppet/cobbler/templates/dnsmasq.template.erb
+index d70f8de..98278e9 100644
+--- a/deployment/puppet/cobbler/templates/dnsmasq.template.erb
++++ b/deployment/puppet/cobbler/templates/dnsmasq.template.erb
+@@ -41,5 +41,7 @@ dhcp-no-override
+ # by assigning IPs one by one instead of using hash algorithm.
+ dhcp-sequential-ip
+
++dhcp-match=set:arm64,93,11
++dhcp-boot=net:arm64,grub/grub-aarch64.efi,boothost,<%= @next_server %>
+ dhcp-option=6,<%= @name_server %>
+ $insert_cobbler_system_definitions
+diff --git a/deployment/puppet/cobbler/templates/efidefault.template.erb b/deployment/puppet/cobbler/templates/efidefault.template.erb
+new file mode 100644
+index 0000000..af09eb8
+--- /dev/null
++++ b/deployment/puppet/cobbler/templates/efidefault.template.erb
+@@ -0,0 +1,5 @@
++set default=1
++set timeout=5
++tr -s ipxe_mac ':' '-' $net_default_mac
++
++$grub_menu_items
+diff --git a/deployment/puppet/cobbler/templates/grublocal.template.erb b/deployment/puppet/cobbler/templates/grublocal.template.erb
+new file mode 100644
+index 0000000..e16c27d
+--- /dev/null
++++ b/deployment/puppet/cobbler/templates/grublocal.template.erb
+@@ -0,0 +1,2 @@
++search --file --no-floppy --set grub_install_device /grub/grub.cfg
++configfile ($grub_install_device)/grub/grub.cfg
+diff --git a/deployment/puppet/cobbler/templates/grubprofile.template.erb b/deployment/puppet/cobbler/templates/grubprofile.template.erb
+new file mode 100644
+index 0000000..eb7557c
+--- /dev/null
++++ b/deployment/puppet/cobbler/templates/grubprofile.template.erb
+@@ -0,0 +1,12 @@
++menuentry '$profile_name' {
++    echo 'Network status:'
++    net_ls_cards
++    net_ls_addr
++    net_ls_routes
++
++    echo 'Loading Linux ...'
++    linux (tftp)$kernel_path $kernel_options BOOTIF=01-$ipxe_mac
++    echo 'Loading initial ramdisk ...'
++    initrd (tftp)$initrd_path
++    echo 'Booting bootstrap image ...'
++}
+diff --git a/deployment/puppet/cobbler/templates/grubsystem.template.erb b/deployment/puppet/cobbler/templates/grubsystem.template.erb
+new file mode 100644
+index 0000000..9a191bc
+--- /dev/null
++++ b/deployment/puppet/cobbler/templates/grubsystem.template.erb
+@@ -0,0 +1,16 @@
++set default=1
++set timeout=5
++tr -s ipxe_mac ':' '-' $net_default_mac
++
++menuentry '$profile_name' {
++    echo 'Network status:'
++    net_ls_cards
++    net_ls_addr
++    net_ls_routes
++
++    echo 'Loading Linux ...'
++    linux (tftp)$kernel_path $kernel_options BOOTIF=01-$ipxe_mac
++    echo 'Loading initial ramdisk ...'
++    initrd (tftp)$initrd_path
++    echo 'Booting bootstrap image ...'
++}
+diff --git a/deployment/puppet/cobbler/templates/preseed/ubuntu-1404.preseed.erb b/deployment/puppet/cobbler/templates/preseed/ubuntu-1404.preseed.erb
+index 6762153..25f8067 100644
+--- a/deployment/puppet/cobbler/templates/preseed/ubuntu-1404.preseed.erb
++++ b/deployment/puppet/cobbler/templates/preseed/ubuntu-1404.preseed.erb
+@@ -66,9 +66,11 @@ d-i apt-setup/mirror/error select Retry
+ # Use network mirror
+ d-i apt-setup/use_mirror boolean true
+
++<% if @arch == 'amd64' %>
+ # Select architecture to amd64. That's very important to dpkg, since
+ # by default we have both amd64 and i386.
+ d-i apt-setup/multiarch string amd64
++<% end %>
+
+ # You can choose to install restricted and universe software, or to install
+ # software from the backports repository.
+diff --git a/deployment/puppet/fuel/manifests/auxiliaryrepos.pp b/deployment/puppet/fuel/manifests/auxiliaryrepos.pp
+index 493ffc6..909fc92 100644
+--- a/deployment/puppet/fuel/manifests/auxiliaryrepos.pp
++++ b/deployment/puppet/fuel/manifests/auxiliaryrepos.pp
+@@ -53,17 +53,21 @@ class fuel::auxiliaryrepos(
+
+   exec { 'create_ubuntu_repo_dirs':
+     path    => '/bin:/sbin:/usr/bin:/usr/sbin',
+-    command => "bash -c \"mkdir -p ${ubuntu_dir}/pool/{main,restricted} ${ubuntu_dir}/dists/auxiliary/{main,restricted}/binary-amd64/\"",
++    command => "bash -c \"mkdir -p ${ubuntu_dir}/pool/{main,restricted} ${ubuntu_dir}/dists/auxiliary/{main,restricted}/binary-{amd64,arm64}/\"",
+     unless  => "test -d ${ubuntu_dir}/pool && \
+       test -d ${ubuntu_dir}/dists/auxiliary/main/binary-amd64 && \
+-      test -d ${ubuntu_dir}/dists/auxiliary/restricted/binary-amd64",
++      test -d ${ubuntu_dir}/dists/auxiliary/restricted/binary-amd64 && \
++      test -d ${ubuntu_dir}/dists/auxiliary/main/binary-arm64 && \
++      test -d ${ubuntu_dir}/dists/auxiliary/restricted/binary-arm64",
+   }
+
+   exec { 'create_ubuntu_repo_Packages':
+     path    => '/bin:/sbin:/usr/bin:/usr/sbin',
+-    command => "bash -c \"touch ${ubuntu_dir}/dists/auxiliary/{main,restricted}/binary-amd64/Packages\"",
++    command => "bash -c \"touch ${ubuntu_dir}/dists/auxiliary/{main,restricted}/binary-{amd64,arm64}/Packages\"",
+     unless  => "test -f ${ubuntu_dir}/dists/auxiliary/main/binary-amd64/Packages && \
+-      test -f ${ubuntu_dir}/dists/auxiliary/restricted/binary-amd64/Packages",
++      test -f ${ubuntu_dir}/dists/auxiliary/restricted/binary-amd64/Packages && \
++      test -f ${ubuntu_dir}/dists/auxiliary/main/binary-arm64/Packages && \
++      test -f ${ubuntu_dir}/dists/auxiliary/restricted/binary-arm64/Packages",
+   }
+
+   file { $release_files:
+diff --git a/deployment/puppet/fuel/manifests/cobbler.pp b/deployment/puppet/fuel/manifests/cobbler.pp
+index b403733..c89027d 100644
+--- a/deployment/puppet/fuel/manifests/cobbler.pp
++++ b/deployment/puppet/fuel/manifests/cobbler.pp
+@@ -114,12 +114,9 @@ class fuel::cobbler(
+     require   => Class['::cobbler::server'],
+   }
+
+-  file { '/var/lib/cobbler/kickstarts/ubuntu-amd64.preseed':
+-    content => template('cobbler/preseed/ubuntu-1404.preseed.erb'),
+-    owner   => 'root',
+-    group   => 'root',
+-    mode    => '0644',
+-    require => Class['::cobbler::server'],
++  fuel::cobbler::preseed { 'ubuntu_1404_x86_64.preseed':
++    path =>'/var/lib/cobbler/kickstarts/ubuntu-amd64.preseed',
++    arch => "amd64",
+   } ->
+
+   cobbler_distro { 'ubuntu_1404_x86_64':
+@@ -132,6 +129,21 @@ class fuel::cobbler(
+     require   => Class['::cobbler::server'],
+   }
+
++  fuel::cobbler::preseed { 'ubuntu_1404_arm64.preseed':
++    path => '/var/lib/cobbler/kickstarts/ubuntu-arm64.preseed',
++    arch => "arm64",
++  } ->
++
++  cobbler_distro { 'ubuntu_1404_arm64':
++    kernel    => "${repo_root}/ubuntu/x86_64/images/linux",
++    initrd    => "${repo_root}/ubuntu/x86_64/images/initrd.gz",
++    arch      => 'x86_64', # Makes Cobbler choose PXE
++    breed     => 'ubuntu',
++    osversion => 'trusty',
++    ksmeta    => '',
++    require   => Class['::cobbler::server'],
++  }
++
+   cobbler_profile { 'centos-x86_64':
+     kickstart => '/var/lib/cobbler/kickstarts/centos-x86_64.ks',
+     kopts     => 'biosdevname=0 sshd=1 dhcptimeout=120',
+@@ -152,6 +164,16 @@ class fuel::cobbler(
+     require   => Cobbler_distro['ubuntu_1404_x86_64'],
+   }
+
++  cobbler_profile { 'ubuntu_1404_arm64':
++    kickstart => '/var/lib/cobbler/kickstarts/ubuntu-arm64.preseed',
++    kopts     => 'console=ttyAMA0,115200 console=ttyS0,115200 net.ifnames=0 biosdevname=0 netcfg/choose_interface=eth0 netcfg/dhcp_timeout=120 netcfg/link_detection_timeout=20',
++    distro    => 'ubuntu_1404_arm64',
++    ksmeta    => '',
++    menu      => false,
++    server    => $real_server,
++    require   => Cobbler_distro['ubuntu_1404_arm64'],
++  }
++
+   cobbler_distro { 'ubuntu_bootstrap':
+     kernel    => "${bootstrap_path}/vmlinuz",
+     initrd    => "${bootstrap_path}/initrd.img",
+@@ -166,7 +188,7 @@ class fuel::cobbler(
+     distro    => 'ubuntu_bootstrap',
+     menu      => true,
+     kickstart => '',
+-    kopts     => extend_kopts($bootstrap_meta['extend_kopts'], "console=ttyS0,9600 console=tty0 panic=60 ethdevice-timeout=${bootstrap_ethdevice_timeout} boot=live toram components fetch=http://${server}:8080/bootstraps/active_bootstrap/root.squashfs biosdevname=0 url=${nailgun_api_url} mco_user=${mco_user} mco_pass=${mco_pass} ip=frommedia"),
++    kopts     => extend_kopts($bootstrap_meta['extend_kopts'], "console=ttyAMA0,115200 console=ttyS0,115200 panic=60 ethdevice-timeout=${bootstrap_ethdevice_timeout} boot=live toram components fetch=http://${server}:8080/bootstraps/active_bootstrap/root.squashfs biosdevname=0 url=${nailgun_api_url} mco_user=${mco_user} mco_pass=${mco_pass} ip=frommedia"),
+     ksmeta    => '',
+     server    => $real_server,
+     require   => Cobbler_distro['ubuntu_bootstrap'],
+diff --git a/deployment/puppet/fuel/manifests/cobbler/preseed.pp b/deployment/puppet/fuel/manifests/cobbler/preseed.pp
+new file mode 100644
+index 0000000..7e1bee9
+--- /dev/null
++++ b/deployment/puppet/fuel/manifests/cobbler/preseed.pp
+@@ -0,0 +1,13 @@
++define fuel::cobbler::preseed(
++  $path,
++  $arch = "amd64",
++) {
++
++  file { $path:
++    content => template('cobbler/preseed/ubuntu-1404.preseed.erb'),
++    owner   => 'root',
++    group   => 'root',
++    mode    => '0644',
++    require => Class['::cobbler::server'],
++  }
++}
diff --git a/patches/fuel-library/multiarch-efi/0002-Add-cobbler-grub-aarch64-to-cobbler-dependencies.patch b/patches/fuel-library/multiarch-efi/0002-Add-cobbler-grub-aarch64-to-cobbler-dependencies.patch
new file mode 100644
index 00000000..14d06ef5
--- /dev/null
+++ b/patches/fuel-library/multiarch-efi/0002-Add-cobbler-grub-aarch64-to-cobbler-dependencies.patch
@@ -0,0 +1,35 @@
+From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+Date: Fri, 13 May 2016 23:06:28 +0200
+Subject: [PATCH] Add cobbler-grub-aarch64 to cobbler dependencies.
+
+The aarch64 grub EFI loader, packaged as an RPM and previously
+installed in the nailgun docker container in /var/lib/cobbler,
+is now added as a cobbler dependency in puppet and installed
+on the Fuel master node.
+
+[Alexandru.Avadanii@enea.com]
+Reworked based on Florin's work for the docker container.
+
+Signed-off-by: Florin Dumitrascu <florin.dumitrascu@enea.com>
+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
+---
+ deployment/puppet/cobbler/manifests/packages.pp | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/deployment/puppet/cobbler/manifests/packages.pp b/deployment/puppet/cobbler/manifests/packages.pp
+index 267c18c..9782c4c 100644
+--- a/deployment/puppet/cobbler/manifests/packages.pp
++++ b/deployment/puppet/cobbler/manifests/packages.pp
+@@ -25,10 +25,10 @@ class cobbler::packages {
+       $pexpect_package     = 'pexpect'
+       case $::operatingsystemrelease {
+         /6.+/: {
+-          $cobbler_additional_packages = ['xinetd', 'tftp-server', 'syslinux', 'wget', 'python-ipaddr','fence-agents', 'bind-utils']
++          $cobbler_additional_packages = ['xinetd', 'tftp-server', 'syslinux', 'wget', 'python-ipaddr','fence-agents', 'bind-utils', 'cobbler-grub-aarch64']
+         }
+         /7.+/: {
+-          $cobbler_additional_packages = ['xinetd', 'tftp-server', 'syslinux', 'wget', 'python-ipaddr','fence-agents-all', 'bind-utils']
++          $cobbler_additional_packages = ['xinetd', 'tftp-server', 'syslinux', 'wget', 'python-ipaddr','fence-agents-all', 'bind-utils', 'cobbler-grub-aarch64']
+         }
+       }
+     }