diff options
80 files changed, 0 insertions, 7879 deletions
diff --git a/.gitmodules b/.gitmodules index a8cfda34..44cc6d80 100644 --- a/.gitmodules +++ b/.gitmodules @@ -2,17 +2,3 @@ path = upstream/fuel url = https://gerrit.opnfv.org/gerrit/fuel branch = master -[submodule "vswitchperf"] - path = upstream/vswitchperf - url = https://gerrit.opnfv.org/gerrit/vswitchperf.git - branch = master -[submodule "fuel-plugin-opendaylight"] - path = upstream/fuel-plugin-opendaylight - url = https://github.com/openstack/fuel-plugin-opendaylight.git - branch = master - ignore = all -[submodule "fuel-plugin-ovs"] - path = upstream/fuel-plugin-ovs - url = https://github.com/openstack/fuel-plugin-ovs - branch = master - ignore = all diff --git a/howto-dea.md b/howto-dea.md deleted file mode 100644 index 5506d62e..00000000 --- a/howto-dea.md +++ /dev/null @@ -1,322 +0,0 @@ -DEA: Deployment Environment Adapter ------------------------------------ - -This file has several sections, some of the sections are self describing: - -``` -title: Deployment Environment Adapter (DEA) -version: -comment: Config for LF POD1 - HA deployment with Ceph and Opendaylight -wanted_release: Kilo on Ubuntu 14.04 -``` - -## The Environment section - -environment: - name: opnfv - mode: ha # noha, no-ha? - net_segment_type: tun # ? - - -## The "Fuel" section - -``` -fuel: - ADMIN_NETWORK: # Static Fuel admin network configuration - cidr: 10.20.0.0/24 # this is the "fuelweb_admin" in the nodes - dhcp_gateway: 10.20.0.2 # nodes us this as default gateway - dhcp_pool_end: 10.20.0.254 # nodes get addresses from here - dhcp_pool_start: 10.20.0.3 # This is the Fuel server IP address - ipaddress: 10.20.0.2 - netmask: 255.255.0.0 # netmask for nodes (otherwise why is it - # needed if CIDR above? - DNS_DOMAIN: domain.tld # - DNS_SEARCH: domain.tld # - DNS_UPSTREAM: 8.8.8.8 # Fuel->Network Settings->Other->DNS Servers - FUEL_ACCESS: - password: admin - user: admin - HOSTNAME: opnfv - NTP1: 0.pool.ntp.org # Fuel->Newtok Setting->Other->NTP server list - NTP2: 1.pool.ntp.org - NTP3: 2.pool.ntp.org -``` - -It would make more sense if NTP was a list: - -``` - NTP: - - 0.pool.ntp.org - - 1.pool.ntp.org - - 2.pool.ntp.org -``` - -Now I don't know if NTP4 would be a valid key... - -## The "node" section - -Then there is the important "node" section: - -``` -node: - - id: 1 - interfaces: <some section describing interfaces> - transformations: <section describing what to do with the interfaces> - role: [controller|compute|cinder|...] - - id: 2 - interfaces: interfaces_1 - transformations: transformations_1 - role: ceph-osd,controller - ... - - id: n - ... -``` - -Now, the "interfaces" section could be something line: - -``` -node: - - id: 1 - interfaces: interfaces_1 - transformations: transformations_1 -``` - -## Interfaces - -In this case we would have a "section" called intefaces_apm, that looks like: - -``` -interfaces_1: - eth0: - - none # I made this up, I don't know if "none" is valid - eth1: - - fuelweb_admin - eth2: - - public - - storage # vlan 2010 - - management # vlan 2011 - - private # vlan 2012 -``` - -This is self describing in a way. eth1 is used as the interface for the -"fuel admin" network, while eth2 will be used for what Fuel calls "public", -"storage", "management" and "private" networks. These match the networks in -the "networks" tab in the Fuel dashboard. - -## Transformations - -For now we won't come into huge detail about the transformations, but they -seem to contain a list of "commands" issued to ovs-vsctl (Open vSwitch). -For example: - -``` -transformations_1: - transformations: - - ... - - action: add-port - bridge: br-mgmt - name: eth1.300 - - ... -``` - -Basically the deploy script will issue the command: - -``` - ovs-vsctl add-port br-mgmt eth1.300 -``` -(or ...eth1 tag=300) - -## The "network" section: - -The networks listed in each of the devices of the "interfaces_1" section -are defined in the "network" section. Inside the "network" section, there is -another section called "networks", with a list of networks defined: - -``` -network: - ... - networks: - - cidr: 192.168.0.0/24 - gateway: null - ip_ranges: - - - 192.168.0.1 - - 192.168.0.254 - meta: - cidr: 192.168.0.0/24 # TBD: Can it be different from previos CIDR? - # Is this the default value in the UI? - configurable: true # TBD, UI? - map_priority: 2 # TBD, UI? - name: management - notation: cidr # TBD, UI? - render_addr_mask: internal # TBD, UI? - render_type: cidr # TBD, UI stuff? - use_gateway: false # Only for public net, or for - vips: # TBD - - haproxy - - vrouter - vlan_start: 101 - name: management - vlan_start: 300 # must match transformations - - cidr: ... - ... -``` - -Let's take the "management" network as an example. Here we define the -netmask and several parameters that will look familiar when looking at the -"Networks" Fuel dashboard tab. The available keys: -- name: the name of the network -- cidr: the CIDR for this network -- gateway: an IP address (only for public network?) -- ip_ranges: a list with the IP ranges available to this network. -- vlan_start: When using vlan tagging, the first vlan tag -- meta: (explained below) - -The purpose of the "meta" key is less obvious here, and some of the data -appears to be redundant. My guess is that it is part of Fuel's user -interface. The CIDR here would be the default and "notation" is probably the -way it is displayed in the form field: - -- cidr: again the same CIDR as above [is this redundant? error prone?] -- configurable: boolean [?] -- map_priority: int [?] -- name: again the same name as above? -- notation: cidr [any other available keys?] -- use_gateway: boolean [apparently only "true "if an IP was given above] -- vips: This seems to be a list of "namespaces" defined later in the -- "network section". -- vlan_start same as above... - -Now if we look back, in the "interfaces_1" section we had this: - -``` -interfaces_1: - eth2: - - management -``` - -This is clearly the network defined above. The same goes for "public", -"storage" and "private". - -## The "network" section continued - -Apart from the definition of each of the networks and required by Fuel, -the "network" section also has a "preamble" with the following parameters -and corresponding setting in Fuel: - -``` -network: - management_vip: 192.168.0.2 # TBD (see vips) - management_vrouter_vip: 192.168.0.1 # TBD - public_vip: 172.30.9.64 # TBD - public_vrouter_vip: 172.30.9.65 # TBD - networking_parameters: # Fuel->Networ->Settings - base_mac: fa:16:3e:00:00:00 # Neutron L2 - configuration_template: null - dns_nameservers: # Neutron L3, guess OS DNS Servers - - 8.8.4.4 - - 8.8.8.8 - floating_ranges: # Neutron L3, floating Network Param - - - 172.30.9.160 # floating IP range start - - 172.30.9.254 # floating IP range end - gre_id_range: # Neutron L2, what if VXLAN? - - 2 # Neutron L2, tunnel ID range start - - 65535 # Neutron L2, tunnel ID range end - # Neutron L3, Internal Network - # Parameters - internal_cidr: 192.168.111.0/24 # internal network CIDR - internal_gateway: 192.168.111.1 # internal network gateway - net_l23_provider: ovs # TBD: must match transformations? - segmentation_type: tun # TBD: what options are there? tun/vlan? - vlan_range: # TBD - - 1000 - - 1030 - vips: - ... -``` - -## The "vips" in the "network" section - -In addition to all the above, the network section contains a "vips" section. -I don't know what they mean, but there are some relations between these -vips, and the networks defined above: - -``` -network: - vips: - management: - ipaddr: 192.168.0.2 # TBD: same as management_vip? - namespace: haproxy # TBD: network namespace? - network_role: mgmt/vip # TBD - node_roles: - - controller # Why do we define it here? - - primary-controller # for an HA environment? - public: - ... - vrouter: - ... - vrouter_pub: - ... -``` - -Also, in contrast to the "networks" section, the "vips" section is not a list, -but a series of records... - -Some Fuel plugins seem to look at this particular setup, one of the examples -in [2], absolute-dashboard-link.pp, reads: - -``` -$os_public_vip = $network_metadata['vips']['public']['ipaddr'] -``` - -If you remember from above, each network has a "metadata" section, this -matches the name of the variable $network_metadata. In that section there is -a "vips" section, that contains a list of "vips", and one of the vips is -"public", and one of the fields is "ipaddr". - -* [1] https://docs.mirantis.com/openstack/fuel/fuel-8.0/file-ref.html#fuel-file-reference-pages -* [2] https://wiki.openstack.org/wiki/Fuel/Plugins - - -## The "Settings" section - -This looks like user interface stuff and default settings. For instance: -settings: - -``` - editable: - ... - additional_components: - ceilometer: - description: If selected, Ceilometer component will be installed - label: Install Ceilometer - type: checkbox - value: false - weight: 40 -``` - -This is clearly the label "Install Ceilometer" in the Fuel web dashboard. - -This looks like an email label entry with the corresponding regex to -validate it: - -``` -settings: - editable: - access: - email: - description: Email address for Administrator - label: Email - regex: - error: Invalid email - source: ^\S+@\S+$ - type: text - value: admin@localhost - weight: 40 - ... -``` - -## Other - -I think most of it, specially the "settings" part, has been machine created. -It would be nice to recreate one of this files from a manual Fuel -deployment. diff --git a/patches/fuel-agent/0001-TEMP-bootstrap-etc-rc.local-force-mco-start.patch b/patches/fuel-agent/0001-TEMP-bootstrap-etc-rc.local-force-mco-start.patch deleted file mode 100644 index ba4c382c..00000000 --- a/patches/fuel-agent/0001-TEMP-bootstrap-etc-rc.local-force-mco-start.patch +++ /dev/null @@ -1,47 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Wed, 22 Feb 2017 21:43:48 +0100 -Subject: [PATCH] TEMP: bootstrap: /etc/rc.local: force mco start - -Temporary workaround for upstream bug: -"Mcollective fails to start in bootstrap of baremetal servers" [1] - -This should be reverted as soon as upstream merges a fix. - -[1] https://bugs.launchpad.net/fuel/newton/+bug/1656150 - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - contrib/fuel_bootstrap/files/xenial/etc/rc.local | 15 +++++++++++++++ - 1 file changed, 15 insertions(+) - -diff --git a/contrib/fuel_bootstrap/files/xenial/etc/rc.local b/contrib/fuel_bootstrap/files/xenial/etc/rc.local -index 9712ba7..12262e3 100755 ---- a/contrib/fuel_bootstrap/files/xenial/etc/rc.local -+++ b/contrib/fuel_bootstrap/files/xenial/etc/rc.local -@@ -2,3 +2,18 @@ - - # Perform fuel bootstrap configuration - /usr/bin/fix-configs-on-startup || /bin/true -+ -+# FIXME: Temporary workaround for: -+# https://bugs.launchpad.net/fuel/newton/+bug/1656150 -+set +e -+until false -+do -+ sleep 60 -+ /etc/init.d/mcollective status > /dev/null 2>&1 -+ if [ $? -eq 0 ] -+ then -+ break -+ else -+ /usr/bin/fix-configs-on-startup || /bin/true -+ fi -+done diff --git a/patches/fuel-agent/arm64-bug-fixes/0001-bootstrap-defaults-Add-lshw-package.patch b/patches/fuel-agent/arm64-bug-fixes/0001-bootstrap-defaults-Add-lshw-package.patch deleted file mode 100644 index 845d8bd2..00000000 --- a/patches/fuel-agent/arm64-bug-fixes/0001-bootstrap-defaults-Add-lshw-package.patch +++ /dev/null @@ -1,36 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Tue, 24 May 2016 16:42:21 +0200 -Subject: [PATCH] bootstrap defaults: Add lshw package. - -Until Ubuntu fixes the following bug on AArch64 [1]: -"AArch64: slow cpuinfo due to redundant loop" -we will be using a patched version of <lshw> from Armband MOS repos. - -This change enforces replacing the lshw from Ubuntu with our custom, -patched package. - -[1] https://bugs.launchpad.net/ubuntu/+source/lshw/+bug/1582181 - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/settings.yaml.sample | 1 + - 1 file changed, 1 insertion(+) - -diff --git a/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/settings.yaml.sample b/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/settings.yaml.sample ---- a/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/settings.yaml.sample -+++ b/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/settings.yaml.sample -@@ -24,5 +24,6 @@ - - live-boot - - live-boot-initramfs-tools -+ - lshw - - mc - - mcollective - - msmtp-mta diff --git a/patches/fuel-agent/cross-bootstrap/0001-Use-qemu-debootstrap-for-image-creation.patch b/patches/fuel-agent/cross-bootstrap/0001-Use-qemu-debootstrap-for-image-creation.patch deleted file mode 100644 index 97875f10..00000000 --- a/patches/fuel-agent/cross-bootstrap/0001-Use-qemu-debootstrap-for-image-creation.patch +++ /dev/null @@ -1,164 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Date: Thu, 25 Feb 2016 13:38:14 +0100 -Subject: [PATCH] Use qemu-debootstrap for image creation - -This commit adds qemu-debootstrap support and a command line argument -`target_arch` which can be used for choosing the architecture to build -the image for. The architecture detection from environment settings is -not yet implemented. ---- - .../fuel_bootstrap_cli/fuel_bootstrap/commands/build.py | 8 ++++++++ - .../fuel_bootstrap/utils/bootstrap_image.py | 3 ++- - debian/control | 2 ++ - fuel_agent/manager.py | 13 ++++++++++++- - fuel_agent/tests/test_build_utils.py | 4 ++-- - fuel_agent/utils/build.py | 2 +- - specs/fuel-agent.spec | 2 ++ - 7 files changed, 29 insertions(+), 5 deletions(-) - -diff --git a/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/commands/build.py b/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/commands/build.py -index b4e9a05..a22d319 100644 ---- a/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/commands/build.py -+++ b/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/commands/build.py -@@ -171,6 +171,14 @@ class BuildCommand(command.Command): - " by ssh still rejected by default! This password actual" - " only for tty login!"), - ) -+ parser.add_argument( -+ '--target_arch', -+ type=str, -+ choices=['arm64', 'amd64'], -+ help="Choose the target architecture for which image is built", -+ default="amd64" -+ ) -+ - return parser - - def take_action(self, parsed_args): -diff --git a/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/utils/bootstrap_image.py b/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/utils/bootstrap_image.py -index c09d421..a535075 100644 ---- a/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/utils/bootstrap_image.py -+++ b/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/utils/bootstrap_image.py -@@ -161,7 +161,8 @@ def make_bootstrap(data): - - LOG.info("Try to build image with data:\n%s", yaml.safe_dump(bootdata)) - -- opts = ['--data_driver', 'bootstrap_build_image'] -+ opts = ['--data_driver', 'bootstrap_build_image', -+ '--target_arch', data['target_arch']] - if data.get('image_build_dir'): - opts.extend(['--image_build_dir', data['image_build_dir']]) - -diff --git a/debian/control b/debian/control -index d24450c..e843980 100644 ---- a/debian/control -+++ b/debian/control -@@ -37,6 +37,8 @@ Pre-Depends: dpkg (>= 1.15.6~) - Depends: bzip2, - cloud-utils, - debootstrap, -+ qemu-user-static, -+ binfmt-support, - dmidecode, - ethtool, - gdisk, -diff --git a/fuel_agent/manager.py b/fuel_agent/manager.py -index df54f65..ba1ab78 100644 ---- a/fuel_agent/manager.py -+++ b/fuel_agent/manager.py -@@ -19,6 +19,7 @@ import signal - import tempfile - - from oslo_config import cfg - from oslo_log import log as logging -+from oslo_config import types - import six - import yaml -@@ -34,5 +35,7 @@ from fuel_agent.utils import md as mu - from fuel_agent.utils import utils - -+ArchType = types.String(choices=['amd64', 'arm64']) -+ - opts = [ - cfg.StrOpt( - 'nc_template_path', -@@ -192,6 +195,13 @@ cli_opts = [ - default='/tmp', - help='Directory where the image is supposed to be built', - ), -+ cfg.Opt( -+ 'target_arch', -+ default='amd64', -+ type=ArchType, -+ help='Architecture for which image will be built using ' -+ 'debootstrap', -+ ), - ] - - CONF = cfg.CONF -@@ -724,7 +734,8 @@ class Manager(object): - LOG.debug('Preventing services from being get started') - bu.suppress_services_start(chroot) - LOG.debug('Installing base operating system using debootstrap') -- bu.run_debootstrap(uri=uri, suite=suite, chroot=chroot, -+ bu.run_debootstrap(uri=uri, suite=suite, arch=CONF.target_arch, -+ chroot=chroot, - attempts=CONF.fetch_packages_attempts, - proxies=proxies.proxies, - direct_repo_addr=proxies.direct_repo_addr_list) -diff --git a/fuel_agent/tests/test_build_utils.py b/fuel_agent/tests/test_build_utils.py -index 54f79f9..0ec466f 100644 ---- a/fuel_agent/tests/test_build_utils.py -+++ b/fuel_agent/tests/test_build_utils.py -@@ -43,7 +43,7 @@ class BuildUtilsTestCase(unittest2.TestCase): - def test_run_debootstrap(self, mock_exec, mock_environ): - bu.run_debootstrap('uri', 'suite', 'chroot', 'arch', attempts=2) - mock_exec.assert_called_once_with( -- 'debootstrap', '--include={0}' -+ 'qemu-debootstrap', '--include={0}' - .format(','.join(bu.ADDITIONAL_DEBOOTSTRAP_PACKAGES)), - '--verbose', '--no-check-gpg', '--arch=arch', - 'suite', 'chroot', 'uri', attempts=2, env_variables={}) -@@ -54,7 +54,7 @@ class BuildUtilsTestCase(unittest2.TestCase): - bu.run_debootstrap('uri', 'suite', 'chroot', 'arch', eatmydata=True, - attempts=2) - mock_exec.assert_called_once_with( -- 'debootstrap', '--include={0}' -+ 'qemu-debootstrap', '--include={0}' - .format(','.join(bu.ADDITIONAL_DEBOOTSTRAP_PACKAGES)), - '--verbose', '--no-check-gpg', '--arch=arch', - '--include=eatmydata', 'suite', -diff --git a/fuel_agent/utils/build.py b/fuel_agent/utils/build.py -index b1ecc0f..2950ad8 100644 ---- a/fuel_agent/utils/build.py -+++ b/fuel_agent/utils/build.py -@@ -76,7 +76,7 @@ def run_debootstrap(uri, suite, chroot, arch='amd64', eatmydata=False, - env_vars['no_proxy'] = ','.join(direct_repo_addr) - LOG.debug('Setting no_proxy for: {0}'.format(env_vars['no_proxy'])) - -- cmds = ['debootstrap', -+ cmds = ['qemu-debootstrap', - '--include={0}'.format(",".join(ADDITIONAL_DEBOOTSTRAP_PACKAGES)), - '--verbose', '--no-check-gpg', - '--arch={0}'.format(arch)] -diff --git a/specs/fuel-agent.spec b/specs/fuel-agent.spec -index 72cd6a1..18af4b9 100644 ---- a/specs/fuel-agent.spec -+++ b/specs/fuel-agent.spec -@@ -50,6 +50,8 @@ Requires: xfsprogs - Requires: pciutils - Requires: ethtool - Requires: debootstrap -+Requires: dpkg -+Requires: qemu-user-static - Requires: xz - Requires: coreutils - Requires: psmisc diff --git a/patches/fuel-agent/cross-bootstrap/0002-Add-FLASH_KERNEL_SKIP-true.patch b/patches/fuel-agent/cross-bootstrap/0002-Add-FLASH_KERNEL_SKIP-true.patch deleted file mode 100644 index d8cd0902..00000000 --- a/patches/fuel-agent/cross-bootstrap/0002-Add-FLASH_KERNEL_SKIP-true.patch +++ /dev/null @@ -1,29 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Thu, 25 Feb 2016 16:01:18 +0100 -Subject: [PATCH] Add FLASH_KERNEL_SKIP=true. - -FIXME: Add nice description of the issue at hand. ---- - fuel_agent/utils/build.py | 1 + - 1 file changed, 1 insertion(+) - -diff --git a/fuel_agent/utils/build.py b/fuel_agent/utils/build.py -index 2950ad8..74cbfd8 100644 ---- a/fuel_agent/utils/build.py -+++ b/fuel_agent/utils/build.py -@@ -93,6 +93,7 @@ def set_apt_get_env(): - # NOTE(agordeev): disable any confirmations/questions from apt-get side - os.environ['DEBIAN_FRONTEND'] = 'noninteractive' - os.environ['DEBCONF_NONINTERACTIVE_SEEN'] = 'true' -+ os.environ['FLASH_KERNEL_SKIP'] = 'true' - os.environ['LC_ALL'] = os.environ['LANG'] = os.environ['LANGUAGE'] = 'C' - - diff --git a/patches/fuel-agent/cross-bootstrap/0003-Fix-qemu-user-static-replacement.patch b/patches/fuel-agent/cross-bootstrap/0003-Fix-qemu-user-static-replacement.patch deleted file mode 100644 index dd406c6b..00000000 --- a/patches/fuel-agent/cross-bootstrap/0003-Fix-qemu-user-static-replacement.patch +++ /dev/null @@ -1,63 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Date: Sun, 6 Mar 2016 16:09:39 +0100 -Subject: [PATCH] Fix qemu-user-static replacement - ---- - fuel_agent/manager.py | 6 ++++++ - fuel_agent/utils/build.py | 18 ++++++++++++++++++ - 2 files changed, 24 insertions(+) - -diff --git a/fuel_agent/manager.py b/fuel_agent/manager.py -index ba1ab78..637c99a 100644 ---- a/fuel_agent/manager.py -+++ b/fuel_agent/manager.py -@@ -760,6 +760,12 @@ class Manager(object): - fu.mount_bind(chroot, '/proc') - bu.populate_basic_dev(chroot) - -+ # we need to make sure that qemu-{target_arch}-static binary is -+ # not replaced inside chroot because we're possibly using it -+ # through qemu-debootstrap -+ LOG.debug('Preventing qemu-user-static replacement inside chroot') -+ bu.prevent_qemu_replacement(chroot, CONF.target_arch) -+ - def destroy_chroot(self, chroot): - # Umount chroot tree and remove images tmp files - if not bu.stop_chrooted_processes(chroot, signal=signal.SIGTERM): -diff --git a/fuel_agent/utils/build.py b/fuel_agent/utils/build.py -index 74cbfd8..1bc0a5f 100644 ---- a/fuel_agent/utils/build.py -+++ b/fuel_agent/utils/build.py -@@ -320,6 +320,24 @@ def populate_basic_dev(chroot): - utils.execute('chroot', chroot, - 'ln', '-s', '/proc/self/fd', '/dev/fd') - -+def prevent_qemu_replacement(chroot, arch): -+ """Prevents qemu-user-static replacement inside chroot. -+ -+ Use dpkg-divert to prevent replacing qemu-user-static binary inside -+ chroot which is necessary for host-side qemu-debootstrap to work -+ properly.""" -+ if arch == 'arm64': -+ qemu = 'qemu-aarch64-static' -+ elif arch == 'amd64': -+ qemu = 'qemu-x86_64-static' -+ utils.execute('chroot', chroot, -+ 'dpkg-divert', '--divert', '/usr/bin/%s.orig' % qemu, -+ '/usr/bin/%s' % qemu) -+ utils.execute('chroot', chroot, -+ 'dpkg-divert', '--divert', '/usr/sbin/update-binfmts.orig', -+ '--rename', '/usr/sbin/update-binfmts') -+ utils.execute('chroot', chroot, 'ln', '-sf', '/bin/true', -+ '/usr/sbin/update-binfmts') - - def create_sparse_tmp_file(dir, suffix, size=8192): - """Creates sparse file. diff --git a/patches/fuel-agent/cross-bootstrap/0004-Prevent-common-cross-debootstrap-newaliases-issue.patch b/patches/fuel-agent/cross-bootstrap/0004-Prevent-common-cross-debootstrap-newaliases-issue.patch deleted file mode 100644 index ffe40b54..00000000 --- a/patches/fuel-agent/cross-bootstrap/0004-Prevent-common-cross-debootstrap-newaliases-issue.patch +++ /dev/null @@ -1,109 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Tue, 8 Mar 2016 00:44:05 +0100 -Subject: [PATCH] Prevent common cross-debootstrap newaliases issue. - -While building target images for a different arch (using qemu-user-static), -the following issue prevents the succesful image build: -newaliases: fatal: inet_addr_local[getifaddrs]: -getifaddrs: Address family not supported by protocol - -For more information, see [1]. - -[1] https://bugs.launchpad.net/ubuntu/+source/postfix/+bug/1531299 ---- - fuel_agent/manager.py | 14 ++++++++++++++ - fuel_agent/utils/build.py | 24 ++++++++++++++++++++++++ - 2 files changed, 38 insertions(+) - -diff --git a/fuel_agent/manager.py b/fuel_agent/manager.py -index 637c99a..86f76b9 100644 ---- a/fuel_agent/manager.py -+++ b/fuel_agent/manager.py -@@ -1037,10 +1037,15 @@ class Manager(object): - direct_repo_addrs=driver_os.proxies.direct_repo_addr_list) - self._update_metadata_with_repos( - metadata, driver_os.repos) -+ # Prevent common cross-debootstraping problem w/ newaliases & qemu -+ # inet_addr_local[getifaddrs]: getifaddrs: Address family not supp -+ LOG.debug('Preventing newaliases from running inside chroot') -+ bu.prevent_qemu_newaliases(chroot) - LOG.debug('Installing packages using apt-get: %s', - ' '.join(packages)) - # disable hosts/resolv files - bu.propagate_host_resolv_conf(chroot) -+ - if hasattr(bs_scheme, 'certs') and bs_scheme.certs: - bu.copy_update_certs(bs_scheme.certs, chroot) - bu.run_apt_get(chroot, packages=packages, -@@ -1084,5 +1089,6 @@ class Manager(object): - # restore disabled hosts/resolv files - bu.restore_resolv_conf(chroot) -+ bu.restore_newaliases(chroot) - metadata['all_packages'] = bu.get_installed_packages(chroot) - # We need to recompress initramfs with new compression: - bu.recompress_initramfs( -@@ -1175,6 +1181,11 @@ class Manager(object): - self._update_metadata_with_repos( - metadata, driver_os.repos) - -+ # Prevent common cross-debootstraping problem w/ newaliases & qemu -+ # inet_addr_local[getifaddrs]: getifaddrs: Address family not supp -+ LOG.debug('Preventing newaliases from running inside chroot') -+ bu.prevent_qemu_newaliases(chroot) -+ - LOG.debug('Installing packages using apt-get: %s', - ' '.join(packages)) - bu.run_apt_get(chroot, packages=packages, -@@ -1187,6 +1198,9 @@ class Manager(object): - allow_unsigned_file=CONF.allow_unsigned_file, - force_ipv4_file=CONF.force_ipv4_file) - -+ LOG.debug('Restoring newaliases command inside chroot') -+ bu.restore_newaliases(chroot) -+ - LOG.debug('Making sure there are no running processes ' - 'inside chroot before trying to umount chroot') - if not bu.stop_chrooted_processes(chroot, signal=signal.SIGTERM): -diff --git a/fuel_agent/utils/build.py b/fuel_agent/utils/build.py -index 1bc0a5f..5761cc5 100644 ---- a/fuel_agent/utils/build.py -+++ b/fuel_agent/utils/build.py -@@ -339,6 +339,30 @@ def prevent_qemu_replacement(chroot, arch): - utils.execute('chroot', chroot, 'ln', '-sf', '/bin/true', - '/usr/sbin/update-binfmts') - -+def prevent_qemu_newaliases(chroot): -+ """Prevents running newaliases under qemu-user-static inside chroot. -+ -+ Use dpkg-divert to prevent running newaliases binary through qemu inside -+ chroot which is necessary to avoid a dpkg --configure error: -+ inet_addr_local[getifaddrs]: getifaddrs: Address family not supported ... -+ """ -+ utils.execute('chroot', chroot, -+ 'dpkg-divert', '--local', '--rename', '--add', -+ '/usr/bin/newaliases') -+ utils.execute('chroot', chroot, 'ln', '-sf', '/bin/true', -+ '/usr/bin/newaliases') -+ -+def restore_newaliases(chroot): -+ """Restores newaliases inside chroot after previous dpkg-divert. -+ -+ opposite to prevent_qemu_newaliases -+ """ -+ utils.execute('chroot', chroot, -+ 'rm', '-f', '/usr/bin/newaliases') -+ utils.execute('chroot', chroot, -+ 'dpkg-divert', '--local', '--rename', '--remove', -+ '/usr/bin/newaliases') -+ - def create_sparse_tmp_file(dir, suffix, size=8192): - """Creates sparse file. - diff --git a/patches/fuel-agent/cross-bootstrap/0005-FIXME-Add-force-yes-to-apt-get-dist-upgrade.patch b/patches/fuel-agent/cross-bootstrap/0005-FIXME-Add-force-yes-to-apt-get-dist-upgrade.patch deleted file mode 100644 index 49e95735..00000000 --- a/patches/fuel-agent/cross-bootstrap/0005-FIXME-Add-force-yes-to-apt-get-dist-upgrade.patch +++ /dev/null @@ -1,39 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Fri, 8 Apr 2016 15:28:43 +0200 -Subject: [PATCH] FIXME: Add --force-yes to apt-get dist-upgrade. - -arm64 udev in Ubuntu Trusty is broken, so we had to provide our own -patched udev package in armband MOS repos. - -Due to dpkg version comparison algorithm, our MOS version of udev -is considered a downgrade, which requires --force-yes for -apt-get dist-upgrade to work and pick up this version, otherwise -bootstrap/target image build would fail with apt-get error code 100. - -This change can be dropped later, if other packages do not manifest -the same behavior. ---- - fuel_agent/utils/build.py | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - -diff --git a/fuel_agent/utils/build.py b/fuel_agent/utils/build.py -index 5761cc5..5557810 100644 ---- a/fuel_agent/utils/build.py -+++ b/fuel_agent/utils/build.py -@@ -107,7 +107,7 @@ def run_apt_get(chroot, packages, eatmydata=False, attempts=10): - time in X times. - """ - for action in ('update', 'dist-upgrade'): -- cmds = ['chroot', chroot, 'apt-get', '-y', action] -+ cmds = ['chroot', chroot, 'apt-get', '-y', '--force-yes', action] - stdout, stderr = utils.execute(*cmds, attempts=attempts) - LOG.debug('Running apt-get %s completed.\nstdout: %s\nstderr: %s', - action, stdout, stderr) diff --git a/patches/fuel-agent/cross-bootstrap/0006-UX-Update-bootstrap-target-build-time-estimate.patch b/patches/fuel-agent/cross-bootstrap/0006-UX-Update-bootstrap-target-build-time-estimate.patch deleted file mode 100644 index cba5f876..00000000 --- a/patches/fuel-agent/cross-bootstrap/0006-UX-Update-bootstrap-target-build-time-estimate.patch +++ /dev/null @@ -1,41 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Mon, 9 May 2016 17:08:06 +0200 -Subject: [PATCH] UX: Update bootstrap/target build time estimate. - -While building for a different architecture (e.g. AArch64 on x86_64), -the bootstrap/target image build may take longer, due to latency -introduced by using qemu-user-static. - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - .../fuel_bootstrap_cli/fuel_bootstrap/utils/bootstrap_image.py | 9 ++++++--- - 1 file changed, 6 insertions(+), 3 deletions(-) - -diff --git a/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/utils/bootstrap_image.py b/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/utils/bootstrap_image.py -index a535075..c7d67e9 100644 ---- a/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/utils/bootstrap_image.py -+++ b/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/utils/bootstrap_image.py -@@ -169,9 +169,12 @@ def make_bootstrap(data): - OSLO_CONF = cfg.CONF - OSLO_CONF(opts, project='fuel-agent') - mngr = manager.Manager(bootdata) -- LOG.info("Build process is in progress. Usually it takes 15-20 minutes." -- " It depends on your internet connection and hardware" -- " performance.") -+ LOG.info("Build process is in progress. Usually it takes 15-20 minutes for" -+ " a native build (x86_64) and/or 30-45 minutes for each" -+ " cross-build (e.g. AArch64)." -+ " It depends on your internet connection, hardware performance" -+ " and selected bootstrap architecture(s)." -+ " This ISO supports AArch64 only.") - mngr.do_mkbootstrap() - - return bootdata['bootstrap']['uuid'], bootdata['output'] diff --git a/patches/fuel-agent/kernel-bump/0001-bootstrap-defaults-Pin-kernel-to-4.8.0-9944.patch b/patches/fuel-agent/kernel-bump/0001-bootstrap-defaults-Pin-kernel-to-4.8.0-9944.patch deleted file mode 100644 index e9a88280..00000000 --- a/patches/fuel-agent/kernel-bump/0001-bootstrap-defaults-Pin-kernel-to-4.8.0-9944.patch +++ /dev/null @@ -1,50 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Tue, 14 Mar 2017 19:56:50 +0100 -Subject: [PATCH] bootstrap defaults: Pin kernel to 4.8.0-9944 - -Since newer kernels have issues with AAVMF + vhost_net, -we need to also temporarily blacklist vhost_net. - -TODO: Remove vhost_net blacklist (later). - -JIRA: https://jira.opnfv.org/browse/ARMBAND-161 -JIRA: https://jira.opnfv.org/browse/ARMBAND-171 -JIRA: https://jira.opnfv.org/browse/ARMBAND-227 -JIRA: https://jira.opnfv.org/browse/ARMBAND-230 -JIRA: https://jira.opnfv.org/browse/ARMBAND-239 -JIRA: https://jira.opnfv.org/browse/ARMBAND-244 - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/settings.yaml.sample | 5 +++-- - 1 files changed, 3 insertions(+), 2 deletions(-) - -diff --git a/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/settings.yaml.sample b/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/settings.yaml.sample -index ff5baa8..42765b6 100644 ---- a/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/settings.yaml.sample -+++ b/contrib/fuel_bootstrap/fuel_bootstrap_cli/fuel_bootstrap/settings.yaml.sample -@@ -13,13 +13,14 @@ - output_dir: /tmp/ - # Defines kernel version --kernel_flavor: linux-image-generic-lts-xenial -+kernel_flavor: linux-image-4.8.0-9944-generic - # Define packages list - packages: - - daemonize - - fuel-agent - - hwloc - - i40e-dkms - - linux-firmware -- - linux-headers-generic -+ - linux-image-extra-4.8.0-9944-generic -+ - linux-headers-4.8.0-9944-generic - - live-boot - - live-boot-initramfs-tools diff --git a/patches/fuel-agent/multiarch-efi/0001-Add-esp-partition-flag.patch b/patches/fuel-agent/multiarch-efi/0001-Add-esp-partition-flag.patch deleted file mode 100644 index 592cdd79..00000000 --- a/patches/fuel-agent/multiarch-efi/0001-Add-esp-partition-flag.patch +++ /dev/null @@ -1,56 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Tue, 8 Mar 2016 16:29:39 +0100 -Subject: [PATCH] Add <esp> partition flag. - -Parted flag <esp> (see [1]) will mark EFI system partition, allowing us -to properly format and mount it during do_partitioning deployment phase. - -[1] https://www.gnu.org/software/parted/manual/html_node/set.html ---- - fuel_agent/drivers/nailgun.py | 2 +- - fuel_agent/utils/partition.py | 4 ++-- - 2 files changed, 3 insertions(+), 3 deletions(-) - -diff --git a/fuel_agent/drivers/nailgun.py b/fuel_agent/drivers/nailgun.py -index ef0e79a..94280e8 100644 ---- a/fuel_agent/drivers/nailgun.py -+++ b/fuel_agent/drivers/nailgun.py -@@ -354,7 +354,7 @@ class Nailgun(BaseDataDriver): - # uefi partition (for future use) - LOG.debug('Adding UEFI partition on disk %s: size=200' % - disk['name']) -- parted.add_partition(size=200) -+ parted.add_partition(size=200, flags=['esp']) - - LOG.debug('Looping over all volumes on disk %s' % disk['name']) - for volume in disk['volumes']: -diff --git a/fuel_agent/utils/partition.py b/fuel_agent/utils/partition.py -index f7126b1..b811511 100644 ---- a/fuel_agent/utils/partition.py -+++ b/fuel_agent/utils/partition.py -@@ -94,7 +94,7 @@ def set_partition_flag(dev, num, flag, state='on'): - :param dev: A device file, e.g. /dev/sda. - :param num: Partition number - :param flag: Flag name. Must be one of 'bios_grub', 'legacy_boot', -- 'boot', 'raid', 'lvm' -+ 'boot', 'raid', 'lvm', 'esp' - :param state: Desiable flag state. 'on' or 'off'. Default is 'on'. - - :returns: None -@@ -104,7 +104,7 @@ def set_partition_flag(dev, num, flag, state='on'): - # parted supports more flags but we are interested in - # setting only this subset of them. - # not all of these flags are compatible with one another. -- if flag not in ('bios_grub', 'legacy_boot', 'boot', 'raid', 'lvm'): -+ if flag not in ('bios_grub', 'legacy_boot', 'boot', 'raid', 'lvm', 'esp'): - raise errors.WrongPartitionSchemeError( - 'Unsupported partition flag: %s' % flag) - if state not in ('on', 'off'): diff --git a/patches/fuel-agent/multiarch-efi/0002-Add-fs-for-efi-partition.patch b/patches/fuel-agent/multiarch-efi/0002-Add-fs-for-efi-partition.patch deleted file mode 100644 index 064fcc58..00000000 --- a/patches/fuel-agent/multiarch-efi/0002-Add-fs-for-efi-partition.patch +++ /dev/null @@ -1,73 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Date: Tue, 8 Mar 2016 21:08:55 +0100 -Subject: [PATCH] Add fs for efi partition - ---- - debian/control | 1 + - fuel_agent/drivers/nailgun.py | 13 ++++++++++--- - specs/fuel-agent.spec | 1 + - 3 files changed, 12 insertions(+), 3 deletions(-) - -diff --git a/debian/control b/debian/control -index e843980..c763ab9 100644 ---- a/debian/control -+++ b/debian/control -@@ -40,5 +40,6 @@ Depends: bzip2, - qemu-user-static, - binfmt-support, - dmidecode, -+ dosfstools, - ethtool, - gdisk, -diff --git a/fuel_agent/drivers/nailgun.py b/fuel_agent/drivers/nailgun.py -index 94280e8..1463f22 100644 ---- a/fuel_agent/drivers/nailgun.py -+++ b/fuel_agent/drivers/nailgun.py -@@ -82,6 +82,9 @@ class Nailgun(BaseDataDriver): - # was already allocated on first matching volume - # or not - self._boot_partition_done = False -+ # this var states whether ESP partition was already -+ # allocated on the first matching volume or not -+ self._esp_partition_done = False - # this var is used as a flag that /boot fs - # has already been added. we need this to - # get rid of md over all disks for /boot partition. -@@ -351,10 +354,14 @@ class Nailgun(BaseDataDriver): - LOG.debug('Adding bios_grub partition on disk %s: size=24' % - disk['name']) - parted.add_partition(size=24, flags=['bios_grub']) -- # uefi partition (for future use) -- LOG.debug('Adding UEFI partition on disk %s: size=200' % -+ # uefi partition - added only once. -+ if self._is_boot_disk(disk) and not self._esp_partition_done: -+ LOG.debug('Adding UEFI partition on disk %s: size=200' % - disk['name']) -- parted.add_partition(size=200, flags=['esp']) -+ prt = parted.add_partition(size=200, flags=['esp']) -+ partition_scheme.add_fs(device=prt.name, mount='/boot/efi', -+ fs_type='vfat') -+ self._esp_partition_done = True - - LOG.debug('Looping over all volumes on disk %s' % disk['name']) - for volume in disk['volumes']: -diff --git a/specs/fuel-agent.spec b/specs/fuel-agent.spec -index 18af4b9..8604e57 100644 ---- a/specs/fuel-agent.spec -+++ b/specs/fuel-agent.spec -@@ -50,6 +50,7 @@ Requires: xfsprogs - Requires: pciutils - Requires: ethtool - Requires: debootstrap -+Requires: dosfstools - Requires: dpkg - Requires: qemu-user-static - Requires: xz diff --git a/patches/fuel-astute/multiarch-efi/0001-Erase-EFI-boot-entry-on-EFI-systems.patch b/patches/fuel-astute/multiarch-efi/0001-Erase-EFI-boot-entry-on-EFI-systems.patch deleted file mode 100644 index 05cc2361..00000000 --- a/patches/fuel-astute/multiarch-efi/0001-Erase-EFI-boot-entry-on-EFI-systems.patch +++ /dev/null @@ -1,51 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Mon, 18 Jul 2016 16:02:20 +0200 -Subject: [PATCH] Erase EFI boot entry on EFI systems. - -On EFI-enabled systemd, grub-install from grub-efi-* package -installs a boot entry named "ubuntu". - -Since this boot entry is saved in board flash memory, erasing -the MBR bootloader code and/or partition signature will not -clear the ubuntu boot entry, leaving it pointing to a loader -on the ESP (EFI System Partition) that will try to load the -kernel/initrd from an erased partition. - -In Fuel 8.0, the whole disk was erased, so the ubuntu EFI boot -entry was skipped due to missing ESP, while for Fuel 9.0 and -above we have to explicitly remove it to keep the system in a -sane state. - -Note: efibootmgr is installed automatically on EFI systems -as a dependency of grub-efi-*. - -Closes: ARMBAND-47 - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - mcagents/erase_node.rb | 4 ++++ - 1 file changed, 4 insertions(+) - -diff --git a/mcagents/erase_node.rb b/mcagents/erase_node.rb -index 3eb98e8..3bac36f 100644 ---- a/mcagents/erase_node.rb -+++ b/mcagents/erase_node.rb -@@ -112,6 +112,10 @@ module MCollective - File.open('/proc/sys/kernel/panic','w') {|file| file.write("10\n")} - - begin -+ # clear out EFI boot entry on EFI-enabled systems -+ system("(which efibootmgr > /dev/null 2>&1 && efibootmgr | "\ -+ "grep -oP '(?<=Boot)[0-9]+(?=.*ubuntu)' | "\ -+ "xargs -I{} efibootmgr --delete-bootnum --bootnum {}) || true") - get_devices(type='all').each do |dev| - debug_msg("erasing bootstrap code area in MBR of #{dev[:name]}") - # clear out the boot code in MBR diff --git a/patches/fuel-library/0001-puppet-database-Fix-Percona-XtraBackup-sync.patch b/patches/fuel-library/0001-puppet-database-Fix-Percona-XtraBackup-sync.patch deleted file mode 100644 index ba307520..00000000 --- a/patches/fuel-library/0001-puppet-database-Fix-Percona-XtraBackup-sync.patch +++ /dev/null @@ -1,62 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Dan Andresan <dan.andresan@enea.com> -Date: Fri, 20 Jan 2017 17:24:40 +0200 -Subject: [PATCH] puppet: database: Fix Percona XtraBackup sync - -Recently, Percona released XtraBackup 2.4.5 which considers that SSL -is enabled if it finds ssl-ca, ssl-cert, ssl-key set up in my.cnf, -even if ssl is set to false in the same config file. Furthermore, -it complains that ssl is deprecated on the stdout instead of stderr, -corrupting xbstream in the process. - -Because ssl was set to false, this patch delete all other ssl-* keys -from the config file, avoiding xtrabackup getting confused. - -See percona xtrabackup bugs here: -https://bugs.launchpad.net/percona-xtrabackup/+bug/1646480 -https://bugs.launchpad.net/percona-xtrabackup/+bug/1647340 - -Closes-bug: https://jira.opnfv.org/browse/ARMBAND-197 - -Signed-off-by: Dan Andresan <dan.andresan@enea.com> ---- - .../puppet/osnailyfacter/manifests/database/database.pp | 14 +++++++++++++- - 1 file changed, 13 insertions(+), 1 deletion(-) - -diff --git a/deployment/puppet/osnailyfacter/manifests/database/database.pp b/deployment/puppet/osnailyfacter/manifests/database/database.pp -index 4fca959..ce406d7 100644 ---- a/deployment/puppet/osnailyfacter/manifests/database/database.pp -+++ b/deployment/puppet/osnailyfacter/manifests/database/database.pp -@@ -253,12 +253,24 @@ class osnailyfacter::database::database { - package_name => $mysql_package_name, - } - -+ # remove ssl - not used and confuse Percona XtraBackup 2.4.5 -+ # see ARMBAND-197: https://jira.opnfv.org/browse/ARMBAND-197 -+ $percona_hotfix = { -+ 'mysqld' => { -+ 'ssl' => undef, -+ 'ssl-ca' => undef, -+ 'ssl-cert' => undef, -+ 'ssl-key' => undef -+ } -+ } -+ - # build our mysql options to be configured in my.cnf - $mysql_override_options = mysql_deepmerge( - $fuel_override_options, - $ignore_db_dir_options, - $binary_logs_options, -- $syslog_options -+ $syslog_options, -+ $percona_hotfix - ) - $galera_options = mysql_deepmerge($wsrep_options, $vendor_override_options) - $override_options = mysql_deepmerge($mysql_override_options, $galera_options) diff --git a/patches/fuel-library/arm64-bug-fixes/0001-nova-config-Disable-usb-tablet-for-aarch64.patch b/patches/fuel-library/arm64-bug-fixes/0001-nova-config-Disable-usb-tablet-for-aarch64.patch deleted file mode 100644 index ae90eb4b..00000000 --- a/patches/fuel-library/arm64-bug-fixes/0001-nova-config-Disable-usb-tablet-for-aarch64.patch +++ /dev/null @@ -1,38 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Date: Tue, 15 Mar 2016 11:33:52 +0100 -Subject: [PATCH] nova config: Disable usb tablet for aarch64 - -[ Alexandru Avadanii ] -Rebased for Newton: reworked use_usb_tablet using pointer_model. - -JIRA: ARMBAND-194 - -Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - deployment/puppet/openstack_tasks/manifests/roles/compute.pp | 4 ++++ - 1 file changed, 4 insertions(+) - -diff --git a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -index 5b247b0..0a1a1c7 100644 ---- a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -+++ b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -@@ -391,6 +391,10 @@ class openstack_tasks::roles::compute { - 'libvirt/live_migration_flag': value => 'VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_PERSIST_DEST'; - 'libvirt/block_migration_flag': value => 'VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_NON_SHARED_INC'; - 'DEFAULT/connection_type': value => 'libvirt'; -+ 'DEFAULT/pointer_model': value => $::architecture ? { -+ /(arm64|aarch64)/ => ps2mouse, -+ default => usbtablet, -+ }; - } - - # TODO (iberezovskiy): rework this option management once it's available in puppet-nova module diff --git a/patches/fuel-library/arm64-bug-fixes/0002-Install-vgabios-and-link-for-aarch64.patch b/patches/fuel-library/arm64-bug-fixes/0002-Install-vgabios-and-link-for-aarch64.patch deleted file mode 100644 index defbcf9b..00000000 --- a/patches/fuel-library/arm64-bug-fixes/0002-Install-vgabios-and-link-for-aarch64.patch +++ /dev/null @@ -1,41 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Tue, 15 Mar 2016 11:33:52 +0100 -Subject: [PATCH] Install vgabios (and link) for aarch64 - -vgabios is needed for standard VGA mode in AArch64 VMs, so -install it by default on AArch64 and create missing link -in </usr/share/qemu> to </usr/share/vgabios/vgabios.bin>. - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> ---- - deployment/puppet/openstack_tasks/manifests/roles/compute.pp | 8 ++++++++ - 1 file changed, 8 insertions(+) - -diff --git a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -index 4240126..b18fa7b 100644 ---- a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -+++ b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -@@ -409,6 +409,14 @@ class openstack_tasks::roles::compute { - libvirt_service_name => 'libvirt-bin', - virtlock_service_name => 'virtlockd', - virtlog_service_name => 'virtlogd', -+ } -> -+ package { 'vgabios': -+ ensure => present; -+ } -> -+ file { '/usr/share/qemu/vgabios-stdvga.bin': -+ ensure => link, -+ target => '/usr/share/vgabios/vgabios.bin', -+ replace => false, - } - - class { '::nova::migration::libvirt': diff --git a/patches/fuel-library/arm64-bug-fixes/0003-nova-Update-vga-console-defaults-for-armv7-aarch64.patch b/patches/fuel-library/arm64-bug-fixes/0003-nova-Update-vga-console-defaults-for-armv7-aarch64.patch deleted file mode 100644 index f3860196..00000000 --- a/patches/fuel-library/arm64-bug-fixes/0003-nova-Update-vga-console-defaults-for-armv7-aarch64.patch +++ /dev/null @@ -1,110 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Date: Tue, 22 Mar 2016 12:05:09 +0100 -Subject: [PATCH] nova: Update vga, console defaults for armv7, aarch64 - -Nova hardcodes default options for both video=cirrus and console. -armv7 and aarch64 VMs require video=vga, and since most applications -use PL011 serial driver for guests, adding console=ttyAMA0 is also -a nice UX addition. ---- - .../openstack/files/nova-libvirt-vga-console.patch | 40 ++++++++++++++++++++++ - .../openstack_tasks/manifests/roles/compute.pp | 19 ++++++++++ - 2 files changed, 59 insertions(+) - create mode 100644 deployment/puppet/openstack/files/nova-libvirt-vga-console.patch - -diff --git a/deployment/puppet/openstack/files/nova-libvirt-vga-console.patch b/deployment/puppet/openstack/files/nova-libvirt-vga-console.patch -new file mode 100644 -index 0000000..3594276 ---- /dev/null -+++ b/deployment/puppet/openstack/files/nova-libvirt-vga-console.patch -@@ -0,0 +1,40 @@ -+From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -+Date: Tue, 22 Mar 2016 12:05:09 +0100 -+Subject: [PATCH] Update vga, console defaults for armv7 and aarch64 -+ -+Nova hardcodes default options for both video=cirrus and console. -+armv7 and aarch64 VMs require video=vga, and since most applications -+use PL011 serial driver for guests, adding console=ttyAMA0 is also -+a nice UX addition. -+ -+Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> -+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -+--- -+ -+diff --git a/virt/libvirt/driver.py b/virt/libvirt/driver.py -+index 920a283..751b140 100644 -+--- a/virt/libvirt/driver.py -++++ b/virt/libvirt/driver.py -+@@ -3834,7 +3834,11 @@ -+ if virt_type == "xen": -+ guest.os_cmdline = "ro root=%s" % root_device_name -+ else: -++ guestarch = libvirt_utils.get_arch(image_meta) -+ guest.os_cmdline = ("root=%s %s" % (root_device_name, CONSOLE)) -++ if guestarch in (arch.ARMV7, arch.AARCH64): -++ # NOTE(armband): ARM v7/v8 use PL011 drv, add ttyAMA0 console -++ guest.os_cmdline += " console=ttyAMA0" -+ if virt_type == "qemu": -+ guest.os_cmdline += " no_timer_check" -+ if instance.ramdisk_id: -+@@ -3972,7 +3972,9 @@ class LibvirtDriver(driver.ComputeDriver): -+ video.type = 'xen' -+ elif CONF.libvirt.virt_type == 'parallels': -+ video.type = 'vga' -+- elif guestarch in (arch.PPC, arch.PPC64, arch.PPC64LE): -++ elif guestarch in (arch.ARMV7, arch.AARCH64, -++ arch.PPC, arch.PPC64, arch.PPC64LE): -++ # NOTE(armband): Added ARM v7/v8, same as on PPC. -+ # NOTE(ldbragst): PowerKVM doesn't support 'cirrus' be default -+ # so use 'vga' instead when running on Power hardware. -+ video.type = 'vga' -diff --git a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -index b18fa7b..2f0f2a8 100644 ---- a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -+++ b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -@@ -158,6 +158,12 @@ class openstack_tasks::roles::compute { - - include ::nova::params - -+ if ! defined(Package['patch']) { -+ package { 'patch': -+ ensure => 'present', -+ } -+ } -+ - case $::osfamily { - 'RedHat': { - # From legacy libvirt.pp -@@ -290,5 +296,6 @@ class openstack_tasks::roles::compute { - } - - $notify_on_state_change = 'vm_and_task_state' -+ $nova_path = '/usr/lib/python2.7/dist-packages/nova' - - class { '::nova': -@@ -311,5 +318,17 @@ class openstack_tasks::roles::compute { - rabbit_heartbeat_timeout_threshold => $rabbit_heartbeat_timeout_threshold, - rabbit_heartbeat_rate => $rabbit_heartbeat_rate, - os_region_name => $region_name, -+ } -> -+ # FIXME(armband): Workaround for missing arm defaults in nova libvirt driver -+ file { "${nova_path}/libvirt-vga-console.patch": -+ ensure => "file", -+ source => "puppet:///modules/openstack/nova-libvirt-vga-console.patch", -+ } -> -+ exec { 'nova libvirt driver patch arm defaults': -+ path => ['/usr/bin'], -+ command => "patch -p1 < ${nova_path}/libvirt-vga-console.patch", -+ unless => "patch -p1 -R -N --dry-run < ${nova_path}/libvirt-vga-console.patch", -+ cwd => $nova_path, -+ require => [Package['patch']], - } - diff --git a/patches/fuel-library/arm64-bug-fixes/0004-AArch64-nova-libvirt-Use-host-model-cpu.patch b/patches/fuel-library/arm64-bug-fixes/0004-AArch64-nova-libvirt-Use-host-model-cpu.patch deleted file mode 100644 index 6f5533fe..00000000 --- a/patches/fuel-library/arm64-bug-fixes/0004-AArch64-nova-libvirt-Use-host-model-cpu.patch +++ /dev/null @@ -1,43 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Thu, 19 Jan 2017 19:19:28 +0100 -Subject: [PATCH] AArch64: nova: libvirt: Use host-model cpu - -Closes-bug: https://jira.opnfv.org/browse/ARMBAND-193 - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - deployment/puppet/openstack_tasks/manifests/roles/compute.pp | 11 ++++++++++- - 1 file changed, 10 insertions(+), 1 deletion(-) - -diff --git a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -index a6f4729..f6e6698 100644 ---- a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -+++ b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -@@ -319,9 +319,18 @@ class openstack_tasks::roles::compute { - # guest OS by using a combination of CPU features and other parameters (such - # as CPUID level) that don't work. Until these issues are fixed, it's a good - # idea to avoid using host-model -+ # NOTE(armband): AArch64: Use host-model cpu - # http://libvirt.org/formatdomain.html#elementsCPU - # https://bugs.launchpad.net/mos/+bug/1618473 -- $libvirt_cpu_mode = 'none' -+ # https://jira.opnfv.org/browse/ARMBAND-193 -+ if str2bool($::is_virtual) { -+ $libvirt_cpu_mode = 'none' -+ } else { -+ $libvirt_cpu_mode = $::architecture ? { -+ /(arm64|aarch64)/ => 'host-model', -+ default => 'none', -+ } -+ } - - # Install / configure nova-compute - diff --git a/patches/fuel-library/arm64-bug-fixes/0005-nova-libvirt-fix-delete-instance-with-nvram.patch b/patches/fuel-library/arm64-bug-fixes/0005-nova-libvirt-fix-delete-instance-with-nvram.patch deleted file mode 100644 index e2d5f73b..00000000 --- a/patches/fuel-library/arm64-bug-fixes/0005-nova-libvirt-fix-delete-instance-with-nvram.patch +++ /dev/null @@ -1,125 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Thu, 19 Jan 2017 23:03:54 +0100 -Subject: [PATCH] nova: libvirt: fix delete instance with nvram - -Backported from [1]. - -Closes-bug: 1567807 - -[1] https://review.openstack.org/#/q/ - 539d381434ccadcdc3f5d58c2705c35558a3a065 - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - .../files/nova-libvirt-delete-with-nvram.patch | 70 ++++++++++++++++++++++ - .../openstack_tasks/manifests/roles/compute.pp | 12 ++++ - 2 files changed, 82 insertions(+) - create mode 100644 deployment/puppet/openstack/files/nova-libvirt-delete-with-nvram.patch - -diff --git a/deployment/puppet/openstack/files/nova-libvirt-delete-with-nvram.patch b/deployment/puppet/openstack/files/nova-libvirt-delete-with-nvram.patch -new file mode 100644 -index 0000000..5d4f67c ---- /dev/null -+++ b/deployment/puppet/openstack/files/nova-libvirt-delete-with-nvram.patch -@@ -0,0 +1,70 @@ -+From: Kevin Zhao <kevin.zhao@linaro.org> -+Date: Thu, 5 Jan 2017 21:32:41 +0000 -+Subject: [PATCH] libvirt: fix nova can't delete the instance with nvram -+ -+Currently libvirt needs a flag when deleting an VM with a nvram file, -+without which nova can't delete an instance booted with UEFI. Add -+deletion flag for NVRAM. Also add a test case. -+ -+[ Alexandru Avadanii ] -+Removed chunks affecting tests and adapted for OPNFV Armband. -+ -+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -+Co-authored-by: Derek Higgins <derekh@redhat.com> -+Change-Id: I46baa952b6c3a1a4c5cf2660931f317cafb5757d -+Closes-Bug: #1567807 -+--- -+ -+diff --git a/virt/libvirt/driver.py b/virt/libvirt/driver.py -+--- a/virt/libvirt/driver.py -++++ b/virt/libvirt/driver.py -+@@ -903,7 +903,8 @@ class LibvirtDriver(driver.ComputeDriver): -+ try: -+ guest = self._host.get_guest(instance) -+ try: -+- guest.delete_configuration() -++ support_uefi = self._has_uefi_support() -++ guest.delete_configuration(support_uefi) -+ except libvirt.libvirtError as e: -+ with excutils.save_and_reraise_exception(): -+ errcode = e.get_error_code() -+@@ -1241,7 +1242,8 @@ class LibvirtDriver(driver.ComputeDriver): -+ # If any part of this block fails, the domain is -+ # re-defined regardless. -+ if guest.has_persistent_configuration(): -+- guest.delete_configuration() -++ support_uefi = self._has_uefi_support() -++ guest.delete_configuration(support_uefi) -+ -+ # Start copy with VIR_DOMAIN_REBASE_REUSE_EXT flag to -+ # allow writing to existing external volume file -+@@ -1760,7 +1762,8 @@ class LibvirtDriver(driver.ComputeDriver): -+ # If any part of this block fails, the domain is -+ # re-defined regardless. -+ if guest.has_persistent_configuration(): -+- guest.delete_configuration() -++ support_uefi = self._has_uefi_support() -++ guest.delete_configuration(support_uefi) -+ -+ # NOTE (rmk): Establish a temporary mirror of our root disk and -+ # issue an abort once we have a complete copy. -+diff --git a/virt/libvirt/guest.py b/virt/libvirt/guest.py -+--- a/virt/libvirt/guest.py -++++ b/virt/libvirt/guest.py -+@@ -262,11 +262,13 @@ class Guest(object): -+ yield VCPUInfo( -+ id=vcpu[0], cpu=vcpu[3], state=vcpu[1], time=vcpu[2]) -+ -+- def delete_configuration(self): -++ def delete_configuration(self, support_uefi=False): -+ """Undefines a domain from hypervisor.""" -+ try: -+- self._domain.undefineFlags( -+- libvirt.VIR_DOMAIN_UNDEFINE_MANAGED_SAVE) -++ flags = libvirt.VIR_DOMAIN_UNDEFINE_MANAGED_SAVE -++ if support_uefi: -++ flags |= libvirt.VIR_DOMAIN_UNDEFINE_NVRAM -++ self._domain.undefineFlags(flags) -+ except libvirt.libvirtError: -+ LOG.debug("Error from libvirt during undefineFlags. %d" -+ "Retrying with undefine", self.id) -diff --git a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -index f6e6698..dd0c034 100644 ---- a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -+++ b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -@@ -301,6 +301,18 @@ class openstack_tasks::roles::compute { - unless => "patch -p1 -R -N --dry-run < ${nova_path}/libvirt-vga-console.patch", - cwd => $nova_path, - require => [Package['patch']], -+ } -> -+ # FIXME(armband): Backport fix: nova delete instance with nvram -+ file { "${nova_path}/nova-libvirt-delete-with-nvram.patch": -+ ensure => "file", -+ source => "puppet:///modules/openstack/nova-libvirt-delete-with-nvram.patch", -+ } -> -+ exec { 'nova libvirt delete instance with nvram': -+ path => ['/usr/bin'], -+ command => "patch -p1 < ${nova_path}/nova-libvirt-delete-with-nvram.patch", -+ unless => "patch -p1 -R -N --dry-run < ${nova_path}/nova-libvirt-delete-with-nvram.patch", -+ cwd => $nova_path, -+ require => [Package['patch']], - } - - class { '::nova::cache': diff --git a/patches/fuel-library/arm64-bug-fixes/0006-FIXME-compute.pp-Temporarily-disable-apparmor.patch b/patches/fuel-library/arm64-bug-fixes/0006-FIXME-compute.pp-Temporarily-disable-apparmor.patch deleted file mode 100644 index 05c143dd..00000000 --- a/patches/fuel-library/arm64-bug-fixes/0006-FIXME-compute.pp-Temporarily-disable-apparmor.patch +++ /dev/null @@ -1,40 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Sun, 26 Mar 2017 20:14:55 +0200 -Subject: [PATCH] FIXME: compute.pp: Temporarily disable apparmor - -Apparmor profiles need to be refreshed for Armband specific -version of libvirt/qemu/etc.; until updated profiles land, -disable libvirt qemu security driver. - -JIRA: https://jira.opnfv.org/browse/ARMBAND-235 - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - deployment/puppet/openstack_tasks/manifests/roles/compute.pp | 4 +++- - 1 file changed, 3 insertions(+), 1 deletion(-) - -diff --git a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -index d81be2a..2844c72 100644 ---- a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -+++ b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -@@ -538,9 +538,11 @@ class openstack_tasks::roles::compute { - require => Package['apparmor'], - } - -+ # FIXME(armband): HACK: Disable apparmor until profiles are updated -+ # https://jira.opnfv.org/browse/ARMBAND-235 - file_line { 'qemu_apparmor': - path => '/etc/libvirt/qemu.conf', -- line => 'security_driver = "apparmor"', -+ line => 'security_driver = "none"', - require => [Package['libvirt'], Service['apparmor']], - notify => Service['libvirt'] - } diff --git a/patches/fuel-library/direct-kernel-boot/0002-nova-Fix-inject-for-direct-boot-with-part-table.patch b/patches/fuel-library/direct-kernel-boot/0002-nova-Fix-inject-for-direct-boot-with-part-table.patch deleted file mode 100644 index f84ed7d4..00000000 --- a/patches/fuel-library/direct-kernel-boot/0002-nova-Fix-inject-for-direct-boot-with-part-table.patch +++ /dev/null @@ -1,98 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Fri, 25 Mar 2016 16:12:08 +0100 -Subject: [PATCH] nova: Fix inject for direct boot with part table. - -See [1] for full bug description. -For now, we detect direct kernel boot with a partition table inside -disk image by passing target_partition when root kernel arg points -to a partition (instead of the whole disk). - -[1] https://bugs.launchpad.net/nova/+bug/1290455 - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> ---- - .../openstack/files/nova-libvirt-inject.patch | 44 ++++++++++++++++++++++ - .../openstack_tasks/manifests/roles/compute.pp | 12 ++++++ - 2 files changed, 56 insertions(+) - create mode 100644 deployment/puppet/openstack/files/nova-libvirt-inject.patch - -diff --git a/deployment/puppet/openstack/files/nova-libvirt-inject.patch b/deployment/puppet/openstack/files/nova-libvirt-inject.patch -new file mode 100644 -index 0000000..b86927e ---- /dev/null -+++ b/deployment/puppet/openstack/files/nova-libvirt-inject.patch -@@ -0,0 +1,44 @@ -+From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -+Date: Tue, 22 Mar 2016 12:05:09 +0100 -+Subject: [PATCH] Fix inject for direct boot with partition table -+ -+See [1] for full bug description. -+For now, we detect direct kernel boot with a partition table inside -+disk image by passing target_partition when root kernel arg points -+to a partition (instead of the whole disk). -+ -+[1] https://bugs.launchpad.net/nova/+bug/1290455 -+ -+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -+Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> -+--- -+ -+diff --git a/virt/libvirt/driver.py b/virt/libvirt/driver.py -+index 920a283..751b140 100644 -+--- a/virt/libvirt/driver.py -++++ b/virt/libvirt/driver.py -+@@ -34,6 +34,7 @@ -+ import mmap -+ import operator -+ import os -++import re -+ import shutil -+ import tempfile -+ import time -+@@ -2820,7 +2821,15 @@ -+ """ -+ # Handles the partition need to be used. -+ target_partition = None -+- if not instance.kernel_id: -++ image_meta = objects.ImageMeta.from_instance(instance) -++ # FIXME(armband): https://bugs.launchpad.net/nova/+bug/1290455 -++ # Support direct kernel boot with a partition table inside disk image: -++ # pass target_partition when root kernel arg points to a partition -++ # (instead of the whole disk). -++ has_partition_table = re.search("root=/dev/.d.([1-9]+)", -++ str(image_meta.properties.get("os_command_line"))) -++ -++ if not instance.kernel_id or has_partition_table: -+ target_partition = CONF.libvirt.inject_partition -+ if target_partition == 0: -+ target_partition = None -diff --git a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -index dc61033..3fcfb51 100644 ---- a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -+++ b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -@@ -319,4 +319,16 @@ class openstack_tasks::roles::compute { - } -> -+ # FIXME(armband): Fix inject for part image direct boot (bug LP #1469308) -+ file { "${nova_path}/libvirt-inject.patch": -+ ensure => "file", -+ source => "puppet:///modules/openstack/nova-libvirt-inject.patch", -+ } -> -+ exec { 'nova libvirt driver patch partitioned injection': -+ path => ['/usr/bin'], -+ command => "patch -p1 < ${nova_path}/libvirt-inject.patch", -+ unless => "patch -p1 -R -N --dry-run < ${nova_path}/libvirt-inject.patch", -+ cwd => $nova_path, -+ require => [Package['patch']], -+ } -> - # FIXME(armband): Workaround for missing arm defaults in nova libvirt driver - file { "${nova_path}/libvirt-vga-console.patch": - ensure => "file", diff --git a/patches/fuel-library/multiarch-efi/0001-Add-arm64-templates-to-cobbler-and-nailgun.patch b/patches/fuel-library/multiarch-efi/0001-Add-arm64-templates-to-cobbler-and-nailgun.patch deleted file mode 100644 index a7959627..00000000 --- a/patches/fuel-library/multiarch-efi/0001-Add-arm64-templates-to-cobbler-and-nailgun.patch +++ /dev/null @@ -1,208 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Date: Sun, 21 Feb 2016 16:29:52 +0100 -Subject: [PATCH] Add arm64 templates to cobbler and nailgun - -Add grub template import to cobbler server manifest - -Add arm64 templates to cobbler and nailgun - -[ Matei Valeanu ] -Add grub template to cobbler for MAC-files -Limitation: grub2 needs to be installed directly on the -root (/) of a device or partition, as the MAC-files will -search for /grub/grub.cfg on the available partitions - -[ Alexandru Avadanii ] -Rebased for Fuel Newton after upstream change: -"Remove deprecated cobbler parts for classic provisioning" [1]. - -[1] https://github.com/openstack/fuel-library/commit/ - e207593d7b3234cfbde6689dbccdcafe6ba67e58 - -Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Signed-off-by: Matei Valeanu <Matei.Valeanu@enea.com> -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - deployment/puppet/cobbler/manifests/server.pp | 36 ++++++++++++++++++++++ - .../puppet/cobbler/templates/dnsmasq.template.erb | 2 ++ - .../cobbler/templates/efidefault.template.erb | 5 +++ - .../cobbler/templates/grublocal.template.erb | 2 ++ - .../cobbler/templates/grubprofile.template.erb | 12 ++++++++ - .../cobbler/templates/grubsystem.template.erb | 16 ++++++++++ - deployment/puppet/fuel/manifests/auxiliaryrepos.pp | 12 +++++--- - deployment/puppet/fuel/manifests/cobbler.pp | 36 +++++++++++++++++----- - 10 files changed, 125 insertions(+), 11 deletions(-) - create mode 100644 deployment/puppet/cobbler/templates/efidefault.template.erb - create mode 100644 deployment/puppet/cobbler/templates/grublocal.template.erb - create mode 100644 deployment/puppet/cobbler/templates/grubprofile.template.erb - create mode 100644 deployment/puppet/cobbler/templates/grubsystem.template.erb - -diff --git a/deployment/puppet/cobbler/manifests/server.pp b/deployment/puppet/cobbler/manifests/server.pp -index 891dff7..52936d5 100644 ---- a/deployment/puppet/cobbler/manifests/server.pp -+++ b/deployment/puppet/cobbler/manifests/server.pp -@@ -297,6 +297,42 @@ class cobbler::server ( - notify => [Service[$cobbler_service], Exec['cobbler_sync']], - } - -+ file { '/etc/cobbler/pxe/efidefault.template': -+ content => template('cobbler/efidefault.template.erb'), -+ owner => 'root', -+ group => 'root', -+ mode => '0644', -+ require => Package[$cobbler::packages::cobbler_package], -+ notify => [Service[$cobbler_service], Exec['cobbler_sync']], -+ } -+ -+ file { '/etc/cobbler/pxe/grubprofile.template': -+ content => template('cobbler/grubprofile.template.erb'), -+ owner => 'root', -+ group => 'root', -+ mode => '0644', -+ require => Package[$cobbler::packages::cobbler_package], -+ notify => [Service[$cobbler_service], Exec['cobbler_sync']], -+ } -+ -+ file { '/etc/cobbler/pxe/grubsystem.template': -+ content => template('cobbler/grubsystem.template.erb'), -+ owner => 'root', -+ group => 'root', -+ mode => '0644', -+ require => Package[$cobbler::packages::cobbler_package], -+ notify => [Service[$cobbler_service], Exec['cobbler_sync']], -+ } -+ -+ file { '/etc/cobbler/pxe/grublocal.template': -+ content => template('cobbler/grublocal.template.erb'), -+ owner => 'root', -+ group => 'root', -+ mode => '0644', -+ require => Package[$cobbler::packages::cobbler_package], -+ notify => [Service[$cobbler_service], Exec['cobbler_sync']], -+ } -+ - exec { '/var/lib/tftpboot/chain.c32': - command => 'cp /usr/share/syslinux/chain.c32 /var/lib/tftpboot/chain.c32', - unless => 'test -e /var/lib/tftpboot/chain.c32', -diff --git a/deployment/puppet/cobbler/templates/dnsmasq.template.erb b/deployment/puppet/cobbler/templates/dnsmasq.template.erb -index d70f8de..98278e9 100644 ---- a/deployment/puppet/cobbler/templates/dnsmasq.template.erb -+++ b/deployment/puppet/cobbler/templates/dnsmasq.template.erb -@@ -41,5 +41,7 @@ dhcp-no-override - # by assigning IPs one by one instead of using hash algorithm. - dhcp-sequential-ip - -+dhcp-match=set:arm64,93,11 -+dhcp-boot=net:arm64,grub/grub-aarch64.efi,boothost,<%= @next_server %> - dhcp-option=6,<%= @name_server %> - $insert_cobbler_system_definitions -diff --git a/deployment/puppet/cobbler/templates/efidefault.template.erb b/deployment/puppet/cobbler/templates/efidefault.template.erb -new file mode 100644 -index 0000000..af09eb8 ---- /dev/null -+++ b/deployment/puppet/cobbler/templates/efidefault.template.erb -@@ -0,0 +1,5 @@ -+set default=1 -+set timeout=5 -+tr -s ipxe_mac ':' '-' $net_default_mac -+ -+$grub_menu_items -diff --git a/deployment/puppet/cobbler/templates/grublocal.template.erb b/deployment/puppet/cobbler/templates/grublocal.template.erb -new file mode 100644 -index 0000000..e16c27d ---- /dev/null -+++ b/deployment/puppet/cobbler/templates/grublocal.template.erb -@@ -0,0 +1,2 @@ -+search --file --no-floppy --set grub_install_device /grub/grub.cfg -+configfile ($grub_install_device)/grub/grub.cfg -diff --git a/deployment/puppet/cobbler/templates/grubprofile.template.erb b/deployment/puppet/cobbler/templates/grubprofile.template.erb -new file mode 100644 -index 0000000..eb7557c ---- /dev/null -+++ b/deployment/puppet/cobbler/templates/grubprofile.template.erb -@@ -0,0 +1,12 @@ -+menuentry '$profile_name' { -+ echo 'Network status:' -+ net_ls_cards -+ net_ls_addr -+ net_ls_routes -+ -+ echo 'Loading Linux ...' -+ linux (tftp)$kernel_path $kernel_options BOOTIF=01-$ipxe_mac -+ echo 'Loading initial ramdisk ...' -+ initrd (tftp)$initrd_path -+ echo 'Booting bootstrap image ...' -+} -diff --git a/deployment/puppet/cobbler/templates/grubsystem.template.erb b/deployment/puppet/cobbler/templates/grubsystem.template.erb -new file mode 100644 -index 0000000..9a191bc ---- /dev/null -+++ b/deployment/puppet/cobbler/templates/grubsystem.template.erb -@@ -0,0 +1,16 @@ -+set default=1 -+set timeout=5 -+tr -s ipxe_mac ':' '-' $net_default_mac -+ -+menuentry '$profile_name' { -+ echo 'Network status:' -+ net_ls_cards -+ net_ls_addr -+ net_ls_routes -+ -+ echo 'Loading Linux ...' -+ linux (tftp)$kernel_path $kernel_options BOOTIF=01-$ipxe_mac -+ echo 'Loading initial ramdisk ...' -+ initrd (tftp)$initrd_path -+ echo 'Booting bootstrap image ...' -+} -diff --git a/deployment/puppet/fuel/manifests/auxiliaryrepos.pp b/deployment/puppet/fuel/manifests/auxiliaryrepos.pp -index 493ffc6..909fc92 100644 ---- a/deployment/puppet/fuel/manifests/auxiliaryrepos.pp -+++ b/deployment/puppet/fuel/manifests/auxiliaryrepos.pp -@@ -53,17 +53,21 @@ class fuel::auxiliaryrepos( - - exec { 'create_ubuntu_repo_dirs': - path => '/bin:/sbin:/usr/bin:/usr/sbin', -- command => "bash -c \"mkdir -p ${ubuntu_dir}/pool/{main,restricted} ${ubuntu_dir}/dists/auxiliary/{main,restricted}/binary-amd64/\"", -+ command => "bash -c \"mkdir -p ${ubuntu_dir}/pool/{main,restricted} ${ubuntu_dir}/dists/auxiliary/{main,restricted}/binary-{amd64,arm64}/\"", - unless => "test -d ${ubuntu_dir}/pool && \ - test -d ${ubuntu_dir}/dists/auxiliary/main/binary-amd64 && \ -- test -d ${ubuntu_dir}/dists/auxiliary/restricted/binary-amd64", -+ test -d ${ubuntu_dir}/dists/auxiliary/restricted/binary-amd64 && \ -+ test -d ${ubuntu_dir}/dists/auxiliary/main/binary-arm64 && \ -+ test -d ${ubuntu_dir}/dists/auxiliary/restricted/binary-arm64", - } - - exec { 'create_ubuntu_repo_Packages': - path => '/bin:/sbin:/usr/bin:/usr/sbin', -- command => "bash -c \"touch ${ubuntu_dir}/dists/auxiliary/{main,restricted}/binary-amd64/Packages\"", -+ command => "bash -c \"touch ${ubuntu_dir}/dists/auxiliary/{main,restricted}/binary-{amd64,arm64}/Packages\"", - unless => "test -f ${ubuntu_dir}/dists/auxiliary/main/binary-amd64/Packages && \ -- test -f ${ubuntu_dir}/dists/auxiliary/restricted/binary-amd64/Packages", -+ test -f ${ubuntu_dir}/dists/auxiliary/restricted/binary-amd64/Packages && \ -+ test -f ${ubuntu_dir}/dists/auxiliary/main/binary-arm64/Packages && \ -+ test -f ${ubuntu_dir}/dists/auxiliary/restricted/binary-arm64/Packages", - } - - file { $release_files: -diff --git a/deployment/puppet/fuel/manifests/cobbler.pp b/deployment/puppet/fuel/manifests/cobbler.pp -index b403733..c89027d 100644 ---- a/deployment/puppet/fuel/manifests/cobbler.pp -+++ b/deployment/puppet/fuel/manifests/cobbler.pp -@@ -166,7 +188,7 @@ class fuel::cobbler( - distro => 'ubuntu_bootstrap', - menu => true, - kickstart => '', -- kopts => extend_kopts($bootstrap_meta['extend_kopts'], "console=ttyS0,9600 console=tty0 panic=60 ethdevice-timeout=${bootstrap_ethdevice_timeout} boot=live toram components fetch=http://${server}:8080/bootstraps/active_bootstrap/root.squashfs biosdevname=0 url=${nailgun_api_url} mco_user=${mco_user} mco_pass=${mco_pass} ip=frommedia"), -+ kopts => extend_kopts($bootstrap_meta['extend_kopts'], "console=ttyAMA0,115200 console=ttyS0,115200 panic=60 ethdevice-timeout=${bootstrap_ethdevice_timeout} boot=live toram components fetch=http://${server}:8080/bootstraps/active_bootstrap/root.squashfs biosdevname=0 url=${nailgun_api_url} mco_user=${mco_user} mco_pass=${mco_pass} ip=frommedia"), - ksmeta => '', - server => $real_server, - require => Cobbler_distro['ubuntu_bootstrap'], diff --git a/patches/fuel-library/multiarch-efi/0002-Add-cobbler-grub-aarch64-to-cobbler-dependencies.patch b/patches/fuel-library/multiarch-efi/0002-Add-cobbler-grub-aarch64-to-cobbler-dependencies.patch deleted file mode 100644 index c85ba0d2..00000000 --- a/patches/fuel-library/multiarch-efi/0002-Add-cobbler-grub-aarch64-to-cobbler-dependencies.patch +++ /dev/null @@ -1,38 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Fri, 13 May 2016 23:06:28 +0200 -Subject: [PATCH] Add cobbler-grub-aarch64 to cobbler dependencies. - -The aarch64 grub EFI loader, packaged as an RPM and previously -installed in the nailgun docker container in /var/lib/cobbler, -is now added as a cobbler dependency in puppet and installed -on the Fuel master node. - -[Alexandru.Avadanii@enea.com] -Reworked based on Florin's work for the docker container. -Rebased for Fuel Newton. - -Signed-off-by: Florin Dumitrascu <florin.dumitrascu@enea.com> -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - deployment/puppet/cobbler/manifests/packages.pp | 2 +- - 1 file changed, 1 insertions(+), 1 deletions(-) - -diff --git a/deployment/puppet/cobbler/manifests/packages.pp b/deployment/puppet/cobbler/manifests/packages.pp -index 267c18c..9782c4c 100644 ---- a/deployment/puppet/cobbler/manifests/packages.pp -+++ b/deployment/puppet/cobbler/manifests/packages.pp -@@ -23,5 +23,5 @@ class cobbler::packages { - $django_package = 'python-django' - $openssh_package = 'openssh-clients' - $pexpect_package = 'pexpect' -- $cobbler_additional_packages = ['xinetd', 'tftp-server', 'syslinux', 'wget', 'python-ipaddr','fence-agents-all', 'bind-utils'] -+ $cobbler_additional_packages = ['xinetd', 'tftp-server', 'syslinux', 'wget', 'python-ipaddr','fence-agents-all', 'bind-utils', 'cobbler-grub-aarch64'] - } diff --git a/patches/fuel-library/multiarch-fuel/0001-Make-qemu-kvm-architecture-aware.patch b/patches/fuel-library/multiarch-fuel/0001-Make-qemu-kvm-architecture-aware.patch deleted file mode 100644 index c5d3479a..00000000 --- a/patches/fuel-library/multiarch-fuel/0001-Make-qemu-kvm-architecture-aware.patch +++ /dev/null @@ -1,37 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Date: Wed, 24 Feb 2016 20:07:06 +0100 -Subject: [PATCH] Make qemu-kvm architecture aware - ---- - deployment/puppet/openstack_tasks/manifests/roles/compute.pp | 9 +++++++-- - 1 file changed, 7 insertions(+), 2 deletions(-) - -diff --git a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -index 3fcfb51..c53409c 100644 ---- a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -+++ b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -@@ -167,9 +167,14 @@ class openstack_tasks::roles::compute { - case $::osfamily { - 'RedHat': { - # From legacy libvirt.pp -+ # Guard against some exotic distros with their `uname -m` -+ $arch = $::architecture ? { -+ /(arm64|aarch64)/ => 'aarch64', -+ default => 'x86_64', -+ } - exec { 'symlink-qemu-kvm': -- command => '/bin/ln -sf /usr/libexec/qemu-kvm /usr/bin/qemu-system-x86_64', -- creates => '/usr/bin/qemu-system-x86_64', -+ command => "/bin/ln -sf /usr/libexec/qemu-kvm /usr/bin/qemu-system-${arch}", -+ creates => "/usr/bin/qemu-system-${arch}", - } - - package { 'avahi': diff --git a/patches/fuel-library/multiarch-fuel/0002-Make-TestVM-creation-architecture-aware.patch b/patches/fuel-library/multiarch-fuel/0002-Make-TestVM-creation-architecture-aware.patch deleted file mode 100644 index 1f4c05f2..00000000 --- a/patches/fuel-library/multiarch-fuel/0002-Make-TestVM-creation-architecture-aware.patch +++ /dev/null @@ -1,139 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Date: Wed, 24 Feb 2016 20:08:24 +0100 -Subject: [PATCH] Make TestVM creation architecture aware - -This depends on cirros-testvm package to contain both amd64 and arm64 -images. ---- - deployment/puppet/openstack/manifests/img/cirros.pp | 8 ++++++-- - .../osnailyfacter/manifests/generate_vms/vm_config.pp | 11 +++++++++++ - deployment/puppet/osnailyfacter/templates/vm_libvirt.erb | 15 ++++++++++++++- - files/fuel-migrate/fuel-migrate | 5 ++++- - 4 files changed, 35 insertions(+), 4 deletions(-) - -diff --git a/deployment/puppet/openstack/manifests/img/cirros.pp b/deployment/puppet/openstack/manifests/img/cirros.pp -index 84e202e..cff06e0 100644 ---- a/deployment/puppet/openstack/manifests/img/cirros.pp -+++ b/deployment/puppet/openstack/manifests/img/cirros.pp -@@ -9,6 +9,10 @@ class openstack::img::cirros ( - $img_name = 'cirros', - $os_name = 'cirros', - ) { -+ $arch = $::architecture ? { -+ /(arm64|aarch64)/ => 'aarch64', -+ default => 'x86_64', -+ } - - package { 'cirros-testvm': - ensure => "present" -@@ -17,14 +21,14 @@ class openstack::img::cirros ( - case $::osfamily { - 'RedHat': { - exec { 'upload-img': -- command => "/usr/bin/glance -N ${os_auth_url} -T ${os_tenant_name} -I ${os_username} -K ${os_password} add name=${img_name} is_public=${public} container_format=${container_format} disk_format=${disk_format} distro=${os_name} < /opt/vm/cirros-x86_64-disk.img", -+ command => "/usr/bin/glance -N ${os_auth_url} -T ${os_tenant_name} -I ${os_username} -K ${os_password} add name=${img_name} is_public=${public} container_format=${container_format} disk_format=${disk_format} distro=${os_name} < /opt/vm/cirros-${arch}-disk.img", - unless => "/usr/bin/glance -N ${os_auth_url} -T ${os_tenant_name} -I ${os_username} -K ${os_password} index && (/usr/bin/glance -N ${os_auth_url} -T ${os_tenant_name} -I ${os_username} -K ${os_password} index | grep ${img_name})", - - } - } - 'Debian': { - exec { 'upload-img': -- command => "/usr/bin/glance -N ${os_auth_url} -T ${os_tenant_name} -I ${os_username} -K ${os_password} add name=${img_name} is_public=${public} container_format=${container_format} disk_format=${disk_format} distro=${os_name} < /usr/share/cirros-testvm/cirros-x86_64-disk.img", -+ command => "/usr/bin/glance -N ${os_auth_url} -T ${os_tenant_name} -I ${os_username} -K ${os_password} add name=${img_name} is_public=${public} container_format=${container_format} disk_format=${disk_format} distro=${os_name} < /usr/share/cirros-testvm/cirros-${arch}-disk.img", - unless => "/usr/bin/glance -N ${os_auth_url} -T ${os_tenant_name} -I ${os_username} -K ${os_password} index && (/usr/bin/glance -N ${os_auth_url} -T ${os_tenant_name} -I ${os_username} -K ${os_password} index | grep ${img_name})", - - } -diff --git a/deployment/puppet/osnailyfacter/manifests/generate_vms/vm_config.pp b/deployment/puppet/osnailyfacter/manifests/generate_vms/vm_config.pp -index 70fef32..dfefcbe 100644 ---- a/deployment/puppet/osnailyfacter/manifests/generate_vms/vm_config.pp -+++ b/deployment/puppet/osnailyfacter/manifests/generate_vms/vm_config.pp -@@ -4,6 +4,17 @@ define osnailyfacter::generate_vms::vm_config( - $details, - $template_dir = '/var/lib/nova', - ) { -+ case $::architecture { -+ 'arm64', 'aarch64': { -+ $machine_arch = 'aarch64' -+ $machine_type = 'virt-2.6' -+ } -+ default: { -+ $machine_arch = 'x86_64' -+ $machine_type = 'pc' -+ } -+ } -+ - file { "${template_dir}/template_${name}_vm.xml": - owner => 'root', - group => 'root', -diff --git a/deployment/puppet/osnailyfacter/templates/vm_libvirt.erb b/deployment/puppet/osnailyfacter/templates/vm_libvirt.erb -index ae2ba5b..64e716d 100644 ---- a/deployment/puppet/osnailyfacter/templates/vm_libvirt.erb -+++ b/deployment/puppet/osnailyfacter/templates/vm_libvirt.erb -@@ -3,14 +3,17 @@ - <memory unit='GiB'><%= @details['mem'] %></memory> - <vcpu placement='static'><%= @details['cpu'] %></vcpu> - <os> -- <type arch='x86_64' machine='pc'>hvm</type> -+ <type arch='<%= @machine_arch %>' machine='<%= @machine_type %>'>hvm</type> - <boot dev='network'/> - <boot dev='hd'/> - </os> - <features> - <acpi/> - <apic/> - <pae/> -+ <% if $machine_arch == 'aarch64' %> -+ <gic version='host' /> -+ <% end %> - </features> - <clock offset='utc'/> - <on_poweroff>destroy</on_poweroff> -@@ -57,7 +66,11 @@ - <input type='keyboard' bus='ps2'/> - <graphics type='vnc' port='-1' autoport='yes'/> - <video> -+<% if $machine_arch == 'aarch64' %> -+ <model type='vga' vram='9216' heads='1'/> -+<% else %> - <model type='cirrus' vram='9216' heads='1'/> -+<% end %> - </video> - <memballoon model='virtio'> - </memballoon> -diff --git a/files/fuel-migrate/fuel-migrate b/files/fuel-migrate/fuel-migrate -index a215e0a..7d5b6cd 100755 ---- a/files/fuel-migrate/fuel-migrate -+++ b/files/fuel-migrate/fuel-migrate -@@ -176,6 +176,8 @@ create_vm(){ - exit 1 - fi - fi -+ # figure out atchitecture of the destination -+ arch="$(virsh -c "${dkvm_uri}" nodeinfo | awk '/CPU model:/ { print $3}')" - # create disk - ssh "${dkvm_ip}" qemu-img create -f raw "${dkvm_folder}/${fvm_name}.img" "${fvm_disk_size}" - # make template for virsh -@@ -185,13 +187,14 @@ create_vm(){ - <memory unit='MiB'>${fvm_ram}</memory> - <vcpu >${fvm_cpu}</vcpu> - <os> -- <type arch='x86_64' >hvm</type> -+ <type arch='${arch}' >hvm</type> - <boot dev='hd'/> - <boot dev='network'/> - </os> - <features> - <acpi/> - <apic/> -+ ${arch/aarch64/<gic version='3'\/>} - </features> - <clock offset='utc'/> - <on_poweroff>destroy</on_poweroff> diff --git a/patches/fuel-library/multiarch-fuel/0003-build_image.pp-Extract-cluster_arch-from-URI.patch b/patches/fuel-library/multiarch-fuel/0003-build_image.pp-Extract-cluster_arch-from-URI.patch deleted file mode 100644 index 2311442b..00000000 --- a/patches/fuel-library/multiarch-fuel/0003-build_image.pp-Extract-cluster_arch-from-URI.patch +++ /dev/null @@ -1,61 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Wed, 15 Mar 2017 23:44:32 +0100 -Subject: [PATCH] build_image.pp: Extract cluster_arch from "/" URI - -Since target image building was moved from nailgun to puppet, we -need to determine the "target_arch" value differently. - -One way to go is to extract it from the root disk image URI, which -is constructed based on static fixture data in nailgun. - -This change relies on the following: -- image_data contains "/", with a properly arch-qualified URI; -- image_data "/" URI arch uses the same format as "target_arch" - (in this case "arm64" instead of "aarch64" or other variations); - -While at it, increase task timeout from 1800s to 3600s. - -NOTE: Unlike the previous approach, this method does not rely on any -other nailgun changes (i.e. adding the "arch" field to the database -model), but only on `fa_build_image --target_arch=...` support. - -JIRA: https://jira.opnfv.org/browse/ARMBAND-240 - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - deployment/puppet/osnailyfacter/manifests/provision/build_image.pp | 7 +++++-- - 1 file changed, 5 insertions(+), 2 deletions(-) - -diff --git a/deployment/puppet/osnailyfacter/manifests/provision/build_image.pp b/deployment/puppet/osnailyfacter/manifests/provision/build_image.pp -index dbfc785..8525e33 100644 ---- a/deployment/puppet/osnailyfacter/manifests/provision/build_image.pp -+++ b/deployment/puppet/osnailyfacter/manifests/provision/build_image.pp -@@ -8,6 +8,9 @@ class osnailyfacter::provision::build_image( - $data = loadyaml($data_file) - } - $cluster_id = $data['cluster']['id'] -+ $root_img_uri_split = split($data['image_data']['/']['uri'], '_') -+ $root_img_suffix_split = split($root_img_uri_split[-1], '[.]') -+ $cluster_arch = $root_img_suffix_split[0] - - if $data['ironic']['enabled'] == true { - # TODO(vsaienko): Use the same system packages for fuel image and ironic bootstrap, but exclude -@@ -52,8 +55,8 @@ class osnailyfacter::provision::build_image( - $extra_params = '--data_driver nailgun_build_image' - - exec { 'generate_image_with_fuel': -- command => "fa_build_image ${build_dir} ${log_params} ${extra_params} --input_data_file ${data_file}", -+ command => "fa_build_image ${build_dir} ${log_params} ${extra_params} --target_arch=${cluster_arch} --input_data_file ${data_file}", - path => ['/bin', '/usr/bin'], -- timeout => 1800, -+ timeout => 3600, - } - } diff --git a/patches/fuel-library/multiarch-mirrors/0001-Add-arm64-for-auxiliary-repos.patch b/patches/fuel-library/multiarch-mirrors/0001-Add-arm64-for-auxiliary-repos.patch deleted file mode 100644 index f6fd0a60..00000000 --- a/patches/fuel-library/multiarch-mirrors/0001-Add-arm64-for-auxiliary-repos.patch +++ /dev/null @@ -1,44 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Date: Wed, 24 Feb 2016 20:05:25 +0100 -Subject: [PATCH] Add arm64 for auxiliary repos - ---- - deployment/puppet/fuel/files/Release-auxiliary | 2 +- - deployment/puppet/fuel/manifests/auxiliaryrepos.pp | 4 +++- - 2 files changed, 4 insertions(+), 2 deletions(-) - -diff --git a/deployment/puppet/fuel/files/Release-auxiliary b/deployment/puppet/fuel/files/Release-auxiliary -index 2d5ce58..03f84b8 100644 ---- a/deployment/puppet/fuel/files/Release-auxiliary -+++ b/deployment/puppet/fuel/files/Release-auxiliary -@@ -3,6 +3,6 @@ Label: auxiliary - Suite: auxiliary - Codename: auxiliary - Date: Mon, 15 Apr 2015 00:00:01 UTC --Architectures: amd64 -+Architectures: amd64 arm64 - Components: main restricted - Description: Auxiliary -diff --git a/deployment/puppet/fuel/manifests/auxiliaryrepos.pp b/deployment/puppet/fuel/manifests/auxiliaryrepos.pp -index 909fc92..4ec5d05 100644 ---- a/deployment/puppet/fuel/manifests/auxiliaryrepos.pp -+++ b/deployment/puppet/fuel/manifests/auxiliaryrepos.pp -@@ -45,7 +45,9 @@ class fuel::auxiliaryrepos( - $release_files = [ - "${ubuntu_dir}/dists/auxiliary/Release", - "${ubuntu_dir}/dists/auxiliary/main/binary-amd64/Release", -- "${ubuntu_dir}/dists/auxiliary/restricted/binary-amd64/Release"] -+ "${ubuntu_dir}/dists/auxiliary/restricted/binary-amd64/Release", -+ "${ubuntu_dir}/dists/auxiliary/main/binary-arm64/Release", -+ "${ubuntu_dir}/dists/auxiliary/restricted/binary-arm64/Release"] - - Exec['create_ubuntu_repo_dirs'] -> - Exec['create_ubuntu_repo_Packages'] -> diff --git a/patches/fuel-library/mysql-sst-provider/0001-Allow-configuring-MySQL-WSREP-SST-provider.patch b/patches/fuel-library/mysql-sst-provider/0001-Allow-configuring-MySQL-WSREP-SST-provider.patch deleted file mode 100644 index 3bf2d842..00000000 --- a/patches/fuel-library/mysql-sst-provider/0001-Allow-configuring-MySQL-WSREP-SST-provider.patch +++ /dev/null @@ -1,67 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Date: Wed, 20 Apr 2016 14:54:42 -0700 -Subject: [PATCH] Allow configuring MySQL WSREP SST provider - -On some arm64 platforms xtrabackup is broken due to an outdated -MySQL/InnoDB code that it uses which is missing. To work around that -this patch allows choosing which WSREP SST provider to use. Aside of -already supported `xtrabackup-v2` and `mysqldump`, the `rsync` method -has been added as it is comparable to xtrabackup in terms of speed (or -is faster), has been validated to work on arm64 platforms where -xtrabackup is broken and it doesn't rely on db mechanisms to perform the -state transfer. - -This patch is tied to a patch in fuel-web that introduces configuration -options for WSREP SST provider and in case nothing is specified, it -will choose `xtrabackup-v2`. - -Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> ---- - deployment/puppet/osnailyfacter/manifests/database/database.pp | 6 ++++-- - 1 file changed, 4 insertions(+), 2 deletions(-) - -diff --git a/deployment/puppet/osnailyfacter/manifests/database/database.pp b/deployment/puppet/osnailyfacter/manifests/database/database.pp -index e741ff8..be19eb5 100644 ---- a/deployment/puppet/osnailyfacter/manifests/database/database.pp -+++ b/deployment/puppet/osnailyfacter/manifests/database/database.pp -@@ -8,5 +8,6 @@ class osnailyfacter::database::database { - $use_syslog = hiera('use_syslog', true) - $mysql_hash = hiera_hash('mysql', {}) -+ $wsrep_hash = hiera_hash('mysql_wsrep', {}) - $debug = pick($mysql_hash['debug'], hiera('debug', false)) - - $mgmt_iface = get_network_role_property('mgmt/database', 'interface') -@@ -17,6 +18,7 @@ class osnailyfacter::database::database { - $mysql_root_password = $mysql_hash['root_password'] - $deb_sysmaint_password = $mysql_hash['wsrep_password'] - $enabled = pick($mysql_hash['enabled'], true) -+ $wsrep_sst_method = pick($wsrep_hash['wsrep_method'], 'xtrabackup-v2') - - $galera_node_address = get_network_role_property('mgmt/database', 'ipaddr') - $galera_nodes = values(get_node_to_ipaddr_map_by_network_role(hiera_hash('database_nodes'), 'mgmt/database')) -@@ -211,7 +213,7 @@ class osnailyfacter::database::database { - 'wsrep_cluster_name' => $galera_cluster_name, - 'wsrep_provider_options' => $wsrep_provider_options, - 'wsrep_slave_threads' => $wsrep_slave_threads, -- 'wsrep_sst_method' => 'xtrabackup-v2', -+ 'wsrep_sst_method' => $wsrep_sst_method, - #TODO (sgolovatiuk): fix this, should be a specific user not root - 'wsrep_sst_auth' => "\"root:${mysql_root_password}\"", - 'wsrep_node_address' => $galera_node_address, -@@ -264,7 +266,7 @@ class osnailyfacter::database::database { - wsrep_group_comm_port => $wsrep_group_comm_port, - bind_address => $galera_node_address, - local_ip => $galera_node_address, -- wsrep_sst_method => 'xtrabackup-v2', -+ wsrep_sst_method => $wsrep_sst_method, - override_options => $override_options, - } - diff --git a/patches/fuel-library/upstream-backports/0001-nova-AArch64-enable-hugepage-support.patch b/patches/fuel-library/upstream-backports/0001-nova-AArch64-enable-hugepage-support.patch deleted file mode 100644 index 03e727be..00000000 --- a/patches/fuel-library/upstream-backports/0001-nova-AArch64-enable-hugepage-support.patch +++ /dev/null @@ -1,98 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Fri, 16 Sep 2016 21:31:56 +0200 -Subject: [PATCH] nova: AArch64: enable hugepage support - -This patch adds AArch64 to the hardcoded lists of valid -architectures for hugepage support. - -Backported upstream patch from [1]. - -[1] https://review.openstack.org/#/c/372304/ - -Closes-bug: ARMBAND-90 - -Signed-off-by: Veena Lingadahalli <vlingadahalli@mvista.com> -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - ...gepage-and-NUMA-support-check-for-aarch64.patch | 41 ++++++++++++++++++++++ - .../openstack_tasks/manifests/roles/compute.pp | 12 +++++++ - 2 files changed, 53 insertions(+) - create mode 100644 deployment/puppet/openstack/files/nova-Adding-hugepage-and-NUMA-support-check-for-aarch64.patch - -diff --git a/deployment/puppet/openstack/files/nova-Adding-hugepage-and-NUMA-support-check-for-aarch64.patch b/deployment/puppet/openstack/files/nova-Adding-hugepage-and-NUMA-support-check-for-aarch64.patch -new file mode 100644 -index 0000000..ba91357 ---- /dev/null -+++ b/deployment/puppet/openstack/files/nova-Adding-hugepage-and-NUMA-support-check-for-aarch64.patch -@@ -0,0 +1,41 @@ -+From: VeenaSL <mveenasl@gmail.com> -+Date: Mon, 19 Sep 2016 13:36:53 +0530 -+Subject: [PATCH] Adding hugepage and NUMA support check for aarch64 -+ -+Nova ignores aarch64 while verifying for hugepage and NUMA support. -+AARCH64 also supports hugepage and NUMA on the same libvirt versions as of x86. -+Hence adding this chek for aarch64 also. -+ -+Change-Id: I7b5ae1dbdca4fdd0aee2eefd4099c4c4953b609a -+Closes-bug: #1623871 -+ -+Signed-off-by: Veena Lingadahalli <vlingadahalli@mvista.com> -+Acked-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -+ -+--- -+ virt/libvirt/driver.py | 6 ++++-- -+ -+diff --git a/virt/libvirt/driver.py b/virt/libvirt/driver.py -+index 5668f57..b0857f3 100644 -+--- a/virt/libvirt/driver.py -++++ b/virt/libvirt/driver.py -+@@ -5108,7 +5108,8 @@ class LibvirtDriver(driver.ComputeDriver): -+ self._bad_libvirt_numa_version_warn = True -+ return False -+ -+- support_matrix = {(arch.I686, arch.X86_64): MIN_LIBVIRT_NUMA_VERSION, -++ support_matrix = {(arch.I686, arch.X86_64, -++ arch.AARCH64): MIN_LIBVIRT_NUMA_VERSION, -+ (arch.PPC64, -+ arch.PPC64LE): MIN_LIBVIRT_NUMA_VERSION_PPC} -+ caps = self._host.get_capabilities() -+@@ -5124,7 +5125,8 @@ class LibvirtDriver(driver.ComputeDriver): -+ def _has_hugepage_support(self): -+ # This means that the host can support multiple values for the size -+ # field in LibvirtConfigGuestMemoryBackingPage -+- supported_archs = [arch.I686, arch.X86_64, arch.PPC64LE, arch.PPC64] -++ supported_archs = [arch.I686, arch.X86_64, arch.PPC64LE, arch.PPC64, -++ arch.AARCH64] -+ caps = self._host.get_capabilities() -+ return ((caps.host.cpu.arch in supported_archs) and -+ self._host.has_min_version(MIN_LIBVIRT_HUGEPAGE_VERSION, -diff --git a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -index 8207261..b5cefdf 100644 ---- a/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -+++ b/deployment/puppet/openstack_tasks/manifests/roles/compute.pp -@@ -313,5 +313,17 @@ class openstack_tasks::roles::compute { - cwd => $nova_path, - require => [Package['patch']], -+ } -> -+ # FIXME(armband): Add AArch64 to arch list for nova hugepage support -+ file { "${nova_path}/nova-Adding-hugepage-and-NUMA-support-check-for-aarch64.patch": -+ ensure => "file", -+ source => "puppet:///modules/openstack/nova-Adding-hugepage-and-NUMA-support-check-for-aarch64.patch", -+ } -> -+ exec { 'nova AArch64 enable hugepage support': -+ path => ['/usr/bin'], -+ command => "patch -p1 < ${nova_path}/nova-Adding-hugepage-and-NUMA-support-check-for-aarch64.patch", -+ unless => "patch -p1 -R -N --dry-run < ${nova_path}/nova-Adding-hugepage-and-NUMA-support-check-for-aarch64.patch", -+ cwd => $nova_path, -+ require => [Package['patch']], - } - - class { '::nova::cache': diff --git a/patches/fuel-main/0001-ntpd-autostart-at-Fuel-master-boot-time.patch b/patches/fuel-main/0001-ntpd-autostart-at-Fuel-master-boot-time.patch deleted file mode 100644 index 5375c82a..00000000 --- a/patches/fuel-main/0001-ntpd-autostart-at-Fuel-master-boot-time.patch +++ /dev/null @@ -1,32 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Fri, 5 May 2017 20:40:19 +0200 -Subject: [PATCH] ntpd autostart at Fuel master boot time - -Closes-bug: 1688631 - -Change-Id: I9bc4e4cb9918bedef2408abb304696fc431ed378 -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - iso/bootstrap_admin_node.sh | 1 + - 1 file changed, 1 insertion(+) - -diff --git a/iso/bootstrap_admin_node.sh b/iso/bootstrap_admin_node.sh -index dbcac5f..f5d1583 100755 ---- a/iso/bootstrap_admin_node.sh -+++ b/iso/bootstrap_admin_node.sh -@@ -542,6 +542,7 @@ fi - # Sync time - systemctl stop ntpd - systemctl start ntpdate || echo "Failed to synchronize time with 'ntpdate'" -+systemctl enable ntpd - systemctl start ntpd - - bash /etc/rc.local diff --git a/patches/fuel-menu/0001-Update-Host-OS-NTP-Servers-to-pool.ntp.org.patch b/patches/fuel-menu/0001-Update-Host-OS-NTP-Servers-to-pool.ntp.org.patch deleted file mode 100644 index c4161c76..00000000 --- a/patches/fuel-menu/0001-Update-Host-OS-NTP-Servers-to-pool.ntp.org.patch +++ /dev/null @@ -1,38 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Catalina Focsa <catalina.focsa@enea.com> -Date: Tue, 4 Apr 2017 14:11:13 +0200 -Subject: [PATCH] Update Host OS NTP Servers to pool.ntp.org - -Old *.fuel.pool.ntp.org values are no longer accessible -LP: https://bugs.launchpad.net/fuel/+bug/1679636 -JIRA: https://jira.opnfv.org/browse/ARMBAND-225 - -Signed-off-by: Catalina Focsa <catalina.focsa@enea.com> ---- - fuelmenu/settings.yaml | 6 +++--- - 1 file changed, 3 insertions(+), 3 deletions(-) - -diff --git a/fuelmenu/settings.yaml b/fuelmenu/settings.yaml -index e93309a..96f4c93 100644 ---- a/fuelmenu/settings.yaml -+++ b/fuelmenu/settings.yaml -@@ -2,9 +2,9 @@ HOSTNAME: "fuel" - DNS_DOMAIN: "domain.tld" - DNS_SEARCH: "domain.tld" - DNS_UPSTREAM: "8.8.8.8" --NTP1: "0.fuel.pool.ntp.org" --NTP2: "1.fuel.pool.ntp.org" --NTP3: "2.fuel.pool.ntp.org" -+NTP1: "0.pool.ntp.org" -+NTP2: "1.pool.ntp.org" -+NTP3: "2.pool.ntp.org" - FUEL_ACCESS: - user: "admin" - password: "admin" diff --git a/patches/fuel-nailgun-agent/0001-ohai-Support-reading-L1-info-from-ethtool.patch b/patches/fuel-nailgun-agent/0001-ohai-Support-reading-L1-info-from-ethtool.patch deleted file mode 100644 index 7067a977..00000000 --- a/patches/fuel-nailgun-agent/0001-ohai-Support-reading-L1-info-from-ethtool.patch +++ /dev/null @@ -1,73 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Enea Armband Devops Team <armband@enea.com> -Date: Sat, 2 Apr 2016 22:57:59 +0200 -Subject: [PATCH] ohai: Support reading L1 info from ethtool. - -Traditional methods of reading the ethernet card speed rely on the -drivers populating the advertised and/or supported link speed lists. - -This is not true for all drivers, especially for some Fibers -that only report the speed via ethtool when the link is up. - -This patch adds support for reading L1 info from ohai, which -supports parsing ethtool speed starting with version amos2 [1]. - -[1] https://linux.enea.com/mos-repos/ubuntu/8.0/pool/main/o/ - ohai/ohai_6.14.0-2~u14.04+mos1+mos8.0+amos2_all.deb ---- - ...ohai-Support-reading-L1-info-from-ethtool.patch | 30 ++++++++++++++++++++++ - debian/patches/series | 1 + - 2 files changed, 31 insertions(+) - create mode 100644 debian/patches/0001-ohai-Support-reading-L1-info-from-ethtool.patch - create mode 100644 debian/patches/series - -diff --git a/debian/patches/0001-ohai-Support-reading-L1-info-from-ethtool.patch b/debian/patches/0001-ohai-Support-reading-L1-info-from-ethtool.patch -new file mode 100644 -index 0000000..8ae8c49 ---- /dev/null -+++ b/debian/patches/0001-ohai-Support-reading-L1-info-from-ethtool.patch -@@ -0,0 +1,30 @@ -+From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -+Date: Sat, 2 Apr 2016 22:54:27 +0200 -+Subject: [PATCH] ohai: Support reading L1 info from ethtool. -+ -+Upstream ohai [1] introduces support for reading layer one information -+from ethtool. Allow nailgun-agent to use that info if present. -+ -+[1] https://github.com/chef/ohai/commit/ -+ afd42e9122f057ab00dd24357c28dc2ad6806434 -+--- -+ agent | 6 +++++- -+ 1 file changed, 5 insertions(+), 1 deletion(-) -+ -+diff --git a/agent b/agent -+index c764a68..978131f 100755 -+--- a/agent -++++ b/agent -+@@ -316,7 +316,11 @@ class NodeAgent -+ int_meta[:current_speed] = int_info.current_mode.speed -+ end -+ rescue -+- int_meta[:current_speed] = nil -++ if intinfo.has_key?(:link_speed) && 0 < intinfo[:link_speed] -++ int_meta[:current_speed] = intinfo[:link_speed] -++ else -++ int_meta[:current_speed] = nil -++ end -+ end -+ elsif (addrinfo[:family] rescue nil) =~ /^inet$/ -+ int_meta[:ip] = addr -diff --git a/debian/patches/series b/debian/patches/series -new file mode 100644 -index 0000000..b221a98 ---- /dev/null -+++ b/debian/patches/series -@@ -0,0 +1 @@ -+0001-ohai-Support-reading-L1-info-from-ethtool.patch diff --git a/patches/fuel-nailgun-agent/0002-Temporary-SR-IOV-Fix-VNICs-broken-filter.patch b/patches/fuel-nailgun-agent/0002-Temporary-SR-IOV-Fix-VNICs-broken-filter.patch deleted file mode 100644 index e2ab3f61..00000000 --- a/patches/fuel-nailgun-agent/0002-Temporary-SR-IOV-Fix-VNICs-broken-filter.patch +++ /dev/null @@ -1,43 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Thu, 7 Jul 2016 13:58:34 +0200 -Subject: [PATCH] Temporary: SR-IOV: Fix VNICs broken filter. - -Some blades have VNICs (VFs on a PCI device), which are not -SR-IOV VFs. - -Bug report: "VNICs ignored due to SR-IOV filtering" [1] - -HACK: Instead of checking for "physfn", we now rely on testing -for "physfn/net" subdir to determine whether current eth is -a SR-IOV VF or something else (thus keeping VNICs). - -FIXME: Revert this patch once upstream closes bug [1]. - -[1] https://bugs.launchpad.net/fuel/+bug/1597775 - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - agent | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - -diff --git a/agent b/agent -index 89ab60f..eb2bfb4 100755 ---- a/agent -+++ b/agent -@@ -406,7 +406,7 @@ class NodeAgent - next if File.exist?("/sys/class/net/#{int}/phy80211") || - File.exist?("/sys/class/net/#{int}/wireless") - # Skip virtual functions -- next if File.exists?("/sys/class/net/#{int}/device/physfn") -+ next if File.exists?("/sys/class/net/#{int}/device/physfn/net") - int_meta = {:name => int} - int_meta[:interface_properties] = {} - int_meta[:state] = intinfo[:state] diff --git a/patches/fuel-nailgun-agent/0003-AArch64-Add-CPU-details-detection.patch b/patches/fuel-nailgun-agent/0003-AArch64-Add-CPU-details-detection.patch deleted file mode 100644 index 0463023f..00000000 --- a/patches/fuel-nailgun-agent/0003-AArch64-Add-CPU-details-detection.patch +++ /dev/null @@ -1,138 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Sun, 15 Jan 2017 08:09:10 +0100 -Subject: [PATCH] AArch64: Add CPU details detection - -[ Alexandru Avadanii ] -Based on Stan's previous patch for ohai, extend fuel-nailgun-agent -to properly support AArch64 CPUs. - -[ Stanislaw Kardach ] -There is currently little human readable detail in /proc/cpuinfo on -arm64 so this patch tries to enchance this information by parsing the -DMI data (using `dmidecode`) and fail gracefully to empty strings -if no information could not be read from there. By no means this -parsing is to be taken as a standardised way of discovering an -arm64 CPU, it is just a suggestion. - -Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - agent | 90 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - 1 file changed, 90 insertions(+) - -diff --git a/agent b/agent -index 0c56264..eeb4d26 100755 ---- a/agent -+++ b/agent -@@ -340,6 +340,15 @@ class NodeAgent - end - - def _get_detailed_cpuinfo -+ case `uname -m` -+ when /aarch64.*/ -+ _get_detailed_cpuinfo_for_arm64 -+ else # default to x86 -+ _get_detailed_cpuinfo_for_x86 -+ end -+ end -+ -+ def _get_detailed_cpuinfo_for_x86 - real = {} - info = {} - info[:total] = 0 -@@ -371,6 +380,87 @@ class NodeAgent - info - end - -+ def _get_detailed_cpuinfo_for_arm64 -+ cpu_implementers = { -+ "0x41" => "ARM", -+ "0x53" => "Samsung", -+ "0x51" => "Qualcom", -+ "0x43" => "Cavium", -+ "0x50" => "APM" -+ } -+ core_models = { -+ "0xd04" => "cortex-a35", -+ "0xd03" => "cortex-a53", -+ "0xd07" => "cortex-a57", -+ "0xd08" => "cortex-a72", -+ "0x001" => "exynos-m1", -+ "0x800" => "qdf24xx", -+ "0x0a1" => "thunderx", -+ "0x000" => "xgene1", -+ "0xd07.0xd03" => "cortex-a57.cortex-a53", -+ "0xd08.0xd03" => "cortex-a72.cortex-a53" -+ } -+ cpuinfo = {} -+ real_cpus = {} -+ cpu_number = -1 -+ current_cpu = nil -+ `dmidecode -t 4 -q`.each_line do |line| -+ case line -+ when /Processor Information/ -+ cpu_number +=1 -+ real_cpus[cpu_number] = {} -+ when /\s*Manufacturer:\s(.+)/ -+ real_cpus[cpu_number][:vendor_id] = $1 -+ when /\s*Family:\s(.+)/ -+ real_cpus[cpu_number][:family] = $1 -+ when /\s*Max Speed:\s(.+)\s*MHz/ -+ real_cpus[cpu_number][:mhz] = $1.strip -+ when /\s*Core Enabled:\s(.+)/ -+ real_cpus[cpu_number][:cores] = $1 -+ end -+ end -+ -+ cpu_number = 0 -+ File.open("/proc/cpuinfo").each do |line| -+ case line -+ when /processor\s+:\s(.+)/ -+ cpuinfo[$1] = {} -+ current_cpu = $1 -+ cpu_number += 1 -+ phys_id = File.read( -+ "/sys/devices/system/cpu/cpu%d/topology/physical_package_id" % -+ $1).strip -+ i = phys_id.to_i -+ cpuinfo[$1][:core_id] = -+ File.read("/sys/devices/system/cpu/cpu%d/topology/core_id" % -+ $1).strip rescue "" -+ cpuinfo[$1][:physical_id] = phys_id if not phys_id.empty? -+ if real_cpus[i].nil? -+ i = 0 -+ end -+ if not real_cpus[i].nil? -+ cpuinfo[$1][:family] = real_cpus[i][:family] rescue "" -+ cpuinfo[$1][:cores] = real_cpus[i][:cores] rescue "" -+ cpuinfo[$1][:mhz] = real_cpus[i][:mhz] rescue "" -+ end -+ when /CPU implementer\s+:\s(.+)/ -+ cpuinfo[current_cpu][:vendor_id] = cpu_implementers[$1] -+ cpuinfo[current_cpu][:vendor_id] ||= real_cpus[cpuinfo[current_cpu][:physical_id].to_i][:vendor_id] rescue nil -+ cpuinfo[current_cpu][:vendor_id] ||= real_cpus[0][:vendor_id] rescue nil -+ cpuinfo[current_cpu][:vendor_id] ||= $1 -+ when /CPU part\s+:\s(.+)/ -+ cpuinfo[current_cpu][:model] = $1 -+ cpuinfo[current_cpu][:model_name] = core_models[$1] -+ cpuinfo[current_cpu][:model_name] ||= "" -+ when /Features\s+:\s(.+)/ -+ cpuinfo[current_cpu][:flags] = $1.split(' ') -+ end -+ end -+ cpuinfo[:total] = cpu_number -+ cpuinfo[:real] = real_cpus.keys.length -+ cpuinfo -+ end -+ - def _get_blkdev_info - info = {} - if File.directory?('/sys/block/') diff --git a/patches/fuel-nailgun-agent/0004-AArch64-Read-hugepage-size-from-proc-meminfo.patch b/patches/fuel-nailgun-agent/0004-AArch64-Read-hugepage-size-from-proc-meminfo.patch deleted file mode 100644 index 384dafca..00000000 --- a/patches/fuel-nailgun-agent/0004-AArch64-Read-hugepage-size-from-proc-meminfo.patch +++ /dev/null @@ -1,38 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Sun, 15 Jan 2017 08:14:12 +0100 -Subject: [PATCH] AArch64: Read hugepage size from /proc/meminfo - -This method should be arch-indepedent, provided /proc/meminfo -reports the correct information. - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - agent | 7 +++++++ - 1 file changed, 7 insertions(+) - -diff --git a/agent b/agent -index eeb4d26..d514de6 100755 ---- a/agent -+++ b/agent -@@ -1399,6 +1399,13 @@ class NodeAgent - def supported_hugepages - return [2048, 1048576] if _get_detailed_cpuinfo['0'][:flags].include?('pdpe1gb') - return [2048] if _get_detailed_cpuinfo['0'][:flags].include?('pse') -+ # AArch64 does not expose CPU flags, but we can rely on /proc/meminfo -+ File.open('/proc/meminfo').each do |l| -+ case l.strip -+ when /Hugepagesize:\s+(\d+)\s+kB/ -+ return [$1.to_i()] -+ end -+ end - [] - end - diff --git a/patches/fuel-ostf/0001-Arch-dependent-Cirros-ssh-password.patch b/patches/fuel-ostf/0001-Arch-dependent-Cirros-ssh-password.patch deleted file mode 100644 index b066cdf8..00000000 --- a/patches/fuel-ostf/0001-Arch-dependent-Cirros-ssh-password.patch +++ /dev/null @@ -1,71 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Nemes <alexandru.nemes@enea.com> -Date: Fri, 5 May 2017 14:16:03 +0300 -Subject: [PATCH] Arch dependent Cirros ssh password - -JIRA: https://jira.opnfv.org/browse/ARMBAND-250 - -Signed-off-by: Alexandru Nemes <alexandru.nemes@enea.com> ---- - fuel_health/nmanager.py | 19 +++++++++++++++++-- - 1 file changed, 17 insertions(+), 2 deletions(-) - -diff --git a/fuel_health/nmanager.py b/fuel_health/nmanager.py -index 74e5a03..5bfb552 100644 ---- a/fuel_health/nmanager.py -+++ b/fuel_health/nmanager.py -@@ -843,12 +843,13 @@ class NovaNetworkScenarioTest(OfficialClientTest): - LOG.exception("") - - command = "ping -q -c1 -w10 8.8.8.8" -+ cirros_pass = self.get_cirros_password() - - return self.retry_command(retries[0], retries[1], - ssh.exec_command_on_vm, - command=command, - user='cirros', -- password='cubswin:)', -+ password=cirros_pass, - vm=ip_address) - - # TODO(???) Allow configuration of execution and sleep duration. -@@ -873,11 +874,13 @@ class NovaNetworkScenarioTest(OfficialClientTest): - except Exception: - LOG.exception("") - -+ cirros_pass = self.get_cirros_password() -+ - return self.retry_command(retries[0], retries[1], - ssh.exec_command_on_vm, - command=cmd, - user='cirros', -- password='cubswin:)', -+ password=cirros_pass, - vm=ip_address) - - # TODO(???) Allow configuration of execution and sleep duration. -@@ -934,6 +937,17 @@ class NovaNetworkScenarioTest(OfficialClientTest): - instance.id, 'ACTIVE') - return instance - -+ def get_cirros_password(self): -+ hypervisor_list = self.compute_client.hypervisors.list() -+ for hypervisor in hypervisor_list: -+ if hypervisor and hypervisor.status == "enabled": -+ cpu_info = hypervisor.cpu_info.lower() -+ LOG.debug('Compute CPU INFO {0}'.format(cpu_info)) -+ break -+ if cpu_info and ("arm" in cpu_info or "aarch64" in cpu_info): -+ return "gocubsgo" -+ return "cubswin:)" -+ - @classmethod - def tearDownClass(cls): - super(NovaNetworkScenarioTest, cls).tearDownClass() diff --git a/patches/fuel-plugin-opendaylight/0001-ODL-leveldb-leveldbjni-Enable-arm64-support.patch b/patches/fuel-plugin-opendaylight/0001-ODL-leveldb-leveldbjni-Enable-arm64-support.patch deleted file mode 100644 index b307af2b..00000000 --- a/patches/fuel-plugin-opendaylight/0001-ODL-leveldb-leveldbjni-Enable-arm64-support.patch +++ /dev/null @@ -1,121 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Florin Dumitrascu <florin.dumitrascu@enea.com> -Date: Thu, 17 Mar 2016 18:15:42 +0100 -Subject: [PATCH] ODL leveldb, leveldbjni: Enable arm64 support. - -ODL depends on a native library (leveldb). - -ODL Beryllium and Boron versions are still using a native x86 version -of this library (wrapped in leveldbjni). - -There is no upstream version of leveldbjni supporting arm64 leveldb, -so we need to compile it when deploying ODL. -This is done when running odl-install puppet manifest. - -For more info about this ODL limitation, see [1]. - -[ Alexandru.Avadanii@enea.com ] -Remove JAVA8 changes after upstream rework. -Convert puppet patching to prebuilt DEB package, as part of [2]. -Rebase for Fuel Newton. - -[ Dan.Andresan@enea.com ] -Modify Puppet recipe to restart Opendaylight service after leveldbjni -This avoids a cache corruption described in [3] -Fix leveldbjni on x86_64, since it should be silently skipped. - -[1] https://bugs.opendaylight.org/show_bug.cgi?id=3973 -[2] https://jira.opnfv.org/browse/ARMBAND-114 -[3] https://jira.opnfv.org/browse/ARMBAND-211 - -Change-Id: Ia7abf375ffd122ff0ac7b12422f64526ff2c78fd -Signed-off-by: Florin Dumitrascu <florin.dumitrascu@enea.com> -Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Signed-off-by: Dan Andresan <dan.andresan@enea.com> ---- - .../puppet/modules/opendaylight/manifests/init.pp | 8 ++++++ - .../modules/opendaylight/manifests/install.pp | 3 ++ - .../modules/opendaylight/manifests/leveldbjni.pp | 32 ++++++++++++++++++++++ - 3 files changed, 43 insertions(+) - create mode 100644 deployment_scripts/puppet/modules/opendaylight/manifests/leveldbjni.pp - -diff --git a/deployment_scripts/puppet/modules/opendaylight/manifests/init.pp b/deployment_scripts/puppet/modules/opendaylight/manifests/init.pp -index 69cc23a..791ceba 100644 ---- a/deployment_scripts/puppet/modules/opendaylight/manifests/init.pp -+++ b/deployment_scripts/puppet/modules/opendaylight/manifests/init.pp -@@ -9,4 +9,12 @@ class opendaylight { - $odl_mgmt_ips = values($odl_mgmt_ips_hash) - $odl_nodes_names = keys($odl_mgmt_ips_hash) - $node_internal_address = $odl_mgmt_ips_hash["node-${node_uid}"] -+ -+ $arch = $::architecture ? { -+ 'aarch64' => $::osfamily ? { -+ 'Debian' => 'arm64', -+ default => 'aarch64', -+ }, -+ default => $::architecture, -+ } - } -diff --git a/deployment_scripts/puppet/modules/opendaylight/manifests/install.pp b/deployment_scripts/puppet/modules/opendaylight/manifests/install.pp -index 2c3d776..22ab204 100644 ---- a/deployment_scripts/puppet/modules/opendaylight/manifests/install.pp -+++ b/deployment_scripts/puppet/modules/opendaylight/manifests/install.pp -@@ -85,9 +85,12 @@ class opendaylight::install ( - value => $enabled_features, - } - -+ include opendaylight::leveldbjni -+ - Package['opendaylight'] -> - Ini_setting <||> -> - Firewall <||> -> -+ Class['opendaylight::leveldbjni'] -> - File <||> ~> - Service['opendaylight'] - } -diff --git a/deployment_scripts/puppet/modules/opendaylight/manifests/leveldbjni.pp b/deployment_scripts/puppet/modules/opendaylight/manifests/leveldbjni.pp -new file mode 100644 -index 0000000..71c3635 ---- /dev/null -+++ b/deployment_scripts/puppet/modules/opendaylight/manifests/leveldbjni.pp -@@ -0,0 +1,32 @@ -+# == Class opendaylight::leveldbjni -+# -+# Manages patching of JNI for leveldb on arm64. -+# -+# It fetches a prebuilt DEB package containing leveldbjni, -+# patched so that it's properly detected by maven on arm64 systems. -+# -+# DEB: http://linux.enea.com/mos-repos/ubuntu/9.0/ \ -+# pool/main/a/armband-odl-leveldb-fix/ -+# JIRA: https://jira.opnfv.org/browse/ARMBAND-114 -+# -+class opendaylight::leveldbjni { -+ case $::osfamily { -+ 'Debian', 'Ubuntu': {} -+ default: { -+ fail('Not supported on OS other than Debian based.') -+ } -+ } -+ -+ if $::osfamily == 'Debian' and $::opendaylight::arch == 'arm64' and ! defined(Package['armband-odl-leveldb-fix']) { -+ exec {'hotfix-odl': -+ command => 'systemctl stop opendaylight; rm -rf /opt/opendaylight/data/cache', -+ onlyif => 'test 0 -eq $(dpkg -l | grep -c armband-odl-leveldb-fix)', -+ path => ['/bin', '/usr/bin'] -+ } -+ -+ package { 'armband-odl-leveldb-fix': -+ ensure => 'present', -+ require => Exec['hotfix-odl'] -+ } -+ } -+} diff --git a/patches/fuel-plugin-opendaylight/0002-install_quagga.sh-Enable-parallel-build.patch b/patches/fuel-plugin-opendaylight/0002-install_quagga.sh-Enable-parallel-build.patch deleted file mode 100644 index 553fb7fc..00000000 --- a/patches/fuel-plugin-opendaylight/0002-install_quagga.sh-Enable-parallel-build.patch +++ /dev/null @@ -1,28 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Thu, 23 Feb 2017 23:45:41 +0000 (+0100) -Subject: install_quagga.sh: Enable parallel build - -install_quagga.sh: Enable parallel build - -Change-Id: I8b961ebacc6ab096200d1e124d8f50ed6e2b8217 -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - -diff --git a/pre_build_hook b/pre_build_hook -index ab8b0a6..20dd715 100755 ---- a/pre_build_hook -+++ b/pre_build_hook -@@ -78,3 +78,5 @@ sed -i 's/cd zrpcd/cd zrpcd \&\& git checkout eddf5bd6305ef52a2d39df99860c734935 - sed -i 's/quagga_110_mpbgp_capnp/95bb0f4a5/' deployment_scripts/install_quagga.sh - # pin thrift to the latest version at this point in time - sed -i 's/cd thrift/cd thrift \&\& git checkout 0.10.0/' deployment_scripts/install_quagga.sh -+# enable parallel build -+sed -i 's|make$|make -j|g' deployment_scripts/install_quagga.sh diff --git a/patches/fuel-plugin-ovs/0001-AArch64-Add-support-for-downloading-prebuilt-DEBs.patch b/patches/fuel-plugin-ovs/0001-AArch64-Add-support-for-downloading-prebuilt-DEBs.patch deleted file mode 100644 index 2a6ad8a7..00000000 --- a/patches/fuel-plugin-ovs/0001-AArch64-Add-support-for-downloading-prebuilt-DEBs.patch +++ /dev/null @@ -1,125 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Mon, 12 Sep 2016 18:02:42 +0200 -Subject: [PATCH] AArch64: Add support for downloading prebuilt DEBs - -Change-Id: I6042f9f7bdde5a15e3cb5a94ab67a990faae0418 -Signed-off-by: Anders Roxell <anders.roxell@enea.com> -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - ovs_build/prebuilt-ovs-fetch.sh | 70 +++++++++++++++++++++++++++++++++++++++++ - pre_build_hook | 10 ++++++ - 2 files changed, 80 insertions(+) - create mode 100755 ovs_build/prebuilt-ovs-fetch.sh - -diff --git a/ovs_build/prebuilt-ovs-fetch.sh b/ovs_build/prebuilt-ovs-fetch.sh -new file mode 100755 -index 0000000..b4f30c5 ---- /dev/null -+++ b/ovs_build/prebuilt-ovs-fetch.sh -@@ -0,0 +1,70 @@ -+#!/bin/bash -+ -+set -eux -+ -+BUILD_DEST=${BUILD_DEST:-/deb} -+BUILD_HOME=${BUILD_HOME:-/tmp/ovs-dpdk} -+BUILD_ARCH=${BUILD_ARCH:-$(dpkg --print-architecture)} -+UBUNTU_ARCH=${UBUNTU_ARCH:-${BUILD_ARCH}} -+ -+# armband_fetch will download all binaries for one debian source package -+# -+# Usage: -+# armband_fetch <src_deb_name> <src_deb_version> \ -+# [<repo_base_url> \ -+# [<repo_Sources_relative_url> \ -+# [<repo_Packages_relative_url>]]] \ -+# -+# e.g.: -+# armband_fetch dpdk 16.07-0ubuntu5~u1604+mos1 \ -+# https://linux.enea.com/mos-repos/ubuntu/10.0 \ -+# dists/mos10.0-fuel-plugin-ovs/main/source/Sources \ -+# dists/mos10.0-fuel-plugin-ovs/main/binary-arm64/Packages -+function armband_fetch() { -+ local name=$1 -+ local ver=${2//\+/\\\+} -+ local repo_base_url=${3:-http://linux.enea.com/mos-repos/ubuntu/10.0} -+ local repo_Sources_url=${4:-dists/mos10.0-fuel-plugin-ovs/main/source/Sources} -+ local repo_Packages_url=${5:-dists/mos10.0-fuel-plugin-ovs/main/binary-arm64/Packages} -+ local found=false -+ -+ # Fetch Sources & Packages files if not already present -+ if [ ! -f Sources -o ! -f Packages ]; then -+ sudo apt-get -y --force-yes install wget -+ wget -c "${repo_base_url}/${repo_Sources_url}" -+ wget -c "${repo_base_url}/${repo_Packages_url}" -+ fi -+ -+ L=$(grep -Pzo "(?<=Package: ${name}\nBinary: )(\N*?)(?=\nVersion: ${ver}\n)" \ -+ Sources | tr -d ',') -+ for deb_binary in ${L}; do -+ D=$(grep -Po "(?<=Filename: )(.*?\/${deb_binary}_${ver}_.*?\.deb)" Packages) || true -+ if [ -n "${D}" ]; then -+ wget -c "${repo_base_url}/${D}" && found=true -+ fi -+ done -+ if [ ${found} = false ]; then -+ echo "ERROR: Could not find DEBs for pkg/ver: ${name}/${ver}" -+ exit 1 -+ fi -+} -+ -+for arch in ${UBUNTU_ARCH}; do -+ if [ "${arch}" = "${BUILD_ARCH}" ]; then -+ # Skip fetching native binaries -+ continue -+ fi -+ -+ cd ${BUILD_DEST}; rm -rf ${BUILD_HOME}; mkdir -p ${BUILD_HOME} -+ cd ${BUILD_HOME} -+ -+ ARMBAND_BASE="http://linux.enea.com/mos-repos/ubuntu/10.0" -+ ARMBAND_SRCS="dists/mos10.0-fuel-plugin-ovs/main/source/Sources" -+ ARMBAND_BINS="dists/mos10.0-fuel-plugin-ovs/main/binary-${arch}/Packages" -+ -+ # OVS with NSH -+ armband_fetch 'openvswitch' '.*nsh' \ -+ "${ARMBAND_BASE}" "${ARMBAND_SRCS}" "${ARMBAND_BINS}" -+ mkdir -p ${BUILD_HOME}/deb; mv *.deb ${BUILD_HOME}/deb; cd ${BUILD_HOME}/deb -+ tar czvf ${BUILD_DEST}/ovs-nsh_${arch}.tar.gz .; -+done -diff --git a/pre_build_hook b/pre_build_hook -index a955678..f9a6010 100755 ---- a/pre_build_hook -+++ b/pre_build_hook -@@ -6,6 +6,11 @@ BUILD_FOR=${BUILD_FOR:-ubuntu} - BUILD_ARCH="$(dpkg --print-architecture)" - DIR="$(dirname `readlink -f $0`)" - -+# Pass target arch(s) (UBUNTU_ARCH) to build scripts. -+# All archs should be represented in `dpkg --print-architecture` format -+# UBUNTU_ARCH holds a space-separated list of target arch(s) -+export UBUNTU_ARCH=${UBUNTU_ARCH:-$(dpkg --print-architecture)} -+ - function build_pkg { - case ${BUILD_ARCH} in - arm64) -@@ -24,6 +29,11 @@ function build_pkg { - rm -rf ${DEB_DIR}; mkdir -p ${DEB_DIR}; chmod 777 ${DEB_DIR}; - sudo docker run -v ${DEB_DIR}:/deb -t ovs_build /ovs_build/build-ovs-nsh.sh - cp -r ${DEB_DIR}/* ${DIR}/repositories/ubuntu/ -+ -+ rm -rf ${DEB_DIR}; mkdir -p ${DEB_DIR}; chmod 777 ${DEB_DIR}; -+ sudo docker run -e "UBUNTU_ARCH=${UBUNTU_ARCH}" -e "BUILD_ARCH=${BUILD_ARCH}" \ -+ -v ${DEB_DIR}:/deb -t ovs_build /ovs_build/prebuilt-ovs-fetch.sh -+ cp -r ${DEB_DIR}/* ${DIR}/repositories/ubuntu/ - rm -rf ${DEB_DIR} - - ;; diff --git a/patches/fuel-plugin-ovs/0002-deb-src-Enable-building-Debian-source-pkgs.patch b/patches/fuel-plugin-ovs/0002-deb-src-Enable-building-Debian-source-pkgs.patch deleted file mode 100644 index 8451c68f..00000000 --- a/patches/fuel-plugin-ovs/0002-deb-src-Enable-building-Debian-source-pkgs.patch +++ /dev/null @@ -1,159 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Wed, 25 Jan 2017 22:11:44 +0100 -Subject: [PATCH] deb-src: Enable building Debian source pkgs - -Armband plans on shipping prebuilt binaries. -Opensource rules dictate they should be accompanied by the -corresponding source packages. - -So, add the possibility to also build Debian sources when a -dummy "ovs_build/build_debian_source" file exists (to avoid -passing an useless env var to each docker run). - -Change-Id: I4e308d84c2eb01af043ed73440c46fc692c30d12 -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - ovs_build/build-ovs-dpdk.sh | 28 +++++++++++++++++----- - ovs_build/build-ovs-nsh.sh | 26 +++++++++++++++++--- - .../debian/patches/series | 1 - - 3 files changed, 45 insertions(+), 10 deletions(-) - delete mode 100644 ovs_build/openvswitch_2.6-dpdk_16.07/debian/patches/series - -diff --git a/ovs_build/build-ovs-dpdk.sh b/ovs_build/build-ovs-dpdk.sh -index 7ce2a62..8dd7430 100755 ---- a/ovs_build/build-ovs-dpdk.sh -+++ b/ovs_build/build-ovs-dpdk.sh -@@ -11,14 +11,30 @@ DEB_ARCH="$(dpkg --print-architecture)" - - export DEB_BUILD_OPTIONS='parallel=8 nocheck' - -+function debian_build() { -+ if [ -f "${BUILD_SRC}/build_debian_source" ]; then -+ dpkg-source -b . -+ cp -f ../*.{dsc,xz} ${BUILD_HOME}/deb -+ fi -+ debian/rules build; fakeroot debian/rules binary -+} -+ -+function debian_src_prepare_ovs() { -+ if [ -f "${BUILD_SRC}/build_debian_source" ]; then -+ suffix="${OVS_VER}.orig.tar.xz" -+ tar cJf ../openvswitch_${suffix} --exclude='./debian' . -+ cd ..; ln -sf openvswitch_${suffix} openvswitch-dpdk_${suffix}; cd - -+ fi -+} -+ - sudo apt-get update - sudo apt-get -y --force-yes install devscripts dpkg-dev git wget dkms - - rm -rf ${BUILD_HOME}; mkdir -p ${BUILD_HOME}/deb - - cd ${BUILD_HOME} --wget -c http://fast.dpdk.org/rel/dpdk-16.07.tar.xz --xz -d dpdk-16.07.tar.xz; tar xvf dpdk-16.07.tar -+wget -c http://fast.dpdk.org/rel/dpdk-16.07.tar.xz -O dpdk_16.07.orig.tar.xz -+tar xJvf dpdk_16.07.orig.tar.xz - cd dpdk-16.07 - cp -r ${BUILD_SRC}/dpdk_16.07.fuel/debian . - cat << EOF > debian/changelog -@@ -45,7 +61,7 @@ sudo apt-get install -y --force-yes debhelper \ - python-sphinx \ - texlive-fonts-recommended \ - texlive-latex-extra --debian/rules build; fakeroot debian/rules binary -+debian_build - - cd ${BUILD_HOME} - sudo apt-get install -y --force-yes hwdata -@@ -73,7 +89,7 @@ sudo apt-get install -y --force-yes autoconf \ - python-six - - git clone https://github.com/openvswitch/ovs.git --cd ovs; git checkout ${OVS_COMMIT}; rm -rf .git -+cd ovs; git checkout ${OVS_COMMIT}; rm -rf .git; debian_src_prepare_ovs - cd ${BUILD_HOME}; cp -r ovs ovs-dpdk - - cd ovs-dpdk -@@ -86,10 +102,10 @@ openvswitch-dpdk (${OVS_VER}-1) unstable; urgency=low - -- Open vSwitch team <dev@openvswitch.org> $(date --rfc-2822) - EOF - --debian/rules build; fakeroot debian/rules binary -+debian_build - - cd ${BUILD_HOME}/ovs --debian/rules build; fakeroot debian/rules binary -+debian_build - - cp -r ${BUILD_HOME}/*.deb ${BUILD_HOME}/deb - cd ${BUILD_HOME}/deb -diff --git a/ovs_build/build-ovs-nsh.sh b/ovs_build/build-ovs-nsh.sh -index dbebdb2..efbabab 100755 ---- a/ovs_build/build-ovs-nsh.sh -+++ b/ovs_build/build-ovs-nsh.sh -@@ -11,6 +11,22 @@ DEB_ARCH="$(dpkg --print-architecture)" - - export DEB_BUILD_OPTIONS='parallel=8 nocheck' - -+function debian_build() { -+ if [ -f "${BUILD_SRC}/build_debian_source" ]; then -+ dpkg-source -b . -+ cp -f ../*.{dsc,xz} ${BUILD_HOME}/deb -+ fi -+ debian/rules build; fakeroot debian/rules binary -+} -+ -+function debian_src_prepare_ovs() { -+ if [ -f "${BUILD_SRC}/build_debian_source" ]; then -+ suffix="${OVS_VER}.orig.tar.xz" -+ tar cJf ../openvswitch_${suffix} --exclude='./debian' . -+ cd ..; ln -sf openvswitch_${suffix} openvswitch-dpdk_${suffix}; cd - -+ fi -+} -+ - sudo apt-get update - sudo apt-get -y --force-yes install devscripts dpkg-dev git wget dkms - -@@ -39,12 +55,16 @@ sudo apt-get install -y --force-yes autoconf \ - python-six - - git clone https://github.com/openvswitch/ovs.git --cd ovs; git checkout ${OVS_COMMIT}; rm -rf .git -+cd ovs; git checkout ${OVS_COMMIT}; rm -rf .git; debian_src_prepare_ovs -+mkdir -p debian/patches; mkdir -p .pc - PATCHES=$(cd ${BUILD_SRC}/ovs_nsh_patches/v2.6.1/; echo *patch) - for patch in ${PATCHES} - do -- patch -p1 < ${BUILD_SRC}/ovs_nsh_patches/v2.6.1/${patch} -+ echo "${patch}" >> debian/patches/series -+ cp ${BUILD_SRC}/ovs_nsh_patches/v2.6.1/${patch} debian/patches/ -+ patch -p1 < debian/patches/${patch} - done -+cp debian/patches/series .pc/applied-patches - - cat << EOF > debian/changelog - openvswitch (${OVS_VER}-1.nsh) unstable; urgency=low -@@ -53,7 +73,7 @@ openvswitch (${OVS_VER}-1.nsh) unstable; urgency=low - - -- Open vSwitch team <dev@openvswitch.org> $(date --rfc-2822) - EOF --debian/rules build; fakeroot debian/rules binary -+debian_build - - cp -r ${BUILD_HOME}/*.deb ${BUILD_HOME}/deb - cd ${BUILD_HOME}/deb -diff --git a/ovs_build/openvswitch_2.6-dpdk_16.07/debian/patches/series b/ovs_build/openvswitch_2.6-dpdk_16.07/debian/patches/series -deleted file mode 100644 -index 8b13789..0000000 ---- a/ovs_build/openvswitch_2.6-dpdk_16.07/debian/patches/series -+++ /dev/null -@@ -1 +0,0 @@ -- diff --git a/patches/fuel-plugin-ovs/0003-build-docker-Use-host-s-network-stack.patch b/patches/fuel-plugin-ovs/0003-build-docker-Use-host-s-network-stack.patch deleted file mode 100644 index ffa9ba77..00000000 --- a/patches/fuel-plugin-ovs/0003-build-docker-Use-host-s-network-stack.patch +++ /dev/null @@ -1,41 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Thu, 15 Sep 2016 19:16:34 +0200 -Subject: [PATCH] build: docker: Use host's network stack - -Similar to Fuel@OPNFV build process, we want to mount /etc/hosts -in the builder container to be able to use local mirror caches. - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - pre_build_hook | 6 ++++-- - 1 file changed, 4 insertions(+), 2 deletions(-) - -diff --git a/pre_build_hook b/pre_build_hook -index f9a6010..76da923 100755 ---- a/pre_build_hook -+++ b/pre_build_hook -@@ -27,12 +27,14 @@ function build_pkg { - DEB_DIR=${DIR}/deb-ovs-dpdk - - rm -rf ${DEB_DIR}; mkdir -p ${DEB_DIR}; chmod 777 ${DEB_DIR}; -- sudo docker run -v ${DEB_DIR}:/deb -t ovs_build /ovs_build/build-ovs-nsh.sh -+ sudo docker run -v ${DEB_DIR}:/deb -v /etc/hosts:/etc/hosts \ -+ -t ovs_build /ovs_build/build-ovs-nsh.sh - cp -r ${DEB_DIR}/* ${DIR}/repositories/ubuntu/ - - rm -rf ${DEB_DIR}; mkdir -p ${DEB_DIR}; chmod 777 ${DEB_DIR}; - sudo docker run -e "UBUNTU_ARCH=${UBUNTU_ARCH}" -e "BUILD_ARCH=${BUILD_ARCH}" \ -- -v ${DEB_DIR}:/deb -t ovs_build /ovs_build/prebuilt-ovs-fetch.sh -+ -v ${DEB_DIR}:/deb -v /etc/hosts:/etc/hosts \ -+ -t ovs_build /ovs_build/prebuilt-ovs-fetch.sh - cp -r ${DEB_DIR}/* ${DIR}/repositories/ubuntu/ - rm -rf ${DEB_DIR} - diff --git a/patches/fuel-web/0001-nailgun-Add-AArch64-Openstack-Newton-release-s.patch b/patches/fuel-web/0001-nailgun-Add-AArch64-Openstack-Newton-release-s.patch deleted file mode 100644 index 3c887278..00000000 --- a/patches/fuel-web/0001-nailgun-Add-AArch64-Openstack-Newton-release-s.patch +++ /dev/null @@ -1,373 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Sun, 18 Dec 2016 21:08:42 +0100 -Subject: [PATCH] nailgun: Add AArch64 Openstack Newton release(s). - -Add the following Openstack releases: -- Newton on Ubuntu 16.04 (aarch64) -- Newton on Ubuntu+UCA 16.04 (aarch64) - -Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - nailgun/nailgun/fixtures/openstack.yaml | 343 ++++++++++++++++++++++++++++++++ - 1 file changed, 343 insertions(+) - -diff --git a/nailgun/nailgun/fixtures/openstack.yaml b/nailgun/nailgun/fixtures/openstack.yaml -index eb33db2..ba967d7 100644 ---- a/nailgun/nailgun/fixtures/openstack.yaml -+++ b/nailgun/nailgun/fixtures/openstack.yaml -@@ -2511,3 +2511,346 @@ - weight: 61 - type: "checkbox" - value: true -+- &ubuntu_aarch64_xenial -+ pk: 4 -+ extend: *base_release -+ fields: -+ name: "Newton on Ubuntu 16.04 (aarch64)" -+ version: "newton-10.0" -+ operating_system: "Ubuntu" -+ description: "This option will install the OpenStack Newton packages using Ubuntu as a base operating system. With high availability features built in, you are getting a robust, enterprise-grade OpenStack deployment." -+ attributes_metadata: -+ editable: -+ kernel_params: -+ kernel: -+ value: "console=ttyAMA0,115200 console=ttyS0,115200 console=tty0 net.ifnames=1 biosdevname=0 rootdelay=90 nomodeset" -+ repo_setup: -+ metadata: -+ label: "Repositories" -+ weight: 50 -+ group: "general" -+ always_editable: true -+ repos: -+ type: "custom_repo_configuration" -+ extra_priority: null -+ description: | -+ Please note: the first repository will be considered the operating system mirror that will be used during node provisioning. -+ To create a local repository mirror on the Fuel master node, please follow the instructions provided by running "fuel-createmirror --help" on the Fuel master node. -+ Please make sure your Fuel master node has Internet access to the repository before attempting to create a mirror. -+ value: -+ - type: "deb" -+ name: "ubuntu-local" -+ uri: "http://{settings.MASTER_IP}:8080/mirrors/ubuntu" -+ suite: "xenial" -+ section: "main" -+ priority: null -+ - type: "deb" -+ name: "ubuntu" -+ uri: "http://ports.ubuntu.com/" -+ suite: "xenial" -+ section: "main universe multiverse" -+ priority: null -+ - type: "deb" -+ name: "ubuntu-updates" -+ uri: "http://ports.ubuntu.com/" -+ suite: "xenial-updates" -+ section: "main universe multiverse" -+ priority: null -+ - type: "deb" -+ name: "ubuntu-security" -+ uri: "http://ports.ubuntu.com/" -+ suite: "xenial-security" -+ section: "main universe multiverse" -+ priority: null -+ - type: "deb" -+ name: "mos" -+ uri: "http://{settings.MASTER_IP}:8080/{cluster.release.version}/ubuntu/x86_64" -+ suite: "mos10.0" -+ section: "main restricted" -+ priority: 1050 -+ - type: "deb" -+ name: "mos-updates" -+ uri: "http://linux.enea.com/mos-repos/ubuntu/{cluster.release.environment_version}" -+ suite: "mos10.0-updates" -+ section: "main restricted" -+ priority: 1050 -+ - type: "deb" -+ name: "mos-security" -+ uri: "http://linux.enea.com/mos-repos/ubuntu/{cluster.release.environment_version}" -+ suite: "mos10.0-security" -+ section: "main restricted" -+ priority: 1050 -+ - type: "deb" -+ name: "mos-holdback" -+ uri: "http://linux.enea.com/mos-repos/ubuntu/{cluster.release.environment_version}" -+ suite: "mos10.0-holdback" -+ section: "main restricted" -+ priority: 1100 -+ - type: "deb" -+ name: "Auxiliary" -+ uri: "http://{settings.MASTER_IP}:8080/{cluster.release.version}/ubuntu/auxiliary" -+ suite: "auxiliary" -+ section: "main restricted" -+ priority: 1150 -+ provision: -+ metadata: -+ label: "Provision" -+ weight: 80 -+ group: "general" -+ restrictions: -+ - condition: "false" -+ action: "hide" -+ packages: -+ label: "Initial packages" -+ weight: 10 -+ type: "textarea" -+ value: | -+ acl -+ anacron -+ bash-completion -+ bridge-utils -+ bsdmainutils -+ build-essential -+ cloud-init -+ curl -+ daemonize -+ debconf-utils -+ gdisk -+ grub-efi-arm64 -+ hwloc -+ linux-firmware -+ linux-headers-generic-lts-xenial -+ linux-image-generic-lts-xenial -+ lvm2 -+ mcollective -+ mdadm -+ multipath-tools -+ multipath-tools-boot -+ nailgun-agent -+ nailgun-mcagents -+ network-checker -+ ntp -+ ntpdate -+ openssh-client -+ openssh-server -+ puppet -+ python-amqp -+ ruby-augeas -+ ruby-ipaddress -+ ruby-json -+ ruby-netaddr -+ ruby-openstack -+ ruby-shadow -+ ruby-stomp -+ telnet -+ ubuntu-minimal -+ ubuntu-standard -+ uuid-runtime -+ vim -+ virt-what -+ vlan -+ generated: -+ cobbler: -+ profile: -+ generator_arg: "ubuntu_bootstrap" -+ provision: -+ codename: "xenial" -+ image_data: -+ /: -+ uri: "http://{settings.MASTER_IP}:8080/targetimages/env_{cluster.id}_ubuntu_1604_arm64.img.gz" -+ format: "ext4" -+ container: "gzip" -+ /boot: -+ uri: "http://{settings.MASTER_IP}:8080/targetimages/env_{cluster.id}_ubuntu_1604_arm64-boot.img.gz" -+ format: "ext2" -+ container: "gzip" -+ node_attributes: -+ cpu_pinning: -+ metadata: -+ group: "nfv" -+ label: "CPU pinning" -+ weight: 10 -+ restrictions: -+ - condition: "settings:common.libvirt_type.value != 'kvm'" -+ action: "hide" -+ nova: -+ weight: 10 -+ description: "Number of CPUs for Nova usage" -+ label: "Nova CPU pinning" -+ type: "number" -+ value: 0 -+ min: 0 -+ dpdk: -+ weight: 20 -+ description: "Number of CPUs for DPDK usage" -+ label: "DPDK CPU pinning" -+ type: "number" -+ value: 0 -+ min: 0 -+ restrictions: -+ - condition: "not ('experimental' in version:feature_groups)" -+ action: "hide" -+ hugepages: -+ metadata: -+ group: "nfv" -+ label: "Huge Pages" -+ weight: 20 -+ restrictions: -+ - condition: "settings:common.libvirt_type.value != 'kvm'" -+ action: "hide" -+ nova: -+ weight: 10 -+ description: "Nova Huge Pages configuration" -+ label: "Nova Huge Pages" -+ type: "custom_hugepages" -+ value: {} -+ dpdk: -+ weight: 20 -+ description: "DPDK Huge Pages per NUMA node in MB" -+ label: "DPDK Huge Pages" -+ type: "number" -+ value: 0 -+ min: 0 -+ restrictions: -+ - condition: "not ('experimental' in version:feature_groups)" -+ action: "hide" -+ networks_metadata: -+ dpdk_drivers: -+ # Sources: -+ # http://dpdk.org/browse/dpdk/tree/doc/guides/rel_notes/known_issues.rst?h=releases&id=7173acefc7cfdfbbb9b91fcba1c9a67adb4c07c9#n168 -+ # http://dpdk.org/browse/dpdk/tree/lib/librte_eal/common/include/rte_pci_dev_ids.h?h=releases&id=7173acefc7cfdfbbb9b91fcba1c9a67adb4c07c9 -+ igb_uio: [ -+ "8086:100e", "8086:100f", "8086:1011", "8086:1010", "8086:1012", -+ "8086:101d", "8086:105e", "8086:105f", "8086:1060", "8086:10d9", -+ "8086:10da", "8086:10a4", "8086:10d5", "8086:10a5", "8086:10bc", -+ "8086:107d", "8086:107e", "8086:107f", "8086:10b9", "8086:109a", -+ "8086:10d3", "8086:10f6", "8086:150c", "8086:10c9", "8086:10e6", -+ "8086:10e7", "8086:10e8", "8086:1526", "8086:150a", "8086:1518", -+ "8086:150d", "8086:10a7", "8086:10a9", "8086:10d6", "8086:150e", -+ "8086:150f", "8086:1510", "8086:1511", "8086:1516", "8086:1527", -+ "8086:1521", "8086:1522", "8086:1523", "8086:1524", "8086:1546", -+ "8086:1533", "8086:1534", "8086:1535", "8086:1536", "8086:1537", -+ "8086:1538", "8086:1539", "8086:1f40", "8086:1f41", "8086:1f45", -+ "8086:0438", "8086:043a", "8086:043c", "8086:0440", "8086:10b6", -+ "8086:1508", "8086:10c6", "8086:10c8", "8086:150b", "8086:10db", -+ "8086:10dd", "8086:10ec", "8086:10f1", "8086:10f4", "8086:10f7", -+ "8086:1514", "8086:1517", "8086:10f9", "8086:10fb", "8086:11a9", -+ "8086:1f72", "8086:17d0", "8086:0470", "8086:152a", "8086:1529", -+ "8086:1507", "8086:154d", "8086:154a", "8086:1558", "8086:1557", -+ "8086:10fc", "8086:151c", "8086:154f", "8086:1528", "8086:1560", -+ "8086:15ae", "8086:1563", "8086:15aa", "8086:15ab", "8086:155d", -+ "8086:1572", "8086:1574", "8086:157f", "8086:1580", "8086:1581", -+ "8086:1583", "8086:1584", "8086:1585", "8086:1586", "8086:15a4", -+ "8086:10ca", "8086:152d", "8086:1520", "8086:152f", "8086:10ed", -+ "8086:152e", "8086:1515", "8086:1530", "8086:1564", "8086:1565", -+ "8086:15a8", "8086:15a9", "8086:154c", "8086:1571", "1af4:1000", -+ "15ad:07b0", "8086:15a5", "1137:0043", "1137:0071", "14e4:168a", -+ "14e4:16a9", "14e4:164f", "14e4:168e", "14e4:16af", "14e4:163d", -+ "14e4:163f", "14e4:168d", "14e4:16a1", "14e4:16a2", "14e4:16ad", -+ "14e4:16ae", "14e4:163e", "14e4:16a4" -+ ] -+- pk: 5 -+ extend: *ubuntu_aarch64_xenial -+ fields: -+ name: "Newton on Ubuntu+UCA 16.04 (aarch64)" -+ version: "newton-10.0" -+ description: "This option will install the OpenStack Newton packages using Ubuntu as a base operating system, including Ubuntu Cloud Archive OpenStack packages. With high availability features built in, you are getting a robust, enterprise-grade OpenStack deployment." -+ attributes_metadata: -+ editable: -+ repo_setup: -+ repos: -+ type: "custom_repo_configuration" -+ extra_priority: null -+ description: | -+ Please note: the first repository will be considered the operating system mirror that will be used during node provisioning. -+ To create a local repository mirror on the Fuel master node, please follow the instructions provided by running "fuel-createmirror --help" on the Fuel master node. -+ Please make sure your Fuel master node has Internet access to the repository before attempting to create a mirror. -+ value: -+ - type: "deb" -+ name: "ubuntu-local" -+ uri: "http://{settings.MASTER_IP}:8080/mirrors/ubuntu" -+ suite: "xenial" -+ section: "main" -+ priority: null -+ - type: "deb" -+ name: "ubuntu" -+ uri: "http://ports.ubuntu.com/" -+ suite: "xenial" -+ section: "main universe multiverse" -+ priority: null -+ - type: "deb" -+ name: "ubuntu-updates" -+ uri: "http://ports.ubuntu.com/" -+ suite: "xenial-updates" -+ section: "main universe multiverse" -+ priority: null -+ - type: "deb" -+ name: "ubuntu-security" -+ uri: "http://ports.ubuntu.com/" -+ suite: "xenial-security" -+ section: "main universe multiverse" -+ priority: null -+ - type: "deb" -+ name: "uca" -+ uri: "http://ubuntu-cloud.archive.canonical.com/ubuntu" -+ suite: "xenial-updates/newton" -+ section: "main" -+ priority: 1080 -+ - type: "deb" -+ name: "uca-proposed" -+ uri: "http://ubuntu-cloud.archive.canonical.com/ubuntu" -+ suite: "xenial-proposed/newton" -+ section: "main" -+ priority: 1080 -+ - type: "deb" -+ name: "mos" -+ uri: "http://{settings.MASTER_IP}:8080/{cluster.release.version}/ubuntu/x86_64" -+ suite: "mos10.0" -+ section: "main restricted" -+ priority: 1050 -+ - type: "deb" -+ name: "mos-updates" -+ uri: "http://linux.enea.com/mos-repos/ubuntu/{cluster.release.environment_version}" -+ suite: "mos10.0-updates" -+ section: "main restricted" -+ priority: 1050 -+ - type: "deb" -+ name: "mos-security" -+ uri: "http://linux.enea.com/mos-repos/ubuntu/{cluster.release.environment_version}" -+ suite: "mos10.0-security" -+ section: "main restricted" -+ priority: 1050 -+ - type: "deb" -+ name: "mos-holdback" -+ uri: "http://linux.enea.com/mos-repos/ubuntu/{cluster.release.environment_version}" -+ suite: "mos10.0-holdback" -+ section: "main restricted" -+ priority: 1100 -+ - type: "deb" -+ name: "Auxiliary" -+ uri: "http://{settings.MASTER_IP}:8080/{cluster.release.version}/ubuntu/auxiliary" -+ suite: "auxiliary" -+ section: "main restricted" -+ priority: 1150 -+ repo_type: -+ type: "hidden" -+ weight: 58 -+ value: "uca" -+ pin_haproxy: -+ label: "HAProxy APT Pinning" -+ description: "Use Fuel HAProxy packages instead of upstream" -+ weight: 59 -+ type: "checkbox" -+ value: true -+ pin_rabbitmq: -+ label: "RabbitMQ APT Pinning" -+ description: "Use Fuel RabbitMQ packages instead of upstream" -+ weight: 60 -+ type: "checkbox" -+ value: true -+ pin_ceph: -+ label: "Ceph APT Pinning" -+ description: "Use Fuel Ceph packages instead of upstream" -+ weight: 61 -+ type: "checkbox" -+ value: true diff --git a/patches/fuel-web/0002-Add-arch-to-nailgun-release-and-target-image.patch b/patches/fuel-web/0002-Add-arch-to-nailgun-release-and-target-image.patch deleted file mode 100644 index df3f96ca..00000000 --- a/patches/fuel-web/0002-Add-arch-to-nailgun-release-and-target-image.patch +++ /dev/null @@ -1,330 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Date: Sun, 18 Dec 2016 21:28:21 +0100 -Subject: [PATCH] Add arch to nailgun release and target image - -This is required so that the TestVM image is created using a cirros -image that is compatible with the architecture of the deployment setup. -As a bonus, it is also used when building the target image. - -[ Alexandru Avadanii ] -Rebased onto Newton. - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> ---- - nailgun/nailgun/consts.py | 5 ++ - .../alembic_migrations/versions/armband.py | 53 ++++++++++++++++++++++ - nailgun/nailgun/db/sqlalchemy/models/release.py | 8 ++++ - nailgun/nailgun/fixtures/openstack.yaml | 5 ++ - .../nailgun/orchestrator/deployment_serializers.py | 8 +++- - .../orchestrator/provisioning_serializers.py | 3 +- - nailgun/nailgun/orchestrator/tasks_templates.py | 5 +- - .../integration/test_cluster_changes_handler.py | 12 ++--- - .../integration/test_orchestrator_serializer.py | 4 +- - nailgun/nailgun/test/unit/test_tasks_templates.py | 6 ++- - 10 files changed, 95 insertions(+), 14 deletions(-) - create mode 100644 nailgun/nailgun/db/migration/alembic_migrations/versions/armband.py - -diff --git a/nailgun/nailgun/consts.py b/nailgun/nailgun/consts.py -index ccbe860..a3875ce 100644 ---- a/nailgun/nailgun/consts.py -+++ b/nailgun/nailgun/consts.py -@@ -38,6 +38,11 @@ RELEASE_OS = Enum( - ) - ) - -+RELEASE_ARCHS = Enum( -+ 'amd64', -+ 'arm64' -+) -+ - CLUSTER_MODES = Enum( - 'multinode', - 'ha_full', -diff --git a/nailgun/nailgun/db/migration/alembic_migrations/versions/armband.py b/nailgun/nailgun/db/migration/alembic_migrations/versions/armband.py -new file mode 100644 -index 0000000..6e42b3f ---- /dev/null -+++ b/nailgun/nailgun/db/migration/alembic_migrations/versions/armband.py -@@ -0,0 +1,53 @@ -+# Copyright 2016 Cavium, Inc. -+# -+# Licensed under the Apache License, Version 2.0 (the "License"); you may -+# not use this file except in compliance with the License. You may obtain -+# a copy of the License at -+# -+# http://www.apache.org/licenses/LICENSE-2.0 -+# -+# Unless required by applicable law or agreed to in writing, software -+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -+# License for the specific language governing permissions and limitations -+# under the License. -+ -+"""Armband patches -+ -+Revision ID: f9b7fd91ac19 -+Revises: c6edea552f1e -+Create Date: 2016-03-01 23:18:58.712617 -+ -+""" -+ -+# revision identifiers, used by Alembic. -+revision = 'f9b7fd91ac19' -+down_revision = 'c6edea552f1e' -+ -+from alembic import op -+from nailgun.utils.migration import drop_enum -+import sqlalchemy as sa -+ -+ENUMS = ( -+ 'release_arch', -+) -+ -+ -+def upgrade(): -+ add_release_arch() -+ -+ -+def downgrade(): -+ remove_release_arch() -+ map(drop_enum, ENUMS) -+ -+ -+def add_release_arch(): -+ arch_enum = sa.Enum('amd64', 'arm64', name='release_arch') -+ arch_enum.create(op.get_bind(), checkfirst=False) -+ op.add_column('releases', sa.Column('arch', arch_enum, nullable=False, -+ server_default='amd64')) -+ -+ -+def remove_release_arch(): -+ op.drop_column('releases', 'arch') -diff --git a/nailgun/nailgun/db/sqlalchemy/models/release.py b/nailgun/nailgun/db/sqlalchemy/models/release.py -index a069b61..882d32c 100644 ---- a/nailgun/nailgun/db/sqlalchemy/models/release.py -+++ b/nailgun/nailgun/db/sqlalchemy/models/release.py -@@ -40,6 +40,14 @@ class Release(Base): - id = Column(Integer, primary_key=True) - name = Column(Unicode(100), nullable=False) - version = Column(String(30), nullable=False) -+ arch = Column( -+ Enum( -+ *consts.RELEASE_ARCHS, -+ name='release_arch' -+ ), -+ nullable=False, -+ default=consts.RELEASE_ARCHS.amd64 -+ ) - description = Column(Unicode) - operating_system = Column(String(50), nullable=False) - state = Column( -diff --git a/nailgun/nailgun/fixtures/openstack.yaml b/nailgun/nailgun/fixtures/openstack.yaml -index ba967d7..c1d2a71 100644 ---- a/nailgun/nailgun/fixtures/openstack.yaml -+++ b/nailgun/nailgun/fixtures/openstack.yaml -@@ -2113,6 +2113,7 @@ - name: "Newton on CentOS 6.5" - state: "unavailable" - version: "newton-10.0" -+ arch: "amd64" - operating_system: "CentOS" - description: "This option will install the OpenStack Mitaka packages using a CentOS based operating system. With high availability features built in, you are getting a robust, enterprise-grade OpenStack deployment." - attributes_metadata: -@@ -2186,6 +2187,7 @@ - fields: - name: "Newton on Ubuntu 16.04" - version: "newton-10.0" -+ arch: "amd64" - operating_system: "Ubuntu" - description: "This option will install the OpenStack Newton packages using Ubuntu as a base operating system. With high availability features built in, you are getting a robust, enterprise-grade OpenStack deployment." - attributes_metadata: -@@ -2417,6 +2419,7 @@ - fields: - name: "Newton on Ubuntu+UCA 16.04" - version: "newton-10.0" -+ arch: "amd64" - description: "This option will install the OpenStack Newton packages using Ubuntu as a base operating system, including Ubuntu Cloud Archive OpenStack packages. With high availability features built in, you are getting a robust, enterprise-grade OpenStack deployment." - attributes_metadata: - editable: -@@ -2517,6 +2520,7 @@ - fields: - name: "Newton on Ubuntu 16.04 (aarch64)" - version: "newton-10.0" -+ arch: "arm64" - operating_system: "Ubuntu" - description: "This option will install the OpenStack Newton packages using Ubuntu as a base operating system. With high availability features built in, you are getting a robust, enterprise-grade OpenStack deployment." - attributes_metadata: -@@ -2754,6 +2758,7 @@ - fields: - name: "Newton on Ubuntu+UCA 16.04 (aarch64)" - version: "newton-10.0" -+ arch: "arm64" - description: "This option will install the OpenStack Newton packages using Ubuntu as a base operating system, including Ubuntu Cloud Archive OpenStack packages. With high availability features built in, you are getting a robust, enterprise-grade OpenStack deployment." - attributes_metadata: - editable: -diff --git a/nailgun/nailgun/orchestrator/deployment_serializers.py b/nailgun/nailgun/orchestrator/deployment_serializers.py -index f983ae3..2f9cec2 100644 ---- a/nailgun/nailgun/orchestrator/deployment_serializers.py -+++ b/nailgun/nailgun/orchestrator/deployment_serializers.py -@@ -261,7 +261,13 @@ class DeploymentMultinodeSerializer(object): - img_dir = '/usr/share/cirros-testvm/' - else: - img_dir = '/opt/vm/' -- image_data['img_path'] = '{0}cirros-x86_64-disk.img'.format(img_dir) -+ arch = node.cluster.release.arch -+ if arch == "amd64": -+ arch = "x86_64" -+ elif arch == "arm64": -+ arch = "aarch64" -+ image_data['img_path'] = '{0}cirros-{1}-disk.img'.format(img_dir, -+ arch) - - properties_data = {} - -diff --git a/nailgun/nailgun/orchestrator/provisioning_serializers.py b/nailgun/nailgun/orchestrator/provisioning_serializers.py -index 7b2c6ac..f2c05e1 100644 ---- a/nailgun/nailgun/orchestrator/provisioning_serializers.py -+++ b/nailgun/nailgun/orchestrator/provisioning_serializers.py -@@ -329,7 +329,8 @@ class ProvisioningSerializer61(ProvisioningSerializer): - attrs['repo_setup']['repos'], - attrs['provision'], - cluster.id, -- packages)) -+ packages, -+ cluster.release.arch)) - - PriorityStrategy().one_by_one(tasks) - return tasks -diff --git a/nailgun/nailgun/orchestrator/tasks_templates.py b/nailgun/nailgun/orchestrator/tasks_templates.py -index 8252891..84bebe6 100644 ---- a/nailgun/nailgun/orchestrator/tasks_templates.py -+++ b/nailgun/nailgun/orchestrator/tasks_templates.py -@@ -208,7 +208,7 @@ def make_reboot_task(uids, task): - - - def make_provisioning_images_task( -- uids, repos, provision_data, cid, packages): -+ uids, repos, provision_data, cid, packages, arch): - conf = { - 'repos': repos, - 'image_data': provision_data['image_data'], -@@ -230,7 +230,8 @@ def make_provisioning_images_task( - "--image_build_dir /var/lib/fuel/ibp " - "--log-file /var/log/fuel-agent-env-{0}.log " - "--data_driver nailgun_build_image " -- "--input_data '{1}'").format(cid, conf), -+ "--target_arch {1} " -+ "--input_data '{2}'").format(cid, arch, conf), - 'timeout': settings.PROVISIONING_IMAGES_BUILD_TIMEOUT, - 'retries': 1}}) - -diff --git a/nailgun/nailgun/test/integration/test_cluster_changes_handler.py b/nailgun/nailgun/test/integration/test_cluster_changes_handler.py -index d7e964e..bce8289 100644 ---- a/nailgun/nailgun/test/integration/test_cluster_changes_handler.py -+++ b/nailgun/nailgun/test/integration/test_cluster_changes_handler.py -@@ -148,7 +148,7 @@ class TestHandlers(BaseIntegrationTest): - common_attrs['last_controller'] = controller_nodes[-1]['name'] - common_attrs['storage']['pg_num'] = 128 - -- common_attrs['test_vm_image'] = { -+ common_attrs['test_vm_image'] = [{ - 'container_format': 'bare', - 'public': 'true', - 'disk_format': 'qcow2', -@@ -164,7 +164,7 @@ class TestHandlers(BaseIntegrationTest): - 'murano_image_info': """'{"title": "Murano Demo", "type":""" - """ "cirros.demo"}'""", - }, -- } -+ }] - - critical_mapping = { - 'primary-controller': True, -@@ -565,7 +565,7 @@ class TestHandlers(BaseIntegrationTest): - common_attrs['last_controller'] = controller_nodes[-1]['name'] - common_attrs['storage']['pg_num'] = 128 - -- common_attrs['test_vm_image'] = { -+ common_attrs['test_vm_image'] = [{ - 'container_format': 'bare', - 'public': 'true', - 'disk_format': 'qcow2', -@@ -581,7 +581,7 @@ class TestHandlers(BaseIntegrationTest): - 'murano_image_info': """'{"title": "Murano Demo", "type":""" - """ "cirros.demo"}'""", - }, -- } -+ }] - - critical_mapping = { - 'primary-controller': True, -@@ -1053,7 +1053,7 @@ class TestHandlers(BaseIntegrationTest): - common_attrs['last_controller'] = controller_nodes[-1]['name'] - common_attrs['storage']['pg_num'] = 128 - -- common_attrs['test_vm_image'] = { -+ common_attrs['test_vm_image'] = [{ - 'container_format': 'bare', - 'public': 'true', - 'disk_format': 'qcow2', -@@ -1069,7 +1069,7 @@ class TestHandlers(BaseIntegrationTest): - 'murano_image_info': """'{"title": "Murano Demo", "type":""" - """ "cirros.demo"}'""", - }, -- } -+ }] - - critical_mapping = { - 'primary-controller': True, -diff --git a/nailgun/nailgun/test/integration/test_orchestrator_serializer.py b/nailgun/nailgun/test/integration/test_orchestrator_serializer.py -index f399602..6034f30 100644 ---- a/nailgun/nailgun/test/integration/test_orchestrator_serializer.py -+++ b/nailgun/nailgun/test/integration/test_orchestrator_serializer.py -@@ -2565,12 +2565,12 @@ class BaseDeploymentSerializer(BaseSerializerTest): - - def check_no_murano_data(self): - glance_properties = self.serializer.generate_test_vm_image_data( -- self.env.nodes[0])['test_vm_image']['glance_properties'] -+ self.env.nodes[0])['test_vm_image'][0]['glance_properties'] - self.assertNotIn('murano_image_info', glance_properties) - - def check_murano_data(self): - glance_properties = self.serializer.generate_test_vm_image_data( -- self.env.nodes[0])['test_vm_image']['glance_properties'] -+ self.env.nodes[0])['test_vm_image'][0]['glance_properties'] - self.assertIn('murano_image_info', glance_properties) - - @staticmethod -diff --git a/nailgun/nailgun/test/unit/test_tasks_templates.py b/nailgun/nailgun/test/unit/test_tasks_templates.py -index 4db183f..4f5d2cb 100644 ---- a/nailgun/nailgun/test/unit/test_tasks_templates.py -+++ b/nailgun/nailgun/test/unit/test_tasks_templates.py -@@ -140,7 +140,8 @@ class TestMakeTask(base.BaseTestCase): - } - }}, - cid=123, -- packages=packages -+ packages=packages, -+ arch='amd64' - ) - - fuel_image_conf = { -@@ -177,7 +178,8 @@ class TestMakeTask(base.BaseTestCase): - cmd = result["parameters"]["cmd"].lstrip( - "fa_build_image --image_build_dir /var/lib/fuel/ibp " - "--log-file /var/log/fuel-agent-env-123.log " -- "--data_driver nailgun_build_image --input_data '").rstrip("'") -+ "--data_driver nailgun_build_image --target_arch amd64" -+ " --input_data '").rstrip("'") - self.assertEqual(jsonutils.loads(cmd), fuel_image_conf) - - def test_generate_ironic_bootstrap_keys_task(self): diff --git a/patches/fuel-web/0003-AArch64-Cirros-TestVM-Use-AAVMF-guest-UEFI.patch b/patches/fuel-web/0003-AArch64-Cirros-TestVM-Use-AAVMF-guest-UEFI.patch deleted file mode 100644 index 64e60b31..00000000 --- a/patches/fuel-web/0003-AArch64-Cirros-TestVM-Use-AAVMF-guest-UEFI.patch +++ /dev/null @@ -1,36 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Sat, 21 Jan 2017 00:01:16 +0100 -Subject: [PATCH] AArch64: Cirros TestVM: Use AAVMF (guest UEFI) - -Based on the direct kernel boot patch by Stan. - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> ---- - nailgun/nailgun/orchestrator/deployment_serializers.py | 7 +++++++ - 1 file changed, 7 insertions(+) - -diff --git a/nailgun/nailgun/orchestrator/deployment_serializers.py b/nailgun/nailgun/orchestrator/deployment_serializers.py -index 77fcba1..fd33b3f 100644 ---- a/nailgun/nailgun/orchestrator/deployment_serializers.py -+++ b/nailgun/nailgun/orchestrator/deployment_serializers.py -@@ -284,4 +284,11 @@ class DeploymentMultinodeSerializer(object): - -+ # AArch64: Use AAVMF (guest UEFI) -+ if arch == 'aarch64': -+ properties_data = { -+ 'hw_firmware_type': 'uefi', -+ 'short_id': 'ubuntu16.04' -+ } -+ - # NOTE(aschultz): properties was added as part of N and should be - # used infavor of glance_properties - image_data['glance_properties'] = self.generate_properties_arguments( diff --git a/patches/fuel-web/0004-nailgun-Openstack-release-Pin-kernel-to-4.8.0-9944.patch b/patches/fuel-web/0004-nailgun-Openstack-release-Pin-kernel-to-4.8.0-9944.patch deleted file mode 100644 index a617c809..00000000 --- a/patches/fuel-web/0004-nailgun-Openstack-release-Pin-kernel-to-4.8.0-9944.patch +++ /dev/null @@ -1,68 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Thu, 23 Mar 2017 01:40:45 +0100 -Subject: [PATCH] nailgun: Openstack release: Pin kernel to 4.8.0-9944 - -Since newer kernels have issues with AAVMF + vhost_net, -we need to also temporarily blacklist vhost_net. - -NOTE: This change also adds linux-image-extra packages to both x86_64 - and AArch64 target image(s). - -TODO: Remove vhost_net blacklist (later). - -JIRA: https://jira.opnfv.org/browse/ARMBAND-161 -JIRA: https://jira.opnfv.org/browse/ARMBAND-171 -JIRA: https://jira.opnfv.org/browse/ARMBAND-227 -JIRA: https://jira.opnfv.org/browse/ARMBAND-230 -JIRA: https://jira.opnfv.org/browse/ARMBAND-239 -JIRA: https://jira.opnfv.org/browse/ARMBAND-244 - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - nailgun/nailgun/fixtures/openstack.yaml | 11 +++++++---- - 1 file changed, 7 insertions(+), 4 deletions(-) - -diff --git a/nailgun/nailgun/fixtures/openstack.yaml b/nailgun/nailgun/fixtures/openstack.yaml -index 5712006..d404168 100644 ---- a/nailgun/nailgun/fixtures/openstack.yaml -+++ b/nailgun/nailgun/fixtures/openstack.yaml -@@ -2164,8 +2164,9 @@ - grub-pc - hwloc - linux-firmware -- linux-headers-generic-lts-xenial -- linux-image-generic-lts-xenial -+ linux-headers-4.8.0-9944-generic -+ linux-image-4.8.0-9944-generic -+ linux-image-extra-4.8.0-9944-generic - lvm2 - mcollective - mdadm -@@ -2492,6 +2493,7 @@ - value: | - acl - anacron -+ armband-vhost-net-blacklist - bash-completion - bridge-utils - bsdmainutils -@@ -2504,8 +2506,9 @@ - grub-efi-arm64 - hwloc - linux-firmware -- linux-headers-generic-lts-xenial -- linux-image-generic-lts-xenial -+ linux-headers-4.8.0-9944-generic -+ linux-image-4.8.0-9944-generic -+ linux-image-extra-4.8.0-9944-generic - lvm2 - mcollective - mdadm diff --git a/patches/fuel-web/0005-Update-Host-OS-NTP-Servers-to-pool.ntp.org.patch b/patches/fuel-web/0005-Update-Host-OS-NTP-Servers-to-pool.ntp.org.patch deleted file mode 100644 index a8a57a42..00000000 --- a/patches/fuel-web/0005-Update-Host-OS-NTP-Servers-to-pool.ntp.org.patch +++ /dev/null @@ -1,36 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Catalina Focsa <catalina.focsa@enea.com> -Date: Mon, 3 Apr 2017 11:30:10 +0200 -Subject: [PATCH] Update Host OS NTP Servers to pool.ntp.org - -Old *.fuel.pool.ntp.org values are no longer accessible -LP: https://bugs.launchpad.net/fuel/+bug/1679636 -JIRA: https://jira.opnfv.org/browse/ARMBAND-225 - -Signed-off-by: Catalina Focsa <catalina.focsa@enea.com> ---- - nailgun/nailgun/settings.yaml | 4 ++-- - 1 file changed, 2 insertions(+), 2 deletions(-) - -diff --git a/nailgun/nailgun/settings.yaml b/nailgun/nailgun/settings.yaml -index 92b3518..1041c66 100644 ---- a/nailgun/nailgun/settings.yaml -+++ b/nailgun/nailgun/settings.yaml -@@ -1100,8 +1100,8 @@ DNS_UPSTREAM: - - '8.8.8.8' - - '8.8.4.4' - NTP_UPSTREAM: -- - '0.fuel.pool.ntp.org' -- - '1.fuel.pool.ntp.org' -+ - '0.pool.ntp.org' -+ - '1.pool.ntp.org' - - # deadlocks detection settings - LOG_DEADLOCKS_WARNINGS: 1 diff --git a/patches/fuel-web/0006-nailgun-Add-NIC-vendorid-and-device-id-for-Cavium-NI.patch b/patches/fuel-web/0006-nailgun-Add-NIC-vendorid-and-device-id-for-Cavium-NI.patch deleted file mode 100644 index 196cea52..00000000 --- a/patches/fuel-web/0006-nailgun-Add-NIC-vendorid-and-device-id-for-Cavium-NI.patch +++ /dev/null @@ -1,50 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Andrei Varvara <andrei.varvara@enea.com> -Date: Thu, 4 May 2017 13:35:32 +0200 -Subject: [PATCH] nailgun: Add NIC vendorid and device id for Cavium NIC - -In order to enable DPDK and ovs-dpdk in Fuel, a checkbox must be enabled -in the interface configuration menu of the nodes. - -This did not work on ThunderX based PODs, because Fuel filters out NICs based -on Vendor Id (currently Intel only) and Product Id. This is implemented -via a static list of known Vid and Pid pairs in openstack.yaml file. - -Added Cavium NIC to the list of DPDK supported NICs so that fuel will -enable the DPDK checkbox. - -Signed-off-by: Andrei Varvara <andrei.varvara@enea.com> ---- - nailgun/nailgun/fixtures/openstack.yaml | 4 ++-- - 1 file changed, 2 insertions(+), 2 deletions(-) - -diff --git a/nailgun/nailgun/fixtures/openstack.yaml b/nailgun/nailgun/fixtures/openstack.yaml -index 0b27611..070004e 100644 ---- a/nailgun/nailgun/fixtures/openstack.yaml -+++ b/nailgun/nailgun/fixtures/openstack.yaml -@@ -2293,7 +2293,7 @@ - "15ad:07b0", "8086:15a5", "1137:0043", "1137:0071", "14e4:168a", - "14e4:16a9", "14e4:164f", "14e4:168e", "14e4:16af", "14e4:163d", - "14e4:163f", "14e4:168d", "14e4:16a1", "14e4:16a2", "14e4:16ad", -- "14e4:16ae", "14e4:163e", "14e4:16a4", "8086:10f8" -+ "14e4:16ae", "14e4:163e", "14e4:16a4", "8086:10f8", "177d:a034" - ] - - pk: 3 - extend: *ubuntu_xenial -@@ -2635,7 +2635,7 @@ - "15ad:07b0", "8086:15a5", "1137:0043", "1137:0071", "14e4:168a", - "14e4:16a9", "14e4:164f", "14e4:168e", "14e4:16af", "14e4:163d", - "14e4:163f", "14e4:168d", "14e4:16a1", "14e4:16a2", "14e4:16ad", -- "14e4:16ae", "14e4:163e", "14e4:16a4" -+ "14e4:16ae", "14e4:163e", "14e4:16a4", "177d:a034" - ] - - pk: 5 - extend: *ubuntu_aarch64_xenial - diff --git a/patches/fuel-web/cross-bootstrap/0000-Add-arch-to-nailgun-release-and-target-image.patch b/patches/fuel-web/cross-bootstrap/0000-Add-arch-to-nailgun-release-and-target-image.patch deleted file mode 120000 index 64ff45c3..00000000 --- a/patches/fuel-web/cross-bootstrap/0000-Add-arch-to-nailgun-release-and-target-image.patch +++ /dev/null @@ -1 +0,0 @@ -../0002-Add-arch-to-nailgun-release-and-target-image.patch
\ No newline at end of file diff --git a/patches/fuel-web/multiarch-fuel/0000-Add-arch-to-nailgun-release-and-target-image.patch b/patches/fuel-web/multiarch-fuel/0000-Add-arch-to-nailgun-release-and-target-image.patch deleted file mode 120000 index 64ff45c3..00000000 --- a/patches/fuel-web/multiarch-fuel/0000-Add-arch-to-nailgun-release-and-target-image.patch +++ /dev/null @@ -1 +0,0 @@ -../0002-Add-arch-to-nailgun-release-and-target-image.patch
\ No newline at end of file diff --git a/patches/fuel-web/multiarch-fuel/0000-nailgun-Add-AArch64-Openstack-Newton-release-s.patch b/patches/fuel-web/multiarch-fuel/0000-nailgun-Add-AArch64-Openstack-Newton-release-s.patch deleted file mode 120000 index 2e9288ee..00000000 --- a/patches/fuel-web/multiarch-fuel/0000-nailgun-Add-AArch64-Openstack-Newton-release-s.patch +++ /dev/null @@ -1 +0,0 @@ -../0001-nailgun-Add-AArch64-Openstack-Newton-release-s.patch
\ No newline at end of file diff --git a/patches/fuel-web/multiarch-fuel/0001-FIXME-Disable-amd64-Newton-on-Ubuntu-for-now.patch b/patches/fuel-web/multiarch-fuel/0001-FIXME-Disable-amd64-Newton-on-Ubuntu-for-now.patch deleted file mode 100644 index abfb9529..00000000 --- a/patches/fuel-web/multiarch-fuel/0001-FIXME-Disable-amd64-Newton-on-Ubuntu-for-now.patch +++ /dev/null @@ -1,40 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Sat, 19 Mar 2016 19:02:15 +0100 -Subject: [PATCH] FIXME: Disable amd64 Newton on Ubuntu for now. - -Until we fix all cohabitation of amd64 and arm64, disable Openstack -releases <Newton on Ubuntu 16.04> and <Newton on Ubuntu+UCA 16.04> -(both x86_64) in openstack.yaml fixture. - -TODO: Propose renaming above releases to include arch in name. -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - nailgun/nailgun/fixtures/openstack.yaml | 2 ++ - 1 file changed, 2 insertions(+) - -diff --git a/nailgun/nailgun/fixtures/openstack.yaml b/nailgun/nailgun/fixtures/openstack.yaml ---- a/nailgun/nailgun/fixtures/openstack.yaml -+++ b/nailgun/nailgun/fixtures/openstack.yaml -@@ -1980,6 +1980,7 @@ - extend: *base_release - fields: - name: "Newton on Ubuntu 16.04" -+ state: "unavailable" - version: "newton-10.0" - arch: "amd64" - operating_system: "Ubuntu" -@@ -2221,5 +2222,6 @@ - fields: - name: "Newton on Ubuntu+UCA 16.04" -+ state: "unavailable" - version: "newton-10.0" - arch: "amd64" - description: "This option will install the OpenStack Newton packages using Ubuntu as a base operating system, including Ubuntu Cloud Archive OpenStack packages. With high availability features built in, you are getting a robust, enterprise-grade OpenStack deployment." diff --git a/patches/fuel-web/mysql-sst-provider/0001-Add-configuration-item-for-WSREP-SST-provider.patch b/patches/fuel-web/mysql-sst-provider/0001-Add-configuration-item-for-WSREP-SST-provider.patch deleted file mode 100644 index 97d9d967..00000000 --- a/patches/fuel-web/mysql-sst-provider/0001-Add-configuration-item-for-WSREP-SST-provider.patch +++ /dev/null @@ -1,53 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Date: Wed, 20 Apr 2016 15:05:15 -0700 -Subject: [PATCH] Add configuration item for WSREP SST provider - -This patch adds a radio button in "OpenStack Services" tab of the -environment settings to configure which WSREP SST provider to use for -MySQL. - -Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> ---- - nailgun/nailgun/fixtures/openstack.yaml | 21 +++++++++++++++++++++ - 1 file changed, 21 insertions(+) - -diff --git a/nailgun/nailgun/fixtures/openstack.yaml b/nailgun/nailgun/fixtures/openstack.yaml -index ba25093..e855cf0 100644 ---- a/nailgun/nailgun/fixtures/openstack.yaml -+++ b/nailgun/nailgun/fixtures/openstack.yaml -@@ -932,6 +932,27 @@ - description: "Name for Mongo replication set" - weight: 30 - type: "text" -+ mysql_wsrep: -+ metadata: -+ label: "Database configuration" -+ weight: 15 -+ group: "openstack_services" -+ wsrep_method: -+ value: "xtrabackup-v2" -+ label: "WSREP SST provider" -+ description: "This setting changes the wsrep_sst_method switch in the MySQL Galera cluster configuration." -+ weight: 10 -+ type: "radio" -+ values: -+ - data: "xtrabackup-v2" -+ label: "xtrabackup-v2" -+ description: "Use Percona xtrabackup v2 provider. This is broken on some ARM64 platforms due to outdated MySQL/InnoDB code used by xtrabackup." -+ - data: "rsync" -+ label: "rsync" -+ description: "Use rsync provider. Use if xtrabackup is broken on your platform." -+ - data: "mysqldump" -+ label: "mysqldump" -+ description: "Use mysqldump provider (untested)." - additional_components: - metadata: - label: "Additional Components" diff --git a/patches/opnfv-fuel/0000-f_odlpluginbuild-Armband-prereq-in-packages.yaml.patch b/patches/opnfv-fuel/0000-f_odlpluginbuild-Armband-prereq-in-packages.yaml.patch deleted file mode 100644 index dcf3b15f..00000000 --- a/patches/opnfv-fuel/0000-f_odlpluginbuild-Armband-prereq-in-packages.yaml.patch +++ /dev/null @@ -1,47 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Sat, 19 Nov 2016 20:39:35 +0100 -Subject: [PATCH] f_odlpluginbuild: Armband prereq in packages.yaml - -We need to explicitily add our Armband leveldb patching prereq to -the ODL plugin's packages.yaml, so they are included in the -offline Ubuntu/MOS mirrors. - -After ARMBAND-114 was implemented, the dependency list is only: -- armband-odl-leveldb-fix (for patching leveldbjni); -- libasound2 (openjdk-8-jre amos version additional dep); - -Note: armband-odl-leveldb-fix MUST be also added to MOS mirror -package list, since it's not present in Ubuntu repos. - -JIRA: ARMBAND-167 -JIRA: ARMBAND-114 - -FIXME: These are only required for arm64, so maybe later we could -filter out other archs by extending the packages.yaml with an <arch> -field. - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - build/f_isoroot/f_odlpluginbuild/packages.yaml | 5 +++++ - 1 file changed, 5 insertions(+) - create mode 100644 build/f_isoroot/f_odlpluginbuild/packages.yaml - -diff --git a/build/f_isoroot/f_odlpluginbuild/packages.yaml b/build/f_isoroot/f_odlpluginbuild/packages.yaml -new file mode 100644 -index 0000000..b1bf086 ---- /dev/null -+++ b/build/f_isoroot/f_odlpluginbuild/packages.yaml -@@ -0,0 +1,5 @@ -+# Armband: openjdk-8-jre depends on libasound2 -+# Armband: leveldb patching requires armband-odl-leveldb-fix -+packages: -+ - armband-odl-leveldb-fix -+ - libasound2 diff --git a/patches/opnfv-fuel/0001-build-Add-ARMBAND_BASE-support.patch b/patches/opnfv-fuel/0001-build-Add-ARMBAND_BASE-support.patch deleted file mode 100644 index c320a8ca..00000000 --- a/patches/opnfv-fuel/0001-build-Add-ARMBAND_BASE-support.patch +++ /dev/null @@ -1,78 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Thu, 11 Aug 2016 15:34:32 +0200 -Subject: [PATCH] build: Add ARMBAND_BASE support - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - build/Makefile | 10 ++++++++++ - build/docker/Dockerfile | 2 +- - build/docker/runcontext | 6 +++++- - 3 files changed, 18 insertions(+), 2 deletions(-) - -diff --git a/build/Makefile b/build/Makefile -index 4454c35..377ecc6 100644 ---- a/build/Makefile -+++ b/build/Makefile -@@ -14,4 +14,9 @@ SHELL = /bin/bash - -+# Only configure Armband specific stuff when ARMBAND_BASE is set -+ifdef ARMBAND_BASE -+ include ${ARMBAND_BASE}/armband-fuel-config.mk -+endif -+ - ############################################################################ - # BEGIN of variables to customize - # -@@ -140,6 +140,7 @@ $(ISOCACHE): - # fuel-main Makefiles do not like `make -C` - cd $(FUEL_MAIN_DIR) && make repos - cp f_repos/.cachefuelinfo gitinfo_fuel.txt -+ test -z $(ARMBAND_BASE) || $(REPOINFO) $(ARMBAND_BASE) >> gitinfo_fuel.txt - - # Repeat build up to three times - sudo -E ./fuel_build_loop -diff --git a/build/docker/Dockerfile b/build/docker/Dockerfile -index b38ea4c..624f233 100644 ---- a/build/docker/Dockerfile -+++ b/build/docker/Dockerfile -@@ -30,7 +30,7 @@ RUN echo "Defaults env_keep += \"ftp_proxy http_proxy https_proxy no_proxy RSYNC - # Keeping PWD is needed to build as root - RUN echo "Defaults env_keep += \"PWD\"" > /etc/sudoers.d/keep-pwd - # Keeping variables for ISO build --RUN echo "Defaults env_keep += \"MIRROR_UBUNTU MIRROR_UBUNTU_ROOT MIRROR_MOS_UBUNTU MIRROR_MOS_UBUNTU_ROOT MIRROR_FUEL LATEST_TARGET_UBUNTU UBUNTU_ARCH\"" > /etc/sudoers.d/keep-mos -+RUN echo "Defaults env_keep += \"MIRROR_UBUNTU MIRROR_UBUNTU_ROOT MIRROR_MOS_UBUNTU MIRROR_MOS_UBUNTU_ROOT MIRROR_FUEL LATEST_TARGET_UBUNTU UBUNTU_ARCH ARMBAND_BASE\"" > /etc/sudoers.d/keep-mos - RUN chmod 0440 /etc/sudoers.d/open-sudo - RUN chmod 0440 /etc/sudoers.d/keep-proxies - RUN chmod 0440 /etc/sudoers.d/keep-pwd -diff --git a/build/docker/runcontext b/build/docker/runcontext -index daad663..e4874df 100755 ---- a/build/docker/runcontext -+++ b/build/docker/runcontext -@@ -42,6 +42,10 @@ GITROOT=`git rev-parse --show-toplevel` - CID_FILE=`mktemp -u -t runcontext.XXXXXXXXXX` - CONTEXT_DIR=`mktemp -d ${GITROOT}/.docker_contextXXXXXX` - -+if [[ $ARMBAND_BASE ]]; then -+ GITROOT=$ARMBAND_BASE -+fi -+ - # If RSYNC_CONNECT_PROG is used, we need to copy all of - # the SSH structure, should one of the keys need to be - # used. -@@ -115,7 +119,7 @@ RUN_CONTEXT_OPT="--cidfile $CID_FILE --privileged=true --rm \ - -e HOME=$HOME -e CACHEDEBUG -e CACHETRANSPORT -e CACHEMAXAGE -e CACHEBASE \ - -e BUILD_FUEL_PLUGINS -e MIRROR_UBUNTU -e MIRROR_UBUNTU_ROOT \ - -e MIRROR_MOS_UBUNTU -e MIRROR_MOS_UBUNTU_ROOT -e MIRROR_FUEL \ -- -e LATEST_TARGET_UBUNTU -e UBUNTU_ARCH -e OPNFV_GIT_SHA \ -+ -e LATEST_TARGET_UBUNTU -e UBUNTU_ARCH -e OPNFV_GIT_SHA -e ARMBAND_BASE \ - -u $USER_ID:$GROUP_ID -w $PWD \ - -v $GITROOT:$GITROOT -v /sys/fs/cgroup:/sys/fs/cgroup:ro $CACHEMOUNT" - diff --git a/patches/opnfv-fuel/0003-dea_base-replace-grub-pc-by-grub-efi-arm64.patch b/patches/opnfv-fuel/0003-dea_base-replace-grub-pc-by-grub-efi-arm64.patch deleted file mode 100644 index e06e69e6..00000000 --- a/patches/opnfv-fuel/0003-dea_base-replace-grub-pc-by-grub-efi-arm64.patch +++ /dev/null @@ -1,28 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Josep Puigdemont <josep.puigdemont@enea.com> -Date: Mon, 11 Jul 2016 15:21:56 +0200 -Subject: [PATCH] dea_base: replace grub-pc by grub-efi-arm64 - -Signed-off-by: Josep Puigdemont <josep.puigdemont@enea.com> ---- - deploy/config/dea_base.yaml | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - -diff --git a/deploy/config/dea_base.yaml b/deploy/config/dea_base.yaml -index 0b8485b..658b328 100644 ---- a/deploy/config/dea_base.yaml -+++ b/deploy/config/dea_base.yaml -@@ -462,5 +462,5 @@ settings: - - gdisk - -- grub-pc -+ grub-efi-arm64 - diff --git a/patches/opnfv-fuel/0004-post-scripts-Allow-SSH-on-non-admin-ifaces.patch b/patches/opnfv-fuel/0004-post-scripts-Allow-SSH-on-non-admin-ifaces.patch deleted file mode 100644 index 157299de..00000000 --- a/patches/opnfv-fuel/0004-post-scripts-Allow-SSH-on-non-admin-ifaces.patch +++ /dev/null @@ -1,55 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Tue, 12 Jul 2016 16:12:25 +0200 -Subject: [PATCH] post-scripts: Allow SSH on non-admin ifaces. - -By default, Fuel 9.0 configures iptables to only accept SSH connections -on admin interface. - -If more than the admin interface is configured (e.g. by transplant script -or manually in fuel menu), whitelist SSH connections on all ifaces. - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - .../post-scripts/10_accept_ssh_all_ifaces.sh | 25 ++++++++++++++++++++++ - 1 file changed, 25 insertions(+) - create mode 100755 build/f_isoroot/f_bootstrap/post-scripts/10_accept_ssh_all_ifaces.sh - -diff --git a/build/f_isoroot/f_bootstrap/post-scripts/10_accept_ssh_all_ifaces.sh b/build/f_isoroot/f_bootstrap/post-scripts/10_accept_ssh_all_ifaces.sh -new file mode 100755 -index 0000000..b551516 ---- /dev/null -+++ b/build/f_isoroot/f_bootstrap/post-scripts/10_accept_ssh_all_ifaces.sh -@@ -0,0 +1,25 @@ -+#/bin/sh -+############################################################################## -+# Copyright (c) 2016 Enea AB and others. -+# Alexandru.Avadanii@enea.com -+# All rights reserved. This program and the accompanying materials -+# are made available under the terms of the Apache License, Version 2.0 -+# which accompanies this distribution, and is available at -+# http://www.apache.org/licenses/LICENSE-2.0 -+############################################################################## -+ -+# Only mess with iptables if we have additional interfaces configured -+if grep -q "DEFROUTE=no" "/etc/sysconfig/network-scripts/ifcfg-eth0"; then -+ echo "iptables: Allow SSH connections on all interfaces" -+ # By default, Fuel 9.0 configures iptables to only accept SSH connections -+ # on admin interface. Whitelist SSH connections on all ifaces. -+ while [ $? -eq 0 ]; do -+ # First, try removing the rule we want to add to prevent duplicates -+ iptables -D INPUT -p tcp --dport ssh -j ACCEPT > /dev/null 2>&1; -+ done -+ iptables -A INPUT -p tcp --dport ssh -j ACCEPT -+ service iptables save -+ echo "iptables: Done configuring SSH" -+else -+ echo "iptables: Skipping configuring SSH for non-admin ifaces" -+fi diff --git a/patches/opnfv-fuel/0005-deploy.py-add-a-time-stamp-to-the-new-ISO-image.patch b/patches/opnfv-fuel/0005-deploy.py-add-a-time-stamp-to-the-new-ISO-image.patch deleted file mode 100644 index d8717156..00000000 --- a/patches/opnfv-fuel/0005-deploy.py-add-a-time-stamp-to-the-new-ISO-image.patch +++ /dev/null @@ -1,47 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Josep Puigdemont <josep.puigdemont@enea.com> -Date: Tue, 17 May 2016 14:40:48 +0200 -Subject: [PATCH] deploy.py: add a time stamp to the new ISO image - -Add a time stamp to the newly create ISO image. This is to prevent name -collisions with other ISO images from earlier deploy jobs. - -Ideally this will only be a temporary need, and releng will provide with -unique IDs for each job that we can use to this effect. - -Signed-off-by: Josep Puigdemont <josep.puigdemont@enea.com> ---- - deploy/deploy.py | 6 ++++-- - 1 file changed, 4 insertions(+), 2 deletions(-) - -diff --git a/deploy/deploy.py b/deploy/deploy.py -index 041ba2f..cebef97 100755 ---- a/deploy/deploy.py -+++ b/deploy/deploy.py -@@ -13,6 +13,7 @@ import os - import io - import re - import sys -+import time - import yaml - import signal - import netaddr -@@ -100,8 +101,9 @@ class AutoDeploy(object): - - def install_fuel_master(self): - log('Install Fuel Master') -- new_iso = ('%s/deploy-%s' -- % (self.tmp_dir, os.path.basename(self.iso_file))) -+ stamp = time.strftime("%Y%m%d%H%M%S") -+ new_iso = ('%s/deploy-%s-%s' -+ % (self.tmp_dir, stamp, os.path.basename(self.iso_file))) - self.patch_iso(new_iso) - self.iso_file = new_iso - self.install_iso() diff --git a/patches/opnfv-fuel/0006-build-docker-Use-host-s-network-stack.patch b/patches/opnfv-fuel/0006-build-docker-Use-host-s-network-stack.patch deleted file mode 100644 index bf52fc23..00000000 --- a/patches/opnfv-fuel/0006-build-docker-Use-host-s-network-stack.patch +++ /dev/null @@ -1,36 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Mon, 29 Aug 2016 22:58:49 +0200 -Subject: [PATCH] build: docker: Use host's /etc/hosts - -The original motivation was pointing to local mirrors via host's -"/etc/hosts", for which an alternative solution tested was -"--net=host" (see [1] for full description), but lead to wierd -loss of connectivity after a few builds. - -[1] https://docs.docker.com/v1.8/articles/networking/ - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - build/docker/runcontext | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - -diff --git a/build/docker/runcontext b/build/docker/runcontext -index 07b0dc1..b0d6d37 100755 ---- a/build/docker/runcontext -+++ b/build/docker/runcontext -@@ -121,6 +121,7 @@ RUN_CONTEXT_OPT="--cidfile $CID_FILE --privileged=true --rm --net=host \ - -e MIRROR_MOS_UBUNTU -e MIRROR_MOS_UBUNTU_ROOT -e MIRROR_FUEL \ - -e LATEST_TARGET_UBUNTU -e UBUNTU_ARCH -e OPNFV_GIT_SHA -e ARMBAND_BASE \ - -u $USER_ID:$GROUP_ID -w $PWD \ -+ -v /etc/hosts:/etc/hosts \ - -v $GITROOT:$GITROOT -v /sys/fs/cgroup:/sys/fs/cgroup:ro $CACHEMOUNT" - - # Passing "debug" puts up an interactive bash shell diff --git a/patches/opnfv-fuel/0007-deploy-Fix-add-bootstrap-DEA-override-delay.patch b/patches/opnfv-fuel/0007-deploy-Fix-add-bootstrap-DEA-override-delay.patch deleted file mode 100644 index 84ede48d..00000000 --- a/patches/opnfv-fuel/0007-deploy-Fix-add-bootstrap-DEA-override-delay.patch +++ /dev/null @@ -1,40 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Wed, 17 Aug 2016 16:18:26 +0200 -Subject: [PATCH] deploy: Fix/add bootstrap DEA override delay - -Previous change adding support for DEA to override bootstrap config -did not account for slow execution on remote servers, so add a -one minute sleep before checking for completition of fuel install. - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - deploy/install_fuel_master.py | 4 +++- - 1 file changed, 3 insertions(+), 1 deletion(-) - -diff --git a/deploy/install_fuel_master.py b/deploy/install_fuel_master.py -index 808d0b1..1a7685a 100644 ---- a/deploy/install_fuel_master.py -+++ b/deploy/install_fuel_master.py -@@ -196,11 +196,13 @@ class InstallFuelMaster(object): - self.work_dir, os.path.basename(self.dea_file))) - - def wait_until_installation_completed(self): -- WAIT_LOOP = 360 -+ WAIT_LOOP = 720 - SLEEP_TIME = 10 - CMD = 'pgrep -f %s' % BOOTSTRAP_ADMIN - - install_completed = False -+ time.sleep(60) -+ self.wait_for_node_up() - with self.ssh: - for i in range(WAIT_LOOP): - ret = self.ssh.exec_cmd(CMD) diff --git a/patches/opnfv-fuel/0008-build-docker-Import-Enea-Armband-keys.patch b/patches/opnfv-fuel/0008-build-docker-Import-Enea-Armband-keys.patch deleted file mode 100644 index fbbaca3b..00000000 --- a/patches/opnfv-fuel/0008-build-docker-Import-Enea-Armband-keys.patch +++ /dev/null @@ -1,31 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Fri, 28 Oct 2016 16:06:20 +0200 -Subject: [PATCH] build/docker: Import Enea Armband keys - -JIRA: ARMBAND-96 - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - build/docker/Dockerfile | 2 ++ - 1 file changed, 2 insertions(+) - -diff --git a/build/docker/Dockerfile b/build/docker/Dockerfile -index 4beb560..04c4d2b 100644 ---- a/build/docker/Dockerfile -+++ b/build/docker/Dockerfile -@@ -33,5 +33,7 @@ RUN git clone -b INSERT_FPB_BRANCH INSERT_FPB_REPO && cd fuel-plugins && \ - # fuel-infra key - RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BCE5CC461FA22B08 -+# Enea Armband key -+RUN apt-key adv --keyserver keys.gnupg.net --recv-keys DEAB90BA798AB1D1 - - RUN echo "ALL ALL=NOPASSWD: ALL" > /etc/sudoers.d/open-sudo - RUN echo "Defaults env_keep += \"ftp_proxy http_proxy https_proxy no_proxy RSYNC_PROXY RSYNC_CONNECT_PROG npm_config_registry\"" > /etc/sudoers.d/keep-proxies diff --git a/patches/opnfv-fuel/0009-f_odlpluginbuild-Always-inherit-Armband-settings.patch b/patches/opnfv-fuel/0009-f_odlpluginbuild-Always-inherit-Armband-settings.patch deleted file mode 100644 index 724ee9c4..00000000 --- a/patches/opnfv-fuel/0009-f_odlpluginbuild-Always-inherit-Armband-settings.patch +++ /dev/null @@ -1,38 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Tue, 24 Jan 2017 19:59:52 +0100 -Subject: [PATCH] f_odlpluginbuild: Always inherit Armband settings - -This should eliminate any possible confusion when building the -ODL plugin manually (outside OPNFV ISO build), which used to -require setting up some additional enviroment variables, like -UBUNTU_ARCH. - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - build/f_isoroot/f_odlpluginbuild/Makefile | 5 +++++ - 1 file changed, 5 insertions(+) - -diff --git a/build/f_isoroot/f_odlpluginbuild/Makefile b/build/f_isoroot/f_odlpluginbuild/Makefile -index d3a7ea8..4c77abc 100644 ---- a/build/f_isoroot/f_odlpluginbuild/Makefile -+++ b/build/f_isoroot/f_odlpluginbuild/Makefile -@@ -9,6 +9,11 @@ - - TOP := $(shell pwd) - -+# Configure Armband specific stuff when ARMBAND_BASE is set -+ifdef ARMBAND_BASE -+ include ${ARMBAND_BASE}/armband-fuel-config.mk -+endif -+ - include config.mk - - .PHONY: all diff --git a/patches/opnfv-fuel/0010-reap.py-Support-reading-MTU-values.patch b/patches/opnfv-fuel/0010-reap.py-Support-reading-MTU-values.patch deleted file mode 100644 index c55824bd..00000000 --- a/patches/opnfv-fuel/0010-reap.py-Support-reading-MTU-values.patch +++ /dev/null @@ -1,104 +0,0 @@ --:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: --: Copyright (c) 2017 Enea AB and others. --: --: All rights reserved. This program and the accompanying materials --: are made available under the terms of the Apache License, Version 2.0 --: which accompanies this distribution, and is available at --: http://www.apache.org/licenses/LICENSE-2.0 --:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Catalina Focsa <catalina.focsa@enea.com> -Date: Wed, 3 May 2017 14:53:30 +0200 -Subject: [PATCH] reap.py: Support reading MTU values - -Add an interfaces_properties list containing -mtu data for each interface - -JIRA: ARMBAND-251 - -Signed-off-by: Catalina Focsa <catalina.focsa@enea.com> ---- - deploy/reap.py | 34 ++++++++++++++++++++++++++++++++++ - 1 file changed, 34 insertions(+) - -diff --git a/deploy/reap.py b/deploy/reap.py -index be72918..fe6bebf 100644 ---- a/deploy/reap.py -+++ b/deploy/reap.py -@@ -143,6 +143,31 @@ class Reap(object): - interfaces[if_name] = interface - return if_name, mac - -+ def reap_properties(self, node_id, properties): -+ properties_name = None -+ node_file = glob.glob('%s/deployment_%s/%s.yaml' -+ % (self.temp_dir, self.env_id, node_id)) -+ with open(node_file[0]) as f: -+ node_config = yaml.load(f) -+ -+ interface = {'interfaces': node_config['network_scheme']['interfaces']} -+ interface_properties = {k:v for k, v in interface['interfaces'].items()} -+ -+ mtu_properties = {} -+ for k, v in interface_properties.items(): -+ if 'mtu' in v: -+ mtu_properties[k]={'mtu':v['mtu']} -+ else: -+ mtu_properties[k] = 'null' -+ interface_properties = mtu_properties -+ -+ properties_name = self.check_dict_exists(properties, interface) if properties else None -+ if not properties_name: -+ properties_name = 'interfaces_properties' -+ properties[properties_name] = interface_properties -+ return properties_name, interface_properties -+ -+ - def reap_transformation(self, node_id, roles, transformations): - main_role = 'controller' if 'controller' in roles else 'compute' - node_file = glob.glob('%s/deployment_%s/%s.yaml' -@@ -172,8 +197,10 @@ class Reap(object): - min_node = min(real_node_ids) - interfaces = {} - transformations = {} -+ properties = {} - dea_nodes = [] - dha_nodes = [] -+ properties_nodes = [] - - for real_node_id in real_node_ids: - node_id = real_node_id - min_node + 1 -@@ -185,14 +212,19 @@ class Reap(object): - dea_node = {'id': node_id, - 'role': roles} - dha_node = {'id': node_id} -+ properties_node = {'id': node_id} - if_name, mac = self.reap_interface(real_node_id, interfaces) - log('reap transformation for node %s' % real_node_id) -+ pr_name, interface_properties = self.reap_properties(real_node_id, properties) - tr_name = self.reap_transformation(real_node_id, roles, - transformations) - dea_node.update( - {'interfaces': if_name, -+ 'properties': pr_name, - 'transformations': tr_name}) - -+ properties_node.update({'interface_configuration': interface_properties}) -+ - dha_node.update( - {'pxeMac': mac if mac else None, - 'ipmiIp': None, -@@ -203,10 +235,12 @@ class Reap(object): - - dea_nodes.append(dea_node) - dha_nodes.append(dha_node) -+ properties_nodes.append(properties_node) - - self.write_yaml(self.dha_file, {'nodes': dha_nodes}, False) - self.write_yaml(self.dea_file, {'nodes': dea_nodes}) - self.write_yaml(self.dea_file, interfaces) -+ self.write_yaml(self.dea_file, {'interfaces_properties': properties_nodes}) - self.write_yaml(self.dea_file, transformations) - self.reap_fuel_node_info() - self.write_yaml(self.dha_file, {'disks': DISKS}) --- -1.9.1 diff --git a/patches/opnfv-fuel/cross-bootstrap/0001-UX-Update-bootstrap-target-build-time-estimate.patch b/patches/opnfv-fuel/cross-bootstrap/0001-UX-Update-bootstrap-target-build-time-estimate.patch deleted file mode 100644 index bc88e124..00000000 --- a/patches/opnfv-fuel/cross-bootstrap/0001-UX-Update-bootstrap-target-build-time-estimate.patch +++ /dev/null @@ -1,46 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Mon, 9 May 2016 17:05:53 +0200 -Subject: [PATCH] UX: Update bootstrap/target build time estimate. - -While building for a different architecture (e.g. AArch64 on x86_64), -the bootstrap/target image build may take longer, due to latency -introduced by using qemu-user-static. - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - .../0001-Patches-for-OPNFV.patch | 15 +++++++++++++++ - 1 file changed, 15 insertions(+) - -diff --git a/build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch b/build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch -index 446d0b6..c2e664c 100644 ---- a/build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch -+++ b/build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch -@@ -10,5 +10,20 @@ diff --git a/iso/bootstrap_admin_node.sh b/iso/bootstrap_admin_node.sh - --- a/iso/bootstrap_admin_node.sh - +++ b/iso/bootstrap_admin_node.sh -+@@ -86,9 +86,11 @@ -+ http://docs.openstack.org/developer/fuel-docs/userdocs/fuel-install-guide/bootstrap/\ -+ bootstrap_troubleshoot.html" -+ bs_progress_message="There is no active bootstrap. Bootstrap image building \ -+-is in progress. Usually it takes 15-20 minutes. It depends on your internet \ -+-connection and hardware performance. After bootstrap image becomes available, \ -+-reboot nodes that failed to be discovered." -++is in progress. Usually it takes 15-20 minutes for a native build (x86_64) \ -++and/or 30-45 minutes for each cross-build (e.g. AArch64). \ -++It depends on your internet connection, hardware performance and selected \ -++bootstrap architecture(s). This ISO supports AArch64 only. After bootstrap \ -++image becomes available, reboot nodes that failed to be discovered." -+ bs_done_message="Default bootstrap image building done. Now you can boot new \ -+ nodes over PXE, they will be discovered and become available for installing \ -+ OpenStack on them" - @@ -347,8 +347,23 @@ fuelmenu --save-only --iface=$ADMIN_INTERFACE || fail - set +x - echo "Done!" diff --git a/patches/opnfv-fuel/cross-bootstrap/0002-Build-bootstrap-image-for-arm64.patch b/patches/opnfv-fuel/cross-bootstrap/0002-Build-bootstrap-image-for-arm64.patch deleted file mode 100644 index ae7de06a..00000000 --- a/patches/opnfv-fuel/cross-bootstrap/0002-Build-bootstrap-image-for-arm64.patch +++ /dev/null @@ -1,35 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Date: Thu, 3 Mar 2016 19:04:07 +0100 -Subject: [PATCH] Build bootstrap image for arm64 - -For now the arch is hardcoded for simplicity. We should detect this -but how? If we're multi-arch then one bootstrap image should be built -per supported arch. This requires Fuel refactoring. ---- - .../0001-Patches-for-OPNFV.patch | 9 +++++++++ - 1 file changed, 9 insertions(+) - -diff --git a/build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch b/build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch ---- a/build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch -+++ b/build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch -@@ -45,3 +45,12 @@ - # Enable iptables - systemctl enable iptables.service -+@@ -484,7 +490,7 @@ -+ local ret=1 -+ echo ${bs_progress_message} >&2 -+ set_ui_bootstrap_error "${bs_progress_message}" >&2 -+- if fuel-bootstrap -v --debug build --activate >>"$bs_build_log" 2>&1; then -++ if fuel-bootstrap -v --debug build --target_arch arm64 --activate >>"$bs_build_log" 2>&1; then -+ ret=0 -+ fuel notify --topic "done" --send "${bs_done_message}" -+ else - @@ -546,6 +560,16 @@ systemctl start ntpd diff --git a/patches/opnfv-fuel/cross-bootstrap/0004-post-scripts-Enable-systemd-binfmt-for-first-boot.patch b/patches/opnfv-fuel/cross-bootstrap/0004-post-scripts-Enable-systemd-binfmt-for-first-boot.patch deleted file mode 100644 index 7158e3ab..00000000 --- a/patches/opnfv-fuel/cross-bootstrap/0004-post-scripts-Enable-systemd-binfmt-for-first-boot.patch +++ /dev/null @@ -1,101 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Thu, 9 Jun 2016 19:46:56 +0200 -Subject: [PATCH] post-scripts: Enable systemd-binfmt for first boot. - -IMPORTANT: -This commit introduces the following file on Fuel Master, -containing the list of arch supported for target nodes: -</etc/fuel_openstack_arch>. - -This file is created AFTER pre.d OPNFV scripts are ran, also by -bootstrap_admin_node.sh, BEFORE the post.d scripts are executed. -NOTE: Arch format is the same used by dpkg (e.g.: "amd64 arm64") - -Based on the number of foreign architectures we should support -(the list of archs in /etc/fuel_openstack_arch minus native arch), -we determine whether cross-builds are expected, in which case -binfmt support should be functional during first Fuel Master boot. - -Since systemd-binfmt service has a series of preconditions for -starting, which are tested early during first boot, some of them -are NOT YET true (e.g. no binfmt handlers are present before -qemu-user-static package is installed), so the service is not -automatically started. Of course, this only affects the first boot. - -Finally, this commit adds an OPNFV post-install script in -/opt/opnfv/bootstrap/post.d, which starts systemd-binfmt service. ---- - .../post-scripts/80_prepare_cross_builds.sh | 32 ++++++++++++++++++++++ - ...PNFV-Additions-to-bootstrap_admin_node.sh.patch | 13 +++++++++ - 2 files changed, 45 insertions(+) - create mode 100755 build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh - -diff --git a/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh b/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh -new file mode 100755 -index 0000000..01eb2e2 ---- /dev/null -+++ b/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh -@@ -0,0 +1,32 @@ -+#/bin/sh -+############################################################################## -+# Copyright (c) 2016 Enea AB and others. -+# Alexandru.Avadanii@enea.com -+# All rights reserved. This program and the accompanying materials -+# are made available under the terms of the Apache License, Version 2.0 -+# which accompanies this distribution, and is available at -+# http://www.apache.org/licenses/LICENSE-2.0 -+############################################################################## -+ -+echo "Preparing for cross-building bootstrap/target images" -+ -+# Fmt handlers are set up by qemu-user-static (after systemd-binfmt -+# prerequisites are checked during first boot of Fuel Master node). -+ -+# Cross-building support provides dpkg, as a fuel-agent dependency -+if which dpkg > /dev/null 2>&1; then -+ # /etc/fuel_openstack_arch is created by bootstrap_admin_node.sh -+ FOREIGN_ARCH_CNT=$(sed -e "s/$(dpkg --print-architecture)//" \ -+ /etc/fuel_openstack_arch | wc -w) -+ if [ ${FOREIGN_ARCH_CNT} -gt 0 ]; then -+ # If no fmt handlers are configured, the service is not started, -+ # so request it explicitly (only necessary for the first boot). -+ systemctl start systemd-binfmt -+ if [ $? -ne 0 ]; then -+ echo "Error starting systemd-binfmt!" -+ exit 1 -+ fi -+ fi -+fi -+ -+echo "Done preparing cross-building" -diff --git a/build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch b/build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch -index 12d0dfb..81d196b 100644 ---- a/build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch -+++ b/build/f_repos/patch/fuel-main/0001-Patches-for-OPNFV.patch -@@ -25,6 +25,18 @@ index 3197c91..db3123d 100755 - bs_done_message="Default bootstrap image building done. Now you can boot new \ - nodes over PXE, they will be discovered and become available for installing \ - OpenStack on them" -+@@ -239,5 +239,11 @@ -+ # /etc/fuel_openstack_version is provided by 'fuel-openstack-metadata' package -+ OPENSTACK_VERSION=$(cat /etc/fuel_openstack_version) -+ -++# FIXME(armband): This part might be moved to an earlier stage later -++# /etc/fuel_openstack_arch is constructed based on local mirror metadata -++grep -oP "^Architectures: \K.*$" \ -++ ${wwwdir}/${OPENSTACK_VERSION}/ubuntu/x86_64/dists/mos${FUEL_RELEASE}/Release > \ -++ /etc/fuel_openstack_arch -++ -+ -+ touch /var/lib/hiera/common.yaml /etc/puppet/hiera.yaml - @@ -347,8 +347,23 @@ fuelmenu --save-only --iface=$ADMIN_INTERFACE || fail - set +x - echo "Done!" diff --git a/patches/opnfv-fuel/cross-bootstrap/0005-mcagent-Increase-max-shell-command-timeout-to-2h.patch b/patches/opnfv-fuel/cross-bootstrap/0005-mcagent-Increase-max-shell-command-timeout-to-2h.patch deleted file mode 100644 index 74713638..00000000 --- a/patches/opnfv-fuel/cross-bootstrap/0005-mcagent-Increase-max-shell-command-timeout-to-2h.patch +++ /dev/null @@ -1,57 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Thu, 9 Jun 2016 21:37:48 +0200 -Subject: [PATCH] mcagent: Increase max shell command timeout to 2h - -Since `execute_shell_command` mcagent is used for building the target -image and the timeout provided in the astute task is just respected on -the Astute level (on mcollective level this timeout is actually a -minumum between value provided and ddl value) we need to increase it -for Armband operation. - -Because Armband builds the target image via `qemu-debootstrap`, it can -take more than 1 hour to finish the building. Therefore aside of -increasing the timeout in the Astute task, the maximum timeout value -for mcagent has to be increased. - -[ Alexandru Avadanii ] -Repackaged, based on Stan's `fuel-astute` patch. - -Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - .../f_bootstrap/post-scripts/80_prepare_cross_builds.sh | 14 ++++++++++++++ - 1 file changed, 14 insertions(+) - -diff --git a/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh b/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh -index 01eb2e2..12f66e6 100755 ---- a/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh -+++ b/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh -@@ -26,6 +26,20 @@ if which dpkg > /dev/null 2>&1; then - echo "Error starting systemd-binfmt!" - exit 1 - fi -+ -+ # Cross-build timeout adjustments -+ # -+ # Since `execute_shell_command` mcagent is used for building the target -+ # image and the timeout provided in the astute task is just respected on -+ # the Astute level (on mcollective level this timeout is actually a -+ # minumum between value provided and ddl value) we need to increase it -+ # for cross-build operations. -+ -+ # Building the target image via `qemu-debootstrap` can take more than -+ # 1 hour. Therefore aside of increasing the timeout in the Astute task, -+ # the maximum timeout value for mcagent has to be increased (use 2h). -+ sed -i.bak -r 's/^(\s+:timeout\s*=>)\s*[[:digit:]]+$/\1 7200/' \ -+ /usr/libexec/mcollective/mcollective/agent/execute_shell_command.ddl - fi - fi - diff --git a/patches/opnfv-fuel/cross-bootstrap/0006-bootstrap-Use-gzip-instead-of-xz-compression.patch b/patches/opnfv-fuel/cross-bootstrap/0006-bootstrap-Use-gzip-instead-of-xz-compression.patch deleted file mode 100644 index a906a7a2..00000000 --- a/patches/opnfv-fuel/cross-bootstrap/0006-bootstrap-Use-gzip-instead-of-xz-compression.patch +++ /dev/null @@ -1,60 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Fri, 10 Jun 2016 22:30:29 +0200 -Subject: [PATCH] bootstrap: Use gzip instead of xz compression. - -bootstrap mksquashfs using qemu-user-static is extremely slow, -taking up to one hour. gzip, on the other hand, is reasonably fast. -According to [1], xz is slower, with not much size gain. - -[1] https://jonathancarter.org/2015/04/06/squashfs-performance-testing/ - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - .../post-scripts/80_prepare_cross_builds.sh | 19 +++++++++++++++++++ - 1 file changed, 19 insertions(+) - -diff --git a/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh b/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh -index 12f66e6..3cb22f3 100755 ---- a/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh -+++ b/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh -@@ -27,6 +27,11 @@ if which dpkg > /dev/null 2>&1; then - exit 1 - fi - -+ # Determine python site-packages directory location -+ PYTHON_SITEPKGS_DIR=$(python -c \ -+ "from distutils.sysconfig import get_python_lib; print(get_python_lib())") -+ [ -d ${PYTHON_SITEPKGS_DIR} ] || exit 1 -+ - # Cross-build timeout adjustments - # - # Since `execute_shell_command` mcagent is used for building the target -@@ -40,6 +45,20 @@ if which dpkg > /dev/null 2>&1; then - # the maximum timeout value for mcagent has to be increased (use 2h). - sed -i.bak -r 's/^(\s+:timeout\s*=>)\s*[[:digit:]]+$/\1 7200/' \ - /usr/libexec/mcollective/mcollective/agent/execute_shell_command.ddl -+ -+ # Bootstrap: Use gzip instead of xz compression. -+ # -+ # bootstrap mksquashfs via `qemu-user-static` is extremely slow, -+ # taking up to one hour. gzip, on the other hand, is reasonably fast. -+ -+ # See the following article for a comparison between gzip and xz -+ # https://jonathancarter.org/2015/04/06/squashfs-performance-testing/ -+ # xz is slower, with very little size gain. -+ if [ -f ${PYTHON_SITEPKGS_DIR}/fuel_bootstrap/consts.py ]; then -+ sed -i.bak -r "s/^(\s+'compress_format'\s*:\s*').*?('.*)$/\1gzip\2/g" \ -+ ${PYTHON_SITEPKGS_DIR}/fuel_bootstrap/consts.py -+ echo "INFO: [xz] => [gzip] updated bootstrap initrd / rootfs compression." -+ fi - fi - fi - diff --git a/patches/opnfv-fuel/cross-bootstrap/0007-Nailgun-Increase-target-image-build-timeout.patch b/patches/opnfv-fuel/cross-bootstrap/0007-Nailgun-Increase-target-image-build-timeout.patch deleted file mode 100644 index c762fabd..00000000 --- a/patches/opnfv-fuel/cross-bootstrap/0007-Nailgun-Increase-target-image-build-timeout.patch +++ /dev/null @@ -1,44 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Sat, 11 Jun 2016 21:37:29 +0200 -Subject: [PATCH] Nailgun: Increase target image build timeout - -Currently we're close to 40 minutes of image building (thanks to -qemu-debootstrap). If network connectivity is a bit slow (happened to -me) it's easy to go over 1h limit. Therefore let's push this to 2 hours -to be safer. - -[ Alexandru Avadanii ] -Moved original patch wrote by Stan to post-install. - -Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - .../f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh | 7 +++++++ - 1 file changed, 7 insertions(+) - -diff --git a/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh b/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh -index 3cb22f3..c106bc3 100755 ---- a/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh -+++ b/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh -@@ -45,6 +45,13 @@ if which dpkg > /dev/null 2>&1; then - # the maximum timeout value for mcagent has to be increased (use 2h). - sed -i.bak -r 's/^(\s+:timeout\s*=>)\s*[[:digit:]]+$/\1 7200/' \ - /usr/libexec/mcollective/mcollective/agent/execute_shell_command.ddl -+ # Update nailgun timeout setting (2h) -+ if [ -f ${PYTHON_SITEPKGS_DIR}/nailgun/settings.yaml ]; then -+ sed -i.bak -r \ -+ 's/^(PROVISIONING_IMAGES_BUILD_TIMEOUT:)\s*[[:digit:]]+$/\1 7200/' \ -+ ${PYTHON_SITEPKGS_DIR}/nailgun/settings.yaml -+ echo "INFO: [1h] => [2h] updated nailgun image build timeout." -+ fi - - # Bootstrap: Use gzip instead of xz compression. - # diff --git a/patches/opnfv-fuel/cross-bootstrap/0008-m1.micro-Increase-profile-RAM-size-to-256MB.patch b/patches/opnfv-fuel/cross-bootstrap/0008-m1.micro-Increase-profile-RAM-size-to-256MB.patch deleted file mode 100644 index de115457..00000000 --- a/patches/opnfv-fuel/cross-bootstrap/0008-m1.micro-Increase-profile-RAM-size-to-256MB.patch +++ /dev/null @@ -1,46 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Sat, 11 Jun 2016 23:34:48 +0200 -Subject: [PATCH] m1.micro: Increase profile RAM size to 256MB. - -TestVM (cirros) on aarch64 requires more than 64MB RAM. -Keep profiles uniform across all archs. - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - .../f_bootstrap/post-scripts/80_prepare_cross_builds.sh | 15 +++++++++++++++ - 1 file changed, 15 insertions(+) - -diff --git a/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh b/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh -index c106bc3..0b48ca2 100755 ---- a/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh -+++ b/build/f_isoroot/f_bootstrap/post-scripts/80_prepare_cross_builds.sh -@@ -66,6 +66,21 @@ if which dpkg > /dev/null 2>&1; then - ${PYTHON_SITEPKGS_DIR}/fuel_bootstrap/consts.py - echo "INFO: [xz] => [gzip] updated bootstrap initrd / rootfs compression." - fi -+ -+ # TestVM (cirros) adjustments for cross-arch -+ # -+ # TestVM (cirros) on aarch64 requires more than 64MB RAM. -+ # Keep profiles uniform across all archs by bumping micro flavor specs. -+ # m1.micro: Increase profile RAM size to 256MB. -+ OPENSTACK_VERSION=$(cat /etc/fuel_openstack_version) -+ CONTROLLER_PP="/etc/puppet/${OPENSTACK_VERSION}/modules/openstack_tasks" -+ CONTROLLER_PP+="/manifests/roles/controller.pp" -+ if [ -f ${CONTROLLER_PP} ]; then -+ sed -i.bak -r \ -+ 's/^(.*ram.*=>)\s+[[:digit:]]+(.*)$/\1 256\2/' \ -+ ${CONTROLLER_PP} -+ echo "INFO: [64] => [256] updated m1.micro profile RAM size (MB)." -+ fi - fi - fi - diff --git a/patches/opnfv-fuel/kernel-bump/0001-dea_base-Pin-kernel-to-4.8.0-9944.patch b/patches/opnfv-fuel/kernel-bump/0001-dea_base-Pin-kernel-to-4.8.0-9944.patch deleted file mode 100644 index 7c38caee..00000000 --- a/patches/opnfv-fuel/kernel-bump/0001-dea_base-Pin-kernel-to-4.8.0-9944.patch +++ /dev/null @@ -1,56 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Tue, 14 Mar 2017 19:56:50 +0100 -Subject: [PATCH] dea_base: Pin kernel to 4.8.0-9944 - -Since newer kernels have issues with AAVMF + vhost_net, -we need to also temporarily blacklist vhost_net. - -TODO: Remove vhost_net blacklist (later). - -JIRA: https://jira.opnfv.org/browse/ARMBAND-161 -JIRA: https://jira.opnfv.org/browse/ARMBAND-171 -JIRA: https://jira.opnfv.org/browse/ARMBAND-227 -JIRA: https://jira.opnfv.org/browse/ARMBAND-230 -JIRA: https://jira.opnfv.org/browse/ARMBAND-239 -JIRA: https://jira.opnfv.org/browse/ARMBAND-244 - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - deploy/config/dea_base.yaml | 8 ++++++-- - 1 files changed, 6 insertions(+), 2 deletions(-) - -diff --git a/deploy/config/dea_base.yaml b/deploy/config/dea_base.yaml -index 386686f..2a5928e 100644 ---- a/deploy/config/dea_base.yaml -+++ b/deploy/config/dea_base.yaml -@@ -534,6 +534,8 @@ settings: - - anacron - -+ armband-vhost-net-blacklist -+ - bash-completion - - bridge-utils -@@ -560,9 +562,11 @@ settings: - - linux-firmware - -- linux-headers-generic-lts-xenial -+ linux-image-4.8.0-9944-generic -+ -+ linux-image-extra-4.8.0-9944-generic - -- linux-image-generic-lts-xenial -+ linux-headers-4.8.0-9944-generic - - lvm2 - diff --git a/patches/opnfv-fuel/multiarch-mirrors/0001-build-install-Add-arm64-deb-repositories-setup.patch b/patches/opnfv-fuel/multiarch-mirrors/0001-build-install-Add-arm64-deb-repositories-setup.patch deleted file mode 100644 index e6182278..00000000 --- a/patches/opnfv-fuel/multiarch-mirrors/0001-build-install-Add-arm64-deb-repositories-setup.patch +++ /dev/null @@ -1,190 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB, Cavium and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Date: Wed, 24 Feb 2016 20:04:03 +0100 -Subject: [PATCH] build/install: Add arm64 deb repositories setup - -If local MOS mirror patching is enabled, we need to regenerate the -repo metadata using apt-ftparchive. - -[ Alexandru Avadanii ] -Fixed non-existing paths for deb files, slightly refactoring our -approach. - -FIXME: Use UBUNTU_ARCH env var instead of hardcoding the arch list. -While fixing that, also take care of missed s/trusty/mos9.0/g -and/or s/xenial/mos10.0/g in {un,}install.sh. - -Closes-bug: ARMBAND-163 - -Signed-off-by: Stanislaw Kardach <stanislaw.kardach@cavium.com> -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - build/f_isoroot/f_kscfg/ks.cfg.patch | 10 ++++++++++ - build/install/apt-ftparchive-deb.conf | 9 ++++----- - build/install/apt-ftparchive-release.conf | 2 +- - build/install/apt-ftparchive-udeb.conf | 10 ++++------ - build/install/install.sh | 22 ++++++++++++++++++---- - build/install/uninstall.sh | 18 ++++++++++-------- - 6 files changed, 47 insertions(+), 24 deletions(-) - -diff --git a/build/f_isoroot/f_kscfg/ks.cfg.patch b/build/f_isoroot/f_kscfg/ks.cfg.patch -index a6840e4..84a241f 100644 ---- a/build/f_isoroot/f_kscfg/ks.cfg.patch -+++ b/build/f_isoroot/f_kscfg/ks.cfg.patch -@@ -1,5 +1,15 @@ - *** /dev/null 2016-04-26 10:10:11.481587709 +0200 - --- ks.cfg 2016-04-26 10:10:11.481587709 +0200 - *************** -+*** 448,453 **** -+--- 448,454 ---- -+ cp ${SOURCE}/.treeinfo ${repodir}/centos/x86_64 -+ -+ # Copying Ubuntu files -++ # FIXME: This is missleading as dist/pool may contain multiple arch! -+ mkdir -p ${repodir}/ubuntu/x86_64/images -+ cp -r ${SOURCE}/ubuntu/dists ${repodir}/ubuntu/x86_64 -+ cp -r ${SOURCE}/ubuntu/pool ${repodir}/ubuntu/x86_64 -+*************** - *** 579,584 **** - --- 579,592 ---- -diff --git a/build/install/apt-ftparchive-deb.conf b/build/install/apt-ftparchive-deb.conf -index edcb1f2..81ead2b 100644 ---- a/build/install/apt-ftparchive-deb.conf -+++ b/build/install/apt-ftparchive-deb.conf -@@ -10,14 +10,13 @@ - - Dir { - ArchiveDir "."; -+ CacheDir "/tmp"; - }; - --TreeDefault { -+Tree "dists/mos10.0" { -+ Architectures "amd64 arm64"; - Directory "pool"; --}; -- --BinDirectory "pool/main" { -- Packages "dists/mos10.0/main/binary-amd64/Packages"; -+ Sections "main"; - }; - - Default { -diff --git a/build/install/apt-ftparchive-release.conf b/build/install/apt-ftparchive-release.conf -index e40ff68..ceaf71a 100644 ---- a/build/install/apt-ftparchive-release.conf -+++ b/build/install/apt-ftparchive-release.conf -@@ -13,6 +13,7 @@ APT::FTPArchive::Release::Label "mos10.0"; - APT::FTPArchive::Release::Suite "mos10.0"; - APT::FTPArchive::Release::Version "16.04"; - APT::FTPArchive::Release::Codename "mos10.0"; --APT::FTPArchive::Release::Architectures "amd64"; -+APT::FTPArchive::Release::Architectures "amd64 arm64"; - APT::FTPArchive::Release::Components "main"; - APT::FTPArchive::Release::Description "Ubuntu 16.04.1 LTS"; -+APT::FTPArchive::Contents "no"; -diff --git a/build/install/apt-ftparchive-udeb.conf b/build/install/apt-ftparchive-udeb.conf -index 3b5b239..8852db7 100644 ---- a/build/install/apt-ftparchive-udeb.conf -+++ b/build/install/apt-ftparchive-udeb.conf -@@ -10,15 +10,13 @@ - - Dir { - ArchiveDir "."; -+ CacheDir "/tmp"; - }; - --TreeDefault { -+Tree "dists/mos10.0" { -+ Architectures "amd64 arm64"; - Directory "pool"; --}; -- --BinDirectory "pool/debian-installer" { -- Packages "dists/xenial/main/debian-installer/binary-amd64/Packages"; -- BinOverride "./indices/override.xenial.main.debian-installer"; -+ Sections "main"; - }; - - Default { -diff --git a/build/install/install.sh b/build/install/install.sh -index d1a49f3..0561207 100755 ---- a/build/install/install.sh -+++ b/build/install/install.sh -@@ -122,12 +122,24 @@ prep_make_live() { - ssh-copy-id root@$FUELHOST - sshfs root@1${FUELHOST}:/ $TMP_HOSTMOUNT - -- if [ -f $REPO/dists/xenial/main/binary-amd64/Packages.backup ]; then -+ for arch in arm64 amd64; do -+ if [ -f $REPO/dists/xenial/main/binary-${arch}/Packages.backup ]; then -+ echo "Error - found backup file for Packages for ${arch}!" -+ exit 1 -+ fi -+ -+ if [ -f $REPO/dists/xenial/main/binary-${arch}/Packages.gz.backup ]; then -+ echo "Error - found backup file for Packages.gz for ${arch}!" -+ exit 1 -+ fi -+ done -+ -+ if [ -f $REPO/dists/xenial/main/binary-arm64/Packages.backup ]; then - echo "Error - found backup file for Packages!" - exit 1 - fi - -- if [ -f $REPO/dists/xenial/main/binary-amd64/Packages.gz.backup ]; then -+ if [ -f $REPO/dists/xenial/main/binary-arm64/Packages.gz.backup ]; then - echo "Error - found backup file for Packages.gz!" - exit 1 - fi -@@ -142,8 +154,10 @@ prep_make_live() { - exit 1 - fi - -- cp $REPO/dists/xenial/main/binary-amd64/Packages $REPO/dists/xenial/main/binary-amd64/Packages.backup -- cp $REPO/dists/xenial/main/binary-amd64/Packages.gz $REPO/dists/xenial/main/binary-amd64/Packages.gz.backup -+ for arch in arm64 amd64; do -+ cp $REPO/dists/xenial/main/binary-${arch}/Packages $REPO/dists/xenial/main/binary-${arch}/Packages.backup -+ cp $REPO/dists/xenial/main/binary-${arch}/Packages.gz $REPO/dists/xenial/main/binary-${arch}/Packages.gz.backup -+ done - cp $REPO/dists/xenial/Release $REPO/dists/xenial/Release.backup - cp -Rvp $DEST/etc/puppet $DEST/etc/puppet.backup - } -diff --git a/build/install/uninstall.sh b/build/install/uninstall.sh -index a9e74bc..e90c632 100755 ---- a/build/install/uninstall.sh -+++ b/build/install/uninstall.sh -@@ -31,15 +31,17 @@ DEST=$MOUNT - REPO=$DEST/var/www/nailgun/ubuntu/fuelweb/x86_64 - - cd $REPO --if [ ! -f $REPO/dists/xenial/main/binary-amd64/Packages.backup ]; then -- echo "Error - didn't find backup file for Packages!" -- exit 1 --fi -+for arch in arm64 amd64; do -+ if [ ! -f $REPO/dists/xenial/main/binary-${arch}/Packages.backup ]; then -+ echo "Error - didn't find backup file for Packages for ${arch}!" -+ exit 1 -+ fi - --if [ ! -f $REPO/dists/xenial/main/binary-amd64/Packages.gz.backup ]; then -- echo "Error - didn't find backup file for Packages.gz!" -- exit 1 --fi -+ if [ ! -f $REPO/dists/xenial/main/binary-${arch}/Packages.gz.backup ]; then -+ echo "Error - didn't find backup file for Packages.gz for ${arch}!" -+ exit 1 -+ fi -+done - - if [ ! -f $REPO/dists/xenial/Release.backup ]; then - echo "Error - didn't find backup file for Release!" diff --git a/patches/opnfv-fuel/upstream-backports/0001-deploy.sh-Remove-check-for-root-rm-umask-0000.patch b/patches/opnfv-fuel/upstream-backports/0001-deploy.sh-Remove-check-for-root-rm-umask-0000.patch deleted file mode 100644 index dcb47a7d..00000000 --- a/patches/opnfv-fuel/upstream-backports/0001-deploy.sh-Remove-check-for-root-rm-umask-0000.patch +++ /dev/null @@ -1,102 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Fri, 6 May 2016 03:07:40 +0200 -Subject: [PATCH] deploy.sh: Remove check for root (rm umask 0000) - -Patch(es) by Josep, signed by Alex only for upstreaming. - -[Alexandru Avadanii] -Squashed two atomic commits into one change. - -Signed-off-by: Josep Puigdemont <josep.puigdemont@enea.com> -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - ci/deploy.sh | 11 ----------- - deploy/deploy-config.py | 1 - - deploy/deploy.py | 2 -- - deploy/environments/virtual_fuel.py | 2 -- - 4 files changed, 16 deletions(-) - -diff --git a/ci/deploy.sh b/ci/deploy.sh -index 8411714..081806c 100755 ---- a/ci/deploy.sh -+++ b/ci/deploy.sh -@@ -84,9 +84,6 @@ Input parameters to the build script is: - -i .iso image to be deployed (needs to be provided in a URI - style, it can be a local resource: file:// or a remote resource http(s)://) - --NOTE: Root priviledges are needed for this script to run -- -- - Examples: - sudo `basename $0` -b file:///home/jenkins/lab-config -l lf -p pod1 -s ha_odl-l3_heat_ceilometer -i file:///home/jenkins/myiso.iso - EOF -@@ -212,11 +209,6 @@ do - esac - done - --if [[ $EUID -ne 0 ]]; then -- echo "This script must be run as root" 1>&2 -- exit 1 --fi -- - if [ -z $BASE_CONFIG_URI ] || [ -z $TARGET_LAB ] || \ - [ -z $TARGET_POD ] || [ -z $DEPLOY_SCENARIO ] || \ - [ -z $ISO ]; then -@@ -231,9 +223,6 @@ fi - # Enable the automatic exit trap - trap do_exit SIGINT SIGTERM EXIT - --# Set no restrictive umask so that Jenkins can removeeee any residuals --umask 0000 -- - clean - - pushd ${DEPLOY_DIR} > /dev/null -diff --git a/deploy/deploy.py b/deploy/deploy.py -index fe213e5..08702d2 100755 ---- a/deploy/deploy.py -+++ b/deploy/deploy.py -@@ -34,7 +34,6 @@ from common import ( - check_dir_exists, - create_dir_if_not_exists, - delete, -- check_if_root, - ArgParser, - ) - -@@ -252,7 +251,6 @@ class AutoDeploy(object): - return 0 - - def run(self): -- check_if_root() - if self.cleanup_only: - self.cleanup_execution_environment() - else: -diff --git a/deploy/environments/virtual_fuel.py b/deploy/environments/virtual_fuel.py -index b1a76e4..4ff68f6 100644 ---- a/deploy/environments/virtual_fuel.py -+++ b/deploy/environments/virtual_fuel.py -@@ -18,7 +18,6 @@ import time - from common import ( - exec_cmd, - check_file_exists, -- check_if_root, - delete, - log, - ) -@@ -135,7 +134,6 @@ class VirtualFuel(ExecutionEnvironment): - vm_definition_overwrite) - - def setup_environment(self): -- check_if_root() - self.cleanup_environment() - self.create_vm() - diff --git a/patches/opnfv-fuel/upstream-backports/0004-Fuel-rework-for-multiple-libvirt.patch b/patches/opnfv-fuel/upstream-backports/0004-Fuel-rework-for-multiple-libvirt.patch deleted file mode 100644 index e9c8e2ce..00000000 --- a/patches/opnfv-fuel/upstream-backports/0004-Fuel-rework-for-multiple-libvirt.patch +++ /dev/null @@ -1,1533 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: cristinapauna <cristina.pauna@enea.com> -Date: Fri, 25 Nov 2016 16:38:46 +0200 -Subject: [PATCH] Fuel rework for multiple libvirt - -The current fuel arhitecture assumes that all the VMs -(Jenkins slave, Fuel master and Target nodes) are all on -the same server. There is code that allows the Fuel master -to be on a different machine, but nothing for the Target nodes. -In Enea-Arm lab we have a different server for each. - -This commit also adds specific templates for a virtual pod in -the Enea-Arm lab. - -Therefore, the following changes are done: -- The default libvirt URI is the machine on which the Target nodes are -- The parameter libvirtUri was added in the dha.yaml file for fuel master -- All virsh calls have "-c" option to explicitly specify the uri; this parameter - is set only when libvirtUri is explicitly set in the yaml file -- The scenarios dha override now point to the arm template files -- The network templates are not needed for ARM, as the network infra is provided - by the server on which the Target node VMs reside; the folder is added here - just for consistency -- The target nodes are created with virsh rather than quemu when the nodes are - not on the same machine as Jenkins slave (similar logic to what is done for - remote Fuel master) - -Initially only ha scenarios were successfully deployed on virtual pods -The default configuration is for ha scenarios and those have 5 target nodes -while on the noha scenarios there are 4. The noha scenarios failed because -it was trying to parse one extra node. - -The dea and dha files need to be modified for each noha scenario so that the -nodes are properly configured. - -Signed-off-by: cristinapauna <cristina.pauna@enea.com> ---- - deploy/dha_adapters/hardware_adapter.py | 5 ++ - deploy/dha_adapters/libvirt_adapter.py | 42 ++++----- - deploy/environments/execution_environment.py | 19 +++-- - deploy/environments/libvirt_environment.py | 59 ++++++++++++- - deploy/environments/virtual_fuel.py | 8 +- - deploy/install_fuel_master.py | 2 +- - deploy/scenario/ha_heat_ceilometer_scenario.yaml | 13 +-- - .../ha_nfv-kvm_heat_ceilometer_scenario.yaml | 13 +-- - .../ha_nfv-ovs_heat_ceilometer_scenario.yaml | 13 +-- - deploy/scenario/ha_odl-bgpvpn_scenario.yaml | 13 +-- - .../ha_odl-l2_heat_ceilometer_scenario.yaml | 13 +-- - .../ha_odl-l2_sfc_heat_ceilometer_scenario.yaml | 13 +-- - .../ha_odl-l3_heat_ceilometer_scenario.yaml | 13 +-- - ...ha_onos_nofeature_heat_ceilometer_scenario.yaml | 13 +-- - .../ha_onos_sfc_heat_ceilometer_scenario.yaml | 13 +-- - .../scenario/ha_vlan_heat_ceilometer_scenario.yaml | 13 +-- - .../scenario/no-ha_heat_ceilometer_scenario.yaml | 25 ++++++ - .../no-ha_nfv-kvm_heat_ceilometer_scenario.yaml | 25 ++++++ - ...a_nfv-kvm_nfv-ovs_heat_ceilometer_scenario.yaml | 25 ++++++ - .../no-ha_nfv-ovs_heat_ceilometer_scenario.yaml | 26 +++++- - deploy/scenario/no-ha_odl-bgpvpn_scenario.yaml | 25 ++++++ - .../no-ha_odl-l2_heat_ceilometer_scenario.yaml | 25 ++++++ - .../no-ha_odl-l2_sfc_heat_ceilometer_scenario.yaml | 25 ++++++ - .../no-ha_odl-l3_heat_ceilometer_scenario.yaml | 25 ++++++ - ...ha_onos_nofeature_heat_ceilometer_scenario.yaml | 26 ++++++ - .../no-ha_onos_sfc_heat_ceilometer_scenario.yaml | 25 ++++++ - .../no-ha_vlan_heat_ceilometer_scenario.yaml | 25 ++++++ - .../arm/virtual_environment/networks/.gitkeep | 0 - .../arm/virtual_environment/vms/compute.xml | 57 +++++++++++++ - .../arm/virtual_environment/vms/controller.xml | 57 +++++++++++++ - .../templates/arm/virtual_environment/vms/fuel.xml | 99 ++++++++++++++++++++++ - 31 files changed, 658 insertions(+), 97 deletions(-) - create mode 100644 deploy/templates/arm/virtual_environment/networks/.gitkeep - create mode 100644 deploy/templates/arm/virtual_environment/vms/compute.xml - create mode 100644 deploy/templates/arm/virtual_environment/vms/controller.xml - create mode 100644 deploy/templates/arm/virtual_environment/vms/fuel.xml - -diff --git a/deploy/dha_adapters/hardware_adapter.py b/deploy/dha_adapters/hardware_adapter.py -index aa59581..cf3d77a 100644 ---- a/deploy/dha_adapters/hardware_adapter.py -+++ b/deploy/dha_adapters/hardware_adapter.py -@@ -63,3 +63,8 @@ class HardwareAdapter(object): - vm_definition = self.dha_struct.get('define_vms') - if vm_definition: - return vm_definition.get(role) -+ def get_node_uri(self, node_id): -+ vm_libvirt_uri = self.get_node_property(node_id, 'libvirtUri') -+ if vm_libvirt_uri: -+ return '-c %s' % vm_libvirt_uri -+ return '' -diff --git a/deploy/dha_adapters/libvirt_adapter.py b/deploy/dha_adapters/libvirt_adapter.py -index a00f091..34535d1 100644 ---- a/deploy/dha_adapters/libvirt_adapter.py -+++ b/deploy/dha_adapters/libvirt_adapter.py -@@ -43,21 +43,21 @@ class LibvirtAdapter(HardwareAdapter): - def node_power_off(self, node_id): - vm_name = self.get_node_property(node_id, 'libvirtName') - log('Power OFF Node %s' % vm_name) -- state = exec_cmd('virsh domstate %s' % vm_name) -+ state = exec_cmd('virsh %s domstate %s' % (self.get_node_uri(node_id), vm_name)) - if state == 'running': -- exec_cmd('virsh destroy %s' % vm_name, False) -+ exec_cmd('virsh %s destroy %s' % (self.get_node_uri(node_id), vm_name), False) - - def node_power_on(self, node_id): - vm_name = self.get_node_property(node_id, 'libvirtName') - log('Power ON Node %s' % vm_name) -- state = exec_cmd('virsh domstate %s' % vm_name) -+ state = exec_cmd('virsh %s domstate %s' % (self.get_node_uri(node_id), vm_name)) - if state == 'shut off': -- exec_cmd('virsh start %s' % vm_name) -+ exec_cmd('virsh %s start %s' % (self.get_node_uri(node_id), vm_name)) - - def node_reset(self, node_id): - vm_name = self.get_node_property(node_id, 'libvirtName') - log('Reset Node %s' % vm_name) -- exec_cmd('virsh reset %s' % vm_name) -+ exec_cmd('virsh %s reset %s' % (self.get_node_uri(node_id), vm_name)) - - def translate(self, boot_order_list): - translated = [] -@@ -73,7 +73,7 @@ class LibvirtAdapter(HardwareAdapter): - vm_name = self.get_node_property(node_id, 'libvirtName') - temp_dir = tempfile.mkdtemp() - log('Set boot order %s on Node %s' % (boot_order_list, vm_name)) -- resp = exec_cmd('virsh dumpxml %s' % vm_name) -+ resp = exec_cmd('virsh %s dumpxml %s' % (self.get_node_uri(node_id), vm_name)) - xml_dump = etree.fromstring(resp, self.parser) - os = xml_dump.xpath('/domain/os') - for o in os: -@@ -92,12 +92,12 @@ class LibvirtAdapter(HardwareAdapter): - xml_file = temp_dir + '/%s.xml' % vm_name - with open(xml_file, 'w') as f: - tree.write(f, pretty_print=True, xml_declaration=True) -- exec_cmd('virsh define %s' % xml_file) -+ exec_cmd('virsh %s define %s' % (self.get_node_uri(node_id), xml_file)) - delete(temp_dir) - - def node_zero_mbr(self, node_id): - vm_name = self.get_node_property(node_id, 'libvirtName') -- resp = exec_cmd('virsh dumpxml %s' % vm_name) -+ resp = exec_cmd('virsh %s dumpxml %s' % (self.get_node_uri(node_id), vm_name)) - xml_dump = etree.fromstring(resp) - disks = xml_dump.xpath('/domain/devices/disk') - for disk in disks: -@@ -114,20 +114,20 @@ class LibvirtAdapter(HardwareAdapter): - - def node_eject_iso(self, node_id): - vm_name = self.get_node_property(node_id, 'libvirtName') -- device = self.get_name_of_device(vm_name, 'cdrom') -- exec_cmd('virsh change-media %s --eject %s --config --live' -- % (vm_name, device), False) -+ device = self.get_name_of_device(vm_name, 'cdrom', node_id) -+ exec_cmd('virsh %s change-media %s --eject %s --config --live' -+ % (self.get_node_uri(node_id), vm_name, device), False) - - def node_insert_iso(self, node_id, iso_file): - vm_name = self.get_node_property(node_id, 'libvirtName') -- device = self.get_name_of_device(vm_name, 'cdrom') -- exec_cmd('virsh change-media %s --insert %s %s' -- % (vm_name, device, iso_file)) -+ device = self.get_name_of_device(vm_name, 'cdrom', node_id) -+ exec_cmd('virsh %s change-media %s --insert %s %s' -+ % (self.get_node_uri(node_id), vm_name, device, iso_file)) - - def get_node_pxe_mac(self, node_id): - mac_list = [] - vm_name = self.get_node_property(node_id, 'libvirtName') -- resp = exec_cmd('virsh dumpxml %s' % vm_name) -+ resp = exec_cmd('virsh %s dumpxml %s' % (self.get_node_uri(node_id), vm_name)) - xml_dump = etree.fromstring(resp) - interfaces = xml_dump.xpath('/domain/devices/interface') - for interface in interfaces: -@@ -136,8 +136,8 @@ class LibvirtAdapter(HardwareAdapter): - mac_list.append(mac.get('address').lower()) - return mac_list - -- def get_name_of_device(self, vm_name, device_type): -- resp = exec_cmd('virsh dumpxml %s' % vm_name) -+ def get_name_of_device(self, vm_name, device_type, node_id): -+ resp = exec_cmd('virsh %s dumpxml %s' % (self.get_node_uri(node_id), vm_name)) - xml_dump = etree.fromstring(resp) - disks = xml_dump.xpath('/domain/devices/disk') - for disk in disks: -@@ -151,7 +151,7 @@ class LibvirtAdapter(HardwareAdapter): - def get_virt_net_conf_dir(self): - return self.dha_struct['virtNetConfDir'] - -- def upload_iso(self, iso_file): -+ def upload_iso(self, node_id, iso_file): - size = os.path.getsize(iso_file) - vol_name = os.path.basename(iso_file) - vol_xml = VOL_XML_TEMPLATE.format(name=vol_name, unit='bytes', -@@ -162,10 +162,10 @@ class LibvirtAdapter(HardwareAdapter): - - log(vol_xml) - pool = DEFAULT_POOL # FIXME -- exec_cmd('virsh vol-create --pool %s %s' % (pool, fname)) -- vol_path = exec_cmd('virsh vol-path --pool %s %s' % (pool, vol_name)) -+ exec_cmd('virsh %s vol-create --pool %s %s' % (self.get_node_uri(node_id), pool, fname)) -+ vol_path = exec_cmd('virsh %s vol-path --pool %s %s' % (self.get_node_uri(node_id), pool, vol_name)) - -- exec_cmd('virsh vol-upload %s %s' % (vol_path, iso_file), -+ exec_cmd('virsh %s vol-upload %s %s' % (self.get_node_uri(node_id), vol_path, iso_file), - attempts=5, delay=10, verbose=True) - - delete(fname) -diff --git a/deploy/environments/execution_environment.py b/deploy/environments/execution_environment.py -index 7a0b474..db25229 100644 ---- a/deploy/environments/execution_environment.py -+++ b/deploy/environments/execution_environment.py -@@ -29,12 +29,12 @@ class ExecutionEnvironment(object): - - def delete_vm(self, node_id): - vm_name = self.dha.get_node_property(node_id, 'libvirtName') -- r, c = exec_cmd('virsh dumpxml %s' % vm_name, False) -+ r, c = exec_cmd('virsh %s dumpxml %s' % (self.dha.get_node_uri(node_id), vm_name), False) - if c: - return -- self.undefine_vm_delete_disk(r, vm_name) -+ self.undefine_vm_delete_disk(r, vm_name, node_id) - -- def undefine_vm_delete_disk(self, printout, vm_name): -+ def undefine_vm_delete_disk(self, printout, vm_name, node_id): - disk_files = [] - xml_dump = etree.fromstring(printout, self.parser) - disks = xml_dump.xpath('/domain/devices/disk') -@@ -45,8 +45,13 @@ class ExecutionEnvironment(object): - if source_file: - disk_files.append(source_file) - log('Deleting VM %s with disks %s' % (vm_name, disk_files)) -- exec_cmd('virsh destroy %s' % vm_name, False) -- exec_cmd('virsh undefine --managed-save --remove-all-storage %s' % vm_name, False) -+ exec_cmd('virsh %s destroy %s' % (self.dha.get_node_uri(node_id), vm_name), False) -+ r, c = exec_cmd('virsh %s undefine --managed-save --remove-all-storage %s' % -+ (self.dha.get_node_uri(node_id), vm_name), False) -+ if c: -+ exec_cmd('virsh %s undefine --managed-save --remove-all-storage --nvram %s' % -+ (self.dha.get_node_uri(node_id), vm_name), False) -+ - for file in disk_files: - delete(file) - -@@ -75,7 +80,7 @@ class ExecutionEnvironment(object): - self.overwrite_xml(xml_element, value) - - def define_vm(self, vm_name, temp_vm_file, disk_path, -- vm_definition_overwrite): -+ vm_definition_overwrite, node_id): - log('Creating VM %s with disks %s' % (vm_name, disk_path)) - with open(temp_vm_file) as f: - vm_xml = etree.parse(f) -@@ -99,4 +104,4 @@ class ExecutionEnvironment(object): - disk.append(source) - with open(temp_vm_file, 'w') as f: - vm_xml.write(f, pretty_print=True, xml_declaration=True) -- exec_cmd('virsh define %s' % temp_vm_file) -+ exec_cmd('virsh %s define %s' % (self.dha.get_node_uri(node_id), temp_vm_file)) -diff --git a/deploy/environments/libvirt_environment.py b/deploy/environments/libvirt_environment.py -index 07a47fd..08d669b 100644 ---- a/deploy/environments/libvirt_environment.py -+++ b/deploy/environments/libvirt_environment.py -@@ -12,6 +12,9 @@ from lxml import etree - import glob - from execution_environment import ExecutionEnvironment - import tempfile -+import os -+import re -+import time - - from common import ( - exec_cmd, -@@ -21,6 +24,25 @@ from common import ( - delete, - ) - -+VOL_XML_TEMPLATE = '''<volume type='file'> -+ <name>{name}</name> -+ <capacity unit='{unit}'>{size!s}</capacity> -+ <target> -+ <format type='{format_type}'/> -+ </target> -+</volume>''' -+ -+DEFAULT_POOL = 'jenkins' -+ -+def get_size_and_unit(s): -+ p = re.compile('^(\d+)\s*(\D+)') -+ m = p.match(s) -+ if m == None: -+ return None, None -+ size = m.groups()[0] -+ unit = m.groups()[1] -+ return size, unit -+ - - class LibvirtEnvironment(ExecutionEnvironment): - -@@ -33,10 +55,39 @@ class LibvirtEnvironment(ExecutionEnvironment): - self.node_ids = self.dha.get_all_node_ids() - self.net_names = self.collect_net_names() - -- def create_storage(self, node_id, disk_path, disk_sizes): -+ def create_volume(self, pool, name, su, node_id, temp_dir, img_type='raw'): -+ log('Creating image using Libvirt volumes in pool %s, name: %s, uri: %s' % -+ (pool, name, self.dha.get_node_uri(node_id))) -+ size, unit = get_size_and_unit(su) -+ if size == None: -+ err('Could not determine size and unit of %s' % s) -+ -+ vol_xml = VOL_XML_TEMPLATE.format(name=name, unit=unit, size=size, -+ format_type=img_type) -+ fname = os.path.join(temp_dir, '%s_vol.xml' % name) -+ with file(fname, 'w') as f: -+ f.write(vol_xml) -+ -+ exec_cmd('virsh %s vol-create --pool %s %s' % -+ (self.dha.get_node_uri(node_id), pool, fname)) -+ vol_path = exec_cmd('virsh %s vol-path --pool %s %s' % -+ (self.dha.get_node_uri(node_id), pool, name)) -+ -+ delete(fname) -+ return vol_path -+ -+ -+ def create_storage(self, node_id, disk_path, disk_sizes, temp_dir): - role = self.dea.get_node_main_role(node_id, self.fuel_node_id) - disk_size = disk_sizes[role] -- exec_cmd('qemu-img create -f raw %s %s' % (disk_path, disk_size)) -+ if os.environ.get('LIBVIRT_DEFAULT_URI') == None: -+ exec_cmd('qemu-img create -f raw %s %s' % (disk_path, disk_size)) -+ else: -+ pool = DEFAULT_POOL # FIXME -+ name = os.path.basename(disk_path) -+ disk_path = self.create_volume(pool, name, disk_size, node_id, temp_dir) -+ return disk_path -+ - - def create_vms(self): - temp_dir = tempfile.mkdtemp() -@@ -48,13 +99,13 @@ class LibvirtEnvironment(ExecutionEnvironment): - node_id, 'libvirtTemplate')) - check_file_exists(vm_template) - disk_path = '%s/%s.raw' % (self.storage_dir, vm_name) -- self.create_storage(node_id, disk_path, disk_sizes) -+ disk_path = self.create_storage(node_id, disk_path, disk_sizes, temp_dir) - temp_vm_file = '%s/%s' % (temp_dir, vm_name) - exec_cmd('cp %s %s' % (vm_template, temp_vm_file)) - vm_definition_overwrite = self.dha.get_vm_definition( - self.dea.get_node_main_role(node_id, self.fuel_node_id)) - self.define_vm(vm_name, temp_vm_file, disk_path, -- vm_definition_overwrite) -+ vm_definition_overwrite, node_id) - delete(temp_dir) - - def start_vms(self): -diff --git a/deploy/environments/virtual_fuel.py b/deploy/environments/virtual_fuel.py -index fcfa532..5f50f40 100644 ---- a/deploy/environments/virtual_fuel.py -+++ b/deploy/environments/virtual_fuel.py -@@ -99,8 +99,10 @@ class VirtualFuel(ExecutionEnvironment): - with file(fname, 'w') as f: - f.write(vol_xml) - -- exec_cmd('virsh vol-create --pool %s %s' % (pool, fname)) -- vol_path = exec_cmd('virsh vol-path --pool %s %s' % (pool, name)) -+ exec_cmd('virsh %s vol-create --pool %s %s' % -+ (self.dha.get_node_uri(self.fuel_node_id), pool, fname)) -+ vol_path = exec_cmd('virsh %s vol-path --pool %s %s' % -+ (self.dha.get_node_uri(self.fuel_node_id), pool, name)) - - delete(fname) - -@@ -131,7 +133,7 @@ class VirtualFuel(ExecutionEnvironment): - vm_definition_overwrite = self.dha.get_vm_definition('fuel') - - self.define_vm(self.vm_name, self.temp_vm_file, disk_path, -- vm_definition_overwrite) -+ vm_definition_overwrite, self.fuel_node_id) - - def setup_environment(self): - self.cleanup_environment() -diff --git a/deploy/install_fuel_master.py b/deploy/install_fuel_master.py -index 2d89c8e..2615818 100644 ---- a/deploy/install_fuel_master.py -+++ b/deploy/install_fuel_master.py -@@ -59,7 +59,7 @@ class InstallFuelMaster(object): - - if os.environ.get('LIBVIRT_DEFAULT_URI'): - log('Upload ISO to pool') -- self.iso_file = self.dha.upload_iso(self.iso_file) -+ self.iso_file = self.dha.upload_iso(self.fuel_node_id, self.iso_file) - else: - log('Zero the MBR') - self.dha.node_zero_mbr(self.fuel_node_id) -diff --git a/deploy/scenario/ha_heat_ceilometer_scenario.yaml b/deploy/scenario/ha_heat_ceilometer_scenario.yaml -index 4051c47..78fa80d 100644 ---- a/deploy/scenario/ha_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/ha_heat_ceilometer_scenario.yaml -@@ -58,22 +58,23 @@ dha-override-config: - nodes: - - id: 1 - libvirtName: controller1 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 2 - libvirtName: controller2 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 3 - libvirtName: controller3 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 4 - libvirtName: compute1 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 5 - libvirtName: compute2 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 6 - libvirtName: fuel-master -- libvirtTemplate: templates/virtual_environment/vms/fuel.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system - isFuel: yes - username: root - password: r00tme -diff --git a/deploy/scenario/ha_nfv-kvm_heat_ceilometer_scenario.yaml b/deploy/scenario/ha_nfv-kvm_heat_ceilometer_scenario.yaml -index 1217f72..bcdda61 100644 ---- a/deploy/scenario/ha_nfv-kvm_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/ha_nfv-kvm_heat_ceilometer_scenario.yaml -@@ -145,22 +145,23 @@ dha-override-config: - nodes: - - id: 1 - libvirtName: controller1 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 2 - libvirtName: controller2 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 3 - libvirtName: controller3 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 4 - libvirtName: compute1 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 5 - libvirtName: compute2 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 6 - libvirtName: fuel-master -- libvirtTemplate: templates/virtual_environment/vms/fuel.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system - isFuel: yes - username: root - password: r00tme -diff --git a/deploy/scenario/ha_nfv-ovs_heat_ceilometer_scenario.yaml b/deploy/scenario/ha_nfv-ovs_heat_ceilometer_scenario.yaml -index 51deb4e..bee2d1a 100644 ---- a/deploy/scenario/ha_nfv-ovs_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/ha_nfv-ovs_heat_ceilometer_scenario.yaml -@@ -107,22 +107,23 @@ dha-override-config: - nodes: - - id: 1 - libvirtName: controller1 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 2 - libvirtName: controller2 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 3 - libvirtName: controller3 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 4 - libvirtName: compute1 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 5 - libvirtName: compute2 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 6 - libvirtName: fuel-master -- libvirtTemplate: templates/virtual_environment/vms/fuel.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system - isFuel: yes - username: root - password: r00tme -diff --git a/deploy/scenario/ha_odl-bgpvpn_scenario.yaml b/deploy/scenario/ha_odl-bgpvpn_scenario.yaml -index 7d52e77..46efc4b 100644 ---- a/deploy/scenario/ha_odl-bgpvpn_scenario.yaml -+++ b/deploy/scenario/ha_odl-bgpvpn_scenario.yaml -@@ -77,22 +77,23 @@ dha-override-config: - nodes: - - id: 1 - libvirtName: controller1 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 2 - libvirtName: controller2 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 3 - libvirtName: controller3 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 4 - libvirtName: compute1 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 5 - libvirtName: compute2 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 6 - libvirtName: fuel-master -- libvirtTemplate: templates/virtual_environment/vms/fuel.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system - isFuel: yes - username: root - password: r00tme -diff --git a/deploy/scenario/ha_odl-l2_heat_ceilometer_scenario.yaml b/deploy/scenario/ha_odl-l2_heat_ceilometer_scenario.yaml -index 90c89ae..50dde30 100644 ---- a/deploy/scenario/ha_odl-l2_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/ha_odl-l2_heat_ceilometer_scenario.yaml -@@ -77,22 +77,23 @@ dha-override-config: - nodes: - - id: 1 - libvirtName: controller1 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 2 - libvirtName: controller2 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 3 - libvirtName: controller3 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 4 - libvirtName: compute1 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 5 - libvirtName: compute2 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 6 - libvirtName: fuel-master -- libvirtTemplate: templates/virtual_environment/vms/fuel.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system - isFuel: yes - username: root - password: r00tme -diff --git a/deploy/scenario/ha_odl-l2_sfc_heat_ceilometer_scenario.yaml b/deploy/scenario/ha_odl-l2_sfc_heat_ceilometer_scenario.yaml -index c478948..e20ec1a 100644 ---- a/deploy/scenario/ha_odl-l2_sfc_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/ha_odl-l2_sfc_heat_ceilometer_scenario.yaml -@@ -95,22 +95,23 @@ dha-override-config: - nodes: - - id: 1 - libvirtName: controller1 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 2 - libvirtName: controller2 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 3 - libvirtName: controller3 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 4 - libvirtName: compute1 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 5 - libvirtName: compute2 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 6 - libvirtName: fuel-master -- libvirtTemplate: templates/virtual_environment/vms/fuel.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system - isFuel: yes - username: root - password: r00tme -diff --git a/deploy/scenario/ha_odl-l3_heat_ceilometer_scenario.yaml b/deploy/scenario/ha_odl-l3_heat_ceilometer_scenario.yaml -index 967e7d2..322dd27 100644 ---- a/deploy/scenario/ha_odl-l3_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/ha_odl-l3_heat_ceilometer_scenario.yaml -@@ -93,22 +93,23 @@ dha-override-config: - nodes: - - id: 1 - libvirtName: controller1 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 2 - libvirtName: controller2 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 3 - libvirtName: controller3 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 4 - libvirtName: compute1 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 5 - libvirtName: compute2 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 6 - libvirtName: fuel-master -- libvirtTemplate: templates/virtual_environment/vms/fuel.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system - isFuel: yes - username: root - password: r00tme -diff --git a/deploy/scenario/ha_onos_nofeature_heat_ceilometer_scenario.yaml b/deploy/scenario/ha_onos_nofeature_heat_ceilometer_scenario.yaml -index f813458..88b8f6b 100644 ---- a/deploy/scenario/ha_onos_nofeature_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/ha_onos_nofeature_heat_ceilometer_scenario.yaml -@@ -88,22 +88,23 @@ dha-override-config: - nodes: - - id: 1 - libvirtName: controller1 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 2 - libvirtName: controller2 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 3 - libvirtName: controller3 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 4 - libvirtName: compute1 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 5 - libvirtName: compute2 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 6 - libvirtName: fuel-master -- libvirtTemplate: templates/virtual_environment/vms/fuel.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system - isFuel: yes - username: root - password: r00tme -diff --git a/deploy/scenario/ha_onos_sfc_heat_ceilometer_scenario.yaml b/deploy/scenario/ha_onos_sfc_heat_ceilometer_scenario.yaml -index 32fbfcf..c390172 100644 ---- a/deploy/scenario/ha_onos_sfc_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/ha_onos_sfc_heat_ceilometer_scenario.yaml -@@ -68,22 +68,23 @@ dha-override-config: - nodes: - - id: 1 - libvirtName: controller1 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 2 - libvirtName: controller2 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 3 - libvirtName: controller3 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 4 - libvirtName: compute1 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 5 - libvirtName: compute2 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 6 - libvirtName: fuel-master -- libvirtTemplate: templates/virtual_environment/vms/fuel.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system - isFuel: yes - username: root - password: r00tme -diff --git a/deploy/scenario/ha_vlan_heat_ceilometer_scenario.yaml b/deploy/scenario/ha_vlan_heat_ceilometer_scenario.yaml -index d1d5191..65b8b20 100644 ---- a/deploy/scenario/ha_vlan_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/ha_vlan_heat_ceilometer_scenario.yaml -@@ -102,22 +102,23 @@ dha-override-config: - nodes: - - id: 1 - libvirtName: controller1 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 2 - libvirtName: controller2 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 3 - libvirtName: controller3 -- libvirtTemplate: templates/virtual_environment/vms/controller.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml - - id: 4 - libvirtName: compute1 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 5 - libvirtName: compute2 -- libvirtTemplate: templates/virtual_environment/vms/compute.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml - - id: 6 - libvirtName: fuel-master -- libvirtTemplate: templates/virtual_environment/vms/fuel.xml -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system - isFuel: yes - username: root - password: r00tme -diff --git a/deploy/scenario/no-ha_heat_ceilometer_scenario.yaml b/deploy/scenario/no-ha_heat_ceilometer_scenario.yaml -index 3dd5f84..d738154 100644 ---- a/deploy/scenario/no-ha_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/no-ha_heat_ceilometer_scenario.yaml -@@ -41,17 +41,42 @@ dea-override-config: - - id: 1 - interfaces: interfaces_1 - role: mongo,controller,congress -+ transformations: transformations_1 - - id: 2 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - id: 3 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - id: 4 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - dha-override-config: -+ nodes: -+ - id: 1 -+ libvirtName: controller1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml -+ - id: 2 -+ libvirtName: compute1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 3 -+ libvirtName: compute2 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 4 -+ libvirtName: compute3 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 5 -+ libvirtName: fuel-master -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system -+ isFuel: yes -+ username: root -+ password: r00tme -+ - # disks: - # contrail: 500G - -diff --git a/deploy/scenario/no-ha_nfv-kvm_heat_ceilometer_scenario.yaml b/deploy/scenario/no-ha_nfv-kvm_heat_ceilometer_scenario.yaml -index 4661a26..7ade095 100644 ---- a/deploy/scenario/no-ha_nfv-kvm_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/no-ha_nfv-kvm_heat_ceilometer_scenario.yaml -@@ -48,15 +48,19 @@ dea-override-config: - - id: 1 - interfaces: interfaces_1 - role: mongo,controller -+ transformations: transformations_1 - - id: 2 - interfaces: interfaces_1 - role: ceph-osd -+ transformations: transformations_2 - - id: 3 - interfaces: interfaces_1 - role: compute -+ transformations: transformations_2 - - id: 4 - interfaces: interfaces_1 - role: compute -+ transformations: transformations_2 - - settings: - editable: -@@ -141,6 +145,27 @@ dea-override-config: - weight: 10 - - dha-override-config: -+ nodes: -+ - id: 1 -+ libvirtName: controller1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml -+ - id: 2 -+ libvirtName: compute1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 3 -+ libvirtName: compute2 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 4 -+ libvirtName: compute3 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 5 -+ libvirtName: fuel-master -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system -+ isFuel: yes -+ username: root -+ password: r00tme -+ - # disks: - # contrail: 500G - -diff --git a/deploy/scenario/no-ha_nfv-kvm_nfv-ovs_heat_ceilometer_scenario.yaml b/deploy/scenario/no-ha_nfv-kvm_nfv-ovs_heat_ceilometer_scenario.yaml -index 87364e8..a3f0864 100644 ---- a/deploy/scenario/no-ha_nfv-kvm_nfv-ovs_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/no-ha_nfv-kvm_nfv-ovs_heat_ceilometer_scenario.yaml -@@ -55,15 +55,19 @@ dea-override-config: - - id: 1 - interfaces: interfaces_1 - role: mongo,controller -+ transformations: transformations_1 - - id: 2 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - id: 3 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - id: 4 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - settings: - editable: - additional_components: -@@ -104,6 +108,27 @@ dea-override-config: - weight: 30 - - dha-override-config: -+ nodes: -+ - id: 1 -+ libvirtName: controller1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml -+ - id: 2 -+ libvirtName: compute1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 3 -+ libvirtName: compute2 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 4 -+ libvirtName: compute3 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 5 -+ libvirtName: fuel-master -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system -+ isFuel: yes -+ username: root -+ password: r00tme -+ - # disks: - # contrail: 500G - -diff --git a/deploy/scenario/no-ha_nfv-ovs_heat_ceilometer_scenario.yaml b/deploy/scenario/no-ha_nfv-ovs_heat_ceilometer_scenario.yaml -index 7ab76cb..f4e0adc 100644 ---- a/deploy/scenario/no-ha_nfv-ovs_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/no-ha_nfv-ovs_heat_ceilometer_scenario.yaml -@@ -56,19 +56,22 @@ dea-override-config: - - id: 1 - interfaces: interfaces_vlan - role: mongo,controller -+ transformations: transformations_1 - - id: 2 - interfaces: interfaces_dpdk - role: ceph-osd,compute - attributes: attributes_1 -+ transformations: transformations_2 - - id: 3 - interfaces: interfaces_dpdk - role: ceph-osd,compute - attributes: attributes_1 -+ transformations: transformations_2 - - id: 4 - interfaces: interfaces_dpdk - role: ceph-osd,compute - attributes: attributes_1 -- -+ transformations: transformations_2 - attributes_1: - hugepages: - dpdk: -@@ -102,6 +105,27 @@ dea-override-config: - vlan_start: null - - dha-override-config: -+ nodes: -+ - id: 1 -+ libvirtName: controller1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml -+ - id: 2 -+ libvirtName: compute1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 3 -+ libvirtName: compute2 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 4 -+ libvirtName: compute3 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 5 -+ libvirtName: fuel-master -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system -+ isFuel: yes -+ username: root -+ password: r00tme -+ - # disks: - # contrail: 500G - -diff --git a/deploy/scenario/no-ha_odl-bgpvpn_scenario.yaml b/deploy/scenario/no-ha_odl-bgpvpn_scenario.yaml -index 6f21ea2..e92a741 100644 ---- a/deploy/scenario/no-ha_odl-bgpvpn_scenario.yaml -+++ b/deploy/scenario/no-ha_odl-bgpvpn_scenario.yaml -@@ -60,22 +60,47 @@ dea-override-config: - - id: 1 - interfaces: interfaces_1 - role: mongo,controller -+ transformations: transformations_1 - - id: 2 - interfaces: interfaces_1 - role: ceph-osd,opendaylight -+ transformations: transformations_2 - - id: 3 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - id: 4 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - settings: - editable: - public_network_assignment: - assign_to_all_nodes: - value: true - - dha-override-config: -+ nodes: -+ - id: 1 -+ libvirtName: controller1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml -+ - id: 2 -+ libvirtName: compute1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 3 -+ libvirtName: compute2 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 4 -+ libvirtName: compute3 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 5 -+ libvirtName: fuel-master -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system -+ isFuel: yes -+ username: root -+ password: r00tme -+ - # disks: - # contrail: 500G - -diff --git a/deploy/scenario/no-ha_odl-l2_heat_ceilometer_scenario.yaml b/deploy/scenario/no-ha_odl-l2_heat_ceilometer_scenario.yaml -index f8787d7..1efa89a 100644 ---- a/deploy/scenario/no-ha_odl-l2_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/no-ha_odl-l2_heat_ceilometer_scenario.yaml -@@ -61,17 +61,42 @@ dea-override-config: - - id: 1 - interfaces: interfaces_1 - role: mongo,controller -+ transformations: transformations_1 - - id: 2 - interfaces: interfaces_1 - role: ceph-osd,opendaylight -+ transformations: transformations_2 - - id: 3 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - id: 4 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - dha-override-config: -+ nodes: -+ - id: 1 -+ libvirtName: controller1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml -+ - id: 2 -+ libvirtName: compute1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 3 -+ libvirtName: compute2 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 4 -+ libvirtName: compute3 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 5 -+ libvirtName: fuel-master -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system -+ isFuel: yes -+ username: root -+ password: r00tme -+ - # disks: - # contrail: 500G - -diff --git a/deploy/scenario/no-ha_odl-l2_sfc_heat_ceilometer_scenario.yaml b/deploy/scenario/no-ha_odl-l2_sfc_heat_ceilometer_scenario.yaml -index 90a45d5..b1fe773 100644 ---- a/deploy/scenario/no-ha_odl-l2_sfc_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/no-ha_odl-l2_sfc_heat_ceilometer_scenario.yaml -@@ -80,17 +80,42 @@ dea-override-config: - - id: 1 - interfaces: interfaces_1 - role: mongo,controller,tacker -+ transformations: transformations_1 - - id: 2 - interfaces: interfaces_1 - role: ceph-osd,opendaylight -+ transformations: transformations_2 - - id: 3 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - id: 4 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - dha-override-config: -+ nodes: -+ - id: 1 -+ libvirtName: controller1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml -+ - id: 2 -+ libvirtName: compute1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 3 -+ libvirtName: compute2 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 4 -+ libvirtName: compute3 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 5 -+ libvirtName: fuel-master -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system -+ isFuel: yes -+ username: root -+ password: r00tme -+ - # disks: - # contrail: 500G - -diff --git a/deploy/scenario/no-ha_odl-l3_heat_ceilometer_scenario.yaml b/deploy/scenario/no-ha_odl-l3_heat_ceilometer_scenario.yaml -index 0c8415f..b59a049 100644 ---- a/deploy/scenario/no-ha_odl-l3_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/no-ha_odl-l3_heat_ceilometer_scenario.yaml -@@ -71,15 +71,19 @@ dea-override-config: - - id: 1 - interfaces: interfaces_1 - role: mongo,controller -+ transformations: transformations_1 - - id: 2 - interfaces: interfaces_1 - role: ceph-osd,opendaylight -+ transformations: transformations_2 - - id: 3 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - id: 4 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - settings: - editable: - public_network_assignment: -@@ -87,6 +91,27 @@ dea-override-config: - value: true - - dha-override-config: -+ nodes: -+ - id: 1 -+ libvirtName: controller1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml -+ - id: 2 -+ libvirtName: compute1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 3 -+ libvirtName: compute2 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 4 -+ libvirtName: compute3 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 5 -+ libvirtName: fuel-master -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system -+ isFuel: yes -+ username: root -+ password: r00tme -+ - # disks: - # contrail: 500G - -diff --git a/deploy/scenario/no-ha_onos_nofeature_heat_ceilometer_scenario.yaml b/deploy/scenario/no-ha_onos_nofeature_heat_ceilometer_scenario.yaml -index f5c00f0..e51dd56 100644 ---- a/deploy/scenario/no-ha_onos_nofeature_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/no-ha_onos_nofeature_heat_ceilometer_scenario.yaml -@@ -68,21 +68,47 @@ dea-override-config: - - id: 1 - interfaces: interfaces_1 - role: mongo,controller,onos -+ transformations: transformations_1 - - id: 2 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - id: 3 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - id: 4 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - settings: - editable: - public_network_assignment: - assign_to_all_nodes: - value: true -+ - dha-override-config: -+ nodes: -+ - id: 1 -+ libvirtName: controller1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml -+ - id: 2 -+ libvirtName: compute1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 3 -+ libvirtName: compute2 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 4 -+ libvirtName: compute3 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 5 -+ libvirtName: fuel-master -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system -+ isFuel: yes -+ username: root -+ password: r00tme -+ - # disks: - # contrail: 500G - -diff --git a/deploy/scenario/no-ha_onos_sfc_heat_ceilometer_scenario.yaml b/deploy/scenario/no-ha_onos_sfc_heat_ceilometer_scenario.yaml -index 587346e..21a2be2 100644 ---- a/deploy/scenario/no-ha_onos_sfc_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/no-ha_onos_sfc_heat_ceilometer_scenario.yaml -@@ -48,21 +48,46 @@ dea-override-config: - - id: 1 - interfaces: interfaces_1 - role: mongo,controller,onos -+ transformations: transformations_1 - - id: 2 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - id: 3 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - id: 4 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - settings: - editable: - public_network_assignment: - assign_to_all_nodes: - value: true - dha-override-config: -+ nodes: -+ - id: 1 -+ libvirtName: controller1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml -+ - id: 2 -+ libvirtName: compute1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 3 -+ libvirtName: compute2 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 4 -+ libvirtName: compute3 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 5 -+ libvirtName: fuel-master -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system -+ isFuel: yes -+ username: root -+ password: r00tme -+ - # disks: - # contrail: 500G - -diff --git a/deploy/scenario/no-ha_vlan_heat_ceilometer_scenario.yaml b/deploy/scenario/no-ha_vlan_heat_ceilometer_scenario.yaml -index 567ea98..ada5b7e 100644 ---- a/deploy/scenario/no-ha_vlan_heat_ceilometer_scenario.yaml -+++ b/deploy/scenario/no-ha_vlan_heat_ceilometer_scenario.yaml -@@ -45,15 +45,19 @@ dea-override-config: - - id: 1 - interfaces: interfaces_1 - role: mongo,controller -+ transformations: transformations_1 - - id: 2 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - id: 3 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - - id: 4 - interfaces: interfaces_1 - role: ceph-osd,compute -+ transformations: transformations_2 - settings: - editable: - additional_components: -@@ -94,6 +98,27 @@ dea-override-config: - weight: 30 - - dha-override-config: -+ nodes: -+ - id: 1 -+ libvirtName: controller1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/controller.xml -+ - id: 2 -+ libvirtName: compute1 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 3 -+ libvirtName: compute2 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 4 -+ libvirtName: compute3 -+ libvirtTemplate: templates/arm/virtual_environment/vms/compute.xml -+ - id: 5 -+ libvirtName: fuel-master -+ libvirtTemplate: templates/arm/virtual_environment/vms/fuel.xml -+ libvirtUri: qemu+ssh://jenkins@10.0.2.5/system -+ isFuel: yes -+ username: root -+ password: r00tme -+ - # These overrides only take effect for virtual deployment scenarios - - ############################################################################## -diff --git a/deploy/templates/arm/virtual_environment/networks/.gitkeep b/deploy/templates/arm/virtual_environment/networks/.gitkeep -new file mode 100644 -index 0000000..e69de29 -diff --git a/deploy/templates/arm/virtual_environment/vms/compute.xml b/deploy/templates/arm/virtual_environment/vms/compute.xml -new file mode 100644 -index 0000000..db3ba29 ---- /dev/null -+++ b/deploy/templates/arm/virtual_environment/vms/compute.xml -@@ -0,0 +1,57 @@ -+<domain type='kvm'> -+ <name>compute</name> -+ <memory unit='KiB'>8392704</memory> -+ <currentMemory unit='KiB'>8392704</currentMemory> -+ <vcpu placement='static'>6</vcpu> -+ <os> -+ <type arch='aarch64' machine='virt-2.6'>hvm</type> -+ <loader readonly='yes' type='pflash'>/usr/share/AAVMF/AAVMF_CODE.fd</loader> -+ </os> -+ <features> -+ <gic version='3'/> -+ </features> -+ <cpu mode='host-model'> -+ <model fallback='allow'/> -+ </cpu> -+ <clock offset='utc'/> -+ <on_poweroff>destroy</on_poweroff> -+ <on_reboot>restart</on_reboot> -+ <on_crash>restart</on_crash> -+ <devices> -+ <emulator>/usr/bin/kvm</emulator> -+ <disk type='file' device='disk'> -+ <driver name='qemu' type='raw' cache='none' io='native'/> -+ <source file='disk.raw'/> -+ <target dev='vda' bus='virtio'/> -+ <boot order='2'/> -+ <address type='virtio-mmio'/> -+ </disk> -+ <controller type='scsi' index='0' model='virtio-scsi'> -+ <address type='virtio-mmio'/> -+ </controller> -+ <controller type='pci' index='0' model='pcie-root'/> -+ <interface type='bridge'> -+ <source bridge='admin8_br'/> -+ <model type='virtio'/> -+ <boot order='1'/> -+ <address type='virtio-mmio'/> -+ </interface> -+ <interface type='bridge'> -+ <source bridge='public8_br'/> -+ <model type='virtio'/> -+ <address type='virtio-mmio'/> -+ </interface> -+ <interface type='bridge'> -+ <source bridge='trunk8_br'/> -+ <model type='virtio'/> -+ <address type='virtio-mmio'/> -+ </interface> -+ <serial type='pty'> -+ <target port='0'/> -+ </serial> -+ <console type='pty'> -+ <target type='serial' port='0'/> -+ </console> -+ </devices> -+</domain> -+ -diff --git a/deploy/templates/arm/virtual_environment/vms/controller.xml b/deploy/templates/arm/virtual_environment/vms/controller.xml -new file mode 100644 -index 0000000..6bd0385 ---- /dev/null -+++ b/deploy/templates/arm/virtual_environment/vms/controller.xml -@@ -0,0 +1,57 @@ -+<domain type='kvm'> -+ <name>controller</name> -+ <memory unit='KiB'>8392704</memory> -+ <currentMemory unit='KiB'>8392704</currentMemory> -+ <vcpu placement='static'>6</vcpu> -+ <os> -+ <type arch='aarch64' machine='virt-2.6'>hvm</type> -+ <loader readonly='yes' type='pflash'>/usr/share/AAVMF/AAVMF_CODE.fd</loader> -+ </os> -+ <features> -+ <gic version='3'/> -+ </features> -+ <cpu mode='host-model'> -+ <model fallback='allow'/> -+ </cpu> -+ <clock offset='utc'/> -+ <on_poweroff>destroy</on_poweroff> -+ <on_reboot>restart</on_reboot> -+ <on_crash>restart</on_crash> -+ <devices> -+ <emulator>/usr/bin/kvm</emulator> -+ <disk type='file' device='disk'> -+ <driver name='qemu' type='raw' cache='none' io='native'/> -+ <source file='disk.raw'/> -+ <target dev='vda' bus='virtio'/> -+ <boot order='2'/> -+ <address type='virtio-mmio'/> -+ </disk> -+ <controller type='scsi' index='0' model='virtio-scsi'> -+ <address type='virtio-mmio'/> -+ </controller> -+ <controller type='pci' index='0' model='pcie-root'/> -+ <interface type='bridge'> -+ <source bridge='admin8_br'/> -+ <model type='virtio'/> -+ <boot order='1'/> -+ <address type='virtio-mmio'/> -+ </interface> -+ <interface type='bridge'> -+ <source bridge='public8_br'/> -+ <model type='virtio'/> -+ <address type='virtio-mmio'/> -+ </interface> -+ <interface type='bridge'> -+ <source bridge='trunk8_br'/> -+ <model type='virtio'/> -+ <address type='virtio-mmio'/> -+ </interface> -+ <serial type='pty'> -+ <target port='0'/> -+ </serial> -+ <console type='pty'> -+ <target type='serial' port='0'/> -+ </console> -+ </devices> -+</domain> -+ -diff --git a/deploy/templates/arm/virtual_environment/vms/fuel.xml b/deploy/templates/arm/virtual_environment/vms/fuel.xml -new file mode 100644 -index 0000000..ad7c67f ---- /dev/null -+++ b/deploy/templates/arm/virtual_environment/vms/fuel.xml -@@ -0,0 +1,99 @@ -+<domain type='kvm'> -+ <name>fuel</name> -+ <memory unit='KiB'>8290304</memory> -+ <currentMemory unit='KiB'>8290304</currentMemory> -+ <vcpu placement='static'>4</vcpu> -+ <resource> -+ <partition>/machine</partition> -+ </resource> -+ <os> -+ <type arch='x86_64' machine='pc-i440fx-rhel7.0.0'>hvm</type> -+ <boot dev='hd'/> -+ <boot dev='cdrom'/> -+ <bootmenu enable='no'/> -+ </os> -+ <features> -+ <acpi/> -+ <apic/> -+ <pae/> -+ </features> -+ <cpu mode='host-model'> -+ <model fallback='allow'/> -+ </cpu> -+ <clock offset='utc'> -+ <timer name='rtc' tickpolicy='catchup'/> -+ <timer name='pit' tickpolicy='delay'/> -+ <timer name='hpet' present='no'/> -+ </clock> -+ <on_poweroff>destroy</on_poweroff> -+ <on_reboot>restart</on_reboot> -+ <on_crash>restart</on_crash> -+ <pm> -+ <suspend-to-mem enabled='no'/> -+ <suspend-to-disk enabled='no'/> -+ </pm> -+ <devices> -+ <emulator>/usr/libexec/qemu-kvm</emulator> -+ <disk type='block' device='cdrom'> -+ <driver name='qemu' type='raw'/> -+ <target dev='hdc' bus='ide'/> -+ <readonly/> -+ </disk> -+ <disk type='file' device='disk'> -+ <driver name='qemu' type='raw' cache='none' io='native'/> -+ <source file='disk.raw'/> -+ <target dev='vda' bus='virtio'/> -+ <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> -+ </disk> -+ <controller type='usb' index='0' model='ich9-ehci1'> -+ <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x7'/> -+ </controller> -+ <controller type='usb' index='0' model='ich9-uhci1'> -+ <master startport='0'/> -+ <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0' multifunction='on'/> -+ </controller> -+ <controller type='usb' index='0' model='ich9-uhci2'> -+ <master startport='2'/> -+ <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x1'/> -+ </controller> -+ <controller type='usb' index='0' model='ich9-uhci3'> -+ <master startport='4'/> -+ <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x2'/> -+ </controller> -+ <controller type='pci' index='0' model='pci-root'/> -+ <controller type='ide' index='0'> -+ <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> -+ </controller> -+ <controller type='virtio-serial' index='0'> -+ <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> -+ </controller> -+ <interface type='bridge'> -+ <source bridge='admin8_br0'/> -+ <model type='virtio'/> -+ <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> -+ </interface> -+ <interface type='bridge'> -+ <source bridge='public8_br0'/> -+ <model type='virtio'/> -+ <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> -+ </interface> -+ <serial type='pty'> -+ <target port='0'/> -+ </serial> -+ <console type='pty'> -+ <target type='serial' port='0'/> -+ </console> -+ <input type='mouse' bus='ps2'/> -+ <input type='keyboard' bus='ps2'/> -+ <graphics type='vnc' port='-1' autoport='yes' listen='127.0.0.1'> -+ <listen type='address' address='127.0.0.1'/> -+ </graphics> -+ <video> -+ <model type='vga' vram='16384' heads='1'/> -+ <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> -+ </video> -+ <memballoon model='virtio'> -+ <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> -+ </memballoon> -+ </devices> -+</domain> diff --git a/patches/opnfv-fuel/upstream-backports/0005-CI-deploy-cache-Store-and-reuse-deploy-artifacts.patch b/patches/opnfv-fuel/upstream-backports/0005-CI-deploy-cache-Store-and-reuse-deploy-artifacts.patch deleted file mode 100644 index 9997dedf..00000000 --- a/patches/opnfv-fuel/upstream-backports/0005-CI-deploy-cache-Store-and-reuse-deploy-artifacts.patch +++ /dev/null @@ -1,786 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Thu, 24 Nov 2016 23:02:04 +0100 -Subject: [PATCH] CI: deploy-cache: Store and reuse deploy artifacts - -Add support for caching deploy artifacts, like bootstraps and -target images, which take a lot of time at each deploy to be built, -considering it requires a cross-debootstrap via qemu-user-static and -binfmt. - -For OPNFV CI, the cache will piggy back on the <iso_mount> mechanism, -and be located at: -/iso_mount/opnfv_ci/<branch>/deploy-cache - -TODO: Use dea interface adapter in target images fingerprinting. -TODO: remote fingerprinting -TODO: differentiate between bootstraps and targetimages, so we don't -end up trying to use one cache artifact type as the other. -TODO: implement sanity checks for bootstrap and target images; -TODO: switch `exec_cmd('mkdir ...')` to `create_dir_if_not_exists`; - -JIRA: ARMBAND-172 -JIRA: ARMBAND-242 - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - ...p_admin_node.sh-deploy_cache-install-hook.patch | 90 ++++++ - ci/deploy.sh | 14 +- - deploy/cloud/configure_settings.py | 4 + - deploy/cloud/deployment.py | 12 + - deploy/deploy.py | 25 +- - deploy/deploy_cache.py | 314 +++++++++++++++++++++ - deploy/deploy_env.py | 13 +- - deploy/install_fuel_master.py | 9 +- - 8 files changed, 472 insertions(+), 9 deletions(-) - create mode 100644 build/f_repos/patch/fuel-main/0006-bootstrap_admin_node.sh-deploy_cache-install-hook.patch - create mode 100644 deploy/deploy_cache.py - -diff --git a/build/f_repos/patch/fuel-main/0006-bootstrap_admin_node.sh-deploy_cache-install-hook.patch b/build/f_repos/patch/fuel-main/0006-bootstrap_admin_node.sh-deploy_cache-install-hook.patch -new file mode 100644 -index 0000000..7acb746 ---- /dev/null -+++ b/build/f_repos/patch/fuel-main/0006-bootstrap_admin_node.sh-deploy_cache-install-hook.patch -@@ -0,0 +1,90 @@ -+From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -+Date: Mon, 28 Nov 2016 14:27:48 +0100 -+Subject: [PATCH] bootstrap_admin_node.sh: deploy_cache install hook -+ -+Tooling on the automatic deploy side was updated to support deploy -+caching of artifacts like bootstrap (and id_rsa keypair), target -+images etc. -+ -+Add installation hook that calls `fuel-bootstrap import` instead of -+`build` when a bootstrap tar is available in the agreed location, -+/var/lib/opnfv/cache/bootstraps/. -+ -+Temporary until Fuel@Openstack fixes Master key propagation to nodes' -+authorized_keys, use Mcollective remote shell execute to add it -+during deployment. -+This might duplicate the entry in authorized_keys during re-deploys. -+ -+JIRA: ARMBAND-172 -+JIRA: ARMBAND-242 -+ -+Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -+--- -+ iso/bootstrap_admin_node.sh | 35 ++++++++++++++++++++++++++++++++++- -+ 1 file changed, 34 insertions(+), 1 deletion(-) -+ -+diff --git a/iso/bootstrap_admin_node.sh b/iso/bootstrap_admin_node.sh -+index 4f5ce4e..4c79552 100755 -+--- a/iso/bootstrap_admin_node.sh -++++ b/iso/bootstrap_admin_node.sh -+@@ -64,6 +64,8 @@ wget \ -+ ASTUTE_YAML='/etc/fuel/astute.yaml' -+ BOOTSTRAP_NODE_CONFIG="/etc/fuel/bootstrap_admin_node.conf" -+ CUSTOM_REPOS="/root/default_deb_repos.yaml" -++OPNFV_CACHE_PATH="/var/cache/opnfv/bootstraps" -++OPNFV_CACHE_TAR="opnfv-bootstraps-cache.tar" -+ bs_build_log='/var/log/fuel-bootstrap-image-build.log' -+ bs_status=0 -+ # Backup network configs to this folder. Folder will be created only if -+@@ -97,6 +99,7 @@ image becomes available, reboot nodes that failed to be discovered." -+ bs_done_message="Default bootstrap image building done. Now you can boot new \ -+ nodes over PXE, they will be discovered and become available for installing \ -+ OpenStack on them" -++bs_cache_message="OPNFV deploy cache: bootstrap image injected." -+ # Update issues messages -+ update_warn_message="There is an issue connecting to update repository of \ -+ your distributions of OpenStack. \ -+@@ -509,12 +512,42 @@ set_ui_bootstrap_error () { -+ EOF -+ } -+ -++function inject_cached_ssh_key () { -++ # FIXME(armband): Propagate master ssh key to nodes' -++ # authorized_keys, until upstream fixes this for image build. -++ local moddir="/etc/puppet/${OPENSTACK_VERSION}/modules/osnailyfacter/modular" -++ cat >> "${moddir}/astute/generate_keys.sh" <<-EOF -++ mco rpc execute_shell_command execute \\ -++ cmd="echo $(cat /root/.ssh/id_rsa.pub) >> /root/.ssh/authorized_keys" -++ EOF -++} -++ -++function inject_cached_ubuntu_bootstrap () { -++ if [ -f "${OPNFV_CACHE_PATH}/${OPNFV_CACHE_TAR}" -a \ -++ -f "${OPNFV_CACHE_PATH}/id_rsa.pub" -a \ -++ -f "${OPNFV_CACHE_PATH}/id_rsa" ]; then -++ if cp "${OPNFV_CACHE_PATH}/id_rsa"* "/root/.ssh/" && \ -++ cp "/root/.ssh/id_rsa.pub" "/root/.ssh/authorized_keys" && \ -++ cp "/root/.ssh/id_rsa.pub" "/etc/cobbler/authorized_keys" && \ -++ sed -i -e "s|\"ssh-rsa .*\"|\"$(cat /root/.ssh/id_rsa.pub)\"|g" \ -++ /etc/nailgun/settings.yaml && \ -++ fuel-bootstrap -v --debug import --activate \ -++ "${OPNFV_CACHE_PATH}/${OPNFV_CACHE_TAR}" >>"$bs_build_log" 2>&1; then -++ inject_cached_ssh_key -++ fuel notify --topic "done" --send "${bs_cache_message}" -++ return 0 -++ fi -++ fi -++ return 1 -++} -++ -+ # Actually build the bootstrap image -+ build_ubuntu_bootstrap () { -+ local ret=1 -+ echo ${bs_progress_message} >&2 -+ set_ui_bootstrap_error "${bs_progress_message}" >&2 -+- if fuel-bootstrap -v --debug build --target_arch arm64 --activate >>"$bs_build_log" 2>&1; then -++ if inject_cached_ubuntu_bootstrap || fuel-bootstrap -v --debug \ -++ build --activate --target_arch arm64 >>"$bs_build_log" 2>&1; then -+ ret=0 -+ fuel notify --topic "done" --send "${bs_done_message}" -+ else -diff --git a/ci/deploy.sh b/ci/deploy.sh -index 081806c..4b1ae0e 100755 ---- a/ci/deploy.sh -+++ b/ci/deploy.sh -@@ -29,7 +29,7 @@ cat << EOF - xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx - `basename $0`: Deploys the Fuel@OPNFV stack - --usage: `basename $0` -b base-uri [-B PXE Bridge] [-f] [-F] [-H] -l lab-name -p pod-name -s deploy-scenario [-S image-dir] [-T timeout] -i iso -+usage: `basename $0` -b base-uri [-B PXE Bridge] [-f] [-F] [-H] -l lab-name -p pod-name -s deploy-scenario [-S image-dir] [-C deploy-cache-dir] [-T timeout] -i iso - -s deployment-scenario [-S optional Deploy-scenario path URI] - [-R optional local relen repo (containing deployment Scenarios] - -@@ -47,6 +47,7 @@ OPTIONS: - -p Pod-name - -s Deploy-scenario short-name/base-file-name - -S Storage dir for VM images -+ -C Deploy cache dir for storing image artifacts - -T Timeout, in minutes, for the deploy. - -i iso url - -@@ -79,6 +80,7 @@ Input parameters to the build script is: - or a deployment short-name as defined by scenario.yaml in the deployment - scenario path. - -S Storage dir for VM images, default is fuel/deploy/images -+-C Deploy cache dir for bootstrap and target image artifacts, optional - -T Timeout, in minutes, for the deploy. It defaults to using the DEPLOY_TIMEOUT - environment variable when defined, or to the default in deploy.py otherwise - -i .iso image to be deployed (needs to be provided in a URI -@@ -116,6 +118,7 @@ FUEL_CREATION_ONLY='' - NO_DEPLOY_ENVIRONMENT='' - STORAGE_DIR='' - DRY_RUN=0 -+DEPLOY_CACHE_DIR='' - if ! [ -z $DEPLOY_TIMEOUT ]; then - DEPLOY_TIMEOUT="-dt $DEPLOY_TIMEOUT" - else -@@ -128,7 +131,7 @@ fi - ############################################################################ - # BEGIN of main - # --while getopts "b:B:dfFHl:L:p:s:S:T:i:he" OPTION -+while getopts "b:B:dfFHl:L:p:s:S:C:T:i:he" OPTION - do - case $OPTION in - b) -@@ -179,6 +182,9 @@ do - STORAGE_DIR="-s ${OPTARG}" - fi - ;; -+ C) -+ DEPLOY_CACHE_DIR="-dc ${OPTARG}" -+ ;; - T) - DEPLOY_TIMEOUT="-dt ${OPTARG}" - ;; -@@ -243,8 +249,8 @@ if [ $DRY_RUN -eq 0 ]; then - ISO=${SCRIPT_PATH}/ISO/image.iso - fi - # Start deployment -- echo "python deploy.py $DEPLOY_LOG $STORAGE_DIR $PXE_BRIDGE $USE_EXISTING_FUEL $FUEL_CREATION_ONLY $NO_HEALTH_CHECK $NO_DEPLOY_ENVIRONMENT -dea ${SCRIPT_PATH}/config/dea.yaml -dha ${SCRIPT_PATH}/config/dha.yaml -iso $ISO $DEPLOY_TIMEOUT" -- python deploy.py $DEPLOY_LOG $STORAGE_DIR $PXE_BRIDGE $USE_EXISTING_FUEL $FUEL_CREATION_ONLY $NO_HEALTH_CHECK $NO_DEPLOY_ENVIRONMENT -dea ${SCRIPT_PATH}/config/dea.yaml -dha ${SCRIPT_PATH}/config/dha.yaml -iso $ISO $DEPLOY_TIMEOUT -+ echo "python deploy.py $DEPLOY_LOG $STORAGE_DIR $PXE_BRIDGE $USE_EXISTING_FUEL $FUEL_CREATION_ONLY $NO_HEALTH_CHECK $NO_DEPLOY_ENVIRONMENT -dea ${SCRIPT_PATH}/config/dea.yaml -dha ${SCRIPT_PATH}/config/dha.yaml -iso $ISO $DEPLOY_TIMEOUT $DEPLOY_CACHE_DIR" -+ python deploy.py $DEPLOY_LOG $STORAGE_DIR $PXE_BRIDGE $USE_EXISTING_FUEL $FUEL_CREATION_ONLY $NO_HEALTH_CHECK $NO_DEPLOY_ENVIRONMENT -dea ${SCRIPT_PATH}/config/dea.yaml -dha ${SCRIPT_PATH}/config/dha.yaml -iso $ISO $DEPLOY_TIMEOUT $DEPLOY_CACHE_DIR - fi - popd > /dev/null - -diff --git a/deploy/cloud/configure_settings.py b/deploy/cloud/configure_settings.py -index b60a60f..4e007e1 100644 ---- a/deploy/cloud/configure_settings.py -+++ b/deploy/cloud/configure_settings.py -@@ -71,5 +71,9 @@ class ConfigureSettings(object): - settings['editable'][plugin]['metadata']['chosen_id'] = orig_dea['editable'][plugin]['metadata']['chosen_id'] - settings['editable'][plugin]['metadata']['versions'][0]['metadata']['plugin_id'] = orig_dea['editable'][plugin]['metadata']['versions'][0]['metadata']['plugin_id'] - -+ # deploy-cache req: pass master id_rsa.pub as authorized key -+ with io.open('/root/.ssh/id_rsa.pub', 'r') as pkey: -+ settings['editable']['operator_user']['authkeys']['value'] = pkey.read() -+ - with io.open(settings_yaml, 'w') as stream: - yaml.dump(settings, stream, default_flow_style=False) -diff --git a/deploy/cloud/deployment.py b/deploy/cloud/deployment.py -index 4329a4c..a84d46c 100644 ---- a/deploy/cloud/deployment.py -+++ b/deploy/cloud/deployment.py -@@ -19,6 +19,8 @@ from common import ( - log, - ) - -+from deploy_cache import DeployCache -+ - SEARCH_TEXT = '(err)' - LOG_FILE = '/var/log/puppet.log' - GREP_LINES_OF_LEADING_CONTEXT = 100 -@@ -51,6 +53,14 @@ class Deployment(object): - self.pattern = re.compile( - '\d\d\d\d-\d\d-\d\d\s\d\d:\d\d:\d\d') - -+ def deploy_cache_install_targetimages(self): -+ log('Using target images from deploy cache') -+ DeployCache.install_targetimages_for_env(self.env_id) -+ -+ def deploy_cache_extract_targetimages(self): -+ log('Collecting Fuel target image files for deploy cache') -+ DeployCache.extract_targetimages_from_env(self.env_id) -+ - def collect_error_logs(self): - for node_id, roles_blade in self.node_id_roles_dict.iteritems(): - log_list = [] -@@ -112,6 +122,7 @@ class Deployment(object): - start = time.time() - - log('Starting deployment of environment %s' % self.env_id) -+ self.deploy_cache_install_targetimages() - deploy_id = None - ready = False - timeout = False -@@ -144,6 +155,7 @@ class Deployment(object): - err('Deployment timed out, environment %s is not operational, ' - 'snapshot will not be performed' - % self.env_id) -+ self.deploy_cache_extract_targetimages() - if ready: - log('Environment %s successfully deployed' - % self.env_id) -diff --git a/deploy/deploy.py b/deploy/deploy.py -index 7648baf..ee3cb7a 100755 ---- a/deploy/deploy.py -+++ b/deploy/deploy.py -@@ -22,6 +22,7 @@ from dea import DeploymentEnvironmentAdapter - from dha import DeploymentHardwareAdapter - from install_fuel_master import InstallFuelMaster - from deploy_env import CloudDeploy -+from deploy_cache import DeployCache - from execution_environment import ExecutionEnvironment - - from common import ( -@@ -61,7 +62,8 @@ class AutoDeploy(object): - def __init__(self, no_fuel, fuel_only, no_health_check, cleanup_only, - cleanup, storage_dir, pxe_bridge, iso_file, dea_file, - dha_file, fuel_plugins_dir, fuel_plugins_conf_dir, -- no_plugins, deploy_timeout, no_deploy_environment, deploy_log): -+ no_plugins, deploy_cache_dir, deploy_timeout, -+ no_deploy_environment, deploy_log): - self.no_fuel = no_fuel - self.fuel_only = fuel_only - self.no_health_check = no_health_check -@@ -75,6 +77,7 @@ class AutoDeploy(object): - self.fuel_plugins_dir = fuel_plugins_dir - self.fuel_plugins_conf_dir = fuel_plugins_conf_dir - self.no_plugins = no_plugins -+ self.deploy_cache_dir = deploy_cache_dir - self.deploy_timeout = deploy_timeout - self.no_deploy_environment = no_deploy_environment - self.deploy_log = deploy_log -@@ -116,7 +119,7 @@ class AutoDeploy(object): - self.fuel_username, self.fuel_password, - self.dea_file, self.fuel_plugins_conf_dir, - WORK_DIR, self.no_health_check, -- self.deploy_timeout, -+ self.deploy_cache_dir, self.deploy_timeout, - self.no_deploy_environment, self.deploy_log) - with old_dep.ssh: - old_dep.check_previous_installation() -@@ -128,6 +131,7 @@ class AutoDeploy(object): - self.fuel_conf['ip'], self.fuel_username, - self.fuel_password, self.fuel_node_id, - self.iso_file, WORK_DIR, -+ self.deploy_cache_dir, - self.fuel_plugins_dir, self.no_plugins) - fuel.install() - -@@ -136,6 +140,7 @@ class AutoDeploy(object): - tmp_new_dir = '%s/newiso' % self.tmp_dir - try: - self.copy(tmp_orig_dir, tmp_new_dir) -+ self.deploy_cache_fingerprints(tmp_new_dir) - self.patch(tmp_new_dir, new_iso) - except Exception as e: - exec_cmd('fusermount -u %s' % tmp_orig_dir, False) -@@ -156,6 +161,12 @@ class AutoDeploy(object): - delete(tmp_orig_dir) - exec_cmd('chmod -R 755 %s' % tmp_new_dir) - -+ def deploy_cache_fingerprints(self, tmp_new_dir): -+ if self.deploy_cache_dir: -+ log('Deploy cache: Collecting fingerprints...') -+ deploy_cache = DeployCache(self.deploy_cache_dir) -+ deploy_cache.do_fingerprints(tmp_new_dir, self.dea_file) -+ - def patch(self, tmp_new_dir, new_iso): - log('Patching...') - patch_dir = '%s/%s' % (CWD, PATCH_DIR) -@@ -218,7 +229,8 @@ class AutoDeploy(object): - dep = CloudDeploy(self.dea, self.dha, self.fuel_conf['ip'], - self.fuel_username, self.fuel_password, - self.dea_file, self.fuel_plugins_conf_dir, -- WORK_DIR, self.no_health_check, self.deploy_timeout, -+ WORK_DIR, self.no_health_check, -+ self.deploy_cache_dir, self.deploy_timeout, - self.no_deploy_environment, self.deploy_log) - return dep.deploy() - -@@ -343,6 +355,8 @@ def parse_arguments(): - help='Fuel Plugins Configuration directory') - parser.add_argument('-np', dest='no_plugins', action='store_true', - default=False, help='Do not install Fuel Plugins') -+ parser.add_argument('-dc', dest='deploy_cache_dir', action='store', -+ help='Deploy Cache Directory') - parser.add_argument('-dt', dest='deploy_timeout', action='store', - default=240, help='Deployment timeout (in minutes) ' - '[default: 240]') -@@ -376,6 +390,10 @@ def parse_arguments(): - for bridge in args.pxe_bridge: - check_bridge(bridge, args.dha_file) - -+ if args.deploy_cache_dir: -+ log('Using deploy cache directory: %s' % args.deploy_cache_dir) -+ create_dir_if_not_exists(args.deploy_cache_dir) -+ - - kwargs = {'no_fuel': args.no_fuel, 'fuel_only': args.fuel_only, - 'no_health_check': args.no_health_check, -@@ -386,6 +404,7 @@ def parse_arguments(): - 'fuel_plugins_dir': args.fuel_plugins_dir, - 'fuel_plugins_conf_dir': args.fuel_plugins_conf_dir, - 'no_plugins': args.no_plugins, -+ 'deploy_cache_dir': args.deploy_cache_dir, - 'deploy_timeout': args.deploy_timeout, - 'no_deploy_environment': args.no_deploy_environment, - 'deploy_log': args.deploy_log} -diff --git a/deploy/deploy_cache.py b/deploy/deploy_cache.py -new file mode 100644 -index 0000000..30bfe30 ---- /dev/null -+++ b/deploy/deploy_cache.py -@@ -0,0 +1,314 @@ -+############################################################################### -+# Copyright (c) 2016 Enea AB and others. -+# Alexandru.Avadanii@enea.com -+# All rights reserved. This program and the accompanying materials -+# are made available under the terms of the Apache License, Version 2.0 -+# which accompanies this distribution, and is available at -+# http://www.apache.org/licenses/LICENSE-2.0 -+############################################################################### -+ -+import glob -+import hashlib -+import io -+import json -+import os -+import shutil -+import yaml -+ -+from common import ( -+ exec_cmd, -+ log, -+) -+ -+############################################################################### -+# Deploy Cache Flow Overview -+############################################################################### -+# 1. do_fingerprints -+# Can be called as soon as a Fuel Master ISO chroot is available. -+# This will gather all required information for uniquely identifying the -+# objects in cache (bootstraps, targetimages). -+# 2. inject_cache -+# Can be called as soon as we have a steady SSH connection to the Fuel -+# Master node. It will inject cached artifacts over SSH, for later install. -+# 3. (external, async) install cached bootstrap instead of building a new one -+# /sbin/bootstrap_admin_node.sh will check for cached bootstrap images -+# (with id_rsa, id_rsa.pub attached) and will install those via -+# $ fuel-bootstrap import opfnv-bootstraps-cache.tar -+# 4. install_targetimages_for_env -+# Should be called before cloud deploy is started, to install env-generic -+# 'env_X_...' cached images for the current environment ID. -+# Static method, to be used on the remote Fuel Master node; does not require -+# access to the deploy cache, it only moves around some local files. -+# 5. extract_targetimages_from_env -+# Should be called at env deploy finish, to prepare artifacts for caching. -+# Static method, same observations as above apply. -+# 6. collect_artifacts -+# Call last, to collect all artifacts. -+############################################################################### -+ -+############################################################################### -+# Deploy cache artifacts: -+# - id_rsa -+# - bootstrap image (Ubuntu) -+# - environment target image (Ubuntu) -+############################################################################### -+# Cache fingerprint covers: -+# - bootstrap: -+# - local mirror contents -+# - FIXME(armband): [disabled] package list (old fuel_bootstrap_cli.yaml) -+# - target image: -+# - local mirror contents -+# - package list (determined from DEA) -+############################################################################### -+# WARN: Cache fingerprint does NOT yet cover: -+# - image_data (always assume the default /boot, /); -+# - output_dir (always assume the default /var/www/nailgun/targetimages; -+# - codename (always assume the default, currently 'trusty'); -+# - extra_dirs: /usr/share/fuel_bootstrap_cli/files/trusty -+# - root_ssh_authorized_file, inluding the contents of /root/.ssh/id_rsa.pub -+# - Auxiliary repo .../mitaka-9.0/ubuntu/auxiliary -+# If the above change without triggering a cache miss, try clearing the cache. -+############################################################################### -+# WARN: Bootstrap caching implies RSA keypair to be reused! -+############################################################################### -+ -+# Local mirrros will be used on Fuel Master for both bootstrap and target image -+# build, from `http://127.0.0.1:8080/...` or `http://10.20.0.2:8080/...`: -+# - MOS .../mitaka-9.0/ubuntu/x86_64 -+# - Ubuntu .../mirrors/ubuntu/ -+# All these reside on Fuel Master at local path: -+NAILGUN_PATH = '/var/www/nailgun/' -+ -+# Artifact names (corresponding to nailgun subdirs) -+MIRRORS = 'mirrors' -+BOOTSTRAPS = 'bootstraps' -+TARGETIMAGES = 'targetimages' -+ -+# Info for collecting RSA keypair -+RSA_KEYPAIR_PATH = '/root/.ssh' -+RSA_KEYPAIR_FILES = ['id_rsa', 'id_rsa.pub'] -+ -+# Relative path for collecting the active bootstrap image(s) after env deploy -+NAILGUN_ACT_BOOTSTRAP_SUBDIR = '%s/active_bootstrap' % BOOTSTRAPS -+ -+# Relative path for collecting target image(s) for deployed enviroment -+NAILGUN_TIMAGES_SUBDIR = TARGETIMAGES -+ -+# FIXME(armband): re-include package list (old fuel_bootstrap_cli.yaml) -+# ISO_BOOTSTRAP_CLI_YAML = '/opnfv/fuel_bootstrap_cli.yaml' -+ -+# OPNFV Deploy Cache path on Fuel Master, where artifacts will be injected -+REMOTE_CACHE_PATH = '/var/cache/opnfv' -+ -+# OPNFV Bootstrap Cache tar archive name, to be used by bootstrap_admin_node.sh -+BOOTSTRAP_ARCHIVE = 'opnfv-bootstraps-cache.tar' -+ -+# Env-ID indep prefix -+ENVX = 'env_X_' -+ -+class DeployCache(object): -+ """OPNFV Deploy Cache - managed storage for cacheable artifacts""" -+ -+ def __init__(self, cache_dir, -+ fingerprints_yaml='deploy_cache_fingerprints.yaml'): -+ self.cache_dir = cache_dir -+ self.fingerprints_yaml = fingerprints_yaml -+ self.fingerprints = {BOOTSTRAPS: None, -+ MIRRORS: None, -+ TARGETIMAGES: None} -+ -+ def __load_fingerprints(self): -+ """Load deploy cache yaml config holding fingerprints""" -+ if os.path.isfile(self.fingerprints_yaml): -+ cache_fingerprints = open(self.fingerprints_yaml).read() -+ self.fingerprints = yaml.load(cache_fingerprints) -+ -+ def __save_fingerprints(self): -+ """Update deploy cache yaml config holding fingerprints""" -+ with open(self.fingerprints_yaml, 'w') as outfile: -+ outfile.write(yaml.safe_dump(self.fingerprints, -+ default_flow_style=False)) -+ -+ def __fingerprint_mirrors(self, chroot_path): -+ """Collect repo mirror fingerprints""" -+ deb_packages = list() -+ # Scan ISO for deb files (MOS mirror + Ubuntu mirror, no plugins) -+ for repo_dir in ['ubuntu', 'opnfv/nailgun/mirrors/ubuntu']: -+ for _, _, files in os.walk(os.path.join(chroot_path, repo_dir)): -+ for fdeb in files: -+ if fdeb.endswith(".deb"): -+ deb_packages.append(fdeb) -+ sorted_debs = json.dumps(deb_packages, sort_keys=True) -+ self.fingerprints[MIRRORS] = hashlib.sha1(sorted_debs).hexdigest() -+ -+ def __fingerprint_bootstrap(self, chroot_path): -+ """Collect bootstrap image metadata fingerprints""" -+ # FIXME(armband): include 'extra_dirs' contents -+ sorted_data = '' -+ # FIXME(armband): re-include package list (old fuel_bootstrap_cli.yaml) -+ # cli_yaml_path = os.path.join(chroot_path, ISO_BOOTSTRAP_CLI_YAML[1:]) -+ # bootstrap_cli_yaml = open(cli_yaml_path).read() -+ # bootstrap_data = yaml.load(bootstrap_cli_yaml) -+ # sorted_data = json.dumps(bootstrap_data, sort_keys=True) -+ self.fingerprints[BOOTSTRAPS] = hashlib.sha1(sorted_data).hexdigest() -+ -+ def __fingerprint_target(self, dea_file): -+ """Collect target image metadata fingerprints""" -+ # FIXME(armband): include 'image_data', 'codename', 'output' -+ with io.open(dea_file) as stream: -+ dea = yaml.load(stream) -+ editable = dea['settings']['editable'] -+ target_data = {'packages': editable['provision']['packages'], -+ 'repos': editable['repo_setup']['repos']} -+ s_data = json.dumps(target_data, sort_keys=True) -+ self.fingerprints[TARGETIMAGES] = hashlib.sha1(s_data).hexdigest() -+ -+ def do_fingerprints(self, chroot_path, dea_file): -+ """Collect SHA1 fingerprints based on chroot contents, DEA settings""" -+ try: -+ self.__load_fingerprints() -+ self.__fingerprint_mirrors(chroot_path) -+ self.__fingerprint_bootstrap(chroot_path) -+ self.__fingerprint_target(dea_file) -+ self.__save_fingerprints() -+ except Exception as ex: -+ log('Failed to get cache fingerprint: %s' % str(ex)) -+ -+ def __lookup_cache(self, sha): -+ """Search for object in cache based on SHA fingerprint""" -+ cache_sha_dir = os.path.join(self.cache_dir, sha) -+ if not os.path.isdir(cache_sha_dir) or not os.listdir(cache_sha_dir): -+ return None -+ return cache_sha_dir -+ -+ def __inject_cache_dir(self, ssh, sha, artifact): -+ """Stage cached object (dir) in Fuel Master OPNFV local cache""" -+ local_path = self.__lookup_cache(sha) -+ if local_path: -+ remote_path = os.path.join(REMOTE_CACHE_PATH, artifact) -+ with ssh: -+ ssh.exec_cmd('mkdir -p %s' % remote_path) -+ for cachedfile in glob.glob('%s/*' % local_path): -+ ssh.scp_put(cachedfile, remote_path) -+ return local_path -+ -+ def __mix_fingerprints(self, f1, f2): -+ """Compute composite fingerprint""" -+ if self.fingerprints[f1] is None or self.fingerprints[f2] is None: -+ return None -+ return hashlib.sha1('%s%s' % -+ (self.fingerprints[f1], self.fingerprints[f2])).hexdigest() -+ -+ def inject_cache(self, ssh): -+ """Lookup artifacts in cache and inject them over SSH/SCP into Fuel""" -+ try: -+ self.__load_fingerprints() -+ for artifact in [BOOTSTRAPS, TARGETIMAGES]: -+ sha = self.__mix_fingerprints(MIRRORS, artifact) -+ if sha is None: -+ log('Missing fingerprint for: %s' % artifact) -+ continue -+ if not self.__inject_cache_dir(ssh, sha, artifact): -+ log('SHA1 not in cache: %s (%s)' % (str(sha), artifact)) -+ else: -+ log('SHA1 injected: %s (%s)' % (str(sha), artifact)) -+ except Exception as ex: -+ log('Failed to inject cached artifacts into Fuel: %s' % str(ex)) -+ -+ def __extract_bootstraps(self, ssh, cache_sha_dir): -+ """Collect bootstrap artifacts from Fuel over SSH/SCP""" -+ remote_tar = os.path.join(REMOTE_CACHE_PATH, BOOTSTRAP_ARCHIVE) -+ local_tar = os.path.join(cache_sha_dir, BOOTSTRAP_ARCHIVE) -+ with ssh: -+ for k in RSA_KEYPAIR_FILES: -+ ssh.scp_get(os.path.join(RSA_KEYPAIR_PATH, k), -+ local=os.path.join(cache_sha_dir, k)) -+ ssh.exec_cmd('mkdir -p %s && cd %s && tar cf %s *' % -+ (REMOTE_CACHE_PATH, -+ os.path.join(NAILGUN_PATH, NAILGUN_ACT_BOOTSTRAP_SUBDIR), -+ remote_tar)) -+ ssh.scp_get(remote_tar, local=local_tar) -+ ssh.exec_cmd('rm -f %s' % remote_tar) -+ -+ def __extract_targetimages(self, ssh, cache_sha_dir): -+ """Collect target image artifacts from Fuel over SSH/SCP""" -+ cti_path = os.path.join(REMOTE_CACHE_PATH, TARGETIMAGES) -+ with ssh: -+ ssh.scp_get('%s/%s*' % (cti_path, ENVX), local=cache_sha_dir) -+ -+ def collect_artifacts(self, ssh): -+ """Collect artifacts from Fuel over SSH/SCP and add them to cache""" -+ try: -+ self.__load_fingerprints() -+ for artifact, func in { -+ BOOTSTRAPS: self.__extract_bootstraps, -+ TARGETIMAGES: self.__extract_targetimages -+ }.iteritems(): -+ sha = self.__mix_fingerprints(MIRRORS, artifact) -+ if sha is None: -+ log('WARN: Skip caching, NO fingerprint: %s' % artifact) -+ continue -+ local_path = self.__lookup_cache(sha) -+ if local_path: -+ log('SHA1 already in cache: %s (%s)' % (str(sha), artifact)) -+ else: -+ log('New cache SHA1: %s (%s)' % (str(sha), artifact)) -+ cache_sha_dir = os.path.join(self.cache_dir, sha) -+ exec_cmd('mkdir -p %s' % cache_sha_dir) -+ func(ssh, cache_sha_dir) -+ except Exception as ex: -+ log('Failed to extract artifacts from Fuel: %s' % str(ex)) -+ -+ @staticmethod -+ def extract_targetimages_from_env(env_id): -+ """Prepare targetimages from env ID for storage in deploy cache -+ -+ NOTE: This method should be executed locally ON the Fuel Master node. -+ WARN: This method overwrites targetimages cache on Fuel Master node. -+ """ -+ env_n = 'env_%s_' % str(env_id) -+ cti_path = os.path.join(REMOTE_CACHE_PATH, TARGETIMAGES) -+ ti_path = os.path.join(NAILGUN_PATH, NAILGUN_TIMAGES_SUBDIR) -+ try: -+ exec_cmd('rm -rf %s && mkdir -p %s' % (cti_path, cti_path)) -+ for root, _, files in os.walk(ti_path): -+ for tif in files: -+ if tif.startswith(env_n): -+ src = os.path.join(root, tif) -+ dest = os.path.join(cti_path, tif.replace(env_n, ENVX)) -+ if tif.endswith('.yaml'): -+ shutil.copy(src, dest) -+ exec_cmd('sed -i "s|%s|%s|g" %s' % -+ (env_n, ENVX, dest)) -+ else: -+ os.link(src, dest) -+ except Exception as ex: -+ log('Failed to extract targetimages artifacts from env %s: %s' % -+ (str(env_id), str(ex))) -+ -+ @staticmethod -+ def install_targetimages_for_env(env_id): -+ """Install targetimages artifacts for a specific env ID -+ -+ NOTE: This method should be executed locally ON the Fuel Master node. -+ """ -+ env_n = 'env_%s_' % str(env_id) -+ cti_path = os.path.join(REMOTE_CACHE_PATH, TARGETIMAGES) -+ ti_path = os.path.join(NAILGUN_PATH, NAILGUN_TIMAGES_SUBDIR) -+ if not os.path.isdir(cti_path): -+ log('%s cache dir not found: %s' % (TARGETIMAGES, cti_path)) -+ else: -+ try: -+ for root, _, files in os.walk(cti_path): -+ for tif in files: -+ src = os.path.join(root, tif) -+ dest = os.path.join(ti_path, tif.replace(ENVX, env_n)) -+ if tif.endswith('.yaml'): -+ shutil.copy(src, dest) -+ exec_cmd('sed -i "s|%s|%s|g" %s' % -+ (ENVX, env_n, dest)) -+ else: -+ os.link(src, dest) -+ except Exception as ex: -+ log('Failed to install targetimages for env %s: %s' % -+ (str(env_id), str(ex))) -diff --git a/deploy/deploy_env.py b/deploy/deploy_env.py -index d374cce..445070a 100644 ---- a/deploy/deploy_env.py -+++ b/deploy/deploy_env.py -@@ -15,6 +15,7 @@ import glob - import time - import shutil - -+from deploy_cache import DeployCache - from ssh_client import SSHClient - - from common import ( -@@ -35,7 +36,8 @@ class CloudDeploy(object): - - def __init__(self, dea, dha, fuel_ip, fuel_username, fuel_password, - dea_file, fuel_plugins_conf_dir, work_dir, no_health_check, -- deploy_timeout, no_deploy_environment, deploy_log): -+ deploy_cache_dir, deploy_timeout, -+ no_deploy_environment, deploy_log): - self.dea = dea - self.dha = dha - self.fuel_ip = fuel_ip -@@ -49,6 +51,8 @@ class CloudDeploy(object): - self.fuel_plugins_conf_dir = fuel_plugins_conf_dir - self.work_dir = work_dir - self.no_health_check = no_health_check -+ self.deploy_cache = ( DeployCache(deploy_cache_dir) -+ if deploy_cache_dir else None ) - self.deploy_timeout = deploy_timeout - self.no_deploy_environment = no_deploy_environment - self.deploy_log = deploy_log -@@ -82,9 +86,14 @@ class CloudDeploy(object): - self.work_dir, os.path.basename(self.dea_file))) - s.scp_put('%s/common.py' % self.file_dir, self.work_dir) - s.scp_put('%s/dea.py' % self.file_dir, self.work_dir) -+ s.scp_put('%s/deploy_cache.py' % self.file_dir, self.work_dir) - for f in glob.glob('%s/cloud/*' % self.file_dir): - s.scp_put(f, self.work_dir) - -+ def deploy_cache_collect_artifacts(self): -+ if self.deploy_cache: -+ self.deploy_cache.collect_artifacts(self.ssh) -+ - def power_off_nodes(self): - for node_id in self.node_ids: - self.dha.node_power_off(node_id) -@@ -281,4 +290,6 @@ class CloudDeploy(object): - - self.get_put_deploy_log() - -+ self.deploy_cache_collect_artifacts() -+ - return rc -diff --git a/deploy/install_fuel_master.py b/deploy/install_fuel_master.py -index b731c6b..83d31fb 100644 ---- a/deploy/install_fuel_master.py -+++ b/deploy/install_fuel_master.py -@@ -10,6 +10,7 @@ - import time - import os - import glob -+from deploy_cache import DeployCache - from ssh_client import SSHClient - from dha_adapters.libvirt_adapter import LibvirtAdapter - -@@ -32,7 +33,7 @@ class InstallFuelMaster(object): - - def __init__(self, dea_file, dha_file, fuel_ip, fuel_username, - fuel_password, fuel_node_id, iso_file, work_dir, -- fuel_plugins_dir, no_plugins): -+ deploy_cache_dir, fuel_plugins_dir, no_plugins): - self.dea_file = dea_file - self.dha = LibvirtAdapter(dha_file) - self.fuel_ip = fuel_ip -@@ -42,6 +43,8 @@ class InstallFuelMaster(object): - self.iso_file = iso_file - self.iso_dir = os.path.dirname(self.iso_file) - self.work_dir = work_dir -+ self.deploy_cache = ( DeployCache(deploy_cache_dir) -+ if deploy_cache_dir else None ) - self.fuel_plugins_dir = fuel_plugins_dir - self.no_plugins = no_plugins - self.file_dir = os.path.dirname(os.path.realpath(__file__)) -@@ -83,6 +86,10 @@ class InstallFuelMaster(object): - log('Wait until Fuel menu is up') - fuel_menu_pid = self.wait_until_fuel_menu_up() - -+ if self.deploy_cache: -+ log('Deploy cache: Injecting bootstraps and targetimages') -+ self.deploy_cache.inject_cache(self.ssh) -+ - log('Inject our own astute.yaml and fuel_bootstrap_cli.yaml settings') - self.inject_own_astute_and_bootstrap_yaml() - diff --git a/patches/vswitchperf/0001-ci-build-vsperf-make-qemu-system-name-arch-indep.patch b/patches/vswitchperf/0001-ci-build-vsperf-make-qemu-system-name-arch-indep.patch deleted file mode 100644 index 3d3785f1..00000000 --- a/patches/vswitchperf/0001-ci-build-vsperf-make-qemu-system-name-arch-indep.patch +++ /dev/null @@ -1,42 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Tue, 28 Jun 2016 21:27:02 +0200 -Subject: [PATCH] ci/build-vsperf: make qemu-system name arch-indep. - -Instead of hardcoding x86_64 into qemu-system name, use `uname -m`. - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - ci/build-vsperf.sh | 7 ++++--- - 1 file changed, 4 insertions(+), 3 deletions(-) - -diff --git a/ci/build-vsperf.sh b/ci/build-vsperf.sh -index a02de00..fa0d46a 100755 ---- a/ci/build-vsperf.sh -+++ b/ci/build-vsperf.sh -@@ -87,14 +87,15 @@ TEST_REPORT_LOG_DIR="${HOME}/opnfv/$PROJECT/results/$BRANCH" - # parameters: - # none - function terminate_vsperf() { -+ local qemu_system_name="qemu-system-$(uname -m)" - sudo pkill stress &> /dev/null - sudo pkill python3 &> /dev/null -- sudo killall -9 qemu-system-x86_64 &> /dev/null -+ sudo killall -9 "${qemu_system_name}" &> /dev/null - - # sometimes qemu resists to terminate, so wait a bit and kill it again -- if pgrep qemu-system-x86_64 &> /dev/null ; then -+ if pgrep "${qemu_system_name}" &> /dev/null ; then - sleep 5 -- sudo killall -9 qemu-system-x86_64 &> /dev/null -+ sudo killall -9 "${qemu_system_name}" &> /dev/null - sleep 5 - fi - diff --git a/patches/vswitchperf/0002-s-u-build_base_machine-Arch-indep-libdir-s.patch b/patches/vswitchperf/0002-s-u-build_base_machine-Arch-indep-libdir-s.patch deleted file mode 100644 index 68ea8e8d..00000000 --- a/patches/vswitchperf/0002-s-u-build_base_machine-Arch-indep-libdir-s.patch +++ /dev/null @@ -1,38 +0,0 @@ -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -: Copyright (c) 2017 Enea AB and others. -: -: All rights reserved. This program and the accompanying materials -: are made available under the terms of the Apache License, Version 2.0 -: which accompanies this distribution, and is available at -: http://www.apache.org/licenses/LICENSE-2.0 -:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -From: Alexandru Avadanii <Alexandru.Avadanii@enea.com> -Date: Tue, 28 Jun 2016 21:35:44 +0200 -Subject: [PATCH] s/u/build_base_machine: Arch indep libdir(s). - -The same fixes should be applied on arm64, for example. - -Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> ---- - systems/ubuntu/build_base_machine.sh | 5 +++-- - 1 file changed, 3 insertions(+), 2 deletions(-) - -diff --git a/systems/ubuntu/build_base_machine.sh b/systems/ubuntu/build_base_machine.sh -index 1b42a79..b0f3740 100755 ---- a/systems/ubuntu/build_base_machine.sh -+++ b/systems/ubuntu/build_base_machine.sh -@@ -54,11 +54,12 @@ apt-get -y install libnuma-dev - # packages related to VM - - # a few manual fix up on a ubuntu --cd /lib/x86_64-linux-gnu -+local arch=$(uname -m) -+cd /lib/${arch}-linux-gnu - ln -sf libssl.so.1.0.0 libssl.so - ln -sf libcrypto.so.1.0.0 libcrypto.so - --cd /usr/lib/x86_64-linux-gnu -+cd /usr/lib/${arch}-linux-gnu - ln -sf libxml2.so.2 libxml2.so - - diff --git a/upstream/fuel-plugin-opendaylight b/upstream/fuel-plugin-opendaylight deleted file mode 160000 -Subproject 323144519475a0ba6be76ea1696b1643232158e diff --git a/upstream/fuel-plugin-ovs b/upstream/fuel-plugin-ovs deleted file mode 160000 -Subproject 7d90856a02fce1a379dcaf2c32dd9d5f6088c22 diff --git a/upstream/vswitchperf b/upstream/vswitchperf deleted file mode 160000 -Subproject 1356d7c2bccabdd6239d1ae8628f869556ed293 |