aboutsummaryrefslogtreecommitdiffstats
path: root/mcp
AgeCommit message (Collapse)AuthorFilesLines
2018-02-01Merge "[patch] Group APT calls for salt formula install" into stable/euphratesAlexandru Avadanii2-2/+71
2018-01-31[patch] Group APT calls for salt formula installAlexandru Avadanii2-2/+71
When installing salt-formula-* on cfg01, we used to call APT for each package. Instead, handle them all at once. While at it, stop using colored output on terminals that don't support it (e.g. 'vt220' used by OPNFV Jenkins). Change-Id: Ib8f2cee9638c43cdf648487bf05b07cd49802d3e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit e1325bf5c526f063e7e16326ad6bfdd1c44d5fec)
2018-01-31[VCP] Catch 'no response' when adding ssh auth keyAlexandru Avadanii1-1/+1
On rare occassions, one or more minions might fail to respond in due time, so catch 'no reponse' using `wait_for`. Change-Id: I8e6b0dc44a39e79c2874ff9a657e152620ba3f13 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit b254aef5a34c7fc96db5c0a330c757fe9d51be76)
2018-01-29[cfg01] apt.conf.d: Dpkg::Use-Pty: 0Alexandru Avadanii2-0/+9
This will silence all 'Reading database ...' noise during Salt formula installation. Change-Id: I734d727194e276443db1e1581f40ec494562196e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 80212433352a59be69f7c82bce8c297bca13eeaf)
2018-01-29[ovs/dpdk] Force up the float-to-ex link onlyMichael Polenchuk1-1/+1
Force up br-ex to br-floating link instead of all networking restart to avoid issues with existing interface routes. Change-Id: I6b8204db6767e1fde964eb1913f885ecb06d0c28 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-01-26Merge "[baremetal] Fix cmp proto dhcp on PXE/admin" into stable/euphratesAlexandru Avadanii2-0/+8
2018-01-26lib.sh: Reduce useless wget dot reportingAlexandru Avadanii1-1/+1
Downloading the base image (usually a few hundred MB) outputs a lot of useless dots to show progress. Switch to 1M per dot (from 1K). Change-Id: I8c525cad0b46e8ba3a7f6da4dd7f8277a49df91f Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 056544990dcdab373c5a12ca1521012e80c404ad)
2018-01-25[baremetal] Fix cmp proto dhcp on PXE/adminAlexandru Avadanii2-0/+8
Although we properly filter the PXE/admin interface in the common openstack_compute_pdf.yml.j2 template and use DHCP instead of manual setup, we failed to do the same in scenario-specific overrides (ODL, OVS), so we end up with 'proto: manual' on PXE/admin on cmp nodes. The fix is trivial and reuses the mechanism in the common class in scenario-specific templates (if interface is PXE/admin, use 'DHCP' instead of 'manual'). This solves the issue of broken connectivity to Salt master after cmp reboot. Change-Id: I1953d03343190acb2efcab4412a3d37e130b0ea9 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 19913423a73f5913f7aa12a0737b2d2d36639efe)
2018-01-24Rectify ODL repository apt keyMichael Polenchuk2-16/+24
Change-Id: Iaa917be9f8f86c328ce4d503923a0d7cca680434 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com> (cherry picked from commit d081166c1cd7698416a7e5cd2ea871b038de35b8)
2018-01-22Merge "reclass: apt_mk_version: 2017.12" into stable/euphratesAlexandru Avadanii3-3/+3
2018-01-22[baremetal] Add upstream (lab) proxy supportAlexandru Avadanii3-0/+26
If upstream proxy is defined in IDF, propagate it to pillar data: - linux:system:proxy:keyserver:http(s) for cfg01, mas01; - maas:region:upstream_proxy for mas01; Sample IDF config: idf: fuel: network: upstream_proxy: address: 10.0.2.2 port: 3128 JIRA: FUEL-317 Change-Id: I12be815e1b4564227fb09c20ce06cd71e7d433b6 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-22[patch] system.repo: Add keyserver proxy supportAlexandru Avadanii6-8/+122
Instead of defining a http proxy for all salt-minion traffic, which also includes some Openstack API accesses we can't filter (no_proxy is not yet supported), add & leverage support for proxy configuration during APT keyserver access / key download. JIRA: FUEL-331 Change-Id: I9470807633596c610cfafb141b139ddda2ff096b Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 6ab6935577900e598ca60aaed14d2e73f7b1633f)
2018-01-21reclass: apt_mk_version: 2017.12Alexandru Avadanii3-3/+3
Although previous commit e3bc5ad changed the first reference of apt-mk repos to '2017.12' from 'nightly', it missed the cluster model. This fixes redeploys with `-f`, which fail due to conflicts between already installed '2017.12' packages and 'nightly' ones. Fixes: e3bc5ad Change-Id: I854bac86feaaa61da0b68d158e270eec1ee0ccb7 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-19Retry cinder volume stateMichael Polenchuk2-2/+5
The service of cinder-volume restarts too quickly after package installation with default/incorrect configuration and goes over restart threshold, so systemd stops attempt to restart any further causing state faulure. To fix it properly the RestartSec (i.e. restart delay) param should be added into cinder-volume.service unit. Change-Id: Ic8591e8ef52a3d439122f276d275e56bd2442ce6 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com> (cherry picked from commit 1ea49591080442d8db86fff60031d3dc41142274)
2018-01-12[baremetal] Disable dhcp offered routesMichael Polenchuk1-0/+5
Prevent dhcp client from setting an unwanted default routes on compute nodes. Conflicts: mcp/reclass/classes/system [stable/euphrates cherry-pick] Drop reclass system submodule bump, only applicable to master. Change-Id: I2529491bbc977647e5f457d5f1ba88b0cc4372ee Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com> (cherry picked from commit 658418ea84e633f5f97a706a075d7e2f24127999)
2018-01-09Merge "[cleanup] Drop cfg01 node def for ODL L2" into stable/euphratesAlexandru Avadanii1-18/+0
2018-01-08Merge "[centos jump] Add missing dependency for nbd build" into stable/euphratesAlexandru Avadanii1-0/+1
2018-01-09[cleanup] Drop cfg01 node def for ODL L2Alexandru Avadanii1-18/+0
Remove obsolete ODL L2 top node definition, which currently breaks `reclass-salt --top`. Change-Id: Ic05f87dd7edea5b9f38e996bb02a3e1062caf98a Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-08[centos jump] Add missing dependency for nbd buildAlexandru Avadanii1-0/+1
Change-Id: Icb23d2e6d3bb6e49b54e2d51cc8a35cb03702fcd Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 60e790a986e64a47f4242766500f1aee6a3083ed)
2018-01-08lib.sh: base image: Prevent using incomplete imgAlexandru Avadanii1-4/+4
In case the previous deploy attempt already copied the base image as the VCP image in order to perform offline operations and failed, leaving an incomplete image in place, current code might try to use it instead of building it from scratch. Use the hash-agnostic link names as checkpoints for successful image handling. Change-Id: I1e99e515e18ba1dec534c520811c127b2b528afe Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 5878c5c0c2ca8c78e9a5124452eefe8290ab63cd)
2018-01-08lib.sh: Add delay after `kpartx -av`Alexandru Avadanii1-0/+1
On rare occassions, mapper bindings created by kpartx take longer to show up, leading to errors when we try to mount them on. Bring back the hardcoded delay to bypass such issues. Change-Id: Ib386c04fc55cd85235a2156dba08fda378e4cdfd Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 6b0d21a70be04666fd60a9d4b05d4ad13e192026)
2018-01-08lib.sh: Fix `modprobe -f` on UbuntuAlexandru Avadanii1-3/+4
For some reason, `modprobe -f` for a clean nbd module (from vanilla Ubuntu) fails with exec format error randomly, while a simple `modprobe` works. Change-Id: I79785e510cab757e2482baf442054be984c24019 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 510a5fe446dea8513060813794c99e856eff1845)
2018-01-08lib.sh: Fix stop condition in wait_forAlexandru Avadanii1-5/+5
Change-Id: Ida693b6dd328db283d6992ac33500f4dd1a73eb8 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 717cb4e0f8d9e430eb31ba643f95682f1592e509)
2018-01-08salt.sh: Fix linux state apply on cfg01Alexandru Avadanii1-4/+4
cfg01 does not repond or is not connected while trying to apply linux state via salt.sh, use wait_for macro to account for this. JIRA: ARMBAND-315 Change-Id: I2d4c63d43f806b65f9ce327f4a00a6334be88750 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry-picked from commit 2bbab1873ef629e89ac92ec195feb15b86cf8295)
2018-01-08lib.sh: Extend wait_for function to catch no respAlexandru Avadanii5-16/+19
wait_for function should be able to also check for minions that did not return or not respond, in addition to the return code. To keep it backwards compatible, condition the new check on the max attempt number being specified in decimal format (e.g. '10.0' unlike old '10'). Change-Id: If2512cf9121cdd795638efe7362ef0485d4e8d91 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 3f559299c232bbb7639d02243c95d6256cdf94d4)
2018-01-08MaaS: Set commission/deploy timeouts via reclassAlexandru Avadanii3-2/+10
Allow end-users to easily change the MaaS commissioning/deploying timeouts by simply editing the reclass model. While at it, use arch-specific values and bump deploy timeout on AArch64 to 20 minutes instead of 15. Change-Id: I37ae434ecebdd64effb007baa06c722b1db15c66 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit ea6ecbb6fb546e27fcc73e699e08c6cad31d2bd0)
2018-01-08x86_64: Revert base, VCP kernel to 4.4/4.8Alexandru Avadanii1-3/+2
Commit 7bfb0ac switched both base and VCP images to using Ubuntu kernel hwe-16.04-edge (4.13.x). Due to testing suites compatibility issues, switch back to: - default Xenial kernel (4.4.x) for base image; - hwe-16.04 (4.8.x) for VCP image (similar to Mirantis image); Fixes: 7bfb0ac Change-Id: I458c42124efa6eda765774fe05cab402d6111f82 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-07user-data.template: Remove salt-minion installAlexandru Avadanii2-12/+1
salt-minion is now pre-provisioned inside the image using qemu-nbd. Revert "lib.sh: Limit envsubst to certain variables" This reverts commit 3a76d07dbd409b781abdb8520f55a1b20edf07db. Change-Id: Icceb8bcf439e28ab01c7731c3602c1113290454d Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit fac41836e91dd56696c3a435bfdbc7adf53899e2)
2018-01-07lib.sh: Cache base image(s) between deploysAlexandru Avadanii1-13/+36
Fingerprint and re-use base image artifacts. Change-Id: Ic7a73c04e27d25addd50e4e9880619a0028956d3 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 1815bdaf0c1765cc1fcb48031f8291409861b0b2)
2018-01-07deploy.sh: Print sysinfoAlexandru Avadanii3-0/+20
JIRA: FUEL-323 Change-Id: I0dcbcfbedc7e9ef013ca50dcc08f804323f91701 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 3aa172fb1659925a837b432649793f9f1f46cca9)
2018-01-06Revert "[baremetal] Retire mas01 NAT"Alexandru Avadanii2-0/+38
Although deploy works now without direct internet access on the cluster nodes, testing suites seem to require it. This reverts commit ed209426e895c7c323d253afd6276bb74df64da0. Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> Change-Id: I35489e18fdd6a4ee6a270e42a3542e5a370bf819
2018-01-05[baremetal] ODL: Move ODL VCP VM to kvm02Alexandru Avadanii1-1/+1
Balance VM distribution on the 3 kvm nodes, as kvm02 has 4 VCP VMs while kvm{01,03} have 5 VCP VMs each (without ODL). Instead of spawning the ODL VCP VM on kvm03, move it to kvm02. Change-Id: Id03b9453ee7c15cd6785c0bc073a38b87034aede Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commmit 62135fd2367ced881bf99dc157fdb4536633d4a0)
2018-01-05[baremetal] Fix Pike reference, should be OcataAlexandru Avadanii2-2/+2
Change-Id: Ifb5216818f67f859d55ccf253eb84101cac0dab4 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-04[centos jump] nbd build: Fix REQ_TYPE_SPECIALAlexandru Avadanii1-1/+5
Change-Id: Ia514418d2aae1b4f7e752d4610fa6c9829c67e51 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commmit 84f58189f269be6633e0c8897d5624d3dfaaa650)
2018-01-04[centos jump] Add nbd build supportAlexandru Avadanii2-2/+74
RHEL distros do not maintain nbd, so add a best-effort function to build it on the fly. Change-Id: Ie0419f0fed8a0b12f6b878b3093d6ca34f72d140 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commmit 924d481ac84e8fad9bf1132e1cff18967887017e)
2018-01-04[baremetal] Switch VCP base image to UCAAlexandru Avadanii4-1/+17
Since Mirantis prebuilt image comes with salt-minion 2016.3 instead of 2016.11 and upgrading it leads to a hard to break catch-22, use the Ubuntu cloud archive image we already download for FN VMs and pre-install: - a newer kernel (hwe-edge); - salt-minion (2016.11); This also implicitly aligns the image handling on AArch64 and x86_64. Change-Id: I86d1c777449d37bdd0348936a598e3ffe9d265af Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit f37103b4d60f057f653e0e5e8b7c6c3023a95b2d)
2018-01-04deploy.sh: Factor out pkg deps into yaml filesAlexandru Avadanii2-0/+70
Instead of hardcoding the packages to install on the jumphost at deploy time in `ci/deploy.sh`, move the requirements list to a dedicated yaml file, so it is decoupled from actual code. While at it, add missing deps: - ipxe-qemu (AArch64 Ubuntu only); Change-Id: I9ee5c0796b70f9d29427d608b6415333c8841e1e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit b57cd65121f61114432b07c04106fc4b8609440e)
2018-01-04base image: Explicitly call kpartxAlexandru Avadanii1-3/+7
qemu-nbd currently available in CentOS 7 does not add partition mappings automatically for NBD devices, so add explicit `kpartx` calls. Change-Id: Ifa79c89b82024602b782c449dbf4de10899e03b5 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 5c890d60fff57b624358f4d4be6865862f4f8415)
2018-01-04AArch64: base image: pre-install salt-minionAlexandru Avadanii3-2/+10
While at it, rename apt repo in foundation node user-data template from "salt" to "saltstack", to align with reclass model naming. Change-Id: I5b216492349ae187b568884b1ab4046c52b1c6b2 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit feb877785544a31a18ae2f0ee2900a82e7ac4337)
2018-01-04Add pre-{install,purge} support for base imageAlexandru Avadanii4-6/+163
Extend <lib.sh> and its invocation from <ci.deploy.sh> with support for modifying foundation node VMs base image prior to using it with: - additional APT GPG keys; - additional APT repos; - packages to pre-install; - packages to pre-remove; - (non-configurable) cloud init datasource via NoCloud only, so VCP VMs won't wait for metadata service; While at it, re-use the resulting image as a base for another round of pre-patching (same operations as above are supported) to provide a base image for VCP VMs. Add AArch64-specific configuration based on new mechanisms: - pre-install linux-image-generic-hwe-16.04-edge (and headers) for foundation node and VCP (common) image (also requires new repo and its key); - pre-install cloud-init for VCP image (it should already be installed, but script needs non-empty config for VCP to create the VCP image and transfer it over to Salt Master); NOTE: cloud-init is required on VCP VMs for DHCP on 1st iface. JIRA: FUEL-309 Change-Id: I7dcaf0ffd9c57009133c6d339496ec831ab14375 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 6ead32f737f39e3d82e1484a9d31f07ed2daeadb)
2018-01-03[baremetal] Move include.proxy yml to proper dirAlexandru Avadanii1-0/+0
Change-Id: Id46b454b224dacdbaf675155c949d0998f0d0036 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-03[baremetal] curtin: Bump salt-minion to 2016.11Alexandru Avadanii1-0/+2
By default, MaaS formula will install Salt minion 2016.3 via curtin on physical nodes. 2016.3 does not properly support proxy_host config option, causing timeouts during `linux.system.repo` SLS apply. Change-Id: I3d6245f0d4b425170c43b3b62a21ad9acc6cb97e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 6e0e518d43cecfb8f981dbd51326d8eae103720c)
2018-01-03[baremetal] Retire mas01 NATAlexandru Avadanii2-38/+0
Isolate networks by retiring NAT on mas01; also cutting direct internet access from cluster nodes that are not facing the public network (prx, cmp). NOTE: Since we are removing mas01 NAT, VCP VMs (except prx which have public IPs) and kvm nodes (cmp also have public IPs) will no longer have direct internet connectivity. Cluster deployment and operations will work without it, but if it is required for different reasons, the MaaS proxy could be enabled by uncommenting the /etc/enviroment section in: - cluster.baremetal-mcp-pike-common-ha.include.proxy.yml JIRA: FUEL-317 Change-Id: I5ed8b420296b27df34a54ec1ebd7b7cf58041425 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 9a6e655e0b851ff6e449027c01ac1a66188b0064)
2018-01-03[patch] Fix OVS ifup workaroundAlexandru Avadanii1-5/+12
Do not assume routes are on the same OVS port as the one currently being configured. Instead, apply the `unless` ifup condition for any OVS port. Change-Id: Iea8084f9e50401d300feb7ed16f90b430680cac5 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit b1849f630a8de3dfce2e91fc375d9e901b90843b)
2018-01-03[baremetal] heat: Switch metadata API URL to mgmtAlexandru Avadanii1-0/+14
Prepare for decoupling management from public (drop mas01 NAT): - ctl: change heat URLs to use new management VIP instead of public; Change-Id: I8e220ee37bd4177c3afd58a9ee401f815d046706 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit d39aaa2b7312c91e6a1851f357cf1d087619cbda)
2018-01-03[baremetal] prx: Add management network VIPAlexandru Avadanii4-3/+38
Include `openstack_web_public_vip` class for setting up the old VIP in the public network, use old class for mgmt VIP. Also change the generic hostname 'prx' to point inside mgmt net. Change-Id: Iff69394f16ede290d149a26b054a85371f00f8e0 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 29c64b73f749364867be78a474410fe062eba99e)
2018-01-03[baremetal] MaaS: Enable HTTP proxyAlexandru Avadanii10-3/+51
Instead of using NAT on the mas01 node for all cluster node outgoing traffic, use the MaaS built-in proxy for APT traffic to leverage its caching capabilities too. Also enable the proxy for salt minions, so they can access public keyservers et al. Cleanup public DNS from kvm nodes, interferes with MaaS proxy. Add example config for global env proxy, but don't enable it: - default environment settings - /etc/environment (via reclass); The MaaS proxy will not be used (at least for now) on nodes: - cfg01; - mas01; NOTE: We can't yet drop the maas.pxe_nat state completely, as certain Openstack services are still accessed via public addresses from ctl nodes. JIRA: FUEL-317 JIRA: FUEL-318 Change-Id: I6c5f6872bb94afb838580571080e808bc262fc68 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 90c0b369c01a2185fe86651f8ad9e0a172d6941d)
2018-01-03[baremetal] cmp: Add missing public gw (default)Alexandru Avadanii3-0/+38
When we dropped the default gw via mas01 NAT, we uncovered a bug, compute nodes do not have the proper public gw set up and used to reach public network via mas01, slowing everything down. Add gw similar to prx nodes. Fixes: d4ab072 Change-Id: I4343c31c376a7a223670cdd623366454396d8d92 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit e9d597db3a9e8a38577908f36f5f700e875d4d78)
2018-01-03[maas] artifact sync: improve barrier conditionAlexandru Avadanii1-3/+2
Simplify wait condition for MaaS service up, since it's fragile and often adds extra time when not really needed. Instead, retry starting boot image import right away. Change-Id: I131d6c82127449cecf6685d4cc7484a366e658c6 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit fedeebd6ee0e3a342777812b7ea90bd4988787ad)
2017-12-31[vcp] Catch 'no response' of salt minion as wellMichael Polenchuk2-6/+1
Salt minion could return 'no response' and cause an unconfigured state of the vcp node(s), so catch this output after linux state as well. Also clean up excess route on proxy nodes. Change-Id: I3183fa09ff41a8f027ee789869bdae0c3962ab8f Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com> (cherry picked from commmit a183db4b3404bd12073b5691eb5d4fbd8135b44b)