summaryrefslogtreecommitdiffstats
path: root/mcp
AgeCommit message (Collapse)AuthorFilesLines
2018-02-19Merge changes from topics 'lab-proxy-keyserver', 'maas-override-failed-testing'Alexandru Avadanii12-7/+40
* changes: [baremetal] Use upstream lab proxy for keyservers [MaaS] Override failed testing by default
2018-02-19Merge "[patch] MaaS: mcp.rsa.pub to auth keys via pillar"Alexandru Avadanii3-46/+2
2018-02-17[baremetal] Use upstream lab proxy for keyserversAlexandru Avadanii8-6/+10
Proxy chainloading does not always work for keyserver requests, so: - mv common.infra.lab_proxy_pdf common.include.lab_proxy_pdf; - mv common.include.proxy common.include.maas_proxy; - include lab_proxy_pdf after each maas_proxy, so if upstream_proxy is defined, it will override the MaaS proxy; JIRA: FUEL-317 Change-Id: I63749be0d350ea73ea2cf6e629710766b14ecb73 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-17[MaaS] Override failed testing by defaultAlexandru Avadanii4-1/+30
Some nodes fail automatic testing done by MaaS during commissioning, although running the testing suites one more time manually works. For now, just override all 'failed testing' nodes unconditionally. JIRA: FUEL-333 Change-Id: I13d3ee3d82550524480aa53aa8752ab90aa940cd Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-17Merge "reclass: maas: Dynamic machine definitions"Alexandru Avadanii2-56/+31
2018-02-16Set nova disk cachemodes to file directsyncMichael Polenchuk4-1/+6
Set nova/libvirt disk cache modes to file directsync to bypass the host page cache & prevent instance's kernel panic due to absent /dev partition. Caching mode directsync effectively turns all guest I/O operations into direct I/O operations on the host, which is the NFS client (virtuals) or GlusterFS (baremetals). Also return back to hwe kernel as a golden mean between GA & Edge ones. Change-Id: I51ab7d0ee71c214ff16b756cfee16e918738b6fd Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-02-15Fix check if node type is baremetal for novcpDelia Popescu1-4/+5
Fixes: 86c8109 Change-Id: I0a947bd7db5dd7cb830cc932f5a0d16831551923 Signed-off-by: Delia Popescu <delia.popescu@enea.com> Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-15Mask opendaylight serviceMichael Polenchuk1-1/+4
In order to avoid using cache data with initial/outdated configuration, mask opendaylight service before package installation. JIRA: FUEL-344 Change-Id: I71eb0b0a5af93d6d21698e76587b32098aba96b4 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-02-13[patch] MaaS: mcp.rsa.pub to auth keys via pillarAlexandru Avadanii3-46/+2
Drop one questionable patch responsible for MaaS node authorized keys to include mcp.rsa.pub by reading the contents of authorized keys on mas01, assuming mcp.rsa.pub will be on the first line. Instead, export the contents of the public key using a shell env var during deploy, which gets expanded via maas_pdf j2 template into a reclass param, leveraging existing salt-formula-maas sshprefs mechanism for delivering the key to MaaS. Since we require the public key to exist before expanding templates, move `generate_ssh_key` call outside the current infrastructure handling block, allowing it to execute during all `deploy.sh` calls, even for dry-runs. Change-Id: I0f53b0f764a2fafd292e0ffd399c284acf61bd30 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-13reclass: maas: Dynamic machine definitionsAlexandru Avadanii2-56/+31
Refactor maas machine definition flow from going through variables in pod_config.yaml to directly handling it in Fuel using a new j2 template. This prepares for future improvements allowing a dynamic compute node count, as well as parameterizing new values via j2. For now, node roles and count are still statically mapped to PDF nodes. JIRA: FUEL-319 Change-Id: I770d82987fcb99792f9d5bc0857ab513f5cd3731 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-13Merge "[centos jump] Add more missing required packages"Alexandru Avadanii1-0/+2
2018-02-09[centos jump] Add more missing required packagesAlexandru Avadanii1-0/+2
- gcc is required for NBD kernel module build; - gdisk is required for resizing rootfs in nbd-mounted chroot; Change-Id: I8863c126b75ce3a15998c03d2d159f53f5006f5d Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-09Merge "[jump] Add simple check for required Linux bridges"Alexandru Avadanii1-3/+39
2018-02-09salt.sh: Fix one more broken online checkAlexandru Avadanii1-1/+1
Fixes: 5f95f52 Change-Id: I53e08a89b2e873829a78ff75c0f3532329ef87a3 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-07[virsh net] Fix virtual node check for public netAlexandru Avadanii1-4/+5
Jinja variables set inside loops do not live outside current iteration, so use a dictionary.update to work around that. Fixes: 3d2c66e Change-Id: I2bef64dddab080486a71952465edc4e528ae9e17 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-07[states] Fix broken online check for bm, vcp nodesAlexandru Avadanii2-5/+5
Previous commit replacing explicit loops with `wait_for` failed to properly escape a nested variable, leading to deploy failure. Also, the logic was flawed, not breaking for offline nodes, rendering the whole barrier check useless. Fixes: 1a0e8e7e Change-Id: I038dbf90fb53c6b61da2e5c9b6867e31d78867af Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-07[jump] Add simple check for required Linux bridgesAlexandru Avadanii1-3/+39
- MaaS requires PXE/admin to be a Linux bridge; - if virtual nodes are present, they should be hooked to a proper Linux bridge for the Public network, but only throw a warning if not (and create a mock public virsh network instead); - if both virtual and baremetal nodes are present, Public bridge is indirectly mandatory (we can't mock it); JIRA: FUEL-339 Change-Id: Idfe99d66c49eadc56cb3d94ca4db3467fb76d388 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-07Merge "[virtual] Rename all to drop virtual prefix"Alexandru Avadanii49-75/+75
2018-02-07Merge "[hybrid] Merge config/scenario/{baremetal,virtual}"Alexandru Avadanii13-30/+28
2018-02-07Merge "[states] maas, vcp: Use `wait_for` in online check"Alexandru Avadanii2-25/+8
2018-02-07Merge "Add NOVCP HA OVS scenario (baremetal, virtual)"Alexandru Avadanii27-11/+807
2018-02-07Switch off broken sphinx stateMichael Polenchuk1-1/+1
Deactivate documentation related optional state until it get fixed in upstream. Change-Id: I5242ed307548c4f37f81d271a1f4f6bee9903f4e Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-02-07Merge changes from topic 'fix-public-bridge'Michael Polenchuk5-4/+24
* changes: [virtual] Add missing ODL dhcp_int [virsh net] public: rm addr cfg for pure baremetal
2018-02-07Merge "[baremetal] cleanup: rm cloud-init iface config"Michael Polenchuk2-6/+0
2018-02-07[baremetal] cleanup: rm cloud-init iface configAlexandru Avadanii2-6/+0
Upstream commit [1] now handles cloud init interface configuration cleanup, so drop our explicit config from reclass. [1] https://github.com/salt-formulas/salt-formula-linux/commit/b333d284 Change-Id: Ifd624f1e884d0d884513a3a7c3e18ffe3f00949a Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[virtual] Rename all to drop virtual prefixAlexandru Avadanii49-75/+75
JIRA: FUEL-322 Change-Id: I99c4b1774a7c3afbc834a6f6e8468f7baf1bc329 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[hybrid] Merge config/scenario/{baremetal,virtual}Alexandru Avadanii13-30/+28
Instead of classifying scenarios by underlying machine type, switch to HA/NOHA differentiantion only. This allows us to add support for hybrid scenarios (with some virtual and some baremetal nodes in the same cluster). To facilitate this, we will template the scenario files, which is a small step towards SDF (Scenario Descriptor File) definition and adoption later. JIRA: FUEL-338 Change-Id: If5787991869a3105d82c27ffa0a86ac79b4b08ba Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[states] maas, vcp: Use `wait_for` in online checkAlexandru Avadanii2-25/+8
Change-Id: I7b583c354843f0116a65b3a31f3be4589087b8a5 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06Add NOVCP HA OVS scenario (baremetal, virtual)Alexandru Avadanii27-11/+807
Add a new class of scenarios, based on existing baremetal HA scenarios, but instead of having a virtualized control plane (VCP), all Openstack controller services will run directly on the cluster nodes. This change adds the common scaffolding, as well as the OVS scenario. The new scenario(s) can be used on full-baremetal clusters, soon on full-virtual clusters and later on hybrid (virt + bare) clusters. This change defines old (current) style scenario definitions for both baremetal and virtual, both named: - os-nosdn-nofeature-novcp-ha; Prerequisites: 1. Merge-able by name reclass.storage.node definitions Each cluster (e.g. database, telemetry) adds its own set of reclass storage node defitions, which for novcp scenarios should be merged into a single node (kvm) based on the 'name' property. This is not currently supported by upstream reclass 'node.sls' high state, so add support for it via an early patch (required before salt-master-init.sh tries to handle reclass.storage). 2. common reclass classes for novcp Some of the classes in `baremetal-...-common-ha` are not fit for novcp as they define VCP-specific config/inheritance, so add new versions of said classes with novcp in mind or adapt old classes: - parameterize ctl hostname in `openstack_compute.yml`; - new `openstack_control_novcp.yml`; - new `openstack_init_novcp.yml`; 3. Handle hard set names in state files for baremetal nodes Some of our state files (e.g. maas) hardcodes baremetal node names to 'kvm', 'cmp', so we need to align the names in novcp scenario with these values to re-use the maas state. As a future improvement we should parameterize these names in all state files. As a consequence, our baremetal controller nodes will also use 'kvm*' hostnames (instead of 'ctl*'). 4. Add 'noifupdown' to all interfaces on kvm nodes to prevent duplicate IPs/routes created at *any* ifup due to /etc/network/route-br-ex. Patch salt-formula-linux to skip network restart on 'noifupdown', also when routes are present on that interface. JIRA: FUEL-310 Change-Id: Ic67778f63e5ee0334dbfe9547c7109ec1a938d61 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[virtual] Add missing ODL dhcp_intAlexandru Avadanii4-4/+14
- add missing network definitions for ODL node's 1st interface; - add missing comments for `notify` global functions; - fix or silence shellcheck issues; JIRA: FUEL-322 Change-Id: Ie3341d29ab12ddf432db603ad865259afb54714e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[virsh net] public: rm addr cfg for pure baremetalAlexandru Avadanii1-0/+10
To prevent adding a gratuitous requirement on public being a real Linux bridge on the jumpserver when baremetal nodes are present, stop configuring any IP address information for the public virsh network, allowing us to keep mocking it like we did before L3 information was read from IDF. JIRA: FUEL-339 Change-Id: Id09ecac14825a80a0770c5969afad79b0235e08e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[HA] Use cluster_public_host for SSL cert fetchAlexandru Avadanii1-5/+7
For VCP-enabled scenarios, `cluster_public_host` and `cluster_vip_address` both point to the public VIP of the cluster. However, for upcoming NOVCP scenarios, `cluster_vip_address` resides inside the management segment, so use `cluster_public_host` instead. JIRA: FUEL-310 Change-Id: I13ef482e2c3116c991dfe91be81d0964f140f8e9 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06Merge "[apt-mk] Switch back to nightly"Alexandru Avadanii3-3/+3
2018-02-06[apt-mk] Switch back to nightlyAlexandru Avadanii3-3/+3
Revert "salt: Use apt-mk 'stable' distribution" Revert "reclass: apt_mk_version: stable" This reverts commit d1b6119e288a31e015573363ce77790fec8684df. This reverts commit 4563ea7d62238e8273d840a8d9c6c1e179ca584e. Change-Id: I383db1f78a087045086096cbc674260b985fd913 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[Horizon] Limit css fixup to Ubuntu packageAlexandru Avadanii1-9/+11
Horizon package from Mirantis mcp-repos does not require the fixup, so limit its application to non-mcp packages. Required for upcoming NOVCP scenarios, where we also have mcp-repos APT source on the proxy nodes. JIRA: FUEL-324 JIRA: FUEL-310 Change-Id: I4399af803c0a17e0aa8f3d7a7330e501a5eedf55 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06lib.sh: modprobe br_netfilter, don't bail on errAlexandru Avadanii1-4/+7
Some sysadmins or distro defaults might blacklist br_netfilter, or it might not be loaded at deploy start, account for these corner cases too. JIRA: FUEL-334 Change-Id: I3ca6cb3848df8d2af1625ff4e3816efe8b320886 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[baremetal] Rename all to drop baremetal prefixAlexandru Avadanii79-238/+238
A few things differ between baremetal and virtual nodes: - provisioning method; - network setup; Since now we support completely dynamic network config based on PDF + IDF, as well as dynamic provisioning of VMs on jumpserver (as virtual cluster nodes), respectively MaaS-driven baremetal provisioning, let's drop the 'baremetal-' prefix from cluster model names and prepare for unified scenarios. Note that some limitations still apply, e.g. virtual nodes are spawned only on jumpserver (localhost) for now. JIRA: FUEL-310 Change-Id: If20077ac37c6f15961468abc58db7e16f2c29260 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[virtual] PDF-based network defs for cluster nodesAlexandru Avadanii15-318/+220
Decouple virtual cluster nodes (ctl, gtw etc.) from opnfv_fn_* vars in favor of parsing PDF/IDF. This is the first step towards unifying baremetal and virtual network definition templates, as well as allowing virtual nodes to run on a remote hypervisor (and eventually with a different arch). opnfv_fn_* vars will still be used for infra VMs spawned on FN (cfg01 and optionally mas01). Adopt new 'net_map.j2' from Pharos submodule for new templates (virt), as well as old ones (baremetal). JIRA: FUEL-322 Change-Id: I150c2416566bbe42ea11cd00f12a8a7bf96776c2 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[virtual] Parameterize cluster model based on PDFAlexandru Avadanii16-80/+58
- 10.1.0.0/24 (internal): * 10.1.0.101 -> opnfv_openstack_compute_node01_tenant_address * 10.1.0.124 -> opnfv_openstack_gateway_node01_tenant_address - 172.16.10.0/24 (mgmt): * 172.16.10.11 -> opnfv_openstack_control_node01_address * 172.16.10.100 -> opnfv_infra_config_address * 172.16.10.101 -> opnfv_openstack_compute_node01_control_address * 172.16.10.111 -> opnfv_opendaylight_server_node01_single_address * 172.16.10.124 -> opnfv_openstack_gateway_node01_address - 10.16.0.0/24 (public): * 10.16.0.11 -> opnfv_openstack_control_node01_external_address * 10.16.0.101 -> opnfv_openstack_compute_node01_external_address * 10.16.0.124 -> opnfv_openstack_gateway_node01_external_address To re-use DPDK config baremetal template, move: - cluster.baremetal-mcp-pike-ovs-dpdk-ha.infra.config_pdf + cluster.all-mcp-arch-common.infra.config_dpdk_pdf Drop unused 'ceilometer_graphite_publisher_host' (172.16.10.107). JIRA: FUEL-322 Change-Id: I3aef3415bd696a7ae5b566af12af4733a50c2135 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[virtual] Change IP addrs to align with baremetalAlexandru Avadanii7-24/+24
To be able to re-use pod_config.yaml parameters generated based on PDF for both baremetal and virtual scenarios without forking it, we first need to align the IP addresses used in virtual deployments. Currently hard set values will be parameterized in an ulterior change. - 10.1.0.0/24 (internal): * 105 -> 101 (cmp01); 106 -> 102 (cmp02); * 110 -> 124 (gtw01); - 172.16.10.0/24 (mgmt): * 101 -> 11 (ctl01); * 105 -> 101 (cmp01); 106 -> 102 (cmp02); * 110 -> 124 (gtw01); - 10.16.0.0/24 (public): * 101 -> 11 (ctl01); * 105 -> 101 (cmp01); 106 -> 102 (cmp02); * 110 -> 124 (gtw01); JIRA: FUEL-322 Change-Id: I5d5def4e92c3462f1a34f73dde65ef7a262a5d62 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[virtual] Split 'pxebr' from 'mcpcontrol' netAlexandru Avadanii8-16/+66
- add new virsh managed network 'pxebr' (to mimic baremetal behavior on virtual PODs, this will be the equivalent of PXE/admin network); - connect 'pxebr' to 3rd interface for cfg01, mas01 for all deploys (used to be baremetal-specific), replacing 'internal'; - keep 'mcpcontrol' connected only to 'cfg01' (+ 'mas01' if present) for initial infrastructure bring-up (1st interface); - switch all virtual cluster nodes to 'pxebr' (1st interface); - use 'pxebr' for all Salt cluster nodes traffic, 'mcpcontrol' only for mas01<=>cfg01 Salt traffic; - convert <user-data.template> to jinja2 and expand it based on PDF instead of using `envsubst`; - split <user-data.sh.j2> into two versions, one for each network used for Salt traffic; - ci/deploy.sh: Read scenario data before template parsing for cluster domain variable, needed in virsh network def; - leave docs diagram refresh to later after all possible deploy types have settled; - limit keyserver proxy usage to nodes where the configured http proxy matches the first nameserver (true for all MaaS-provisioned nodes), so we can re-use the same pillar for FN VMs and baremetal nodes; - add PXE/admin IP on cfg01's 3rd interface and switch other vnodes `salt_master_host` to point to it; JIRA: FUEL-322 Change-Id: Ie4f7aedddf2ef81046f1127b377d88dce79f0fda Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[FN VM] Reboot VMs on jump, wait for all onlineAlexandru Avadanii5-14/+35
- apply `linux` state on cfg01 first, so PXE/admin IP is added and FN VM minions are available; - add barrier and wait for all FN VMs to register with cfg01; - use batch-mode execution while applying `linux.network` on FN VMs; - retry all states executed via <salt.sh> on FN VMs; JIRA: FUEL-310 Change-Id: I72e1c565370072500df1d486fe76e6315f583c75 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[PDF] Switch to generate_config, unify templatesAlexandru Avadanii9-39/+83
- move bash template handling (previously expanded via `envsubst`) to lib.sh; - move j2 template handling to lib.sh; - move virsh network templates to 'mcp/scripts/virsh_net' subdir; - switch virsh network templates from `envsubst` expansion to j2 and leverage generate_config.py, similar to PDF Fuel installer adapter; - add relevant runtime env vars (e.g. SALT_MASTER, MAAS_IP) on the fly to PDF, to consume them in templates like params coming from PDF; - parameterize virsh network definitions based on PDF (mgmt, public); JIRA: FUEL-322 Change-Id: Ib94e78fc4f25797b9354a0552e884104da5d0003 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05deploy.sh: Move notify() to globals.shAlexandru Avadanii1-0/+28
Extend `notify` to 4 variants: - notify_i = inline (no newline) colored output; - notify = `notify_i` + trailing '\n'; - notify_n = `notify` + extra '\n' before and after; - notify_e = `notify` + stderr output + exit; This allows us to remove '\n' and cleanup the code a bit. While at it, fix some 'NOTE' messages going to stderr instead of stdout. Change-Id: I682e3344ae9e307c4a68ab31c7766bc91b12ee58 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[vPDF] Use local-virtual1, unify pkg requirementsAlexandru Avadanii2-16/+16
Until PDF/IDF land in Pharos for all our virtual PODs, use a common vPDF we already provide as an example to mimic the old hardcoded behavior while leveraging PDF/IDF parameterization. As a consequence, python requirements previously only needed for baremetal should now also be installed for virtual deploys too. JIRA: FUEL-322 Change-Id: Ied1c907275285a9086450a15491ae516a0db1be2 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[vPDF] Add experimental vPOD lab configAlexandru Avadanii2-0/+206
JIRA: FUEL-322 Change-Id: I1482badbbbf66b4855faf6daf486520fc71e09b0 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[baremetal] Retire example pod_config.yamlAlexandru Avadanii1-111/+0
It is easier to just generate the `pod_config.yaml` file than to maintain it, so let's remove it. While at it, link sample PDF/IDF inside pharos git submodule, so we don't have to pass a different lab-config URI to use the sample. To generate pod_config.yml for the sample PDF/IDF: $ ./ci/deploy.sh -l local -p pod1 -s os-odl-nofeature-ha -d $ cat mcp/deploy/images/pod_config.yml JIRA: FUEL-322 Change-Id: If5898f92ef54bebc31d57f9632959e9093a89250 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[PDF] pod1: Refresh PDF, IDF examplesAlexandru Avadanii2-65/+66
Sync latest changes from pharos git repo for our sample PDF/IDF: - move net_config from PDF to IDF; - minor cleanup; JIRA: FUEL-322 Change-Id: If6865ac61a4942a1dd5daf7081fd8faa67e0e7bf Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-03Merge "[baremetal] Fix: wrong nic name idf index"Alexandru Avadanii2-2/+2
2018-02-02[baremetal] Fix: wrong nic name idf indexGuillermo Herrero2-2/+2
This was only affecting pod deployments with different board models, under the current limited support: - 3 KVMs will be same model and have the same NIC names - 2 Compute nodes will be the same model and have same NIC names For the computes nodes, br-mesh NIC name was wrong due to incorrect idf mapping Change-Id: I9685b35cb23b03be9fc0e6fe16c0712a9ad70e19 Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>