aboutsummaryrefslogtreecommitdiffstats
path: root/mcp
AgeCommit message (Collapse)AuthorFilesLines
2018-02-07[states] Fix broken online check for bm, vcp nodesAlexandru Avadanii2-5/+5
Previous commit replacing explicit loops with `wait_for` failed to properly escape a nested variable, leading to deploy failure. Also, the logic was flawed, not breaking for offline nodes, rendering the whole barrier check useless. Fixes: 1a0e8e7e Change-Id: I038dbf90fb53c6b61da2e5c9b6867e31d78867af Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-07[jump] Add simple check for required Linux bridgesAlexandru Avadanii1-3/+39
- MaaS requires PXE/admin to be a Linux bridge; - if virtual nodes are present, they should be hooked to a proper Linux bridge for the Public network, but only throw a warning if not (and create a mock public virsh network instead); - if both virtual and baremetal nodes are present, Public bridge is indirectly mandatory (we can't mock it); JIRA: FUEL-339 Change-Id: Idfe99d66c49eadc56cb3d94ca4db3467fb76d388 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-07Merge "[virtual] Rename all to drop virtual prefix"Alexandru Avadanii49-75/+75
2018-02-07Merge "[hybrid] Merge config/scenario/{baremetal,virtual}"Alexandru Avadanii13-30/+28
2018-02-07Merge "[states] maas, vcp: Use `wait_for` in online check"Alexandru Avadanii2-25/+8
2018-02-07Merge "Add NOVCP HA OVS scenario (baremetal, virtual)"Alexandru Avadanii27-11/+807
2018-02-07Switch off broken sphinx stateMichael Polenchuk1-1/+1
Deactivate documentation related optional state until it get fixed in upstream. Change-Id: I5242ed307548c4f37f81d271a1f4f6bee9903f4e Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-02-07Merge changes from topic 'fix-public-bridge'Michael Polenchuk5-4/+24
* changes: [virtual] Add missing ODL dhcp_int [virsh net] public: rm addr cfg for pure baremetal
2018-02-07Merge "[baremetal] cleanup: rm cloud-init iface config"Michael Polenchuk2-6/+0
2018-02-07[baremetal] cleanup: rm cloud-init iface configAlexandru Avadanii2-6/+0
Upstream commit [1] now handles cloud init interface configuration cleanup, so drop our explicit config from reclass. [1] https://github.com/salt-formulas/salt-formula-linux/commit/b333d284 Change-Id: Ifd624f1e884d0d884513a3a7c3e18ffe3f00949a Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[virtual] Rename all to drop virtual prefixAlexandru Avadanii49-75/+75
JIRA: FUEL-322 Change-Id: I99c4b1774a7c3afbc834a6f6e8468f7baf1bc329 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[hybrid] Merge config/scenario/{baremetal,virtual}Alexandru Avadanii13-30/+28
Instead of classifying scenarios by underlying machine type, switch to HA/NOHA differentiantion only. This allows us to add support for hybrid scenarios (with some virtual and some baremetal nodes in the same cluster). To facilitate this, we will template the scenario files, which is a small step towards SDF (Scenario Descriptor File) definition and adoption later. JIRA: FUEL-338 Change-Id: If5787991869a3105d82c27ffa0a86ac79b4b08ba Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[states] maas, vcp: Use `wait_for` in online checkAlexandru Avadanii2-25/+8
Change-Id: I7b583c354843f0116a65b3a31f3be4589087b8a5 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06Add NOVCP HA OVS scenario (baremetal, virtual)Alexandru Avadanii27-11/+807
Add a new class of scenarios, based on existing baremetal HA scenarios, but instead of having a virtualized control plane (VCP), all Openstack controller services will run directly on the cluster nodes. This change adds the common scaffolding, as well as the OVS scenario. The new scenario(s) can be used on full-baremetal clusters, soon on full-virtual clusters and later on hybrid (virt + bare) clusters. This change defines old (current) style scenario definitions for both baremetal and virtual, both named: - os-nosdn-nofeature-novcp-ha; Prerequisites: 1. Merge-able by name reclass.storage.node definitions Each cluster (e.g. database, telemetry) adds its own set of reclass storage node defitions, which for novcp scenarios should be merged into a single node (kvm) based on the 'name' property. This is not currently supported by upstream reclass 'node.sls' high state, so add support for it via an early patch (required before salt-master-init.sh tries to handle reclass.storage). 2. common reclass classes for novcp Some of the classes in `baremetal-...-common-ha` are not fit for novcp as they define VCP-specific config/inheritance, so add new versions of said classes with novcp in mind or adapt old classes: - parameterize ctl hostname in `openstack_compute.yml`; - new `openstack_control_novcp.yml`; - new `openstack_init_novcp.yml`; 3. Handle hard set names in state files for baremetal nodes Some of our state files (e.g. maas) hardcodes baremetal node names to 'kvm', 'cmp', so we need to align the names in novcp scenario with these values to re-use the maas state. As a future improvement we should parameterize these names in all state files. As a consequence, our baremetal controller nodes will also use 'kvm*' hostnames (instead of 'ctl*'). 4. Add 'noifupdown' to all interfaces on kvm nodes to prevent duplicate IPs/routes created at *any* ifup due to /etc/network/route-br-ex. Patch salt-formula-linux to skip network restart on 'noifupdown', also when routes are present on that interface. JIRA: FUEL-310 Change-Id: Ic67778f63e5ee0334dbfe9547c7109ec1a938d61 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[virtual] Add missing ODL dhcp_intAlexandru Avadanii4-4/+14
- add missing network definitions for ODL node's 1st interface; - add missing comments for `notify` global functions; - fix or silence shellcheck issues; JIRA: FUEL-322 Change-Id: Ie3341d29ab12ddf432db603ad865259afb54714e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[virsh net] public: rm addr cfg for pure baremetalAlexandru Avadanii1-0/+10
To prevent adding a gratuitous requirement on public being a real Linux bridge on the jumpserver when baremetal nodes are present, stop configuring any IP address information for the public virsh network, allowing us to keep mocking it like we did before L3 information was read from IDF. JIRA: FUEL-339 Change-Id: Id09ecac14825a80a0770c5969afad79b0235e08e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[HA] Use cluster_public_host for SSL cert fetchAlexandru Avadanii1-5/+7
For VCP-enabled scenarios, `cluster_public_host` and `cluster_vip_address` both point to the public VIP of the cluster. However, for upcoming NOVCP scenarios, `cluster_vip_address` resides inside the management segment, so use `cluster_public_host` instead. JIRA: FUEL-310 Change-Id: I13ef482e2c3116c991dfe91be81d0964f140f8e9 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06Merge "[apt-mk] Switch back to nightly"Alexandru Avadanii3-3/+3
2018-02-06[apt-mk] Switch back to nightlyAlexandru Avadanii3-3/+3
Revert "salt: Use apt-mk 'stable' distribution" Revert "reclass: apt_mk_version: stable" This reverts commit d1b6119e288a31e015573363ce77790fec8684df. This reverts commit 4563ea7d62238e8273d840a8d9c6c1e179ca584e. Change-Id: I383db1f78a087045086096cbc674260b985fd913 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[Horizon] Limit css fixup to Ubuntu packageAlexandru Avadanii1-9/+11
Horizon package from Mirantis mcp-repos does not require the fixup, so limit its application to non-mcp packages. Required for upcoming NOVCP scenarios, where we also have mcp-repos APT source on the proxy nodes. JIRA: FUEL-324 JIRA: FUEL-310 Change-Id: I4399af803c0a17e0aa8f3d7a7330e501a5eedf55 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06lib.sh: modprobe br_netfilter, don't bail on errAlexandru Avadanii1-4/+7
Some sysadmins or distro defaults might blacklist br_netfilter, or it might not be loaded at deploy start, account for these corner cases too. JIRA: FUEL-334 Change-Id: I3ca6cb3848df8d2af1625ff4e3816efe8b320886 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[baremetal] Rename all to drop baremetal prefixAlexandru Avadanii79-238/+238
A few things differ between baremetal and virtual nodes: - provisioning method; - network setup; Since now we support completely dynamic network config based on PDF + IDF, as well as dynamic provisioning of VMs on jumpserver (as virtual cluster nodes), respectively MaaS-driven baremetal provisioning, let's drop the 'baremetal-' prefix from cluster model names and prepare for unified scenarios. Note that some limitations still apply, e.g. virtual nodes are spawned only on jumpserver (localhost) for now. JIRA: FUEL-310 Change-Id: If20077ac37c6f15961468abc58db7e16f2c29260 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[virtual] PDF-based network defs for cluster nodesAlexandru Avadanii15-318/+220
Decouple virtual cluster nodes (ctl, gtw etc.) from opnfv_fn_* vars in favor of parsing PDF/IDF. This is the first step towards unifying baremetal and virtual network definition templates, as well as allowing virtual nodes to run on a remote hypervisor (and eventually with a different arch). opnfv_fn_* vars will still be used for infra VMs spawned on FN (cfg01 and optionally mas01). Adopt new 'net_map.j2' from Pharos submodule for new templates (virt), as well as old ones (baremetal). JIRA: FUEL-322 Change-Id: I150c2416566bbe42ea11cd00f12a8a7bf96776c2 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[virtual] Parameterize cluster model based on PDFAlexandru Avadanii16-80/+58
- 10.1.0.0/24 (internal): * 10.1.0.101 -> opnfv_openstack_compute_node01_tenant_address * 10.1.0.124 -> opnfv_openstack_gateway_node01_tenant_address - 172.16.10.0/24 (mgmt): * 172.16.10.11 -> opnfv_openstack_control_node01_address * 172.16.10.100 -> opnfv_infra_config_address * 172.16.10.101 -> opnfv_openstack_compute_node01_control_address * 172.16.10.111 -> opnfv_opendaylight_server_node01_single_address * 172.16.10.124 -> opnfv_openstack_gateway_node01_address - 10.16.0.0/24 (public): * 10.16.0.11 -> opnfv_openstack_control_node01_external_address * 10.16.0.101 -> opnfv_openstack_compute_node01_external_address * 10.16.0.124 -> opnfv_openstack_gateway_node01_external_address To re-use DPDK config baremetal template, move: - cluster.baremetal-mcp-pike-ovs-dpdk-ha.infra.config_pdf + cluster.all-mcp-arch-common.infra.config_dpdk_pdf Drop unused 'ceilometer_graphite_publisher_host' (172.16.10.107). JIRA: FUEL-322 Change-Id: I3aef3415bd696a7ae5b566af12af4733a50c2135 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[virtual] Change IP addrs to align with baremetalAlexandru Avadanii7-24/+24
To be able to re-use pod_config.yaml parameters generated based on PDF for both baremetal and virtual scenarios without forking it, we first need to align the IP addresses used in virtual deployments. Currently hard set values will be parameterized in an ulterior change. - 10.1.0.0/24 (internal): * 105 -> 101 (cmp01); 106 -> 102 (cmp02); * 110 -> 124 (gtw01); - 172.16.10.0/24 (mgmt): * 101 -> 11 (ctl01); * 105 -> 101 (cmp01); 106 -> 102 (cmp02); * 110 -> 124 (gtw01); - 10.16.0.0/24 (public): * 101 -> 11 (ctl01); * 105 -> 101 (cmp01); 106 -> 102 (cmp02); * 110 -> 124 (gtw01); JIRA: FUEL-322 Change-Id: I5d5def4e92c3462f1a34f73dde65ef7a262a5d62 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[virtual] Split 'pxebr' from 'mcpcontrol' netAlexandru Avadanii8-16/+66
- add new virsh managed network 'pxebr' (to mimic baremetal behavior on virtual PODs, this will be the equivalent of PXE/admin network); - connect 'pxebr' to 3rd interface for cfg01, mas01 for all deploys (used to be baremetal-specific), replacing 'internal'; - keep 'mcpcontrol' connected only to 'cfg01' (+ 'mas01' if present) for initial infrastructure bring-up (1st interface); - switch all virtual cluster nodes to 'pxebr' (1st interface); - use 'pxebr' for all Salt cluster nodes traffic, 'mcpcontrol' only for mas01<=>cfg01 Salt traffic; - convert <user-data.template> to jinja2 and expand it based on PDF instead of using `envsubst`; - split <user-data.sh.j2> into two versions, one for each network used for Salt traffic; - ci/deploy.sh: Read scenario data before template parsing for cluster domain variable, needed in virsh network def; - leave docs diagram refresh to later after all possible deploy types have settled; - limit keyserver proxy usage to nodes where the configured http proxy matches the first nameserver (true for all MaaS-provisioned nodes), so we can re-use the same pillar for FN VMs and baremetal nodes; - add PXE/admin IP on cfg01's 3rd interface and switch other vnodes `salt_master_host` to point to it; JIRA: FUEL-322 Change-Id: Ie4f7aedddf2ef81046f1127b377d88dce79f0fda Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[FN VM] Reboot VMs on jump, wait for all onlineAlexandru Avadanii5-14/+35
- apply `linux` state on cfg01 first, so PXE/admin IP is added and FN VM minions are available; - add barrier and wait for all FN VMs to register with cfg01; - use batch-mode execution while applying `linux.network` on FN VMs; - retry all states executed via <salt.sh> on FN VMs; JIRA: FUEL-310 Change-Id: I72e1c565370072500df1d486fe76e6315f583c75 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[PDF] Switch to generate_config, unify templatesAlexandru Avadanii9-39/+83
- move bash template handling (previously expanded via `envsubst`) to lib.sh; - move j2 template handling to lib.sh; - move virsh network templates to 'mcp/scripts/virsh_net' subdir; - switch virsh network templates from `envsubst` expansion to j2 and leverage generate_config.py, similar to PDF Fuel installer adapter; - add relevant runtime env vars (e.g. SALT_MASTER, MAAS_IP) on the fly to PDF, to consume them in templates like params coming from PDF; - parameterize virsh network definitions based on PDF (mgmt, public); JIRA: FUEL-322 Change-Id: Ib94e78fc4f25797b9354a0552e884104da5d0003 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05deploy.sh: Move notify() to globals.shAlexandru Avadanii1-0/+28
Extend `notify` to 4 variants: - notify_i = inline (no newline) colored output; - notify = `notify_i` + trailing '\n'; - notify_n = `notify` + extra '\n' before and after; - notify_e = `notify` + stderr output + exit; This allows us to remove '\n' and cleanup the code a bit. While at it, fix some 'NOTE' messages going to stderr instead of stdout. Change-Id: I682e3344ae9e307c4a68ab31c7766bc91b12ee58 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[vPDF] Use local-virtual1, unify pkg requirementsAlexandru Avadanii2-16/+16
Until PDF/IDF land in Pharos for all our virtual PODs, use a common vPDF we already provide as an example to mimic the old hardcoded behavior while leveraging PDF/IDF parameterization. As a consequence, python requirements previously only needed for baremetal should now also be installed for virtual deploys too. JIRA: FUEL-322 Change-Id: Ied1c907275285a9086450a15491ae516a0db1be2 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[vPDF] Add experimental vPOD lab configAlexandru Avadanii2-0/+206
JIRA: FUEL-322 Change-Id: I1482badbbbf66b4855faf6daf486520fc71e09b0 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[baremetal] Retire example pod_config.yamlAlexandru Avadanii1-111/+0
It is easier to just generate the `pod_config.yaml` file than to maintain it, so let's remove it. While at it, link sample PDF/IDF inside pharos git submodule, so we don't have to pass a different lab-config URI to use the sample. To generate pod_config.yml for the sample PDF/IDF: $ ./ci/deploy.sh -l local -p pod1 -s os-odl-nofeature-ha -d $ cat mcp/deploy/images/pod_config.yml JIRA: FUEL-322 Change-Id: If5898f92ef54bebc31d57f9632959e9093a89250 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[PDF] pod1: Refresh PDF, IDF examplesAlexandru Avadanii2-65/+66
Sync latest changes from pharos git repo for our sample PDF/IDF: - move net_config from PDF to IDF; - minor cleanup; JIRA: FUEL-322 Change-Id: If6865ac61a4942a1dd5daf7081fd8faa67e0e7bf Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-03Merge "[baremetal] Fix: wrong nic name idf index"Alexandru Avadanii2-2/+2
2018-02-02[baremetal] Fix: wrong nic name idf indexGuillermo Herrero2-2/+2
This was only affecting pod deployments with different board models, under the current limited support: - 3 KVMs will be same model and have the same NIC names - 2 Compute nodes will be the same model and have same NIC names For the computes nodes, br-mesh NIC name was wrong due to incorrect idf mapping Change-Id: I9685b35cb23b03be9fc0e6fe16c0712a9ad70e19 Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>
2018-02-02Merge "deploy.sh: Disable net.bridge.bridge-nf-call"Michael Polenchuk1-0/+9
2018-02-02Revert "[FN VMs] remove graphics"Alexandru Avadanii13-35/+55
RHEL family virtualization tools reserve 02:00 PCI slot for VGA, even if 'nographics' is specified when creating the VM (in case the user wants to later hook a video card, which usually *requires* PCI slot2). Debian systems do not follow this rule (tested with libvirt 1.x, 2.x, 3.x), hence 1st NIC lands on PCI slot 2 (and get eth name 'ens2'). To align the behavior across all possible jumpserver distros, bring back the virtio video. This reverts commit 738f6c3b68d1179de1ff790f9e72c25f10874da4. Change-Id: Ifd855c12e04aec1ff0ab047b13f8081365741889 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-01deploy.sh: Disable net.bridge.bridge-nf-callAlexandru Avadanii1-0/+9
JIRA: FUEL-334 Change-Id: I6d2499053dcfb7f99593fcd5c948b569bdcb9c9b Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-01[baremetal] Allow MaaS timeouts to be set via IDFAlexandru Avadanii3-10/+2
Bump Pharos git submodule to pick up support for MaaS timeout parameterization, as well as new IDF for lf-pod2. Drop arch-specific MaaS timeouts, as they are now configurable on a per-POD basis. Sample usage (via IDF): idf: fuel: maas: # MaaS timeouts (in minutes) timeout_comissioning: 10 timeout_deploying: 15 Change-Id: I8fafa336b0bc64d705f6c2e40fc3dfb85672fb15 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-31Turn off Retpoline and KPTI protectionMichael Polenchuk6-28/+79
Based on Canonical research (https://goo.gl/QJykMa) there is low-risk of attack for private clouds environments, therefore turn off the related kernel patches & regain performance back. Change-Id: I661fa127241e327b07d21a29d58d584997607123 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-01-31Merge "[patch] Group APT calls for salt formula install"Michael Polenchuk2-2/+71
2018-01-31Merge "[VCP] Catch 'no response' when adding ssh auth key"Michael Polenchuk1-1/+1
2018-01-31[VCP] Catch 'no response' when adding ssh auth keyAlexandru Avadanii1-1/+1
On rare occassions, one or more minions might fail to respond in due time, so catch 'no reponse' using `wait_for`. Change-Id: I8e6b0dc44a39e79c2874ff9a657e152620ba3f13 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-30[patch] Group APT calls for salt formula installAlexandru Avadanii2-2/+71
When installing salt-formula-* on cfg01, we used to call APT for each package. Instead, handle them all at once. While at it, stop using colored output on terminals that don't support it (e.g. 'vt220' used by OPNFV Jenkins). Change-Id: Ib8f2cee9638c43cdf648487bf05b07cd49802d3e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-30Update reclass system modelMichael Polenchuk2-4/+5
Get the latest reclass system model to be in sync with salt formulas code. Adjust keystone v3 admin endpoint patch. Change-Id: Ia7ce6187b875cc9e18af2784ef1eb5df1c145c7d Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-01-29[patch] Point neutron opendaylight configs to pikeMichael Polenchuk1-12/+12
Change-Id: I3fae38da88b170b84ea926abf93da8b178e9ca8a Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-01-29Merge "[FN VMs] remove graphics"Alexandru Avadanii13-55/+35
2018-01-29Merge "[cfg01] apt.conf.d: Dpkg::Use-Pty: 0"Michael Polenchuk2-0/+9
2018-01-29[FN VMs] remove graphicsAlexandru Avadanii13-55/+35
Since VCP VMs (created via salt formula) do not have a video controller defined in their domain XMLs, network devices end on different PCI slots and hence have different names assigned (ens2+ vs foundation node VMs, which start with ens3). To align network interface names for VMs on jumpserver vs kvm nodes, and reduce confusion, remove the video controller from FN VMs. This allows some cleanup: - drop extra AArch64 args from virt-install; - unify 'opnfv_vcp_vm_*' and 'opnfv_fn_vm_*' variables; Change-Id: I0d108b00914b3eaaa03b67c652174f8ed4573118 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-01-28[cfg01] apt.conf.d: Dpkg::Use-Pty: 0Alexandru Avadanii2-0/+9
This will silence all 'Reading database ...' noise during Salt formula installation. Change-Id: I734d727194e276443db1e1581f40ec494562196e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>