summaryrefslogtreecommitdiffstats
path: root/mcp/scripts
AgeCommit message (Collapse)AuthorFilesLines
2018-06-08[state] Stop expecting linux.network failuresAlexandru Avadanii1-1/+1
Now that the `linux.network` state applies cleanly on all nodes, stop expecting failures to prevent false-positive passes. Change-Id: I8a283474e12b85deb95815e4e7a0af4a74d143ac Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-06-07[noha] Update OpenStack version to QueensMichael Polenchuk1-2/+2
* make cluster names version-agnostic * pre-install oslo-templates formula * salt version update to 2017.7 * system reclass update Change-Id: I903895f6bcd98a1062e66e3f14d23ec04dc1b869 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-06-04[submodule] Bump Pharos for ericsson-pod1 updatesAlexandru Avadanii1-0/+0
This bump also brings in: - arm-pod{8,9} config; - UNH lab config; - NTP config in IDFs for existing PODs; - basic vPDF support; Change-Id: Ifc16616bc8a13de2bdf048971448d4853e93880e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-05-17[noha] Bring in gnocchi/panko servicesMichael Polenchuk1-1/+1
JIRA: FUEL-372 Change-Id: I4e322a4a2c84843e9350fe9b3b849cd0c5244a12 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-04-26[jump] Print CPU, RAM, disk info for debugAlexandru Avadanii1-0/+10
JIRA: FUEL-323 Change-Id: I74f92fbdf84762540401fe07a6eb424bdc7aa664 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-04-25[submodule] Bump Pharos for zte-pod1 IDF updateAlexandru Avadanii1-0/+0
Change-Id: I4942a1c3e12db304db11fbc231ca26ff87e3b6bb Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-04-18[reclass] Parameterize tenant VLAN rangesAlexandru Avadanii1-0/+0
Instead of hardcoding the VLAN ranges in reclass, read them from IDF. This is required when VLAN segmentation is used instead of VXLAN, as lab switches should be configured appropiately (e.g. Armband PODs use VLANs 2000+ instead of 1000-1030). Change-Id: I009ccea9b328c40f055a745791eafe3393b96390 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-04-13[submodule] Bump PharosCristina Pauna1-0/+0
Change-Id: Ib5f7a9d7874704b11b10cdc1eefaae19de2a629e Signed-off-by: Cristina Pauna <cristina.pauna@enea.com>
2018-04-05[submodule] Bump Pharos for Armband IDF changesAlexandru Avadanii1-0/+0
Bring in arm-pod5 hugepage configuration et al. Change-Id: If8a2f9cd84b400c5a8a58fe477f8ef7a12c42bf8 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-18lib.sh: Skip modprobing nbd, loop if inbuiltAlexandru Avadanii2-2/+4
While at it, skip waiting for Salt master when deleting UEFI stale entries if it doesn't respond to ping. Also, use https for fetching Armband GPG key to bypass yet another hks issue behind proxies/firewalls that block hks port. Change-Id: I400cbe3257094b62c96b302a3c81c5ffd1ba4755 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-13salt.sh: Copy mcp.rsa to cfg01Alexandru Avadanii2-0/+2
- let rsync transfer mcp.rsa* to cfg01; - symlink mcp.rsa in /root; Change-Id: I18cb70306faf9d166e1110e374a9fe16ac4449de Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-11lib.sh: Add missing `sudo` for `losetup`Alexandru Avadanii1-2/+2
Change-Id: I1c5e3d7a0dbac14bf242730d6ac8d2b1d0817907 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-11[IDF] Move ip-range start higher for mgmt, publicAlexandru Avadanii1-0/+0
- move mgmt start to .10; - move public start to .100; - sync local idf-pod1; JIRA: ARMBAND-365 Change-Id: I5d93c4f486daf280d8bdbe92f5aa7edd8d020db0 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-07[IDF] net_config: Add support for custom ip-rangeAlexandru Avadanii1-0/+0
- submodule: bump Pharos to pick up installer adapter support, as well as IDF updates for Ericsson baremetal pod1; - labs/local/virtual: Bump mgmt, public networks start addresses from .1 to .10, similar to ericsson-pod1; While at it, drop patch now upstream and instead adopt the new param 'neutron:server:root_helper_daemon'. JIRA: FUEL-351 Change-Id: I9bc244a7fd8698861a390ed2b6a27804be46c285 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-07Merge "[novcp] Add deploy argument `-N` (experimental)"Alexandru Avadanii2-1/+2
2018-03-07Merge "[vnode parsing] Move core logic to j2"Alexandru Avadanii5-67/+144
2018-03-07Clean up aodh formula symlinkMichael Polenchuk1-4/+0
Change-Id: I1d84b5d4e3eb0de1c3ebf6fc5a538ee6714d53b2 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-03-07[novcp] Add deploy argument `-N` (experimental)Alexandru Avadanii2-1/+2
Instead of duplicating scenarios for NOVCP, allow it to be specified using a new deploy argument, `-N`. Things are getting convoluted, so instead of creating dedicated '*_pdf.yml.j2' files for each group of similar features, apply the templating in-place and rename all affected files to ".yml.j2". This breaks .gitignore assumption of hiding only "*_pdf.yml" files, so extend (manually) the <mcp/reclass/classes/cluster/.gitignore> with `git ls-files --exclude-standard -o` after an expansion. - ha: move nfv.cpu_pinning to j2, conditioned by 'baremetal'; - ha: add cmp00* vnode definitions (hugepages need more RAM); - labs/local: enable hugepage params for non-dpdk noha; - salt.sh: add route_wrapper to all non-infra VMs; This change extends novcp support to all HA scenarios. Change-Id: I7a80415ac33367ab227ececb4ffb1bc026546d36 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-07[vnode parsing] Move core logic to j2Alexandru Avadanii5-67/+144
j2/python is easier to read and manipulate strings, although it does need some special care about undefined dict keys. With this in place, deploy.sh only contains the higher level logic for the deployment process. - merge arch-specific default configuration files into a singular file with arch name as main dict key of old config (also avoids creating duplicate 'virtual' YAML keys in $LOCAL_PDF); - move template handling to separate <lib_template.sh>; - decouple tight bash ordering of scenario expansion -> parse_yaml -> variable export (e.g. CLUSTER_DOMAIN) -> re-use in cluster j2s; however we can't parse *all* j2s in one go, as scenario j2s might expand to YAMLs needed while expanding cluster j2; - split `do_templates` into separate functions for each stage, with no coupling between them other then call order; Change-Id: I4b5e804094c00e5e918caf769fd85fa52181ad76 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-04[jump req] Move pkg installation to lib.shAlexandru Avadanii1-0/+15
Change-Id: I687b73b256aca78c9d41d4bcd49bfbde51278b51 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-04[deploy.sh] Factor out jump bridge parsing to j2Alexandru Avadanii2-0/+21
Change-Id: Ie4d8e70866d533d195a6e80cdfecbdb00a3027ce Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-03[submodule] Bump Pharos for IDF updatesAlexandru Avadanii1-0/+0
Change-Id: I9e71d9006bad53cc79805ed9255b53b6c5705725 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-03[j2 parsing] Adopt new generate_config featuresAlexandru Avadanii3-16/+14
- bump Pharos git submodule for j2 'do' extension + batch mode; - adopt j2 'do' in our templates; - use int filter for 'native' vlan check; - lib.sh: adopt `-i` to remove `ln` hack for net_map.j2; - lib.sh: adopt `-b` to speedup template parsing; NOTE: Bumping Pharos will also bring in the latest changes in pod_config.yml.j2, which include massive IP shifts and updates. JIRA: FUEL-335 Change-Id: I7d3a997b3d8659d5f09f867870fb3a148c1ec6df Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-02Install missing python module of jsonschemaMichael Polenchuk2-0/+2
Change-Id: Ib903863a7a46008c93ee795083712a50a022a41d Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-02-28lib.sh: Validate pdf/idf with pharos schemaGuillermo Herrero1-1/+15
Run the pharos yaml schema validation for configuration files before expanding them JIRA: FUEL-341 Change-Id: Ia1d69f53265876683a1b6674665a9594ba7dae16 Signed-off-by: Guillermo Herrero <guillermo.herrero@enea.com>
2018-02-27[submodule] Bump Pharos git repo for IDF updatesAlexandru Avadanii1-0/+0
Change-Id: I8d526d51a3e18652c59bad9d7c3c384aa6583566 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-26[patch] cleanup: Drop LVM filter disableAlexandru Avadanii1-0/+0
Now we explicitly add a LVM PV on /dev/sda{1,2} for Cinder storage, so we can safely drop the patch disabling LVM volume filtering. If we later move the PV to a different disk, we can just add the VG and LV definitions to linux:storage via reclass, and Salt will skip setting them up (as they're already created by MaaS), yet keep the filtering sane. While at it, fix 'nova_cpu_pinning' param expr; constructs based on reclass interpolation (e.g. '${_param:x}') do not work when parameters are passed via reclass.storage templating, so change reclass interpolation syntax with classic YAML anchors. Fixes: 672ae12 Change-Id: Ieb41635ddeb630543d7e4d1079f45d636d9a43af Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-25[ovs/dpdk] Parameterize node-specific compute argsCristina Pauna1-0/+0
- node-specific parameters (nova pinning, hugepages, dpdk) should be configurable via IDF, on a per-node basis; - keep default settings for lf-pod2, with and without DPDK, override them for virtual deploys via local-virtual1 IDF; - leave neutron_tenant_* vars hardcoded for now, as they are required on both ctl and cmp nodes - this way we'll deal stricly with cmp params, so we can nicely pass them via config.yml to reclass per-node (and not per-role), allowing mixed computes later; - add compute params for ovs/odl-noha, preparing them for deployment on baremetal later. JIRA: ARMBAND-343 Change-Id: I89a58b9565679ab3882d85f07ae817690ae85c67 Signed-off-by: Cristina Pauna <cristina.pauna@enea.com>
2018-02-25[salt.sh] Add explicit apt-get updateAlexandru Avadanii1-0/+1
Change-Id: I17e66457f90818bad4d746300bb49e9d5179c239 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-22[noHA] Add dedicated cinder volume for virtual cmpAlexandru Avadanii1-0/+11
Replace loop device LVM-backed cinder volume with a dedicated /dev/vdb drive. This is another small step towards bringing noHA to baremetal. Change-Id: I80f9c2bee42e933a36ab7a8f9b4c5247d1652b42 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-22[HA] [cinder] Fix rootfs part name for sda > 2TBAlexandru Avadanii1-0/+0
On systems with more than 2TB on /dev/sda, MaaS automatically creates a 1MB partition with bios_grub label as /dev/sda1. Account for this and expect rootfs to reside on /dev/sda2 (assume all compute nodes have the same behavior wrt this issue). Ideally, later we will switch to explicit partitioning via MaaS, or to a new baremetal provisioner like Ironic. Requires bumping Pharos git submodule to pick up the new j2 filter. Fixes: f6f8a56 JIRA: FUEL-330 Change-Id: I12f5046cf69bf2f52c8b8dc331b90048df643646 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-20[salt.sh] Armband extra repo for forked reclassAlexandru Avadanii1-0/+5
- add Armband nightly/extra DEB repository; - install forked and updated reclass 1.5.2 which includes: * better error reporting; * support for deleting existing keys during list interpolation; * various other improvements and optimizations; While at it, update copyright year for patches. JIRA: FUEL-345 Change-Id: I00d8b625fe191648e7ea34b3dd4c8375691384e6 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-13Merge "[centos jump] Add more missing required packages"Alexandru Avadanii1-0/+2
2018-02-09[centos jump] Add more missing required packagesAlexandru Avadanii1-0/+2
- gcc is required for NBD kernel module build; - gdisk is required for resizing rootfs in nbd-mounted chroot; Change-Id: I8863c126b75ce3a15998c03d2d159f53f5006f5d Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-09Merge "[jump] Add simple check for required Linux bridges"Alexandru Avadanii1-3/+39
2018-02-09salt.sh: Fix one more broken online checkAlexandru Avadanii1-1/+1
Fixes: 5f95f52 Change-Id: I53e08a89b2e873829a78ff75c0f3532329ef87a3 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-07[virsh net] Fix virtual node check for public netAlexandru Avadanii1-4/+5
Jinja variables set inside loops do not live outside current iteration, so use a dictionary.update to work around that. Fixes: 3d2c66e Change-Id: I2bef64dddab080486a71952465edc4e528ae9e17 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-07[jump] Add simple check for required Linux bridgesAlexandru Avadanii1-3/+39
- MaaS requires PXE/admin to be a Linux bridge; - if virtual nodes are present, they should be hooked to a proper Linux bridge for the Public network, but only throw a warning if not (and create a mock public virsh network instead); - if both virtual and baremetal nodes are present, Public bridge is indirectly mandatory (we can't mock it); JIRA: FUEL-339 Change-Id: Idfe99d66c49eadc56cb3d94ca4db3467fb76d388 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[hybrid] Merge config/scenario/{baremetal,virtual}Alexandru Avadanii2-4/+0
Instead of classifying scenarios by underlying machine type, switch to HA/NOHA differentiantion only. This allows us to add support for hybrid scenarios (with some virtual and some baremetal nodes in the same cluster). To facilitate this, we will template the scenario files, which is a small step towards SDF (Scenario Descriptor File) definition and adoption later. JIRA: FUEL-338 Change-Id: If5787991869a3105d82c27ffa0a86ac79b4b08ba Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06Add NOVCP HA OVS scenario (baremetal, virtual)Alexandru Avadanii2-6/+5
Add a new class of scenarios, based on existing baremetal HA scenarios, but instead of having a virtualized control plane (VCP), all Openstack controller services will run directly on the cluster nodes. This change adds the common scaffolding, as well as the OVS scenario. The new scenario(s) can be used on full-baremetal clusters, soon on full-virtual clusters and later on hybrid (virt + bare) clusters. This change defines old (current) style scenario definitions for both baremetal and virtual, both named: - os-nosdn-nofeature-novcp-ha; Prerequisites: 1. Merge-able by name reclass.storage.node definitions Each cluster (e.g. database, telemetry) adds its own set of reclass storage node defitions, which for novcp scenarios should be merged into a single node (kvm) based on the 'name' property. This is not currently supported by upstream reclass 'node.sls' high state, so add support for it via an early patch (required before salt-master-init.sh tries to handle reclass.storage). 2. common reclass classes for novcp Some of the classes in `baremetal-...-common-ha` are not fit for novcp as they define VCP-specific config/inheritance, so add new versions of said classes with novcp in mind or adapt old classes: - parameterize ctl hostname in `openstack_compute.yml`; - new `openstack_control_novcp.yml`; - new `openstack_init_novcp.yml`; 3. Handle hard set names in state files for baremetal nodes Some of our state files (e.g. maas) hardcodes baremetal node names to 'kvm', 'cmp', so we need to align the names in novcp scenario with these values to re-use the maas state. As a future improvement we should parameterize these names in all state files. As a consequence, our baremetal controller nodes will also use 'kvm*' hostnames (instead of 'ctl*'). 4. Add 'noifupdown' to all interfaces on kvm nodes to prevent duplicate IPs/routes created at *any* ifup due to /etc/network/route-br-ex. Patch salt-formula-linux to skip network restart on 'noifupdown', also when routes are present on that interface. JIRA: FUEL-310 Change-Id: Ic67778f63e5ee0334dbfe9547c7109ec1a938d61 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[virtual] Add missing ODL dhcp_intAlexandru Avadanii3-4/+9
- add missing network definitions for ODL node's 1st interface; - add missing comments for `notify` global functions; - fix or silence shellcheck issues; JIRA: FUEL-322 Change-Id: Ie3341d29ab12ddf432db603ad865259afb54714e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06[virsh net] public: rm addr cfg for pure baremetalAlexandru Avadanii1-0/+10
To prevent adding a gratuitous requirement on public being a real Linux bridge on the jumpserver when baremetal nodes are present, stop configuring any IP address information for the public virsh network, allowing us to keep mocking it like we did before L3 information was read from IDF. JIRA: FUEL-339 Change-Id: Id09ecac14825a80a0770c5969afad79b0235e08e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06Merge "[apt-mk] Switch back to nightly"Alexandru Avadanii1-1/+1
2018-02-06[apt-mk] Switch back to nightlyAlexandru Avadanii1-1/+1
Revert "salt: Use apt-mk 'stable' distribution" Revert "reclass: apt_mk_version: stable" This reverts commit d1b6119e288a31e015573363ce77790fec8684df. This reverts commit 4563ea7d62238e8273d840a8d9c6c1e179ca584e. Change-Id: I383db1f78a087045086096cbc674260b985fd913 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-06lib.sh: modprobe br_netfilter, don't bail on errAlexandru Avadanii1-4/+7
Some sysadmins or distro defaults might blacklist br_netfilter, or it might not be loaded at deploy start, account for these corner cases too. JIRA: FUEL-334 Change-Id: I3ca6cb3848df8d2af1625ff4e3816efe8b320886 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[virtual] PDF-based network defs for cluster nodesAlexandru Avadanii1-0/+3
Decouple virtual cluster nodes (ctl, gtw etc.) from opnfv_fn_* vars in favor of parsing PDF/IDF. This is the first step towards unifying baremetal and virtual network definition templates, as well as allowing virtual nodes to run on a remote hypervisor (and eventually with a different arch). opnfv_fn_* vars will still be used for infra VMs spawned on FN (cfg01 and optionally mas01). Adopt new 'net_map.j2' from Pharos submodule for new templates (virt), as well as old ones (baremetal). JIRA: FUEL-322 Change-Id: I150c2416566bbe42ea11cd00f12a8a7bf96776c2 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[virtual] Split 'pxebr' from 'mcpcontrol' netAlexandru Avadanii5-9/+55
- add new virsh managed network 'pxebr' (to mimic baremetal behavior on virtual PODs, this will be the equivalent of PXE/admin network); - connect 'pxebr' to 3rd interface for cfg01, mas01 for all deploys (used to be baremetal-specific), replacing 'internal'; - keep 'mcpcontrol' connected only to 'cfg01' (+ 'mas01' if present) for initial infrastructure bring-up (1st interface); - switch all virtual cluster nodes to 'pxebr' (1st interface); - use 'pxebr' for all Salt cluster nodes traffic, 'mcpcontrol' only for mas01<=>cfg01 Salt traffic; - convert <user-data.template> to jinja2 and expand it based on PDF instead of using `envsubst`; - split <user-data.sh.j2> into two versions, one for each network used for Salt traffic; - ci/deploy.sh: Read scenario data before template parsing for cluster domain variable, needed in virsh network def; - leave docs diagram refresh to later after all possible deploy types have settled; - limit keyserver proxy usage to nodes where the configured http proxy matches the first nameserver (true for all MaaS-provisioned nodes), so we can re-use the same pillar for FN VMs and baremetal nodes; - add PXE/admin IP on cfg01's 3rd interface and switch other vnodes `salt_master_host` to point to it; JIRA: FUEL-322 Change-Id: Ie4f7aedddf2ef81046f1127b377d88dce79f0fda Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[FN VM] Reboot VMs on jump, wait for all onlineAlexandru Avadanii1-10/+17
- apply `linux` state on cfg01 first, so PXE/admin IP is added and FN VM minions are available; - add barrier and wait for all FN VMs to register with cfg01; - use batch-mode execution while applying `linux.network` on FN VMs; - retry all states executed via <salt.sh> on FN VMs; JIRA: FUEL-310 Change-Id: I72e1c565370072500df1d486fe76e6315f583c75 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05[PDF] Switch to generate_config, unify templatesAlexandru Avadanii6-8/+77
- move bash template handling (previously expanded via `envsubst`) to lib.sh; - move j2 template handling to lib.sh; - move virsh network templates to 'mcp/scripts/virsh_net' subdir; - switch virsh network templates from `envsubst` expansion to j2 and leverage generate_config.py, similar to PDF Fuel installer adapter; - add relevant runtime env vars (e.g. SALT_MASTER, MAAS_IP) on the fly to PDF, to consume them in templates like params coming from PDF; - parameterize virsh network definitions based on PDF (mgmt, public); JIRA: FUEL-322 Change-Id: Ib94e78fc4f25797b9354a0552e884104da5d0003 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-05deploy.sh: Move notify() to globals.shAlexandru Avadanii1-0/+28
Extend `notify` to 4 variants: - notify_i = inline (no newline) colored output; - notify = `notify_i` + trailing '\n'; - notify_n = `notify` + extra '\n' before and after; - notify_e = `notify` + stderr output + exit; This allows us to remove '\n' and cleanup the code a bit. While at it, fix some 'NOTE' messages going to stderr instead of stdout. Change-Id: I682e3344ae9e307c4a68ab31c7766bc91b12ee58 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>