aboutsummaryrefslogtreecommitdiffstats
path: root/mcp/scripts
AgeCommit message (Collapse)AuthorFilesLines
2018-08-31[docker] Use distro docker-compose if not brokenAlexandru Avadanii1-9/+14
While at it, use explicit `docker-compose pull` call to maintain backwards compatibility with older docker-compose (1.8.0 on aarch64). JIRA: FUEL-383 Change-Id: I0f6ac4fb59c0a286ac99a1a0dc4484d00f3837fe Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-29[salt-formulas] Add & enable armband formulaGuillermo Herrero1-1/+1
AArch64 specific formula, mostly tweaking nova conf / installing virtualization layer prerequisites: - install qemu-efi; - install vgabios; - fix missing link for vgabios binary blob; - nova conf: cpu_model=cortex-a57 (only for virtual deploys); - nova conf: virt_type=qemu (only for virtual deploys); - nova compute conf: virt_type=qemu (only for virtual deploys); - nova conf: pointer_model=ps2mouse since AArch64 has no USB tablet; [1] https://github.com/openstack/nova/commit/f0f0953 Change-Id: I40515bdbd941850b103a86d51b347cc8610f5741 Signed-off-by: Guillermo Herrero <Guillermo.Herrero@enea.com> Signed-off-by: Charalampos Kominos <Charalampos.Kominos@enea.com> Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-29[docker] Switch to containerized Salt MasterAlexandru Avadanii3-138/+41
* Refactor OPNFV salt-formulas mechanism to resemble upstream git structure: - git submodules: add new submodule for each formula we patch; - create salt-formula-x directories for OPNFV formulas; - move mcp/metadata/service contents to their each formula subdir; - use `make patches-import` for patches previously handled by patch.sh; - retire patch.sh * states: add virtual_init: - mostly based on old salt.sh, which is now obsolete; - exclude salt-master service restart (it would kill the container); * scenarios: cleanup (rm cfg01 virtual node def), adopt virtual_init; * reclass: align our model with prebuilt container's Salt config: - drop linux:network pillar data (handled by Docker); - stop applying linux.system state on cfg01; - align salt user homedir; - drop salt-formula packages (preprovisioned); * minor plumbing in deploy.sh and lib.sh; JIRA: FUEL-383 Change-Id: I28708a9b399d3f19012212c71966ebda9d6fc0ac Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-29[docker] Add docker-compose definitionsAlexandru Avadanii4-1/+146
Add configuration items for bringing up the Salt master Docker container using docker-compose: - mount <storage dir>/hosts as /etc/hosts - mount <storage dir>/salt as /etc/salt: * semi-persistent generated Salt information; - mount <storage dir>/nodes as /srv/salt/reclass/nodes: * semi-persistent generated reclass node data storage; - mount Fuel@OPNFV git repo in /root/fuel; - mount all other previously rsynced artifacts; - hook container to 3 networks/bridges: mcpcontrol, PXE/admin and management: * container IP addresses are managed by Docker, so no DHCP is used; - override upstream entrypoint.sh by mounting our own version on top; * run-time patch Salt to fix certain init system interactions due to 'Tini' looking like Upstart but behaving a bit differently; * work around directory traversal issues in Salt/reclass by using extensive `cp` (to be later mitigated at container build time); * overwrite reclass.system with our own version (patched), later to fetched pre-patched during container build; * overwrite salt-formulas with our patched versions; * create missing links for salt-formulas-* packages (e.g. gnocchi) which are not available as git repos upstream; - add mcp.rsa.pub to 'ubuntu' user's authorized_keys for SSH login to master node to remain backwards compatible; - mount configuration files for semi-persistent state preservation at container destroy/rebuild (using '-f' deploy argument); JIRA: FUEL-383 Change-Id: Ie17e578a7ebeb071b1c0e0a49ad58ffa6bbc89bb Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-29[lib.sh] Add veth pair handling supportAlexandru Avadanii1-2/+10
Upcoming docker integration requires special care for network interfaces Docker will hook to, so virsh networks can be mixed with Docker networks. We'll use veth pair(s) to hook together the Docker network(s) to their counterpart virsh-managed Linux bridge(s). JIRA: FUEL-383 Change-Id: I1a714dc8be73e79539d85bacac311d1ed05cc18d Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-29[jump vnet] Workaround MTU set race conditionAlexandru Avadanii1-2/+4
Older libvirt (1.x, 2.x, even older 3.x) try to enforce their own MTU (1500), overriding it back to 1500 after the udev rule finished. Delay link modification with 1 second and extend its patten to also include libvirt's own tap interfaces (<network name>-nic). Change-Id: I3050c11de7f376df02cbf203e9106b61c560683f Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-27[jump vnet] Fix udev rule event for older libvirtAlexandru Avadanii1-1/+1
Older libvirt seems to raise 'change' events instead of 'add', so the MTU bump of the libvirt managed tap interfaces was not properly applied. Change-Id: Ie1a4f1c4f235ad4f83bf9ee218a40dcedc1ce914 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-21[deploy.sh] Install Docker if not presentAlexandru Avadanii1-0/+11
JIRA: FUEL-383 Change-Id: I19d27ca59a3f24d1bd66e39457a6ca267bccce19 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-21[jump req] Add build/deploy specific requirementsAlexandru Avadanii3-3/+14
Add support for different prerequisites depending on the current operation (docker build or cluster deploy). Leverage the new support to pre-install upcoming deps: - python-pip (build); - docker-compose (deploy); JIRA: FUEL-383 Change-Id: Ic3e6062b1943e3584f0b1f80d2e33b8812defced Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-13[jump] Enable jumbo frames for vnet devicesAlexandru Avadanii1-0/+7
By default, vnet devices have a MTU of 1500 on the host side, causing issue with larger packets traversing the bridges between guest VMs when guest VMs have jumbo frames enabled. JIRA: FUEL-336 JIRA: FUEL-367 JIRA: FUEL-382 [1] http://linuxaleph.blogspot.com/2013/01/ how-to-network-jumbo-frames-to-kvm-guest.html [2] https://packetpushers.net/udev/ Change-Id: I941ac9cf764e3b3fa2d6463be5363b5459775f29 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-09[IDF] infra VMs: Allow trunking mgmt networkAlexandru Avadanii1-0/+0
Due to design limitations in idf.net_config, for a hybrid POD we need to trunk certain network segments for the VMs running on the jumpserver, including mgmt network going to cfg01, mas01. Add mgmt VLAN support for cfg01, mas01 gated by a flag in IDF: idf.fuel.jumphost.trunks.mgmt: True JIRA: FUEL-338 Change-Id: I6903f9b70e5f8a88618bce28e21c7c0631a05065 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-09Update Salt version to 2017.7Michael Polenchuk1-1/+1
Salt 2016.x has a bug with states ordering coming from 'include' statement. Glance/Heat DB sync is applied before package setup although clearly specified 'require' in state. Change-Id: Ic5f7ce4a7623fb208c0a5ba366802b7e02fa8b9f Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-08-07[states] baremetal_init: Parameterize kvm hostsAlexandru Avadanii1-0/+1
Instead of hardcoding the 'kvm' hostnames, use new targeting mechanism based on scenario-specific node names, preparing for baremetal noha scenario integration. JIRA: FUEL-382 Change-Id: If336aa1ac130749e4df7bffaf27a55513dd4f267 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-07[scenarios] Differentiate virtual node rolesAlexandru Avadanii2-6/+36
Split scenario yaml definitions for virtual.nodes based on node role ('infra', 'control' or 'compute'), to be leveraged later to contruct node lists based on said role. This moves the responsability of filtering node names in scenario files (based on 'virtual' or 'baremetal' type) to xdf_data.sh.j2, simplifying scenario templates. By keeping all nodes (both virtual and baremetal) in scenario files, we can later determine the role (and implicitly the hostname) for a MaaS-managed node based on its index in the virtual.nodes.control structure. JIRA: FUEL-382 Change-Id: I1f83a307631f4166ee1c57ef598c44876b962f97 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-03[lib.sh] Reset virtual nodes after MaaS installAlexandru Avadanii1-0/+17
For hybrid PODs (e.g. x86_64 jumpserver + control nodes, aarch64 baremetal compute nodes), the virtual nodes rely on MaaS DHCP to be up when the OS boots, so issue a `virsh reset` accordingly. Instead of checking for online nodes using `test.ping`, use `saltutil.sync_all` to also sync Salt state modules to the virtual nodes (usually handled by baremetal_init state in HA deploys). JIRA: FUEL-338 Change-Id: If689d057dc4438102c3a7428a97b9638e21bfdc5 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-03[lib.sh] Support older jumpserver libvirtAlexandru Avadanii1-1/+2
`virsh undefine` argument `--nvram` is only supported by newer versions of libvirt. Although this is mandatory for AArch64, for x86_64 this is not a blocker (since we don't enable OVMF for the VMs on the jumpserver). Change-Id: I3a82bc54b36228980a41d77a463a7558a685c03d Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-07-26Revert "[salt master] Disable broken _orchestration.conf"Michael Polenchuk1-1/+1
This reverts commit 8ac1c0fa491ab5de2eaf128374f9c73202df1751. Upstream commit has been reverted, Change-Id: I3b20edd450034dc79ef15b46c1ff845f81aaf981 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-07-26[salt master] Disable broken _orchestration.confAlexandru Avadanii1-1/+1
The new orchestration configuration file includes multiple fragments from all installed formulas; breaking due to an unguarded dependency include in keystone formula meta/salt.yml. Temporarily disable the new config generation until fixed upstream. Change-Id: I8476e5b60b531f08f779df2826b23cd42a043d9b Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-07-24Merge "[salt-formulas] opnfv: Drop obsolete route_wrapper"Alexandru Avadanii1-1/+0
2018-07-16[salt-formulas] opnfv: Drop obsolete route_wrapperAlexandru Avadanii1-1/+0
Change-Id: I84a4789ff2155d7c14f9ffd9bfe54c5bca7a0d4f Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-07-12[salt.sh] Ignoring files using rsyncJunaid Ali1-1/+7
Currently rsync is not ignoring '.gitignore' file due to the specification of absolute path. This patch fixes the issue and also ignores the temporary directory if that is created inside the cloned fuel repo PatchSet2: Shell check PatchSet3: Minor improvements PatchSet4: Rebasing the commit Change-Id: I05fc39236af1bc6f8eabb668304ba5a32dec0737 Signed-off-by: Junaid Ali <junaidali.yahya@gmail.com>
2018-06-29Revert "[salt-formulas] Switch to 'testing' dist"Alexandru Avadanii1-1/+1
This reverts commit a8f434118013a69295bdc0f9fc8294c539742c01. This reverts commit b52ef6ea62f43de4b91e2becd92a3282e569925c. Change-Id: I604df3f50febb049f1545a8141c5abb013cf6c69 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-06-29sysinfo_print: Dump all iptables rulesAlexandru Avadanii1-0/+1
Change-Id: I0ce6d21e14efe6e5236759c8d8b3439f09e89675 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-06-28[salt-formulas] Switch to 'testing' distAlexandru Avadanii1-1/+1
Nightly repo dist is currently broken. Change-Id: I9235c2c95b4edebf56d20e33932237f18a8e63d5 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-06-28[submodule] Bump Pharos for arm-virtual2 ip-rangeAlexandru Avadanii1-0/+0
Change-Id: I77859a2670e07668913232d482853d77cfc8b149 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-06-21[submodule] Bump Pharos for unh-pod1 updatesAlexandru Avadanii1-0/+0
Change-Id: Ic6705c64dd3582b562e96ff79fd7fc83789533a6 Signed-off-by: Paul Vaduva <Paul.Vaduva@enea.com> Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-06-19Enforce static configuration instead of DHCPAlexandru Avadanii8-38/+14
- noha: 'accept_policy: open_mode' to align with ha scenarios; - s/cmp01/cmp001/g to align all scenarios and allow code reuse; - rename network params: s/dhcp/mcpcontrol/g, cleanup; - computes XDF data: drop 'opnfv_*' layer of params, cleanup; - local vPDF: add comments with default roles by node index; - parameterize all netmasks; - drop unused address/netmask for 'proto: manual' interfaces; - virsh_net: cleanup definitions, remove hardcodes, align IP on jumpserver and DHCP range with MaaS for pxebr; - maas: parameterize hardcoded '/24' cidr for PXE/admin, refactor maas.region.machines parameterization; - merge <all-mcp-arch-common/infra/config_*pdf.yaml.j2> templates; - move reclass.storage definitions of compute nodes to common dir; - drop 'openstack_compute_*' reclass params in favor of expanding them via j2 directly in reclass.storage params; - adopt `nm.cluster.has_*_nodes` where possible; - obsolete `runtime.yml` from reclass model; - refactor arch-specific reclass param selection; - remove unused defaults in favor of mandatory IDF properties; - noha: prepare for baremetal node support in cinder_lvm_devices; - interfaces: add interface_mtu and 'noifupdown: true' everywhere; - interfaces: use j2 macros to generate eth/vlan config; - states cleanup: remove DHCP route disable workaround on prx/cmp; - allow configuring NTP servers via: `idf.fuel.network.ntp_strata_host{1,2}`; - ovs_bridge: Allow setting gateway, dns-nameservers - apache: Adjust module list for novcp class inheritance; - glusterfs PPA: pin with same prio of MCP repos for novcp scenario; JIRA: FUEL-319 JIRA: FUEL-326 JIRA: FUEL-337 Change-Id: Ia6ad64ba8cade85a75fb22c9a2505decc3834360 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-06-14Get back to salt 2016.11Michael Polenchuk1-1/+1
Change-Id: Id024ed22dd1760f41ae18aeb8e680c2f07a5dc63 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-06-08[state] Stop expecting linux.network failuresAlexandru Avadanii1-1/+1
Now that the `linux.network` state applies cleanly on all nodes, stop expecting failures to prevent false-positive passes. Change-Id: I8a283474e12b85deb95815e4e7a0af4a74d143ac Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-06-07[noha] Update OpenStack version to QueensMichael Polenchuk1-2/+2
* make cluster names version-agnostic * pre-install oslo-templates formula * salt version update to 2017.7 * system reclass update Change-Id: I903895f6bcd98a1062e66e3f14d23ec04dc1b869 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-06-04[submodule] Bump Pharos for ericsson-pod1 updatesAlexandru Avadanii1-0/+0
This bump also brings in: - arm-pod{8,9} config; - UNH lab config; - NTP config in IDFs for existing PODs; - basic vPDF support; Change-Id: Ifc16616bc8a13de2bdf048971448d4853e93880e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-05-17[noha] Bring in gnocchi/panko servicesMichael Polenchuk1-1/+1
JIRA: FUEL-372 Change-Id: I4e322a4a2c84843e9350fe9b3b849cd0c5244a12 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-04-26[jump] Print CPU, RAM, disk info for debugAlexandru Avadanii1-0/+10
JIRA: FUEL-323 Change-Id: I74f92fbdf84762540401fe07a6eb424bdc7aa664 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-04-25[submodule] Bump Pharos for zte-pod1 IDF updateAlexandru Avadanii1-0/+0
Change-Id: I4942a1c3e12db304db11fbc231ca26ff87e3b6bb Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-04-18[reclass] Parameterize tenant VLAN rangesAlexandru Avadanii1-0/+0
Instead of hardcoding the VLAN ranges in reclass, read them from IDF. This is required when VLAN segmentation is used instead of VXLAN, as lab switches should be configured appropiately (e.g. Armband PODs use VLANs 2000+ instead of 1000-1030). Change-Id: I009ccea9b328c40f055a745791eafe3393b96390 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-04-13[submodule] Bump PharosCristina Pauna1-0/+0
Change-Id: Ib5f7a9d7874704b11b10cdc1eefaae19de2a629e Signed-off-by: Cristina Pauna <cristina.pauna@enea.com>
2018-04-05[submodule] Bump Pharos for Armband IDF changesAlexandru Avadanii1-0/+0
Bring in arm-pod5 hugepage configuration et al. Change-Id: If8a2f9cd84b400c5a8a58fe477f8ef7a12c42bf8 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-18lib.sh: Skip modprobing nbd, loop if inbuiltAlexandru Avadanii2-2/+4
While at it, skip waiting for Salt master when deleting UEFI stale entries if it doesn't respond to ping. Also, use https for fetching Armband GPG key to bypass yet another hks issue behind proxies/firewalls that block hks port. Change-Id: I400cbe3257094b62c96b302a3c81c5ffd1ba4755 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-13salt.sh: Copy mcp.rsa to cfg01Alexandru Avadanii2-0/+2
- let rsync transfer mcp.rsa* to cfg01; - symlink mcp.rsa in /root; Change-Id: I18cb70306faf9d166e1110e374a9fe16ac4449de Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-11lib.sh: Add missing `sudo` for `losetup`Alexandru Avadanii1-2/+2
Change-Id: I1c5e3d7a0dbac14bf242730d6ac8d2b1d0817907 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-11[IDF] Move ip-range start higher for mgmt, publicAlexandru Avadanii1-0/+0
- move mgmt start to .10; - move public start to .100; - sync local idf-pod1; JIRA: ARMBAND-365 Change-Id: I5d93c4f486daf280d8bdbe92f5aa7edd8d020db0 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-07[IDF] net_config: Add support for custom ip-rangeAlexandru Avadanii1-0/+0
- submodule: bump Pharos to pick up installer adapter support, as well as IDF updates for Ericsson baremetal pod1; - labs/local/virtual: Bump mgmt, public networks start addresses from .1 to .10, similar to ericsson-pod1; While at it, drop patch now upstream and instead adopt the new param 'neutron:server:root_helper_daemon'. JIRA: FUEL-351 Change-Id: I9bc244a7fd8698861a390ed2b6a27804be46c285 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-07Merge "[novcp] Add deploy argument `-N` (experimental)"Alexandru Avadanii2-1/+2
2018-03-07Merge "[vnode parsing] Move core logic to j2"Alexandru Avadanii5-67/+144
2018-03-07Clean up aodh formula symlinkMichael Polenchuk1-4/+0
Change-Id: I1d84b5d4e3eb0de1c3ebf6fc5a538ee6714d53b2 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-03-07[novcp] Add deploy argument `-N` (experimental)Alexandru Avadanii2-1/+2
Instead of duplicating scenarios for NOVCP, allow it to be specified using a new deploy argument, `-N`. Things are getting convoluted, so instead of creating dedicated '*_pdf.yml.j2' files for each group of similar features, apply the templating in-place and rename all affected files to ".yml.j2". This breaks .gitignore assumption of hiding only "*_pdf.yml" files, so extend (manually) the <mcp/reclass/classes/cluster/.gitignore> with `git ls-files --exclude-standard -o` after an expansion. - ha: move nfv.cpu_pinning to j2, conditioned by 'baremetal'; - ha: add cmp00* vnode definitions (hugepages need more RAM); - labs/local: enable hugepage params for non-dpdk noha; - salt.sh: add route_wrapper to all non-infra VMs; This change extends novcp support to all HA scenarios. Change-Id: I7a80415ac33367ab227ececb4ffb1bc026546d36 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-07[vnode parsing] Move core logic to j2Alexandru Avadanii5-67/+144
j2/python is easier to read and manipulate strings, although it does need some special care about undefined dict keys. With this in place, deploy.sh only contains the higher level logic for the deployment process. - merge arch-specific default configuration files into a singular file with arch name as main dict key of old config (also avoids creating duplicate 'virtual' YAML keys in $LOCAL_PDF); - move template handling to separate <lib_template.sh>; - decouple tight bash ordering of scenario expansion -> parse_yaml -> variable export (e.g. CLUSTER_DOMAIN) -> re-use in cluster j2s; however we can't parse *all* j2s in one go, as scenario j2s might expand to YAMLs needed while expanding cluster j2; - split `do_templates` into separate functions for each stage, with no coupling between them other then call order; Change-Id: I4b5e804094c00e5e918caf769fd85fa52181ad76 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-04[jump req] Move pkg installation to lib.shAlexandru Avadanii1-0/+15
Change-Id: I687b73b256aca78c9d41d4bcd49bfbde51278b51 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-04[deploy.sh] Factor out jump bridge parsing to j2Alexandru Avadanii2-0/+21
Change-Id: Ie4d8e70866d533d195a6e80cdfecbdb00a3027ce Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-03[submodule] Bump Pharos for IDF updatesAlexandru Avadanii1-0/+0
Change-Id: I9e71d9006bad53cc79805ed9255b53b6c5705725 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>