summaryrefslogtreecommitdiffstats
path: root/mcp/scripts
AgeCommit message (Collapse)AuthorFilesLines
2017-12-18[baremetal] Move salt master IP to PXE/adminAlexandru Avadanii1-4/+3
Use PXE/admin network for salt traffic from/to all minions except cfg01, mas01. This allows us to drop the route to admin net from cfg01. Change-Id: Ic2526f1ff77afe5d92ced900971f4c8f78d2d8a2 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-12-18[baremetal] MaaS: Fix DHCP dynamic reserved rangeAlexandru Avadanii1-0/+0
- patch MaaS to default to `DHCP` instead of `AUTO` for physical PXE interfaces (all IPs will be handed out by MaaS DHCP *inside* the defined dynamic DHCP IP range); - reduce range to silence bogus MaaS warning about address exhaustion; - regenerate pod_config.yml.example to reflect the changes; - drop `opnfv_infra_maas_pxe_address` (duplicate of `opnfv_infra_maas_node01_deploy_address`); - add `opnfv_infra_config_pxe_address` for future usage; - while at it, fix missing patch copyright; JIRA: FUEL-316 Change-Id: I81fad333e77f7c8508cd2b2b267c7b39c130e3e1 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-12-12deploy.sh: Factor out pkg deps into yaml filesAlexandru Avadanii2-0/+70
Instead of hardcoding the packages to install on the jumphost at deploy time in `ci/deploy.sh`, move the requirements list to a dedicated yaml file, so it is decoupled from actual code. While at it, add missing deps: - ipxe-qemu (AArch64 Ubuntu only); Change-Id: I9ee5c0796b70f9d29427d608b6415333c8841e1e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-12-11[baremetal] Move all MaaS PXE net config to PDFAlexandru Avadanii1-3/+0
- s/opnfv_maas_pxe_/opnfv_infra_maas_pxe_/g to align with other vars; - patches: pharos: Add MaaS PXE network to installer adapter; - runtime.yml{,.template}: move to installer adapter, update pod_config.yml example; - drop MAAS_PXE_NETWORK global env var, now read strictly from PDF; JIRA: FUEL-313 Change-Id: I46d7510bd53fba7890c411d36bc28fd6ff6f3648 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-12-11patches: pharos: Bump & drop upstream patchesAlexandru Avadanii1-0/+0
All our staged patches for Pharos repo are now upstream, so bump the git submodule and remove the patch files. Change-Id: I0d68eb3bdd9abfa286c3640acc1f13ce6100801d Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-12-09submodule: pharos: Bump to fix run without eyamlAlexandru Avadanii1-0/+0
Most Fuel PODs do not use encrypted PDFs, so bump Pharos git submodule to include updated `generate_config.py` to silently skip decryption in case eyaml is not installed. Change-Id: I845a32c7001123a93e38197e0daf53d18c797aed Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-12-08[baremetal] PDF: Use net_admin for MaaS PXE netAlexandru Avadanii1-1/+1
While at it, fix another small issue where a variable (parameters__param_opnfv_maas_pxe_address) was used without being populated (defined in 'runtime.yml', which is not read using `parse_yaml`, like the scenario yml or the PDF expanded class, pod_config.yml). JIRA: FUEL-313 Change-Id: Iee88601d5420f55572bf90c8cf330afbedfb2e21 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-12-07patches: pharos: Drop patches merged upstreamAlexandru Avadanii1-0/+0
Also, remove redundant pharos patch that adds prx mgmt IPs: - "Re-assign mgmt network to proxy nodes" as those values are set already (to different values!) by patch: + "extend public gateway support" While at it, `make patches-export` should clean the patch dir first. Change-Id: Ice106e5d48c7b4cd90ffc6af7441199034d4f546 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-12-07Merge "base image: Explicitly call kpartx"Michael Polenchuk1-3/+7
2017-12-07Merge "lib.sh: persistent cfg01, mas01 mcpcontrol"Michael Polenchuk1-2/+2
2017-12-01salt.sh: Force ocata link creationAlexandru Avadanii1-1/+1
Required for re-deploying with `ci/deploy.sh -f`. Change-Id: I8e6d95f17f57b9dab3257fb4c8ccd1f962dacb45 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-29Merge "AArch64: base image: pre-install salt-minion"Michael Polenchuk2-2/+8
2017-11-28[virtual] Update openstack version to pikeMichael Polenchuk1-1/+5
Change-Id: I1df0228cb44bf9122aaf93dd25fc16a0d26a5240 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-11-28base image: Explicitly call kpartxAlexandru Avadanii1-3/+7
qemu-nbd currently available in CentOS 7 does not add partition mappings automatically for NBD devices, so add explicit `kpartx` calls. Change-Id: Ifa79c89b82024602b782c449dbf4de10899e03b5 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-27lib.sh: persistent cfg01, mas01 mcpcontrolAlexandru Avadanii1-2/+2
cfg01, mas01 DHCP leases in mcpcontrol virtual network should be persistent (if cfg01 IP changes, minions can't find Salt Master). Change-Id: I497207ebe1537af94fd92de12491664d17ad3144 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-27AArch64: base image: pre-install salt-minionAlexandru Avadanii2-2/+8
While at it, rename apt repo in foundation node user-data template from "salt" to "saltstack", to align with reclass model naming. Change-Id: I5b216492349ae187b568884b1ab4046c52b1c6b2 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-27Add pre-{install,purge} support for base imageAlexandru Avadanii2-5/+144
Extend <lib.sh> and its invocation from <ci.deploy.sh> with support for modifying foundation node VMs base image prior to using it with: - additional APT GPG keys; - additional APT repos; - packages to pre-install; - packages to pre-remove; - (non-configurable) cloud init datasource via NoCloud only, so VCP VMs won't wait for metadata service; While at it, re-use the resulting image as a base for another round of pre-patching (same operations as above are supported) to provide a base image for VCP VMs. Add AArch64-specific configuration based on new mechanisms: - pre-install linux-image-generic-hwe-16.04-edge (and headers) for foundation node and VCP (common) image (also requires new repo and its key); - pre-install cloud-init for VCP image (it should already be installed, but script needs non-empty config for VCP to create the VCP image and transfer it over to Salt Master); NOTE: cloud-init is required on VCP VMs for DHCP on 1st iface. JIRA: FUEL-309 Change-Id: I7dcaf0ffd9c57009133c6d339496ec831ab14375 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-23salt.sh: Rework `git -C` for older git compatAlexandru Avadanii1-1/+1
CentOS 7.4 provides git 1.8.3, while `git -C` support was introduced in 1.8.5, so rework <salt.sh> to not rely on this parameter. Change-Id: Ied88466b2b3d91a3317922fa08f60afe934358a0 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-21ci/deploy.sh: Add new `-E` arg for env eraseAlexandru Avadanii1-12/+38
NOTE: In order to undefine VCP VMs with NVRAM (e.g. AArch64 VMs using AAVMF), an additional parameter should be passed to libvirt by Salt virt core module (equivalent to `virsh undefine --nvram`). While at it, pass CI_DEBUG, ERASE_ENV enviroment variables to state execution, and stop force-applying patches. Also refactor the rsync between foundation node and Salt master, so the whole git repo is copied as </root/opnfv>, and <root/fuel> becomes a link to it; useful for Armband, where 'fuel' is a git submodule. Fix .git paths after rsync, so git submodules work as expected in cfg01 repos. JIRA: FUEL-307 Change-Id: Ic62f03e786581c019168c50ccc50107238021d7f Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-12salt formulas: Pin via reclass instead of salt.shAlexandru Avadanii1-8/+0
Previous commit pinned the salt-formula-package by installing a specific version after OPNFV-specific patches were applied by `patch.sh`. To prevent patch bypass, move the pin to reclass model. While at it, also move salt-formula-neutron pinning to reclass to bypass wrong minimum length of 2 for version-pinned formula pkgs. Change-Id: I35c2773c130d79ec919685cd771311d6c11e11a8 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-11salt.sh: Pin MaaS formula to last known stable verAlexandru Avadanii1-0/+3
Recent upstream changes broke MaaS machine status parsing. Pin the formula to the previous version until upstream provides a fixed package or a repo snapshot baseline. Change-Id: I26ebd60f40812ae47da0ca28d19c9fdcac7920da Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-08lib.sh: Cleanup previous /boot/efi contentsAlexandru Avadanii1-1/+2
Some UEFI firmwares insist on scanning removable drives, even when boot entries were deleted from UEFI boot list (board flash). To work around this, remove contents of </boot/efi/*>, so scanning won't identify any valid EFI binaries. Another option would be erasing partition tables, but identifying the underlying disk(s) is more complicated, especially when using LVM/RAID etc. Change-Id: I9949b99b139b1642e3bd8f04de3bd5ef74d1ecc5 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-07Revert "Freeze the galera formula to avoid update"Michael Polenchuk1-4/+0
This reverts commit f49340e1aac3f5e8bb009a9d0d35aa6fedba7384. Version has been set back to 5.6 in upstream. Change-Id: I06d6865e59cfbc5055322eb31cbb237075ebdcb1 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-10-31Freeze the galera formula to avoid updateMichael Polenchuk1-0/+4
Change-Id: Ibf03a7dc2ed58a749476ae0bdc6cad2bba8783fb Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-10-31Connect proxy nodes to public networkMichael Polenchuk1-0/+0
cloudify vnf requires public enpoints to be available from instance with manager, so connect nodes w/ nginx to routed public network. JIRA: FUEL-286 Change-Id: Id0caa699519b9b46dad51deac6955afe6a0050de Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-10-28lib.sh: rm Ubuntu boot entry on EFI systemsAlexandru Avadanii1-0/+10
On EFI-enabled systems, grub-install from grub-efi-* package installs a boot entry named "ubuntu". MaaS relies on IPMI to set boot order to PXE first; however on systems with buggy firmware or without full IPMI support, that fails, leading to booting Ubuntu from hard disk instead. Work around this by clearing any previous Ubuntu boot entry from board flash, before starting a new baremetal deploy. NOTE: This only runs against nodes that are online from a previous deploy. Closes: ARMBAND-47 Change-Id: I1c4ece09e42845ce2a1b7119ec69e46e5ca12376 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-27Merge "lib.sh: Fix compatibility with bash 4.2"Alexandru Avadanii1-16/+16
2017-10-27lib.sh: Fix compatibility with bash 4.2Alexandru Avadanii1-16/+16
JIRA: FUEL-296 Change-Id: Ide9f9333fe9b44ff6b78678064f8e67f05aabd42 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-26AArch64: Switch jumpserver VMs to virtio videoAlexandru Avadanii1-2/+2
Drop vgabios dependency by switching video from VGA to virtio for all VMs spawned on the jumpserver. NOTE: This requires virtualization packages on the jumpserver to be up to date (e.g. libvirt, QEMU). JIRA: ARMBAND-306 Change-Id: I73913e1ae8584f4e73b92994f78f7ec363cba3ec Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-20lib.sh: Fix locals override in nested invocationsAlexandru Avadanii1-11/+14
'wait_for' bash function is nested in another 'wait_for' call in some places, which leads to inner calls interfering with outer calls by overriding the locally scoped variables, including the 'attempt' internal counter. In some cases, the outer 'wait_for' would exit after a single attempt. Fix that by running all contents of `wait_for` inside a subshell, which inherits outer calls variables, but does not override them when the inner call is finished. Change-Id: I450eda3d023af2380c61ee930071fbfc393a5645 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-18Horizon: service: Default to v2 APIAlexandru Avadanii1-1/+1
Horizon service definition hardcodes the API version to v3, and also overrides the value in our reclass model during interpolation. Adjust the default to '2'. While at it, move 'reclass' patching later in `salt.sh`, after the salt formula packages are installed. JIRA: FUEL-284 Change-Id: I6f63b4d00ae85c82b076b3efef857e7cc4f3fd35 Signed-off-by: Ting Wu <ting.wu@enea.com> Signed-off-by: Junaid Ali <junaidali.yahya@gmail.com> Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-16Merge "salt.sh: DEBIAN_FRONTEND=noninteractive"Alexandru Avadanii1-0/+1
2017-10-16[ovs/dpdk] Return true on linux.network stateMichael Polenchuk1-1/+1
linux.network state is expected to be down until node reboot to activate required hugepages therefore return true in any case. Change-Id: I61deac12b9ac212c5f79cdcce30cda8e5d24413b Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-10-16salt.sh: DEBIAN_FRONTEND=noninteractiveAlexandru Avadanii1-0/+1
Change-Id: Ibb1e78563043298dffdb4b7e9091d3f582e189fc Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-15states: Break on error, retry states up to 5 timesAlexandru Avadanii5-12/+20
While applying scenario states, break on error, and retry failed state up to 5 times. Apply the same behavior for `salt.sh`. Add new deploy parameter, '-D', backed up by 'CI_DEBUG' env var, which gates deploy sh scripts logging (set -x). Also extend '-f' deploy parameter, allowing it to be specified more than once; the first occurence will skip infra VM creation, but still sync reclass & other config from local repo, while a second occurence will also disable config sync. To prevent glusterfs client state from failing due to non-existent nova user/group, move it after nova:compute's nova state is applied. Change-Id: I234e126e16be0e133d878957bd88fed946955de8 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-14Add license headers where missingAlexandru Avadanii7-0/+53
While at it, compact 'set' into bash shebang where possible and add `make patches-copyright` target to simplify adding patch license headers. Change-Id: I0c841de72e5709e5eef915a52c5ec4a7fc0f7c37 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-14Drop infinite loops in favor of finite wait_forAlexandru Avadanii2-4/+3
While at it, fix some shellcheck warnings, and s/fgrep/grep -F/g. Change-Id: I093b7b4c196731b1ecc0c27a4111955b2e412762 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-14Merge "states: Split virtual_control_plane from maas"Alexandru Avadanii1-10/+23
2017-10-14states: Split virtual_control_plane from maasAlexandru Avadanii1-10/+23
We should eventually also support baremetal deploys without a virtualized control plane (VCP), so decouple MaaS provisioning from VCP provisioning. While at it, move "wait_for" bash function from maas state to common library file, lib.sh. Change-Id: I32c33135655cb6aceae901a5f92b51265a8c84b4 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-13patch.sh: Exit on patch apply failureAlexandru Avadanii1-1/+2
Patches that fail to apply are silently skipped. Since retrying is not possible, deploy moves on, making the problem hard to observe. Change-Id: I3fe01cef6b245de53767a660b9b4fdfae0d5d0ff Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-05log.sh: Fix sourcing of globals.shAlexandru Avadanii1-2/+2
Fix sourcing of globals.sh from log.sh when running with a CWD different from scripts dir. Change-Id: I680531022d30b57f650a612493f40fdb3e073dc2 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-04Merge "ci/deploy.sh: Move global vars to separate file"Alexandru Avadanii2-0/+23
2017-10-04Merge "Identify jump host bridges based on IDF / PDF nets"Alexandru Avadanii3-9/+3
2017-10-03ci/deploy.sh: Move global vars to separate fileAlexandru Avadanii2-0/+23
log.sh should also work as standalone script, so factor out global variables from <ci/deploy.sh> into <mcp/scripts/globals.sh>, which should be sourced by log.sh when required variables are not set. Change-Id: I5a5bb85ee5835f567d6910b11c137d95e2147714 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-03Identify jump host bridges based on IDF / PDF netsAlexandru Avadanii3-9/+3
- minor refactor of runtime templates parsing to allow var expansion; - parse <pod_config.yml> into shell vars, match dynamically networks from PDF to IP addresses on bridges of current jumphost; - keep old '-B' parameter in <ci/deploy.sh>, use it for providing fallback values in case there's no bridge name specified via IDF and no IP on the jumphost for one or more of the PDF networks; - re-enable dry-run to ease testing of the above; - add sample 'idf-pod1.yaml' to <mcp/config/labs/local>; The new behavior will try to determine the jump host bridge names: 1. Based on IDF mapping, if available 2. Based on PDF network matching with IP addrs on jumphost; 3. Fallback to values passed via '-B'; 4. Fallback to default values hardcoded in the deploy script; Later, we will drop MaaS network env vars in favor of PDF vars, once the PDF template is generating them. Change-Id: If9cd65d310c02965b2e2bfa06a0d7e0f97f1dd48 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-03Apply neutron formula patch after formula updateMichael Polenchuk1-0/+1
Change-Id: I8dba3676adc8cc49731d91db7cc028a9c5b1627d Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-10-03salt.sh: Stick to older salt-formula-neutronAlexandru Avadanii1-0/+4
Recent changes to salt-formula-neutron, more precisely [1], broke the compatibility with our current reclass model [2], which was not updated to reflect the formula change. Instead of locally patching the reclass model, install the latest compatible version of the Neutron formula: - 2016.12.1+201709251458.f0607d9-1xenial1; [1] https://github.com/salt-formulas/salt-formula-neutron/commit/ 79ffa26858d682f404984175dc1ab93863a149ad [2] https://github.com/Mirantis/reclass-system-salt-model Change-Id: Ia1bc9de9a1543f7888688be76ff3513e6b4f062b Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-27pharos git submodule: Bump for latest PDF adapterAlexandru Avadanii1-0/+0
Change-Id: I658c1dfeb369adf74038abd0ce1b593d40303527 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-27Merge "PDF parsing support"Alexandru Avadanii2-5/+7
2017-09-27PDF parsing supportAlexandru Avadanii2-5/+7
- add new git submodule pointing to OPNFV Pharos; - use Pharos 'generate_config.sh' to parse the PDF using the Fuel installer adapter and generate <pod_config.yml> dynamically; - build <pod_config.yml> outside current git repo and sync separately to prevent sensitive data leak; - add <pod1.yaml> PDF sample based on LF-POD2, should be used with 'ci/deploy.sh -l local -p pod1'; Change-Id: I4e1b95f180bcd5ade5d86f516628eb8edbe64b1c Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>