aboutsummaryrefslogtreecommitdiffstats
path: root/mcp/scripts/salt.sh
AgeCommit message (Collapse)AuthorFilesLines
2018-01-04Add pre-{install,purge} support for base imageAlexandru Avadanii1-0/+12
Extend <lib.sh> and its invocation from <ci.deploy.sh> with support for modifying foundation node VMs base image prior to using it with: - additional APT GPG keys; - additional APT repos; - packages to pre-install; - packages to pre-remove; - (non-configurable) cloud init datasource via NoCloud only, so VCP VMs won't wait for metadata service; While at it, re-use the resulting image as a base for another round of pre-patching (same operations as above are supported) to provide a base image for VCP VMs. Add AArch64-specific configuration based on new mechanisms: - pre-install linux-image-generic-hwe-16.04-edge (and headers) for foundation node and VCP (common) image (also requires new repo and its key); - pre-install cloud-init for VCP image (it should already be installed, but script needs non-empty config for VCP to create the VCP image and transfer it over to Salt Master); NOTE: cloud-init is required on VCP VMs for DHCP on 1st iface. JIRA: FUEL-309 Change-Id: I7dcaf0ffd9c57009133c6d339496ec831ab14375 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 6ead32f737f39e3d82e1484a9d31f07ed2daeadb)
2017-12-31salt.sh: Use salt-call to apply linux sls on cfg01Alexandru Avadanii1-1/+1
Also, retry applying linux high state up to twice, due to rare spurious failures with 'No reponse' status. Change-Id: Ic7839a5c9501673cb127412136afb91e05f87a7e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commmit 4509936347b429fb36a27844a63d808f057fd61e)
2017-12-30salt: Use apt-mk '2017.12' distributionAlexandru Avadanii1-1/+1
'nightly' repo dist from apt-mk is broken, so switch to '2017.12'. Change-Id: Ie12dfc2a499910b8b98a63886ba16044e66435f5 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-24salt.sh: Rework `git -C` for older git compatAlexandru Avadanii1-1/+1
CentOS 7.4 provides git 1.8.3, while `git -C` support was introduced in 1.8.5, so rework <salt.sh> to not rely on this parameter. Change-Id: Ied88466b2b3d91a3317922fa08f60afe934358a0 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 0adbe359a0e448e0a8007b0041890e33a612a4dc)
2017-11-22ci/deploy.sh: Add new `-E` arg for env eraseAlexandru Avadanii1-12/+38
NOTE: In order to undefine VCP VMs with NVRAM (e.g. AArch64 VMs using AAVMF), an additional parameter should be passed to libvirt by Salt virt core module (equivalent to `virsh undefine --nvram`). While at it, pass CI_DEBUG, ERASE_ENV enviroment variables to state execution, and stop force-applying patches. Also refactor the rsync between foundation node and Salt master, so the whole git repo is copied as </root/opnfv>, and <root/fuel> becomes a link to it; useful for Armband, where 'fuel' is a git submodule. Fix .git paths after rsync, so git submodules work as expected in cfg01 repos. JIRA: FUEL-307 Change-Id: Ic62f03e786581c019168c50ccc50107238021d7f Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 77942178b3aff6adc83b5f83645acfff467fa76a)
2017-11-12salt formulas: Pin via reclass instead of salt.shAlexandru Avadanii1-8/+0
Previous commit pinned the salt-formula-package by installing a specific version after OPNFV-specific patches were applied by `patch.sh`. To prevent patch bypass, move the pin to reclass model. While at it, also move salt-formula-neutron pinning to reclass to bypass wrong minimum length of 2 for version-pinned formula pkgs. Change-Id: I35c2773c130d79ec919685cd771311d6c11e11a8 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 1058c7ee35d64f38d978db2b3f840afca2facdef)
2017-11-11salt.sh: Pin MaaS formula to last known stable verAlexandru Avadanii1-0/+3
Recent upstream changes broke MaaS machine status parsing. Pin the formula to the previous version until upstream provides a fixed package or a repo snapshot baseline. Change-Id: I26ebd60f40812ae47da0ca28d19c9fdcac7920da Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit c4b87dbc189e2a2f4d586d41ef5db2166c42231d)
2017-10-18Horizon: service: Default to v2 APIAlexandru Avadanii1-1/+1
Horizon service definition hardcodes the API version to v3, and also overrides the value in our reclass model during interpolation. Adjust the default to '2'. While at it, move 'reclass' patching later in `salt.sh`, after the salt formula packages are installed. JIRA: FUEL-284 Change-Id: I6f63b4d00ae85c82b076b3efef857e7cc4f3fd35 Signed-off-by: Ting Wu <ting.wu@enea.com> Signed-off-by: Junaid Ali <junaidali.yahya@gmail.com> Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 68ff467c4c2d9d0f201e2912c97616402f8109a3)
2017-10-18Merge "states: Break on error, retry states up to 5 times" into stable/euphratesAlexandru Avadanii1-5/+7
2017-10-16salt.sh: DEBIAN_FRONTEND=noninteractiveAlexandru Avadanii1-0/+1
Change-Id: Ibb1e78563043298dffdb4b7e9091d3f582e189fc Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 882220ff185d7bcc15e6c5f75edaee454c933956)
2017-10-16[ovs/dpdk] Return true on linux.network stateMichael Polenchuk1-1/+1
linux.network state is expected to be down until node reboot to activate required hugepages therefore return true in any case. Change-Id: I61deac12b9ac212c5f79cdcce30cda8e5d24413b Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com> (cherry picked from commit 044f49fd195526d8c00967fb9a0b6b1ab0a36c14)
2017-10-15states: Break on error, retry states up to 5 timesAlexandru Avadanii1-5/+7
While applying scenario states, break on error, and retry failed state up to 5 times. Apply the same behavior for `salt.sh`. Add new deploy parameter, '-D', backed up by 'CI_DEBUG' env var, which gates deploy sh scripts logging (set -x). Also extend '-f' deploy parameter, allowing it to be specified more than once; the first occurence will skip infra VM creation, but still sync reclass & other config from local repo, while a second occurence will also disable config sync. To prevent glusterfs client state from failing due to non-existent nova user/group, move it after nova:compute's nova state is applied. Change-Id: I234e126e16be0e133d878957bd88fed946955de8 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 23f9ef727310a8ef449372313785241df6557559)
2017-10-14Drop infinite loops in favor of finite wait_forAlexandru Avadanii1-1/+1
While at it, fix some shellcheck warnings, and s/fgrep/grep -F/g. Change-Id: I093b7b4c196731b1ecc0c27a4111955b2e412762 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 089585ad5370109d4c2a476b94b0017153d7713d)
2017-10-13patch.sh: Exit on patch apply failureAlexandru Avadanii1-1/+2
Patches that fail to apply are silently skipped. Since retrying is not possible, deploy moves on, making the problem hard to observe. Change-Id: I3fe01cef6b245de53767a660b9b4fdfae0d5d0ff Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit bd0259e7bb5f045a1ba7958d79039848d062aad1)
2017-10-03Apply neutron formula patch after formula updateMichael Polenchuk1-0/+1
Change-Id: I8dba3676adc8cc49731d91db7cc028a9c5b1627d Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com> (cherry picked from commit 952b5cc60febe14df428f92280448327572d5c24)
2017-10-02salt.sh: Stick to older salt-formula-neutronAlexandru Avadanii1-0/+4
Recent changes to salt-formula-neutron, more precisely [1], broke the compatibility with our current reclass model [2], which was not updated to reflect the formula change. Instead of locally patching the reclass model, install the latest compatible version of the Neutron formula: - 2016.12.1+201709251458.f0607d9-1xenial1; [1] https://github.com/salt-formulas/salt-formula-neutron/commit/ 79ffa26858d682f404984175dc1ab93863a149ad [2] https://github.com/Mirantis/reclass-system-salt-model Change-Id: Ia1bc9de9a1543f7888688be76ff3513e6b4f062b Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit f2b1d6cfe31a451b0608b9a2aadead2dcf4cfabb)
2017-09-27PDF parsing supportAlexandru Avadanii1-5/+7
- add new git submodule pointing to OPNFV Pharos; - use Pharos 'generate_config.sh' to parse the PDF using the Fuel installer adapter and generate <pod_config.yml> dynamically; - build <pod_config.yml> outside current git repo and sync separately to prevent sensitive data leak; - add <pod1.yaml> PDF sample based on LF-POD2, should be used with 'ci/deploy.sh -l local -p pod1'; Change-Id: I4e1b95f180bcd5ade5d86f516628eb8edbe64b1c Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 36e4b9754dedef27637c09b510829f686bd469e4)
2017-09-26Run packages upgrade on openstack nodes onlyMichael Polenchuk1-2/+1
Change-Id: I53ac0be519df1bb39a6a56e236285fce95228bd4 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com> (cherry picked from commit c15004f402555df4886b18bba9a837288c6ef250)
2017-09-22Adjust memory allocation sizeMichael Polenchuk1-0/+2
* [baremetal] add memory to contollers & salt master * tune up sysctl vm.dirty* for compute nodes * upgrade packages to get the latest versions (https://bugs.launchpad.net/cinder/+bug/1641312) Change-Id: I9ad22206f2f3f11e1da3f93c7a0931c592adf1cf Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com> (cherry picked from commit 87310fb8edfe49b9621fe4410ae52d989072e3c5)
2017-09-13[virtual] Setup nfs for nova instancesMichael Polenchuk1-1/+2
Configure nfs shared storage for nova instances to support live migration feature. Change-Id: I777d5725e45d03325507d3ecc8dd59a28cbc188f Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-09-12salt-master-init.sh: Use stable 2016.11 salt verAlexandru Avadanii1-5/+3
The <salt-master-init.sh> script default to using stable 2016.3 version of Salt via BOOTSTRAP_SALTSTACK_OPTS. However, since we manually add the 2016.11 repos beforehand, the correct version (2016.11) ends up being installed. This change merely drops one uselss repo from salt master apt sources. While at it, use HTTPS for repo.saltstack.com DEB repository. Since we already add the repo to apt sources via <user-data.sh>, pass "-r" on all architectures, skipping repo (re)addition. Change-Id: I7fbe0b646b2f2a8e952718a61ff8c180116f8f27 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-11reclass: Multiarch support for arch specific paramAlexandru Avadanii1-0/+2
Introduce a simple mechanism that simulates an 'if-arch-then' cond for reclass models: - add new <all-mcp-ocata-common> class hierarchy; - at runtime (via <salt.sh>) make 'all-mcp-ocata-common.arch' point to 'all-mcp-ocata-common.$(uname -i)' dynamically; - inherit new 'arch' class in all cluster models; - factor out current x86_64 default for "salt_control_xenial_image"; - add AArch64 default for param "salt_control_xenial_image"; Change-Id: I3b239b28d0fd1cc2ced8579e2e93b764eb71ffc3 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-09log.sh: Collect /var/log from cluster nodesAlexandru Avadanii1-2/+0
Bring back <ci/deploy.sh> parameter `-L`, which is already passed by releng deploy script. Since we need to enable file_recv on salt master, do so by adding a new class in <virtual-mcp-ocata-common/infra/config.yml>, which will be inherited by all scenario reclass models. JIRA: FUEL-288 Change-Id: I9b649315984fb8580b63883d0e06c1e1efd11057 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-05Merge "salt.sh: Execute deepclean before importing patches"Michael Polenchuk1-1/+1
2017-09-04salt.sh: Execute deepclean before importing patchesJunaid Ali1-1/+1
Deepcleaning can be useful when deploying from an existing local repo Change-Id: I3908d6533c744da7cc863a250f4158c6c0e8e169 Signed-off-by: Junaid Ali <junaidali.yahya@gmail.com>
2017-09-01salt.sh, user-data: Add Saltstack arm64 repoAlexandru Avadanii1-0/+4
These changes will only be applied on AArch64, based on `uname -m`. While at it, add arm64 suppport to salt-master-setup.sh. Upstream commit [1] broke Salt bootstrap on AArch64, by introducing an architecture condition that is too strict to allow Debian package installation (even if we provide our own repo). Add "arm64" to the list of supported architectures. This needs to be done on the fly, as the bootstrap script is fetched using `curl` from <salt-master-setup.sh>. [1] https://github.com/saltstack/salt-bootstrap/commit/caa6d7d Change-Id: Id706a74a52ffe2f8b8c9dd3b9f70c78f35b2f745 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-31mcp/deploy/scripts: Move to git submoduleAlexandru Avadanii1-4/+2
Change-Id: I65d1f5680000011493bde17a249a87738ebfdd96 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-30Update salt init scripts sourceMichael Polenchuk1-3/+2
Change-Id: I269e397b78d55794b1c49bf582cc0e663cbe9ca6 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-08-11Revert "Switch distrib revision to testing"Michael Polenchuk1-1/+1
There is no much difference between nightly vs testing repos. This reverts commit 052a1626da71d232431b3a9faf576a76db61f10e. Change-Id: I40f60ddc2c9e1e59c7c4eb0be59ae19b00b7fb9b Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-08-09Synchronize down all of the dynamic modulesMichael Polenchuk1-0/+1
Sync all of the salt dynamic modules to make sure custom modules, states, grains & etc. are on the target nodes. Change-Id: I04f2ac0465d5381c4c07623e9383619d9b9731de Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-08-08Switch distrib revision to testingMichael Polenchuk1-1/+1
In order to avoid bleeding edge issues with nightly version of salt formulas, switch to "safe" testing repo source. Change-Id: I9d12f337c3952711cb8843943c23315795064016 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-08-01salt.sh: Drop upstream clone in favor of local gitAlexandru Avadanii1-10/+34
salt.sh currently clones the full Fuel@OPNFV git repo from upstream public mirror, preventing us from testing locally edited or new patches. Instead, bring back git submodule handling from old f_repos, clone and patch each submodule locally, then copy the whole parent repo over to cfg01. This is also a first step towards implementing offline deploy support. NOTE: This adds new deploy prerequisite packages: - git (for submodule clone/update); - make (for submodule patching); - rsync (for parent repo replication to cfg01); NOTE: Parent repository is expected to be a git repo, in order to work with git submodules. While at it, perform some minor related changes: - add deploy artifacts (ISOs, qcow2 files) to .gitignore, also used to filter-out such files during rsync to cfg01; - remove obsolete Fuel patches (old f_repos mechanism); - rename "reclass-system-salt-model" submodule; Change-Id: I6210d80d41010b2802e4f1b31acf249a18db7963 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-07-31mcp/scripts: shellcheck cleanupAlexandru Avadanii1-1/+2
Change-Id: I280d540b461341e14de654186016248eba4d3521 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-07-12Apply reclass patches before salt master initMichael Polenchuk1-8/+7
Change-Id: I7bb984880e98b7bdec9aa0b3895a3be9fd75cac0 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-07-11Apply upstream patchesMichael Polenchuk1-0/+2
For opendaylight support: * neutron formula * reclass system model Plus missing apply of haproxy state. Change-Id: Ic9e3672b51d5331656d636c44c3f891e49437e23 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-07-05Turn on opendaylight scenarioMichael Polenchuk1-0/+15
* fix formula & reclass cluster model * bring in running states Change-Id: I8e66e69045f5c745f9aa6f59f7ce6d66b5bf1c95 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-07-03Rewrite infra deployment scriptsMichael Polenchuk1-0/+23
* bring in scenario files * shift infra code into functions Change-Id: I650a26d03d842c3afcc7fcb97b84ef4826827a38 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>