aboutsummaryrefslogtreecommitdiffstats
path: root/mcp/scripts
AgeCommit message (Collapse)AuthorFilesLines
2017-10-04Identify jump host bridges based on IDF / PDF netsAlexandru Avadanii3-9/+3
- minor refactor of runtime templates parsing to allow var expansion; - parse <pod_config.yml> into shell vars, match dynamically networks from PDF to IP addresses on bridges of current jumphost; - keep old '-B' parameter in <ci/deploy.sh>, use it for providing fallback values in case there's no bridge name specified via IDF and no IP on the jumphost for one or more of the PDF networks; - re-enable dry-run to ease testing of the above; - add sample 'idf-pod1.yaml' to <mcp/config/labs/local>; The new behavior will try to determine the jump host bridge names: 1. Based on IDF mapping, if available 2. Based on PDF network matching with IP addrs on jumphost; 3. Fallback to values passed via '-B'; 4. Fallback to default values hardcoded in the deploy script; Later, we will drop MaaS network env vars in favor of PDF vars, once the PDF template is generating them. Change-Id: If9cd65d310c02965b2e2bfa06a0d7e0f97f1dd48 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 8ec927497b7ee0fd3b7346e957878173b080ef6a)
2017-10-03Apply neutron formula patch after formula updateMichael Polenchuk1-0/+1
Change-Id: I8dba3676adc8cc49731d91db7cc028a9c5b1627d Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com> (cherry picked from commit 952b5cc60febe14df428f92280448327572d5c24)
2017-10-02salt.sh: Stick to older salt-formula-neutronAlexandru Avadanii1-0/+4
Recent changes to salt-formula-neutron, more precisely [1], broke the compatibility with our current reclass model [2], which was not updated to reflect the formula change. Instead of locally patching the reclass model, install the latest compatible version of the Neutron formula: - 2016.12.1+201709251458.f0607d9-1xenial1; [1] https://github.com/salt-formulas/salt-formula-neutron/commit/ 79ffa26858d682f404984175dc1ab93863a149ad [2] https://github.com/Mirantis/reclass-system-salt-model Change-Id: Ia1bc9de9a1543f7888688be76ff3513e6b4f062b Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit f2b1d6cfe31a451b0608b9a2aadead2dcf4cfabb)
2017-09-29pharos git submodule: Bump for latest PDF adapterAlexandru Avadanii1-0/+0
Change-Id: I658c1dfeb369adf74038abd0ce1b593d40303527 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 5ad2e311bbc1326493386aeee4d87a9091337adf)
2017-09-29Merge "lib.sh: Use host bridge in favor of virsh network" into stable/euphratesAlexandru Avadanii1-4/+5
2017-09-27PDF parsing supportAlexandru Avadanii2-5/+7
- add new git submodule pointing to OPNFV Pharos; - use Pharos 'generate_config.sh' to parse the PDF using the Fuel installer adapter and generate <pod_config.yml> dynamically; - build <pod_config.yml> outside current git repo and sync separately to prevent sensitive data leak; - add <pod1.yaml> PDF sample based on LF-POD2, should be used with 'ci/deploy.sh -l local -p pod1'; Change-Id: I4e1b95f180bcd5ade5d86f516628eb8edbe64b1c Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 36e4b9754dedef27637c09b510829f686bd469e4)
2017-09-27lib.sh: Use host bridge in favor of virsh networkAlexandru Avadanii1-4/+5
If a linux bridge already exists on the jump host, with the same name as the value passed via `-B` or the default, use it instead of trying to create a virsh network. Change-Id: If7f0753ec366d429ff101c2b37dd68f895903404 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 96cccf2b507949978c1b37c65961a5182ffbbbfd)
2017-09-26Run packages upgrade on openstack nodes onlyMichael Polenchuk1-2/+1
Change-Id: I53ac0be519df1bb39a6a56e236285fce95228bd4 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com> (cherry picked from commit c15004f402555df4886b18bba9a837288c6ef250)
2017-09-26Fix or silence all yamllint warningsAlexandru Avadanii1-1/+1
Change-Id: Iface28ab770beee00374afb902ef4f9c983538f5 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com> (cherry picked from commit 597e4b55f57001ead8e90f30e2e3211c7d705ca8)
2017-09-22Adjust memory allocation sizeMichael Polenchuk1-0/+2
* [baremetal] add memory to contollers & salt master * tune up sysctl vm.dirty* for compute nodes * upgrade packages to get the latest versions (https://bugs.launchpad.net/cinder/+bug/1641312) Change-Id: I9ad22206f2f3f11e1da3f93c7a0931c592adf1cf Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com> (cherry picked from commit 87310fb8edfe49b9621fe4410ae52d989072e3c5)
2017-09-13[virtual] Setup nfs for nova instancesMichael Polenchuk1-1/+2
Configure nfs shared storage for nova instances to support live migration feature. Change-Id: I777d5725e45d03325507d3ecc8dd59a28cbc188f Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-09-12Merge "salt-master-init.sh: Use stable 2016.11 salt ver"Alexandru Avadanii2-6/+4
2017-09-12reclass, states: Parametrize runtime configurationAlexandru Avadanii2-10/+10
mcpcontrol virsh network, as well as MaaS PXE network are installer specific, and not POD specific. Therefore, these should be easily parametrized without the PDF, using only installer inputs (e.g. env vars passed via Jenkins). - add new <all-mcp-ocata-common.opnfv.runtime> reclass class; - parametrize at runtime new reclass class based on global vars; - factor out MaaS deploy address / config using new mechanism; - parametrize at runtime virsh network definitions based on template; - add new "maas.pxe_route" sls for configuring routing on cfg01; - replace env vars with the new sls in "maas" state; NOTE: baremetal parametrization will be handled later. Change-Id: Ifd61143d818fb088b3f4395388ba769bbc49156e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-12salt-master-init.sh: Use stable 2016.11 salt verAlexandru Avadanii2-6/+4
The <salt-master-init.sh> script default to using stable 2016.3 version of Salt via BOOTSTRAP_SALTSTACK_OPTS. However, since we manually add the 2016.11 repos beforehand, the correct version (2016.11) ends up being installed. This change merely drops one uselss repo from salt master apt sources. While at it, use HTTPS for repo.saltstack.com DEB repository. Since we already add the repo to apt sources via <user-data.sh>, pass "-r" on all architectures, skipping repo (re)addition. Change-Id: I7fbe0b646b2f2a8e952718a61ff8c180116f8f27 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-11salt master, maas: Move mcpcontrol to 10.20.0.0/24Alexandru Avadanii2-3/+4
Use INSTALLER_IP Jenkins param instead of SALT_MASTER_IP, allowing us to drop SALT_MASTER_IP completely from releng. mcpcontrol IP changes: - 192.168.10.100 becomes 10.20.0.2 (align with legacy Fuel master); - 192.168.10.3 becomes 10.20.0.3 (baremetal MaaS address); JIRA: FUEL-285 Change-Id: I6e2d44c3a8b43846196bd64191735214167a76ce Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-11reclass: Multiarch support for arch specific paramAlexandru Avadanii1-0/+2
Introduce a simple mechanism that simulates an 'if-arch-then' cond for reclass models: - add new <all-mcp-ocata-common> class hierarchy; - at runtime (via <salt.sh>) make 'all-mcp-ocata-common.arch' point to 'all-mcp-ocata-common.$(uname -i)' dynamically; - inherit new 'arch' class in all cluster models; - factor out current x86_64 default for "salt_control_xenial_image"; - add AArch64 default for param "salt_control_xenial_image"; Change-Id: I3b239b28d0fd1cc2ced8579e2e93b764eb71ffc3 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-11Merge "log.sh: Collect /var/log from cluster nodes"Michael Polenchuk2-2/+35
2017-09-09log.sh: Collect /var/log from cluster nodesAlexandru Avadanii2-2/+35
Bring back <ci/deploy.sh> parameter `-L`, which is already passed by releng deploy script. Since we need to enable file_recv on salt master, do so by adding a new class in <virtual-mcp-ocata-common/infra/config.yml>, which will be inherited by all scenario reclass models. JIRA: FUEL-288 Change-Id: I9b649315984fb8580b63883d0e06c1e1efd11057 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-08bash scripts: Fix remaining shellcheck warn/errsAlexandru Avadanii1-1/+1
Fix/silence all shellcheck errors, except for scripts in <prototypes/sfc_tacker>. Change-Id: Idc317cdba0f69b78299f2d3665e72ffc19dd8af5 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-08Merge "Move mcp.rsa to /var/lib/opnfv"Alexandru Avadanii1-1/+1
2017-09-08lib.sh: Remove all storage on undefineAlexandru Avadanii1-1/+4
While executing the deploy script via sudo will raise no issues with re-using old storage volume files, non-root deploys fail to change ownership of said files. Properly clean after ourselves when destroying and undefining VMs. JIRA: ARMBAND-311 Change-Id: I7fdd269dccbfd5ab5285d852ac7dacfc4dc5ccd7 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-07Move mcp.rsa to /var/lib/opnfvAlexandru Avadanii1-1/+1
JIRA: FUEL-280 Change-Id: I1e07b0e1597b2a1e4a92e1274f89dda62cb81bb8 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-05Merge "ci/deploy.sh, lib.sh: Align SSH_KEY with Jenkins"Michael Polenchuk1-14/+21
2017-09-05Merge "salt.sh: Execute deepclean before importing patches"Michael Polenchuk1-1/+1
2017-09-05ci/deploy.sh, lib.sh: Align SSH_KEY with JenkinsAlexandru Avadanii1-14/+21
Jenkins defines a build param also called SSH_KEY, which holds the full path to our MCP private RSA key, as opposed to Fuel@OPNFV, which uses the file basename as SSH_KEY. Switch to using full path in SSH_KEY, while also moving the key outside /tmp. NOTE: When running `ci/deploy.sh` with sudo, key will land in /root/opnfv/mcp.rsa unless SSH_KEY is set via env var. While at it, bring back VM image storage dir param, allowing us to drop one more reference to /tmp. JIRA: FUEL-280 Change-Id: Id1cfa6dbe7b2e6f3915b22281da5957333718401 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-04lib.sh: Limit envsubst to certain variablesAlexandru Avadanii1-1/+2
This will prevent envsubst from expanding the SALT_REPO local variable in user-data.template. Change-Id: Idcc3ea827293ac0d3b4059776f90bbbcf807daec Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-04salt.sh: Execute deepclean before importing patchesJunaid Ali1-1/+1
Deepcleaning can be useful when deploying from an existing local repo Change-Id: I3908d6533c744da7cc863a250f4158c6c0e8e169 Signed-off-by: Junaid Ali <junaidali.yahya@gmail.com>
2017-09-01salt.sh, user-data: Add Saltstack arm64 repoAlexandru Avadanii2-2/+10
These changes will only be applied on AArch64, based on `uname -m`. While at it, add arm64 suppport to salt-master-setup.sh. Upstream commit [1] broke Salt bootstrap on AArch64, by introducing an architecture condition that is too strict to allow Debian package installation (even if we provide our own repo). Add "arm64" to the list of supported architectures. This needs to be done on the fly, as the bootstrap script is fetched using `curl` from <salt-master-setup.sh>. [1] https://github.com/saltstack/salt-bootstrap/commit/caa6d7d Change-Id: Id706a74a52ffe2f8b8c9dd3b9f70c78f35b2f745 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-31mcp/deploy/scripts: Move to git submoduleAlexandru Avadanii1-4/+2
Change-Id: I65d1f5680000011493bde17a249a87738ebfdd96 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-30Update salt init scripts sourceMichael Polenchuk1-3/+2
Change-Id: I269e397b78d55794b1c49bf582cc0e663cbe9ca6 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-08-29lib.sh: Reuse /tmp/mcp.rsa if presentAlexandru Avadanii1-0/+5
Long-term, /tmp/mcp.rsa should be moved to a persistent location [1], and made configurable via env var / other mechanisms. This will allow us to: - use an existing keypair (provided by end-user in expected path); - login to previous deployment machines (e.g. to cleanup UEFI boot entries before destroying the cluster and rebuilding it); - split deploy in re-entrant stages (salt master only, cluster nodes only; similar to old Fuel, where we could reuse old Fuel VM); [1] https://jira.opnfv.org/browse/FUEL-280 Change-Id: I1e53321ed1cfc217ff95e809c867fa3370c479c9 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-29[virtual] Apply ceilometer/aodh servicesMichael Polenchuk1-1/+1
* run ceilometer/aodh states * wrap common virtual cluster options * get the source image based on timestamps Change-Id: I88f1d63ed4a94eba4ec0a9cf33d36d51c75ae355 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-08-17Merge "Bring in baremetal support"Alexandru Avadanii2-24/+30
2017-08-17lib.sh: Fix mcp key owner for OPNFV Jenkins jobsAlexandru Avadanii1-1/+4
In case of non-root deploys (i.e. running `ci/deploy.sh` without sudo), the OPNFV Jenkins slave executes it under a user shell, under a root shell, e.g.: $ sudo sudo -u jenkins bash $ echo $USER jenkins $ echo $SUDO_USER root Adjust lib.sh to only use SUDO_USER when it is not "root". Change-Id: I6dd1c8a5dd3a3b02cfb760818d66e99f49709a5b Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-17Bring in baremetal supportAlexandru Avadanii2-24/+30
- ci/deploy.sh: fail if default scenario file is missing; - start by copying reclass/classes/cluster/virtual-mcp-ocata-ovs as classes/cluster/baremetal-mcp-ocata-ovs; - add new state (maas) that will handle MaaS configuration; - Split PXE network in two for baremetal: * rename old "pxe" virtual network to "mcpcontrol", make it non-configurable and identical for baremetal/virtual deploys; * new "pxebr" bridge is dedicated for MaaS fabric network, which comes with its own DHCP, TFTP etc.; - Drop hardcoded PXE gateway & static IP for MaaS node, since "mcpcontrol" remains a NAT-ed virtual network, with its own DHCP; - Keep internet access available on first interfaces for cfg01/mas01; - Align MaaS IP addrs (all x.y.z.3), add public IP for easy debug via MaaS dashboard; - Add static IP in new network segment (192.168.11.3/24) on MaaS node's PXE interface; - Set MaaS PXE interface MTU 1500 (weird network errors with jumbo); - MaaS node: Add NAT iptables traffic forward from "mcpcontrol" to "pxebr" interfaces; - MaaS: Add harcoded lf-pod2 machine info (fixed identation in v6); - Switch our targeted scenario to HA; * scenario: s/os-nosdn-nofeature-noha/os-nosdn-nofeature-ha/ - maas region: Use mcp.rsa.pub from ~ubuntu/.ssh/authorized_keys; - add route for 192.168.11.0/24 via mas01 on cfg01; - fix race condition on kvm nodes network setup: * add "noifupdown" support in salt formula for linux.network; * keep primary eth/br-mgmt unconfigured till reboot; TODO: - Read all this info from PDF (Pod Descriptor File) later; - investigate leftover references to eno2, eth3; - add public network interfaces config, IPs; - improve wait conditions for MaaS commision/deploy; - report upstream breakage in system.single; Change-Id: Ie8dd584b140991d2bd992acdfe47f5644bf51409 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com> Signed-off-by: Guillermo Herrero <Guillermo.Herrero@enea.com> Signed-off-by: Charalampos Kominos <Charalampos.Kominos@enea.com> Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-11Merge "lib.sh: AArch64: Use VGA video mode for guests"Alexandru Avadanii1-1/+9
2017-08-11lib.sh: AArch64: Use VGA video mode for guestsAlexandru Avadanii1-1/+9
On AArch64, there is no Cirrus video, so use VGA mode instead. On Debian systems, also fix a missing link for vgabios-stdvga.bin. Based on previous Armband work from [1]. [1] https://github.com/opnfv/armband/blob/danube.2.0/patches/fuel-library/ arm64-bug-fixes/0002-Install-vgabios-and-link-for-aarch64.patch Change-Id: Ia84ea5536b68e14993b719488375b0041183a767 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-11Revert "Switch distrib revision to testing"Michael Polenchuk1-1/+1
There is no much difference between nightly vs testing repos. This reverts commit 052a1626da71d232431b3a9faf576a76db61f10e. Change-Id: I40f60ddc2c9e1e59c7c4eb0be59ae19b00b7fb9b Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-08-09Synchronize down all of the dynamic modulesMichael Polenchuk1-0/+1
Sync all of the salt dynamic modules to make sure custom modules, states, grains & etc. are on the target nodes. Change-Id: I04f2ac0465d5381c4c07623e9383619d9b9731de Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-08-08Switch distrib revision to testingMichael Polenchuk1-1/+1
In order to avoid bleeding edge issues with nightly version of salt formulas, switch to "safe" testing repo source. Change-Id: I9d12f337c3952711cb8843943c23315795064016 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-08-01salt.sh: Drop upstream clone in favor of local gitAlexandru Avadanii1-10/+34
salt.sh currently clones the full Fuel@OPNFV git repo from upstream public mirror, preventing us from testing locally edited or new patches. Instead, bring back git submodule handling from old f_repos, clone and patch each submodule locally, then copy the whole parent repo over to cfg01. This is also a first step towards implementing offline deploy support. NOTE: This adds new deploy prerequisite packages: - git (for submodule clone/update); - make (for submodule patching); - rsync (for parent repo replication to cfg01); NOTE: Parent repository is expected to be a git repo, in order to work with git submodules. While at it, perform some minor related changes: - add deploy artifacts (ISOs, qcow2 files) to .gitignore, also used to filter-out such files during rsync to cfg01; - remove obsolete Fuel patches (old f_repos mechanism); - rename "reclass-system-salt-model" submodule; Change-Id: I6210d80d41010b2802e4f1b31acf249a18db7963 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-07-31mcp/scripts: shellcheck cleanupAlexandru Avadanii3-37/+43
Change-Id: I280d540b461341e14de654186016248eba4d3521 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-07-31ci/deploy.sh: Rework bridge arguments for MCPAlexandru Avadanii1-19/+35
Change-Id: I20d6dedeaa31f7986eaa35be49c5388c3fdb4b83 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-07-31lib.sh: Use host-passthrough when spawning VMsAlexandru Avadanii1-1/+3
virsh defaults to using "host-model" instead of "host-passthrough", which not only might lead to a small performance hit on x86, it is also causing compatibility issues with libvirt 3.x. Address this by explicitly requesting "host-passthrough" as the CPU model for virt-install. Change-Id: Ia7c5e8ff906c02fed2193c268cbb3594c5607bc6 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-07-31Set key owner to original user in case of sudoMichael Polenchuk1-1/+3
Change-Id: If964d36ad4f0cb6f1b2caf8544d4e35487f59c6c Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-07-28mcp/user-data: Pin saltstack repo to 2016.11Alexandru Avadanii1-2/+2
Latest saltstack repository recently pushed a new version of "salt-api" which breaks compatibility with current Fuel@OPNFV code. For now, let's pin saltstack repo to the most recent stable version. Change-Id: I2a7e11d4fb50a8c62220f3f29ded5955fa69aef0 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-07-26lib.sh: cleanup_vms: Also remove VMs that use UEFIAlexandru Avadanii1-2/+4
Pass `--nvram` to `virsh undefine`, so guest VMs booted using UEFI (OVMF on x86 or AAVMF on AArch64) can be destroyed. This does not affect VMs without UEFI. While at it, fix logical error in previous commit adding support for cleaning up stopped VMs. Change-Id: I44c20566469f29a8683d81e3641c69da572ccd4e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-07-26lib.sh: cleanup_vms: Also remove stopped MCP VMsAlexandru Avadanii1-1/+1
In case the cfg01 & co. VMs are in "shut off" state, `virsh --name` will ommit them, which leads to `cleanup_vms` leaving behind stale VMs. Add `--all` arg to list all VMs, including stopped ones. Change-Id: Ia550a10eb0b40138f87ade709336c2871a8b1cd8 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-07-19[bugfix]invalid user ‘jenkins’zhihui wu1-1/+1
See https://build.opnfv.org/ci/job/fuel-deploy-zte-pod1-daily-master/456/console In lib.sh, generate_ssh_key() assumes that "jenkins" is the current user name. But on zte-pod1, the ci user name isn't jenkins. It is better to use $USER replaced "jenkins". JIRA:FUEL-273 Change-Id: I6ec6847eccd055b8b4062dd202f8f0a24ba6dd73 Signed-off-by: zhihui wu <wu.zhihui1@zte.com.cn>
2017-07-15mcp: lib.sh: virt-install --noautoconsoleAlexandru Avadanii1-1/+2
When X server is available, virt-install tries opening virt-viewer automatically (for each newly created VM), which blocks the deployment until the user closes the virt-viewer window. Without X, virt-install just throws a warning and moves on. Fix both of the above by passing the "--noautoconsole" argument to virt-install. Change-Id: I7df839aa902f7629dec11d24905fc05d1070f9db Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>