aboutsummaryrefslogtreecommitdiffstats
path: root/ci
AgeCommit message (Collapse)AuthorFilesLines
2017-12-04Rename reclass models to add "-ha" or "-noha"Alexandru Avadanii1-3/+1
Parse all reclass j2 templates, not only common + current scenario (useful when adding new scenarios later). JIRA: FUEL-310 Change-Id: I8e87af702f83c42cb8f766bf6f121449aa5f2c26 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-29Merge "AArch64: base image: pre-install salt-minion"Michael Polenchuk1-2/+2
2017-11-28[virtual] Update openstack version to pikeMichael Polenchuk1-1/+1
Change-Id: I1df0228cb44bf9122aaf93dd25fc16a0d26a5240 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2017-11-27AArch64: base image: pre-install salt-minionAlexandru Avadanii1-2/+2
While at it, rename apt repo in foundation node user-data template from "salt" to "saltstack", to align with reclass model naming. Change-Id: I5b216492349ae187b568884b1ab4046c52b1c6b2 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-27Add pre-{install,purge} support for base imageAlexandru Avadanii1-2/+21
Extend <lib.sh> and its invocation from <ci.deploy.sh> with support for modifying foundation node VMs base image prior to using it with: - additional APT GPG keys; - additional APT repos; - packages to pre-install; - packages to pre-remove; - (non-configurable) cloud init datasource via NoCloud only, so VCP VMs won't wait for metadata service; While at it, re-use the resulting image as a base for another round of pre-patching (same operations as above are supported) to provide a base image for VCP VMs. Add AArch64-specific configuration based on new mechanisms: - pre-install linux-image-generic-hwe-16.04-edge (and headers) for foundation node and VCP (common) image (also requires new repo and its key); - pre-install cloud-init for VCP image (it should already be installed, but script needs non-empty config for VCP to create the VCP image and transfer it over to Salt Master); NOTE: cloud-init is required on VCP VMs for DHCP on 1st iface. JIRA: FUEL-309 Change-Id: I7dcaf0ffd9c57009133c6d339496ec831ab14375 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-21ci/deploy.sh: Add new `-E` arg for env eraseAlexandru Avadanii1-4/+14
NOTE: In order to undefine VCP VMs with NVRAM (e.g. AArch64 VMs using AAVMF), an additional parameter should be passed to libvirt by Salt virt core module (equivalent to `virsh undefine --nvram`). While at it, pass CI_DEBUG, ERASE_ENV enviroment variables to state execution, and stop force-applying patches. Also refactor the rsync between foundation node and Salt master, so the whole git repo is copied as </root/opnfv>, and <root/fuel> becomes a link to it; useful for Armband, where 'fuel' is a git submodule. Fix .git paths after rsync, so git submodules work as expected in cfg01 repos. JIRA: FUEL-307 Change-Id: Ic62f03e786581c019168c50ccc50107238021d7f Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-10AArch64: deploy.sh: Add AAVMF install prereqAlexandru Avadanii1-0/+6
Change-Id: I7b3bb3eac1b4665dc9c08eaf6f5af336cde31f87 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-10deploy.sh: Check hypervisor after prereq installAlexandru Avadanii1-7/+9
On a clean system (i.e. no libvirt installed), the prerequisites should be installed first, before checking for hypervisor access. While at it, add 'libvirt' to the list of prerequisites, as it is not automatically required by the others. Add a note in the documentation about libvirtd service not being automatically started after installation on CentOS. Change-Id: I29d94d4a56cd823362811e8fc9c076c83fb6c04c Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-11-09[baremetal] package: install python-yamlTaseer Ahmed1-2/+2
- Required for PDF parsing Change-Id: I95205e62efab3e9920687866d3a53a135828a218 Signed-off-by: Taseer Ahmed <taseer94@gmail.com>
2017-10-27Merge "ci/deploy.sh: Add PDF parsing dependencies"Alexandru Avadanii1-0/+8
2017-10-27Merge "lib.sh: Fix compatibility with bash 4.2"Alexandru Avadanii1-8/+9
2017-10-27ci/deploy.sh: Add PDF parsing dependenciesAlexandru Avadanii1-0/+8
The following packages are required for PDF parsing: - python - python-ipaddress - python-jinja2 Note that `python-ipaddress` is not available on older Ubuntu distributions (e.g. Trusty), in which case `pip` should be used to install it. An alternative would be to use python3 packages. Update the installation instructions to specify minimum expected requirements on the jump server. JIRA: FUEL-292 Change-Id: I19fdcc163dcfdee45ac6fa3f5e09a28638d6035c Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-27Merge "AArch64: Switch jumpserver VMs to virtio video"Alexandru Avadanii1-6/+0
2017-10-27lib.sh: Fix compatibility with bash 4.2Alexandru Avadanii1-8/+9
JIRA: FUEL-296 Change-Id: Ide9f9333fe9b44ff6b78678064f8e67f05aabd42 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-26AArch64: Switch jumpserver VMs to virtio videoAlexandru Avadanii1-6/+0
Drop vgabios dependency by switching video from VGA to virtio for all VMs spawned on the jumpserver. NOTE: This requires virtualization packages on the jumpserver to be up to date (e.g. libvirt, QEMU). JIRA: ARMBAND-306 Change-Id: I73913e1ae8584f4e73b92994f78f7ec363cba3ec Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-26Add missing installation dependencyTaseer Ahmed1-2/+2
- uuidgen is required for installation - Add installation package for both Debian and RHEL based linux distribitions Change-Id: I9c9bf04e64dc6be1b4e8fb92aea531eccbea33a5 Signed-off-by: Taseer Ahmed <taseer94@gmail.com>
2017-10-18[baremetal] PDF-based network configAlexandru Avadanii1-0/+14
This change extends current PDF/IDF support with: - dynamic interface / vlan definition; - dynamic interface and/or bridge allocation of installer networks on top of interfaces (physical or vlans); This allows us to drop hardcoded interface names and vlan / bridge configuration in favor of a runtime determined model based on PDF/IDF. For now, we duplicate common jinja variable definitions in each template, but this will later be moved to a common include file. JIRA: FUEL-275 Change-Id: Ia80a66dbdf898b0bd0a4fb99d069ce9ebe33fa65 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-15states: Break on error, retry states up to 5 timesAlexandru Avadanii1-18/+33
While applying scenario states, break on error, and retry failed state up to 5 times. Apply the same behavior for `salt.sh`. Add new deploy parameter, '-D', backed up by 'CI_DEBUG' env var, which gates deploy sh scripts logging (set -x). Also extend '-f' deploy parameter, allowing it to be specified more than once; the first occurence will skip infra VM creation, but still sync reclass & other config from local repo, while a second occurence will also disable config sync. To prevent glusterfs client state from failing due to non-existent nova user/group, move it after nova:compute's nova state is applied. Change-Id: I234e126e16be0e133d878957bd88fed946955de8 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-14Add license headers where missingAlexandru Avadanii1-2/+1
While at it, compact 'set' into bash shebang where possible and add `make patches-copyright` target to simplify adding patch license headers. Change-Id: I0c841de72e5709e5eef915a52c5ec4a7fc0f7c37 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-14states: Split virtual_control_plane from maasAlexandru Avadanii1-1/+1
We should eventually also support baremetal deploys without a virtualized control plane (VCP), so decouple MaaS provisioning from VCP provisioning. While at it, move "wait_for" bash function from maas state to common library file, lib.sh. Change-Id: I32c33135655cb6aceae901a5f92b51265a8c84b4 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-13ci/deploy.sh: Add installation succesful messageAlexandru Avadanii1-6/+9
Change-Id: Id366854c4ccb3b835f4a54ddea924a3a2cc30c7d Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-08ci/deploy.sh: Drop unsupported optionsAlexandru Avadanii1-34/+5
Drop support for unsupported arguments: - timeout; - iso URI; While at it, disable shell debugging during PDF parsing to prevent sensitive information leaking via debug logs. Change-Id: I6b40f3e3749a3e162326728e670a176c8fa3c4bb Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-03ci/deploy.sh: Move global vars to separate fileAlexandru Avadanii1-11/+1
log.sh should also work as standalone script, so factor out global variables from <ci/deploy.sh> into <mcp/scripts/globals.sh>, which should be sourced by log.sh when required variables are not set. Change-Id: I5a5bb85ee5835f567d6910b11c137d95e2147714 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-10-03Identify jump host bridges based on IDF / PDF netsAlexandru Avadanii1-15/+50
- minor refactor of runtime templates parsing to allow var expansion; - parse <pod_config.yml> into shell vars, match dynamically networks from PDF to IP addresses on bridges of current jumphost; - keep old '-B' parameter in <ci/deploy.sh>, use it for providing fallback values in case there's no bridge name specified via IDF and no IP on the jumphost for one or more of the PDF networks; - re-enable dry-run to ease testing of the above; - add sample 'idf-pod1.yaml' to <mcp/config/labs/local>; The new behavior will try to determine the jump host bridge names: 1. Based on IDF mapping, if available 2. Based on PDF network matching with IP addrs on jumphost; 3. Fallback to values passed via '-B'; 4. Fallback to default values hardcoded in the deploy script; Later, we will drop MaaS network env vars in favor of PDF vars, once the PDF template is generating them. Change-Id: If9cd65d310c02965b2e2bfa06a0d7e0f97f1dd48 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-30Enable options for customizing deploy workflowYujun Zhang1-33/+40
Change-Id: I1011734895771fdaf653c59348a9ce0aeeff47a7 Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
2017-09-27PDF parsing supportAlexandru Avadanii1-10/+36
- add new git submodule pointing to OPNFV Pharos; - use Pharos 'generate_config.sh' to parse the PDF using the Fuel installer adapter and generate <pod_config.yml> dynamically; - build <pod_config.yml> outside current git repo and sync separately to prevent sensitive data leak; - add <pod1.yaml> PDF sample based on LF-POD2, should be used with 'ci/deploy.sh -l local -p pod1'; Change-Id: I4e1b95f180bcd5ade5d86f516628eb8edbe64b1c Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-12reclass, states: Parametrize runtime configurationAlexandru Avadanii1-0/+10
mcpcontrol virsh network, as well as MaaS PXE network are installer specific, and not POD specific. Therefore, these should be easily parametrized without the PDF, using only installer inputs (e.g. env vars passed via Jenkins). - add new <all-mcp-ocata-common.opnfv.runtime> reclass class; - parametrize at runtime new reclass class based on global vars; - factor out MaaS deploy address / config using new mechanism; - parametrize at runtime virsh network definitions based on template; - add new "maas.pxe_route" sls for configuring routing on cfg01; - replace env vars with the new sls in "maas" state; NOTE: baremetal parametrization will be handled later. Change-Id: Ifd61143d818fb088b3f4395388ba769bbc49156e Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-11salt master, maas: Move mcpcontrol to 10.20.0.0/24Alexandru Avadanii1-2/+2
Use INSTALLER_IP Jenkins param instead of SALT_MASTER_IP, allowing us to drop SALT_MASTER_IP completely from releng. mcpcontrol IP changes: - 192.168.10.100 becomes 10.20.0.2 (align with legacy Fuel master); - 192.168.10.3 becomes 10.20.0.3 (baremetal MaaS address); JIRA: FUEL-285 Change-Id: I6e2d44c3a8b43846196bd64191735214167a76ce Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-11README files: Convert from markdown to RSTAlexandru Avadanii1-25/+33
Change-Id: I176f583e8adb7ba717932de7c27283f696ded548 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-11Merge "Add & fix licensing in README.rst files, align ext"Alexandru Avadanii1-9/+3
2017-09-11Merge "scenario: Add AArch64 defaults"Michael Polenchuk1-4/+11
2017-09-11Merge "log.sh: Collect /var/log from cluster nodes"Michael Polenchuk1-5/+8
2017-09-11Add & fix licensing in README.rst files, align extAlexandru Avadanii1-9/+3
Rename all variations of README files to README.rst. Fix license headers according to OPNFV Contribution Guidelines [1]. Markdown syntax will be switched to rst in an ulterior change. [1] https://wiki.opnfv.org/display/DEV/Contribution+Guidelines Change-Id: I2c71fd62b38f218545507c79c39e07dbd35f7c88 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-10scenario: Add AArch64 defaultsAlexandru Avadanii1-4/+11
Move scenario defaults.yml out of virtual/baremetal dirs, unifying them into a single, arch-specific defaults configuration file. Add AArch64 equivalent config, and arch-qualify default filename. For AArch64, bump VCPU number from 2 to 6. Change-Id: If9b39850af96e4c46e868b1d532bd301f9fd9f26 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-09log.sh: Collect /var/log from cluster nodesAlexandru Avadanii1-5/+8
Bring back <ci/deploy.sh> parameter `-L`, which is already passed by releng deploy script. Since we need to enable file_recv on salt master, do so by adding a new class in <virtual-mcp-ocata-common/infra/config.yml>, which will be inherited by all scenario reclass models. JIRA: FUEL-288 Change-Id: I9b649315984fb8580b63883d0e06c1e1efd11057 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-08bash scripts: Fix remaining shellcheck warn/errsAlexandru Avadanii1-1/+1
Fix/silence all shellcheck errors, except for scripts in <prototypes/sfc_tacker>. Change-Id: Idc317cdba0f69b78299f2d3665e72ffc19dd8af5 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-07Move mcp.rsa to /var/lib/opnfvAlexandru Avadanii1-5/+2
JIRA: FUEL-280 Change-Id: I1e07b0e1597b2a1e4a92e1274f89dda62cb81bb8 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-05cleanup: Rename ODL-L3 to ODL, drop ODL-L2Alexandru Avadanii1-3/+3
JIRA: FUEL-279 Change-Id: Ia8bed5e09937a1ffd29a15d40f610d4b0c8f0a17 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-05ci/deploy.sh: Remove extra '-s' in image dir pathAlexandru Avadanii1-1/+1
Change-Id: Iabadd8565b7fcac6d3361200f30310391daf6648 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-05Merge "build, deploy: Remove obsolete Fuel@Openstack code"Alexandru Avadanii3-327/+19
2017-09-05ci/deploy.sh, lib.sh: Align SSH_KEY with JenkinsAlexandru Avadanii1-10/+13
Jenkins defines a build param also called SSH_KEY, which holds the full path to our MCP private RSA key, as opposed to Fuel@OPNFV, which uses the file basename as SSH_KEY. Switch to using full path in SSH_KEY, while also moving the key outside /tmp. NOTE: When running `ci/deploy.sh` with sudo, key will land in /root/opnfv/mcp.rsa unless SSH_KEY is set via env var. While at it, bring back VM image storage dir param, allowing us to drop one more reference to /tmp. JIRA: FUEL-280 Change-Id: Id1cfa6dbe7b2e6f3915b22281da5957333718401 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-09-03build, deploy: Remove obsolete Fuel@Openstack codeAlexandru Avadanii3-327/+19
JIRA: FUEL-278 Change-Id: I30c04c325de5ac97aee172386de43201988646c5 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-23ci/deploy.sh, states: bash debug, continue on errAlexandru Avadanii1-1/+1
Since we don't `set -e` in state files, applying each state will always succeed unless the last instruction in the state fails. Make this uniform by always succeeding in applying the state. While at it, enable bash debugging logs, for better readability of deploy log files. Change-Id: I3cf4886f6d73c6fd1380df1a4e1413334bec1701 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-18ci/deploy.sh: yum install --skip-brokenAlexandru Avadanii1-2/+2
Change-Id: I8098eff73ab15fe58cb357db99df76e1bd160652 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-17Bring in baremetal supportAlexandru Avadanii1-4/+13
- ci/deploy.sh: fail if default scenario file is missing; - start by copying reclass/classes/cluster/virtual-mcp-ocata-ovs as classes/cluster/baremetal-mcp-ocata-ovs; - add new state (maas) that will handle MaaS configuration; - Split PXE network in two for baremetal: * rename old "pxe" virtual network to "mcpcontrol", make it non-configurable and identical for baremetal/virtual deploys; * new "pxebr" bridge is dedicated for MaaS fabric network, which comes with its own DHCP, TFTP etc.; - Drop hardcoded PXE gateway & static IP for MaaS node, since "mcpcontrol" remains a NAT-ed virtual network, with its own DHCP; - Keep internet access available on first interfaces for cfg01/mas01; - Align MaaS IP addrs (all x.y.z.3), add public IP for easy debug via MaaS dashboard; - Add static IP in new network segment (192.168.11.3/24) on MaaS node's PXE interface; - Set MaaS PXE interface MTU 1500 (weird network errors with jumbo); - MaaS node: Add NAT iptables traffic forward from "mcpcontrol" to "pxebr" interfaces; - MaaS: Add harcoded lf-pod2 machine info (fixed identation in v6); - Switch our targeted scenario to HA; * scenario: s/os-nosdn-nofeature-noha/os-nosdn-nofeature-ha/ - maas region: Use mcp.rsa.pub from ~ubuntu/.ssh/authorized_keys; - add route for 192.168.11.0/24 via mas01 on cfg01; - fix race condition on kvm nodes network setup: * add "noifupdown" support in salt formula for linux.network; * keep primary eth/br-mgmt unconfigured till reboot; TODO: - Read all this info from PDF (Pod Descriptor File) later; - investigate leftover references to eno2, eth3; - add public network interfaces config, IPs; - improve wait conditions for MaaS commision/deploy; - report upstream breakage in system.single; Change-Id: Ie8dd584b140991d2bd992acdfe47f5644bf51409 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com> Signed-off-by: Guillermo Herrero <Guillermo.Herrero@enea.com> Signed-off-by: Charalampos Kominos <Charalampos.Kominos@enea.com> Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-11lib.sh: AArch64: Use VGA video mode for guestsAlexandru Avadanii1-0/+6
On AArch64, there is no Cirrus video, so use VGA mode instead. On Debian systems, also fix a missing link for vgabios-stdvga.bin. Based on previous Armband work from [1]. [1] https://github.com/opnfv/armband/blob/danube.2.0/patches/fuel-library/ arm64-bug-fixes/0002-Install-vgabios-and-link-for-aarch64.patch Change-Id: Ia84ea5536b68e14993b719488375b0041183a767 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-06Merge "ci/deploy.sh: Allow non-root deploys"Alexandru Avadanii1-5/+10
2017-08-01mcp/config: Move scenario defs to separate dirAlexandru Avadanii1-3/+7
Prepare for adding baremetal scenarios by moving existing scenario definitions (intended for deploys on virtual PODs) to their own subdirectory. Change-Id: Iaa75e0b0a9876e4f263fc6bbae796d97848ef99b Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-01ci/deploy.sh: Allow non-root deploysAlexandru Avadanii1-5/+10
Currently, deploying Fuel@OPNFV requires the `ci/deploy.sh` to be ran as root user, for: - hypervisor access (virsh); - installing prerequisite packages; We can support non-root deploys if we relax above restriction into: - current user should have hypervisor access beforehand (new check added); - current user has passwordless sudo rights, for installing packages; Later, we should move the package installation out of the deploy script (`ci/deploy.sh`), making the whole deploy possible without sudo, only hypervisor access. After this change, the old way (sudo ci/deploy.sh) should work just like it used to before, so no regression/behavior change should be introduced with this. Change-Id: Ib2ea96aa7b6d67ba0606318407a8adeb51e6969d Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2017-08-01ci/deploy.sh: Fix "notify" on dumb terminalsAlexandru Avadanii1-1/+1
Dumb terminals (which do not support colors), like "vt220", make `tput setaf` return with an error code, hence bypassing the actual print. Change-Id: Id838583beb30a810aec478ed80d452c6f3203e0f Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>