aboutsummaryrefslogtreecommitdiffstats
path: root/ci
AgeCommit message (Collapse)AuthorFilesLines
2019-05-16[lib] Add uninstall/cleanup optionAlexandru Avadanii1-0/+7
When multiple installers are used on the same jumpserver, it is useful to have the ability of automatic cleanup after a previous deploy. Change-Id: Ib3249f53ee9d6b1ba2409dd71bd13480536faedc Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2019-05-07Merge "[docs] Update documentation for Hunter"Alexandru Avadanii1-2/+2
2019-05-07[docs] Update documentation for HunterCristina Pauna1-2/+2
Updated the documentation for the Hunter release plus one minor change of wording in the deploy script as we no longer install just Openstack Change-Id: I853f5536b0f4a89a8c20af0a9650372690ef7c99 Signed-off-by: Cristina Pauna <cristina.pauna@enea.com>
2019-04-08[virtual] Parameterize scenarios based on PDF/IDFAlexandru Avadanii1-2/+1
NOTE: only os-nosdn-nofeature-noha is parameterized for now. - move config drive & disk creation from prepare_vms to create_vms; - make default disk size(s) configurable based on scenario defaults and vPDF; * compute nodes require 2 disks to be defined in vPDF, since the pillar reclass model assumes /dev/vdb is reserved for cinder; * if multiple disks are defined in vPDF, they are created and attached accordinly (only ctl01 and cmp nodes are parameterized in this change; only for the os-nosdn-nofeature-noha scenario); - vCPU specifications are deduced based on vPDF (sockets, cores); * threads/core is hard set to 2 since vPDF does not have a key for it; * NUMA resources are distributed evenly based on the number of sockets configured in PDF; * no less than the mininum requirement for a scenario is allocated (e.g. if PDF specifies 2 cores, but the scenario requires at least 4 cores, the larger value will be used); - RAM is deduced based on PDF (but no less than the mininum req is allocated, e.g. if PDF specifies 2GB RAM for computes, but the scenario requires at least 8GB, the larger value will be used); Change-Id: I97188aa2a1006865b8429eb6483e10c76795f7d2 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2019-02-14[baremetal] Containerize MaaSAlexandru Avadanii1-2/+2
- replace mas01 VM with a Docker container; - drop `mcpcontrol` virsh-managed network, including special handling previously required for it across all scripts; - drop infrastructure VMs handling from scripts, the only VMs we still handle are cluster VMs for virtual and/or hybrid deployments; - drop SSH server from mas01; - stop running linux state on mas01, as all prerequisites are properly handled durin Docker build or via entrypoint.sh - for completeness, we still keep pillar data in sync with the actual contents of mas01 configuration, so running the state manually would still work; - make port 5240 available on the jumpserver for MaaS dashboard access; - docs: update diagrams and text to reflect the new changes; Change-Id: I6d9424995e9a90c530fd7577edf401d552bab929 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2019-02-07[docker] Add MaaS container build supportAlexandru Avadanii1-1/+11
Change-Id: I7709c9ca9e701b656447154919eb084a710f49af Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2019-01-29[deploy] Allow only operating system installAlexandru Avadanii1-6/+9
Extend one of the existing deployment arguments to allow the installation of only the operating system and infrastructure networks, skipping cloud setup. Change-Id: Ibc5d0f324ed15b66f809839cfce49a0324b6fe4d Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-12-14[docker] deploy.sh: Use tag for stable branchAlexandru Avadanii1-1/+3
Outside OPNFV Jenkins (i.e. when manually cloning the OPNFV Fuel repo and starting a deploy), the Docker tag used to default to 'latest' unless the user specifically set it to 'gambia'. Rely on 'defaultbranch' setting in .gitreview to determine the appropiate Docker tag. Change-Id: I7e6b0706597d84d7cd5dc077499da78031aa61af Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-11-21[docker] build: Allow cache invalidationAlexandru Avadanii1-4/+7
While at it, fix emoji issues with latest virtualenv [1]. JIRA: FUEL-398 [1] https://github.com/pypa/pipenv/issues/3223 Change-Id: Ice5937222bf75c1ddadc6b9f1994635bc10faf57 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-11-05[docs] Refresh for Gambia releaseAlexandru Avadanii2-82/+78
- s/Fuel@OPNFV/OPNFV Fuel/g; - added README files for ci/scenarios/patches directories; - refresh & simplify cluster overview diagrams; - unify labels across docs; - fix TOC numbering; - remove local labs PDF/IDF files, as they are merely duplicates of Pharos files included as a git submodule; JIRA: FUEL-397 Change-Id: I87f61938eeb67f13fd9205d5226a30f02e55d267 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-09-26[ha] Fix missing aodh_version paramAlexandru Avadanii1-1/+1
_param:aodh_version was lost during a recent refactor, bring it back. While at it, also make chown in entrypoint.sh recursive to prepare for non-sudo deployments. Fixes: c0de0902 Change-Id: I41b225c4a3f15269aa156a1c33412206beff6ee9 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-09-24[lib.sh] Split into multiple files for readabilityAlexandru Avadanii2-1/+4
lib.sh got pretty big over time, making it hard to maintain. Since most of the functions defined now in lib.sh are only required during build/deploy and not in state files, move them to a new file. While at it, prepare for running build/deploy as non-root and set a default connection string for virsh instead of using user specific config in ~/.config/libvirt/libvirt.conf, which caused end user experience issues in the past. Change-Id: Id8c2a8139e4bfdb99af2b0fad73b911ffa18ebea Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-09-22[build] Use python -m pip workaroundAlexandru Avadanii1-5/+5
pipenv depends on pip, which conflicts with some distro versions of pip, see bug [1]. While at it, workaround lib64 issues on CentOS by passing `--always-copy` when creating the venv [2]. [1] https://github.com/pyenv/pyenv/issues/1141 [2] https://github.com/pypa/pipenv/issues/1929 Change-Id: I6af1a5fdbe8d37f75ab8b09b050449ad432191ad Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-09-21[deploy] Collect logs only if cluster is deployedAlexandru Avadanii1-2/+2
Change-Id: Iaa60008e234a506b05e7b5249c85fd1d6fa63d56 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-09-21Merge "[state] virtual_init: Limit to mas01 on baremetal"Alexandru Avadanii1-0/+1
2018-09-20[state] virtual_init: Limit to mas01 on baremetalAlexandru Avadanii1-0/+1
`virtual_init` state file tries to ping all FN VMs, but that won't work on hybrid PODs since all FN VMs but mas01 require MaaS DHCP to be already configured (i.e. FN VMs in question will be reset after mas01 is fully configured). Limit virtual node queries in `virtual_init` to mas01 VM, as the rest of FN VMs will be handled via `baremetal_init` state. While at it, move _param:apt_mk_version def to common reclass to avoid an undef reference in NOHA hybrid deployments; set MCP_VCP to 0 for non-HA scenarios. JIRA: FUEL-385 Change-Id: I582bca6864e9bfed23baf26f9b66e6e95e986c58 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-09-20[infra] Bind mas01 mcpcontrol DHCP to MAAS_IPAlexandru Avadanii1-0/+1
Bring back MAAS_IP global env var and use it for mas01 VM IP addr in mcpcontrol network to prevent salt minion signature change. Partially-reverts: b666bc50 Change-Id: I5c7668393fe66287bd3ecdc75dd3195d5a89a8f3 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-09-19Merge "Use common way to detect dpdk feature"Alexandru Avadanii1-3/+2
2018-09-19Merge "[cleanup] Move default storage dir to /var/lib"Michael Polenchuk1-3/+8
2018-09-19Use common way to detect dpdk featureMichael Polenchuk1-3/+2
Change-Id: I55a3c10f275079b11b7456b28a2c846cb33c204a Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-09-18Make docker isolation chain go firstMichael Polenchuk1-1/+5
Restart docker service to refresh FORWARD chain and insert docker related rules on top. Change-Id: I971840f5979636c4ea8ae4d66a82982c24aa5f66 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-09-12[docker] Relax verify check for docker pullAlexandru Avadanii1-1/+2
JIRA: FUEL-383 Change-Id: I9203aa8d20def5b78d261f8c6847ddc576f0feb7 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-09-11[cleanup] Move default storage dir to /var/libAlexandru Avadanii1-3/+8
Since we switched to dockerized Salt master and mounting the full contents of the current git repo inside cfg01's /root/fuel dir, exposing temporary deploy artifacts should be avoided, i.e. the default temporary storage location (configurable via '-S' deploy arg) should be moved outside the git repo. JIRA: FUEL-383 Change-Id: I32f7197b018fd853867de42f5618650ac9022dc9 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-09-11[cleanup] FN VMs: Fold user-data templatesAlexandru Avadanii1-1/+0
While at it, retire obsolete MAAS_IP global variable and let mas01 VM get a DHCP address from virsh-managed mcpcontrol network. Change-Id: Ifd85dbcab10894a5d0d675d37f0c35f09776d9b4 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-09-10Merge "[nosdn-noha] Meet EPA testcases requirements (NUMA)"Alexandru Avadanii1-2/+11
2018-09-10Merge "[submodule] Bump Pharos for maas:machines sync"Alexandru Avadanii1-3/+3
2018-09-10[nosdn-noha] Meet EPA testcases requirements (NUMA)Dimitrios Markou1-2/+11
Install in a local directory a newer version of virt-manager to workaround obsolete Ubuntu versions lacking --cpu cellN.* support. This change only affects CPU cfg of virtual compute nodes in nosdn-nofeature-noha scenarios with: - set default cpu_topology to dual socket (2 cores, 2 sockets, 2 threads); - bump default RAM to 16GB; - define 2 NUMA cells, each with half the resources; To keep the old behavior available (single socket), a new deploy argument has been added (`-m`). The RAM change is not configurable via deploy args. NOTE: The CPU topology for virtual nodes should later be read from PDF instead of hardcoding it on a per-scenario basis in the installer. NOTE: Default 'ram' unit is MiB, while cellN.memory default unit is 'KiB'. JIRA: FUEL-385 Change-Id: I7ca268b0a2052524cb7187a5cf9b6fa8a382c9f9 Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com> Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-09-10[docker] Add MCP_DOCKER_TAG env varAlexandru Avadanii1-1/+2
Allow skipping docker pull for verify jobs by setting the new env var to 'verify'. JIRA: FUEL-383 Change-Id: If8e2f66b5ccdac5c3911eeabfc2ba9c0eba61093 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-09-06[submodule] Bump Pharos for maas:machines syncAlexandru Avadanii1-3/+3
Drop duplicate maas:machines definitions which could cause conflicts in rare corner cases. Slightly refactor j2 template expansion to make `conf.virtual.nodes` available during first stage. Change-Id: I04d56e346b12c6eb97da5c0c0ab1e3446e5fc1b8 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-29[deploy.sh] Cleanup deploy arguments, rework -ffAlexandru Avadanii1-15/+13
* refactor & extend `-f` deploy argument; * retire INFRA_CREATION_ONLY env var, duplicated by NO_DEPLOY_ENVIRONMENT; * `-F` and `-e` deploy arguments are now equivalent; JIRA: FUEL-383 Change-Id: Ifc1527fa1e7d7486d1b7600772e2c5de34b1e52c Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-29[docker] Switch to containerized Salt MasterAlexandru Avadanii1-6/+5
* Refactor OPNFV salt-formulas mechanism to resemble upstream git structure: - git submodules: add new submodule for each formula we patch; - create salt-formula-x directories for OPNFV formulas; - move mcp/metadata/service contents to their each formula subdir; - use `make patches-import` for patches previously handled by patch.sh; - retire patch.sh * states: add virtual_init: - mostly based on old salt.sh, which is now obsolete; - exclude salt-master service restart (it would kill the container); * scenarios: cleanup (rm cfg01 virtual node def), adopt virtual_init; * reclass: align our model with prebuilt container's Salt config: - drop linux:network pillar data (handled by Docker); - stop applying linux.system state on cfg01; - align salt user homedir; - drop salt-formula packages (preprovisioned); * minor plumbing in deploy.sh and lib.sh; JIRA: FUEL-383 Change-Id: I28708a9b399d3f19012212c71966ebda9d6fc0ac Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-25[build.sh] Fix quotes, missing deb arch for repoAlexandru Avadanii1-2/+2
JIRA: FUEL-383 Change-Id: Ie4374d44993bd738b90e9b6e357014d0c5657da0 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-25[build.sh] Use pipenv run instead of shellAlexandru Avadanii1-4/+4
JIRA: FUEL-383 Change-Id: Ia1b58f9e7eefd9f3efc4b5241ec496c545ccee25 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-24[build.sh] Add local python bin dir to PATHAlexandru Avadanii1-0/+2
JIRA: FUEL-383 Change-Id: I5596ed6139317d72fceae3af89e71181aefbd6a3 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-24[build.sh] Enable push, minor fixesAlexandru Avadanii1-2/+9
* default to enable pushing to public docker registry; * fix prerequisites path; * fix unknown terminal in Jenkins jobs; JIRA: FUEL-383 Change-Id: Ie27dfbca6965c218475060da390a82921799b044 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-23[docker build] Install OpenSSH serverAlexandru Avadanii1-1/+1
While at it, create the 'ubuntu' user with passwordless sudo and preinstall salt-formula-gnocchi (missing git repo workaround), as well as various useful packages (e.g. net-tools). JIRA: FUEL-383 Change-Id: I5902c37110331acfd3fd4fccb92104de7b5ece6b Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-21[submodule] Add docker-salt-formulas, ci/build.shAlexandru Avadanii1-0/+79
- add new git submodule pointing to upstream docker build scripts; - add patch extending Docker tags with an '-(arch)' suffix, aligning with OPNFV tagging requiremnts; - add <ci/build.sh> wrapper for starting Docker builds; - install build-specific distro package requirements, as well as pip-managed packages (e.g. pipenv); JIRA: FUEL-383 Change-Id: Id4fc886206d7eaf7e6d02810380f2391609ba405 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-21[deploy.sh] Install Docker if not presentAlexandru Avadanii1-0/+1
JIRA: FUEL-383 Change-Id: I19d27ca59a3f24d1bd66e39457a6ca267bccce19 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-21[jump req] Add build/deploy specific requirementsAlexandru Avadanii1-1/+1
Add support for different prerequisites depending on the current operation (docker build or cluster deploy). Leverage the new support to pre-install upcoming deps: - python-pip (build); - docker-compose (deploy); JIRA: FUEL-383 Change-Id: Ic3e6062b1943e3584f0b1f80d2e33b8812defced Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-19[deploy.sh] Prefix and export more env varsAlexandru Avadanii1-13/+14
Upcoming support for containerized Salt Master node requires access to certain variables during j2 interpolation, so also export: - MCP_STORAGE_DIR; - MCP_REPO_ROOT_PATH; JIRA: FUEL-383 Change-Id: I584c0bf8133b5ae6178d97da5b44d345e45a0222 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-13[jump] Enable jumbo frames for vnet devicesAlexandru Avadanii1-0/+1
By default, vnet devices have a MTU of 1500 on the host side, causing issue with larger packets traversing the bridges between guest VMs when guest VMs have jumbo frames enabled. JIRA: FUEL-336 JIRA: FUEL-367 JIRA: FUEL-382 [1] http://linuxaleph.blogspot.com/2013/01/ how-to-network-jumbo-frames-to-kvm-guest.html [2] https://packetpushers.net/udev/ Change-Id: I941ac9cf764e3b3fa2d6463be5363b5459775f29 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-08-03[lib.sh] Reset virtual nodes after MaaS installAlexandru Avadanii1-0/+5
For hybrid PODs (e.g. x86_64 jumpserver + control nodes, aarch64 baremetal compute nodes), the virtual nodes rely on MaaS DHCP to be up when the OS boots, so issue a `virsh reset` accordingly. Instead of checking for online nodes using `test.ping`, use `saltutil.sync_all` to also sync Salt state modules to the virtual nodes (usually handled by baremetal_init state in HA deploys). JIRA: FUEL-338 Change-Id: If689d057dc4438102c3a7428a97b9638e21bfdc5 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-06-25[noha] Adopt vPDF usageAlexandru Avadanii1-5/+3
Previously, all virtual deployments relied on lab configuration from <./mcp/config> using 'local-virtual1' as POD name. To keep using that default sample config, one can still pass the appropiate arguments to <ci/deploy.sh> as in: $ ./ci/deploy.sh -l local -p virtual1 [...] This changed introduces support for vPOD-specific lab config, e.g.: $ ./ci/deploy.sh -l ericsson -p virtual4 Change-Id: I6ecb8e71d7e923e4b29b4ab0693c1afa5e19b582 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-05-08[virtual] Bring in OpenDaylight DPDK scenarioMichael Polenchuk1-1/+2
Change-Id: If77ac85fa86e0a1a18c0cc2abff77d876cdb9e93 Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
2018-03-07[novcp] Add deploy argument `-N` (experimental)Alexandru Avadanii1-4/+11
Instead of duplicating scenarios for NOVCP, allow it to be specified using a new deploy argument, `-N`. Things are getting convoluted, so instead of creating dedicated '*_pdf.yml.j2' files for each group of similar features, apply the templating in-place and rename all affected files to ".yml.j2". This breaks .gitignore assumption of hiding only "*_pdf.yml" files, so extend (manually) the <mcp/reclass/classes/cluster/.gitignore> with `git ls-files --exclude-standard -o` after an expansion. - ha: move nfv.cpu_pinning to j2, conditioned by 'baremetal'; - ha: add cmp00* vnode definitions (hugepages need more RAM); - labs/local: enable hugepage params for non-dpdk noha; - salt.sh: add route_wrapper to all non-infra VMs; This change extends novcp support to all HA scenarios. Change-Id: I7a80415ac33367ab227ececb4ffb1bc026546d36 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-07[vnode parsing] Move core logic to j2Alexandru Avadanii1-47/+13
j2/python is easier to read and manipulate strings, although it does need some special care about undefined dict keys. With this in place, deploy.sh only contains the higher level logic for the deployment process. - merge arch-specific default configuration files into a singular file with arch name as main dict key of old config (also avoids creating duplicate 'virtual' YAML keys in $LOCAL_PDF); - move template handling to separate <lib_template.sh>; - decouple tight bash ordering of scenario expansion -> parse_yaml -> variable export (e.g. CLUSTER_DOMAIN) -> re-use in cluster j2s; however we can't parse *all* j2s in one go, as scenario j2s might expand to YAMLs needed while expanding cluster j2; - split `do_templates` into separate functions for each stage, with no coupling between them other then call order; Change-Id: I4b5e804094c00e5e918caf769fd85fa52181ad76 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-04[jump req] Move pkg installation to lib.shAlexandru Avadanii1-12/+1
Change-Id: I687b73b256aca78c9d41d4bcd49bfbde51278b51 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-03-04[deploy.sh] Factor out jump bridge parsing to j2Alexandru Avadanii1-14/+2
Change-Id: Ie4d8e70866d533d195a6e80cdfecbdb00a3027ce Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-13[patch] MaaS: mcp.rsa.pub to auth keys via pillarAlexandru Avadanii1-1/+4
Drop one questionable patch responsible for MaaS node authorized keys to include mcp.rsa.pub by reading the contents of authorized keys on mas01, assuming mcp.rsa.pub will be on the first line. Instead, export the contents of the public key using a shell env var during deploy, which gets expanded via maas_pdf j2 template into a reclass param, leveraging existing salt-formula-maas sshprefs mechanism for delivering the key to MaaS. Since we require the public key to exist before expanding templates, move `generate_ssh_key` call outside the current infrastructure handling block, allowing it to execute during all `deploy.sh` calls, even for dry-runs. Change-Id: I0f53b0f764a2fafd292e0ffd399c284acf61bd30 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
2018-02-07[jump] Add simple check for required Linux bridgesAlexandru Avadanii1-0/+3
- MaaS requires PXE/admin to be a Linux bridge; - if virtual nodes are present, they should be hooked to a proper Linux bridge for the Public network, but only throw a warning if not (and create a mock public virsh network instead); - if both virtual and baremetal nodes are present, Public bridge is indirectly mandatory (we can't mock it); JIRA: FUEL-339 Change-Id: Idfe99d66c49eadc56cb3d94ca4db3467fb76d388 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>