Age | Commit message (Collapse) | Author | Files | Lines |
|
On some aarch64 platforms (e.g. ThunderX 1), lvcreate manifests some
spurious timing issues resulting in incomplete/corrupted LVM thin
creation and eventually to transaction ID mismatch between userspace
and kernel space.
This eventually leads to cinder-volume issues, either when creating
the thin storage pool (vgroot-pool) and/or when creating the LVs
inside said pool.
The issue manifests spuriously on Ubuntu Bionic + UCA, so until a
working combination of userspace/kernel is found, work around this
by bumping the kernel package to hwe-18.04 (kernel 5.0),
effectively bypassing the timing issues during volume creation.
This affects all cluster machines (both HA and NOHA scenarios,
baremetal and virtual, x86_64 and aarch64, baremetal and virtualized
nodes).
Note: Ubuntu Bionic cloud image partition handling requires e2fsprogs
1.43, not currently available on Ubuntu Xenial / CentOS 7.
Change-Id: I839e03080104c391fe18185b9544c9df43c114e6
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Change-Id: I9c1e97144ffd46040d32a0edf8253fc393b73c89
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
- reclass: iec: CentOS compatibility changes:
* drop `proto: static` in favor of letting the linux formula set
the appropiate default based on target OS;
* replace `proto: manual` with `proto: none` on RHEL systems;
* system.file: Avoid using non-existing `shadow` group for system
files;
* load br_netfilter kernel module to avoid `linux.network` state
failures;
* disable `at`, `cron` due to incomplete defaults in
salt-formula-linux (since we don't use them on iec nodes anyway);
- jumpserver/VCP VMs: centos: enable predictable interface names:
* CentOS cloud image defaults to old 'eth' naming scheme;
* add necessary kernel boot options via linux state;
* cleanup auto-generated udev rules for old eth interface names;
- salt-formula-linux: network: RHEL: Set bridge for member interfaces
* Find the bridge containing the interface being currently
configured (if any) and pass it to the `network.managed` Salt call;
- deploy.sh: Add new deploy argument `-o` for specifying the operating
system to preinstall on jumpserver and/or VCP VMs;
* defaults to 'ubuntu1604';
* only iec scenarios will also support 'centos' for now;
- user-data: minor tweaks for CentOS compatability:
* use `systemctl` instead of `service` utility;
* explicitly enable `salt-minion` service, since it defaults to
disabled on RHEL systems;
* explicitly call `ldconfig` to work around stale cache on RHEL,
preventing `salt-minion` from using OpenSSL library;
- states: virtual_init: Skip non-existing sysctl options on CentOS:
* CentOS currently uses a 3.x kernel which lacks certain sysctl
options that were only introduced in 4.x kernels, so skip them;
- state: akraino_iec: Add centos support:
* move iec repo to `/var/lib/akraino/iec` on both Salt Master and
cluster nodes;
- scenario defaults: Add CentOS configuration:
* OS-dependent configuration split;
* CentOS base image, default packages etc.;
- AArch64 deploy requirements: Add `xz` dependency
* CentOS AArch64 cloud image is archived using xz, install xz tools
for decompression;
- xdf_data: Make yaml parsing OS agnostic:
* rename `apt` to `repo` where appropiate;
* OS-dependent configuration parsing;
- lib_jump_deploy: CentOS handling changes:
* skip filesystem resize of cloud image for CentOS;
* add repo handling, package intallation/removal handling for CentOS;
* unxz base image if necessary (CentOS AArch64 cloud image);
Change-Id: Ic3538bacd53198701ff4ef77db62218eabc662e7
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
When multiple installers are used on the same jumpserver, it is
useful to have the ability of automatic cleanup after a previous
deploy.
Change-Id: Ib3249f53ee9d6b1ba2409dd71bd13480536faedc
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
|
|
Updated the documentation for the Hunter release plus one minor
change of wording in the deploy script as we no longer install
just Openstack
Change-Id: I853f5536b0f4a89a8c20af0a9650372690ef7c99
Signed-off-by: Cristina Pauna <cristina.pauna@enea.com>
|
|
NOTE: only os-nosdn-nofeature-noha is parameterized for now.
- move config drive & disk creation from prepare_vms to create_vms;
- make default disk size(s) configurable based on scenario defaults
and vPDF;
* compute nodes require 2 disks to be defined in vPDF, since the
pillar reclass model assumes /dev/vdb is reserved for cinder;
* if multiple disks are defined in vPDF, they are created and
attached accordinly (only ctl01 and cmp nodes are parameterized
in this change; only for the os-nosdn-nofeature-noha scenario);
- vCPU specifications are deduced based on vPDF (sockets, cores);
* threads/core is hard set to 2 since vPDF does not have a key
for it;
* NUMA resources are distributed evenly based on the number of
sockets configured in PDF;
* no less than the mininum requirement for a scenario is allocated
(e.g. if PDF specifies 2 cores, but the scenario requires at
least 4 cores, the larger value will be used);
- RAM is deduced based on PDF (but no less than the mininum req is
allocated, e.g. if PDF specifies 2GB RAM for computes, but the
scenario requires at least 8GB, the larger value will be used);
Change-Id: I97188aa2a1006865b8429eb6483e10c76795f7d2
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- replace mas01 VM with a Docker container;
- drop `mcpcontrol` virsh-managed network, including special handling
previously required for it across all scripts;
- drop infrastructure VMs handling from scripts, the only VMs we still
handle are cluster VMs for virtual and/or hybrid deployments;
- drop SSH server from mas01;
- stop running linux state on mas01, as all prerequisites are properly
handled durin Docker build or via entrypoint.sh - for completeness,
we still keep pillar data in sync with the actual contents of mas01
configuration, so running the state manually would still work;
- make port 5240 available on the jumpserver for MaaS dashboard access;
- docs: update diagrams and text to reflect the new changes;
Change-Id: I6d9424995e9a90c530fd7577edf401d552bab929
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Change-Id: I7709c9ca9e701b656447154919eb084a710f49af
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Extend one of the existing deployment arguments to allow the
installation of only the operating system and infrastructure networks,
skipping cloud setup.
Change-Id: Ibc5d0f324ed15b66f809839cfce49a0324b6fe4d
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Outside OPNFV Jenkins (i.e. when manually cloning the OPNFV Fuel
repo and starting a deploy), the Docker tag used to default to
'latest' unless the user specifically set it to 'gambia'.
Rely on 'defaultbranch' setting in .gitreview to determine the
appropiate Docker tag.
Change-Id: I7e6b0706597d84d7cd5dc077499da78031aa61af
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
While at it, fix emoji issues with latest virtualenv [1].
JIRA: FUEL-398
[1] https://github.com/pypa/pipenv/issues/3223
Change-Id: Ice5937222bf75c1ddadc6b9f1994635bc10faf57
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- s/Fuel@OPNFV/OPNFV Fuel/g;
- added README files for ci/scenarios/patches directories;
- refresh & simplify cluster overview diagrams;
- unify labels across docs;
- fix TOC numbering;
- remove local labs PDF/IDF files, as they are merely duplicates of
Pharos files included as a git submodule;
JIRA: FUEL-397
Change-Id: I87f61938eeb67f13fd9205d5226a30f02e55d267
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
_param:aodh_version was lost during a recent refactor, bring it back.
While at it, also make chown in entrypoint.sh recursive to prepare
for non-sudo deployments.
Fixes: c0de0902
Change-Id: I41b225c4a3f15269aa156a1c33412206beff6ee9
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
lib.sh got pretty big over time, making it hard to maintain.
Since most of the functions defined now in lib.sh are only required
during build/deploy and not in state files, move them to a new file.
While at it, prepare for running build/deploy as non-root and
set a default connection string for virsh instead of using
user specific config in ~/.config/libvirt/libvirt.conf, which
caused end user experience issues in the past.
Change-Id: Id8c2a8139e4bfdb99af2b0fad73b911ffa18ebea
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
pipenv depends on pip, which conflicts with some distro versions of
pip, see bug [1].
While at it, workaround lib64 issues on CentOS by passing
`--always-copy` when creating the venv [2].
[1] https://github.com/pyenv/pyenv/issues/1141
[2] https://github.com/pypa/pipenv/issues/1929
Change-Id: I6af1a5fdbe8d37f75ab8b09b050449ad432191ad
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Change-Id: Iaa60008e234a506b05e7b5249c85fd1d6fa63d56
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
|
|
`virtual_init` state file tries to ping all FN VMs, but that won't
work on hybrid PODs since all FN VMs but mas01 require MaaS DHCP to
be already configured (i.e. FN VMs in question will be reset after
mas01 is fully configured).
Limit virtual node queries in `virtual_init` to mas01 VM, as the rest
of FN VMs will be handled via `baremetal_init` state.
While at it, move _param:apt_mk_version def to common reclass to
avoid an undef reference in NOHA hybrid deployments; set MCP_VCP to
0 for non-HA scenarios.
JIRA: FUEL-385
Change-Id: I582bca6864e9bfed23baf26f9b66e6e95e986c58
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Bring back MAAS_IP global env var and use it for mas01 VM IP addr
in mcpcontrol network to prevent salt minion signature change.
Partially-reverts: b666bc50
Change-Id: I5c7668393fe66287bd3ecdc75dd3195d5a89a8f3
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
|
|
|
|
Change-Id: I55a3c10f275079b11b7456b28a2c846cb33c204a
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
Restart docker service to refresh FORWARD chain
and insert docker related rules on top.
Change-Id: I971840f5979636c4ea8ae4d66a82982c24aa5f66
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
JIRA: FUEL-383
Change-Id: I9203aa8d20def5b78d261f8c6847ddc576f0feb7
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Since we switched to dockerized Salt master and mounting the full
contents of the current git repo inside cfg01's /root/fuel dir,
exposing temporary deploy artifacts should be avoided, i.e. the
default temporary storage location (configurable via '-S' deploy arg)
should be moved outside the git repo.
JIRA: FUEL-383
Change-Id: I32f7197b018fd853867de42f5618650ac9022dc9
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
While at it, retire obsolete MAAS_IP global variable and let mas01
VM get a DHCP address from virsh-managed mcpcontrol network.
Change-Id: Ifd85dbcab10894a5d0d675d37f0c35f09776d9b4
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
|
|
|
|
Install in a local directory a newer version of virt-manager
to workaround obsolete Ubuntu versions lacking --cpu cellN.* support.
This change only affects CPU cfg of virtual compute nodes in
nosdn-nofeature-noha scenarios with:
- set default cpu_topology to dual socket (2 cores, 2 sockets,
2 threads);
- bump default RAM to 16GB;
- define 2 NUMA cells, each with half the resources;
To keep the old behavior available (single socket), a new deploy
argument has been added (`-m`). The RAM change is not configurable
via deploy args.
NOTE: The CPU topology for virtual nodes should later be read from
PDF instead of hardcoding it on a per-scenario basis in the installer.
NOTE: Default 'ram' unit is MiB, while cellN.memory default unit is
'KiB'.
JIRA: FUEL-385
Change-Id: I7ca268b0a2052524cb7187a5cf9b6fa8a382c9f9
Signed-off-by: Dimitrios Markou <mardim@intracom-telecom.com>
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Allow skipping docker pull for verify jobs by setting the new env
var to 'verify'.
JIRA: FUEL-383
Change-Id: If8e2f66b5ccdac5c3911eeabfc2ba9c0eba61093
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Drop duplicate maas:machines definitions which could cause conflicts
in rare corner cases.
Slightly refactor j2 template expansion to make `conf.virtual.nodes`
available during first stage.
Change-Id: I04d56e346b12c6eb97da5c0c0ab1e3446e5fc1b8
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
* refactor & extend `-f` deploy argument;
* retire INFRA_CREATION_ONLY env var, duplicated by
NO_DEPLOY_ENVIRONMENT;
* `-F` and `-e` deploy arguments are now equivalent;
JIRA: FUEL-383
Change-Id: Ifc1527fa1e7d7486d1b7600772e2c5de34b1e52c
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
* Refactor OPNFV salt-formulas mechanism to resemble upstream git
structure:
- git submodules: add new submodule for each formula we patch;
- create salt-formula-x directories for OPNFV formulas;
- move mcp/metadata/service contents to their each formula subdir;
- use `make patches-import` for patches previously handled by
patch.sh;
- retire patch.sh
* states: add virtual_init:
- mostly based on old salt.sh, which is now obsolete;
- exclude salt-master service restart (it would kill the container);
* scenarios: cleanup (rm cfg01 virtual node def), adopt virtual_init;
* reclass: align our model with prebuilt container's Salt config:
- drop linux:network pillar data (handled by Docker);
- stop applying linux.system state on cfg01;
- align salt user homedir;
- drop salt-formula packages (preprovisioned);
* minor plumbing in deploy.sh and lib.sh;
JIRA: FUEL-383
Change-Id: I28708a9b399d3f19012212c71966ebda9d6fc0ac
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
JIRA: FUEL-383
Change-Id: Ie4374d44993bd738b90e9b6e357014d0c5657da0
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
JIRA: FUEL-383
Change-Id: Ia1b58f9e7eefd9f3efc4b5241ec496c545ccee25
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
JIRA: FUEL-383
Change-Id: I5596ed6139317d72fceae3af89e71181aefbd6a3
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
* default to enable pushing to public docker registry;
* fix prerequisites path;
* fix unknown terminal in Jenkins jobs;
JIRA: FUEL-383
Change-Id: Ie27dfbca6965c218475060da390a82921799b044
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
While at it, create the 'ubuntu' user with passwordless sudo
and preinstall salt-formula-gnocchi (missing git repo workaround),
as well as various useful packages (e.g. net-tools).
JIRA: FUEL-383
Change-Id: I5902c37110331acfd3fd4fccb92104de7b5ece6b
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- add new git submodule pointing to upstream docker build scripts;
- add patch extending Docker tags with an '-(arch)' suffix,
aligning with OPNFV tagging requiremnts;
- add <ci/build.sh> wrapper for starting Docker builds;
- install build-specific distro package requirements, as well as
pip-managed packages (e.g. pipenv);
JIRA: FUEL-383
Change-Id: Id4fc886206d7eaf7e6d02810380f2391609ba405
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
JIRA: FUEL-383
Change-Id: I19d27ca59a3f24d1bd66e39457a6ca267bccce19
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Add support for different prerequisites depending on the current
operation (docker build or cluster deploy).
Leverage the new support to pre-install upcoming deps:
- python-pip (build);
- docker-compose (deploy);
JIRA: FUEL-383
Change-Id: Ic3e6062b1943e3584f0b1f80d2e33b8812defced
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Upcoming support for containerized Salt Master node requires access
to certain variables during j2 interpolation, so also export:
- MCP_STORAGE_DIR;
- MCP_REPO_ROOT_PATH;
JIRA: FUEL-383
Change-Id: I584c0bf8133b5ae6178d97da5b44d345e45a0222
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
By default, vnet devices have a MTU of 1500 on the host side, causing
issue with larger packets traversing the bridges between guest VMs
when guest VMs have jumbo frames enabled.
JIRA: FUEL-336
JIRA: FUEL-367
JIRA: FUEL-382
[1] http://linuxaleph.blogspot.com/2013/01/
how-to-network-jumbo-frames-to-kvm-guest.html
[2] https://packetpushers.net/udev/
Change-Id: I941ac9cf764e3b3fa2d6463be5363b5459775f29
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
For hybrid PODs (e.g. x86_64 jumpserver + control nodes, aarch64
baremetal compute nodes), the virtual nodes rely on MaaS DHCP to be
up when the OS boots, so issue a `virsh reset` accordingly.
Instead of checking for online nodes using `test.ping`, use
`saltutil.sync_all` to also sync Salt state modules to the virtual
nodes (usually handled by baremetal_init state in HA deploys).
JIRA: FUEL-338
Change-Id: If689d057dc4438102c3a7428a97b9638e21bfdc5
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Previously, all virtual deployments relied on lab configuration from
<./mcp/config> using 'local-virtual1' as POD name.
To keep using that default sample config, one can still pass the
appropiate arguments to <ci/deploy.sh> as in:
$ ./ci/deploy.sh -l local -p virtual1 [...]
This changed introduces support for vPOD-specific lab config, e.g.:
$ ./ci/deploy.sh -l ericsson -p virtual4
Change-Id: I6ecb8e71d7e923e4b29b4ab0693c1afa5e19b582
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Change-Id: If77ac85fa86e0a1a18c0cc2abff77d876cdb9e93
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
Instead of duplicating scenarios for NOVCP, allow it to be specified
using a new deploy argument, `-N`.
Things are getting convoluted, so instead of creating dedicated
'*_pdf.yml.j2' files for each group of similar features, apply the
templating in-place and rename all affected files to ".yml.j2".
This breaks .gitignore assumption of hiding only "*_pdf.yml" files,
so extend (manually) the <mcp/reclass/classes/cluster/.gitignore>
with `git ls-files --exclude-standard -o` after an expansion.
- ha: move nfv.cpu_pinning to j2, conditioned by 'baremetal';
- ha: add cmp00* vnode definitions (hugepages need more RAM);
- labs/local: enable hugepage params for non-dpdk noha;
- salt.sh: add route_wrapper to all non-infra VMs;
This change extends novcp support to all HA scenarios.
Change-Id: I7a80415ac33367ab227ececb4ffb1bc026546d36
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
j2/python is easier to read and manipulate strings, although it does
need some special care about undefined dict keys.
With this in place, deploy.sh only contains the higher level logic for
the deployment process.
- merge arch-specific default configuration files into a singular
file with arch name as main dict key of old config (also avoids
creating duplicate 'virtual' YAML keys in $LOCAL_PDF);
- move template handling to separate <lib_template.sh>;
- decouple tight bash ordering of scenario expansion -> parse_yaml ->
variable export (e.g. CLUSTER_DOMAIN) -> re-use in cluster j2s;
however we can't parse *all* j2s in one go, as scenario j2s might
expand to YAMLs needed while expanding cluster j2;
- split `do_templates` into separate functions for each stage, with
no coupling between them other then call order;
Change-Id: I4b5e804094c00e5e918caf769fd85fa52181ad76
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Change-Id: I687b73b256aca78c9d41d4bcd49bfbde51278b51
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|