Age | Commit message (Collapse) | Author | Files | Lines |
|
Allow skipping docker pull for verify jobs by setting the new env
var to 'verify'.
JIRA: FUEL-383
Change-Id: If8e2f66b5ccdac5c3911eeabfc2ba9c0eba61093
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
* refactor & extend `-f` deploy argument;
* retire INFRA_CREATION_ONLY env var, duplicated by
NO_DEPLOY_ENVIRONMENT;
* `-F` and `-e` deploy arguments are now equivalent;
JIRA: FUEL-383
Change-Id: Ifc1527fa1e7d7486d1b7600772e2c5de34b1e52c
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
* Refactor OPNFV salt-formulas mechanism to resemble upstream git
structure:
- git submodules: add new submodule for each formula we patch;
- create salt-formula-x directories for OPNFV formulas;
- move mcp/metadata/service contents to their each formula subdir;
- use `make patches-import` for patches previously handled by
patch.sh;
- retire patch.sh
* states: add virtual_init:
- mostly based on old salt.sh, which is now obsolete;
- exclude salt-master service restart (it would kill the container);
* scenarios: cleanup (rm cfg01 virtual node def), adopt virtual_init;
* reclass: align our model with prebuilt container's Salt config:
- drop linux:network pillar data (handled by Docker);
- stop applying linux.system state on cfg01;
- align salt user homedir;
- drop salt-formula packages (preprovisioned);
* minor plumbing in deploy.sh and lib.sh;
JIRA: FUEL-383
Change-Id: I28708a9b399d3f19012212c71966ebda9d6fc0ac
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
JIRA: FUEL-383
Change-Id: Ie4374d44993bd738b90e9b6e357014d0c5657da0
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
JIRA: FUEL-383
Change-Id: Ia1b58f9e7eefd9f3efc4b5241ec496c545ccee25
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
JIRA: FUEL-383
Change-Id: I5596ed6139317d72fceae3af89e71181aefbd6a3
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
* default to enable pushing to public docker registry;
* fix prerequisites path;
* fix unknown terminal in Jenkins jobs;
JIRA: FUEL-383
Change-Id: Ie27dfbca6965c218475060da390a82921799b044
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
While at it, create the 'ubuntu' user with passwordless sudo
and preinstall salt-formula-gnocchi (missing git repo workaround),
as well as various useful packages (e.g. net-tools).
JIRA: FUEL-383
Change-Id: I5902c37110331acfd3fd4fccb92104de7b5ece6b
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- add new git submodule pointing to upstream docker build scripts;
- add patch extending Docker tags with an '-(arch)' suffix,
aligning with OPNFV tagging requiremnts;
- add <ci/build.sh> wrapper for starting Docker builds;
- install build-specific distro package requirements, as well as
pip-managed packages (e.g. pipenv);
JIRA: FUEL-383
Change-Id: Id4fc886206d7eaf7e6d02810380f2391609ba405
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
JIRA: FUEL-383
Change-Id: I19d27ca59a3f24d1bd66e39457a6ca267bccce19
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Add support for different prerequisites depending on the current
operation (docker build or cluster deploy).
Leverage the new support to pre-install upcoming deps:
- python-pip (build);
- docker-compose (deploy);
JIRA: FUEL-383
Change-Id: Ic3e6062b1943e3584f0b1f80d2e33b8812defced
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Upcoming support for containerized Salt Master node requires access
to certain variables during j2 interpolation, so also export:
- MCP_STORAGE_DIR;
- MCP_REPO_ROOT_PATH;
JIRA: FUEL-383
Change-Id: I584c0bf8133b5ae6178d97da5b44d345e45a0222
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
By default, vnet devices have a MTU of 1500 on the host side, causing
issue with larger packets traversing the bridges between guest VMs
when guest VMs have jumbo frames enabled.
JIRA: FUEL-336
JIRA: FUEL-367
JIRA: FUEL-382
[1] http://linuxaleph.blogspot.com/2013/01/
how-to-network-jumbo-frames-to-kvm-guest.html
[2] https://packetpushers.net/udev/
Change-Id: I941ac9cf764e3b3fa2d6463be5363b5459775f29
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
For hybrid PODs (e.g. x86_64 jumpserver + control nodes, aarch64
baremetal compute nodes), the virtual nodes rely on MaaS DHCP to be
up when the OS boots, so issue a `virsh reset` accordingly.
Instead of checking for online nodes using `test.ping`, use
`saltutil.sync_all` to also sync Salt state modules to the virtual
nodes (usually handled by baremetal_init state in HA deploys).
JIRA: FUEL-338
Change-Id: If689d057dc4438102c3a7428a97b9638e21bfdc5
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Previously, all virtual deployments relied on lab configuration from
<./mcp/config> using 'local-virtual1' as POD name.
To keep using that default sample config, one can still pass the
appropiate arguments to <ci/deploy.sh> as in:
$ ./ci/deploy.sh -l local -p virtual1 [...]
This changed introduces support for vPOD-specific lab config, e.g.:
$ ./ci/deploy.sh -l ericsson -p virtual4
Change-Id: I6ecb8e71d7e923e4b29b4ab0693c1afa5e19b582
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Change-Id: If77ac85fa86e0a1a18c0cc2abff77d876cdb9e93
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
Instead of duplicating scenarios for NOVCP, allow it to be specified
using a new deploy argument, `-N`.
Things are getting convoluted, so instead of creating dedicated
'*_pdf.yml.j2' files for each group of similar features, apply the
templating in-place and rename all affected files to ".yml.j2".
This breaks .gitignore assumption of hiding only "*_pdf.yml" files,
so extend (manually) the <mcp/reclass/classes/cluster/.gitignore>
with `git ls-files --exclude-standard -o` after an expansion.
- ha: move nfv.cpu_pinning to j2, conditioned by 'baremetal';
- ha: add cmp00* vnode definitions (hugepages need more RAM);
- labs/local: enable hugepage params for non-dpdk noha;
- salt.sh: add route_wrapper to all non-infra VMs;
This change extends novcp support to all HA scenarios.
Change-Id: I7a80415ac33367ab227ececb4ffb1bc026546d36
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
j2/python is easier to read and manipulate strings, although it does
need some special care about undefined dict keys.
With this in place, deploy.sh only contains the higher level logic for
the deployment process.
- merge arch-specific default configuration files into a singular
file with arch name as main dict key of old config (also avoids
creating duplicate 'virtual' YAML keys in $LOCAL_PDF);
- move template handling to separate <lib_template.sh>;
- decouple tight bash ordering of scenario expansion -> parse_yaml ->
variable export (e.g. CLUSTER_DOMAIN) -> re-use in cluster j2s;
however we can't parse *all* j2s in one go, as scenario j2s might
expand to YAMLs needed while expanding cluster j2;
- split `do_templates` into separate functions for each stage, with
no coupling between them other then call order;
Change-Id: I4b5e804094c00e5e918caf769fd85fa52181ad76
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Change-Id: I687b73b256aca78c9d41d4bcd49bfbde51278b51
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Change-Id: Ie4d8e70866d533d195a6e80cdfecbdb00a3027ce
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Drop one questionable patch responsible for MaaS node authorized
keys to include mcp.rsa.pub by reading the contents of authorized
keys on mas01, assuming mcp.rsa.pub will be on the first line.
Instead, export the contents of the public key using a shell env
var during deploy, which gets expanded via maas_pdf j2 template
into a reclass param, leveraging existing salt-formula-maas sshprefs
mechanism for delivering the key to MaaS.
Since we require the public key to exist before expanding templates,
move `generate_ssh_key` call outside the current infrastructure
handling block, allowing it to execute during all `deploy.sh` calls,
even for dry-runs.
Change-Id: I0f53b0f764a2fafd292e0ffd399c284acf61bd30
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- MaaS requires PXE/admin to be a Linux bridge;
- if virtual nodes are present, they should be hooked to a proper
Linux bridge for the Public network, but only throw a warning if
not (and create a mock public virsh network instead);
- if both virtual and baremetal nodes are present, Public bridge is
indirectly mandatory (we can't mock it);
JIRA: FUEL-339
Change-Id: Idfe99d66c49eadc56cb3d94ca4db3467fb76d388
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Instead of classifying scenarios by underlying machine type, switch
to HA/NOHA differentiantion only.
This allows us to add support for hybrid scenarios (with some virtual
and some baremetal nodes in the same cluster).
To facilitate this, we will template the scenario files, which is a
small step towards SDF (Scenario Descriptor File) definition and
adoption later.
JIRA: FUEL-338
Change-Id: If5787991869a3105d82c27ffa0a86ac79b4b08ba
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- add missing network definitions for ODL node's 1st interface;
- add missing comments for `notify` global functions;
- fix or silence shellcheck issues;
JIRA: FUEL-322
Change-Id: Ie3341d29ab12ddf432db603ad865259afb54714e
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- add new virsh managed network 'pxebr' (to mimic baremetal behavior
on virtual PODs, this will be the equivalent of PXE/admin network);
- connect 'pxebr' to 3rd interface for cfg01, mas01 for all deploys
(used to be baremetal-specific), replacing 'internal';
- keep 'mcpcontrol' connected only to 'cfg01' (+ 'mas01' if present)
for initial infrastructure bring-up (1st interface);
- switch all virtual cluster nodes to 'pxebr' (1st interface);
- use 'pxebr' for all Salt cluster nodes traffic, 'mcpcontrol' only
for mas01<=>cfg01 Salt traffic;
- convert <user-data.template> to jinja2 and expand it based on PDF
instead of using `envsubst`;
- split <user-data.sh.j2> into two versions, one for each network
used for Salt traffic;
- ci/deploy.sh: Read scenario data before template parsing for
cluster domain variable, needed in virsh network def;
- leave docs diagram refresh to later after all possible deploy types
have settled;
- limit keyserver proxy usage to nodes where the configured http proxy
matches the first nameserver (true for all MaaS-provisioned nodes),
so we can re-use the same pillar for FN VMs and baremetal nodes;
- add PXE/admin IP on cfg01's 3rd interface and switch other vnodes
`salt_master_host` to point to it;
JIRA: FUEL-322
Change-Id: Ie4f7aedddf2ef81046f1127b377d88dce79f0fda
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- apply `linux` state on cfg01 first, so PXE/admin IP is added and
FN VM minions are available;
- add barrier and wait for all FN VMs to register with cfg01;
- use batch-mode execution while applying `linux.network` on FN VMs;
- retry all states executed via <salt.sh> on FN VMs;
JIRA: FUEL-310
Change-Id: I72e1c565370072500df1d486fe76e6315f583c75
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- move bash template handling (previously expanded via `envsubst`)
to lib.sh;
- move j2 template handling to lib.sh;
- move virsh network templates to 'mcp/scripts/virsh_net' subdir;
- switch virsh network templates from `envsubst` expansion to j2 and
leverage generate_config.py, similar to PDF Fuel installer adapter;
- add relevant runtime env vars (e.g. SALT_MASTER, MAAS_IP) on the fly
to PDF, to consume them in templates like params coming from PDF;
- parameterize virsh network definitions based on PDF (mgmt, public);
JIRA: FUEL-322
Change-Id: Ib94e78fc4f25797b9354a0552e884104da5d0003
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Extend `notify` to 4 variants:
- notify_i = inline (no newline) colored output;
- notify = `notify_i` + trailing '\n';
- notify_n = `notify` + extra '\n' before and after;
- notify_e = `notify` + stderr output + exit;
This allows us to remove '\n' and cleanup the code a bit.
While at it, fix some 'NOTE' messages going to stderr instead of
stdout.
Change-Id: I682e3344ae9e307c4a68ab31c7766bc91b12ee58
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- hard requiremenet of PDF/IDF configuration for all deployments;
- expand j2 templates for virtual deploys too;
Since until now we used the same model for *all* virtual PODs, one
of the PDF/IDF sets for existing vPODs (e.g. ericsson-virtual3) can
be re-used practically on any host, without defining new vPODs.
JIRA: FUEL-322
Change-Id: Iac6aab91b6958d0e5e175ed142da6aafadc6fac6
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Until PDF/IDF land in Pharos for all our virtual PODs, use a common
vPDF we already provide as an example to mimic the old hardcoded
behavior while leveraging PDF/IDF parameterization.
As a consequence, python requirements previously only needed for
baremetal should now also be installed for virtual deploys too.
JIRA: FUEL-322
Change-Id: Ied1c907275285a9086450a15491ae516a0db1be2
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
It is easier to just generate the `pod_config.yaml` file than to
maintain it, so let's remove it.
While at it, link sample PDF/IDF inside pharos git submodule, so we
don't have to pass a different lab-config URI to use the sample.
To generate pod_config.yml for the sample PDF/IDF:
$ ./ci/deploy.sh -l local -p pod1 -s os-odl-nofeature-ha -d
$ cat mcp/deploy/images/pod_config.yml
JIRA: FUEL-322
Change-Id: If5898f92ef54bebc31d57f9632959e9093a89250
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Sync latest changes from pharos git repo for our sample PDF/IDF:
- move net_config from PDF to IDF;
- minor cleanup;
JIRA: FUEL-322
Change-Id: If6865ac61a4942a1dd5daf7081fd8faa67e0e7bf
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
JIRA: FUEL-334
Change-Id: I6d2499053dcfb7f99593fcd5c948b569bdcb9c9b
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
All bridge information should be defined via IDF.
Keeping `-B` support in deploy.sh only creates confusion.
While at it, remove automatic bridge detection based on network addr.
Change-Id: Id8b54e4887a97092daed5beb31fde4edd13057f1
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
|
|
- switch from securedlab to pharos as lab-config structure;
- accomodate the move net_config from PDF to IDF in j2 templates;
Change-Id: Ib04e4fb384568a6efd9e78a080857b663521ae88
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Previously, when a requested scenario was missing, we used to default
to the simplest scenario.
Now that we have a critical mass of scenario definitions, retire
this fallback mechanism.
Change-Id: I3ba6b04290806ff78b3ec22b90fa054d45602869
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
JIRA: FUEL-323
Change-Id: I0dcbcfbedc7e9ef013ca50dcc08f804323f91701
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Cleanup sequence might print some warnings/errors that are misleading
for new users, so let's silence them.
While at it, remove unused `cleanup` function stub.
Change-Id: Ic4225f7d4b701cd15d96e1bfc5eecfe2c4291eb6
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Change-Id: I4d41ff8b9f7c1b59a2c1a400b9335f17b8c542be
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Instead of hardcoding the packages to install on the jumphost at
deploy time in `ci/deploy.sh`, move the requirements list to a
dedicated yaml file, so it is decoupled from actual code.
While at it, add missing deps:
- ipxe-qemu (AArch64 Ubuntu only);
Change-Id: I9ee5c0796b70f9d29427d608b6415333c8841e1e
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- s/opnfv_maas_pxe_/opnfv_infra_maas_pxe_/g to align with other vars;
- patches: pharos: Add MaaS PXE network to installer adapter;
- runtime.yml{,.template}: move to installer adapter, update
pod_config.yml example;
- drop MAAS_PXE_NETWORK global env var, now read strictly from PDF;
JIRA: FUEL-313
Change-Id: I46d7510bd53fba7890c411d36bc28fd6ff6f3648
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
While at it, fix another small issue where a variable
(parameters__param_opnfv_maas_pxe_address) was used without being
populated (defined in 'runtime.yml', which is not read using
`parse_yaml`, like the scenario yml or the PDF expanded class,
pod_config.yml).
JIRA: FUEL-313
Change-Id: Iee88601d5420f55572bf90c8cf330afbedfb2e21
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
|
|
Parse all reclass j2 templates, not only common + current scenario
(useful when adding new scenarios later).
JIRA: FUEL-310
Change-Id: I8e87af702f83c42cb8f766bf6f121449aa5f2c26
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
|
|
Change-Id: I1df0228cb44bf9122aaf93dd25fc16a0d26a5240
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
qemu-nbd currently available in CentOS 7 does not add partition
mappings automatically for NBD devices, so add explicit `kpartx`
calls.
Change-Id: Ifa79c89b82024602b782c449dbf4de10899e03b5
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
While at it, rename apt repo in foundation node user-data template
from "salt" to "saltstack", to align with reclass model naming.
Change-Id: I5b216492349ae187b568884b1ab4046c52b1c6b2
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Extend <lib.sh> and its invocation from <ci.deploy.sh> with
support for modifying foundation node VMs base image prior to
using it with:
- additional APT GPG keys;
- additional APT repos;
- packages to pre-install;
- packages to pre-remove;
- (non-configurable) cloud init datasource via NoCloud only,
so VCP VMs won't wait for metadata service;
While at it, re-use the resulting image as a base for another
round of pre-patching (same operations as above are supported)
to provide a base image for VCP VMs.
Add AArch64-specific configuration based on new mechanisms:
- pre-install linux-image-generic-hwe-16.04-edge (and headers)
for foundation node and VCP (common) image (also requires new
repo and its key);
- pre-install cloud-init for VCP image (it should already be
installed, but script needs non-empty config for VCP to create
the VCP image and transfer it over to Salt Master);
NOTE: cloud-init is required on VCP VMs for DHCP on 1st iface.
JIRA: FUEL-309
Change-Id: I7dcaf0ffd9c57009133c6d339496ec831ab14375
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|