Age | Commit message (Collapse) | Author | Files | Lines |
|
Make the bulk of scenario files static again by shifting out all
common virtual nodes (mas01) and states (virtual_init, maas etc.)
to default.yaml(.j2).
This allows us to parse scenario-specific data during first j2
expansion, preparing for the new Pharos installer adapter that
relies on `conf.virtual.nodes.control` length to construct the
proper list of MaaS node definitions (kvm{01,02,03} vs {ctl01,gtw01}).
Change-Id: I666ab5bd6bb2a42f98646af51950f6b9fffa0e8b
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
curtin generates armband_3.list, which we aligned in the reclass
model before, but not in the defaults yml used by cfg01/VCP VMs.
Change-Id: Ibe8d5b79465b508e41c20ec08a98a96a1510da8d
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
* Refactor OPNFV salt-formulas mechanism to resemble upstream git
structure:
- git submodules: add new submodule for each formula we patch;
- create salt-formula-x directories for OPNFV formulas;
- move mcp/metadata/service contents to their each formula subdir;
- use `make patches-import` for patches previously handled by
patch.sh;
- retire patch.sh
* states: add virtual_init:
- mostly based on old salt.sh, which is now obsolete;
- exclude salt-master service restart (it would kill the container);
* scenarios: cleanup (rm cfg01 virtual node def), adopt virtual_init;
* reclass: align our model with prebuilt container's Salt config:
- drop linux:network pillar data (handled by Docker);
- stop applying linux.system state on cfg01;
- align salt user homedir;
- drop salt-formula packages (preprovisioned);
* minor plumbing in deploy.sh and lib.sh;
JIRA: FUEL-383
Change-Id: I28708a9b399d3f19012212c71966ebda9d6fc0ac
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- dummy copy of os-nosdn-nofeature-ha masquerading as
os-nosdn-vpp-ha scenario placeholder
- dummy copy of os-nosdn-nofeature-noha masquerading as
os-nosdn-vpp-noha scenario placeholder
Change-Id: I63a302c98e6e176a480fbc110012ac66749ee88f
Signed-off-by: Cristina Pauna <cristina.pauna@enea.com>
|
|
JIRA: ARMBAND-398
Change-Id: I329346755348525f00602d7f89af295c141a20d5
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Salt 2016.x has a bug with states ordering coming from 'include'
statement. Glance/Heat DB sync is applied before package setup
although clearly specified 'require' in state.
Change-Id: Ic5f7ce4a7623fb208c0a5ba366802b7e02fa8b9f
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
JIRA: FUEL-382
Change-Id: Ib1f905bec87ce2afe643b4aa3b0201d7100be16d
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Split scenario yaml definitions for virtual.nodes based on node
role ('infra', 'control' or 'compute'), to be leveraged later to
contruct node lists based on said role.
This moves the responsability of filtering node names in scenario
files (based on 'virtual' or 'baremetal' type) to xdf_data.sh.j2,
simplifying scenario templates.
By keeping all nodes (both virtual and baremetal) in scenario files,
we can later determine the role (and implicitly the hostname) for a
MaaS-managed node based on its index in the virtual.nodes.control
structure.
JIRA: FUEL-382
Change-Id: I1f83a307631f4166ee1c57ef598c44876b962f97
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Mirantis repositories shadow certain packages from Armband repos
(e.g. qemu-efi) if pinning is not set.
Since MaaS does not allow configuring repo priorities via its API,
use Salt pillar data to re-define the Armband repo and set its prio.
Change-Id: Ic743f4b684b3f13552792f5f04097fac73171b37
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Change-Id: Ie707de90617e7080a5eb87cad69a604cf5e0f9fc
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
- noha: 'accept_policy: open_mode' to align with ha scenarios;
- s/cmp01/cmp001/g to align all scenarios and allow code reuse;
- rename network params: s/dhcp/mcpcontrol/g, cleanup;
- computes XDF data: drop 'opnfv_*' layer of params, cleanup;
- local vPDF: add comments with default roles by node index;
- parameterize all netmasks;
- drop unused address/netmask for 'proto: manual' interfaces;
- virsh_net: cleanup definitions, remove hardcodes, align IP on
jumpserver and DHCP range with MaaS for pxebr;
- maas: parameterize hardcoded '/24' cidr for PXE/admin, refactor
maas.region.machines parameterization;
- merge <all-mcp-arch-common/infra/config_*pdf.yaml.j2> templates;
- move reclass.storage definitions of compute nodes to common dir;
- drop 'openstack_compute_*' reclass params in favor of expanding
them via j2 directly in reclass.storage params;
- adopt `nm.cluster.has_*_nodes` where possible;
- obsolete `runtime.yml` from reclass model;
- refactor arch-specific reclass param selection;
- remove unused defaults in favor of mandatory IDF properties;
- noha: prepare for baremetal node support in cinder_lvm_devices;
- interfaces: add interface_mtu and 'noifupdown: true' everywhere;
- interfaces: use j2 macros to generate eth/vlan config;
- states cleanup: remove DHCP route disable workaround on prx/cmp;
- allow configuring NTP servers via:
`idf.fuel.network.ntp_strata_host{1,2}`;
- ovs_bridge: Allow setting gateway, dns-nameservers
- apache: Adjust module list for novcp class inheritance;
- glusterfs PPA: pin with same prio of MCP repos for novcp scenario;
JIRA: FUEL-319
JIRA: FUEL-326
JIRA: FUEL-337
Change-Id: Ia6ad64ba8cade85a75fb22c9a2505decc3834360
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Change-Id: Id024ed22dd1760f41ae18aeb8e680c2f07a5dc63
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
* make cluster names version-agnostic
* switch to mirror.mirantis.com
* system reclass update
Change-Id: I96394b3c9d8d38321289af8e3a4e8bbc5dd7a408
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
* make cluster names version-agnostic
* pre-install oslo-templates formula
* salt version update to 2017.7
* system reclass update
Change-Id: I903895f6bcd98a1062e66e3f14d23ec04dc1b869
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
Align kernel versions across architectures (where possible).
Change-Id: I66a822611eb5e46f90b62f5d36df571ae75dcba3
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
To work around the missing dependency of 'python-tornado' on
'python-features' provided by the binary-specific DEB package
in amd64 Saltstack repository, pre-install the req explicitly
on FN (jumpserver) and VCP VMs via lib.sh.
JIRA: ARMBAND-382
Change-Id: I2dbd08a58f6cf57c2c48c2d7fc8d19ad4872e800
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Change-Id: If77ac85fa86e0a1a18c0cc2abff77d876cdb9e93
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
* setup HWE kernel to get suitable conntrack module
* clean out outdated state with ovn ctl options
* point SB remote source to local mgmt network
Change-Id: I8986c227ce0a9a3b7ab3faf382760ec32e6e7c00
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
JIRA: FUEL-362
Change-Id: Ib2621bca72d1ba376af5d369edcf5fcf37e9788b
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
* return back opendaylight state after neutron setup
* sleep for awhile to let neutron api reconnect to the ODL
controller and agents to register on server
Change-Id: Ife0c7d3cc20574b0733e8e3064843c680379cc84
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
Change-Id: Ia517b7cf1723a5afaf43cb0709716f3a67a29e9f
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
* employ GA kernel for baremetal computes as well
* setup/start opendaylight server after ovs host config
Change-Id: Ic772aed544b17be02e6ca9ccd175f2288b2128a8
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
Change-Id: I078e11219fb8dea4505c46e7f75c295c5a72c59b
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
Instead of duplicating scenarios for NOVCP, allow it to be specified
using a new deploy argument, `-N`.
Things are getting convoluted, so instead of creating dedicated
'*_pdf.yml.j2' files for each group of similar features, apply the
templating in-place and rename all affected files to ".yml.j2".
This breaks .gitignore assumption of hiding only "*_pdf.yml" files,
so extend (manually) the <mcp/reclass/classes/cluster/.gitignore>
with `git ls-files --exclude-standard -o` after an expansion.
- ha: move nfv.cpu_pinning to j2, conditioned by 'baremetal';
- ha: add cmp00* vnode definitions (hugepages need more RAM);
- labs/local: enable hugepage params for non-dpdk noha;
- salt.sh: add route_wrapper to all non-infra VMs;
This change extends novcp support to all HA scenarios.
Change-Id: I7a80415ac33367ab227ececb4ffb1bc026546d36
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
j2/python is easier to read and manipulate strings, although it does
need some special care about undefined dict keys.
With this in place, deploy.sh only contains the higher level logic for
the deployment process.
- merge arch-specific default configuration files into a singular
file with arch name as main dict key of old config (also avoids
creating duplicate 'virtual' YAML keys in $LOCAL_PDF);
- move template handling to separate <lib_template.sh>;
- decouple tight bash ordering of scenario expansion -> parse_yaml ->
variable export (e.g. CLUSTER_DOMAIN) -> re-use in cluster j2s;
however we can't parse *all* j2s in one go, as scenario j2s might
expand to YAMLs needed while expanding cluster j2;
- split `do_templates` into separate functions for each stage, with
no coupling between them other then call order;
Change-Id: I4b5e804094c00e5e918caf769fd85fa52181ad76
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- dummy copy of os-nosdn-nofeature-ha masquerading as
os-ovn-nofeature-ha scenario placeholder;
JIRA: FUEL-329
Change-Id: Ic5e6d65cf057289bb13ce2fa157b6ba53d7dab80
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- bump Pharos git submodule for j2 'do' extension + batch mode;
- adopt j2 'do' in our templates;
- use int filter for 'native' vlan check;
- lib.sh: adopt `-i` to remove `ln` hack for net_map.j2;
- lib.sh: adopt `-b` to speedup template parsing;
NOTE: Bumping Pharos will also bring in the latest changes in
pod_config.yml.j2, which include massive IP shifts and updates.
JIRA: FUEL-335
Change-Id: I7d3a997b3d8659d5f09f867870fb3a148c1ec6df
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
- fix `route-br-ex` if-up.d script failing when route already exists
by adding a wrapper around distro's '/sbin/route' binary in
'/usr/local/sbin/route', exploiting default order in Ubuntu PATH;
- fix 'br-prv' duplicate entry in 'interfaces.d/ifcfg-br-prv' and
'interfaces' caused by upstream bug [1];
- add barrier waiting for all baremetal nodes online before attempting
reboot, trying to catch rare failures which are undetectable in logs
as both a succesful reboot and a disconneted minion report 'n/c';
With the above in place, networking service should no longer fail
to start on cmp nodes w/ DPDK.
[1] https://github.com/saltstack/salt/issues/40262
Change-Id: I6d4895376ce323c14c997e6c9af2ea3eeeee0184
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Virtual compute nodes goes to kernel panic during nova instances ops
under hwe kernel, so use the default one from xenial image.
Change-Id: Iae100b68208cc1fb9e43e45f385e762cdbd6573a
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
Set nova/libvirt disk cache modes to file directsync to bypass the
host page cache & prevent instance's kernel panic due to absent
/dev partition. Caching mode directsync effectively turns all guest
I/O operations into direct I/O operations on the host, which is the
NFS client (virtuals) or GlusterFS (baremetals).
Also return back to hwe kernel as a golden mean between GA & Edge ones.
Change-Id: I51ab7d0ee71c214ff16b756cfee16e918738b6fd
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
Fixes: 86c8109
Change-Id: I0a947bd7db5dd7cb830cc932f5a0d16831551923
Signed-off-by: Delia Popescu <delia.popescu@enea.com>
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
JIRA: FUEL-322
Change-Id: I99c4b1774a7c3afbc834a6f6e8468f7baf1bc329
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Instead of classifying scenarios by underlying machine type, switch
to HA/NOHA differentiantion only.
This allows us to add support for hybrid scenarios (with some virtual
and some baremetal nodes in the same cluster).
To facilitate this, we will template the scenario files, which is a
small step towards SDF (Scenario Descriptor File) definition and
adoption later.
JIRA: FUEL-338
Change-Id: If5787991869a3105d82c27ffa0a86ac79b4b08ba
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Add a new class of scenarios, based on existing baremetal HA
scenarios, but instead of having a virtualized control plane (VCP),
all Openstack controller services will run directly on the cluster
nodes.
This change adds the common scaffolding, as well as the OVS scenario.
The new scenario(s) can be used on full-baremetal clusters, soon on
full-virtual clusters and later on hybrid (virt + bare) clusters.
This change defines old (current) style scenario definitions for
both baremetal and virtual, both named:
- os-nosdn-nofeature-novcp-ha;
Prerequisites:
1. Merge-able by name reclass.storage.node definitions
Each cluster (e.g. database, telemetry) adds its own set of
reclass storage node defitions, which for novcp scenarios should
be merged into a single node (kvm) based on the 'name' property.
This is not currently supported by upstream reclass 'node.sls'
high state, so add support for it via an early patch (required
before salt-master-init.sh tries to handle reclass.storage).
2. common reclass classes for novcp
Some of the classes in `baremetal-...-common-ha` are not fit for
novcp as they define VCP-specific config/inheritance, so add new
versions of said classes with novcp in mind or adapt old classes:
- parameterize ctl hostname in `openstack_compute.yml`;
- new `openstack_control_novcp.yml`;
- new `openstack_init_novcp.yml`;
3. Handle hard set names in state files for baremetal nodes
Some of our state files (e.g. maas) hardcodes baremetal node names
to 'kvm', 'cmp', so we need to align the names in novcp scenario
with these values to re-use the maas state. As a future improvement
we should parameterize these names in all state files.
As a consequence, our baremetal controller nodes will also use
'kvm*' hostnames (instead of 'ctl*').
4. Add 'noifupdown' to all interfaces on kvm nodes to prevent duplicate
IPs/routes created at *any* ifup due to /etc/network/route-br-ex.
Patch salt-formula-linux to skip network restart on 'noifupdown',
also when routes are present on that interface.
JIRA: FUEL-310
Change-Id: Ic67778f63e5ee0334dbfe9547c7109ec1a938d61
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
A few things differ between baremetal and virtual nodes:
- provisioning method;
- network setup;
Since now we support completely dynamic network config based on PDF +
IDF, as well as dynamic provisioning of VMs on jumpserver (as virtual
cluster nodes), respectively MaaS-driven baremetal provisioning, let's
drop the 'baremetal-' prefix from cluster model names and prepare for
unified scenarios.
Note that some limitations still apply, e.g. virtual nodes are spawned
only on jumpserver (localhost) for now.
JIRA: FUEL-310
Change-Id: If20077ac37c6f15961468abc58db7e16f2c29260
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
* switch ovs/dpdk scenario from vlan to vxlan mode
* force br-ex interface to mitigate race with incorrect state
* remove dpdk packages list (already in upstream)
Change-Id: Ib827cef2d67879fd2a86d286ca2118b22493274d
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
- openvswitch 2.8 officially supports kernel versions from 3.10 to 4.12
- ODL baremetal scenario is acting up with floating/public SNAT
flow under hwe edge kernel 4.13
Change-Id: I099d528b3b1c2ea34f8f856cd60f809f90defea6
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
Since Mirantis prebuilt image comes with salt-minion 2016.3 instead
of 2016.11 and upgrading it leads to a hard to break catch-22, use
the Ubuntu cloud archive image we already download for FN VMs and
pre-install:
- a newer kernel (hwe-edge);
- salt-minion (2016.11);
This also implicitly aligns the image handling on AArch64 and x86_64.
Change-Id: I86d1c777449d37bdd0348936a598e3ffe9d265af
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Change-Id: Ib9021ee3ca15c05cc137ae42c263383acb4393bd
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
* rename openstack state name with noha suffix
* increase vcpus for compute nodes
Change-Id: I03386c4c1c92d329d847aa506589823e57644ef4
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
OVN based scenario doesn't require conventional gateway
node since connectivity to external networks and routing
occurs on compute nodes.
Change-Id: I81e0d497170d5ffb067adf13b0e46290525f26a6
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
Armband restructured its repos to support UCA-only virtual deploys,
propagate the change to our base image pre-build. Drop apt-mk repos,
should only be added later by Salt, when MCP repos are also present.
JIRA: ARMBAND-338
Change-Id: Ieafa6d3fb0599679341053187b4a7d778af86428
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
We hard set some repo URLs used for base image pre-building, so bump
those to Pike, now that Armband repos for Pike are populated.
JIRA: ARMBAND-335
Change-Id: Ibd7267b628985a3c7a33d9d8a93c824077c69f10
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Change-Id: If4208e937734df72b10aad259c65a2617ff5e19c
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
To align with new cluster naming convention, rename 'openstack'
state file to 'openstack_noha'.
While at it, factor out baremetal setup from 'virtual_control_plane'
into a new state that will be reused in upcoming scenarios, remove
useless sync_all (automatically done after node reboot).
FUEL-310
Change-Id: I6d7e5db8f09305f2fd8eeca0199a2e85b08d2202
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Parse all reclass j2 templates, not only common + current scenario
(useful when adding new scenarios later).
JIRA: FUEL-310
Change-Id: I8e87af702f83c42cb8f766bf6f121449aa5f2c26
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
|
|
Change-Id: I1df0228cb44bf9122aaf93dd25fc16a0d26a5240
Signed-off-by: Michael Polenchuk <mpolenchuk@mirantis.com>
|
|
While at it, rename apt repo in foundation node user-data template
from "salt" to "saltstack", to align with reclass model naming.
Change-Id: I5b216492349ae187b568884b1ab4046c52b1c6b2
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|
|
Extend <lib.sh> and its invocation from <ci.deploy.sh> with
support for modifying foundation node VMs base image prior to
using it with:
- additional APT GPG keys;
- additional APT repos;
- packages to pre-install;
- packages to pre-remove;
- (non-configurable) cloud init datasource via NoCloud only,
so VCP VMs won't wait for metadata service;
While at it, re-use the resulting image as a base for another
round of pre-patching (same operations as above are supported)
to provide a base image for VCP VMs.
Add AArch64-specific configuration based on new mechanisms:
- pre-install linux-image-generic-hwe-16.04-edge (and headers)
for foundation node and VCP (common) image (also requires new
repo and its key);
- pre-install cloud-init for VCP image (it should already be
installed, but script needs non-empty config for VCP to create
the VCP image and transfer it over to Salt Master);
NOTE: cloud-init is required on VCP VMs for DHCP on 1st iface.
JIRA: FUEL-309
Change-Id: I7dcaf0ffd9c57009133c6d339496ec831ab14375
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
|