| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Makes it possible to resolve network subnets within a service
template; the data is transported into a new property ServiceData
wired into every service which hopefully is generic enough to
be extended in the future and transport more data.
Data can be consumed in service templates to set config values
which need to know what is the subnet where a deamon operates (for
example the Ceph Public vs Cluster network).
Change-Id: I28e21c46f1ef609517175f7e7ee19e28d1c0cba2
|
|
|
|
With the merging of Iad3e9b215c6f21ba761c8360bb7ed531e34520e6 the
roles_data.yaml should be generated with tripleoclient rather than
edited. This change adds in a pep8 task to verify that the appropriate
role files in roles/ have been modified to match how our default
roles_data.yaml is constructed. Additionally this change adds a new tox
target called 'genrolesdata' that will all you to automatically generate
roles_data.yaml and roles_data_undercloud.yaml
Change-Id: I5eb15443a131a122d1a4abf6fc15a3ac3e15941b
Related-Blueprint: example-custom-role-environments
|
|
Render all per-network resources and interfaces via j2 to enable
future support for custom networks via network_data.yaml
Note this doesn't enable custom networks for the built-in roles
as we skip j2 rendering for them, this will be resolved by converting
them to use the generic role template instead of the hard-coded
ones listed in the j2_excludes.yaml.
Depends-On: I18fa3829ff38ac200550d8e36bbe334c0005da22
Change-Id: I49565f9389f3ec9aef4861e23a3bed64a85501e6
Partially-Implements: blueprint composable-networks
|
|
|
|
Currently we only consume the name with a special-case
for the disable constraints boolean, but it will be more
flexible if we consume the whole roles_data mapping for
each role, so that e.g composable networks and other
per-role customizations can be expressed in these
templates
Partially-Implements: blueprint composable-networks
Depends-On: Id1249b78b3dd87e91d572ffa31b7a541f3cde2c7
Change-Id: I355534ec456479944f66106e957404a660d8f2d2
|
|
This is necessary for accessing the bind mounted hieradata in the
container in order to determine if the node is the primary node.
With the new validation added to yaml-validate.py, we could spot
potential issues in sahara-api and keystone bootstrap tasks.
The keystone one is a false positive, as the image defaults to the root
user in order to be able to run apache. Still, it is better to be
consistent here and specify the root user nonetheless.
Change-Id: Ib0ff9748d5406f507261e506c19b96750b10e846
Closes-Bug: #1697917
|
|
Mounting host volumes when running containers via puppet_config already
works and is supported with docker-puppet.py. However, the validation in
yaml-validate.py does not allow it. This patch makes it allowed by the
validation.
It is sometimes necessary since some puppet modules expect to make
persistent file system changes other than just configuration data under
/etc.
In particular, ironic inspector expects to configure a http and tftp
root director with an ipxe configuration. See:
https://github.com/openstack/puppet-ironic/blob/master/manifests/inspector.pp
These changes would be lost if the value for those directories are not
mounted as host volumes.
Change-Id: Ie51c653f4c666fbaaef0ea80990e2e61f4b1353b
|
|
This commit consistently defines a heat template parameter in the form
of DockerXXXConfigImage where XXX represents the name of the
config_volume that is used by docker-puppet.
The goal is to mitigate hard to debug errors where the templates would
set different defaults for the image docker-puppet.py uses to run, for
the same config_volume name.
This fixes a couple of inconsistencies on the way.
Change-Id: I212020a76622a03521385a6cae4ce73e51ce5b6b
Closes-Bug: #1699791
|
|
Many of our parameters are defined in multiple templates, but
currently there is no easy way of checking that all of those
definitions match. It can be confusing when a parameter is defined
one way in one file and another way in a different file. For example,
the NovaWorkers description is:
Number of workers for Nova API service.
and
Number of workers for Nova Placement API service.
and
Number of workers for Nova Conductor service.
Which is it actually? All of them. That one parameter controls
the workers for all of the nova services, and its description should
reflect that, no matter which template you happen to look at.
This change adds a check to yaml-validate.py to catch these sorts of
inconsistencies and allow us to eventually prevent new ones from
getting into the templates.
An exclusion mechanism is included because there are some parameter
definitions we probably can't/shouldn't change. In particular, this
includes the network cidrs which are defaulted to ipv4 addresses in
the ipv4 net-iso templates and ipv6 in the ipv6 templates. It's
possible a user would be relying on one of those defaults in their
configuration, so if we change it they might break.
To get around that, the tool explicitly ignores the default field of
those parameters, while still checking the description and type fields
so we maintain some sanity. There may be other parameters where this
is an issue, but those can be added later as they are found.
For the moment any inconsistencies are soft-fails. A failure message
will be printed, but the return value will not be affected so we can
add the tool without first having to fix every divergent parameter
definition in tripleo-heat-templates (and there appear to be plenty).
This will allow us to gradually fix the parameters over time, and
once that is done we can make this a hard-fail.
Change-Id: Ib8b2cb5e610022d2bbcec9f2e2d30d9a7c2be511
Partial-Bug: 1700664
|
|
|
|
We're not going to want to list every single sample environment in
a single file, so let's also take a directory and just read every
yaml file in it. This commit adds support for that as well as
some initial environments to demonstrate its use.
Change-Id: If2c608f2a61fc5e16784ab594d23f1fa335e1d3c
|
|
Move to one common services.yaml not only reduces the duplication, but it
should improve performance for the docker/services.yaml case, because we were
creating two ResourceChains with $many services which we know can be really
slow (especially since we seem to be missing concurrent: true on one)
Change-Id: I76f188438bfc6449b152c2861d99738e6eb3c61b
|
|
When a service is enabled on multiple roles, the parameters for the
service will be global. This change enables an option to provide
role specific parameter to services and other templates.
Two new parameters - RoleName and RoleParameters, are added to the
service template. RoleName provides the role name of on which the
current instance of the service is being applied on. RoleParameters
provides the list of parameters which are configured specific to the
role in the environment file, like below:
parameters_default:
# Default value for applied to all roles
NovaReservedHostMemory: 2048
ComputeDpdkParameters:
# Applied only to ComputeDpdk role
NovaReservedHostMemory: 4096
In above sample, the cluster contains 2 roles - Compute, ComputeDpdk.
The values of ComputeDpdkParameters will be passed on to the templates
as RoleParameters while creating the stack for ComputeDpdk role. The
parameter which supports role specific configuration, should find the
parameter first in in the RoleParameters list, if not found, then the
default (for all roles) should be used.
Implements: blueprint tripleo-derive-parameters
Change-Id: I72376a803ec6b2ed93903cc0c95a6ffce718b6dc
|
|
|
|
|
|
* Split it to REQUIRED/OPTIONAL
* Move puppet_tags to OPTIONAL as it already has a
default set of tags that need not to be repeated
explicitly.
Change-Id: Ib70176f1edf61228771c983b0c3231fb7939a316
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
|
|
1.As mentioned in [1], we should avoid using
six.iteritems to achieve iterators. We can
use dict.items instead, as it will return
iterators in PY3 as well. And dict.items/keys
will more readable. 2.In py2, the performance
about list should be negligible, see the link [2].
[1] https://wiki.openstack.org/wiki/Python3
[2] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html
Change-Id: I7c8f540eced0731f4dfb1dfd045828b5a9bb6c67
|
|
Note: since it replaces rabbitmq, in order to aim for the smallest
amount of changes the service_name is called 'rabbitmq' so all the
other services do not need additional logic to use qdr.
Depends-On: Idecbbabdd4f06a37ff0cfb34dc23732b1176a608
Change-Id: I27f01d2570fa32de91ffe1991dc873cdf2293dbc
|
|
We've decided to use volumes for configuration wherever possible.
This means moving away from kolla_config blocks in the templates.
Update pep8 to reflect this.
Change-Id: If1ec40d0e5a515eed35e0cd04711079294f358c3
|
|
This section will be needed for TLS-everywhere. So it should be added as
optional in the yaml-validate.
Change-Id: Ic6ea563b6c8e454cb51f640bb5aaa3adda82a5dd
|
|
This implements a host_prep_tasks hook where we can specify Ansible
tasks to perform on the host before deploying containerized
services. The hook runs in a single step, the assumption is that we will
mostly use the hook for creating per-service directories on the host to
ensure we are able to mount them into the containers. (We cannot do this
operation via Puppet because all containerized services run their Puppet
within a config container, so Puppet doesn't have access to host's
filesystem.)
Change-Id: I7d8bac39e0cd422fd651eefe29f7d10941ab4a1a
|
|
This patch adds the beginning of a set of unit tests
for the new docker services templates. This should help
us the new interfaces as they evolve.
Change-Id: I98a73cf090ebab4593a682f5f34c0950d37e010c
|
|
This moves the hard-coded networks from the default environment,
and provides the first step towards enabling composable networks.
Co-Author: Dan Sneddon <dsneddon@redhat.com>
Partial-Bug: #1633090
Depends-On: I9f818912bd8e2a3220e41c8ccbbab3d9063b4d72
Change-Id: I7793b8badede5450b05437c84d9b40c28de7546b
|
|
Until bug #1635409 is fixed we'll have to keep the default list
of services deployed by hyperconverged-ceph.yaml in sync with the
ServicesDefault list provided in roles_data.yaml
This change adds some logic in the templates validation script to
ensure that is preserved with future updates.
Change-Id: Ib767f9a24c3541b16f96bd6b6455cf797113fbd8
|
|
When fixing LP#1643487 we added ?bind_address to all DB URIs.
Since this clashes with Cellsv2 due to the URIs becoming host
dependent, we need a new approach to pass bind_address to pymysql
that leaves the DB URIs host-independent.
In change Iff8bd2d9ee85f7bb1445aa2e1b3cfbff1f397b18 we first create a
/etc/my.cnf.d/tripleo.cnf file with a [tripleo] section with the correct
bind-address option.
In this change we make sure that the DB URIs will point to the added
file and to the specific section containing the necessary bind-address
option. We do introduce a new MySQLClient profile which will hold all
this more client-specific configuration so that this change can fit
better in the composable roles work. Also, in the future it might
contain the necessary configuration for SSL for example.
Note that in case the /etc/my.cnf.d/tripleo.cnf file does not exist
(because it is created via the mysqlclient profile), things keep on
working as usual and the bind-address option simply won't be set, which
has no impact on hosts where there are no VIPs.
Co-Authored-By: Damien Ciabrini <dciabrin@redhat.com>
Change-Id: Ieac33efe38f32e949fd89545eb1cd8e0fe114a12
Related-Bug: #1643487
Closes-Bug: #1663181
Closes-Bug: #1664524
Depends-On: Iff8bd2d9ee85f7bb1445aa2e1b3cfbff1f397b18
|
|
This reverts the changes in https://review.openstack.org/414629 for nova as
they are incompatible with cell_v2.
This is a temporary fix for HA while a long-term solution is developed.
Change-Id: I79d30a2d76a354999152c0c997ea77f104c51027
Related-bug: #1643487
Closes-bug: #1662344
|
|
Currently we are applying this validation for the services templates, this
submission moves it to run with all templates.
Also fixed those templates not using the alias name.
Change-Id: I3a2c0ce6adcc8061fdc51f73fdc6b9748c0fead9
|
|
This replicates the behavior of the custom Jinja2 loader from tripleo-common to
allow template validation on the local filesystem using tox.
Change-Id: I27683ab31187c6334dc5b4b5363a3347874b9a90
Partially-Implements: blueprint overcloud-upgrades-per-service
Depends-On: Idc5c3f49c7a2fc7f3622c76da001992cc657384e
|
|
this attempts to make the error message more useful. This error message
happens if the environment files containing endpoint map overrides
haven't been updated to match the base endpoint map (or the defaults).
Change-Id: If53d3a9d7848aed62ebb235afe8b14c18d1b284d
|
|
|
|
|
|
Allow for passing the output_dir in the process-templates scripts so
that it doesn't overwrites the templates in the src dir. This is a
desired feature when running the script from a t-h-t installed
system-wide.
Change-Id: I47994d34f47a4084a11124bc9075cb2f457889ea
|
|
Don't walk through hidden files. This avoids going through the .git,
.tox and other hidden directories that we don't care about.
Change-Id: I34b83229775d221299c8b572a7049175debac99d
|
|
Quick verification to check that the release name
is used instead the date.
Im also adding here all the updated templates required
to pass the check and merge this check as soon as possible.
Change-Id: Ifdc9ac4a9d0a4872d3e21672c93fc87da2e68a4e
|
|
This validation checks that the TLS-related environment files contain
all of the services defined in the base endpoint map. This will
hopefully help to keep them updated.
Change-Id: I58df72e104d8eb74e577484405f15e0a6f92d0ce
|
|
Add ReNo support to manage release notes.
http://docs.openstack.org/developer/reno/
Change-Id: Ie5154d909e616e4e7e813052f9c121d6ac5b0875
|
|
The error path for this fails because we don't import six
or install it in the templates tox venv
Change-Id: Ie9f46332f2b03d48a1b0a4a432e9721757833569
|
|
When a service connects to the database VIP from the node hosting this
VIP, the resulting TCP socket has a src address which is by default
bound to the VIP as well. If the VIP is failed over to another node
while the socket's Send-Q is not empty, TCP keepalive won't engage and
the service will become unavailable for a very long time (by default
more than 10m).
To prevent failover issues, DB connections should have the src address
of their TCP socket bound to the IP of the network interface used for
MySQL traffic. This is achieved by passing a new option to the
database connection URIs. This option is available starting from
PyMySQL 0.7.9-2.
We use a new intermediate variable in hiera to hold the IP to be used
as a source address for all DB connections. All services adapt their
database URI accordingly.
Moreover, a new YAML validation check is added to guarantee that new
services will construct their database URI appropriately.
Change-Id: Ic69de63acbfb992314ea30a3a9b17c0b5341c035
Closes-Bug: #1643487
|
|
This patch adds a local version of our template processing
routine so that developers can more quickly view the templates
that are actually getting generated. I've noticed multiple developers
now do a full deployment with 'overcloud deploy' only to download
the swift container with the generated templates. This simple task
avoids that step by allowing developers to generate it locally.
It also aims to preserve the ability to use t-h-t templates directly
with Heat (instead of going through Mistral) should users wish to do that.
The new undercloud heat installer requires the ability to generate
templates without requiring Mistral and Swift to do so.
Ideally the Mistral API workflow would use this same code
so perhaps in the future we might modify that routine to:
-download swift tarball containing the templates
-run this local routine that lives in t-h-t
-re-upload the tarball of templates to the swift container
Change-Id: Ie664c9c5f455b7320a58a26f35bc403355408d9b
|
|
Wire in os-net-config via a normal script heat deployment, which has the
following advantages:
1. Improved error path, currently o-a-c deployments don't report any
errors, thus hang and eventually the deployment times out
2. It's far more hackable from a deployer perspective, e.g it's
much easier to change the os-net-config options or include a
mapping file
3. Reduces our dependencies on o-a-c (it's only os-net-config and hiera
which requires it), although the script does currently still use oac to
get the metadata IP.
4. May enable passing os-net-config yaml via a json parameter in future,
reducing the need for resource_registry mappings (although we'll have to
support that for backwards compatibility)
The script used is based directly on 20-os-net-config (from t-i-e
at cf94c5e, we can probably improve this now that we have an error path,
but for this initial commit it's a straight copy other than the changes to
replace o-a-c for rendering the json config file.
Co-Authored-By: Steven Hardy <shardy@redhat.com>
Change-Id: I0ed08332cfc49a579de2e83960f0d8047690b97a
|
|
The first step of generating the Service chain resources via j2,
we'll then incrementally convert other resources to be created
in a similar way.
Partially-Implements: blueprint custom-roles
Depends-On: I81239991f36ed5f6453184bf9cffe930832cb68b
Change-Id: Iafa9b2afddf18a5a9833ec472a552fb256338b38
|
|
Skips the unused parameter warning for required parameters.
Change-Id: I71ad4ab9f6e6c63e3f01b8cc9c72262f1958331e
|
|
This patch adds a new DefaultPasswords parameter to
composable services. This is needed to help provide
access to top level password resources that overcloud.yaml
currently manages (passwords for Rabbit, Mysql, etc.).
Moving the RandomString resources into composable services
would cause them to regenerate within the stack. With this
approach we can leave them where they are while we deprecate
the top level mechanism and move the code that uses the
passwords into the composable services.
Change-Id: I4f21603c58a169a093962594e860933306879e3f
|
|
This will be needed to pick the network where the service has
to bind to from within the service template.
Change-Id: I52652e1ad8c7b360efd2c7af199e35932aaaea8c
|
|
Currently we use hyphens, e.g cinder-api, but in overcloud.yaml
we have a lot of references to services (e.g for AllNodesConfig)
by underscore, e.g cinder_api. To enable dynamic generation of
this data, we need the service name in underscore format.
Change-Id: Ief13dfe5d8d7691dfe2534ad5c39d7eacbcb6f70
|
|
This patch adds a new service_name section to each composable
service. We now have an explicit unit test check to ensure that
service_name exists in tools/yaml-validate.py.
This patch also wires service_names into hieradata on each
of the roles so that tools can access the deployed services locally
during deployment and upgrades.
Change-Id: I60861c5aa760534db3e314bba16a13b90ea72f0c
|
|
Output a warning for parameters which look unused, this should help
developers clean up the template a bit, and eventually could maybe
be developed further into something we can use for gating.
Change-Id: Ide4fbe3c85854cbddee44801d39ae73003d63bb8
|
|
For developer usage it's helpful to have the choice to provide either
an individual files, list of files, or some mix of files and directories
as you don't necessarily want to walk everything all the time.
Change-Id: I050de123bba51402a0dbb42d71e97fd27d7ce4bc
|
|
Adds a "validate" tox env for basic sanity checking of templates.
Currently it just validates that all of the .yaml files are in fact
valid YAML. In the future we might want to add more, but this
seemed like a reasonable start.
Change-Id: I8091bbad0003b150e23dae5de4f465053c982229
|
Giulio Fidente
Jenkins
Alex Schultz
Steven Hardy
Martin André
James Slagle
Ben Nemec
Saravanan KR
Bogdan Dobrelya
loooosy
John Eckersberg
Ian Main
Juan Antonio Osorio Robles
Jiri Stransky
Dan Prince
Michele Baldessari
Oliver Walsh
Carlos Camacho
Flavio Percoco
Emilien Macchi
Damien Ciabrini
Brent Eagles