summaryrefslogtreecommitdiffstats
path: root/docker/services
diff options
context:
space:
mode:
authorDan Prince <dprince@redhat.com>2017-01-03 22:21:44 -0500
committerDan Prince <dprince@redhat.com>2017-02-15 12:56:44 -0500
commitad2ea290bed17bff9b53ac225d3604ed642ee8bc (patch)
tree75e24d906374e3b17d0a9463a6c00bc44c031913 /docker/services
parentb06e49302d510d0e5f28c585c41c52ef5bc8ad13 (diff)
docker: new hybrid deployment architecture and configuration
This patch implements a new docker deployment architecture that should us to install docker services in a stepwise manner alongside of baremetal puppet services. This works by using Yaql to select docker specific services (docker/services/*.yaml) vs the puppet specific ones and then applying the selected Json to relevant Heat software deployments for docker and baremetal puppet in a stepwise fashion. Additionally the new architecture leverages new composable services interfaces from Newton to allow configuration of per-service container configuration sets (directories that are bind mounted into kolla containers) by using the Kolla containers themselves. It does this by spinning up a throw away "configuration only" version of the container being configured itself, then running the puppet apply in that container and copying the generated config files into /var/lib/config-data. This avoids having to install all of the OpenStack dependency packages in the heat-agent-container itself (our previous approach) and should allow us to configure a much wider variety of container config files that would otherwise be impossible with the previous shared approach. The new approach (combined) should allow us to configure containers in both the undercloud and overcloud and incrementally add CI coverage to services as we containerize them. Co-Authored-By: Martin André <m.andre@redhat.com> Co-Authored-By: Ian Main <imain@redhat.com> Co-Authored-By: Flavio Percoco <flavio@redhat.com> Change-Id: Ibcff99f03e6751fbf3197adefd5d344178b71fc2
Diffstat (limited to 'docker/services')
-rw-r--r--docker/services/README.rst121
-rw-r--r--docker/services/neutron-ovs-agent.yaml36
-rw-r--r--docker/services/nova-compute.yaml38
-rw-r--r--docker/services/nova-libvirt.yaml48
-rw-r--r--docker/services/services.yaml10
5 files changed, 154 insertions, 99 deletions
diff --git a/docker/services/README.rst b/docker/services/README.rst
index 60719bfc..edaa5ee9 100644
--- a/docker/services/README.rst
+++ b/docker/services/README.rst
@@ -1,65 +1,104 @@
-========
-services
-========
+===============
+Docker Services
+===============
-A TripleO nested stack Heat template that encapsulates generic configuration
-data to configure a specific service. This generally includes everything
-needed to configure the service excluding the local bind ports which
-are still managed in the per-node role templates directly (controller.yaml,
-compute.yaml, etc.). All other (global) service settings go into
-the puppet/service templates.
+TripleO docker services are currently built on top of the puppet services.
+To do this each of the docker services includes the output of the
+t-h-t puppet/service templates where appropriate.
-Input Parameters
-----------------
+In general global docker specific service settings should reside in these
+templates (templates in the docker/services directory.) The required and
+optional items are specified in the docker settings section below.
-Each service may define its own input parameters and defaults.
-Operators will use the parameter_defaults section of any Heat
-environment to set per service parameters.
+If you are adding a config setting that applies to both docker and
+baremetal that setting should (so long as we use puppet) go into the
+puppet/services templates themselves.
-Config Settings
----------------
+Building Kolla Images
+---------------------
+
+TripleO currently relies on Kolla docker containers. Kolla supports container
+customization and we are making use of this feature within TripleO to inject
+puppet (our configuration tool of choice) into the Kolla base images. To
+build Kolla images for TripleO adjust your kolla config to build your
+centos base image with puppet using the example below:
+
+.. code-block::
+
+$ cat template-overrides.j2
+{% extends parent_template %}
+{% set base_centos_binary_packages_append = ['puppet'] %}
-Each service may define a config_settings output variable which returns
-Hiera settings to be configured.
+kolla-build --base centos --template-override template-overrides.j2
-Steps
------
+..
+
+Docker settings
+---------------
Each service may define an output variable which returns a puppet manifest
snippet that will run at each of the following steps. Earlier manifests
are re-asserted when applying latter ones.
- * config_settings: Custom hiera settings for this service. These are
- used to generate configs.
+ * config_settings: This setting is generally inherited from the
+ puppet/services templates and only need to be appended
+ to on accasion if docker specific config settings are required.
+
+ * step_config: This setting controls the manifest that is used to
+ create docker config files via puppet. The puppet tags below are
+ used along with this manifest to generate a config directory for
+ this container.
* kolla_config: Contains YAML that represents how to map config files
into the kolla container. This config file is typically mapped into
the container itself at the /var/lib/kolla/config_files/config.json
location and drives how kolla's external config mechanisms work.
- * step_config: A puppet manifest that is used to step through the deployment
- sequence. Each sequence is given a "step" (via hiera('step') that provides
- information for when puppet classes should activate themselves.
+ * docker_image: The full name of the docker image that will be used.
- * docker_compose:
+ * docker_config: Data that is passed to the docker-cmd hook to configure
+ a container, or step of containers at each step. See the available steps
+ below and the related docker-cmd hook documentation in the heat-agents
+ project.
- * container_name:
+ * puppet_tags: Puppet resource tag names that are used to generate config
+ files with puppet. Only the named config resources are used to generate
+ a config file. Any service that specifies tags will have the default
+ tags of 'file,concat,file_line' appended to the setting.
+ Example: keystone_config
- * volumes:
+ * config_volume: The name of the volume (directory) where config files
+ will be generated for this service. Use this as the location to
+ bind mount into the running Kolla container for configuration.
-Steps correlate to the following:
-
- 1) Service configuration generation with puppet.
-
- 2) Early Openstack Service setup (database init?)
-
- 3) Early containerized networking services startup (OVS)
+ * config_image: The name of the docker image that will be used for
+ generating configuration files. This is often the same value as
+ 'docker_image' above but some containers share a common set of
+ config files which are generated in a common base container.
- 4) Network configuration
+Docker steps
+------------
+Similar to baremetal docker containers are brought up in a stepwise manner.
+The current architecture supports bringing up baremetal services alongside
+of containers. For each step the baremetal puppet manifests are executed
+first and then any docker containers are brought up afterwards.
- 5) General OpenStack Services
-
- 6) Service activation (Pacemaker)
-
- 7) Fencing (Pacemaker)
+Steps correlate to the following:
+ Pre) Containers config files generated per hiera settings.
+ 1) Load Balancer configuration baremetal
+ a) step 1 baremetal
+ b) step 1 containers
+ 2) Core Services (Database/Rabbit/NTP/etc.)
+ a) step 2 baremetal
+ b) step 2 containers
+ 3) Early Openstack Service setup (Ringbuilder, etc.)
+ a) step 3 baremetal
+ b) step 3 containers
+ 4) General OpenStack Services
+ a) step 4 baremetal
+ b) step 4 containers
+ c) Keystone containers post initialization (tenant,service,endpoint creation)
+ 5) Service activation (Pacemaker)
+ a) step 5 baremetal
+ b) step 5 containers
diff --git a/docker/services/neutron-ovs-agent.yaml b/docker/services/neutron-ovs-agent.yaml
index 0a061f6c..ab99da5e 100644
--- a/docker/services/neutron-ovs-agent.yaml
+++ b/docker/services/neutron-ovs-agent.yaml
@@ -10,7 +10,7 @@ parameters:
type: string
DockerOpenvswitchImage:
description: image
- default: 'centos-binary-neutron-openvswitch-agent'
+ default: 'centos-binary-neutron-openvswitch-agent:latest'
type: string
ServiceNetMap:
default: {}
@@ -32,53 +32,53 @@ resources:
NeutronOvsAgentBase:
type: ../../puppet/services/neutron-ovs-agent.yaml
properties:
+ EndpointMap: {get_param: EndpointMap}
ServiceNetMap: {get_param: ServiceNetMap}
DefaultPasswords: {get_param: DefaultPasswords}
- EndpointMap: {get_param: EndpointMap}
outputs:
role_data:
description: Role data for Neutron openvswitch service
value:
+ service_name: {get_attr: [NeutronOvsAgentBase, role_data, service_name]}
config_settings: {get_attr: [NeutronOvsAgentBase, role_data, config_settings]}
step_config: {get_attr: [NeutronOvsAgentBase, role_data, step_config]}
+ docker_image: &neutron_ovs_agent_image
+ list_join:
+ - '/'
+ - [ {get_param: DockerNamespace}, {get_param: DockerOpenvswitchImage} ]
puppet_tags: neutron_config,neutron_agent_ovs,neutron_plugin_ml2
+ config_volume: neutron
+ config_image: *neutron_ovs_agent_image
kolla_config:
- /var/lib/etc-data/json-config/neutron-openvswitch-agent.json:
- command: /usr/bin/neutron-openvswitch-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-file /etc/neutron/plugins/ml2/ml2_conf.ini
+ /var/lib/kolla/config_files/neutron-openvswitch-agent.json:
+ command: /usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-file /etc/neutron/plugins/ml2/ml2_conf.ini
config_files:
- dest: /etc/neutron/neutron.conf
owner: neutron
perm: '0600'
- source: /var/lib/kolla/config_files/neutron.conf
+ source: /var/lib/kolla/config_files/src/etc/neutron/neutron.conf
- dest: /etc/neutron/plugins/ml2/openvswitch_agent.ini
owner: neutron
perm: '0600'
- source: /var/lib/kolla/config_files/openvswitch_agent.ini
+ source: /var/lib/kolla/config_files/src/etc/neutron/plugins/ml2/openvswitch_agent.ini
- dest: /etc/neutron/plugins/ml2/ml2_conf.ini
owner: neutron
perm: '0600'
- source: /var/lib/kolla/config_files/ml2_conf.ini
+ source: /var/lib/kolla/config_files/src/etc/neutron/plugins/ml2/ml2_conf.ini
docker_config:
- step_1:
+ step_4:
neutronovsagent:
- image:
- list_join:
- - '/'
- - [ {get_param: DockerNamespace}, {get_param: DockerOpenvswitchImage} ]
+ image: *neutron_ovs_agent_image
net: host
pid: host
privileged: true
restart: always
volumes:
- - /var/lib/etc-data/json-config/neutron-openvswitch-agent.json:/var/lib/kolla/config_files/config.json
- - /var/lib/etc-data/neutron/neutron.conf:/var/lib/kolla/config_files/neutron.conf:ro
- - /var/lib/etc-data/neutron/plugins/ml2/ml2_conf.ini:/var/lib/kolla/config_files/ml2_conf.ini:ro
- - /var/lib/etc-data/neutron/plugins/ml2/openvswitch_agent.ini:/var/lib/kolla/config_files/openvswitch_agent.ini:ro
+ - /var/lib/kolla/config_files/neutron-openvswitch-agent.json:/var/lib/kolla/config_files/config.json:ro
+ - /var/lib/config-data/neutron:/var/lib/kolla/config_files/src:ro
- /etc/localtime:/etc/localtime:ro
- /lib/modules:/lib/modules:ro
- /run:/run
- - logs:/var/log/kolla/
environment:
- KOLLA_CONFIG_STRATEGY=COPY_ALWAYS
- step_2: {}
diff --git a/docker/services/nova-compute.yaml b/docker/services/nova-compute.yaml
index e765609e..8eebc397 100644
--- a/docker/services/nova-compute.yaml
+++ b/docker/services/nova-compute.yaml
@@ -10,7 +10,7 @@ parameters:
type: string
DockerNovaComputeImage:
description: image
- default: 'centos-binary-nova-compute'
+ default: 'centos-binary-nova-compute:latest'
type: string
ServiceNetMap:
default: {}
@@ -29,53 +29,57 @@ parameters:
resources:
+
NovaComputeBase:
type: ../../puppet/services/nova-compute.yaml
properties:
EndpointMap: {get_param: EndpointMap}
+ ServiceNetMap: {get_param: ServiceNetMap}
+ DefaultPasswords: {get_param: DefaultPasswords}
outputs:
role_data:
description: Role data for the Nova Compute service.
value:
+ service_name: {get_attr: [NovaComputeBase, role_data, service_name]}
config_settings: {get_attr: [NovaComputeBase, role_data, config_settings]}
step_config: {get_attr: [NovaComputeBase, role_data, step_config]}
puppet_tags: nova_config,nova_paste_api_ini
+ docker_image: &nova_compute_image
+ list_join:
+ - '/'
+ - [ {get_param: DockerNamespace}, {get_param: DockerNovaComputeImage} ]
+ config_volume: nova_libvirt
+ config_image: *nova_compute_image
kolla_config:
- /var/lib/etc-data/json-config/nova-compute.json:
+ /var/lib/kolla/config_files/nova-compute.json:
command: /usr/bin/nova-compute --config-file /etc/nova/nova.conf --config-file /etc/nova/rootwrap.conf
config_files:
- dest: /etc/nova/nova.conf
owner: nova
perm: '0600'
- source: /var/lib/kolla/config_files/nova.conf
+ source: /var/lib/kolla/config_files/src/etc/nova/nova.conf
- dest: /etc/nova/rootwrap.conf
owner: nova
perm: '0600'
- source: /var/lib/kolla/config_files/rootwrap.conf
+ source: /var/lib/kolla/config_files/src/etc/nova/rootwrap.conf
docker_config:
- step_1:
+ step_4:
novacompute:
- image:
- list_join:
- - '/'
- - [ {get_param: DockerNamespace}, {get_param: DockerNovaComputeImage} ]
+ image: *nova_compute_image
net: host
privileged: true
user: root
restart: always
volumes:
- - /var/lib/etc-data/json-config/nova-compute.json:/var/lib/kolla/config_files/config.json
- - /var/lib/etc-data/nova/nova.conf:/var/lib/kolla/config_files/nova.conf:ro
- - /var/lib/etc-data/nova/rootwrap.conf:/var/lib/kolla/config_files/rootwrap.conf:ro
+ - /var/lib/kolla/config_files/nova-compute.json:/var/lib/kolla/config_files/config.json:ro
+ - /var/lib/config-data/nova_libvirt:/var/lib/kolla/config_files/src:ro
+ - /dev:/dev
+ - /etc/iscsi:/etc/iscsi
- /etc/localtime:/etc/localtime:ro
- /lib/modules:/lib/modules:ro
- /run:/run
- - /dev:/dev
- - logs:/var/log/kolla/
- - /etc/iscsi:/etc/iscsi
+ - /var/lib/nova:/var/lib/nova
- libvirtd:/var/lib/libvirt
- - nova_compute:/var/lib/nova/
environment:
- KOLLA_CONFIG_STRATEGY=COPY_ALWAYS
- step_2: {}
diff --git a/docker/services/nova-libvirt.yaml b/docker/services/nova-libvirt.yaml
index 004d624a..d6e7dc76 100644
--- a/docker/services/nova-libvirt.yaml
+++ b/docker/services/nova-libvirt.yaml
@@ -10,7 +10,13 @@ parameters:
type: string
DockerLibvirtImage:
description: image
- default: 'centos-binary-libvirt'
+ default: 'centos-binary-nova-libvirt:latest'
+ type: string
+ # we configure libvirt via the nova-compute container due to coupling
+ # in the puppet modules
+ DockerNovaComputeImage:
+ description: image
+ default: 'centos-binary-nova-compute:latest'
type: string
ServiceNetMap:
default: {}
@@ -33,50 +39,54 @@ resources:
type: ../../puppet/services/nova-libvirt.yaml
properties:
EndpointMap: {get_param: EndpointMap}
+ ServiceNetMap: {get_param: ServiceNetMap}
+ DefaultPasswords: {get_param: DefaultPasswords}
outputs:
role_data:
description: Role data for the Libvirt service.
value:
+ service_name: {get_attr: [NovaLibvirtBase, role_data, service_name]}
config_settings: {get_attr: [NovaLibvirtBase, role_data, config_settings]}
step_config: {get_attr: [NovaLibvirtBase, role_data, step_config]}
+ docker_image: &libvirt_image
+ list_join:
+ - '/'
+ - [ {get_param: DockerNamespace}, {get_param: DockerLibvirtImage} ]
puppet_tags: nova_config
+ config_volume: nova_libvirt
+ config_image:
+ list_join:
+ - '/'
+ - [ {get_param: DockerNamespace}, {get_param: DockerNovaComputeImage} ]
kolla_config:
- /var/lib/etc-data/json-config/nova-libvirt.json:
+ /var/lib/kolla/config_files/nova-libvirt.json:
command: /usr/sbin/libvirtd --config /etc/libvirt/libvirtd.conf
config_files:
- dest: /etc/libvirt/libvirtd.conf
owner: root
perm: '0644'
- source: /var/lib/kolla/config_files/libvirtd.conf
+ source: /var/lib/kolla/config_files/src/etc/libvirt/libvirtd.conf
docker_config:
- step_1:
+ step_3:
nova_libvirt:
- image:
- list_join:
- - '/'
- - [ {get_param: DockerNamespace}, {get_param: DockerLibvirtImage} ]
+ image: *libvirt_image
net: host
pid: host
privileged: true
restart: always
volumes:
- - /var/lib/etc-data/json-config/nova-libvirt.json:/var/lib/kolla/config_files/config.json
- - /var/lib/etc-data/libvirt/libvirtd.conf:/var/lib/kolla/config_files/libvirtd.conf
- # NOTE(mandre) Ideally the qemu.conf file is mounted in
- # /var/lib/kolla/config_files and copied to the right place but
- # copy-json.py doesn't allow us to do that without appending the
- # file as an additional config on the CLI
- - /var/lib/etc-data/libvirt/qemu.conf:/etc/libvirt/qemu.conf:ro
+ - /var/lib/kolla/config_files/nova-libvirt.json:/var/lib/kolla/config_files/config.json:ro
+ - /var/lib/config-data/nova_libvirt:/var/lib/kolla/config_files/src:ro
+ - /dev:/dev
- /etc/localtime:/etc/localtime:ro
- /lib/modules:/lib/modules:ro
- /run:/run
- - /dev:/dev
- /sys/fs/cgroup:/sys/fs/cgroup
- - logs:/var/log/kolla/
+ - /var/lib/nova:/var/lib/nova
+ # Needed to use host's virtlogd
+ - /var/run/libvirt:/var/run/libvirt
- libvirtd:/var/lib/libvirt
- - nova_compute:/var/lib/nova/
- nova_libvirt_qemu:/etc/libvirt/qemu
environment:
- KOLLA_CONFIG_STRATEGY=COPY_ALWAYS
- step_2: {}
diff --git a/docker/services/services.yaml b/docker/services/services.yaml
index 8c31107f..8e899024 100644
--- a/docker/services/services.yaml
+++ b/docker/services/services.yaml
@@ -66,10 +66,12 @@ outputs:
global_config_settings:
{get_attr: [PuppetServices, role_data, global_config_settings]}
step_config:
- {get_attr: [PuppetServices, role_data, step_config]}
- puppet_tags: {list_join: [",", {get_attr: [ServiceChain, role_data, puppet_tags]}]}
+ {get_attr: [ServiceChain, role_data, step_config]}
+ docker_image: {get_attr: [ServiceChain, role_data, docker_image]}
+ puppet_tags: {get_attr: [ServiceChain, role_data, puppet_tags]}
+ config_volume: {get_attr: [ServiceChain, role_data, config_volume]}
+ config_image: {get_attr: [ServiceChain, role_data, config_image]}
kolla_config:
map_merge: {get_attr: [ServiceChain, role_data, kolla_config]}
docker_config:
- step_1: {map_merge: {get_attr: [ServiceChain, role_data, docker_config, step_1]}}
- step_2: {map_merge: {get_attr: [ServiceChain, role_data, docker_config, step_2]}}
+ {get_attr: [ServiceChain, role_data, docker_config]}