Merge "docker: new hybrid deployment architecture and configuration"

5 files changed, 154 insertions, 99 deletions

diff --git a/docker/services/README.rst b/docker/services/README.rst
index 60719bfc..edaa5ee9 100644
--- a/docker/services/README.rst
+++ b/docker/services/README.rst
@@ -1,65 +1,104 @@
-========
-services
-========
+===============
+Docker Services
+===============
 
-A TripleO nested stack Heat template that encapsulates generic configuration
-data to configure a specific service. This generally includes everything
-needed to configure the service excluding the local bind ports which
-are still managed in the per-node role templates directly (controller.yaml,
-compute.yaml, etc.). All other (global) service settings go into
-the puppet/service templates.
+TripleO docker services are currently built on top of the puppet services.
+To do this each of the docker services includes the output of the
+t-h-t puppet/service templates where appropriate.
 
-Input Parameters
-----------------
+In general global docker specific service settings should reside in these
+templates (templates in the docker/services directory.) The required and
+optional items are specified in the docker settings section below.
 
-Each service may define its own input parameters and defaults.
-Operators will use the parameter_defaults section of any Heat
-environment to set per service parameters.
+If you are adding a config setting that applies to both docker and
+baremetal that setting should (so long as we use puppet) go into the
+puppet/services templates themselves.
 
-Config Settings
----------------
+Building Kolla Images
+---------------------
+
+TripleO currently relies on Kolla docker containers. Kolla supports container
+customization and we are making use of this feature within TripleO to inject
+puppet (our configuration tool of choice) into the Kolla base images. To
+build Kolla images for TripleO adjust your kolla config to build your
+centos base image with puppet using the example below:
+
+.. code-block::
+
+$ cat template-overrides.j2
+{% extends parent_template %}
+{% set base_centos_binary_packages_append = ['puppet'] %}
 
-Each service may define a config_settings output variable which returns
-Hiera settings to be configured.
+kolla-build --base centos --template-override template-overrides.j2
 
-Steps
------
+..
 
+
+Docker settings
+---------------
 Each service may define an output variable which returns a puppet manifest
 snippet that will run at each of the following steps. Earlier manifests
 are re-asserted when applying latter ones.
 
- * config_settings: Custom hiera settings for this service. These are
-   used to generate configs.
+ * config_settings: This setting is generally inherited from the
+   puppet/services templates and only need to be appended
+   to on accasion if docker specific config settings are required.
+
+ * step_config: This setting controls the manifest that is used to
+   create docker config files via puppet. The puppet tags below are
+   used along with this manifest to generate a config directory for
+   this container.
 
  * kolla_config: Contains YAML that represents how to map config files
    into the kolla container. This config file is typically mapped into
    the container itself at the /var/lib/kolla/config_files/config.json
    location and drives how kolla's external config mechanisms work.
 
- * step_config: A puppet manifest that is used to step through the deployment
-   sequence. Each sequence is given a "step" (via hiera('step') that provides
-   information for when puppet classes should activate themselves.
+ * docker_image: The full name of the docker image that will be used.
 
- * docker_compose:
+ * docker_config: Data that is passed to the docker-cmd hook to configure
+   a container, or step of containers at each step. See the available steps
+   below and the related docker-cmd hook documentation in the heat-agents
+   project.
 
- * container_name:
+ * puppet_tags: Puppet resource tag names that are used to generate config
+   files with puppet. Only the named config resources are used to generate
+   a config file. Any service that specifies tags will have the default
+   tags of 'file,concat,file_line' appended to the setting.
+   Example: keystone_config
 
- * volumes:
+ * config_volume: The name of the volume (directory) where config files
+   will be generated for this service. Use this as the location to
+   bind mount into the running Kolla container for configuration.
 
-Steps correlate to the following:
-
-   1) Service configuration generation with puppet.
-
-   2) Early Openstack Service setup (database init?)
-
-   3) Early containerized networking services startup (OVS)
+ * config_image: The name of the docker image that will be used for
+   generating configuration files. This is often the same value as
+   'docker_image' above but some containers share a common set of
+   config files which are generated in a common base container.
 
-   4) Network configuration
+Docker steps
+------------
+Similar to baremetal docker containers are brought up in a stepwise manner.
+The current architecture supports bringing up baremetal services alongside
+of containers. For each step the baremetal puppet manifests are executed
+first and then any docker containers are brought up afterwards.
 
-   5) General OpenStack Services
-
-   6) Service activation (Pacemaker)
-
-   7) Fencing (Pacemaker)
+Steps correlate to the following:
 
+   Pre) Containers config files generated per hiera settings.
+   1) Load Balancer configuration baremetal
+     a) step 1 baremetal
+     b) step 1 containers
+   2) Core Services (Database/Rabbit/NTP/etc.)
+     a) step 2 baremetal
+     b) step 2 containers
+   3) Early Openstack Service setup (Ringbuilder, etc.)
+     a) step 3 baremetal
+     b) step 3 containers
+   4) General OpenStack Services
+     a) step 4 baremetal
+     b) step 4 containers
+     c) Keystone containers post initialization (tenant,service,endpoint creation)
+   5) Service activation (Pacemaker)
+     a) step 5 baremetal
+     b) step 5 containers
diff --git a/docker/services/neutron-ovs-agent.yaml b/docker/services/neutron-ovs-agent.yaml
index 0a061f6c..ab99da5e 100644
--- a/docker/services/neutron-ovs-agent.yaml
+++ b/docker/services/neutron-ovs-agent.yaml
@@ -10,7 +10,7 @@ parameters:
     type: string
   DockerOpenvswitchImage:
     description: image
-    default: 'centos-binary-neutron-openvswitch-agent'
+    default: 'centos-binary-neutron-openvswitch-agent:latest'
     type: string
   ServiceNetMap:
     default: {}
@@ -32,53 +32,53 @@ resources:
   NeutronOvsAgentBase:
     type: ../../puppet/services/neutron-ovs-agent.yaml
     properties:
+      EndpointMap: {get_param: EndpointMap}
       ServiceNetMap: {get_param: ServiceNetMap}
       DefaultPasswords: {get_param: DefaultPasswords}
-      EndpointMap: {get_param: EndpointMap}
 
 outputs:
   role_data:
     description: Role data for Neutron openvswitch service
     value:
+      service_name: {get_attr: [NeutronOvsAgentBase, role_data, service_name]}
       config_settings: {get_attr: [NeutronOvsAgentBase, role_data, config_settings]}
       step_config: {get_attr: [NeutronOvsAgentBase, role_data, step_config]}
+      docker_image: &neutron_ovs_agent_image
+        list_join:
+        - '/'
+        - [ {get_param: DockerNamespace}, {get_param: DockerOpenvswitchImage} ]
       puppet_tags: neutron_config,neutron_agent_ovs,neutron_plugin_ml2
+      config_volume: neutron
+      config_image: *neutron_ovs_agent_image
       kolla_config:
-        /var/lib/etc-data/json-config/neutron-openvswitch-agent.json:
-           command: /usr/bin/neutron-openvswitch-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-file /etc/neutron/plugins/ml2/ml2_conf.ini
+        /var/lib/kolla/config_files/neutron-openvswitch-agent.json:
+           command: /usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-file /etc/neutron/plugins/ml2/ml2_conf.ini
            config_files:
            - dest: /etc/neutron/neutron.conf
              owner: neutron
              perm: '0600'
-             source: /var/lib/kolla/config_files/neutron.conf
+             source: /var/lib/kolla/config_files/src/etc/neutron/neutron.conf
            - dest: /etc/neutron/plugins/ml2/openvswitch_agent.ini
              owner: neutron
              perm: '0600'
-             source: /var/lib/kolla/config_files/openvswitch_agent.ini
+             source: /var/lib/kolla/config_files/src/etc/neutron/plugins/ml2/openvswitch_agent.ini
            - dest: /etc/neutron/plugins/ml2/ml2_conf.ini
              owner: neutron
              perm: '0600'
-             source: /var/lib/kolla/config_files/ml2_conf.ini
+             source: /var/lib/kolla/config_files/src/etc/neutron/plugins/ml2/ml2_conf.ini
       docker_config:
-        step_1:
+        step_4:
           neutronovsagent:
-            image:
-              list_join:
-              - '/'
-              - [ {get_param: DockerNamespace}, {get_param: DockerOpenvswitchImage} ]
+            image: *neutron_ovs_agent_image
             net: host
             pid: host
             privileged: true
             restart: always
             volumes:
-              - /var/lib/etc-data/json-config/neutron-openvswitch-agent.json:/var/lib/kolla/config_files/config.json
-              - /var/lib/etc-data/neutron/neutron.conf:/var/lib/kolla/config_files/neutron.conf:ro
-              - /var/lib/etc-data/neutron/plugins/ml2/ml2_conf.ini:/var/lib/kolla/config_files/ml2_conf.ini:ro
-              - /var/lib/etc-data/neutron/plugins/ml2/openvswitch_agent.ini:/var/lib/kolla/config_files/openvswitch_agent.ini:ro
+              - /var/lib/kolla/config_files/neutron-openvswitch-agent.json:/var/lib/kolla/config_files/config.json:ro
+              - /var/lib/config-data/neutron:/var/lib/kolla/config_files/src:ro
               - /etc/localtime:/etc/localtime:ro
               - /lib/modules:/lib/modules:ro
               - /run:/run
-              - logs:/var/log/kolla/
             environment:
               - KOLLA_CONFIG_STRATEGY=COPY_ALWAYS
-        step_2: {}
diff --git a/docker/services/nova-compute.yaml b/docker/services/nova-compute.yaml
index e765609e..8eebc397 100644
--- a/docker/services/nova-compute.yaml
+++ b/docker/services/nova-compute.yaml
@@ -10,7 +10,7 @@ parameters:
     type: string
   DockerNovaComputeImage:
     description: image
-    default: 'centos-binary-nova-compute'
+    default: 'centos-binary-nova-compute:latest'
     type: string
   ServiceNetMap:
     default: {}
@@ -29,53 +29,57 @@ parameters:
 
 resources:
 
+
   NovaComputeBase:
     type: ../../puppet/services/nova-compute.yaml
     properties:
       EndpointMap: {get_param: EndpointMap}
+      ServiceNetMap: {get_param: ServiceNetMap}
+      DefaultPasswords: {get_param: DefaultPasswords}
 
 outputs:
   role_data:
     description: Role data for the Nova Compute service.
     value:
+      service_name: {get_attr: [NovaComputeBase, role_data, service_name]}
       config_settings: {get_attr: [NovaComputeBase, role_data, config_settings]}
       step_config: {get_attr: [NovaComputeBase, role_data, step_config]}
       puppet_tags: nova_config,nova_paste_api_ini
+      docker_image: &nova_compute_image
+        list_join:
+        - '/'
+        - [ {get_param: DockerNamespace}, {get_param: DockerNovaComputeImage} ]
+      config_volume: nova_libvirt
+      config_image: *nova_compute_image
       kolla_config:
-        /var/lib/etc-data/json-config/nova-compute.json:
+        /var/lib/kolla/config_files/nova-compute.json:
            command: /usr/bin/nova-compute --config-file /etc/nova/nova.conf --config-file /etc/nova/rootwrap.conf
            config_files:
            - dest: /etc/nova/nova.conf
              owner: nova
              perm: '0600'
-             source: /var/lib/kolla/config_files/nova.conf
+             source: /var/lib/kolla/config_files/src/etc/nova/nova.conf
            - dest: /etc/nova/rootwrap.conf
              owner: nova
              perm: '0600'
-             source: /var/lib/kolla/config_files/rootwrap.conf
+             source: /var/lib/kolla/config_files/src/etc/nova/rootwrap.conf
       docker_config:
-        step_1:
+        step_4:
           novacompute:
-            image:
-              list_join:
-              - '/'
-              - [ {get_param: DockerNamespace}, {get_param: DockerNovaComputeImage} ]
+            image: *nova_compute_image
             net: host
             privileged: true
             user: root
             restart: always
             volumes:
-              - /var/lib/etc-data/json-config/nova-compute.json:/var/lib/kolla/config_files/config.json
-              - /var/lib/etc-data/nova/nova.conf:/var/lib/kolla/config_files/nova.conf:ro
-              - /var/lib/etc-data/nova/rootwrap.conf:/var/lib/kolla/config_files/rootwrap.conf:ro
+              - /var/lib/kolla/config_files/nova-compute.json:/var/lib/kolla/config_files/config.json:ro
+              - /var/lib/config-data/nova_libvirt:/var/lib/kolla/config_files/src:ro
+              - /dev:/dev
+              - /etc/iscsi:/etc/iscsi
               - /etc/localtime:/etc/localtime:ro
               - /lib/modules:/lib/modules:ro
               - /run:/run
-              - /dev:/dev
-              - logs:/var/log/kolla/
-              - /etc/iscsi:/etc/iscsi
+              - /var/lib/nova:/var/lib/nova
               - libvirtd:/var/lib/libvirt
-              - nova_compute:/var/lib/nova/
             environment:
              - KOLLA_CONFIG_STRATEGY=COPY_ALWAYS
-        step_2: {}
diff --git a/docker/services/nova-libvirt.yaml b/docker/services/nova-libvirt.yaml
index 004d624a..d6e7dc76 100644
--- a/docker/services/nova-libvirt.yaml
+++ b/docker/services/nova-libvirt.yaml
@@ -10,7 +10,13 @@ parameters:
     type: string
   DockerLibvirtImage:
     description: image
-    default: 'centos-binary-libvirt'
+    default: 'centos-binary-nova-libvirt:latest'
+    type: string
+  # we configure libvirt via the nova-compute container due to coupling
+  # in the puppet modules
+  DockerNovaComputeImage:
+    description: image
+    default: 'centos-binary-nova-compute:latest'
     type: string
   ServiceNetMap:
     default: {}
@@ -33,50 +39,54 @@ resources:
     type: ../../puppet/services/nova-libvirt.yaml
     properties:
       EndpointMap: {get_param: EndpointMap}
+      ServiceNetMap: {get_param: ServiceNetMap}
+      DefaultPasswords: {get_param: DefaultPasswords}
 
 outputs:
   role_data:
     description: Role data for the Libvirt service.
     value:
+      service_name: {get_attr: [NovaLibvirtBase, role_data, service_name]}
       config_settings: {get_attr: [NovaLibvirtBase, role_data, config_settings]}
       step_config: {get_attr: [NovaLibvirtBase, role_data, step_config]}
+      docker_image: &libvirt_image
+        list_join:
+        - '/'
+        - [ {get_param: DockerNamespace}, {get_param: DockerLibvirtImage} ]
       puppet_tags: nova_config
+      config_volume: nova_libvirt
+      config_image:
+        list_join:
+        - '/'
+        - [ {get_param: DockerNamespace}, {get_param: DockerNovaComputeImage} ]
       kolla_config:
-        /var/lib/etc-data/json-config/nova-libvirt.json:
+        /var/lib/kolla/config_files/nova-libvirt.json:
            command: /usr/sbin/libvirtd --config /etc/libvirt/libvirtd.conf
            config_files:
            - dest: /etc/libvirt/libvirtd.conf
              owner: root
              perm: '0644'
-             source: /var/lib/kolla/config_files/libvirtd.conf
+             source: /var/lib/kolla/config_files/src/etc/libvirt/libvirtd.conf
       docker_config:
-        step_1:
+        step_3:
           nova_libvirt:
-            image:
-              list_join:
-                - '/'
-                - [ {get_param: DockerNamespace}, {get_param: DockerLibvirtImage} ]
+            image: *libvirt_image
             net: host
             pid: host
             privileged: true
             restart: always
             volumes:
-              - /var/lib/etc-data/json-config/nova-libvirt.json:/var/lib/kolla/config_files/config.json
-              - /var/lib/etc-data/libvirt/libvirtd.conf:/var/lib/kolla/config_files/libvirtd.conf
-              # NOTE(mandre) Ideally the qemu.conf file is mounted in
-              # /var/lib/kolla/config_files and copied to the right place but
-              # copy-json.py doesn't allow us to do that without appending the
-              # file as an additional config on the CLI
-              - /var/lib/etc-data/libvirt/qemu.conf:/etc/libvirt/qemu.conf:ro
+              - /var/lib/kolla/config_files/nova-libvirt.json:/var/lib/kolla/config_files/config.json:ro
+              - /var/lib/config-data/nova_libvirt:/var/lib/kolla/config_files/src:ro
+              - /dev:/dev
               - /etc/localtime:/etc/localtime:ro
               - /lib/modules:/lib/modules:ro
               - /run:/run
-              - /dev:/dev
               - /sys/fs/cgroup:/sys/fs/cgroup
-              - logs:/var/log/kolla/
+              - /var/lib/nova:/var/lib/nova
+              # Needed to use host's virtlogd
+              - /var/run/libvirt:/var/run/libvirt
               - libvirtd:/var/lib/libvirt
-              - nova_compute:/var/lib/nova/
               - nova_libvirt_qemu:/etc/libvirt/qemu
             environment:
               - KOLLA_CONFIG_STRATEGY=COPY_ALWAYS
-        step_2: {}
diff --git a/docker/services/services.yaml b/docker/services/services.yaml
index 8c31107f..8e899024 100644
--- a/docker/services/services.yaml
+++ b/docker/services/services.yaml
@@ -66,10 +66,12 @@ outputs:
       global_config_settings:
         {get_attr: [PuppetServices, role_data, global_config_settings]}
       step_config:
-        {get_attr: [PuppetServices, role_data, step_config]}
-      puppet_tags: {list_join: [",", {get_attr: [ServiceChain, role_data, puppet_tags]}]}
+        {get_attr: [ServiceChain, role_data, step_config]}
+      docker_image: {get_attr: [ServiceChain, role_data, docker_image]}
+      puppet_tags: {get_attr: [ServiceChain, role_data, puppet_tags]}
+      config_volume: {get_attr: [ServiceChain, role_data, config_volume]}
+      config_image: {get_attr: [ServiceChain, role_data, config_image]}
       kolla_config:
         map_merge: {get_attr: [ServiceChain, role_data, kolla_config]}
       docker_config:
-        step_1: {map_merge: {get_attr: [ServiceChain, role_data, docker_config, step_1]}}
-        step_2: {map_merge: {get_attr: [ServiceChain, role_data, docker_config, step_2]}}
+        {get_attr: [ServiceChain, role_data, docker_config]}