aboutsummaryrefslogtreecommitdiffstats
path: root/docker/services/swift-storage.yaml
AgeCommit message (Collapse)AuthorFilesLines
2017-10-19Disable xinetd class when creating swift-storage puppet configurationMichele Baldessari1-1/+4
Due to missing puppet invocation with --detailed-exitcodes we ignored a large amount of puppet errors during deploy. Swift storage fails during the puppet_config step with the following error: Debug: /Stage[main]/Swift::Storage::Object/Swift::Storage::Generic[object]/Package[swift-object]: Not tagged with file, file_line, concat, augeas, cron, swif t_proxy_config, swift_config, swift_container_config, swift_container_sync_realms_config, swift_account_config, swift_object_config, swift_object_expirer_con fig, rsync::server Debug: /Stage[main]/Swift::Storage::Object/Swift::Storage::Generic[object]/Package[swift-object]: Resource is being skipped, unscheduling all events Debug: Executing: '/usr/bin/systemctl is-active xinetd' Debug: Executing: '/usr/bin/systemctl is-enabled xinetd' Debug: Executing: '/usr/bin/systemctl unmask xinetd' Debug: Executing: '/usr/bin/systemctl start xinetd' Debug: Runing journalctl command to get logs for systemd start failure: journalctl -n 50 --since '5 minutes ago' -u xinetd --no-pager Debug: Executing: 'journalctl -n 50 --since '5 minutes ago' -u xinetd --no-pager' Error: Systemd start for xinetd failed! The problem is that by using the rsync::server tag we end up including the xinetd class automatically which will try to start a service inside a container. By nooping the xinetd class, we're able avoid systemctl calls and have a successfuly deployment. The resulting swift_rsync container seems to work correctly: [root@overcloud-controller-0 ~]# docker exec -it swift_rsync /bin/bash -c "ps -axuwf" USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 10 0.0 0.0 47444 1624 pts/1 Rs+ 18:16 0:00 ps -axuwf root 1 0.0 0.0 188 4 ? Ss 17:27 0:00 /usr/local/bin/dumb-init /bin/bash /usr/local/bin/kolla_start root 6 0.0 0.0 11036 924 ? Ss 17:27 0:00 /usr/bin/rsync --daemon --no-detach --config=/etc/rsyncd.conf [root@overcloud-controller-0 ~]# docker logs swift_rsync 2>&1|tail -n4 INFO:__main__:Deleting /etc/rsyncd.conf INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/rsyncd.conf to /etc/rsyncd.conf INFO:__main__:Writing out command to execute Running command: '/usr/bin/rsync --daemon --no-detach --config=/etc/rsyncd.conf' Change-Id: I5e43e8fd61e002d2acc56a7de52e6aae64ab60be Closes-Bug: #1723463 (cherry picked from commit b5eeeab73e12efecc86ea7deebc105eee0739510)
2017-10-09Remove package if service stopped and disabledmarios1-1/+13
Adds a UpgradeRemoveUnusedPackages param to use in the ansible when conditional for the removal Adds package removal to step2 right after a service is stopped and disabled on step2. Package updates happen in step3 so ideally remove before that. The package removal task has ignore_errors true so dependencies or other issue removing packages will not fail the upgrade workflow. Also adds this to the upgrade environment files for visibility and defaulting false Change-Id: Ie4e4a2d41f7752c5a13507a7c15c6f68e203cfca Related-Bug: 1701501 (cherry picked from commit ce0ef2fa207698c1ae61c1620fe3c5e8d1c7bfca)
2017-10-09docker: add logging(source & groups)Juan Badia Payno1-0/+2
The services that docker depends on, have logging_sources and logging_groups; but those are not set on the docker outputs so they are not used when dockers are deployed. Added logging_source & logging_groups as docker optional parameters in tools/yaml-validate.py Closes-Bug: #1718110 Change-Id: I8795eaf4bd06051e9b94aa50450dee0d8761e526 (cherry picked from commit 5dbe1121e98a794ec6a6387ff56ee34314177567)
2017-07-18Stop also openstack-swift-object-expirer when upgrading swift servicesCarlos Camacho1-0/+1
openstack-swift-object-expirer is not stopped when running the upgrade tasks so forth when changing to containers the service is still running after upgrading to docker. This service is added by default here: https://review.openstack.org/#/c/404149 But it wasnt stopped when running the upgrade tasks. Related also to this RHBZ#1470005 Change-Id: I8d5f195095d702057c3b2741127e7338d7451aad Closes-Bug: 1699444
2017-07-15Merge "Use a single configuration file for specifying docker containers."Jenkins1-28/+5
2017-07-14Use a single configuration file for specifying docker containers.Ian Main1-28/+5
This removes the default container names from all the templates and uses a single environment file to specify the full container name and registry from which to pull. Also does away with most of DockerNamespace. Change-Id: Ieaedac33f0a25a352ab432cdb00b5c888be4ba27 Depends-On: Ibc108871ebc2beb1baae437105b2da1d0123ba60 Co-Authored-By: Dan Prince <dprince@redhat.com> Co-Authored-By: Steve Baker <sbaker@redhat.com>
2017-07-14Adds network/cidr mapping into a new service propertyGiulio Fidente1-0/+5
Makes it possible to resolve network subnets within a service template; the data is transported into a new property ServiceData wired into every service which hopefully is generic enough to be extended in the future and transport more data. Data can be consumed in service templates to set config values which need to know what is the subnet where a deamon operates (for example the Ceph Public vs Cluster network). Change-Id: I28e21c46f1ef609517175f7e7ee19e28d1c0cba2
2017-07-12Run rsync for Swift without xinetdChristian Schwede1-4/+4
The default in non-containerized environments is to run rsync within xinetd for Red Hat-based deployments, however in an containerized environment this is not really needed. Therefore run rsync directly without being started by xinetd. Change-Id: I08abd917eba08d1192437ddf96c71b06d099a3f8
2017-07-10Copy only generated puppet files into the containerMartin André1-14/+84
This solves a problem with bind-mounts when the containers are holding files descriptors open. At the same time this makes the template more robust to puppet changes since new config files will be available in the containers without needing to update the templates. Partial-Bug: #1698323 Change-Id: Ia4ad6d77387e3dc354cd131c2f9756939fb8f736
2017-06-30Merge "Add missing xinetd/rsync container for Swift"Jenkins1-1/+21
2017-06-28Add missing xinetd/rsync container for SwiftChristian Schwede1-1/+21
Swift object replication relies on the rsync server, which is run by xinetd. This patch adds the missing container and configuration. Note that xinetd needs bind to a privileged port (873) and has to be started as root therefore. Change-Id: I7655c9dd116c0130035d8a2fae81148171ae6448
2017-06-28Add heat parameter for all of config_volume imagesMartin André1-3/+10
This commit consistently defines a heat template parameter in the form of DockerXXXConfigImage where XXX represents the name of the config_volume that is used by docker-puppet. The goal is to mitigate hard to debug errors where the templates would set different defaults for the image docker-puppet.py uses to run, for the same config_volume name. This fixes a couple of inconsistencies on the way. Change-Id: I212020a76622a03521385a6cae4ce73e51ce5b6b Closes-Bug: #1699791
2017-06-08Fix containerized SwiftRawDisks usageChristian Schwede1-1/+22
This patch partitions the defined devices and mounts them on the hostnode. It also disables the mount_check inside Swift because it is currently not possible to detect wether a given directory is a mounted device or not. This is just a workaround until a better solution has been implemented in Swift itself. Change-Id: I6e8e1328d7ffb18bb96ed1a940013dbb8b6b433e
2017-05-15Add missing type for RoleParameters parameterMartin André1-0/+1
This was forgotten in I72376a803ec6b2ed93903cc0c95a6ffce718b6dc and broke containerized deployment. Change-Id: I599a87bf06efbfefd3067c77ed6ca866505900f9 Closes-Bug: #1690870
2017-05-15Add role specific information to the service templateSaravanan KR1-0/+9
When a service is enabled on multiple roles, the parameters for the service will be global. This change enables an option to provide role specific parameter to services and other templates. Two new parameters - RoleName and RoleParameters, are added to the service template. RoleName provides the role name of on which the current instance of the service is being applied on. RoleParameters provides the list of parameters which are configured specific to the role in the environment file, like below: parameters_default: # Default value for applied to all roles NovaReservedHostMemory: 2048 ComputeDpdkParameters: # Applied only to ComputeDpdk role NovaReservedHostMemory: 4096 In above sample, the cluster contains 2 roles - Compute, ComputeDpdk. The values of ComputeDpdkParameters will be passed on to the templates as RoleParameters while creating the stack for ComputeDpdk role. The parameter which supports role specific configuration, should find the parameter first in in the RoleParameters list, if not found, then the default (for all roles) should be used. Implements: blueprint tripleo-derive-parameters Change-Id: I72376a803ec6b2ed93903cc0c95a6ffce718b6dc
2017-05-05Mount hostpath logs on /var/logFlavio Percoco1-2/+22
Some containers are using the logs named volume for collecting logs written to `/var/log`. We should make this consistent for all the containers. This patch also cleans up some mounts that weren't needed for some services. For example, glance-api doesn't need `/run` to be mounted. Other changes: * Rework log volumes to hostpath mounts to omit slow COW writes. * Add kolla_config's permission and host_prep_tasks create and manage hostpath mounted log dirs permissions. * Rework data owning init containers to kolla_config permissions * When a step wants KOLLA_BOOTSTRAP or DB sync, use logs data owning init containers to set permissions for logs. This is required because kolla bootsrap and DB sync runs before the kolla config stage and there is yet permissions set for logs. * In order to address hybrid cases for host services vs containerized ones to access logs having different UIDs, persist containerized services' logs into separate directories (an upgrade impact) * Ensure host prep tasks to create /var/log/containers/ and /var/lib/ sub-directories for services * Fix missing /etc/httpd, /var/www config-data mounts for zaqar/ironic * Fix YAML indentation and drop strings quotation. Co-authored-by: Bogdan Dobrelya <bdobreli@redhat.com> Partial blueprint containerized-services-logs Change-Id: I53e737120bf0121bd28667f355b6f29f1b2a6b82
2017-05-02Move containers common volumes from yaql to list_concatJuan Antonio Osorio Robles1-131/+105
list_concat was introduced recently and is able to replace the yaql calls for concatenating lists. Change-Id: Id3a80a0e1e4c25b6d838898757c69ec99d0cd826
2017-04-18Introduce common resources for docker templatesJuan Antonio Osorio Robles1-91/+133
This enables common resources that the docker templates might need. The initial resource only is common volumes, and two volumes are introduced (localtime and hosts). Change-Id: Ic55af32803f9493a61f9b57aff849bfc6187d992
2017-04-03Remove kolla_config copy from servicesMartin André1-13/+0
Simplify the config of the containerized services by bind mounting in the configurations instead of specifying them all in kolla config. This is change is useful to limit the side effects of generating the config files and running the container is two separate steps as config directories are now bind-mounted inside the container instead of having files being copied to the container. We've seen examples of Apache's mod_ssl configuration file present on the container preventing it to start when puppet configured apache not to load the ssl module (in case TLS is disabled). Co-Authored-By: Ian Main <imain@redhat.com> Change-Id: I4ec5dd8b360faea71a044894a61790997f54d48a
2017-03-21Merge "Cleanup docker services templates"Jenkins1-48/+15
2017-03-16Keep existing data for containerized SwiftJiri Stransky1-15/+20
Use mounts instead of docker volumes to preserve existing data when moving from baremetal to containerized Swift. Change-Id: Ib7cbca2ef674a0245a67b69ee2c77f574d74c181
2017-03-10Cleanup docker services templatesMartin André1-48/+15
Use yaml anchors wherever possible for image definition and drop unused anchors. Renamed parameters to Docker*ConfigImage to clarify that an image is specifically used to generate configuration files. Change-Id: I388bd59de7f1d36a3a881fbb723ba5bcba09e637
2017-03-10Remove docker_image sections (unused)Dan Prince1-5/+4
We don't use docker_image for anything. It is a remant of the pre-composable docker templates and we can now remove it. This patch removes references to the 'docker_image' section from docker/post.yaml and all of the docker/services* templates. Change-Id: I208c1ef1550ab39ab0ee47ab282f9b1937379810
2017-03-06Enable composable upgrades for docker service templatesSteven Hardy1-0/+17
This aligns the docker based services with the new composable upgrades architecture we landed for ocata, and does a first-pass adding upgrade_tasks for the services (these may change, atm we only disable the service on the host). To run the upgrade workflow you basically do two steps: openstack overcloud deploy --templates \ -e environments/major-upgrade-composable-steps-docker.yaml This will run the ansible upgrade steps we define via upgrade_tasks then run the normal docker PostDeploySteps to bring up the containers. For the puppet workflow there's then an operator driven step where compute nodes (and potentially storage nodes) are upgrades in batches and finally you do: openstack overcloud deploy --templates \ -e environments/major-upgrade-converge-docker.yaml In the puppet case this re-applies puppet to unpin the nova RPC API so I guess it'll restart the nova containers this affects but otherwise will be a no-op (we also disable the ansible steps at this point. Depends-On: I9057d47eea15c8ba92ca34717b6b5965d4425ab1 Change-Id: Ia50169819cb959025866348b11337728f8ed5c9e
2017-03-01Put docker puppet config in puppet_config dictSteve Baker1-4/+7
This approach removes the need for the yaql zip to build the docker-puppet data by building the data in a puppet_config dict. This allows a future change to make docker-puppet.py only accept dict data. Currently the step_config is left where it is and referenced inside puppet_config, but feedback is welcome whether this is necessary or desirable. Change-Id: I4a4d7a6fd2735cb841174af305dbb62e0b3d3e8c
2017-02-22Containerized Swift Services.Ian Main1-0/+343
Running swift in containers. Oh yeah. Co-Authored-by: Dan Prince <dprince@redhat.com> Change-Id: I2d96514fb7aa51dffe8fe293bc950e0e99df5e94