diff options
| -rw-r--r-- | deployed-server/README.rst | 129 | ||||
| -rw-r--r-- | deployed-server/deployed-server-config.yaml | 22 | ||||
| -rw-r--r-- | deployed-server/deployed-server.yaml | 122 | ||||
| -rwxr-xr-x | deployed-server/scripts/get-occ-config.sh | 113 | ||||
| -rw-r--r-- | environments/deployed-server-environment.yaml | 3 | ||||
| -rw-r--r-- | net-config-static-bridge-with-external-dhcp.yaml | 99 | ||||
| -rw-r--r-- | overcloud-resource-registry-puppet.yaml | 3 | ||||
| -rw-r--r-- | puppet/ceph-storage.yaml | 2 | ||||
| -rw-r--r-- | puppet/cinder-storage.yaml | 2 | ||||
| -rw-r--r-- | puppet/compute.yaml | 2 | ||||
| -rw-r--r-- | puppet/controller.yaml | 2 |
11 files changed, 495 insertions, 4 deletions
diff --git a/deployed-server/README.rst b/deployed-server/README.rst new file mode 100644 index 00000000..ce74e77b --- /dev/null +++ b/deployed-server/README.rst @@ -0,0 +1,129 @@ +TripleO with Deployed Servers +============================= + +The deployed-server set of templates can be used to deploy TripleO via +tripleo-heat-templates to servers that are already installed with a base +operating system. + +When OS::TripleO::Server is mapped to the deployed-server.yaml template via the +provided deployed-server-environment.yaml resource registry, Nova and Ironic +are not used to create any server instances. Heat continues to create the +SoftwareDeployment resources, and they are made available to the already +deployed and running servers. + +Template Usage +-------------- +To use these templates pass the included environment file to the deployment +command:: + + -e deployed-server/deployed-server-environment.yaml + +Deployed Server configuration +----------------------------- +It is currently assumed that the deployed servers being used have the required +set of software and packages already installed on them. These exact +requirements must match how such a server would look if it were deployed the +standard way via Ironic using the TripleO overcloud-full image. + +An easy way to help get this setup for development is to use an overcloud-full +image from an already existing TripleO setup. Create the vm's for the already +deployed server, and use the overcloud-full image as their disk. + +Each server must have a fqdn set that resolves to an IP address on a routable +network (e.g., the hostname should not resolve to 127.0.0.1). The hostname +will be detected on each server via the hostnamectl --static command. + +Each server also must have a route to the configured IP address on the +undercloud where the OpenStack services are listening. This is the value for +local_ip in the undercloud.conf. + +It's recommended that each server have at least 2 nic's. One used for external +management such as ssh, and one used for the OpenStack deployment itself. Since +the overcloud deployment will reconfigure networking on the configured nic to +be used by OpenStack, the external management nic is needed as a fallback so +that all connectivity is not lost in case of a configuration error. Be sure to +use correct nic config templates as needed, since the nodes will not receive +dhcp from the undercloud neutron-dhcp-agent service. + +For example, the net-config-static-bridge.yaml template could be used for +controllers, and the net-config-static.yaml template could be used for computes +by specifying: + +resource_registry: + OS::TripleO::Controller::Net::SoftwareConfig: /home/stack/deployed-server/tripleo-heat-templates/net-config-static-bridge.yaml + OS::TripleO::Compute::Net::SoftwareConfig: /home/stack/deployed-server/tripleo-heat-templates/net-config-static.yaml + +In a setup where the first nic on the servers is used for external management, +set the nic's to be used for OpenStack to nic2: + +parameter_defaults: + NeutronPublicInterface: nic2 + HypervisorNeutronPublicInterface: nic2 + +The above nic config templates also require a route to the ctlplane network to +be defined. Define the needed parameters as necessary for your environment, for +example: + +parameter_defaults: + ControlPlaneDefaultRoute: 192.168.122.130 + ControlPlaneSubnetCidr: "24" + EC2MetadataIp: "192.0.2.1" + +In this example, 192.168.122.130 is the external management IP of an +undercloud, thus it is the default route for the configured local_ip value of +192.0.2.1. + + +os-collect-config +----------------- +os-collect-config on each deployed server must be manually configured to poll +the Heat API for the available SoftwareDeployments. An example configuration +for /etc/os-collect-config.conf looks like: + + [DEFAULT] + collectors=heat + command=os-refresh-config + + [heat] + # you can get these values from stackrc on the undercloud + user_id=<a user that can connect to heat> # note this must be the ID, not the username + password=<a password> + auth_url=<keystone url> + project_id=<project_id> # note, this must be the ID, not project name + stack_id=<stack_id> + resource_name=<resource_name> + +Note that the stack_id value is the id of the nested stack containing the +resource (identified by resource_name) implemented by the deployed-server.yaml +templates. + +Once the configuration for os-collect-config has been defined, the service +needs to be restarted. Once restarted, it will start polling Heat and applying +the SoftwareDeployments. + +A sample script at deployed-server/scripts/get-occ-config.sh is included that +will automatically generate the os-collect-config configuration needed on each +server, ssh to each server, copy the configuration, and restart the +os-collect-config service. + +.. warning:: + The get-occ-config.sh script is not intended for production use, as it + copies admin credentials to each of the deployed nodes. + +The script can only be used once the stack id's of the nested deployed-server +stacks have been created via Heat. This usually only takes a couple of minutes +once the deployment command has been started. Once the following output is seen +from the deployment command, the script should be ready to run: + + [Controller]: CREATE_IN_PROGRESS state changed + [NovaCompute]: CREATE_IN_PROGRESS state changed + +The user running the script must be able to ssh as root to each server. Define +the hostnames of the deployed servers you intend to use for each role type:: + + export controller_hosts="controller0 controller1 controller2" + export compute_hosts="compute0" + +Then run the script on the undercloud with a stackrc file sourced, and +the script will copy the needed os-collect-config.conf configuration to each +server and restart the os-collect-config service. diff --git a/deployed-server/deployed-server-config.yaml b/deployed-server/deployed-server-config.yaml new file mode 100644 index 00000000..8c59dc72 --- /dev/null +++ b/deployed-server/deployed-server-config.yaml @@ -0,0 +1,22 @@ +heat_template_version: 2014-10-16 +parameters: + user_data_format: + type: string + default: SOFTWARE_CONFIG + +resources: + # We just need something which returns a unique ID, but we can't + # use RandomString because RefId returns the value, not the physical + # resource ID, SoftwareConfig should work as it returns a UUID + deployed-server-config: + type: OS::Heat::SoftwareConfig + +outputs: + # FIXME(shardy) this is needed because TemplateResource returns an + # ARN not a UUID, which overflows the Deployment server_id column.. + user_data_format: + value: SOFTWARE_CONFIG + OS::stack_id: + value: {get_resource: deployed-server-config} + + diff --git a/deployed-server/deployed-server.yaml b/deployed-server/deployed-server.yaml new file mode 100644 index 00000000..81941047 --- /dev/null +++ b/deployed-server/deployed-server.yaml @@ -0,0 +1,122 @@ +heat_template_version: 2014-10-16 +parameters: + image: + type: string + default: unused + flavor: + type: string + default: unused + key_name: + type: string + default: unused + security_groups: + type: json + default: [] + # Require this so we can validate the parent passes the + # correct value + user_data_format: + type: string + user_data: + type: string + default: '' + name: + type: string + default: '' + image_update_policy: + type: string + default: '' + networks: + type: comma_delimited_list + default: '' + metadata: + type: json + default: {} + software_config_transport: + default: POLL_SERVER_CFN + type: string + scheduler_hints: + type: json + description: Optional scheduler hints to pass to nova + default: {} + +resources: + # We just need something which returns a unique ID, but we can't + # use RandomString because RefId returns the value, not the physical + # resource ID, SoftwareConfig should work as it returns a UUID + deployed-server: + type: OS::TripleO::DeployedServerConfig + properties: + user_data_format: SOFTWARE_CONFIG + + InstanceIdConfig: + type: OS::Heat::StructuredConfig + properties: + group: os-apply-config + config: + instance-id: {get_attr: [deployed-server, "OS::stack_id"]} + + InstanceIdDeployment: + type: OS::Heat::StructuredDeployment + properties: + config: {get_resource: InstanceIdConfig} + server: {get_resource: deployed-server} + + HostsEntryConfig: + type: OS::Heat::SoftwareConfig + properties: + group: script + config: | + #!/bin/bash + set -eux + mkdir -p $heat_outputs_path + host=$(hostnamectl --static) + echo -n "$host " > $heat_outputs_path.hosts_entry + host_ip=$(python -c "import socket; print socket.gethostbyname(\"$host\")") + echo -n "$host_ip " >> $heat_outputs_path.hosts_entry + echo >> $heat_outputs_path.hosts_entry + cat $heat_outputs_path.hosts_entry + echo -n $host_ip > $heat_outputs_path.ip_address + cat $heat_outputs_path.ip_address + echo -n $host > $heat_outputs_path.hostname + cat $heat_outputs_path.hostname + outputs: + - name: hosts_entry + description: hosts_entry + - name: ip_address + description: ip_address + - name: hostname + description: hostname + + HostsEntryDeployment: + type: OS::Heat::SoftwareDeployment + properties: + config: {get_resource: HostsEntryConfig} + server: {get_resource: deployed-server} + + ControlPlanePort: + type: OS::Neutron::Port + properties: + network: ctlplane + name: + list_join: + - '-' + - - {get_attr: [HostsEntryDeployment, hostname]} + - ctlplane + - port + replacement_policy: AUTO + +outputs: + # FIXME(shardy) this is needed because TemplateResource returns an + # ARN not a UUID, which overflows the Deployment server_id column.. + OS::stack_id: + value: {get_attr: [deployed-server, "OS::stack_id"]} + networks: + value: + ctlplane: + - {get_attr: [ControlPlanePort, fixed_ips, 0, ip_address]} + name: + value: {get_attr: [HostsEntryDeployment, hostname]} + hosts_entry: + value: {get_attr: [HostsEntryDeployment, hosts_entry]} + ip_address: + value: {get_attr: [HostsEntryDeployment, ip_address]} diff --git a/deployed-server/scripts/get-occ-config.sh b/deployed-server/scripts/get-occ-config.sh new file mode 100755 index 00000000..b4cedea2 --- /dev/null +++ b/deployed-server/scripts/get-occ-config.sh @@ -0,0 +1,113 @@ +#!/bin/bash + +set -eux + +SLEEP_TIME=5 + +CONTROLLER_HOSTS=${CONTROLLER_HOSTS:-""} +COMPUTE_HOSTS=${COMPUTE_HOSTS:-""} +BLOCKSTORAGE_HOSTS=${BLOCKSTORAGE_HOSTS:-""} +OBJECTSTORAGE_HOSTS=${OBJECTSTORAGE_HOSTS:-""} +CEPHSTORAGE_HOSTS=${CEPHSTORAGE_HOSTS:-""} +SUBNODES_SSH_KEY=${SUBNODES_SSH_KEY:-"~/.ssh/id_rsa"} +SSH_OPTIONS="-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=Verbose -o PasswordAuthentication=no -o ConnectionAttempts=32" + +read -a Controller_hosts_a <<< $CONTROLLER_HOSTS +read -a Compute_hosts_a <<< $COMPUTE_HOSTS +read -a BlockStorage_hosts_a <<< $BLOCKSTORAGE_HOSTS +read -a ObjectStorage_hosts_a <<< $OBJECTSTORAGE_HOSTS +read -a CephStorage_hosts_a <<< $CEPHSTORAGE_HOSTS + +roles="Controller Compute BlockStorage ObjectStorage CephStorage" +admin_user_id=$(openstack user show admin -c id -f value) +admin_project_id=$(openstack project show admin -c id -f value) + +function check_stack { + local stack_to_check=$1 + + if [ "$stack_to_check" = "|" ]; then + echo Stack not created + return 1 + fi + + echo Checking if $1 stack is created + set +e + heat resource-list $stack_to_check + rc=$? + set -e + + if [ ! "$rc" = "0" ]; then + echo Stack $1 not yet created + fi + + return $rc +} + + +for role in $roles; do + while ! check_stack overcloud; do + sleep $SLEEP_TIME + done + + rg_stack=$(heat resource-list overcloud | grep " $role " | awk '{print $4}') + while ! check_stack $rg_stack; do + sleep $SLEEP_TIME + rg_stack=$(heat resource-list overcloud | grep " $role " | awk '{print $4}') + done + + stacks=$(heat resource-list $rg_stack | grep OS::TripleO::$role | awk '{print $4}') + + i=0 + + for stack in $stacks; do + server_resource_name=$role + if [ "$server_resource_name" = "Compute" ]; then + server_resource_name="NovaCompute" + fi + + server_stack=$(heat resource-list $stack | grep " $server_resource_name " | awk '{print $4}') + while ! check_stack $server_stack; do + sleep $SLEEP_TIME + server_stack=$(heat resource-list $stack | grep " $server_resource_name " | awk '{print $4}') + done + + deployed_server_stack=$(heat resource-list $server_stack | grep "deployed-server" | awk '{print $4}') + + echo "======================" + echo "$role$i os-collect-config.conf configuration:" + + config=" +[DEFAULT] +collectors=heat +command=os-refresh-config +polling_interval=30 + +[heat] +user_id=$admin_user_id +password=$OS_PASSWORD +auth_url=$OS_AUTH_URL +project_id=$admin_project_id +stack_id=$deployed_server_stack +resource_name=deployed-server-config" + + echo "$config" + echo "======================" + echo + + + host= + eval host=\${${role}_hosts_a[i]} + if [ -n "$host" ]; then + # Delete the os-collect-config.conf template so our file won't get + # overwritten + ssh $SSH_OPTIONS -i $SUBNODES_SSH_KEY $host sudo /bin/rm -f /usr/libexec/os-apply-config/templates/etc/os-collect-config.conf + ssh $SSH_OPTIONS -i $SUBNODES_SSH_KEY $host "echo \"$config\" > os-collect-config.conf" + ssh $SSH_OPTIONS -i $SUBNODES_SSH_KEY $host sudo cp os-collect-config.conf /etc/os-collect-config.conf + ssh $SSH_OPTIONS -i $SUBNODES_SSH_KEY $host sudo systemctl restart os-collect-config + fi + + let i+=1 + + done + +done diff --git a/environments/deployed-server-environment.yaml b/environments/deployed-server-environment.yaml new file mode 100644 index 00000000..3c9e3459 --- /dev/null +++ b/environments/deployed-server-environment.yaml @@ -0,0 +1,3 @@ +resource_registry: + OS::TripleO::Server: ../deployed-server/deployed-server.yaml + OS::TripleO::DeployedServerConfig: ../deployed-server/deployed-server-config.yaml diff --git a/net-config-static-bridge-with-external-dhcp.yaml b/net-config-static-bridge-with-external-dhcp.yaml new file mode 100644 index 00000000..6dbe5982 --- /dev/null +++ b/net-config-static-bridge-with-external-dhcp.yaml @@ -0,0 +1,99 @@ +heat_template_version: 2015-04-30 + +description: > + Software Config to drive os-net-config for a simple bridge configured + with a static IP address for the ctlplane network. + +parameters: + ControlPlaneIp: + default: '' + description: IP address/subnet on the ctlplane network + type: string + ExternalIpSubnet: + default: '' + description: IP address/subnet on the external network + type: string + InternalApiIpSubnet: + default: '' + description: IP address/subnet on the internal API network + type: string + StorageIpSubnet: + default: '' + description: IP address/subnet on the storage network + type: string + StorageMgmtIpSubnet: + default: '' + description: IP address/subnet on the storage mgmt network + type: string + TenantIpSubnet: + default: '' + description: IP address/subnet on the tenant network + type: string + ManagementIpSubnet: + default: '' + description: IP address/subnet on the management network + type: string + ControlPlaneSubnetCidr: # Override this via parameter_defaults + default: '24' + description: The subnet CIDR of the control plane network. + type: string + ControlPlaneDefaultRoute: # Override this via parameter_defaults + description: The default route of the control plane network. + type: string + DnsServers: # Override this via parameter_defaults + default: [] + description: A list of DNS servers (2 max for some implementations) that will be added to resolv.conf. + type: comma_delimited_list + EC2MetadataIp: # Override this via parameter_defaults + description: The IP address of the EC2 metadata server. + type: string + +resources: + OsNetConfigImpl: + type: OS::Heat::StructuredConfig + properties: + group: os-apply-config + config: + os_net_config: + network_config: + - + type: ovs_bridge + name: {get_input: bridge_name} + use_dhcp: true + members: + - + type: interface + name: {get_input: interface_name} + # force the MAC address of the bridge to this interface + primary: true + - + type: interface + # would like to do the following, but can't b/c of: + # https://bugs.launchpad.net/heat/+bug/1344284 + # name: + # list_join: + # - '/' + # - - {get_input: bridge_name} + # - ':0' + # So, just hardcode to br-ex:0 for now, br-ex is hardcoded in + # controller.yaml anyway. + name: br-ex:0 + addresses: + - + ip_netmask: + list_join: + - '/' + - - {get_param: ControlPlaneIp} + - {get_param: ControlPlaneSubnetCidr} + routes: + - + ip_netmask: 169.254.169.254/32 + next_hop: {get_param: EC2MetadataIp} + - + default: true + next_hop: {get_param: ControlPlaneDefaultRoute} + +outputs: + OS::stack_id: + description: The OsNetConfigImpl resource. + value: {get_resource: OsNetConfigImpl} diff --git a/overcloud-resource-registry-puppet.yaml b/overcloud-resource-registry-puppet.yaml index 591b0300..4e98e84b 100644 --- a/overcloud-resource-registry-puppet.yaml +++ b/overcloud-resource-registry-puppet.yaml @@ -28,6 +28,9 @@ resource_registry: OS::TripleO::Tasks::ControllerPrePuppet: OS::Heat::None OS::TripleO::Tasks::ControllerPostPuppet: OS::Heat::None + + OS::TripleO::Server: OS::Nova::Server + # This creates the "heat-admin" user for all OS images by default # To disable, replace with firstboot/userdata_default.yaml OS::TripleO::NodeAdminUserData: firstboot/userdata_heat_admin.yaml diff --git a/puppet/ceph-storage.yaml b/puppet/ceph-storage.yaml index 8a43b673..baf5bf48 100644 --- a/puppet/ceph-storage.yaml +++ b/puppet/ceph-storage.yaml @@ -98,7 +98,7 @@ parameters: resources: CephStorage: - type: OS::Nova::Server + type: OS::TripleO::Server metadata: os-collect-config: command: {get_param: ConfigCommand} diff --git a/puppet/cinder-storage.yaml b/puppet/cinder-storage.yaml index d0f562ed..9bbef86a 100644 --- a/puppet/cinder-storage.yaml +++ b/puppet/cinder-storage.yaml @@ -98,7 +98,7 @@ parameters: resources: BlockStorage: - type: OS::Nova::Server + type: OS::TripleO::Server metadata: os-collect-config: command: {get_param: ConfigCommand} diff --git a/puppet/compute.yaml b/puppet/compute.yaml index 7ff0676b..75d20bfe 100644 --- a/puppet/compute.yaml +++ b/puppet/compute.yaml @@ -324,7 +324,7 @@ parameters: resources: NovaCompute: - type: OS::Nova::Server + type: OS::TripleO::Server metadata: os-collect-config: command: {get_param: ConfigCommand} diff --git a/puppet/controller.yaml b/puppet/controller.yaml index 0d6e4028..862f7c72 100644 --- a/puppet/controller.yaml +++ b/puppet/controller.yaml @@ -404,7 +404,7 @@ parameter_groups: resources: Controller: - type: OS::Nova::Server + type: OS::TripleO::Server metadata: os-collect-config: command: {get_param: ConfigCommand} |