Add NOVCP HA OVS scenario (baremetal, virtual)

Add a new class of scenarios, based on existing baremetal HA
scenarios, but instead of having a virtualized control plane (VCP),
all Openstack controller services will run directly on the cluster
nodes.
This change adds the common scaffolding, as well as the OVS scenario.

The new scenario(s) can be used on full-baremetal clusters, soon on
full-virtual clusters and later on hybrid (virt + bare) clusters.
This change defines old (current) style scenario definitions for
both baremetal and virtual, both named:
- os-nosdn-nofeature-novcp-ha;

Prerequisites:
1. Merge-able by name reclass.storage.node definitions
   Each cluster (e.g. database, telemetry) adds its own set of
   reclass storage node defitions, which for novcp scenarios should
   be merged into a single node (kvm) based on the 'name' property.
   This is not currently supported by upstream reclass 'node.sls'
   high state, so add support for it via an early patch (required
   before salt-master-init.sh tries to handle reclass.storage).
2. common reclass classes for novcp
   Some of the classes in `baremetal-...-common-ha` are not fit for
   novcp as they define VCP-specific config/inheritance, so add new
   versions of said classes with novcp in mind or adapt old classes:
   - parameterize ctl hostname in `openstack_compute.yml`;
   - new `openstack_control_novcp.yml`;
   - new `openstack_init_novcp.yml`;
3. Handle hard set names in state files for baremetal nodes
   Some of our state files (e.g. maas) hardcodes baremetal node names
   to 'kvm', 'cmp', so we need to align the names in novcp scenario
   with these values to re-use the maas state. As a future improvement
   we should parameterize these names in all state files.
   As a consequence, our baremetal controller nodes will also use
   'kvm*' hostnames (instead of 'ctl*').
4. Add 'noifupdown' to all interfaces on kvm nodes to prevent duplicate
   IPs/routes created at *any* ifup due to /etc/network/route-br-ex.
   Patch salt-formula-linux to skip network restart on 'noifupdown',
   also when routes are present on that interface.

JIRA: FUEL-310

Change-Id: Ic67778f63e5ee0334dbfe9547c7109ec1a938d61
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>

13 files changed, 227 insertions, 0 deletions

diff --git a/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/infra/config.yml b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/infra/config.yml
new file mode 100644
index 000000000..5681e5a6b
--- /dev/null
+++ b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/infra/config.yml
@@ -0,0 +1,24 @@
+##############################################################################
+# Copyright (c) 2018 Mirantis Inc., Enea AB and others.
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+---
+classes:
+  - cluster.mcp-pike-common-ha.infra.config
+  - cluster.mcp-pike-ovs-novcp-ha.infra
+parameters:
+  reclass:
+    storage:
+      node:
+        openstack_control_node01:  # openstack_proxy_node01
+          params:
+            external_address: ${_param:openstack_proxy_node01_address}
+        openstack_control_node02:  # no proxy role
+          params:
+            external_address: 0.0.0.0
+        openstack_control_node03:  # openstack_proxy_node02
+          params:
+            external_address: ${_param:openstack_proxy_node02_address}
diff --git a/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/infra/init.yml b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/infra/init.yml
new file mode 100644
index 000000000..23c299cbc
--- /dev/null
+++ b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/infra/init.yml
@@ -0,0 +1,18 @@
+##############################################################################
+# Copyright (c) 2018 Mirantis Inc., Enea AB and others.
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+---
+classes:
+  - cluster.mcp-pike-common-ha.infra
+  - cluster.mcp-pike-ovs-novcp-ha.openstack
+parameters:
+  _param:
+    cluster_name: mcp-pike-ovs-novcp-ha
+    # For NOVCP, we override kvm addresses to overlap with ctl
+    infra_kvm_node01_address: ${_param:openstack_control_node01_address}
+    infra_kvm_node02_address: ${_param:openstack_control_node02_address}
+    infra_kvm_node03_address: ${_param:openstack_control_node03_address}
diff --git a/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/infra/kvm.yml b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/infra/kvm.yml
new file mode 100644
index 000000000..887af029d
--- /dev/null
+++ b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/infra/kvm.yml
@@ -0,0 +1,12 @@
+##############################################################################
+# Copyright (c) 2018 Mirantis Inc., Enea AB and others.
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+---
+# NOTE(armband): we don't want to pull in salt.control for novcp
+# classes:
+#   - cluster.mcp-pike-common-ha.infra.kvm
+#   - cluster.mcp-pike-ovs-novcp-ha.infra
diff --git a/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/infra/maas.yml b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/infra/maas.yml
new file mode 100644
index 000000000..e694679b9
--- /dev/null
+++ b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/infra/maas.yml
@@ -0,0 +1,11 @@
+##############################################################################
+# Copyright (c) 2018 Mirantis Inc., Enea AB and others.
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+---
+classes:
+  - cluster.mcp-pike-common-ha.infra.maas
+  - cluster.mcp-pike-ovs-novcp-ha.infra
diff --git a/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/init.yml b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/init.yml
new file mode 100644
index 000000000..234418d9c
--- /dev/null
+++ b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/init.yml
@@ -0,0 +1,12 @@
+##############################################################################
+# Copyright (c) 2018 Mirantis Inc., Enea AB and others.
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+---
+classes:
+  - cluster.all-mcp-arch-common
+  - cluster.mcp-pike-ovs-novcp-ha.infra
+  - cluster.mcp-pike-ovs-novcp-ha.openstack
diff --git a/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/compute.yml b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/compute.yml
new file mode 100644
index 000000000..afe1cc5a6
--- /dev/null
+++ b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/compute.yml
@@ -0,0 +1,17 @@
+##############################################################################
+# Copyright (c) 2018 Mirantis Inc., Enea AB and others.
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+---
+classes:
+  - cluster.mcp-pike-common-ha.openstack_compute
+  - cluster.mcp-pike-ovs-novcp-ha.openstack.compute_pdf
+  - cluster.mcp-pike-ovs-novcp-ha.infra
+parameters:
+  nova:
+    compute:
+      libvirt_service: libvirtd
+      libvirt_bin: /etc/default/libvirtd
diff --git a/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/compute_pdf.yml.j2 b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/compute_pdf.yml.j2
new file mode 120000
index 000000000..d43451a2a
--- /dev/null
+++ b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/compute_pdf.yml.j2
@@ -0,0 +1 @@
+../../mcp-pike-ovs-ha/openstack/compute_pdf.yml.j2
\ No newline at end of file
diff --git a/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/control.yml b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/control.yml
new file mode 100644
index 000000000..4cd5b9629
--- /dev/null
+++ b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/control.yml
@@ -0,0 +1,19 @@
+##############################################################################
+# Copyright (c) 2018 Mirantis Inc., Enea AB and others.
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+---
+classes:
+  - system.neutron.control.openvswitch.cluster
+  - cluster.mcp-pike-common-ha.openstack_control_novcp
+  - cluster.mcp-pike-ovs-novcp-ha
+parameters:
+  apache:
+    server:
+      modules:
+        # NOTE(armband): We first override mods to ~, then to this to drop ssl
+        - rewrite
+        - wsgi
diff --git a/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/database.yml b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/database.yml
new file mode 100644
index 000000000..163769524
--- /dev/null
+++ b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/database.yml
@@ -0,0 +1,10 @@
+##############################################################################
+# Copyright (c) 2018 Mirantis Inc., Enea AB and others.
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+---
+classes:
+  - cluster.mcp-pike-common-ha.openstack_database
diff --git a/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/init.yml b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/init.yml
new file mode 100644
index 000000000..7540c0074
--- /dev/null
+++ b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/init.yml
@@ -0,0 +1,24 @@
+##############################################################################
+# Copyright (c) 2018 Mirantis Inc., Enea AB and others.
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+---
+classes:
+  - cluster.mcp-pike-common-ha.openstack_init_novcp
+parameters:
+  _param:
+    neutron_tenant_network_types: "flat,vxlan"
+    nova_cpu_pinning: "1,2,3,4,5,7,8,9,10,11"
+    compute_hugepages_size: 1G
+    compute_hugepages_count: 16
+    compute_hugepages_mount: /mnt/hugepages_1G
+    compute_kernel_isolcpu: ${_param:nova_cpu_pinning}
+  apache:
+    server:
+      # NOTE(armband): override these to ~ first, so we can later drop ssl/443
+      bind:
+        ports: ~
+      modules: ~
diff --git a/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/message_queue.yml b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/message_queue.yml
new file mode 100644
index 000000000..0fd4a8a54
--- /dev/null
+++ b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/message_queue.yml
@@ -0,0 +1,10 @@
+##############################################################################
+# Copyright (c) 2018 Mirantis Inc., Enea AB and others.
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+---
+classes:
+  - cluster.mcp-pike-common-ha.openstack_message_queue
diff --git a/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/proxy.yml b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/proxy.yml
new file mode 100644
index 000000000..89b095430
--- /dev/null
+++ b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/proxy.yml
@@ -0,0 +1,59 @@
+##############################################################################
+# Copyright (c) 2018 Mirantis Inc., Enea AB and others.
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+---
+classes:
+  - cluster.mcp-pike-common-ha.openstack_proxy
+  - system.keepalived.cluster.instance.openstack_web_public_vip
+parameters:
+  linux:
+    # Set up routes similar to prx*ovs-ha
+    network:
+      interface:
+        br-ex:
+          route:
+            public:
+              address: 0.0.0.0
+              netmask: 0.0.0.0
+              gateway: ${_param:opnfv_net_public_gw}
+  nginx:
+    server:
+      # NOTE(armband): Define host.address for all proxies for uniformity
+      site:
+        nginx_proxy_novnc: &nginx_openstack_proxy_address
+          host:
+            address: ${_param:openstack_proxy_address}
+        nginx_proxy_openstack_api_aodh:
+          <<: *nginx_openstack_proxy_address
+        nginx_proxy_openstack_api_ceilometer:
+          <<: *nginx_openstack_proxy_address
+        nginx_proxy_openstack_api_cinder:
+          <<: *nginx_openstack_proxy_address
+        nginx_proxy_openstack_api_glance:
+          <<: *nginx_openstack_proxy_address
+        nginx_proxy_openstack_api_heat:
+          <<: *nginx_openstack_proxy_address
+        nginx_proxy_openstack_api_heat_cfn:
+          <<: *nginx_openstack_proxy_address
+        nginx_proxy_openstack_api_heat_cloudwatch:
+          <<: *nginx_openstack_proxy_address
+        nginx_proxy_openstack_api_keystone:
+          <<: *nginx_openstack_proxy_address
+        nginx_proxy_openstack_api_keystone_private:
+          <<: *nginx_openstack_proxy_address
+        nginx_proxy_openstack_api_neutron:
+          <<: *nginx_openstack_proxy_address
+        nginx_proxy_openstack_api_nova:
+          <<: *nginx_openstack_proxy_address
+        nginx_proxy_openstack_api_nova_ec2:
+          <<: *nginx_openstack_proxy_address
+        nginx_proxy_openstack_web:
+          <<: *nginx_openstack_proxy_address
+        nginx_ssl_redirect_openstack_web:
+          <<: *nginx_openstack_proxy_address
+        nginx_static_reclass_doc:
+          <<: *nginx_openstack_proxy_address
diff --git a/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/telemetry.yml b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/telemetry.yml
new file mode 100644
index 000000000..1cb72ac5c
--- /dev/null
+++ b/mcp/reclass/classes/cluster/mcp-pike-ovs-novcp-ha/openstack/telemetry.yml
@@ -0,0 +1,10 @@
+##############################################################################
+# Copyright (c) 2018 Mirantis Inc., Enea AB and others.
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+---
+classes:
+  - cluster.mcp-pike-common-ha.openstack_telemetry