[docker] Cleanup, minor fixes, formula bump

* ship prebuilt salt master conf for better readability:
  - enable x509.sign_remote_certificate (for prx VCP nodes);
* refactor Salt master CA handling:
  - preinstall `salt_minion_dependency_packages` and
   `salt_minion_reclass_dependencies` inside docker image;
  - persistent /etc/pki;
  - run salt.minion on cfg01 to generate master keys;
* bump container formulas to 1 Sep 2018 versions or newer:
  - inject date into Docker makefile, forcing a fresh fetch of all
    salt formulas from upstream git repos;
* workaround broken salt-formula-designate's meta/sphinx.yml:
  - the DEB package version of salt-formula-designate uses `cmd.shell`
    to query dpkg on the minion, while the git repo version still
    uses `cmd.run`, running into parsing issues;
  - temporarily disable sphinx metadata generation for designate until
    upstream git repo syncs with the DEB version;
* upstream: salt-formula-salt AArch64 salt.control.virt support:
  - retire salt-formula-salt git submodule and related patches;
* skip installing reclass distro package (already installed via pip
  inside the container);
* limit initial pillar_refresh call to nodes on jumphost;
* remove unused salt-formula-nova git submodule;

JIRA: FUEL-383

Change-Id: I883b825e556f887a5e31f8a43676dcd8ece6dfde
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>

3 files changed, 22 insertions, 164 deletions

diff --git a/mcp/patches/docker/0002-OPNFV-package-installation-Ubuntu-user.patch b/mcp/patches/docker/0002-OPNFV-package-installation-Ubuntu-user.patch
index 51931b507..c983ad728 100644
--- a/mcp/patches/docker/0002-OPNFV-package-installation-Ubuntu-user.patch
+++ b/mcp/patches/docker/0002-OPNFV-package-installation-Ubuntu-user.patch
@@ -16,17 +16,26 @@ Subject: [PATCH] OPNFV package installation, Ubuntu user
   workaround);
 * While at it, create 'ubuntu' user so other OPNFV projects don't
   have to switch to 'root' login;
+* Preinstall `salt_minion_dependency_packages` and
+  `salt_minion_reclass_dependencies`;
 
 Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
 ---
- DockerMake.yml | 28 ++++++++++++++++++++++++++++
- 1 file changed, 28 insertions(+)
+ DockerMake.yml | 36 +++++++++++++++++++++++++++++++++++-
+ 1 file changed, 35 insertions(+), 1 deletion(-)
 
 diff --git a/DockerMake.yml b/DockerMake.yml
-index 2c75586..4883e2c 100644
+index 2c75586..8fb460d 100644
 --- a/DockerMake.yml
 +++ b/DockerMake.yml
-@@ -108,6 +108,34 @@ salt-formulas:
+@@ -102,12 +102,46 @@ salt-formulas:
+     ENV SALT_ENV_PATH_ $SALT_ENV_PATH_
+     ARG RECLASS_BASE="/srv/salt/reclass"
+     ENV RECLASS_BASE $RECLASS_BASE
+-    RUN echo "Layer python/salt module prerequisites, formulas" \
++    RUN echo "Layer python/salt module prerequisites, formulas (1 Sep 2018)" \
+       && mkdir -p /srv/salt \
+       && curl -sSqL https://raw.githubusercontent.com/salt-formulas/salt-formulas-scripts/master/formula-fetch.sh -o /srv/salt/formula-fetch.sh \
        && bash -c 'source /srv/salt/formula-fetch.sh && setupPyEnv && fetchAll' \
        && eval ${LAYER_CLEANUP}
 
@@ -53,6 +62,12 @@ index 2c75586..4883e2c 100644
 +           kmod \
 +           net-tools \
 +           openssh-server \
++           python-m2crypto \
++           python-msgpack \
++           python-netaddr \
++           python-oauth \
++           python-psutil \
++           python-yaml \
 +      && useradd -m ubuntu \
 +      && echo 'ubuntu ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/ubuntu \
 +      && eval ${LAYER_CLEANUP}
diff --git a/mcp/patches/salt-formula-linux/0002-Set-ovs-bridges-as-L3-interfaces.patch b/mcp/patches/salt-formula-linux/0002-Set-ovs-bridges-as-L3-interfaces.patch
index 9de6325ed..390a0bfcd 100644
--- a/mcp/patches/salt-formula-linux/0002-Set-ovs-bridges-as-L3-interfaces.patch
+++ b/mcp/patches/salt-formula-linux/0002-Set-ovs-bridges-as-L3-interfaces.patch
@@ -12,15 +12,15 @@ Subject: [PATCH] Set ovs bridges as L3 interfaces
 
 Change-Id: I1e83129cc184cf481bea21d7aa452bf60d9e0499
 ---
- linux/files/ovs_bridge      | 19 +++++++++++++++++++
+ linux/files/ovs_bridge      | 18 ++++++++++++++++++
  linux/files/ovs_port        |  7 ++++++-
  linux/network/interface.sls | 28 ++++++++++++++++++++++++++++
- 3 files changed, 53 insertions(+), 1 deletion(-)
+ 3 files changed, 52 insertions(+), 1 deletion(-)
  create mode 100644 linux/files/ovs_bridge
 
 diff --git a/linux/files/ovs_bridge b/linux/files/ovs_bridge
 new file mode 100644
-index 0000000..073c91c
+index 0000000..d33864c
 --- /dev/null
 +++ b/linux/files/ovs_bridge
 @@ -0,0 +1,18 @@
diff --git a/mcp/patches/salt-formula-salt/0001-libvirt-xml-pass-loader-virt-machine-cpu-mode.patch b/mcp/patches/salt-formula-salt/0001-libvirt-xml-pass-loader-virt-machine-cpu-mode.patch
deleted file mode 100644
index effab8743..000000000
--- a/mcp/patches/salt-formula-salt/0001-libvirt-xml-pass-loader-virt-machine-cpu-mode.patch
+++ /dev/null
@@ -1,157 +0,0 @@
-::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
-: Copyright (c) 2018 Mirantis Inc., Enea AB and others.
-:
-: All rights reserved. This program and the accompanying materials
-: are made available under the terms of the Apache License, Version 2.0
-: which accompanies this distribution, and is available at
-: http://www.apache.org/licenses/LICENSE-2.0
-::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
-From: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
-Date: Sun, 24 Jun 2018 20:36:44 +0200
-Subject: [PATCH] libvirt xml: pass loader, virt machine, cpu mode
-
-- libvirt xml: pass loader param to vm
-  Based on upstream commit [1].
-- libvirt xml: pass virt machine type
-- libvirt xml: pass cpu mode to vm
-- virt module: Allow NVRAM unlinking on DOM undefine
-  UEFI-enabled VMs usually have pflash (NVRAM) devices attached,
-  which require one additional libvirt flag to be passed at 'undefine'.
-  This is usually the case for AArch64 (arm64) VMs, where AAVMF (AA64
-  UEFI) is the only supported guest bootloader.
-
-[1] https://github.com/saltstack/salt/commit/9cace9adb
-
-Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
----
- README.rst                           |  7 +++++
- _modules/virtng.py                   | 40 +++++++++++++++++++++++++++-
- salt/control/virt.sls                |  9 +++++++
- tests/pillar/control_virt_custom.sls |  6 +++++
- 4 files changed, 61 insertions(+), 1 deletion(-)
-
-diff --git a/README.rst b/README.rst
-index fd15b19..7f8f4a4 100644
---- a/README.rst
-+++ b/README.rst
-@@ -453,6 +453,13 @@ Control VM provisioning:
-                   rate:
-                     period: '1800'
-                     bytes: '1500'
-+                # Custom per-node loader definition (e.g. for AArch64 UEFI)
-+                loader:
-+                  readonly: yes
-+                  type: pflash
-+                  path: /usr/share/AAVMF/AAVMF_CODE.fd
-+                machine: virt-2.11  # Custom per-node virt machine type
-+                cpu_mode: host-passthrough
-                 mac:
-                   nic01: AC:DE:48:AA:AA:AA
-                   nic02: AC:DE:48:AA:AA:BB
-diff --git a/_modules/virtng.py b/_modules/virtng.py
-index ce09508..6abd0eb 100644
---- a/_modules/virtng.py
-+++ b/_modules/virtng.py
-@@ -530,6 +530,9 @@ def init(name,
-          disk='default',
-          saltenv='base',
-          rng=None,
-+         loader=None,
-+         machine=None,
-+         cpu_mode=None,
-          **kwargs):
-     '''
-     Initialize a new vm
-@@ -649,6 +652,37 @@ def init(name,
-
-     xml = _gen_xml(name, cpu, mem, diskp, nicp, hypervisor, **kwargs)
-
-+    # TODO: Remove this code and refactor module, when salt-common would have updated libvirt_domain.jinja template
-+    if cpu_mode:
-+        xml_doc = minidom.parseString(xml)
-+        cpu_xml = xml_doc.createElement("cpu")
-+        cpu_xml.setAttribute('mode', cpu_mode)
-+        xml_doc.getElementsByTagName("domain")[0].appendChild(cpu_xml)
-+        xml = xml_doc.toxml()
-+
-+    # TODO: Remove this code and refactor module, when salt-common would have updated libvirt_domain.jinja template
-+    if machine:
-+        xml_doc = minidom.parseString(xml)
-+        os_xml = xml_doc.getElementsByTagName("domain")[0].getElementsByTagName("os")[0]
-+        os_xml.getElementsByTagName("type")[0].setAttribute('machine', machine)
-+        xml = xml_doc.toxml()
-+
-+    # TODO: Remove this code and refactor module, when salt-common would have updated libvirt_domain.jinja template
-+    if loader and 'path' not in loader:
-+        log.info('`path` is a required property of `loader`, and cannot be found. Skipping loader configuration')
-+        loader = None
-+    elif loader:
-+        xml_doc = minidom.parseString(xml)
-+        loader_xml = xml_doc.createElement("loader")
-+        for key, val in loader.items():
-+            if key == 'path':
-+                continue
-+            loader_xml.setAttribute(key, val)
-+        loader_path_xml = xml_doc.createTextNode(loader['path'])
-+        loader_xml.appendChild(loader_path_xml)
-+        xml_doc.getElementsByTagName("domain")[0].getElementsByTagName("os")[0].appendChild(loader_xml)
-+        xml = xml_doc.toxml()
-+
-     # TODO: Remove this code and refactor module, when salt-common would have updated libvirt_domain.jinja template
-     for _nic in nicp:
-         if _nic['virtualport']:
-@@ -1552,7 +1586,11 @@ def undefine(vm_):
-         salt '*' virtng.undefine <vm name>
-     '''
-     dom = _get_dom(vm_)
--    return dom.undefine() == 0
-+    if getattr(libvirt, 'VIR_DOMAIN_UNDEFINE_NVRAM', False):
-+        # This one is only in 1.2.8+
-+        return dom.undefineFlags(libvirt.VIR_DOMAIN_UNDEFINE_NVRAM) == 0
-+    else:
-+        return dom.undefine() == 0
-
-
- def purge(vm_, dirs=False):
-diff --git a/salt/control/virt.sls b/salt/control/virt.sls
-index a2e56ff..1bcca95 100644
---- a/salt/control/virt.sls
-+++ b/salt/control/virt.sls
-@@ -58,6 +58,15 @@ salt_control_virt_{{ cluster_name }}_{{ node_name }}:
-   {%- elif rng is defined %}
-   - rng: {{ rng }}
-   {%- endif %}
-+  {%- if  node.loader is defined %}
-+  - loader: {{  node.loader }}
-+  {%- endif %}
-+  {%- if  node.machine is defined %}
-+  - machine: {{ node.machine }}
-+  {%- endif %}
-+  {%- if  node.cpu_mode is defined %}
-+  - cpu_mode: {{ node.cpu_mode }}
-+  {%- endif %}
-   - kwargs:
-       seed: True
-       serial_type: pty
-diff --git a/tests/pillar/control_virt_custom.sls b/tests/pillar/control_virt_custom.sls
-index 71cf37f..dcfafbd 100644
---- a/tests/pillar/control_virt_custom.sls
-+++ b/tests/pillar/control_virt_custom.sls
-@@ -63,11 +63,17 @@ salt:
-             image: ubuntu.qcow
-             size: medium
-             img_dest: /var/lib/libvirt/ssdimages
-+            machine: virt-2.11
-+            cpu_mode: host-passthrough
-           ubuntu2:
-             provider: node02.domain.com
-             image: bubuntu.qcomw
-             size: small
-             img_dest: /var/lib/libvirt/hddimages
-+            loader:
-+              readonly: yes
-+              type: pflash
-+              path: /usr/share/AAVMF/AAVMF_CODE.fd
-           ubuntu3:
-             provider: node03.domain.com
-             image: meowbuntu.qcom2