TOPIC: ODL DOCKER

AUTHOR: DANIEL SMITH
*** PLEASE MERGE **

UPDATED TO REFLECT INPUT FRMO J. BJUREL
  - fixed whitespaces
  - fixed location of .erb file (they should have been in templates - the directory was there just int he wrong spot)
  - removed opcheck.pp from common opnfv class
  - Removed Debug from fuel/build/Makefile :)

UPDATE: Input from S. Berg and F. Bockners incorporated.  Merge required before we can refactor to common puppet manifest directory

This patchset delivers the folowing functionality:
	- implementation of the common/opnfv-puppet structure / move of .pp files and update of f_odl_docker to build / source from there
	- creation of f_odl_doc subclass
	- fetch of latest stable release of ODL
	- fetch of latest docker binary release (TODO: this will be changed in next patchset push)
	- build of docker container with all needed libs and port exposure for DLUX and OVSDB/ODL with Openstack Integration (OVS Manager)
	- deployment of both the target odl docker image as well as the docker runtime binary to the control nodes via puppet script.

GENERATES:
	- docker-latest - binary of docker for use on control node
	- odl_docker_image.tar - a docker container with a ODL controller running DLUX and OVSDB

ENABLE / DISABLE:
        - Comment/Un-comment "SUBDIRS += f_odl_doc in base (fuel/build/) Makefile

Breakdown of Update / Edits per File:
=====================================
fuel/build/Makefile
	- Modified include to capture the newly created f_odl_doc directory

fuel/build/f_odl_docker/Makefile
	- Fetches libraries and produces two outputs:
		docker-latest - binary of docker (actually lxc-docker cause ODL Container is running 12.04 (precise) libraries - i.e java7, tz 12.04, etc)
		odl_docker_image.tar - this is a docker image defined in ./dockefile/Dockerfile and contains the ODL distro + setup and deployment scripts for
					runtime on the target control node.

fuel/build/f_odl_docker/dockerfile/Dockerfile
	- This Dockerfile defined the packages for use in the Docker Container that will run ODL with DLUX and OVSDB submodules.  It also defines the ports to be
		exposed to the HOST OS (and thus as well through docker the ODL Controller exists in a private, but routable via but the fuel (10.20.0.0/16) and the
		tenants public network since docker handles the mapping (see the docker run command in the start_odl_docker.sh script)

fuel/build/f_odl_docker/dockerfile/check_feature.sh
	- Simple expect script that starts up a client and checks that the features are installed (used during visual demo only)
		TODO - Replace with API call to ODL KARAF to install features (LOOKUP - Dont know how to address Karaf programatically - LOOKUP)

fuel/build/f_odl_docker/dockerfile/speak.sh
	- Expect script called by odl_start_docker.sh once karaf is up to install the features that we need (runtime inside container script) called via ENTRYPOINT in Dockerfile
		at runtime on control node.

fuel/build/f_odl_docker/dockerfile/start_odl_docker.sh
	- This is the CMD/ENTRYPOINT defined in docker and is what is called from the controller when you start the container (note: This runs inside the conatiner), not
		to be confused with staring the actual container on the control node).  This script fires up Karaf first time, loads DLUX and OVSDB modules
		and monitors that the container is up.  The container itself is started on the control node via /opt/opnfv/start_odl_container.sh which includes the syntax
		for the port mapping (RANDOM or 1:1).
		TODO -
			integrate into controller monitor script to ensure better handling (stop, start, monitor) of docker processes
			remove expect helper scripts and replace with API/JSON or some other appropriate method to KARAF

fuel/build/f_odl_docker/puppet/modules/opnfv/manifests/odl_doc.pp
	- This puppet manifest defined where the docker binary and docker image should be placed on the target control node. /opt/opnfv/start_odl_container.sh will install
		docker binary package (if necessary) and load ODL docker image into docker, start the image.  This file just ensures placement in /opt/opnfv/odl_docker

fuel/build/f_opnfv_puppet/puppet/modules/opnfv/manifests/init.pp (MODIFICATION):
	- Removed previous includes and updated to have only f_odl_doc added

fuel/build/f_odl_doc/scripts/start_odl_container.sh
	- this is the control script that will start the docker container (to be run on the control node), this is deployed
		this is deployed to /opt/opnfv on the control node via odl_doc puppet manifest file.

JIRA:

DEPRECATED:
Deleted files are no longer needed due to new implementation of ODL.

Change-Id: I26c13cc468a2aba18af78b7a3c78a719033f03e0
Signed-off-by: Daniel Smith <daniel.smith@ericsson.com>

10 files changed, 416 insertions, 10 deletions

diff --git a/common/puppet-opnfv/manifests/init.pp b/common/puppet-opnfv/manifests/init.pp
index f1b4576..6e211da 100644
--- a/common/puppet-opnfv/manifests/init.pp
+++ b/common/puppet-opnfv/manifests/init.pp
@@ -14,17 +14,27 @@
 
 
 class opnfv {
-    exec {'disable selinux':
-        command => '/usr/sbin/setenforce 0',
-        unless => '/usr/sbin/getenforce | grep Permissive',
-    }
-    include stdlib
-    stage { 'presetup':
-      before => Stage['setup'],
+     if $::osfamily == 'Fuel' {
+	include opnfv::resolver
+	include opnfv::ntp
+	include opnfv::add_packages
+	include opnfv::odl_docker
+	include opnfv::opncheck
     }
 
-    class { "opnfv::repo":
-     stage => presetup,
-    }
+   if $::osfamily == 'RedHat' {
+
+    	exec {'disable selinux':
+       		command => '/usr/sbin/setenforce 0',
+        	unless => '/usr/sbin/getenforce | grep Permissive',
+    	}
+    	include stdlib
+    	stage { 'presetup':
+      		before => Stage['setup'],
+    	}
 
+    	class { "opnfv::repo":
+     		stage => presetup,
+    	}
+   }
 }
diff --git a/common/puppet-opnfv/manifests/ntp.pp b/common/puppet-opnfv/manifests/ntp.pp
new file mode 100644
index 0000000..c27175e
--- /dev/null
+++ b/common/puppet-opnfv/manifests/ntp.pp
@@ -0,0 +1,72 @@
+##############################################################################
+# Copyright (c) 2015 Ericsson AB and others.
+# stefan.k.berg@ericsson.com
+# jonas.bjurel@ericsson.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+
+# Class: Ntp
+#
+# Add Ntp content passed through astute.yaml into ntp.conf depending on the role
+#
+# Suitable yaml content:
+# <begin>
+# opnfv:
+#   ntp:
+#     controller: |
+#      line 1
+#      line 2
+#    compute: |
+#      line 1
+#      line 2
+# <end>
+#
+#
+#
+
+class opnfv::ntp(
+  $file='/etc/ntp.conf'
+) {
+
+  if $::fuel_settings['role'] {
+    if ($::fuel_settings['opnfv'] and
+    $::fuel_settings['opnfv']['ntp']) {
+      case $::fuel_settings['role'] {
+        /controller/: {
+          if $::fuel_settings['opnfv']['ntp']['controller'] {
+            $template = 'opnfv/ntp.conf.controller.erb'
+            $file_content = $::fuel_settings['opnfv']['ntp']['controller']
+          }
+        }
+        /compute/:    {
+          if $::fuel_settings['opnfv']['ntp']['compute'] {
+            $template = 'opnfv/ntp.conf.compute.erb'
+            $file_content = $::fuel_settings['opnfv']['ntp']['compute']
+          }
+        }
+      }
+    }
+  }
+
+  if $file_content {
+    package { 'ntp':
+      ensure => installed,
+    }
+
+    file { $file:
+      content => template($template),
+      notify  => Service['ntp'],
+    }
+
+    service { 'ntp':
+      ensure  => running,
+      enable  => true,
+      require => [ Package['ntp'], File[$file]]
+    }
+  }
+}
+
+
diff --git a/common/puppet-opnfv/manifests/odl_docker.pp b/common/puppet-opnfv/manifests/odl_docker.pp
new file mode 100644
index 0000000..6e70ba0
--- /dev/null
+++ b/common/puppet-opnfv/manifests/odl_docker.pp
@@ -0,0 +1,50 @@
+##############################################################################
+# Copyright (c) 2015 Ericsson AB and others.
+# daniel.smith@ericsson.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+
+class opnfv::odl_docker
+{
+    case $::fuel_settings['role'] {
+      /controller/: {
+
+        file { "/opt":
+                ensure => "directory",
+             }
+
+        file { "/opt/opnfv":
+                ensure => "directory",
+                owner => "root",
+                group => "root",
+                mode => 777,
+             }
+
+        file { "/opt/opnfv/odl":
+                ensure => "directory",
+             }
+
+        file { "/opt/opnfv/odl/odl_docker_image.tar":
+                ensure => present,
+                source => "/etc/puppet/modules/opnfv/odl_docker/odl_docker_image.tar",
+                mode => 750,
+             }
+
+        file { "/opt/opnfv/odl/docker-latest":
+                ensure => present,
+                source => "/etc/puppet/modules/opnfv/odl_docker/docker-latest",
+                mode => 750,
+             }
+
+        file { "/opt/opnfv/odl/start_odl_conatiner.sh":
+                ensure => present,
+                source => "/etc/puppet/modules/opnfv/scripts/start_odl_container.sh",
+                mode => 750,
+             }
+  }
+ }
+}
+
diff --git a/common/puppet-opnfv/manifests/resolver.pp b/common/puppet-opnfv/manifests/resolver.pp
new file mode 100644
index 0000000..2951f7e
--- /dev/null
+++ b/common/puppet-opnfv/manifests/resolver.pp
@@ -0,0 +1,62 @@
+##############################################################################
+# Copyright (c) 2015 Ericsson AB and others.
+# stefan.k.berg@ericsson.com
+# jonas.bjurel@ericsson.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+
+# Class: opnfv::resolver
+#
+# Add resolver content passed through astute.yaml into resolv.conf
+# depending on the role
+#
+# Suitable yaml content:
+# <begin>
+# opnfv:
+#  dns:
+#    compute:
+#    - 100.100.100.2
+#    - 100.100.100.3
+#    controller:
+#    - 100.100.100.102
+#    - 100.100.100.104
+# <end>
+#
+#
+#
+
+class opnfv::resolver()
+{
+  if $::fuel_settings['role'] {
+    if $::fuel_settings['role']  == 'primary-controller' {
+      $role = 'controller'
+    } else {
+      $role = $::fuel_settings['role']
+    }
+
+    if ($::fuel_settings['opnfv']
+        and $::fuel_settings['opnfv']['dns']
+        and $::fuel_settings['opnfv']['dns'][$role]) {
+      $nameservers=$::fuel_settings['opnfv']['dns'][$role]
+
+      file { '/etc/resolv.conf':
+            owner   => root,
+            group   => root,
+            mode    => '0644',
+            content => template('opnfv/resolv.conf.erb'),
+      }
+# /etc/resolv.conf is re-generated at each boot by resolvconf, so we
+# need to store there as well.
+      file { '/etc/resolvconf/resolv.conf.d/head':
+            owner   => root,
+            group   => root,
+            mode    => '0644',
+            content => template('opnfv/resolv.conf.erb'),
+      }
+    }
+  }
+}
+
diff --git a/common/puppet-opnfv/manifests/templates/dockerfile/Dockerfile b/common/puppet-opnfv/manifests/templates/dockerfile/Dockerfile
new file mode 100644
index 0000000..6b2f241
--- /dev/null
+++ b/common/puppet-opnfv/manifests/templates/dockerfile/Dockerfile
@@ -0,0 +1,82 @@
+####################################################################
+# Copyright (c) 2015 Ericsson AB and others.
+# daniel.smith@ericsson.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+#
+#  DOCKERFILE TO CREATE ODL IN CONTAINER AND EXPOSE DLUX AND OVSDB TO ODL
+#
+#############################################################################
+
+
+#Set the base image - note: the current release of Karaf is using Jdk7 and alot of 12.04, so we will use it rather than 14.04 and backport a ton of stuff
+FROM ubuntu:12.04
+
+# Maintainer Info
+MAINTAINER Daniel Smith
+
+
+#Run apt-get update one start just to check for updates when building
+RUN echo "Updating APT"
+RUN apt-get update
+RUN echo "Adding wget"
+RUN apt-get install -y wget
+RUN apt-get install -y net-tools
+RUN apt-get install -y openjdk-7-jre
+RUN apt-get install -y openjdk-7-jdk
+RUN apt-get install -y openssh-server
+RUN apt-get install -y vim
+RUN apt-get install -y expect
+RUN apt-get install -y daemontools
+RUN mkdir -p /opt/odl_source
+RUN bash -c 'echo "export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64" >> ~/.bashrc'
+
+
+
+#Now lets got and fetch the ODL distribution
+RUN echo "Fetching ODL"
+RUN wget https://nexus.opendaylight.org/content/groups/public/org/opendaylight/integration/distribution-karaf/0.2.2-Helium-SR2/distribution-karaf-0.2.2-Helium-SR2.tar.gz -O /opt/odl_source/distribution-karaf-0.2.2-Helium-SR2.tar.gz
+
+RUN echo "Untarring ODL inplace"
+RUN mkdir -p /opt/odl
+RUN tar zxvf /opt/odl_source/distribution-karaf-0.2.2-Helium-SR2.tar.gz -C /opt/odl
+
+RUN echo "Installing DLUX and other features into ODL"
+#COPY dockerfile/container_scripts/start_odl_docker.sh /etc/init.d/start_odl_docker.sh
+COPY container_scripts/start_odl_docker_container.sh /etc/init.d/
+COPY container_scripts/speak.sh /etc/init.d/
+#COPY dockerfile/container_scripts/speak.sh /etc/init.d/speak.sh
+RUN chmod 777 /etc/init.d/start_odl_docker_container.sh
+RUN chmod 777 /etc/init.d/speak.sh
+
+
+
+# Expose the ports
+
+# PORTS FOR BASE SYSTEM AND DLUX
+EXPOSE 8101
+EXPOSE 6633
+EXPOSE 1099
+EXPOSE 43506
+EXPOSE 8181
+EXPOSE 8185
+EXPOSE 9000
+EXPOSE 39378
+EXPOSE 33714
+EXPOSE 44444
+EXPOSE 6653
+
+# PORTS FOR OVSDB AND ODL CONTROL
+EXPOSE 12001
+EXPOSE 6640
+EXPOSE 8080
+EXPOSE 7800
+EXPOSE 55130
+EXPOSE 52150
+EXPOSE 36826
+
+# set the ENTRYPOINT - An entry point allows us to run this container as an exectuable
+CMD ["/etc/init.d/start_odl_docker.sh"]
diff --git a/common/puppet-opnfv/manifests/templates/dockerfile/container_scripts/check_feature.sh b/common/puppet-opnfv/manifests/templates/dockerfile/container_scripts/check_feature.sh
new file mode 100644
index 0000000..5062a46
--- /dev/null
+++ b/common/puppet-opnfv/manifests/templates/dockerfile/container_scripts/check_feature.sh
@@ -0,0 +1,18 @@
+##############################################################################
+# Copyright (c) 2015 Ericsson AB and others.
+# daniel.smith@ericsson.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+
+#!/usr/bin/expect
+spawn /opt/odl/distribution-karaf-0.2.2-Helium-SR2/bin/client
+expect "root>"
+send "feature:list | grep -i odl-restconf\r"
+send "\r\r\r"
+expect "root>"
+send "logout\r"
+
+
diff --git a/common/puppet-opnfv/manifests/templates/dockerfile/container_scripts/speak.sh b/common/puppet-opnfv/manifests/templates/dockerfile/container_scripts/speak.sh
new file mode 100644
index 0000000..d5fb217
--- /dev/null
+++ b/common/puppet-opnfv/manifests/templates/dockerfile/container_scripts/speak.sh
@@ -0,0 +1,20 @@
+#!/usr/bin/expect
+##############################################################################
+# Copyright (c) 2015 Ericsson AB and others.
+# daniel.smith@ericsson.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+#
+# Simple expect script to start up ODL client and load feature set for DLUX and OVSDB
+#  NOTE: THIS WILL BE REPLACED WITH A PROGRAMATIC METHOD SHORTLY
+#################################################################################
+
+spawn /opt/odl/distribution-karaf-0.2.2-Helium-SR2/bin/client
+expect "root>"
+send "feature:install odl-base-all odl-aaa-authn odl-restconf odl-nsf-all odl-adsal-northbound odl-mdsal-apidocs  odl-ovsdb-openstack odl-ovsdb-northbound odl-dlux-core"
+send "\r\r\r"
+expect "root>"
+send "logout\r"
diff --git a/common/puppet-opnfv/manifests/templates/dockerfile/container_scripts/start_odl_docker_container.sh b/common/puppet-opnfv/manifests/templates/dockerfile/container_scripts/start_odl_docker_container.sh
new file mode 100644
index 0000000..46049be
--- /dev/null
+++ b/common/puppet-opnfv/manifests/templates/dockerfile/container_scripts/start_odl_docker_container.sh
@@ -0,0 +1,48 @@
+#!/bin/bash
+##############################################################################
+# Copyright (c) 2015 Ericsson AB and others.
+# daniel.smith@ericsson.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+#
+# Simple expect script to start up ODL client and load feature set for DLUX and OVSDB
+#  NOTE: THIS WILL BE REPLACED WITH A PROGRAMATIC METHOD SHORTLY
+#################################################################################
+#  Start up script for calling karaf / ODL inside a docker container.
+#
+#  This script will also call a couple expect scripts to load the feature set that we want
+
+
+#ENV
+export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
+
+#MAIN
+echo "Starting up the da Sheilds..."
+/opt/odl/distribution-karaf-0.2.2-Helium-SR2/bin/karaf server &
+echo "Sleeping 5 bad hack"
+sleep 10
+echo "should see stuff listening now"
+netstat -na
+echo " should see proess running for karaf"
+ps -efa
+echo " Starting the packages we want"
+/etc/init.d/speak.sh
+echo "Printout the status - if its right, you should see 8181 appear now"
+netstat -na
+ps -efa
+
+
+
+## This is a loop that keeps our container going currently, prinout the "status of karaf" to the docker logs every minute
+## Cheap - but effective
+while true;
+do
+        echo "Checking status of ODL:"
+        /opt/odl/distribution-karaf-0.2.2-Helium-SR2/bin/status
+        sleep 60
+done
+
+
diff --git a/common/puppet-opnfv/manifests/templates/ntp.conf.compute.erb b/common/puppet-opnfv/manifests/templates/ntp.conf.compute.erb
new file mode 100644
index 0000000..ac65293
--- /dev/null
+++ b/common/puppet-opnfv/manifests/templates/ntp.conf.compute.erb
@@ -0,0 +1,22 @@
+##############################################################################
+# Copyright (c) 2015 Ericsson AB and others.
+# stefan.k.berg@ericsson.com
+# jonas.bjurel@ericsson.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+
+tinker panic 0
+driftfile /var/lib/ntp/ntp.drift
+statistics loopstats peerstats clockstats
+filegen loopstats file loopstats type day enable
+filegen peerstats file peerstats type day enable
+filegen clockstats file clockstats type day enable
+restrict -4 default kod notrap nomodify nopeer noquery
+restrict -6 default kod notrap nomodify nopeer noquery
+restrict 127.0.0.1
+restrict ::1
+<%= @file_content %>
+
diff --git a/common/puppet-opnfv/manifests/templates/ntp.conf.controller.erb b/common/puppet-opnfv/manifests/templates/ntp.conf.controller.erb
new file mode 100644
index 0000000..ac65293
--- /dev/null
+++ b/common/puppet-opnfv/manifests/templates/ntp.conf.controller.erb
@@ -0,0 +1,22 @@
+##############################################################################
+# Copyright (c) 2015 Ericsson AB and others.
+# stefan.k.berg@ericsson.com
+# jonas.bjurel@ericsson.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+
+tinker panic 0
+driftfile /var/lib/ntp/ntp.drift
+statistics loopstats peerstats clockstats
+filegen loopstats file loopstats type day enable
+filegen peerstats file peerstats type day enable
+filegen clockstats file clockstats type day enable
+restrict -4 default kod notrap nomodify nopeer noquery
+restrict -6 default kod notrap nomodify nopeer noquery
+restrict 127.0.0.1
+restrict ::1
+<%= @file_content %>
+