Rebase fuel-plugin for kvmfornfv to fuel 9.0

1. Uplifit fuel-plugin for kvmfornfv to fuel 9.0 due to that fuel 9.0 will be the
   deployment tool of OPNFV Colorado release.
2. Fixed quirk for kernel.
3. Added all tools and libs for building OVS module.
4. Allow KVM developer to build fuel-plugin-kvm after they modify the kerenl
   code without commiting their change into the repo first.  So, they can test
   their code change by fuel-plugin-kvm till they satisfy with their change,
   then commit to the repo.
5. The final code built into OPNFV fuel iso will depend on the commit ID for
   kvmfornfv speicifed in fuel for OPNFV source tree.

Change-Id: Iaf9ff49c69df374d0757884cfdac4cccee3eebe4
Signed-off-by: davidjchou <david.j.chou@intel.com>

11 files changed, 122 insertions, 226 deletions

diff --git a/fuel-plugin/Dockerfile b/fuel-plugin/Dockerfile
index b71a09271..ef2e90b65 100644
--- a/fuel-plugin/Dockerfile
+++ b/fuel-plugin/Dockerfile
@@ -1,10 +1,5 @@
 #!/bin/bash
 
 FROM ubuntu:14.04.3
-RUN apt-get update
-RUN apt-get install -y git fakeroot build-essential ncurses-dev xz-utils kernel-package bc autoconf
 RUN echo "ALL ALL=NOPASSWD: ALL" > /etc/sudoers.d/open-sudo
 RUN chmod 0440 /etc/sudoers.d/open-sudo
-ADD ./build_kvm.sh /root/build_kvm.sh
-RUN chmod +x /root/build_kvm.sh
-RUN /root/build_kvm.sh
diff --git a/fuel-plugin/README.md b/fuel-plugin/README.md
index 810d8a021..77dee17c2 100644
--- a/fuel-plugin/README.md
+++ b/fuel-plugin/README.md
@@ -14,7 +14,7 @@ Requirements
 
 | Requirement                      | Version/Comment |
 |----------------------------------|-----------------|
-| Mirantis OpenStack compatibility | 8.0             |
+| Mirantis OpenStack compatibility | 9.0             |
 
 Recommendations
 ---------------
@@ -26,43 +26,40 @@ Limitations
 
 None.
 
-Installation Guide
-==================
-
-KVM plugin installation
-----------------------------------------
-
-1. Clone the fuel-plugin repo from https://gerrit.opnfv.org/gerrit/kvmfornfv/fuel-plugin:
-
-        git clone https://gerrit.opnfv.org/gerrit/kvmfornfv/fuel-plugin
-
-2. Install the Fuel Plugin Builder:
-
-        pip install fuel-plugin-builder
-
-3. Build KVM Fuel plugin:
-
-        fpb --build fuel-plugin-kvm/
-
-4. The *fuel-plugin-kvm-[x.x.x].rpm* plugin package will be created in the plugin folder.
-  
-5. Move this file to the Fuel Master node with secure copy (scp):
-
-        scp fuel-plugin-kvm-[x.x.x].rpm root@<the_Fuel_Master_node_IP address>:/tmp
-
-6. While logged in Fuel Master install the KVM plugin:
-
-        fuel plugins --install fuel-plugin-kvm-[x.x.x].rpm
-
-7. Check if the plugin was installed successfully:
+Build Guide
+===========
 
-        fuel plugins
+Buiding system pre-requistes
+----------------------------
+1. Ubuntu 14.04 LTS desktop or server
+2. Minimum 4 CPU cores, 6 GB RAM, and 200 GB available hard drive space
+3. "VirtualBox" and "vagrant" installed
+
+Buid instruction
+----------------
+1. Clone the kvmfornfv repo from https://gerrit.opnfv.org/gerrit/kvmfornfv by
+   "git clone https://gerrit.opnfv.org/gerrit/kvmfornfv"
+2. You can modify the kernel code in kvmfornfv/kernel as you want
+3. Go to kvmfornfv/fuel-plugin/vagrant, type "vagrant destroy -f; vagrant up;
+   estroy -f", the building will start
+4. When the building completes, you should find the built fuel-plugin-kvm in
+   kvmfornfv/fuel-plugin with the name as "fuel-plugin-kvm-0.9-0.9.0-1.noarch.rpm",
+   where "0.9-0.9.0-1" is the version information for this plugin, this version info
+   may be changed in future. The built plugin incules the changes you made.
 
-        id | name             | version | package_version
-        ---|------------------|---------|----------------
-        1  | fuel-plugin-kvm | 0.5.2   | 3.0.0
+Installation Guide
+==================
+1. Move the built fuel-pluginn-kvm to the Fuel Master node with secure copy (scp):
+      scp fuel-plugin-kvm-0.9-0.9.0-1.noarch.rpm root@<the_Fuel_Master_node_IP address>:
+2. While logged in Fuel Masternode, install the KVM plugin by typing:
+        fuel plugins --install fuel-plugin-kvm-0.9-0.9.0-1.noarch.rpm
+3. Check if the plugin was installed successfully by typing "fuel plugins", the folowing
+   should appear:
 
-8. Plugin is ready to use and can be enabled on the Settings tab of the Fuel web UI.
+        id | name             | version | package_version | release
+        ---+------------------+---------+-----------------+--------------------
+        1  | fuel-plugin-kvm  | 0.9.0   | 4.0.0           | ubuntu (mitaka-9.0)
+4. Plugin is ready to use and can be enabled on the Settings tab of the Fuel web UI.
 
 
 User Guide
@@ -70,66 +67,19 @@ User Guide
 
 KVM plugin configuration
 ---------------------------------------------
-
 1. Create a new environment with the Fuel UI wizard.
 2. Click on the Settings tab of the Fuel web UI.
-3. Scroll down the page, select the plugin checkbox. 
-
+3. Scroll down the page, select the plugin checkbox.
 
-Build options
--------------
-
-It is possible to modify process of building plugin by setting environment variables. Look into [pre_build_hook file](pre_build_hook) for more details.
-
-Dependencies
-------------
-
-If you plan to use plugin in environment without internet access or/and CentOS environment modify build command:
-
-     INCLUDE_DEPENDENCIES=true fpb --build fuel-plugin-kvm/
-
-Pre build script will try download required dependencies so it become part of the compiled plugin.
-
-Note: List of packages for [ubuntu](kvm_package/ubuntu/dependencies.txt) and [centos](kvm_package/centos/dependencies.txt) may need to be modified if packages in centos or ubuntu repositories will change.
 
 Testing
 -------
-
 None.
 
 Known issues
 ------------
-
 None.
 
-
-
-Development
-===========
-
-The *OpenStack Development Mailing List* is the preferred way to communicate,
-emails should be sent to `openstack-dev@lists.openstack.org` with the subject
-prefixed by `[fuel][plugins][kvm]`.
-
-Reporting Bugs
---------------
-
-Bugs should be filled on the [Launchpad fuel-plugins project](
-https://bugs.launchpad.net/fuel-plugins) (not GitHub) with the tag `kvm`.
-
-
-Contributing
-------------
-
-If you would like to contribute to the development of this Fuel plugin you must
-follow the [OpenStack development workflow](
-http://docs.openstack.org/infra/manual/developers.html#development-workflow).
-
-Patch reviews take place on the [OpenStack gerrit](
-https://review.openstack.org/#/q/status:open+project:stackforge/fuel-plugin-kvm,n,z)
-system.
-
 Contributors
 ------------
-
-* ling.y.yu@intel.com,ruijing.guo@intel.com
+* davi.j.chou@intel.com, ruijing.guo@intel.comi, ling.y.yu@intel.com
diff --git a/fuel-plugin/build_kvm.sh b/fuel-plugin/build_kvm.sh
index 0ea75a0e4..9984f3bab 100755
--- a/fuel-plugin/build_kvm.sh
+++ b/fuel-plugin/build_kvm.sh
@@ -1,133 +1,70 @@
 #!/bin/bash
 
-KVM_COMMIT="0e68cb048bb8aadb14675f5d4286d8ab2fc35449"
+SRC=/
+CONFIG="arch/x86/configs/opnfv.config"
+VERSION="1.0.OPNFV"
 OVS_COMMIT="4ff6642f3c1dd8949c2f42b3310ee2523ee970a6"
-KEEP=no
 
 quirks() {
-
-	#
-	# Apply out of tree patches
-	#
-	for i in $SRC/kvmfornfv/patches/$1/*.patch
-	do
-		if [ -f "$i" ]
-		then
-			echo "Applying: $i"
-			patch -p1 <$i
-		fi
-	done
+#
+# Apply out of tree patches
+#
+for i in $SRC/kvmfornfv/patches/$1/*.patch
+do
+    if [ -f "$i" ]
+    then
+        echo "Applying: $i"
+        patch -p1 <$i
+    fi
+done
 }
 
-for i
-do
-	case $i in
+apt-get update
+apt-get install -y git fakeroot build-essential ncurses-dev xz-utils kernel-package bc autoconf automake libtool python python-pip
 
-	-k)	KEEP=yes
-		shift
-		;;
+cd $SRC
 
-	-c)	KVM_COMMIT=$2
-		shift;shift
-		;;
+# Get the Open VSwitch sources
+rm -rf ovs
+git clone https://github.com/openvswitch/ovs.git
+cd ovs; git checkout $OVS_COMMIT
 
-	-o)	OVS_COMMIT=$2
-		shift;shift
-		;;
+cd $SRC/kvmfornfv/
+quirks kernel
 
-	esac
-done
+cd kernel
 
-SRC=${1:-/root}
-CONFIG=${2:-arch/x86/configs/opnfv.config}
-VERSION=${3:-1.0.OPNFV}
-
-# Check for necessary build tools
-if ! type git >/dev/null 2>/dev/null
-then
-	echo "Build tools missing, run the command
-
-apt-get install git fakeroot build-essential ncurses-dev xz-utils kernel-package automake
-
-as root and try again"
-	exit 1
-fi
-
-# Make sure the source dir exists
-if [ ! -d $SRC ]
-then
-	echo "$SRC: no such directory"
-	exit 1
-fi
-
-(
-	cd $SRC
-
-	# Get the Open VSwitch sources
-	if [ ! -d ovs ]
-	then
-		git clone https://github.com/openvswitch/ovs.git
-	fi
-
-	# Get the KVM for NFV kernel sources
-	if [ ! -d kvmfornfv ]
-	then
-		git clone https://gerrit.opnfv.org/gerrit/kvmfornfv
-	fi
-	cd kvmfornfv
-	git pull
-	if [ x$KVM_COMMIT != x ]
-	then
-		git checkout $KVM_COMMIT
-	else
-		git reset --hard
-	fi
-	cd kernel
-
-	# Workaround build bug on Ubuntu 14.04
-	cat <<EOF > arch/x86/boot/install.sh
+# Workaround build bug on Ubuntu 14.04
+cat <<EOF > arch/x86/boot/install.sh
 #!/bin/sh
 cp -a -- "\$2" "\$4/vmlinuz-\$1"
 EOF
 
-	quirks kernel
-
-	# Configure the kernel
-	cp $CONFIG .config
-
-	make oldconfig </dev/null
-
-	# Build the kernel debs
-	if [ $KEEP = no ]
-	then
-		make-kpkg clean
-	fi
-	fakeroot make-kpkg --initrd --revision=$VERSION kernel_image kernel_headers
-	git checkout arch/x86/boot/install.sh
-	git checkout fs/xfs/xfs_super.c
-
-	# Build OVS kernel modules
-	cd ../../ovs
-	if [ x$OVS_COMMIT != x ]
-	then
-		git checkout $OVS_COMMIT
-	else
-		git reset --hard
-	fi
-
-	quirks ovs
-
-	./boot.sh
-	./configure --with-linux=$SRC/kvmfornfv/kernel
-	make
-
-	# Add OVS kernel modules to kernel deb
-	dpkg-deb -x $SRC/kvmfornfv/linux-image*.deb ovs.$$
-	dpkg-deb --control $SRC/kvmfornfv/linux-image*.deb ovs.$$/DEBIAN
-	cp datapath/linux/*.ko ovs.$$/lib/modules/*/kernel/net/openvswitch
-	depmod -b ovs.$$ -a `ls ovs.$$/lib/modules`
-	dpkg-deb -b ovs.$$ $SRC/kvmfornfv/linux-image*.deb
-	rm -rf ovs.$$
-)
-
-mv $SRC/kvmfornfv/*.deb .
+# Configure the kernel
+cp $CONFIG .config
+
+make oldconfig </dev/null
+
+# Build the kernel debs
+make-kpkg clean
+
+fakeroot make-kpkg --initrd --revision=$VERSION kernel_image kernel_headers
+
+# Build OVS kernel modules
+cd ../../ovs
+
+quirks ovs
+pip install six
+
+./boot.sh
+./configure --with-linux=$SRC/kvmfornfv/kernel
+make
+
+# Add OVS kernel modules to kernel deb
+dpkg-deb -x $SRC/kvmfornfv/linux-image*.deb ovs.$$
+dpkg-deb --control $SRC/kvmfornfv/linux-image*.deb ovs.$$/DEBIAN
+cp datapath/linux/*.ko ovs.$$/lib/modules/*/kernel/net/openvswitch
+depmod -b ovs.$$ -a `ls ovs.$$/lib/modules`
+dpkg-deb -b ovs.$$ $SRC/kvmfornfv/linux-image*.deb
+rm -rf ovs.$$
+
diff --git a/fuel-plugin/deployment_scripts/puppet/manifests/kvm-install.pp b/fuel-plugin/deployment_scripts/puppet/manifests/kvm-install.pp
index 23902cc37..a396e9472 100644
--- a/fuel-plugin/deployment_scripts/puppet/manifests/kvm-install.pp
+++ b/fuel-plugin/deployment_scripts/puppet/manifests/kvm-install.pp
@@ -1,4 +1,4 @@
-$fuel_settings = parseyaml(file('/etc/compute.yaml'))
+$fuel_settings = parseyaml(file('/etc/astute.yaml'))
 if $operatingsystem == 'Ubuntu' {
         if $fuel_settings['fuel-plugin-kvm']['use_kvm'] {
                 package { 'linux-headers-4.4.6-rt14nfv':
diff --git a/fuel-plugin/deployment_tasks.yaml b/fuel-plugin/deployment_tasks.yaml
index fe51488c7..ee9e998fa 100644
--- a/fuel-plugin/deployment_tasks.yaml
+++ b/fuel-plugin/deployment_tasks.yaml
@@ -1 +1,10 @@
-[]
+- id: kvm_install
+  type: puppet
+  version: 2.0.0
+  groups: [compute]
+  required_for: [pre_deployment_end]
+  requires: [pre_deployment_start]
+  parameters:
+    puppet_manifest: puppet/manifests/kvm-install.pp
+    puppet_modules: puppet/modules:/etc/puppet/modules
+    timeout: 720
diff --git a/fuel-plugin/metadata.yaml b/fuel-plugin/metadata.yaml
index 4d9e492d8..04beeed9b 100644
--- a/fuel-plugin/metadata.yaml
+++ b/fuel-plugin/metadata.yaml
@@ -3,11 +3,11 @@ name: fuel-plugin-kvm
 # Human-readable name for your plugin
 title: fuel-plugin-kvm
 # Plugin version
-version: '0.5.2'
+version: '0.9.0'
 # Description
 description: 'This plugin provides to deploy kvm'
 # Required fuel version
-fuel_version: ['8.0']
+fuel_version: ['9.0']
 # Specify license of your plugin
 licenses: ['Apache License Version 2.0']
 # Specify author or company name
@@ -21,10 +21,12 @@ groups: ['hypervisor']
 # The plugin is compatible with releases in the list
 releases:
   - os: ubuntu
-    version: liberty-8.0
+    version: mitaka-9.0
     mode: ['ha']
     deployment_scripts_path: deployment_scripts/
     repository_path: repositories/ubuntu
 
 # Version of plugin package
-package_version: '3.0.0'
+package_version: '4.0.0'
+
+is_hotpluggable: false
diff --git a/fuel-plugin/pre_build_hook b/fuel-plugin/pre_build_hook
index 0dfc36cbc..a3c883283 100755
--- a/fuel-plugin/pre_build_hook
+++ b/fuel-plugin/pre_build_hook
@@ -8,10 +8,14 @@ DIR="$(dirname `readlink -f $0`)"
 function build_pkg {
   case $1 in
     ubuntu)
+      rm -rf repositories/ubuntu; mkdir -p repositories/ubuntu
       sudo docker build -t kvm .
-      container_id=`sudo docker run -d kvm`
-      sudo docker cp $container_id:/linux-headers-4.4.6-rt14nfv_1.0.OPNFV_amd64.deb ${DIR}/repositories/ubuntu/
-      sudo docker cp $container_id:/linux-image-4.4.6-rt14nfv_1.0.OPNFV_amd64.deb ${DIR}/repositories/ubuntu/
+
+      # run /kvmfornfv/fuel-plugin/build_kvm.sh in docker
+      sudo docker run -v ${DIR}/..:/kvmfornfv -t  kvm /kvmfornfv/fuel-plugin/build_kvm.sh
+      # debug in console by the following command
+      # sudo docker run -v /kvmfornfv:/kvmfornfv -ti  kvm  /bin/bash
+      cp ${DIR}/../*.deb repositories/ubuntu
     ;;
     *) echo "Not supported system"; exit 1;;
   esac
diff --git a/fuel-plugin/repositories/ubuntu/README b/fuel-plugin/repositories/ubuntu/README
deleted file mode 100644
index 11e6dd081..000000000
--- a/fuel-plugin/repositories/ubuntu/README
+++ /dev/null
@@ -1 +0,0 @@
-This directory holds kvm deb packages
diff --git a/fuel-plugin/tasks.yaml b/fuel-plugin/tasks.yaml
index c5f6ba067..fe51488c7 100644
--- a/fuel-plugin/tasks.yaml
+++ b/fuel-plugin/tasks.yaml
@@ -1,8 +1 @@
-- id: kvm_install
-  role: ['compute']
-  stage: pre_deployment
-  type: puppet
-  parameters:
-    puppet_manifest: puppet/manifests/kvm-install.pp
-    puppet_modules: puppet/modules:/etc/puppet/modules
-    timeout: 720
+[]
diff --git a/fuel-plugin/vagrant/Vagrantfile b/fuel-plugin/vagrant/Vagrantfile
index 0505333e0..f9d17fc4e 100644
--- a/fuel-plugin/vagrant/Vagrantfile
+++ b/fuel-plugin/vagrant/Vagrantfile
@@ -11,7 +11,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
 
   config.vm.define "fuel" do | h |
     h.vm.host_name = "fuel"
-    h.vm.provision :shell, :inline => "/vagrant/build_fuel_plugin.sh"
+    h.vm.provision :shell, :inline => "/vagrant/build_fuel_plugin.sh", privileged: false
     h.vm.synced_folder "../..", "/kvmfornfv"
     h.vm.provider :virtualbox do |v|
       v.customize ["modifyvm", :id, "--memory", 4096]
diff --git a/fuel-plugin/vagrant/build_fuel_plugin.sh b/fuel-plugin/vagrant/build_fuel_plugin.sh
index 257b3ca81..b18ae173b 100755
--- a/fuel-plugin/vagrant/build_fuel_plugin.sh
+++ b/fuel-plugin/vagrant/build_fuel_plugin.sh
@@ -9,4 +9,11 @@ sudo apt-get install ruby -y
 sudo gem install rubygems-update
 sudo gem install fpm
 sudo apt-get install docker.io -y
-cd /kvmfornfv/fuel-plugin; fpb --debug --build .
+cd /home/vagrant
+# Will build fuel-plugin-kvm in guest VM local directory, not change host
+cp -r /kvmfornfv .
+cd kvmfornfv/fuel-plugin
+fpb --debug --build .
+# Copy the built fuel-plugin-kvm back to the host
+rm /kvmfornfv/fuel-plugin/fuel-plugin-kvm*.rpm
+cp fuel-plugin-kvm*.rpm /kvmfornfv/fuel-plugin/.