Synch stable/brahmaputra with master

Several commits over recent months have not been cherry-picked to
stable/brahmaputra. As some of these have affected gerrit ci a set of
intermediate patches will not pass gerrit ci. So this is a merge commit
of the outstanding patches that need to be cherry-picked and brings
stable/b to be identical to master commit
4e07dd488cb78b8333763418c4ccc25e05766956 Tue May 10 18:31:20 2016 -0400
"Fix for new location of OVS and DPDK COPR repos."

The patches merged into this patch are:
        0001-Build-OVS-with-dpdk.patch
        0002-Fix-pathname-to-script.patch
        0003-Fix-uploading-of-files-and-remove-date-from-file-nam.patch
        0004-OVS-Archive-file-name-changed.patch
        0005-Fixes-RPM-build-problem.-Use-correct-copr-repo.-Fix-.patch
        0006-Fixed-base-name-of-intermediate-tarball-name.patch
        0007-Fix-build-problem-by-freezing-copr-commits.patch
        0008-fuel-plugin-Pick-updated-libpcre-packages.patch
        0009-Fix-for-new-location-of-OVS-and-DPDK-COPR-repos.patch

Plus final manual changes (add ci/clean.sh) to bring stable brahmaputra
identical to master.

Change-Id: e7ac7b4cb358b907b8756c4b95cc6b6618a76913
Signed-off-by: Billy O'Mahony <billy.o.mahony@intel.com>
Signed-off-by: Tom Herbert <therbert@redhat.com>
Reviewed-by: Jonas Bjurel <jonas.bjurel@ericsson.com>
Reviewed-by: Mark D. Gray <mark.d.gray@intel.com>

Build OVS with dpdk

Builds OVS with DPDK. Uses dpdk 2.2.0 and Open vSwitch 2.5.
Includes Multi-Queue and recent patches.
Open vSwitch RPM is now built with DPDK dataplane.
Upstream copr (Fedora) spec files and patches are used for building.

Also includes fixes of RPM building and command line options.
Adds dpdk arg to RPM test script.

Use "pure" COPR spec files without modifications. Fixed some function problems.
Addressed reviewer comments. Delete local copies of spec files
Build dpdk snapshot instead of using upstream snapshot.
Fix clean script.

"stack" VM with its default configuration doesn't support sse3 instruction set
required by DPDK.  Include workaround to test RPM script in host instead of VM.

Install prereqs for dpdk build.
Use 'set -e' in all scripts for error termination.
Add PyQt4 Pre-req.
Move functions and respond to reviewers comments.

Change-Id: I8d5892ff61a814271972e8d2279628d645b2831b
Signed-off-by: Thomas F Herbert <therbert@redhat.com>
Conflicts:
	build/BuildAndTestOVS.sh
	build/build_ovs_rpm.sh
	build/config
	build/instack_ovs.sh
	build/test_ovs_rpm.sh
	ci/buildovs.sh

Fix pathname to script. This patch fixes bug that is currently blocking the daily build.

Change-Id: If3624ea4e93bc42c5fe633b0c30d506f2b70efd0
Signed-off-by: Thomas F Herbert <therbert@redhat.com>

Fix uploading of files and remove date from file name

Change-Id: I33c3c1bbb5c9e7cd45e693de2db3230887cde7da
Signed-off-by: Thomas F Herbert <therbert@redhat.com>

OVS Archive file name changed. Change message to call archive snapshot

Change-Id: Ia07f6e38e2108f061dd616aa393cd4d98d7e8f11
Signed-off-by: Thomas F Herbert <therbert@redhat.com>

Fixes RPM build problem. Use correct copr repo. Fix some version number parsing and tarball and rpm naming that was specific to old repo and old DPDK version. Now supports DPDK 16.04-rc4.

Change-Id: Ia64b38d676e945df27917f4d9aa90ba4872c8cb5
Signed-off-by: Thomas F Herbert <therbert@redhat.com>

Fixed base name of intermediate tarball name.

Change-Id: I23e60800e83f216474328c461a2ac7aff453412f
Signed-off-by: Thomas F Herbert <therbert@redhat.com>

Fix build problem by freezing copr commits Use correct variable for copr dpdk version

Change-Id: I09c5e7060924771a75376aebe5bf3bde897c38eb
Signed-off-by: Thomas F Herbert <therbert@redhat.com>

fuel plugin: Pick updated libpcre packages.

Account for changes in SR3 iso that caused the plugin deploy to fail
when it tried to downgrade libpcre packages.

Change-Id: I6a11fc2e887a76cf879b1c1a398784da691b8329
Signed-off-by: Billy O'Mahony <billy.o.mahony@intel.com>
Reviewed-by: Jonas Bjurel <jonas.bjurel@ericsson.com>
Reviewed-by: Mark D. Gray <mark.d.gray@intel.com>
Reviewed-by: Tom Herbert <therbert@redhat.com>
Reviewed-by: Michal Skalski <mskalski@mirantis.com>

Fix for new location of OVS and DPDK COPR repos.

Change-Id: Ibee81cbee778bb1b542317ed41b94d01f86426bc
Signed-off-by: Thomas F Herbert <therbert@redhat.com>

Final manual changes to bring stable/brahmaputra into line with master.

Signed-off-by: Billy O'Mahony <billy.o.mahony@intel.com>

4 files changed, 97 insertions, 110 deletions

diff --git a/ci/build.sh b/ci/build.sh
index c1655ad..2723abe 100755
--- a/ci/build.sh
+++ b/ci/build.sh
@@ -13,6 +13,7 @@
 #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 #    See the License for the specific language governing permissions and
 #    limitations under the License.
+set -e
 
 echo "Hello OVSNFV community!"
 
diff --git a/ci/buildovs.sh b/ci/buildovs.sh
index 77cb96d..5270f03 100755
--- a/ci/buildovs.sh
+++ b/ci/buildovs.sh
@@ -1,12 +1,23 @@
 #!/bin/bash
-##############################################################################
-# Copyright (c) 2015 Red Hat Inc. and others.
-# therbert@redhat.com
-# All rights reserved. This program and the accompanying materials
-# are made available under the terms of the Apache License, Version 2.0
-# which accompanies this distribution, and is available at
-# http://www.apache.org/licenses/LICENSE-2.0
-##############################################################################
+
+# Copyright (c) 2016 Open Platform for NFV Project, Inc. and its contributors
+#
+#    Licensed under the Apache License, Version 2.0 (the "License");
+#    you may not use this file except in compliance with the License.
+#    You may obtain a copy of the License at
+#
+#        http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS,
+#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#    See the License for the specific language governing permissions and
+#    limitations under the License.
+
+set -e
+
+echo "==================================="
+echo executing $0 $@
 
 # Check to verify that I am being run by Jenkins CI.
 
@@ -16,9 +27,7 @@ if [ -z ${WORKSPACE+1} ]; then
 fi
 
 
-if [ -z ${OVSTAG+1} ]; then
-    export TAG=master
-else
+if [ ${OVSTAG} ]; then
     export TAG=$OVSTAG
 fi
 
@@ -38,7 +47,6 @@ if [ ! -f $BUILD_BASE/config ]; then
 fi
 
 export PATH=$PATH:$WORKSPACE/ci:$BUILD_BASE
-
 source $BUILD_BASE/config
 
 cd $BUILD_BASE
@@ -51,10 +59,6 @@ export CACHE_DIR=$TOPDIR/cache
 export TMPDIR=$TOPDIR/scratch
 export RPMDIR=$TOPDIR/rpmbuild
 
-echo "--------------------------------------------------"
-echo "Build OVS RPM from upstream git $TAG"
-echo
-
 mkdir -p $RPMDIR/RPMS
 mkdir -p $RPMDIR/SOURCES
 mkdir -p $RPMDIR/SPECS
@@ -65,97 +69,25 @@ then
     mkdir -p $TMP_RELEASE_DIR
 fi
 
-# Centos build server should support the following build prerequisites
-
-# yum install gcc make python-devel openssl-devel kernel-devel graphviz \
-# kernel-debug-devel autoconf automake rpm-build redhat-rpm-config \
-# libtool
-
-if [ -d $TMPDIR ]
-then
-    rm -rf $TMPDIR
-fi
-
-mkdir $TMPDIR
-
-cd $TMPDIR
-
-echo "---------------------"
-echo "Clone git repo $TAG"
-echo
-git clone https://github.com/openvswitch/ovs.git
-
-cd ovs
-echo "--------------------"
-echo "Checkout OVS $TAG"
-echo
-if [[ ! "$TAG" =~ "master" ]]; then
-    git checkout $TAG
-fi
-./boot.sh
-./configure
-echo "--------------------"
-echo "Make OVS $TAG"
-echo
-make
 #
-# Get version for master
+# Build ovs rpm with DPDK
 #
-echo "--------------------"
-echo "Get OVS version for $TAG"
+echo =============================================
+echo =======Build ovs rpm with DPDK and test in VM.
 echo
-if [[ "$TAG" =~ "master" ]]; then
-    v=$($TMPDIR/ovs/utilities/ovs-vsctl --version | head -1 | cut -d' ' -f4)
-    export VERSION=$v
-else
-    export VERSION=${TAG:1}
-fi
-
-echo "--------------------"
-echo "OVS version is $VERSION"
-echo
-echo "--------------------"
-echo "Make OVS distribution $TAG"
-echo
-
-make dist
-
-cd $TMPDIR/ovs
-
-cp openvswitch-$VERSION.tar.gz $TOPDIR/rpmbuild/SOURCES
-cp openvswitch-$VERSION.tar.gz $TMPDIR
-
-cd $TMPDIR
-tar -xzf openvswitch-$VERSION.tar.gz
-
-cd $TMPDIR/openvswitch-$VERSION
-
-
-echo "--------------------"
-echo "Build OVS RPM"
-echo
-
-if [ ! -z ${NOCHECK+1} ]; then
-    # Build RPM without checks
-    #
-    rpmbuild -bb --define "_topdir `echo $RPMDIR`" --without check rhel/openvswitch.spec
-else
-    rpmbuild -bb --define "_topdir `echo $RPMDIR`" rhel/openvswitch.spec
-fi
+$BUILD_BASE/BuildAndTestOVS.sh -d -g master -p none -t
+#
+# Build special version of ovs with patches --TODO
+#
 
 # Once build is done copy product to artifactory.
+# and cleanup
 
-echo "---------------------------------------"
-echo "Copy RPM into $TMP_RELEASE_DIR"
-echo
-cp $RPMDIR/RPMS/x86_64/*.rpm $TMP_RELEASE_DIR
-
-# cleanup
 
 echo "---------------------------------------"
-echo "Cleanup $TMP_RELEASE_DIR"
+echo "Cleanup temporary dirs"
 echo
-cd $BUILDDIR
+cd $BUILD_BASE
 
 if [ -d $TMPDIR ]
 then
@@ -166,7 +98,20 @@ fi
 # copy artifacts.
 
 if [[ "$JOB_NAME" =~ "daily" ]]; then
-    upload_artifacts.sh
+    $BUILD_BASE/../ci/upload_artifacts.sh
 fi
 
+if [ -d $TMP_RELEASE_DIR ]; then
+    rm -rf $CACHE_RELEASE_DIR
+fi
+
+if [ -d $RPMDIR ]; then
+    rm -rf $RPMDIR
+fi
+
+# Destroy VM if one has been deployed. Also remove any local installation of
+# DPDK and OVS
+#
+sudo $BUILD_BASE/../ci/clean.sh
+
 exit 0
diff --git a/ci/clean.sh b/ci/clean.sh
new file mode 100755
index 0000000..b1033b0
--- /dev/null
+++ b/ci/clean.sh
@@ -0,0 +1,43 @@
+#!/usr/bin/env bash
+##############################################################################
+# Copyright (c) 2016 Red Hat Inc. and others.
+# therbert@redhat.com
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+##############################################################################
+#Clean script to uninstall provisioning server for Apex
+#author: Dan Radez (dradez@redhat.com)
+#
+vm_index=4
+
+# Clean off instack VM
+virsh destroy instack 2> /dev/null || echo -n ''
+virsh undefine instack --remove-all-storage 2> /dev/null || echo -n ''
+virsh vol-delete instack.qcow2 --pool default 2> /dev/null
+rm -f /var/lib/libvirt/images/instack.qcow2 2> /dev/null
+
+# Clean off baremetal VMs in case they exist
+for i in $(seq 0 $vm_index); do
+  virsh destroy baremetalbrbm_brbm1_$i 2> /dev/null || echo -n ''
+  virsh undefine baremetalbrbm_brbm1_$i --remove-all-storage 2> /dev/null || echo -n ''
+  virsh vol-delete baremetalbrbm_brbm1_${i}.qcow2 --pool default 2> /dev/null
+  rm -f /var/lib/libvirt/images/baremetalbrbm_brbm1_${i}.qcow2 2> /dev/null
+done
+
+# Clean off brbm bridges
+virsh net-destroy brbm 2> /dev/null
+virsh net-undefine brbm 2> /dev/null
+vs-vsctl del-br brbm 2> /dev/null
+
+virsh net-destroy brbm1 2> /dev/null
+virsh net-undefine brbm1 2> /dev/null
+vs-vsctl del-br brbm1 2> /dev/null
+
+# clean pub keys from root's auth keys
+sed -i '/stack@instack.localdomain/d' /root/.ssh/authorized_keys
+sed -i '/virtual-power-key/d' /root/.ssh/authorized_keys
+
+
+echo "Cleanup Completed"
diff --git a/ci/upload_artifacts.sh b/ci/upload_artifacts.sh
index 470773d..45128bd 100755
--- a/ci/upload_artifacts.sh
+++ b/ci/upload_artifacts.sh
@@ -19,21 +19,19 @@ set -o nounset
 set -o pipefail
 
 # log info to console
-echo "Uploading the OVS $VERSION RPM artifacts. "
+echo "Uploading the OVS and DPDK RPM artifacts. "
 echo "-----------------------------------------"
 echo
 
-export RPMFILE_D=openvswitch-debuginfo-$VERSION-1.x86_64.rpm
-export RPMFILE=openvswitch-$VERSION-1.x86_64.rpm
-
-# upload artifact and additional files to google storage
-echo gsutil cp $TMP_RELEASE_DIR/$RPMFILE_D gs://$GS_URL/opnfv-$DATE-$RPMFILE_D
-gsutil cp $TMP_RELEASE_DIR/$RPMFILE_D gs://$GS_URL/opnfv-$DATE-$RPMFILE_D
-
-echo gsutil cp $TMP_RELEASE_DIR/$RPMFILE gs://$GS_URL/opnfv-$DATE-$RPMFILE
-gsutil cp $TMP_RELEASE_DIR/$RPMFILE gs://$GS_URL/opnfv-$DATE-$RPMFILE
+cd $TMP_RELEASE_DIR
+for i in `ls *.rpm`
+do
+    echo copying $i to gs://$GS_URL/ovs4opnfv
+    gsutil cp $TMP_RELEASE_DIR/$i gs://$GS_URL/ovs4opnfv-$i
+    echo
+done
 
 echo
 echo "------------------------------------------------------"
 echo "Done!"
-echo "Artifacts are available as http://$GS_URL/opnfv-$DATE-$RPMFILE"
+echo "Artifacts are available in http://$GS_URL/ovs4opnfv/*.rpm"