nfvbenchvm: add option to use local nfvbench code

build-image.sh has a new option -s to build the generator VM image from
the local nfvbench code (Git working copy) instead of the master branch
of nfvbench repository on OPNFV Gerrit.  This is useful for testing
during the development phases.

When this option is used, the version of the generator VM image is
extended with nfvbench development version number to be able to
distinguish the development images from the latest published image.

Change-Id: If21cbf2be59ff4f4110ebc157af0f934b98bddcb
Signed-off-by: Gwenael Lambrouin <gwenael.lambrouin@orange.com>

3 files changed, 122 insertions, 29 deletions

diff --git a/nfvbenchvm/README.rst b/nfvbenchvm/README.rst
index 29ec91f..0b6fd37 100644
--- a/nfvbenchvm/README.rst
+++ b/nfvbenchvm/README.rst
@@ -4,6 +4,7 @@ NFVBENCH VM IMAGES FOR OPENSTACK
 This repo will build two centos 7 images with:
     - testpmd and VPP installed for loop VM use case
     - NFVbench and TRex installed for generator VM use case
+
 These VMs will come with a pre-canned user/password: nfvbench/nfvbench
 
 BUILD INSTRUCTIONS
@@ -19,7 +20,7 @@ Pre-requisites
     - qemu-img (CentOs) or qemu-utils (Ubuntu)
     - kpartx
 
-.. note:: the image build process is based on `diskimage-builder
+.. note:: The image build process is based on `diskimage-builder
           <https://docs.openstack.org/diskimage-builder/latest/index.html>`_
           that will be installed in a Python virtual environment by nfvbenchvm
           build script build-image.sh.
@@ -35,12 +36,31 @@ Build the image
 - cd dib
 - update the version number for the image (if needed) by modifying __version__ in build-image.sh
 - setup your http_proxy if needed
-- to build loop VM image only:
-    - `bash build-image.sh -l`
-- to build generator VM image only:
-    - `bash build-image.sh -g`
-- to build both images only:
-    - `bash build-image.sh`
+- run ``build-image.sh`` to build the images.  A few examples:
+
+    - to build all the images and publish the code to Google cloud storage:
+        - ``bash build-image.sh``
+    - to build and publish only the loop VM:
+        - ``bash build-image.sh -l``
+    - to build and publish only the generator VM:
+        - ``bash build-image.sh -g``
+    - to build the generator VM without publishing it:
+        - ``bash build-image.sh -gv``
+
+.. note:: Run ``bash build-image.sh`` -h to see all options available.
+
+.. note:: By default, the generator VM image embeds the latest nfvbench version
+          found on the master branch of OPNFV Gerrit repository
+          https://gerrit.opnfv.org/gerrit/nfvbench.
+
+          During development phases, it is also possible to build the image with
+          all the committed changes found in the current working copy of
+          nfvbench (local code).  To do that, run the image build with the ``-s``
+          option, for instance: ``bash build-image.sh -gvs``.
+
+          In that case, the version of the generator VM image will be extended
+          with nfvbench development version number to be able to distinguish the
+          development images from the latest published image.
 
 LOOP VM IMAGE INSTANCE AND CONFIG
 =================================
@@ -245,7 +265,7 @@ To check NFVbench is up and running use REST request:
 
 .. code-block:: bash
 
-curl -XGET '<management_ip>:<port>/status'
+    curl -XGET '<management_ip>:<port>/status'
 
 Example of answer:
 
@@ -263,7 +283,7 @@ To start a test run using NFVbench API use this type of REST request:
 
 .. code-block:: bash
 
-curl -XPOST '<management_ip>:<port>/start_run' -H "Content-Type: application/json" -d @nfvbenchconfig.json
+    curl -XPOST '<management_ip>:<port>/start_run' -H "Content-Type: application/json" -d @nfvbenchconfig.json
 
 Example of return when the submission is successful:
 
@@ -283,7 +303,7 @@ To start a test run using Xtesting python library and NFVbench API use this type
 
 .. code-block:: bash
 
-run_tests -t nfvbench-demo
+    run_tests -t nfvbench-demo
 
 .. note:: `-t` option determine which test case to be runned by Xtesting
  (see `xtesting/testcases.yaml` file content to see available list of test cases)
diff --git a/nfvbenchvm/dib/build-image.sh b/nfvbenchvm/dib/build-image.sh
index 445807b..8ee9bf3 100755
--- a/nfvbenchvm/dib/build-image.sh
+++ b/nfvbenchvm/dib/build-image.sh
@@ -11,8 +11,8 @@ set -euo pipefail
 
 DEBUG=no
 verify_only=0
-generator_only=0
-loopvm_only=0
+build_generator=0
+build_loopvm=0
 __prefix__=""
 
 # Artifact URL
@@ -23,6 +23,9 @@ __version__=0.15
 loopvm_image_name=nfvbenchvm_centos-$__version__
 generator_image_name=nfvbenchvm_centos-generator-$__version__
 
+# Default values for nfvbenchvm dib element variables
+export DIB_NFVBENCH_CODE_ORIGIN=opnfv-gerrit
+
 
 # ----------------------------------------------------------------------------
 # Parse command line options and configure the script
@@ -38,6 +41,8 @@ OPTIONS
     -l: build NFVbench loop VM image
     -g: build NFVbench generator image
     -v: verify only (build but do not push to google storage)
+    -s: use local nfvbench code instead of cloning from OPNFV gerrit
+        (only relevant for NFVbench generator image)
 
     -t: enable debug trace (set -x + DIB_DEBUG_TRACE=1)
     -d: start debug shell in image chroot in case of build error
@@ -46,17 +51,20 @@ EOF
     exit 1
 }
 
-while getopts ":lgvtdh" opt; do
+while getopts ":lgvstdh" opt; do
     case $opt in
         l)
-            loopvm_only=1
+            build_loopvm=1
             ;;
         g)
-            generator_only=1
+            build_generator=1
             ;;
         v)
             verify_only=1
             ;;
+        s)
+            export DIB_NFVBENCH_CODE_ORIGIN=static
+            ;;
         t)
             set -x
             export DIB_DEBUG_TRACE=1
@@ -76,6 +84,42 @@ while getopts ":lgvtdh" opt; do
 done
 
 
+# Build all VM images if the image to build is not specified on the CLI
+if [[ $build_generator -eq 0 ]] && [[ $build_loopvm -eq 0 ]]; then
+    build_generator=1
+    build_loopvm=1
+fi
+
+if [[ "${DIB_NFVBENCH_CODE_ORIGIN}" == "static" ]] && [[ $build_generator -eq 0 ]]; then
+    echo "Error: option -s is only relevant to the build of the generator image"
+    exit 1
+fi
+
+
+# ----------------------------------------------------------------------------
+# Copy local nfvbench code to elements/nfvbenchvm/static/opt/nfvbench
+# ----------------------------------------------------------------------------
+
+function copy_local_nfvbench_code_to_static_dir {
+    echo "Copy local nfvbench code to elements/nfvbenchvm/static/opt"
+    # Create elements/nfvbenchvm/static/opt/ directory if it does not exist and
+    # move there
+    pushd $(dirname $0)/elements/nfvbenchvm/static
+    [ -d opt ] || mkdir opt
+    cd opt
+
+    # Remove nfvbench code if it is already there
+    [ -d nfvbench ] && rm -rf nfvbench
+
+    # Use git to "copy" the local nfvbench code.
+    # This will include all the committed changes of the current branch.
+    git clone ../../../../../.. nfvbench
+
+    # Go back to the current directory when this function was called
+    popd
+}
+
+
 # ----------------------------------------------------------------------------
 # Configure and start the nfvbenchvm image build
 # ----------------------------------------------------------------------------
@@ -172,18 +216,31 @@ function build_image {
 # Main program
 # ----------------------------------------------------------------------------
 
-if [ ! $generator_only -eq 1 ] && [ ! $loopvm_only -eq 1 ]; then
-   echo "Build loop VM image"
-   build_image $loopvm_image_name
-   echo "Build generator image"
-   build_image $generator_image_name
-else
-    if [ $loopvm_only -eq 1 ]; then
-       echo "Build loop VM image"
-       build_image $loopvm_image_name
-    fi
-    if [ $generator_only -eq 1 ]; then
-       echo "Build generator image"
-       build_image $generator_image_name
+if [ $build_loopvm -eq 1 ]; then
+    echo "Build loop VM image"
+    build_image $loopvm_image_name
+fi
+
+if [ $build_generator -eq 1 ]; then
+    echo "Build generator image"
+
+    if [[ "${DIB_NFVBENCH_CODE_ORIGIN}" == "static" ]]; then
+        echo "Use local nfvbench code"
+        copy_local_nfvbench_code_to_static_dir
+
+        # Append nfvbench version number to the image name:
+        # during development, this is useful to distinguish the development
+        # images from the latest published image.
+        #
+        # To avoid confusion, we use the same versioning as nfvbench (see
+        # nfvbench/__init__.py), although "git describe" would give us a better
+        # number with respect to uniqueness.  So we will typically get something
+        # like "5.0.4.dev31" where "5.0.4" is the latest annotated tag ("5.0.3")
+        # plus one and where dev31 indicates the number of commits (31) since
+        # that tag.
+        nfvbench_version=$(python -c 'import pbr.version; print(pbr.version.VersionInfo("nfvbench").version_string_with_vcs())')
+        generator_image_name="${generator_image_name}-${nfvbench_version}"
     fi
+
+    build_image $generator_image_name
 fi
diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/52-nfvbench-script b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/52-nfvbench-script
index 2182c83..37af915 100755
--- a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/52-nfvbench-script
+++ b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/52-nfvbench-script
@@ -24,7 +24,23 @@ rm -rf /opt/trex/$TREX_VER/automation/trex_control_plane/interactive/trex
 
 # NFVbench installation
 cd /opt
-git clone https://gerrit.opnfv.org/gerrit/nfvbench
+if [[ "${DIB_NFVBENCH_CODE_ORIGIN}" == "static" ]]; then
+    # nfvbench code has been copied by the install-static element to
+    # /opt/nfvbench without the .git/ directory.  But pip will need that .git/
+    # directory to compute nfvbench version, so will now finish the incomplete
+    # job of install-static:
+    STATIC_NFVBENCH_CODE="$(dirname $0)/../static/opt/nfvbench"
+    [ -d "${STATIC_NFVBENCH_CODE}" ] || {
+        echo "Error: directory ${STATIC_NFVBENCH_CODE} missing."
+        echo "    You requested a build from local nfvbench code with"
+        echo "    DIB_NFVBENCH_CODE_ORIGIN=static, but you likely forgot"
+        echo "    to clone nfvbench code in elements/nfvbenchvm/static/opt/nfvbench"
+        exit 1
+    }
+    rsync -lr "${STATIC_NFVBENCH_CODE}"/ /opt/nfvbench/
+else
+    git clone https://gerrit.opnfv.org/gerrit/nfvbench
+fi
 cd nfvbench/
 python3 -m pip install . --use-deprecated=legacy-resolver
 cp xtesting/testcases.yaml /usr/local/lib/python3.6/site-packages/xtesting/ci/testcases.yaml