diff options
Diffstat (limited to 'nfvbenchvm')
17 files changed, 326 insertions, 382 deletions
diff --git a/nfvbenchvm/README.rst b/nfvbenchvm/README.rst deleted file mode 100644 index dad9f46..0000000 --- a/nfvbenchvm/README.rst +++ /dev/null @@ -1,316 +0,0 @@ -NFVBENCH VM IMAGES FOR OPENSTACK -++++++++++++++++++++++++++++++++ - -This repo will build two centos 7 images with: - - testpmd and VPP installed for loop VM use case - - NFVbench and TRex installed for generator VM use case -These VMs will come with a pre-canned user/password: nfvbench/nfvbench - -BUILD INSTRUCTIONS -================== - -Pre-requisites --------------- -- must run on Linux -- the following packages must be installed prior to using this script: - - git - - qemu-utils - - kpartx - -Build the image ---------------- -- cd dib -- update the version number for the image (if needed) by modifying __version__ in build-image.sh -- setup your http_proxy if needed -- to build loop VM image only: - - `bash build-image.sh -l` -- to build generator VM image only: - - `bash build-image.sh -g` -- to build both images only: - - `bash build-image.sh` - -LOOP VM IMAGE INSTANCE AND CONFIG -================================= - -Interface Requirements ----------------------- -The instance must be launched using OpenStack with 2 network interfaces. -For best performance, it should use a flavor with: - -- 2 vCPU -- 4 GB RAM -- cpu pinning set to exclusive - -Auto-configuration ------------------- -nfvbench VM will automatically find the two virtual interfaces to use, and use the forwarder specifed in the config file. - -In the case testpmd is used, testpmd will be launched with mac forwarding mode where the destination macs rewritten according to the config file. - -In the case VPP is used, VPP will set up a L3 router, and forwarding traffic from one port to the other. - -nfvbenchvm Config ------------------ -nfvbenchvm config file is located at ``/etc/nfvbenchvm.conf``. - -.. code-block:: bash - - FORWARDER=testpmd - INTF_MAC1=FA:16:3E:A2:30:41 - INTF_MAC2=FA:16:3E:10:DA:10 - TG_MAC1=00:10:94:00:0A:00 - TG_MAC2=00:11:94:00:0A:00 - VNF_GATEWAY1_CIDR=1.1.0.2/8 - VNF_GATEWAY2_CIDR=2.2.0.2/8 - TG_NET1=10.0.0.0/8 - TG_NET2=20.0.0.0/8 - TG_GATEWAY1_IP=1.1.0.100 - TG_GATEWAY2_IP=2.2.0.100 - - -Launching nfvbenchvm VM ------------------------ - -Normally this image will be used together with NFVBench, and the required configurations will be automatically generated and pushed to VM by NFVBench. If launched manually, no forwarder will be run. Users will have the full control to run either testpmd or VPP via VNC console. - -To check if testpmd is running, you can run this command in VNC console: - -.. code-block:: bash - - sudo screen -r testpmd - -To check if VPP is running, you can run this command in VNC console: - -.. code-block:: bash - - service vpp status - - -Hardcoded Username and Password --------------------------------- -- Username: nfvbench -- Password: nfvbench - - -GENERATOR IMAGE INSTANCE AND CONFIG -=================================== - -Pre-requisites --------------- -To use openstack APIs, NFVbench generator VM will use `clouds.yaml` file as openstack configuration. -The OpenStack clouds configuration from clouds.yaml file to use. -clouds.yaml file must be in one of the following paths: -- ~/.config/openstack -- /etc/openstack - -Example of `clouds.yaml`: - -.. code-block:: yaml - - clouds: - devstack: - auth: - auth_url: http://192.168.122.10:35357/ - project_name: demo - username: demo - password: 0penstack - region_name: RegionOne - -.. note:: Add `CLOUD_DETAIL` property with the accurate value for your openstack configuration (`devstack` in the above example) in ``/etc/nfvbenchvm.conf`` - -Interface Requirements ----------------------- -The instance must be launched using OpenStack with 2 network interfaces for dataplane traffic (using SR-IOV function) and 1 management interface to control nfvbench. -For best performance, it should use network interfaces for dataplane traffic with a `vnic_type` to `direct-physical` (or `direct` if physical function is not possible) -and a flavor with: - -- 6 vCPU -- 8 GB RAM -- cpu pinning set to exclusive - -.. note:: For the management interface: any interface type can be used. This interface required a routable IP (through floating IP or direct) and an access to the openstack APIs. -.. note:: CPU pinning: 1 core dedicated for guest OS and NFVbench process, other provided cores are used by TRex - -Template of a genarator profile using CPU pinning: - -.. code-block:: bash - - generator_profile: - - name: {{name}} - tool: {{tool}} - ip: {{ip}} - zmq_pub_port: {{zmq_pub_port}} - zmq_rpc_port: {{zmq_rpc_port}} - software_mode: {{software_mode}} - cores: {{CORES}} - platform: - master_thread_id: '0' - latency_thread_id: '1' - dual_if: - - socket: 0 - threads: [{{CORE_THREADS}}] - - interfaces: - - port: 0 - pci: "{{PCI_ADDRESS_1}}" - switch: - - port: 1 - pci: "{{PCI_ADDRESS_2}}" - switch: - intf_speed: - -.. note:: `CORE_THREADS` value is determined automatically based on the cores available on the VM starting from 2 to last worker core available. - -Auto-configuration ------------------- -nfvbench VM will automatically find the two virtual interfaces to use for dataplane based on MAC addresses or openstack port name (see config part below). -This applies to the management interface as well. - -nfvbenchvm Config ------------------ -nfvbenchvm config file is located at ``/etc/nfvbenchvm.conf``. - -Example of configuration: - -.. code-block:: bash - - ACTION=e2e - LOOPBACK_INTF_MAC1=FA:16:3E:A2:30:41 - LOOPBACK_INTF_MAC2=FA:16:3E:10:DA:10 - E2E_INTF_MAC1=FA:16:3E:B0:E2:43 - E2E_INTF_MAC2=FA:16:3E:D3:6A:FC - -.. note:: `ACTION` parameter is not mandatory but will permit to start NFVbench with the accurate ports (loopback or e2e). -.. note:: Set of MAC parameters cannot be used in parallel as only one NFVbench/TRex process is running. -.. note:: Switching from `loopback` to `e2e` action can be done manually using `/nfvbench/start-nfvbench.sh <action>` with the accurate keyword for `action` parameter. This script will restart NFVbench with the good set of MAC. - -nfvbenchvm config file with management interface: - -.. code-block:: bash - - ACTION=e2e - LOOPBACK_INTF_MAC1=FA:16:3E:A2:30:41 - LOOPBACK_INTF_MAC2=FA:16:3E:10:DA:10 - INTF_MAC_MGMT=FA:16:3E:06:11:8A - INTF_MGMT_CIDR=172.20.56.228/2 - INTF_MGMT_IP_GW=172.20.56.225 - -.. note:: `INTF_MGMT_IP_GW` and `INTF_MGMT_CIDR` parameters are used by the VM to automatically configure virtual interface and route to allow an external access through SSH. - - -Using pre-created direct-physical ports on openstack, mac addresses value are only known when VM is deployed. In this case, you can pass the port name in config: - -.. code-block:: bash - - LOOPBACK_PORT_NAME1=nfvbench-pf1 - LOOPBACK_PORT_NAME2=nfvbench-pf2 - E2E_PORT_NAME1=nfvbench-pf1 - E2E_PORT_NAME1=nfvbench-pf3 - INTF_MAC_MGMT=FA:16:3E:06:11:8A - INTF_MGMT_CIDR=172.20.56.228/2 - INTF_MGMT_IP_GW=172.20.56.225 - DNS_SERVERS=8.8.8.8,dns.server.com - -.. note:: A management interface is required to automatically find the virtual interface to use according to the MAC address provided (see `INTF_MAC_MGMT` parameter). -.. note:: NFVbench VM will call openstack API through the management interface to retrieve mac address for these ports -.. note:: If openstack API required a host name resolution, add the parameter DNS_SERVERS to add IP or DNS server names (multiple servers can be added separated by a `,`) - -Control nfvbenchvm VM and run test ----------------------------------- - -By default, NFVbench will be started in server mode (`--server`) and will act as an API. - -NFVbench VM will be accessible through SSH or HTTP using the management interface IP. - -NFVbench API endpoint is : `http://<management_ip>:<port>` -.. note:: by default port value is 7555 - -Get NFVbench status -^^^^^^^^^^^^^^^^^^^ - -To check NFVbench is up and running use REST request: - -.. code-block:: bash - -curl -XGET '<management_ip>:<port>/status' - -Example of answer: - -.. code-block:: bash - - { - "error_message": "nfvbench run still pending", - "status": "PENDING" - } - -Start NFVbench test -^^^^^^^^^^^^^^^^^^^ - -To start a test run using NFVbench API use this type of REST request: - -.. code-block:: bash - -curl -XPOST '<management_ip>:<port>/start_run' -H "Content-Type: application/json" -d @nfvbenchconfig.json - -Example of return when the submission is successful: - -.. code-block:: bash - - { - "error_message": "NFVbench run still pending", - "request_id": "42cccb7effdc43caa47f722f0ca8ec96", - "status": "PENDING" - } - - -Start NFVbench test using Xtesting -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To start a test run using Xtesting python library and NFVbench API use this type of command on the VM: - -.. code-block:: bash - -run_tests -t nfvbench-demo - -.. note:: `-t` option determine which test case to be runned by Xtesting - (see `xtesting/testcases.yaml` file content to see available list of test cases) - - -Connect to the VM using SSH keypair -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If a key is provided at VM creation you can use it to log on the VM using `cloud-user` username: - -.. code-block:: bash - - ssh -i key.pem cloud-user@<management_ip> - - -Connect to VM using SSH username/password -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -VM is accessible over SSH using the hardcoded username and password (see below): - -.. code-block:: bash - - ssh nfvbench@<management_ip> - - -Launching nfvbenchvm VM ------------------------ - -Normally this image will be deployed using Ansible role, and the required configurations will be automatically generated and pushed to VM by Ansible. -If launched manually, users will have the full control to configure and run NFVbench via VNC console. - -To check if NFVbench is running, you can run this command in VNC console: - -.. code-block:: bash - - sudo screen -r nfvbench - - -Hardcoded Username and Password --------------------------------- -- Username: nfvbench -- Password: nfvbench - diff --git a/nfvbenchvm/dib/build-image.sh b/nfvbenchvm/dib/build-image.sh index 6339f01..c320192 100755 --- a/nfvbenchvm/dib/build-image.sh +++ b/nfvbenchvm/dib/build-image.sh @@ -3,39 +3,80 @@ # A shell script to build the VPP VM image or NFVbench+TRex VM image using diskinage-builder # # The following packages must be installed prior to using this script: -# sudo apt-get -y install python-virtualenv qemu-utils kpartx +# Ubuntu: sudo apt-get -y install python3 python3-venv qemu-utils kpartx +# CentOS: sudo yum install -y python3 qemu-img kpartx -usage() { - echo "Usage: $0 [-l] [-g] [-v]" - echo " -l build NFVbench loop VM image" - echo " -g build NFVbench generator image" - echo " -v verify only (build but do not push to google storage)" - exit 1 -} +# Stop on error (see https://wizardzines.com/comics/bash-errors/) +set -euo pipefail +DEBUG=no verify_only=0 -generator_only=0 -loopvm_only=0 +build_generator=0 +build_loopvm=0 __prefix__="" + +# Artifact URL +gs_url=artifacts.opnfv.org/nfvbench/images + +# image version number +__loopvm_version__=0.16 +__generator_version__=0.15 +loopvm_image_name=nfvbenchvm_centos-$__loopvm_version__ +generator_image_name=nfvbenchvm_centos-generator-$__generator_version__ + +# Default values for nfvbenchvm dib element variables +export DIB_NFVBENCH_CODE_ORIGIN=opnfv-gerrit + + # ---------------------------------------------------------------------------- # Parse command line options and configure the script # ---------------------------------------------------------------------------- -while getopts ":hglv" opt; do +usage() { + cat <<EOF +$(basename $0) - build NFVbench VM images +Usage: + $(basename $0) [OPTIONS] + +OPTIONS + -l: build NFVbench loop VM image + -g: build NFVbench generator image + -v: verify only (build but do not push to google storage) + -s: use local nfvbench code instead of cloning from OPNFV gerrit + (only relevant for NFVbench generator image) + + -t: enable debug trace (set -x + DIB_DEBUG_TRACE=1) + -d: start debug shell in image chroot in case of build error + -h: show this help message +EOF + exit 1 +} + +while getopts ":lgvstdh" opt; do case $opt in - h) - usage - exit 0 + l) + build_loopvm=1 ;; g) - generator_only=1 - ;; - l) - loopvm_only=1 + build_generator=1 ;; v) verify_only=1 ;; + s) + export DIB_NFVBENCH_CODE_ORIGIN=static + ;; + t) + set -x + export DIB_DEBUG_TRACE=1 + ;; + d) + DEBUG=yes + ;; + h) + usage + exit 0 + ;; ?) usage exit 1 @@ -43,22 +84,53 @@ while getopts ":hglv" opt; do esac done -set -e -# Artifact URL -gs_url=artifacts.opnfv.org/nfvbench/images +# Build all VM images if the image to build is not specified on the CLI +if [[ $build_generator -eq 0 ]] && [[ $build_loopvm -eq 0 ]]; then + build_generator=1 + build_loopvm=1 +fi -# image version number -__version__=0.15 -loopvm_image_name=nfvbenchvm_centos-$__version__ -generator_image_name=nfvbenchvm_centos-generator-$__version__ +if [[ "${DIB_NFVBENCH_CODE_ORIGIN}" == "static" ]] && [[ $build_generator -eq 0 ]]; then + echo "Error: option -s is only relevant to the build of the generator image" + exit 1 +fi + + +# ---------------------------------------------------------------------------- +# Copy local nfvbench code to elements/nfvbenchvm/static/opt/nfvbench +# ---------------------------------------------------------------------------- + +function copy_local_nfvbench_code_to_static_dir { + echo "Copy local nfvbench code to elements/nfvbenchvm/static/opt" + # Create elements/nfvbenchvm/static/opt/ directory if it does not exist and + # move there + pushd $(dirname $0)/elements/nfvbenchvm/static + [ -d opt ] || mkdir opt + cd opt + + # Remove nfvbench code if it is already there + [ -d nfvbench ] && rm -rf nfvbench + + # Use git to "copy" the local nfvbench code. + # This will include all the committed changes of the current branch. + git clone ../../../../../.. nfvbench + + # Go back to the current directory when this function was called + popd +} + + +# ---------------------------------------------------------------------------- +# Configure and start the nfvbenchvm image build +# ---------------------------------------------------------------------------- function build_image { # if image exists skip building echo "Checking if image exists in google storage..." if command -v gsutil >/dev/null; then if gsutil -q stat gs://$gs_url/$1.qcow2; then - echo "Image already exists at http://$gs_url/$1.qcow2" + echo "Image already exists at https://$gs_url/$1.qcow2" echo "Build is skipped" exit 0 fi @@ -72,14 +144,13 @@ function build_image { if [ -f $1.qcow2 ]; then echo "Image $1.qcow2 already exists locally" else - # install diskimage-builder if [ -d dib-venv ]; then . dib-venv/bin/activate else - virtualenv dib-venv + python3 -m venv dib-venv . dib-venv/bin/activate - pip install diskimage-builder + pip install diskimage-builder==3.16.0 fi # Add nfvbenchvm_centos elements directory to the DIB elements path @@ -110,8 +181,18 @@ function build_image { export TREX_VER=$(awk '/ENV TREX_VER/ {print $3}' ../../docker/Dockerfile | sed 's/"//g' | sed 's/\r//g') fi + # Specify CentOS version + export DIB_RELEASE=7 + + # Debug on error: if an error occurs during the build, disk-image-create + # will drop us in a Bash inside the chroot, and we will be able to inspect + # the current state of the image. + if [[ "${DEBUG}" == "yes" ]]; then + export break=after-error + fi + echo "Building $1.qcow2..." - time disk-image-create -o $1 centos7 nfvbenchvm + time disk-image-create -o $1 centos nfvbenchvm fi ls -l $1.qcow2 @@ -123,7 +204,7 @@ function build_image { if command -v gsutil >/dev/null; then echo "Uploading $1.qcow2..." gsutil cp $1.qcow2 gs://$gs_url/$1.qcow2 - echo "You can access to image at http://$gs_url/$1.qcow2" + echo "You can access to image at https://$gs_url/$1.qcow2" else echo "Cannot upload new image to the OPNFV artifact repository (gsutil not available)" exit 1 @@ -132,18 +213,35 @@ function build_image { } -if [ ! $generator_only -eq 1 ] && [ ! $loopvm_only -eq 1 ]; then - echo "Build loop VM image" - build_image $loopvm_image_name - echo "Build generator image" - build_image $generator_image_name -else - if [ $loopvm_only -eq 1 ]; then - echo "Build loop VM image" - build_image $loopvm_image_name - fi - if [ $generator_only -eq 1 ]; then - echo "Build generator image" - build_image $generator_image_name +# ---------------------------------------------------------------------------- +# Main program +# ---------------------------------------------------------------------------- + +if [ $build_loopvm -eq 1 ]; then + echo "Build loop VM image" + build_image $loopvm_image_name +fi + +if [ $build_generator -eq 1 ]; then + echo "Build generator image" + + if [[ "${DIB_NFVBENCH_CODE_ORIGIN}" == "static" ]]; then + echo "Use local nfvbench code" + copy_local_nfvbench_code_to_static_dir + + # Append nfvbench version number to the image name: + # during development, this is useful to distinguish the development + # images from the latest published image. + # + # To avoid confusion, we use the same versioning as nfvbench (see + # nfvbench/__init__.py), although "git describe" would give us a better + # number with respect to uniqueness. So we will typically get something + # like "5.0.4.dev31" where "5.0.4" is the latest annotated tag ("5.0.3") + # plus one and where dev31 indicates the number of commits (31) since + # that tag. + nfvbench_version=$(python -c 'import pbr.version; print(pbr.version.VersionInfo("nfvbench").version_string_with_vcs())') + generator_image_name="${generator_image_name}-${nfvbench_version}" fi -fi
\ No newline at end of file + + build_image $generator_image_name +fi diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/element-deps b/nfvbenchvm/dib/elements/nfvbenchvm/element-deps index c6be0aa..7fd8e5b 100644 --- a/nfvbenchvm/dib/elements/nfvbenchvm/element-deps +++ b/nfvbenchvm/dib/elements/nfvbenchvm/element-deps @@ -1,4 +1,5 @@ vm +block-device-mbr cloud-init-datasources install-static package-installs diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/fdio-release.repo b/nfvbenchvm/dib/elements/nfvbenchvm/fdio-release.repo index 3ad12fb..6b980c1 100644 --- a/nfvbenchvm/dib/elements/nfvbenchvm/fdio-release.repo +++ b/nfvbenchvm/dib/elements/nfvbenchvm/fdio-release.repo @@ -1,5 +1,11 @@ [fdio-release] -name=fd.io release branch latest merge -baseurl=https://nexus.fd.io/content/repositories/fd.io.stable.1710.centos7/ -enabled=1 +name=FD.io release 17.10 binary RPM package repository for CentOS 7 +baseurl=https://packagecloud.io/fdio/1710/el/7/$basearch +repo_gpgcheck=1 gpgcheck=0 +enabled=1 +gpgkey=https://packagecloud.io/fdio/1710/gpgkey +sslverify=1 +sslcacert=/etc/pki/tls/certs/ca-bundle.crt +metadata_expire=300 + diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/finalise.d/51-add-cpu-isolation b/nfvbenchvm/dib/elements/nfvbenchvm/finalise.d/51-add-cpu-isolation index 12ffcdc..e9cf3c5 100755 --- a/nfvbenchvm/dib/elements/nfvbenchvm/finalise.d/51-add-cpu-isolation +++ b/nfvbenchvm/dib/elements/nfvbenchvm/finalise.d/51-add-cpu-isolation @@ -1,5 +1,12 @@ #!/bin/bash +if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then + set -x +fi + +# Stop on error +set -euo pipefail + grubby --update-kernel=ALL --args="isolcpus=1-7 rcu_nocbs=1 nohz=on nohz_full=1 nmi_watchdog=0" grubby --update-kernel=ALL --args="default_hugepagesz=1G hugepagesz=1G hugepages=4" grubby --update-kernel=ALL --args="intel_iommu=on iommu=pt" diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/finalise.d/52-change-resolution b/nfvbenchvm/dib/elements/nfvbenchvm/finalise.d/52-change-resolution index 26f2f6a..d01b907 100755 --- a/nfvbenchvm/dib/elements/nfvbenchvm/finalise.d/52-change-resolution +++ b/nfvbenchvm/dib/elements/nfvbenchvm/finalise.d/52-change-resolution @@ -1,3 +1,10 @@ #!/bin/bash +if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then + set -x +fi + +# Stop on error +set -euo pipefail + grubby --update-kernel=ALL --args="vga=792" diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/finalise.d/53-boot-from-new-kernel b/nfvbenchvm/dib/elements/nfvbenchvm/finalise.d/53-boot-from-new-kernel index 6607b0b..e461801 100755 --- a/nfvbenchvm/dib/elements/nfvbenchvm/finalise.d/53-boot-from-new-kernel +++ b/nfvbenchvm/dib/elements/nfvbenchvm/finalise.d/53-boot-from-new-kernel @@ -1,3 +1,10 @@ #!/bin/bash +if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then + set -x +fi + +# Stop on error +set -euo pipefail + grubby --set-default-index=0 diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/package-installs.yaml b/nfvbenchvm/dib/elements/nfvbenchvm/package-installs.yaml index 60efec1..6f86027 100644 --- a/nfvbenchvm/dib/elements/nfvbenchvm/package-installs.yaml +++ b/nfvbenchvm/dib/elements/nfvbenchvm/package-installs.yaml @@ -17,4 +17,5 @@ openssh-server: dpdk-tools: git: python3-dev: -libpython3.6-dev:
\ No newline at end of file +libpython3.6-dev: +python3-pip: diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/01-update-kernel b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/01-update-kernel index 8094006..0c32b5f 100755 --- a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/01-update-kernel +++ b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/01-update-kernel @@ -1,5 +1,12 @@ #!/bin/bash +if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then + set -x +fi + +# Stop on error +set -euo pipefail + if [ $DIB_USE_ELREPO_KERNEL != "True" ]; then exit 0 fi diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/02-pip-package b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/02-pip-package index 9707841..ee7c5f2 100755 --- a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/02-pip-package +++ b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/02-pip-package @@ -1,8 +1,12 @@ #!/bin/bash -cd /tmp -wget https://bootstrap.pypa.io/get-pip.py -python3 get-pip.py +if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then + set -x +fi -pip3 install setuptools wheel pbr -pip3 install pyyaml +# Stop on error +set -euo pipefail + +python3 -m pip install --upgrade pip +python3 -m pip install setuptools wheel pbr +python3 -m pip install pyyaml diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/03-copy-rc-local b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/03-copy-rc-local index 3311530..4a8e66b 100755 --- a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/03-copy-rc-local +++ b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/03-copy-rc-local @@ -1,8 +1,15 @@ #!/bin/bash +if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then + set -x +fi + +# Stop on error +set -euo pipefail + # set accurate rc.local file corresponding to current image built if [ $DIB_DEV_IMAGE = "loopvm" ]; then mv /etc/rc.d/rc.local.loopvm /etc/rc.d/rc.local else mv /etc/rc.d/rc.local.generator /etc/rc.d/rc.local -fi
\ No newline at end of file +fi diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/04-add-execute-attribute b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/04-add-execute-attribute index 666d9dc..52aed8a 100755 --- a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/04-add-execute-attribute +++ b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/04-add-execute-attribute @@ -1,5 +1,12 @@ #!/bin/bash +if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then + set -x +fi + +# Stop on error +set -euo pipefail + chmod +x /etc/rc.d/rc.local chmod +x /etc/sysconfig/network-scripts/ifcfg-eth0 chmod +x /etc/profile.d/nfvbench.sh diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/52-nfvbench-script b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/52-nfvbench-script index 9162952..4edc493 100755 --- a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/52-nfvbench-script +++ b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/52-nfvbench-script @@ -1,5 +1,12 @@ #!/bin/bash +if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then + set -x +fi + +# Make sure the disk image build fails if nfvbench installation fails +set -euo pipefail + if [ $DIB_DEV_IMAGE != "generator" ]; then exit 0 fi @@ -17,15 +24,33 @@ rm -rf /opt/trex/$TREX_VER/automation/trex_control_plane/interactive/trex # NFVbench installation cd /opt -git clone https://gerrit.opnfv.org/gerrit/nfvbench +if [[ "${DIB_NFVBENCH_CODE_ORIGIN}" == "static" ]]; then + # nfvbench code has been copied by the install-static element to + # /opt/nfvbench without the .git/ directory. But pip will need that .git/ + # directory to compute nfvbench version, so will now finish the incomplete + # job of install-static: + STATIC_NFVBENCH_CODE="$(dirname $0)/../static/opt/nfvbench" + [ -d "${STATIC_NFVBENCH_CODE}" ] || { + echo "Error: directory ${STATIC_NFVBENCH_CODE} missing." + echo " You requested a build from local nfvbench code with" + echo " DIB_NFVBENCH_CODE_ORIGIN=static, but you likely forgot" + echo " to clone nfvbench code in elements/nfvbenchvm/static/opt/nfvbench" + exit 1 + } + rsync -lr "${STATIC_NFVBENCH_CODE}"/ /opt/nfvbench/ +else + git clone https://gerrit.opnfv.org/gerrit/nfvbench +fi cd nfvbench/ -pip3 install . --use-deprecated=legacy-resolver +python3 -m pip install . --use-deprecated=legacy-resolver cp xtesting/testcases.yaml /usr/local/lib/python3.6/site-packages/xtesting/ci/testcases.yaml python3 ./docker/cleanup_generators.py rm -rf /opt/nfvbench/.git rm -rf /opt/nfvbench/nfvbench +rm -rf /opt/nfvbench/behave_tests # symlink to NFVbench sources ln -s /usr/local/lib/python3.6/site-packages/nfvbench /opt/nfvbench/nfvbench +ln -s /usr/local/lib/python3.6/site-packages/behave_tests /opt/nfvbench/behave_tests # persist env variables echo "export TREX_VER=\"$TREX_VER\"" >> /etc/profile.d/nfvbench.sh echo "export TREX_EXT_LIBS=\"/opt/trex/$TREX_VER/external_libs\"" >> /etc/profile.d/nfvbench.sh diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/53-sshd-script b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/53-sshd-script index 64e8877..0c7dfd5 100755 --- a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/53-sshd-script +++ b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/53-sshd-script @@ -1,4 +1,11 @@ #!/bin/bash +if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then + set -x +fi + +# Stop on error +set -euo pipefail + # Set UseDNS no value in sshd_config to reduce time to connect -echo "UseDNS no" >> /etc/ssh/sshd_config
\ No newline at end of file +echo "UseDNS no" >> /etc/ssh/sshd_config diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/99-cleanup b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/99-cleanup index e48ca52..dbb7342 100755 --- a/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/99-cleanup +++ b/nfvbenchvm/dib/elements/nfvbenchvm/post-install.d/99-cleanup @@ -1,3 +1,10 @@ #!/bin/bash +if [ ${DIB_DEBUG_TRACE:-1} -gt 0 ]; then + set -x +fi + +# Stop on error +set -euo pipefail + yum erase -y python-devel libyaml-devel numactl-devel kernel-devel kernel-headers kernel-lt-headers kernel-lt-devel gcc git python3-dev libpython3.6-dev diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/static/etc/rc.d/rc.local.generator b/nfvbenchvm/dib/elements/nfvbenchvm/static/etc/rc.d/rc.local.generator index 633403c..9ac23a3 100644 --- a/nfvbenchvm/dib/elements/nfvbenchvm/static/etc/rc.d/rc.local.generator +++ b/nfvbenchvm/dib/elements/nfvbenchvm/static/etc/rc.d/rc.local.generator @@ -86,6 +86,24 @@ if [ $INTF_MGMT_CIDR ] && [ $INTF_MGMT_IP_GW ]; then logger "NFVBENCHVM ERROR: Cannot find eth port for management port" return 1 fi + + # By default, configure the MTU of the management interface to the + # conservative value of 1500: this will reduce the risk to get an + # unmanageable VM in some setups. + # + # To set the MTU to a different value, configure the INTF_MGMT_MTU variable + # in /etc/nfvbenchvm.conf. If INTF_MGMT_MTU is set to the special value + # "auto", the MTU will not be configured and it will keep the value set by + # the hypervisor ("legacy" nfvbenchvm behavior). If INTF_MGMT_MTU is unset, + # the MTU will be set to 1500. In other cases, the MTU will be set to the + # value of INTF_MGMT_MTU. + # + if [[ -z "$INTF_MGMT_MTU" ]]; then + ip link set $ETH_PORT mtu 1500 + elif [[ "$INTF_MGMT_MTU" != "auto" ]]; then + ip link set $ETH_PORT mtu $INTF_MGMT_MTU + fi + ip addr add $INTF_MGMT_CIDR dev $ETH_PORT ip link set $ETH_PORT up ip route add default via $INTF_MGMT_IP_GW dev $ETH_PORT @@ -101,4 +119,4 @@ else /nfvbench/start-nfvbench.sh fi -exit 0
\ No newline at end of file +exit 0 diff --git a/nfvbenchvm/dib/elements/nfvbenchvm/static/etc/rc.d/rc.local.loopvm b/nfvbenchvm/dib/elements/nfvbenchvm/static/etc/rc.d/rc.local.loopvm index ecc8b05..181ff2a 100644 --- a/nfvbenchvm/dib/elements/nfvbenchvm/static/etc/rc.d/rc.local.loopvm +++ b/nfvbenchvm/dib/elements/nfvbenchvm/static/etc/rc.d/rc.local.loopvm @@ -137,6 +137,24 @@ if [ $INTF_MGMT_CIDR ] && [ $INTF_MGMT_IP_GW ]; then else ETH_PORT="eth0" fi + + # By default, configure the MTU of the management interface to the + # conservative value of 1500: this will reduce the risk to get an + # unmanageable VM in some setups. + # + # To set the MTU to a different value, configure the INTF_MGMT_MTU variable + # in /etc/nfvbenchvm.conf. If INTF_MGMT_MTU is set to the special value + # "auto", the MTU will not be configured and it will keep the value set by + # the hypervisor ("legacy" nfvbenchvm behavior). If INTF_MGMT_MTU is unset, + # the MTU will be set to 1500. In other cases, the MTU will be set to the + # value of INTF_MGMT_MTU. + # + if [[ -z "$INTF_MGMT_MTU" ]]; then + ip link set $ETH_PORT mtu 1500 + elif [[ "$INTF_MGMT_MTU" != "auto" ]]; then + ip link set $ETH_PORT mtu $INTF_MGMT_MTU + fi + ip addr add $INTF_MGMT_CIDR dev $ETH_PORT ip link set $ETH_PORT up ip route add default via $INTF_MGMT_IP_GW dev $ETH_PORT @@ -164,6 +182,29 @@ else logger "NFVBENCHVM ERROR: VM MAC Addresses missing in $NFVBENCH_CONF" fi +wait_vpp_service() { + # Wait for at most wait_max=$1 seconds until VPP service is ready. Exit + # with code 1 if timeout is reached. + # + # Because VPP systemd unit has Type=simple, systemctl will report the + # service to be active has soon as it is forked. This does not mean that + # the service is ready, and actually it takes some times before vppctl can + # succesfully connect to VPP client socket /run/vpp/cli.sock. + local wait_max=$1 + + local wait_time=0 + while ! vppctl show int; do + if [[ $wait_time -ge $wait_max ]]; then + # Log error to both system log and standard error output + logger -s "NFVBENCHVM ERROR: VPP service still not ready after $wait_max seconds." \ + "Exiting $(basename $0)." + exit 1 + fi + sleep 1 + wait_time=$(( wait_time + 1 )) + done +} + if [ $PCI_ADDRESS_1 ] && [ $PCI_ADDRESS_2 ]; then logger "NFVBENCHVM: Using pci $PCI_ADDRESS_1 ($INTF_MAC1)" logger "NFVBENCHVM: Using pci $PCI_ADDRESS_2 ($INTF_MAC2)" @@ -205,12 +246,22 @@ if [ $PCI_ADDRESS_1 ] && [ $PCI_ADDRESS_2 ]; then sed -i "s/{{WORKER_CORES}}/$WORKER_CORES/g" /etc/vpp/startup.conf sed -i "s/{{VIF_MQ_SIZE}}/${VIF_MQ_SIZE}/g" /etc/vpp/startup.conf sed -i "s/{{NUM_MBUFS}}/${NUM_MBUFS}/g" /etc/vpp/startup.conf - service vpp start - sleep 10 + systemctl start vpp + # Wait until VPP service is ready for at most 30 seconds + wait_vpp_service 30 - INTFS=`vppctl show int | grep Ethernet | xargs` - INTF_1=`echo $INTFS | awk '{ print $1 }'` - INTF_2=`echo $INTFS | awk '{ print $4 }'` + VPPCTL_OUTPUT=$(vppctl show int) + INTFS=$(echo "$VPPCTL_OUTPUT" | grep Ethernet | xargs) + INTF_1=$(echo $INTFS | awk '{ print $1 }') + INTF_2=$(echo $INTFS | awk '{ print $4 }') + if [[ -z "$INTF_1" ]] || [[ -z "$INTF_2" ]]; then + # Log error to both system log and standard error output + logger -s "NFVBENCHVM DEBUG: \"vppctl show int\" output:" + logger -s "NFVBENCHVM DEBUG: $VPPCTL_OUTPUT" + logger -s "NFVBENCHVM ERROR: vppctl does not show the two Ethernet interfaces we expect." \ + "Exiting $(basename $0)." + exit 1 + fi if [ -z "${TG_MAC1}" ]; then # vm.conf does not support lines commented with #, so # we need to remove the line to set the static ARP entry. @@ -231,7 +282,7 @@ if [ $PCI_ADDRESS_1 ] && [ $PCI_ADDRESS_2 ]; then sed -i "s/{{TG_NET2}}/${TG_NET2//\//\/}/g" /etc/vpp/vm.conf sed -i "s/{{TG_GATEWAY1_IP}}/${TG_GATEWAY1_IP}/g" /etc/vpp/vm.conf sed -i "s/{{TG_GATEWAY2_IP}}/${TG_GATEWAY2_IP}/g" /etc/vpp/vm.conf - service vpp restart + systemctl restart vpp logger "NFVBENCHVM: vpp service restarted" else echo "ERROR: Unknown forwarder value. Accepted values: testpmd or vpp" @@ -244,4 +295,4 @@ else logger "NFVBENCHVM ERROR: Cannot find PCI Address from MAC" fi -exit 0
\ No newline at end of file +exit 0 |