diff options
author | Yury Kylulin <yury.kylulin@intel.com> | 2019-12-03 18:29:00 +0300 |
---|---|---|
committer | Patrice Buriez <patrice.buriez@intel.com> | 2020-01-31 19:09:11 +0100 |
commit | 855816b08aaa8529dfcccef91890bbdae0a1dd3e (patch) | |
tree | 8db6b6982467011ddbb83e89d6a23c76e0f64cf1 /VNFs/DPPD-PROX/helper-scripts/rapid | |
parent | 5d3bcf89c23b72fcdc0c32306a843bb3a0f64dcf (diff) |
Add support for kubernetes deployment in rapid scripts.
In case of kubernetes based deployments procedure for test environment
creation is different. Please refer to README.k8s file for more
details.
Dockerfile and dockerimage.sh files can be used to create a docker
image with prox.
There is separate createrapidk8s.py script which is used to create
appropriate number of PODs for testing based on the rapid.pods
configuration.
port_info is a helper application to identify port MAC address.
Change-Id: I73611f066ac54260aa11dd17c173379df06f46a5
Signed-off-by: Yury Kylulin <yury.kylulin@intel.com>
Signed-off-by: Patrice Buriez <patrice.buriez@intel.com>
Diffstat (limited to 'VNFs/DPPD-PROX/helper-scripts/rapid')
26 files changed, 1251 insertions, 95 deletions
diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/Dockerfile b/VNFs/DPPD-PROX/helper-scripts/rapid/Dockerfile new file mode 100644 index 00000000..9f2161fe --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/Dockerfile @@ -0,0 +1,54 @@ +## +## Copyright (c) 2019 Intel Corporation +## +## Licensed under the Apache License, Version 2.0 (the "License"); +## you may not use this file except in compliance with the License. +## You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## + +################################################## +# Build all components in separate builder image # +################################################## +FROM centos:7 as builder + +ARG BUILD_DIR=/opt/rapid + +COPY ./port_info ${BUILD_DIR}/port_info + +COPY ./deploycentostools.sh ${BUILD_DIR}/ +RUN chmod +x ${BUILD_DIR}/deploycentostools.sh \ + && ${BUILD_DIR}/deploycentostools.sh -k deploy + +############################# +# Create slim runtime image # +############################# +FROM centos:7 + +ARG BUILD_DIR=/opt/rapid + +COPY ./deploycentostools.sh ${BUILD_DIR}/ +COPY --from=builder ${BUILD_DIR}/install_components.tgz ${BUILD_DIR}/install_components.tgz + +RUN chmod +x ${BUILD_DIR}/deploycentostools.sh \ + && ${BUILD_DIR}/deploycentostools.sh -k runtime_image + +# Expose SSH and PROX ports +EXPOSE 22 8474 + +# Copy SSH keys +COPY ./rapid_rsa_key.pub /home/centos/.ssh/authorized_keys +COPY ./rapid_rsa_key.pub /root/.ssh/authorized_keys + +# Copy startup script +COPY ./start.sh /start.sh +RUN chmod +x /start.sh + +ENTRYPOINT ["/start.sh"] diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/README.k8s b/VNFs/DPPD-PROX/helper-scripts/rapid/README.k8s new file mode 100644 index 00000000..e1abbe75 --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/README.k8s @@ -0,0 +1,94 @@ +## +## Copyright (c) 2019 Intel Corporation +## +## Licensed under the Apache License, Version 2.0 (the "License"); +## you may not use this file except in compliance with the License. +## You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## + +############################################################################### +# REQUIREMENTS # +############################################################################### +1. Working Kubernetes cluster. It can be set up using Intel Container Bare +Metal Reference Architecture https://github.com/intel/container-experience-kits + +2. 1024x 2M hugepages must be configured on the nodes + +3. SRIOV Network Device Plugin for Kubernetes installed +https://github.com/intel/sriov-network-device-plugin. + +4. SRIOV VFs configured and rebind to the vfio-pci module +As an example, SRIOV VFs (rebind to the vfio-pci driver) pool is named as +intel.com/intel_sriov_vfio. + +Network attachment definition is named as +k8s.v1.cni.cncf.io/networks: intel-sriov-vfio. + +5. PROX image created and pushed to the local registry or distributed and +loaded on all of the testing nodes. + +############################################################################### +# PROX IMAGE BUILD # +############################################################################### +Run +# dockerimage.sh build +to build PROX image. + +After the successfull build prox.tar will be created and can be used to load +image on the k8s nodes or it can be pushed to the local repository using +# dockerimage.sh push + +############################################################################### +# TESTING # +############################################################################### +1. Edit rapidpods file and set the right name (nodeSelector_hostname) for the +nodes on which you want to execute test PODs. + +# kubectl get nodes -o wide +NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME +k8s-master1 Ready master 7d13h v1.13.5 10.10.0.10 <none> CentOS Linux 7 (Core) 3.10.0-1062.4.1.el7.x86_64 docker://18.6.2 +k8s-node1 Ready node 7d13h v1.13.5 10.10.0.12 <none> CentOS Linux 7 (Core) 3.10.0-1062.4.1.el7.x86_64 docker://18.6.2 +k8s-node2 Ready node 7d13h v1.13.5 10.10.0.13 <none> CentOS Linux 7 (Core) 3.10.0-1062.4.1.el7.x86_64 docker://18.6.2 + +Set the right IP addresses (dp_ip) to use by the PODs for the Dataplane network. + +2. Edit pod-rapid.yaml file and set correct + - image name (image: localhost:5000/prox:latest) + - network attachment definition in metadata->annotation section + (k8s.v1.cni.cncf.io/networks: intel-sriov-vfio) + - SRIOV VFs resources attached to the vfio-pci driver + (intel.com/intel_sriov_vfio: '1') + +3. Copy SSH private key in the rapid_rsa_key file + +4. Run createrapidk8s.py to create test PODs according to the configuration from +rapid.pods file. + +# ./createrapidk8s.py + +Check for rapid PODs. They should be up and running. + +# kubectl get pods -o wide +NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES +pod-rapid-1 1/1 Running 0 18h 10.244.2.87 k8s-node1 <none> <none> +pod-rapid-2 1/1 Running 0 18h 10.244.1.40 k8s-node2 <none> <none> +pod-rapid-3 1/1 Running 0 18h 10.244.1.39 k8s-node2 <none> <none> + +5. Run test case. + +# ./runrapid.py --test basicrapid.test + +############################################################################### +# NOTES # +############################################################################### +If layer 2 tests are planned to be executed MAC adresses must be +preconfigured for the SRIOV VFs to avoid issues with randomly generated MACs +each time when the PROX starts. diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/centos.json b/VNFs/DPPD-PROX/helper-scripts/rapid/centos.json index 066f0ea4..92617b86 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/centos.json +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/centos.json @@ -38,7 +38,9 @@ }, { "type": "shell", - "script": "deploycentostools.sh" + "inline": [ + "chmod a+x /home/centos/deploycentostools.sh", + "/home/centos/deploycentostools.sh -u deploy" ] } ] } diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/createrapid.py b/VNFs/DPPD-PROX/helper-scripts/rapid/createrapid.py index ccb500aa..a7b1ec6c 100755 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/createrapid.py +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/createrapid.py @@ -264,14 +264,14 @@ cmd = 'openstack keypair list -f value -c Name' log.debug (cmd) KeyExist = subprocess.check_output(cmd , shell=True).decode().strip() if key in KeyExist: - log.info("Key ("+key+") already installed") + log.info("Key (" + key + ") already installed") else: log.info('Creating key ...') - cmd = 'openstack keypair create '+ key + '>' +key+'.pem' + cmd = 'openstack keypair create ' + key + '>' + key + '.pem' log.debug(cmd) subprocess.check_call(cmd , shell=True) - cmd = 'chmod 600 ' +key+'.pem' - subprocess.check_call(cmd , shell=True) + cmd = 'chmod 600 ' + key + '.pem' + subprocess.check_call(cmd, shell=True) cmd = 'openstack keypair list -f value -c Name' log.debug(cmd) KeyExist = subprocess.check_output(cmd , shell=True).decode().strip() @@ -388,7 +388,7 @@ for vm in range(1, int(total_number_of_VMs)+1): log.info('%s: (admin IP: %s), (dataplane IP: %s), (dataplane MAC: %s)' % (ServerName[vm-1],vmAdminIP,vmDPIP,vmDPmac)) config.add_section('ssh') -config.set('ssh', 'key', key+'.pem') +config.set('ssh', 'key', key + '.pem') config.set('ssh', 'user', 'centos') config.add_section('Varia') config.set('Varia', 'VIM', 'OpenStack') diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/createrapidk8s.py b/VNFs/DPPD-PROX/helper-scripts/rapid/createrapidk8s.py new file mode 100755 index 00000000..e096c82e --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/createrapidk8s.py @@ -0,0 +1,37 @@ +#!/usr/bin/env python2.7 + +## +## Copyright (c) 2019 Intel Corporation +## +## Licensed under the Apache License, Version 2.0 (the "License"); +## you may not use this file except in compliance with the License. +## You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## + +from k8sdeployment import K8sDeployment + +# Config file name for deployment creation +CREATE_CONFIG_FILE_NAME = "rapid.pods" + +# Config file name for runrapid script +RUN_CONFIG_FILE_NAME = "rapid.env" + +# Create a new deployment +deployment = K8sDeployment() + +# Load config file with test environment description +deployment.load_create_config(CREATE_CONFIG_FILE_NAME) + +# Create PODs for test +deployment.create_pods() + +# Save config file for runrapid script +deployment.save_runtime_config(RUN_CONFIG_FILE_NAME) diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/deploycentostools.sh b/VNFs/DPPD-PROX/helper-scripts/rapid/deploycentostools.sh index 1d260d61..5e2cf3d1 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/deploycentostools.sh +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/deploycentostools.sh @@ -15,10 +15,12 @@ ## limitations under the License. ## +# Directory for package build BUILD_DIR="/opt/rapid" -WORK_DIR="/home/centos" # Directory where the packer tool has copied some files (e.g. check_prox_system_setup.sh) - # Runtime scripts are assuming ${WORK_DIR} as the directory for PROX. Check the rundir variable in runrapid.py. Should be the same! - # This variable is defined in 4 different places and should have the same value: centos.json, deploycentos.sh, check_prox_system_setup.sh and runrapid.py +# Directory where the packer tool has copied some files (e.g. check_prox_system_setup.sh) +# Runtime scripts are assuming ${WORK_DIR} as the directory for PROX. Check the rundir variable in runrapid.py. Should be the same! +# This variable is defined in 4 different places and should have the same value: centos.json, deploycentostools.sh, check_prox_system_setup.sh and runrapid.py +WORK_DIR="/home/centos" DPDK_VERSION="19.05" PROX_COMMIT="b71a4cfd" PROX_CHECKOUT="git checkout ${PROX_COMMIT}" @@ -29,22 +31,42 @@ MULTI_BUFFER_LIB_VER="0.52" export RTE_SDK="${BUILD_DIR}/dpdk-${DPDK_VERSION}" export RTE_TARGET="x86_64-native-linuxapp-gcc" +# By default, do not update OS +OS_UPDATE="n" +# By default, asumming that we are in the VM +K8S_ENV="n" + +# If already running from root, no need for sudo +SUDO="" +[ $(id -u) -ne 0 ] && SUDO="sudo" + function os_pkgs_install() { - sudo yum install -y deltarpm yum-utils + ${SUDO} yum install -y deltarpm yum-utils + # NASM repository for AESNI MB library - sudo yum-config-manager --add-repo http://www.nasm.us/nasm.repo + ${SUDO} yum-config-manager --add-repo http://www.nasm.us/nasm.repo - sudo yum update -y - sudo yum install -y git wget gcc unzip libpcap-devel ncurses-devel \ + [ "${OS_UPDATE}" == "y" ] && ${SUDO} yum update -y + ${SUDO} yum install -y git wget gcc unzip libpcap-devel ncurses-devel \ libedit-devel lua-devel kernel-devel iperf3 pciutils \ - numactl-devel vim tuna openssl-devel nasm wireshark + numactl-devel vim tuna openssl-devel nasm wireshark \ + make +} + +function k8s_os_pkgs_runtime_install() +{ + [ "${OS_UPDATE}" == "y" ] && ${SUDO} yum update -y + + # Install required dynamically linked libraries + required packages + ${SUDO} yum install -y numactl-libs libpcap openssh openssh-server \ + openssh-clients sudo } function os_cfg() { # huge pages to be used by DPDK - sudo sh -c '(echo "vm.nr_hugepages = 1024") > /etc/sysctl.conf' + ${SUDO} sh -c '(echo "vm.nr_hugepages = 1024") > /etc/sysctl.conf' # Enabling tuned with the realtime-virtual-guest profile pushd ${BUILD_DIR} > /dev/null 2>&1 @@ -52,26 +74,44 @@ function os_cfg() wget http://linuxsoft.cern.ch/cern/centos/7/rt/x86_64/Packages/tuned-profiles-nfv-guest-2.8.0-5.el7_4.2.noarch.rpm # Install with --nodeps. The latest CentOS cloud images come with a tuned version higher than 2.8. These 2 packages however # do not depend on v2.8 and also work with tuned 2.9. Need to be careful in the future - sudo rpm -ivh ${BUILD_DIR}/tuned-profiles-realtime-2.8.0-5.el7_4.2.noarch.rpm --nodeps - sudo rpm -ivh ${BUILD_DIR}/tuned-profiles-nfv-guest-2.8.0-5.el7_4.2.noarch.rpm --nodeps + ${SUDO} rpm -ivh ${BUILD_DIR}/tuned-profiles-realtime-2.8.0-5.el7_4.2.noarch.rpm --nodeps + ${SUDO} rpm -ivh ${BUILD_DIR}/tuned-profiles-nfv-guest-2.8.0-5.el7_4.2.noarch.rpm --nodeps # Although we do no know how many cores the VM will have when begin deployed for real testing, we already put a number for the # isolated CPUs so we can start the realtime-virtual-guest profile. If we don't, that command will fail. # When the VM will be instantiated, the check_kernel_params service will check for the real number of cores available to this VM # and update the realtime-virtual-guest-variables.conf accordingly. - echo "isolated_cores=1" | sudo tee -a /etc/tuned/realtime-virtual-guest-variables.conf - sudo tuned-adm profile realtime-virtual-guest + echo "isolated_cores=1" | ${SUDO} tee -a /etc/tuned/realtime-virtual-guest-variables.conf + ${SUDO} tuned-adm profile realtime-virtual-guest # Install the check_tuned_params service to make sure that the grub cmd line has the right cpus in isolcpu. The actual number of cpu's # assigned to this VM depends on the flavor used. We don't know at this time what that will be. - sudo chmod +x ${WORK_DIR}/check_prox_system_setup.sh - sudo cp -r ${WORK_DIR}/check_prox_system_setup.sh /usr/local/libexec/ - sudo cp -r ${WORK_DIR}/check-prox-system-setup.service /etc/systemd/system/ - sudo systemctl daemon-reload - sudo systemctl enable check-prox-system-setup.service + ${SUDO} chmod +x ${WORK_DIR}/check_prox_system_setup.sh + ${SUDO} cp -r ${WORK_DIR}/check_prox_system_setup.sh /usr/local/libexec/ + ${SUDO} cp -r ${WORK_DIR}/check-prox-system-setup.service /etc/systemd/system/ + ${SUDO} systemctl daemon-reload + ${SUDO} systemctl enable check-prox-system-setup.service popd > /dev/null 2>&1 } +function k8s_os_cfg() +{ + [ ! -f /etc/ssh/ssh_host_rsa_key ] && ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -N '' + [ ! -f /etc/ssh/ssh_host_ecdsa_key ] && ssh-keygen -t ecdsa -f /etc/ssh/ssh_host_ecdsa_key -N '' + [ ! -f /etc/ssh/ssh_host_ed25519_key ] && ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key -N '' + + [ ! -d /var/run/sshd ] && mkdir -p /var/run/sshd + + USER_NAME="centos" + USER_PWD="centos" + + useradd -m -d /home/${USER_NAME} -s /bin/bash -U ${USER_NAME} + echo "${USER_NAME}:${USER_PWD}" | chpasswd + usermod -aG wheel ${USER_NAME} + + echo "%wheel ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/wheelnopass +} + function mblib_install() { export AESNI_MULTI_BUFFER_LIB_PATH="${BUILD_DIR}/intel-ipsec-mb-${MULTI_BUFFER_LIB_VER}" @@ -82,7 +122,7 @@ function mblib_install() unzip v${MULTI_BUFFER_LIB_VER}.zip pushd ${AESNI_MULTI_BUFFER_LIB_PATH} make -j`getconf _NPROCESSORS_ONLN` - sudo make install + ${SUDO} make install popd > /dev/null 2>&1 popd > /dev/null 2>&1 } @@ -99,15 +139,23 @@ function dpdk_install() tar -xf ./dpdk-${DPDK_VERSION}.tar.xz popd > /dev/null 2>&1 - sudo ln -s ${RTE_SDK} ${WORK_DIR}/dpdk + ${SUDO} ln -s ${RTE_SDK} ${WORK_DIR}/dpdk pushd ${RTE_SDK} > /dev/null 2>&1 make config T=${RTE_TARGET} # The next sed lines make sure that we can compile DPDK 17.11 with a relatively new OS. Using a newer DPDK (18.5) should also resolve this issue - #sudo sed -i '/CONFIG_RTE_LIBRTE_KNI=y/c\CONFIG_RTE_LIBRTE_KNI=n' ${RTE_SDK}/build/.config - #sudo sed -i '/CONFIG_RTE_LIBRTE_PMD_KNI=y/c\CONFIG_RTE_LIBRTE_PMD_KNI=n' ${RTE_SDK}/build/.config - #sudo sed -i '/CONFIG_RTE_KNI_KMOD=y/c\CONFIG_RTE_KNI_KMOD=n' ${RTE_SDK}/build/.config - #sudo sed -i '/CONFIG_RTE_KNI_PREEMPT_DEFAULT=y/c\CONFIG_RTE_KNI_PREEMPT_DEFAULT=n' ${RTE_SDK}/build/.config + #${SUDO} sed -i '/CONFIG_RTE_LIBRTE_KNI=y/c\CONFIG_RTE_LIBRTE_KNI=n' ${RTE_SDK}/build/.config + #${SUDO} sed -i '/CONFIG_RTE_LIBRTE_PMD_KNI=y/c\CONFIG_RTE_LIBRTE_PMD_KNI=n' ${RTE_SDK}/build/.config + #${SUDO} sed -i '/CONFIG_RTE_KNI_KMOD=y/c\CONFIG_RTE_KNI_KMOD=n' ${RTE_SDK}/build/.config + #${SUDO} sed -i '/CONFIG_RTE_KNI_PREEMPT_DEFAULT=y/c\CONFIG_RTE_KNI_PREEMPT_DEFAULT=n' ${RTE_SDK}/build/.config + + # For Kubernetes environment we use host vfio module + if [ "${K8S_ENV}" == "y" ]; then + sed -i 's/CONFIG_RTE_EAL_IGB_UIO=y/CONFIG_RTE_EAL_IGB_UIO=n/g' ${RTE_SDK}/build/.config + sed -i 's/CONFIG_RTE_LIBRTE_KNI=y/CONFIG_RTE_LIBRTE_KNI=n/g' ${RTE_SDK}/build/.config + sed -i 's/CONFIG_RTE_KNI_KMOD=y/CONFIG_RTE_KNI_KMOD=n/g' ${RTE_SDK}/build/.config + fi + # Compile with MB library sed -i '/CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n/c\CONFIG_RTE_LIBRTE_PMD_AESNI_MB=y' ${RTE_SDK}/build/.config make -j`getconf _NPROCESSORS_ONLN` @@ -128,23 +176,125 @@ function prox_install() # Clone and compile PROX pushd ${BUILD_DIR} > /dev/null 2>&1 git clone https://git.opnfv.org/samplevnf - pushd ${BUILD_DIR}/samplevnf/VNFs/DPPD-PROX + pushd ${BUILD_DIR}/samplevnf/VNFs/DPPD-PROX > /dev/null 2>&1 ${PROX_CHECKOUT} popd > /dev/null 2>&1 prox_compile - sudo ln -s ${BUILD_DIR}/samplevnf/VNFs/DPPD-PROX ${WORK_DIR}/prox + ${SUDO} cp ${BUILD_DIR}/samplevnf/VNFs/DPPD-PROX/build/app/prox ${WORK_DIR}/prox + popd > /dev/null 2>&1 +} + +function port_info_build() +{ + [ ! -d ${BUILD_DIR}/port_info ] && echo "Skipping port_info compilation..." && return + + pushd ${BUILD_DIR}/port_info > /dev/null 2>&1 + make + ${SUDO} cp ${BUILD_DIR}/port_info/build/app/port_info ${WORK_DIR}/port_info popd > /dev/null 2>&1 } -if [ "$1" == "compile" ]; then +function create_minimal_install() +{ + ldd ${WORK_DIR}/prox | awk '{ if ($(NF-1) != "=>") print $(NF-1) }' >> ${BUILD_DIR}/list_of_install_components + + echo "${WORK_DIR}/prox" >> ${BUILD_DIR}/list_of_install_components + echo "${WORK_DIR}/port_info" >> ${BUILD_DIR}/list_of_install_components + + tar -czvhf ${BUILD_DIR}/install_components.tgz -T ${BUILD_DIR}/list_of_install_components +} + +function cleanup() +{ + ${SUDO} yum autoremove -y + ${SUDO} yum clean all + ${SUDO} rm -rf /var/cache/yum +} + +function k8s_runtime_image() +{ + k8s_os_pkgs_runtime_install + k8s_os_cfg + cleanup + + pushd / > /dev/null 2>&1 + tar -xvf ${BUILD_DIR}/install_components.tgz --skip-old-files + popd > /dev/null 2>&1 + + ldconfig + + #rm -rf ${BUILD_DIR}/install_components.tgz +} + +function print_usage() +{ + echo "Usage: ${0} [OPTIONS] [COMMAND]" + echo "Options:" + echo " -u, --update Full OS update" + echo " -k, --kubernetes Build for Kubernetes environment" + echo "Commands:" + echo " deploy Run through all deployment steps" + echo " compile PROX compile only" + echo " runtime_image Apply runtime configuration only" +} + +COMMAND="" +# Parse options and comman +for opt in "$@"; do + case ${opt} in + -u|--update) + echo 'Full OS update will be done!' + OS_UPDATE="y" + ;; + -k|--kubernetes) + echo "Kubernetes environment is set!" + K8S_ENV="y" + ;; + compile) + COMMAND="compile" + ;; + runtime_image) + COMMAND="runtime_image" + ;; + deploy) + COMMAND="deploy" + ;; + *) + echo "Unknown option/command ${opt}" + print_usage + exit 1 + ;; + esac +done + +if [ "${COMMAND}" == "compile" ]; then + echo "PROX compile only..." prox_compile -else - [ ! -d ${BUILD_DIR} ] && sudo mkdir -p ${BUILD_DIR} - sudo chmod 0777 ${BUILD_DIR} +elif [ "${COMMAND}" == "runtime_image" ]; then + echo "Runtime image intallation and configuration..." + k8s_runtime_image +elif [ "${COMMAND}" == "deploy" ]; then + [ ! -d ${BUILD_DIR} ] && ${SUDO} mkdir -p ${BUILD_DIR} + ${SUDO} chmod 0777 ${BUILD_DIR} os_pkgs_install - os_cfg + + if [ "${K8S_ENV}" == "y" ]; then + k8s_os_cfg + else + os_cfg + fi + mblib_install dpdk_install prox_install + + if [ "${K8S_ENV}" == "y" ]; then + port_info_build + create_minimal_install + fi + + cleanup +else + print_usage fi diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/dockerimage.sh b/VNFs/DPPD-PROX/helper-scripts/rapid/dockerimage.sh new file mode 100755 index 00000000..3d77aaed --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/dockerimage.sh @@ -0,0 +1,95 @@ +#!/usr/bin/env bash +## +## Copyright (c) 2010-2019 Intel Corporation +## +## Licensed under the Apache License, Version 2.0 (the "License"); +## you may not use this file except in compliance with the License. +## You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## + +PROX_DEPLOY_DIR="." +PROX_IMAGE_NAME="prox_slim" +RSA_KEY_FILE_NAME="rapid_rsa_key" + +DOCKERFILE="Dockerfile" +DOCKER_REGISTRY="localhost:5000" + +USE_DOCKER_CACHE="n" + +function create_ssh_key() +{ + if [ -f ./${RSA_KEY_FILE_NAME} ]; then + read -p "RSA key already exist! Do you want to remove it (yYnN)?" -n 1 -r + + if [ "$REPLY" == "y" ] || [ "$REPLY" == "Y" ]; then + echo "Removing existing key..." + sleep 3 + + [ -f "./${RSA_KEY_FILE_NAME}" ] && rm -rf ./${RSA_KEY_FILE_NAME} + [ -f "./${RSA_KEY_FILE_NAME}.pub" ] && rm -rf ./${RSA_KEY_FILE_NAME}.pub + else + echo "Using existing key..." + return + fi + fi + + echo "Generating new RSA key..." + ssh-keygen -t rsa -b 4096 -N "" -f ./${RSA_KEY_FILE_NAME} +} + +function build_prox_image() +{ + if [ "${USE_DOCKER_CACHE}" == "y" ]; then + echo "Building image using cache..." + docker build --rm -t ${PROX_IMAGE_NAME}:latest -f ${DOCKERFILE} ${PROX_DEPLOY_DIR} + else + echo "Building image without cache..." + docker build --no-cache --rm -t ${PROX_IMAGE_NAME}:latest -f ${DOCKERFILE} ${PROX_DEPLOY_DIR} + fi +} + +function save_prox_image() +{ + echo "Saving image ${PROX_IMAGE_NAME}:latest to ./${PROX_IMAGE_NAME}.tar" + docker save -o ./${PROX_IMAGE_NAME}.tar ${PROX_IMAGE_NAME}:latest +} + +function load_prox_image() +{ + echo "Loading image ./${PROX_IMAGE_NAME}.tar" + docker load -i ./${PROX_IMAGE_NAME}.tar +} + +function push_prox_image() +{ + docker tag ${PROX_IMAGE_NAME}:latest ${DOCKER_REGISTRY}/${PROX_IMAGE_NAME} + docker push ${DOCKER_REGISTRY}/${PROX_IMAGE_NAME} +} + +function print_help() +{ + echo "${0}: [build|load|push]" + echo " build: build and save image ${PROX_IMAGE_NAME}:latest using ${DOCKERFILE}" + echo " load: load saved image from ${PROX_IMAGE_NAME}.tar file in the local registry" + echo " push: tag and push local ${PROX_IMAGE_NAME}:latest image in the ${DOCKER_REGISTRY}/${PROX_IMAGE_NAME} registry" +} + +if [ "$1" == "build" ]; then + create_ssh_key + build_prox_image + save_prox_image +elif [ "$1" == "load" ]; then + load_prox_image +elif [ "$1" == "push" ]; then + push_prox_image +else + print_help +fi diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/gen.cfg b/VNFs/DPPD-PROX/helper-scripts/rapid/gen.cfg index 775eef9a..4944f017 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/gen.cfg +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/gen.cfg @@ -14,12 +14,13 @@ ;; limitations under the License. ;; +[lua] +dofile("parameters.lua") + [eal options] -n=4 ; force number of memory channels no-output=no ; disable DPDK debug output - -[lua] -dofile("parameters.lua") +eal=--proc-type auto ${eal} [port 0] name=p0 diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/gen_gw.cfg b/VNFs/DPPD-PROX/helper-scripts/rapid/gen_gw.cfg index c79e144a..b4843acf 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/gen_gw.cfg +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/gen_gw.cfg @@ -14,12 +14,13 @@ ;; limitations under the License. ;; +[lua] +dofile("parameters.lua") + [eal options] -n=4 ; force number of memory channels no-output=no ; disable DPDK debug output - -[lua] -dofile("parameters.lua") +eal=--proc-type auto ${eal} [port 0] name=p0 diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/impair.cfg b/VNFs/DPPD-PROX/helper-scripts/rapid/impair.cfg index 8e86c8e1..183fe77b 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/impair.cfg +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/impair.cfg @@ -14,12 +14,13 @@ ;; limitations under the License. ;; +[lua] +dofile("parameters.lua") + [eal options] -n=4 ; force number of memory channels no-output=no ; disable DPDK debug output - -[lua] -dofile("parameters.lua") +eal=--proc-type auto ${eal} [port 0] name=if0 diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/irq.cfg b/VNFs/DPPD-PROX/helper-scripts/rapid/irq.cfg index 8c1331e9..4e9af96b 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/irq.cfg +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/irq.cfg @@ -14,12 +14,13 @@ ;; limitations under the License. ;; +[lua] +dofile("parameters.lua") + [eal options] -n=4 ; force number of memory channels no-output=no ; disable DPDK debug output - -[lua] -dofile("parameters.lua") +eal=--proc-type auto ${eal} [;port 0] name=p0 diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/k8sdeployment.py b/VNFs/DPPD-PROX/helper-scripts/rapid/k8sdeployment.py new file mode 100644 index 00000000..caf2d2e2 --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/k8sdeployment.py @@ -0,0 +1,196 @@ +#!/usr/bin/env python2.7 + +## +## Copyright (c) 2019 Intel Corporation +## +## Licensed under the Apache License, Version 2.0 (the "License"); +## you may not use this file except in compliance with the License. +## You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## + +import sys +from kubernetes import client, config +import ConfigParser +import logging +from logging import handlers + +from pod import Pod + +class K8sDeployment: + """Deployment class to create containers for test execution in Kubernetes + environment. + """ + LOG_FILE_NAME = "createrapidk8s.log" + SSH_PRIVATE_KEY = "./rapid_rsa_key" + SSH_USER = "centos" + + POD_YAML_TEMPLATE_FILE_NAME = "pod-rapid.yaml" + + _log = None + _create_config = None + _runtime_config = None + _total_number_of_pods = 0 + _pods = [] + + def __init__(self): + # Configure logger + self._log = logging.getLogger("k8srapid") + self._log.setLevel(logging.DEBUG) + + console_formatter = logging.Formatter("%(message)s") + console_handler = logging.StreamHandler(sys.stdout) + console_handler.setLevel(logging.DEBUG) + console_handler.setFormatter(console_formatter) + + file_formatter = logging.Formatter("%(asctime)s - " + "%(levelname)s - " + "%(message)s") + file_handler = logging.handlers.RotatingFileHandler(self.LOG_FILE_NAME, + backupCount=10) + file_handler.setLevel(logging.DEBUG) + file_handler.setFormatter(file_formatter) + + self._log.addHandler(file_handler) + self._log.addHandler(console_handler) + + # Initialize k8s plugin + config.load_kube_config() + Pod.k8s_CoreV1Api = client.CoreV1Api() + + def load_create_config(self, config_file_name): + """Read and parse configuration file for the test environment. + """ + self._log.info("Loading configuration file %s", config_file_name) + self._create_config = ConfigParser.RawConfigParser() + try: + self._create_config.read(config_file_name) + except Exception as e: + self._log.error("Failed to read config file!\n%s\n" % e) + return -1 + + # Now parse config file content + # Parse [DEFAULT] section + if self._create_config.has_option("DEFAULT", "total_number_of_pods"): + self._total_number_of_pods = self._create_config.getint( + "DEFAULT", "total_number_of_pods") + else: + self._log.error("No option total_number_of_pods in DEFAULT section") + return -1 + + self._log.debug("Total number of pods %d" % self._total_number_of_pods) + + # Parse [PODx] sections + for i in range(1, int(self._total_number_of_pods) + 1): + # Search for POD name + if self._create_config.has_option("POD%d" % i, + "name"): + pod_name = self._create_config.get( + "POD%d" % i, "name") + else: + pod_name = "pod-rapid-%d" % i + + # Search for POD hostname + if self._create_config.has_option("POD%d" % i, + "nodeSelector_hostname"): + pod_nodeselector_hostname = self._create_config.get( + "POD%d" % i, "nodeSelector_hostname") + else: + pod_nodeselector_hostname = None + + # Search for POD dataplane static IP + if self._create_config.has_option("POD%d" % i, + "dp_ip"): + pod_dp_ip = self._create_config.get( + "POD%d" % i, "dp_ip") + else: + pod_dp_ip = None + + pod = Pod(pod_name) + pod.set_nodeselector(pod_nodeselector_hostname) + pod.set_dp_ip(pod_dp_ip) + pod.set_id(i) + + # Add POD to the list of PODs which need to be created + self._pods.append(pod) + + return 0 + + def create_pods(self): + """ Create test PODs and wait for them to start. + Collect information for tests to run. + """ + self._log.info("Creating PODs...") + + # Create PODs using template from yaml file + for pod in self._pods: + self._log.info("Creating POD %s...", pod.get_name()) + pod.create_from_yaml(K8sDeployment.POD_YAML_TEMPLATE_FILE_NAME) + + # Wait for PODs to start + for pod in self._pods: + pod.wait_for_start() + + # Collect information from started PODs for test execution + for pod in self._pods: + pod.set_ssh_credentials(K8sDeployment.SSH_USER, K8sDeployment.SSH_PRIVATE_KEY) + pod.get_sriov_dev_mac() + + def save_runtime_config(self, config_file_name): + self._log.info("Saving config %s for runrapid script...", + config_file_name) + self._runtime_config = ConfigParser.RawConfigParser() + + # Section [DEFAULT] +# self._runtime_config.set("DEFAULT", +# "total_number_of_test_machines", +# self._total_number_of_pods) + + # Section [ssh] + self._runtime_config.add_section("ssh") + self._runtime_config.set("ssh", + "key", + K8sDeployment.SSH_PRIVATE_KEY) + self._runtime_config.set("ssh", + "user", + K8sDeployment.SSH_USER) + + # Section [rapid] + self._runtime_config.add_section("rapid") + self._runtime_config.set("rapid", + "total_number_of_machines", + self._total_number_of_pods) + + # Export information about each pod + # Sections [Mx] + for pod in self._pods: + self._runtime_config.add_section("M%d" % pod.get_id()) + self._runtime_config.set("M%d" % pod.get_id(), + "admin_ip", pod.get_admin_ip()) + self._runtime_config.set("M%d" % pod.get_id(), + "dp_mac", pod.get_dp_mac()) + self._runtime_config.set("M%d" % pod.get_id(), + "dp_pci_dev", pod.get_dp_pci_dev()) + self._runtime_config.set("M%d" % pod.get_id(), + "dp_ip", pod.get_dp_ip()) + + # Section [Varia] + self._runtime_config.add_section("Varia") + self._runtime_config.set("Varia", + "vim", + "kubernetes") + + # Write runtime config file + with open(config_file_name, "w") as file: + self._runtime_config.write(file) + + def delete_pods(self): + for pod in self._pods: + pod.terminate() diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/l2gen.cfg b/VNFs/DPPD-PROX/helper-scripts/rapid/l2gen.cfg index 37612c3d..b1f624c1 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/l2gen.cfg +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/l2gen.cfg @@ -14,12 +14,13 @@ ;; limitations under the License. ;; +[lua] +dofile("parameters.lua") + [eal options] -n=4 ; force number of memory channels no-output=no ; disable DPDK debug output - -[lua] -dofile("parameters.lua") +eal=--proc-type auto ${eal} [port 0] name=p0 diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/l2gen_bare.cfg b/VNFs/DPPD-PROX/helper-scripts/rapid/l2gen_bare.cfg index 380b6646..57a7094d 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/l2gen_bare.cfg +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/l2gen_bare.cfg @@ -14,12 +14,13 @@ ;; limitations under the License. ;; +[lua] +dofile("parameters.lua") + [eal options] -n=4 ; force number of memory channels no-output=no ; disable DPDK debug output - -[lua] -dofile("parameters.lua") +eal=--proc-type auto ${eal} [port 0] name=p0 diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/l2swap.cfg b/VNFs/DPPD-PROX/helper-scripts/rapid/l2swap.cfg index 2c8ce091..17396381 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/l2swap.cfg +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/l2swap.cfg @@ -14,12 +14,13 @@ ;; limitations under the License. ;; +[lua] +dofile("parameters.lua") + [eal options] -n=4 ; force number of memory channels no-output=no ; disable DPDK debug output - -[lua] -dofile("parameters.lua") +eal=--proc-type auto ${eal} [port 0] name=if0 diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/pod-rapid.yaml b/VNFs/DPPD-PROX/helper-scripts/rapid/pod-rapid.yaml new file mode 100644 index 00000000..5ce09071 --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/pod-rapid.yaml @@ -0,0 +1,30 @@ +apiVersion: v1 +kind: Pod +metadata: + name: pod-rapid- + annotations: + k8s.v1.cni.cncf.io/networks: intel-sriov-vfio +spec: + containers: + - name: pod-rapid + image: localhost:5000/prox_slim:latest + imagePullPolicy: Always + securityContext: + capabilities: + add: ["IPC_LOCK"] + volumeMounts: + - mountPath: /dev/hugepages + name: hugepages + resources: + requests: + hugepages-2Mi: 512Mi + memory: 1Gi + intel.com/intel_sriov_vfio: '1' + limits: + hugepages-2Mi: 512Mi + intel.com/intel_sriov_vfio: '1' + volumes: + - name: hugepages + emptyDir: + medium: HugePages + restartPolicy: Never diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/pod.py b/VNFs/DPPD-PROX/helper-scripts/rapid/pod.py new file mode 100644 index 00000000..359523af --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/pod.py @@ -0,0 +1,203 @@ +#!/usr/bin/env python2.7 + +## +## Copyright (c) 2019 Intel Corporation +## +## Licensed under the Apache License, Version 2.0 (the "License"); +## you may not use this file except in compliance with the License. +## You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## + +from os import path +import time, yaml +import logging +from kubernetes import client, config + +from sshclient import SSHClient + +class Pod: + """Class which represents test pods. + For example with traffic gen, forward/swap applications, etc + """ + k8s_CoreV1Api = None + + _log = None + + _name = "pod" + _namespace = "default" + _nodeSelector_hostname = None + _last_status = None + _id = None + _admin_ip = None + _dp_ip = None + + _ssh_client = None + + _sriov_vf = None + _sriov_vf_mac = None + + def __init__(self, name, namespace = "default", logger_name = "k8srapid"): + self._log = logging.getLogger(logger_name) + + self._name = name + self._namespace = namespace + self._ssh_client = SSHClient(logger_name = logger_name) + + def __del__(self): + """Destroy POD. Do a cleanup. + """ + if self._ssh_client is not None: + self._ssh_client.disconnect() + + def create_from_yaml(self, file_name): + """Load POD description from yaml file. + """ + with open(path.join(path.dirname(__file__), file_name)) as yaml_file: + self.body = yaml.safe_load(yaml_file) + + self.body["metadata"]["name"] = self._name + + if (self._nodeSelector_hostname is not None): + if ("nodeSelector" not in self.body["spec"]): + self.body["spec"]["nodeSelector"] = {} + self.body["spec"]["nodeSelector"]["kubernetes.io/hostname"] = self._nodeSelector_hostname + self._log.debug("Creating POD, body:\n%s" % self.body) + + try: + self.k8s_CoreV1Api.create_namespaced_pod(body = self.body, + namespace = self._namespace) + except client.rest.ApiException as e: + self._log.error("Couldn't create POD %s!\n%s\n" % (self._name, e)) + + def terminate(self): + """Terminate POD. Close SSH connection. + """ + if self._ssh_client is not None: + self._ssh_client.disconnect() + self.k8s_CoreV1Api.delete_namespaced_pod(name = self._name, + namespace = self._namespace) + + def update_admin_ip(self): + """Check for admin IP address assigned by k8s. + """ + try: + pod = self.k8s_CoreV1Api.read_namespaced_pod_status(name = self._name, namespace = self._namespace) + self._admin_ip = pod.status.pod_ip + except client.rest.ApiException as e: + self._log.error("Couldn't update POD %s admin IP!\n%s\n" % (self._name, e)) + + def wait_for_start(self): + """Wait for POD to start. + """ + self._log.info("Waiting for POD %s to start..." % self._name) + while True: + self.get_status() + if (self._last_status == "Running" or self._last_status == "Failed" + or self._last_status == "Unknown"): + break + else: + time.sleep(3) + + self.update_admin_ip() + + return self._last_status + + def ssh_run_cmd(self, cmd): + """Execute command for POD via SSH connection. + SSH credentials should be configured before use of this function. + """ + self._ssh_client.run_cmd(cmd) + + def get_name(self): + return self._name + + def get_admin_ip(self): + return self._admin_ip + + def get_dp_ip(self): + return self._dp_ip + + def get_dp_mac(self): + return self._sriov_vf_mac + + def get_dp_pci_dev(self): + return self._sriov_vf + + def get_id(self): + return self._id + + def get_status(self): + """Get current status fro the pod. + """ + try: + pod = self.k8s_CoreV1Api.read_namespaced_pod_status(name = self._name, + namespace = self._namespace) + except client.rest.ApiException as e: + self._log.error("Couldn't read POD %s status!\n%s\n" % (self._name, e)) + + self._last_status = pod.status.phase + return self._last_status + + def get_sriov_dev_mac(self): + """Get assigned by k8s SRIOV network device plugin SRIOV VF devices. + Return 0 in case of sucessfull configuration. + Otherwise return -1. + """ + self._log.info("Checking assigned SRIOV VF for POD %s" % self._name) + ret = self._ssh_client.run_cmd("cat /opt/k8s_sriov_device_plugin_envs") + if ret != 0: + self._log.error("Failed to check assigned SRIOV VF!" + "Error %s" % self._ssh_client.get_error()) + return -1 + + cmd_output = self._ssh_client.get_output().decode("utf-8").rstrip() + self._log.debug("Environment variable %s" % cmd_output) + + # Parse environment variable + cmd_output = cmd_output.split("=")[1] + self._sriov_vf = cmd_output.split(",")[0] + self._log.debug("Using first SRIOV VF %s" % self._sriov_vf) + + self._log.info("Getting MAC address for assigned SRIOV VF %s" % self._sriov_vf) + self._ssh_client.run_cmd("sudo /home/centos/port_info -n 4 -w %s" % self._sriov_vf) + if ret != 0: + self._log.error("Failed to get MAC address!" + "Error %s" % self._ssh_client.get_error()) + return -1 + + # Parse MAC address + cmd_output = self._ssh_client.get_output().decode("utf-8").rstrip() + self._log.debug(cmd_output) + cmd_output = cmd_output.splitlines() + for line in cmd_output: + if line.startswith("Port 0 MAC: "): + self._sriov_vf_mac = line[12:] + + self._log.debug("MAC %s" % self._sriov_vf_mac) + + def set_dp_ip(self, dp_ip): + self._dp_ip = dp_ip + + def set_id(self, pod_id): + self._id = pod_id + + def set_nodeselector(self, hostname): + """Set hostname on which POD will be executed. + """ + self._nodeSelector_hostname = hostname + + def set_ssh_credentials(self, user, rsa_private_key): + """Set SSH credentials for the SSH connection to the POD. + """ + self.update_admin_ip() + self._ssh_client.set_credentials(ip = self._admin_ip, + user = user, + rsa_private_key = rsa_private_key) diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/port_info/Makefile b/VNFs/DPPD-PROX/helper-scripts/rapid/port_info/Makefile new file mode 100644 index 00000000..39ebd354 --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/port_info/Makefile @@ -0,0 +1,42 @@ +## +## Copyright (c) 2019 Intel Corporation +## +## Licensed under the Apache License, Version 2.0 (the "License"); +## you may not use this file except in compliance with the License. +## You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## + +ifeq ($(RTE_SDK),) +$(error "Please define RTE_SDK environment variable") +endif + +# Default target, can be overridden by command line or environment +RTE_TARGET ?= x86_64-native-linuxapp-gcc + +include $(RTE_SDK)/mk/rte.vars.mk + +# binary name +APP = port_info + +# all source are stored in SRCS-y +SRCS-y := port_info.c + +CFLAGS += $(WERROR_FLAGS) + +# workaround for a gcc bug with noreturn attribute +# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603 +ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y) +CFLAGS_main.o += -Wno-return-type +endif + +EXTRA_CFLAGS += -O3 -g -Wfatal-errors + +include $(RTE_SDK)/mk/rte.extapp.mk diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/port_info/port_info.c b/VNFs/DPPD-PROX/helper-scripts/rapid/port_info/port_info.c new file mode 100644 index 00000000..6f270136 --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/port_info/port_info.c @@ -0,0 +1,61 @@ +/* +// Copyright (c) 2019 Intel Corporation +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +*/ + +#include <stdint.h> +#include <inttypes.h> +#include <rte_eal.h> +#include <rte_ethdev.h> + +static const uint16_t rx_rings = 1, tx_rings = 1; +static const struct rte_eth_conf port_conf = { .link_speeds = ETH_LINK_SPEED_AUTONEG }; + +static inline int +port_info(void) +{ + uint8_t port_id; + int ret_val; + + RTE_ETH_FOREACH_DEV(port_id) { + ret_val = rte_eth_dev_configure(port_id, rx_rings, tx_rings, &port_conf); + if (ret_val != 0) + return ret_val; + + struct ether_addr addr; + rte_eth_macaddr_get(port_id, &addr); + printf("Port %u MAC: %02" PRIx8 ":%02" PRIx8 ":%02" PRIx8 + ":%02" PRIx8 ":%02" PRIx8 ":%02" PRIx8 "\n", + (unsigned) port_id, + addr.addr_bytes[0], addr.addr_bytes[1], + addr.addr_bytes[2], addr.addr_bytes[3], + addr.addr_bytes[4], addr.addr_bytes[5]); + } + + return 0; +} + +int +main(int argc, char *argv[]) +{ + /* Initialize the Environment Abstraction Layer (EAL). */ + int ret = rte_eal_init(argc, argv); + if (ret < 0) + rte_exit(EXIT_FAILURE, "Error with EAL initialization\n"); + + argc -= ret; + argv += ret; + + return port_info(); +} diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/rapid.pods b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid.pods new file mode 100644 index 00000000..918125ae --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/rapid.pods @@ -0,0 +1,30 @@ +## +## Copyright (c) 2019 Intel Corporation +## +## Licensed under the Apache License, Version 2.0 (the "License"); +## you may not use this file except in compliance with the License. +## You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## + +[DEFAULT] +total_number_of_pods=3 + +[POD1] +nodeSelector_hostname=k8s-node1 +dp_ip=192.168.30.11 + +[POD2] +nodeSelector_hostname=k8s-node2 +dp_ip=192.168.30.12 + +[POD3] +nodeSelector_hostname=k8s-node2 +dp_ip=192.168.30.13 diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/runrapid.py b/VNFs/DPPD-PROX/helper-scripts/rapid/runrapid.py index 124a2342..62164c4b 100755 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/runrapid.py +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/runrapid.py @@ -978,6 +978,7 @@ vmDPIP =[] vmAdminIP =[] vmDPmac =[] hexDPIP =[] +vmDPPCIDEV =[] config_file =[] prox_socket =[] prox_launch_exit =[] @@ -1011,6 +1012,7 @@ config = ConfigParser.RawConfigParser() config.read(env) machine_map = ConfigParser.RawConfigParser() machine_map.read(machine_map_file) +vim_type = config.get('Varia', 'vim') key = config.get('ssh', 'key') user = config.get('ssh', 'user') total_number_of_machines = config.get('rapid', 'total_number_of_machines') @@ -1023,6 +1025,8 @@ for vm in range(1, int(total_number_of_machines)+1): vmDPIP.append(config.get('M%d'%vm, 'dp_ip')) ip = vmDPIP[-1].split('.') hexDPIP.append(hex(int(ip[0]))[2:].zfill(2) + ' ' + hex(int(ip[1]))[2:].zfill(2) + ' ' + hex(int(ip[2]))[2:].zfill(2) + ' ' + hex(int(ip[3]))[2:].zfill(2)) + if (vim_type == "kubernetes"): + vmDPPCIDEV.append(config.get('M%d'%vm, 'dp_pci_dev')) machine_index = [] for vm in range(1, int(required_number_of_test_machines)+1): machine_index.append(int(machine_map.get('TestM%d'%vm, 'machine_index'))-1) @@ -1040,6 +1044,10 @@ for vm in range(1, int(required_number_of_test_machines)+1): f.write('name="%s"\n'% testconfig.get('TestM%d'%vm, 'name')) f.write('local_ip="%s"\n'% vmDPIP[machine_index[vm-1]]) f.write('local_hex_ip="%s"\n'% hexDPIP[machine_index[vm-1]]) + if (vim_type == "kubernetes"): + f.write("eal=\"--socket-mem=512,0 --file-prefix %s-%s-%s --pci-whitelist %s\"\n" % (env, test_file, vm, vmDPPCIDEV[machine_index[vm-1]])) + else: + f.write("eal=\"\"\n") if testconfig.has_option('TestM%d'%vm, 'cores'): cores.append(ast.literal_eval(testconfig.get('TestM%d'%vm, 'cores'))) f.write('cores="%s"\n'% ','.join(map(str, cores[-1]))) @@ -1133,42 +1141,44 @@ for vm in range(0, int(required_number_of_test_machines)): if prox_socket[vm]: clients.append(prox_ctrl(vmAdminIP[machine_index[vm]], key,user)) connect_client(clients[-1]) + if (vim_type == "OpenStack"): # Creating script to bind the right network interface to the poll mode driver - devbindfile = '{}_{}_devbindvm{}.sh'.format(env,test_file, vm+1) - with open(devbindfile, "w") as f: - newText= 'link="$(ip -o link | grep '+vmDPmac[machine_index[vm]]+' |cut -d":" -f 2)"\n' - f.write(newText) - newText= 'if [ -n "$link" ];\n' - f.write(newText) - newText= 'then\n' - f.write(newText) - newText= ' echo Need to bind\n' - f.write(newText) - newText= ' sudo ' + rundir + '/dpdk/usertools/dpdk-devbind.py --force --bind igb_uio $('+rundir+'/dpdk/usertools/dpdk-devbind.py --status |grep $link | cut -d" " -f 1)\n' - f.write(newText) - newText= 'else\n' - f.write(newText) - newText= ' echo Assuming port is already bound to DPDK\n' - f.write(newText) - newText= 'fi\n' - f.write(newText) - newText= 'exit 0\n' - f.write(newText) - st = os.stat(devbindfile) - os.chmod(devbindfile, st.st_mode | stat.S_IEXEC) - clients[-1].scp_put('./%s'%devbindfile, rundir+'/devbind.sh') - cmd = 'sudo ' + rundir+ '/devbind.sh' - clients[-1].run_cmd(cmd) - log.debug("devbind.sh running on VM%d"%(vm+1)) + devbindfile = '{}_{}_devbindvm{}.sh'.format(env,test_file, vm+1) + with open(devbindfile, "w") as f: + newText= 'link="$(ip -o link | grep '+vmDPmac[machine_index[vm]]+' |cut -d":" -f 2)"\n' + f.write(newText) + newText= 'if [ -n "$link" ];\n' + f.write(newText) + newText= 'then\n' + f.write(newText) + newText= ' echo Need to bind\n' + f.write(newText) + newText= ' sudo ' + rundir + '/dpdk/usertools/dpdk-devbind.py --force --bind igb_uio $('+rundir+'/dpdk/usertools/dpdk-devbind.py --status |grep $link | cut -d" " -f 1)\n' + f.write(newText) + newText= 'else\n' + f.write(newText) + newText= ' echo Assuming port is already bound to DPDK\n' + f.write(newText) + newText= 'fi\n' + f.write(newText) + newText= 'exit 0\n' + f.write(newText) + st = os.stat(devbindfile) + os.chmod(devbindfile, st.st_mode | stat.S_IEXEC) + clients[-1].scp_put('./%s'%devbindfile, rundir+'/devbind.sh') + cmd = 'sudo ' + rundir+ '/devbind.sh' + clients[-1].run_cmd(cmd) + log.debug("devbind.sh running on VM%d"%(vm+1)) + clients[-1].scp_put('./%s'%config_file[vm], rundir+'/%s'%config_file[vm]) clients[-1].scp_put('./{}_{}_parameters{}.lua'.format(env,test_file, vm+1), rundir + '/parameters.lua') if not configonly: if prox_launch_exit[vm]: log.debug("Starting PROX on VM%d"%(vm+1)) if auto_start[vm]: - cmd = 'sudo ' +rundir + '/prox/build/prox -t -o cli -f ' + rundir + '/%s'%config_file[vm] + cmd = 'sudo ' + rundir + '/prox -t -o cli -f ' + rundir + '/%s'%config_file[vm] else: - cmd = 'sudo ' +rundir + '/prox/build/prox -e -t -o cli -f ' + rundir + '/%s'%config_file[vm] + cmd = 'sudo ' + rundir + '/prox -e -t -o cli -f ' + rundir + '/%s'%config_file[vm] clients[-1].fork_cmd(cmd, 'PROX Testing on TestM%d'%(vm+1)) socks_control.append(prox_launch_exit[vm]) socks.append(connect_socket(clients[-1])) diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/secgw1.cfg b/VNFs/DPPD-PROX/helper-scripts/rapid/secgw1.cfg index 0ffafb1b..38f36a59 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/secgw1.cfg +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/secgw1.cfg @@ -18,12 +18,13 @@ ; This is sample ESP config. ;; +[lua] +dofile("parameters.lua") + [eal options] -n=4 ; force number of memory channels no-output=no ; disable DPDK debug output - -[lua] -dofile("parameters.lua") +eal=--proc-type auto ${eal} [port 0] name=if diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/secgw2.cfg b/VNFs/DPPD-PROX/helper-scripts/rapid/secgw2.cfg index dacbf08b..daeb18a8 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/secgw2.cfg +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/secgw2.cfg @@ -17,13 +17,15 @@ ;; ; This is sample ESP config. ;; -[eal options] --n=4 ; force number of memory channels -no-output=no ; disable DPDK debug output [lua] dofile("parameters.lua") +[eal options] +-n=4 ; force number of memory channels +no-output=no ; disable DPDK debug output +eal=--proc-type auto ${eal} + [port 0] name=if mac=hardware diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/sshclient.py b/VNFs/DPPD-PROX/helper-scripts/rapid/sshclient.py new file mode 100644 index 00000000..c781271e --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/sshclient.py @@ -0,0 +1,113 @@ +#!/usr/bin/env python2.7 + +## +## Copyright (c) 2019 Intel Corporation +## +## Licensed under the Apache License, Version 2.0 (the "License"); +## you may not use this file except in compliance with the License. +## You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## + +import paramiko +import logging + +class SSHClient: + """Wrapper class for paramiko module to connect via SSH + """ + _log = None + + _ip = None + _user = None + _rsa_private_key = None + _timeout = None + _ssh = None + _connected = False + + _output = None + _error = None + + def __init__(self, ip=None, user=None, rsa_private_key=None, timeout=15, logger_name=None): + self._ip = ip + self._user = user + self._rsa_private_key = rsa_private_key + self._timeout = timeout + + if (logger_name is not None): + self._log = logging.getLogger(logger_name) + + self._connected = False + + def set_credentials(self, ip, user, rsa_private_key): + self._ip = ip + self._user = user + self._rsa_private_key = rsa_private_key + + def connect(self): + if self._connected: + if (self._log is not None): + self._log.debug("Already connected!") + return + + if ((self._ip is None) or (self._user is None) or + (self._rsa_private_key is None)): + if (self._log is not None): + self._log.error("Wrong parameter! IP %s, user %s, RSA private key %s" + % (self._ip, self._user, self._rsa_private_key)) + self._connected = False + return + + self._ssh = paramiko.SSHClient() + self._ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) + private_key = paramiko.RSAKey.from_private_key_file(self._rsa_private_key) + + try: + self._ssh.connect(hostname = self._ip, username = self._user, pkey = private_key) + except Exception as e: + if (self._log is not None): + self._log.error("Failed to connect to the host! IP %s, user %s, RSA private key %s\n%s" + % (self._ip, self._user, self._rsa_private_key, e)) + self._connected = False + self._ssh.close() + return + + self._connected = True + + def disconnect(self): + if self._connected: + self._connected = False + self._ssh.close() + + def run_cmd(self, cmd): + self.connect() + + if self._connected is not True: + return -1 + + try: + ret = 0 + _stdin, stdout, stderr = self._ssh.exec_command(cmd, timeout = self._timeout) + self._output = stdout.read() + self._error = stderr.read() + except Exception as e: + if (self._log is not None): + self._log.error("Failed to execute command! IP %s, cmd %s\n%s" + % (self._ip, cmd, e)) + ret = -1 + + self.disconnect() + + return ret + + def get_output(self): + return self._output + + def get_error(self): + return self._error diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/start.sh b/VNFs/DPPD-PROX/helper-scripts/rapid/start.sh new file mode 100755 index 00000000..742983ec --- /dev/null +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/start.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash +## +## Copyright (c) 2010-2019 Intel Corporation +## +## Licensed under the Apache License, Version 2.0 (the "License"); +## you may not use this file except in compliance with the License. +## You may obtain a copy of the License at +## +## http://www.apache.org/licenses/LICENSE-2.0 +## +## Unless required by applicable law or agreed to in writing, software +## distributed under the License is distributed on an "AS IS" BASIS, +## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +## See the License for the specific language governing permissions and +## limitations under the License. +## + +function save_k8s_envs() +{ + printenv | grep "PCIDEVICE_INTEL_COM" > /opt/k8s_sriov_device_plugin_envs +} + +save_k8s_envs + +# Start SSH server in background +/usr/sbin/sshd + +exec sleep infinity diff --git a/VNFs/DPPD-PROX/helper-scripts/rapid/swap.cfg b/VNFs/DPPD-PROX/helper-scripts/rapid/swap.cfg index 284294e0..b72fc001 100644 --- a/VNFs/DPPD-PROX/helper-scripts/rapid/swap.cfg +++ b/VNFs/DPPD-PROX/helper-scripts/rapid/swap.cfg @@ -14,12 +14,13 @@ ;; limitations under the License. ;; +[lua] +dofile("parameters.lua") + [eal options] -n=4 ; force number of memory channels no-output=no ; disable DPDK debug output - -[lua] -dofile("parameters.lua") +eal=--proc-type auto ${eal} [port 0] name=if0 |