Improved image build & packet loss measurements

Reworked deploycentos1.sh and deploycentos2.sh which is installing
a PROX version that deals with non data plane packet counting for
improved packet loss measurements. Fixed some issues with the packer
scripts.

Change-Id: I64aab09d364ab5abbbc5fc75c73afcbe11fed681
Signed-off-by: Luc Provoost <luc.provoost@intel.com>

22 files changed, 139 insertions, 84 deletions

diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/README b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/README
index d58b4bf3..0dea6565 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/README
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/README
@@ -24,7 +24,7 @@ Copy the files in a directory on a machine that can run the OpenStack CLI
 commands and that can reach the OpenStack public network.
 
 You will need an image that has the PROX tool installed.
-A good way to do this is to use the packer tool to build an image for a target of your choice.
+The best way to do this is to use the packer tool to build an image for a target of your choice.
 You can also build this image manually by executing all the commands described in the deploycentos1.sh
 and deploycentos2.sh file.
 The default name of the qcow2 file should is rapidVM.qcow2
@@ -36,12 +36,19 @@ You should now source the proper .rc file so Packer can connect to your OpenStac
 There are 3 files: centos.json, deploycentos1.sh and deploycentos2.sh, allowing you to create
 an image automatically. Run
   # packer build centos.json
+Edit centos.json to reflect the settings of your environment: The following fields need to
+be the ID's of your system:
+  - "source_image": Needs to be the id of the Centos cloud image
+  - "flavor": Needs to be the ID of the flavor existing in your OpenStack environment that will be used
+    to start the VM in which we will install all tools
+  - "networks": ID of the network that will be used for the VM
+  - "use_floating_ip": true or false
+  - "floating_ip_pool": ID of the floating ip pool in case floating ip are being used
+  - "security_groups": ID of the security group being used
+
 Note that this procedure is not only installing the necessary tools to run PROX,
 but also does some system optimizations (tuned).
 
-Now that the image exists in OpenStack, source the openrc file of the OpenStack
-environment so that the OpenStack CLI commands can be run:
-  # source openrc
 Now you can run the createrapid.py file. Use help for more info on the usage:
   # ./createrapid.py --help
 
@@ -70,8 +77,6 @@ The actual test that is running is described in <TEST>.test.
 Notes about prox_user_data.sh script:
 - The script contains commands that will be executed using cloud-init at
   startup of the VMs.
-- The script also assumes some specific DPDK directory and tools which might
-  change over different DPDK release. This release has been tested with DPDK-17.02.
 - huge pages are allocated for DPDK on node 0 (hard-coded) in the VM.
 
 Note on using SRIOV ports:
@@ -100,7 +105,7 @@ An example of the env file generated by createrapid.py can be found below.
 Note that this file can be created manually in case the stack is created in a
 different way (not using the createrapid.py). This can be useful in case you are
 not using OpenStack as a VIM or when using special configurations that cannot be
-achieved using creater needed for runrapid are:
+achieved using createrapid.py. Fields needed for runrapid are:
 * all info in the [Mx] sections
 * the key information in the [OpenStack] section
 * the total_number_of_vms information in the [rapid] section
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/basicrapid.test b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/basicrapid.test
index f97d8217..8d73711b 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/basicrapid.test
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/basicrapid.test
@@ -26,10 +26,10 @@ group1cores = not_used
 group2cores = not_used
 group3cores = not_used
 drop_rate_treshold = 0.1
-lat_avg_treshold = 50
+lat_avg_treshold = 200
 lat_max_treshold = 1000
 accuracy = 0.1
-startspeed = 100
+startspeed = 10
 
 
 [TestM1]
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/centos.json b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/centos.json
index efab7ad3..1676f2f7 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/centos.json
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/centos.json
@@ -5,7 +5,7 @@
    {
 "type": "openstack",
 "ssh_username": "centos",
-"image_name": "ProxDPDK1802VM",
+"image_name": "PROXffc6be26-DPDK1808VM",
 "source_image": "aa0f8333-d8e1-42db-8210-6b39d9ee7267",
 "flavor": "7aaec028-eccc-4612-8f9b-29a1e039943a",
 "networks": "97bbe96f-ee1b-4789-ace8-e922f76bea3f",
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/createrapid.py b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/createrapid.py
index 8fab2b6c..06e2f232 100755
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/createrapid.py
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/createrapid.py
@@ -31,7 +31,7 @@ from logging import handlers
 from prox_ctrl import prox_ctrl
 import ConfigParser
 
-version="18.10.15"
+version="19.1.10"
 stack = "rapid" #Default string for stack. This is not an OpenStack Heat stack, just a group of VMs
 vms = "rapidVMs" #Default string for vms file
 key = "prox" # default name for kay
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/deploycentos1.sh b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/deploycentos1.sh
index ff1c1c14..c65a4833 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/deploycentos1.sh
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/deploycentos1.sh
@@ -15,7 +15,6 @@
 ## See the License for the specific language governing permissions and
 ## limitations under the License.
 ##
-sudo sh -c '(echo "export RTE_TARGET=\"build\"";echo "export RTE_SDK=\"/root/dpdk\"";echo "export AESNI_MULTI_BUFFER_LIB_PATH=\"/home/centos/intel-ipsec-mb-0.49\"";) >> /root/.bashrc'
 sudo yum install deltarpm -y
 sudo yum update -y
 sudo yum-config-manager --add-repo http://www.nasm.us/nasm.repo
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/deploycentos2.sh b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/deploycentos2.sh
index 34fc5514..9a3157f6 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/deploycentos2.sh
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/deploycentos2.sh
@@ -19,7 +19,10 @@
 # That file could contain DNS information coming from the dataplane which might be wrong. A solution is to confire the correct DNS for the dataplne
 # in OpenStack.  DNS might be slowing down ssh access. We don't need that for our dataplane benchmarking purposes
 # sudo sed -i '/#UseDNS yes/c\UseDNS no' /etc/ssh/sshd_config
-
+sudo sh -c '(echo "export RTE_TARGET=\"build\"";echo "export RTE_SDK=\"/root/dpdk\"";echo "export AESNI_MULTI_BUFFER_LIB_PATH=\"/home/centos/intel-ipsec-mb-0.50\"";) >> /root/.bashrc'
+export RTE_TARGET=build
+export RTE_SDK=/home/centos/dpdk
+export AESNI_MULTI_BUFFER_LIB_PATH=/home/centos/intel-ipsec-mb-0.50
 # Mounting huge pages to be used by DPDK
 sudo mkdir -p /mnt/huge
 sudo umount `awk '/hugetlbfs/ { print $2 }' /proc/mounts` >/dev/null 2>&1
@@ -30,18 +33,16 @@ sudo sh -c '(echo "vm.nr_hugepages = 1024") > /etc/sysctl.conf'
 cd /home/centos
 wget https://github.com/01org/intel-ipsec-mb/archive/v0.50.zip
 unzip v0.50.zip
-# AESNI_MULTI_BUFFER_LIB_PATH should be already set in deploycentos1.sh
-export  AESNI_MULTI_BUFFER_LIB_PATH=/home/centos/intel-ipsec-mb-0.50
 cd $AESNI_MULTI_BUFFER_LIB_PATH
 make
 sudo make install
 # Clone and compile DPDK
 cd /home/centos/
 git clone http://dpdk.org/git/dpdk
-cd dpdk
-git checkout v18.05
-export RTE_TARGET=build
-export RTE_SDK=/home/centos/dpdk
+# Runtime scripts are assuming /root as the directory for PROX
+sudo ln -s /home/centos/dpdk /root/dpdk
+cd $RTE_SDK
+git checkout v18.08
 make config T=x86_64-native-linuxapp-gcc
 # The next sed lines make sure that we can compile DPDK 17.11 with a relatively new OS. Using a newer DPDK (18.5) should also resolve this issue
 #sudo sed -i '/CONFIG_RTE_LIBRTE_KNI=y/c\CONFIG_RTE_LIBRTE_KNI=n' /home/centos/dpdk/build/.config
@@ -51,17 +52,13 @@ make config T=x86_64-native-linuxapp-gcc
 # Compile with MB library
 sed -i '/CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n/c\CONFIG_RTE_LIBRTE_PMD_AESNI_MB=y' /home/centos/dpdk/build/.config
 make 
-# Runtime scripts are assuming /root as the directory for PROX
-sudo ln -s /home/centos/dpdk /root/dpdk
 
 # Clone and compile PROX
 cd /home/centos
 git clone https://git.opnfv.org/samplevnf
 cd /home/centos/samplevnf/VNFs/DPPD-PROX
-git checkout 4d59d3530d1c41734f15423142e64eb9c929c717
-# Compiling PROX with the crc=soft option because offloaded CRC calculation causes problems on multiple VIM environments. This will of course slow
-# down the performance of the generator.
-make crc=soft
+git checkout ffc6be26
+make
 sudo ln -s /home/centos/samplevnf/VNFs/DPPD-PROX /root/prox
 
 # Enabling tuned with the realtime-virtual-guest profile
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/gen.cfg b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/gen.cfg
index a87ce758..fd8eadbf 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/gen.cfg
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/gen.cfg
@@ -23,6 +23,9 @@ dofile("parameters.lua")
 
 [port 0]
 name=p0
+rx desc=2048
+tx desc=2048
+vlan=yes
 
 [variables]
 $mbs=8
@@ -36,7 +39,7 @@ name=${name}
 [core 0]
 mode=master
 
-[core ${group1}]
+[core $group1]
 name=p0
 task=0
 mode=gen
@@ -49,14 +52,14 @@ pkt size=60
 local ipv4=${local_ip}
 min bulk size=$mbs
 max bulk size=16
-drop=no
+drop=yes
 lat pos=42
 accuracy pos=46
 packet id pos=50
 signature=0x6789abcd
 signature pos=56
 
-[core ${group2}]
+[core $group2]
 name=lat
 task=0
 mode=lat
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/gen_gw.cfg b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/gen_gw.cfg
index 7feaa7fd..4d5c7076 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/gen_gw.cfg
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/gen_gw.cfg
@@ -23,6 +23,9 @@ dofile("parameters.lua")
 
 [port 0]
 name=p0
+rx desc=2048
+tx desc=2048
+vlan=yes
 
 [variables]
 $mbs=8
@@ -36,7 +39,7 @@ name=${name}
 [core 0]
 mode=master
 
-[core ${group1}]
+[core $group1]
 name=p0
 task=0
 mode=gen
@@ -49,14 +52,14 @@ gateway ipv4=${gw_ip}
 local ipv4=${local_ip}
 min bulk size=$mbs
 max bulk size=16
-drop=no
+drop=yes
 lat pos=42
 accuracy pos=46
 packet id pos=50
 signature=0x6789abcd
 signature pos=56
 
-[core ${group2}]
+[core $group2]
 name=lat
 task=0
 mode=lat
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/impair.cfg b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/impair.cfg
index 8fefd41f..c41c479c 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/impair.cfg
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/impair.cfg
@@ -24,9 +24,13 @@ dofile("parameters.lua")
 [port 0]
 name=if0
 mac=hardware
+rx desc=2048
+tx desc=2048
+vlan=yes
+
 
 [defaults]
-mempool size=2K
+mempool size=4K
 
 [global]
 name=${name}
@@ -34,7 +38,7 @@ name=${name}
 [core 0]
 mode=master
 
-[core ${group1}]
+[core $group1]
 name=impair
 task=0
 mode=impair
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/impair.test b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/impair.test
index 5f75e00d..ff6624b7 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/impair.test
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/impair.test
@@ -26,8 +26,8 @@ group1cores = not_used
 group2cores = not_used
 group3cores = not_used
 drop_rate_treshold = 1
-lat_avg_treshold = 100
-lat_max_treshold = 800
+lat_avg_treshold = 1200
+lat_max_treshold = 2000
 accuracy = 0.01
 startspeed = 10
 
@@ -54,4 +54,4 @@ config_file = swap.cfg
 group1cores = [1]
 
 [test1]
-cmd=run_impairtest(socks[0],socks[2],8)
+cmd=run_impairtest(socks[0],socks[2])
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/irq.cfg b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/irq.cfg
index 3ae539c5..304312f2 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/irq.cfg
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/irq.cfg
@@ -36,7 +36,7 @@ name=${name}
 [core 0]
 mode=master
 
-[core ${group1}]
+[core $group1]
 name=irq
 task=0
 mode=irq
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/irq.test b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/irq.test
index 26ae3de3..d5ba9dff 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/irq.test
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/irq.test
@@ -26,8 +26,8 @@ group1cores = not_used
 group2cores = not_used
 group3cores = not_used
 drop_rate_treshold = 1
-lat_avg_treshold = 100
-lat_max_treshold = 800
+lat_avg_treshold = 200
+lat_max_treshold = 1000
 accuracy = 0.01
 startspeed = 10
 
@@ -35,18 +35,12 @@ startspeed = 10
 name = InterruptTesting
 machine_index = 1
 config_file = irq.cfg
-group1cores = [1,2,3,4,5,6,7]
+group1cores = [1,2,3]
 
 [TestM2]
 name = InterruptTesting
 machine_index = 2
 config_file = irq.cfg
-group1cores = [1,2,3,4,5,6,7]
-
-[TestM3]
-name = InterruptTesting
-machine_index = 3
-config_file = irq.cfg
 group1cores = [1,2,3]
 
 
@@ -54,6 +48,4 @@ group1cores = [1,2,3]
 cmd=run_irqtest(socks[0])
 [test2]
 cmd=run_irqtest(socks[1])
-[test3]
-cmd=run_irqtest(socks[2])
 
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/l2gen.cfg b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/l2gen.cfg
index 9e7bf90e..f055c6aa 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/l2gen.cfg
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/l2gen.cfg
@@ -23,6 +23,9 @@ dofile("parameters.lua")
 
 [port 0]
 name=p0
+rx desc=2048
+tx desc=2048
+vlan=yes
 
 [variables]
 $mbs=8
@@ -36,7 +39,7 @@ name=${name}
 [core 0]
 mode=master
 
-[core ${group1}]
+[core $group1]
 name=p0
 task=0
 mode=gen
@@ -48,14 +51,14 @@ pkt size=60
 local ipv4=${local_ip}
 min bulk size=$mbs
 max bulk size=16
-drop=no
+drop=yes
 lat pos=42
 accuracy pos=46
 packet id pos=50
 signature=0x6789abcd
 signature pos=56
 
-[core ${group2}]
+[core $group2]
 name=lat
 task=0
 mode=lat
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/l2swap.cfg b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/l2swap.cfg
index c02556d9..98246aba 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/l2swap.cfg
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/l2swap.cfg
@@ -24,9 +24,12 @@ dofile("parameters.lua")
 [port 0]
 name=if0
 mac=hardware
+rx desc=2048
+tx desc=2048
+vlan=yes
 
 [defaults]
-mempool size=2K
+mempool size=4K
 
 [global]
 name=${name}
@@ -34,11 +37,10 @@ name=${name}
 [core 0]
 mode=master
 
-[core ${group1}]
+[core $group1]
 name=swap
 task=0
 mode=swap
 rx port=if0
 tx port=if0
 drop=no
-
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/l2zeroloss.test b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/l2zeroloss.test
index 691af092..9049877e 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/l2zeroloss.test
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/l2zeroloss.test
@@ -16,7 +16,7 @@
 
 [DEFAULT]
 name = L2BasicSwapTesting
-number_of_tests = 1
+number_of_tests = 3
 total_number_of_test_machines = 2
 init_code=init_test()
 dest_vm = not_used
@@ -26,8 +26,8 @@ group1cores = not_used
 group2cores = not_used
 group3cores = not_used
 drop_rate_treshold = 0
-lat_avg_treshold = 100
-lat_max_treshold = 800
+lat_avg_treshold = 200
+lat_max_treshold = 1000
 accuracy = 0.1
 startspeed = 10
 
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/prox_ctrl.py b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/prox_ctrl.py
index 059cbf71..70395097 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/prox_ctrl.py
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/prox_ctrl.py
@@ -208,16 +208,18 @@ class prox_sock(object):
         return buckets
 
     def core_stats(self, cores, task=0):
-        rx = tx = drop = tsc = hz = 0
-        self._send('core stats %s %s' % (','.join(map(str, cores)), task))
+        rx = tx = drop = tsc = hz = rx_non_dp = tx_non_dp = 0
+        self._send('dp core stats %s %s' % (','.join(map(str, cores)), task))
         for core in cores:
             stats = self._recv().split(',')
             rx += int(stats[0])
             tx += int(stats[1])
-            drop += int(stats[2])
-            tsc = int(stats[3])
-            hz = int(stats[4])
-        return rx, tx, drop, tsc, hz
+            rx_non_dp += int(stats[2])
+            tx_non_dp += int(stats[3])
+            drop += int(stats[4])
+            tsc = int(stats[5])
+            hz = int(stats[6])
+        return rx-rx_non_dp, tx-tx_non_dp, drop, tsc, hz
 
     def set_random(self, cores, task, offset, mask, length):
         self._send('set random %s %s %s %s %s' % (','.join(map(str, cores)), task, offset, mask, length))
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/rapidVMs.vms b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/rapidVMs.vms
index 4c197d08..dc22a9c5 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/rapidVMs.vms
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/rapidVMs.vms
@@ -17,12 +17,13 @@
 
 [DEFAULT]
 total_number_of_vms=3
-flavor_info=--ram 4096 --disk 20 --vcpus 8
+flavor_info=--ram 4096 --disk 20 --vcpus 4
 ;flavor_meta_data=--property hw:mem_page_size=large --property hw:cpu_policy=dedicated --property hw:cpu_thread_policy=isolate --property hw:numa_nodes=1 --property hw:numa_cpus.0=0,1,2,3 --property hw:numa_mempolicy=strict --property hw:numa_mem.0=4096
 ;flavor_meta_data=--property hw:mem_page_size=large --property hw:cpu_policy=dedicated --property hw:cpu_thread_policy=isolate
 ;flavor_meta_data=--property hw:mem_page_size=large --property hw:cpu_policy=dedicated
 flavor_meta_data=--property hw:mem_page_size=large --property hw:cpu_policy=dedicated --property hw:cpu_realtime=yes --property hw:cpu_realtime_mask=^0
-boot_info=--availability-zone nova --security-group default
+;boot_info=--availability-zone nova --security-group default
+boot_info=--availability-zone L27 --security-group default
 SRIOV_port=NO
 
 [VM1]
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/runrapid.py b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/runrapid.py
index 5208698b..fdf7b8f7 100755
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/runrapid.py
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/runrapid.py
@@ -34,7 +34,7 @@ import ast
 import atexit
 import csv
 
-version="18.10.15"
+version="19.1.10"
 env = "rapid" #Default string for environment
 test = "basicrapid" #Default string for test
 loglevel="DEBUG" # sets log level for writing to file
@@ -259,15 +259,15 @@ def get_pps(speed,size):
 	return (speed * 100.0 / (8*(size+24)))
 
 def run_speedtest(gensock,sutsock):
+	maxspeed = speed = STARTSPEED
+	minspeed = 0
+	size=175
+	attempts = 0
         log.info("+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+")
-        log.info("| Generator is sending UDP (1 flow) packets (64 bytes) to SUT. SUT sends packets back                                                                                          |")
+        log.info("| Generator is sending UDP (1 flow) packets ("+ '{:>5}'.format(size+4) +" bytes) to SUT. SUT sends packets back                                                                                       |")
 	log.info("+--------+--------------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+------------+------------+")
 	log.info("| Test   |  Speed requested   | Sent to NIC    |  Sent by Gen   | Forward by SUT |  Rec. by Gen   |  Avg. Latency  |  Max. Latency  |  Packets Lost  | Loss Ratio | Result     |")
 	log.info("+--------+--------------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+------------+------------+")
-	maxspeed = speed = STARTSPEED
-	minspeed = 0
-	size=60
-	attempts = 0
 	endpps_sut_tx_str = 'NO_RESULTS'
 	gensock.set_size(gencores,0,size) # This is setting the frame size
 	gensock.set_value(gencores,0,16,(size-14),2) # 18 is the difference between the frame size and IP size = size of (MAC addresses, ethertype and FCS)
@@ -280,7 +280,7 @@ def run_speedtest(gensock,sutsock):
                 # Start generating packets at requested speed (in % of a 10Gb/s link)
                 gensock.speed(speed, gencores)
                 time.sleep(1)
-                # Get statistics now that the generation is stable and NO ARP messages any more
+                # Get statistics now that the generation is stable and initial ARP messages are dealt with.
 		pps_req_tx,pps_tx,pps_sut_tx_str,pps_rx,lat_avg,lat_max, abs_dropped, abs_tx = run_iteration(gensock,sutsock)
 		drop_rate = 100.0*abs_dropped/abs_tx
         	if ((get_pps(speed,size) - pps_tx)/get_pps(speed,size))<0.001 and ((drop_rate < DROP_RATE_TRESHOLD) or (abs_dropped==DROP_RATE_TRESHOLD ==0)) and (lat_avg< LAT_AVG_TRESHOLD) and (lat_max < LAT_MAX_TRESHOLD):
@@ -315,7 +315,7 @@ def run_speedtest(gensock,sutsock):
 				speed_prefix = bcolors.ENDC
 			else:
 				speed_prefix = bcolors.FAIL
-	                log.info('|{:>7}'.format(str(attempts))+" | " + '{:>5.1f}'.format(speed) + '% '+speed_prefix +'{:>6.3f}'.format(get_pps(speed,size)) + ' Mpps | '+ '{:>9.3f}'.format(pps_req_tx)+' Mpps | '+ bcolors.ENDC + '{:>9.3f}'.format(pps_tx) +' Mpps | ' + '{:>9}'.format(pps_sut_tx_str) +' Mpps | '+ '{:>9.3f}'.format(pps_rx)+' Mpps | '+lat_avg_prefix+ '{:>9.0f}'.format(lat_avg)+' us   | '+lat_max_prefix+ '{:>9.0f}'.format(lat_max)+' us   | '+ abs_drop_rate_prefix + '{:>14d}'.format(abs_dropped)+drop_rate_prefix+ ' |''{:>9.2f}'.format(drop_rate)+bcolors.ENDC+ '%  | FAILED     |')
+	                log.info('|{:>7}'.format(str(attempts))+" | " + '{:>5.1f}'.format(speed) + '% '+speed_prefix +'{:>6.3f}'.format(get_pps(speed,size)) + ' Mpps | '+ '{:>9.3f}'.format(pps_req_tx)+' Mpps | ' + '{:>9.3f}'.format(pps_tx) +' Mpps | '+ bcolors.ENDC  + '{:>9}'.format(pps_sut_tx_str) +' Mpps | '+ '{:>9.3f}'.format(pps_rx)+' Mpps | '+lat_avg_prefix+ '{:>9.0f}'.format(lat_avg)+' us   | '+lat_max_prefix+ '{:>9.0f}'.format(lat_max)+' us   | '+ abs_drop_rate_prefix + '{:>14d}'.format(abs_dropped)+drop_rate_prefix+ ' |''{:>9.2f}'.format(drop_rate)+bcolors.ENDC+ '%  | FAILED     |')
 			success = False 
 		speed,minspeed,maxspeed = new_speed(speed,minspeed,maxspeed,success)
 	if endpps_sut_tx_str <>  'NO_RESULTS':
@@ -340,6 +340,7 @@ def run_flowtest(gensock,sutsock):
 	gensock.set_value(gencores,0,38,(size-34),2) # 38 is the difference between the frame size and UDP size = 18 + size of IP header (=20)
 	# This will only work when using sending UDP packets. For different protocls and ehternet types, we would need a differnt calculation
 	for flow_number in sorted(flows.iterkeys()):
+		attempts = 0
 		gensock.reset_stats()
 		if sutsock!='none':
 			sutsock.reset_stats()
@@ -350,15 +351,17 @@ def run_flowtest(gensock,sutsock):
 		maxspeed = speed = STARTSPEED
 		minspeed = 0
 		while (maxspeed-minspeed > ACCURACY):
+                	attempts += 1
 			print(str(flow_number)+' flows: Measurement ongoing at speed: ' + str(round(speed,2)) + '%      ',end='\r')
 			sys.stdout.flush()
 			# Start generating packets at requested speed (in % of a 10Gb/s link)
 			gensock.speed(speed, gencores)
 			time.sleep(1)
-			# Get statistics now that the generation is stable and NO ARP messages any more
+			# Get statistics now that the generation is stable and initial ARP messages are dealt with
 			pps_req_tx,pps_tx,pps_sut_tx_str,pps_rx,lat_avg,lat_max, abs_dropped, abs_tx = run_iteration(gensock,sutsock)
 			drop_rate = 100.0*abs_dropped/abs_tx
 	        	if ((get_pps(speed,size) - pps_tx)/get_pps(speed,size))<0.001 and ((drop_rate < DROP_RATE_TRESHOLD) or (abs_dropped==DROP_RATE_TRESHOLD ==0)) and (lat_avg< LAT_AVG_TRESHOLD) and (lat_max < LAT_MAX_TRESHOLD):
+	                	log.debug('|{:>7}'.format(str(attempts))+" | " + '{:>5.1f}'.format(speed) + '% ' +'{:>6.3f}'.format(get_pps(speed,size)) + ' Mpps | '+ '{:>9.3f}'.format(pps_req_tx)+' Mpps | '+ '{:>9.3f}'.format(pps_tx) +' Mpps | ' + '{:>9}'.format(pps_sut_tx_str) +' Mpps | '+ '{:>9.3f}'.format(pps_rx)+' Mpps | '+ '{:>9.0f}'.format(lat_avg)+' us   | '+  '{:>9.0f}'.format(lat_max)+' us   | '+ '{:>14d}'.format(abs_dropped)+ ' |''{:>9.2f}'.format(drop_rate)+ '%  | SUCCESS    |')
 				endspeed = speed
 				endpps_req_tx = pps_req_tx
 				endpps_tx = pps_tx
@@ -370,6 +373,26 @@ def run_flowtest(gensock,sutsock):
 				enddrop_rate = drop_rate
 				success = True
 			else:
+				abs_drop_rate_prefix = bcolors.ENDC
+				if ((abs_dropped>0) and (DROP_RATE_TRESHOLD ==0)):
+					abs_drop_rate_prefix = bcolors.FAIL
+				if (drop_rate < DROP_RATE_TRESHOLD):
+					drop_rate_prefix = bcolors.ENDC
+				else:
+					drop_rate_prefix = bcolors.FAIL
+				if (lat_avg< LAT_AVG_TRESHOLD):
+					lat_avg_prefix = bcolors.ENDC
+				else:
+					lat_avg_prefix = bcolors.FAIL
+				if (lat_max< LAT_MAX_TRESHOLD):
+					lat_max_prefix = bcolors.ENDC
+				else:
+					lat_max_prefix = bcolors.FAIL
+				if (((get_pps(speed,size) - pps_tx)/get_pps(speed,size))<0.001):
+					speed_prefix = bcolors.ENDC
+				else:
+					speed_prefix = bcolors.FAIL
+				log.debug('|{:>7}'.format(str(attempts))+" | " + '{:>5.1f}'.format(speed) + '% '+speed_prefix +'{:>6.3f}'.format(get_pps(speed,size)) + ' Mpps | '+ '{:>9.3f}'.format(pps_req_tx)+' Mpps | ' + '{:>9.3f}'.format(pps_tx) +' Mpps | '+ bcolors.ENDC  + '{:>9}'.format(pps_sut_tx_str) +' Mpps | '+ '{:>9.3f}'.format(pps_rx)+' Mpps | '+lat_avg_prefix+ '{:>9.0f}'.format(lat_avg)+' us   | '+lat_max_prefix+ '{:>9.0f}'.format(lat_max)+' us   | '+ abs_drop_rate_prefix + '{:>14d}'.format(abs_dropped)+drop_rate_prefix+ ' |''{:>9.2f}'.format(drop_rate)+bcolors.ENDC+ '%  | FAILED     |')
 				success = False 
 			speed,minspeed,maxspeed = new_speed(speed,minspeed,maxspeed,success)
 		if endpps_sut_tx_str <>  'NO_RESULTS':
@@ -390,6 +413,7 @@ def run_sizetest(gensock,sutsock):
 #	sizes=[1020,508,252,124,60]
 #	sizes=[124,60]
 	for size in sizes:
+		attempts = 0
 		gensock.reset_stats()
 		if sutsock!='none':
 			sutsock.reset_stats()
@@ -401,14 +425,16 @@ def run_sizetest(gensock,sutsock):
 		maxspeed = speed = STARTSPEED
 		minspeed = 0
 		while (maxspeed-minspeed > ACCURACY):
+                	attempts += 1
 			print(str(size+4)+' bytes: Measurement ongoing at speed: ' + str(round(speed,2)) + '%      ',end='\r')
 			sys.stdout.flush()
 			# Start generating packets at requested speed (in % of a 10Gb/s link)
 			gensock.speed(speed, gencores)
-			# Get statistics now that the generation is stable and NO ARP messages any more
+			# Get statistics now that the generation is stable and initial ARP messages are dealt with
 			pps_req_tx,pps_tx,pps_sut_tx_str,pps_rx,lat_avg,lat_max, abs_dropped, abs_tx = run_iteration(gensock,sutsock)
 			drop_rate = 100.0*abs_dropped/abs_tx
 	        	if ((get_pps(speed,size) - pps_tx)/get_pps(speed,size))<0.001 and ((drop_rate < DROP_RATE_TRESHOLD) or (abs_dropped==DROP_RATE_TRESHOLD ==0)) and (lat_avg< LAT_AVG_TRESHOLD) and (lat_max < LAT_MAX_TRESHOLD):
+	                	log.debug('|{:>7}'.format(str(attempts))+" | " + '{:>5.1f}'.format(speed) + '% ' +'{:>6.3f}'.format(get_pps(speed,size)) + ' Mpps | '+ '{:>9.3f}'.format(pps_req_tx)+' Mpps | '+ '{:>9.3f}'.format(pps_tx) +' Mpps | ' + '{:>9}'.format(pps_sut_tx_str) +' Mpps | '+ '{:>9.3f}'.format(pps_rx)+' Mpps | '+ '{:>9.0f}'.format(lat_avg)+' us   | '+  '{:>9.0f}'.format(lat_max)+' us   | '+ '{:>14d}'.format(abs_dropped)+ ' |''{:>9.2f}'.format(drop_rate)+ '%  | SUCCESS    |')
 				endspeed = speed
 				endpps_req_tx = pps_req_tx
 				endpps_tx = pps_tx
@@ -420,6 +446,26 @@ def run_sizetest(gensock,sutsock):
 				enddrop_rate = drop_rate
 				success = True
 			else:
+				abs_drop_rate_prefix = bcolors.ENDC
+				if ((abs_dropped>0) and (DROP_RATE_TRESHOLD ==0)):
+					abs_drop_rate_prefix = bcolors.FAIL
+				if (drop_rate < DROP_RATE_TRESHOLD):
+					drop_rate_prefix = bcolors.ENDC
+				else:
+					drop_rate_prefix = bcolors.FAIL
+				if (lat_avg< LAT_AVG_TRESHOLD):
+					lat_avg_prefix = bcolors.ENDC
+				else:
+					lat_avg_prefix = bcolors.FAIL
+				if (lat_max< LAT_MAX_TRESHOLD):
+					lat_max_prefix = bcolors.ENDC
+				else:
+					lat_max_prefix = bcolors.FAIL
+				if (((get_pps(speed,size) - pps_tx)/get_pps(speed,size))<0.001):
+					speed_prefix = bcolors.ENDC
+				else:
+					speed_prefix = bcolors.FAIL
+				log.debug('|{:>7}'.format(str(attempts))+" | " + '{:>5.1f}'.format(speed) + '% '+speed_prefix +'{:>6.3f}'.format(get_pps(speed,size)) + ' Mpps | '+ '{:>9.3f}'.format(pps_req_tx)+' Mpps | ' + '{:>9.3f}'.format(pps_tx) +' Mpps | '+ bcolors.ENDC  + '{:>9}'.format(pps_sut_tx_str) +' Mpps | '+ '{:>9.3f}'.format(pps_rx)+' Mpps | '+lat_avg_prefix+ '{:>9.0f}'.format(lat_avg)+' us   | '+lat_max_prefix+ '{:>9.0f}'.format(lat_max)+' us   | '+ abs_drop_rate_prefix + '{:>14d}'.format(abs_dropped)+drop_rate_prefix+ ' |''{:>9.2f}'.format(drop_rate)+bcolors.ENDC+ '%  | FAILED     |')
 				success = False 
 			speed,minspeed,maxspeed = new_speed(speed,minspeed,maxspeed,success)
 		if endpps_sut_tx_str <>  'NO_RESULTS':
@@ -465,7 +511,7 @@ def run_irqtest(sock):
 				irq[i][j] = str(round(diff/float(runtime), 2))
 	log.info('\n'.join([''.join(['{:>12}'.format(item) for item in row]) for row in irq]))
 
-def run_impairtest(gensock,sutsock,speed):
+def run_impairtest(gensock,sutsock):
         log.info("+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+")
         log.info("| Generator is sending UDP (1 flow) packets (64 bytes) to SUT via GW dropping and delaying packets. SUT sends packets back. Use ctrl-c to stop the test           |")
 	log.info("+--------+--------------------+----------------+----------------+----------------+----------------+----------------+----------------+----------------+------------+")
@@ -477,6 +523,7 @@ def run_impairtest(gensock,sutsock,speed):
 	gensock.set_value(gencores,0,16,(size-14),2) # 18 is the difference between the frame size and IP size = size of (MAC addresses, ethertype and FCS)
 	gensock.set_value(gencores,0,38,(size-34),2) # 38 is the difference between the frame size and UDP size = 18 + size of IP header (=20)
 	# This will only work when using sending UDP packets. For different protocols and ethernet types, we would need a different calculation
+	speed = STARTSPEED
         gensock.speed(speed, gencores)
         while True:
                 attempts += 1
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/secgw.test b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/secgw.test
index cd2483ae..7cfa7e6d 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/secgw.test
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/secgw.test
@@ -26,8 +26,8 @@ group1cores = not_used
 group2cores = not_used
 group3cores = not_used
 drop_rate_treshold = 0.01
-lat_avg_treshold = 100
-lat_max_treshold = 800
+lat_avg_treshold = 200
+lat_max_treshold = 1000
 accuracy = 0.01
 startspeed = 10
 
@@ -57,6 +57,3 @@ group1cores = [1]
 
 [test1]
 cmd=run_speedtest(socks[0],socks[2])
-
-[test2]
-cmd=run_sizetest(socks[0],socks[2])
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/secgw1.cfg b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/secgw1.cfg
index 30abb8f7..7788b386 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/secgw1.cfg
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/secgw1.cfg
@@ -43,7 +43,7 @@ name=${name}
 [core 0]
 mode=master
 
-[core ${group1}]
+[core $group1]
 name=esp_enc
 task=0
 mode=esp_enc
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/secgw2.cfg b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/secgw2.cfg
index a361e875..dfaf253a 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/secgw2.cfg
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/secgw2.cfg
@@ -42,7 +42,7 @@ name=${name}
 [core 0]
 mode=master
 
-[core ${group1}]
+[core $group1]
 name=esp_dec
 task=0
 mode=esp_dec
diff --git a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/swap.cfg b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/swap.cfg
index 4229c207..2f736b16 100644
--- a/VNFs/DPPD-PROX/helper-scripts/openstackrapid/swap.cfg
+++ b/VNFs/DPPD-PROX/helper-scripts/openstackrapid/swap.cfg
@@ -34,7 +34,7 @@ name=${name}
 [core 0]
 mode=master
 
-[core ${group1}]
+[core $group1]
 name=swap
 task=0
 mode=swap