Documentation: Installation and Results for vTC test cases

Change-Id: I6b0d11255b4edfe9fe5070f880ad75ca597e1960
Signed-off-by: Vincenzo Riccobene <vincenzox.m.riccobene@intel.com>

4 files changed, 252 insertions, 39 deletions

diff --git a/docs/results/results.rst b/docs/results/results.rst
index 08acab573..f046e626b 100644
--- a/docs/results/results.rst
+++ b/docs/results/results.rst
@@ -38,6 +38,7 @@ cases suite:
    fuel-os-odl_l2-nofeature-ha.rst
    fuel-os-onos-nofeature-ha.rst
    joid-os-odl_l2-nofeature-ha.rst
+   yardstick-opnfv-vtc.rst
 
 
 Limitations
diff --git a/docs/results/yardstick-opnfv-vtc.rst b/docs/results/yardstick-opnfv-vtc.rst
index 50eb414bb..768cb69df 100644
--- a/docs/results/yardstick-opnfv-vtc.rst
+++ b/docs/results/yardstick-opnfv-vtc.rst
@@ -2,10 +2,14 @@
 .. License.
 .. http://creativecommons.org/licenses/by/4.0
 
+.. _Dashboard006: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-tc006
+.. _Dashboard007: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-tc007
+.. _Dashboard020: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-tc020
+.. _Dashboard021: http://testresults.opnfv.org/grafana/dashboard/db/yardstick-tc021
 
-===================================
-Test Results for yardstick-opnfv-ha
-===================================
+====================================
+Test Results for yardstick-opnfv-vtc
+====================================
 
 .. toctree::
    :maxdepth: 2
@@ -23,16 +27,184 @@ Overview of test results
 
 .. general on metrics collected, number of iterations
 
+The virtual Traffic Classifier (vtc) Scenario supported by Yardstick is used by 4 Test Cases:
+
+- TC006
+- TC007
+- TC020
+- TC021
+
+
+* TC006
+
+TC006 is the Virtual Traffic Classifier Data Plane Throughput Benchmarking Test.
+It collects measures about the end-to-end throughput supported by the
+virtual Traffic Classifier (vTC).
+Results of the test are shown in the Dashboard006_
+The throughput is expressed as percentage of the available bandwidth on the NIC.
+
+
+* TC007
+
+TC007 is the Virtual Traffic Classifier Data Plane Throughput Benchmarking in presence of
+noisy neighbors Test.
+It collects measures about the end-to-end throughput supported by the
+virtual Traffic Classifier when a user-defined number of noisy neighbors is deployed.
+Results of the test are shown in the Dashboard007_
+The throughput is expressed as percentage of the available bandwidth on the NIC.
+
+
+* TC020
+
+TC020 is the Virtual Traffic Classifier Instantiation Test.
+It verifies that a newly instantiated vTC is alive and functional and its instantiation
+is correctly supported by the underlying infrastructure.
+Results of the test are shown in the Dashboard020_
+
+
+* TC021
+
+TC021 is the Virtual Traffic Classifier Instantiation in presence of noisy neighbors Test.
+It verifies that a newly instantiated vTC is alive and functional and its instantiation
+is correctly supported by the underlying infrastructure when noisy neighbors are present.
+Results of the test are shown in the Dashboard021_
+
+
 Detailed test results
 ---------------------
 
-.. info on lab, installer, scenario
+* TC006
+
+The results for TC006 have been obtained using the following test case
+configuration:
+
+- Context: Dummy
+- Scenario: vtc_throughput
+- Network Techology: SR-IOV
+- vTC Flavor: m1.large
+
+
+* TC007
+
+The results for TC007 have been obtained using the following test case
+configuration:
+
+- Context: Dummy
+- Scenario: vtc_throughput_noisy
+- Network Techology: SR-IOV
+- vTC Flavor: m1.large
+- Number of noisy neighbors: 2
+- Number of cores per neighbor: 2
+- Amount of RAM per neighbor: 1G
+
+
+* TC020
+
+The results for TC020 have been obtained using the following test case
+configuration:
+
+The results listed in previous section have been obtained using the following
+test case configuration:
+
+- Context: Dummy
+- Scenario: vtc_instantiation_validation
+- Network Techology: SR-IOV
+- vTC Flavor: m1.large
+
+
+* TC021
+
+The results listed in previous section have been obtained using the following
+test case configuration:
+
+- Context: Dummy
+- Scenario: vtc_instantiation_validation
+- Network Techology: SR-IOV
+- vTC Flavor: m1.large
+- Number of noisy neighbors: 2
+- Number of cores per neighbor: 2
+- Amount of RAM per neighbor: 1G
+
+
+For all the test cases, the user can specify different values for the parameters.
 
 Rationale for decisions
 -----------------------
-.. result analysis, pass/fail
+
+* TC006
+
+The result of the test is a number between 0 and 100 which represents the percentage of bandwidth
+available on the NIC that corresponds to the supported throughput by the vTC.
+
+
+* TC007
+
+The result of the test is a number between 0 and 100 which represents the percentage of bandwidth
+available on the NIC that corresponds to the supported throughput by the vTC.
+
+* TC020
+
+The execution of the test is done as described in the following:
+
+- The vTC is deployed on the OpenStack testbed;
+- Some traffic is sent to the vTC;
+- The vTC changes the header of the packets and sends them back to the packet generator;
+- The packet generator checks that all the packets are received correctly and have been changed
+correctly by the vTC.
+
+The test is declared as PASSED if all the packets are correcly received by the packet generator
+and they have been modified by the virtual Traffic Classifier as required.
+
+
+* TC021
+
+The execution of the test is done as described in the following:
+
+- The vTC is deployed on the OpenStack testbed;
+- The noisy neighbors are deployed as requested by the user;
+- Some traffic is sent to the vTC;
+- The vTC change the header of the packets and sends them back to the packet generator;
+- The packet generator checks that all the packets are received correctly and have been changed
+correctly by the vTC
+
+The test is declared as PASSED if all the packets are correcly received by the packet generator
+and they have been modified by the virtual Traffic Classifier as required.
+
 
 Conclusions and recommendations
 -------------------------------
 
-.. did the expected behavior occured?
+* TC006
+
+The obtained results show that the virtual Traffic Classifier can support up to 4 Gbps
+(40% of the available bandwidth) correspond to the expected behaviour of the virtual
+Traffic Classifier.
+Using the configuration with SR-IOV and large flavor, the expected throughput should
+generally be in the range between 3 and 4 Gbps.
+
+
+* TC007
+
+These results correspond to the configuration in which the virtual Traffic Classifier uses SR-IOV
+Virtual Functions and the flavor is set to large for the virtual machine.
+The throughput is in the range between 2.5 Gbps and 3.7 Gbps.
+This shows that the effect of 2 noisy neighbors reduces the throughput of
+the service between 10 and 20%.
+Increasing number of neihbours would have a higher impact on the performance.
+
+
+* TC020
+
+The obtained results correspond to the expected behaviour of the virtual Traffic Classifier.
+Using the configuration with SR-IOV and large flavor, the expected result is that the vTC is
+correctly instantiated, it is able to receive and send packets using SR-IOV technology
+and to forward packets back to the packet generator changing the TCP/IP header as required.
+
+
+* TC021
+
+The obtained results correspond to the expected behaviour of the virtual Traffic Classifier.
+Using the configuration with SR-IOV and large flavor, the expected result is that the vTC is
+correctly instantiated, it is able to receive and send packets using SR-IOV technology
+and to forward packets back to the packet generator changing the TCP/IP header as required,
+also in presence of noisy neighbors.
diff --git a/docs/userguide/apexlake_installation.rst b/docs/userguide/apexlake_installation.rst
index 4870c2e21..d4493e0f8 100644
--- a/docs/userguide/apexlake_installation.rst
+++ b/docs/userguide/apexlake_installation.rst
@@ -7,6 +7,7 @@
 .. _DPDK: http://dpdk.org/doc/nics
 .. _DPDK-pktgen: https://github.com/Pktgen/Pktgen-DPDK/
 .. _SRIOV: https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking
+.. _PORTSEC: https://wiki.openstack.org/wiki/Neutron/ML2PortSecurityExtensionDriver
 .. _here: https://wiki.opnfv.org/vtc
 
 
@@ -67,19 +68,24 @@ The example provided is based on Ubuntu and needs to be executed in root mode.
     apt-get install tcpreplay
     apt-get install libpcap-dev
 
-2. Install the Framework on the Target System.
-
-After entering the Apexlake directory, run the following command.
+2. Source OpenStack openrc file.
 
 ::
 
-    python setup.py install
+    source openrc
 
-3. Source OpenStack openrc file.
+3. Configure Openstack Neutron
 
-::
+In order to support traffic generation and management by the virtual
+Traffic Classifier, the configuration of the port security driver
+extension is required for Neutron.
+
+For further details please follow the following link: PORTSEC_
+This step can be skipped in case the target OpenStack is Juno or Kilo release,
+but it is required to support Liberty.
+It is therefore required to indicate the release version in the configuration
+file located in ./yardstick/vTC/apexlake/apexlake.conf
 
-    source openrc
 
 4. Create Two Networks based on VLANs in Neutron.
 
@@ -88,33 +94,49 @@ node, two networks must be created via Neutron and mapped to the VLAN IDs
 that were previously used in the configuration of the physical switch.
 The following shows the typical set of commands required to configure Neutron
 correctly.
+The physical switches need to be configured accordingly.
 
 ::
 
-    VLAN_1=2025
-    VLAN_2=2021
+    VLAN_1=2032
+    VLAN_2=2033
+    PHYSNET=physnet2
     neutron net-create apexlake_inbound_network \
             --provider:network_type vlan \
             --provider:segmentation_id $VLAN_1 \
-            --provider:physical_network physnet1
+            --provider:physical_network $PHYSNET
 
     neutron subnet-create apexlake_inbound_network \
             192.168.0.0/24 --name apexlake_inbound_subnet
 
     neutron net-create apexlake_outbound_network \
             --provider:network_type vlan \
-            --provider:physical_network physnet1
-
-    neutron net-create apexlake_inbound_network \
-            --provider:network_type vlan \
             --provider:segmentation_id $VLAN_2 \
-            --provider:physical_network physnet1
+            --provider:physical_network $PHYSNET
 
     neutron subnet-create apexlake_outbound_network 192.168.1.0/24 \
             --name apexlake_outbound_subnet
 
 
-5. Configure the Test Cases
+5. Download Ubuntu Cloud Image and load it on Glance
+
+The virtual Traffic Classifier is supported on top of Ubuntu 14.04 cloud image.
+The image can be downloaded on the local machine and loaded on Glance
+using the following commands:
+
+::
+
+    wget cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img
+    glance image-create \
+            --name ubuntu1404 \
+            --is-public true \
+            --disk-format qcow \
+            --container-format bare \
+            --file trusty-server-cloudimg-amd64-disk1.img
+
+
+
+6. Configure the Test Cases
 
 The VLAN tags must also be included in the test case Yardstick yaml file
 as parameters for the following test cases:
@@ -216,6 +238,7 @@ The following is the list of commands required to download and install smroute.
     cd ~
     git clone https://github.com/troglobit/smcroute.git
     cd smcroute
+    git reset --hard c3f5c56
     sed -i 's/aclocal-1.11/aclocal/g' ./autogen.sh
     sed -i 's/automake-1.11/automake/g' ./autogen.sh
     ./autogen.sh
@@ -224,6 +247,7 @@ The following is the list of commands required to download and install smroute.
     sudo make install
     cd ..
 
+It is required to do the reset to the specified commit ID.
 It is also requires the creation a configuration file using the following
 command:
 
@@ -260,3 +284,17 @@ compatible NIC is required.
 NIC configuration depends on model and vendor. After proper configuration to
 support :term:`SR-IOV`, a proper configuration of OpenStack is required.
 For further information, please refer to the SRIOV_ configuration guide
+
+Finalize installation the framework on the system
+=================================================
+
+The installation of the framework on the system requires the setup of the project.
+After entering into the apexlake directory, it is sufficient to run the following
+command.
+
+::
+
+    python setup.py install
+
+Since some elements are copied into the /tmp directory (see configuration file)
+it could be necessary to repeat this step after a reboot of the host.
diff --git a/yardstick/vTC/apexlake/README.rst b/yardstick/vTC/apexlake/README.rst
index 956186a8e..06a795c12 100644
--- a/yardstick/vTC/apexlake/README.rst
+++ b/yardstick/vTC/apexlake/README.rst
@@ -46,24 +46,13 @@ The following example is provided for Ubuntu and need to be executed as root.
     apt-get install tcpreplay
     apt-get install libpcap-dev
 
-2. Install the framework on the system.
-
-The installation of the framework on the system requires the setup of the project.
-After entering into the apexlake directory, it is sufficient to run the following command.
-::
-
-    python setup.py install
-
-Since some elements are copied into the /tmp directory (see configuration file) it could be necessary
-to repeat this step after a reboot of the host.
-
-3. Source OpenStack openrc file.
+2. Source OpenStack openrc file.
 
 ::
 
     source openrc
 
-4. Configure Openstack Neutron
+3. Configure Openstack Neutron
 
 In order to support traffic generation and management by the virtual Traffic Classifier, 
 the configuration of the port security driver extension is required for Neutron.
@@ -72,7 +61,7 @@ This step can be skipped in case the target OpenStack is Juno or Kilo release,
 but it is required to support Liberty.
 It is therefore required to indicate the release version in the configuration file apexlake.conf.
 
-5. Create 2 Networks based on VLANs in Neutron.
+4. Create 2 Networks based on VLANs in Neutron.
 
 In order for the network communication between the packet generator and the Compute node to
 work fine, it is required to create through Neutron two networks and map those on the VLAN IDs
@@ -99,7 +88,7 @@ The underlying switch needs to be configured accordingly.
     neutron subnet-create apexlake_outbound_network 192.168.1.0/24 \
             --name apexlake_outbound_subnet
 
-6. Download Ubuntu Cloud Image and load it on Glance
+5. Download Ubuntu Cloud Image and load it on Glance
 
 The virtual Traffic Classifier is supported on top of Ubuntu 14.04 cloud image.
 The image can be downloaded on the local machine and loaded on Glance using the following commands:
@@ -113,7 +102,7 @@ The image can be downloaded on the local machine and loaded on Glance using the
             --container-format bare \
             --file trusty-server-cloudimg-amd64-disk1.img
 
-7. Configure the Test Cases.
+6. Configure the Test Cases.
 
 The VLAN tags are also required into the test case Yardstick yaml file as parameters the following test cases:
     - TC 006
@@ -204,7 +193,7 @@ In the following a list of commands to be ran to download and install smroute is
     sudo make install
     cd ..
 
-It is important to do the reset to the specified commit ID.
+It is required to do the reset to the specified commit ID.
 It is also required to create a configuration file using the following command:
 
     SMCROUTE_NIC=(name of the nic)
@@ -234,3 +223,16 @@ In order to enable SR-IOV interfaces on the physical NIC of the compute node, a
 NIC configuration depends on model and vendor. After proper configuration to support SR-IOV,
 a proper configuration of openstack is required.
 For further information, please look at the _SRIOV configuration guide
+
+
+Finalize installation the framework on the system
+=================================================
+
+The installation of the framework on the system requires the setup of the project.
+After entering into the apexlake directory, it is sufficient to run the following command.
+::
+
+    python setup.py install
+
+Since some elements are copied into the /tmp directory (see configuration file) it could be necessary
+to repeat this step after a reboot of the host.