Added initial draft for samplevnf useguide

JIRA: SAMPLEVNF-77
Change-Id: Ib537b36ae82361754762a627ae942d99b4966856
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

9 files changed, 921 insertions, 0 deletions

diff --git a/docs/testing/user/userguide/01-introduction.rst b/docs/testing/user/userguide/01-introduction.rst
new file mode 100755
index 00000000..bb92af6f
--- /dev/null
+++ b/docs/testing/user/userguide/01-introduction.rst
@@ -0,0 +1,62 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation and others.
+
+============
+Introduction
+============
+
+**Welcome to SampleVNF's documentation !**
+
+.. _Pharos: https://wiki.opnfv.org/pharos
+.. _SampleVNF: https://wiki.opnfv.org/samplevnf
+.. _Technical_Briefs: https://wiki.opnfv.org/display/SAM/Technical+Briefs+of+VNFs
+
+SampleVNF_ is an OPNFV Project.
+
+The project's goal is to provides a placeholder for various sample VNF
+(Virtual Network Function (:term:`VNF`)) development which includes example
+reference architecture and optimization methods related to VNF/Network service
+for high performance VNFs. This project provides benefits to other OPNFV
+projects like Functest, Models, yardstick etc to perform real life
+use-case based testing and VNF/NFVi characterization for the same.
+
+The Project's scope to create a repository of sample VNFs to help VNF
+benchmarking and NFVi characterization with real world traffic and host a
+common development environment for developing the VNF using optimized libraries.
+Also, Develop a test framework in yardstick  to enable
+Virtual Network Function (:term:`VNF`) / Network Function Virtualization Infrastructure
+(:term:`NFVI`) verification.
+
+*SampleVNF* is used in OPNFV for characterization of NFVi/VNF on OPNFV infrastructure
+and some of the OPNFV features.
+
+.. seealso:: Pharos_ for information on OPNFV community labs and this
+   Technical_Briefs_ for an overview of *SampleVNF*
+
+
+About This Document
+===================
+
+This document consists of the following chapters:
+
+* Chapter :doc:`01-introduction` provides a brief introduction to *SampleVNF*
+  project's background and describes the structure of this document.
+
+* Chapter :doc:`02-methodology` describes the methodology implemented by the
+  *SampleVNF* Project for :term:`VNF` and :term:`NFVI` verification.
+
+* Chapter :doc:`03-architecture` provides information on the software architecture
+  of *SampleVNF*.
+
+* Chapter :doc:`04-installation` provides instructions to install *SampleVNF*.
+
+* Chapter :doc:`05-BKMs` provides example on how installing and running *SampleVNF*.
+
+Contact SampleVNF
+=================
+
+Feedback? `Contact us`_
+
+.. _Contact us: opnfv-users@lists.opnfv.org
diff --git a/docs/testing/user/userguide/02-methodology.rst b/docs/testing/user/userguide/02-methodology.rst
new file mode 100644
index 00000000..9f377d8d
--- /dev/null
+++ b/docs/testing/user/userguide/02-methodology.rst
@@ -0,0 +1,85 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation and others.
+
+===========
+Methodology
+===========
+
+Abstract
+========
+
+This chapter describes the methodology/overview of SampleVNF project from
+the perspective of a :term:`VNF` and verifying the :term:`NFVI`
+
+Overview
+========
+This project provides a placeholder for various sample VNF (Virtual Network Function (:term:`VNF`))
+development which includes example reference architecture and optimization methods
+related to VNF/Network service for high performance VNFs.
+
+The sample VNFs are Open Source approximations* of Telco grade :term:`VNF`
+using optimized VNF + NFVi Infrastructure libraries, with Performance Characterization of Sample† Traffic Flows.
+• * Not a commercial product. Encourage the community to contribute and close the feature gaps.
+• † No Vendor/Proprietary Workloads 
+
+ETSI-NFV
+========
+
+.. _NFV-TST001: http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_NFV-TST001v010101p.pdf
+.. _SampleVNFtst: https://wiki.opnfv.org/display/SAM/Technical+Briefs+of+VNFs
+.. _Yardstick_NSB: http://artifacts.opnfv.org/yardstick/docs/testing_user_userguide/index.html#document-13-nsb-overview
+
+SampleVNF Test Infrastructure (NSB (Yardstick_NSB_))in yardstick helps to facilitate
+consistent/repeatable methodologies for characterizing & validating the
+sample VNFs (:term:`VNF`) through OPEN SOURCE VNF approximations.
+
+Network Service Benchmarking in yardstick framework follows ETSI GS NFV-TST001_
+to verify/characterize both :term:`NFVI` & :term:`VNF`
+
+The document ETSI GS NFV-TST001_, "Pre-deployment Testing; Report on Validation
+of NFV Environments and Services", recommends methods for pre-deployment
+testing of the functional components of an NFV environment.
+
+The SampleVNF project implements the methodology described in chapter 13 of Yardstick_NSB_,
+"Pre-deployment validation of NFV infrastructure".
+
+The methodology consists in decomposing the typical :term:`VNF` work-load
+performance metrics into a number of characteristics/performance vectors, which
+each can be represented by distinct test-cases.
+
+.. seealso:: SampleVNFtst_ for material on alignment ETSI TST001 and SampleVNF.
+
+Metrics
+=======
+
+The metrics, as defined by ETSI GS NFV-TST001, are shown in
+:ref:`Table1 <table2_1>`.
+
+.. _table2_1:
+
+**Table 1 - Performance/Speed Metrics**
+
++---------+-------------------------------------------------------------------+
+| Category| Performance/Speed                                                 |
+|         |                                                                   |
++---------+-------------------------------------------------------------------+
+| Network | * Throughput per NFVI node (frames/byte per second)               |
+|         | * Throughput provided to a VM (frames/byte per second)            |
+|         | * Latency per traffic flow                                        |
+|         | * Latency between VMs                                             |
+|         | * Latency between NFVI nodes                                      |
+|         | * Packet delay variation (jitter) between VMs                     |
+|         | * Packet delay variation (jitter) between NFVI nodes              |
+|         |                                                                   |
++---------+-------------------------------------------------------------------+
+
+.. note:: The description in this OPNFV document is intended as a reference for
+  users to understand the scope of the SampleVNF Project and the
+  deliverables of the SampleVNF framework. For complete description of
+  the methodology, please refer to the ETSI document.
+
+.. rubric:: Footnotes
+.. [1] To be included in future deliveries.
+
diff --git a/docs/testing/user/userguide/03-architecture.rst b/docs/testing/user/userguide/03-architecture.rst
new file mode 100755
index 00000000..4c4b7e61
--- /dev/null
+++ b/docs/testing/user/userguide/03-architecture.rst
@@ -0,0 +1,113 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation and others.
+
+============
+Architecture
+============
+
+Abstract
+========
+This chapter describes the samplevnf  software architecture.
+we will introduce it VNFs. More technical details will be introduced in this chapter.
+
+Overview
+========
+
+Architecture overview
+---------------------
+This project provides a placeholder for various sample VNF (Virtual Network Function)
+development which includes example reference architecture and optimization methods
+related to VNF/Network service for high performance VNFs.
+
+The sample VNFs are Open Source approximations* of Telco grade VNF’s using
+optimized VNF + NFVi Infrastructure libraries, with Performance Characterization
+of Sample† Traffic Flows.
+
+::
+
+ * Not a commercial product. Encourage the community to contribute and close the feature gaps.
+ † No Vendor/Proprietary Workloads
+
+t helps to facilitate deterministic & repeatable bench-marking on Industry
+standard high volume Servers. It augments well with a Test Infrastructure to
+help facilitate consistent/repeatable methodologies for characterizing &
+validating the sample VNFs through OPEN SOURCE VNF approximations and test tools.
+The VNFs belongs to this project are never meant for field deployment.
+All the VNF source code part of this project requires Apache License Version 2.0.
+
+Supported deployment:
+----------------------
+* Bare-Metal - All VNFs can run on a Bare-Metal DUT
+* Standalone Virtualization: All VNFs can run on SV like VPP as switch, ovs,
+  ovs-dpdk, srioc
+* Openstack: Latest Openstack supported
+
+VNF supported
+-------------
+ - Carrier Grade Network Address Translation (CG-NAT) VNF
+   ::
+      The Carrier Grade Network Address and port Translation (vCG-NAPT) is a
+      VNF approximation extending the life of the service providers IPv4 network
+      infrastructure and mitigate IPv4 address exhaustion by using address and
+      port translation in large scale. It processes the traffic in both the directions.
+      It also supports the connectivity between the IPv6 access network to
+      IPv4 data network using the IPv6 to IPv4 address translation and vice versa.
+ - Firewall (vFW) VNF
+   ::
+      The Virtual Firewall (vFW) is a VNF approximation serving as a state full
+      L3/L4 packet filter with connection tracking enabled for TCP, UDP and ICMP.
+      The VNF could be a part of Network Services (industry use-cases) deployed
+      to secure the enterprise network from un-trusted network.
+ - Access Control List (vACL) VNF
+   ::
+      The vACL vNF is implemented as a DPDK application using VNF Infrastructure
+      Library (VIL). The VIL implements common VNF internal, optimized for
+      Intel Architecture functions like load balancing between cores, IPv4/IPv6
+      stack features, and interface to NFV infrastructure like OVS or SRIOV.
+ - UDP_Replay
+   ::
+      The UDP Replay is implemented as a DPDK application using VNF Infrastructure
+      Library (VIL). Performs as a refelector of all the traffic on given port.
+ - Prox - Packet pROcessing eXecution engine.
+   ::
+      Packet pROcessing eXecution Engine (PROX) which is a DPDK application.
+      PROX can do operations on packets in a highly configurable manner.
+      The PROX application is also displaying performance statistics that can
+      be used for performance investigations. 
+      Intel® DPPD - PROX is an application built on top of DPDK which allows
+      creating software architectures, such as the one depicted below, through
+      small and readable configuration files.
+
+Test Framework
+--------------
+
+.. _Yardstick_NSB: http://artifacts.opnfv.org/yardstick/docs/testing_user_userguide/index.html#document-13-nsb-overview
+
+SampleVNF Test Infrastructure (NSB (Yardstick_NSB_))in yardstick helps to facilitate
+consistent/repeatable methodologies for characterizing & validating the
+sample VNFs (:term:`VNF`) through OPEN SOURCE VNF approximations.
+
+
+Network Service Benchmarking in yardstick framework follows ETSI GS NFV-TST001_
+to verify/characterize both :term:`NFVI` & :term:`VNF`
+
+For more inforamtion refer, 
+
+SampleVNF Directory structure
+=============================
+
+**samplevnf/** - SampleVNF main directory.
+
+*common/* - Common re-useable code like arp, nd, packet fwd etc
+
+*docs/* - All documentation is stored here, such as configuration guides,
+          user guides and SampleVNF descriptions.
+
+*tools/* - Currently contains tools to build image for VMs which are deployed
+           by Heat. Currently contains helper scripts like install, setup env 
+
+*VNFs/* - all VNF source code directory.
+
+*VNF_Catalogue/* - Collection of all  Open Source VNFs
diff --git a/docs/testing/user/userguide/04-installation.rst b/docs/testing/user/userguide/04-installation.rst
new file mode 100644
index 00000000..265f4e61
--- /dev/null
+++ b/docs/testing/user/userguide/04-installation.rst
@@ -0,0 +1,258 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation and others.
+
+SampleVNF Installation
+======================
+
+
+Abstract
+--------
+
+This project provides a placeholder for various sample VNF
+(Virtual Network Function (:term `VNF`)) development which includes example
+reference architecture and optimization methods related to VNF/Network service
+for high performance VNFs.
+The sample VNFs are Open Source approximations* of Telco grade VNF’s using
+optimized VNF + NFVi Infrastructure libraries, with Performance Characterization
+of Sample† Traffic Flows.
+
+::
+  • * Not a commercial product. Encourage the community to contribute and close the feature gaps.
+  • † No Vendor/Proprietary Workloads 
+
+SampleVNF supports installation directly in Ubuntu. The installation procedure
+are detailed in the sections below.
+
+The steps needed to run SampleVNF are:
+1. Install and Build SampleVNF.
+2. deploy the VNF on the target and modify the config based on the
+   Network under test
+3. Run the traffic generator to generate the traffic.
+
+Prerequisites
+-------------
+
+Supported Test setup:
+--------------------
+The device under test (DUT) consists of a system following;
+  * A single or dual processor and PCH chip, except for System on Chip (SoC) cases
+  * DRAM memory size and frequency (normally single DIMM per channel)
+  * Specific Intel Network Interface Cards (NICs)
+  * BIOS settings noting those that updated from the basic settings
+  * DPDK build configuration settings, and commands used for tests
+Connected to the DUT is an IXIA* or Software Traffic generator like pktgen or TRex,
+simulation platform to generate packet traffic to the DUT ports and
+determine the throughput/latency at the tester side.
+
+Below are the supported/tested (:term `VNF`) deployment type.
+.. image:: images/deploy_type.png
+   :width: 800px
+   :alt: SampleVNF supported topology
+
+Hardware & Software Ingredients
+-------------------------------
+.. code-block:: console
+   +-----------+------------------+
+   | Item      | Description      |
+   +-----------+------------------+
+   | Memory    | Min 20GB         |
+   +-----------+------------------+
+   | NICs      | 2 x 10G          |
+   +-----------+------------------+
+   | OS        | Ubuntu 16.04 LTS |
+   +-----------+------------------+
+   | kernel    |  4.4.0-34-generic|
+   +-----------+------------------+
+   |DPD        | 17.02            |
+   +-----------+------------------+
+
+   Boot and BIOS settings
+   +------------------+---------------------------------------------------+
+   | Boot settings    | default_hugepagesz=1G hugepagesz=1G hugepages=16  |
+   |                  | hugepagesz=2M hugepages=2048 isolcpus=1-11,22-33  |
+   |                  | nohz_full=1-11,22-33 rcu_nocbs=1-11,22-33         |
+   |                  | Note: nohz_full and rcu_nocbs is to disable Linux*|
+   |                  | kernel interrupts, and it’s import                |
+   +------------------+---------------------------------------------------+
+   |BIOS              | CPU Power and Performance Policy <Performance>    |
+   |                  | CPU C-state Disabled                              |
+   |                  | CPU P-state Disabled                              |
+   |                  | Enhanced Intel® Speedstep® Tech Disabled          |
+   |                  | Hyper-Threading Technology (If supported) Enable  |
+   |                  | Virtualization Techology Enable                   |
+   |                  | Coherency Enable                                  |
+   |                  | Turbo Boost Disabled                              |
+   +------------------+---------------------------------------------------+
+
+Network Topology for testing VNFs
+---------------------------------
+The ethernet cables should be connected between traffic generator and the VNF server (BM,
+SRIOV or OVS) setup based on the test profile.
+
+The connectivity could be
+1. Single port pair : One pair ports used for traffic 
+   ::
+     e.g. Single port pair link0 and link1 of VNF are used
+     TG:port 0 ------ VNF:Port 0
+     TG:port 1 ------ VNF:Port 1
+
+2. Multi port pair :  More than one pair of traffic
+   ::
+     e.g. Two port pair link 0, link1, link2 and link3 of VNF are used
+     TG:port 0 ------ VNF:Port 0
+     TG:port 1 ------ VNF:Port 1
+     TG:port 2 ------ VNF:Port 2 
+     TG:port 3 ------ VNF:Port 3
+
+ * Bare-Metal
+   Refer: http://fast.dpdk.org/doc/pdf-guides/ to setup the DUT for VNF to run 
+
+ * Standalone Virtualization - PHY-VM-PHY
+   * SRIOV
+     Refer below link to setup sriov
+     https://software.intel.com/en-us/articles/using-sr-iov-to-share-an-ethernet-port-among-multiple-vms
+
+   * OVS/OVS/DPDK
+     Refer below link to setup ovs/ovs-dpdk
+     http://docs.openvswitch.org/en/latest/intro/install/general/
+     http://docs.openvswitch.org/en/latest/intro/install/dpdk/
+
+ * Openstack
+     Use OPNFV installer to deploy the openstack.
+    
+
+Build VNFs on the DUT:
+----------------------
+ * Clone sampleVNF project repository  - git clone https://git.opnfv.org/samplevnf
+ Auto Build
+ ----------
+   * Interactive options: 
+         ::
+           ./tools/vnf_build.sh -i
+           Follow the steps in the screen from option [1] –> [9] and
+           select option [8] to build the vnfs.
+           It will automatically download selected DPDK version and any
+           required patches and will setup everything and build VNFs.
+
+           Following are the options for setup:
+           ----------------------------------------------------------
+            Step 1: Environment setup.
+            ----------------------------------------------------------
+            [1] Check OS and network connection
+            [2] Select DPDK RTE version
+
+          ----------------------------------------------------------
+           Step 2: Download and Install
+           ----------------------------------------------------------
+           [3] Agree to download
+           [4] Download packages
+           [5] Download DPDK zip
+           [6] Build and Install DPDK
+           [7] Setup hugepages
+
+           ----------------------------------------------------------
+            Step 3: Build VNFs
+            ----------------------------------------------------------
+            [8] Build all VNFs (vACL, vCGNAPT, vFW, UDP_Replay, DPPD-PROX)
+
+            [9] Exit Script
+    * non-Interactive options:
+          ::
+            ./tools/vnf_build.sh -s -d=<dpdk version eg 17.02>
+Manual Build
+------------
+   ::
+      1.Download DPDK supported version from dpdk.org
+        http://dpdk.org/browse/dpdk/snapshot/dpdk-$DPDK_RTE_VER.zip
+        unzip dpdk-$DPDK_RTE_VER.zip and apply dpdk patches only in case of 16.04 (Not required for other DPDK versions)
+        cd dpdk
+        make config T=x86_64-native-linuxapp-gcc O=x86_64-native-linuxapp-gcc
+        cd x86_64-native-linuxapp-gcc
+        make -j
+      2.Setup huge pages
+        For 1G/2M hugepage sizes, for example 1G pages, the size must be specified
+        explicitly and can also be optionally set as the default hugepage size
+        for the system. For example, to reserve 8G of hugepage memory in the form
+        of eight 1G pages, the following options should be passed to the
+        kernel: * default_hugepagesz=1G hugepagesz=1G hugepages=8 hugepagesz=2M hugepages=2048
+      3.Add this to Go to /etc/default/grub configuration file.
+        Append “default_hugepagesz=1G hugepagesz=1G hugepages=8 hugepagesz=2M hugepages=2048”to the GRUB_CMDLINE_LINUX entry.
+      4.Setup Environment Variable
+        export RTE_SDK=<samplevnf>/dpdk
+        export RTE_TARGET=x86_64-native-linuxapp-gcc
+        export VNF_CORE=<samplevnf>
+        or using ./tools/setenv.sh
+      5.Build vACL VNFs
+        cd <samplevnf>/VNFs/vACL
+        make clean
+        make
+        The vACL executable will be created at the following location
+        <samplevnf>/VNFs/vACL/build/vACL
+
+Standalone virtualization/Openstack:
+ ::
+  * Build image from yardstick
+    git clone https://git.opnfv.org/yardstick
+  * cd yardstick and run
+    ./tools/yardstick-img-modify tools/ubuntu-server-cloudimg-samplevnf-modify.sh
+
+Modify Scripts as per Traffic Generator Settings
+---------------------------------------------------------
+
+e.g: vFW
+Modify the configuration according to system test configuration.
+
+traffic_type selction in config file:
+The traffic_type parmeter should be set to 4 (IPv4) or 6 (IPv6)
+traffic type.
+
+There are many other vFW parameters which can be changed
+in the config file for simulating different traffic conditions like
+timeouts.
+
+Modify the scripts according to system test configuration.
+
+The routeadd and arpadd settings should be updated as per the
+traffic generator settings.
+
+; routeadd <port #> <ipv4 nhip address in decimal> <Mask>
+routeadd 0 202.16.100.20 0xff000000
+routeadd 1 172.16.40.20 0xff000000
+
+;routeadd <port #> <ipv6 nhip address in hex> <Depth>
+;routeadd 0 fec0::6a05:caff:fe30:21b0 64
+;routeadd 1 2012::6a05:caff:fe30:2081 64
+
+; IPv4 static ARP
+;p 1 arpadd 1 172.16.40.20 00:00:00:00:00:04
+;p 1 arpadd 0 202.16.100.20 00:00:00:00:00:01
+
+; IPv6 static ARP
+;p 1 arpadd 0 fec0::6a05:caff:fe30:21b0 00:00:00:00:00:01
+;p 1 arpadd 1 2012::6a05:caff:fe30:2081 00:00:00:00:00:04
+
+The vFW supports  enabling/disabling of multiple features
+like firewall, conntrack, synproxy and debug.
+Thease features can be enabled/disabled through scripts or
+CLI commands.
+
+
+ACL rules can be modified based on the test scenarios.
+The default rules are provided for reference.
+
+
+Commands to run vFW
+----------------------------
+::
+  SW_LoadB
+  ./vFW -p <port mask> -f <config> -s <script>
+ 
+  HW_LoadB
+  ./vFW -p <port mask> -f <config> -s <script> -hwlb <num_WT>
+
+
+Proxy Support (**Todo**)
+---------------------------
+
diff --git a/docs/testing/user/userguide/05-BKMs.rst b/docs/testing/user/userguide/05-BKMs.rst
new file mode 100644
index 00000000..289bdad2
--- /dev/null
+++ b/docs/testing/user/userguide/05-BKMs.rst
@@ -0,0 +1,277 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation and others.
+
+SampleVNF BKMs - Example how to run VNF vFW
+============================================
+
+
+Abstract
+--------
+
+This project provides a placeholder for various sample VNF
+(Virtual Network Function (:term `VNF`)) development which includes example
+reference architecture and optimization methods related to VNF/Network service
+for high performance VNFs.
+The sample VNFs are Open Source approximations* of Telco grade VNF’s using
+optimized VNF + NFVi Infrastructure libraries, with Performance Characterization
+of Sample† Traffic Flows.
+
+::
+  • * Not a commercial product. Encourage the community to contribute and close the feature gaps.
+  • † No Vendor/Proprietary Workloads 
+
+SampleVNF supports installation directly in Ubuntu. The installation procedure
+are detailed in the sections below.
+
+The steps needed to run SampleVNF are:
+1. Install and Build SampleVNF.
+2. deploy the VNF on the target and modify the config based on the
+   Network under test
+3. Run the traffic generator to generate the traffic.
+
+Prerequisites
+-------------
+
+Supported Test setup:
+--------------------
+The device under test (DUT) consists of a system following;
+  * A single or dual processor and PCH chip, except for System on Chip (SoC) cases
+  * DRAM memory size and frequency (normally single DIMM per channel)
+  * Specific Intel Network Interface Cards (NICs)
+  * BIOS settings noting those that updated from the basic settings
+  * DPDK build configuration settings, and commands used for tests
+Connected to the DUT is an IXIA* or Software Traffic generator like pktgen or TRex,
+simulation platform to generate packet traffic to the DUT ports and
+determine the throughput/latency at the tester side.
+
+Below are the supported/tested (:term `VNF`) deployment type.
+.. image:: images/deploy_type.png
+   :width: 800px
+   :alt: SampleVNF supported topology
+
+Hardware & Software Ingredients
+-------------------------------
+.. code-block:: console
+   +-----------+------------------+
+   | Item      | Description      |
+   +-----------+------------------+
+   | Memory    | Min 20GB         |
+   +-----------+------------------+
+   | NICs      | 2 x 10G          |
+   +-----------+------------------+
+   | OS        | Ubuntu 16.04 LTS |
+   +-----------+------------------+
+   | kernel    |  4.4.0-34-generic|
+   +-----------+------------------+
+   |DPD        | 17.02            |
+   +-----------+------------------+
+
+   Boot and BIOS settings
+   +------------------+---------------------------------------------------+
+   | Boot settings    | default_hugepagesz=1G hugepagesz=1G hugepages=16  |
+   |                  | hugepagesz=2M hugepages=2048 isolcpus=1-11,22-33  |
+   |                  | nohz_full=1-11,22-33 rcu_nocbs=1-11,22-33         |
+   |                  | Note: nohz_full and rcu_nocbs is to disable Linux*|
+   |                  | kernel interrupts, and it’s import                |
+   +------------------+---------------------------------------------------+
+   |BIOS              | CPU Power and Performance Policy <Performance>    |
+   |                  | CPU C-state Disabled                              |
+   |                  | CPU P-state Disabled                              |
+   |                  | Enhanced Intel® Speedstep® Tech Disabled          |
+   |                  | Hyper-Threading Technology (If supported) Enable  |
+   |                  | Virtualization Techology Enable                   |
+   |                  | Coherency Enable                                  |
+   |                  | Turbo Boost Disabled                              |
+   +------------------+---------------------------------------------------+
+
+Network Topology for testing VNFs
+---------------------------------
+The ethernet cables should be connected between traffic generator and the VNF server (BM,
+SRIOV or OVS) setup based on the test profile.
+
+The connectivity could be
+1. Single port pair : One pair ports used for traffic 
+   ::
+     e.g. Single port pair link0 and link1 of VNF are used
+     TG:port 0 ------ VNF:Port 0
+     TG:port 1 ------ VNF:Port 1
+
+2. Multi port pair :  More than one pair of traffic
+   ::
+     e.g. Two port pair link 0, link1, link2 and link3 of VNF are used
+     TG:port 0 ------ VNF:Port 0
+     TG:port 1 ------ VNF:Port 1
+     TG:port 2 ------ VNF:Port 2 
+     TG:port 3 ------ VNF:Port 3
+
+ * Bare-Metal
+   Refer: http://fast.dpdk.org/doc/pdf-guides/ to setup the DUT for VNF to run 
+
+ * Stadalone Virtualization - PHY-VM-PHY
+   * SRIOV
+     Refer below link to setup sriov
+     https://software.intel.com/en-us/articles/using-sr-iov-to-share-an-ethernet-port-among-multiple-vms
+
+   * OVS/OVS/DPDK
+     Refer below link to setup ovs/ovs-dpdk
+     http://docs.openvswitch.org/en/latest/intro/install/general/
+     http://docs.openvswitch.org/en/latest/intro/install/dpdk/
+
+ * Openstack
+     use OPNFV installer to deploy the openstack.
+    
+Traffic generator and VNF Setup details
+----------------------------------------
+
+step 0:  Preparing hardware connection.
+         Connect Traffic generator and VNF system back to back as shown in previous section e.g. Bare-Metal Configuration
+         TRex port 0 ↔ (VNF Port 0) ↔ (VNF Port 1) ↔ TRex port 1
+
+step 1: Setting up Traffic generator (TRex) (Refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html)
+        TRex Software preparations
+        --------------------------
+        a. Install the OS (Bare metal Linux, not VM!)
+        b. Obtain the latest TRex package: wget https://trex-tgn.cisco.com/trex/release/latest
+        c. Untar the package: tar -xzf latest
+        d. Change dir to unzipped TRex
+        e. Create config file using command: sudo python dpdk_setup_ports.py -i
+           In case of Ubuntu 16 need python3
+           See paragraph config creation for detailed step-by-step
+step 2: Setting up VNF
+
+        Deployment type - Bare-Metal:
+        ----------------------------
+        a. Clone sampleVNF project repository  - git clone https://git.opnfv.org/samplevnf
+        Auto Build
+          * Interactive options: 
+                ./tools/vnf_build.sh -i
+                Follow the steps in the screen from option [1] –> [9] and select option [8] to build the vnfs. It will automatically download selected DPDK version and any required patches and will setup everything and build VNFs.
+                Following are the options for setup:
+                ----------------------------------------------------------
+                Step 1: Environment setup.
+                ----------------------------------------------------------
+                [1] Check OS and network connection
+                [2] Select DPDK RTE version
+
+                ----------------------------------------------------------
+                Step 2: Download and Install
+                ----------------------------------------------------------
+                [3] Agree to download
+                [4] Download packages
+                [5] Download DPDK zip
+                [6] Build and Install DPDK
+                [7] Setup hugepages
+
+                ----------------------------------------------------------
+                Step 3: Build VNFs
+                ----------------------------------------------------------
+                [8] Build all VNFs (vACL, vCGNAPT, vFW, UDP_Replay, DPPD-PROX)
+
+                [9] Exit Script
+          * non-Interactive options:
+                ./tools/vnf_build.sh -s -d=<dpdk version eg 17.02>
+        Manual Build
+           1. Download DPDK supported version from dpdk.org
+              http://dpdk.org/browse/dpdk/snapshot/dpdk-$DPDK_RTE_VER.zip
+              unzip dpdk-$DPDK_RTE_VER.zip and apply dpdk patches only in case of 16.04 (Not required for other DPDK versions)
+              cd dpdk
+              make config T=x86_64-native-linuxapp-gcc O=x86_64-native-linuxapp-gcc
+              cd x86_64-native-linuxapp-gcc
+              make
+           2. Setup huge pages
+              For 1G/2M hugepage sizes, for example 1G pages, the size must be
+              specified explicitly and can also be optionally set as the
+              default hugepage size for the system. For example, to reserve 8G
+              of hugepage memory in the form of eight 1G pages, the following
+              options should be passed to the kernel: * default_hugepagesz=1G
+              hugepagesz=1G hugepages=8 hugepagesz=2M hugepages=2048
+           3. Add this to Go to /etc/default/grub configuration file.
+              Append “default_hugepagesz=1G hugepagesz=1G hugepages=8 hugepagesz=2M hugepages=2048”
+              to the GRUB_CMDLINE_LINUX entry.
+           4. Setup Environment Variable
+              export RTE_SDK=<samplevnf>/dpdk
+              export RTE_TARGET=x86_64-native-linuxapp-gcc
+              export VNF_CORE=<samplevnf>
+              or using ./tools/setenv.sh
+           5. Build vFW VNFs
+              cd <samplevnf>/VNFs/vFW
+              make clean
+              make
+              The vFW executable will be created at the following location
+              <samplevnf>/VNFs/vFW/build/vFW
+
+step 3: Running VNF
+  a. Setup Port to run VNF
+        For DPDK versions 17.xx
+        1. cd <samplevnf>/dpdk
+        2. ./usertools/dpdk-devbind.py --status <--- List the network device
+        3. ./usertools/dpdk-devbind.py -b igb_uio <PCI Port 0> <PCI Port 1>
+        .. _More details: http://dpdk.org/doc/guides-17.05/linux_gsg/build_dpdk.html#binding-and-unbinding-network-ports-to-from-the-kernel-modules 
+  b. Prepare script to enalble VNF to route the packets
+        ::
+          cd <samplevnf>/VNFs/vFW/config
+          Open -> VFW_SWLB_SinglePortPair_script.tc. Replace the bold items based on your setting.
+
+           link 0 config <VNF port 0 IP eg 202.16.100.10> 8
+           link 0 up
+           link 1 down
+           link 1 config <VNF port 0 IP eg 172.16.40.10> 8
+           link 1 up
+           ; routeadd <port #> <ipv4 nhip address in decimal> <Mask>
+           routeadd 0 <traffic generator port 0 IP eg 202.16.100.20> 0xff000000
+           routeadd 1 <traffic generator port 1 IP eg 172.16.40.20> 0xff000000
+
+           ; IPv4 static ARP; disable if dynamic arp is enabled.
+           p 1 arpadd 0 <traffic generator port 0 IP eg 202.16.100.20> <traffic generator port 0 MAC>
+           p 1 arpadd 1  <traffic generator port 1 IP eg 172.16.40.20> <traffic generator port 1 MAC>
+           p action add 0 accept
+           p action add 0 fwd 0
+           p action add 0 count
+           p action add 1 accept
+           p action add 1 fwd 1
+           p action add 1 count
+           p action add 2 drop
+           p action add 2 count
+           p action add 0 conntrack
+           p action add 1 conntrack
+           p action add 2 conntrack
+           p action add 3 conntrack
+           ; IPv4 rules
+           p vfw add 1 <traffic generator port 0 IP eg 202.16.100.20> 8 <traffic generator port 1 IP eg 172.16.40.20> 8 0 65535 67 69 0 0 2
+           p vfw add 2 <traffic generator port 0 IP eg 202.16.100.20> 8 <traffic generator port 1 IP eg 172.16.40.20> 8 0 65535 0 65535 0 0 1
+           p vfw add 2 <traffic generator port 1 IP eg 172.16.40.20> 8 <traffic generator port 0 IP eg 202.16.100.20> 8 0 65535 0 65535 0 0 0
+           p vfw applyruleset
+   c. Run below cmd to launch the VNF. Please make sure both hugepages and ports to be used are bind to dpdk.
+      ::
+        cd <samplevnf>/VNFs/vFW/
+        ./build/vFW -p 0x3 -f ./config/VFW_SWLB_SinglePortPair_4Thread.cfg  -s ./config/VFW_SWLB_SinglePortPair_script.tc
+
+step 4: Run Test using traffic geneator
+
+On traffic generator system:
+        cd <trex eg v2.28/stl>
+        Update the bench.py to generate the traffic. 
+
+        class STLBench(object):
+        ip_range = {}
+        ip_range['src'] = {'start': '<traffic generator port 0 IP eg 202.16.100.20>', 'end': '<traffic generator port 0 IP eg 202.16.100.20>'}
+        ip_range['dst'] = {'start': '<traffic generator port 1 IP eg 172.16.40.20>', 'end': '<traffic generator port 1 IP eg 172.16.40.20>'}
+        cd <trex eg v2.28>
+        Run the TRex server: sudo ./t-rex-64 -i -c 7
+        In another shell run TRex console: trex-console
+        The console can be run from another computer with -s argument, --help for more info.
+        Other options for TRex client are automation or GUI
+        In the console, run "tui" command, and then send the traffic with commands like:
+        start -f stl/bench.py -m 50% --port 0 3 -t size=590,vm=var1
+        For more details refer: https://trex-tgn.cisco.com/trex/doc/trex_stateless_bench.html 
+
+Deployment type Standalone Virtualization/Openstack:
+Step 0: Setup the NFVi Infrastrucutre. (Refer installation section)
+Step 1: Build SampleVNF enabled VM
+  * Build image from yardstick
+    git clone https://git.opnfv.org/yardstick
+  * cd yardstick and run
+    ./tools/yardstick-img-modify tools/ubuntu-server-cloudimg-samplevnf-modify.sh
+Step 3: Follow steps 0 to 4 in above section to run the VNF.
diff --git a/docs/testing/user/userguide/glossary.rst b/docs/testing/user/userguide/glossary.rst
new file mode 100644
index 00000000..e54794ee
--- /dev/null
+++ b/docs/testing/user/userguide/glossary.rst
@@ -0,0 +1,62 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation and others.
+
+========
+Glossary
+========
+
+.. glossary::
+   :sorted:
+
+   TRex
+     Realistic traffic generator
+
+   API
+     Application Programming Interface
+
+   DPI
+     Deep Packet Inspection
+
+   DPDK
+     Data Plane Development Kit
+
+   NIC
+     Network Interface Controller
+
+   BNG
+     Broadband Network Gateway
+
+   VM
+     Virtual Machine
+
+   VNF
+     Virtual Network Function
+
+   VNFC
+     Virtual Network Function Component
+
+   NFVI
+     Network Function Virtualization Infrastructure
+
+   SR-IOV
+     Single Root IO Virtualization
+
+   SUT
+     System Under Test
+
+   ToS
+     Type of Service
+
+   vFW
+     Virtual Firewall
+
+   vCGNAPT
+     Virtual Carrier Grade Network Address and port Translation
+
+   vACL
+     Virtual Access Control List
+
+   PROX
+     Packet pROcessing eXecution engine
diff --git a/docs/testing/user/userguide/images/deploy_type.png b/docs/testing/user/userguide/images/deploy_type.png
new file mode 100644
index 00000000..9f5aad8b
--- /dev/null
+++ b/docs/testing/user/userguide/images/deploy_type.png
diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst
index 1998be3b..e21f1faa 100644
--- a/docs/testing/user/userguide/index.rst
+++ b/docs/testing/user/userguide/index.rst
@@ -3,3 +3,19 @@
 .. (c) OPNFV, Intel Corporation and others.
 
 .. OPNFV SAMPLEVNF Documentation master file.
+
+===========================================
+SampleVNF User Guide
+===========================================
+
+.. toctree::
+   :maxdepth: 4
+   :numbered:
+
+   01-introduction
+   02-methodology
+   03-architecture
+   04-installation
+   05-BKMs
+   glossary
+   references
diff --git a/docs/testing/user/userguide/references.rst b/docs/testing/user/userguide/references.rst
new file mode 100644
index 00000000..30f6e604
--- /dev/null
+++ b/docs/testing/user/userguide/references.rst
@@ -0,0 +1,48 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International
+.. License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation and others.
+
+==========
+References
+==========
+
+
+OPNFV
+=====
+
+* Yardstick wiki: https://wiki.opnfv.org/yardstick
+* SampleVNF wiki: https://wiki.opnfv.org/samplevnf
+
+References used in Test Cases
+=============================
+
+* TRex: https://trex-tgn.cisco.com/
+* DPDKpktgen: https://github.com/Pktgen/Pktgen-DPDK/
+* DPDK: http://dpdk.org
+* DPDK supported NICs: http://dpdk.org/doc/nics
+* fdisk: http://www.tldp.org/HOWTO/Partition/fdisk_partitioning.html
+* fio: http://www.bluestop.org/fio/HOWTO.txt
+* free: http://manpages.ubuntu.com/manpages/trusty/en/man1/free.1.html
+* iperf3: https://iperf.fr/
+* Lmbench man-pages: http://manpages.ubuntu.com/manpages/trusty/lat_mem_rd.8.html
+* Memory bandwidth man-pages: http://manpages.ubuntu.com/manpages/trusty/bw_mem.8.html
+* mpstat man-pages: http://manpages.ubuntu.com/manpages/trusty/man1/mpstat.1.html
+* pktgen: https://www.kernel.org/doc/Documentation/networking/pktgen.txt
+* RAMspeed: http://alasir.com/software/ramspeed/
+* SR-IOV: https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking
+* Storperf: https://wiki.opnfv.org/display/storperf/Storperf
+* unixbench: https://github.com/kdlucas/byte-unixbench/blob/master/UnixBench
+
+
+Research
+========
+*
+
+Standards
+=========
+
+* ETSI NFV: http://www.etsi.org/technologies-clusters/technologies/nfv
+* ETSI GS-NFV TST 001: http://www.etsi.org/deliver/etsi_gs/NFV-TST/001_099/001/01.01.01_60/gs_NFV-TST001v010101p.pdf
+* RFC2544: https://www.ietf.org/rfc/rfc2544.txt
+