Added developer user guide draft for samplevnfs

Change-Id: I15a5f7ae8e3aac5d509b7a722f7032fbd2da69d0
Signed-off-by: Deepak S <deepak.s@linux.intel.com>

6 files changed, 416 insertions, 10 deletions

diff --git a/docs/testing/developer/design/01-Overview.rst b/docs/testing/developer/design/01-Overview.rst
new file mode 100644
index 00000000..e4c7a984
--- /dev/null
+++ b/docs/testing/developer/design/01-Overview.rst
@@ -0,0 +1,51 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation and others.
+
+.. OPNFV SAMPLEVNF Documentation design file.
+
+============
+Introduction
+============
+
+**Welcome to SampleVNF's developer getstarted documentation !**
+
+.. _Pharos: https://wiki.opnfv.org/pharos
+.. _SampleVNF: https://wiki.opnfv.org/samplevnf
+.. _Technical_Briefs: https://wiki.opnfv.org/display/SAM/Technical+Briefs+of+VNFs
+
+Overview:
+---------
+
+This project provides a placeholder for various sample VNF (Virtual Network Function)
+development which includes example reference architecture and optimization methods
+related to VNF/Network service for high performance VNFs. This project provides
+benefits to other OPNFV projects like Functest, Models, yardstick etc to perform
+real life use-case based testing and NFVi characterization for the same.
+
+The sample VNFs are Open Source approximations* of Telco grade VNF’s using optimized
+VNF + NFVi Infrastructure libraries, with Performance Characterization of Sample† Traffic Flows.
+  * * Not a commercial product. Encourage the community to contribute and close the feature gaps.
+  * † No Vendor/Proprietary Workloads 
+
+It helps to facilitate deterministic & repeatable bench-marking on
+Industry standard high volume Servers. It augments well with a Test Infrastructure
+to help facilitate consistent/repeatable methodologies for characterizing & validating
+the sample VNFs through OPEN SOURCE VNF approximations and test tools.
+The VNFs belongs to this project are never meant for field deployment.
+All the VNF source code part of this project requires Apache License Version 2.0.
+
+Scope:
+-----
+The Scope of samplevnf project as follows"
+To create a repository of sample VNFs to help VNF benchmarking and NFVi
+characterization with real world traffic.
+Host a common development environment for developing the VNF using optimized libraries
+Integrate into CI tool chain and existing test frameworks for VNF feature and deployment testing
+
+Testability:
+-----------
+Network Service Testing framework added into the Yardstick will be used as a test
+tool for Functional/Performance verification of all the sample VNFs. 
+Planning to extend the same to FuncTest and Models project to include the testcases
+related to sample VNFs.
diff --git a/docs/testing/developer/design/02-Get_started_Guide.rst b/docs/testing/developer/design/02-Get_started_Guide.rst
new file mode 100644
index 00000000..4bcff899
--- /dev/null
+++ b/docs/testing/developer/design/02-Get_started_Guide.rst
@@ -0,0 +1,154 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation and others.
+
+.. OPNFV SAMPLEVNF Documentation design file.
+
+====================================
+Get started as a SampleVNF developer
+===================================
+
+.. _SampleVNF: https://wiki.opnfv.org/samplevnf
+.. _Gerrit: https://www.gerritcodereview.com/
+.. _JIRA: https://jira.opnfv.org
+.. _Technical_Briefs: https://wiki.opnfv.org/display/SAM/Technical+Briefs+of+VNFs
+
+Prerequisite knowledge
+-----------------------
+Development/Contribution to SampleVNF requires knowledge of networking technologies including
+knowledge of network protocols and hands-on experience with relevant open-source
+software, such as Linux*, SDN, NFVI and the DPDK (if VNF is based on DPDK libraries).
+Developer needs debugging and benchmarking skils, as well as understanding of NFVi
+infrastructure across multiple domains.
+
+There are many ways to contribute to samplevnf.
+ * Develop new test cases in samplevnf
+ * Review code changes
+ * Develop/contribute to existing VNFs or new VNFs
+ * Write samplevnf documentation
+
+Techical Briefs of existsin VNFs in Technical_Briefs_
+
+
+Get Started:
+----------
+Where can I find some help to start?
+You can also directly contact us by mail with [SampleVNF] prefix in the title at
+opnfv-tech-discuss@lists.opnfv.org or on the IRC chan #opnfv-samplevnf.
+
+How TOs
+How can I contribute to SampleVNF?
+If you are already a contributor of any OPNFV project, you can contribute to samplevnf.
+If you are totally new to OPNFV, you must first create your Linux Foundation account,
+then contact us in order to declare you in the repository database.
+
+We distinguish 2 levels of contributors:
+the standard contributor can push patch and vote +1/0/-1 on any samplevnf patch
+The commitor can vote -2/-1/0/+1/+2 and merge
+SampleVNF commitors are promoted by the samplevnf contributors.
+
+Gerrit & JIRA
+OPNFV uses Gerrit_ for web based code review and repository management for the
+Git Version Control System. You can access OPNFV Gerrit from this link.
+Please note that you need to have Linux Foundation ID in order to use OPNFV Gerrit.
+You can get one from this link.
+
+OPNFV uses JIRA_ for issue management. An important principle of change management
+is to have two-way trace-ability between issue management (i.e. JIRA_) and the code repository (via Gerrit).
+In this way, individual commits can be traced to JIRA issues and we also know which
+commits were used to resolve a JIRA issue.
+If you want to contribute to samplevnf, you can pick a issue from SampleVNF's
+JIRA dashboard or you can create you own issue and submit it to JIRA.
+ 
+Submitting code to Gerrit
+Installing and configuring Git and Git-Review is necessary in order to submit code to Gerrit.
+The Getting to the code page will provide you with some help for that.
+ 
+Comitting the code with Git
+Open a terminal window and set the project's directory to the working directory using the cd command.
+In this case "/home/opnfv/samplevnf" is the path to the samplevnf project folder on my computer.
+Replace this with the path of your own project.
+
+::
+  cd /home/opnfv/samplevnf
+
+Tell Git which files you would like to take into account for the next commit.
+This is called 'staging' the files, by placing them into the staging area,
+using the 'git add' command (or the synonym 'git stage' command).
+
+::
+  git add samplevnf/samples/sample.yaml
+  ...
+
+Alternatively, you can choose to stage all files that have been modified
+(that is the files you have worked on) since the last time you generated a commit, by using the -a argument.
+
+::
+  git add -a
+
+Git won't let you push (upload) any code to Gerrit if you haven't pulled the latest changes first.
+So the next step is to pull (download) the latest changes made to the project by other collaborators using the 'pull' command.
+
+::
+  git pull
+
+Now that you have the latest version of the project and you have staged the files you wish to push,
+it is time to actually commit your work to your local Git repository.
+
+::
+  git commit --signoff -m "Title of change
+     
+  Test of change that describes in high level what
+  was done. There is a lot of documentation in code
+  so you do not need to repeat it here.
+     
+  JIRA: SAMPLEVNF-XXX"
+
+The message that is required for the commit should follow a specific set of rules.
+This practice allows to standardize the description messages attached to the commits,
+and eventually navigate among the latter more easily.
+
+Verify your patch locally before submitting
+Once you finish a patch, you can submit it to Gerrit for code review.
+A developer sends a new patch to Gerrit will trigger patch verify job on Jenkins CI.
+
+Pushing the code to Gerrit for review
+Now that the code has been comitted into your local Git repository the following
+step is to push it online to Gerrit for it to be reviewed. The command we will use is 'git review'.
+
+::
+  git review
+This will automatically push your local commit into Gerrit.
+
+Code review
+You can add Samplevnf committers and contributors to review your codes.
+ 
+Modifying the code under review in Gerrit
+At the same time the code is being reviewed in Gerrit, you may need to edit it to
+make some changes and then send it back for review. The following steps go through the procedure.
+Once you have modified/edited your code files under your IDE, you will have to stage them.
+The 'status' command is very helpful at this point as it provides an overview of Git's current state.
+
+::
+  git status
+The output of the command provides us with the files that have been modified after the latest commit.
+ 
+You can now stage the files that have been modified as part of the Gerrit code review
+edition/modification/improvement using git add command.
+It is now time to commit the newly modified files, but the objective here is not to
+create a new commit, we simply want to inject the new changes into the previous commit.
+You can achieve that with the '--amend' option on the 'commit' command:
+
+::
+  git commit --amend
+
+If the commit was successful, the 'status' command should not return the updated
+files as about to be commited.
+ 
+The final step consists in pushing the newly modified commit to Gerrit.
+
+::
+  git review
+
+References
+[1]: http://artifacts.opnfv.org/samplevnf/docs/testing_user_userguide_vACL/index.html
diff --git a/docs/testing/developer/design/04-SampleVNF_Desgin.rest b/docs/testing/developer/design/04-SampleVNF_Desgin.rest
new file mode 100644
index 00000000..6c39da73
--- /dev/null
+++ b/docs/testing/developer/design/04-SampleVNF_Desgin.rest
@@ -0,0 +1,123 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation and others.
+
+.. OPNFV SAMPLEVNF Documentation design file.
+
+===================================
+SampleVNF Highlevel Desing
+===================================
+
+vFW - Design
+=============
+
+Requirements
+-----------------
+Following are the design requierments of the vFW.
+
+-  The firewall will examine packets and verify that they are appropriate for the
+   current state of the connection. Inappropriate packets will be discarded, and
+   counter(s) incremented.
+-  Support both IPv4 and IPv6 traffic type for TCP/UDP/ICMP.
+-  All packet inspection features like firewall, synproxy, connection tracker
+   in this component may be turned off or on through CLI commands
+-  The Static filtering is done thorugh ACL using DPDK libraries. The rules
+   can be added/modified through CLI commands.
+-  Multiple instance of the vFW Pipeline running on multipe cores should be
+   supported for scaling the performance scaling.
+-  Should follow the DPDK IP pipeline framework
+-  Sould use the DPDK libraries and functionalities for better performance
+-  The memory should be allocated in Hugepages using DPDK RTE calls for better
+   performance.
+   
+
+High Level Design
+=================
+
+The Firewall performs basic filtering for malformed packets and dynamic packet
+filtering incoming packets using the connection tracker library.
+The connection data will be stored using a DPDK hash table. There will be one
+entry in the hash table for each connection. The hash key will be based on
+source address/port,destination address/port, and protocol of a packet. The
+hash key will be processed to allow a single entry to be used, regardless of
+which direction the packet is flowing (thus changing source and destination).
+The ACL is implemented as libray stattically linked to vFW, which is used for
+used for rule based packet filtering.
+
+TCP connections and UDP pseudo connections will be tracked separately even if
+theaddresses and ports are identical. Including the protocol in the hash key
+will ensure this.
+
+The Input FIFO contains all the incoming packets for vFW filtering.  The vFW
+Filter has no dependency on which component has written to the Input FIFO.
+Packets will be dequeued from the FIFO in bulk for processing by the vFW.
+Packets will be enqueued to the output FIFO.
+
+The software or hardware loadbalancing can be used for traffic distribution
+across multiple worker threads. The hardware loadbalancing require ethernet
+flow director support from hardware (eg. Fortville x710 NIC card).
+The Input and Output FIFOs will be implemented using DPDK Ring Buffers.
+
+Components of vFW
+=================
+
+In vFW, each component is constructed using packet framework pipelines.
+It includes Rx and Tx Driver, Master pipeline, load balancer pipeline and
+vfw worker pipeline components. A Pipeline framework is a collection of input
+ports, table(s),output ports and actions (functions).
+
+---------------------------
+Receive and Transmit Driver
+---------------------------
+Packets will be received in bulk and provided to LoadBalancer(LB) thread.
+Transimit takes packets from worker threads in a dedicated ring and sent to
+hardware queue.
+
+---------------
+Master Pipeline
+---------------
+The Master component is part of all the IP Pipeline applications. This component
+does not process any packets and should configure with Core 0, to allow
+other cores for processing of the traffic. This component is responsible for
+1. Initializing each component of the Pipeline application in different threads
+2. Providing CLI shell for the user control/debug
+3. Propagating the commands from user to the corresponding components
+
+----------------
+ARPICMP Pipeline
+----------------
+This pipeline processes the APRICMP packets.
+
+--------------
+TXRX Pipelines
+--------------
+The TXTX and RXRX pipelines are pass through pipelines to forward both ingress
+and egress traffic to Loadbalancer. This is required when the Software
+Loadbalancer is used.
+
+----------------------
+Load Balancer Pipeline
+----------------------
+The vFW support both hardware and software balancing for load balancing of
+traffic across multiple VNF threads. The Hardware load balancing require support
+from hardware like Flow Director for steering of packets to application through
+hardware queues.
+
+The Software Load balancer is also supported if hardware load balancing can't be
+used for any reason. The TXRX along with LOADB pipeline provides support for
+software load balancing by distributing the flows to Multiple vFW worker
+threads.
+Loadbalancer (HW or SW) distributes traffic based on the 5 tuple (src addr, src
+port, dest addr, dest port and protocol) applying an XOR logic distributing to
+active worker threads, thereby maintaining an affinity of flows to worker
+threads.
+
+------------
+vFW Pipeline
+------------
+The vFW performs the basic packet filtering and will drop the invalid and
+malformed packets.The Dynamic packet filtering done using the connection tracker
+library. The packets are processed in bulk and Hash table is used to maintain
+the connection details.
+Every TCP/UDP packets are passed through connection tracker library for valid
+connection. The ACL library integrated to firewall provide rule based filtering.
diff --git a/docs/testing/developer/design/index.rst b/docs/testing/developer/design/index.rst
deleted file mode 100644
index 03070d59..00000000
--- a/docs/testing/developer/design/index.rst
+++ /dev/null
@@ -1,5 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Intel Corporation and others.
-
-.. OPNFV SAMPLEVNF Documentation design file.
diff --git a/docs/testing/developer/requirements/03-Requirements.rst b/docs/testing/developer/requirements/03-Requirements.rst
new file mode 100644
index 00000000..ce43e06a
--- /dev/null
+++ b/docs/testing/developer/requirements/03-Requirements.rst
@@ -0,0 +1,88 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. (c) OPNFV, Intel Corporation and others.
+
+.. OPNFV SAMPLEVNF Documentation design file.
+
+============
+Requirements
+============
+
+**Required Test setup:**
+
+.. _SampleVNF: https://wiki.opnfv.org/samplevnf
+.. _Technical_Briefs: https://wiki.opnfv.org/display/SAM/Technical+Briefs+of+VNFs
+
+Supported Test setup:
+--------------------
+The device under test (DUT) consists of a system following;
+  * A single or dual processor and PCH chip, except for System on Chip (SoC) cases
+  * DRAM memory size and frequency (normally single DIMM per channel)
+  * Specific Intel Network Interface Cards (NICs)
+  * BIOS settings noting those that updated from the basic settings
+  * DPDK build configuration settings, and commands used for tests
+Connected to the DUT is an IXIA* or Software Traffic generator like pktgen or TRex,
+simulation platform to generate packet traffic to the DUT ports and
+determine the throughput/latency at the tester side.
+
+Below are the supported/tested (:term `VNF`) deployment type.
+.. image:: images/deploy_type.png
+   :width: 800px
+   :alt: SampleVNF supported topology
+
+Hardware & Software Ingredients
+-------------------------------
+.. code-block:: console
+   +---------------+------------------+
+   | Item          | Description      |
+   +---------------+------------------+
+   | Memory        | Min 20GB         |
+   +---------------+------------------+
+   | NICs          | 2 x 10G          |
+   +---------------+------------------+
+   | HostOS/Guest  | Ubuntu 16.04 LTS |
+   +---------------+------------------+
+   | kernel        | >4.4.0-34-generic|
+   +---------------+------------------+
+   |DPDK           | >17.02           |
+   +---------------+------------------+
+
+   Boot and BIOS settings
+   +------------------+---------------------------------------------------+
+   | Boot settings    | default_hugepagesz=1G hugepagesz=1G hugepages=16  |
+   |                  | hugepagesz=2M hugepages=2048 isolcpus=1-11,22-33  |
+   |                  | nohz_full=1-11,22-33 rcu_nocbs=1-11,22-33         |
+   |                  | Note: nohz_full and rcu_nocbs is to disable Linux*|
+   |                  | kernel interrupts, and it’s import                |
+   +------------------+---------------------------------------------------+
+   |BIOS              | CPU Power and Performance Policy <Performance>    |
+   |                  | CPU C-state Disabled                              |
+   |                  | CPU P-state Disabled                              |
+   |                  | Enhanced Intel® Speedstep® Tech Disabled          |
+   |                  | Hyper-Threading Technology (If supported) Enable  |
+   |                  | Virtualization Techology Enable                   |
+   |                  | Coherency Enable                                  |
+   |                  | Turbo Boost Disabled                              |
+   +------------------+---------------------------------------------------+
+
+Network Topology for testing VNFs
+---------------------------------
+The ethernet cables should be connected between traffic generator and the VNF server (BM,
+SRIOV or OVS) setup based on the test profile.
+
+The connectivity could be
+1. Single port pair : One pair ports used for traffic 
+   ::
+     e.g. Single port pair link0 and link1 of VNF are used
+     TG:port 0 <------> VNF:Port 0
+     TG:port 1 <------> VNF:Port 1
+
+2. Multi port pair :  More than one pair of traffic
+   ::
+     e.g. Two port pair link 0, link1, link2 and link3 of VNF are used
+     TG:port 0 <------> VNF:Port 0
+     TG:port 1 <------> VNF:Port 1
+     TG:port 2 <------> VNF:Port 2 
+     TG:port 3 <------> VNF:Port 3
+For openstack/Standalone virtualization, installation please refer the openstack guide and ovs-dpdk/sriov github. 
+(TBA - Add link to guide)
diff --git a/docs/testing/developer/requirements/index.rst b/docs/testing/developer/requirements/index.rst
deleted file mode 100644
index 8efa8620..00000000
--- a/docs/testing/developer/requirements/index.rst
+++ /dev/null
@@ -1,5 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International License.
-.. http://creativecommons.org/licenses/by/4.0
-.. (c) OPNFV, Intel Corporation and others.
-
-.. OPNFV SAMPLEVNF Documentation requirements file.