Clover Doc Patches: Update release note, user guide, and add Clovisor

configuration guide

Change-Id: I65da13cb9ce503f9b5b2fd2457f147549a13b9f6
Signed-off-by: Stephen Wong <stephen.kf.wong@gmail.com>

4 files changed, 211 insertions, 47 deletions

diff --git a/docs/release/configguide/clovisor_config_guide.rst b/docs/release/configguide/clovisor_config_guide.rst
new file mode 100644
index 0000000..9b5f4a3
--- /dev/null
+++ b/docs/release/configguide/clovisor_config_guide.rst
@@ -0,0 +1,156 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. SPDX-License-Identifier CC-BY-4.0
+.. (c) Authors of Clover
+
+.. _clovisor_config_guide:
+
+============================
+Clovisor Configuration Guide
+============================
+
+Clovisor requires minimal to no configurations to function as a network tracer.
+It expects configurations to be set at a redis sever running at clover-system
+namespace.
+
+No Configuration
+================
+
+If redis server isn't running as service name **redis** in namespace
+**clover-system** or there isn't any configuration related to Clovisor in that
+redis service, then Clovisor would monitor all pods under the **default**
+namespace. The traces would be sent to **jaeger-collector** service under the
+**clover-system** namespace
+
+Using redis-cli
+===============
+
+Install ``redis-cli`` on the client machine, and look up redis IP address:
+
+.. code-block:: bash
+
+    $ kubectl get services -n clover-system
+
+which one may get something like the following:
+
+.. code-block:: bash
+
+    $
+    NAME      TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
+    redis     ClusterIP   10.109.151.40   <none>        6379/TCP   16s
+
+if (like above), the external IP isn't visible, one may be able to get the pod
+IP address directly via the pod (for example, it works with Flannel as CNI
+plugin):
+
+.. code-block:: bash
+
+    $ kubectl get pods -n clover-system -o=wide
+    NAME      READY     STATUS    RESTARTS   AGE       IP             NODE
+    redis     2/2       Running   0          34m       10.244.0.187   clover1804
+
+and one can connect to redis via::
+
+    redis-cli -h 10.244.0.187 -p 6379
+
+Jaeger Collector Configuration
+==============================
+
+Clovisor allows user to specify the Jaeger service for which Clovisor would send
+the network traces to. This is configured via setting the values for
+keys **clovisor_jaeger_collector** and **clovisor_jaeger_agent**::
+
+    redis> SET clovisor_jaeger_collector "jaeger-collector.istio-system:14268"
+    "OK"
+    redis> SET clovisor_jaeger_agent "jaeger-agent.istio-system:6831"
+    "OK"
+
+Configure Monitoring Namespace and Labels
+=========================================
+
+Configruation Value String Format:
+----------------------------------
+
+    <namespace>[:label-key:label-value]
+
+User can configure namespace(s) for Clovisor to tap into via adding namespace
+configuration in redis list **clovisor_labels**::
+
+    redis> LPUSH clovisor_labels "my-namespace"
+    (integer) 1
+
+the above command will cause Clovisor to **NOT** monitor the pods in **default**
+namespace, and only monitor the pods under **my-namespace**.
+
+If user wants to monitor both 'default' and 'my-namespace', she needs to
+explicitly add 'default' namespace back to the list::
+
+    redis> LPUSH clovisor_labels "default"
+    (integer) 2
+    redis> LRANGE clovisor_labels 0 -1
+    1.) "default"
+    2.) "my-namespace"
+
+Clovisor allows user to optionally specify which label match on pods to further
+filter the pods to monitor::
+
+    redis> LPUSH clovisor_labels "my-2nd-ns:app:database"
+    (integer) 1
+
+the above configuration would result in Clovisor only monitoring pods in
+my-2nd-ns namespace which matches the label "app:database"
+
+User can specify multiple labels to filter via adding more configuration
+entries::
+
+    redis> LPUSH clovisor_labels "my-2nd-ns:app:web"
+    (integer) 2
+    redis> LRANGE clovisor_labels 0 -1
+    1.) "my-2nd-ns:app:web"
+    2.) "my-2nd-ns:app:database"
+
+the result is that Clovisor would monitor pods under namespace my-2nd-ns which
+match **EITHER** app:database **OR** app:web
+
+Currently Clovisor does **NOT** support filtering of more than one label per
+filter, i.e., no configuration option to specify a case where a pod in a
+namespace needs to be matched with TWO or more labels to be monitored
+
+Configure Egress Match IP address, Port Number, and Matching Pods
+=================================================================
+
+Configruation Value String Format:
+----------------------------------
+
+    <IP Address>:<TCP Port Number>[:<Pod Name Prefix>]
+
+By default, Clovisor only traces packets that goes to a pod via its service
+port, and the response packets, i.e., from pod back to client. User can
+configure tracing packet going **OUT** of the pod to the next microservice, or
+an external service also via the **clovior_egress_match** list::
+
+    redis> LPUSH clovior_egress_match "10.0.0.1:3456"
+    (integer) 1
+
+the command above will cause Clovisor to trace packet going out of ALL pods
+under monitoring to match IP address 10.0.0.1 and destination TCP port 3456 on
+the **EGRESS** side --- that is, packets going out of the pod.
+
+User can also choose to ignore the outbound IP address, and only specify the
+port to trace via setting IP address to zero::
+
+    redis> LPUSH clovior_egress_match "0:3456"
+    (integer) 1
+
+the command above will cause Clovisor to trace packets going out of all the pods
+under monitoring that match destination TCP port 3456.
+
+User can further specify a specific pod prefix for such egress rule to be
+applied::
+
+    redis> LPUSH clovior_egress_match "0:3456:proxy"
+    (integer) 1
+
+the command above will cause Clovisor to trace packets going out of pods under
+monitoring which have name starting with the string "proxy" that match destination
+TCP port 3456
diff --git a/docs/release/configguide/index.rst b/docs/release/configguide/index.rst
index ec7b16a..41c1eca 100644
--- a/docs/release/configguide/index.rst
+++ b/docs/release/configguide/index.rst
@@ -19,3 +19,4 @@ Clover Configuration Guide
    visibility_config_guide.rst
    modsecurity_config_guide.rst
    spinnaker_config_guide.rst
+   clovisor_config_guide.rst
diff --git a/docs/release/release-notes/release-notes.rst b/docs/release/release-notes/release-notes.rst
index f345f61..9dd15b5 100644
--- a/docs/release/release-notes/release-notes.rst
+++ b/docs/release/release-notes/release-notes.rst
@@ -4,7 +4,7 @@
 .. (c) Authors of Clover
 
 
-This document provides Clover project's release notes for the OPNFV Fraser release.
+This document provides Clover project's release notes for the OPNFV Gambia release.
 
 .. contents::
    :depth: 3
@@ -18,24 +18,37 @@ Version history
 | **Date**           | **Ver.**           | **Author**         | **Comment**        |
 |                    |                    |                    |                    |
 +--------------------+--------------------+--------------------+--------------------+
-| 2018-03-14         | Fraser 1.0         | Stephen Wong       | First draft        |
+| 2018-03-14         | Gambia 1.0         | Stephen Wong       | First draft        |
 |                    |                    |                    |                    |
 +--------------------+--------------------+--------------------+--------------------+
 
 Important notes
 ===============
 
-The Clover project for OPNFV Fraser can ONLY be run on Kubernetes version 1.9 or
-later
+The Clover project for OPNFV Gambia is tested on Kubernetes version 1.9 and
+1.11. It is only tested on Istio 1.0.
 
 Summary
 =======
 
-Clover Fraser release provides tools for installation and validation of various
-upstream cloud native projects including Istio, fluentd, Jaegar, and Prometheus.
-In addition, the Fraser release also includes a sample VNF, its Kubernetes
-manifest, simple tools to validate route rules from Istio, as well as an
-example A-B testing framework.
+Clover Gambia release further enhances the Fraser release by providing various
+tools to help operators deploy cloud native network functions. These tools
+include
+
+#. Collector: gathers and collects metrics and traces from Prometheus and
+   Jaeger, respectively, and provides a single access point for such data
+#. Visibility: utilizes an analytic engine to correlate and organize data
+   gathered by the collector
+#. CLI: comprehensive Clover CLI called cloverctl, offering a single management
+   tool for operating Clover toolset
+#. Network Tracing: CNI plugin agnostic network tracing tool
+#. Extended HTTP Security: integrate modsecurity (Web Application Firewall) and
+   Snort with Istio gateway via Istio newly added mechanisms to redirect and
+   mirror traffic to the network functions
+#. HTTP Test Client: bundle JMeter as test client for testing
+#. UI: developmental / sample UI to offer single pane view of Clover system
+#. Spinnaker Integration: provides automated / programmable cloud provider
+   add/update/delete; sample pipeline and installation scripts
 
 Release Data
 ============
@@ -47,13 +60,13 @@ Release Data
 | **Repo/commit-ID**                   |                                      |
 |                                      |                                      |
 +--------------------------------------+--------------------------------------+
-| **Release designation**              | Fraser                               |
+| **Release designation**              | Gambia                               |
 |                                      |                                      |
 +--------------------------------------+--------------------------------------+
-| **Release date**                     | 2018-04-27
+| **Release date**                     | 2018-11-09
 |                                      |                                      |
 +--------------------------------------+--------------------------------------+
-| **Purpose of the delivery**          | OPNFV Fraser release                 |
+| **Purpose of the delivery**          | OPNFV Gambia release                 |
 |                                      |                                      |
 +--------------------------------------+--------------------------------------+
 
@@ -62,18 +75,20 @@ Version change
 
 Module version changes
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-OPNFV Fraser marks the first release for Clover
+Clover Gambia release will no longer support Istio 0.6, the version of Istio
+supported by Clover Gambia release
 
 Document version changes
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-OPNFV Fraser marks the first release for Clover
+Clover Gambia has updated the config guide and user guide accordingly, including
+new documents for the new features
 
 Reason for version
 ^^^^^^^^^^^^^^^^^^^^
 
 Feature additions
 ~~~~~~~~~~~~~~~~~~~~~~~
-<None> (no backlog)
+See Summary above
 
 Bug corrections
 ~~~~~~~~~~~~~~~~~~~~~
diff --git a/docs/release/userguide/userguide.rst b/docs/release/userguide/userguide.rst
index d99359b..7c2b012 100644
--- a/docs/release/userguide/userguide.rst
+++ b/docs/release/userguide/userguide.rst
@@ -5,56 +5,48 @@
 
 
 ================================================================
-Clover User Guide (Fraser Release)
+Clover User Guide (Gambia Release)
 ================================================================
 
-This document provides the Clover user guide for the OPNFV Fraser release.
+This document provides the Clover user guide for the OPNFV Gambia release.
 
 Description
 ===========
 
-As project Clover's first release, the Fraser release includes installation and simple
-validation of foundational upstream projects including Istio, fluentd, Jaeger, and
-Prometheus. The Clover Fraser release also provides a sample set of web-oriented network
-services, which follow a micro-service design pattern, its Kubernetes manifest, and an
-automated script to demonstrate a sample A-B testing use-case. The A-B sample script
-validates performance criteria using Istio request routing functionality leveraging
-the sample services deployed within Istio and the tracing data available within Jaeger.
+Clover Gambia builds on previous release to further enhance the toolset for
+cloud native network functions operations. The two emphasis on the release are:
+#. Integration of Spinnaker to support continuous delivery
+#. Centralizing Operational Data for Visibility
 
-What is in Fraser?
+What is in Gambia?
 ==================
 
  * Sample micro-service composed VNF named Service Delivery Controller (SDC)
 
- * Logging module: fluentd and elasticsearch Kubernetes manifests,
-   and fluentd installation validation
+ * Istio 1.0 support
 
- * Tracing module: Jaeger Kubernetes manifest, installation validation,
-   Jaegar tracing query tools, and module for trace data output to datastore
+ * clover-collector: gathers and collects metrics and traces from Prometheus and
+   Jaeger, and provides a single access point for such data
 
- * Monitoring module: Prometheus Kubernetes manifest, installation
-   validation, and sample Prometheous query of Istio related metrics
+ * Visibility: utilizes an analytic engine to correlate and organize data
+   collected by clover-collector
 
- * Istio route-rules sample yaml and validation tools
+ * cloverctl: Clover's new CLI
 
- * Test scripts
+ * Clovisor: Clover's cloud native, CNI-plugin agnostic network tracing tool
 
- * Sample code for an A-B testing demo shown during ONS North America 2018
+ * Integration of HTTP Security Modules with Istio 1.0
 
-Usage
-=====
+ * JMeter: integrating jmeter as test client
+
+ * Clover UI: sample UI to offer single pane view / configuration point of the
+   Clover system
 
- * Python modules to validate installation of fluentd logging, Jaeger tracing, and
-   Prometheus monitoring. Deployment and validation instructions can be found at:
-   :ref:`logging`, :ref:`tracing`, and :ref:`monitoring` respectively.
+ * Spinnaker Integration: add ability to add/update/delete cloud provider via
+   cloverctl, and sample pipeline utilized by Clover project to deploy SDC
 
- * Deployment and usage of SDC sample
-   - Services designed and implemented with micro-service design pattern
-   - Tested and validated via Istio service mesh tools
-   Detailed usage instructions for the sample can be found at :ref:`sdc_config_guide`
 
- * An example use-case for A-B testing. Detailed usage instructions for this sample A-B
-   validation can be found at: :ref:`a_b_config_guide`
+Usage
+=====
 
- * Sample tool to validate Istio route rules:
-   tools/python clover_validate_route_rules.py -s <service name> -t <test id>
+ * Please refer to configguildes for usage detail on various modules