Merge "Initial commit of Jmeter validation config guide"

1 files changed, 298 insertions, 0 deletions

diff --git a/docs/release/configguide/jmeter_config_guide.rst b/docs/release/configguide/jmeter_config_guide.rst
new file mode 100644
index 0000000..de1d2f5
--- /dev/null
+++ b/docs/release/configguide/jmeter_config_guide.rst
@@ -0,0 +1,298 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. SPDX-License-Identifier CC-BY-4.0
+.. (c) Authors of Clover
+
+.. _jmeter_config_guide:
+
+=======================================
+JMeter Validation Configuration Guide
+=======================================
+
+This document provides a guide to use the JMeter validation service, which is introduced in
+the Clover Gambia release.
+
+Overview
+=========
+
+Apache JMeter is a mature, open source application that supports web client emulation. Its
+functionality has been integrated into the Clover project to allow various CI validations
+and performance tests to be performed. The system under test can either be REST services/APIs
+directly or a set of L7 network services. In the latter scenario, Clover nginx servers may
+be employed as an endpoint to allow traffic to be sent end-to-end across a service chain.
+
+The Clover JMeter integration is packaged as docker containers with manifests to deploy
+in a Kubernetes (k8s) cluster. The Clover CLI (**cloverctl**) can be used to configure and
+control the JMeter service within the k8s cluster via **clover-controller**.
+
+The Clover JMeter integration has the following attributes:
+
+ * **Master/Slave Architecture:** uses the native master/slave implementation of JMeter. The master
+   and slaves have distinct OPNFV docker containers for rapid deployment and usage. Slaves allow
+   the scale of the emulation to be increased linearly for performance testing. However, for
+   functional validations and modest scale, the master may be employed without any slaves.
+
+ * **Test Creation & Control:** JMeter makes use of a rich XML-based test plan. While this offers
+   a plethora of configurable options, it can be daunting for a beginner user to edit directly.
+   Clover provides an abstracted yaml syntax exposing a subset of the available configuration
+   parameters. JMeter test plans are generated on the master and tests can be started from
+   **cloverctl** CLI.
+
+ * **Result Collection:** summary log results and detailed per-request results can be retrieved
+   from the JMeter master during and after tests from the **cloverctl** or from a REST API exposed
+   via **clover-controller**.
+
+.. image:: imgs/jmeter_overview.png
+    :align: center
+    :scale: 100%
+
+Deploying Clover JMeter service
+===============================
+
+Prerequisites
+-------------
+
+The following assumptions must be met before continuing on to deployment:
+
+ * Installation of Docker has already been performed. It's preferable to install Docker CE.
+ * Installation of k8s in a single-node or multi-node cluster.
+ * Clover CLI (**cloverctl**) has been downloaded and setup. Instructions to deploy can be found
+   at :ref:`controller_services_controller`
+ * The **clover-controller** service is deployed in the k8s cluster the validation services will
+   be deployed in. Instructions to deploy can be found at :ref:`controller_services_controller`.
+
+Deploy with Clover CLI
+-----------------------
+
+The easiest way to deploy Clover JMeter validation services into your k8s cluster is to use the
+**cloverctl** CLI using the following command:
+
+.. code-block:: bash
+
+    $ cloverctl create system validation
+
+Container images with the Gambia release tag will pulled if the tag is unspecified. The release
+tag is **opnfv-7.0.0** for the Gambia release. To deploy the latest containers from master, use
+the command shown below::
+
+    $ cloverctl create system validation -t latest
+
+The Clover CLI will add master/slave pods to the  k8s cluster in the default namespace.
+
+The JMeter master/slave docker images will automatically be pulled from the OPNFV public
+Dockerhub registry. Deployments and respective services will be created with three slave
+replica pods added with the **clover-jmeter-slave** prefix. A single master pod will be
+created with the **clover-jmeter-master** prefix.
+
+Deploy from source
+------------------
+
+To continue to deploy from the source code, clone the Clover git repository and navigate
+within to the directory, as shown below:
+
+.. code-block:: bash
+
+    $ git clone https://gerrit.opnfv.org/gerrit/clover
+    $ cd clover/clover/tools/jmeter/yaml
+    $ git checkout stable/gambia
+
+To deploy the master use the following two commands, which will create a manifest with
+the Gambia release tags and creates the deployment in the k8s cluster::
+
+    $ python render_master.py --image_tag=opnfv-7.0.0 --image_path=opnfv
+    $ kubectl create -f clover-jmeter-master.yaml
+
+JMeter can be injected into an Istio service mesh. To deploy in the default
+namespace within the service mesh, use the following command for manual
+sidecar injection::
+
+    $ istioctl kube-inject -f clover-jmeter-master.yaml | kubectl apply -f -
+
+**Note, when injecting JMeter into the service mesh, only the master will function for
+the Clover integration, as master-slave communication is known not to function with the Java
+RMI API. Ensure 'istioctl' is in your path for the above command.**
+
+To deploy slave replicas, render the manifest yaml and create in k8s adjusting the
+``--replica_count`` value for the number of slave pods desired::
+
+    $ python render_slave.py --image_tag=opnfv-7.0.0 --image_path=opnfv --replica_count=3
+    $ kubectl create -f clover-jmeter-slave.yaml
+
+Verifying the deployment
+------------------------
+
+To verify the validation services are deployed, ensure the following pods are present
+with the command below:
+
+.. code-block:: bash
+
+    $ kubectl get pod --all-namespaces
+
+The listing below must include the following pods assuming deployment in the default
+namespace:
+
+.. code-block:: bash
+
+    NAMESPACE   NAME                                       READY      STATUS
+    default     clover-jmeter-master-688677c96f-8nnnr       1/1       Running
+    default     clover-jmeter-slave-7f9695d56-8xh67         1/1       Running
+    default     clover-jmeter-slave-7f9695d56-fmpz5         1/1       Running
+    default     clover-jmeter-slave-7f9695d56-kg76s         1/1       Running
+    default     clover-jmeter-slave-7f9695d56-qfgqj         1/1       Running
+
+Using JMeter Validation
+=======================
+
+Creating a test plan
+--------------------
+
+To employ a test plan that can be used against the :ref:`sdc_config_guide` sample, navigate to
+ cloverctl yaml directory and use the sample named 'jmeter_testplan.yaml', which is shown below.
+
+.. code-block:: bash
+
+    load_spec:
+        num_threads: 5
+        loops: 2
+        ramp_time: 60
+        duration: 80
+    url_list:
+        - name: url1
+          url: http://proxy-access-control.default:9180
+          method: GET
+          user-agent: chrome
+        - name: url2
+          url: http://proxy-access-control.default:9180
+          method: GET
+          user-agent: safari
+
+The composition of the yaml file breaks down as follows:
+ * ``load_spec`` section of the yaml defines the load profile of the test.
+ * `num_threads`` parameter defines the maximum number of clients/users the test will emulate.
+ * ``ramp_time`` determines the rate at which threads/users will be setup.
+ * ``loop`` parameter reruns the same test and can be set to 0 to loop forever.
+ * ``duration`` parameter is used to limit the test run time and be used as a hard cutoff when
+   using loop forever.
+ * ``url_list`` section of the yaml defines a set of HTTP requests that each user will perform.
+   It includes the request URL that is given a name (used as reference in detailed per-request
+   results) and the HTTP method to use (ex. GET, POST). The ``user-agent`` parameter allows this
+   HTTP header to be specified per request and can be used to emulate browsers and devices.
+
+The ``url`` syntax is <domain or IP>:<port #>. The colon port number may be omitted if port 80
+is intended.
+
+The test plan yaml is an abstraction of the JMeter XML syntax (uses .jmx extension) and can be
+pushed to the master using the **cloverctl** CLI with the following command:
+
+.. code-block:: bash
+
+    $ cloverctl create testplan –f jmeter_testplan.yaml
+
+The test plan can now be executed and will automatically be distributed to available JMeter slaves.
+
+Starting the test
+-----------------
+
+Once a test plan has been created on the JMeter master, a test can be started for the test plan
+with the following command:
+
+.. code-block:: bash
+
+    $ cloverctl start testplan
+
+The test will be executed from the **clover-jmeter-master** pod, whereby HTTP requests will
+originate directly from the master. The number of aggregate threads/users and request rates
+can be scaled by increasing the thread count or decreasing the ramp time respectively in the
+test plan yaml. However, the scale of the test can also be controlled by adding slaves to the
+test. When slaves are employed, the master will only be used to control slaves and will not be
+a source of traffic. Each slave pod will execute the test plan in its entirety.
+
+To execute tests using slaves, add the flag '-s' to the start command from the Clover CLI as shown
+below:
+
+.. code-block:: bash
+
+    $ cloverctl start testplan –s <slave count>
+
+The **clover-jmeter-slave** pods must be deployed in advance before executing the above command. If
+the steps outlined in section `Deploy with Clover CLI`_ have been followed, three slaves will
+have already been deployed.
+
+Retrieving Results
+------------------
+
+Results for the test can be obtained by executing the following command:
+
+.. code-block:: bash
+
+    $ cloverctl get testresult
+    $ cloverctl get testresult log
+
+The bottom of the log will display a summary of the test results, as shown below::
+
+    3  in 00:00:00 =  111.1/s Avg:     7 Min:     6 Max:     8 Err:     0 (0.00%)
+    20 in 00:00:48 =    0.4/s Avg:    10 Min:     6 Max:    31 Err:     0 (0.00%)
+
+Each row of the summary table is a snapshot in time with the final numbers in the last row.
+In this example, 20 requests (5 users/threads x 2 URLs) x loops) were sent successfully
+with no HTTP responses with invalid/error (4xx/5xx) status codes. Longer tests will produce
+a larger number of snapshot rows. Minimum, maximum and average response times are output per
+snapshot.
+
+To obtain detailed, per-request results use the ``detail`` option, as shown below::
+
+    $ cloverctl get testresult detail
+
+    1541567388622,14,url1,200,OK,ThreadGroup 1-4,text,true,,843,0,1,1,14,0,0
+    1541567388637,8,url2,200,OK,ThreadGroup 1-4,text,true,,843,0,1,1,8,0,0
+    1541567388646,6,url1,200,OK,ThreadGroup 1-4,text,true,,843,0,1,1,6,0,0
+    1541567388653,7,url2,200,OK,ThreadGroup 1-4,text,true,,843,0,1,1,7,0,0
+    1541567400622,12,url1,200,OK,ThreadGroup 1-5,text,true,,843,0,1,1,12,0,0
+    1541567400637,8,url2,200,OK,ThreadGroup 1-5,text,true,,843,0,1,1,8,0,0
+    1541567400645,7,url1,200,OK,ThreadGroup 1-5,text,true,,843,0,1,1,7,0,0
+    1541567400653,6,url2,200,OK,ThreadGroup 1-5,text,true,,843,0,1,1,6,0,0
+
+Columns are broken down on the following fields:
+ * timeStamp, elapsed, label, responseCode, responseMessage, threadName, dataType, success
+ * failureMessage  bytes, sentBytes, grpThreads, allThreads, Latency, IdleTime, Connect
+
+``elapsed`` or ``Latency`` values are in milliseconds.
+
+Uninstall from Kubernetes environment
+=====================================
+
+Delete with Clover CLI
+-----------------------
+
+When you're finished working with JMeter validation services, you can uninstall it with the
+following command:
+
+.. code-block:: bash
+
+     $ cloverctl delete system validation
+
+The command above will remove the clover-jmeter-master and clover-jmeter-slave deployment
+and service resources from the current k8s context.
+
+Delete from source
+------------------
+
+The JMeter validation services can be uninstalled from the source code using the commands below:
+
+.. code-block:: bash
+
+    $ cd clover/samples/scenarios
+    $ kubectl delete -f clover-jmeter-master.yaml
+    $ kubectl delete -f clover-jmeter-slave.yaml
+
+Uninstall from Docker environment
+=================================
+
+The OPNFV docker images can be removed with the following commands from nodes
+in the k8s cluster.
+
+.. code-block:: bash
+
+    $ docker rmi opnfv/clover-jmeter-master
+    $ docker rmi opnfv/clover-jmeter-slave
+    $ docker rmi opnfv/clover-controller