summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2018-11-09Merge "Initial commit of visibility services config guide"Stephen Wong10-7/+410
2018-11-09Initial commit of visibility services config guideearrage10-7/+410
- Updates to jmeter validation and controller services guides - Image for jmeter guide Change-Id: I175c4052f005d6fbebde8afbcb2bda1df4a4d6d9 Signed-off-by: earrage <eddie.arrage@huawei.com>
2018-11-09change the doc for the istio 1.0Ace Lee3-53/+11
prometheus has been installed in istio 1.0 so we need to delete the view.sh file. but we need to change to to NodePort way to expose the port Change-Id: I8ae6bc1c251ba5effbc94efe984384af6e64adfc Signed-off-by: Ace Lee <liyin11@huawei.com>
2018-11-08Merge "Add create/delete of clover-system services from CLI"Stephen Wong26-244/+936
2018-11-08Fix format problem on user guideStephen Wong1-0/+1
Change-Id: I474ccaed8fa16d934f770b555211b6a30ba7bfae Signed-off-by: Stephen Wong <stephen.kf.wong@gmail.com>
2018-11-08Clover Doc Patches: Update release note, user guide, and add ClovisorStephen Wong4-47/+211
configuration guide Change-Id: I65da13cb9ce503f9b5b2fd2457f147549a13b9f6 Signed-off-by: Stephen Wong <stephen.kf.wong@gmail.com>
2018-11-08Merge "Initial commit for Clovisor design doc"Stephen Wong2-0/+206
2018-11-08Merge "Initial commit of controller services config guide"Stephen Wong2-0/+186
2018-11-08Merge "Initial commit of Jmeter validation config guide"Stephen Wong1-0/+298
2018-11-08correcting reference labelSofia Wallin2-7/+9
Change-Id: I9b354e85f7f86872617904dd84ff5f0c516e1ffb Signed-off-by: Sofia Wallin <sofia.wallin@est.tech>
2018-11-07Initial commit of controller services config guideearrage2-0/+186
- Controller services lump cloverctl CLI and clover-controller together and this config guide provides brief overview and deploy/delete instructions - Intended to be one of the first guides a user follows, as controller is prerequisite for many other Clover services - Add new, existing config guides to index including forthcoming jmeter, visibility guides - Embedded images forthcoming Change-Id: Ifef0248d9370df862a1d192b979606e443a82e95 Signed-off-by: earrage <eddie.arrage@huawei.com>
2018-11-07Initial commit of Jmeter validation config guideearrage1-0/+298
- Provides brief overview and usage instructions including: - delete/deploy in k8s using cloverctl CLI and source - Generate the test plan on the master - Start tests on master or with slaves - Retrieve results using cloverctl and interpret - Embedded images forthcoming Change-Id: Ibbfa21f5d73c76c16ffac115ad0164c5cf210236 Signed-off-by: earrage <eddie.arrage@huawei.com>
2018-11-07Test cloverctl tarball download from gitearrage1-0/+0
- This file will be updated when outstanding patches are merged. - This is a test for the config guide docs to validate download procedure. Change-Id: Iebc79b14b1e74ca09f85f1f5846b0f4f43c782ba Signed-off-by: earrage <eddie.arrage@huawei.com>
2018-11-07Initial commit for Clovisor design docStephen Wong2-0/+206
Change-Id: I9e878b0302623266a7c900f5ca53825d7a3492c3 Signed-off-by: Stephen Wong <stephen.kf.wong@gmail.com>
2018-11-07Merge "Add ability to read controller nodeport IP from file"Stephen Wong29-133/+237
2018-11-07Merge "Fix Docker build of Clovisor Docker image on OPNFV Jenkins"Stephen Wong1-1/+2
2018-11-06Add create/delete of clover-system services from CLIearrage26-244/+936
- Improve usability when deploying or deleting various Clover components using cloverctl - Add yaml for each of the clover-system services including controller, collector, spark, redis, cassandra, clovisor and jmeter separated by k8s resource and available under a single yaml directory - Deploy/delete yaml for services in various subsets including: - visibility(controller, collector, spark, redis, cassandra) - datastore(redis, cassandra) - validation(jmeter master/slaves(4)) - clovisor (individually) - collector (individually) - controller (individually) Ex. cloverctl create system controller Ex. cloverctl create system visibility - Ability to add nodeport for controller separately (cloverctl create system controller nodeport) - Use native client-go methods to create/delete all k8s resources used by Clover including: - pods, deployments, statefulsets, daemonsets, services - serviceaccounts, clusterrolebindings - Above allows yaml override of values such as image value specified in deployment or daemonset. This is broken down by: - tag (ex. latest, opnfv-7.0.0) or repo'(ex. opnfv, localhost:5000) that can be added with CLI flags as shown below: (cloverctl create system visibility -t latest -r localhost:5000) - defaults to -r opnfv -t latest - Creates/deletes clover-system namespace - Also added ability to create/delete lb service for controller external access when running on environment such as GKE (cloverctl <create/delete> system controller lb) Change-Id: I2a3c6c80035d4663fa38368b3ff13e9a14090a47 Signed-off-by: earrage <eddie.arrage@huawei.com>
2018-11-05Add ability to read controller nodeport IP from fileearrage29-133/+237
- Formalize use of cloverctl config file using viper - Default name of config file is '.cloverctl.yaml' - Look for file in home directory or relative to cloverctl executable - Single value in config is for 'ControllerIP: <IP Address>' - Still try and retrieve port value of clover-controller nodeport automatically - Still try and retrieve IP address of clover-controller LB if GKE environment is being used and override config file value if found in current kube context - Improve error messaging if controller connect fails - Clean up the CLI help output - Modify jmeter 'get testresult' command to avoid the use of flags and use arguments instead Change-Id: Id95d1b3d1079ac84996681db62da1f9b6b8a88b5 Signed-off-by: earrage <eddie.arrage@huawei.com>
2018-11-03Fix Docker build of Clovisor Docker image on OPNFV JenkinsStephen Wong1-1/+2
Change-Id: Ied0ef2ccd4eb95cace68826a1494e82f2c18d95c Signed-off-by: Stephen Wong <stephen.kf.wong@gmail.com>
2018-11-02Modify container tag to opnfv in spark-submitearrage4-28/+28
- Change tag from local repo to opnfv for clover-spark container, which spark-submit references - Move update of configured visibility services into loop Change-Id: I9d09a851977c6006ea2cbba45d7a28be6a4660b9 Signed-off-by: earrage <eddie.arrage@huawei.com>
2018-11-02Add minor enhancements to jmeter configurationearrage7-17/+75
- Add user-agent field to jmeter config yaml and jmeter jmx template that can be configured per URL in testplan - Add duration time that can be used in combination with ramp time. Duration is the total time of the test while ramp time governs how much time is allotted to establish all configured requests/connections - Explicitly define clover-jmeter-master service name with default namespace for clover-controller in clover-system namespace to reach - Add missing request_counts.html file Change-Id: I5e2d388db0b73ea6c28482c94c33cc2a01f1b52e Signed-off-by: earrage <eddie.arrage@huawei.com>
2018-11-01Merge "CLOVER-43: Initial commit for Clovisor"Stephen Wong12-0/+1619
2018-11-01Merge "Initial UI dashboard added for visibility"Stephen Wong24-67/+18284
2018-11-01Merge "Update modsecurity image docker registry"Stephen Wong1-1/+1
2018-11-01Merge "Initial commit for Spark to analyze visibility data"Stephen Wong13-0/+532
2018-11-01Merge "Add initial Istio validation testcase for CI"Stephen Wong5-17/+83
2018-11-01CLOVER-43: Initial commit for ClovisorStephen Wong12-0/+1619
Part of Clover's initial task (during project proposal) is to explore cloud native network tracing tools. Clover the project proposes Clovisor as a way for Clover to use the IOvisor (https://www.iovisor.org/) to perform network tracing. This is the first commit of this module Please note that the BPF code used in this commit were only tested with Linux kernel 4.14+ --- hence, if ubuntu is used, and one prefers LTS, the requirement is 18.04+ A different patch will track the redis and k8s API server watcher code Also, a design doc will be added in the future (TBD) Signed-off-by: Stephen Wong <stephen.kf.wong@gmail.com> Change-Id: I30d9f9d474b8703097c470d39628e86bc788f9b6
2018-10-31Initial UI dashboard added for visibilityearrage24-67/+18284
- Add foundation, jquery and jquery-ui open source js/css packages for skin, layout and AJAX - Add plotly and candela js open source graphing packages - Build core visibility view and provide toplevel templates, headers, menu navigation, etc. - Update collector/visibility APIs to align to latest Spark analytics and UI requirements - Refresh page elements at 5 second interval. - Visibility controls to stop continual page refresh and clear visibility. - Multi-select and update service set to analyze. - System count updates for tracing, spans, metrics schemas. - Min/max/avg response times per service (aggregate) table. - Graphs for: per service request counts, user-agent percentages, per service/URL counts, per URL/status code counts, per URL counts - Tabs for distinct URLs, status codes, user-agents, upstream cluster, operation names, nodeids (service names) - Table for metrics counters specified in visibility configuration. - Metrics over time scatter plots for tracked metrics. Change-Id: Ie3ea12fd32e0274ac52f78becc4a56c4c4134660 Signed-off-by: earrage <eddie.arrage@huawei.com>
2018-10-30Update modsecurity image docker registryJingLu51-1/+1
The clover-ns-modsecurity-crs docker image is now built by Jenkins and uploaded to the OPNFV docker hub. This patch updates the modsecurity deployment kubernetes manifest to pull image from OPNFV docker hub. Change-Id: I7e8afd27f726600de9247fc98fe3f5d491000f1c Signed-off-by: JingLu5 <lvjing5@huawei.com>
2018-10-27Merge "Fix broken jmeter binary links"Stephen Wong2-2/+2
2018-10-26Docs IndexTrevor Bramwell2-1/+19
Adds the missing index.rst needed to build the documentation. Change-Id: I6ff27873559718a5ca4e3120646a6ef347120c8a Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
2018-10-22Fix broken jmeter binary linksearrage2-2/+2
- Docker container builds are failing due to expired links for the jmeter binary for jmeter-master and jmeter-slave - Updated link provided Change-Id: Icb364bc234b118e0c00359a04b51461142f832af Signed-off-by: earrage <eddie.arrage@huawei.com>
2018-10-22Initial commit for Spark to analyze visibility dataearrage13-0/+532
- Add Apache Spark 2.3 with native Kubernetes support. - Runs self contained within K8s cluster in clover-system namespace. One container (clover-spark) includes Clover Spark JAR artifact. This container interacts with the K8s API to spawn a spark-driver pod. This pod in turn spawns executor pods to execute Spark jobs. - Currently JAR is included in source for convenience and must be built with sbt (install sbt and execute sbt package) - Includes JAR from DataStax to provide Cassandra connector to analyze Cassandra schemas as RDDs (Resilient Distributed Dataset). - Includes Redis interface JAR to write analyzed data back to visibility (UI, CLI or API). - Second container (clover-spark-submit) submits Spark jobs continuously to allow Spark to be operated entirely within the cluster. - Two Spark jobs (CloverSlow, CloverFast) allows some analytics to be provided in real-time and other analytics to be provided over longer horizons. - Each Spark job spawns two executor pods. - Includes yaml manifest to deploy clover-spark-submit with the necessary RBAC permissions to interact with the K8s API. - Data analyzed includes tracing and metrics schemas obtained by clover-collector and written to Cassandra. - Docker builds of clover-spark and clover-spark-submit are provided and will be pushed as OPNFV DockerHub images in a separate patch. Change-Id: I2e92c41fd75d4ebba948c0f8cb60face57005e50 Signed-off-by: earrage <eddie.arrage@huawei.com>
2018-10-21Merge "Add visibility API in controller and CLI"Stephen Wong9-37/+231
2018-10-21Merge "Add nginx services control functionality"Stephen Wong11-42/+230
2018-10-21Merge "Improve data ingestion reliability and functionality"Stephen Wong8-86/+128
2018-10-21Merge "Remove former check_envs call"Stephen Wong1-2/+0
2018-10-21Merge "Updated from global requirements"Stephen Wong1-3/+3
2018-10-21Merge "Add spinnaker configguide"Stephen Wong4-1/+257
2018-10-19Add initial Istio validation testcase for CIJingLu55-17/+83
This work adds testcase for validating Istio installation Change-Id: Ie895b27eb4b3fbec23cf9519699f5fe37236153c Signed-off-by: JingLu5 <lvjing5@huawei.com>
2018-10-17Remove former check_envs callCédric Ollivier1-2/+0
Else it raises exceptions in __init__(). Change-Id: I4af1f59fbb9a94da1df132e750525fcdb5d0917b Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-10-17Add spinnaker configguidewutianwei4-1/+257
- Supply manifest for creation PV - Add the spinnaker as a service documentation - set the ci.jenkins.enabled to false Change-Id: I66d91aa133085c6c5910913da4303c05fcba9800 Signed-off-by: wutianwei <wutianwei1@huawei.com>
2018-10-16Updated from global requirementsCédric Ollivier1-3/+3
Change-Id: I81e92ff3db9ca401cc700ff4a9c28c740f7e9060 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-10-15Add visibility API in controller and CLIearrage9-37/+231
- Modify get visibility to retrieve config and stats - Add visibility REST API in controller to clear, set, and get from redis - Add example yaml to set visibility (service list by name, metric suffixes/prefixes, and custom metrics) from CLI - Modify example yaml to start visibility (collector) for Istio 1.0 from CLI Change-Id: I43304ff6d70bb4b817b345b9c383ce3621fb06c7 Signed-off-by: earrage <eddie.arrage@huawei.com>
2018-10-11Add nginx services control functionalityearrage11-42/+230
- Modify nginx server template to use clover-controller in clover-system namespace for file upload - Add ability to set nginx server paths and move uploaded files in clover-controller and set/get file upload metadata - Add cloverctl commands to set nginx server and lb configurations - Add example yaml files to configure nginx server and lb v1/v2 from cloverctl - Modify service definition for http-lb versions to distinguish the two versions for GRPC messaging from clover-controller in SDC deployment yaml Change-Id: I5c6866c5ff3de358939c58ea8c0bde64a69c6eca Signed-off-by: earrage <eddie.arrage@huawei.com>
2018-10-10Improve data ingestion reliability and functionalityearrage8-86/+128
- Modify deployment namespace to clover-system and account for cassandra moving to the clover-system namespace - Increase k8s compute resource assigned to cassandra to deal with performance issues - Add additional fields (user-agent, request/response size, status codes) to span schema definition and modify primary keys - Improve exception handling to prevent collect process from crashing - Minor changes to support tracing/monitoring with Istio 1.0 - Inhibit logging for debug messages - Increase time back and number of traces to fetch in each sampling interval to deal with Jaeger REST interface returning trace data out of order under load (tested to 300 conn/sec; 12K connections currently) - Move trace insert into batch mode to cassandra - Read visibility services to analyze from redis rather than defaults (cloverctl, UI or clover-controller REST will set) - Remove local directory copies in docker build, as image is based on base clover container Change-Id: Ibae98ef5057e52a6eeddd9ebbcfaeb644caec36c Signed-off-by: earrage <eddie.arrage@huawei.com>
2018-10-06Updated from global requirementsCédric Ollivier1-1/+1
Change-Id: I92fc0937130b0c70dc5888b64b1082e8816969d4 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-09-19Merge "Local Documentation Builds"Stephen Wong5-1/+24
2018-09-19Merge "Add ModSecurity config guide"Stephen Wong8-3/+370
2018-09-19Merge "Spinnaker as a Service"Stephen Wong22-5/+1416