Initial commit for Clover Collector

- Added a container named clover-collector using clover
container as a base with build script
- GRPC server to manage collector process
- Cassandra DB client interface to initialize visibility keyspace
- Init messaging adds table schemas for tracing - traces & spans
- Adds table for monitoring - metrics
- Does not implement Cassandra server but developed using
public Cassandra docker container
- Collector process in simple loop that periodically fetches
traces and monitoring data and inserts to Cassandra - not optimized
for batch retrieval yet for monitoring
- CLI interface added to collector process and used
by GRPC server for configuration
- Simple GRPC client script to test GRPC server and start/stop
of collector process
- Collector process can be configured with access for tracing,
monitoring and Cassandra
- Added a return value in monitoring query method

- Added ability to truncate tracing, metrics and spans tables
in cql
- Added cql prepared statements and batch insert for metrics
and spans
- Align cql connection to cql deployment within k8s
- Fix issue with cql host list using ast and collect process
args with background argument
- Added redis interface to accept service/metric list
externally for monitoring (will work in conjunction
with clover-controller)
- Use k8s DNS names and default ports for monitoring, tracing
and cassandra
- Added yaml manifest renderer/template for collector

Change-Id: I3e4353e28844c4ce9c185ff4638012b66c7fff67
Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>

1 files changed, 73 insertions, 0 deletions

diff --git a/clover/collector/yaml/render_yaml.py b/clover/collector/yaml/render_yaml.py
new file mode 100644
index 0000000..c1d8be7
--- /dev/null
+++ b/clover/collector/yaml/render_yaml.py
@@ -0,0 +1,73 @@
+# Copyright (c) Authors of Clover
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Apache License, Version 2.0
+# which accompanies this distribution, and is available at
+# http://www.apache.org/licenses/LICENSE-2.0
+
+import argparse
+
+from jinja2 import Template
+
+
+def render_yaml(args):
+    template_file = 'manifest.template'
+    out_file = args['deploy_name'] + '.yaml'
+
+    try:
+        with open(template_file) as f:
+            tmpl = Template(f.read())
+        output = tmpl.render(
+            image_path=args['image_path'],
+            image_name=args['image_name'],
+            image_tag=args['image_tag'],
+            deploy_name=args['deploy_name'],
+            grpc_port=args['grpc_port'],
+            monitor_port=args['monitor_port'],
+            redis_port=args['redis_port'],
+            cass_port=args['cass_port'],
+            trace_port=args['trace_port']
+        )
+        with open(out_file, "wb") as fh:
+            fh.write(output)
+        return "Generated manifest for {}".format(args['deploy_name'])
+    except Exception as e:
+        print(e)
+        return "Unable to generate manifest for {}".format(
+                                        args['deploy_name'])
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+            '--image_name', default='clover-collector',
+            help='The image name to use')
+    parser.add_argument(
+            # '--image_path', default='opnfv',
+            '--image_path', default='localhost:5000',
+            help='The path to the image to use')
+    parser.add_argument(
+            # '--image_tag', default='opnfv-6.0.0',
+            '--image_tag', default='latest',
+            help='The image tag to use')
+    parser.add_argument(
+            '--deploy_name', default='clover-collector',
+            help='The k8s deploy name to use')
+    parser.add_argument(
+            '--redis_port', default='6379',
+            help='The redis port to connect for management')
+    parser.add_argument(
+            '--monitor_port', default='9090',
+            help='The Prometheus monitoring port')
+    parser.add_argument(
+            '--grpc_port', default='50054',
+            help='The GRPC server port for collector management')
+    parser.add_argument(
+            '--trace_port', default='16686',
+            help='The Jaeger tracing port')
+    parser.add_argument(
+            '--cass_port', default='9042',
+            help='The Cassandra port')
+
+    args = parser.parse_args()
+    print(render_yaml(vars(args)))