| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
OPNFV codebase; likewise for *.jar files in source directories
Change-Id: I11362ae444a299a5fef9e3f87e4fa84b7bc79c07
Signed-off-by: Stephen Wong <stephen.kf.wong@gmail.com>
|
|
- Change tag from local repo to opnfv for clover-spark
container, which spark-submit references
- Move update of configured visibility services into
loop
Change-Id: I9d09a851977c6006ea2cbba45d7a28be6a4660b9
Signed-off-by: earrage <eddie.arrage@huawei.com>
|
|
- Add Apache Spark 2.3 with native Kubernetes support.
- Runs self contained within K8s cluster in clover-system
namespace. One container (clover-spark) includes Clover Spark
JAR artifact. This container interacts with the
K8s API to spawn a spark-driver pod. This pod in turn spawns executor
pods to execute Spark jobs.
- Currently JAR is included in source for convenience and must be
built with sbt (install sbt and execute sbt package)
- Includes JAR from DataStax to provide Cassandra connector to analyze
Cassandra schemas as RDDs (Resilient Distributed Dataset).
- Includes Redis interface JAR to write analyzed data back to visibility
(UI, CLI or API).
- Second container (clover-spark-submit) submits Spark jobs
continuously to allow Spark to be operated entirely within the cluster.
- Two Spark jobs (CloverSlow, CloverFast) allows some analytics to be
provided in real-time and other analytics to be provided over longer
horizons.
- Each Spark job spawns two executor pods.
- Includes yaml manifest to deploy clover-spark-submit with the
necessary RBAC permissions to interact with the K8s API.
- Data analyzed includes tracing and metrics schemas obtained by
clover-collector and written to Cassandra.
- Docker builds of clover-spark and clover-spark-submit are provided
and will be pushed as OPNFV DockerHub images in a separate patch.
Change-Id: I2e92c41fd75d4ebba948c0f8cb60face57005e50
Signed-off-by: earrage <eddie.arrage@huawei.com>
|
Stephen Wong
earrage