summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2018-04-25Merge "Experimental commit for A-B testing with Clover Fraser release and on ↵Stephen Wong15-4/+646
the SDC application"
2018-04-25Merge "Created config guide for SDC sample app"Stephen Wong6-0/+546
2018-04-25Created config guide for SDC sample appEddie Arrage6-0/+546
- Overview with micro-service diagram - Source diagram file in GIMP with layers for editing by others - Deploying the sample - Using the sample - Exposing Jaeger Tracing and Prometheus monitoring browser interfaces - Modifying run-time micro-service configuration including modifying load balancer server list and adding a custom snort rule - Uninstalling the sample - Updated overview with service description, table and traffic flow description, general edits - Link to A/B configuration guide and doc index file - Additional edits - Diagram for Jaeger UI with SDC Change-Id: I5d851316c05a9e1bd48c8aab5511a98116e6893d Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-04-24Simplification of Clover Fraser official docsStephen Wong4-4/+4
(Double commit from master:I89adbef74aa74071a055dcdf62aa0925e263ffe3, gerrit 56167) Change-Id: I86f12d5ba67f09177eca758c184c614ee9d6dd9d Signed-off-by: Stephen Wong <stephen.kf.wong@gmail.com>
2018-04-24Merge "Fix snort rule with blank content & WR packet in alert"Stephen Wong2-12/+20
2018-04-23Fraser official docs simplificationStephen Wong3-181/+165
[TBD]: link to configguide for sample VNF, link to A-B testing configguide Change-Id: I89adbef74aa74071a055dcdf62aa0925e263ffe3 Signed-off-by: Stephen Wong <stephen.kf.wong@gmail.com>
2018-04-23Experimental commit for A-B testing with Clover Fraser releaseStephen Wong15-4/+646
and on the SDC application Change-Id: I6e1bd84a6d674a2c4c4484722b20415f5402a59c Signed-off-by: Stephen Wong <stephen.kf.wong@gmail.com>
2018-04-24Fix snort rule with blank content & WR packet in alertEddie Arrage2-12/+20
- Fix bug with addition of content field in rule definition that causes rules with a blank content fields to inhibit snort from starting successfully. - Write more of the packet data for snort alert into Redis - Above includes X-Real-IP, X-Forwarded-For header fields for http traffic from proxy that shows source IP Some packet data is missing in alerts from snort. Change-Id: I2c5c29e514d1ca9e8e5b9b3f7990afa87c6311b9 Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-04-19Merge "Add section to help user understand how logging works"Stephen Wong1-0/+136
2018-04-19Merge "Cover validation of elasticsearch cluster"Stephen Wong4-0/+49
2018-04-18Minor edits to release notes and user guidesrpaik2-11/+11
Change-Id: I093fb995f108194367334c4215780b5884d0207a Signed-off-by: rpaik <rpaik@linuxfoundation.org>
2018-04-17Cover validation of elasticsearch clusterYujun Zhang4-0/+49
- cluster health is not red - indics found - log entry created by istio found - requests in and out http load balance matches pytest is used as the test runner and wrapped in `validate.py` Change-Id: Iad540b69d05118fadc97df679cf3424513c15e38 Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
2018-04-17Merge "Remove unused argument"Stephen Wong1-1/+0
2018-04-17Merge "Added dependent python packages to Clover container"Stephen Wong1-1/+2
2018-04-17Merge "Extended snort rule add to allow content field"Stephen Wong4-11/+36
2018-04-17Add section to help user understand how logging worksYujun Zhang1-0/+136
Change-Id: Iebfb747450cc08e930eabd36a87670236b23ffff Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
2018-04-16Extended snort rule add to allow content fieldEddie Arrage4-11/+36
- Exposed the 'content' field in the GRPC server AddRules method - Allows the 'MALWARE-CNC User-Agent ASafaWeb Scan' signature in the community rules to be copied to local rules - Above ensures more deterministic alerts by snort each time the signature is hit - Added here to support the SDC configuration guide, which details how to add this scan rule via GRPC client script Change-Id: I6945c1e500075444134543bb9eb6003a03f1d5cc Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-04-16Added dependent python packages to Clover containerEddie Arrage1-1/+2
- Added pip grpcio and argparse packages to docker build - Allows service (nginx/snort) client sample scripts to be executed using the Clover container without having to clone the repo Change-Id: Ifeda6d58a9a381cb80372255f41ad703a089ea4b Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-04-16Remove unused argumentYujun Zhang1-1/+0
`BRANCH` is no longer required since we copy source code from working directory instead of remote git repository. Change-Id: I44776538a9efbca72e8d165e7790603cdafbe395 Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
2018-04-15Merge "Add SDC deploy/clean scripts"Stephen Wong3-0/+80
2018-04-14Merge "Add support for install istio of specified version"Stephen Wong2-6/+8
2018-04-13Add SDC deploy/clean scriptsEddie Arrage3-0/+80
- Added deploy/clean scripts for use in Clover container - Deployment of entire SDC scenario - Deployment includes istio install for manual sidecar injection without TLS authentication (deploy.sh) - Added Jaeger tracing and Prometheus monitoring install (view.sh) - Exposes NodePort for monitoring/tracing to access UIs outside of cluster - Clean.sh attempts to remove all of the above Change-Id: Id9548a77d71465a814a6e0cb1cbdf02d37235590 Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-04-13Add support for install istio of specified versionYujun Zhang2-6/+8
Change-Id: Ibfe0002daff58d30e7fffbb8828d8853a7e963a6 Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
2018-04-12Updated from global requirementsCédric Ollivier1-1/+1
Change-Id: Ieeaf87ab920f1862e3a1b9ac3316d387ff64954f Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-04-12Merge "Update from requirements"Stephen Wong2-15/+79
2018-04-12Merge "Create index to be included in opnfvdocs site"Stephen Wong2-0/+28
2018-04-12Merge "Modified validation script for tracing to support CI"Stephen Wong3-17/+43
2018-04-12Update from requirementsYujun Zhang2-15/+79
Pipfile.lock ensures a consistent environment Change-Id: Id2e544c77a67ce8fa010fba9c357735496f62a26 Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
2018-04-12Create index to be included in opnfvdocs siteYujun Zhang2-0/+28
Change-Id: Icbfe547697a8d879f4af8d9f9fbde2211b63129c Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
2018-04-11Modified validation script for tracing to support CIEddie Arrage3-17/+43
- Changed default Jaeger ports to 16686 for use with basic kubernetes port-forward and CI scripts - Added CLI to validate script to disable istio service check by default. This requires at least a single http request to istio-ingress after Jaeger deployment. It can be enabled with 'python validate.py -s'. Port and IP address for Jaeger can optionally be specified with '-ip' and '-port' options - Modified tracing doc to add k8s port-forward example in addition to k8s expose Change-Id: I10fb4d3cccfa50370d44ed7446f67a49c538bba9 Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-04-10Updated from global requirementsCédric Ollivier1-2/+2
Change-Id: I9e6c925744ed928f7a5c6fbe54942e8b3895b1b9 Signed-off-by: Cédric Ollivier <cedric.ollivier@orange.com>
2018-04-05Add existing public redis pod/serviceEddie Arrage1-0/+45
- Use a community yaml for redis in k8s as simple data store - Redis can be used for tracing and also by the snort-ids to store alerts that can be processed by other services - If flannel is used, the redis CLI can be accessed on the host OS with redis-cli -h <flannel ip> - Within the k8s cluster, the redis service can be accessed with DNS using name 'redis' - The same yaml for redis is also included in toplevel manifest for SDC scenario. Included here if intention is to use separately (tracing only) Change-Id: Ibad283a4cc8938fe01f5de6b7743bdb5511be3af Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-04-05Merge "Fix Nginx lb in k8s/istio"Stephen Wong5-24/+56
2018-04-05Merge "Added toplevel manifests for SDC sample scenario"Stephen Wong2-1/+722
2018-04-05Merge "Aligned snort-ids default config for SDC scenario"Stephen Wong3-4/+12
2018-04-05Merge "Move design document to dedicated folder"Stephen Wong4-0/+15
2018-04-04Add clover initial docker image build scriptQiLiang9-14/+106
- install dependent deb/pip packages - install basic tools istioctl, kubectl - install clover source code - build/upload docker image script - update requirements.txt - update module import path - To use this image use need setup kube-config file. e.g. `docker run -v /root/config:/root/.kube/config -it clover bash` Change-Id: I91044bb99ce8e2b785ef03212d961a97b3d42233 Signed-off-by: QiLiang <liangqi1@huawei.com>
2018-04-04Move design document to dedicated folderYujun Zhang4-0/+15
Change-Id: I20c85b7116cd2b29d0efcaae5ee0b0381a685bbb Signed-off-by: Yujun Zhang <zhang.yujunz@zte.com.cn>
2018-04-04Aligned snort-ids default config for SDC scenarioEddie Arrage3-4/+12
- Modified snort-ids alert process to use k8s DNS name 'proxy-access-control' to align with SDC scenario naming - Added default port 50054 to the manifest yaml template and rendering script for communication with proxy-access-control Change-Id: Ib04ee75e5d8ea9921b16b3b4469bed87b1cd2018 Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-04-03Fix Nginx lb in k8s/istioEddie Arrage5-24/+56
- Provide workaround to make nginx lb work properly - nginx_client sample can modify default load balancing from three to two servers at runtime - Ensure port 9180 is used for default deploy for lb and servers - Modify render_yaml to specify deploy_name so that clover-server1, 2, 3 can be used for default lb config - Ensure proxy template is aligned to lb to allow the source IP from originating host to be propagated to final destination - Fix default nginx proxy server_name to 'proxy-access-control' and change default proxy destination to 'http-lb' - Split lb service_type to 'lbv1' and 'lbv2' to provide an example of how to modify the run-time configuration of the load balancer after deployment - modify http-lb-v2 to use clover-server4/5 instead of the defualt clover-server1/2/3 - modify http-lb-v1 to use clover-server1/2 instead of 1/2/3 - Aligned pod IP retrival method with nginx_client.py Change-Id: I73fa60a69c93ae1e82a477ef6283c00f67a21360 Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-04-03Added toplevel manifests for SDC sample scenarioEddie Arrage2-1/+722
- Added missing k8s manifest yaml files for overall service delivery controller scenario - cannot be deployed coherently without this manifest - One file for private docker registry and one for opnfv public registry - Outlined in JIRA ticket CLOVER-16 and validated per description - Includes ingress rule, community redis pod/service and deployments for http-lb (v1/v2), snort-ids, proxy-access-controller, and clover-server1-5 - All above pod/deployment naming matches default container configuration - Tested with istio manual injection Change-Id: Ia03782b38020d744ab00c99adbf4832d15bbd9f3 Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-03-31Merge "Clover initial commit for servicemesh/route_rules, ↵Stephen Wong8-0/+578
orchestration/kube_client, and tools/clover_validate_rr"
2018-03-30Clover initial commit for servicemesh/route_rules,Stephen Wong8-0/+578
orchestration/kube_client, and tools/clover_validate_rr Add an 'orchestration' directory. Please note that 'orchestration' does NOT mean Clover does any orchestration --- similar to how Clover doesn't by itself implement tracing or logging, orchestration is a directory for code related to Docker orchestration client --- such as k8s client kube_client utilizes the Kubernetes python client (a dependency) to perform tasks against Kubernetes API server. For this commit, it is only tested for weighted route rule verification, it does three tasks: (1) get a list of pods under a namespace --- pod dictionary now only contains pod name and label dictionary: used to match pod name with the node name in traces from OpenTracing (2) check to see if a particular pod is up in a particular namespace: used to check if Istio pods are running in istio-system namespace (3) check if a container exists in a list of pods under a namespace: used to check if application pods have istio-proxy container running route_rule directly invokes istioctl as there isn't any Istio Python client yet. Currently it reads and parses routerules from Istio, and validates if a particular trace result matches the routerules Finally, a sample tool clover_validate_rr is provided. This tool assumes a previous test has been ran (with an id with both the route-rule-under-test and corresponding traces are stored --- currently the assumption is tests were ran with redis-master running on system). The tool can be invoked: python clover_validate_rr.py -t <test-id> -s <service name> where test-id is the ID of the test (most likely uuid) and service name is the name of the service running in the Kubernetes cluster upon which test traces should be fetched against Change-Id: Ic8ab6efc23c71ac4643bee796ef986a86f6fc7dd Signed-off-by: Stephen Wong <stephen.kf.wong@gmail.com>
2018-03-31Merge "Added initial nginx services"Stephen Wong20-0/+1255
2018-03-31Develop snort IDS and content inspect serviceEddie Arrage17-0/+1319
- Initial commit to show potential structure of a sample service - This wil be part of a larger sample application currently dubbed Service Delivery Controller - Docker container needs to be built and employs open-source Linux packages - Service is deployable in Istio service mesh using provided yaml - Control snort daemon and add custom rules with GRPC messaging - Process snort alerts actively and send to redis and upstream service mesh components - Integrates a web server for better HTTP signature detection - Improved build script for CI with variables - Render k8s yaml snort manifest dynamically with command line options - Improve snort_client sample script for runtime modifications including passing args on CLI, error checking - Update nginx proxy interface - Added logging to snort server and alert process Change-Id: Ic56f9fcd9ed21f64b84b85ac8ee280d69af7b7c9 Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-03-30Merge "Add Functest test hook 2"Stephen Wong1-2/+38
2018-03-30Merge "Initial commit for logging installation and validataion"Stephen Wong6-0/+526
2018-03-30Merge "Initial commit for tracing"Stephen Wong4-0/+358
2018-03-30Added initial nginx servicesEddie Arrage20-0/+1255
- Proxy allows ingress traffic to be sent to another element in service mesh - Mirroring is also in the default configuration - Default configuration is to proxy to a clover-server and mirror to snort-ids - A location_path (URI in HTTP requests) can be reconfigured to restrict proxing; default to '/' - A proxy_path can be reconfigured to specify an alternate destination - A mirror path can be reconfigured to specify where traffic will be spanned - The default server_port (listen port) for the proxy is 9180 but can be reconfigured - The default server_name is http-proxy but can be reconfigured - Reconfiguration is done over GRPC with jinja2 template for nginx - Currently snort ids sends alerts to proxy with stub code in GRPC - Refactored the code to have a nginx base with subservices - Proxy, Load Balancer (lb), and Server can share code - mainly GRPC server - Nginx subservices have separate docker builds - Improved build scripts for CI - Render yaml manifests dynamically - Improve nginx_client for runtime modifications (but not really useful yet) Change-Id: Icbff6890021bcc8a8da4690c9261205d6e1ca43a Signed-off-by: Eddie Arrage <eddie.arrage@huawei.com>
2018-03-30Add Functest test hook 2QiLiang1-2/+38
- Temporary run functest testcase after all clover env setup. - TODO: Use jenkins to trigger functest job. Change-Id: I5f620496d747c4d742c7bbf8bb825616f8c69499 Signed-off-by: QiLiang <liangqi1@huawei.com>