diff options
| author |   Koren Lev <korenlev@gmail.com> | 2017-09-26 20:41:34 +0300 |
|---|---|---|
| committer |   Yaron Yogev <yaronyogev@gmail.com> | 2017-10-03 09:45:52 +0000 |
| commit | fbbaf20912c79bd99a5c3696850d70c11965f56b (patch) | |
| tree | 91a55c3b911a137dbd5835a65bd0090cd0d17e5d | |
| parent | ee2bdf83ae1f31933c7cde2ce2eef1a9b23d64e3 (diff) | |
new rst format for docs
Change-Id: Ic0e74f3099db56e49bde55aa060a13e9588284d5
Signed-off-by: Koren Lev <korenlev@gmail.com>
(cherry picked from commit 0c5426cd309d720db1e30641e43d311ee0b751b0)
41 files changed, 6246 insertions, 0 deletions
diff --git a/docs/release/about.rst b/docs/release/about.rst new file mode 100644 index 0000000..164818a --- /dev/null +++ b/docs/release/about.rst @@ -0,0 +1,234 @@ +###############################################################################
+# Copyright (c) 2017 Koren Lev (Cisco Systems), Yaron Yogev (Cisco Systems) #
+# and others #
+# #
+# All rights reserved. This program and the accompanying materials #
+# are made available under the terms of the Apache License, Version 2.0 #
+# which accompanies this distribution, and is available at #
+# http://www.apache.org/licenses/LICENSE-2.0 #
+###############################################################################
+| Calipso.io
+| Product Description and Value
+
+|image0|
+
+Virtual and Physical networking low level details and inter-connections,
+dependencies in OpenStack, Docker or Kubernetes environments are
+currently invisible and abstracted, by design, so data is not exposed
+through any API or UI.
+
+During virtual networking failures, troubleshooting takes substantial
+amount of time due to manual discovery and analysis.
+
+Maintenance work needs to happen in the data center, virtual and
+physical networking (controlled or not) are impacted.
+
+Most of the times, the impact of any of the above scenarios is
+catastrophic.
+
+Project “Calipso” tries to illuminate complex virtual networking with
+real time operational state visibility for large and highly distributed
+Virtual Infrastructure Management (VIM).
+
+*Customer needs during maintenance:*
+
+Visualize the networking topology, easily pinpointing the location
+needed for maintenance and show the impact of maintenance work needed in
+that location.
+
+Administrator can plan ahead easily and report up his command chain the
+detailed impact – Calipso substantially lower the admin time and
+overhead needed for that reporting.
+
+*Customer need during troubleshooting:*
+
+Visualize and pinpointing the exact location of the failure in the
+networking chain, using a suspected ‘focal point’ (ex: a VM that cannot
+communicate).
+
+Monitor the networking location and alerting till the problem is
+resolved. Calipso also covers pinpointing the root cause.
+
+*Calipso is for multiple distributions/plugins and many virtual
+environment variances:*
+
+We built a fully tested unified model to deal with many variances.
+
+Supporting in initial release: VPP, OVS, LXB with all type drivers
+possible, onto 5 different OS distributions, totaling to more than 60
+variances (see Calipso-model guide).
+
+New classes per object, link and topology can be programmed (see
+development guide).
+
+*Detailed Monitoring:*
+
+Calipso provides visible insights using smart discovery and virtual
+topological representation in graphs, with monitoring per object in the
+graph inventory to reduce error vectors and troubleshooting, maintenance
+cycles for VIM operators and administrators.
+
+*We believe that Stability is driven by accurate Visibility*.
+
+Table of Contents
+
+Calipso.io Product Description and Value 1
+
+1 About 4
+
+1.1 Project Description 4
+
+2 Main modules 5
+
+2.1 High level module descriptions 5
+
+2.2 High level functionality 5
+
+3 Customer Requirements 6
+
+3.1 Releases and Distributions 7
+
+About
+=====
+
+Project Description
+-------------------
+
+Calipso interfaces with the virtual infrastructure (like OpenStack)
+through API, DB and CLI adapters, discovers the specific
+distribution/plugins in-use, their versions and based on that collects
+detailed data regarding running objects in the underlying workers and
+processes running on the different hosts. Calipso analyzes the inventory
+for inter-relationships and keeps them in a common and highly adaptive
+data model.
+
+Calipso then represents the inter-connections as real-time topologies
+using automatic updates per changes in VIM, monitors the related objects
+and analyzes the data for impact and root-cause analysis.
+
+This is done with the objective to lower and potentially eliminate
+complexity and lack of visibility from the VIM layers as well as to
+offer a common and coherent representation of all physical and virtual
+network components used under the VIM, all exposed through an API.
+
+Calipso is developed to work with different OpenStack flavors, plugins
+and installers.
+
+Calipso is developed to save network admins discovery and
+troubleshooting cycles of the networking aspects. Calipso helps estimate
+the impact of several micro failure in the infrastructure to allow
+appropriate resolutions.
+
+Calipso focuses on scenarios, which requires VIM/OpenStack maintenance
+and troubleshooting enhancements using operations dashboards i.e.
+connectivity, topology and related stats – as well as their correlation.
+
+|image1|
+
+ Main modules
+=============
+
+High level module descriptions
+------------------------------
+
+Calipso modules included with initial release:
+
+- *Scanning*: detailed inventory discovery and inter-connection
+ analysis, smart/logical and automated learning from the VIM, based on
+ specific environment version/type etc.
+
+- *Listening*: Attach to VIM message BUS and update changes in real
+ time.
+
+- *Visualization*: represent the result of the discovery in browsable
+ graph topology and tree.
+
+- *Monitoring*: Health and status for all discovered objects and
+ inter-connections: use the discovered data to configure monitoring
+ agents and gather monitoring results.
+
+- *Analysis*: some traffic analysis, impact and root-cause analysis for
+ troubleshooting.
+
+- *API:* allow integration with Calipso application’s inventory and
+ monitoring results.
+
+- *Database*: Mongo based
+
+- *LDAP*: pre-built integration for smooth attachment to corporate
+ directories.
+
+For Monitoring we are planning to utilize the work done by ‘Sensu’ and
+‘Barometer’.
+
+The project also develops required enhancements to individual components
+in OpenStack like Neutron, Telemetry API and the different OpenStack
+monitoring agents in order to provide a baseline for “Operations APIs”.
+
+High level functionality
+-------------------------
+
+*Scanning*:
+
+Calipso uses API, Database and Command-Line adapters for interfacing
+with the Cloud infrastructure to logically discover every networking
+component and it's relationships with others, building a smart topology
+and inventory.
+
+*Automated setup*:
+
+Calipso uses Sensu framework for Monitoring. It automatically deploys
+and configures the necessary configuration files on all hosts, writes
+customized checks and handlers to setup monitoring per inventory object.
+
+*Modeled analysis*:
+
+Calipso uses a unique logical model to help facilitate the topology
+discovery, analysis of inter-connections and dependencies. Impact
+Analysis is embedded, other types of analysis is possible through a
+plugin framework.
+
+*Visualization:*
+
+Using its unique dependency model calipso visualize topological
+inventory and monitoring results, in a highly customizable and modeled
+UI framework
+
+*Monitoring*:
+
+After collecting the data, from processes and workers provisioned by the
+cloud management systems, calipso dynamically checks for health and
+availability, as a baseline for SLA monitoring.
+
+*Reporting:*
+
+Calipso allows networking administrators to operate, plan for
+maintenance or troubleshooting and provides an easy to use hierarchical
+representation of all the virtual networking components.
+
+Customer Requirements
+=====================
+
+We identified an operational challenge: lack of visibility that leads to
+limited stability.
+
+The lack of operational tooling coupled with the reality of deployment
+tools really needs to get solved to decrease the complexity as well as
+assist not only deploying but also supporting OpenStack and other cloud
+stacks.
+
+Calispo integrates well with installers like Apex to offer enhanced day
+2 operations.
+
+Releases and Distributions
+--------------------------
+
+Calipso is distributed for enterprises - ‘S’ release, through
+calipso.io, and for service providers - ‘P’ release, through OPNFV.
+
+.. |image0| image:: media/image1.png
+ :width: 6.50000in
+ :height: 4.27153in
+.. |image1| image:: media/image2.png
+ :width: 6.50000in
+ :height: 3.52153in
diff --git a/docs/release/admin-guide.pdf b/docs/release/admin-guide.pdf Binary files differindex e16228c..5c42733 100644 --- a/docs/release/admin-guide.pdf +++ b/docs/release/admin-guide.pdf diff --git a/docs/release/admin-guide.rst b/docs/release/admin-guide.rst new file mode 100644 index 0000000..9afa451 --- /dev/null +++ b/docs/release/admin-guide.rst @@ -0,0 +1,1361 @@ +############################################################################### +# Copyright (c) 2017 Koren Lev (Cisco Systems), Yaron Yogev (Cisco Systems) # +# and others # +# # +# All rights reserved. This program and the accompanying materials # +# are made available under the terms of the Apache License, Version 2.0 # +# which accompanies this distribution, and is available at # +# http://www.apache.org/licenses/LICENSE-2.0 # +############################################################################### +| Calipso.io +| Administration Guide + +|image0| + +Project “Calipso” tries to illuminate complex virtual networking with +real time operational state visibility for large and highly distributed +Virtual Infrastructure Management (VIM). + +Calipso provides visible insights using smart discovery and virtual +topological representation in graphs, with monitoring per object in the +graph inventory to reduce error vectors and troubleshooting, maintenance +cycles for VIM operators and administrators. + +Calipso model, described in this document, was *built for +multi-environment and many VIM variances*, the model was tested +successfully (as of Aug 27\ :sup:`th`) against 60 different VIM +variances (Distributions, Versions, Networking Drivers and Types). + +Table of Contents + +Calipso.io Administration Guide 1 + +1 Environments config 3 + +2 UI overview 5 + +2.1 User management 7 + +2.2 Logging in and out 8 + +2.3 Messaging check 9 + +2.4 Adding a new environment 9 + +3 Preparing an environment for scanning 10 + +3.1 Where to deploy Calipso application 10 + +3.2 Environment setup 10 + +3.3 Filling the environment config data 11 + +3.4 Testing the connections 11 + +4 Links and Cliques 12 + +4.1 Adding environment clique\_types 13 + +5 Environment scanning 14 + +5.1 UI scanning request 14 + +5.2 UI scan schedule request 16 + +5.3 API scanning request 17 + +5.4 CLI scanning in the calipso-scan container 18 + +5.4.1 Clique Scanning 19 + +5.4.2 Viewing results 20 + +6 Editing or deleting environments 20 + +7 Event-based scanning 21 + +7.1 Enabling event-based scanning 21 + +7.2 Event-based handling details 22 + +8 ACI scanning 34 + +9 Monitoring enablement 36 + +10 Modules data flows 38 + +Environments config +=================== + + Environment is defined as a certain type of Virtual Infrastructure + facility the runs under a single unified Management (like an + OpenStack facility). + + Everything in Calipso application rely on environments config, this + is maintained in the **“environments\_config”** collection in the + mongo Calipso DB. + + Environment configs are pushed down to Calipso DB either through UI + or API (and only in OPNFV case Calipso provides an automated program + to build all needed environments\_config parameters for an ‘Apex’ + distribution automatically). + + When scanning and discovering items Calipso uses this configuration + document for successful scanning results, here is an example of an + environment config document: + + **{ ** + + **"name": "DEMO-ENVIRONMENT-SCHEME", ** + + **"enable\_monitoring": true, ** + + **"last\_scanned": "filled-by-scanning", ** + + **"app\_path": "/home/scan/calipso\_prod/app", ** + + **"type": "environment", ** + + **"distribution": "Mirantis", ** + + **"distribution\_version": "8.0”, ** + + **"mechanism\_drivers": ["OVS”], ** + + **"type\_drivers": "vxlan"** + + **"operational": "stopped", ** + + **"listen": true, ** + + **"scanned": false, ** + + **"configuration": [** + + **{** + + **"name": "OpenStack", ** + + **"port":”5000”, ** + + **"user": "adminuser", ** + + **"pwd": "dummy\_pwd", ** + + **"host": "10.0.0.1", ** + + **"admin\_token": "dummy\_token"** + + **}, ** + + **{** + + **"name": "mysql", ** + + **"pwd": "dummy\_pwd", ** + + **"host": "10.0.0.1", ** + + **"port": “3307”, ** + + **"user": "mysqluser"** + + **}, ** + + **{** + + **"name": "CLI", ** + + **"user": "sshuser", ** + + **"host": "10.0.0.1", ** + + **"pwd": "dummy\_pwd"** + + **}, ** + + **{** + + **"name": "AMQP", ** + + **"pwd": "dummy\_pwd", ** + + **"host": "10.0.0.1", ** + + **"port": “5673”, ** + + **"user": "rabbitmquser"** + + **}, ** + + **{** + + **"name": "Monitoring", ** + + **"ssh\_user": "root", ** + + **"server\_ip": "10.0.0.1", ** + + **"ssh\_password": "dummy\_pwd", ** + + **"rabbitmq\_pass": "dummy\_pwd", ** + + **"rabbitmq\_user": "sensu", ** + + **"rabbitmq\_port": “5671”, ** + + **"provision": "None", ** + + **"env\_type": "production", ** + + **"ssh\_port": “20022”, ** + + **"config\_folder": "/local\_dir/sensu\_config", ** + + **"server\_name": "sensu\_server", ** + + **"type": "Sensu", ** + + **"api\_port": NumberInt(4567)** + + **}, ** + + **{** + + **"name": "ACI", ** + + **"user": "admin", ** + + **"host": "10.1.1.104", ** + + **"pwd": "dummy\_pwd"** + + **}** + + **], ** + + **"user": "wNLeBJxNDyw8G7Ssg", ** + + **"auth": {** + + **"view-env": [** + + **"wNLeBJxNDyw8G7Ssg"** + + **], ** + + **"edit-env": [** + + **"wNLeBJxNDyw8G7Ssg"** + + **]** + + **}, ** + + **}** + + Here is a brief explanation of the purpose of major keys in this + environment configuration doc: + + **Distribution**: captures type of VIM, used for scanning of + objects, links and cliques. + + **Distribution\_version**: captures version of VIM distribution, + used for scanning of objects, links and cliques. + + **Mechanism\_driver**: captures virtual switch type used by the VIM, + used for scanning of objects, links and cliques. + + **Type\_driver**: captures virtual switch tunneling type used by the + switch, used for scanning of objects, links and cliques. + + **Listen**: defines whether or not to use Calipso listener against + the VIM BUS for updating inventory in real-time from VIM events. + + **Scanned**: defines whether or not Calipso ran a full and a + successful scan against this environment. + + **Last\_scanned**: end time of last scan. + + **Operational**: defines whether or not VIM environment endpoints + are up and running. + + **Enable\_monitoring**: defines whether or not Calipso should deploy + monitoring of the inventory objects running inside all environment + hosts. + + **Configuration-OpenStack**: defines credentials for OpenStack API + endpoints access. + + **Configuration-mysql**: defines credentials for OpenStack DB + access. + + **Configuration-CLI**: defines credentials for servers CLI access. + + **Configuration-AMQP**: defines credentials for OpenStack BUS + access. + + **Configuration-Monitoring**: defines credentials and setup for + Calipso sensu server (see monitoring-guide for details). + + **Configuration-ACI**: defines credentials for ACI switched + management API, if exists. + + **User and auth**: used for UI authorizations to view and edit this + environment. + + **App-path**: defines the root directory of the scanning + application. + + \* This guide will help you understand how-to add new environment + through the provided Calispo UI module and then how-to use this + environment (and potentially many others) for scanning and real-time + inventories collection. + +UI overview +============ + + Cloud administrator can use the Calipso UI for he’s daily tasks. + Once Calipso containers are running (see quickstart-guide) the UI + will be available at: + + http://server-ip:80 , default login credentials: admin/123456. + + Before logging in, while at the main landing page, a generic + information is provided. + + Post login, at the main dashboard you can click on “Get started” and + view a short guide for using some of the basic UI functions, + available at: + `server-ip/getstarted <http://korlev-calipso-dev.cisco.com/getstarted>`__. + + The main areas of interest are shown in the following screenshot: + + *Main areas on UI:* + + |image1| + + *Main areas details:* + + **Navigation Tree(1):** Hierarchy searching through the inventory + using objects and parents details, to lookup a focal point of + interest for graphing or data gathering. + + **Main functions (2):** Jumping between highest level dashboard (all + environments), specific environment and some generic help is + provided in this area. + + **Environment Summary (3):** The central area where the data is + exposed, either through graph or through widget-attribute-listing. + + **Search engine (4):** Finding interesting focal points faster + through basic object naming lookups, then clicking on results to get + transferred directly to that specific object dashboard. Searches are + conducted across all environments. + + **More settings (5):** In this area the main collections of data are + exposed, like scans, schedules, messaging, clique\_types, + link\_types and others. + + **Graph or Data toggle (6):** When focusing on a certain focal + point, this button allows changing from a graph-view to simple + data-view per request, if no graph is available for a certain object + the data-view is used by default, if information is missing try this + button first to make sure the correct view is chosen. + +User management +--------------- + + The first place an administrator might use is the user’s + configurations, this is where a basic RBAC is provided for + authorizing access to the UI functions. Use the ‘settings’ button + and choose ‘users’ to access: + + |image2| + + Editing the admin user password is allowed here: + +|image3| + + Note: + + The ‘admin’ user is allowed all functions on all environments, you + shouldn’t change this behavior and you should never delete this + user, or you’ll need re-install Calipso. + + Adding new user is provided when clicking the “Create new user” + option: + + *Creating a new user:* + +|image4| + + Before environments are configured there is not a lot of options + here, once environments are defined (one or more), users can be + allowed to edit or view-only those environments. + + Logging in and out +------------------- + + To logout and re-login with different user credentials you can click + the username option and choose to sign out: + + |image5| + +Messaging check +--------------- + + When calispo-scan and calipso-listen containers are running, they + provide basic messages on their processes status, this should be + exposed thorough the messaging system up to the UI, to validate this + choose ‘messages’ from the settings button: + +|image6| + +Adding a new environment +------------------------ + + As explained above, environment configuration is the pre requisite + for any Calipso data gathering, goto “My Environments” -> and “Add + new Environment” to start building the environment configuration + scheme: + +|image7| + +Note: this is automated with OPNFV apex distro, where Calipso +auto-discovers all credentials + +Preparing an environment for scanning +===================================== + + Some preparation is needed for allowing Calipso to successfully + gather data from the underlying systems running in the virtual + infrastructure environment. This chapter explain the basic + requirements and provide recommendations. + +Where to deploy Calipso application +----------------------------------- + + Calipso application replaces the manual discovery steps typically + done by the administrator on every maintenance and troubleshooting + cycles, It needs to have the administrators privileges and is most + accurate when placed on one of the controllers or a“jump server” + deployed as part of the cloud virtual infrastructure, Calipso calls + this server a “Master host”. + + Consider Calipso as yet another cloud infrastructure module, similar + to neutron, nova. + + Per supported distributions we recommend installing the Calipso + application at: + +1. Mirantis: on the ‘Fuel’ or ‘MCP’ server. + +2. RDO/Packstack: where the ansible playbooks are deployed. + +3. Canonical/Ubuntu: on the juju server. + +4. Triple-O/Apex: on the jump host server. + +Environment setup +----------------- + + The following steps should be taken to enable Calispo’s scanner and + listener to connect to the environment controllers and compute + hosts: + +1. OpenStack API endpoints : Remote access user accessible from the + master host with the required credentials and allows typical ports: + 5000, 35357, 8777, 8773, 8774, 8775, 9696 + +2. OpenStack DB (MariaDB or MySQL): Remote access user accessible from + the master host to ports 3306 or 3307 allowed access to all Databases + as read-only. + +3. Master host SSH access: Remote access user with sudo privileges + accessible from the master host through either user/pass or rsa keys, + the master host itself should then be allowed access using rsa-keys + (password-less) to all other infrastructure hosts, all allowing to + run sudo CLI commands over tty, when commands entered from the master + host source itself. + +4. AMQP message BUS (like Rabbitmq): allowed remote access from the + master host to listen for all events generated using a guest account + with a password. + +5. Physical switch controller (like ACI): admin user/pass accessed from + master host. + + *Note: The current lack of operational toolsets like Calipso forces + the use of the above scanning methods, the purpose of Calipso is to + deploy its scanning engine as an agent on all environment hosts, in + such scenario the requirements above might be deprecated and the + scanning itself can be made more efficient.* + +Filling the environment config data +----------------------------------- + + As explained in chapter 1 above, environment configuration is the + pre requisite and all data required is modeled as described. See + api-guide for details on submitting those details through calispo + api module. When using the UI module, follow the sections tabs and + fill the needed data per help messages and the explanations in + chapter 1. + + Only the AMQP, Monitoring and ACI sections in environment\_config + documents are optional, per the requirements detailed below on this + guide. + +Testing the connections +----------------------- + + Before submitting the environment\_config document it is wise to + test the connections. Each section tab in the environment + configuration has an optional butting for testing the connection + tagged “test connection”. When this button is clicked, a check is + made to make sure all needed data is entered correctly, then a + request is sent down to mongoDB to the “connection\_tests” + collection. Then the calispo scanning module will make the required + test and will push back a response message alerting whether or not + this connection is possible with the provided details and + credentials. + + *Test connection per configuration section:* + +|image8| + + With the above tool, the administrator can be assured that Calipso + scanning will be successful and the results will be an accurate + representation of the state of he’s live environment. + +Links and Cliques +================== + + A very powerful capability in Calipso allows it to be very adaptive + and support many variances of VIM environments, this capability lies + in its objects, links and cliques models enabling the scanning of + data and analysis of inter-connections and creation of many types of + topology graphs.. + + Please refer to calipso-model document for more details. + + The UI allows viewing and editing of Link types and Clique types + through the settings options: + + *Link types:* + +|image9| + + Note: + + We currently recommend not to add nor edit the Link types pre-built + in Calipso’s latest release (allowed only for the ‘admin’ user), as + it is tested and proven to support more than 60 popular VIM + variances. + + An administrator might choose to define several environment specific + **Clique types** for creating favorite graphs using the focal\_point + objects and link\_types lists already built-in: + + Adding environment clique\_types +---------------------------------- + + Use either the API or the UI to define specific environment + clique\_types. + + For adding clique\_types, use settings menu and choose “Create new + clique type” option, then provide a specific environment name (per + previous environment configurations), define a focal\_point (like: + instance, or other object types) and a list of resulted link\_types + to include in the final topology graph. Refer to calipso-model + document for more details. + + Clique\_types are needed for accurate graph buildup, before sending + a scan request. + + Several defaults are provided with each new Calipso release. + + *Clique types:* + +|image10| + + Note: ask calipso developers for recommended clique\_types + (pre-built in several Calipso deployments), per distribution + variance, fully tested by Calipso developers: + +Environment scanning +==================== + + Once environment is setup correctly, environment\_config data is + filled and tested, scanning can start. This is can be done with the + following four options: + +1. UI scanning request + +2. UI scan schedule request + +3. API scanning or scheduling request. + +4. CLI scanning in the calipso-scan container. + + The following sections with describe those scanning options. + +UI scanning request +------------------- + + This can be accomplished after environment configuration has been + submitted, the environment name will be listed under “My + environment” and the administrator can choose it from the list and + login to the specific environment dashboard: + +|image11| + + Onces inside a specific environment dashboard the administrator can + click the scanning button the go into scanning request wizards: + +|image12| + + In most cases, the only step needed to send a scanning request is to + use all default options and click the “Submit” button: + +|image13| + + Scanning request will propagate into the “scans” collection and will + be handled by scan\_manager in the calipso-scan container. + + *Scan options*: + + **Log level**: determines the level and details of the scanning + logs. + + **Clear data**: empty historical inventories related to that + specific environment, before scanning. + + **Only inventory**: creates inventory objects without analyzing for + links. + + **Only links**: create links from pre-existing inventory, does not + build graph topologies. + + **Only Cliques**: create graph topologies from pre-existing + inventory and links. + +UI scan schedule request +------------------------ + + Scanning can be used periodically to dynamically update the + inventories per changes in the underlying virtual environment + infrastructure. This can be defined using scan scheduling and can be + combined with the above one time scanning request. + + |image14| + + Scheduled scans has the same options as in single scan request, + while choosing a specific environment to schedule on and providing + frequency details, timer is counted from the submission time, scan + scheduling requests are propagated to the “scheduled\_scans” + collection in the Calispo mongoDB and handled by scan\_manager in + the calispo-scan container. + +API scanning request +-------------------- + + Follow api-guide for details on submitting scanning request through + Calipso API. + +CLI scanning in the calipso-scan container +------------------------------------------ + + When using the UI for scanning messages are populated in the + “Messages” menu item and includes several details for successful + scanning and some alerts. When more detailed debugging of the + scanning process is needed, administrator can login directly to the + calispo-scan container and run the scanning manually using CLI: + +- Login to calispo-scan container running on the installed host: + + **ssh scan@localhost –p 3002** , using default-password: ‘scan’ + +- Move to the calipso scan application location: + + **cd /home/scan/calipso\_prod/app/discover** + +- Run the scan.py application with the basic default options: + + **python3 ./scan.py -m /local\_dir/calipso\_mongo\_access.conf -e + Mirantis-8** + + Default options: -m points to the default location of mongoDB access + details, -e points to the specific environment name, as submitted to + mongoDB through UI or API. + + Other optional scanning parameters, can be used for detailed + debugging: + + | The scan.py script is located in directory app/discover in the + Calipso repository. + | To show the help information, run scan.py with –help option, here + is the results + + : + + Usage: scan.py [-h] [-c [CGI]] [-m [MONGO\_CONFIG]] [-e [ENV]] [-t + [TYPE]] + + [-y [INVENTORY]] [-s] [-i [ID]] [-p [PARENT\_ID]] + + [-a [PARENT\_TYPE]] [-f [ID\_FIELD]] [-l [LOGLEVEL]] + + [--inventory\_only] [--links\_only] [--cliques\_only] + [--clear] + + + + Optional arguments: + + -h, --help show this help message and exit + + -c [CGI], --cgi [CGI] + + read argument from CGI (true/false) + (default: false) + + -m [MONGO\_CONFIG], --mongo\_config [MONGO\_CONFIG] + + name of config file with MongoDB server + access details + + -e [ENV], --env [ENV] + + name of environment to scan (default: WebEX- + + Mirantis@Cisco) + + -t [TYPE], --type [TYPE] + + type of object to scan (default: + environment) + + -y [INVENTORY], --inventory [INVENTORY] + + name of inventory collection (default: + 'inventory') + + -s, --scan\_self scan changes to a specific object (default: + False) + + -i [ID], --id [ID] ID of object to scan (when scan\_self=true) + + -p [PARENT\_ID], --parent\_id [PARENT\_ID] + + ID of parent object (when scan\_self=true) + + -a [PARENT\_TYPE], --parent\_type [PARENT\_TYPE] + + type of parent object (when scan\_self=true) + + -f [ID\_FIELD], --id\_field [ID\_FIELD] + + name of ID field (when scan\_self=true) + (default: 'id', + + use 'name' for projects) + + -l [LOGLEVEL], --loglevel [LOGLEVEL] + + logging level (default: 'INFO') + + --inventory\_only do only scan to inventory (default: False) + + --links\_only do only links creation (default: False) + + --cliques\_only do only cliques creation (default: False) + + --clear clear all data prior to scanning (default: + False) + + A simple scan.py run will look, by default, for a local MongoDB + server. Assuming running this from within the scan container + running, the administrator needs to point it to use the specific + MongoDB server. This is done using the Mongo access config file + created by the installer (see install-guide for details):: + + ./scan.py -m your\_mongo\_access.conf + + Environment needs to be specified explicitly, no default environment + is used by scanner. + + By default, the inventory collection, named 'inventory', along with + the accompanying collections: "links", "cliques", "clique\_types" + and "clique\_constraints" are used to place initial scanning data + results. + + | As a more granular scan example, for debugging purposes, using + environment "RDO-packstack-Mitaka", pointing scanning results to + an inventory collection named "RDO": + | The accompanying collections will be automatically created and + renamed accordingly: + | "RDO\_links", "RDO\_cliques", "RDO\_clique\_types" and + "RDO\_clique\_constraints". + + Another parameter in use here is --clear, which is good for + development: it clears all the previous data from the data + collections (inventory, links & cliques). + + scan.py -m your\_mongo\_access.conf -e RDO-packstack-Mitaka -y RDO + –clear + + Log level will provide the necessary details for cases of scan + debugging. + +Clique Scanning +~~~~~~~~~~~~~~~ + + | For creating cliques based on the discovered objects and links, + clique\_types must be defined for the given environment (or a + default “ANY” environment clique\_types will be used) + | A clique type specifies the link types used in building a clique + (graph topology) for a specific focal point object type. + | For example, it can define that for instance objects we want to + have the following link types: + +- instance-vnic + +- vnic-vconnector + +- vconnector-vedge + +- vedge-host\_pnic + +- host\_pnic-network + + See calipso-model guide for more details on cliques and links. + + As in many cases the same clique types are used, we can simply copy + the clique\_types documents from an existing clique\_types + collection. For example, using MongoChef: + +- Click the existing clique types collection + +- Right click the results area + +- Choose export + +- Click 'next' all the time (JSON format, to clipboard) + +- Select JSON format and "Overwrite document with the same \_id" + +- Right click the target collection + +- Choose import, then JSON and clipboard + +- Note that the name of the target collection should have the prefix + name of your collection's name. For example, you create a + collection named your\_test, then your clique types collection's + name must be your\_test\_clique\_types. + + Now run scan.py again to have it create cliques-only from that data. + +Viewing results +~~~~~~~~~~~~~~~ + + Scan results are written into the collections in the ‘Calispo’ DB on + the MongoDB database. + + In our example, we use the MongoDB database server on + “install-hostname”\ `http://korlev-osdna-devtest.cisco.com/ <http://korlev-osdna-devtest.cisco.com>`__, + so we can connect to it by Mongo client, such as Mongochef and + investigate the specific collections for details. + +Editing or deleting environments +================================ + + Inside a specific environment dashboard optional buttons are + available for deleting and editing the environment configurations: + +|image15| + +Note: Deleting an environment does not empty the inventories of previous +scan results, this can be accomplished in future scans when using the +--clear options. + +Event-based scanning +==================== + + For dynamic discovery and real-time updates of the inventories + Calipso also provides event-based scanning with event\_manager + application in the calipso-listen container. + + Event\_manager listens to the VIM AMQP BUS and based on the events + updates the inventories and also kickoff automatic scanning of a + specific object and its dependencies. + +Enabling event-based scanning +----------------------------- + + Per environment, administrator can define the option of event-based + scanning, using either UI or API to configure that parameter in the + specific environment configuration: + +|image16| + + In cases where event-based scanning is not supported for a specific + distribution variance the checkbox for event based scan will be + grayed out. When checked, the AMQP section becomes mandatory. + + This behavior is maintained through the “supported\_environments” + collection and explained in more details in the calipso-model + document. + +Event-based handling details +---------------------------- + + The event-based scanning module needs more work to adapt to the + changes in any specific distribution variance, this is where we + would like some community support to help us maintain data without + the need for full or partial scanning through scheduling. + + The following diagram illustrates event-based scanning module + functions on top of the regular scanning module functions: + +|image17| + + In the following tables, some of the current capabilities of + event-handling and event-based scanning in Calipso are explained: (NOTE: see pdf version of this guide for better tables view) + ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| # | Event name | AMQP event | Handler | Workflow | Scans | Notes | ++==========================+===========================+=====================================+=========================================+==================================================================================================================================================================================================================================================================================+======================================================================================================+==========================================================================================================================================================================================================================================================================================================================================+ +| **Instance** | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 1 | Create Instance | compute.instance.create.end | EventInstanceAdd | 1. Get *instances\_root* from inventory | **Yes** | ** ** | +| | | | | | | | +| | | | | 2. If *instance\_root* is None, log error, **return None** | | {by object id: 2, | | +| | | | | | | links: 1, | | +| | | | | 3. Create ScanInstancesRoot object. | | cliques: 1, | | +| | | | | | | from queue: ?} | | +| | | | | 4. Scan instances root (and only new instance as a child) | | | +| | | | | | | | +| | | | | 5. Scan from queue | | | +| | | | | | | | +| | | | | 6. Get *host* from inventory | | | +| | | | | | | | +| | | | | 7. Scan host (and only children of types ‘vconnectors\_folder’ and ‘vedges\_folder’ | | | +| | | | | | | | +| | | | | 8. Scan from queue | | | +| | | | | | | | +| | | | | 9. Scan links | | | +| | | | | | | | +| | | | | 10. Scan cliques | | | +| | | | | | | | +| | | | | 11. **Return True** | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 2 | Update Instance | compute.instance.rebuild.end | EventInstanceUpdate | 1. If state == ‘building’, **return None** | **Yes** (if #1 is used) | The only fields that are updated: *name*, *object\_name* and *name\_path* | +| | | | | | | | +| | | compute.instance.update | | 2. If state == ‘active’ and old\_state == ‘building’, call *EventInstanceAdd* (see #1), **return None** | **No** (otherwise) | | +| | | | | | | | +| | | | | 3. If state == ‘deleted’ and old\_state == ‘active’, call *EventInstanceDelete* (see #2), **return None** | | | +| | | | | | | | +| | | | | 4. Get *instance* from inventory | | | +| | | | | | | | +| | | | | 5. If *instance* is None, log error, **return None** | | | +| | | | | | | | +| | | | | 6. Update several fields in *instance*. | | | +| | | | | | | | +| | | | | 7. If *name\_path* has changed, update relevant names and *name\_path* for descendants | | | +| | | | | | | | +| | | | | 8. Update *instance* in db | | | +| | | | | | | | +| | | | | 9. **Return None** | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 3 | Delete Instance | compute.instance.delete.end | EventInstanceDelete (EventDeleteBase) | 1. Extract *id* from payload | **No** | delete\_handler() is expanded later | +| | | | | | | | +| | | | | 2. Execute *self.delete\_handler()* | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Instance Lifecycle** | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 4 | Instance Down | compute.instance.shutdown.start | **Not implemented** | | | | +| | | | | | | | +| | | compute.instance.power\_off.start | | | | | +| | | | | | | | +| | | compute.instance.suspend.start | | | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 5 | Instance Up | compute.instance.power\_on.end | **Not implemented** | | | | +| | | | | | | | +| | | compute.instance.suspend.end | | | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Region** | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 6 | Add Region | servergroup.create | **Not implemented** | | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 7 | Update Region | servergroup.update | **Not implemented** | ** ** | ** ** | ** ** | +| | | | | | | | +| | | servergroup.addmember | | | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 8 | Delete Region | servergroup.delete | **Not implemented** | ** ** | ** ** | ** ** | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Network** | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 9 | Add Network | network.create.end | EventNetworkAdd | 1. If network with specified *id* already exists, log error and **return None** | **No** | ** ** | +| | | | | | | | +| | | | | 2. Parse incoming data and create a *network* dict | | | +| | | | | | | | +| | | | | 3. Save *network* in db | | | +| | | | | | | | +| | | | | 4. **Return None** | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 10 | Update Network | network.update.end | EventNetworkUpdate | 1. Get *network\_document* from db | **No** | The only fields that are updated: *name*, *object\_name*, *name\_path* and *admin\_state\_up* | +| | | | | | | | +| | | | | 2. If *network\_document* doesn’t exist, log error and **return None** | | | +| | | | | | | | +| | | | | 3. If name has changed, update relevant names and *name\_path* for descendants | | | +| | | | | | | | +| | | | | 4. Update *admin\_state\_up* from payload | | | +| | | | | | | | +| | | | | 5. Update *network\_document* in db | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 11 | Delete Network | network.delete.end | EventNetworkDelete (EventDeleteBase) | 1. Extract *network\_id* from payload | **No** | delete\_handler() is expanded later | +| | | | | | | | +| | | | | 2. Execute *self.delete\_handler()* | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Subnet** | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 12 | Add Subnet | subnet.create.end | EventSubnetAdd | 1. Get *network\_document* from db | **Yes** {cliques: 1} | 1. I don’t fully understand what `*these lines* <https://cto-github.cisco.com/OSDNA/OSDNA/blob/b8246e3b19732d2f30922791ade23a94b4f52426/app/discover/events/event_subnet_add.py#L123-L126>`__ do. We make sure *ApiAccess.regions* variable is not empty, but why? The widespread usage of static variables is not a good sign anyway. | +| | | | | | | | +| | | | | 2. If *network\_document* doesn’t exist, log error and **return None** | | 2. For some reason `*the comment* <https://cto-github.cisco.com/OSDNA/OSDNA/blob/b8246e3b19732d2f30922791ade23a94b4f52426/app/discover/events/event_subnet_add.py#L132>`__ before those lines states we “scan for links” but it looks like we just add them. | +| | | | | | | | +| | | | | 3. Update *network\_document* with new subnet | | | +| | | | | | | | +| | | | | 4. If *dhcp\_enable* is *True*, we update parent network (***note 1***) and add the following children docs: *ports\_folder*, *port\_document*, *network\_services\_folder*, *dhcp\_document*, *vnic\_folder* and *vnic\_document*. | | | +| | | | | | | | +| | | | | 5. Add links for *pnics* and *vservice\_vnics* (***note 2***) | | | +| | | | | | | | +| | | | | 6. Scan cliques | | | +| | | | | | | | +| | | | | 7. **Return None** | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 13 | Update Subnet | subnet.update.end | EventSubnetUpdate | 1. Get *network\_document* from db | **Yes** {cliques: 1} (only if dhcp status has *switched* to True) | 1. If subnet name has changed, we set it in *subnets* object inside *network\_document* by new key, but don’t remove the old one. A bug? | +| | | | | | | | +| | | | | 2. If *network\_document* doesn’t exist, log error and **return None** | | | +| | | | | | | | +| | | | | 3. If we don’t have a matching subnet in *network\_document[‘subnets’]*, **return None** | | | +| | | | | | | | +| | | | | 4. If subnet has *enable\_dhcp* set to *True* and it wasn’t so before: | | | +| | | | | | | | +| | | | | 4.1. Add dhcp document | | | +| | | | | | | | +| | | | | 4.2. Make sure ApiAccess.regions is not empty | | | +| | | | | | | | +| | | | | 4.3. Add port document | | | +| | | | | | | | +| | | | | 4.4. If port has been added, add vnic document, add links and scan cliques. | | | +| | | | | | | | +| | | | | 5. Is subnet has *enable\_dhcp* set to *False* and it wasn’t so before: | | | +| | | | | | | | +| | | | | 5.1. Delete dhcp document | | | +| | | | | | | | +| | | | | 5.2. Delete port binding to dhcp server if exists | | | +| | | | | | | | +| | | | | 6. If name hasn’t changed, update it by its key in *subnets*. Otherwise, set it by the new key in *subnets*. (***note 1***) | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 14 | Delete Subnet | subnet.delete.end | EventSubnetDelete | 1. Get *network\_document* from db | **No** | | +| | | | | | | | +| | | | | 2. If *network\_document* doesn’t exist, log error and **return None** | | | +| | | | | | | | +| | | | | 3. Delete subnet id from *network\_document[‘subnet\_ids’]* | | | +| | | | | | | | +| | | | | 4. If subnet exists in *network\_document[‘subnets’]*, remove its cidr from *network\_document[‘cidrs’]* | | | +| | | | | | | | +| | | | | and remove itself from *network\_document[‘subnets’]* | | | +| | | | | | | | +| | | | | 5. Update *network\_document* in db | | | +| | | | | | | | +| | | | | 6. If no subnets are left in *network\_document*, delete related vservice dhcp, port and vnic documents | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Port** | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 15 | Create Port | port.create.end | EventPortAdd | 1. Check if ports folder exists, create if not. | **Yes** {cliques: 1} | 1. The port and (maybe) port folder will still persist in db even if we abort the execution on step 6. See idea 1 for details. | +| | | | | | | | +| | | | | 2. Add port document to db | (only if ‘compute’ is in port[‘device\_owner’] and instance\_root is not None (see steps 3 and 6)) | | +| | | | | | | | +| | | | | 3. If ‘compute’ is *not* in port[‘device\_owner’], **return None** | | | +| | | | | | | | +| | | | | 4. Get *old\_instance\_doc* (updated instance document) from db | | | +| | | | | | | | +| | | | | 5. Get *instances\_root* from db | | | +| | | | | | | | +| | | | | 6. If *instances\_root* is None, log error and **return None** (***note 1***) | | | +| | | | | | | | +| | | | | 7. Use an *ApiFetchHostInstances* fetcher to get data for instance with id equal to the device from payload. | | | +| | | | | | | | +| | | | | 8. If such instance exists, update *old\_instance\_doc*\ ’s fields *network\_info*, *network* and possibly *mac\_address* with their counterparts from fetched instance. Update *old\_instance\_doc* in db | | | +| | | | | | | | +| | | | | 9. Use a *CliFetchInstanceVnics/CliFetchInstanceVnicsVpp* fetcher to get *vnic* with *mac\_address* equal to the port’s mac address | | | +| | | | | | | | +| | | | | 10. If such vnic exists, update its data and update in db | | | +| | | | | | | | +| | | | | 11. Add new links using *FindLinksForInstanceVnics* and *FindLinksForVedges* classes | | | +| | | | | | | | +| | | | | 12. Scan cliques | | | +| | | | | | | | +| | | | | 13. **Return True** | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 16 | Update Port | port.update.end | EventPortUpdate | 1. Get *port* from db | **No** | | +| | | | | | | | +| | | | | 2. If *port* doesn’t exist, log error and **return None** | | | +| | | | | | | | +| | | | | 3. Update port data (*name*, *admin\_state\_up*, *status*, *binding:vnic\_type*) in db | | | +| | | | | | | | +| | | | | 4. **Return None** | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 17 | Delete Port | port.delete.end | EventPortDelete (EventDeleteBase) | 1. Get *port* from db | **No** | delete\_handler() is expanded later | +| | | | | | | | +| | | | | 2. If *port* doesn’t exist, log error and **return None** | | | +| | | | | | | | +| | | | | 3. If ‘compute’ is in port[‘device\_owner’], do the following: | | | +| | | | | | | | +| | | | | 3.1. Get *instance* document for the port from db. If it doesn’t exist, to step 4. | | | +| | | | | | | | +| | | | | 3.2. Remove port from *network\_info* of *instance* | | | +| | | | | | | | +| | | | | 3.3. If it was the last port for network in instance doc, remove network from the doc | | | +| | | | | | | | +| | | | | 3.4. If port’s *mac\_address* is equal to *instance\_doc*\ ’s one, then fetch an *instance* with the same id as *instance\_doc* using *ApiFetchHostInstances* fetcher. If *instance* exists and ‘mac\_address’ not in *instance*, set *instance\_doc*\ ’s mac\_address to None | | | +| | | | | | | | +| | | | | 3.5. Save *instance\_docs* in db | | | +| | | | | | | | +| | | | | 4. Delete port from db | | | +| | | | | | | | +| | | | | 5. Delete related vnic from db | | | +| | | | | | | | +| | | | | 6. Execute *self.delete\_handler(vnic)* *for vnic* | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Router** | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 18 | Add Router | router.create.end | EventRouterAdd | 1. Get *host* by id from db | **Yes** {cliques: 1} | 1. Looks like code author confused a lot of stuff here. This class needs to be reviewed thoroughly. | +| | | | | | | | +| | | | | 2. Fetch *router\_doc* using a *CliFetchHostVservice* | | 2. Step **3.7** never returns anything for some reason (a bug?) | +| | | | | | | | +| | | | | 3. If *router\_doc* contains *‘external\_gateway\_info’*: | | 3. Why are we adding router document again? It shouldn’t be added again on step **4** if it was already added on step **3.1**. Probably an ‘else’ clause is missing | +| | | | | | | | +| | | | | 3.1. Add router document (*with network*) to db | | | +| | | | | | | | +| | | | | 3.2. Add children documents: | | | +| | | | | | | | +| | | | | 3.3. If no ports folder exists for this router, create one | | | +| | | | | | | | +| | | | | 3.4. Add router *port* to db | | | +| | | | | | | | +| | | | | 3.5. Add *vnics folder* for router to db | | | +| | | | | | | | +| | | | | 3.6. If port was successfully added (**3.4**), try to add *vnic document* for router to db two times (??) | | | +| | | | | | | | +| | | | | 3.7. If port wasn’t successfully added, try adding *vnics\_folder* again (???) (***note 1***) | | | +| | | | | | | | +| | | | | 3.8. If step **3.7** returned False (***Note 2***), try to add *vnic\_document* again (??) | | | +| | | | | | | | +| | | | | 4. Add router document (*without network*) to db (**Note 3**) | | | +| | | | | | | | +| | | | | 5. Add relevant links for the new router | | | +| | | | | | | | +| | | | | 6. Scan cliques | | | +| | | | | | | | +| | | | | 7. **Return None** | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 19 | Update Router | router.update.end | EventRouterUpdate | 1. Get *router\_doc* from db | **Yes** {cliques: 1} | | +| | | | | | | | +| | | | | 2. If *router\_doc* doesn’t exist, log error and **return None** | | | +| | | | | | | | +| | | | | 3. If payload router data doesn’t have *external\_gateway\_info*, do the following: | | | +| | | | | | | | +| | | | | 3.1. If *router\_doc* has a *‘gw\_port\_id’* key, delete relevant port. | | | +| | | | | | | | +| | | | | 3.2. If *router\_doc* has a *‘network’*: | | | +| | | | | | | | +| | | | | 3.2.1. If a port was deleted on step **3.1**, remove its *‘network\_id’* from *router\_doc[‘network’]* | | | +| | | | | | | | +| | | | | 3.2.2. Delete related links | | | +| | | | | | | | +| | | | | 4. If payload router data has *external\_gateway\_info*, do the following: | | | +| | | | | | | | +| | | | | 4.1. Add new network id to *router\_doc* networks | | | +| | | | | | | | +| | | | | 4.2. Use *CliFetchHostVservice* to fetch gateway port and update it in *router\_doc* | | | +| | | | | | | | +| | | | | 4.3. Add children documents for router (see **#18** steps **3.2**-**3.8**) | | | +| | | | | | | | +| | | | | 4.4. Add relevant links | | | +| | | | | | | | +| | | | | 5. Update *router\_doc* in db | | | +| | | | | | | | +| | | | | 6. Scan cliques | | | +| | | | | | | | +| | | | | 7. **Return None** | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 20 | Delete Router | router.delete.end | EventRouterDelete (EventDeleteBase) | 1. Extract *router\_id* from payload | **No** | delete\_handler() is expanded later | +| | | | | | | | +| | | | | 2. Execute *self.delete\_handler()* | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Router Interface** | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 21 | Add Router Interface | router.interface.create | EventInterfaceAdd | 1. Get *network\_doc* from db based on subnet id from interface payload | **Yes** {cliques: 1} | 1. Log message states that we should abort interface adding, though the code does nothing to support that. Moreover, router\_doc can’t be empty at that moment because it’s referenced before. | +| | | | | | | | +| | | | | 2. If *network\_doc* doesn’t exist, **return None** | | | +| | | | | | | | +| | | | | 3. Make sure ApiAccess.regions is not empty (?) | | | +| | | | | | | | +| | | | | 4. Add router-interface port document in db | | | +| | | | | | | | +| | | | | 5. Add vnic document for interface. If unsuccessful, try again after a small delay | | | +| | | | | | | | +| | | | | 6. Update router: | | | +| | | | | | | | +| | | | | 6.1. If router\_doc is an empty type, log an error and continue to step **7** (***Note 1***) | | | +| | | | | | | | +| | | | | 6.2. Add new network id to *router\_doc* network list | | | +| | | | | | | | +| | | | | 6.3. If gateway port is in both router\_doc and db, continue to step **6.7** | | | +| | | | | | | | +| | | | | 6.4. Fetch *router* using *CliFetchHostVservice*, set gateway port in *router\_doc* to the one from fetched *router* | | | +| | | | | | | | +| | | | | 6.5. Add gateway port to db | | | +| | | | | | | | +| | | | | 6.6. Add vnic document for router. If unsuccessful, try again after a small delay | | | +| | | | | | | | +| | | | | 6.7. Update *router\_id* in db | | | +| | | | | | | | +| | | | | 7. Add relevant links | | | +| | | | | | | | +| | | | | 8. Scan cliques | | | +| | | | | | | | +| | | | | 9. **Return None** | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| 22 | Delete Router Interface | router.interface.delete | EventInterfaceDelete | 1. Get *port\_doc* by payload port id from db | **No** | | +| | | | | | | | +| | | | | 2. If *port\_doc* doesn’t exist, log an error and **return None** | | | +| | | | | | | | +| | | | | 3. Update relevant router by removing network id of *port\_doc* | | | +| | | | | | | | +| | | | | 4. Delete port by executing *EventPortDelete().delete\_port()* | | | ++--------------------------+---------------------------+-------------------------------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +ACI scanning +============ + + For dynamic discovery and real-time updates of physical switches and + connections between physical switches ports and host ports (pNICs), + Calispo provides an option to integrate with the Cisco data center + switches controller called “ACI APIC”. + + This is an optional parameter and once checked details on the ACI + server and API credentials needs to be provided: + +|image18| + + The results of this integration (when ACI switches are used in that + specific VIM environment) are extremely valuable as it maps out and + monitors virtual-to-physical connectivity across the entire data + center environment, both internal and external. + + Example graph generated in such environments: + +|image19| + +|image20| + + |image21| + +Monitoring enablement +====================== + + For dynamic discovery of real-time statuses and states of physical + and virtual components and thier connections Calispo provides an + option to automatically integrate with the Sensu framework, + customized and adapted from the Calispo model and design concepts. + Follow the monitoring-guide for details on this optional module. + + Enabling Monitoring through UI, using environment configuration + wizard: + +|image22| + + Modules data flows +=================== + +Calipso modules/containers and the VIM layers have some +inter-dependencies, illustrated in the following diagram: + +|image23| + +.. |image0| image:: media/image1.png + :width: 6.50000in + :height: 4.27153in +.. |image1| image:: media/image21.png + :width: 6.50000in + :height: 3.30347in +.. |image2| image:: media/image22.png + :width: 2.73924in + :height: 4.03075in +.. |image3| image:: media/image23.png + :width: 7.12500in + :height: 0.85278in +.. |image4| image:: media/image24.png + :width: 6.50000in + :height: 2.79167in +.. |image5| image:: media/image25.png + :width: 4.97854in + :height: 2.08307in +.. |image6| image:: media/image26.png + :width: 7.27906in + :height: 1.92708in +.. |image7| image:: media/image27.png + :width: 5.21767in + :height: 3.98349in +.. |image8| image:: media/image28.png + :width: 6.96114in + :height: 3.29167in +.. |image9| image:: media/image29.png + :width: 7.22987in + :height: 2.91667in +.. |image10| image:: media/image30.png + :width: 7.12651in + :height: 1.89583in +.. |image11| image:: media/image31.png + :width: 7.13041in + :height: 2.90625in +.. |image12| image:: media/image32.png + :width: 7.04193in + :height: 1.55208in +.. |image13| image:: media/image33.png + :width: 5.63542in + :height: 4.62032in +.. |image14| image:: media/image34.png + :width: 6.36846in + :height: 5.61458in +.. |image15| image:: media/image35.png + :width: 6.78043in + :height: 1.12500in +.. |image16| image:: media/image36.png + :width: 6.88648in + :height: 3.88542in +.. |image17| image:: media/image37.png + :width: 7.19278in + :height: 3.95833in +.. |image18| image:: media/image38.png + :width: 7.10748in + :height: 2.84375in +.. |image19| image:: media/image39.png + :width: 7.01329in + :height: 3.83333in +.. |image20| image:: media/image40.png + :width: 6.50000in + :height: 3.41181in +.. |image21| image:: media/image41.png + :width: 6.50000in + :height: 3.33681in +.. |image22| image:: media/image42.png + :width: 7.26121in + :height: 7.04167in +.. |image23| image:: media/image43.png + :width: 7.06279in + :height: 3.94792in diff --git a/docs/release/api-guide.rst b/docs/release/api-guide.rst new file mode 100644 index 0000000..24c04fb --- /dev/null +++ b/docs/release/api-guide.rst @@ -0,0 +1,2145 @@ +############################################################################### +# Copyright (c) 2017 Koren Lev (Cisco Systems), Yaron Yogev (Cisco Systems) # +# and others # +# # +# All rights reserved. This program and the accompanying materials # +# are made available under the terms of the Apache License, Version 2.0 # +# which accompanies this distribution, and is available at # +# http://www.apache.org/licenses/LICENSE-2.0 # +############################################################################### +| Calipso.io +| API Guide + +|image0| + +Project “Calipso” tries to illuminate complex virtual networking with +real time operational state visibility for large and highly distributed +Virtual Infrastructure Management (VIM). + +We believe that Stability is driven by accurate Visibility. + +Calipso provides visible insights using smart discovery and virtual +topological representation in graphs, with monitoring per object in the +graph inventory to reduce error vectors and troubleshooting, maintenance +cycles for VIM operators and administrators. + +Table of Contents + +Calipso.io API Guide 1 + +1 Pre Requisites 3 + +1.1 Calipso API container 3 + +2 Overview 3 + +2.1 Introduction 3 + +2.2 HTTP Standards 4 + +2.3 Calipso API module Code 4 + +3 Starting the Calipso API server 4 + +3.1 Authentication 4 + +3.2 Database 5 + +3.3 Running the API Server 5 + +4 Using the Calipso API server 6 + +4.1 Authentication 6 + +4.2 Messages 9 + +4.3 Inventory 14 + +4.4 Links 17 + +4.5 Cliques 20 + +4.6 Clique\_types 23 + +4.7 Clique\_constraints 26 + +4.8 Scans 29 + +4.9 Scheduled\_scans 32 + +4.10 Constants 35 + +4.11 Monitoring\_Config\_Templates 37 + +4.12 Aggregates 39 + +4.13 Environment\_configs 42 + +Pre Requisites +=============== + +Calipso API container +---------------------- + + Calipso’s main application is written with Python3.5 for Linux + Servers, tested successfully on Centos 7.3 and Ubuntu 16.04. When + running using micro-services many of the required software packages + and libraries are delivered per micro service, including the API + module case. In a monolithic case dependencies are needed. + + Here is a list of the required software packages for the API, and + the official supported steps required to install them: + +1. Python3.5.x for Linux : + https://docs.python.org/3.5/using/unix.html#on-linux + +2. Pip for Python3 : https://docs.python.org/3/installing/index.html + +3. Python3 packages to install using pip3 : + +4. falcon (1.1.0) + +5. pymongo (3.4.0) + +6. gunicorn (19.6.0) + +7. ldap3 (2.1.1) + +8. setuptools (34.3.2) + +9. python3-dateutil (2.5.3-2) + +10. bcrypt (3.1.1) + + You should use pip3 python package manager to install the specific + version of the library. Calipso project uses Python 3, so + package installation should look like this: + + pip3 install falcon==1.1.0 + + The versions of the Python packages specified above are the ones + that were used in the development of the API, other versions might + also be compatible. + + This document describes how to setup Calipso API container for + development against the API. + +Overview +========= + +Introduction +------------ + + The Calipso API provides access to the Calipso data stored in the + MongoDB. + + Calispo API uses + `falcon <https://falconframework.org/>`__ (https://falconframework.org) + web framework and `gunicorn <http://gunicorn.org/>`__ + (http://gunicorn.org) WSGI server. + + The authentication of the Calipso API is based on LDAP (Lightweight + Directory Access Protocol). It can therefore interface with any + directory servers which implements the LDAP protocol, e.g. OpenLDAP, + Active Directory etc. Calipso app offers and uses the LDAP built-in + container by default to make sure this integration is fully tested, + but it is possible to interface to other existing directories. + +HTTP Standards +-------------- + + | The Calipso API supports standard \ `HTTP + methods <https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html>`__ + described here: + https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html. + | At present two types of operations are supported: GET (retrieve + data) and POST (create a new data object). + +Calipso API module Code +----------------------- + + Clipso API code is currently located in opnfv repository. + + Run the following command to get the source code: + + git + clone \ `**https://git.opnfv.org/calipso/** <https://git.opnfv.org/calipso/>`__ + + The source code of the API is located in the app/api directory + sub-tree. + +Starting the Calipso API server +================================ + +Authentication +--------------- + + Calipso API uses LDAP as the protocol to implement the + authentication, so you can use any LDAP directory server as the + authentication backend, like OpenLDAP and `Microsoft + AD <https://msdn.microsoft.com/en-us/library/bb742424.aspx>`__. You + can edit the ldap.conf file which is located in app/config directory + to configure LDAP server options (see details in quickstart-guide): + + | # url for connecting to the LDAP server (customize to your own as + needed): + | url ldap\_url + | # LDAP attribute mapped to user id, must not be a multivalued + attributes: + | user\_id\_attribute CN + | # LDAP attribute mapped to user password: + | user\_pass\_attribute userPassword + | # LDAP objectclass for user + | user\_objectclass inetOrgPerson + | # Search base for users + | user\_tree\_dn OU=Employees,OU=Example Users,DC=exmaple,DC=com + | query\_scope one + | # Valid options for tls\_req\_cert are demand, never, and allow + | tls\_req\_cert demand + | # CA certificate file path for communicating with LDAP servers. + | tls\_cacertfile ca\_cert\_file\_path + | group\_member\_attribute member + + Calipso currently implements the basic authentication, the client + send the query request with its username and password in the auth + header, if the user can be bound to the LDAP server, authentication + succeeds otherwise fails. Other methods will be supported in future + releases. + +Database +-------- + + Calipso API query for and retrieves data from MongoDB container, the + data in the MongoDB comes from the results of Calipso scanning, + monitoring or the user inputs from the API. All modules of a single + Calipso instance of the application must point to the same MongoDB + used by the scanning and monitoring modules. Installation and + testing of mongoDB is covered in install-guide and quickstart-guide. + +Running the API Server +---------------------- + + The entry point (initial command) running the Calipso API + application is the server.py script in the app/api directory. + Options for running the API server can be listed using: python3 + server.py –help. Here is the current options available: + + | -m [MONGO\_CONFIG], --mongo\_config [MONGO\_CONFIG] + | name of config file with mongo access details + | --ldap\_config [LDAP\_CONFIG] + | name of the config file with ldap server config + | details + | -l [LOGLEVEL], --loglevel [LOGLEVEL] logging level (default: + 'INFO') + | -b [BIND], --bind [BIND] + | binding address of the API server (default: + 127.0.0.1:8000) + | -y [INVENTORY], --inventory [INVENTORY] + | name of inventory collection (default: + 'inventory') + + For testing, you can simply run the API server by: + + python3 app/api/server.py + + This will start a HTTP server listening on \ http://localhost:8000, + if you want to change the binding address of the server, you can run + it using this command: + + python3 server.py --bind ip\_address/server\_name:port\_number + + You can also use your own configuration files for LDAP server and + MongoDB, just add --mongo\_config and --ldap\_config options in your + command: + + python3 server.py --mongo\_config your\_mongo\_config\_file\_path + --ldap\_config your\_ldap\_config\_file\_path + + --inventory option is used to set the collection names the server + uses for the API, as per the quickstart-guide this will default to + **/local\_dir/calipso\_mongo\_access.conf** and + **/local\_dir/ldap.conf** mounted inside the API container. + + Notes: the --inventory argument can only change the collection names + of the inventory, links, link\_types, clique\_types, + clique\_constraints, cliques, constants and scans collections, names + of the monitoring\_config\_templates, environments\_config and + messages collections will remain at the root level across releases. + +Using the Calipso API server +============================= + +The following covers the currently available requests and responses on +the Calipso API + +Authentication +-------------- + +**POST** /auth/tokens + +Description: get token with password and username or a valid token. + +Normal response code: 201 + +Error response code: badRequest(400), unauthorized(401) + +**Request** + ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++=========================+==========+============+==================================================================================================================================+ +| auth(Mandatory) | body | object | An auth object that contains the authentication information | ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------+ +| methods(Mandatory) | body | array | The authentication methods. For password authentication, specify password, for token authentication, specify token. | ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------+ +| credentials(Optional) | body | object | Credentials object which contains the username and password, it must be provided when getting the token with user credentials. | ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------+ +| token(Optional) | body | string | The token of the user, it must be provided when getting the user with an existing valid token. | ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------+ + +**Response** + ++---------------+----------+------------+--------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++===============+==========+============+============================================================================================================================================+ +| token | body | string | Token for the user. | ++---------------+----------+------------+--------------------------------------------------------------------------------------------------------------------------------------------+ +| issued-at | body | string | The date and time when the token was issued. the date and time format follows \ `*ISO 8610* <https://en.wikipedia.org/wiki/ISO_8601>`__: | +| | | | | +| | | | YYYY-MM-DDThh:mm:ss.sss+hhmm | ++---------------+----------+------------+--------------------------------------------------------------------------------------------------------------------------------------------+ +| expires\_at | body | string | The date and time when the token expires. the date and time format follows \ `*ISO 8610* <https://en.wikipedia.org/wiki/ISO_8601>`__: | +| | | | | +| | | | YYYY-MM-DDThh:mm:ss.sss+hhmm | ++---------------+----------+------------+--------------------------------------------------------------------------------------------------------------------------------------------+ +| method | body | string | The method which achieves the token. | ++---------------+----------+------------+--------------------------------------------------------------------------------------------------------------------------------------------+ + +** +Examples** + +**Get token with credentials:** + +Post\ ** **\ `*http://korlev-osdna-staging1.cisco.com:8000/auth/tokens* <http://korlev-osdna-staging1.cisco.com:8000/auth/tokens>`__ + +| { +| "auth": { +| "methods": ["credentials"], +| "credentials": { +| "username": "username", +| "password": "password" +| } +| } +| } + +**Get token with token** + +post\ ** **\ http://korlev-calipso-staging1.cisco.com:8000/auth/tokens + +| { +| "auth": { +| "methods": ["token"], +| "token": "17dfa88789aa47f6bb8501865d905f13" +| } +| } + +** +** + +**DELETE** /auth/tokens + +Description: delete token with a valid token. + +Normal response code: 200 + +Error response code: badRequest(400), unauthorized(401) + +**Request** + ++----------------+----------+------------+-------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++================+==========+============+=============================================================+ +| X-Auth-Token | header | string | A valid authentication token that is doing to be deleted. | ++----------------+----------+------------+-------------------------------------------------------------+ + +**Response** + +200 OK will be returned when the delete succeed + +Messages +--------- + +**GET **/messages + +Description: get message details with environment name and message id, +or get a list of messages with filters except id. + +Normal response code: 200 + +Error response code: badRequest(400), unauthorized(401), notFound(404) + +**Request** + ++------------------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++====================================+==========+============+==================================================================================================================================================================================================================================+ +| env\_name(Mandatory) | query | string | Environment name of the messages. e.g. "Mirantis-Liberty-API". | ++------------------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| id (Optional) | query | string | ID of the message. | ++------------------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| source\_system (Optional) | query | string | Source system of the message, e.g. "OpenStack". | ++------------------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| start\_time (Optional) | query | string | Start time of the messages, when this parameter is specified, the messages after that time will be returned, the date and time format follows `*ISO 8610: * <https://en.wikipedia.org/wiki/ISO_8601>`__ | +| | | | | +| | | | YYYY-MM-DDThh:mm:ss.sss\ *+*\ hhmm | +| | | | | +| | | | The *+*\ hhmm value, if included, returns the time zone as an offset from UTC, For example, 2017-01-25T09:45:33.000-0500. If you omit the time zone, the UTC time is assumed. | ++------------------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| end\_time (Optional) | query | string | End time of the message, when this parameter is specified, the messages before that time will be returned, the date and time format follows \ `*ISO 8610* <https://en.wikipedia.org/wiki/ISO_8601>`__: | +| | | | | +| | | | YYYY-MM-DDThh:mm:ss.sss\ *+*\ hhmm | +| | | | | +| | | | The *+*\ hhmm value, if included, returns the time zone as an offset from UTC, For example, 2017-01-25T09:45:33.000-0500. If you omit the time zone, the UTC time is assumed. | ++------------------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| level (Optional) | query | string | The severity of the messages, we accept the severities strings described in `*RFC 5424* <https://tools.ietf.org/html/rfc5424>`__, possible values are "panic", "alert", "crit", "error", "warn", "notice", "info" and "debug". | ++------------------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| related\_object (Optional) | query | string | ID of the object related to the message. | ++------------------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| related\_object\_type (Optional) | query | string | Type of the object related to the message, possible values are "vnic", "vconnector", "vedge", "instance", "vservice", "host\_pnic", "network", "port", "otep" and "agent". | ++------------------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page (Optional) | query | int | Which page will to be returned, the default is first page, if the page is larger than the maximum page of the query, and it will return an empty result set (Page start from 0). | ++------------------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page\_size (Optional) | query | int | Size of each page, the default is 1000. | ++------------------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +**Response ** + ++-------------------------+----------+------------+---------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++=========================+==========+============+===================================================+ +| environment | body | string | Environment name of the message. | ++-------------------------+----------+------------+---------------------------------------------------+ +| id | body | string | ID of the message. | ++-------------------------+----------+------------+---------------------------------------------------+ +| \_id | body | string | MongoDB ObjectId of the message. | ++-------------------------+----------+------------+---------------------------------------------------+ +| timestamp | body | string | Timestamp of message. | ++-------------------------+----------+------------+---------------------------------------------------+ +| viewed | body | boolean | Indicates whether the message has been viewed. | ++-------------------------+----------+------------+---------------------------------------------------+ +| display\_context | body | string | The content which will be displayed. | ++-------------------------+----------+------------+---------------------------------------------------+ +| message | body | object | Message object. | ++-------------------------+----------+------------+---------------------------------------------------+ +| source\_system | body | string | Source system of the message, e.g. "OpenStack". | ++-------------------------+----------+------------+---------------------------------------------------+ +| level | body | string | The severity of the message. | ++-------------------------+----------+------------+---------------------------------------------------+ +| related\_object | body | string | Related object of the message. | ++-------------------------+----------+------------+---------------------------------------------------+ +| related\_object\_type | body | string | Type of the related object. | ++-------------------------+----------+------------+---------------------------------------------------+ +| messages | body | array | List of message ids which match the filters. | ++-------------------------+----------+------------+---------------------------------------------------+ + +**Examples** + +**Example Get Messages ** + +**Request:** + +http://korlev-calipso-testing.cisco.com:8000/messages?env_name=Mirantis-Liberty-API&start_time=2017-01-25T14:28:32.400Z&end_time=2017-01-25T14:28:42.400Z + +**Response:** + +| { +| messages: [ + + | { + | "level": "info", + | "environment": "Mirantis-Liberty", + | "id": "3c64fe31-ca3b-49a3-b5d3-c485d7a452e7", + | "source\_system": "OpenStack" + | }, + | { + | "level": "info", + | "environment": "Mirantis-Liberty", + | "id": "c7071ec0-04db-4820-92ff-3ed2b916738f", + | "source\_system": "OpenStack" + | }, + +| ] +| } + +**Example Get Message Details** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/messages?env_name=Mirantis-Liberty-API&id=80b5e074-0f1a-4b67-810c-fa9c92d41a98 + +**Response** + +| { +| "related\_object\_type": "instance", +| "source\_system": "OpenStack", +| "level": "info", +| "timestamp": "2017-01-25T14:28:33.057000", +| "\_id": "588926916a283a8bee15cfc6", +| "viewed": true, +| "display\_context": "\*", +| "related\_object": "97a1e179-6a42-4c7b-bced-4f64bd9e4b6b", +| "environment": "Mirantis-Liberty-API", +| "message": { +| "\_context\_show\_deleted": false, +| "\_context\_user\_name": "admin", +| "\_context\_project\_id": "a3efb05cd0484bf0b600e45dab09276d", +| "\_context\_service\_catalog": [ +| { +| "type": "volume", +| "endpoints": [ +| { +| "internalURL": + "`*http://192.168.0.2:8776/v1/a3efb05cd0484bf0b600e45dab09276d* <http://192.168.0.2:8776/v1/a3efb05cd0484bf0b600e45dab09276d>`__", +| "publicURL": + "`*http://172.16.0.3:8776/v1/a3efb05cd0484bf0b600e45dab09276d* <http://172.16.0.3:8776/v1/a3efb05cd0484bf0b600e45dab09276d>`__", +| "adminURL": + "`*http://192.168.0.2:8776/v1/a3efb05cd0484bf0b600e45dab09276d* <http://192.168.0.2:8776/v1/a3efb05cd0484bf0b600e45dab09276d>`__", +| "region": "RegionOne" +| } +| ], +| "name": "cinder" +| }, +| { +| "type": "volumev2", +| "endpoints": [ +| { +| "internalURL": + "`*http://192.168.0.2:8776/v2/a3efb05cd0484bf0b600e45dab09276d* <http://192.168.0.2:8776/v2/a3efb05cd0484bf0b600e45dab09276d>`__", +| "publicURL": + "`*http://172.16.0.3:8776/v2/a3efb05cd0484bf0b600e45dab09276d* <http://172.16.0.3:8776/v2/a3efb05cd0484bf0b600e45dab09276d>`__", +| "adminURL": + "`*http://192.168.0.2:8776/v2/a3efb05cd0484bf0b600e45dab09276d* <http://192.168.0.2:8776/v2/a3efb05cd0484bf0b600e45dab09276d>`__", +| "region": "RegionOne" +| } +| ], +| "name": "cinderv2" +| } +| ], +| "\_context\_user\_identity": "a864d9560b3048e9864118555bb9614c + a3efb05cd0484bf0b600e45dab09276d - - -", +| "\_context\_project\_domain": null, +| "\_context\_is\_admin": true, +| "\_context\_instance\_lock\_checked": false, +| "\_context\_timestamp": "2017-01-25T22:27:08.773313", +| "priority": "INFO", +| "\_context\_project\_name": "project-osdna", +| "publisher\_id": + "`*compute.node-1.cisco.com* <http://compute.node-1.cisco.com>`__", +| "\_context\_read\_only": false, +| "message\_id": "80b5e074-0f1a-4b67-810c-fa9c92d41a98", +| "\_context\_user\_id": "a864d9560b3048e9864118555bb9614c", +| "\_context\_quota\_class": null, +| "\_context\_tenant": "a3efb05cd0484bf0b600e45dab09276d", +| "\_context\_remote\_address": "192.168.0.2", +| "\_context\_request\_id": "req-2955726b-f227-4eac-9826-b675f5345ceb", +| "\_context\_auth\_token": + "gAAAAABYiSVcHmaq1TWwNc1\_QLlKhdUeC1-M6zBebXyoXN4D0vMlxisny9Q61crBzqwSyY\_Eqd\_yjrL8GvxatWI1WI1uG4VeWU6axbLe\_k5FaXS4RVOP83yR6eh5g\_qXQtsNapQufZB1paypZm8YGERRvR-vV5Ee76aTSkytVjwOBeipr9D0dXd-wHcRnSNkTD76nFbGKTu\_", +| "\_context\_user\_domain": null, +| "payload": { +| "image\_meta": { +| "container\_format": "bare", +| "disk\_format": "qcow2", +| "min\_ram": "64", +| "base\_image\_ref": "5f048984-37d1-4952-8b8a-9acb0237bad7", +| "min\_disk": "0" +| }, +| "display\_name": "test", +| "terminated\_at": "", +| "access\_ip\_v6": null, +| "architecture": null, +| "image\_ref\_url": + "`*http://192.168.0.3:9292/images/5f048984-37d1-4952-8b8a-9acb0237bad7* <http://192.168.0.3:9292/images/5f048984-37d1-4952-8b8a-9acb0237bad7>`__", +| "audit\_period\_beginning": "2017-01-01T00:00:00.000000", +| "metadata": {}, +| "node": "`*node-2.cisco.com* <http://node-2.cisco.com>`__", +| "audit\_period\_ending": "2017-01-25T22:27:12.888042", +| "instance\_type": "m1.micro", +| "ramdisk\_id": "", +| "availability\_zone": "nova", +| "kernel\_id": "", +| "hostname": "test", +| "vcpus": 1, +| "bandwidth": {}, +| "user\_id": "a864d9560b3048e9864118555bb9614c", +| "state\_description": "block\_device\_mapping", +| "old\_state": "building", +| "root\_gb": 0, +| "instance\_flavor\_id": "8784e0b5-7d17-4281-a509-f49d6fd102f9", +| "cell\_name": "", +| "reservation\_id": "r-zt7sh7vy", +| "access\_ip\_v4": null, +| "deleted\_at": "", +| "tenant\_id": "a3efb05cd0484bf0b600e45dab09276d", +| "disk\_gb": 0, +| "instance\_id": "97a1e179-6a42-4c7b-bced-4f64bd9e4b6b", +| "host": "`*node-2.cisco.com* <http://node-2.cisco.com>`__", +| "memory\_mb": 64, +| "os\_type": null, +| "old\_task\_state": "block\_device\_mapping", +| "state": "building", +| "instance\_type\_id": 6, +| "launched\_at": "", +| "ephemeral\_gb": 0, +| "created\_at": "2017-01-25 22:27:09+00:00", +| "progress": "", +| "new\_task\_state": "block\_device\_mapping" +| }, +| "\_context\_read\_deleted": "no", +| "event\_type": "compute.instance.update", +| "\_context\_roles": [ +| "admin", +| "\_member\_" +| ], +| "\_context\_user": "a864d9560b3048e9864118555bb9614c", +| "timestamp": "2017-01-25 22:27:12.912744", +| "\_unique\_id": "d6dff97e6f71401bb8890057f872644f", +| "\_context\_resource\_uuid": null, +| "\_context\_domain": null +| }, +| "id": "80b5e074-0f1a-4b67-810c-fa9c92d41a98" +| } + +Inventory +--------- + +**GET **/inventory** ** + +Description: get object details with environment name and id of the +object, or get a list of objects with filters except id. + +Normal response code: 200 + +Error response code: badRequest(400), unauthorized(401), notFound(404) + +**Request** + ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++===========================+==========+============+==================================================================================================================================================================================================================================================================================================================================+ +| env\_name (Mandatory) | query | string | Environment of the objects. e.g. "Mirantis-Liberty-API". | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| id (Optional) | query | string | ID of the object. e.g. "`*node-2.cisco.com* <http://node-2.cisco.com>`__". | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| parent\_id (Optional) | query | string | ID of the parent object. e.g. "nova". | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| id\_path (Optional) | query | string | ID path of the object. e.g. "/Mirantis-Liberty-API/Mirantis-Liberty-API-regions/RegionOne/RegionOne-availability\_zones/nova/`*node-2.cisco.com* <http://node-2.cisco.com>`__". | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| parent\_path (Optional) | query | string | ID path of the parent object. "/Mirantis-Liberty-API/Mirantis-Liberty-API-regions/RegionOne/RegionOne-availability\_zones/nova". | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| sub\_tree (Optional) | query | boolean | If it is true and the parent\_path is specified, it will return the whole sub-tree of that parent object which includes the parent itself, If it is false and the parent\_path is specified, it will only return the siblings of that parent (just the children of that parent node), the default value of sub\_tree is false. | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page (Optional) | query | int | Which page is to be returned, the default is the first page, if the page is larger than the maximum page of the query, it will return an empty set, (page starts from 0). | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page\_size (Optional) | query | int | Size of each page, the default is 1000. | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +**Response ** + ++-----------------+----------+------------+--------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++=================+==========+============+==================================================+ +| environment | body | string | Environment name of the object. | ++-----------------+----------+------------+--------------------------------------------------+ +| id | body | string | ID of the object. | ++-----------------+----------+------------+--------------------------------------------------+ +| \_id | body | string | MongoDB ObjectId of the object. | ++-----------------+----------+------------+--------------------------------------------------+ +| type | body | string | Type of the object. | ++-----------------+----------+------------+--------------------------------------------------+ +| parent\_type | body | string | Type of the parent object. | ++-----------------+----------+------------+--------------------------------------------------+ +| parent\_id | body | string | ID of the parent object. | ++-----------------+----------+------------+--------------------------------------------------+ +| name\_path | body | string | Name path of the object. | ++-----------------+----------+------------+--------------------------------------------------+ +| last\_scanned | body | string | Time of last scanning. | ++-----------------+----------+------------+--------------------------------------------------+ +| name | body | string | Name of the object. | ++-----------------+----------+------------+--------------------------------------------------+ +| id\_path | body | string | ID path of the object. | ++-----------------+----------+------------+--------------------------------------------------+ +| objects | body | array | The list of object IDs that match the filters. | ++-----------------+----------+------------+--------------------------------------------------+ + +**Examples** + +**Example Get Objects ** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/inventory?env_name=Mirantis-Liberty-API&parent_path=/Mirantis-Liberty-API/Mirantis-Liberty-API-regions/RegionOne&sub_tree=false + +**Response** + +{ + + "objects": [ + + | { + | "id": "Mirantis-Liberty-regions", + | "name": "Regions", + | "name\_path": "/Mirantis-Liberty/Regions" + | }, + | { + | "id": "Mirantis-Liberty-projects", + | "name": "Projects", + | "name\_path": "/Mirantis-Liberty/Projects" + | } + + ] + +} + +**Examples Get Object Details** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/inventory?env_name=Mirantis-Liberty-API&id=node-2.cisco.com + +**Response** + +| { +| 'ip\_address': '192.168.0.5', +| 'services': { +| 'nova-compute': { +| 'active': True, +| 'updated\_at': '2017-01-20T23:03:57.000000', +| 'available': True +| } +| }, +| 'name': '`*node-2.cisco.com* <http://node-2.cisco.com>`__', +| 'id\_path': + '/Mirantis-Liberty-API/Mirantis-Liberty-API-regions/RegionOne/RegionOne-availability\_zones/nova/`*node-2.cisco.com* <http://node-2.cisco.com>`__', +| 'show\_in\_tree': True, +| 'os\_id': '1', +| 'object\_name': '`*node-2.cisco.com* <http://node-2.cisco.com>`__', +| '\_id': '588297ae6a283a8bee15cc0d', +| 'host\_type': [ +| 'Compute' +| ], +| 'name\_path': '/Mirantis-Liberty-API/Regions/RegionOne/Availability + Zones/nova/\ `*node-2.cisco.com* <http://node-2.cisco.com>`__', +| 'parent\_type': 'availability\_zone', +| 'zone': 'nova', +| 'parent\_id': 'nova', +| 'host': '`*node-2.cisco.com* <http://node-2.cisco.com>`__', +| 'last\_scanned': '2017-01-20T15:05:18.501000', +| 'id': '`*node-2.cisco.com* <http://node-2.cisco.com>`__', +| 'environment': 'Mirantis-Liberty-API', +| 'type': 'host' +| } + +Links +----- + +**GET **/links + +Description: get link details with environment name and id of the link, +or get a list of links with filters except id + +Normal response code: 200 + +Error response code: badRequest(400), unauthorized(401), notFound(404) + +**Request** + ++-------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++=========================+==========+============+=====================================================================================================================================================================================================================================================+ +| env\_name (Mandatory) | query | string | Environment of the links. e.g. "Mirantis-Liberty-API". | ++-------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| id (Optional) | query | string | ID of the link, it must be a string which can be converted to MongoDB ObjectId. | ++-------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| host (Optional) | query | string | Host of the link. e.g. "`*node-1.cisco.com* <http://node-1.cisco.com>`__". | ++-------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| link\_type (Optional) | query | string | Type of the link, some possible values for that are "instance-vnic", "otep-vconnector", "otep-host\_pnic", "host\_pnic-network", "vedge-otep", "vnic-vconnector", "vconnector-host\_pnic", "vnic-vedge", "vedge-host\_pnic" and "vservice-vnic" . | ++-------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| link\_name (Optional) | query | string | Name of the link. e.g. "Segment-2". | ++-------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| source\_id (Optional) | query | string | ID of the source object of the link. e.g. "qdhcp-4f4bf8b5-ca42-411a-9f64-5b214d1f1c71". | ++-------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| target\_id (Optional) | query | string | ID of the target object of the link. "tap708d399a-20". | ++-------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| state (Optional) | query | string | State of the link, "up" or "down". | ++-------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| attributes | query | object | The attributes of the link, e.g. the network attribute of the link is attributes:network="4f4bf8b5-ca42-411a-9f64-5b214d1f1c71". | ++-------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page (Optional) | query | int | Which page is to be returned, the default is first page, when the page is larger than the maximum page of the query, | +| | | | | +| | | | it will return an empty set. (Page starts from 0). | ++-------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page\_size (Optional) | query | int | Size of each page, the default is 1000. | ++-------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +**Response ** + ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++=================+==========+============+====================================================================================================================================================================================================================================================+ +| id | body | string | ID of the link. | ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| \_id | body | string | MongoDB ObjectId of the link. | ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| environment | body | string | Environment of the link. | ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| source\_id | body | string | ID of the source object of the link. | ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| target\_id | body | string | ID of the target object of the link. | ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| source | body | string | MongoDB ObjectId of the source object. | ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| target | body | string | MongoDB ObjectId of the target object. | ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| source\_label | body | string | Descriptive text for the source object. | ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| target\_label | body | string | Descriptive text for the target object. | ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| link\_weight | body | string | Weight of the link. | ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| link\_type | body | string | Type of the link, some possible values for that are "instance-vnic", "otep-vconnector", "otep-host\_pnic", "host\_pnic-network", "vedge-otep", "vnic-vconnector", "vconnector-host\_pnic", "vnic-vedge", "vedge-host\_pnic" and "vservice-vnic". | ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| state | body | string | State of the link, "up" or "down". | ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| attributes | body | object | The attributes of the link. | ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| links | body | array | List of link IDs which match the filters. | ++-----------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +**Examples** + +**Example Get Link Ids** + +**Request** + +`*http://korlev-calipso-testing.cisco.com:8000/links?env\_name=Mirantis-Liberty-API&host=node-2.cisco.com* <http://korlev-osdna-testing.cisco.com:8000/links?env_name=Mirantis-Liberty-API&host=node-2.cisco.com>`__ + +**Response** + +{ + + "links": [ + + | { + | "id": "58ca73ae3a8a836d10ff3b45", + | "host": "`*node-1.cisco.com* <http://node-1.cisco.com>`__", + | "link\_type": "host\_pnic-network", + | "link\_name": "Segment-103", + | "environment": "Mirantis-Liberty" + | } + + ] + +} + +**Example Get Link Details** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/links?env_name=Mirantis-Liberty-API&id=5882982c6a283a8bee15cc62 + +**Response** + +| { +| "target\_id": "6d0250ae-e7df-4b30-aa89-d9fcc22e6371", +| "target": "58a23ff16a283a8bee15d3e6", +| "link\_type": "vnic-vedge", +| "link\_name": + "`*qr-24364cd7-ab-node-1.cisco.com* <http://qr-24364cd7-ab-node-1.cisco.com>`__-OVS-3", +| "environment": "Mirantis-Liberty-API", +| "\_id": "58a240646a283a8bee15d438", +| "source\_label": "fa:16:3e:38:11:c9", +| "state": "up", +| "link\_weight": 0, +| "id": "58a240646a283a8bee15d438", +| "host": "`*node-1.cisco.com* <http://node-1.cisco.com>`__", +| "source": "58a23fd46a283a8bee15d3c6", +| "target\_label": "10", +| "attributes": {}, +| "source\_id": "qr-24364cd7-ab" +| } + +Cliques +------- + +**GET **/cliques + +Description: get clique details with environment name and clique id, or +get a list of cliques with filters except id + +Normal response code: 200 + +Error response code: badRequest(400), unauthorized(401), notFound(404) + +**Request** + ++---------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++=================================+==========+============+=====================================================================================================================================================================================================================================================================================================================================================================+ +| env\_name (Mandatory) | query | string | Environment of the cliques. e.g. "Mirantis-Liberty-API". | ++---------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| id (Optional) | query | string | ID of the clique, it must be a string that can be converted to Mongo ObjectID. | ++---------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| focal\_point (Optional) | query | string | MongoDB ObjectId of the focal point object, it must be a string that can be converted to Mongo ObjectID. | ++---------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| focal\_point\_type (Optional) | query | string | Type of the focal point object, some possible values are "vnic", "vconnector", "vedge", "instance", "vservice", "host\_pnic", "network", "port", "otep" and "agent". | ++---------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| link\_type(Optional) | query | string | Type of the link, when this filter is specified, it will return all the cliques which contain the specific type of the link, some possible values for link\_type are "instance-vnic", "otep-vconnector", "otep-host\_pnic", "host\_pnic-network", "vedge-otep", "vnic-vconnector", "vconnector-host\_pnic", "vnic-vedge", "vedge-host\_pnic" and "vservice-vnic". | ++---------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| link\_id (Optional) | query | string | MongoDB ObjectId of the link, it must be a string that can be converted to MongoDB ID, when this filter is specified, it will return all the cliques which contain that specific link. | ++---------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page (Optional) | query | int | The page is to be returned, the default is the first page, if the page is larger than the maximum page of the query, it will return an empty set. (Page starts from 0). | ++---------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page\_size (Optional) | query | int | The size of each page, the default is 1000. | ++---------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +**Response** + ++----------------------+----------+------------+---------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++======================+==========+============+=========================================================+ +| id | body | string | ID of the clique. | ++----------------------+----------+------------+---------------------------------------------------------+ +| \_id | body | string | MongoDB ObjectId of the clique. | ++----------------------+----------+------------+---------------------------------------------------------+ +| environment | body | string | Environment of the clique. | ++----------------------+----------+------------+---------------------------------------------------------+ +| focal\_point | body | string | Object ID of the focal point. | ++----------------------+----------+------------+---------------------------------------------------------+ +| focal\_point\_type | body | string | Type of the focal point object, e.g. "vservice". | ++----------------------+----------+------------+---------------------------------------------------------+ +| links | body | array | List of MongoDB ObjectIds of the links in the clique. | ++----------------------+----------+------------+---------------------------------------------------------+ +| links\_detailed | body | array | Details of the links in the clique. | ++----------------------+----------+------------+---------------------------------------------------------+ +| constraints | body | object | Constraints of the clique. | ++----------------------+----------+------------+---------------------------------------------------------+ +| cliques | body | array | The list of clique ids that match the filters. | ++----------------------+----------+------------+---------------------------------------------------------+ + +**Examples** + +**Example Get Cliques** + +**Request** + +`*http://10.56.20.32:8000/cliques?env\_name=Mirantis-Liberty-API&link\_id=58a2405a6a283a8bee15d42f* <http://10.56.20.32:8000/cliques?env_name=Mirantis-Liberty-API&link_id=58a2405a6a283a8bee15d42f>`__ + +**Response** + +{ + + "cliques": [ + + | { + | "link\_types": [ + | "instance-vnic", + | "vservice-vnic", + | "vnic-vconnector" + | ], + | "environment": "Mirantis-Liberty", + | "focal\_point\_type": "vnic", + | "id": "576c119a3f4173144c7a75c5" + | }, + + | { + | "link\_types": [ + | "vnic-vconnector", + | "vconnector-vedge" + | ], + | "environment": "Mirantis-Liberty", + | "focal\_point\_type": "vconnector", + | "id": "576c119a3f4173144c7a75c6" + | } + + ] + +} + +**Example Get Clique Details** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/cliques?env_name=Mirantis-Liberty-API&id=58a2406e6a283a8bee15d43f + +**Response** + +| { +| 'id': '58867db16a283a8bee15cd2b', +| 'focal\_point\_type': 'host\_pnic', +| 'environment': 'Mirantis-Liberty', +| '\_id': '58867db16a283a8bee15cd2b', +| 'links\_detailed': [ +| { +| 'state': 'up', +| 'attributes': { +| 'network': 'e180ce1c-eebc-4034-9e50-b3bab1c13979' +| }, +| 'target': '58867cc86a283a8bee15cc92', +| 'source': '58867d166a283a8bee15ccd0', +| 'host': '`*node-1.cisco.com* <http://node-1.cisco.com>`__', +| 'link\_type': 'host\_pnic-network', +| 'target\_id': 'e180ce1c-eebc-4034-9e50-b3bab1c13979', +| 'source\_id': + 'eno16777728.103@eno16777728-00:50:56:ac:e8:97', +| 'link\_weight': 0, +| 'environment': 'Mirantis-Liberty', +| '\_id': '58867d646a283a8bee15ccf3', +| 'target\_label': '', +| 'link\_name': 'Segment-None', +| 'source\_label': '' +| } +| ], + +| 'links': [ +| '58867d646a283a8bee15ccf3' +| ], +| 'focal\_point': '58867d166a283a8bee15ccd0', +| 'constraints': { +| } + +} + +Clique\_types +------------- + +**GET **/clique\_types + +Description: get clique\_type details with environment name and +clique\_type id, or get a list of clique\_types with filters except id + +Normal response code: 200 + +Error response code: badRequest(400), unauthorized(401), notFound(404) + +**Request** + ++---------------------------------+----------+------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++=================================+==========+============+==============================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+ +| env\_name | query | string | Environment of the clique\_types. e.g. "Mirantis-Liberty-API" | ++---------------------------------+----------+------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| id | query | string | ID of the clique\_type, it must be a string that can be converted to the MongoDB ObjectID. | ++---------------------------------+----------+------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| focal\_point\_type (Optional) | query | string | Type of the focal point object, some possible values for it are "vnic", "vconnector", "vedge", "instance", "vservice", "host\_pnic", "network", "port", "otep" and "agent". | ++---------------------------------+----------+------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| link\_type(Optional) | query | string | Type of the link, when this filter is specified, it will return all the clique\_types which contain the specific link\_type in its link\_types array. Some possible values of the link\_type are "instance-vnic", "otep-vconnector", "otep-host\_pnic", "host\_pnic-network", "vedge-otep", "vnic-vconnector", "vconnector-host\_pnic", "vnic-vedge", "vedge-host\_pnic" and "vservice-vnic". Repeat link\_type several times to specify multiple link\_types, e.g link\_type=instance-vnic&link\_type=host\_pnic-network. | ++---------------------------------+----------+------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page\_size(Optional) | query | int | Size of each page, the default is 1000. | ++---------------------------------+----------+------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page (Optional) | query | int | Which page is to be returned, the default is first page, if the page is larger than the maximum page of the query, it will return an empty result set. (Page starts from 0). | ++---------------------------------+----------+------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +**Response** + ++----------------------+----------+------------+--------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++======================+==========+============+====================================================================+ +| id | body | string | ID of the clique\_type. | ++----------------------+----------+------------+--------------------------------------------------------------------+ +| \_id | body | string | MongoDB ObjectId of the clique\_type | ++----------------------+----------+------------+--------------------------------------------------------------------+ +| environment | body | string | Environment of the clique\_type. | ++----------------------+----------+------------+--------------------------------------------------------------------+ +| focal\_point\_type | body | string | Type of the focal point, e.g. "vnic". | ++----------------------+----------+------------+--------------------------------------------------------------------+ +| link\_types | body | array | List of link\_types of the clique\_type. | ++----------------------+----------+------------+--------------------------------------------------------------------+ +| name | body | string | Name of the clique\_type. | ++----------------------+----------+------------+--------------------------------------------------------------------+ +| clique\_types | body | array | List of clique\_type ids of clique types that match the filters. | ++----------------------+----------+------------+--------------------------------------------------------------------+ + +**Examples** + +**Example Get Clique\_types** + +**Request** + +`*** *** <http://korlev-osdna-testing.cisco.com:8000/clique_types?env_name=Mirantis-Liberty-API&id=&focal_point_type=&link_type=instance-vnic&page=&page_size=3&link_type=&link_type=pnic-network>`__\ http://korlev-calipso-testing.cisco.com:8000/clique_types?env_name=Mirantis-Liberty-API&link_type=instance-vnic&page_size=3&link_type=host_pnic-network + +`**Response** <http://korlev-osdna-testing.cisco.com:8000/clique_types?env_name=Mirantis-Liberty-API&link_type=instance-vnic&page_size=3&link_type=pnic-network>`__ + +{ + + "clique\_types": [ + + | { + | "environment": "Mirantis-Liberty", + | "focal\_point\_type": "host\_pnic", + | "id": "58ca73ae3a8a836d10ff3b80" + | } + + ] + +} + +**Example Get Clique\_type Details** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/clique_types?env_name=Mirantis-Liberty-API&id=585b183c761b05789ee3c659 + +**Response** + +| { +| 'id': '585b183c761b05789ee3c659', +| 'focal\_point\_type': 'vnic', +| 'environment': 'Mirantis-Liberty-API', +| '\_id': '585b183c761b05789ee3c659', +| 'link\_types': [ +| 'instance-vnic', +| 'vservice-vnic', +| 'vnic-vconnector' +| ], +| 'name': 'vnic\_clique' +| } + +**POST **/clique\_types + +Description: Create a new clique\_type + +Normal response code: 201(Created) + +Error response code: badRequest(400), unauthorized(401), conflict(409) + +**Request** + ++---------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++=================================+==========+============+===========================================================================================================================================================================================================================================================================+ +| environment(Mandatory) | body | string | Environment of the system, the environment must be the existing environment in the system. | ++---------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| focal\_point\_type(Mandatory) | body | string | Type of the focal point, some possible values are "vnic", "vconnector", "vedge", "instance", "vservice", "host\_pnic", "network", "port", "otep" and "agent". | ++---------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| link\_types(Mandatory) | body | array | Link\_types of the clique\_type, some possible values of the link\_type are "instance-vnic", "otep-vconnector", "otep-host\_pnic", "host\_pnic-network", "vedge-otep", "vnic-vconnector", "vconnector-host\_pnic", "vnic-vedge", "vedge-host\_pnic" and "vservice-vnic" | ++---------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| name(Mandatory) | body | string | Name of the clique type, e.g. "instance\_vconnector\_clique" | ++---------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +**Request Example** + +**post **\ http://korlev-calipso-testing.cisco.com:8000/clique\_types + +| { +| "environment" : "RDO-packstack-Mitaka", +| "focal\_point\_type" : "instance", +| "link\_types" : [ +| "instance-vnic", +| "vnic-vconnector", +| "vconnector-vedge", +| "vedge-otep", +| "otep-host\_pnic", +| "host\_pnic-network" +| ], +| "name" : "instance\_vconnector\_clique" +| } + +**Response** + +**Successful Example** + +| { +| "message": "created a new clique\_type for environment + Mirantis-Liberty" +| } + +Clique\_constraints +------------------- + +**GET **/clique\_constraints + +Description: get clique\_constraint details with clique\_constraint id, +or get a list of clique\_constraints with filters except id. + +Normal response code: 200 + +Error response code: badRequest(400), unauthorized(401), notFound(404) + +Note: this is not environment specific so query starts with parameter, +not env\_name (as with all others), example: + +http://korlev-calipso-testing.cisco.com:8000/clique_constraints?focal_point_type=instance + +**Request** + ++---------------------------------+----------+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++=================================+==========+============+================================================================================================================================================================================+ +| id (Optional) | query | string | ID of the clique\_constraint, it must be a string that can be converted to MongoDB ObjectId. | ++---------------------------------+----------+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| focal\_point\_type (Optional) | query | string | Type of the focal\_point, some possible values for that are "vnic", "vconnector", "vedge", "instance", "vservice", "host\_pnic", "network", "port", "otep" and "agent". | ++---------------------------------+----------+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| constraint(Optional) | query | string | Constraint of the cliques, repeat this filter several times to specify multiple constraints. e.g | +| | | | | +| | | | constraint=network&constraint=host\_pnic. | ++---------------------------------+----------+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page (Optional) | query | int | Which page is to be returned, the default is the first page, if the page is larger than the maximum page of the query, the last page will be returned. (Page starts from 0.) | ++---------------------------------+----------+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page\_size (Optional) | query | int | Size of each page, the default is 1000 | ++---------------------------------+----------+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +**Response ** + ++-----------------------+----------+------------+----------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++=======================+==========+============+==========================================================+ +| id | body | string | Object id of the clique constraint. | ++-----------------------+----------+------------+----------------------------------------------------------+ +| \_id | body | string | MongoDB ObjectId of the clique\_constraint. | ++-----------------------+----------+------------+----------------------------------------------------------+ +| focal\_point\_type | body | string | Type of the focal point object. | ++-----------------------+----------+------------+----------------------------------------------------------+ +| constraints | body | array | Constraints of the clique. | ++-----------------------+----------+------------+----------------------------------------------------------+ +| clique\_constraints | body | array | List of clique constraints ids that match the filters. | ++-----------------------+----------+------------+----------------------------------------------------------+ + +**Examples** + +**Example Get Clique\_constraints** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/clique_constraints?constraint=host_pnic&constraint=network + +**Response** + +{ + + "clique\_constraints": [ + + | { + | "id": "576a4176a83d5313f21971f5" + | }, + | { + | "id": "576ac7069f6ba3074882b2eb" + | } + + ] + +} + +**Example Get Clique\_constraint Details** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/clique_constraints?id=576a4176a83d5313f21971f5 + +`**Response** <http://korlev-osdna-testing.cisco.com:8000/clique_constraints?focal_point_type=&constraint=&id=576a4176a83d5313f21971f5&constraint=&page=&page_size=>`__ + +| { +| "\_id": "576a4176a83d5313f21971f5", +| "constraints": [ +| "network", +| "host\_pnic" +| ], +| "id": "576a4176a83d5313f21971f5", +| "focal\_point\_type": "instance" +| } + +Scans +----- + +**GET **/scans + +Description: get scan details with environment name and scan id, or get +a list of scans with filters except id + +Normal response code: 200 + +Error response code: badRequest (400), unauthorized (401), notFound(404) + +**Request** + ++--------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++==========================+==========+============+=============================================================================================================================================================================+ +| env\_name (Mandatory) | query | string | Environment of the scans. e.g. "Mirantis-Liberty". | ++--------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| id (Optional) | query | string | ID of the scan, it must be a string that can be converted MongoDB ObjectId. | ++--------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| base\_object(Optional) | query | string | ID of the scanned base object. e.g. "`*node-2.cisco.com* <http://node-2.cisco.com>`__". | ++--------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| status (Optional) | query | string | Status of the scans, the possible values for the status are "draft", "pending", "running", "completed", "failed" and "aborted". | ++--------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page (Optional) | query | int | Which page is to be returned, the default is the first page, if the page is larger than the maximum page of the query, it will return an empty set. (Page starts from 0.) | ++--------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page\_size (Optional) | query | int | Size of each page, the default is 1000. | ++--------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +**Response** + ++-------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++=========================+==========+============+===========================================================================================================================+ +| status | body | string | The current status of the scan, possible values are "draft", "pending", "running", "completed", "failed" and "aborted". | ++-------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| log\_level | body | string | Logging level of the scanning, the possible values are "CRITICAL", "ERROR", "WARNING", "INFO", "DEBUG" and "NOTSET". | ++-------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| clear | body | boolean | Indicates whether it needs to clear all the data before scanning. | ++-------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| scan\_only\_inventory | body | boolean | Only scan and store data in the inventory. | ++-------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| scan\_only\_links | body | boolean | Limit the scan to find only missing links. | ++-------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| scan\_only\_cliques | body | boolean | Limit the scan to find only missing cliques. | ++-------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| scan\_completed | body | boolean | Indicates if the scan completed | ++-------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| submit\_timestamp | body | string | Submit timestamp of the scan | ++-------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| environment | body | string | Environment name of the scan | ++-------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| inventory | body | string | Name of the inventory collection. | ++-------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| object\_id | body | string | Base object of the scan | ++-------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ + +**Examples** + +**Example Get Scans** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/scans?status=completed&env_name=Mirantis-Liberty&base_object=ff + +**Response** + +| { +| "scans": [ + +| { +| "status": "pending", +| "environment": "Mirantis-Liberty", +| "id": "58c96a075eb66a121cc4e75f", +| "scan\_completed": true +| } + + ] + +} + +**Example Get Scan Details** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/scans?env_name=Mirantis-Liberty&id=589a49cf2e8f4d154386c725 + +**Response** + +| { +| "scan\_only\_cliques": true, +| "object\_id": "ff", +| "start\_timestamp": "2017-01-28T01:02:47.352000", +| "submit\_timestamp": null, +| "clear": true, +| "\_id": "589a49cf2e8f4d154386c725", +| "environment": "Mirantis-Liberty", +| "scan\_only\_links": true, +| "id": "589a49cf2e8f4d154386c725", +| "inventory": "update-test", +| "scan\_only\_inventory": true, +| "log\_level": "warning", +| "status": "completed", +| "end\_timestamp": "2017-01-28T01:07:54.011000" +| } + +**POST **/scans + +Description: create a new scan (ask calipso to scan an environment for +detailed data gathering). + +Normal response code: 201(Created) + +Error response code: badRequest (400), unauthorized (401) + +**Request ** + ++------------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++====================================+==========+============+===========================================================================================================================+ +| status (mandatory) | body | string | The current status of the scan, possible values are "draft", "pending", "running", "completed", "failed" and "aborted". | ++------------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| log\_level (optional) | body | string | Logging level of the scanning, the possible values are "critical", "error", "warning", "info", "debug" and "notset". | ++------------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| clear (optional) | body | boolean | Indicates whether it needs to clear all the data before scanning. | ++------------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| scan\_only\_inventory (optional) | body | boolean | Only scan and store data in the inventory. | ++------------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| scan\_only\_links (optional) | body | boolean | Limit the scan to find only missing links. | ++------------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| scan\_only\_cliques (optional) | body | boolean | Limit the scan to find only missing cliques. | ++------------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| environment (mandatory) | body | string | Environment name of the scan | ++------------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| inventory (optional) | body | string | Name of the inventory collection. | ++------------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ +| object\_id (optional) | body | string | Base object of the scan | ++------------------------------------+----------+------------+---------------------------------------------------------------------------------------------------------------------------+ + +**Request Example** + +**post + **\ http://korlev-calipso-testing.cisco.com:8000/\ `*scans* <http://korlev-osdna-testing.cisco.com:8000/scans>`__ + +| { +| "status" : "pending", +| "log\_level" : "warning", +| "clear" : true, +| "scan\_only\_inventory" : true, +| "environment" : "Mirantis-Liberty", +| "inventory" : "koren", +| "object\_id" : "ff" +| } + +**Response** + +**Successful Example** + +| { +| "message": "created a new scan for environment + Mirantis-Liberty" +| } + +Scheduled\_scans +---------------- + +**GET **/scheduled\_scans + +Description: get scheduled\_scan details with environment name and +scheduled\_scan id, or get a list of scheduled\_scans with filters +except id + +Normal response code: 200 + +Error response code: badRequest (400), unauthorized (401), notFound(404) + +**Request** + ++--------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++==========================+==========+============+=============================================================================================================================================================================+ +| environment(Mandatory) | query | string | Environment of the scheduled\_scans. e.g. "Mirantis-Liberty". | ++--------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| id (Optional) | query | string | ID of the scheduled\_scan, it must be a string that can be converted to MongoDB ObjectId. | ++--------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| freq (Optional) | query | string | Frequency of the scheduled\_scans, the possible values for the freq are "HOURLY", "DAILY", "WEEKLY", "MONTHLY", and "YEARLY". | ++--------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page (Optional) | query | int | Which page is to be returned, the default is the first page, if the page is larger than the maximum page of the query, it will return an empty set. (Page starts from 0.) | ++--------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page\_size (Optional) | query | int | Size of each page, the default is 1000. | ++--------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +**Response** + ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++=========================+==========+============+==============================================================================================================================================+ +| freq | body | string | The frequency of the scheduled\_scan, possible values are "HOURLY", "DAILY", "WEEKLY", "MONTHLY", and "YEARLY". | ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------+ +| log\_level | body | string | Logging level of the scheduled\_scan, the possible values are "critical", "error", "warning", "info", "debug" and "notset". | ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------+ +| clear | body | boolean | Indicates whether it needs to clear all the data before scanning. | ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------+ +| scan\_only\_inventory | body | boolean | Only scan and store data in the inventory. | ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------+ +| scan\_only\_links | body | boolean | Limit the scan to find only missing links. | ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------+ +| scan\_only\_cliques | body | boolean | Limit the scan to find only missing cliques. | ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------+ +| submit\_timestamp | body | string | Submitted timestamp of the scheduled\_scan | ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------+ +| environment | body | string | Environment name of the scheduled\_scan | ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------+ +| scheduled\_timestamp | body | string | Scheduled time for the scanning, it should follows `*ISO 8610: * <https://en.wikipedia.org/wiki/ISO_8601>`__\ YYYY-MM-DDThh:mm:ss.sss+hhmm | ++-------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------------------------------+ + +**Examples** + +**Example Get Scheduled\_scans** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/scheduled_scans?environment=Mirantis-Liberty + +**Response** + +| { +| "scheduled\_scans": [ + + { + +| "freq":"WEEKLY", +| "environment": "Mirantis-Liberty", +| "id": "58c96a075eb66a121cc4e75f", +| "scheduled\_timestamp": "2017-01-28T01:07:54.011000" +| } + + ] + +} + +**Example Get Scheduled\_Scan Details** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/scheduled_scans?environment=Mirantis-Liberty&id=589a49cf2e8f4d154386c725 + +**Response** + +| { +| "scan\_only\_cliques": true, +| "scheduled\_timestamp": "2017-01-28T01:02:47.352000", +| "submit\_timestamp": 2017-01-27T01:07:54.011000"", +| "clear": true, +| "\_id": "589a49cf2e8f4d154386c725", +| "environment": "Mirantis-Liberty", +| "scan\_only\_links":false, +| "id": "589a49cf2e8f4d154386c725", +| "scan\_only\_inventory":false, +| "log\_level": "warning", +| "freq": "WEEKLY" +| } + +**POST **/scheduled\_scans + +Description: create a new scheduled\_scan (request calipso to scan in a +future date). + +Normal response code: 201(Created) + +Error response code: badRequest (400), unauthorized (401) + +**Request ** + ++------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++====================================+==========+============+=====================================================================================================================================================+ +| log\_level (optional) | body | string | Logging level of the scheduled\_scan, the possible values are "critical", "error", "warning", "info", "debug" and "notset". | ++------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+ +| clear (optional) | body | boolean | Indicates whether it needs to clear all the data before scanning. | ++------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+ +| scan\_only\_inventory (optional) | body | boolean | Only scan and store data in the inventory. | ++------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+ +| scan\_only\_links (optional) | body | boolean | Limit the scan to find only missing links. | ++------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+ +| scan\_only\_cliques (optional) | body | boolean | Limit the scan to find only missing cliques. | ++------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+ +| environment (mandatory) | body | string | Environment name of the scan | ++------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+ +| freq(mandatory) | body | string | The frequency of the scheduled\_scan, possible values are "HOURLY", "DAILY", "WEEKLY", "MONTHLY", and "YEARLY". | ++------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+ +| submit\_timestamp(mandatory) | body | string | Submitted time for the scheduled\_scan, it should follows `*ISO 8610: * <https://en.wikipedia.org/wiki/ISO_8601>`__\ YYYY-MM-DDThh:mm:ss.sss+hhmm | ++------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------+ + +** +Post** http://korlev-calipso-testing.cisco.com:8000/scheduled_scans + +| { +| "freq" : "WEEKLY", +| "log\_level" : "warning", +| "clear" : true, +| "scan\_only\_inventory" : true, +| "environment" : "Mirantis-Liberty", +| "submit\_timestamp" : "2017-01-28T01:07:54.011000" +| } + +**Response** + +**Successful Example** + +| { +| "message": "created a new scheduled\_scan for environment + Mirantis-Liberty" +| } + +Constants +--------- + +**GET **/constants + +Description: get constant details with name (constants are used by ui +and event/scan managers) + +Normal response code: 200 + +Error response code: badRequest(400), unauthorized(401), notFound(404) + +**Request** + ++--------------------+----------+------------+-----------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++====================+==========+============+===============================================+ +| name (Mandatory) | query | string | Name of the constant. e.g. "distributions". | ++--------------------+----------+------------+-----------------------------------------------+ + +**Response** + ++------------+----------+------------+-------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++============+==========+============+=====================================+ +| id | body | string | ID of the constant. | ++------------+----------+------------+-------------------------------------+ +| \_id | body | string | MongoDB ObjectId of the constant. | ++------------+----------+------------+-------------------------------------+ +| name | body | string | Name of the constant. | ++------------+----------+------------+-------------------------------------+ +| data | body | array | Data of the constant. | ++------------+----------+------------+-------------------------------------+ + +**Examples** + +**Example Get Constant Details ** + +**Request** + +`*http://korlev-osdna-testing.cisco.com:8000/constants?name=link\_states* <http://korlev-osdna-testing.cisco.com:8000/constants?name=link_states>`__ + +**Response** + +| { +| "\_id": "588796ac2e8f4d02b8e7aa2a", +| "data": [ +| { +| "value": "up", +| "label": "up" +| }, +| { +| "value": "down", +| "label": "down" +| } +| ], +| "id": "588796ac2e8f4d02b8e7aa2a", +| "name": "link\_states" +| } + +Monitoring\_Config\_Templates +----------------------------- + +**GET **/monitoring\_config\_templates + +Description: get monitoring\_config\_template details with template id, +or get a list of templates with filters except id (see +monitoring-guide). + +Normal response code: 200 + +Error response code: badRequest(400), unauthorized(401), notFound(404) + + +**Request** + ++------------------------+----------+------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++========================+==========+============+========================================================================================================================================================================================================================+ +| id (Optional) | query | string | ID of the monitoring config template, it must be a string that can be converted MongoDB ObjectId | ++------------------------+----------+------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| order (Optional) | query | int | Order by which templates are applied, 1 is the OSDNA default template. Templates that the user added later we use higher order and will override matching attributes in the default templates or add new attributes. | ++------------------------+----------+------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| side (Optional) | query | string | The side which runs the monitoring, the possible values are "client" and "server". | ++------------------------+----------+------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| type (Optional) | query | string | The name of the config file, e.g. "client.json". | ++------------------------+----------+------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page (Optional) | query | int | Which page is to be returned, the default is the first page, if the page is larger than the maximum page of the query, it will return an empty result set. (Page starts from 0). | ++------------------------+----------+------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page\_size(Optional) | query | int | Size of each page, the default is 1000. | ++------------------------+----------+------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +**Response ** + ++----------------------+----------+------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++======================+==========+============+=========================================================================================================================================================================================================================+ +| id | body | string | ID of the monitoring\_config\_template. | ++----------------------+----------+------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| \_id | body | srting | MongoDB ObjectId of the monitoring\_config\_template. | ++----------------------+----------+------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| monitoring\_system | body | string | System that we use to do the monitoring, e.g, "Sensu". | ++----------------------+----------+------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| order | body | string | Order by which templates are applied, 1 is the OSDNA default templates. Templates that the user added later we use higher order and will override matching attributes in the default templates or add new attributes. | ++----------------------+----------+------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| config | body | object | Configuration of the monitoring. | ++----------------------+----------+------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| side | body | string | The side which runs the monitoring. | ++----------------------+----------+------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| type | body | string | The name of the config file, e.g. "client.json". | ++----------------------+----------+------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +**Examples** + +**Example Get Monitoring\_config\_templates** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/monitoring_config_templates?side=client&order=1&type=rabbitmq.json&page=0&page_size=1 + +**Response** + +| { +| "monitoring\_config\_templates": [ + + | { + | "type": "rabbitmq.json", + | "side": "client", + | "id": "583711893e149c14785d6daa" + | } + +| ] +| } + +**Example Get Monitoring\_config\_template Details** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/monitoring_config_templates?id=583711893e149c14785d6daa + +**Response** + +| { +| "order": "1", +| "monitoring\_system": "sensu", +| "\_id": "583711893e149c14785d6daa", +| "side": "client", +| "type": "rabbitmq.json", +| "config": { +| "rabbitmq": { +| "host": "{server\_ip}", +| "vhost": "/sensu", +| "password": "{rabbitmq\_pass}", +| "user": "{rabbitmq\_user}", +| "port": 5672 +| } +| }, +| "id": "583711893e149c14785d6daa" +| } + +Aggregates +---------- + +**GET **/aggregates + +Description: List some aggregated information about environment, message +or constant. + +Normal response code: 200 + +Error response code: badRequest(400), unauthorized(401), notFound(404) + +**Request** + ++------------------------+----------+------------+--------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++========================+==========+============+==============================================================================================================+ +| env\_name (Optional) | query | string | Environment name, if the aggregate type is "environment", this value must be specified. | ++------------------------+----------+------------+--------------------------------------------------------------------------------------------------------------+ +| type (Optional) | query | string | Type of aggregate, currently we support three types of aggregate, "environment", "message" and "constant". | ++------------------------+----------+------------+--------------------------------------------------------------------------------------------------------------+ + +**Response ** + ++------------------------+----------+------------+------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++========================+==========+============+============================================================================================================+ +| type | body | string | Type of aggregate, we support three types of aggregates now, "environment", "message" and "constant". | ++------------------------+----------+------------+------------------------------------------------------------------------------------------------------------+ +| env\_name (Optional) | body | string | Environment name of the aggregate, when the aggregate type is "environment", this attribute will appear. | ++------------------------+----------+------------+------------------------------------------------------------------------------------------------------------+ +| aggregates | body | object | The aggregates information. | ++------------------------+----------+------------+------------------------------------------------------------------------------------------------------------+ + +**Examples** + +**Example Get Environment Aggregate ** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/aggregates?env_name=Mirantis-Liberty-API&type=environment + +**Response** + +| { +| "env\_name": "Mirantis-Liberty-API", +| "type": "environment", +| "aggregates": { +| "object\_types": { +| "projects\_folder": 1, +| "instances\_folder": 3, +| "otep": 3, +| "region": 1, +| "vedge": 3, +| "networks\_folder": 2, +| "project": 2, +| "vconnectors\_folder": 3, +| "availability\_zone": 2, +| "vedges\_folder": 3, +| "regions\_folder": 1, +| "network": 3, +| "vnics\_folder": 6, +| "instance": 2, +| "vservice": 4, +| "availability\_zones\_folder": 1, +| "vnic": 8, +| "vservices\_folder": 3, +| "port": 9, +| "pnics\_folder": 3, +| "network\_services\_folder": 3, +| "ports\_folder": 3, +| "host": 3, +| "vconnector": 6, +| "network\_agent": 6, +| "aggregates\_folder": 1, +| "pnic": 15, +| "network\_agents\_folder": 3, +| "vservice\_miscellenaous\_folder": 1 +| } +| } +| } + +**Example Get Messages Aggregate** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/aggregates?type=message + +**Response** + +{ + + "type": "message", + + "aggregates": { + + "levels": { + + "warn": 5, + + "info": 10, + + "error": 10 + + }, + + "environments": { + + "Mirantis-Liberty-API": 5, + + "Mirantis-Liberty": 10 + + } + + } + +} + +**Example Get Constants Aggregate** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/aggregates?type=constant + +**Response** + +| { +| "type": "constant", +| "aggregates": { +| "names": { +| "link\_states": 2, +| "scan\_statuses": 6, +| "type\_drivers": 5, +| "log\_levels": 6, +| "monitoring\_sides": 2, +| "mechanism\_drivers": 5, +| "messages\_severity": 8, +| "distributions": 16, +| "link\_types": 11, +| "object\_types": 10 +| } +| } +| } + +Environment\_configs +-------------------- + +**GET **/environment\_configs + +Description: get environment\_config details with name, or get a list of +environments\_config with filters except name + +Normal response code: 200 + +Error response code: badRequest(400), unauthorized(401), notFound(404) + +**Request** + ++-------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++=====================================+==========+============+===========================================================================================================================================================================================================+ +| name(Optional) | query | string | Name of the environment. | ++-------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| distribution(Optional) | query | string | The distribution of the OpenStack environment, it must be one of the distributions we support, e.g "Mirantis-8.0".(you can get all the supported distributions by querying the distributions constants) | ++-------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| mechanism\_drivers(Optional) | query | string | The mechanism drivers of the environment, it should be one of the drivers in mechanism\_drivers constants, e.g "ovs". | ++-------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| type\_drivers(Optional) | query | string | 'flat', 'gre', 'vlan', 'vxlan'. | ++-------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| user(Optional) | query | string | name of the environment user | ++-------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| listen(Optional) | query | boolean | Indicates whether the environment is being listened. | ++-------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| scanned(Optional) | query | boolean | Indicates whether the environment has been scanned. | ++-------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| monitoring\_setup\_done(Optional) | query | boolean | Indicates whether the monitoring setup has been done. | ++-------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| operational(Optional) | query | string | operational status of the environment, the possible statuses are "stopped", "running" and "error". | ++-------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page(Optional) | query | int | Which page is to be returned, the default is the first page, if the page is larger than the maximum page of the query, it will return an empty result set. (Page starts from 0). | ++-------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| page\_size(Optional) | query | int | Size of each page, the default is 1000. | ++-------------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +**Response** + ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++===========================+==========+============+======================================================================================================================+ +| configuration | body | array | List of configurations of the environment, including configurations of mysql, OpenStack, CLI, AMQP and Monitoring. | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------+ +| distribution | body | string | The distribution of the OpenStack environment, it must be one of the distributions we support, e.g "Mirantis-8.0". | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------+ +| last\_scanned | body | string | The date of last time scanning the environment, the format of the date is MM/DD/YY. | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------+ +| mechanism\_dirvers | body | array | The mechanism drivers of the environment, it should be one of the drivers in mechanism\_drivers constants. | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------+ +| monitoring\_setup\_done | body | boolean | Indicates whether the monitoring setup has been done. | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------+ +| name | body | string | Name of the environment. | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------+ +| operational | body | boolean | Indicates if the environment is operational. | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------+ +| scanned | body | boolean | Indicates whether the environment has been scanned. | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------+ +| type | body | string | Production, testing, development, etc. | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------+ +| type\_drivers | body | string | 'flat', 'gre', 'vlan', 'vxlan'. | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------+ +| user | body | string | The user of the environment. | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------+ +| listen | body | boolean | Indicates whether the environment is being listened. | ++---------------------------+----------+------------+----------------------------------------------------------------------------------------------------------------------+ + +**Examples** + +**Example Get Environments config** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/environment_configs?mechanism_drivers=ovs + +`**Response** <http://korlev-osdna-testing.cisco.com:8000/environment_configs?mechanism_drivers=ovs&name=>`__ + +| { +| environment\_configs: [ + + | { + | "distribution": "Canonical-icehouse", + | "name": "thundercloud" + | } + +| ] +| } + +**Example Environment config Details** + +**Request** + +http://korlev-calipso-testing.cisco.com:8000/environment_configs?name=Mirantis-Mitaka-2 + +**Response** + +| { +| "type\_drivers": "vxlan", +| "name": "Mirantis-Mitaka-2", +| "app\_path": "/home/yarony/osdna\_prod/app", +| "scanned": true, +| "type": "environment", +| "user": "test", +| "distribution": "Mirantis-9.1", +| "monitoring\_setup\_done": true, +| "listen": true, +| "mechanism\_drivers": [ +| "ovs" +| ], +| "configuration": [ +| { +| "name": "mysql", +| "user": "root", +| "host": "10.56.31.244", +| "port": "3307", +| "password": "TsbQPwP2VPIUlcFShkCFwBjX" +| }, +| { +| "name": "CLI", +| "user": "root", +| "host": "10.56.31.244", +| "key": "/home/ilia/Mirantis\_Mitaka\_id\_rsa" +| }, +| { +| "password": "G1VfxeJmtK5vIyNNMP4qZmXB", +| "user": "nova", +| "name": "AMQP", +| "port": "5673", +| "host": "10.56.31.244" +| }, +| { +| "server\_ip": + "`*korlev-nsxe1.cisco.com* <http://korlev-nsxe1.cisco.com>`__", +| "name": "Monitoring", +| "port": "4567", +| "env\_type": "development", +| "rabbitmq\_pass": "sensuaccess", +| "rabbitmq\_user": "sensu", +| "provision": "DB", +| "server\_name": "devtest-sensu", +| "type": "Sensu", +| "config\_folder": "/tmp/sensu\_test" +| }, +| { +| "user": "admin", +| "name": "OpenStack", +| "port": "5000", +| "admin\_token": "qoeROniLLwFmoGixgun5AXaV", +| "host": "10.56.31.244", +| "pwd": "admin" +| } +| ], +| "\_id": "582d77ee3e149c1318b3aa54", +| "operational": "yes" +| } + +**POST **/environment\_configs + +Description: create a new environment configuration. + +Normal response code: 201(Created) + +Error response code: +badRequest(400), unauthorized(401), notFound(404), conflict(409) + +**Request** + ++---------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| **Name** | **In** | **Type** | **Description** | ++=================================+==========+============+===========================================================================================================================================================================================================+ +| configuration(Mandatory) | body | array | List of configurations of the environment, including configurations of mysql(mandatory), OpenStack(mandatory), CLI(mandatory), AMQP(mandatory) and Monitoring(Optional). | ++---------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| distribution(Mandatory) | body | string | The distribution of the OpenStack environment, it must be one of the distributions we support, e.g "Mirantis-8.0".(you can get all the supported distributions by querying the distributions constants) | ++---------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| last\_scanned(Optional) | body | string | The date and time of last scanning, it should follows `*ISO 8610: * <https://en.wikipedia.org/wiki/ISO_8601>`__ | +| | | | | +| | | | YYYY-MM-DDThh:mm:ss.sss\ *+*\ hhmm | ++---------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| mechanism\_dirvers(Mandatory) | body | array | The mechanism drivers of the environment, it should be one of the drivers in mechanism\_drivers constants, e.g "OVS". | ++---------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| name(Mandatory) | body | string | Name of the environment. | ++---------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| operational(Mandatory) | body | boolean | Indicates if the environment is operational. e.g. true. | ++---------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| scanned(Optional) | body | boolean | Indicates whether the environment has been scanned. | ++---------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| listen(Mandatory) | body | boolean | Indicates if the environment need to been listened. | ++---------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| user(Optional) | body | string | The user of the environment. | ++---------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| app\_path(Mandatory) | body | string | The path that the app is located in. | ++---------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| type(Mandatory) | body | string | Production, testing, development, etc. | ++---------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| type\_drivers(Mandatory) | body | string | 'flat', 'gre', 'vlan', 'vxlan'. | ++---------------------------------+----------+------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +**Request Example** + +**Post** http://korlev-calipso-testing:8000/environment_configs + +| { +| "app\_path" : "/home/korenlev/OSDNA/app/", +| "configuration" : [ +| { +| "host" : "172.23.165.21", +| "name" : "mysql", +| "password" : "password", +| "port" : NumberInt(3306), +| "user" : "root", +| "schema" : "nova" +| }, +| { +| "name" : "OpenStack", +| "host" : "172.23.165.21", +| "admin\_token" : "TL4T0I7qYNiUifH", +| "admin\_project" : "admin", +| "port" : "5000", +| "user" : "admin", +| "pwd" : "admin" +| }, +| { +| "host" : "172.23.165.21", +| "key" : "/home/yarony/.ssh/juju\_id\_rsa", +| "name" : "CLI", +| "user" : "ubuntu" +| }, +| { +| "name" : "AMQP", +| "host" : "10.0.0.1", +| "port" : "5673", +| "user" : "User", +| "password" : "abcd1234" +| }, +| { +| "config\_folder" : "/tmp/sensu\_test\_liberty", +| "provision" : "None", +| "env\_type" : "development", +| "name" : "Monitoring", +| "port" : "4567", +| "rabbitmq\_pass" : "sensuaccess", +| "rabbitmq\_user" : "sensu", +| "server\_ip" : + "`*korlev.cisco.com* <http://korlev.cisco.com>`__", +| "server\_name" : "devtest-sensu", +| "type" : "Sensu" +| } +| ], +| "distribution" : "Canonical-icehouse", +| "last\_scanned" : "2017-02-13T16:07:15Z", +| "listen" : true, +| "mechanism\_drivers" : [ +| "OVS" +| ], +| "name" : "thundercloud", +| "operational" : "yes", +| "scanned" : false, +| "type" : "environment", +| "type\_drivers" : "gre", +| "user" : "WS7j8oTbWPf3LbNne" +| } + +**Response ** + +**Successful Example** + +| { +| "message": "created environment\_config for Mirantis-Liberty" +| } + +.. |image0| image:: media/image1.png + :width: 6.50000in + :height: 4.27153in diff --git a/docs/release/calipso-model.rst b/docs/release/calipso-model.rst new file mode 100644 index 0000000..d7ee8f7 --- /dev/null +++ b/docs/release/calipso-model.rst @@ -0,0 +1,926 @@ +############################################################################### +# Copyright (c) 2017 Koren Lev (Cisco Systems), Yaron Yogev (Cisco Systems) # +# and others # +# # +# All rights reserved. This program and the accompanying materials # +# are made available under the terms of the Apache License, Version 2.0 # +# which accompanies this distribution, and is available at # +# http://www.apache.org/licenses/LICENSE-2.0 # +############################################################################### +| Calipso.io +| Objects Model + +|image0| + +Project “Calipso” tries to illuminate complex virtual networking with +real time operational state visibility for large and highly distributed +Virtual Infrastructure Management (VIM). + +Calipso provides visible insights using smart discovery and virtual +topological representation in graphs, with monitoring per object in the +graph inventory to reduce error vectors and troubleshooting, maintenance +cycles for VIM operators and administrators. + +Calipso model, described in this document, was *built for +multi-environment and many VIM variances*, the model was tested +successfully (as of Aug 27\ :sup:`th`) against 60 different VIM +variances (Distributions, Versions, Networking Drivers and Types). + +Table of Contents + +Calipso.io Objects Model 1 + +1 Environments config 4 + +2 Inventory objects 6 + +2.1 Host 6 + +2.2 physical NIC (pNIC) 7 + +2.3 Bond 7 + +2.4 Instance 7 + +2.5 virtual Service (vService) 7 + +2.6 Network 7 + +2.7 virtual NIC (vNIC) 7 + +2.8 Port 8 + +2.9 virtual Connector (vConnector) 8 + +2.10 virtual Edge (vEdge) 8 + +2.11 Overlay-Tunnel-Endpoint (OTEP) 8 + +2.12 Network\_segment 8 + +2.13 Network\_Agent 8 + +2.14 Looking up Calipso objects details 9 + +3 Link Objects 10 + +3.1 Link types 11 + +4 Clique objects 11 + +4.1 Clique types 11 + +5 Supported Environments 12 + +6 System collections 14 + +6.1 Attributes\_for\_hover\_on\_data 14 + +6.2 Clique\_constraints 14 + +6.3 Connection\_tests 14 + +6.4 Messages 14 + +6.5 Network\_agent\_types 14 + +6.6 Roles, Users 15 + +6.7 Statistics 15 + +6.8 Constants 15 + +6.9 Constants-env\_types 15 + +6.10 Constants-log\_levels 15 + +6.11 Constants-mechanism\_drivers 15 + +6.12 Constants-type\_drivers 15 + +6.13 Constants-environment\_monitoring\_types 15 + +6.14 Constants-link\_states 15 + +6.15 Constants-environment\_provision\_types 15 + +6.16 Constants-environment\_operational\_status 16 + +6.17 Constants-link\_types 16 + +6.18 Constants-monitoring\_sides 16 + +6.19 Constants-object\_types 16 + +6.20 Constants-scans\_statuses 16 + +6.21 Constants-distributions 16 + +6.22 Constants-distribution\_versions 16 + +6.23 Constants-message\_source\_systems 16 + +6.24 Constants-object\_types\_for\_links 16 + +6.25 Constants-scan\_object\_types 17 + +Environments config +=================== + + Environment is defined as a certain type of Virtual Infrastructure + facility the runs under a single unified Management (like an + OpenStack facility). + + Everything in Calipso application rely on environments config, this + is maintained in the **“environments\_config”** collection in the + mongo Calipso DB. + + Environment configs are pushed down to Calipso DB either through UI + or API (and only in OPNFV case Calipso provides an automated program + to build all needed environments\_config parameters for an ‘Apex’ + distribution automatically). + + When scanning and discovering items Calipso uses this configuration + document for successful scanning results, here is an example of an + environment config document: + + **{ ** + + **"name": "DEMO-ENVIRONMENT-SCHEME", ** + + **"enable\_monitoring": true, ** + + **"last\_scanned": "filled-by-scanning", ** + + **"app\_path": "/home/scan/calipso\_prod/app", ** + + **"type": "environment", ** + + **"distribution": "Mirantis", ** + + **"distribution\_version": "8.0”, ** + + **"mechanism\_drivers": ["OVS”], ** + + **"type\_drivers": "vxlan"** + + **"operational": "stopped", ** + + **"listen": true, ** + + **"scanned": false, ** + + **"configuration": [** + + **{** + + **"name": "OpenStack", ** + + **"port":”5000”, ** + + **"user": "adminuser", ** + + **"pwd": "dummy\_pwd", ** + + **"host": "10.0.0.1", ** + + **"admin\_token": "dummy\_token"** + + **}, ** + + **{** + + **"name": "mysql", ** + + **"pwd": "dummy\_pwd", ** + + **"host": "10.0.0.1", ** + + **"port": “3307”, ** + + **"user": "mysqluser"** + + **}, ** + + **{** + + **"name": "CLI", ** + + **"user": "sshuser", ** + + **"host": "10.0.0.1", ** + + **"pwd": "dummy\_pwd"** + + **}, ** + + **{** + + **"name": "AMQP", ** + + **"pwd": "dummy\_pwd", ** + + **"host": "10.0.0.1", ** + + **"port": “5673”, ** + + **"user": "rabbitmquser"** + + **}, ** + + **{** + + **"name": "Monitoring", ** + + **"ssh\_user": "root", ** + + **"server\_ip": "10.0.0.1", ** + + **"ssh\_password": "dummy\_pwd", ** + + **"rabbitmq\_pass": "dummy\_pwd", ** + + **"rabbitmq\_user": "sensu", ** + + **"rabbitmq\_port": “5671”, ** + + **"provision": "None", ** + + **"env\_type": "production", ** + + **"ssh\_port": “20022”, ** + + **"config\_folder": "/local\_dir/sensu\_config", ** + + **"server\_name": "sensu\_server", ** + + **"type": "Sensu", ** + + **"api\_port": NumberInt(4567)** + + **}, ** + + **{** + + **"name": "ACI", ** + + **"user": "admin", ** + + **"host": "10.1.1.104", ** + + **"pwd": "dummy\_pwd"** + + **}** + + **], ** + + **"user": "wNLeBJxNDyw8G7Ssg", ** + + **"auth": {** + + **"view-env": [** + + **"wNLeBJxNDyw8G7Ssg"** + + **], ** + + **"edit-env": [** + + **"wNLeBJxNDyw8G7Ssg"** + + **]** + + **}, ** + + **}** + + Here is a brief explanation of the purpose of major keys in this + environment configuration doc: + + **Distribution**: captures type of VIM, used for scanning of + objects, links and cliques. + + **Distribution\_version**: captures version of VIM distribution, + used for scanning of objects, links and cliques. + + **Mechanism\_driver**: captures virtual switch type used by the VIM, + used for scanning of objects, links and cliques. + + **Type\_driver**: captures virtual switch tunneling type used by the + switch, used for scanning of objects, links and cliques. + + **Listen**: defines whether or not to use Calipso listener against + the VIM BUS for updating inventory in real-time from VIM events. + + **Scanned**: defines whether or not Calipso ran a full and a + successful scan against this environment. + + **Last\_scanned**: end time of last scan. + + **Operational**: defines whether or not VIM environment endpoints + are up and running. + + **Enable\_monitoring**: defines whether or not Calipso should deploy + monitoring of the inventory objects running inside all environment + hosts. + + **Configuration-OpenStack**: defines credentials for OpenStack API + endpoints access. + + **Configuration-mysql**: defines credentials for OpenStack DB + access. + + **Configuration-CLI**: defines credentials for servers CLI access. + + **Configuration-AMQP**: defines credentials for OpenStack BUS + access. + + **Configuration-Monitoring**: defines credentials and setup for + Calipso sensu server (see monitoring-guide for details). + + **Configuration-ACI**: defines credentials for ACI switched + management API, if exists. + + **User and auth**: used for UI authorizations to view and edit this + environment. + + **App-path**: defines the root directory of the scanning + application. + +Inventory objects +================= + + Calipso’s success in scanning, discovering and analyzing many (60 as + of 27\ :sup:`th` Aug 2017) variances of virtual infrastructures lies + with its objects model and relationship definitions (model was + tested even against a vSphere VMware environment). + + Those objects are the real-time processes and systems that are built + by workers and agents on the virtual infrastructure servers. + + All Calipso objects are maintained in the **“inventory”** + collection. + + Here are the major objects defined in Calipso inventory in order to + capture the real-time state of networking: + +Host +---- + + It’s the physical server that runs all virtual objects, typically a + hypervisor or a containers hosting machine. + + It’s typically a bare-metal server, in some cases it might be + virtual (running “nesting” VMs as second virtualization layer inside + it). + +physical NIC (pNIC) +------------------- + + It’s the physical Ethernet Network Interface Card attached to the + Host, typically several of those are available on a host, in some + cases few of those are grouped (bundled) together into etherchannel + bond interfaces. + + For capturing data from real infrastructure devices Calipso created + 2 types of pNICs: host\_pnic (pNICs on the servers) and switch\_pnic + (pNICs on the physical switches). Calipso currently discovers host + to switch physical connections only in some types of switches (Cisco + ACI as of Aug 27\ :sup:`th` 2017). + +Bond +---- + + It’s a logical Network Interface using etherchannel standard + protovcols to form a group of pNICs providing enhanced throughput + for communications to/from the host. + + Calipso currently maintains bond details inside a host\_pnic object. + +Instance +-------- + + It’s the virtual server created for running a certain application or + function. Typically it’s a Virtual Machine, sometimes it’s a + Container. + +virtual Service (vService) +-------------------------- + + It’s a process/system that provides some type of networking service + to instances running on networks, some might be deployed as + namespaces and some might deploy as VM or Container, for example: + DHCP server, Router, Firewall, Load-Balancer, VPN service and + others. Calipso categorized vServices accordingly. + +Network +------- + + It’s an abstracted object, illustrating and representing all the + components (see below) that builds and provides communication + services for several instances and vServices. + +virtual NIC (vNIC) +------------------ + + There are 2 types - instance vNIC and vService vNIC: + +- Instance vNIC: It’s the virtual Network Interface Card attached to + the Instance and used by it for communications from/to that instance. + +- vService vNIC: It’s the virtual Network Interface Card attached to + the vService used by it for communications from/to that vService. + +Port +---- + + It’s an abstracted object representing the attachment point for an + instance or a vService into the network, in reality it’s fulfilled + by deployment of vNICs on hosts. + +virtual Connector (vConnector) +------------------------------ + + It’s a process/system that provides layer 2 isolation for a specific + network inside the host (isolating traffic from other networks). + Examples: Linux Bridge, Bridge-group, port-group etc. + +virtual Edge (vEdge) +-------------------- + + It’s a process/system that provides switching and routing services + for instances and/or vServices running on a specific host. It + function as an edge device between virtual components running on + that host and the pNICs on that host, making sure traffic is + maintained and still isolated across different networks. + + Examples: Open Virtual Switch, Midonet, VPP. + +Overlay-Tunnel-Endpoint (OTEP) +------------------------------ + + It’s an abstracted object representing the end-point on the host + that runs a certain tunneling technology to provide isolation across + networks and hosts for packets leaving and entering the pNICs of a + specific host. Examples: VXLAN tunnels endpoints, GRE tunnels + endpoints etc. + +Network\_segment +---------------- + + It’s the specific segment used inside the “overlay tunnel” to + represent traffic from a specific network, this depends on the + specific type (encapsulation) of the OTEP. + + Calipso currently maintains segments details inside a network + object. + +Network\_Agent +-------------- + + It’s a controlling software running on the hosts for orchestrating + the lifecycle of the above virtual components. Examples: DHCP agent, + L3 agent, OVS agent, Metadata agent etc. + +Looking up Calipso objects details +---------------------------------- + + As explained in more details in Calipso admin-guide, the underlying + database used is mongoDB. All major objects discovered by Calipso + scanning module are maintained in the “inventory” collection and + those document includes detailed attributes captured from the + infrastructure about those objects, here are the main objects + quarries to use for grabbing each of the above object types from + Calipso’s inventory: + + **{type:"vnic"}** + + **{type:"vservice"}** + + **{type:"instance"}** + + **{type:"host\_pnic"}** + + **{type:"switch\_pnic"}** + + **{type:"vconnector"}** + + **{type:"vedge"}** + + **{type:"network"}** + + **{type:"network\_agent"}** + + **{type:"otep"}** + + **{type:"host"}** + + **{type:"port"}** + + All Calipso modules (visualization, monitoring and analysis) rely on + those objects as baseline inventory items for any further + computation. + + Here is an example of a query made using mongo Chef Client + application: + + |image1| + + \* See Calipso API-guide for details on looking up those objects + through the Calipso API. + + The following simplified UML illustrates the way Calipso objects + relationships are maintained in a VIM of type OpenStack: + +|image2| + +Link Objects +============ + + Calipso analyzes all objects in its inventory for relationships, + finding in real-time, which object is attached to which object and + then creates a link object representing this relationship. This + analysis finds a link that is “single hop away” - a connection from + certain object to certain object that is attached to it directly. + + Derived relationships (A to B and B to C = A to C) is maintained as + ‘cliques’. + + Links objects are maintained in the **“links”** collection. + +Link types +---------- + + Based on the specific VIM distribution, distribution version, + mechanism driver and type driver a set of links are discovered + automatically by Calipso scanning module. Each link type is + bi-directional, it means that if a connection is discovered from A + to B, a connection also exists from B to A. + + Here is the list of link types that might be discovered from a + certain environment in the current release: + + **{"link\_type": "instance-vnic"}** + + **{"link\_type": "vnic-vconnector"}** + + **{"link\_type": "vconnector-vedge"}** + + **{"link\_type": "vedge-host\_pnic"}** + + **{"link\_type: "host\_pnic-network"}** + + **{"link\_type": "vedge-otep"}** + + **{"link\_type": "otep-vconnector"}** + + **{"link\_type": "otep-host\_pnic"}** + + **{"link\_type": "vconnector-host\_pnic"}** + + **{"link\_type": "vnic-vedge"}** + + **{"link\_type": "vservice-vnic"}** + + **{"link\_type": "switch\_pnic-host\_pnic"}** + + **{"link\_type": "switch\_pnic-switch\_pnic"}** + + **{"link\_type": "switch\_pnic-switch"}** + + A successful completion of scanning and discovery means that all + inventory objects, link objects and clique objects (see below) are + found and accurately representing real-time state of the virtual + networking on the specific environment. + +Clique objects +============== + + Cliques are lists of links. Clique represent a certain path in the + virtual networking infrastructure that an administrator is + interested in, this is made to allow easier searching and finding of + certain points of interest (“focal point”). + +Clique types +------------ + + Based on the specific VIM distribution, distribution version, + mechanism driver and type driver variance, Calipso scanning module + search for specific cliques using a model that is pre-populated in + its **“clique\_types”** collection, and it depends on the + environment variance, here is an example of a clique\_type: + + **{ ** + + **"environment" : "Apex-Euphrates", ** + + **"link\_types" : [** + + **"instance-vnic", ** + + **"vnic-vconnector", ** + + **"vconnector-vedge", ** + + **"vedge-otep", ** + + **"otep-host\_pnic", ** + + **"host\_pnic-network"** + + **], ** + + **"name": "instance\_clique\_for\_opnfv", ** + + **"focal\_point\_type": "instance"** + + **}** + + The above model instruct the Calipso scanner to create cliques with + the above list of link types for a “focal\_point” that is an + “instance” type of object. We believe this is a highly customized + model for analysis of dependencies for many use cases. We have + included several clique types, common across variances supported in + this release. + + The cliques themselves are then maintained in the **“cliques**\ ” + collection. + + To clarify this concept, here is an example for an implementation + use case in the Calipso UI module: + + When the user of the UI clicks on a certain object of type=instance, + he expresses he’s wish to see a graph representing the path taken by + traffic from that specific instance (as the root source of traffic, + on that specific network) all the way down to the host pNIC and the + (abstracted) network itself. + + A successful completion of scanning and discovery means that all + inventory objects, link objects and clique objects (based on the + environment clique types) are found and accurately representing + real-time state of the virtual networking on the specific + environment. + +Supported Environments +====================== + + As of Aug 27\ :sup:`th` 2017, Calipso application supports 60 + different VIM environment variances and with each release the + purpose of the application is to maintain support and add more + variances per the VIM development cycles. The latest supported + variance and the specific functions of Calipso available for that + specific variance is captured in the **“supported\_environments”** + collection, here are two examples of that ‘supported’ model: + + **1.** + + **{ ** + + **"environment" : {** + + **"distribution" : "Apex", ** + + **"distribution\_version" : ["Euphrates"], ** + + **"mechanism\_drivers" : "OVS", ** + + **"type\_drivers" : "vxlan"** + + **}, ** + + **"features" : {** + + **"listening" : true, ** + + **"scanning" : true, ** + + **"monitoring" : false** + + **}** + + **}** + + **2.** + + **{ ** + + **"environment" : {** + + **"distribution" : "Mirantis", ** + + **"distribution\_version": ["6.0", "7.0", "8.0", "9.0", "9.1", + "10.0"], ** + + **"mechanism\_drivers" : "OVS", ** + + **"type\_drivers" : "vxlan"** + + **}, ** + + **"features" : {** + + **"listening" : true, ** + + **"scanning" : true, ** + + **"monitoring" : true** + + **}** + + **}** + + The examples above defines for Calipso application that: + +1. For an ‘Apex’ environment of version ‘Euphrates’ using OVS and vxlan, + Calipso can scan/discover all details (objects, links, cliques) but + is not yet monitoring those discovered objects. + +2. For a “Mirantis” environment of versions 6.0 to 10.0 using OVS and + vxlan, Calipso can scan/discover all details (objects, links, + cliques) and also monitor those discovered objects. + +With each calipso release more “supported\_environments” should be +added. + +System collections +================== + +Calipso uses other system collections to maintain its data for scanning, +event handling, monitoring and for helping to operate the API and UI +modules, here is the recent list of collections not covered yet in other +written guides: + +Attributes\_for\_hover\_on\_data +-------------------------------- + +This collection maintains a list of documents describing what will be +presented on the UI popup screen when the use hover-on a specific object +type, it details which parameters or attributed from the object’s data +will be shown on the screen, making this popup fully customized. + +Clique\_constraints +------------------- + +Defines the logic on which cliques are built, currently network is the +main focus of the UI (central point of connection for all cliques in the +system), but this is customizable. + +When building a clique graph, Calipso defaults to traversing all nodes +edges (links) in the graph. + +In some cases we want to limit the graph so it will not expand too much +(nor forever). + +For example: when we build the graph for a specific instance, we limit +the graph to only take objects from the network on which this instance +resides - otherwise the graph will show objects related to other +instances. + +The constraint validation is done by checking value V from the focal +point F on the links. + +For example, if an n instance has network X, we check that each link we +use either has network X (attribute “network” has value X), or does not +have the “network” attribute. + +Connection\_tests +----------------- + +This collection keeps requests from the UI to test the different +adapters (API, DB, CLI etc) connections to the underlying VIM, making +sure dynamic and real-time data is maintained. + +Messages +-------- + +Aggregates all loggings from the different systems, source\_system of +logs currently defined as “OpenStack” (the VIM), “Sensu” (the Monitoring +module) and “Calipso” (logs of the application itself. Messages have 6 +levels of severity and can be browsed in the UI and through Calipso API. + +Network\_agent\_types +--------------------- + +Lists the types of networking agents supported on the VIM (per +distribution and version). + +Roles, Users +------------ + +Basic RBAC facility to authorize calispo UI users for certain calipso +functionalities on the UI. + +Statistics +---------- + +Built for detailed analysis and future functionalities, used today for +traffic analysis (capturing samples of throughputs per session on VPP +based environments). + +Constants +--------- + +This is an aggregated collection for many types of documents that are +required mostly by the UI and basic functionality on some scanning +classes (‘fetchers’). + +Constants-env\_types +-------------------- + +Type of environments to allow for configuration on sensu monitoring +framework. + +Constants-log\_levels +--------------------- + +Severity levels for messages generated. + +Constants-mechanism\_drivers +---------------------------- + +Mechanism-drivers allowed for UI users. + +Constants-type\_drivers +----------------------- + +Type-drivers allowed for UI users. + +Constants-environment\_monitoring\_types +---------------------------------------- + +Currently only “Sensu” is available, might be used for other monitoring +systems integrations. + +Constants-link\_states +---------------------- + +Provides statuses for link objects, based on monitoring results. + +Constants-environment\_provision\_types +--------------------------------------- + +The types of deployment options available for monitoring (see +monitoring-guide for details). + +Constants-environment\_operational\_status +------------------------------------------ + +Captures the overall (aggregated) status of a curtained environment. + +Constants-link\_types +--------------------- + +Lists the connections and relationships options for objects in the +inventory. + +Constants-monitoring\_sides +--------------------------- + +Used for monitoring auto configurations of clients and servers. + +Constants-object\_types +----------------------- + +Lists the type of objects supported through scanning (inventory +objects). + +Constants-scans\_statuses +------------------------- + +During scans, several statuses are shown on the UI, based on the +specific stage and results. + +Constants-distributions +----------------------- + +Lists the VIM distributions. + +Constants-distribution\_versions +-------------------------------- + +Lists the VIM different versions of different distributions. + +Constants-message\_source\_systems +---------------------------------- + +The list of systems that can generate logs and messages. + +Constants-object\_types\_for\_links +----------------------------------- + +Object\_types used only for link popups on UI. + +Constants-scan\_object\_types +----------------------------- + +Object\_types used during scanning, see development-guide for details. + +.. |image0| image:: media/image1.png + :width: 6.50000in + :height: 4.27153in +.. |image1| image:: media/image8.png + :width: 6.50000in + :height: 2.43750in +.. |image2| image:: media/image9.png + :width: 7.02325in + :height: 5.22917in diff --git a/docs/release/install-guide.rst b/docs/release/install-guide.rst new file mode 100644 index 0000000..6027960 --- /dev/null +++ b/docs/release/install-guide.rst @@ -0,0 +1,343 @@ +############################################################################### +# Copyright (c) 2017 Koren Lev (Cisco Systems), Yaron Yogev (Cisco Systems) # +# and others # +# # +# All rights reserved. This program and the accompanying materials # +# are made available under the terms of the Apache License, Version 2.0 # +# which accompanies this distribution, and is available at # +# http://www.apache.org/licenses/LICENSE-2.0 # +############################################################################### +| Calipso.io +| Installation Guide + +|image0| + +Project “Calipso” tries to illuminate complex virtual networking with +real time operational state visibility for large and highly distributed +Virtual Infrastructure Management (VIM). + +We believe that Stability is driven by accurate Visibility. + +Calipso provides visible insights using smart discovery and virtual +topological representation in graphs, with monitoring per object in the +graph inventory to reduce error vectors and troubleshooting, maintenance +cycles for VIM operators and administrators. + +Table of Contents + +Calipso.io Installation Guide 1 + +1 Pre Requisites 3 + +1.1 Pre Requisites for Calipso “all in one” application 3 + +1.2 Pre Requisites for Calipso UI application 3 + +2 Installation Options 4 + +2.1 Monolithic App 4 + +2.2 Micro Services App, single line install 4 + +2.3 Micro Services App, customized single line install 5 + +2.4 Micro Services App, customized interactive install 6 + +3 OPNFV Options 7 + +3.1 APEX scenarios 7 + +3.2 Fuel scenarios 7 + +Pre Requisites +=============== + +Pre Requisites for Calipso “all in one” application +---------------------------------------------------- + + Calipso’s main application is written with Python3.5 for Linux + Servers, tested successfully on Centos 7.3 and Ubuntu 16.04. When + running using micro-services many of the required software packages + and libraries are delivered per micro service, but for an “all in + one” application case there are several dependencies. + + Here is a list of the required software packages, and the official + supported steps required to install them: + +1. Python3.5.x for Linux : + https://docs.python.org/3.5/using/unix.html#on-linux + +2. Pip for Python3 : https://docs.python.org/3/installing/index.html + +3. Python3 packages to install using pip3 : + + **sudo pip3 install falcon (>1.1.0)** + + **sudo pip3 install pymongo (>3.4.0)** + + **sudo pip3 install gunicorn (>19.6.0)** + + **sudo pip3 install ldap3 (>2.1.1)** + + **sudo pip3 install setuptools (>34.3.2)** + + **sudo pip3 install python3-dateutil (>2.5.3-2)** + + **sudo pip3 install bcrypt (>3.1.1)** + + **sudo pip3 install bson** + + **sudo pip3 install websocket** + + **sudo pip3 install datetime** + + **sudo pip3 install typing** + + **sudo pip3 install kombu** + + **sudo pip3 install boltons** + + **sudo pip3 install paramiko** + + **sudo pip3 install requests ** + + **sudo pip3 install httplib2** + + **sudo pip3 install mysql.connector** + + **sudo pip3 install xmltodict** + + **sudo pip3 install cryptography** + + **sudo pip3 install docker** + +1. Git : https://git-scm.com/book/en/v2/Getting-Started-Installing-Git + +2. Docker : https://docs.docker.com/engine/installation/ + +Pre Requisites for Calipso UI application +------------------------------------------ + + Calipso UI is developed and maintained using Meteor Framework + (https://www.meteor.com/tutorials). For stability and manageability + reasons we decided to always build the latest Calipso UI as a Docker + container pre-parameterized for stable and supported behavior. The + required steps for installing the Calipso UI with several options + are listed below. + +Installation Options +==================== + +Monolithic App +--------------- + + For development use, one might require Calipso to be installed as a + Monolithic Application, to do that all you need is a server + installed with Calipso pre-requisites and cloning of Calipso’s + public repository, here are the required steps for this option: + +1. Create a user named ‘\ **calipso**\ ’ and give it **sudo** access, + login as ‘calipso’ user. + +2. Create those directories as the ‘calipso’ user : **mkdir -p log & + mkdir log/calipso** + +3. Clone calipso main application from the latest public repository: + + **git clone https://git.opnfv.org/calipso/** + +1. Move to the default install directory: **cd calipso** + +2. Setup Python3 environment for calipso: + + **export PYTHONPATH=/home/calipso/calipso/app** + +1. Follow quick-start guide on how to use calipso modules for monolithic + scenario, and run each module manually. + +Micro Services App, single line install +--------------------------------------- + + For most users, this will be the fastest and more reliable install + option. We currently have Calipso divided into 7 major containers, + those are installed using a single installer. The Calipso containers + are pre-packaged and fully customized per our design needs. Here are + the required steps for installation using this option: + +1. Follow steps 1- 4 per section 2.1 above. + +2. Install Docker : https://docs.docker.com/engine/installation/ + +3. Install the following python3 libraries using pip3 : docker, pymongo + +4. Although Calipso installer can download all needed containers, if + they doesn’t exist locally already, we recommend doing a manual + download of all 7 containers, providing better control and logging: + + **sudo docker login** # use your DockerHub username and password to + login. + + **sudo docker pull korenlev/calipso:scan** # scan container used to + scan VIM + + **sudo docker pull korenlev/calipso:listen** # listen container to + attach to VIM’s BUS. + + **sudo docker pull korenlev/calipso:api** # api container for + application integration + + **sudo docker pull korenlev/calipso:sensu** # sensu server container + for monitoring + + **sudo docker pull korenlev/calipso:mongo** # calipso mongo DB + container + + **sudo docker pull korenlev/calipso:ui** # calipso ui container + + **sudo docker pull korenlev/calipso:ldap** # calipso ldap container + +1. Check that all containers were downloaded and registered + successfully: + + **sudo docker images** + + Expected results (As of Aug 2017): + + **REPOSITORY TAG IMAGE ID CREATED SIZE** + + **korenlev/calipso listen 12086aaedbc3 6 hours ago 1.05GB** + + **korenlev/calipso api 34c4c6c1b03e 6 hours ago 992MB** + + **korenlev/calipso scan 1ee60c4e61d5 6 hours ago 1.1GB** + + **korenlev/calipso sensu a8a17168197a 6 hours ago 1.65GB** + + **korenlev/calipso mongo 17f2d62f4445 22 hours ago 1.31GB** + + **korenlev/calipso ui ab37b366e812 11 days ago 270MB** + + **korenlev/calipso ldap 316bc94b25ad 2 months ago 269MB** + +1. Run the calipso installer using single line arguments: + + **python3 calipso/app/install/calipso-installer.py--command + start-all --copy q** + + This should launch all calipso modules in sequence along with all + needed configuration files placed in /home/calipso. + +Micro Services App, customized single line install +-------------------------------------------------- + + Calipso app includes the following directory in its default + structure (as of Aug 2017): + + **app/install/db,** this directory holds the initial Database scheme + and files needed as an initial data for starting Calipso + application. + + Calipso Database container (calipso-mongo) comes pre-packaged with + all the necessary initial scheme and files, but in some development + cases might not be synchronized with the latest ones supported. For + this reason, the installer has an option to copy files from the + above directory into the Database after runtime. + + You can run calipso installer using the following single line + arguments: + +1. **--command start-all \| stop-all** + + This will either start (docker run) or stop (docker kill and remove) + all calipso containers\ **,** a mandatory attribute for a single line + install option. + +2. **--copy q \| c ** + + This will either copy all files from app/install/db into mongoDB or + skip that step, a mandatory attribute for a single line install + option. + +3. **--hostname ** + + Allows to enter an IP address or hostname where container will be + deployed, an optional argument, default IP 172.17.0.1 (docker0 + default) is deployed if not used. + +4. **--webport ** + + Allows to enter a TCP port to be used for calipso UI on the host, an + optional argument, default 80 (http default) is deployed if not used. + +5. **--dbport ** + + Allows to enter a TCP port to be used for mongoDB port on the host, + an optional argument, default 27017 (mongo default) is deployed if + not used. + +6. **--dbuser ** + + Allows to enter a username to be used for mongoDB access on the host, + an optional argument, default ‘calipso’ (calipso-mongo container’s + default) is deployed if not used. + +7. **--dbpassword ** + + Allows to enter a password to be used for mongoDB access on the host, + an optional argument, default ‘calipso\_default’ (calipso-mongo + container’s default) is deployed if not used. + +Micro Services App, customized interactive install +-------------------------------------------------- + + Calipso’s application containers can be initiated and stopped + individually for testing purposes, this option is available through + interactive install, run calipso-installer.py with no argument to + kickstart the interactive process, allowing the following steps: + +1. **Action? (stop, start, or 'q' to quit):** + +2. **Container? (all, calipso-mongo, calipso-scan, calipso-listen, + calipso-ldap, calipso-api, calipso-sensu, calipso-ui or 'q' to + quit):** + +3. **create initial calipso DB ? (copy json files from 'db' folder to + mongoDB - 'c' to copy, 'q' to skip):** + +*Note*: based on the arguments input (or defaults), calipso installer +automatically creates and place 2 configuration files under +/**home/calipso**: **ldap.conf** and **calipso\_mongo\_access.conf**, +those are mandatory configuration files used by calipso containers to +interact with each other! + +OPNFV Options +============= + +Although calipso is designed for any VIM and for enterprise use-cases +too, service providers may use additional capability to install calipso +with Apex for OPNFV. + +APEX scenarios +--------------- + + When using apex to install OPNFV, the Triple-O based OpenStack is + installed automatically and calipso installation can be initiated + automatically after apex completes the VIM installation process for + a certain scenario. + + In this case calipso-configuration.py can be used for creating a new + environment automatically into mongoDB and UI of Calipso (instead of + using the calipso UI to do that as typical user would do), then + detailed scanning can start immediately, the following options are + available for calipso-configuration.py: + + TBD + +Fuel scenarios +--------------- + + TBD + +.. |image0| image:: media/image1.png + :width: 6.50000in + :height: 4.27153in diff --git a/docs/release/media/image1.png b/docs/release/media/image1.png Binary files differnew file mode 100644 index 0000000..67d026c --- /dev/null +++ b/docs/release/media/image1.png diff --git a/docs/release/media/image10.png b/docs/release/media/image10.png Binary files differnew file mode 100644 index 0000000..8207c07 --- /dev/null +++ b/docs/release/media/image10.png diff --git a/docs/release/media/image2.png b/docs/release/media/image2.png Binary files differnew file mode 100644 index 0000000..fe4b318 --- /dev/null +++ b/docs/release/media/image2.png diff --git a/docs/release/media/image21.png b/docs/release/media/image21.png Binary files differnew file mode 100644 index 0000000..dd510d1 --- /dev/null +++ b/docs/release/media/image21.png diff --git a/docs/release/media/image22.png b/docs/release/media/image22.png Binary files differnew file mode 100644 index 0000000..059e695 --- /dev/null +++ b/docs/release/media/image22.png diff --git a/docs/release/media/image23.png b/docs/release/media/image23.png Binary files differnew file mode 100644 index 0000000..6626de9 --- /dev/null +++ b/docs/release/media/image23.png diff --git a/docs/release/media/image24.png b/docs/release/media/image24.png Binary files differnew file mode 100644 index 0000000..94be4ff --- /dev/null +++ b/docs/release/media/image24.png diff --git a/docs/release/media/image25.png b/docs/release/media/image25.png Binary files differnew file mode 100644 index 0000000..707b078 --- /dev/null +++ b/docs/release/media/image25.png diff --git a/docs/release/media/image26.png b/docs/release/media/image26.png Binary files differnew file mode 100644 index 0000000..0bd7285 --- /dev/null +++ b/docs/release/media/image26.png diff --git a/docs/release/media/image27.png b/docs/release/media/image27.png Binary files differnew file mode 100644 index 0000000..b1b7d55 --- /dev/null +++ b/docs/release/media/image27.png diff --git a/docs/release/media/image28.png b/docs/release/media/image28.png Binary files differnew file mode 100644 index 0000000..4a8f63b --- /dev/null +++ b/docs/release/media/image28.png diff --git a/docs/release/media/image29.png b/docs/release/media/image29.png Binary files differnew file mode 100644 index 0000000..a2fed02 --- /dev/null +++ b/docs/release/media/image29.png diff --git a/docs/release/media/image3.png b/docs/release/media/image3.png Binary files differnew file mode 100644 index 0000000..3caa404 --- /dev/null +++ b/docs/release/media/image3.png diff --git a/docs/release/media/image30.png b/docs/release/media/image30.png Binary files differnew file mode 100644 index 0000000..68f47da --- /dev/null +++ b/docs/release/media/image30.png diff --git a/docs/release/media/image31.png b/docs/release/media/image31.png Binary files differnew file mode 100644 index 0000000..b9a7dcf --- /dev/null +++ b/docs/release/media/image31.png diff --git a/docs/release/media/image32.png b/docs/release/media/image32.png Binary files differnew file mode 100644 index 0000000..7da333e --- /dev/null +++ b/docs/release/media/image32.png diff --git a/docs/release/media/image33.png b/docs/release/media/image33.png Binary files differnew file mode 100644 index 0000000..8c56dd1 --- /dev/null +++ b/docs/release/media/image33.png diff --git a/docs/release/media/image34.png b/docs/release/media/image34.png Binary files differnew file mode 100644 index 0000000..854debd --- /dev/null +++ b/docs/release/media/image34.png diff --git a/docs/release/media/image35.png b/docs/release/media/image35.png Binary files differnew file mode 100644 index 0000000..afd1e7d --- /dev/null +++ b/docs/release/media/image35.png diff --git a/docs/release/media/image36.png b/docs/release/media/image36.png Binary files differnew file mode 100644 index 0000000..ba662b3 --- /dev/null +++ b/docs/release/media/image36.png diff --git a/docs/release/media/image37.png b/docs/release/media/image37.png Binary files differnew file mode 100644 index 0000000..3f08d11 --- /dev/null +++ b/docs/release/media/image37.png diff --git a/docs/release/media/image38.png b/docs/release/media/image38.png Binary files differnew file mode 100644 index 0000000..12d80d8 --- /dev/null +++ b/docs/release/media/image38.png diff --git a/docs/release/media/image39.png b/docs/release/media/image39.png Binary files differnew file mode 100644 index 0000000..b961c6c --- /dev/null +++ b/docs/release/media/image39.png diff --git a/docs/release/media/image4.png b/docs/release/media/image4.png Binary files differnew file mode 100644 index 0000000..5ee593f --- /dev/null +++ b/docs/release/media/image4.png diff --git a/docs/release/media/image40.png b/docs/release/media/image40.png Binary files differnew file mode 100644 index 0000000..9b4f15a --- /dev/null +++ b/docs/release/media/image40.png diff --git a/docs/release/media/image41.png b/docs/release/media/image41.png Binary files differnew file mode 100644 index 0000000..f49dd45 --- /dev/null +++ b/docs/release/media/image41.png diff --git a/docs/release/media/image42.png b/docs/release/media/image42.png Binary files differnew file mode 100644 index 0000000..4966d67 --- /dev/null +++ b/docs/release/media/image42.png diff --git a/docs/release/media/image43.png b/docs/release/media/image43.png Binary files differnew file mode 100644 index 0000000..7430501 --- /dev/null +++ b/docs/release/media/image43.png diff --git a/docs/release/media/image5.png b/docs/release/media/image5.png Binary files differnew file mode 100644 index 0000000..d414891 --- /dev/null +++ b/docs/release/media/image5.png diff --git a/docs/release/media/image6.png b/docs/release/media/image6.png Binary files differnew file mode 100644 index 0000000..f3a3f42 --- /dev/null +++ b/docs/release/media/image6.png diff --git a/docs/release/media/image7.png b/docs/release/media/image7.png Binary files differnew file mode 100644 index 0000000..f4423e6 --- /dev/null +++ b/docs/release/media/image7.png diff --git a/docs/release/media/image8.png b/docs/release/media/image8.png Binary files differnew file mode 100644 index 0000000..3a682fd --- /dev/null +++ b/docs/release/media/image8.png diff --git a/docs/release/media/image9.png b/docs/release/media/image9.png Binary files differnew file mode 100644 index 0000000..627b970 --- /dev/null +++ b/docs/release/media/image9.png diff --git a/docs/release/monitoring-guide.rst b/docs/release/monitoring-guide.rst new file mode 100644 index 0000000..2532848 --- /dev/null +++ b/docs/release/monitoring-guide.rst @@ -0,0 +1,663 @@ +############################################################################### +# Copyright (c) 2017 Koren Lev (Cisco Systems), Yaron Yogev (Cisco Systems) # +# and others # +# # +# All rights reserved. This program and the accompanying materials # +# are made available under the terms of the Apache License, Version 2.0 # +# which accompanies this distribution, and is available at # +# http://www.apache.org/licenses/LICENSE-2.0 # +############################################################################### +| Calipso.io +| Monitoring Guide + +|image0| + +Project “Calipso” tries to illuminate complex virtual networking with +real time operational state visibility for large and highly distributed +Virtual Infrastructure Management (VIM). + +We believe that Stability is driven by accurate Visibility. + +Calipso provides visible insights using smart discovery and virtual +topological representation in graphs, with monitoring per object in the +graph inventory to reduce error vectors and troubleshooting, maintenance +cycles for VIM operators and administrators. + +Table of Contents + +Calipso.io Monitoring Guide 1 + +1 Monitoring deployment options 3 + +1.1 Calipso monitoring provisioning 3 + +1.2 Calipso-sensu container 5 + +2 Monitoring configurations 5 + +2.1 Calipso monitoring configuration templates 6 + +2.2 Calipso Apex monitoring integration 11 + +Monitoring deployment options +============================== + + Calipso uses sensu framework that includes a customized calipso + sensu server and a set of customized clients, for details on the + framework see: + + https://sensuapp.org/docs/latest/overview/what-is-sensu.html + + Calipso uses this framework but further configure and customizes the + install, per: + + https://sensuapp.org/docs/1.0/installation/ + + Server and clients are deployed with rabbitmq over SSL as the + client-server transport: + + https://sensuapp.org/docs/1.0/reference/ssl.html + + Calipso then deploys customized checks and handlers as new types of + plugins: + + http://sensu-plugins.io/plugins/ + + In order to monitor all virtual components as described in + Calipso-model document, Calipso offers two major functions: + +1. Calipso scan module: customizes and automatically deploys sensu, + config files and all needed plugins and scripts on all clients and + the central server. + +2. Calipso sensu module: listens for the results of the customized + events and updates the inventories with state and statuses, while + generating all related messages. + +Calipso monitoring provisioning +------------------------------- + + Calipso administrator should instruct the main scanning engine + whether or not to deploy monitoring across the VIM environment + hosts. This is done first by the Calipso scan container that uses a + set of “environment\_config” parameters (see calipso-model guide for + details). + + A full scan of an environment is a mandatory pre-requisite for + monitoring, because calipso monitoring provisioning uses inventory + discovered data as values of several configuration items to be + deployed onto the sensu framework. + + Here are the important parts of an “environment\_config” document + that needs to be defined either through UI or API as a pre-requisite + for monitoring provisioning: + + { + + **"enable\_monitoring": true**, // this must be “true” for any + monitoring result + + **"name"** **: "my-env-with-monitoring",** // name is needed as + usual + + **"distribution" :** // other details removed …. + + **"configuration" : [** // in configuration: a monitoring section is + defined + + **{** + + **"name" : "Monitoring",** // name of this configuration section + + **"type" : "Sensu",** // instruct to use a sensu type of monitoring + framework + + **"rabbitmq\_port" : 5671,** // the SSL port used by sensu clients + to server + + **"rabbitmq\_pass" : "dummy\_pwd",** // clients-to-server rabbitmq + password + + **"rabbitmq\_user" : "sensu",** // clients-to-server rabbitmq + username + + **"server\_ip" : "10.0.0.1",** // the calipso sensu container ip + address + + **"ssh\_user": "root",** // user used inside the sensu server + (container) + + **"ssh\_password": "dummy\_pwd",** // root pwd used inside the sensu + server + + **"ssh\_port" : 20022, //** ssh port where sensu container offers + ssh access + + **"server\_name”: "sensu\_server", /**/ name to use for the sensu + server api + + **"env\_type" : "production",** // a type of environment to setup + (sensu’s ‘type’) + + **"config\_folder" : "/local\_dir/sensu\_config",** // location to + place config files + + **"api\_port" : 4567** // sensu server api port to connect to + + **"provision" : "None" // \*\* see provision options described next + ! \*\*** + + **}, ** + + **]** + + \*\* **Provision** options are: + +1. **“None”:** does not deploy any monitoring configurations + +2. **“DB”:** creates all necessary configuration files in the mongoDB, + placed in “monitoring\_config” collection. + +3. **“Files”:** creates all necessary configuration files in the + mongoDB, placed in “monitoring\_config” collection and also in + ‘config\_folder’ location on the server. + +4. **“Deploy”:** Does what “Files” is doing, but also goes and deploys + all configuration files and scripts on all servers on the + environment. + + *This is a high level representation of the calipso monitoring + provisioning logic:* + + |image1| + +Calipso-sensu container +----------------------- + + Once sensu clients and all needed configurations and plugins are + deployed properly, the sensu server should start receiving results + and update the relevant inventory objects with their states and + statuses. + + The calipso-sensu container is a pre-built sensu server customized + for calipso design. + + The following services are exposed on the calipso-sensu container: + +1. Calipso monitoring handling app running in: + /home/scan/calipso\_prod/app/monitoring/handlers/monitor.py + +2. Sensu-api service accessible on port 4567. + +3. Calipso container bash access through ssh on port 20022 (see + quickstart-guide). + +4. Sensu-server service listening as rabbitmq over ssl on port 5671. + +5. Rabbitmq management server on port 15672 (http://server-ip:15672/ to + access). + +6. Sensu uchiwa UI listening on port 3000 (http://server-ip:3000 to + access). + + All the above services are maintained by calipso and customized for + calipso’s virtual inventory monitoring design. + +Monitoring configurations +========================== + + You can access calipso-sensus container by ssh, using: + + **ssh scan@localhost -p 20022** with a default password “scan”. + + Inside the calipso-sensu container the calipso monitoring + application is maintained at: + + **/home/scan/calipso\_prod/app/monitoring** + + SSL keys are shipped with the calipso-sensu container and maintained + at: + + **/etc/sensu/ssl ** + + Check scripts used to run a health check against monitored objects + are maintained at: + + **/home/scan/calipso\_prod/app/monitoring/checks** + + Handling scripts to grab and handle checks results from clients are + maintained at: + + **/home/scan/calipso\_prod/app/monitoring/handlers** + + Configuration deployment setup scripts are maintained at: + + **/home/scan/calipso\_prod/app/monitoring/setup** + +Calipso monitoring configuration templates +------------------------------------------ + + Calipso application maintained a customizable model for all the + configuration files deployed onto the sensu server and clients at + provisioning stage of its main scanning engine. + + The following configuration files are customizable: + + *Server side:* + + **redis.json** + + **transport,json** + + **rabbitmq.json** + + **api.json** + + **client.json** + + **filters.json** + + **handlers.json** + + *Clients side:* + + **transport,json** + + **rabbitmq.json** + + **client.json** + + **customized checks to integrate into client.json ** + + We have defined a default set of configuration parameters in those + files per our knowledge and best practices, so a typical user would + not need to edit any of the above set of configuration files and + their options. In case any of these needs customizations we offer a + very granular model maintained in Calipso mongoDB under the + **“monitoring\_config\_templates”** collection. + + Here is an example of such template, and its options: + + **"type" : "client.json",** // this will be used for building + client.json files + + **"order" : "1",** // if several of these templates are configured, + take a priority to use + + **"side" : "client", //**\ client.json file deployed onto the client + side (env hosts) + + **"config" : {** // the actual configuration txt follows … + + **"client" : {** + + **"address" : "{client\_name}",** // take the name of host from scan + inventory + + **"subscriptions" : [** //a future option + + **], ** + + **"environment" : "{env\_name}",** // name of the VIM environment + for logs + + **"name" : "{client\_name}" //** take the name of host from scan + inventory + + **}, ** + + **"api" : {** + + **"host" : "{server\_ip}", //** take the server\_ip of + environment\_config + + **"port" : NumberInt(4567)** // use this port for interfacing with + the server + + **}** + + **}, ** + + **"monitoring\_system" : "sensu"** + + **}** + + The above is just a simple example, login to mongoDB and check + “monitoring\_config\_templates” collection for recent information on + deployment files and their configuration options. + + The results of the monitoring provisioning are placed, by default, + in mongoDB at the collection – “\ **monitoring\_config**\ ”, locally + on the calipso-scan container at **/local\_dir/sensu\_config** and + finally on the server (calipso-sensu container) and on all the hosts + (clients). Here is an example of the resulted client.json file on + one of the environment hosts (example deployment from real VIM + environment): + + **{** + + **"api": {** + + **"host": "korlev-calipso-dev.cisco.com",** + + **"port": 4567** + + **},** + + **"checks": {** + + **"host\_pnic\_eno16777728-00---..58..---50---..58..---56---..58..---ac---..58..---e8---..58..---97": + {** + + **"command": "check\_pnic\_ovs.py eno16777728",** + + **"handlers": [** + + **"file",** + + **"osdna-monitor"** + + **],** + + **"interval": 15,** + + **"standalone": true,** + + **"subscribers": [** + + **"base"** + + **],** + + **"type": "metric"** + + **},** + + **"host\_pnic\_eno33554952-00---..58..---50---..58..---56---..58..---ac---..58..---c9---..58..---a2": + {** + + **"command": "check\_pnic\_ovs.py eno33554952",** + + **"handlers": [** + + **"file",** + + **"osdna-monitor"** + + **],** + + **"interval": 15,** + + **"standalone": true,** + + **"subscribers": [** + + **"base"** + + **],** + + **"type": "metric"** + + **},** + + **"otep\_node-6.cisco.com-otep\_vxlan-c0a80201": {** + + **"command": "check\_ping.py -c 10 -i 0.5 -p 4f532d444e41 -w 10 -s + 256 -f 192.168.2.2 -t 192.168.2.1 -W 1%/301.11/600 -C + 10%/1020.12/2000",** + + **"handlers": [** + + **"default",** + + **"file",** + + **"osdna-monitor"** + + **],** + + **"interval": 15,** + + **"standalone": true,** + + **"subscribers": [** + + **"base"** + + **],** + + **"type": "metric"** + + **},** + + **"otep\_node-6.cisco.com-otep\_vxlan-c0a80203": {** + + **"command": "check\_ping.py -c 10 -i 0.5 -p 4f532d444e41 -w 10 -s + 256 -f 192.168.2.2 -t 192.168.2.3 -W 1%/301.11/600 -C + 10%/1020.12/2000",** + + **"handlers": [** + + **"default",** + + **"file",** + + **"osdna-monitor"** + + **],** + + **"interval": 15,** + + **"standalone": true,** + + **"subscribers": [** + + **"base"** + + **],** + + **"type": "metric"** + + **},** + + **"vedge\_bc865c43-3dc5-4940-af1d-b4be59df1bd0": {** + + **"command": "check\_vedge\_ovs.py",** + + **"handlers": [** + + **"default",** + + **"file",** + + **"osdna-monitor"** + + **],** + + **"interval": 15,** + + **"standalone": true,** + + **"subscribers": [** + + **"base"** + + **],** + + **"type": "metric"** + + **},** + + **"vservice\_qdhcp-6c5ddc76-fcd7-4bdd-bff4-1d08b88b96ca": {** + + **"command": "PYTHONPATH=/etc/sensu/plugins check\_vservice.py dhcp + qdhcp-6c5ddc76-fcd7-4bdd-bff4-1d08b88b96ca",** + + **"handlers": [** + + **"default",** + + **"file",** + + **"osdna-monitor"** + + **],** + + **"interval": 15,** + + **"standalone": true,** + + **"subscribers": [** + + **"base"** + + **],** + + **"type": "metric"** + + **},** + + **"vservice\_qdhcp-721f9c95-3042-4840-b8a4-83968c1e92b6": {** + + **"command": "PYTHONPATH=/etc/sensu/plugins check\_vservice.py dhcp + qdhcp-721f9c95-3042-4840-b8a4-83968c1e92b6",** + + **"handlers": [** + + **"default",** + + **"file",** + + **"osdna-monitor"** + + **],** + + **"interval": 15,** + + **"standalone": true,** + + **"subscribers": [** + + **"base"** + + **],** + + **"type": "metric"** + + **},** + + **"vservice\_qdhcp-cc7ea40b-bb11-4b51-8e51-1a3b7abd283d": {** + + **"command": "PYTHONPATH=/etc/sensu/plugins check\_vservice.py dhcp + qdhcp-cc7ea40b-bb11-4b51-8e51-1a3b7abd283d",** + + **"handlers": [** + + **"default",** + + **"file",** + + **"osdna-monitor"** + + **],** + + **"interval": 15,** + + **"standalone": true,** + + **"subscribers": [** + + **"base"** + + **],** + + **"type": "metric"** + + **},** + + **"vservice\_qrouter-1833846f-573e-45ef-8c87-3f7df530cdbd": {** + + **"command": "PYTHONPATH=/etc/sensu/plugins check\_vservice.py + router qrouter-1833846f-573e-45ef-8c87-3f7df530cdbd",** + + **"handlers": [** + + **"default",** + + **"file",** + + **"osdna-monitor"** + + **],** + + **"interval": 15,** + + **"standalone": true,** + + **"subscribers": [** + + **"base"** + + **],** + + **"type": "metric"** + + **}** + + **},** + + **"client": {** + + **"address": "Mirantis-Liberty-node-6.cisco.com",** + + **"environment": "Mirantis-Liberty",** + + **"name": "Mirantis-Liberty-node-6.cisco.com",** + + **"subscriptions": []** + + **}** + + **}** + + All sensu configuration files, keys and scripts are eventually + deployed, both on clients and on server side at the following + locations: + + *SSL keys:* **/etc/sensu/ssl** + + *Calipso monitoring checks*: **/etc/sensu/plugins ** + + *Configuration files:* **/etc/sensu/conf.d ** + + The calipso-scan container is in-charge of the actual deployment (in + case environment\_config is configured with needed details and + “provision” = “Deploy. + + Calipso-scan then uses the calipso-sensu as the target “sensu + server” to deploy and all the environment pre-discovered hosts as + “sensu clients” to deploy, all pointing their monitoring results + back to the calipso-sensu container. + + Calipso-scan uses ssh to access all hosts (through the master-host, + see admin-guide) and also to access calipso-sensu container on port + 20022 to upload all customized files and places them in the above + locations. + +Calipso Apex monitoring integration +----------------------------------- + + For OPNFV version ‘P’ of the calipso application, farther automation + has been developed for ‘zero touch’ automation. Calipso has a + built-in ‘apex-configurator’ that runs at apex install phase + (current scenario: os-nosdn-calipso-noha) and deploys the sensu + clients themselves with all needed configurations, per apex install + parameters and customizes the calipso-sensu container accordingly…no + manual UI or API steps are needed for end-to-end functionality. + +.. |image0| image:: media/image1.png + :width: 6.50000in + :height: 4.27153in +.. |image1| image:: media/image10.png + :width: 6.50000in + :height: 3.62708in diff --git a/docs/release/quickstart-guide.rst b/docs/release/quickstart-guide.rst new file mode 100644 index 0000000..19366a2 --- /dev/null +++ b/docs/release/quickstart-guide.rst @@ -0,0 +1,574 @@ +############################################################################### +# Copyright (c) 2017 Koren Lev (Cisco Systems), Yaron Yogev (Cisco Systems) # +# and others # +# # +# All rights reserved. This program and the accompanying materials # +# are made available under the terms of the Apache License, Version 2.0 # +# which accompanies this distribution, and is available at # +# http://www.apache.org/licenses/LICENSE-2.0 # +############################################################################### +| Calipso.io +| Quick Start Guide + +|image0| + +Project “Calipso” tries to illuminate complex virtual networking with +real time operational state visibility for large and highly distributed +Virtual Infrastructure Management (VIM). + +We believe that Stability is driven by accurate Visibility. + +Calipso provides visible insights using smart discovery and virtual +topological representation in graphs, with monitoring per object in the +graph inventory to reduce error vectors and troubleshooting, maintenance +cycles for VIM operators and administrators. + +Table of Contents + +Calipso.io Quick Start Guide 1 + +1 Getting started 3 + +1.1 Post installation tools 3 + +1.2 Calipso containers details 3 + +1.3 Calipso containers access 5 + +2 Validating Calipso app 5 + +2.1 Validating calipso-mongo module 5 + +2.2 Validating calipso-scan module 7 + +2.3 Validating calipso-listen module 8 + +2.4 Validating calipso-api module 9 + +2.5 Validating calipso-sensu module 9 + +2.6 Validating calipso-ui module 10 + +2.7 Validating calipso-ldap module 10 + +Getting started +=============== + +Post installation tools +------------------------ + + Calipso administrator should first complete installation as per + install-guide document. + + After all calipso containers are running she can start examining the + application using the following suggested tools: + +1. MongoChef : https://studio3t.com/download/ as a useful GUI client to + interact with calipso mongoDB module. + +2. Web Browser to access calipso-UI at the default localtion: + http://server-IP + +3. SSH client to access other calipso containers as needed. + +4. Python3 toolsets for debugging and development as needed. + +Calipso containers details +-------------------------- + + Calipso is currently made of the following 7 containers: + +1. Mongo: holds and maintains calipso’s data inventories. + +2. LDAP: holds and maintains calipso’s user directories. + +3. Scan: deals with automatic discovery of virtual networking from VIMs. + +4. Listen: deals with automatic updating of virtual networking into + inventories. + +5. API: runs calipso’s RESTful API server. + +6. UI: runs calipso’s GUI/web server. + +7. Sensu: runs calipso’s monitoring server. + + After successful installation Calipso containers should have been + downloaded, registered and started, here are the images used: + + **sudo docker images** + + Expected results (as of Aug 2017): + + **REPOSITORY TAG IMAGE ID CREATED SIZE** + + **korenlev/calipso listen 12086aaedbc3 6 hours ago 1.05GB** + + **korenlev/calipso api 34c4c6c1b03e 6 hours ago 992MB** + + **korenlev/calipso scan 1ee60c4e61d5 6 hours ago 1.1GB** + + **korenlev/calipso sensu a8a17168197a 6 hours ago 1.65GB** + + **korenlev/calipso mongo 17f2d62f4445 22 hours ago 1.31GB** + + **korenlev/calipso ui ab37b366e812 11 days ago 270MB** + + **korenlev/calipso ldap 316bc94b25ad 2 months ago 269MB** + + Typically Calipso application is fully operational at this stage and + you can jump to ‘Using Calipso’ section to learn how to use it, the + following explains how the containers are deployed by + calipso-installer.py for general reference. + + Checking the running containers status and ports in use: + + **sudo docker ps** + + Expected results and details (as of Aug 2017): + +|image2| + + The above listed TCP ports are used by default on the hosts to map + to each calipso container, you should be familiar with these + mappings of ports per container. + + Checking running containers entry-points (The commands used inside + the container): + + **sudo docker inspect [container-ID]** + + Expected results (as of Aug 2017): + +|image3| + + Calipso containers configuration can be listed with **docker + inspect**, summarized in the table above. In a none-containerized + deployment (see ‘Monolithic app install option in the install-guide) + these are the individual commands that are needed to run calipso + manually for special development needs. + + The ‘calipso-sensu’ is built using sensu framework customized for + calipso monitoring design, ‘calipso-ui’ is built using meteor + framework, ‘calipso-ldap’ is built using pre-defined open-ldap + container, and as such those three are only supported as pre-built + containers. + + Administrator should be aware of the following details deployed in + the containers: + +1. calipso-api, calipso-sensu, calipso-scan and calipso-listen maps host + directory **/home/calipso as volume /local\_dir** inside the + container. + + They use **calipso\_mongo\_access.conf** and **ldap.conf** files for + configuration. + + They use **/home/scan/calipso\_prod/app** as the main PYTHONPATH + needed to run the different python modules per container. + +2. Calipso-sensu is using the ‘supervisord’ process to control all sensu + server processes needed for calipso and the calipso event handler on + this container. + +3. Calipso-ldap can be used as standalone, but is a pre-requisite for + calipso-api. + +4. Calipso-ui needs calipso-mongo with latest scheme, to run and offer + UI services. + +Calipso containers access +------------------------- + + The different Calipso containers are also accessible using SSH and + pre-defined default credentials, here is the access details: + + Calipso-listen: ssh scan@localhost –p 50022 , password = scan + + Calipso-scan: ssh scan@localhost –p 30022 , password = scan + + Calipso-api: ssh scan@localhost –p 40022 , password = scan + + Calipso-sensu: ssh scan@localhost –p 20022 , password = scan + + Calipso-ui: only accessible through web browser + + Calipso-ldap: only accessible through ldap tools. + + Calipso-mongo: only accessible through mongo clients like MongoChef. + +Validating Calipso app +====================== + +Validating calipso-mongo module +------------------------------- + + Using MongoChef client, create a new connection pointing to the + server where calipso-mongo container is running, using port 27017 + and the following default credentials: + + Host IP=server\_IP and TCP port=27017 + + Username : calipso + + Password : calipso\_default + + Auto-DB: calipso + + Defaults are also configured into + /home/calipso/calipso\_mongo\_access.conf. + + The following is a screenshot of a correct connection setup in + MongoChef: + + |image4| + + When clicking on the new defined connection the calipso DB should be + listed: + + |image5| + + At this stage you can checkout calipso-mongo collections data and + validate as needed. + +Validating calipso-scan module +------------------------------ + + Scan container is running the main calipso scanning engine that + receives requests to scan a specific VIM environment, this command + will validate that the main scan\_manager.py process is running and + waiting for scan requests: + + **sudo docker ps** **# grab the containerID of calipso-scan** + + **sudo docker logs bf5f2020028a #containerID for example** + + Expected results: + + **2017-08-28 06:11:39,231 INFO: Using inventory collection: + inventory** + + **2017-08-28 06:11:39,231 INFO: Using links collection: links** + + **2017-08-28 06:11:39,231 INFO: Using link\_types collection: + link\_types** + + **2017-08-28 06:11:39,231 INFO: Using clique\_types collection: + clique\_types** + + **2017-08-28 06:11:39,231 INFO: Using clique\_constraints + collection: clique\_constraints** + + **2017-08-28 06:11:39,231 INFO: Using cliques collection: cliques** + + **2017-08-28 06:11:39,232 INFO: Using monitoring\_config collection: + monitoring\_config** + + **2017-08-28 06:11:39,232 INFO: Using constants collection: + constants** + + **2017-08-28 06:11:39,232 INFO: Using scans collection: scans** + + **2017-08-28 06:11:39,232 INFO: Using messages collection: + messages** + + **2017-08-28 06:11:39,232 INFO: Using monitoring\_config\_templates + collection: monitoring\_config\_templates** + + **2017-08-28 06:11:39,232 INFO: Using environments\_config + collection: environments\_config** + + **2017-08-28 06:11:39,232 INFO: Using supported\_environments + collection: supported\_environments** + + **2017-08-28 06:11:39,233 INFO: Started ScanManager with following + configuration:** + + **Mongo config file path: /local\_dir/calipso\_mongo\_access.conf** + + **Scans collection: scans** + + **Environments collection: environments\_config** + + **Polling interval: 1 second(s)** + + The above logs basically shows that scan\_manager.py is running and + listening to scan requests (should they come in through into ‘scans’ + collection for specific environment listed in ‘environments\_config’ + collection, refer to use-guide for details). + +Validating calipso-listen module +-------------------------------- + + Listen container is running the main calipso event\_manager engine + that listens for events on a specific VIM BUS environment, this + command will validate that the main event\_manager.py process is + running and waiting for events from the BUS: + + **2017-08-28 06:11:35,572 INFO: Using inventory collection: + inventory** + + **2017-08-28 06:11:35,572 INFO: Using links collection: links** + + **2017-08-28 06:11:35,572 INFO: Using link\_types collection: + link\_types** + + **2017-08-28 06:11:35,572 INFO: Using clique\_types collection: + clique\_types** + + **2017-08-28 06:11:35,572 INFO: Using clique\_constraints + collection: clique\_constraints** + + **2017-08-28 06:11:35,573 INFO: Using cliques collection: cliques** + + **2017-08-28 06:11:35,573 INFO: Using monitoring\_config collection: + monitoring\_config** + + **2017-08-28 06:11:35,573 INFO: Using constants collection: + constants** + + **2017-08-28 06:11:35,573 INFO: Using scans collection: scans** + + **2017-08-28 06:11:35,573 INFO: Using messages collection: + messages** + + **2017-08-28 06:11:35,573 INFO: Using monitoring\_config\_templates + collection: monitoring\_config\_templates** + + **2017-08-28 06:11:35,573 INFO: Using environments\_config + collection: environments\_config** + + **2017-08-28 06:11:35,574 INFO: Using supported\_environments + collection: supported\_environments** + + **2017-08-28 06:11:35,574 INFO: Started EventManager with following + configuration:** + + **Mongo config file path: /local\_dir/calipso\_mongo\_access.conf** + + **Collection: environments\_config** + + **Polling interval: 5 second(s)** + + The above logs basically shows that event\_manager.py is running and + listening to event (should they come in through from VIM BUS) and + listed in ‘environments\_config’ collection, refer to use-guide for + details). + +Validating calipso-api module +----------------------------- + + Scan container is running the main calipso API that allows + applications to integrate with calipso inventory and functions, this + command will validate it is operational: + + **sudo docker ps** **# grab the containerID of calipso-scan** + + **sudo docker logs bf5f2020028c #containerID for example** + + Expected results: + + **2017-08-28 06:11:38,118 INFO: Using inventory collection: + inventory** + + **2017-08-28 06:11:38,119 INFO: Using links collection: links** + + **2017-08-28 06:11:38,119 INFO: Using link\_types collection: + link\_types** + + **2017-08-28 06:11:38,119 INFO: Using clique\_types collection: + clique\_types** + + **2017-08-28 06:11:38,120 INFO: Using clique\_constraints + collection: clique\_constraints** + + **2017-08-28 06:11:38,120 INFO: Using cliques collection: cliques** + + **2017-08-28 06:11:38,121 INFO: Using monitoring\_config collection: + monitoring\_config** + + **2017-08-28 06:11:38,121 INFO: Using constants collection: + constants** + + **2017-08-28 06:11:38,121 INFO: Using scans collection: scans** + + **2017-08-28 06:11:38,121 INFO: Using messages collection: + messages** + + **2017-08-28 06:11:38,121 INFO: Using monitoring\_config\_templates + collection: monitoring\_config\_templates** + + **2017-08-28 06:11:38,122 INFO: Using environments\_config + collection: environments\_config** + + **2017-08-28 06:11:38,122 INFO: Using supported\_environments + collection: supported\_environments** + + **[2017-08-28 06:11:38 +0000] [6] [INFO] Starting gunicorn 19.4.5** + + **[2017-08-28 06:11:38 +0000] [6] [INFO] Listening at: + http://0.0.0.0:8000 (6)** + + **[2017-08-28 06:11:38 +0000] [6] [INFO] Using worker: sync** + + **[2017-08-28 06:11:38 +0000] [12] [INFO] Booting worker with pid: + 12** + + The above logs basically shows that the calipso api is running and + listening on port 8000 for requests. + +Validating calipso-sensu module +------------------------------- + + Sensu container is running several servers (currently unified into + one for simplicity) and the calipso event handler (refer to + use-guide for details), here is how to validate it is operational: + + **ssh scan@localhost -p 20022 # default password = scan** + + **sudo /etc/init.d/sensu-client status** + + **sudo /etc/init.d/sensu-server status** + + **sudo /etc/init.d/sensu-api status** + + **sudo /etc/init.d/uchiwa status** + + **sudo /etc/init.d/rabbitmq-server status** + + Expected results: + + **Each of the above should return a pid and a ‘running’ state +** + + **ls /home/scan/calipso\_prod/app/monitoring/handlers # should list + monitor.py module.** + + The above logs basically shows that calipso-sensu is running and + listening to monitoring events from sensu-clients on VIM hosts, + refer to use-guide for details). + +Validating calipso-ui module +---------------------------- + + UI container is running several JS process with the back-end + mongoDB, it needs data to run and it will not run if any connection + with DB is lost, this is per design. To validate operational state + of the UI simply point a Web Browser to : http://server-IP:80 and + expect a login page. Use admin/123456 as default credentials to + login: + + |image6| + +Validating calipso-ldap module +------------------------------ + + LDAP container is running a common user directory for integration + with UI and API modules, it is placed with calipso to validate + interaction with LDAP. The main configuration needed for + communication with it is stored by calipso installer in + /home/calipso/ldap.conf and accessed by the API module. We assume in + production use-cases a corporate LDAP server might be used instead, + in that case ldap.conf needs to be changed and point to the + corporate server. + + To validate LDAP container, you will need to install + openldap-clients, using: + + **yum -y install openldap-clients / apt-get install + openldap-clients** + + Search all LDAP users inside that ldap server: + + **ldapsearch -H ldap://localhost -LL -b ou=Users,dc=openstack,dc=org + x** + + Admin user details on this container (user=admin, pass=password): + + **LDAP username : cn=admin,dc=openstack,dc=org** + + **cn=admin,dc=openstack,dc=org's password : password** + + **Account BaseDN [DC=168,DC=56,DC=153:49154]: + ou=Users,dc=openstack,dc=org** + + **Group BaseDN [ou=Users,dc=openstack,dc=org]:** + + Add a new user (admin credentials needed to bind to ldap and add + users): + + Create a **/tmp/adduser.ldif** file, use this example: + + **dn: cn=Myname,ou=Users,dc=openstack,dc=org // which org, which ou + etc ...** + + **objectclass: inetOrgPerson** + + **cn: Myname // match the dn details !** + + **sn: Koren** + + **uid: korlev** + + **userpassword: mypassword // the password** + + **carlicense: MYCAR123** + + **homephone: 555-111-2222** + + **mail: korlev@cisco.com** + + **description: koren guy** + + **ou: calipso Department** + + Run this command to add the above user attributes into the ldap + server: + + **ldapadd -x -D cn=admin,dc=openstack,dc=org -w password -c -f + /tmp/adduser.ldif** // for example, the above file is used and the + admin bind credentials who is, by default, authorized to add users. + + You should see **"user added"** message if successful + + Validate users against this LDAP container: + + Wrong credentials: + + **ldapwhoami -x -D cn=Koren,ou=Users,dc=openstack,dc=org -w + korlevwrong** + + **Response: ldap\_bind: Invalid credentials (49)** + + Correct credentials: + + **ldapwhoami -x -D cn=Koren,ou=Users,dc=openstack,dc=org -w korlev** + + **Response: dn:cn=Koren,ou=Users,dc=openstack,dc=org** + + The reply ou/dc details can be used by any application (UI and API + etc) for mapping users to some application specific group… + +- If all the above validations passed, Calipso is now fully functional, + refer to admin-guide for more details. + +.. |image0| image:: media/image1.png + :width: 6.50000in + :height: 4.27153in +.. |image1| image:: media/image2.png + :width: 7.34814in + :height: 2.09375in +.. |image2| image:: media/image3.png + :width: 7.13920in + :height: 3.94792in +.. |image3| image:: media/image4.png + :width: 6.21875in + :height: 3.50203in +.. |image4| image:: media/image5.png + :width: 4.60359in + :height: 4.32238in +.. |image5| image:: media/image6.png + :width: 6.50000in + :height: 1.55903in +.. |image6| image:: media/image7.png + :width: 6.50000in + :height: 1.55903in |
