aboutsummaryrefslogtreecommitdiffstats
path: root/docs/release/quickstart-guide.rst
blob: 413ad26432efb4af5d89e0d4a7972aa4907a74da (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
| Calipso.io
| Quick Start Guide

Copyright (c) 2017 Koren Lev (Cisco Systems), Yaron Yogev (Cisco Systems) and others                                                                
All rights reserved. This program and the accompanying materials           
are made available under the terms of the Apache License, Version 2.0       
which accompanies this distribution, and is available at                    
http://www.apache.org/licenses/LICENSE-2.0

|image0|

Project “Calipso” tries to illuminate complex virtual networking with
real time operational state visibility for large and highly distributed
Virtual Infrastructure Management (VIM).

We believe that Stability is driven by accurate Visibility.

Calipso provides visible insights using smart discovery and virtual
topological representation in graphs, with monitoring per object in the
graph inventory to reduce error vectors and troubleshooting, maintenance
cycles for VIM operators and administrators.

Table of Contents

Calipso.io Quick Start Guide 1

1 Getting started 3

1.1 Post installation tools 3

1.2 Calipso containers details 3

1.3 Calipso containers access 5

2 Validating Calipso app 5

2.1 Validating calipso-mongo module 5

2.2 Validating calipso-scan module 7

2.3 Validating calipso-listen module 8

2.4 Validating calipso-api module 9

2.5 Validating calipso-sensu module 9

2.6 Validating calipso-ui module 10

2.7 Validating calipso-ldap module 10

Getting started
===============

Post installation tools 
------------------------

    Calipso administrator should first complete installation as per
    install-guide document.

    After all calipso containers are running she can start examining the
    application using the following suggested tools:

1. MongoChef : https://studio3t.com/download/ as a useful GUI client to
   interact with calipso mongoDB module.

2. Web Browser to access calipso-UI at the default localtion:
   http://server-IP

3. SSH client to access other calipso containers as needed.

4. Python3 toolsets for debugging and development as needed.

Calipso containers details
--------------------------

    Calipso is currently made of the following 7 containers:

1. Mongo: holds and maintains calipso’s data inventories.

2. LDAP: holds and maintains calipso’s user directories.

3. Scan: deals with automatic discovery of virtual networking from VIMs.

4. Listen: deals with automatic updating of virtual networking into
   inventories.

5. API: runs calipso’s RESTful API server.

6. UI: runs calipso’s GUI/web server.

7. Sensu: runs calipso’s monitoring server.

    After successful installation Calipso containers should have been
    downloaded, registered and started, here are the images used:

    **sudo docker images**

    Expected results (as of Aug 2017):

    **REPOSITORY        TAG       IMAGE    ID CREATED     SIZE**

    **korenlev/calipso listen 12086aaedbc3 6 hours ago 1.05GB**

    **korenlev/calipso api 34c4c6c1b03e 6 hours ago 992MB**

    **korenlev/calipso scan 1ee60c4e61d5 6 hours ago 1.1GB**

    **korenlev/calipso sensu a8a17168197a 6 hours ago 1.65GB**

    **korenlev/calipso mongo 17f2d62f4445 22 hours ago 1.31GB**

    **korenlev/calipso ui ab37b366e812 11 days ago 270MB**

    **korenlev/calipso ldap 316bc94b25ad 2 months ago 269MB**

    Typically Calipso application is fully operational at this stage and
    you can jump to ‘Using Calipso’ section to learn how to use it, the
    following explains how the containers are deployed by
    calipso-installer.py for general reference.

    Checking the running containers status and ports in use:

    **sudo docker ps**

    Expected results and details (as of Aug 2017):

|image2|

    The above listed TCP ports are used by default on the hosts to map
    to each calipso container, you should be familiar with these
    mappings of ports per container.

    Checking running containers entry-points (The commands used inside
    the container):

    **sudo docker inspect [container-ID]**

    Expected results (as of Aug 2017):

|image3|

    Calipso containers configuration can be listed with **docker
    inspect**, summarized in the table above. In a none-containerized
    deployment (see ‘Monolithic app install option in the install-guide)
    these are the individual commands that are needed to run calipso
    manually for special development needs.

    The ‘calipso-sensu’ is built using sensu framework customized for
    calipso monitoring design, ‘calipso-ui’ is built using meteor
    framework, ‘calipso-ldap’ is built using pre-defined open-ldap
    container, and as such those three are only supported as pre-built
    containers.

    Administrator should be aware of the following details deployed in
    the containers:

1. calipso-api, calipso-sensu, calipso-scan and calipso-listen maps host
   directory **/home/calipso as volume /local\_dir** inside the
   container.

   They use **calipso\_mongo\_access.conf** and **ldap.conf** files for
   configuration.

   They use **/home/scan/calipso\_prod/app** as the main PYTHONPATH
   needed to run the different python modules per container.

2. Calipso-sensu is using the ‘supervisord’ process to control all sensu
   server processes needed for calipso and the calipso event handler on
   this container.

3. Calipso-ldap can be used as standalone, but is a pre-requisite for
   calipso-api.

4. Calipso-ui needs calipso-mongo with latest scheme, to run and offer
   UI services.

Calipso containers access
-------------------------

    The different Calipso containers are also accessible using SSH and
    pre-defined default credentials, here is the access details:

    Calipso-listen: ssh scan@localhost –p 50022 , password = scan

    Calipso-scan: ssh scan@localhost –p 30022 , password = scan

    Calipso-api: ssh scan@localhost –p 40022 , password = scan

    Calipso-sensu: ssh scan@localhost –p 20022 , password = scan

    Calipso-ui: only accessible through web browser

    Calipso-ldap: only accessible through ldap tools.

    Calipso-mongo: only accessible through mongo clients like MongoChef.

Validating Calipso app
======================

Validating calipso-mongo module
-------------------------------

    Using MongoChef client, create a new connection pointing to the
    server where calipso-mongo container is running, using port 27017
    and the following default credentials:

    Host IP=server\_IP and TCP port=27017

    Username : calipso

    Password : calipso\_default

    Auto-DB: calipso

    Defaults are also configured into
    /home/calipso/calipso\_mongo\_access.conf.

    The following is a screenshot of a correct connection setup in
    MongoChef:

    |image4|

    When clicking on the new defined connection the calipso DB should be
    listed:

    |image5|

    At this stage you can checkout calipso-mongo collections data and
    validate as needed.

Validating calipso-scan module
------------------------------

    Scan container is running the main calipso scanning engine that
    receives requests to scan a specific VIM environment, this command
    will validate that the main scan\_manager.py process is running and
    waiting for scan requests:

    **sudo docker ps** **# grab the containerID of calipso-scan**

    **sudo docker logs bf5f2020028a #containerID for example**

    Expected results:

    **2017-08-28 06:11:39,231 INFO: Using inventory collection:
    inventory**

    **2017-08-28 06:11:39,231 INFO: Using links collection: links**

    **2017-08-28 06:11:39,231 INFO: Using link\_types collection:
    link\_types**

    **2017-08-28 06:11:39,231 INFO: Using clique\_types collection:
    clique\_types**

    **2017-08-28 06:11:39,231 INFO: Using clique\_constraints
    collection: clique\_constraints**

    **2017-08-28 06:11:39,231 INFO: Using cliques collection: cliques**

    **2017-08-28 06:11:39,232 INFO: Using monitoring\_config collection:
    monitoring\_config**

    **2017-08-28 06:11:39,232 INFO: Using constants collection:
    constants**

    **2017-08-28 06:11:39,232 INFO: Using scans collection: scans**

    **2017-08-28 06:11:39,232 INFO: Using messages collection:
    messages**

    **2017-08-28 06:11:39,232 INFO: Using monitoring\_config\_templates
    collection: monitoring\_config\_templates**

    **2017-08-28 06:11:39,232 INFO: Using environments\_config
    collection: environments\_config**

    **2017-08-28 06:11:39,232 INFO: Using supported\_environments
    collection: supported\_environments**

    **2017-08-28 06:11:39,233 INFO: Started ScanManager with following
    configuration:**

    **Mongo config file path: /local\_dir/calipso\_mongo\_access.conf**

    **Scans collection: scans**

    **Environments collection: environments\_config**

    **Polling interval: 1 second(s)**

    The above logs basically shows that scan\_manager.py is running and
    listening to scan requests (should they come in through into ‘scans’
    collection for specific environment listed in ‘environments\_config’
    collection, refer to use-guide for details).

Validating calipso-listen module
--------------------------------

    Listen container is running the main calipso event\_manager engine
    that listens for events on a specific VIM BUS environment, this
    command will validate that the main event\_manager.py process is
    running and waiting for events from the BUS:

    **2017-08-28 06:11:35,572 INFO: Using inventory collection:
    inventory**

    **2017-08-28 06:11:35,572 INFO: Using links collection: links**

    **2017-08-28 06:11:35,572 INFO: Using link\_types collection:
    link\_types**

    **2017-08-28 06:11:35,572 INFO: Using clique\_types collection:
    clique\_types**

    **2017-08-28 06:11:35,572 INFO: Using clique\_constraints
    collection: clique\_constraints**

    **2017-08-28 06:11:35,573 INFO: Using cliques collection: cliques**

    **2017-08-28 06:11:35,573 INFO: Using monitoring\_config collection:
    monitoring\_config**

    **2017-08-28 06:11:35,573 INFO: Using constants collection:
    constants**

    **2017-08-28 06:11:35,573 INFO: Using scans collection: scans**

    **2017-08-28 06:11:35,573 INFO: Using messages collection:
    messages**

    **2017-08-28 06:11:35,573 INFO: Using monitoring\_config\_templates
    collection: monitoring\_config\_templates**

    **2017-08-28 06:11:35,573 INFO: Using environments\_config
    collection: environments\_config**

    **2017-08-28 06:11:35,574 INFO: Using supported\_environments
    collection: supported\_environments**

    **2017-08-28 06:11:35,574 INFO: Started EventManager with following
    configuration:**

    **Mongo config file path: /local\_dir/calipso\_mongo\_access.conf**

    **Collection: environments\_config**

    **Polling interval: 5 second(s)**

    The above logs basically shows that event\_manager.py is running and
    listening to event (should they come in through from VIM BUS) and
    listed in ‘environments\_config’ collection, refer to use-guide for
    details).

Validating calipso-api module
-----------------------------

    Scan container is running the main calipso API that allows
    applications to integrate with calipso inventory and functions, this
    command will validate it is operational:

    **sudo docker ps** **# grab the containerID of calipso-scan**

    **sudo docker logs bf5f2020028c #containerID for example**

    Expected results:

    **2017-08-28 06:11:38,118 INFO: Using inventory collection:
    inventory**

    **2017-08-28 06:11:38,119 INFO: Using links collection: links**

    **2017-08-28 06:11:38,119 INFO: Using link\_types collection:
    link\_types**

    **2017-08-28 06:11:38,119 INFO: Using clique\_types collection:
    clique\_types**

    **2017-08-28 06:11:38,120 INFO: Using clique\_constraints
    collection: clique\_constraints**

    **2017-08-28 06:11:38,120 INFO: Using cliques collection: cliques**

    **2017-08-28 06:11:38,121 INFO: Using monitoring\_config collection:
    monitoring\_config**

    **2017-08-28 06:11:38,121 INFO: Using constants collection:
    constants**

    **2017-08-28 06:11:38,121 INFO: Using scans collection: scans**

    **2017-08-28 06:11:38,121 INFO: Using messages collection:
    messages**

    **2017-08-28 06:11:38,121 INFO: Using monitoring\_config\_templates
    collection: monitoring\_config\_templates**

    **2017-08-28 06:11:38,122 INFO: Using environments\_config
    collection: environments\_config**

    **2017-08-28 06:11:38,122 INFO: Using supported\_environments
    collection: supported\_environments**

    **[2017-08-28 06:11:38 +0000] [6] [INFO] Starting gunicorn 19.4.5**

    **[2017-08-28 06:11:38 +0000] [6] [INFO] Listening at:
    http://0.0.0.0:8000 (6)**

    **[2017-08-28 06:11:38 +0000] [6] [INFO] Using worker: sync**

    **[2017-08-28 06:11:38 +0000] [12] [INFO] Booting worker with pid:
    12**

    The above logs basically shows that the calipso api is running and
    listening on port 8000 for requests.

Validating calipso-sensu module
-------------------------------

    Sensu container is running several servers (currently unified into
    one for simplicity) and the calipso event handler (refer to
    use-guide for details), here is how to validate it is operational:

    **ssh scan@localhost -p 20022 # default password = scan**

    **sudo /etc/init.d/sensu-client status**

    **sudo /etc/init.d/sensu-server status**

    **sudo /etc/init.d/sensu-api status**

    **sudo /etc/init.d/uchiwa status**

    **sudo /etc/init.d/rabbitmq-server status**

    Expected results:

    **Each of the above should return a pid and a ‘running’ state +**

    **ls /home/scan/calipso\_prod/app/monitoring/handlers # should list
    monitor.py module.**

    The above logs basically shows that calipso-sensu is running and
    listening to monitoring events from sensu-clients on VIM hosts,
    refer to use-guide for details).

Validating calipso-ui module
----------------------------

    UI container is running several JS process with the back-end
    mongoDB, it needs data to run and it will not run if any connection
    with DB is lost, this is per design. To validate operational state
    of the UI simply point a Web Browser to : http://server-IP:80 and
    expect a login page. Use admin/123456 as default credentials to
    login:

    |image6|

Validating calipso-ldap module
------------------------------

    LDAP container is running a common user directory for integration
    with UI and API modules, it is placed with calipso to validate
    interaction with LDAP. The main configuration needed for
    communication with it is stored by calipso installer in
    /home/calipso/ldap.conf and accessed by the API module. We assume in
    production use-cases a corporate LDAP server might be used instead,
    in that case ldap.conf needs to be changed and point to the
    corporate server.

    To validate LDAP container, you will need to install
    openldap-clients, using:

    **yum -y install openldap-clients / apt-get install
    openldap-clients**

    Search all LDAP users inside that ldap server:

    **ldapsearch -H ldap://localhost -LL -b ou=Users,dc=openstack,dc=org
    x**

    Admin user details on this container (user=admin, pass=password):

    **LDAP username : cn=admin,dc=openstack,dc=org**

    **cn=admin,dc=openstack,dc=org's password : password**

    **Account BaseDN [DC=168,DC=56,DC=153:49154]:
    ou=Users,dc=openstack,dc=org**

    **Group BaseDN [ou=Users,dc=openstack,dc=org]:**

    Add a new user (admin credentials needed to bind to ldap and add
    users):

    Create a **/tmp/adduser.ldif** file, use this example:

    **dn: cn=Myname,ou=Users,dc=openstack,dc=org // which org, which ou
    etc ...**

    **objectclass: inetOrgPerson**

    **cn: Myname // match the dn details !**

    **sn: Koren**

    **uid: korlev**

    **userpassword: mypassword // the password**

    **carlicense: MYCAR123**

    **homephone: 555-111-2222**

    **mail: korlev@cisco.com**

    **description: koren guy**

    **ou: calipso Department**

    Run this command to add the above user attributes into the ldap
    server:

    **ldapadd -x -D cn=admin,dc=openstack,dc=org -w password -c -f
    /tmp/adduser.ldif** // for example, the above file is used and the
    admin bind credentials who is, by default, authorized to add users.

    You should see **"user added"** message if successful

    Validate users against this LDAP container:

    Wrong credentials:

    **ldapwhoami -x -D cn=Koren,ou=Users,dc=openstack,dc=org -w
    korlevwrong**

    **Response: ldap\_bind: Invalid credentials (49)**

    Correct credentials:

    **ldapwhoami -x -D cn=Koren,ou=Users,dc=openstack,dc=org -w korlev**

    **Response: dn:cn=Koren,ou=Users,dc=openstack,dc=org**

    The reply ou/dc details can be used by any application (UI and API
    etc) for mapping users to some application specific group…

-  If all the above validations passed, Calipso is now fully functional,
   refer to admin-guide for more details.

.. |image0| image:: media/image1.png
   :width: 6.50000in
   :height: 4.27153in
.. |image1| image:: media/image2.png
   :width: 7.34814in
   :height: 2.09375in
.. |image2| image:: media/image3.png
   :width: 7.13920in
   :height: 3.94792in
.. |image3| image:: media/image4.png
   :width: 6.21875in
   :height: 3.50203in
.. |image4| image:: media/image5.png
   :width: 4.60359in
   :height: 4.32238in
.. |image5| image:: media/image6.png
   :width: 6.50000in
   :height: 1.55903in
.. |image6| image:: media/image7.png
   :width: 6.50000in
   :height: 1.55903in