diff options
| author |   Ruan HE <ruan.he@orange.com> | 2016-06-09 08:12:34 +0000 |
|---|---|---|
| committer |   Gerrit Code Review <gerrit@172.30.200.206> | 2016-06-09 08:12:34 +0000 |
| commit | 4bc079a2664f9a407e332291f34d174625a9d5ea (patch) | |
| tree | 7481cd5d0a9b3ce37c44c797a1e0d39881221cbe /keystone-moon/doc/source | |
| parent | 2f179c5790fbbf6144205d3c6e5089e6eb5f048a (diff) | |
| parent | 2e7b4f2027a1147ca28301e4f88adf8274b39a1f (diff) | |
Merge "Update Keystone core to Mitaka."
Diffstat (limited to 'keystone-moon/doc/source')
26 files changed, 1202 insertions, 298 deletions
diff --git a/keystone-moon/doc/source/apache-httpd.rst b/keystone-moon/doc/source/apache-httpd.rst index dbebc86e..1436ddad 100644 --- a/keystone-moon/doc/source/apache-httpd.rst +++ b/keystone-moon/doc/source/apache-httpd.rst @@ -19,17 +19,73 @@ Running Keystone in HTTPD ========================= -.. WARNING:: +mod_proxy_uwsgi +--------------- + +The recommended keystone deployment is to have a real web server such as Apache +HTTPD or nginx handle the HTTP connections and proxy requests to an independent +keystone server (or servers) running under a wsgi container such as uwsgi or +gunicorn. The typical deployment will have several applications proxied by the +web server (for example horizon on /dashboard and keystone on /identity, +/identity_admin, port :5000, and :35357). Proxying allows the applications to +be shut down and restarted independently, and a problem in one application +isn't going to affect the web server or other applications. The servers can +easily be run in their own virtualenvs. + +The httpd/ directory contains sample files for configuring HTTPD to proxy +requests to keystone servers running under uwsgi. + +Copy the `httpd/uwsgi-keystone.conf` sample configuration file to the +appropriate location for your Apache server, on Debian/Ubuntu systems it is:: + + /etc/apache2/sites-available/uwsgi-keystone.conf + +On Red Hat based systems it is:: + + /etc/httpd/conf.d/uwsgi-keystone.conf + +Update the file to match your system configuration. Enable TLS by supplying the +correct certificates. + +Enable mod_proxy_uwsgi. + +* On Ubuntu the required package is libapache2-mod-proxy-uwsgi; enable using + ``sudo a2enmod proxy`` +* On Fedora the required package is mod_proxy_uwsgi; enable by creating a file + ``/etc/httpd/conf.modules.d/11-proxy_uwsgi.conf`` containing + ``LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so`` - Running Keystone under HTTPD in the recommended (and tested) configuration - does not support the use of ``Transfer-Encoding: chunked``. This is due to - a limitation with the WSGI spec and the implementation used by - ``mod_wsgi``. It is recommended that all clients assume Keystone will not - support ``Transfer-Encoding: chunked``. +Enable the site by creating a symlink from the file in ``sites-available`` to +``sites-enabled``, for example, on Debian/Ubuntu systems +(not required on Red Hat based systems):: + + ln -s /etc/apache2/sites-available/uwsgi-keystone.conf /etc/apache2/sites-enabled/ + +Start or restart HTTPD to pick up the new configuration. + +Now configure and start the uwsgi services. Copy the +`httpd/keystone-uwsgi-admin.ini` and `httpd/keystone-uwsgi-public.ini` files to +`/etc/keystone`. Update the files to match your system configuration (for +example, you'll want to set the number of threads for the public and admin +servers). +Start up the keystone servers using uwsgi:: + + $ sudo pip install uwsgi + $ uwsgi /etc/keystone/keystone-uwsgi-admin.ini + $ uwsgi /etc/keystone/keystone-uwsgi-public.ini + + +mod_wsgi +-------- + +.. WARNING:: -Files ------ + Running Keystone under HTTPD in this configuration does not support the use + of ``Transfer-Encoding: chunked``. This is due to a limitation with the + WSGI spec and the implementation used by ``mod_wsgi``. It is recommended + that all clients assume Keystone will not support + ``Transfer-Encoding: chunked``. Copy the ``httpd/wsgi-keystone.conf`` sample configuration file to the appropriate location for your Apache server, on Debian/Ubuntu systems @@ -55,7 +111,7 @@ Enable the site by creating a symlink from the file in ``sites-available`` to ``sites-enabled``, for example, on Debian/Ubuntu systems (not required on Red Hat based systems):: - ln -s /etc/apache2/sites-available/keystone.conf /etc/apache2/sites-enabled/ + ln -s /etc/apache2/sites-available/wsgi-keystone.conf /etc/apache2/sites-enabled/ Restart Apache to have it start serving keystone. diff --git a/keystone-moon/doc/source/api_curl_examples.rst b/keystone-moon/doc/source/api_curl_examples.rst index c88c7fd0..066efe97 100644 --- a/keystone-moon/doc/source/api_curl_examples.rst +++ b/keystone-moon/doc/source/api_curl_examples.rst @@ -28,7 +28,7 @@ Tokens Default scope ------------- -Get an token with default scope (may be unscoped): +Get a token with default scope (may be unscoped): .. code-block:: bash diff --git a/keystone-moon/doc/source/architecture.rst b/keystone-moon/doc/source/architecture.rst index c119e2bd..773aa6d4 100644 --- a/keystone-moon/doc/source/architecture.rst +++ b/keystone-moon/doc/source/architecture.rst @@ -165,10 +165,18 @@ The corresponding drivers for the services are: * :mod:`keystone.assignment.core.AssignmentDriverV8` * :mod:`keystone.assignment.core.RoleDriverV8` * :mod:`keystone.catalog.core.CatalogDriverV8` +* :mod:`keystone.credential.core.CredentialDriverV8` +* :mod:`keystone.endpoint_policy.core.EndpointPolicyDriverV8` +* :mod:`keystone.federation.core.FederationDriverV8` * :mod:`keystone.identity.core.IdentityDriverV8` +* :mod:`keystone.identity.core.MappingDriverV8` +* :mod:`keystone.oauth1.core.Oauth1DriverV8` * :mod:`keystone.policy.core.PolicyDriverV8` +* :mod:`keystone.resource.core.DomainConfigDriverV8` * :mod:`keystone.resource.core.ResourceDriverV8` +* :mod:`keystone.revoke.core.RevokeDriverV8` * :mod:`keystone.token.core.TokenDriverV8` +* :mod:`keystone.trust.core.TrustDriverV8` If you implement a backend driver for one of the Keystone services, you're expected to subclass from these classes. diff --git a/keystone-moon/doc/source/auth-totp.rst b/keystone-moon/doc/source/auth-totp.rst new file mode 100644 index 00000000..4e81757f --- /dev/null +++ b/keystone-moon/doc/source/auth-totp.rst @@ -0,0 +1,136 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +=================================== +Time-based One-time Password (TOTP) +=================================== + +Configuring TOTP +================ + +TOTP is not enabled in Keystone by default. To enable it add the ``totp`` +authentication method to the ``[auth]`` section in ``keystone.conf``: + +.. code-block:: ini + + [auth] + methods = external,password,token,oauth1,totp + +For a user to have access to TOTP, he must have configured TOTP credentials in +Keystone and a TOTP device (i.e. `Google Authenticator`_). + +.. _Google Authenticator: http://www.google.com/2step + +TOTP uses a base32 encoded string for the secret. The secret must be at least +148 bits (16 bytes). The following python code can be used to generate a TOTP +secret: + +.. code-block:: python + + import base64 + message = '1234567890123456' + print base64.b32encode(message).rstrip('=') + +Example output:: + + GEZDGNBVGY3TQOJQGEZDGNBVGY + +This generated secret can then be used to add new 'totp' credentials to a +specific user. + +Create a TOTP credential +------------------------ + +Create ``totp`` credentials for user: + +.. code-block:: bash + + USER_ID=b7793000f8d84c79af4e215e9da78654 + SECRET=GEZDGNBVGY3TQOJQGEZDGNBVGY + + curl -i \ + -H "Content-Type: application/json" \ + -d ' + { + "credential": { + "blob": "'$SECRET'", + "type": "totp", + "user_id": "'$USER_ID'" + } + }' \ + http://localhost:5000/v3/credentials ; echo + +Google Authenticator +-------------------- + +On a device install Google Authenticator and inside the app click on 'Set up +account' and then click on 'Enter provided key'. In the input fields enter +account name and secret. Optionally a QR code can be generated programatically +to avoid having to type the information. + +QR code +------- + +Create TOTP QR code for device: + +.. code-block:: python + + import qrcode + + secret='GEZDGNBVGY3TQOJQGEZDGNBVGY' + uri = 'otpauth://totp/{name}?secret={secret}&issuer={issuer}'.format( + name='name', + secret=secret, + issuer='Keystone') + + img = qrcode.make(uri) + img.save('totp.png') + +In Google Authenticator app click on 'Set up account' and then click on 'Scan +a barcode', and then scan the 'totp.png' image. This should create a new TOTP +entry in the application. + +Authenticate with TOTP +====================== + +Google Authenticator will generate a 6 digit PIN (passcode) every few seconds. +Use the passcode and your user ID to authenticate using the ``totp`` method. + +Tokens +------ + +Get a token with default scope (may be unscoped) using totp: + +.. code-block:: bash + + USER_ID=b7793000f8d84c79af4e215e9da78654 + PASSCODE=012345 + + curl -i \ + -H "Content-Type: application/json" \ + -d ' + { "auth": { + "identity": { + "methods": [ + "totp" + ], + "totp": { + "user": { + "id": "'$USER_ID'", + "passcode": "'$PASSCODE'" + } + } + } + } + }' \ + http://localhost:5000/v3/auth/tokens ; echo diff --git a/keystone-moon/doc/source/community.rst b/keystone-moon/doc/source/community.rst index dfb0870f..f3296efb 100644 --- a/keystone-moon/doc/source/community.rst +++ b/keystone-moon/doc/source/community.rst @@ -29,7 +29,6 @@ Our community welcomes all people interested in open source cloud computing, and there are no formal membership requirements. The best way to join the community is to talk with others online or at a meetup and offer contributions through Launchpad_, the wiki_, or blogs. We welcome all types of contributions, - from blueprint designs to documentation to testing to deployment scripts. .. _Launchpad: https://launchpad.net/keystone @@ -96,6 +95,5 @@ Twitter ------- Because all the cool kids do it: `@openstack <http://twitter.com/openstack>`_. -Also follow the `#openstack <http://search.twitter.com/search?q=%23openstack>`_ +Also follow the `#openstack <https://twitter.com/search?q=%23openstack>`_ tag for relevant tweets. - diff --git a/keystone-moon/doc/source/conf.py b/keystone-moon/doc/source/conf.py index 7cca2857..1037c39e 100644 --- a/keystone-moon/doc/source/conf.py +++ b/keystone-moon/doc/source/conf.py @@ -24,8 +24,7 @@ # All configuration values have a default; values that are commented out # serve to show the default. -import os - +import subprocess # NOTE(dstanek): adds _ to the builtins so keystone modules can be imported __builtins__['_'] = str @@ -42,9 +41,13 @@ extensions = ['sphinx.ext.autodoc', 'sphinx.ext.todo', 'sphinx.ext.coverage', 'sphinx.ext.viewcode', + 'oslo_config.sphinxconfiggen', 'oslosphinx', ] +config_generator_config_file = '../../config-generator/keystone.conf' +sample_config_basename = '_static/keystone' + todo_include_todos = True # Add any paths that contain templates here, relative to this directory. @@ -148,13 +151,15 @@ man_pages = [ # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". -#html_static_path = ['images'] +html_static_path = ['_static'] # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' -git_cmd = "git log --pretty=format:'%ad, commit %h' --date=local -n1" -html_last_updated_fmt = os.popen(git_cmd).read() +git_cmd = ["git", "log", "--pretty=format:'%ad, commit %h'", "--date=local", + "-n1"] +html_last_updated_fmt = subprocess.Popen( + git_cmd, stdout=subprocess.PIPE).communicate()[0] # If true, SmartyPants will be used to convert quotes and dashes to # typographically correct entities. diff --git a/keystone-moon/doc/source/configuration.rst b/keystone-moon/doc/source/configuration.rst index 574b26be..e78c0ac6 100644 --- a/keystone-moon/doc/source/configuration.rst +++ b/keystone-moon/doc/source/configuration.rst @@ -52,8 +52,8 @@ Starting and Stopping Keystone under Eventlet Running keystone under eventlet has been deprecated as of the Kilo release. Support for utilizing eventlet will be removed as of the M-release. The - recommended deployment is to run keystone in a WSGI server - (e.g. ``mod_wsgi`` under ``HTTPD``). + recommended deployment is to run keystone in a WSGI server such as Apache + httpd with ``mod_wsgi``. Keystone can be run using either its built-in eventlet server or it can be run embedded in a web server. While the eventlet server is convenient and easy to @@ -73,7 +73,7 @@ services are configured to run in a single process. .. NOTE:: - The separation into ``admin`` and ``main`` interfaces is an historical + The separation into ``admin`` and ``main`` interfaces is a historical anomaly. The new V3 API provides the same interface on both the admin and main interfaces (this can be configured in ``keystone-paste.ini``, but the default is to have both the same). The V2.0 API provides a limited public @@ -113,8 +113,8 @@ The primary configuration file is organized into the following sections: * ``[cache]`` - Caching layer configuration * ``[catalog]`` - Service catalog driver configuration * ``[credential]`` - Credential system driver configuration -* ``[endpoint_filter]`` - Endpoint filtering extension configuration -* ``[endpoint_policy]`` - Endpoint policy extension configuration +* ``[endpoint_filter]`` - Endpoint filtering configuration +* ``[endpoint_policy]`` - Endpoint policy configuration * ``[eventlet_server]`` - Eventlet server configuration * ``[eventlet_server_ssl]`` - Eventlet server SSL configuration * ``[federation]`` - Federation driver configuration @@ -124,7 +124,7 @@ The primary configuration file is organized into the following sections: * ``[ldap]`` - LDAP configuration options * ``[memcache]`` - Memcache configuration options * ``[oauth1]`` - OAuth 1.0a system driver configuration -* ``[os_inherit]`` - Inherited role assignment extension +* ``[os_inherit]`` - Inherited role assignment configuration * ``[paste_deploy]`` - Pointer to the PasteDeploy configuration file * ``[policy]`` - Policy system driver configuration for RBAC * ``[resource]`` - Resource system driver configuration @@ -134,7 +134,7 @@ The primary configuration file is organized into the following sections: * ``[signing]`` - Cryptographic signatures for PKI based tokens * ``[ssl]`` - SSL certificate generation configuration * ``[token]`` - Token driver & token provider configuration -* ``[trust]`` - Trust extension configuration +* ``[trust]`` - Trust configuration The Keystone primary configuration file is expected to be named ``keystone.conf``. When starting Keystone, you can specify a different @@ -423,7 +423,7 @@ Token Persistence Driver Keystone supports customizable token persistence drivers. These can be specified in the ``[token]`` section of the configuration file. Keystone provides three non-test persistence backends. These can be set with the -``[token]\driver`` configuration option. +``[token] driver`` configuration option. The drivers Keystone provides are: @@ -438,7 +438,7 @@ The drivers Keystone provides are: * ``memcache`` - The memcached based token persistence backend. This backend relies on ``dogpile.cache`` and stores the token data in a set of memcached - servers. The servers URLs are specified in the ``[memcache]\servers`` + servers. The servers URLs are specified in the ``[memcache] servers`` configuration option in the Keystone config. Implemented by :class:`keystone.token.persistence.backends.memcache.Token` @@ -446,9 +446,10 @@ The drivers Keystone provides are: .. WARNING:: It is recommended you use the ``memcache_pool`` backend instead of ``memcache`` as the token persistence driver if you are deploying Keystone - under eventlet instead of Apache + mod_wsgi. This recommendation is due to - known issues with the use of ``thread.local`` under eventlet that can allow - the leaking of memcache client objects and consumption of extra sockets. + under eventlet instead of Apache httpd with ``mod_wsgi``. This + recommendation is due to known issues with the use of ``thread.local`` + under eventlet that can allow the leaking of memcache client objects and + consumption of extra sockets. Token Provider @@ -539,7 +540,8 @@ disabled. backend will need to be specified. Current functional backends are: * ``dogpile.cache.memcached`` - Memcached backend using the standard - `python-memcached`_ library + `python-memcached`_ library (recommended for use with Apache httpd with + ``mod_wsgi``) * ``dogpile.cache.pylibmc`` - Memcached backend using the `pylibmc`_ library * ``dogpile.cache.bmemcached`` - Memcached using `python-binary-memcached`_ @@ -548,7 +550,7 @@ disabled. * ``dogpile.cache.dbm`` - local DBM file backend * ``dogpile.cache.memory`` - in-memory cache * ``keystone.cache.mongo`` - MongoDB as caching backend - * ``keystone.cache.memcache_pool`` - An eventlet safe implementation of + * ``keystone.cache.memcache_pool`` - An eventlet-safe implementation of ``dogpile.cache.memcached``. This implementation also provides client connection re-use. @@ -651,7 +653,6 @@ For more information about the different backends (and configuration options): .. _`dogpile.cache.backends.redis`: http://dogpilecache.readthedocs.org/en/latest/api.html#redis-backends .. _`dogpile.cache.backends.file`: http://dogpilecache.readthedocs.org/en/latest/api.html#file-backends .. _`ProxyBackends`: http://dogpilecache.readthedocs.org/en/latest/api.html#proxy-backends -.. _`PyMongo API`: http://api.mongodb.org/python/current/api/pymongo/index.html Certificates for PKI @@ -897,17 +898,46 @@ Another such example is `available in devstack (files/default_catalog.templates) <https://git.openstack.org/cgit/openstack-dev/devstack/tree/files/default_catalog.templates>`_. +Endpoint Filtering enables creation of ad-hoc catalogs for each project-scoped +token request. + +Configure the endpoint filter catalog driver in the ``[catalog]`` section. +For example: + +.. code-block:: ini + + [catalog] + driver = catalog_sql + +In the ``[endpoint_filter]`` section, set ``return_all_endpoints_if_no_filter`` +to ``False`` to return an empty catalog if no associations are made. +For example: + +.. code-block:: ini + + [endpoint_filter] + return_all_endpoints_if_no_filter = False + +See `API Specification for Endpoint Filtering <http://specs.openstack.org/ +openstack/keystone-specs/api/v3/identity-api-v3-os-ep-filter-ext.html>`_ for +the details of API definition. + +.. NOTE:: Support status for Endpoint Filtering + + *Experimental* (Icehouse, Juno) + *Stable* (Kilo) + Logging ------- Logging is configured externally to the rest of Keystone. Configure the path to -your logging configuration file using the ``[DEFAULT] log_config`` option of -``keystone.conf``. If you wish to route all your logging through syslog, set -the ``[DEFAULT] use_syslog`` option. +your logging configuration file using the ``[DEFAULT] log_config_append`` +option of ``keystone.conf``. If you wish to route all your logging through +syslog, set the ``[DEFAULT] use_syslog`` option. -A sample ``log_config`` file is included with the project at +A sample ``log_config_append`` file is included with the project at ``etc/logging.conf.sample``. Like other OpenStack projects, Keystone uses the -`Python logging module`, which includes extensive configuration options for +`Python logging module`_, which includes extensive configuration options for choosing the output levels and formats. .. _Paste: http://pythonpaste.org/ @@ -926,7 +956,7 @@ this section. Here is the description of each of them and their purpose: The SSL configuration options available to the eventlet server (``keystone-all``) described here are severely limited. A secure deployment should have Keystone running in a web server (such as Apache - HTTPd), or behind an SSL terminator. When running Keystone in a web server + httpd), or behind an SSL terminator. When running Keystone in a web server or behind an SSL terminator the options described in this section have no effect and SSL is configured in the web server or SSL terminator. @@ -1005,28 +1035,11 @@ and is only recommended for developments environment. We do not recommend using ``ssl_setup`` for production environments. -User CRUD extension for the V2.0 API +User CRUD additions for the V2.0 API ------------------------------------ -.. NOTE:: - - The core V3 API includes user operations so no extension needs to be - enabled for the V3 API. - -For the V2.0 API, Keystone provides a user CRUD filter that can be added to the -public_api pipeline. This user crud filter allows users to use a HTTP PATCH to -change their own password. To enable this extension you should define a -user_crud_extension filter, insert it after the ``*_body`` middleware and -before the ``public_service`` app in the public_api WSGI pipeline in -``keystone-paste.ini`` e.g.: - -.. code-block:: ini - - [filter:user_crud_extension] - paste.filter_factory = keystone.contrib.user_crud:CrudExtension.factory - - [pipeline:public_api] - pipeline = url_normalize token_auth admin_token_auth json_body debug ec2_extension user_crud_extension public_service +For the V2.0 API, Keystone provides an additional capability that allows users +to use a HTTP PATCH to change their own password. Each user can then change their own password with a HTTP PATCH : @@ -1039,19 +1052,112 @@ In addition to changing their password all of the user's current tokens will be revoked. -Inherited Role Assignment Extension ------------------------------------ +Inherited Role Assignments +-------------------------- -Keystone provides an optional extension that adds the capability to assign -roles on a project or domain that, rather than affect the project or domain -itself, are instead inherited to the project subtree or to all projects owned -by that domain. This extension is disabled by default, but can be enabled by -including the following in ``keystone.conf``: +Keystone provides an optional capability to assign roles on a project or domain +that, rather than affect the project or domain itself, are instead inherited to +the project subtree or to all projects owned by that domain. This capability is +enabled by default, but can be disabled by including the following in +``keystone.conf``: .. code-block:: ini [os_inherit] - enabled = True + enabled = False + + +Endpoint Policy +--------------- + +The Endpoint Policy feature provides associations between service endpoints +and policies that are already stored in the Identity server and referenced +by a policy ID. + +Configure the endpoint policy backend driver in the ``[endpoint_policy]`` +section. For example: + +.. code-block:: ini + + [endpoint_policy] + driver = sql + +See `API Specification for Endpoint Policy <http://specs.openstack.org/ +openstack/keystone-specs/api/v3/identity-api-v3-os-endpoint-policy.html>`_ +for the details of API definition. + +.. NOTE:: Support status for Endpoint Policy + + *Experimental* (Juno) + *Stable* (Kilo) + + +OAuth1 1.0a +----------- + +The OAuth 1.0a feature provides the ability for Identity users to delegate +roles to third party consumers via the OAuth 1.0a specification. + +To enable OAuth1: + +1. Add the oauth1 driver to the ``[oauth1]`` section in ``keystone.conf``. For + example: + +.. code-block:: ini + + [oauth1] + driver = sql + +2. Add the ``oauth1`` authentication method to the ``[auth]`` section in + ``keystone.conf``: + +.. code-block:: ini + + [auth] + methods = external,password,token,oauth1 + +3. If deploying under Apache httpd with ``mod_wsgi``, set the + `WSGIPassAuthorization` to allow the OAuth Authorization headers to pass + through `mod_wsgi`. For example, add the following to the keystone virtual + host file: + +.. code-block:: ini + + WSGIPassAuthorization On + +See `API Specification for OAuth 1.0a <http://specs.openstack.org/openstack/ +keystone-specs/api/v3/identity-api-v3-os-oauth1-ext.html>`_ for the details of +API definition. + +.. NOTE:: Support status for OAuth 1.0a + + *Experimental* (Havana, Icehouse) + *Stable* (Juno) + + +Revocation Events +----------------- + +The Revocation Events feature provides a list of token revocations. Each event +expresses a set of criteria which describes a set of tokens that are +no longer valid. + +Add the revoke backend driver to the ``[revoke]`` section in +``keystone.conf``. For example: + +.. code-block:: ini + + [revoke] + driver = sql + +See `API Specification for Revocation Events <https://specs.openstack.org/ +openstack/keystone-specs/api/v3/identity-api-v3-os-revoke-ext.html>`_ for +the details of API definition. + +.. NOTE:: Support status for Revocation Events + + *Experimental* (Juno) + *Stable* (Kilo) Token Binding @@ -1114,6 +1220,39 @@ If a response to ``list_{entity}`` call has been truncated, then the response status code will still be 200 (OK), but the ``truncated`` attribute in the collection will be set to ``true``. + +URL safe naming of projects and domains +--------------------------------------- + +In the future, keystone may offer the ability to identify a project in a +hierarchy via a URL style of naming from the root of the hierarchy (for example +specifying 'projectA/projectB/projectC' as the project name in an +authentication request). In order to prepare for this, keystone supports the +optional ability to ensure both projects and domains are named without +including any of the reserverd characters specified in section 2.2 of +`rfc3986 <http://tools.ietf.org/html/rfc3986>`_. + +The safety of the names of projects and domains can be controlled via two +configuration options: + +.. code-block:: ini + + [resource] + project_name_url_safe = off + domain_name_url_safe = off + +When set to ``off`` (which is the default), no checking is done on the URL +safeness of names. When set to ``new``, an attempt to create a new project or +domain with an unsafe name (or update the name of a project or domain to be +unsafe) will cause a status code of 400 (Bad Request) to be returned. Setting +the configuration option to ``strict`` will, in addition to preventing the +creation and updating of entities with unsafe names, cause an authentication +attempt which specifies a project or domain name that is unsafe to return a +status code of 401 (Unauthorized). + +It is recommended that installations take the steps necessary to where they +can run with both options set to ``strict`` as soon as is practical. + Sample Configuration Files -------------------------- @@ -1124,6 +1263,7 @@ files for each Server application. * ``etc/keystone-paste.ini`` * ``etc/logging.conf.sample`` * ``etc/default_catalog.templates`` +* ``etc/sso_callback_template.html`` .. _`API protection with RBAC`: @@ -1189,6 +1329,7 @@ are filtered out (e.g. user passwords). List of object attributes: * role: + * target.role.domain_id * target.role.id * target.role.name @@ -1441,14 +1582,6 @@ as follows: $ openstack --os-username=admin --os-password=secrete --os-project-name=admin --os-auth-url=http://localhost:35357/v2.0 user list $ openstack --os-username=admin --os-password=secrete --os-project-name=admin --os-auth-url=http://localhost:35357/v2.0 project create demo -For additional examples using ``python-keystoneclient`` refer to -`python-keystoneclient examples`_, likewise, for additional examples using -``python-openstackclient``, refer to `python-openstackclient examples`_. - -.. _`python-keystoneclient examples`: cli_examples.html#using-python-keystoneclient-v2-0 -.. _`python-openstackclient examples`: cli_examples.html#using-python-openstackclient-v3 - - Removing Expired Tokens ======================= @@ -1506,12 +1639,6 @@ The corresponding entries in the Keystone configuration file are: user_tree_dn = ou=Users,dc=openstack,dc=org user_objectclass = inetOrgPerson - project_tree_dn = ou=Projects,dc=openstack,dc=org - project_objectclass = groupOfNames - - role_tree_dn = ou=Roles,dc=openstack,dc=org - role_objectclass = organizationalRole - The default object classes and attributes are intentionally simplistic. They reflect the common standard objects according to the LDAP RFCs. However, in a live deployment, the correct attributes can be overridden to support a @@ -1539,14 +1666,6 @@ and you have only read access, in such case the configuration is: user_allow_update = False user_allow_delete = False - project_allow_create = True - project_allow_update = True - project_allow_delete = True - - role_allow_create = True - role_allow_update = True - role_allow_delete = True - There are some configuration options for filtering users, tenants and roles, if the backend is providing too much output, in such case the configuration will look like: @@ -1555,8 +1674,6 @@ look like: [ldap] user_filter = (memberof=CN=openstack-users,OU=workgroups,DC=openstack,DC=org) - project_filter = - role_filter = In case that the directory server does not have an attribute enabled of type boolean for the user, there is several configuration parameters that can be @@ -1588,26 +1705,15 @@ specified classes in the LDAP module so you can configure them like: .. code-block:: ini [ldap] - user_objectclass = person - user_id_attribute = cn - user_name_attribute = cn - user_mail_attribute = mail - user_enabled_attribute = userAccountControl - user_enabled_mask = 2 - user_enabled_default = 512 - user_attribute_ignore = tenant_id,tenants - project_objectclass = groupOfNames - project_id_attribute = cn - project_member_attribute = member - project_name_attribute = ou - project_desc_attribute = description - project_enabled_attribute = extensionName - project_attribute_ignore = - role_objectclass = organizationalRole - role_id_attribute = cn - role_name_attribute = ou - role_member_attribute = roleOccupant - role_attribute_ignore = + user_objectclass = person + user_id_attribute = cn + user_name_attribute = cn + user_description_attribute = displayName + user_mail_attribute = mail + user_enabled_attribute = userAccountControl + user_enabled_mask = 2 + user_enabled_default = 512 + user_attribute_ignore = tenant_id,tenants Debugging LDAP -------------- @@ -1632,14 +1738,13 @@ Enabled Emulation ----------------- Some directory servers do not provide any enabled attribute. For these servers, -the ``user_enabled_emulation`` and ``project_enabled_emulation`` attributes -have been created. They are enabled by setting their respective flags to True. -Then the attributes ``user_enabled_emulation_dn`` and -``project_enabled_emulation_dn`` may be set to specify how the enabled users -and projects (tenants) are selected. These attributes work by using a -``groupOfNames`` entry and adding whichever users or projects (tenants) that -you want enabled to the respective group with the ``member`` attribute. For -example, this will mark any user who is a member of ``enabled_users`` as enabled: +the ``user_enabled_emulation`` attribute has been created. It is enabled by +setting the respective flags to True. Then the attribute +``user_enabled_emulation_dn`` may be set to specify how the enabled users are +selected. This attribute works by using a ``groupOfNames`` entry and adding +whichever users or that you want enabled to the respective group with the +``member`` attribute. For example, this will mark any user who is a member of +``enabled_users`` as enabled: .. code-block:: ini @@ -1647,15 +1752,14 @@ example, this will mark any user who is a member of ``enabled_users`` as enabled user_enabled_emulation = True user_enabled_emulation_dn = cn=enabled_users,cn=groups,dc=openstack,dc=org -The default values for user and project (tenant) enabled emulation DN is -``cn=enabled_users,$user_tree_dn`` and ``cn=enabled_tenants,$project_tree_dn`` -respectively. +The default values for user enabled emulation DN is +``cn=enabled_users,$user_tree_dn``. + If a different LDAP schema is used for group membership, it is possible to use the ``group_objectclass`` and ``group_member_attribute`` attributes to determine membership in the enabled emulation group by setting the -``user_enabled_emulation_use_group_config`` and -``project_enabled_emulation_use_group_config`` attributes to True. +``user_enabled_emulation_use_group_config`` attribute to True. Secure Connection ----------------- @@ -1760,7 +1864,7 @@ with the user's DN and provided password. This kind of authentication bind can fill up the pool pretty quickly, so a separate pool is provided for end user authentication bind calls. If a deployment does not want to use a pool for those binds, then it can disable pooling selectively by setting -``use_auth_pool`` to false. If a deployment wants to use a pool for those +``use_auth_pool`` to false. If a deployment wants to use a pool for those authentication binds, then ``use_auth_pool`` needs to be set to true. For the authentication pool, a different pool size (``auth_pool_size``) and connection lifetime (``auth_pool_connection_lifetime``) can be specified. With an enabled @@ -1805,3 +1909,16 @@ Connection pool configuration is part of the ``[ldap]`` configuration section: # End user auth connection lifetime in seconds. (integer value) auth_pool_connection_lifetime=60 +Specifying Multiple LDAP servers +-------------------------------- + +Multiple LDAP server URLs can be provided to keystone to provide +high-availability support for a single LDAP backend. To specify multiple LDAP +servers, simply change the ``url`` option in the ``[ldap]`` section. The new +option should list the different servers, each separated by a comma. For +example: + +.. code-block:: ini + + [ldap] + url = "ldap://localhost,ldap://backup.localhost" diff --git a/keystone-moon/doc/source/configure_federation.rst b/keystone-moon/doc/source/configure_federation.rst index 09d0984d..644d3175 100644 --- a/keystone-moon/doc/source/configure_federation.rst +++ b/keystone-moon/doc/source/configure_federation.rst @@ -90,14 +90,19 @@ configure ``federation``. Configure authentication drivers in ``keystone.conf`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. NOTE:: + ``saml2`` has been deprecated as of the Mitaka release. Support for the + ``saml2`` wrapper will be removed as of the "O" release. The recommended authentication method + is ``mapped``, which supports ``saml2``. + Add the authentication methods to the ``[auth]`` section in ``keystone.conf``. Names should be equal to protocol names added via Identity API v3. Here we use -examples ``saml2`` and ``openid``. +examples ``mapped`` and ``openid``. .. code-block:: bash [auth] - methods = external,password,token,saml2,openid + methods = external,password,token,mapped,openid Create keystone groups and assign roles ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -182,7 +187,7 @@ issue an HTTP POST request with authentication data in the request body. To start federated authentication a user must access the dedicated URL with Identity Provider's and Protocol's identifiers stored within a protected URL. The URL has a format of: -``/v3/OS-FEDERATION/identity_providers/{identity_provider}/protocols/{protocol}/auth``. +``/v3/OS-FEDERATION/identity_providers/{idp_id}/protocols/{protocol_id}/auth``. In this instance we follow a standard SAML2 authentication procedure, that is, the user will be redirected to the Identity Provider's authentication webpage @@ -207,7 +212,7 @@ SAML authentication procedure. .. code-block:: bash - $ curl -X GET -D - http://localhost:5000/v3/OS-FEDERATION/identity_providers/{identity_provider}/protocols/{protocol}/auth + $ curl -X GET -D - http://localhost:5000/v3/OS-FEDERATION/identity_providers/{idp_id}/protocols/{protocol_id}/auth Determine accessible resources ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -251,7 +256,7 @@ Example cURL .. code-block:: bash - $ curl -X POST -H "Content-Type: application/json" -d '{"auth":{"identity":{"methods":["saml2"],"saml2":{"id":"<unscoped_token_id>"}},"scope":{"project":{"domain": {"name": "Default"},"name":"service"}}}}' -D - http://localhost:5000/v3/auth/tokens + $ curl -X POST -H "Content-Type: application/json" -d '{"auth":{"identity":{"methods":["mapped"],"saml2":{"id":"<unscoped_token_id>"}},"scope":{"project":{"domain": {"name": "Default"},"name":"service"}}}}' -D - http://localhost:5000/v3/auth/tokens -------------------------------------- Keystone as an Identity Provider (IdP) @@ -332,7 +337,7 @@ Create a Service Provider (SP) ------------------------------ In this example we are creating a new Service Provider with an ID of ``BETA``, -a ``sp_url`` of ``http://beta.example.com/Shibboleth.sso/POST/ECP`` and a +a ``sp_url`` of ``http://beta.example.com/Shibboleth.sso/SAML2/ECP`` and a ``auth_url`` of ``http://beta.example.com:5000/v3/OS-FEDERATION/identity_providers/beta/protocols/saml2/auth`` . The ``sp_url`` will be used when creating a SAML assertion for ``BETA`` and signed by the current keystone IdP. The ``auth_url`` is used to retrieve the @@ -345,8 +350,8 @@ field is optional we are passing it set to ``true`` otherwise it will be set to $ curl -s -X PUT \ -H "X-Auth-Token: $OS_TOKEN" \ -H "Content-Type: application/json" \ - -d '{"service_provider": {"auth_url": "http://beta.example.com:5000/v3/OS-FEDERATION/identity_providers/beta/protocols/saml2/auth", "sp_url": "https://example.com:5000/Shibboleth.sso/SAML2/ECP", "enabled": true}' \ - http://localhost:5000/v3/service_providers/BETA | python -mjson.tool + -d '{"service_provider": {"auth_url": "http://beta.example.com:5000/v3/OS-FEDERATION/identity_providers/beta/protocols/saml2/auth", "sp_url": "https://example.com:5000/Shibboleth.sso/SAML2/ECP", "enabled": true}}' \ + http://localhost:5000/v3/OS-FEDERATION/service_providers/BETA | python -mjson.tool Testing it all out ------------------ diff --git a/keystone-moon/doc/source/configure_tokenless_x509.rst b/keystone-moon/doc/source/configure_tokenless_x509.rst index 40b9fd20..614b1179 100644 --- a/keystone-moon/doc/source/configure_tokenless_x509.rst +++ b/keystone-moon/doc/source/configure_tokenless_x509.rst @@ -87,7 +87,7 @@ The following options can be defined in `keystone.conf`: * ``issuer_attribute`` - The issuer attribute that is served as an IdP ID for the X.509 tokenless authorization along with the protocol to look up its corresponding mapping. It is the environment variable in the WSGI - enviornment that references to the Issuer of the client certificate. It + environment that references to the Issuer of the client certificate. It defaults to ``SSL_CLIENT_I_DN``. This is a sample configuration for two `trusted_issuer` and a `protocol` set @@ -127,7 +127,7 @@ like in a certificate. C=US The issuer DN should be constructed as a string that contains no spaces -and have the right order seperated by commas like the example below. +and have the right order separated by commas like the example below. Please be aware that ``emailAddress`` and ``ST`` should be used instead of ``E`` and ``S`` that are shown in the above example. The following is the sample Python code used to create the IdP ID. diff --git a/keystone-moon/doc/source/configuringservices.rst b/keystone-moon/doc/source/configuringservices.rst index 3ffa13e7..40fe03a2 100644 --- a/keystone-moon/doc/source/configuringservices.rst +++ b/keystone-moon/doc/source/configuringservices.rst @@ -38,8 +38,80 @@ The middleware will pass those data down to the service as headers. More details on the architecture of that setup is described in the `authentication middleware documentation`_. -Setting up credentials -====================== +Setting up credentials with ``keystone-manage bootstrap`` +========================================================= + +Setting up projects, users, and roles +------------------------------------- + +The ``keystone-manage bootstrap`` command will create a user, project and role, +and will assign the newly created role to the newly created user on the newly +created project. By default, the names of these new resources will be called +``admin``. + +The defaults may be overridden by calling ``--bootstrap-username``, +``--bootstrap-project-name`` and ``--bootstrap-role-name``. Each of these have +an environment variable equivalent: ``OS_BOOTSTRAP_USERNAME``, +``OS_BOOTSTRAP_PROJECT_NAME`` and ``OS_BOOTSTRAP_ROLE_NAME``. + +A user password must also be supplied. This can be passed in as either +``--bootstrap-password``, or set as an environment variable using +``OS_BOOTSTRAP_PASSWORD``. + +Optionally, if specified by ``--bootstrap-public-url``, +``--bootstrap-admin-url`` and/or ``--bootstrap-internal-url`` or the equivalent +environment variables, the command will create an identity service with the +specified endpoint information. You may also configure the +``--bootstrap-region-id`` and ``--bootstrap-service-name`` for the endpoints to +your deployment's requirements. + +.. NOTE:: + + It is strongly encouraged to configure the identity service and its + endpoints while bootstrapping keystone. + +Minimally, keystone can be bootstrapped with: + +.. code-block:: bash + + $ keystone-manage bootstrap --bootstrap-password s3cr3t + +Verbosely, keystone can be bootstrapped with: + +.. code-block:: bash + + $ keystone-manage bootstrap --bootstrap-password s3cr3t + --bootstrap-username admin \ + --bootstrap-project-name admin \ + --bootstrap-role-name admin \ + --bootstrap-service-name keystone \ + --bootstrap-region-id RegionOne \ + --bootstrap-admin-url http://localhost:35357 \ + --bootstrap-public-url http://localhost:5000 \ + --bootstrap-internal-url http://localhost:5000 + +This will create an ``admin`` user with the ``admin`` role on the ``admin`` +project. The user will have the password specified in the command. Note that +both the user and the project will be created in the ``default`` domain. By not +creating an endpoint in the catalog users will need to provide endpoint +overrides to perform additional identity operations. + +By creating an ``admin`` user and an identity endpoint deployers may +authenticate to keystone and perform identity operations like creating +additional services and endpoints using that ``admin`` user. This will preclude +the need to ever use or configure the ``admin_token`` (described below). + +To test a proper configuration, a user can use OpenStackClient CLI: + +.. code-block:: bash + + $ openstack project list --os-username admin --os-project-name admin \ + --os-user-domain-id default --os-project-domain-id default \ + --os-identity-api-version 3 --os-auth-url http://localhost:5000 \ + --os-password s3cr3t + +Setting up credentials with Admin Token +======================================= Admin Token ----------- diff --git a/keystone-moon/doc/source/developing.rst b/keystone-moon/doc/source/developing.rst index 50fed9e5..d49d1e14 100644 --- a/keystone-moon/doc/source/developing.rst +++ b/keystone-moon/doc/source/developing.rst @@ -21,10 +21,9 @@ Developing with Keystone Setup ----- -Get your development environment set up according to :doc:`setup`. The -instructions from here will assume that you have installed Keystone into a -virtualenv. If you chose not to, simply exclude "tools/with_venv.sh" from the -example commands below. +Get your development environment set up according to +:doc:`devref/development.environment`. It is recommended that you install +Keystone into a virtualenv. Configuring Keystone @@ -48,7 +47,7 @@ To run the Keystone Admin and API server instances, use: .. code-block:: bash - $ tools/with_venv.sh keystone-all + $ keystone-all This runs Keystone with the configuration the etc/ directory of the project. See :doc:`configuration` for details on how Keystone is configured. By default, @@ -72,7 +71,7 @@ place: .. code-block:: bash - $ bin/keystone-manage db_sync + $ keystone-manage db_sync .. _`python-keystoneclient`: https://git.openstack.org/cgit/openstack/python-keystoneclient .. _`openstackclient`: https://git.openstack.org/cgit/openstack/python-openstackclient @@ -100,15 +99,7 @@ the script, the version is the number before the underline. For example, if the script is named ``001_add_X_table.py`` then the version of the SQL migration is ``1``. -.. _SQLAlchemy-migrate: http://code.google.com/p/sqlalchemy-migrate/ - -Extensions should be created as directories under ``keystone/contrib``. An -extension that requires SQL migrations should not change the common repository, -but should instead have its own repository. This repository must be in the -extension's directory in ``keystone/contrib/<extension>/migrate_repo``. In -addition, it needs a subdirectory named ``versions``. For example, if the -extension name is ``my_extension`` then the directory structure would be -``keystone/contrib/my_extension/migrate_repo/versions/``. +.. _SQLAlchemy-migrate: https://git.openstack.org/cgit/openstack/sqlalchemy-migrate For the migration to work, both the ``migrate_repo`` and ``versions`` subdirectories must have ``__init__.py`` files. SQLAlchemy-migrate will look @@ -121,10 +112,7 @@ the minimal set of values is:: version_table=migrate_version required_dbs=[] -The directory ``keystone/contrib/example`` contains a sample extension -migration. - -For core components, to run a migration for upgrade, simply run: +To run a migration for upgrade, simply run: .. code-block:: bash @@ -134,22 +122,73 @@ For core components, to run a migration for upgrade, simply run: If no version is specified, then the most recent migration will be used. -For extensions, migrations must be explicitly run for each extension individually. -To run a migration for a specific extension, simply run: +.. NOTE:: -.. code-block:: bash + Schema downgrades are not supported. - $ keystone-manage db_sync --extension <name> +.. _online-migration: -.. NOTE:: +From Mitaka release, we are starting to write the migration scripts in a +backward compatible way to support `online schema migration`_. The following +guidelines for schema and data migrations should be followed: - The meaning of "extension" here has been changed since all of the - "extension" are loaded and the migrations are run by default, but - the source is maintained in a separate directory. +* Additive schema migrations - In general, almost all schema migrations should + be additive. Put simply, they should only create elements like columns, + indices, and tables. -.. NOTE:: +* Subtractive schema migrations - To remove an element like a column or table: + + #. Expand phase: The element must be deprecated and retained for backward + compatibility. This allows for graceful upgrade from X release to X+1. + + #. Migrate phase: Data migration must completely migrate data from the old + version of the schema to the new version. Data migrations should have the + ability to run online, while the service is operating normally, so the + keystone service implementation (typically the SQLAlchemy model) has to + be aware that data should be retrieved and/or written from/to more than + one place and format, to maintain consistency (see examples below). + + #. Contract phase: The column can then be removed with a schema migration at + the start of X+2. Contract phase can't happen if the data migration isn't + finished (see last point in this section). + +* Release notes - There should be a release note in case an operation is + "blocking", "expensive", or both. You can find information on which DDL + operations are expensive in `MySQL docs`_. Other supported SQL DBs support + `transactional DDL`_, and experienced DBA's know to take advantage of this + feature. + +* Constraints - When adding a foreign or unique key constraint, the schema + migration code needs to handle possible problems with data before applying + the constraint. For example, a unique constraint must clean up duplicate + records before applying said constraint. + +* Data migrations - should be done in an online fashion by custom code in the + SQLAlchemy layer that handles moving data between the old and new portions + of the schema. In addition, for each type of data migration performed, + a keystone-manage command can be added for the operator to manually request + that rows be migrated (see examples below, like the nova flavor migration). + +* All schema migrations should be idempotent. For example, a migration + should check if an element exists in the schema before attempting to add + it. This logic comes for free in the autogenerated workflow of + the online migrations. + +* Before running `contract` in the expand/migrate/contract schema migration + workflow, the remaining data migrations should be performed by the contract + script. Alternatively, running a relevant keystone-manage migration should + be enforced, to ensure that all remaining data migrations are completed. + It is a good practice to move data out of the old columns, and ensure they + are filled with null values before removing them. + +A good example of an online schema migration is documented in a `cinder spec`_. +See more examples in :doc:`online_schema_migration_examples`. + +.. _`online schema migration`: https://specs.openstack.org/openstack/keystone-specs/specs/mitaka/online-schema-migration.html +.. _`MySQL docs`: https://dev.mysql.com/doc/refman/5.7/en/innodb-create-index-overview.html +.. _`transactional DDL`: https://wiki.postgresql.org/wiki/Transactional_DDL_in_PostgreSQL:_A_Competitive_Analysis +.. _`cinder spec`: https://specs.openstack.org/openstack/cinder-specs/specs/mitaka/online-schema-upgrades.html - Schema downgrades are not supported for both core components and extensions. Initial Sample Data ------------------- @@ -159,7 +198,7 @@ data for use with keystone: .. code-block:: bash - $ OS_TOKEN=ADMIN tools/with_venv.sh tools/sample_data.sh + $ OS_TOKEN=ADMIN tools/sample_data.sh Notice it requires a service token read from an environment variable for authentication. The default value "ADMIN" is from the ``admin_token`` @@ -170,13 +209,13 @@ Once run, you can see the sample data that has been created by using the .. code-block:: bash - $ tools/with_venv.sh openstack --os-token ADMIN --os-url http://127.0.0.1:35357/v2.0/ user list + $ openstack --os-token ADMIN --os-url http://127.0.0.1:35357/v2.0/ user list The `openstackclient`_ can be installed using the following: .. code-block:: bash - $ tools/with_venv.sh pip install python-openstackclient + $ pip install python-openstackclient Filtering responsibilities between controllers and drivers ---------------------------------------------------------- @@ -247,7 +286,8 @@ Running Tests ============= Before running tests, you should have ``tox`` installed and available in your -environment (in addition to the other external dependencies in :doc:`setup`): +environment (in addition to the other external dependencies in +:doc:`devref/development.environment`): .. code-block:: bash @@ -328,7 +368,7 @@ Not all of the tests in the keystone/tests/unit directory are strictly unit tests. Keystone intentionally includes tests that run the service locally and drives the entire configuration to achieve basic functional testing. -For the functional tests, an in-memory key-value store or in-memory sqlite +For the functional tests, an in-memory key-value store or in-memory SQLite database is used to keep the tests fast. Within the tests directory, the general structure of the backend tests is a @@ -819,3 +859,44 @@ The documentation is generated with Sphinx using the tox command. To create HTM $ tox -e docs The results are in the doc/build/html and doc/build/man directories respectively. + + +Release Notes +------------- + +The release notes for a patch should be included in the patch. If not, the +release notes should be in a follow-on review. + +If the following applies to the patch, a release note is required: + +* The deployer needs to take an action when upgrading +* The backend driver interface changes +* A new feature is implemented +* Function was removed (hopefully it was deprecated) +* Current behavior is changed +* A new config option is added that the deployer should consider changing from + the default +* A security bug is fixed + +A release note is suggested if a long-standing or important bug is fixed. +Otherwise, a release note is not required. + +Keystone uses `reno <http://docs.openstack.org/developer/reno/usage.html>`_ to +generate release notes. Please read the docs for details. In summary, use + +.. code-block:: bash + + $ tox -e venv -- reno new <bug-,bp-,whatever> + +Then edit the sample file that was created and push it with your change. + +To see the results: + +.. code-block:: bash + + $ git commit # Commit the change because reno scans git log. + + $ tox -e releasenotes + +Then look at the generated release notes files in releasenotes/build/html in +your favorite browser. diff --git a/keystone-moon/doc/source/developing_drivers.rst b/keystone-moon/doc/source/developing_drivers.rst index 1e3996de..38cd7319 100644 --- a/keystone-moon/doc/source/developing_drivers.rst +++ b/keystone-moon/doc/source/developing_drivers.rst @@ -25,7 +25,7 @@ Each major subsystem (that has data access needs) implements the data access by using drivers. Some examples of Keystone's drivers: - :class:`keystone.identity.backends.ldap.Identity` -- :class:`keystone.token.providers.fernet.Provider` +- :class:`keystone.token.providers.fernet.core.Provider` - :class:`keystone.contrib.federation.backends.sql.Federation` In/Out of Tree diff --git a/keystone-moon/doc/source/devref/development.environment.rst b/keystone-moon/doc/source/devref/development.environment.rst new file mode 100644 index 00000000..2718966a --- /dev/null +++ b/keystone-moon/doc/source/devref/development.environment.rst @@ -0,0 +1,175 @@ +.. + Copyright 2011-2012 OpenStack Foundation + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +============================================= +Setting up a Keystone development environment +============================================= + +This document describes getting the source from keystone's `Git repository`_ +for development purposes. + +To install Keystone from packaging, refer instead to Keystone's `User +Documentation`_. + +.. _`Git Repository`: http://git.openstack.org/cgit/openstack/keystone +.. _`User Documentation`: http://docs.openstack.org/ + +Prerequisites +============= + +This document assumes you are using Ubuntu, Fedora or openSUSE (SLE) + +And that you have the following tools available on your system: + +- Python_ 2.7 and 3.4 +- git_ +- setuptools_ +- pip_ +- msgfmt (part of the gettext package) +- virtualenv_ +- tox_ + +**Reminder**: If you're successfully using a different platform, or a +different version of the above, please document your configuration here! + +.. _Python: http://www.python.org/ +.. _git: http://git-scm.com/ +.. _setuptools: http://pypi.python.org/pypi/setuptools +.. _tox: https://pypi.python.org/pypi/tox + +Getting the latest code +======================= + +Make a clone of the code from our `Git repository`: + +.. code-block:: bash + + $ git clone https://git.openstack.org/openstack/keystone.git + +When that is complete, you can: + +.. code-block:: bash + + $ cd keystone + +Installing dependencies +======================= + +Keystone maintains two lists of dependencies:: + + requirements.txt + test-requirements.txt + +The first is the list of dependencies needed for running keystone, the second list includes dependencies used for active development and testing of Keystone itself. + +These dependencies can be installed from PyPi_ using the Python tool pip_. + +.. _PyPi: http://pypi.python.org/ +.. _pip: http://pypi.python.org/pypi/pip + +However, your system *may* need additional dependencies that `pip` (and by +extension, PyPi) cannot satisfy. These dependencies should be installed +prior to using `pip`, and the installation method may vary depending on +your platform. + +Ubuntu 14.04, 15.10: + +.. code-block:: bash + + $ sudo apt-get install python-dev python3-dev libxml2-dev libxslt1-dev \ + libsasl2-dev libsqlite3-dev libssl-dev libldap2-dev libffi-dev + + +Fedora 19+: + +.. code-block:: bash + + $ sudo yum install python-lxml python-greenlet-devel python-ldap \ + sqlite-devel openldap-devel python-devel libxslt-devel \ + openssl-devel libffi-devel + +openSUSE 13.2 (SLE 12): + +.. code-block:: bash + + $ sudo zypper install libxslt-devel openldap2-devel libopenssl-devel \ + python-devel python-greenlet-devel python-ldap python-lxml \ + python-pysqlite sqlite3-devel + +PyPi Packages and VirtualEnv +---------------------------- + +We recommend establishing a virtualenv to run Keystone within. virtualenv +limits the Python environment to just what you're installing as dependencies, +useful to keep a clean environment for working on Keystone. + +.. code-block:: bash + + $ tox -e venv --notest + +This will create a local virtual environment in the directory ``.tox``. +Once created, you can activate this virtualenv for your current shell using: + +.. code-block:: bash + + $ source .tox/venv/bin/activate + +The virtual environment can be disabled using the command: + +.. code-block:: bash + + $ deactivate + +You can also use ``tox -e venv`` to prefix commands so that they run +within the virtual environment. For more information on virtual environments, +see virtualenv_. + +.. _virtualenv: http://www.virtualenv.org/ + +If you want to run Keystone outside of a virtualenv, you can install the +dependencies directly into your system from the requirements files: + +.. code-block:: bash + + # Install the dependencies for running keystone + $ pip install -r requirements.txt + + # Install the dependencies for developing, testing, and running keystone + $ pip install -r test-requirements.txt + + # Use 'python setup.py' to link Keystone into Python's site-packages + $ python setup.py develop + + +Verifying Keystone is set up +============================ + +Once set up, either directly or within a virtualenv, you should be able to +invoke Python and import the libraries. If you're using a virtualenv, don't +forget to activate it: + +.. code-block:: bash + + $ source .tox/venv/bin/activate + +You should then be able to `import keystone` using Python without issue: + +.. code-block:: bash + + $ python -c "import keystone" + +If you can import Keystone without a traceback, you should be ready to move on +to :doc:`../developing`. diff --git a/keystone-moon/doc/source/event_notifications.rst b/keystone-moon/doc/source/event_notifications.rst index 740986b1..d9225c56 100644 --- a/keystone-moon/doc/source/event_notifications.rst +++ b/keystone-moon/doc/source/event_notifications.rst @@ -28,7 +28,7 @@ The supported operations between the two types of notification formats are documented below. Common Notification Structure -============================== +============================= Notifications generated by Keystone are generated in JSON format. An external application can format them into ATOM format and publish them as a feed. @@ -365,7 +365,7 @@ the unique identifier of the resource type. .. code-block:: javascript { - "event_type": "identity.created.role_assignment", + "event_type": "identity.role_assignment.created", "message_id": "a5901371-d5fd-b3bb-448f-a14dead6f4cb", "payload": { "typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event", @@ -414,3 +414,26 @@ ensures this has an immediate impact on the accessibility of the project's resources by revoking tokens with authorization on the project, but should **not** have a direct impact on the projects resources (in other words, virtual machines should **not** be deleted). + +Opting out of certain notifications +=================================== + +There are many notifications that Keystone emits and some deployers may only +care about certain events. In Keystone there is a way to opt-out of certain +notifications. In ``/etc/keystone/keystone.conf`` you can set ``opt_out`` to +the event you wish to opt-out of. It is possible to opt-out of multiple events. + +Example: + +.. code-block:: ini + + [DEFAULT] + notification_opt_out = identity.user.created + notification_opt_out = identity.role_assignment.created + notification_opt_out = identity.authenticate.pending + +This will opt-out notifications for user creation, role assignment creation and +successful authentications. For a list of event types that can be used, refer +to: `Telemetry Measurements`_. + +.. _Telemetry Measurements: http://docs.openstack.org/admin-guide-cloud/telemetry-measurements.html#openstack-identity diff --git a/keystone-moon/doc/source/extensions.rst b/keystone-moon/doc/source/extensions.rst index 0a6b34a2..4d171f05 100644 --- a/keystone-moon/doc/source/extensions.rst +++ b/keystone-moon/doc/source/extensions.rst @@ -43,99 +43,3 @@ Removal Process It is not intended that functionality should stay in experimental for a long period, functionality that stays `experimental` for more than **two** releases would be expected to make a transition to either `stable` or `out-of-tree`. - -Current Extensions -================== - ------------------- -Endpoint Filtering ------------------- - -The Endpoint Filtering extension enables creation of ad-hoc catalogs for each -project-scoped token request. - -.. NOTE:: Support status for Endpoint Filtering - - *Experimental* (Icehouse, Juno) - *Stable* (Kilo) - -.. toctree:: - :maxdepth: 1 - - extensions/endpoint_filter.rst - -* `API Specification for Endpoint Filtering <http://specs.openstack.org/openstack/keystone-specs/api/v3/identity-api-v3-os-ep-filter-ext.html>`__ - ---------------- -Endpoint Policy ---------------- - -The Endpoint Policy extension provides associations between service endpoints -and policies that are already stored in the Identity server and referenced by -a policy ID. - -.. NOTE:: Support status for Endpoint Policy - - *Experimental* (Juno) - *Stable* (Kilo) - -.. toctree:: - :maxdepth: 1 - - extensions/endpoint_policy.rst - -* `API Specification for Endpoint Policy <http://specs.openstack.org/openstack/keystone-specs/api/v3/identity-api-v3-os-endpoint-policy.html>`__ - -------- -Inherit -------- - -The Inherit extension provides the ability for projects to inherit role -assignments from their owning domain, or from projects higher in the -hierarchy. - -.. NOTE:: Support status for Inherit - - *Experimental* (Havava, Icehouse) - *Stable* (Juno) - -* `API Specification for Inherit <http://specs.openstack.org/openstack/keystone-specs/api/v3/identity-api-v3-os-inherit-ext.html>`__ - ----------- -OAuth 1.0a ----------- - -The OAuth 1.0a extension provides the ability for Identity users to delegate -roles to third party consumers via the OAuth 1.0a specification. - -.. NOTE:: Support status for OAuth 1.0a - - *Experimental* (Havana, Icehouse) - *Stable* (Juno) - -.. toctree:: - :maxdepth: 1 - - extensions/oauth1.rst - -* `API Specification for OAuth 1.0a <http://specs.openstack.org/openstack/keystone-specs/api/v3/identity-api-v3-os-oauth1-ext.html>`__ - ------------------ -Revocation Events ------------------ - -The Revocation Events extension provides a list of token revocations. Each -event expresses a set of criteria which describes a set of tokens that are -no longer valid. - -.. NOTE:: Support status for Revocation Events - - *Experimental* (Juno) - *Stable* (Kilo) - -.. toctree:: - :maxdepth: 1 - - extensions/revoke.rst - -* `API Specification for Revocation Events <http://specs.openstack.org/openstack/keystone-specs/api/v3/identity-api-v3-os-revoke-ext.html>`__ diff --git a/keystone-moon/doc/source/federation/shibboleth.rst b/keystone-moon/doc/source/federation/shibboleth.rst index d67cfa1a..b82bd703 100644 --- a/keystone-moon/doc/source/federation/shibboleth.rst +++ b/keystone-moon/doc/source/federation/shibboleth.rst @@ -48,10 +48,13 @@ a *<Location>* directive for each identity provider:: ShibRequestSetting requireSession 1 ShibRequestSetting applicationId idp_1 AuthType shibboleth - ShibRequireAll On - ShibRequireSession On ShibExportAssertion Off Require valid-user + + <IfVersion < 2.4> + ShibRequireSession On + ShibRequireAll On + </IfVersion> </Location> .. NOTE:: @@ -61,7 +64,7 @@ a *<Location>* directive for each identity provider:: The same name is used inside the shibboleth2.xml configuration file but they could be different. * The ``ShibRequireSession`` and ``ShibRequireAll`` rules are invalid in - Apache 2.4+ and should be dropped in that specific setup. + Apache 2.4+. * You are advised to carefully examine `Shibboleth Apache configuration documentation <https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPApacheConfig>`_ @@ -265,7 +268,7 @@ environment variable is present so make sure Shibboleth doesn't set the ``/etc/shibboleth/shibboleth2.xml`` configuration file and remove the ``REMOTE_USER`` directives. -Examine your attributes map file ``/etc/shibboleth/attributes-map.xml`` and adjust +Examine your attributes map file ``/etc/shibboleth/attribute-map.xml`` and adjust your requirements if needed. For more information see `attributes documentation <https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPAddAttribute>`_ diff --git a/keystone-moon/doc/source/http-api.rst b/keystone-moon/doc/source/http-api.rst index a31b5e69..3b915881 100644 --- a/keystone-moon/doc/source/http-api.rst +++ b/keystone-moon/doc/source/http-api.rst @@ -73,14 +73,14 @@ method: .. code-block:: ini [app:service_v3] - paste.app_factory = keystone.service:v3_app_factory + use = egg:keystone#service_v3 Then define a v3 pipeline, which terminates with the v3 application you defined above: .. code-block:: ini - [app:app_v3] + [pipeline:api_v3] pipeline = ... service_v3 Replace "..." with whatever middleware you'd like to run in front of the API diff --git a/keystone-moon/doc/source/index.rst b/keystone-moon/doc/source/index.rst index 511bc89f..00e55176 100644 --- a/keystone-moon/doc/source/index.rst +++ b/keystone-moon/doc/source/index.rst @@ -47,7 +47,7 @@ Getting Started .. toctree:: :maxdepth: 1 - setup + devref/development.environment installing configuration policy_mapping @@ -55,6 +55,7 @@ Getting Started mapping_combinations mapping_schema configure_tokenless_x509 + auth-totp configuringservices extensions key_terms @@ -80,11 +81,20 @@ Developers Documentation middlewarearchitecture http-api api_curl_examples - cli_examples apache-httpd external-auth event_notifications - extension_development + services + online_schema_migration_examples + + +Sample Configuration File +========================= + +.. toctree:: + :maxdepth: 1 + + sample_config Code Documentation ================== diff --git a/keystone-moon/doc/source/installing.rst b/keystone-moon/doc/source/installing.rst index e38663b7..b00ac6d2 100644 --- a/keystone-moon/doc/source/installing.rst +++ b/keystone-moon/doc/source/installing.rst @@ -21,7 +21,7 @@ Installing Keystone This document describes how to install Keystone in order to use it. If you are intending to develop on or with Keystone, please read :doc:`developing` and -:doc:`setup`. +:doc:`devref/development.environment`. Installing from Source ---------------------- @@ -49,7 +49,14 @@ Install the Keystone web service: .. code-block:: bash - $ python setup.py install + $ pip install . + +.. NOTE:: + + This step is guaranteed to fail if you do not have the proper binary + dependencies already installed on your development system. Maintaining a + list of platform-specific dependencies is outside the scope of this + documentation, but is within scope of DEVSTACK_. You should have all the pieces you need to run Keystone installed on your system. The following commands should be available on the command-line path: @@ -92,8 +99,7 @@ bash script called `lib/keystone`_ Installing from packages: Ubuntu -------------------------------- -Ubuntu is providing packages for Keystone for Precise. To install keystone -on Ubuntu: +To install keystone on Ubuntu: .. code-block:: bash @@ -101,9 +107,8 @@ on Ubuntu: In using Ubuntu's packages, the packages will set up a user account for the Keystone service (`keystone`), and place default configurations in -``/etc/keystone``. The Debian installer will also ask you about configuration -options for setting up and running Keystone. As of this writing, the defaults -for Keystone backends are all SQL based, stored locally in a SQLite. +``/etc/keystone``. As of this writing, the defaults for Keystone backends are +all SQL based, stored locally in SQLite. Once installed, you still need to initialize data in Keystone, which you can find described in :doc:`configuringservices`. @@ -123,4 +128,4 @@ To install the packages: Once installed, you still need to initialize data in Keystone, which you can find described in :doc:`configuringservices`. -.. _`OpenStack Install Guide`: http://docs.openstack.org/juno/install-guide/install/yum/content/keystone-install.html +.. _`OpenStack Install Guide`: http://docs.openstack.org/liberty/install-guide-rdo/keystone-install.html diff --git a/keystone-moon/doc/source/key_terms.rst b/keystone-moon/doc/source/key_terms.rst index 93aec532..11ae576e 100644 --- a/keystone-moon/doc/source/key_terms.rst +++ b/keystone-moon/doc/source/key_terms.rst @@ -28,7 +28,7 @@ globally unique, but only unique to their domain. Resources ========= -The Identity portion of keystone includes ``Projects`` and ``Domains``, and +The Resources portion of keystone includes ``Projects`` and ``Domains``, and are commonly stored in an SQL backend. Projects (Tenants) diff --git a/keystone-moon/doc/source/man/keystone-manage.rst b/keystone-moon/doc/source/man/keystone-manage.rst index 2ef2d51a..a69cf374 100644 --- a/keystone-moon/doc/source/man/keystone-manage.rst +++ b/keystone-moon/doc/source/man/keystone-manage.rst @@ -7,9 +7,9 @@ Keystone Management Utility --------------------------- :Author: openstack@lists.openstack.org -:Date: 2015-10-15 +:Date: 2016-4-7 :Copyright: OpenStack Foundation -:Version: 8.0.0 +:Version: 9.0.0 :Manual section: 1 :Manual group: cloud computing @@ -40,6 +40,7 @@ Invoking ``keystone-manage`` by itself will give you some usage information. Available commands: +* ``bootstrap``: Perform the basic bootstrap process. * ``db_sync``: Sync the database. * ``db_version``: Print the current migration version of the database. * ``domain_config_upload``: Upload domain configuration file. @@ -47,7 +48,7 @@ Available commands: * ``fernet_setup``: Setup a Fernet key repository. * ``mapping_purge``: Purge the identity mapping table. * ``mapping_engine``: Test your federation mapping rules. -* ``pki_setup``: Initialize the certificates used to sign tokens. +* ``pki_setup``: Initialize the certificates used to sign tokens. **deprecated** * ``saml_idp_metadata``: Generate identity provider metadata. * ``ssl_setup``: Generate certificates for SSL. * ``token_flush``: Purge expired tokens. diff --git a/keystone-moon/doc/source/mapping_combinations.rst b/keystone-moon/doc/source/mapping_combinations.rst index 9aa411ad..1b275a4a 100644 --- a/keystone-moon/doc/source/mapping_combinations.rst +++ b/keystone-moon/doc/source/mapping_combinations.rst @@ -206,12 +206,12 @@ In ``<other_condition>`` shown below, please supply one of the following: { "user": { "name": "{0}" - }, - "groups": { - "name": "{1}", - "domain": { - "id": "0cd5e9" - } + } + }, + { + "groups": "{1}", + "domain": { + "id": "0cd5e9" } } ], @@ -594,4 +594,57 @@ global username mapping. ] } +Keystone to Keystone +-------------------- + +Keystone to Keystone federation also utilizes mappings, but has some +differences. + +An attribute file (``/etc/shibboleth/attribute-map.xml``) is used to add +attributes to the Keystone Identity Provider. Attributes look as follows: + +.. code-block:: xml + + <Attribute name="openstack_user" id="openstack_user"/> + <Attribute name="openstack_user_domain" id="openstack_user_domain"/> + +The Keystone Service Provider must contain a mapping as shown below. +``openstack_user``, and ``openstack_user_domain`` match to the attribute +names we have in the Identity Provider. It will map any user with the name +``user1`` or ``admin`` in the ``openstack_user`` attribute and +``openstack_domain`` attribute ``default`` to a group with id ``abc1234``. + +.. code-block:: javascript + + { + rules = [ + { + "local": [ + { + "group": { + "id": "abc1234" + } + } + ], + "remote": [ + { + "type": "openstack_user", + "any_one_of": [ + "user1", + "admin" + ] + }, + { + "type":"openstack_user_domain", + "any_one_of": [ + "Default" + ] + } + ] + } + ] + } +The possible attributes that can be used in a mapping are `openstack_user`, +`openstack_user_domain`, `openstack_roles`, `openstack_project`, and +`openstack_project_domain`. diff --git a/keystone-moon/doc/source/online_schema_migration_examples.rst b/keystone-moon/doc/source/online_schema_migration_examples.rst new file mode 100644 index 00000000..0e0fb399 --- /dev/null +++ b/keystone-moon/doc/source/online_schema_migration_examples.rst @@ -0,0 +1,24 @@ + +==================================== +Online SQL schema migration examples +==================================== + +This document links to several examples implementing online SQL schema +migrations to facilitate simultaneously running OpenStack services in +different versions with the same DB schema. + + +* Nova `data migration example + <http://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/flavor-from-sysmeta-to-blob.html>`_ +* Nova `data migration enforcement example + <https://review.openstack.org/#/c/174480/15/nova/db/sqlalchemy/migrate_repo/versions/291_enforce_flavors_migrated.py>`_ + of sqlalchemy migrate/deprecated scripts +* Nova `flavor migration spec + <http://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/flavor-from-sysmeta-to-blob.html>`_ + example of data migrations in the object layer +* Cinder `online schema upgrades spec <https://specs.openstack.org/openstack/cinder-specs/specs/mitaka/online-schema-upgrades.html>`_ + example of migrating a column to a many-to-many relation table + + +For documentation on how to make online migrations move on to +:ref:`Database Schema Migrations <online-migration>`. diff --git a/keystone-moon/doc/source/policy_mapping.rst b/keystone-moon/doc/source/policy_mapping.rst index 9b11efd6..2d3cd60a 100644 --- a/keystone-moon/doc/source/policy_mapping.rst +++ b/keystone-moon/doc/source/policy_mapping.rst @@ -73,12 +73,26 @@ identity:create_role POST /v3/roles identity:update_role PATCH /v3/roles/{role_id} identity:delete_role DELETE /v3/roles/{role_id} +identity:get_domain_role GET /v3/roles/{role_id} where role.domain_id is not null +identity:list_domain_roles GET /v3/roles?domain_id where role.domain_id is not null +identity:create_domain_role POST /v3/roles where role.domain_id is not null +identity:update_domain_role PATCH /v3/roles/{role_id} where role.domain_id is not null +identity:delete_domain_role DELETE /v3/roles/{role_id} where role.domain_id is not null + +identity:get_implied_role GET /v3/roles/{prior_role_id}/implies/{implied_role_id} +identity:list_implied_roles GET /v3/roles/{prior_role_id}/implies +identity:create_implied_role PUT /v3/roles/{prior_role_id}/implies/{implied_role_id} +identity:delete_implied_role DELETE /v3/roles/{prior_role_id}/implies/{implied_role_id} +identity:list_role_inference_rules GET /v3/role_inferences +identity:check_implied_role HEAD /v3/roles/{prior_role_id}/implies/{implied_role_id} + identity:check_grant GET `grant_resources`_ identity:list_grants GET `grant_collections`_ identity:create_grant PUT `grant_resources`_ identity:revoke_grant DELETE `grant_resources`_ identity:list_role_assignments GET /v3/role_assignments +identity:list_role_assignments_for_tree GET /v3/role_assignments?include_subtree identity:get_policy GET /v3/policy/{policy_id} identity:list_policies GET /v3/policy @@ -185,7 +199,9 @@ identity:update_domain_config - PATCH /v3/domains/{ identity:delete_domain_config - DELETE /v3/domains/{domain_id}/config - DELETE /v3/domains/{domain_id}/config/{group} - DELETE /v3/domains/{domain_id}/config/{group}/{option} - +identity:get_domain_config_default - GET /v3/domains/config/default + - GET /v3/domains/config/{group}/default + - GET /v3/domains/config/{group}/{option}/default ========================================================= === .. _grant_resources: diff --git a/keystone-moon/doc/source/sample_config.rst b/keystone-moon/doc/source/sample_config.rst new file mode 100644 index 00000000..b170f848 --- /dev/null +++ b/keystone-moon/doc/source/sample_config.rst @@ -0,0 +1,12 @@ +============================== +Keystone Configuration Options +============================== + +The following is a sample keystone configuration for adaptation and use. It is +auto-generated from keystone when this documentation is built, so if you are +having issues with an option, please compare your version of keystone with the +version of this documentation. + +The sample configuration can also be viewed in `file form <_static/keystone.conf.sample>`_. + +.. literalinclude:: _static/keystone.conf.sample diff --git a/keystone-moon/doc/source/services.rst b/keystone-moon/doc/source/services.rst new file mode 100644 index 00000000..2c71e450 --- /dev/null +++ b/keystone-moon/doc/source/services.rst @@ -0,0 +1,200 @@ +=========================== +Keystone for other services +=========================== + +This document provides a summary of some things that other services need to +know about how keystone works, and specifically about how they can take +advantage of the v3 API. + +The v3 API was introduced as a stable API in the Grizzly release and included +in the default pipeline ever since. Until recently, its use has been hidden +from other services because the ``auth_token`` middleware translated the token +format so that both versions look the same. Once the services need to make use +of v3 features they need to know about how it works. + + +Glossary +======== + +Service + OpenStack services like identity, compute, image, etc. + +Project + A project provides namespace and resource isolation for groups of OpenStack + entities. Users must be assigned a role on a project in order to interact + with it. Prior to the introduction of the v3 API, projects were referred to + as tenants and the term is still used in reference to the v2.0 API. + + +Domains +======= + +A major new feature in v3 is domains. Every project, user, and user group is +owned by a domain (reflected by their ``domain_id`` value) which provides them +their own namespace. For example, unlike in v2.0, usernames are no longer +unique across the deployment. You can have two users with the same name, but +they must be in different domains. However, user IDs are assigned to users by +keystone and are expected to be unique across the deployment. All of this logic +applies to both projects and user groups as well. Note that roles are *not* +namespaced by domains. + +One of the great things about domains is that you can have one domain backed by +SQL (for service users) and another backed by LDAP (the cloud is deployed into +existing infrastructure). + +The "default" domain +==================== + +Conventionally the "default" domain has a domain ID of ``default`` and a domain +name of ``Default``. It is created by ``keystone-manage db_sync`` and thus +should always exist, although the domain ID is configurable in +``keystone.conf`` and the domain name is mutable through the v3 API. + +Because only the v3 API is domain-aware, we must work to avoid perceived +namespace conflicts to v2.0 clients. The solution to this is to have a single +domain serve as the implied namespace for all user and tenant references in +v2.0. Thus, v2.0 clients can continue to be domain-unaware and avoid the +security risk posed by potential namespace conflicts. *This is the only purpose +of the default domain.* + +For example, I could otherwise create a domain in v3, create a user in that +domain called "admin", authenticate using v2.0, and a domain-unaware v2.0 +client might assume I'm the same "admin" user it has seen before and grant me +escalated privileges. Instead, users outside of the default domain simply +cannot authenticate against v2.0, nor can such tokens with references to users +and projects outside the default domain be validated on the v2.0 API. + +From a v2.0 client's perspective, there's no way to specify the domain, so v2.0 +operations implicitly work against the default domain. So if your client is +only capable of using v2.0 and you need to get a token, then you can only get +tokens for users and tenants (projects) in the default domain. In the real +world, this means that if your default domain is backed by SQL and you have a +separate domain for LDAP users, then you can't authenticate as an LDAP user +using v2.0. Conversely, if your default domain is backed by a read-only LDAP +driver, then you won't be able to create the service users using v2.0 clients +because any SQL-backed domain is unreachable. + +From a v3 client's perspective, the default domain is not special, other than +the fact that such a domain can generally be assumed to exist (assuming the +deployment is also running the v2.0 API). It would be reasonable for a v3 +client to assume a default user domain ID of ``default`` and a default project +domain ID of ``default`` unless overridden by more specific configuration. + +To summarize, avoiding namespace conflicts in the v2.0 API is achieved by +limiting the v2.0 API and its clients to working with users and projects which +are namespaced by a single, arbitrary domain in v3. + +Token differences +================= + +The keystone service runs both v2.0 and v3, where v2.0 requests go to the +``/v2.0`` endpoint and v3 requests go to the ``/v3`` endpoint. If you're using +the default pipeline that ships with keystone, then you don't need "enable" the +v3 API in keystone, as it runs by default as a parallel pipeline to the v2.0 +API. + +If you get a token using the v2.0 API, then you can use it to do v3 operations +(such as list users). The reverse, using a v3 token against v2.0, is possible +only in certain circumstances. For example, if you're using a project-scoped +token wherein the user and project are both owned by the "default" domain, +everything will work. Otherwise, token validation against the v2.0 API will +fail. + +You can get a v2.0 token using ``POST /v2.0/tokens``. You can get a v3 token +using ``POST /v3/auth/tokens``. Note that the responses are significantly +different. For example, the service catalog is in a different format, and the +v3 token conveys additional context (such as the user's domain and the +project's domain). + +Domain-scoped tokens +-------------------- + +Domain-scoped tokens are scoped to a domain rather than a project. These are +useful for operating against keystone but are generally useless in other +services that don't have use cases for domain-level operations. Unless a +service has a real case for handling such authorization, they don't need to +concern themselves with domain-scoped tokens. + + +Auth Token middleware +===================== + +The ``auth_token`` middleware handles token validation for the different +services. Conceptually, what happens is that ``auth_token`` pulls the token out +of the ``X-Auth-Token`` request header, validates the token using keystone, +produces information about the identity (the API user) and authorization +context (the project, roles, etc) of the token, and sets environment variables +with that data. The services typically take the environment variables, put them +in the service's "context", and use the context for policy enforcement via +``oslo.policy``. + +Service tokens +-------------- + +Service tokens are a feature where the ``auth_token`` middleware will also +accept a service token in the ``X-Service-Token`` header. It does the same +thing with the service token as the user token, but the results of the token +are passed separately in environment variables for the service token (the +service user, project, and roles). If the service knows about these then it can +put this info in its "context" and use it for policy checks. For example, +assuming there's a special policy rule called ``service_role`` that works like +the ``role`` rule except checks the service roles, you could have an +``oslo.policy`` rule like ``service_role:service and user_id:%(user_id)s`` such +that a service token is required along with the user owning the object. + +v2.0 or v3? +----------- + +By default, the ``auth_token`` middleware will use discovery to determine the +best available API to use, or can be explicitly configured to use either v2.0 +or v3. When discovery is used, ``auth_token`` will use v3 if keystone reports +that v3 is available. If ``auth_token`` is configured to use v2.0, then it will +fail when it receives a v3 token wherein the user is not in the default domain +(for example, the domain that heat creates users in). So if at all possible, +the ``auth_token`` middleware should be allowed to use v3. + +Additionally, as other services begin to utilize features which are only found +in the v3 API, you'll need to use the v3 API in order to utilize those +services. For example, heat creates users in an isolated domain, and thus +requires the v3 API. + +Do this, not that +================= + +Config options for authentication +--------------------------------- + +If you need to get a token, don't define options for username and password and +get a token using v2.0. We've got an interface for using authentication plugins +where there's an option for that supports v2.0 or v3 and potentially other +authentication mechanisms (X.509 client certs!). + +If your config file doesn't have the domain for the user, it's not going to be +able to use v3 for authentication. + +Picking the version +------------------- + +Use version discovery to figure out what version the identity server supports +rather than configuring the version. + +Use OpenStack CLI not keystone CLI +---------------------------------- + +The keystone CLI is deprecated and will be removed soon. The `OpenStack CLI +<http://docs.openstack.org/developer/python-openstackclient/>`_ has all the +keystone CLI commands and even supports v3. + + +Hierarchical Multitenancy +========================= + +This feature allows maintenance of a hierarchy of projects with "parent" +projects operating as domains. + +The token format is the same (the token doesn't contain any info about the +hierarchy). If the service needs to know the hierarchy it will have to use the +v3 API to fetch the hierarchy. + +While you can't use v2.0 to set up the hierarchy, you can get a v2.0 token +scoped to a project that's part of a hierarchy. |