diff options
| author |   Zhou Ya <zhou.ya@zte.com.cn> | 2016-11-24 13:57:31 +0800 |
|---|---|---|
| committer | Zhou Ya <zhou.ya@zte.com.cn> | 2016-11-28 15:25:44 +0800 |
| commit | 3f262284f3ce78ce23b4e3c8e9fed112fc56e37d (patch) | |
| tree | a00651eaff5d1aa60f1972c6b7172abcd1562297 | |
| parent | 7db4ee4e743d8ec55a4552560427c0ff37ec6de5 (diff) | |
add escalator cli framework
JIRA:ESCALATOR-36
This patch will support escalatorclient,and this is just the frame of
escalatorclient,with this code you can use 'python setup.py sdist' to
generate escalatorclient package.
Change-Id: Id7b602345f7cb78bb548b589d1297a201056699a
Signed-off-by: Zhou Ya <zhou.ya@zte.com.cn>
42 files changed, 5232 insertions, 0 deletions
diff --git a/client/AUTHORS b/client/AUTHORS new file mode 100644 index 0000000..22952a6 --- /dev/null +++ b/client/AUTHORS @@ -0,0 +1,11 @@ +Aric Gardner <agardner@linuxfoundation.org> +Jie Hu <hu.jie@zte.com.cn> +Zhou Ya <zhou.ya@zte.com.cn> +Liyi Meng <liyi.meng@ericsson.com> +Maria Toeroe <Maria.Toeroe@ericsson.com> +Ryota MIBU <r-mibu@cq.jp.nec.com> +SerenaFeng <feng.xiaoewi@zte.com.cn> +chaozhong-zte <chao.zhong@zte.com.cn> +hujie <hu.jie@zte.com.cn> +wangguobing <wang.guobing1@zte.com.cn> +zhang-jun3g <zhang.jun3g@zte.com.cn> diff --git a/client/ChangeLog b/client/ChangeLog new file mode 100644 index 0000000..5ea65ab --- /dev/null +++ b/client/ChangeLog @@ -0,0 +1,8 @@ +CHANGES +======= + +1.0.0 +------ + +* Add escalatorclient + diff --git a/client/LICENSE b/client/LICENSE new file mode 100644 index 0000000..67db858 --- /dev/null +++ b/client/LICENSE @@ -0,0 +1,175 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. diff --git a/client/MANIFEST.in b/client/MANIFEST.in new file mode 100644 index 0000000..ae484e5 --- /dev/null +++ b/client/MANIFEST.in @@ -0,0 +1,10 @@ +include ChangeLog +include MANIFEST.in pylintrc +include AUTHORS +include LICENSE +include ChangeLog +include babel.cfg tox.ini +graft docs +graft etc +graft escalator/locale +global-exclude *.pyc diff --git a/client/PKG-INFO b/client/PKG-INFO new file mode 100644 index 0000000..ecd5763 --- /dev/null +++ b/client/PKG-INFO @@ -0,0 +1,30 @@ +Metadata-Version: 1.1 +Name: escalatorclient +Version: 1.0.0 +Summary: escalator Escalator Client Library +Home-page: http://www.opnfv.org/ +Author: OPNFV +Author-email: opnfv-tech-discuss@lists.opnfv.org +License: Apache License, Version 2.0 +Description: Python bindings to the escalator Escalator Client + ============================================= + + This is a client library for escalator built on the Escalator Client. It provides a Python API (the ``escalatorclient`` module) and a command-line tool (``escalator``). This library fully supports the v1 Escalator Client, while support for the v2 Client is in progress. + + Development takes place via the usual OPNFV processes as outlined in the `developer guide <http://docs.openstack.org/infra/manual/developers.html>`_. + + +Platform: UNKNOWN +Classifier: Development Status :: 5 - Production/Stable +Classifier: Environment :: Console +Classifier: Environment :: OPNFV +Classifier: Intended Audience :: Information Technology +Classifier: Intended Audience :: System Administrators +Classifier: License :: OSI Approved :: Apache Software License +Classifier: Operating System :: POSIX :: Linux +Classifier: Programming Language :: Python +Classifier: Programming Language :: Python :: 2 +Classifier: Programming Language :: Python :: 2.7 +Classifier: Programming Language :: Python :: 2.6 +Classifier: Programming Language :: Python :: 3 +Classifier: Programming Language :: Python :: 3.3 diff --git a/client/README.rst b/client/README.rst new file mode 100644 index 0000000..ba5a5d7 --- /dev/null +++ b/client/README.rst @@ -0,0 +1,6 @@ +Python bindings to the Escalator Client +============================================= + +This is a client library for Escalator built on the Escalator Client. +It provides a Python API (the ``escalatorclient`` module) and a command-line tool (``escalator``). +This library fully supports the Escalator Client. diff --git a/client/babel.cfg b/client/babel.cfg new file mode 100644 index 0000000..efceab8 --- /dev/null +++ b/client/babel.cfg @@ -0,0 +1 @@ +[python: **.py] diff --git a/client/doc/Makefile b/client/doc/Makefile new file mode 100644 index 0000000..430e5a3 --- /dev/null +++ b/client/doc/Makefile @@ -0,0 +1,90 @@ +# Makefile for Sphinx documentation +# + +# You can set these variables from the command line. +SPHINXOPTS = +SPHINXBUILD = sphinx-build +SPHINXSOURCE = source +PAPER = +BUILDDIR = build + +# Internal variables. +PAPEROPT_a4 = -D latex_paper_size=a4 +PAPEROPT_letter = -D latex_paper_size=letter +ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) $(SPHINXSOURCE) + +.PHONY: help clean html dirhtml pickle json htmlhelp qthelp latex changes linkcheck doctest + +help: + @echo "Please use \`make <target>' where <target> is one of" + @echo " html to make standalone HTML files" + @echo " dirhtml to make HTML files named index.html in directories" + @echo " pickle to make pickle files" + @echo " json to make JSON files" + @echo " htmlhelp to make HTML files and a HTML help project" + @echo " qthelp to make HTML files and a qthelp project" + @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" + @echo " changes to make an overview of all changed/added/deprecated items" + @echo " linkcheck to check all external links for integrity" + @echo " doctest to run all doctests embedded in the documentation (if enabled)" + +clean: + -rm -rf $(BUILDDIR)/* + +html: + $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html + @echo + @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." + +dirhtml: + $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml + @echo + @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." + +pickle: + $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle + @echo + @echo "Build finished; now you can process the pickle files." + +json: + $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json + @echo + @echo "Build finished; now you can process the JSON files." + +htmlhelp: + $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp + @echo + @echo "Build finished; now you can run HTML Help Workshop with the" \ + ".hhp project file in $(BUILDDIR)/htmlhelp." + +qthelp: + $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp + @echo + @echo "Build finished; now you can run "qcollectiongenerator" with the" \ + ".qhcp project file in $(BUILDDIR)/qthelp, like this:" + @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/python-keystoneclient.qhcp" + @echo "To view the help file:" + @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/python-keystoneclient.qhc" + +latex: + $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex + @echo + @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." + @echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \ + "run these through (pdf)latex." + +changes: + $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes + @echo + @echo "The overview file is in $(BUILDDIR)/changes." + +linkcheck: + $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck + @echo + @echo "Link check complete; look for any errors in the above output " \ + "or in $(BUILDDIR)/linkcheck/output.txt." + +doctest: + $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest + @echo "Testing of doctests in the sources finished, look at the " \ + "results in $(BUILDDIR)/doctest/output.txt." diff --git a/client/doc/source/apiv2.rst b/client/doc/source/apiv2.rst new file mode 100644 index 0000000..0695b2a --- /dev/null +++ b/client/doc/source/apiv2.rst @@ -0,0 +1,27 @@ +Python API v1 +============= + +To create a client:: + + from keystoneclient.auth.identity import v2 as identity + from keystoneclient import session + from escalatorclient import Client + + auth = identity.Password(auth_url=AUTH_URL, + username=USERNAME, + password=PASSWORD, + tenant_name=PROJECT_ID) + + sess = session.Session(auth=auth) + token = auth.get_token(sess) + + escalator = Client('1', endpoint=OS_IMAGE_ENDPOINT, token=token) + + +List +---- +List nodes you can access:: + + for node in escalator.nodes.list(): + print node + diff --git a/client/doc/source/conf.py b/client/doc/source/conf.py new file mode 100644 index 0000000..1cfaad2 --- /dev/null +++ b/client/doc/source/conf.py @@ -0,0 +1,72 @@ +# -*- coding: utf-8 -*- +# + +import os +import sys + +sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), + '..', '..'))) + + +# -- General configuration ---------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. +extensions = ['sphinx.ext.autodoc', 'oslosphinx'] + +# autodoc generation is a bit aggressive and a nuisance when doing heavy +# text edit cycles. +# execute "export SPHINX_DEBUG=1" in your terminal to disable + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# The suffix of source filenames. +source_suffix = '.rst' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +project = 'python-escalatorclient' +copyright = u'OpenStack Foundation' + +# If true, '()' will be appended to :func: etc. cross-reference text. +add_function_parentheses = True + +# If true, the current module name will be prepended to all description +# unit titles (such as .. function::). +add_module_names = True + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +# Grouping the document tree for man pages. +# List of tuples 'sourcefile', 'target', u'title', u'Authors name', 'manual' + +man_pages = [ + ('man/escalator', 'escalator', u'Client for OpenStack Images API', + [u'OpenStack Foundation'], 1), +] +# -- Options for HTML output -------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. Major themes that come with +# Sphinx are currently 'default' and 'sphinxdoc'. +# html_theme = 'nature' + +# Output file base name for HTML help builder. +htmlhelp_basename = '%sdoc' % project + + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, author, documentclass +# [howto/manual]). +latex_documents = [ + ( + 'index', + '%s.tex' % project, + u'%s Documentation' % project, + u'OpenStack Foundation', + 'manual' + ), +] diff --git a/client/doc/source/index.rst b/client/doc/source/index.rst new file mode 100644 index 0000000..ec7d4ef --- /dev/null +++ b/client/doc/source/index.rst @@ -0,0 +1,37 @@ +Python API +========== +In order to use the python api directly, you must first obtain an auth token and identify which endpoint you wish to speak to. Once you have done so, you can use the API like so:: + + >>> from escalatorclient import Client + >>> escalator = Client('1', endpoint=OS_IMAGE_ENDPOINT, token=OS_AUTH_TOKEN) + >>> image = escalator.images.create(name="My Test Image") + >>> print image.status + 'queued' + >>> image.update(data=open('/tmp/myimage.iso', 'rb')) + >>> print image.status + 'active' + >>> image.update(properties=dict(my_custom_property='value')) + >>> with open('/tmp/copyimage.iso', 'wb') as f: + for chunk in image.data: + f.write(chunk) + >>> image.delete() + +For an API v2 example see also :doc:`apiv2`. + +Command-line Tool +================= +In order to use the CLI, you must provide your OpenStack username, password, tenant, and auth endpoint. Use the corresponding configuration options (``--os-username``, ``--os-password``, ``--os-tenant-id``, and ``--os-auth-url``) or set them in environment variables:: + + export OS_USERNAME=user + export OS_PASSWORD=pass + export OS_TENANT_ID=b363706f891f48019483f8bd6503c54b + export OS_AUTH_URL=http://auth.example.com:5000/v2.0 + +The command line tool will attempt to reauthenticate using your provided credentials for every request. You can override this behavior by manually supplying an auth token using ``--os-image-url`` and ``--os-auth-token``. You can alternatively set these environment variables:: + + export OS_IMAGE_URL=http://escalator.example.org:9292/ + export OS_AUTH_TOKEN=3bcc3d3a03f44e3d8377f9247b0ad155 + +Once you've configured your authentication parameters, you can run ``escalator help`` to see a complete listing of available commands. + +See also :doc:`/man/escalator`.
\ No newline at end of file diff --git a/client/doc/source/man/escalator.rst b/client/doc/source/man/escalator.rst new file mode 100644 index 0000000..40536ec --- /dev/null +++ b/client/doc/source/man/escalator.rst @@ -0,0 +1,87 @@ +============================== +:program:`escalator` CLI man page +============================== + +.. program:: escalator +.. highlight:: bash + +SYNOPSIS +======== + +:program:`escalator` [options] <command> [command-options] + +:program:`escalator help` + +:program:`escalator help` <command> + + +DESCRIPTION +=========== + +The :program:`escalator` command line utility interacts with OpenStack Images +Service (escalator). + +In order to use the CLI, you must provide your OpenStack username, password, +project (historically called tenant), and auth endpoint. You can use +configuration options :option:`--os-username`, :option:`--os-password`, +:option:`--os-tenant-id`, and :option:`--os-auth-url` or set corresponding +environment variables:: + + export OS_USERNAME=user + export OS_PASSWORD=pass + export OS_TENANT_ID=b363706f891f48019483f8bd6503c54b + export OS_AUTH_URL=http://auth.example.com:5000/v2.0 + +The command line tool will attempt to reauthenticate using provided +credentials for every request. You can override this behavior by manually +supplying an auth token using :option:`--os-image-url` and +:option:`--os-auth-token` or by setting corresponding environment variables:: + + export OS_IMAGE_URL=http://escalator.example.org:9292/ + export OS_AUTH_TOKEN=3bcc3d3a03f44e3d8377f9247b0ad155 + + +You can select an API version to use by :option:`--os-image-api-version` +option or by setting corresponding environment variable:: + + export OS_IMAGE_API_VERSION=2 + +OPTIONS +======= + +To get a list of available commands and options run:: + + escalator help + +To get usage and options of a command:: + + escalator help <command> + + +EXAMPLES +======== + +Get information about image-create command:: + + escalator help image-create + +See available images:: + + escalator image-list + +Create new image:: + + escalator image-create --name foo --disk-format=qcow2 \ + --container-format=bare --is-public=True \ + --copy-from http://somewhere.net/foo.img + +Describe a specific image:: + + escalator image-show foo + + +BUGS +==== + +escalator client is hosted in Launchpad so you can view current bugs at +https://bugs.launchpad.net/python-escalatorclient/. diff --git a/client/escalatorclient/__init__.py b/client/escalatorclient/__init__.py new file mode 100644 index 0000000..4b95f8a --- /dev/null +++ b/client/escalatorclient/__init__.py @@ -0,0 +1,31 @@ +# Copyright 2016 OPNFV Foundation +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +# NOTE(bcwaldon): this try/except block is needed to run setup.py due to +# its need to import local code before installing required dependencies +try: + import escalatorclient.client + Client = escalatorclient.client.Client +except ImportError: + import warnings + warnings.warn("Could not import escalatorclient.client", ImportWarning) + +import pbr.version + +version_info = pbr.version.VersionInfo('python-escalatorclient') + +try: + __version__ = version_info.version_string() +except AttributeError: + __version__ = None diff --git a/client/escalatorclient/_i18n.py b/client/escalatorclient/_i18n.py new file mode 100644 index 0000000..bbabb98 --- /dev/null +++ b/client/escalatorclient/_i18n.py @@ -0,0 +1,34 @@ +# Copyright 2012 OpenStack Foundation +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. +try: + import oslo_i18n as i18n +except ImportError: + from oslo import i18n + + +_translators = i18n.TranslatorFactory(domain='escalatorclient') + +# The primary translation function using the well-known name "_" +_ = _translators.primary + +# Translators for log levels. +# +# The abbreviated names are meant to reflect the usual use of a short +# name like '_'. The "L" is for "log" and the other letter comes from +# the level. +_LI = _translators.log_info +_LW = _translators.log_warning +_LE = _translators.log_error +_LC = _translators.log_critical diff --git a/client/escalatorclient/client.py b/client/escalatorclient/client.py new file mode 100644 index 0000000..b11e23b --- /dev/null +++ b/client/escalatorclient/client.py @@ -0,0 +1,39 @@ +# Copyright 2012 OpenStack Foundation +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +import warnings + +from escalatorclient.common import utils + + +def Client(version=None, endpoint=None, *args, **kwargs): + if version is not None: + warnings.warn(("`version` keyword is being deprecated. Please pass the" + " version as part of the URL. " + "http://$HOST:$PORT/v$VERSION_NUMBER"), + DeprecationWarning) + + endpoint, url_version = utils.strip_version(endpoint) + + if not url_version and not version: + msg = ("Please provide either the version or an url with the form " + "http://$HOST:$PORT/v$VERSION_NUMBER") + raise RuntimeError(msg) + + version = int(version or url_version) + + module = utils.import_versioned_module(version, 'client') + client_class = getattr(module, 'Client') + return client_class(endpoint, *args, **kwargs) diff --git a/client/escalatorclient/common/__init__.py b/client/escalatorclient/common/__init__.py new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/client/escalatorclient/common/__init__.py diff --git a/client/escalatorclient/common/base.py b/client/escalatorclient/common/base.py new file mode 100644 index 0000000..329ca6b --- /dev/null +++ b/client/escalatorclient/common/base.py @@ -0,0 +1,35 @@ +# Copyright 2012 OpenStack Foundation +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +""" +Base utilities to build API operation managers and objects on top of. + +DEPRECATED post v.0.12.0. Use 'escalatorclient.openstack.common.apiclient.base' +instead of this module." +""" + +import warnings + +from escalatorclient.openstack.common.apiclient import base + + +warnings.warn("The 'escalatorclient.common.base' module is deprecated post " + "v.0.12.0. Use 'escalatorclient.openstack.common.apiclient.base' " + "instead of this one.", DeprecationWarning) + + +getid = base.getid +Manager = base.ManagerWithFind +Resource = base.Resource diff --git a/client/escalatorclient/common/http.py b/client/escalatorclient/common/http.py new file mode 100644 index 0000000..301eedb --- /dev/null +++ b/client/escalatorclient/common/http.py @@ -0,0 +1,288 @@ +# Copyright 2012 OpenStack Foundation +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +import copy +import logging +import socket +from oslo_utils import encodeutils +from escalatorclient.common import https +from escalatorclient.common.utils import safe_header +from escalatorclient import exc +from oslo_utils import importutils +from oslo_utils import netutils +from simplejson import decoder +import requests +try: + from requests.packages.urllib3.exceptions import ProtocolError +except ImportError: + ProtocolError = requests.exceptions.ConnectionError +import six +from six.moves.urllib import parse + +try: + import json +except ImportError: + import simplejson as json + +# Python 2.5 compat fix +if not hasattr(parse, 'parse_qsl'): + import cgi + parse.parse_qsl = cgi.parse_qsl + + +osprofiler_web = importutils.try_import("osprofiler.web") + +LOG = logging.getLogger(__name__) +USER_AGENT = 'python-escalatorclient' +CHUNKSIZE = 1024 * 64 # 64kB + + +class HTTPClient(object): + + def __init__(self, endpoint, **kwargs): + self.endpoint = endpoint + self.identity_headers = kwargs.get('identity_headers') + self.auth_token = kwargs.get('token') + if self.identity_headers: + if self.identity_headers.get('X-Auth-Token'): + self.auth_token = self.identity_headers.get('X-Auth-Token') + del self.identity_headers['X-Auth-Token'] + + self.session = requests.Session() + self.session.headers["User-Agent"] = USER_AGENT + + if self.auth_token: + self.session.headers["X-Auth-Token"] = self.auth_token + + self.timeout = float(kwargs.get('timeout', 600)) + + if self.endpoint.startswith("https"): + compression = kwargs.get('ssl_compression', True) + + if not compression: + self.session.mount("escalator+https://", https.HTTPSAdapter()) + self.endpoint = 'escalator+' + self.endpoint + + self.session.verify = ( + kwargs.get('cacert', requests.certs.where()), + kwargs.get('insecure', False)) + + else: + if kwargs.get('insecure', False) is True: + self.session.verify = False + else: + if kwargs.get('cacert', None) is not '': + self.session.verify = kwargs.get('cacert', True) + + self.session.cert = (kwargs.get('cert_file'), + kwargs.get('key_file')) + + @staticmethod + def parse_endpoint(endpoint): + return netutils.urlsplit(endpoint) + + def log_curl_request(self, method, url, headers, data, kwargs): + curl = ['curl -g -i -X %s' % method] + + headers = copy.deepcopy(headers) + headers.update(self.session.headers) + + for (key, value) in six.iteritems(headers): + header = '-H \'%s: %s\'' % safe_header(key, value) + curl.append(header) + + if not self.session.verify: + curl.append('-k') + else: + if isinstance(self.session.verify, six.string_types): + curl.append(' --cacert %s' % self.session.verify) + + if self.session.cert: + curl.append(' --cert %s --key %s' % self.session.cert) + + if data and isinstance(data, six.string_types): + curl.append('-d \'%s\'' % data) + + curl.append(url) + + msg = ' '.join([encodeutils.safe_decode(item, errors='ignore') + for item in curl]) + LOG.debug(msg) + + @staticmethod + def log_http_response(resp, body=None): + status = (resp.raw.version / 10.0, resp.status_code, resp.reason) + dump = ['\nHTTP/%.1f %s %s' % status] + headers = resp.headers.items() + dump.extend(['%s: %s' % safe_header(k, v) for k, v in headers]) + dump.append('') + if body: + body = encodeutils.safe_decode(body) + dump.extend([body, '']) + LOG.debug('\n'.join([encodeutils.safe_decode(x, errors='ignore') + for x in dump])) + + @staticmethod + def encode_headers(headers): + """Encodes headers. + + Note: This should be used right before + sending anything out. + + :param headers: Headers to encode + :returns: Dictionary with encoded headers' + names and values + """ + return dict((encodeutils.safe_encode(h), encodeutils.safe_encode(v)) + for h, v in six.iteritems(headers) if v is not None) + + def _request(self, method, url, **kwargs): + """Send an http request with the specified characteristics. + Wrapper around httplib.HTTP(S)Connection.request to handle tasks such + as setting headers and error handling. + """ + # Copy the kwargs so we can reuse the original in case of redirects + headers = kwargs.pop("headers", {}) + headers = headers and copy.deepcopy(headers) or {} + + if self.identity_headers: + for k, v in six.iteritems(self.identity_headers): + headers.setdefault(k, v) + + # Default Content-Type is octet-stream + content_type = headers.get('Content-Type', 'application/octet-stream') + + def chunk_body(body): + chunk = body + while chunk: + chunk = body.read(CHUNKSIZE) + if chunk == '': + break + yield chunk + + data = kwargs.pop("data", None) + if data is not None and not isinstance(data, six.string_types): + try: + data = json.dumps(data) + content_type = 'application/json' + except TypeError: + # Here we assume it's + # a file-like object + # and we'll chunk it + data = chunk_body(data) + + headers['Content-Type'] = content_type + stream = True if content_type == 'application/octet-stream' else False + + if osprofiler_web: + headers.update(osprofiler_web.get_trace_id_headers()) + + # Note(flaper87): Before letting headers / url fly, + # they should be encoded otherwise httplib will + # complain. + headers = self.encode_headers(headers) + + try: + if self.endpoint.endswith("/") or url.startswith("/"): + conn_url = "%s%s" % (self.endpoint, url) + else: + conn_url = "%s/%s" % (self.endpoint, url) + self.log_curl_request(method, conn_url, headers, data, kwargs) + resp = self.session.request(method, + conn_url, + data=data, + stream=stream, + headers=headers, + **kwargs) + except requests.exceptions.Timeout as e: + message = ("Error communicating with %(endpoint)s %(e)s" % + dict(url=conn_url, e=e)) + raise exc.InvalidEndpoint(message=message) + except (requests.exceptions.ConnectionError, ProtocolError) as e: + message = ("Error finding address for %(url)s: %(e)s" % + dict(url=conn_url, e=e)) + raise exc.CommunicationError(message=message) + except socket.gaierror as e: + message = "Error finding address for %s: %s" % ( + self.endpoint_hostname, e) + raise exc.InvalidEndpoint(message=message) + except (socket.error, socket.timeout) as e: + endpoint = self.endpoint + message = ("Error communicating with %(endpoint)s %(e)s" % + {'endpoint': endpoint, 'e': e}) + raise exc.CommunicationError(message=message) + + if not resp.ok: + LOG.debug("Request returned failure status %s." % resp.status_code) + raise exc.from_response(resp, resp.text) + elif resp.status_code == requests.codes.MULTIPLE_CHOICES: + raise exc.from_response(resp) + + content_type = resp.headers.get('Content-Type') + + # Read body into string if it isn't obviously image data + if content_type == 'application/octet-stream': + # Do not read all response in memory when + # downloading an image. + body_iter = _close_after_stream(resp, CHUNKSIZE) + self.log_http_response(resp) + else: + content = resp.text + self.log_http_response(resp, content) + if content_type and content_type.startswith('application/json'): + # Let's use requests json method, + # it should take care of response + # encoding + try: + body_iter = resp.json() + except decoder.JSONDecodeError: + status_body = {'status_code': resp.status_code} + return resp, status_body + else: + body_iter = six.StringIO(content) + try: + body_iter = json.loads(''.join([c for c in body_iter])) + except ValueError: + body_iter = None + return resp, body_iter + + def head(self, url, **kwargs): + return self._request('HEAD', url, **kwargs) + + def get(self, url, **kwargs): + return self._request('GET', url, **kwargs) + + def post(self, url, **kwargs): + return self._request('POST', url, **kwargs) + + def put(self, url, **kwargs): + return self._request('PUT', url, **kwargs) + + def patch(self, url, **kwargs): + return self._request('PATCH', url, **kwargs) + + def delete(self, url, **kwargs): + return self._request('DELETE', url, **kwargs) + + +def _close_after_stream(response, chunk_size): + """Iterate over the content and ensure the response is closed after.""" + # Yield each chunk in the response body + for chunk in response.iter_content(chunk_size=chunk_size): + yield chunk + # Once we're done streaming the body, ensure everything is closed. + # This will return the connection to the HTTPConnectionPool in urllib3 + # and ideally reduce the number of HTTPConnectionPool full warnings. + response.close() diff --git a/client/escalatorclient/common/https.py b/client/escalatorclient/common/https.py new file mode 100644 index 0000000..55769a0 --- /dev/null +++ b/client/escalatorclient/common/https.py @@ -0,0 +1,349 @@ +# Copyright 2014 Red Hat, Inc +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +import socket +import ssl +import struct + +import OpenSSL +from requests import adapters +from requests import compat +try: + from requests.packages.urllib3 import connectionpool +except ImportError: + from urllib3 import connectionpool + +from oslo_utils import encodeutils +import six +# NOTE(jokke): simplified transition to py3, behaves like py2 xrange +from six.moves import range + +from escalatorclient.common import utils + +try: + from eventlet import patcher + # Handle case where we are running in a monkey patched environment + if patcher.is_monkey_patched('socket'): + from eventlet.green.httplib import HTTPSConnection + from eventlet.green.OpenSSL.SSL import GreenConnection as Connection + from eventlet.greenio import GreenSocket + # TODO(mclaren): A getsockopt workaround: see 'getsockopt' doc string + GreenSocket.getsockopt = utils.getsockopt + else: + raise ImportError +except ImportError: + try: + from httplib import HTTPSConnection + except ImportError: + from http.client import HTTPSConnection + from OpenSSL.SSL import Connection as Connection + + +from escalatorclient import exc + + +def verify_callback(host=None): + """ + We use a partial around the 'real' verify_callback function + so that we can stash the host value without holding a + reference on the VerifiedHTTPSConnection. + """ + def wrapper(connection, x509, errnum, + depth, preverify_ok, host=host): + return do_verify_callback(connection, x509, errnum, + depth, preverify_ok, host=host) + return wrapper + + +def do_verify_callback(connection, x509, errnum, + depth, preverify_ok, host=None): + """ + Verify the server's SSL certificate. + + This is a standalone function rather than a method to avoid + issues around closing sockets if a reference is held on + a VerifiedHTTPSConnection by the callback function. + """ + if x509.has_expired(): + msg = "SSL Certificate expired on '%s'" % x509.get_notAfter() + raise exc.SSLCertificateError(msg) + + if depth == 0 and preverify_ok: + # We verify that the host matches against the last + # certificate in the chain + return host_matches_cert(host, x509) + else: + # Pass through OpenSSL's default result + return preverify_ok + + +def host_matches_cert(host, x509): + """ + Verify that the x509 certificate we have received + from 'host' correctly identifies the server we are + connecting to, ie that the certificate's Common Name + or a Subject Alternative Name matches 'host'. + """ + def check_match(name): + # Directly match the name + if name == host: + return True + + # Support single wildcard matching + if name.startswith('*.') and host.find('.') > 0: + if name[2:] == host.split('.', 1)[1]: + return True + + common_name = x509.get_subject().commonName + + # First see if we can match the CN + if check_match(common_name): + return True + # Also try Subject Alternative Names for a match + san_list = None + for i in range(x509.get_extension_count()): + ext = x509.get_extension(i) + if ext.get_short_name() == b'subjectAltName': + san_list = str(ext) + for san in ''.join(san_list.split()).split(','): + if san.startswith('DNS:'): + if check_match(san.split(':', 1)[1]): + return True + + # Server certificate does not match host + msg = ('Host "%s" does not match x509 certificate contents: ' + 'CommonName "%s"' % (host, common_name)) + if san_list is not None: + msg = msg + ', subjectAltName "%s"' % san_list + raise exc.SSLCertificateError(msg) + + +def to_bytes(s): + if isinstance(s, six.string_types): + return six.b(s) + else: + return s + + +class HTTPSAdapter(adapters.HTTPAdapter): + """ + This adapter will be used just when + ssl compression should be disabled. + + The init method overwrites the default + https pool by setting escalatorclient's + one. + """ + + def request_url(self, request, proxies): + # NOTE(flaper87): Make sure the url is encoded, otherwise + # python's standard httplib will fail with a TypeError. + url = super(HTTPSAdapter, self).request_url(request, proxies) + return encodeutils.safe_encode(url) + + def _create_escalator_httpsconnectionpool(self, url): + kw = self.poolmanager.connection_kw + # Parse the url to get the scheme, host, and port + parsed = compat.urlparse(url) + # If there is no port specified, we should use the standard HTTPS port + port = parsed.port or 443 + pool = HTTPSConnectionPool(parsed.host, port, **kw) + + with self.poolmanager.pools.lock: + self.poolmanager.pools[(parsed.scheme, parsed.host, port)] = pool + + return pool + + def get_connection(self, url, proxies=None): + try: + return super(HTTPSAdapter, self).get_connection(url, proxies) + except KeyError: + # NOTE(sigamvirus24): This works around modifying a module global + # which fixes bug #1396550 + # The scheme is most likely escalator+https but check anyway + if not url.startswith('escalator+https://'): + raise + + return self._create_escalator_httpsconnectionpool(url) + + def cert_verify(self, conn, url, verify, cert): + super(HTTPSAdapter, self).cert_verify(conn, url, verify, cert) + conn.ca_certs = verify[0] + conn.insecure = verify[1] + + +class HTTPSConnectionPool(connectionpool.HTTPSConnectionPool): + """ + HTTPSConnectionPool will be instantiated when a new + connection is requested to the HTTPSAdapter.This + implementation overwrites the _new_conn method and + returns an instances of escalatorclient's VerifiedHTTPSConnection + which handles no compression. + + ssl_compression is hard-coded to False because this will + be used just when the user sets --no-ssl-compression. + """ + + scheme = 'escalator+https' + + def _new_conn(self): + self.num_connections += 1 + return VerifiedHTTPSConnection(host=self.host, + port=self.port, + key_file=self.key_file, + cert_file=self.cert_file, + cacert=self.ca_certs, + insecure=self.insecure, + ssl_compression=False) + + +class OpenSSLConnectionDelegator(object): + """ + An OpenSSL.SSL.Connection delegator. + + Supplies an additional 'makefile' method which httplib requires + and is not present in OpenSSL.SSL.Connection. + + Note: Since it is not possible to inherit from OpenSSL.SSL.Connection + a delegator must be used. + """ + def __init__(self, *args, **kwargs): + self.connection = Connection(*args, **kwargs) + + def __getattr__(self, name): + return getattr(self.connection, name) + + def makefile(self, *args, **kwargs): + return socket._fileobject(self.connection, *args, **kwargs) + + +class VerifiedHTTPSConnection(HTTPSConnection): + """ + Extended HTTPSConnection which uses the OpenSSL library + for enhanced SSL support. + Note: Much of this functionality can eventually be replaced + with native Python 3.3 code. + """ + # Restrict the set of client supported cipher suites + CIPHERS = 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:'\ + 'eCDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:'\ + 'RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS' + + def __init__(self, host, port=None, key_file=None, cert_file=None, + cacert=None, timeout=None, insecure=False, + ssl_compression=True): + # List of exceptions reported by Python3 instead of + # SSLConfigurationError + if six.PY3: + excp_lst = (TypeError, IOError, ssl.SSLError) + # https.py:250:36: F821 undefined name 'FileNotFoundError' + else: + # NOTE(jamespage) + # Accomodate changes in behaviour for pep-0467, introduced + # in python 2.7.9. + # https://github.com/python/peps/blob/master/pep-0476.txt + excp_lst = (TypeError, IOError, ssl.SSLError) + try: + HTTPSConnection.__init__(self, host, port, + key_file=key_file, + cert_file=cert_file) + self.key_file = key_file + self.cert_file = cert_file + self.timeout = timeout + self.insecure = insecure + # NOTE(flaper87): `is_verified` is needed for + # requests' urllib3. If insecure is True then + # the request is not `verified`, hence `not insecure` + self.is_verified = not insecure + self.ssl_compression = ssl_compression + self.cacert = None if cacert is None else str(cacert) + self.set_context() + # ssl exceptions are reported in various form in Python 3 + # so to be compatible, we report the same kind as under + # Python2 + except excp_lst as e: + raise exc.SSLConfigurationError(str(e)) + + def set_context(self): + """ + Set up the OpenSSL context. + """ + self.context = OpenSSL.SSL.Context(OpenSSL.SSL.SSLv23_METHOD) + self.context.set_cipher_list(self.CIPHERS) + + if self.ssl_compression is False: + self.context.set_options(0x20000) # SSL_OP_NO_COMPRESSION + + if self.insecure is not True: + self.context.set_verify(OpenSSL.SSL.VERIFY_PEER, + verify_callback(host=self.host)) + else: + self.context.set_verify(OpenSSL.SSL.VERIFY_NONE, + lambda *args: True) + + if self.cert_file: + try: + self.context.use_certificate_file(self.cert_file) + except Exception as e: + msg = 'Unable to load cert from "%s" %s' % (self.cert_file, e) + raise exc.SSLConfigurationError(msg) + if self.key_file is None: + # We support having key and cert in same file + try: + self.context.use_privatekey_file(self.cert_file) + except Exception as e: + msg = ('No key file specified and unable to load key ' + 'from "%s" %s' % (self.cert_file, e)) + raise exc.SSLConfigurationError(msg) + + if self.key_file: + try: + self.context.use_privatekey_file(self.key_file) + except Exception as e: + msg = 'Unable to load key from "%s" %s' % (self.key_file, e) + raise exc.SSLConfigurationError(msg) + + if self.cacert: + try: + self.context.load_verify_locations(to_bytes(self.cacert)) + except Exception as e: + msg = 'Unable to load CA from "%s" %s' % (self.cacert, e) + raise exc.SSLConfigurationError(msg) + else: + self.context.set_default_verify_paths() + + def connect(self): + """ + Connect to an SSL port using the OpenSSL library and apply + per-connection parameters. + """ + result = socket.getaddrinfo(self.host, self.port, 0, + socket.SOCK_STREAM) + if result: + socket_family = result[0][0] + if socket_family == socket.AF_INET6: + sock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM) + else: + sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + else: + # If due to some reason the address lookup fails - we still connect + # to IPv4 socket. This retains the older behavior. + sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + if self.timeout is not None: + # '0' microseconds + sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVTIMEO, + struct.pack('LL', self.timeout, 0)) + self.sock = OpenSSLConnectionDelegator(self.context, sock) + self.sock.connect((self.host, self.port)) diff --git a/client/escalatorclient/common/utils.py b/client/escalatorclient/common/utils.py new file mode 100644 index 0000000..0156d31 --- /dev/null +++ b/client/escalatorclient/common/utils.py @@ -0,0 +1,462 @@ +# Copyright 2012 OpenStack Foundation +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +from __future__ import print_function + +import errno +import hashlib +import json +import os +import re +import sys +import threading +import uuid +from oslo_utils import encodeutils +from oslo_utils import strutils +import prettytable +import six + +from escalatorclient import exc +from oslo_utils import importutils + +if os.name == 'nt': + import msvcrt +else: + msvcrt = None + + +_memoized_property_lock = threading.Lock() + +SENSITIVE_HEADERS = ('X-Auth-Token', ) + + +# Decorator for cli-args +def arg(*args, **kwargs): + def _decorator(func): + # Because of the sematics of decorator composition if we just append + # to the options list positional options will appear to be backwards. + func.__dict__.setdefault('arguments', []).insert(0, (args, kwargs)) + return func + return _decorator + + +def schema_args(schema_getter, omit=None): + omit = omit or [] + typemap = { + 'string': str, + 'integer': int, + 'boolean': strutils.bool_from_string, + 'array': list + } + + def _decorator(func): + schema = schema_getter() + if schema is None: + param = '<unavailable>' + kwargs = { + 'help': ("Please run with connection parameters set to " + "retrieve the schema for generating help for this " + "command") + } + func.__dict__.setdefault('arguments', []).insert(0, ((param, ), + kwargs)) + else: + properties = schema.get('properties', {}) + for name, property in six.iteritems(properties): + if name in omit: + continue + param = '--' + name.replace('_', '-') + kwargs = {} + + type_str = property.get('type', 'string') + + if isinstance(type_str, list): + # NOTE(flaper87): This means the server has + # returned something like `['null', 'string']`, + # therfore we use the first non-`null` type as + # the valid type. + for t in type_str: + if t != 'null': + type_str = t + break + + if type_str == 'array': + items = property.get('items') + kwargs['type'] = typemap.get(items.get('type')) + kwargs['nargs'] = '+' + else: + kwargs['type'] = typemap.get(type_str) + + if type_str == 'boolean': + kwargs['metavar'] = '[True|False]' + else: + kwargs['metavar'] = '<%s>' % name.upper() + + description = property.get('description', "") + if 'enum' in property: + if len(description): + description += " " + + # NOTE(flaper87): Make sure all values are `str/unicode` + # for the `join` to succeed. Enum types can also be `None` + # therfore, join's call would fail without the following + # list comprehension + vals = [six.text_type(val) for val in property.get('enum')] + description += ('Valid values: ' + ', '.join(vals)) + kwargs['help'] = description + + func.__dict__.setdefault('arguments', + []).insert(0, ((param, ), kwargs)) + return func + + return _decorator + + +def pretty_choice_list(l): + return ', '.join("'%s'" % i for i in l) + + +def print_list(objs, fields, formatters=None, field_settings=None, + conver_field=True): + formatters = formatters or {} + field_settings = field_settings or {} + pt = prettytable.PrettyTable([f for f in fields], caching=False) + pt.align = 'l' + + for o in objs: + row = [] + for field in fields: + if field in field_settings: + for setting, value in six.iteritems(field_settings[field]): + setting_dict = getattr(pt, setting) + setting_dict[field] = value + + if field in formatters: + row.append(formatters[field](o)) + else: + if conver_field: + field_name = field.lower().replace(' ', '_') + else: + field_name = field.replace(' ', '_') + data = getattr(o, field_name, None) + row.append(data) + pt.add_row(row) + + print(encodeutils.safe_decode(pt.get_string())) + + +def print_dict(d, max_column_width=80): + pt = prettytable.PrettyTable(['Property', 'Value'], caching=False) + pt.align = 'l' + pt.max_width = max_column_width + for k, v in six.iteritems(d): + if isinstance(v, (dict, list)): + v = json.dumps(v) + pt.add_row([k, v]) + print(encodeutils.safe_decode(pt.get_string(sortby='Property'))) + + +def find_resource(manager, id): + """Helper for the _find_* methods.""" + # first try to get entity as integer id + try: + if isinstance(id, int) or id.isdigit(): + return manager.get(int(id)) + except exc.NotFound: + pass + + # now try to get entity as uuid + try: + # This must be unicode for Python 3 compatibility. + # If you pass a bytestring to uuid.UUID, you will get a TypeError + uuid.UUID(encodeutils.safe_decode(id)) + return manager.get(id) + except (ValueError, exc.NotFound): + msg = ("id %s is error " % id) + raise exc.CommandError(msg) + + # finally try to find entity by name + matches = list(manager.list(filters={'name': id})) + num_matches = len(matches) + if num_matches == 0: + msg = "No %s with a name or ID of '%s' exists." % \ + (manager.resource_class.__name__.lower(), id) + raise exc.CommandError(msg) + elif num_matches > 1: + msg = ("Multiple %s matches found for '%s', use an ID to be more" + " specific." % (manager.resource_class.__name__.lower(), + id)) + raise exc.CommandError(msg) + else: + return matches[0] + + +def skip_authentication(f): + """Function decorator used to indicate a caller may be unauthenticated.""" + f.require_authentication = False + return f + + +def is_authentication_required(f): + """Checks to see if the function requires authentication. + + Use the skip_authentication decorator to indicate a caller may + skip the authentication step. + """ + return getattr(f, 'require_authentication', True) + + +def env(*vars, **kwargs): + """Search for the first defined of possibly many env vars + + Returns the first environment variable defined in vars, or + returns the default defined in kwargs. + """ + for v in vars: + value = os.environ.get(v, None) + if value: + return value + return kwargs.get('default', '') + + +def import_versioned_module(version, submodule=None): + module = 'escalatorclient.v%s' % version + if submodule: + module = '.'.join((module, submodule)) + return importutils.import_module(module) + + +def exit(msg='', exit_code=1): + if msg: + print(encodeutils.safe_decode(msg), file=sys.stderr) + sys.exit(exit_code) + + +def save_image(data, path): + """ + Save an image to the specified path. + + :param data: binary data of the image + :param path: path to save the image to + """ + if path is None: + image = sys.stdout + else: + image = open(path, 'wb') + try: + for chunk in data: + image.write(chunk) + finally: + if path is not None: + image.close() + + +def make_size_human_readable(size): + suffix = ['B', 'kB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB'] + base = 1024.0 + + index = 0 + while size >= base: + index = index + 1 + size = size / base + + padded = '%.1f' % size + stripped = padded.rstrip('0').rstrip('.') + + return '%s%s' % (stripped, suffix[index]) + + +def getsockopt(self, *args, **kwargs): + """ + A function which allows us to monkey patch eventlet's + GreenSocket, adding a required 'getsockopt' method. + TODO: (mclaren) we can remove this once the eventlet fix + (https://bitbucket.org/eventlet/eventlet/commits/609f230) + lands in mainstream packages. + """ + return self.fd.getsockopt(*args, **kwargs) + + +def exception_to_str(exc): + try: + error = six.text_type(exc) + except UnicodeError: + try: + error = str(exc) + except UnicodeError: + error = ("Caught '%(exception)s' exception." % + {"exception": exc.__class__.__name__}) + return encodeutils.safe_decode(error, errors='ignore') + + +def get_file_size(file_obj): + """ + Analyze file-like object and attempt to determine its size. + + :param file_obj: file-like object. + :retval The file's size or None if it cannot be determined. + """ + if (hasattr(file_obj, 'seek') and hasattr(file_obj, 'tell') and + (six.PY2 or six.PY3 and file_obj.seekable())): + try: + curr = file_obj.tell() + file_obj.seek(0, os.SEEK_END) + size = file_obj.tell() + file_obj.seek(curr) + return size + except IOError as e: + if e.errno == errno.ESPIPE: + # Illegal seek. This means the file object + # is a pipe (e.g. the user is trying + # to pipe image data to the client, + # echo testdata | bin/escalator add blah...), or + # that file object is empty, or that a file-like + # object which doesn't support 'seek/tell' has + # been supplied. + return + else: + raise + + +def get_data_file(args): + if args.file: + return open(args.file, 'rb') + else: + # distinguish cases where: + # (1) stdin is not valid (as in cron jobs): + # escalator ... <&- + # (2) image data is provided through standard input: + # escalator ... < /tmp/file or cat /tmp/file | escalator ... + # (3) no image data provided: + # escalator ... + try: + os.fstat(0) + except OSError: + # (1) stdin is not valid (closed...) + return None + if not sys.stdin.isatty(): + # (2) image data is provided through standard input + if msvcrt: + msvcrt.setmode(sys.stdin.fileno(), os.O_BINARY) + return sys.stdin + else: + # (3) no image data provided + return None + + +def strip_version(endpoint): + """Strip version from the last component of endpoint if present.""" + # NOTE(flaper87): This shouldn't be necessary if + # we make endpoint the first argument. However, we + # can't do that just yet because we need to keep + # backwards compatibility. + if not isinstance(endpoint, six.string_types): + raise ValueError("Expected endpoint") + + version = None + # Get rid of trailing '/' if present + endpoint = endpoint.rstrip('/') + url_bits = endpoint.split('/') + # regex to match 'v1' or 'v2.0' etc + if re.match('v\d+\.?\d*', url_bits[-1]): + version = float(url_bits[-1].lstrip('v')) + endpoint = '/'.join(url_bits[:-1]) + return endpoint, version + + +def print_image(image_obj, max_col_width=None): + ignore = ['self', 'access', 'file', 'schema'] + image = dict([item for item in six.iteritems(image_obj) + if item[0] not in ignore]) + if str(max_col_width).isdigit(): + print_dict(image, max_column_width=max_col_width) + else: + print_dict(image) + + +def integrity_iter(iter, checksum): + """ + Check image data integrity. + + :raises: IOError + """ + md5sum = hashlib.md5() + for chunk in iter: + yield chunk + if isinstance(chunk, six.string_types): + chunk = six.b(chunk) + md5sum.update(chunk) + md5sum = md5sum.hexdigest() + if md5sum != checksum: + raise IOError(errno.EPIPE, + 'Corrupt image download. Checksum was %s expected %s' % + (md5sum, checksum)) + + +def memoized_property(fn): + attr_name = '_lazy_once_' + fn.__name__ + + @property + def _memoized_property(self): + if hasattr(self, attr_name): + return getattr(self, attr_name) + else: + with _memoized_property_lock: + if not hasattr(self, attr_name): + setattr(self, attr_name, fn(self)) + return getattr(self, attr_name) + return _memoized_property + + +def safe_header(name, value): + if name in SENSITIVE_HEADERS: + v = value.encode('utf-8') + h = hashlib.sha1(v) + d = h.hexdigest() + return name, "{SHA1}%s" % d + else: + return name, value + + +def to_str(value): + if value is None: + return value + if not isinstance(value, six.string_types): + return str(value) + return value + + +def get_host_min_mac(host_interfaces): + mac_list = [interface['mac'] for interface in + host_interfaces if interface.get('mac')] + if mac_list: + return min(mac_list) + else: + return None + + +class IterableWithLength(object): + def __init__(self, iterable, length): + self.iterable = iterable + self.length = length + + def __iter__(self): + return self.iterable + + def next(self): + return next(self.iterable) + + def __len__(self): + return self.length diff --git a/client/escalatorclient/exc.py b/client/escalatorclient/exc.py new file mode 100644 index 0000000..06a9126 --- /dev/null +++ b/client/escalatorclient/exc.py @@ -0,0 +1,201 @@ +# Copyright 2012 OpenStack Foundation +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +import re +import sys + + +class BaseException(Exception): + """An error occurred.""" + def __init__(self, message=None): + self.message = message + + def __str__(self): + return self.message or self.__class__.__doc__ + + +class CommandError(BaseException): + """Invalid usage of CLI.""" + + +class InvalidEndpoint(BaseException): + """The provided endpoint is invalid.""" + + +class CommunicationError(BaseException): + """Unable to communicate with server.""" + + +class ClientException(Exception): + """DEPRECATED!""" + + +class HTTPException(ClientException): + """Base exception for all HTTP-derived exceptions.""" + code = 'N/A' + + def __init__(self, details=None): + self.details = details or self.__class__.__name__ + + def __str__(self): + return "%s (HTTP %s)" % (self.details, self.code) + + +class HTTPMultipleChoices(HTTPException): + code = 300 + + def __str__(self): + self.details = ("Requested version of OpenStack Images API is not " + "available.") + return "%s (HTTP %s) %s" % (self.__class__.__name__, self.code, + self.details) + + +class BadRequest(HTTPException): + """DEPRECATED!""" + code = 400 + + +class HTTPBadRequest(BadRequest): + pass + + +class Unauthorized(HTTPException): + """DEPRECATED!""" + code = 401 + + +class HTTPUnauthorized(Unauthorized): + pass + + +class Forbidden(HTTPException): + """DEPRECATED!""" + code = 403 + + +class HTTPForbidden(Forbidden): + pass + + +class NotFound(HTTPException): + """DEPRECATED!""" + code = 404 + + +class HTTPNotFound(NotFound): + pass + + +class HTTPMethodNotAllowed(HTTPException): + code = 405 + + +class Conflict(HTTPException): + """DEPRECATED!""" + code = 409 + + +class HTTPConflict(Conflict): + pass + + +class OverLimit(HTTPException): + """DEPRECATED!""" + code = 413 + + +class HTTPOverLimit(OverLimit): + pass + + +class HTTPInternalServerError(HTTPException): + code = 500 + + +class HTTPNotImplemented(HTTPException): + code = 501 + + +class HTTPBadGateway(HTTPException): + code = 502 + + +class ServiceUnavailable(HTTPException): + """DEPRECATED!""" + code = 503 + + +class HTTPServiceUnavailable(ServiceUnavailable): + pass + + +# NOTE(bcwaldon): Build a mapping of HTTP codes to corresponding exception +# classes +_code_map = {} +for obj_name in dir(sys.modules[__name__]): + if obj_name.startswith('HTTP'): + obj = getattr(sys.modules[__name__], obj_name) + _code_map[obj.code] = obj + + +def from_response(response, body=None): + """Return an instance of an HTTPException based on httplib response.""" + cls = _code_map.get(response.status_code, HTTPException) + if body and 'json' in response.headers['content-type']: + # Iterate over the nested objects and retreive the "message" attribute. + messages = [obj.get('message') for obj in response.json().values()] + # Join all of the messages together nicely and filter out any objects + # that don't have a "message" attr. + details = '\n'.join(i for i in messages if i is not None) + return cls(details=details) + elif body and 'html' in response.headers['content-type']: + # Split the lines, strip whitespace and inline HTML from the response. + details = [re.sub(r'<.+?>', '', i.strip()) + for i in response.text.splitlines()] + details = [i for i in details if i] + # Remove duplicates from the list. + details_seen = set() + details_temp = [] + for i in details: + if i not in details_seen: + details_temp.append(i) + details_seen.add(i) + # Return joined string separated by colons. + details = ': '.join(details_temp) + return cls(details=details) + elif body: + details = body.replace('\n\n', '\n') + return cls(details=details) + + return cls() + + +class NoTokenLookupException(Exception): + """DEPRECATED!""" + pass + + +class EndpointNotFound(Exception): + """DEPRECATED!""" + pass + + +class SSLConfigurationError(BaseException): + pass + + +class SSLCertificateError(BaseException): + pass diff --git a/client/escalatorclient/openstack/__init__.py b/client/escalatorclient/openstack/__init__.py new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/client/escalatorclient/openstack/__init__.py diff --git a/client/escalatorclient/openstack/common/__init__.py b/client/escalatorclient/openstack/common/__init__.py new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/client/escalatorclient/openstack/common/__init__.py diff --git a/client/escalatorclient/openstack/common/_i18n.py b/client/escalatorclient/openstack/common/_i18n.py new file mode 100644 index 0000000..95d1792 --- /dev/null +++ b/client/escalatorclient/openstack/common/_i18n.py @@ -0,0 +1,45 @@ +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +"""oslo.i18n integration module. + +See http://docs.openstack.org/developer/oslo.i18n/usage.html + +""" + +try: + import oslo.i18n + + # NOTE(dhellmann): This reference to o-s-l-o will be replaced by the + # application name when this module is synced into the separate + # repository. It is OK to have more than one translation function + # using the same domain, since there will still only be one message + # catalog. + _translators = oslo.i18n.TranslatorFactory(domain='escalatorclient') + + # The primary translation function using the well-known name "_" + _ = _translators.primary + + # Translators for log levels. + # + # The abbreviated names are meant to reflect the usual use of a short + # name like '_'. The "L" is for "log" and the other letter comes from + # the level. + _LI = _translators.log_info + _LW = _translators.log_warning + _LE = _translators.log_error + _LC = _translators.log_critical +except ImportError: + # NOTE(dims): Support for cases where a project wants to use + # code from oslo-incubator, but is not ready to be internationalized + # (like tempest) + _ = _LI = _LW = _LE = _LC = lambda x: x diff --git a/client/escalatorclient/openstack/common/apiclient/__init__.py b/client/escalatorclient/openstack/common/apiclient/__init__.py new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/client/escalatorclient/openstack/common/apiclient/__init__.py diff --git a/client/escalatorclient/openstack/common/apiclient/auth.py b/client/escalatorclient/openstack/common/apiclient/auth.py new file mode 100644 index 0000000..4d29dcf --- /dev/null +++ b/client/escalatorclient/openstack/common/apiclient/auth.py @@ -0,0 +1,234 @@ +# Copyright 2013 OpenStack Foundation +# Copyright 2013 Spanish National Research Council. +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +# E0202: An attribute inherited from %s hide this method +# pylint: disable=E0202 + +######################################################################## +# +# THIS MODULE IS DEPRECATED +# +# Please refer to +# https://etherpad.openstack.org/p/kilo-escalatorclient-library-proposals for +# the discussion leading to this deprecation. +# +# We recommend checking out the python-openstacksdk project +# (https://launchpad.net/python-openstacksdk) instead. +# +######################################################################## + +import abc +import argparse +import os + +import six +from stevedore import extension + +from escalatorclient.openstack.common.apiclient import exceptions + + +_discovered_plugins = {} + + +def discover_auth_systems(): + """Discover the available auth-systems. + + This won't take into account the old style auth-systems. + """ + global _discovered_plugins + _discovered_plugins = {} + + def add_plugin(ext): + _discovered_plugins[ext.name] = ext.plugin + + ep_namespace = "escalatorclient.openstack.common.apiclient.auth" + mgr = extension.ExtensionManager(ep_namespace) + mgr.map(add_plugin) + + +def load_auth_system_opts(parser): + """Load options needed by the available auth-systems into a parser. + + This function will try to populate the parser with options from the + available plugins. + """ + group = parser.add_argument_group("Common auth options") + BaseAuthPlugin.add_common_opts(group) + for name, auth_plugin in six.iteritems(_discovered_plugins): + group = parser.add_argument_group( + "Auth-system '%s' options" % name, + conflict_handler="resolve") + auth_plugin.add_opts(group) + + +def load_plugin(auth_system): + try: + plugin_class = _discovered_plugins[auth_system] + except KeyError: + raise exceptions.AuthSystemNotFound(auth_system) + return plugin_class(auth_system=auth_system) + + +def load_plugin_from_args(args): + """Load required plugin and populate it with options. + + Try to guess auth system if it is not specified. Systems are tried in + alphabetical order. + + :type args: argparse.Namespace + :raises: AuthPluginOptionsMissing + """ + auth_system = args.os_auth_system + if auth_system: + plugin = load_plugin(auth_system) + plugin.parse_opts(args) + plugin.sufficient_options() + return plugin + + for plugin_auth_system in sorted(six.iterkeys(_discovered_plugins)): + plugin_class = _discovered_plugins[plugin_auth_system] + plugin = plugin_class() + plugin.parse_opts(args) + try: + plugin.sufficient_options() + except exceptions.AuthPluginOptionsMissing: + continue + return plugin + raise exceptions.AuthPluginOptionsMissing(["auth_system"]) + + +@six.add_metaclass(abc.ABCMeta) +class BaseAuthPlugin(object): + """Base class for authentication plugins. + + An authentication plugin needs to override at least the authenticate + method to be a valid plugin. + """ + + auth_system = None + opt_names = [] + common_opt_names = [ + "auth_system", + "username", + "password", + "tenant_name", + "token", + "auth_url", + ] + + def __init__(self, auth_system=None, **kwargs): + self.auth_system = auth_system or self.auth_system + self.opts = dict((name, kwargs.get(name)) + for name in self.opt_names) + + @staticmethod + def _parser_add_opt(parser, opt): + """Add an option to parser in two variants. + + :param opt: option name (with underscores) + """ + dashed_opt = opt.replace("_", "-") + env_var = "OS_%s" % opt.upper() + arg_default = os.environ.get(env_var, "") + arg_help = "Defaults to env[%s]." % env_var + parser.add_argument( + "--os-%s" % dashed_opt, + metavar="<%s>" % dashed_opt, + default=arg_default, + help=arg_help) + parser.add_argument( + "--os_%s" % opt, + metavar="<%s>" % dashed_opt, + help=argparse.SUPPRESS) + + @classmethod + def add_opts(cls, parser): + """Populate the parser with the options for this plugin. + """ + for opt in cls.opt_names: + # use `BaseAuthPlugin.common_opt_names` since it is never + # changed in child classes + if opt not in BaseAuthPlugin.common_opt_names: + cls._parser_add_opt(parser, opt) + + @classmethod + def add_common_opts(cls, parser): + """Add options that are common for several plugins. + """ + for opt in cls.common_opt_names: + cls._parser_add_opt(parser, opt) + + @staticmethod + def get_opt(opt_name, args): + """Return option name and value. + + :param opt_name: name of the option, e.g., "username" + :param args: parsed arguments + """ + return (opt_name, getattr(args, "os_%s" % opt_name, None)) + + def parse_opts(self, args): + """Parse the actual auth-system options if any. + + This method is expected to populate the attribute `self.opts` with a + dict containing the options and values needed to make authentication. + """ + self.opts.update(dict(self.get_opt(opt_name, args) + for opt_name in self.opt_names)) + + def authenticate(self, http_client): + """Authenticate using plugin defined method. + + The method usually analyses `self.opts` and performs + a request to authentication server. + + :param http_client: client object that needs authentication + :type http_client: HTTPClient + :raises: AuthorizationFailure + """ + self.sufficient_options() + self._do_authenticate(http_client) + + @abc.abstractmethod + def _do_authenticate(self, http_client): + """Protected method for authentication. + """ + + def sufficient_options(self): + """Check if all required options are present. + + :raises: AuthPluginOptionsMissing + """ + missing = [opt + for opt in self.opt_names + if not self.opts.get(opt)] + if missing: + raise exceptions.AuthPluginOptionsMissing(missing) + + @abc.abstractmethod + def token_and_endpoint(self, endpoint_type, service_type): + """Return token and endpoint. + + :param service_type: Service type of the endpoint + :type service_type: string + :param endpoint_type: Type of endpoint. + Possible values: public or publicURL, + internal or internalURL, + admin or adminURL + :type endpoint_type: string + :returns: tuple of token and endpoint strings + :raises: EndpointException + """ diff --git a/client/escalatorclient/openstack/common/apiclient/base.py b/client/escalatorclient/openstack/common/apiclient/base.py new file mode 100644 index 0000000..eb7218b --- /dev/null +++ b/client/escalatorclient/openstack/common/apiclient/base.py @@ -0,0 +1,532 @@ +# Copyright 2010 Jacob Kaplan-Moss +# Copyright 2011 OpenStack Foundation +# Copyright 2012 Grid Dynamics +# Copyright 2013 OpenStack Foundation +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +""" +Base utilities to build API operation managers and objects on top of. +""" + +######################################################################## +# +# THIS MODULE IS DEPRECATED +# +# Please refer to +# https://etherpad.openstack.org/p/kilo-escalatorclient-library-proposals for +# the discussion leading to this deprecation. +# +# We recommend checking out the python-openstacksdk project +# (https://launchpad.net/python-openstacksdk) instead. +# +######################################################################## + + +# E1102: %s is not callable +# pylint: disable=E1102 + +import abc +import copy + +from oslo_utils import strutils +import six +from six.moves.urllib import parse + +from escalatorclient.openstack.common._i18n import _ +from escalatorclient.openstack.common.apiclient import exceptions + + +def getid(obj): + """Return id if argument is a Resource. + + Abstracts the common pattern of allowing both an object or an object's ID + (UUID) as a parameter when dealing with relationships. + """ + try: + if obj.uuid: + return obj.uuid + except AttributeError: + pass + try: + return obj.id + except AttributeError: + return obj + + +# TODO(aababilov): call run_hooks() in HookableMixin's child classes +class HookableMixin(object): + """Mixin so classes can register and run hooks.""" + _hooks_map = {} + + @classmethod + def add_hook(cls, hook_type, hook_func): + """Add a new hook of specified type. + + :param cls: class that registers hooks + :param hook_type: hook type, e.g., '__pre_parse_args__' + :param hook_func: hook function + """ + if hook_type not in cls._hooks_map: + cls._hooks_map[hook_type] = [] + + cls._hooks_map[hook_type].append(hook_func) + + @classmethod + def run_hooks(cls, hook_type, *args, **kwargs): + """Run all hooks of specified type. + + :param cls: class that registers hooks + :param hook_type: hook type, e.g., '__pre_parse_args__' + :param args: args to be passed to every hook function + :param kwargs: kwargs to be passed to every hook function + """ + hook_funcs = cls._hooks_map.get(hook_type) or [] + for hook_func in hook_funcs: + hook_func(*args, **kwargs) + + +class BaseManager(HookableMixin): + """Basic manager type providing common operations. + + Managers interact with a particular type of API (servers, flavors, images, + etc.) and provide CRUD operations for them. + """ + resource_class = None + + def __init__(self, client): + """Initializes BaseManager with `client`. + + :param client: instance of BaseClient descendant for HTTP requests + """ + super(BaseManager, self).__init__() + self.client = client + + def _list(self, url, response_key=None, obj_class=None, json=None): + """List the collection. + + :param url: a partial URL, e.g., '/servers' + :param response_key: the key to be looked up in response dictionary, + e.g., 'servers'. If response_key is None - all response body + will be used. + :param obj_class: class for constructing the returned objects + (self.resource_class will be used by default) + :param json: data that will be encoded as JSON and passed in POST + request (GET will be sent by default) + """ + if json: + body = self.client.post(url, json=json).json() + else: + body = self.client.get(url).json() + + if obj_class is None: + obj_class = self.resource_class + + data = body[response_key] if response_key is not None else body + # NOTE(ja): keystone returns values as list as {'values': [ ... ]} + # unlike other services which just return the list... + try: + data = data['values'] + except (KeyError, TypeError): + pass + + return [obj_class(self, res, loaded=True) for res in data if res] + + def _get(self, url, response_key=None): + """Get an object from collection. + + :param url: a partial URL, e.g., '/servers' + :param response_key: the key to be looked up in response dictionary, + e.g., 'server'. If response_key is None - all response body + will be used. + """ + body = self.client.get(url).json() + data = body[response_key] if response_key is not None else body + return self.resource_class(self, data, loaded=True) + + def _head(self, url): + """Retrieve request headers for an object. + + :param url: a partial URL, e.g., '/servers' + """ + resp = self.client.head(url) + return resp.status_code == 204 + + def _post(self, url, json, response_key=None, return_raw=False): + """Create an object. + + :param url: a partial URL, e.g., '/servers' + :param json: data that will be encoded as JSON and passed in POST + request (GET will be sent by default) + :param response_key: the key to be looked up in response dictionary, + e.g., 'server'. If response_key is None - all response body + will be used. + :param return_raw: flag to force returning raw JSON instead of + Python object of self.resource_class + """ + body = self.client.post(url, json=json).json() + data = body[response_key] if response_key is not None else body + if return_raw: + return data + return self.resource_class(self, data) + + def _put(self, url, json=None, response_key=None): + """Update an object with PUT method. + + :param url: a partial URL, e.g., '/servers' + :param json: data that will be encoded as JSON and passed in POST + request (GET will be sent by default) + :param response_key: the key to be looked up in response dictionary, + e.g., 'servers'. If response_key is None - all response body + will be used. + """ + resp = self.client.put(url, json=json) + # PUT requests may not return a body + if resp.content: + body = resp.json() + if response_key is not None: + return self.resource_class(self, body[response_key]) + else: + return self.resource_class(self, body) + + def _patch(self, url, json=None, response_key=None): + """Update an object with PATCH method. + + :param url: a partial URL, e.g., '/servers' + :param json: data that will be encoded as JSON and passed in POST + request (GET will be sent by default) + :param response_key: the key to be looked up in response dictionary, + e.g., 'servers'. If response_key is None - all response body + will be used. + """ + body = self.client.patch(url, json=json).json() + if response_key is not None: + return self.resource_class(self, body[response_key]) + else: + return self.resource_class(self, body) + + def _delete(self, url): + """Delete an object. + + :param url: a partial URL, e.g., '/servers/my-server' + """ + return self.client.delete(url) + + +@six.add_metaclass(abc.ABCMeta) +class ManagerWithFind(BaseManager): + """Manager with additional `find()`/`findall()` methods.""" + + @abc.abstractmethod + def list(self): + pass + + def find(self, **kwargs): + """Find a single item with attributes matching ``**kwargs``. + + This isn't very efficient: it loads the entire list then filters on + the Python side. + """ + matches = self.findall(**kwargs) + num_matches = len(matches) + if num_matches == 0: + msg = _("No %(name)s matching %(args)s.") % { + 'name': self.resource_class.__name__, + 'args': kwargs + } + raise exceptions.NotFound(msg) + elif num_matches > 1: + raise exceptions.NoUniqueMatch() + else: + return matches[0] + + def findall(self, **kwargs): + """Find all items with attributes matching ``**kwargs``. + + This isn't very efficient: it loads the entire list then filters on + the Python side. + """ + found = [] + searches = kwargs.items() + + for obj in self.list(): + try: + if all(getattr(obj, attr) == value + for (attr, value) in searches): + found.append(obj) + except AttributeError: + continue + + return found + + +class CrudManager(BaseManager): + """Base manager class for manipulating entities. + + Children of this class are expected to define a `collection_key` and `key`. + + - `collection_key`: Usually a plural noun by convention (e.g. `entities`); + used to refer collections in both URL's (e.g. `/v3/entities`) and JSON + objects containing a list of member resources (e.g. `{'entities': [{}, + {}, {}]}`). + - `key`: Usually a singular noun by convention (e.g. `entity`); used to + refer to an individual member of the collection. + + """ + collection_key = None + key = None + + def build_url(self, base_url=None, **kwargs): + """Builds a resource URL for the given kwargs. + + Given an example collection where `collection_key = 'entities'` and + `key = 'entity'`, the following URL's could be generated. + + By default, the URL will represent a collection of entities, e.g.:: + + /entities + + If kwargs contains an `entity_id`, then the URL will represent a + specific member, e.g.:: + + /entities/{entity_id} + + :param base_url: if provided, the generated URL will be appended to it + """ + url = base_url if base_url is not None else '' + + url += '/%s' % self.collection_key + + # do we have a specific entity? + entity_id = kwargs.get('%s_id' % self.key) + if entity_id is not None: + url += '/%s' % entity_id + + return url + + def _filter_kwargs(self, kwargs): + """Drop null values and handle ids.""" + for key, ref in six.iteritems(kwargs.copy()): + if ref is None: + kwargs.pop(key) + else: + if isinstance(ref, Resource): + kwargs.pop(key) + kwargs['%s_id' % key] = getid(ref) + return kwargs + + def create(self, **kwargs): + kwargs = self._filter_kwargs(kwargs) + return self._post( + self.build_url(**kwargs), + {self.key: kwargs}, + self.key) + + def get(self, **kwargs): + kwargs = self._filter_kwargs(kwargs) + return self._get( + self.build_url(**kwargs), + self.key) + + def head(self, **kwargs): + kwargs = self._filter_kwargs(kwargs) + return self._head(self.build_url(**kwargs)) + + def list(self, base_url=None, **kwargs): + """List the collection. + + :param base_url: if provided, the generated URL will be appended to it + """ + kwargs = self._filter_kwargs(kwargs) + + return self._list( + '%(base_url)s%(query)s' % { + 'base_url': self.build_url(base_url=base_url, **kwargs), + 'query': '?%s' % parse.urlencode(kwargs) if kwargs else '', + }, + self.collection_key) + + def put(self, base_url=None, **kwargs): + """Update an element. + + :param base_url: if provided, the generated URL will be appended to it + """ + kwargs = self._filter_kwargs(kwargs) + + return self._put(self.build_url(base_url=base_url, **kwargs)) + + def update(self, **kwargs): + kwargs = self._filter_kwargs(kwargs) + params = kwargs.copy() + params.pop('%s_id' % self.key) + + return self._patch( + self.build_url(**kwargs), + {self.key: params}, + self.key) + + def delete(self, **kwargs): + kwargs = self._filter_kwargs(kwargs) + + return self._delete( + self.build_url(**kwargs)) + + def find(self, base_url=None, **kwargs): + """Find a single item with attributes matching ``**kwargs``. + + :param base_url: if provided, the generated URL will be appended to it + """ + kwargs = self._filter_kwargs(kwargs) + + rl = self._list( + '%(base_url)s%(query)s' % { + 'base_url': self.build_url(base_url=base_url, **kwargs), + 'query': '?%s' % parse.urlencode(kwargs) if kwargs else '', + }, + self.collection_key) + num = len(rl) + + if num == 0: + msg = _("No %(name)s matching %(args)s.") % { + 'name': self.resource_class.__name__, + 'args': kwargs + } + raise exceptions.NotFound(404, msg) + elif num > 1: + raise exceptions.NoUniqueMatch + else: + return rl[0] + + +class Extension(HookableMixin): + """Extension descriptor.""" + + SUPPORTED_HOOKS = ('__pre_parse_args__', '__post_parse_args__') + manager_class = None + + def __init__(self, name, module): + super(Extension, self).__init__() + self.name = name + self.module = module + self._parse_extension_module() + + def _parse_extension_module(self): + self.manager_class = None + for attr_name, attr_value in self.module.__dict__.items(): + if attr_name in self.SUPPORTED_HOOKS: + self.add_hook(attr_name, attr_value) + else: + try: + if issubclass(attr_value, BaseManager): + self.manager_class = attr_value + except TypeError: + pass + + def __repr__(self): + return "<Extension '%s'>" % self.name + + +class Resource(object): + """Base class for OpenStack resources (tenant, user, etc.). + + This is pretty much just a bag for attributes. + """ + + HUMAN_ID = False + NAME_ATTR = 'name' + + def __init__(self, manager, info, loaded=False): + """Populate and bind to a manager. + + :param manager: BaseManager object + :param info: dictionary representing resource attributes + :param loaded: prevent lazy-loading if set to True + """ + self.manager = manager + self._info = info + self._add_details(info) + self._loaded = loaded + + def __repr__(self): + reprkeys = sorted(k + for k in self.__dict__.keys() + if k[0] != '_' and k != 'manager') + info = ", ".join("%s=%s" % (k, getattr(self, k)) for k in reprkeys) + return "<%s %s>" % (self.__class__.__name__, info) + + @property + def human_id(self): + """Human-readable ID which can be used for bash completion. + """ + if self.HUMAN_ID: + name = getattr(self, self.NAME_ATTR, None) + if name is not None: + return strutils.to_slug(name) + return None + + def _add_details(self, info): + for (k, v) in six.iteritems(info): + try: + setattr(self, k, v) + self._info[k] = v + except AttributeError: + # In this case we already defined the attribute on the class + pass + + def __getattr__(self, k): + if k not in self.__dict__: + # NOTE(bcwaldon): disallow lazy-loading if already loaded once + if not self.is_loaded(): + self.get() + return self.__getattr__(k) + + raise AttributeError(k) + else: + return self.__dict__[k] + + def get(self): + """Support for lazy loading details. + + Some clients, such as novaclient have the option to lazy load the + details, details which can be loaded with this function. + """ + # set_loaded() first ... so if we have to bail, we know we tried. + self.set_loaded(True) + if not hasattr(self.manager, 'get'): + return + + new = self.manager.get(self.id) + if new: + self._add_details(new._info) + self._add_details( + {'x_request_id': self.manager.client.last_request_id}) + + def __eq__(self, other): + if not isinstance(other, Resource): + return NotImplemented + # two resources of different types are not equal + if not isinstance(other, self.__class__): + return False + if hasattr(self, 'id') and hasattr(other, 'id'): + return self.id == other.id + return self._info == other._info + + def is_loaded(self): + return self._loaded + + def set_loaded(self, val): + self._loaded = val + + def to_dict(self): + return copy.deepcopy(self._info) diff --git a/client/escalatorclient/openstack/common/apiclient/client.py b/client/escalatorclient/openstack/common/apiclient/client.py new file mode 100644 index 0000000..d478989 --- /dev/null +++ b/client/escalatorclient/openstack/common/apiclient/client.py @@ -0,0 +1,388 @@ +# Copyright 2010 Jacob Kaplan-Moss +# Copyright 2011 OpenStack Foundation +# Copyright 2011 Piston Cloud Computing, Inc. +# Copyright 2013 Alessio Ababilov +# Copyright 2013 Grid Dynamics +# Copyright 2013 OpenStack Foundation +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +""" +OpenStack Client interface. Handles the REST calls and responses. +""" + +# E0202: An attribute inherited from %s hide this method +# pylint: disable=E0202 + +import hashlib +import logging +import time + +try: + import simplejson as json +except ImportError: + import json + +from oslo_utils import encodeutils +from oslo_utils import importutils +import requests + +from escalatorclient.openstack.common._i18n import _ +from escalatorclient.openstack.common.apiclient import exceptions + +_logger = logging.getLogger(__name__) +SENSITIVE_HEADERS = ('X-Auth-Token', 'X-Subject-Token',) + + +class HTTPClient(object): + """This client handles sending HTTP requests to OpenStack servers. + + Features: + + - share authentication information between several clients to different + services (e.g., for compute and image clients); + - reissue authentication request for expired tokens; + - encode/decode JSON bodies; + - raise exceptions on HTTP errors; + - pluggable authentication; + - store authentication information in a keyring; + - store time spent for requests; + - register clients for particular services, so one can use + `http_client.identity` or `http_client.compute`; + - log requests and responses in a format that is easy to copy-and-paste + into terminal and send the same request with curl. + """ + + user_agent = "escalatorclient.openstack.common.apiclient" + + def __init__(self, + auth_plugin, + region_name=None, + endpoint_type="publicURL", + original_ip=None, + verify=True, + cert=None, + timeout=None, + timings=False, + keyring_saver=None, + debug=False, + user_agent=None, + http=None): + self.auth_plugin = auth_plugin + + self.endpoint_type = endpoint_type + self.region_name = region_name + + self.original_ip = original_ip + self.timeout = timeout + self.verify = verify + self.cert = cert + + self.keyring_saver = keyring_saver + self.debug = debug + self.user_agent = user_agent or self.user_agent + + self.times = [] # [("item", starttime, endtime), ...] + self.timings = timings + + # requests within the same session can reuse TCP connections from pool + self.http = http or requests.Session() + + self.cached_token = None + self.last_request_id = None + + def _safe_header(self, name, value): + if name in SENSITIVE_HEADERS: + # because in python3 byte string handling is ... ug + v = value.encode('utf-8') + h = hashlib.sha1(v) + d = h.hexdigest() + return encodeutils.safe_decode(name), "{SHA1}%s" % d + else: + return (encodeutils.safe_decode(name), + encodeutils.safe_decode(value)) + + def _http_log_req(self, method, url, kwargs): + if not self.debug: + return + + string_parts = [ + "curl -g -i", + "-X '%s'" % method, + "'%s'" % url, + ] + + for element in kwargs['headers']: + header = ("-H '%s: %s'" % + self._safe_header(element, kwargs['headers'][element])) + string_parts.append(header) + + _logger.debug("REQ: %s" % " ".join(string_parts)) + if 'data' in kwargs: + _logger.debug("REQ BODY: %s\n" % (kwargs['data'])) + + def _http_log_resp(self, resp): + if not self.debug: + return + _logger.debug( + "RESP: [%s] %s\n", + resp.status_code, + resp.headers) + if resp._content_consumed: + _logger.debug( + "RESP BODY: %s\n", + resp.text) + + def serialize(self, kwargs): + if kwargs.get('json') is not None: + kwargs['headers']['Content-Type'] = 'application/json' + kwargs['data'] = json.dumps(kwargs['json']) + try: + del kwargs['json'] + except KeyError: + pass + + def get_timings(self): + return self.times + + def reset_timings(self): + self.times = [] + + def request(self, method, url, **kwargs): + """Send an http request with the specified characteristics. + + Wrapper around `requests.Session.request` to handle tasks such as + setting headers, JSON encoding/decoding, and error handling. + + :param method: method of HTTP request + :param url: URL of HTTP request + :param kwargs: any other parameter that can be passed to + requests.Session.request (such as `headers`) or `json` + that will be encoded as JSON and used as `data` argument + """ + kwargs.setdefault("headers", {}) + kwargs["headers"]["User-Agent"] = self.user_agent + if self.original_ip: + kwargs["headers"]["Forwarded"] = "for=%s;by=%s" % ( + self.original_ip, self.user_agent) + if self.timeout is not None: + kwargs.setdefault("timeout", self.timeout) + kwargs.setdefault("verify", self.verify) + if self.cert is not None: + kwargs.setdefault("cert", self.cert) + self.serialize(kwargs) + + self._http_log_req(method, url, kwargs) + if self.timings: + start_time = time.time() + resp = self.http.request(method, url, **kwargs) + if self.timings: + self.times.append(("%s %s" % (method, url), + start_time, time.time())) + self._http_log_resp(resp) + + self.last_request_id = resp.headers.get('x-openstack-request-id') + + if resp.status_code >= 400: + _logger.debug( + "Request returned failure status: %s", + resp.status_code) + raise exceptions.from_response(resp, method, url) + + return resp + + @staticmethod + def concat_url(endpoint, url): + """Concatenate endpoint and final URL. + + E.g., "http://keystone/v2.0/" and "/tokens" are concatenated to + "http://keystone/v2.0/tokens". + + :param endpoint: the base URL + :param url: the final URL + """ + return "%s/%s" % (endpoint.rstrip("/"), url.strip("/")) + + def client_request(self, client, method, url, **kwargs): + """Send an http request using `client`'s endpoint and specified `url`. + + If request was rejected as unauthorized (possibly because the token is + expired), issue one authorization attempt and send the request once + again. + + :param client: instance of BaseClient descendant + :param method: method of HTTP request + :param url: URL of HTTP request + :param kwargs: any other parameter that can be passed to + `HTTPClient.request` + """ + + filter_args = { + "endpoint_type": client.endpoint_type or self.endpoint_type, + "service_type": client.service_type, + } + token, endpoint = (self.cached_token, client.cached_endpoint) + just_authenticated = False + if not (token and endpoint): + try: + token, endpoint = self.auth_plugin.token_and_endpoint( + **filter_args) + except exceptions.EndpointException: + pass + if not (token and endpoint): + self.authenticate() + just_authenticated = True + token, endpoint = self.auth_plugin.token_and_endpoint( + **filter_args) + if not (token and endpoint): + raise exceptions.AuthorizationFailure( + _("Cannot find endpoint or token for request")) + + old_token_endpoint = (token, endpoint) + kwargs.setdefault("headers", {})["X-Auth-Token"] = token + self.cached_token = token + client.cached_endpoint = endpoint + # Perform the request once. If we get Unauthorized, then it + # might be because the auth token expired, so try to + # re-authenticate and try again. If it still fails, bail. + try: + return self.request( + method, self.concat_url(endpoint, url), **kwargs) + except exceptions.Unauthorized as unauth_ex: + if just_authenticated: + raise + self.cached_token = None + client.cached_endpoint = None + if self.auth_plugin.opts.get('token'): + self.auth_plugin.opts['token'] = None + if self.auth_plugin.opts.get('endpoint'): + self.auth_plugin.opts['endpoint'] = None + self.authenticate() + try: + token, endpoint = self.auth_plugin.token_and_endpoint( + **filter_args) + except exceptions.EndpointException: + raise unauth_ex + if (not (token and endpoint) or + old_token_endpoint == (token, endpoint)): + raise unauth_ex + self.cached_token = token + client.cached_endpoint = endpoint + kwargs["headers"]["X-Auth-Token"] = token + return self.request( + method, self.concat_url(endpoint, url), **kwargs) + + def add_client(self, base_client_instance): + """Add a new instance of :class:`BaseClient` descendant. + + `self` will store a reference to `base_client_instance`. + + Example: + + >>> def test_clients(): + ... from keystoneclient.auth import keystone + ... from openstack.common.apiclient import client + ... auth = keystone.KeystoneAuthPlugin( + ... username="user", password="pass", tenant_name="tenant", + ... auth_url="http://auth:5000/v2.0") + ... openstack_client = client.HTTPClient(auth) + ... # create nova client + ... from novaclient.v1_1 import client + ... client.Client(openstack_client) + ... # create keystone client + ... from keystoneclient.v2_0 import client + ... client.Client(openstack_client) + ... # use them + ... openstack_client.identity.tenants.list() + ... openstack_client.compute.servers.list() + """ + service_type = base_client_instance.service_type + if service_type and not hasattr(self, service_type): + setattr(self, service_type, base_client_instance) + + def authenticate(self): + self.auth_plugin.authenticate(self) + # Store the authentication results in the keyring for later requests + if self.keyring_saver: + self.keyring_saver.save(self) + + +class BaseClient(object): + """Top-level object to access the OpenStack API. + + This client uses :class:`HTTPClient` to send requests. :class:`HTTPClient` + will handle a bunch of issues such as authentication. + """ + + service_type = None + endpoint_type = None # "publicURL" will be used + cached_endpoint = None + + def __init__(self, http_client, extensions=None): + self.http_client = http_client + http_client.add_client(self) + + # Add in any extensions... + if extensions: + for extension in extensions: + if extension.manager_class: + setattr(self, extension.name, + extension.manager_class(self)) + + def client_request(self, method, url, **kwargs): + return self.http_client.client_request( + self, method, url, **kwargs) + + @property + def last_request_id(self): + return self.http_client.last_request_id + + def head(self, url, **kwargs): + return self.client_request("HEAD", url, **kwargs) + + def get(self, url, **kwargs): + return self.client_request("GET", url, **kwargs) + + def post(self, url, **kwargs): + return self.client_request("POST", url, **kwargs) + + def put(self, url, **kwargs): + return self.client_request("PUT", url, **kwargs) + + def delete(self, url, **kwargs): + return self.client_request("DELETE", url, **kwargs) + + def patch(self, url, **kwargs): + return self.client_request("PATCH", url, **kwargs) + + @staticmethod + def get_class(api_name, version, version_map): + """Returns the client class for the requested API version + + :param api_name: the name of the API, e.g. 'compute', 'image', etc + :param version: the requested API version + :param version_map: a dict of client classes keyed by version + :rtype: a client class for the requested API version + """ + try: + client_path = version_map[str(version)] + except (KeyError, ValueError): + msg = _("Invalid %(api_name)s client version '%(version)s'. " + "Must be one of: %(version_map)s") % { + 'api_name': api_name, + 'version': version, + 'version_map': ', '.join(version_map.keys())} + raise exceptions.UnsupportedVersion(msg) + + return importutils.import_class(client_path) diff --git a/client/escalatorclient/openstack/common/apiclient/exceptions.py b/client/escalatorclient/openstack/common/apiclient/exceptions.py new file mode 100644 index 0000000..bcda21d --- /dev/null +++ b/client/escalatorclient/openstack/common/apiclient/exceptions.py @@ -0,0 +1,479 @@ +# Copyright 2010 Jacob Kaplan-Moss +# Copyright 2011 Nebula, Inc. +# Copyright 2013 Alessio Ababilov +# Copyright 2013 OpenStack Foundation +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +""" +Exception definitions. +""" + +######################################################################## +# +# THIS MODULE IS DEPRECATED +# +# Please refer to +# https://etherpad.openstack.org/p/kilo-escalatorclient-library-proposals for +# the discussion leading to this deprecation. +# +# We recommend checking out the python-openstacksdk project +# (https://launchpad.net/python-openstacksdk) instead. +# +######################################################################## + +import inspect +import sys + +import six + +from escalatorclient.openstack.common._i18n import _ + + +class ClientException(Exception): + """The base exception class for all exceptions this library raises. + """ + pass + + +class ValidationError(ClientException): + """Error in validation on API client side.""" + pass + + +class UnsupportedVersion(ClientException): + """User is trying to use an unsupported version of the API.""" + pass + + +class CommandError(ClientException): + """Error in CLI tool.""" + pass + + +class AuthorizationFailure(ClientException): + """Cannot authorize API client.""" + pass + + +class ConnectionError(ClientException): + """Cannot connect to API service.""" + pass + + +class ConnectionRefused(ConnectionError): + """Connection refused while trying to connect to API service.""" + pass + + +class AuthPluginOptionsMissing(AuthorizationFailure): + """Auth plugin misses some options.""" + def __init__(self, opt_names): + super(AuthPluginOptionsMissing, self).__init__( + _("Authentication failed. Missing options: %s") % + ", ".join(opt_names)) + self.opt_names = opt_names + + +class AuthSystemNotFound(AuthorizationFailure): + """User has specified an AuthSystem that is not installed.""" + def __init__(self, auth_system): + super(AuthSystemNotFound, self).__init__( + _("AuthSystemNotFound: %r") % auth_system) + self.auth_system = auth_system + + +class NoUniqueMatch(ClientException): + """Multiple entities found instead of one.""" + pass + + +class EndpointException(ClientException): + """Something is rotten in Service Catalog.""" + pass + + +class EndpointNotFound(EndpointException): + """Could not find requested endpoint in Service Catalog.""" + pass + + +class AmbiguousEndpoints(EndpointException): + """Found more than one matching endpoint in Service Catalog.""" + def __init__(self, endpoints=None): + super(AmbiguousEndpoints, self).__init__( + _("AmbiguousEndpoints: %r") % endpoints) + self.endpoints = endpoints + + +class HttpError(ClientException): + """The base exception class for all HTTP exceptions. + """ + http_status = 0 + message = _("HTTP Error") + + def __init__(self, message=None, details=None, + response=None, request_id=None, + url=None, method=None, http_status=None): + self.http_status = http_status or self.http_status + self.message = message or self.message + self.details = details + self.request_id = request_id + self.response = response + self.url = url + self.method = method + formatted_string = "%s (HTTP %s)" % (self.message, self.http_status) + if request_id: + formatted_string += " (Request-ID: %s)" % request_id + super(HttpError, self).__init__(formatted_string) + + +class HTTPRedirection(HttpError): + """HTTP Redirection.""" + message = _("HTTP Redirection") + + +class HTTPClientError(HttpError): + """Client-side HTTP error. + + Exception for cases in which the client seems to have erred. + """ + message = _("HTTP Client Error") + + +class HttpServerError(HttpError): + """Server-side HTTP error. + + Exception for cases in which the server is aware that it has + erred or is incapable of performing the request. + """ + message = _("HTTP Server Error") + + +class MultipleChoices(HTTPRedirection): + """HTTP 300 - Multiple Choices. + + Indicates multiple options for the resource that the client may follow. + """ + + http_status = 300 + message = _("Multiple Choices") + + +class BadRequest(HTTPClientError): + """HTTP 400 - Bad Request. + + The request cannot be fulfilled due to bad syntax. + """ + http_status = 400 + message = _("Bad Request") + + +class Unauthorized(HTTPClientError): + """HTTP 401 - Unauthorized. + + Similar to 403 Forbidden, but specifically for use when authentication + is required and has failed or has not yet been provided. + """ + http_status = 401 + message = _("Unauthorized") + + +class PaymentRequired(HTTPClientError): + """HTTP 402 - Payment Required. + + Reserved for future use. + """ + http_status = 402 + message = _("Payment Required") + + +class Forbidden(HTTPClientError): + """HTTP 403 - Forbidden. + + The request was a valid request, but the server is refusing to respond + to it. + """ + http_status = 403 + message = _("Forbidden") + + +class NotFound(HTTPClientError): + """HTTP 404 - Not Found. + + The requested resource could not be found but may be available again + in the future. + """ + http_status = 404 + message = _("Not Found") + + +class MethodNotAllowed(HTTPClientError): + """HTTP 405 - Method Not Allowed. + + A request was made of a resource using a request method not supported + by that resource. + """ + http_status = 405 + message = _("Method Not Allowed") + + +class NotAcceptable(HTTPClientError): + """HTTP 406 - Not Acceptable. + + The requested resource is only capable of generating content not + acceptable according to the Accept headers sent in the request. + """ + http_status = 406 + message = _("Not Acceptable") + + +class ProxyAuthenticationRequired(HTTPClientError): + """HTTP 407 - Proxy Authentication Required. + + The client must first authenticate itself with the proxy. + """ + http_status = 407 + message = _("Proxy Authentication Required") + + +class RequestTimeout(HTTPClientError): + """HTTP 408 - Request Timeout. + + The server timed out waiting for the request. + """ + http_status = 408 + message = _("Request Timeout") + + +class Conflict(HTTPClientError): + """HTTP 409 - Conflict. + + Indicates that the request could not be processed because of conflict + in the request, such as an edit conflict. + """ + http_status = 409 + message = _("Conflict") + + +class Gone(HTTPClientError): + """HTTP 410 - Gone. + + Indicates that the resource requested is no longer available and will + not be available again. + """ + http_status = 410 + message = _("Gone") + + +class LengthRequired(HTTPClientError): + """HTTP 411 - Length Required. + + The request did not specify the length of its content, which is + required by the requested resource. + """ + http_status = 411 + message = _("Length Required") + + +class PreconditionFailed(HTTPClientError): + """HTTP 412 - Precondition Failed. + + The server does not meet one of the preconditions that the requester + put on the request. + """ + http_status = 412 + message = _("Precondition Failed") + + +class RequestEntityTooLarge(HTTPClientError): + """HTTP 413 - Request Entity Too Large. + + The request is larger than the server is willing or able to process. + """ + http_status = 413 + message = _("Request Entity Too Large") + + def __init__(self, *args, **kwargs): + try: + self.retry_after = int(kwargs.pop('retry_after')) + except (KeyError, ValueError): + self.retry_after = 0 + + super(RequestEntityTooLarge, self).__init__(*args, **kwargs) + + +class RequestUriTooLong(HTTPClientError): + """HTTP 414 - Request-URI Too Long. + + The URI provided was too long for the server to process. + """ + http_status = 414 + message = _("Request-URI Too Long") + + +class UnsupportedMediaType(HTTPClientError): + """HTTP 415 - Unsupported Media Type. + + The request entity has a media type which the server or resource does + not support. + """ + http_status = 415 + message = _("Unsupported Media Type") + + +class RequestedRangeNotSatisfiable(HTTPClientError): + """HTTP 416 - Requested Range Not Satisfiable. + + The client has asked for a portion of the file, but the server cannot + supply that portion. + """ + http_status = 416 + message = _("Requested Range Not Satisfiable") + + +class ExpectationFailed(HTTPClientError): + """HTTP 417 - Expectation Failed. + + The server cannot meet the requirements of the Expect request-header field. + """ + http_status = 417 + message = _("Expectation Failed") + + +class UnprocessableEntity(HTTPClientError): + """HTTP 422 - Unprocessable Entity. + + The request was well-formed but was unable to be followed due to semantic + errors. + """ + http_status = 422 + message = _("Unprocessable Entity") + + +class InternalServerError(HttpServerError): + """HTTP 500 - Internal Server Error. + + A generic error message, given when no more specific message is suitable. + """ + http_status = 500 + message = _("Internal Server Error") + + +# NotImplemented is a python keyword. +class HttpNotImplemented(HttpServerError): + """HTTP 501 - Not Implemented. + + The server either does not recognize the request method, or it lacks + the ability to fulfill the request. + """ + http_status = 501 + message = _("Not Implemented") + + +class BadGateway(HttpServerError): + """HTTP 502 - Bad Gateway. + + The server was acting as a gateway or proxy and received an invalid + response from the upstream server. + """ + http_status = 502 + message = _("Bad Gateway") + + +class ServiceUnavailable(HttpServerError): + """HTTP 503 - Service Unavailable. + + The server is currently unavailable. + """ + http_status = 503 + message = _("Service Unavailable") + + +class GatewayTimeout(HttpServerError): + """HTTP 504 - Gateway Timeout. + + The server was acting as a gateway or proxy and did not receive a timely + response from the upstream server. + """ + http_status = 504 + message = _("Gateway Timeout") + + +class HttpVersionNotSupported(HttpServerError): + """HTTP 505 - HttpVersion Not Supported. + + The server does not support the HTTP protocol version used in the request. + """ + http_status = 505 + message = _("HTTP Version Not Supported") + + +# _code_map contains all the classes that have http_status attribute. +_code_map = dict( + (getattr(obj, 'http_status', None), obj) + for name, obj in six.iteritems(vars(sys.modules[__name__])) + if inspect.isclass(obj) and getattr(obj, 'http_status', False) +) + + +def from_response(response, method, url): + """Returns an instance of :class:`HttpError` or subclass based on response. + + :param response: instance of `requests.Response` class + :param method: HTTP method used for request + :param url: URL used for request + """ + + req_id = response.headers.get("x-openstack-request-id") + # NOTE(hdd) true for older versions of nova and cinder + if not req_id: + req_id = response.headers.get("x-compute-request-id") + kwargs = { + "http_status": response.status_code, + "response": response, + "method": method, + "url": url, + "request_id": req_id, + } + if "retry-after" in response.headers: + kwargs["retry_after"] = response.headers["retry-after"] + + content_type = response.headers.get("Content-Type", "") + if content_type.startswith("application/json"): + try: + body = response.json() + except ValueError: + pass + else: + if isinstance(body, dict): + error = body.get(list(body)[0]) + if isinstance(error, dict): + kwargs["message"] = (error.get("message") or + error.get("faultstring")) + kwargs["details"] = (error.get("details") or + six.text_type(body)) + elif content_type.startswith("text/"): + kwargs["details"] = response.text + + try: + cls = _code_map[response.status_code] + except KeyError: + if 500 <= response.status_code < 600: + cls = HttpServerError + elif 400 <= response.status_code < 500: + cls = HTTPClientError + else: + cls = HttpError + return cls(**kwargs) diff --git a/client/escalatorclient/openstack/common/apiclient/utils.py b/client/escalatorclient/openstack/common/apiclient/utils.py new file mode 100644 index 0000000..c0f612a --- /dev/null +++ b/client/escalatorclient/openstack/common/apiclient/utils.py @@ -0,0 +1,100 @@ +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +######################################################################## +# +# THIS MODULE IS DEPRECATED +# +# Please refer to +# https://etherpad.openstack.org/p/kilo-escalatorclient-library-proposals for +# the discussion leading to this deprecation. +# +# We recommend checking out the python-openstacksdk project +# (https://launchpad.net/python-openstacksdk) instead. +# +######################################################################## + +from oslo_utils import encodeutils +from oslo_utils import uuidutils +import six + +from escalatorclient.openstack.common._i18n import _ +from escalatorclient.openstack.common.apiclient import exceptions + + +def find_resource(manager, name_or_id, **find_args): + """Look for resource in a given manager. + + Used as a helper for the _find_* methods. + Example: + + .. code-block:: python + + def _find_hypervisor(cs, hypervisor): + #Get a hypervisor by name or ID. + return cliutils.find_resource(cs.hypervisors, hypervisor) + """ + # first try to get entity as integer id + try: + return manager.get(int(name_or_id)) + except (TypeError, ValueError, exceptions.NotFound): + pass + + # now try to get entity as uuid + try: + if six.PY2: + tmp_id = encodeutils.safe_encode(name_or_id) + else: + tmp_id = encodeutils.safe_decode(name_or_id) + + if uuidutils.is_uuid_like(tmp_id): + return manager.get(tmp_id) + except (TypeError, ValueError, exceptions.NotFound): + pass + + # for str id which is not uuid + if getattr(manager, 'is_alphanum_id_allowed', False): + try: + return manager.get(name_or_id) + except exceptions.NotFound: + pass + + try: + try: + return manager.find(human_id=name_or_id, **find_args) + except exceptions.NotFound: + pass + + # finally try to find entity by name + try: + resource = getattr(manager, 'resource_class', None) + name_attr = resource.NAME_ATTR if resource else 'name' + kwargs = {name_attr: name_or_id} + kwargs.update(find_args) + return manager.find(**kwargs) + except exceptions.NotFound: + msg = _("No %(name)s with a name or " + "ID of '%(name_or_id)s' exists.") % \ + { + "name": manager.resource_class.__name__.lower(), + "name_or_id": name_or_id + } + raise exceptions.CommandError(msg) + except exceptions.NoUniqueMatch: + msg = _("Multiple %(name)s matches found for " + "'%(name_or_id)s', use an ID to be more specific.") % \ + { + "name": manager.resource_class.__name__.lower(), + "name_or_id": name_or_id + } + raise exceptions.CommandError(msg) diff --git a/client/escalatorclient/shell.py b/client/escalatorclient/shell.py new file mode 100644 index 0000000..8f452b0 --- /dev/null +++ b/client/escalatorclient/shell.py @@ -0,0 +1,713 @@ +# Copyright 2012 OpenStack Foundation +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +""" +Command-line interface to the OpenStack Images API. +""" + +from __future__ import print_function + +import argparse +import copy +import getpass +import json +import logging +import os +from os.path import expanduser +import sys +import traceback + +from oslo_utils import encodeutils +from oslo_utils import importutils +import six.moves.urllib.parse as urlparse + +import escalatorclient +from escalatorclient import _i18n +from escalatorclient.common import utils +from escalatorclient import exc + +from keystoneclient.auth.identity import v2 as v2_auth +from keystoneclient.auth.identity import v3 as v3_auth +from keystoneclient import discover +from keystoneclient.openstack.common.apiclient import exceptions as ks_exc +from keystoneclient import session + +osprofiler_profiler = importutils.try_import("osprofiler.profiler") +_ = _i18n._ + + +class escalatorShell(object): + + def _append_global_identity_args(self, parser): + # FIXME(bobt): these are global identity (Keystone) arguments which + # should be consistent and shared by all service clients. Therefore, + # they should be provided by python-keystoneclient. We will need to + # refactor this code once this functionality is avaible in + # python-keystoneclient. See + # + # https://bugs.launchpad.net/python-keystoneclient/+bug/1332337 + # + parser.add_argument('-k', '--insecure', + default=False, + action='store_true', + help='Explicitly allow escalatorclient to perform ' + '\"insecure SSL\" (https) requests. The server\'s ' + 'certificate will not be verified against any ' + 'certificate authorities. This option should ' + 'be used with caution.') + + parser.add_argument('--os-cert', + help='Path of certificate file to use in SSL ' + 'connection. This file can optionally be ' + 'prepended with the private key.') + + parser.add_argument('--cert-file', + dest='os_cert', + help='DEPRECATED! Use --os-cert.') + + parser.add_argument('--os-key', + help='Path of client key to use in SSL ' + 'connection. This option is not necessary ' + 'if your key is prepended to your cert file.') + + parser.add_argument('--key-file', + dest='os_key', + help='DEPRECATED! Use --os-key.') + + parser.add_argument('--os-cacert', + metavar='<ca-certificate-file>', + dest='os_cacert', + default=utils.env('OS_CACERT'), + help='Path of CA TLS certificate(s) used to ' + 'verify the remote server\'s certificate. ' + 'Without this option escalator looks for the ' + 'default system CA certificates.') + + parser.add_argument('--ca-file', + dest='os_cacert', + help='DEPRECATED! Use --os-cacert.') + + parser.add_argument('--os-username', + default=utils.env('OS_USERNAME'), + help='Defaults to env[OS_USERNAME].') + + parser.add_argument('--os_username', + help=argparse.SUPPRESS) + + parser.add_argument('--os-user-id', + default=utils.env('OS_USER_ID'), + help='Defaults to env[OS_USER_ID].') + + parser.add_argument('--os-user-domain-id', + default=utils.env('OS_USER_DOMAIN_ID'), + help='Defaults to env[OS_USER_DOMAIN_ID].') + + parser.add_argument('--os-user-domain-name', + default=utils.env('OS_USER_DOMAIN_NAME'), + help='Defaults to env[OS_USER_DOMAIN_NAME].') + + parser.add_argument('--os-project-id', + default=utils.env('OS_PROJECT_ID'), + help='Another way to specify tenant ID. ' + 'This option is mutually exclusive with ' + ' --os-tenant-id. ' + 'Defaults to env[OS_PROJECT_ID].') + + parser.add_argument('--os-project-name', + default=utils.env('OS_PROJECT_NAME'), + help='Another way to specify tenant name. ' + 'This option is mutually exclusive with ' + ' --os-tenant-name. ' + 'Defaults to env[OS_PROJECT_NAME].') + + parser.add_argument('--os-project-domain-id', + default=utils.env('OS_PROJECT_DOMAIN_ID'), + help='Defaults to env[OS_PROJECT_DOMAIN_ID].') + + parser.add_argument('--os-project-domain-name', + default=utils.env('OS_PROJECT_DOMAIN_NAME'), + help='Defaults to env[OS_PROJECT_DOMAIN_NAME].') + + parser.add_argument('--os-password', + default=utils.env('OS_PASSWORD'), + help='Defaults to env[OS_PASSWORD].') + + parser.add_argument('--os_password', + help=argparse.SUPPRESS) + + parser.add_argument('--os-tenant-id', + default=utils.env('OS_TENANT_ID'), + help='Defaults to env[OS_TENANT_ID].') + + parser.add_argument('--os_tenant_id', + help=argparse.SUPPRESS) + + parser.add_argument('--os-tenant-name', + default=utils.env('OS_TENANT_NAME'), + help='Defaults to env[OS_TENANT_NAME].') + + parser.add_argument('--os_tenant_name', + help=argparse.SUPPRESS) + + parser.add_argument('--os-auth-url', + default=utils.env('OS_AUTH_URL'), + help='Defaults to env[OS_AUTH_URL].') + + parser.add_argument('--os_auth_url', + help=argparse.SUPPRESS) + + parser.add_argument('--os-region-name', + default=utils.env('OS_REGION_NAME'), + help='Defaults to env[OS_REGION_NAME].') + + parser.add_argument('--os_region_name', + help=argparse.SUPPRESS) + + parser.add_argument('--os-auth-token', + default=utils.env('OS_AUTH_TOKEN'), + help='Defaults to env[OS_AUTH_TOKEN].') + + parser.add_argument('--os_auth_token', + help=argparse.SUPPRESS) + + parser.add_argument('--os-service-type', + default=utils.env('OS_SERVICE_TYPE'), + help='Defaults to env[OS_SERVICE_TYPE].') + + parser.add_argument('--os_service_type', + help=argparse.SUPPRESS) + + parser.add_argument('--os-endpoint-type', + default=utils.env('OS_ENDPOINT_TYPE'), + help='Defaults to env[OS_ENDPOINT_TYPE].') + + parser.add_argument('--os_endpoint_type', + help=argparse.SUPPRESS) + + parser.add_argument('--os-endpoint', + default=utils.env('OS_ENDPOINT'), + help='Defaults to env[OS_ENDPOINT].') + + parser.add_argument('--os_endpoint', + help=argparse.SUPPRESS) + + def get_base_parser(self): + parser = argparse.ArgumentParser( + prog='escalator', + description=__doc__.strip(), + epilog='See "escalator help COMMAND" ' + 'for help on a specific command.', + add_help=False, + formatter_class=HelpFormatter, + ) + + # Global arguments + parser.add_argument('-h', '--help', + action='store_true', + help=argparse.SUPPRESS, + ) + + parser.add_argument('-d', '--debug', + default=bool(utils.env('ESCALATORCLIENT_DEBUG')), + action='store_true', + help='Defaults to env[ESCALATORCLIENT_DEBUG].') + + parser.add_argument('-v', '--verbose', + default=False, action="store_true", + help="Print more verbose output") + + parser.add_argument('--get-schema', + default=False, action="store_true", + dest='get_schema', + help='Ignores cached copy and forces retrieval ' + 'of schema that generates portions of the ' + 'help text. Ignored with API version 1.') + + parser.add_argument('--timeout', + default=600, + help='Number of seconds to wait for a response') + + parser.add_argument('--no-ssl-compression', + dest='ssl_compression', + default=True, action='store_false', + help='Disable SSL compression when using https.') + + parser.add_argument('-f', '--force', + dest='force', + default=False, action='store_true', + help='Prevent select actions from requesting ' + 'user confirmation.') + + parser.add_argument('--os-image-url', + default=utils.env('OS_IMAGE_URL'), + help=('Defaults to env[OS_IMAGE_URL]. ' + 'If the provided image url contains ' + 'a version number and ' + '`--os-image-api-version` is omitted ' + 'the version of the URL will be picked as ' + 'the image api version to use.')) + + parser.add_argument('--os_image_url', + help=argparse.SUPPRESS) + + parser.add_argument('--os-image-api-version', + default=utils.env('OS_IMAGE_API_VERSION', + default=None), + help='Defaults to env[OS_IMAGE_API_VERSION] or 1.') + + parser.add_argument('--os_image_api_version', + help=argparse.SUPPRESS) + + if osprofiler_profiler: + parser.add_argument('--profile', + metavar='HMAC_KEY', + help='HMAC key to use for encrypting context ' + 'data for performance profiling of operation. ' + 'This key should be the value of HMAC key ' + 'configured in osprofiler middleware in ' + 'escalator, it is specified in paste ' + 'configuration file at ' + '/etc/escalator/api-paste.ini and ' + '/etc/escalator/registry-paste.ini. Without key ' + 'the profiling will not be triggered even ' + 'if osprofiler is enabled on server side.') + + # FIXME(bobt): this method should come from python-keystoneclient + self._append_global_identity_args(parser) + + return parser + + def get_subcommand_parser(self, version): + parser = self.get_base_parser() + + self.subcommands = {} + subparsers = parser.add_subparsers(metavar='<subcommand>') + try: + submodule = utils.import_versioned_module(version, 'shell') + except ImportError: + print('"%s" is not a supported API version. Example ' + 'values are "1" or "2".' % version) + utils.exit() + + self._find_actions(subparsers, submodule) + self._find_actions(subparsers, self) + + self._add_bash_completion_subparser(subparsers) + + return parser + + def _find_actions(self, subparsers, actions_module): + for attr in (a for a in dir(actions_module) if a.startswith('do_')): + # I prefer to be hypen-separated instead of underscores. + command = attr[3:].replace('_', '-') + callback = getattr(actions_module, attr) + desc = callback.__doc__ or '' + help = desc.strip().split('\n')[0] + arguments = getattr(callback, 'arguments', []) + + subparser = subparsers.add_parser(command, + help=help, + description=desc, + add_help=False, + formatter_class=HelpFormatter + ) + subparser.add_argument('-h', '--help', + action='help', + help=argparse.SUPPRESS, + ) + self.subcommands[command] = subparser + for (args, kwargs) in arguments: + subparser.add_argument(*args, **kwargs) + subparser.set_defaults(func=callback) + + def _add_bash_completion_subparser(self, subparsers): + subparser = subparsers.add_parser('bash_completion', + add_help=False, + formatter_class=HelpFormatter) + self.subcommands['bash_completion'] = subparser + subparser.set_defaults(func=self.do_bash_completion) + + def _get_image_url(self, args): + """Translate the available url-related options into a single string. + + Return the endpoint that should be used to talk to escalator if a + clear decision can be made. Otherwise, return None. + """ + if args.os_image_url: + return args.os_image_url + else: + return None + + def _discover_auth_versions(self, session, auth_url): + # discover the API versions the server is supporting base on the + # given URL + v2_auth_url = None + v3_auth_url = None + try: + ks_discover = discover.Discover(session=session, auth_url=auth_url) + v2_auth_url = ks_discover.url_for('2.0') + v3_auth_url = ks_discover.url_for('3.0') + except ks_exc.ClientException as e: + # Identity service may not support discover API version. + # Lets trying to figure out the API version from the original URL. + url_parts = urlparse.urlparse(auth_url) + (scheme, netloc, path, params, query, fragment) = url_parts + path = path.lower() + if path.startswith('/v3'): + v3_auth_url = auth_url + elif path.startswith('/v2'): + v2_auth_url = auth_url + else: + # not enough information to determine the auth version + msg = ('Unable to determine the Keystone version ' + 'to authenticate with using the given ' + 'auth_url. Identity service may not support API ' + 'version discovery. Please provide a versioned ' + 'auth_url instead. error=%s') % (e) + raise exc.CommandError(msg) + + return (v2_auth_url, v3_auth_url) + + def _get_keystone_session(self, **kwargs): + ks_session = session.Session.construct(kwargs) + + # discover the supported keystone versions using the given auth url + auth_url = kwargs.pop('auth_url', None) + (v2_auth_url, v3_auth_url) = self._discover_auth_versions( + session=ks_session, + auth_url=auth_url) + + # Determine which authentication plugin to use. First inspect the + # auth_url to see the supported version. If both v3 and v2 are + # supported, then use the highest version if possible. + user_id = kwargs.pop('user_id', None) + username = kwargs.pop('username', None) + password = kwargs.pop('password', None) + user_domain_name = kwargs.pop('user_domain_name', None) + user_domain_id = kwargs.pop('user_domain_id', None) + # project and tenant can be used interchangeably + project_id = (kwargs.pop('project_id', None) or + kwargs.pop('tenant_id', None)) + project_name = (kwargs.pop('project_name', None) or + kwargs.pop('tenant_name', None)) + project_domain_id = kwargs.pop('project_domain_id', None) + project_domain_name = kwargs.pop('project_domain_name', None) + auth = None + + use_domain = (user_domain_id or + user_domain_name or + project_domain_id or + project_domain_name) + use_v3 = v3_auth_url and (use_domain or (not v2_auth_url)) + use_v2 = v2_auth_url and not use_domain + + if use_v3: + auth = v3_auth.Password( + v3_auth_url, + user_id=user_id, + username=username, + password=password, + user_domain_id=user_domain_id, + user_domain_name=user_domain_name, + project_id=project_id, + project_name=project_name, + project_domain_id=project_domain_id, + project_domain_name=project_domain_name) + elif use_v2: + auth = v2_auth.Password( + v2_auth_url, + username, + password, + tenant_id=project_id, + tenant_name=project_name) + else: + # if we get here it means domain information is provided + # (caller meant to use Keystone V3) but the auth url is + # actually Keystone V2. Obviously we can't authenticate a V3 + # user using V2. + exc.CommandError("Credential and auth_url mismatch. The given " + "auth_url is using Keystone V2 endpoint, which " + "may not able to handle Keystone V3 credentials. " + "Please provide a correct Keystone V3 auth_url.") + + ks_session.auth = auth + return ks_session + + def _get_endpoint_and_token(self, args, force_auth=False): + image_url = self._get_image_url(args) + auth_token = args.os_auth_token + + auth_reqd = force_auth or\ + (utils.is_authentication_required(args.func) and not + (auth_token and image_url)) + + if not auth_reqd: + endpoint = image_url + token = args.os_auth_token + else: + + if not args.os_username: + raise exc.CommandError( + _("You must provide a username via" + " either --os-username or " + "env[OS_USERNAME]")) + + if not args.os_password: + # No password, If we've got a tty, try prompting for it + if hasattr(sys.stdin, 'isatty') and sys.stdin.isatty(): + # Check for Ctl-D + try: + args.os_password = getpass.getpass('OS Password: ') + except EOFError: + pass + # No password because we didn't have a tty or the + # user Ctl-D when prompted. + if not args.os_password: + raise exc.CommandError( + _("You must provide a password via " + "either --os-password, " + "env[OS_PASSWORD], " + "or prompted response")) + + # Validate password flow auth + project_info = ( + args.os_tenant_name or args.os_tenant_id or ( + args.os_project_name and ( + args.os_project_domain_name or + args.os_project_domain_id + ) + ) or args.os_project_id + ) + + if not project_info: + # tenant is deprecated in Keystone v3. Use the latest + # terminology instead. + raise exc.CommandError( + _("You must provide a project_id or project_name (" + "with project_domain_name or project_domain_id) " + "via " + " --os-project-id (env[OS_PROJECT_ID])" + " --os-project-name (env[OS_PROJECT_NAME])," + " --os-project-domain-id " + "(env[OS_PROJECT_DOMAIN_ID])" + " --os-project-domain-name " + "(env[OS_PROJECT_DOMAIN_NAME])")) + + if not args.os_auth_url: + raise exc.CommandError( + _("You must provide an auth url via" + " either --os-auth-url or " + "via env[OS_AUTH_URL]")) + + kwargs = { + 'auth_url': args.os_auth_url, + 'username': args.os_username, + 'user_id': args.os_user_id, + 'user_domain_id': args.os_user_domain_id, + 'user_domain_name': args.os_user_domain_name, + 'password': args.os_password, + 'tenant_name': args.os_tenant_name, + 'tenant_id': args.os_tenant_id, + 'project_name': args.os_project_name, + 'project_id': args.os_project_id, + 'project_domain_name': args.os_project_domain_name, + 'project_domain_id': args.os_project_domain_id, + 'insecure': args.insecure, + 'cacert': args.os_cacert, + 'cert': args.os_cert, + 'key': args.os_key + } + ks_session = self._get_keystone_session(**kwargs) + token = args.os_auth_token or ks_session.get_token() + + endpoint_type = args.os_endpoint_type or 'public' + service_type = args.os_service_type or 'image' + endpoint = args.os_image_url or ks_session.get_endpoint( + service_type=service_type, + interface=endpoint_type, + region_name=args.os_region_name) + + return endpoint, token + + def _get_versioned_client(self, api_version, args, force_auth=False): + # ndpoint, token = self._get_endpoint_and_token( + # args,force_auth=force_auth) + # endpoint = "http://10.43.175.62:19292" + endpoint = args.os_endpoint + # print endpoint + kwargs = { + # 'token': token, + 'insecure': args.insecure, + 'timeout': args.timeout, + 'cacert': args.os_cacert, + 'cert': args.os_cert, + 'key': args.os_key, + 'ssl_compression': args.ssl_compression + } + client = escalatorclient.Client(api_version, endpoint, **kwargs) + return client + + def _cache_schemas(self, options, home_dir='~/.escalatorclient'): + homedir = expanduser(home_dir) + if not os.path.exists(homedir): + os.makedirs(homedir) + + resources = ['image', 'metadefs/namespace', 'metadefs/resource_type'] + schema_file_paths = [homedir + os.sep + x + '_schema.json' + for x in ['image', 'namespace', 'resource_type']] + + client = None + for resource, schema_file_path in zip(resources, schema_file_paths): + if (not os.path.exists(schema_file_path)) or options.get_schema: + try: + if not client: + client = self._get_versioned_client('2', options, + force_auth=True) + schema = client.schemas.get(resource) + + with open(schema_file_path, 'w') as f: + f.write(json.dumps(schema.raw())) + except Exception: + # NOTE(esheffield) do nothing here, we'll get a message + # later if the schema is missing + pass + + def main(self, argv): + # Parse args once to find version + + # NOTE(flepied) Under Python3, parsed arguments are removed + # from the list so make a copy for the first parsing + base_argv = copy.deepcopy(argv) + parser = self.get_base_parser() + (options, args) = parser.parse_known_args(base_argv) + + try: + # NOTE(flaper87): Try to get the version from the + # image-url first. If no version was specified, fallback + # to the api-image-version arg. If both of these fail then + # fallback to the minimum supported one and let keystone + # do the magic. + endpoint = self._get_image_url(options) + endpoint, url_version = utils.strip_version(endpoint) + except ValueError: + # NOTE(flaper87): ValueError is raised if no endpoint is povided + url_version = None + + # build available subcommands based on version + try: + api_version = int(options.os_image_api_version or url_version or 1) + except ValueError: + print("Invalid API version parameter") + utils.exit() + + if api_version == 2: + self._cache_schemas(options) + + subcommand_parser = self.get_subcommand_parser(api_version) + self.parser = subcommand_parser + + # Handle top-level --help/-h before attempting to parse + # a command off the command line + if options.help or not argv: + self.do_help(options) + return 0 + + # Parse args again and call whatever callback was selected + args = subcommand_parser.parse_args(argv) + + # Short-circuit and deal with help command right away. + if args.func == self.do_help: + self.do_help(args) + return 0 + elif args.func == self.do_bash_completion: + self.do_bash_completion(args) + return 0 + + LOG = logging.getLogger('escalatorclient') + LOG.addHandler(logging.StreamHandler()) + LOG.setLevel(logging.DEBUG if args.debug else logging.INFO) + + profile = osprofiler_profiler and options.profile + if profile: + osprofiler_profiler.init(options.profile) + + client = self._get_versioned_client(api_version, args, + force_auth=False) + + try: + args.func(client, args) + except exc.Unauthorized: + raise exc.CommandError("Invalid OpenStack Identity credentials.") + except Exception: + # NOTE(kragniz) Print any exceptions raised to stderr if the + # --debug flag is set + if args.debug: + traceback.print_exc() + raise + finally: + if profile: + trace_id = osprofiler_profiler.get().get_base_id() + print("Profiling trace ID: %s" % trace_id) + print("To display trace use next command:\n" + "osprofiler trace show --html %s " % trace_id) + + @utils.arg('command', metavar='<subcommand>', nargs='?', + help='Display help for <subcommand>.') + def do_help(self, args): + """ + Display help about this program or one of its subcommands. + """ + if getattr(args, 'command', None): + if args.command in self.subcommands: + self.subcommands[args.command].print_help() + else: + raise exc.CommandError("'%s' is not a valid subcommand" % + args.command) + else: + self.parser.print_help() + + def do_bash_completion(self, _args): + """Prints arguments for bash_completion. + + Prints all of the commands and options to stdout so that the + escalator.bash_completion script doesn't have to hard code them. + """ + commands = set() + options = set() + for sc_str, sc in self.subcommands.items(): + commands.add(sc_str) + for option in sc._optionals._option_string_actions.keys(): + options.add(option) + + commands.remove('bash_completion') + commands.remove('bash-completion') + print(' '.join(commands | options)) + + +class HelpFormatter(argparse.HelpFormatter): + + def start_section(self, heading): + # Title-case the headings + heading = '%s%s' % (heading[0].upper(), heading[1:]) + super(HelpFormatter, self).start_section(heading) + + +def main(): + try: + escalatorShell().main(map(encodeutils.safe_decode, sys.argv[1:])) + except KeyboardInterrupt: + utils.exit('... terminating escalator client', exit_code=130) + except Exception as e: + utils.exit(utils.exception_to_str(e)) diff --git a/client/escalatorclient/v1/__init__.py b/client/escalatorclient/v1/__init__.py new file mode 100644 index 0000000..cd35765 --- /dev/null +++ b/client/escalatorclient/v1/__init__.py @@ -0,0 +1,16 @@ +# Copyright 2012 OpenStack Foundation +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +from escalatorclient.v1.client import Client # noqa diff --git a/client/escalatorclient/v1/client.py b/client/escalatorclient/v1/client.py new file mode 100644 index 0000000..f74300f --- /dev/null +++ b/client/escalatorclient/v1/client.py @@ -0,0 +1,36 @@ +# Copyright 2012 OpenStack Foundation +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +from escalatorclient.common import http +from escalatorclient.common import utils +from escalatorclient.v1.versions import VersionManager + + +class Client(object): + """Client for the escalator v1 API. + + :param string endpoint: A user-supplied endpoint URL for the escalator + service. + :param string token: Token for authentication. + :param integer timeout: Allows customization of the timeout for client + http requests. (optional) + """ + + def __init__(self, endpoint, *args, **kwargs): + """Initialize a new client for the escalator v1 API.""" + endpoint, version = utils.strip_version(endpoint) + self.version = version or 1.0 + self.http_client = http.HTTPClient(endpoint, *args, **kwargs) + self.node = VersionManager(self.http_client) diff --git a/client/escalatorclient/v1/shell.py b/client/escalatorclient/v1/shell.py new file mode 100644 index 0000000..9f9ea4f --- /dev/null +++ b/client/escalatorclient/v1/shell.py @@ -0,0 +1,182 @@ +# Copyright 2012 OpenStack Foundation +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +from __future__ import print_function + +import copy +import functools +import pprint +import os +import json + +from oslo_utils import encodeutils +from oslo_utils import strutils +import escalatorclient.v1.versions +from escalatorclient.common import utils +from escalatorclient import exc + +_bool_strict = functools.partial(strutils.bool_from_string, strict=True) + + +def _escalator_show(escalator, max_column_width=80): + info = copy.deepcopy(escalator._info) + exclusive_field = ('deleted', 'deleted_at') + for field in exclusive_field: + if field in info: + info.pop(field) + utils.print_dict(info, max_column_width=max_column_width) + + +@utils.arg('--type', metavar='<TYPE>', + help='Type of escalator version, supported type are "internal": ' + 'the internal version of escalator.') +def do_version(dc, args): + """Get version of escalator.""" + fields = dict(filter(lambda x: x[1] is not None, vars(args).items())) + + # Filter out values we can't use + VERSION_PARAMS = escalatorclient.v1.version.VERSION_PARAMS + fields = dict(filter(lambda x: x[0] in VERSION_PARAMS, fields.items())) + version = dc.version.version(**fields) + _escalator_show(version) + + +@utils.arg('id', metavar='<ID>', + help='Filter version to those that have this id.') +def do_version_detail(dc, args): + """Get backend_types of escalator.""" + version = utils.find_resource(dc.versions, args.id) + _escalator_show(version) + + +@utils.arg('name', metavar='<NAME>', + help='name of version.') +@utils.arg('type', metavar='<TYPE>', + help='version type.eg redhat7.0...') +@utils.arg('--size', metavar='<SIZE>', + help='size of the version file.') +@utils.arg('--checksum', metavar='<CHECKSUM>', + help='md5 of version file') +@utils.arg('--version', metavar='<VERSION>', + help='version number of version file') +@utils.arg('--description', metavar='<DESCRIPTION>', + help='description of version file') +@utils.arg('--status', metavar='<STATUS>', + help='version file status.default:init') +def do_version_add(dc, args): + """Add a version.""" + + fields = dict(filter(lambda x: x[1] is not None, vars(args).items())) + + # Filter out values we can't use + CREATE_PARAMS = escalatorclient.v1.versions.CREATE_PARAMS + fields = dict(filter(lambda x: x[0] in CREATE_PARAMS, fields.items())) + + version = dc.versions.add(**fields) + _escalator_show(version) + + +@utils.arg('id', metavar='<ID>', + help='ID of versions.') +@utils.arg('--name', metavar='<NAME>', + help='name of version.') +@utils.arg('--type', metavar='<TYPE>', + help='version type.eg redhat7.0...') +@utils.arg('--size', metavar='<SIZE>', + help='size of the version file.') +@utils.arg('--checksum', metavar='<CHECKSUM>', + help='md5 of version file') +@utils.arg('--version', metavar='<VERSION>', + help='version number of version file') +@utils.arg('--description', metavar='<DESCRIPTION>', + help='description of version file') +@utils.arg('--status', metavar='<STATUS>', + help='version file status.default:init') +def do_version_update(dc, args): + """Add a version.""" + + fields = dict(filter(lambda x: x[1] is not None, vars(args).items())) + + # Filter out values we can't use + CREATE_PARAMS = escalatorclient.v1.versions.CREATE_PARAMS + fields = dict(filter(lambda x: x[0] in CREATE_PARAMS, fields.items())) + version_id = fields.get('id', None) + version = dc.versions.update(version_id, **fields) + _escalator_show(version) + + +@utils.arg('id', metavar='<ID>', nargs='+', + help='ID of versions.') +def do_version_delete(dc, args): + """Delete specified template(s).""" + fields = dict(filter(lambda x: x[1] is not None, vars(args).items())) + versions = fields.get('id', None) + for version in versions: + try: + if args.verbose: + print('Requesting version delete for %s ...' % + encodeutils.safe_decode(version), end=' ') + dc.versions.delete(version) + if args.verbose: + print('[Done]') + except exc.HTTPException as e: + if args.verbose: + print('[Fail]') + print('%s: Unable to delete version %s' % (e, version)) + + +@utils.arg('--name', metavar='<NAME>', + help='Filter version to those that have this name.') +@utils.arg('--status', metavar='<STATUS>', + help='Filter version status.') +@utils.arg('--type', metavar='<type>', + help='Filter by type.') +@utils.arg('--version', metavar='<version>', + help='Filter by version number.') +@utils.arg('--page-size', metavar='<SIZE>', default=None, type=int, + help='Number to request in each paginated request.') +@utils.arg('--sort-key', default='name', + choices=escalatorclient.v1.versions.SORT_KEY_VALUES, + help='Sort version list by specified field.') +@utils.arg('--sort-dir', default='asc', + choices=escalatorclient.v1.versions.SORT_DIR_VALUES, + help='Sort version list in specified direction.') +def do_version_list(dc, args): + """List hosts you can access.""" + filter_keys = ['name', 'type', 'status', 'version'] + filter_items = [(key, getattr(args, key)) for key in filter_keys] + filters = dict([item for item in filter_items if item[1] is not None]) + + kwargs = {'filters': filters} + if args.page_size is not None: + kwargs['page_size'] = args.page_size + + kwargs['sort_key'] = args.sort_key + kwargs['sort_dir'] = args.sort_dir + + versions = dc.versions.list(**kwargs) + + columns = ['ID', 'NAME', 'TYPE', 'VERSION', 'size', + 'checksum', 'description', 'status', 'VERSION_PATCH'] + + utils.print_list(versions, columns) + + +@utils.arg('id', metavar='<ID>', + help='Filter version patch to those that have this id.') +def do_version_patch_detail(dc, args): + """Get version_patch of escalator.""" + version = utils.find_resource(dc.version_patchs, args.id) + _escalator_show(version) diff --git a/client/escalatorclient/v1/versions.py b/client/escalatorclient/v1/versions.py new file mode 100644 index 0000000..f54ea23 --- /dev/null +++ b/client/escalatorclient/v1/versions.py @@ -0,0 +1,294 @@ +# Copyright 2012 OpenStack Foundation +# All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +import copy + +from oslo_utils import encodeutils +from oslo_utils import strutils +import six +import six.moves.urllib.parse as urlparse + +from escalatorclient.common import utils +from escalatorclient.openstack.common.apiclient import base + +CREATE_PARAMS = ('id', 'name', 'description', 'type', 'version', 'size', + 'checksum', 'status', 'os_status', 'version_patch') + +DEFAULT_PAGE_SIZE = 200 +VERSION_PARAMS = ('type') +SORT_DIR_VALUES = ('asc', 'desc') +SORT_KEY_VALUES = ( + 'name', 'id', 'cluster_id', 'created_at', 'updated_at', 'status') + +OS_REQ_ID_HDR = 'x-openstack-request-id' + + +class Version(base.Resource): + + def __repr__(self): + return "<Version %s>" % self._info + + def update(self, **fields): + self.manager.update(self, **fields) + + def delete(self, **kwargs): + return self.manager.delete(self) + + def data(self, **kwargs): + return self.manager.data(self, **kwargs) + + +class VersionManager(base.ManagerWithFind): + resource_class = Version + + def _list(self, url, response_key, obj_class=None, body=None): + resp, body = self.client.get(url) + + if obj_class is None: + obj_class = self.resource_class + + data = body[response_key] + return ([obj_class(self, res, loaded=True) for res in data if res], + resp) + + def _version_meta_from_headers(self, headers): + meta = {'properties': {}} + safe_decode = encodeutils.safe_decode + for key, value in six.iteritems(headers): + value = safe_decode(value, incoming='utf-8') + if key.startswith('x-image-meta-property-'): + _key = safe_decode(key[22:], incoming='utf-8') + meta['properties'][_key] = value + elif key.startswith('x-image-meta-'): + _key = safe_decode(key[13:], incoming='utf-8') + meta[_key] = value + + for key in ['is_public', 'protected', 'deleted']: + if key in meta: + meta[key] = strutils.bool_from_string(meta[key]) + + return self._format_version_meta_for_user(meta) + + def _version_meta_to_headers(self, fields): + headers = {} + fields_copy = copy.deepcopy(fields) + for key, value in six.iteritems(fields_copy): + headers['%s' % key] = utils.to_str(value) + return headers + + @staticmethod + def _format_version_meta_for_user(meta): + for key in ['size', 'min_ram', 'min_disk']: + if key in meta: + try: + meta[key] = int(meta[key]) if meta[key] else 0 + except ValueError: + pass + return meta + + def get(self, version, **kwargs): + """Get the metadata for a specific version. + + :param version: image object or id to look up + :rtype: :class:`version` + """ + version_id = base.getid(version) + resp, body = self.client.get('/v1/versions/%s' + % urlparse.quote(str(version_id))) + # meta = self._version_meta_from_headers(resp.headers) + return_request_id = kwargs.get('return_req_id', None) + if return_request_id is not None: + return_request_id.append(resp.headers.get(OS_REQ_ID_HDR, None)) + # return version(self, meta) + return Version(self, self._format_version_meta_for_user( + body['version'])) + + def _build_params(self, parameters): + params = {'limit': parameters.get('page_size', DEFAULT_PAGE_SIZE)} + + if 'marker' in parameters: + params['marker'] = parameters['marker'] + + sort_key = parameters.get('sort_key') + if sort_key is not None: + if sort_key in SORT_KEY_VALUES: + params['sort_key'] = sort_key + else: + raise ValueError('sort_key must be one of the following: %s.' + % ', '.join(SORT_KEY_VALUES)) + + sort_dir = parameters.get('sort_dir') + if sort_dir is not None: + if sort_dir in SORT_DIR_VALUES: + params['sort_dir'] = sort_dir + else: + raise ValueError('sort_dir must be one of the following: %s.' + % ', '.join(SORT_DIR_VALUES)) + + filters = parameters.get('filters', {}) + params.update(filters) + + return params + + def list(self, **kwargs): + """Get a list of versions. + + :param page_size: number of items to request in each paginated request + :param limit: maximum number of versions to return + :param marker:begin returning versions that appear later in version + list than that represented by this version id + :param filters: dict of direct comparison filters that mimics the + structure of an version object + :param return_request_id: If an empty list is provided, populate this + list with the request ID value from the header + x-openstack-request-id + :rtype: list of :class:`version` + """ + absolute_limit = kwargs.get('limit') + page_size = kwargs.get('page_size', DEFAULT_PAGE_SIZE) + + def paginate(qp, return_request_id=None): + for param, value in six.iteritems(qp): + if isinstance(value, six.string_types): + # Note(flaper87) Url encoding should + # be moved inside http utils, at least + # shouldn't be here. + # + # Making sure all params are str before + # trying to encode them + qp[param] = encodeutils.safe_decode(value) + + url = '/v1/versions?%s' % urlparse.urlencode(qp) + versions, resp = self._list(url, "versions") + + if return_request_id is not None: + return_request_id.append(resp.headers.get(OS_REQ_ID_HDR, None)) + + for version in versions: + yield version + + return_request_id = kwargs.get('return_req_id', None) + + params = self._build_params(kwargs) + + seen = 0 + while True: + seen_last_page = 0 + filtered = 0 + for version in paginate(params, return_request_id): + last_version = version.id + + if (absolute_limit is not None and + seen + seen_last_page >= absolute_limit): + # Note(kragniz): we've seen enough images + return + else: + seen_last_page += 1 + yield version + + seen += seen_last_page + + if seen_last_page + filtered == 0: + # Note(kragniz): we didn't get any versions in the last page + return + + if absolute_limit is not None and seen >= absolute_limit: + # Note(kragniz): reached the limit of versions to return + return + + if page_size and seen_last_page + filtered < page_size: + # Note(kragniz): we've reached the last page of the versions + return + + # Note(kragniz): there are more versions to come + params['marker'] = last_version + seen_last_page = 0 + + def add(self, **kwargs): + """Add a version + + TODO(bcwaldon): document accepted params + """ + + fields = {} + for field in kwargs: + if field in CREATE_PARAMS: + fields[field] = kwargs[field] + elif field == 'return_req_id': + continue + else: + msg = 'create() got an unexpected keyword argument \'%s\'' + raise TypeError(msg % field) + + hdrs = self._version_meta_to_headers(fields) + + resp, body = self.client.post('/v1/versions', + headers=None, + data=hdrs) + return_request_id = kwargs.get('return_req_id', None) + if return_request_id is not None: + return_request_id.append(resp.headers.get(OS_REQ_ID_HDR, None)) + + return Version(self, self._format_version_meta_for_user( + body['version'])) + + def delete(self, version, **kwargs): + """Delete an version.""" + url = "/v1/versions/%s" % base.getid(version) + resp, body = self.client.delete(url) + return_request_id = kwargs.get('return_req_id', None) + if return_request_id is not None: + return_request_id.append(resp.headers.get(OS_REQ_ID_HDR, None)) + + def update(self, version, **kwargs): + """Update an version + + TODO(bcwaldon): document accepted params + """ + hdrs = {} + fields = {} + for field in kwargs: + if field in CREATE_PARAMS: + fields[field] = kwargs[field] + elif field == 'return_req_id': + continue + hdrs.update(self._version_meta_to_headers(fields)) + + url = '/v1/versions/%s' % base.getid(version) + resp, body = self.client.put(url, headers=None, data=hdrs) + return_request_id = kwargs.get('return_req_id', None) + if return_request_id is not None: + return_request_id.append(resp.headers.get(OS_REQ_ID_HDR, None)) + + return Version(self, self._format_version_meta_for_user( + body['version_meta'])) + + def version(self, **kwargs): + """Get internal or external version of escalator. + + TODO(bcwaldon): document accepted params + """ + fields = {} + for field in kwargs: + if field in VERSION_PARAMS: + fields[field] = kwargs[field] + else: + msg = 'install() got an unexpected keyword argument \'%s\'' + raise TypeError(msg % field) + + url = '/v1/version' + hdrs = self._restore_meta_to_headers(fields) + resp, body = self.client.post(url, headers=None, data=hdrs) + return Version(self, body) diff --git a/client/pylintrc b/client/pylintrc new file mode 100644 index 0000000..6b073fd --- /dev/null +++ b/client/pylintrc @@ -0,0 +1,27 @@ +[Messages Control] +# W0511: TODOs in code comments are fine. +# W0142: *args and **kwargs are fine. +# W0622: Redefining id is fine. +disable-msg=W0511,W0142,W0622 + +[Basic] +# Variable names can be 1 to 31 characters long, with lowercase and underscores +variable-rgx=[a-z_][a-z0-9_]{0,30}$ + +# Argument names can be 2 to 31 characters long, with lowercase and underscores +argument-rgx=[a-z_][a-z0-9_]{1,30}$ + +# Method names should be at least 3 characters long +# and be lowercased with underscores +method-rgx=[a-z_][a-z0-9_]{2,50}$ + +# Module names matching nova-* are ok (files in bin/) +module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+)|(nova-[a-z0-9_-]+))$ + +# Don't require docstrings on tests. +no-docstring-rgx=((__.*__)|([tT]est.*)|setUp|tearDown)$ + +[Design] +max-public-methods=100 +min-public-methods=0 +max-args=6 diff --git a/client/requirements.txt b/client/requirements.txt new file mode 100644 index 0000000..c34e04a --- /dev/null +++ b/client/requirements.txt @@ -0,0 +1,14 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +pbr>=0.6,!=0.7,<1.0 +Babel>=1.3 +argparse +PrettyTable>=0.7,<0.8 +python-keystoneclient>=1.0.0 +pyOpenSSL>=0.11 +requests>=2.2.0,!=2.4.0 +warlock>=1.0.1,<2 +six>=1.7.0 +oslo.utils>=1.2.0 # Apache-2.0 +oslo.i18n>=1.3.0 # Apache-2.0 diff --git a/client/run_tests.sh b/client/run_tests.sh new file mode 100644 index 0000000..80edda6 --- /dev/null +++ b/client/run_tests.sh @@ -0,0 +1,49 @@ +#!/bin/bash + +function usage { + echo "Usage: $0 [OPTION]..." + echo "Run python-escalatorclient's test suite(s)" + echo "" + echo " -p, --pep8 Just run flake8" + echo " -h, --help Print this usage message" + echo "" + echo "This script is deprecated and currently retained for compatibility." + echo 'You can run the full test suite for multiple environments by running "tox".' + echo 'You can run tests for only python 2.7 by running "tox -e py27", or run only' + echo 'the flake8 tests with "tox -e pep8".' + exit +} + +command -v tox > /dev/null 2>&1 +if [ $? -ne 0 ]; then + echo 'This script requires "tox" to run.' + echo 'You can install it with "pip install tox".' + exit 1; +fi + +just_pep8=0 + +function process_option { + case "$1" in + -h|--help) usage;; + -p|--pep8) let just_pep8=1;; + esac +} + +for arg in "$@"; do + process_option $arg +done + +if [ $just_pep8 -eq 1 ]; then + tox -e pep8 + exit +fi + +tox -e py27 $toxargs 2>&1 | tee run_tests.err.log || exit +if [ ${PIPESTATUS[0]} -ne 0 ]; then + exit ${PIPESTATUS[0]} +fi + +if [ -z "$toxargs" ]; then + tox -e pep8 +fi diff --git a/client/setup.cfg b/client/setup.cfg new file mode 100644 index 0000000..165fb68 --- /dev/null +++ b/client/setup.cfg @@ -0,0 +1,46 @@ +[metadata] +name = escalatorclient +summary = Escalator Client Library +description-file = + README.rst +license = Apache License, Version 2.0 +author = OpenStack +author-email = openstack-dev@lists.openstack.org +home-page = http://www.openstack.org/ +classifier = + Development Status :: 5 - Production/Stable + Environment :: Console + Environment :: OpenStack + Intended Audience :: Information Technology + Intended Audience :: System Administrators + License :: OSI Approved :: Apache Software License + Operating System :: POSIX :: Linux + Programming Language :: Python + Programming Language :: Python :: 2 + Programming Language :: Python :: 2.7 + Programming Language :: Python :: 2.6 + Programming Language :: Python :: 3 + Programming Language :: Python :: 3.3 + +[files] +packages = + escalatorclient + +[global] +setup-hooks = + pbr.hooks.setup_hook + +[entry_points] +console_scripts = + escalator = escalatorclient.shell:main + +[build_sphinx] +source-dir = doc/source +build-dir = doc/build +all_files = 1 + +[upload_sphinx] +upload-dir = doc/build/html + +[wheel] +universal = 1 diff --git a/client/setup.py b/client/setup.py new file mode 100644 index 0000000..7363757 --- /dev/null +++ b/client/setup.py @@ -0,0 +1,30 @@ +#!/usr/bin/env python +# Copyright (c) 2013 Hewlett-Packard Development Company, L.P. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or +# implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT +import setuptools + +# In python < 2.7.4, a lazy loading of package `pbr` will break +# setuptools if some other modules registered functions in `atexit`. +# solution from: http://bugs.python.org/issue15881#msg170215 +try: + import multiprocessing # noqa +except ImportError: + pass + +setuptools.setup( + setup_requires=['pbr'], + pbr=True) diff --git a/client/test-requirements.txt b/client/test-requirements.txt new file mode 100644 index 0000000..06cb4aa --- /dev/null +++ b/client/test-requirements.txt @@ -0,0 +1,13 @@ +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +hacking>=0.8.0,<0.9 + +coverage>=3.6 +discover +mox3>=0.7.0 +mock>=1.0 +oslosphinx>=2.2.0 # Apache-2.0 +sphinx>=1.1.2,!=1.2.0,!=1.3b1,<1.3 +testrepository>=0.0.18 +testtools>=0.9.36,!=1.2.0 diff --git a/client/tox.ini b/client/tox.ini new file mode 100644 index 0000000..eca4fd9 --- /dev/null +++ b/client/tox.ini @@ -0,0 +1,41 @@ +[tox] +envlist = py26,py27,py33,py34,pypy,pep8 +minversion = 1.6 +skipsdist = True + +[testenv] +usedevelop = True +install_command = pip install -U {opts} {packages} +setenv = VIRTUAL_ENV={envdir} + OS_STDOUT_NOCAPTURE=False + OS_STDERR_NOCAPTURE=False + PYTHONHASHSEED=0 + +deps = -r{toxinidir}/requirements.txt + -r{toxinidir}/test-requirements.txt +commands = python setup.py testr --testr-args='{posargs}' + +[testenv:pep8] +commands = flake8 + +[testenv:venv] +commands = {posargs} + +[testenv:cover] +commands = python setup.py testr --coverage --testr-args='{posargs}' + +[testenv:docs] +commands= + python setup.py build_sphinx + +[tox:jenkins] +downloadcache = ~/cache/pip + +[flake8] +# H233 Python 3.x incompatible use of print operator +# H302 import only modules +# H303 no wildcard import +# H404 multi line docstring should start with a summary +ignore = F403,F812,H233,H302,H303,H404,F401,E731 +show-source = True +exclude = .venv,.tox,dist,doc,*egg,build |