From c1f5f3c5966da1805da36ee1ef739bd658251eb8 Mon Sep 17 00:00:00 2001 From: Emma Foley Date: Thu, 2 Aug 2018 18:20:15 +0100 Subject: [docs] Add developer guidelines for contribution A new section was added to the devguide, stating expectations for code contributions. * What to include in a change * Code style guidelines * How to run tests JIRA: YARDSTICK-1335 Change-Id: Idfe7790042a3a3fe6cd8763374b10f2268bd63cd Signed-off-by: Emma Foley --- docs/testing/developer/devguide/devguide.rst | 89 ++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) (limited to 'docs') diff --git a/docs/testing/developer/devguide/devguide.rst b/docs/testing/developer/devguide/devguide.rst index 91f2c2148..43672cbda 100755 --- a/docs/testing/developer/devguide/devguide.rst +++ b/docs/testing/developer/devguide/devguide.rst @@ -449,6 +449,10 @@ Verify your patch:: It is used in CI but also by the CLI. +For more details on ``tox`` and tests, please refer to the `Running tests`_ +and `working with tox`_ sections below, which describe the different available +environments. + Submit the code with Git ++++++++++++++++++++++++ @@ -566,6 +570,91 @@ The process for backporting is as follows: A backported change needs a ``+1`` and a ``+2`` from a committer who didn’t propose the change (i.e. minimum 3 people involved). +Development guidelines +---------------------- +This section provides guidelines and best practices for feature development +and bug fixing in Yardstick. + +In general, bug fixes should be submitted as a single patch. + +When developing larger features, all commits on the local topic branch can be +submitted together, by running ``git review`` on the tip of the branch. This +creates a chain of related patches in gerrit. + +Each commit should contain one logical change and the author should aim for no +more than 300 lines of code per commit. This helps to make the changes easier +to review. + +Each feature should have the following: + +* Feature/bug fix code +* Unit tests (both positive and negative) +* Functional tests (optional) +* Sample testcases (if applicable) +* Documentation +* Update to release notes + +Coding style +~~~~~~~~~~~~ +.. _`OpenStack Style Guidelines`: https://docs.openstack.org/hacking/latest/user/hacking.html +.. _`OPNFV coding guidelines`: https://wiki.opnfv.org/display/DEV/Contribution+Guidelines + +Please follow the `OpenStack Style Guidelines`_ for code contributions (the +section on Internationalization (i18n) Strings is not applicable). + +When writing commit message, the `OPNFV coding guidelines`_ on git commit +message style should also be used. + +Running tests +~~~~~~~~~~~~~ +Once your patch has been submitted, a number of tests will be run by Jenkins +CI to verify the patch. Before submitting your patch, you should run these +tests locally. You can do this using ``tox``, which has a number of different +test environments defined in ``tox.ini``. +Calling ``tox`` without any additional arguments runs the default set of +tests (unit tests, functional tests, coverage and pylint). + +If some tests are failing, you can save time and select test environments +individually, by passing one or more of the following command-line options to +``tox``: + +* ``-e py27``: Unit tests using Python 2.7 +* ``-e py3``: Unit tests using Python 3 +* ``-e pep8``: Linter and style checks on updated files +* ``-e functional``: Functional tests using Python 2.7 +* ``-e functional-py3``: Functional tests using Python 3 +* ``-e coverage``: Code coverage checks + +.. note:: You need to stage your changes prior to running coverage for those + changes to be checked. + +In addition to the tests run by Jenkins (listed above), there are a number of +other test environments defined. + +* ``-e pep8-full``: Linter and style checks are run on the whole repo (not + just on updated files) +* ``-e os-requirements``: Check that the requirements are compatible with + OpenStack requirements. + +Working with tox +++++++++++++++++ +.. _virtualenv: https://virtualenv.pypa.io/en/stable/ + +``tox`` uses `virtualenv`_ to create isolated Python environments to run the +tests in. The test environments are located at +``.tox/`` e.g. ``.tox/py27``. + +If requirements are changed, you will need to recreate the tox test +environment to make sure the new requirements are installed. This is done by +passing the additional ``-r`` command-line option to ``tox``:: + + tox -r -e ... + +This can also be achieved by deleting the test environments manually before +running ``tox``:: + + rm -rf .tox/ + rm -rf .tox/py27 Plugins ------- -- cgit 1.2.3-korg From b7d1b4c159ff66396d6c035d6cc0df115b3cf5e3 Mon Sep 17 00:00:00 2001 From: Emma Foley Date: Wed, 15 Aug 2018 09:55:07 +0100 Subject: [docs] Add conventions for writing unit tests. Added details of conventions used when writing unit tests. JIRA: YARDSTICK-1335 Change-Id: I7bb141126968c88c8a2d7e0dce4f33ed6f79cee4 Signed-off-by: Emma Foley --- docs/testing/developer/devguide/devguide.rst | 51 ++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) (limited to 'docs') diff --git a/docs/testing/developer/devguide/devguide.rst b/docs/testing/developer/devguide/devguide.rst index 43672cbda..4fe01c12b 100755 --- a/docs/testing/developer/devguide/devguide.rst +++ b/docs/testing/developer/devguide/devguide.rst @@ -656,6 +656,57 @@ running ``tox``:: rm -rf .tox/ rm -rf .tox/py27 +Writing unit tests +~~~~~~~~~~~~~~~~~~ +For each change submitted, a set of unit tests should be submitted, which +should include both positive and negative testing. + +In order to help identify which tests are needed, follow the guidelines below. + +* In general, there should be a separate test for each branching point, return + value and input set. +* Negative tests should be written to make sure exceptions are raised and/or + handled appropriately. + +The following convention should be used for naming tests:: + + test__ + +The comment gives more information on the nature of the test, the side effect +being checked, or the parameter being modified:: + + test_my_method_runtime_error + test_my_method_invalid_credentials + test_my_method_param1_none + +Mocking ++++++++ +The ``mock`` library is used for unit testing to stub out external libraries. + +The following conventions are used in Yardstick: + +* Use ``mock.patch.object`` instead of ``mock.patch``. + +* When naming mocked classes/functions, use ``mock_`` + e.g. ``mock_subprocess_call`` + +* Avoid decorating classes with mocks. Apply the mocking in ``setUp()``:: + + @mock.patch.object(ssh, 'SSH') + class MyClassTestCase(unittest.TestCase): + + should be:: + + class MyClassTestCase(unittest.TestCase): + def setUp(self): + self._mock_ssh = mock.patch.object(ssh, 'SSH') + self.mock_ssh = self._mock_ssh.start() + + self.addCleanup(self._stop_mocks) + + def _stop_mocks(self): + self._mock_ssh.stop() + Plugins ------- -- cgit 1.2.3-korg From ddeab7c51942b19c1c88efe58339964afca941bc Mon Sep 17 00:00:00 2001 From: Stepan Andrushko Date: Fri, 14 Sep 2018 20:05:13 +0300 Subject: [docs] Update on Yardstick installation Added details how to install Yardstick using install.yaml. Updated: - 04-installation.rst - 13-nsb-installation.rst JIRA: YARDSTICK-1335 Change-Id: I526c328f1b197833a2cf496cfef15ba89c11fab1 Signed-off-by: Stepan Andrushko --- docs/testing/user/userguide/04-installation.rst | 109 +++++++++++++++++++++ .../testing/user/userguide/13-nsb-installation.rst | 4 + 2 files changed, 113 insertions(+) (limited to 'docs') diff --git a/docs/testing/user/userguide/04-installation.rst b/docs/testing/user/userguide/04-installation.rst index a4846230e..d97078909 100644 --- a/docs/testing/user/userguide/04-installation.rst +++ b/docs/testing/user/userguide/04-installation.rst @@ -444,6 +444,115 @@ These configuration files can be found in the ``samples`` directory. Default location for the output is ``/tmp/yardstick.out``. +Automatic installation of Yardstick using ansible +------------------------------------------------- + +Automatic installation can be used as an alternative to the manual. +Yardstick can be installed on the bare metal and to the container. Yardstick +container can be either pulled or built. + +Bare metal installation +^^^^^^^^^^^^^^^^^^^^^^^ + +Use ansible script ``install.yaml`` to install Yardstick on Ubuntu server: + +.. code-block:: console + + ansible-playbook -i install-inventory.ini install.yaml \ + -e YARDSTICK_DIR= + +.. note:: By default ``INSTALLATION_MODE`` is ``baremetal``. + +.. note:: By default Ubuntu 16.04 is chosen (xenial). It can be changed to + Ubuntu 18.04 (bionic) by passing ``-e OS_RELEASE=bionic`` parameter. + +.. note:: To install Yardstick in virtual environment pass parameter + ``-e VIRTUAL_ENVIRONMENT=True``. + +To build Yardstick NSB image pass ``IMG_PROPERTY=nsb`` as input parameter: + +.. code-block:: console + + ansible-playbook -i install-inventory.ini install.yaml \ + -e IMAGE_PROPERTY=nsb \ + -e YARDSTICK_DIR= + +.. note:: In this ``INSTALLATION_MODE`` mode either Yardstick image or SampleVNF + images will be built. Image type is defined by parameter ``IMAGE_PROPERTY``. + By default Yardstick image will be built. + +Container installation +^^^^^^^^^^^^^^^^^^^^^^ + +Use ansible script ``install.yaml`` to pull or build Yardstick +container. To pull Yardstick image and start container run: + +.. code-block:: console + + ansible-playbook -i install-inventory.ini install.yaml \ + -e YARDSTICK_DIR= \ + -e INSTALLATION_MODE=container_pull + +.. note:: In this ``INSTALLATION_MODE`` mode either Yardstick image or SampleVNF + images will be built. Image type is defined by variable ``IMG_PROPERTY`` in + file ``ansible/group_vars/all.yml``. By default Yardstick image will be + built. + +.. note:: Open question: How to know if Docker image is built on Ubuntu 16.04 and 18.04? + Do we need separate tag to be used? + +To build Yardstick image run: + +.. code-block:: console + + ansible-playbook -i install-inventory.ini install.yaml \ + -e YARDSTICK_DIR= \ + -e INSTALLATION_MODE=container + +.. note:: In this ``INSTALLATION_MODE`` mode neither Yardstick image nor SampleVNF + image will be built. + +.. note:: By default Ubuntu 16.04 is chosen (xenial). It can be changed to + Ubuntu 18.04 (bionic) by passing ``-e OS_RELEASE=bionic`` parameter. + +Parameters for ``install.yaml`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Description of the parameters used with ``install.yaml`` script + + +-------------------------+-------------------------------------------------+ + | Parameters | Detail | + +=========================+=================================================+ + | -i install-inventory.ini| Installs package dependency to remote servers | + | | Mandatory parameter | + | | By default no remote servers are provided | + | | Needed packages will be installed on localhost | + +-------------------------+-------------------------------------------------+ + | -e YARDSTICK_DIR | Path to Yardstick folder | + | | Mandatory parameter | + +-------------------------+-------------------------------------------------+ + | -e INSTALLATION_MODE | baremetal: Yardstick is installed to the bare | + | | metal | + | | Default parameter | + | +-------------------------------------------------+ + | | container: Yardstick is installed in container | + | | Container is built from Dockerfile | + | +-------------------------------------------------+ + | | container_pull: Yardstick is installed in | + | | container | + | | Container is pulled from docker hub | + +-------------------------+-------------------------------------------------+ + | -e OS_RELEASE | xenial or bionic: Ubuntu version to be used | + | | Default is Ubuntu 16.04 (xenial) | + +-------------------------+-------------------------------------------------+ + | -e IMAGE_PROPERTY | normal or nsb: Type of the VM image to be built | + | | Default image is Yardstick | + +-------------------------+-------------------------------------------------+ + | -e VIRTUAL_ENVIRONMENT | False or True: Whether install in virtualenv | + | | Default is False | + +-------------------------+-------------------------------------------------+ + + Deploy InfluxDB and Grafana using Docker ---------------------------------------- diff --git a/docs/testing/user/userguide/13-nsb-installation.rst b/docs/testing/user/userguide/13-nsb-installation.rst index fb68fbf21..0b76cdd30 100644 --- a/docs/testing/user/userguide/13-nsb-installation.rst +++ b/docs/testing/user/userguide/13-nsb-installation.rst @@ -168,6 +168,10 @@ It will also automatically download all the packages needed for NSB Testing setup. Refer chapter :doc:`04-installation` for more on docker **Install Yardstick using Docker (recommended)** +Another way to execute an installation for a Bare-Metal or a Standalone context +is to use ansible script ``install.yaml``. Refer chapter :doc:`04-installation` +for more details. + System Topology: ================ -- cgit 1.2.3-korg