summaryrefslogtreecommitdiffstats
path: root/docs/testing/developer
diff options
context:
space:
mode:
authorMorgan Richomme <morgan.richomme@orange.com>2017-09-12 11:39:11 +0200
committerJose Lausuch <jalausuch@suse.com>2017-10-11 14:04:18 +0000
commit21d8ff3fb13dac52cb3e906e968ee69104012082 (patch)
tree4ea58dbf17e4ed2e31fc86d744a103ed94a749f6 /docs/testing/developer
parentb45746ee8dcf62cf11b860e4c4b797779a08abc7 (diff)
Update Functest documentation for Euphrates
- Better description of Alpine docker (default artifact) - Reorganization of config and user guide - Upgrade of the developer guide Change-Id: Ie8e10f027bfcdb01c992cd161a1af2d6d6e29536 Signed-off-by: Morgan Richomme <morgan.richomme@orange.com> (cherry picked from commit 5504724dc69096b36948de9bd07b82f5058242f0)
Diffstat (limited to 'docs/testing/developer')
-rw-r--r--docs/testing/developer/devguide/index.rst86
1 files changed, 49 insertions, 37 deletions
diff --git a/docs/testing/developer/devguide/index.rst b/docs/testing/developer/devguide/index.rst
index 8a8b09cb1..1eae33f65 100644
--- a/docs/testing/developer/devguide/index.rst
+++ b/docs/testing/developer/devguide/index.rst
@@ -1,9 +1,9 @@
.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. SPDX-License-Identifier: CC-BY-4.0
-******************************
-OPNFV FUNCTEST developer guide
-******************************
+************************
+Functest Developer Guide
+************************
.. toctree::
:numbered:
@@ -32,7 +32,7 @@ Introduction
============
Functest is a project dealing with functional testing.
-Functest produces its own internal test cases but can also be considered
+The project produces its own internal test cases but can also be considered
as a framework to support feature and VNF onboarding project testing.
Therefore there are many ways to contribute to Functest. You can:
@@ -44,8 +44,10 @@ Therefore there are many ways to contribute to Functest. You can:
Additional tasks involving Functest but addressing all the test projects
may also be mentioned:
- * Develop the API / Test collection framework
- * Develop dashboards or automatic reporting portals
+ * The API / Test collection framework
+ * The dashboards
+ * The automatic reporting portals
+ * The testcase catalog
This document describes how, as a developer, you may interact with the
Functest project. The first section details the main working areas of
@@ -69,17 +71,22 @@ Until Danube, Functest produced 2 docker files based on Ubuntu 14.04:
* aarch64 Functest: https://hub.docker.com/r/opnfv/functest_aarch64/
In Euphrates Alpine containers have been introduced in order to lighten the
-container and manage testing slicing, the new containers are created according
+container and manage testing slicing. The new containers are created according
to the different tiers:
* functest-core: https://hub.docker.com/r/opnfv/functest-core/
* functest-healthcheck: https://hub.docker.com/r/opnfv/functest-healthcheck/
* functest-smoke: https://hub.docker.com/r/opnfv/functest-smoke/
- * functest-features: TODO
- * functest-components: TODO
- * functest-vnf: TODO
+ * functest-features: https://hub.docker.com/r/opnfv/functest-features/
+ * functest-components: https://hub.docker.com/r/opnfv/functest-components/
+ * functest-vnf: https://hub.docker.com/r/opnfv/functest-vnf/
+ * functest-parser: https://hub.docker.com/r/opnfv/functest-parser/
+ * functest-restapi: https://hub.docker.com/r/opnfv/functest-restapi/
-Functest can be described as follow::
+Standalone functest dockers are maintained for Euphrates but Alpine containers
+are recommended.
+
+Functest can be described as follow:
+----------------------+
| |
@@ -87,8 +94,8 @@ Functest can be described as follow::
| | | | Public | |
| | Tools | +------------------+ OPNFV |
| | Scripts | | | System Under Test |
- | | Scenarios | +------------------+ |
- | | | | Management | |
+ | | Scenarios | | | |
+ | | | | | |
| +--------------+ | +-------------------+
| |
| Functest Docker |
@@ -101,8 +108,8 @@ The internal test cases in Euphrates are:
* api_check
- * cloudify_ims
* connection_check
+ * snaps_health_check
* vping_ssh
* vping_userdata
* odl
@@ -110,9 +117,8 @@ The internal test cases in Euphrates are:
* odl-fds
* rally_full
* rally_sanity
- * snaps_health_check
- * tempest_full_parallel
* tempest_smoke_serial
+ * tempest_full_parallel
* cloudify_ims
By internal, we mean that this particular test cases have been developed and/or
@@ -124,7 +130,7 @@ just integrated in Functest).
The structure of this repository is detailed in `[1]`_.
The main internal test cases are in the opnfv_tests subfolder of the
-repository, the internal test cases are:
+repository, the internal test cases can be grouped by domain:
* sdn: odl, odl_netvirt, odl_fds
* openstack: api_check, connection_check, snaps_health_check, vping_ssh, vping_userdata, tempest_*, rally_*
@@ -145,23 +151,29 @@ The external test cases are:
* doctor
* domino
* fds
- * orchestra_ims
* parser
* promise
* refstack_defcore
* snaps_smoke
* functest-odl-sfc
- * vyos_vrouter
+ * orchestra_clearwaterims
+ * orchestra_openims
+ * cloudify_vrouter
+ * juju_vepc
External test cases integrated in previous versions but not released in
Euphrates:
* copper
+ * moon
* netready
* security_scan
The code to run these test cases is hosted in the repository of the project.
+Please note that orchestra test cases are hosted in Functest repository and not
+in orchestra repository. Cloudify_vrouter and juju_vepc code is also hosted in
+functest as there are no dedicated projects.
Functest framework
@@ -174,7 +186,7 @@ a CLI to prepare the environment and run tests.
It simplifies the integration of external test suites in CI pipeline and provide
commodity tools to collect and display results.
-Since Colorado, test categories also known as tiers have been created to
+Since Colorado, test categories also known as **tiers** have been created to
group similar tests, provide consistent sub-lists and at the end optimize
test duration for CI (see How To section).
@@ -205,7 +217,7 @@ introduced:
The goal is to unify the way to run tests in Functest.
-Feature, unit and vnf_base inherit from testcase::
+Feature, unit and vnf_base inherit from testcase:
+-----------------------------------------+
| |
@@ -232,34 +244,34 @@ Feature, unit and vnf_base inherit from testcase::
Testcase
-========
+--------
.. raw:: html
:url: http://artifacts.opnfv.org/functest/docs/apidoc/functest.core.testcase.html
Feature
-=======
+-------
.. raw:: html
:url: http://artifacts.opnfv.org/functest/docs/apidoc/functest.core.feature.html
Unit
-====
+----
.. raw:: html
:url: http://artifacts.opnfv.org/functest/docs/apidoc/functest.core.unit.html
VNF
-===
+---
.. raw:: html
:url: http://artifacts.opnfv.org/functest/docs/apidoc/functest.core.vnf.html
-see `Functest framework overview`_ to get code samples
+see `[5]`_ to get code samples
Functest util classes
=====================
In order to simplify the creation of test cases, Functest develops also some
-functions that can be used by any feature or internal test cases.
+functions that are used by internal test cases.
Several features are supported such as logger, configuration management and
Openstack capabilities (snapshot, clean, tacker,..).
These functions can be found under <repo>/functest/utils and can be described as
@@ -276,14 +288,14 @@ follows::
|-- openstack_tacker.py
`-- openstack_utils.py
-Please note that it is possible to use snaps utils. SNAPS `[4]`_ is an OPNFV
-project providing OpenStack utils.
+It is recommended to use the SNAPS-OO library for deploying OpenStack instances.
+SNAPS `[4]`_ is an OPNFV project providing OpenStack utils.
TestAPI
=======
Functest is using the Test collection framework and the TestAPI developed by
-the OPNFV community. See `OPNFV Test collection framework`_ for details.
+the OPNFV community. See `[6]`_ for details.
Reporting
@@ -295,15 +307,15 @@ jinja2 templates `[3]`_.
Dashboard
=========
-Additional dashboarding is managed at the testing group level, see
-`OPNFV Testing dashboard`_
+Additional dashboarding is managed at the testing group level, see `[7]`_ for
+details.
=======
How TOs
=======
-See `How to section`_ on Functest wiki
+See How to section on Functest wiki `[8]`_
==========
@@ -318,12 +330,12 @@ _`[3]`: https://git.opnfv.org/cgit/releng/tree/utils/test/reporting
_`[4]`: https://git.opnfv.org/snaps/
-_`Functest framework overview` : http://testresults.opnfv.org/functest/framework/index.html
+_`[5]` : http://testresults.opnfv.org/functest/framework/index.html
-_`OPNFV Test collection framework`: TODO
+_`[6]`: http://docs.opnfv.org/en/latest/testing/testing-dev.html
-_`OPNFV Testing dashboard`: https://opnfv.biterg.io/goto/283dba93ca18e95964f852c63af1d1ba
+_`[7]`: https://opnfv.biterg.io/goto/283dba93ca18e95964f852c63af1d1ba
-_`How to section`: https://wiki.opnfv.org/pages/viewpage.action?pageId=7768932
+_`[8]`: https://wiki.opnfv.org/pages/viewpage.action?pageId=7768932
IRC support chan: #opnfv-functest