summaryrefslogtreecommitdiffstats
path: root/docs/testing
diff options
context:
space:
mode:
Diffstat (limited to 'docs/testing')
-rw-r--r--docs/testing/developer/devguide/index.rst17
-rw-r--r--docs/testing/user/configguide/configguide.rst48
-rw-r--r--docs/testing/user/userguide/index.rst13
-rw-r--r--docs/testing/user/userguide/test_details.rst45
-rw-r--r--docs/testing/user/userguide/test_overview.rst3
-rw-r--r--docs/testing/user/userguide/troubleshooting.rst37
6 files changed, 92 insertions, 71 deletions
diff --git a/docs/testing/developer/devguide/index.rst b/docs/testing/developer/devguide/index.rst
index 1eae33f65..96ba53e6d 100644
--- a/docs/testing/developer/devguide/index.rst
+++ b/docs/testing/developer/devguide/index.rst
@@ -197,12 +197,7 @@ The tiers are:
* smoke
* features
* components
- * performance
* vnf
- * stress
-
-Note Functest deals with healthcheck, smoke, features, components and vnf tiers.
-Performance and stress tiers are out of scope.
Functest abstraction classes
============================
@@ -217,7 +212,7 @@ introduced:
The goal is to unify the way to run tests in Functest.
-Feature, unit and vnf_base inherit from testcase:
+Feature, unit and vnf_base inherit from testcase::
+-----------------------------------------+
| |
@@ -225,12 +220,12 @@ Feature, unit and vnf_base inherit from testcase:
| |
| - init() |
| - run() |
- | - publish_report() |
- | - check_criteria() |
+ | - push_to_db() |
+ | - is_successful() |
| |
+-----------------------------------------+
- | |
- V V
+ | | |
+ V V V
+--------------------+ +--------------+ +--------------------------+
| | | | | |
| feature | | unit | | vnf |
@@ -280,7 +275,7 @@ follows::
functest/utils/
|-- config.py
|-- constants.py
- |-- decoratos.py
+ |-- decorators.py
|-- env.py
|-- functest_utils.py
|-- openstack_clean.py
diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst
index 9d0179c80..e41700b23 100644
--- a/docs/testing/user/configguide/configguide.rst
+++ b/docs/testing/user/configguide/configguide.rst
@@ -353,16 +353,14 @@ includes two main directories:
src and repos directories are used to host third party code used for the tests.
-The functest code is under /usr/lib/python2.7/site-packages/functest
-The structure can be described as follows::
+The structure of functest repo can be described as follows::
|-- INFO
|-- LICENSE
|-- api
| `-- apidoc
|-- build.sh
- |-- commons
- | |-- docker
+ |-- docker
| |-- Dockerfile
| |-- Dockerfile.aarch64.patch
| |-- components
@@ -373,7 +371,9 @@ The structure can be described as follows::
| |-- healthcheck
| |-- smoke
| |-- vnf
- | `-- thirdparty-requirements.txt
+ | |-- parser
+ | |-- restapi
+ | |-- thirdparty-requirements.txt
|-- docs
| |-- com
| |-- images
@@ -392,13 +392,20 @@ The structure can be described as follows::
| |-- urls.py
| |-- common
| | |-- api_utils.py
- | | `-- error.py
+ | | |-- thread.py
| `-- resources
| `-- v1
| |-- creds.py
| |-- envs.py
| |-- testcases.py
- | `-- tiers.py
+ | |-- tiers.py
+ | |-- tasks.py
+ | `-- database
+ | |-- db.py
+ | `-- v1
+ | |-- handlers.py
+ | |-- models.py
+ | `-- swagger
|-- ci
│   |-- check_deployment.py
│   |-- config_aarch64_patch.yaml
@@ -412,36 +419,32 @@ The structure can be described as follows::
│   |-- run_tests.py
│   |-- testcases.yaml
│   |-- tier_builder.py
- │   `-- tier_handler.py
+ │   |-- tier_handler.py
|-- cli
│   |-- cli_base.py
│   |-- commands
│   │   |-- cli_env.py
│   │   |-- cli_os.py
│   │   |-- cli_testcase.py
- │   │   `-- cli_tier.py
+ │   │   |-- cli_tier.py
│   |-- functest-complete.sh
|-- core
│   |-- feature.py
│   |-- testcase.py
│   |-- unit.py
- │   `-- vnf.py
+ │   |-- vnf.py
|-- energy
│   |-- energy.py
- │   `-- energy.pyc
|-- opnfv_tests
- │   |-- mano
- │   │   |-- orchestra.py
- │   |-- openstack
- │   │   |-- rally
- │   │   |-- refstack_client
- │   │   |-- snaps
- │   │   |-- tempest
- │   │   `-- vping
- │   |-- sdn
+ │   `-- openstack
+ │      |-- rally
+ │      |-- refstack_client
+ │      |-- snaps
+ │      |-- tempest
+ │      |-- vping
+ │   `-- sdn
│   │   `-- odl
│   `-- vnf
- │   |-- aaa
│   |-- ims
│   `-- router
|-- tests
@@ -490,7 +493,8 @@ We may distinguish several directories, the first level has 5 directories:
* **functest**: This directory contains all the code needed to run
functest internal cases and OPNFV onboarded feature or VNF test cases.
-Functest directory has 7 sub-directories:
+Functest directory has 7 sub-directories, which is located under
+/usr/lib/python2.7/site-packages/functest:
* **api**: This directory is dedicated for the internal Functest API and the
API (framework) documentations.
* **ci**: This directory contains test structure definition files
diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst
index c0c763651..2addfaea8 100644
--- a/docs/testing/user/userguide/index.rst
+++ b/docs/testing/user/userguide/index.rst
@@ -10,19 +10,6 @@ Functest User Guide
.. toctree::
:maxdepth: 2
-Version history
-===============
-+------------+----------+------------------+----------------------------------+
-| **Date** | **Ver.** | **Author** | **Comment** |
-| | | | |
-+------------+----------+------------------+----------------------------------+
-| 2016-08-17 | 1.0.0 | Juha Haapavirta | Colorado release |
-| | | Column Gaynor | |
-+------------+----------+------------------+----------------------------------+
-| 2017-01-23 | 1.0.1 | Morgan Richomme | Adaptations for Danube |
-+------------+----------+------------------+----------------------------------+
-| 2017-08-16 | 1.0.2 | Morgan Richomme | Adaptations for Euphrates |
-+------------+----------+------------------+----------------------------------+
Introduction
diff --git a/docs/testing/user/userguide/test_details.rst b/docs/testing/user/userguide/test_details.rst
index c9ef63d38..56cdd3b66 100644
--- a/docs/testing/user/userguide/test_details.rst
+++ b/docs/testing/user/userguide/test_details.rst
@@ -426,28 +426,29 @@ The Clearwater architecture is described as follows:
:align: center
:alt: vIMS architecture
- cloudify_ims_perf
- ^^^^^^^^^^^^
- This testcase extends the cloudify_ims test case.
- The first part is similar but the testing part is different.
- The testing part consists in automating a realistic signaling load on the vIMS
- using an Ixia loader (proprietary tools)
- - You need to have access to an Ixia licence server
- defined in the configuration file.
-
- To start this test you need to have access to an Ixia licence server and have ixia image locally
- -
- case_name: cloudify_ims_perf
- project_name: functest
- criteria: 100
- blocking: false
- description: ''
- dependencies:
- installer: ''
- scenario: 'o'
- run:
- module: 'functest.opnfv_tests.vnf.ims.cloudify_ims_perf'
- class: 'CloudifyImsPerf'
+
+cloudify_ims_perf
+^^^^^^^^^^^^^^^^^
+This testcase extends the cloudify_ims test case.
+The first part is similar but the testing part is different.
+The testing part consists in automating a realistic signaling load on the vIMS
+using an Ixia loader (proprietary tools)
+ - You need to have access to an Ixia licence server defined in the configuration
+ file.
+
+To start this test you need to have access to an Ixia licence server and have ixia image locally
+ -
+ case_name: cloudify_ims_perf
+ project_name: functest
+ criteria: 100
+ blocking: false
+ description: ''
+ dependencies:
+ installer: ''
+ scenario: ''
+ run:
+ module: 'functest.opnfv_tests.vnf.ims.cloudify_ims_perf'
+ class: 'CloudifyImsPerf'
orchestra_openims
^^^^^^^^^^^^^^^^^
diff --git a/docs/testing/user/userguide/test_overview.rst b/docs/testing/user/userguide/test_overview.rst
index 2fca325a2..a23df02db 100644
--- a/docs/testing/user/userguide/test_overview.rst
+++ b/docs/testing/user/userguide/test_overview.rst
@@ -77,9 +77,6 @@ validate the scenario for the release.
| | | rally_full | Run the OpenStack testing tool |
| | | | benchmarking OpenStack modules |
| | | | See the Rally documents `[3]`_ |
-| | +----------------+----------------------------------+
-| | | tempest_custom | Allow to run a customized list |
-| | | | of Tempest cases |
+-------------+---------------+----------------+----------------------------------+
| Controllers | smoke | odl | Opendaylight Test suite |
| | | | Limited test suite to check the |
diff --git a/docs/testing/user/userguide/troubleshooting.rst b/docs/testing/user/userguide/troubleshooting.rst
index 29facc294..3ceb914ca 100644
--- a/docs/testing/user/userguide/troubleshooting.rst
+++ b/docs/testing/user/userguide/troubleshooting.rst
@@ -249,6 +249,43 @@ related REST API requests/responses are output to the console. More detailed deb
information can be found from tempest.log file stored into related Rally deployment
folder.
+Functest offers a possibility to test a customized list of Tempest test cases.
+To enable that, add a new entry in docker/components/testcases.yaml on the "components" container
+with the following content::
+
+ -
+ case_name: tempest_custom
+ project_name: functest
+ criteria: 100
+ blocking: false
+ description: >-
+ The test case allows running a customized list of tempest
+ test cases
+ dependencies:
+ installer: ''
+ scenario: ''
+ run:
+ module: 'functest.opnfv_tests.openstack.tempest.tempest'
+ class: 'TempestCustom'
+
+Also, a list of the Tempest test cases must be provided to the container or modify
+the existing one in
+/usr/lib/python2.7/site-packages/functest/opnfv_tests/openstack/tempest/custom_tests/test_list.txt
+
+Example of custom list of tests 'my-custom-tempest-tests.txt'::
+
+ tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops[compute,id-7fff3fb3-91d8-4fd0-bd7d-0204f1f180ba,network,smoke]
+ tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,id-f323b3ba-82f8-4db7-8ea6-6a895869ec49,network,smoke]
+
+This is an example of running a customized list of Tempest tests in Functest::
+
+ sudo docker run --env-file env \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
+ -v $(pwd)/images:/home/opnfv/functest/images \
+ -v $(pwd)/my-custom-testcases.yaml:/usr/lib/python2.7/site-packages/functest/ci/testcases.yaml \
+ -v $(pwd)/my-custom-tempest-tests.txt:/usr/lib/python2.7/site-packages/functest/opnfv_tests/openstack/tempest/custom_tests/test_list.txt \
+ opnfv/functest-components /bin/bash -c "prepare_env start && run_tests -t tempest_custom"
+
Rally
^^^^^