summaryrefslogtreecommitdiffstats
path: root/docs
diff options
context:
space:
mode:
Diffstat (limited to 'docs')
-rw-r--r--docs/testing/user/configguide/configguide.rst13
-rw-r--r--docs/testing/user/userguide/index.rst13
-rw-r--r--docs/testing/user/userguide/test_details.rst46
-rw-r--r--docs/testing/user/userguide/test_overview.rst3
-rw-r--r--docs/testing/user/userguide/test_results.rst6
-rw-r--r--docs/testing/user/userguide/troubleshooting.rst37
6 files changed, 57 insertions, 61 deletions
diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst
index e41700b23..b82376dbd 100644
--- a/docs/testing/user/configguide/configguide.rst
+++ b/docs/testing/user/configguide/configguide.rst
@@ -360,6 +360,7 @@ The structure of functest repo can be described as follows::
|-- api
| `-- apidoc
|-- build.sh
+ |-- commons
|-- docker
| |-- Dockerfile
| |-- Dockerfile.aarch64.patch
@@ -483,11 +484,10 @@ The structure of functest repo can be described as follows::
We may distinguish several directories, the first level has 5 directories:
* **api**: This directory is dedicated to the API (framework) documentations.
-* **commons**: This directory is dedicated for storage of traffic
- profile or any other test inputs that could be reused by any test
- project.
+* **commons**: This directory is dedicated for storage of traffic profile or
+ any other test inputs that could be reused by any test project.
* **docker**: This directory includes the needed files and tools to
- build the Functest Docker image.
+ build the Functest Docker images.
* **docs**: This directory includes documentation: Release Notes,
User Guide, Configuration Guide and Developer Guide.
* **functest**: This directory contains all the code needed to run
@@ -495,8 +495,7 @@ We may distinguish several directories, the first level has 5 directories:
Functest directory has 7 sub-directories, which is located under
/usr/lib/python2.7/site-packages/functest:
- * **api**: This directory is dedicated for the internal Functest API and the
- API (framework) documentations.
+ * **api**: This directory is dedicated for the internal Functest API.
* **ci**: This directory contains test structure definition files
(e.g <filename>.yaml) and bash shell/python scripts used to
configure and execute Functional tests. The test execution script
@@ -508,7 +507,7 @@ Functest directory has 7 sub-directories, which is located under
the integration of internal, feature or vnf cases.
* **opnfv_tests**: This directory includes the scripts required by
Functest internal test cases and other feature projects test cases.
- * **tests**: This directory includes the functest unit tests
+ * **tests**: This directory includes the functest unit tests.
* **utils**: this directory holds Python source code for some general
purpose helper utilities, which testers can also re-use in their
own test code. See for an example the Openstack helper utility:
diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst
index c0c763651..2addfaea8 100644
--- a/docs/testing/user/userguide/index.rst
+++ b/docs/testing/user/userguide/index.rst
@@ -10,19 +10,6 @@ Functest User Guide
.. toctree::
:maxdepth: 2
-Version history
-===============
-+------------+----------+------------------+----------------------------------+
-| **Date** | **Ver.** | **Author** | **Comment** |
-| | | | |
-+------------+----------+------------------+----------------------------------+
-| 2016-08-17 | 1.0.0 | Juha Haapavirta | Colorado release |
-| | | Column Gaynor | |
-+------------+----------+------------------+----------------------------------+
-| 2017-01-23 | 1.0.1 | Morgan Richomme | Adaptations for Danube |
-+------------+----------+------------------+----------------------------------+
-| 2017-08-16 | 1.0.2 | Morgan Richomme | Adaptations for Euphrates |
-+------------+----------+------------------+----------------------------------+
Introduction
diff --git a/docs/testing/user/userguide/test_details.rst b/docs/testing/user/userguide/test_details.rst
index 56cdd3b66..33af31ba9 100644
--- a/docs/testing/user/userguide/test_details.rst
+++ b/docs/testing/user/userguide/test_details.rst
@@ -434,18 +434,23 @@ The first part is similar but the testing part is different.
The testing part consists in automating a realistic signaling load on the vIMS
using an Ixia loader (proprietary tools)
- You need to have access to an Ixia licence server defined in the configuration
- file.
+ file and have ixia image locally.
+
+This test case is available but not declared in testcases.yaml. The declaration
+of the testcase is simple, connect to your functest-vnf docker, add the following
+section in /usr/lib/python2.7/site-packacges/functest/ci/testcases.yaml::
-To start this test you need to have access to an Ixia licence server and have ixia image locally
-
case_name: cloudify_ims_perf
project_name: functest
- criteria: 100
+ criteria: 80
blocking: false
- description: ''
+ description: >-
+ Stress tests based on Cloudify. Ixia loader images and access to Ixia
+ server license.
dependencies:
installer: ''
- scenario: ''
+ scenario: 'os-nosdn-nofeature-ha'
run:
module: 'functest.opnfv_tests.vnf.ims.cloudify_ims_perf'
class: 'CloudifyImsPerf'
@@ -460,15 +465,8 @@ orchestra_clearwaterims
Orchestra test case deals with the deployment of Clearwater vIMS with OpenBaton
orchestrator.
-parser
-^^^^^^
-
-See parser user guide for details.
-
-
vyos-vrouter
^^^^^^^^^^^^
-
This test case deals with the deployment and the test of vyos vrouter with
Cloudify orchestrator. The test case can do testing for interchangeability of
BGP Protocol using vyos.
@@ -486,30 +484,6 @@ The Workflow is as follows:
The vyos-vrouter architecture is described in `[14]`_
-cloudify_ims_perf
-^^^^^^^^^^^^^^^^^
-
-This test case is available but not declared in testcases.yaml. If you want to
-run it you need to get the Ixia loader images and have access to an Ixia license
-server.
-
-The declaration of the testcase is simple, connect to your functest-vnf docker,
-add the following section in /usr/lib/python2.7/site-packacges/functest/ci/testcases.yaml::
-
- case_name: cloudify_ims_perf
- project_name: functest
- criteria: 80
- blocking: false
- description: >-
- Stress tests based on Cloudify. Ixia loader images and access to Ixia
- server license.
- dependencies:
- installer: ''
- scenario: 'os-nosdn-nofeature-ha'
- run:
- module: 'functest.opnfv_tests.vnf.ims.cloudify_ims_perf'
- class: 'CloudifyImsPerf'
-
.. _`[2]`: http://docs.openstack.org/developer/tempest/overview.html
.. _`[3]`: https://rally.readthedocs.org/en/latest/index.html
diff --git a/docs/testing/user/userguide/test_overview.rst b/docs/testing/user/userguide/test_overview.rst
index 2fca325a2..a23df02db 100644
--- a/docs/testing/user/userguide/test_overview.rst
+++ b/docs/testing/user/userguide/test_overview.rst
@@ -77,9 +77,6 @@ validate the scenario for the release.
| | | rally_full | Run the OpenStack testing tool |
| | | | benchmarking OpenStack modules |
| | | | See the Rally documents `[3]`_ |
-| | +----------------+----------------------------------+
-| | | tempest_custom | Allow to run a customized list |
-| | | | of Tempest cases |
+-------------+---------------+----------------+----------------------------------+
| Controllers | smoke | odl | Opendaylight Test suite |
| | | | Limited test suite to check the |
diff --git a/docs/testing/user/userguide/test_results.rst b/docs/testing/user/userguide/test_results.rst
index 6129ef3a0..3941ba0a1 100644
--- a/docs/testing/user/userguide/test_results.rst
+++ b/docs/testing/user/userguide/test_results.rst
@@ -37,8 +37,10 @@ at the end of the job and can be described as follow::
| rally_sanity | functest | smoke | 15:16 | PASS |
| refstack_defcore | functest | smoke | 15:55 | PASS |
| snaps_smoke | functest | smoke | 26:45 | FAIL |
- | cloudify_ims | functest | vnf | 83:33 | FAIL |
- | orchestra_ims | functest | vnf | 11:32 | FAIL |
+ | cloudify_ims | functest | vnf | 23:56 | PASS |
+ | orchestra_openims | orchestra | vnf | 15:07 | PASS |
+ | orchestra_clearwaterims | orchestra | vnf | 19:10 | PASS |
+ | vyos_vrouter | functest | vnf | 00:00 | SKIP |
+------------------------------+------------------+---------------------+------------------+----------------+
Results are automatically pushed to the test results database, some additional
diff --git a/docs/testing/user/userguide/troubleshooting.rst b/docs/testing/user/userguide/troubleshooting.rst
index 29facc294..20879bf6e 100644
--- a/docs/testing/user/userguide/troubleshooting.rst
+++ b/docs/testing/user/userguide/troubleshooting.rst
@@ -249,6 +249,43 @@ related REST API requests/responses are output to the console. More detailed deb
information can be found from tempest.log file stored into related Rally deployment
folder.
+Functest offers a possibility to test a customized list of Tempest test cases.
+To enable that, add a new entry in docker/components/testcases.yaml on the "components" container
+with the following content::
+
+ -
+ case_name: tempest_custom
+ project_name: functest
+ criteria: 100
+ blocking: false
+ description: >-
+ The test case allows running a customized list of tempest
+ test cases
+ dependencies:
+ installer: ''
+ scenario: ''
+ run:
+ module: 'functest.opnfv_tests.openstack.tempest.tempest'
+ class: 'TempestCustom'
+
+Also, a list of the Tempest test cases must be provided to the container or modify
+the existing one in
+/usr/lib/python2.7/site-packages/functest/opnfv_tests/openstack/tempest/custom_tests/test_list.txt
+
+Example of custom list of tests 'my-custom-tempest-tests.txt'::
+
+ tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops[compute,id-7fff3fb3-91d8-4fd0-bd7d-0204f1f180ba,network,smoke]
+ tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,id-f323b3ba-82f8-4db7-8ea6-6a895869ec49,network,smoke]
+
+This is an example of running a customized list of Tempest tests in Functest::
+
+ sudo docker run --env-file env \
+ -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \
+ -v $(pwd)/images:/home/opnfv/functest/images \
+ -v $(pwd)/my-custom-testcases.yaml:/usr/lib/python2.7/site-packages/functest/ci/testcases.yaml \
+ -v $(pwd)/my-custom-tempest-tests.txt:/usr/lib/python2.7/site-packages/functest/opnfv_tests/openstack/tempest/custom_tests/test_list.txt \
+ opnfv/functest-components /bin/bash -c "prepare_env start && run_tests -t tempest_custom"
+
Rally
^^^^^