diff options
Diffstat (limited to 'docs/testing/user')
| -rw-r--r-- | docs/testing/user/configguide/configguide.rst | 13 | ||||
| -rw-r--r-- | docs/testing/user/userguide/index.rst | 13 | ||||
| -rw-r--r-- | docs/testing/user/userguide/test_details.rst | 46 | ||||
| -rw-r--r-- | docs/testing/user/userguide/test_overview.rst | 3 | ||||
| -rw-r--r-- | docs/testing/user/userguide/test_results.rst | 6 | ||||
| -rw-r--r-- | docs/testing/user/userguide/troubleshooting.rst | 37 |
6 files changed, 57 insertions, 61 deletions
diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst index e41700b23..b82376dbd 100644 --- a/docs/testing/user/configguide/configguide.rst +++ b/docs/testing/user/configguide/configguide.rst @@ -360,6 +360,7 @@ The structure of functest repo can be described as follows:: |-- api | `-- apidoc |-- build.sh + |-- commons |-- docker | |-- Dockerfile | |-- Dockerfile.aarch64.patch @@ -483,11 +484,10 @@ The structure of functest repo can be described as follows:: We may distinguish several directories, the first level has 5 directories: * **api**: This directory is dedicated to the API (framework) documentations. -* **commons**: This directory is dedicated for storage of traffic - profile or any other test inputs that could be reused by any test - project. +* **commons**: This directory is dedicated for storage of traffic profile or + any other test inputs that could be reused by any test project. * **docker**: This directory includes the needed files and tools to - build the Functest Docker image. + build the Functest Docker images. * **docs**: This directory includes documentation: Release Notes, User Guide, Configuration Guide and Developer Guide. * **functest**: This directory contains all the code needed to run @@ -495,8 +495,7 @@ We may distinguish several directories, the first level has 5 directories: Functest directory has 7 sub-directories, which is located under /usr/lib/python2.7/site-packages/functest: - * **api**: This directory is dedicated for the internal Functest API and the - API (framework) documentations. + * **api**: This directory is dedicated for the internal Functest API. * **ci**: This directory contains test structure definition files (e.g <filename>.yaml) and bash shell/python scripts used to configure and execute Functional tests. The test execution script @@ -508,7 +507,7 @@ Functest directory has 7 sub-directories, which is located under the integration of internal, feature or vnf cases. * **opnfv_tests**: This directory includes the scripts required by Functest internal test cases and other feature projects test cases. - * **tests**: This directory includes the functest unit tests + * **tests**: This directory includes the functest unit tests. * **utils**: this directory holds Python source code for some general purpose helper utilities, which testers can also re-use in their own test code. See for an example the Openstack helper utility: diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst index c0c763651..2addfaea8 100644 --- a/docs/testing/user/userguide/index.rst +++ b/docs/testing/user/userguide/index.rst @@ -10,19 +10,6 @@ Functest User Guide .. toctree:: :maxdepth: 2 -Version history -=============== -+------------+----------+------------------+----------------------------------+ -| **Date** | **Ver.** | **Author** | **Comment** | -| | | | | -+------------+----------+------------------+----------------------------------+ -| 2016-08-17 | 1.0.0 | Juha Haapavirta | Colorado release | -| | | Column Gaynor | | -+------------+----------+------------------+----------------------------------+ -| 2017-01-23 | 1.0.1 | Morgan Richomme | Adaptations for Danube | -+------------+----------+------------------+----------------------------------+ -| 2017-08-16 | 1.0.2 | Morgan Richomme | Adaptations for Euphrates | -+------------+----------+------------------+----------------------------------+ Introduction diff --git a/docs/testing/user/userguide/test_details.rst b/docs/testing/user/userguide/test_details.rst index 56cdd3b66..33af31ba9 100644 --- a/docs/testing/user/userguide/test_details.rst +++ b/docs/testing/user/userguide/test_details.rst @@ -434,18 +434,23 @@ The first part is similar but the testing part is different. The testing part consists in automating a realistic signaling load on the vIMS using an Ixia loader (proprietary tools) - You need to have access to an Ixia licence server defined in the configuration - file. + file and have ixia image locally. + +This test case is available but not declared in testcases.yaml. The declaration +of the testcase is simple, connect to your functest-vnf docker, add the following +section in /usr/lib/python2.7/site-packacges/functest/ci/testcases.yaml:: -To start this test you need to have access to an Ixia licence server and have ixia image locally - case_name: cloudify_ims_perf project_name: functest - criteria: 100 + criteria: 80 blocking: false - description: '' + description: >- + Stress tests based on Cloudify. Ixia loader images and access to Ixia + server license. dependencies: installer: '' - scenario: '' + scenario: 'os-nosdn-nofeature-ha' run: module: 'functest.opnfv_tests.vnf.ims.cloudify_ims_perf' class: 'CloudifyImsPerf' @@ -460,15 +465,8 @@ orchestra_clearwaterims Orchestra test case deals with the deployment of Clearwater vIMS with OpenBaton orchestrator. -parser -^^^^^^ - -See parser user guide for details. - - vyos-vrouter ^^^^^^^^^^^^ - This test case deals with the deployment and the test of vyos vrouter with Cloudify orchestrator. The test case can do testing for interchangeability of BGP Protocol using vyos. @@ -486,30 +484,6 @@ The Workflow is as follows: The vyos-vrouter architecture is described in `[14]`_ -cloudify_ims_perf -^^^^^^^^^^^^^^^^^ - -This test case is available but not declared in testcases.yaml. If you want to -run it you need to get the Ixia loader images and have access to an Ixia license -server. - -The declaration of the testcase is simple, connect to your functest-vnf docker, -add the following section in /usr/lib/python2.7/site-packacges/functest/ci/testcases.yaml:: - - case_name: cloudify_ims_perf - project_name: functest - criteria: 80 - blocking: false - description: >- - Stress tests based on Cloudify. Ixia loader images and access to Ixia - server license. - dependencies: - installer: '' - scenario: 'os-nosdn-nofeature-ha' - run: - module: 'functest.opnfv_tests.vnf.ims.cloudify_ims_perf' - class: 'CloudifyImsPerf' - .. _`[2]`: http://docs.openstack.org/developer/tempest/overview.html .. _`[3]`: https://rally.readthedocs.org/en/latest/index.html diff --git a/docs/testing/user/userguide/test_overview.rst b/docs/testing/user/userguide/test_overview.rst index 2fca325a2..a23df02db 100644 --- a/docs/testing/user/userguide/test_overview.rst +++ b/docs/testing/user/userguide/test_overview.rst @@ -77,9 +77,6 @@ validate the scenario for the release. | | | rally_full | Run the OpenStack testing tool | | | | | benchmarking OpenStack modules | | | | | See the Rally documents `[3]`_ | -| | +----------------+----------------------------------+ -| | | tempest_custom | Allow to run a customized list | -| | | | of Tempest cases | +-------------+---------------+----------------+----------------------------------+ | Controllers | smoke | odl | Opendaylight Test suite | | | | | Limited test suite to check the | diff --git a/docs/testing/user/userguide/test_results.rst b/docs/testing/user/userguide/test_results.rst index 6129ef3a0..3941ba0a1 100644 --- a/docs/testing/user/userguide/test_results.rst +++ b/docs/testing/user/userguide/test_results.rst @@ -37,8 +37,10 @@ at the end of the job and can be described as follow:: | rally_sanity | functest | smoke | 15:16 | PASS | | refstack_defcore | functest | smoke | 15:55 | PASS | | snaps_smoke | functest | smoke | 26:45 | FAIL | - | cloudify_ims | functest | vnf | 83:33 | FAIL | - | orchestra_ims | functest | vnf | 11:32 | FAIL | + | cloudify_ims | functest | vnf | 23:56 | PASS | + | orchestra_openims | orchestra | vnf | 15:07 | PASS | + | orchestra_clearwaterims | orchestra | vnf | 19:10 | PASS | + | vyos_vrouter | functest | vnf | 00:00 | SKIP | +------------------------------+------------------+---------------------+------------------+----------------+ Results are automatically pushed to the test results database, some additional diff --git a/docs/testing/user/userguide/troubleshooting.rst b/docs/testing/user/userguide/troubleshooting.rst index 29facc294..20879bf6e 100644 --- a/docs/testing/user/userguide/troubleshooting.rst +++ b/docs/testing/user/userguide/troubleshooting.rst @@ -249,6 +249,43 @@ related REST API requests/responses are output to the console. More detailed deb information can be found from tempest.log file stored into related Rally deployment folder. +Functest offers a possibility to test a customized list of Tempest test cases. +To enable that, add a new entry in docker/components/testcases.yaml on the "components" container +with the following content:: + + - + case_name: tempest_custom + project_name: functest + criteria: 100 + blocking: false + description: >- + The test case allows running a customized list of tempest + test cases + dependencies: + installer: '' + scenario: '' + run: + module: 'functest.opnfv_tests.openstack.tempest.tempest' + class: 'TempestCustom' + +Also, a list of the Tempest test cases must be provided to the container or modify +the existing one in +/usr/lib/python2.7/site-packages/functest/opnfv_tests/openstack/tempest/custom_tests/test_list.txt + +Example of custom list of tests 'my-custom-tempest-tests.txt':: + + tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_ops[compute,id-7fff3fb3-91d8-4fd0-bd7d-0204f1f180ba,network,smoke] + tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops[compute,id-f323b3ba-82f8-4db7-8ea6-6a895869ec49,network,smoke] + +This is an example of running a customized list of Tempest tests in Functest:: + + sudo docker run --env-file env \ + -v $(pwd)/openstack.creds:/home/opnfv/functest/conf/openstack.creds \ + -v $(pwd)/images:/home/opnfv/functest/images \ + -v $(pwd)/my-custom-testcases.yaml:/usr/lib/python2.7/site-packages/functest/ci/testcases.yaml \ + -v $(pwd)/my-custom-tempest-tests.txt:/usr/lib/python2.7/site-packages/functest/opnfv_tests/openstack/tempest/custom_tests/test_list.txt \ + opnfv/functest-components /bin/bash -c "prepare_env start && run_tests -t tempest_custom" + Rally ^^^^^ |