aboutsummaryrefslogtreecommitdiffstats
path: root/docs/testing
diff options
context:
space:
mode:
authorMatthewLi <matthew.lijun@huawei.com>2017-04-06 04:26:37 -0400
committerMatthewLi <matthew.lijun@huawei.com>2017-05-03 03:08:18 -0400
commit11826f38829dd5952830d6d2e50ac4918be528f1 (patch)
treecafc3412eaf73a1043771d6abb3ef5a8bb3b8bb6 /docs/testing
parent7c79dd75b519ac22483de76b4433a9923da68ef6 (diff)
refstack-client userguide
JIRA: FUNCTEST-758 container setup steps are skipped, since they already included in the functest userguide documents. Change-Id: I729d4c3174b03706624a7235da24cb3043b28c2c Signed-off-by: MatthewLi <matthew.lijun@huawei.com>
Diffstat (limited to 'docs/testing')
-rw-r--r--docs/testing/user/configguide/configguide.rst1
-rw-r--r--docs/testing/user/userguide/index.rst120
2 files changed, 119 insertions, 2 deletions
diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst
index e3485be40..9a1749789 100644
--- a/docs/testing/user/configguide/configguide.rst
+++ b/docs/testing/user/configguide/configguide.rst
@@ -336,6 +336,7 @@ should now be in place::
|-- parser
|-- promise
|-- rally
+ |-- refstack-client
|-- releng
|-- sdnvpn
|-- securityscanning
diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst
index 6c9c2de83..590043879 100644
--- a/docs/testing/user/userguide/index.rst
+++ b/docs/testing/user/userguide/index.rst
@@ -247,13 +247,125 @@ The Rally testcases are distributed accross two Tiers:
NOTE: Test case 'rally_sanity' executes a limited number of Rally smoke test
cases. Test case 'rally_full' executes the full defined set of Rally tests.
+
+Refstack-client to run Defcore testcases
+-----------------------------------------
+
+Refstack-client `[8]`_ is a command line utility that allows you to
+execute Tempest test runs based on configurations you specify.
+It is the official tool to run Defcore `[9]`_ testcases,
+which focuses on testing interoperability between OpenStack clouds.
+
+Refstack-client is integrated in Functest, consumed by Dovetail, which
+intends to define and provide a set of OPNFV related validation criteria
+that will provide input for the evaluation of the use of OPNFV trademarks.
+This progress is under the guideline of Compliance Verification Program(CVP).
+
+Defcore testcases
+^^^^^^^^^^^^^^^^^^
+
+*Danube Release*
+
+Set of DefCore tempest test cases not flagged and required.
+According to `[10]`_, some tests are still flagged due to outstanding bugs
+in the Tempest library, particularly tests that require SSH. Refstack developers
+are working on correcting these bugs upstream. Please note that although some tests
+are flagged because of bugs, there is still an expectation that the capabilities
+covered by the tests are available. It only contains Openstack core compute
+(no object storage). The approved guidelines (2016.08) are valid for Kilo,
+Liberty, Mitaka and Newton releases of OpenStack.
+The list can be generated using the Rest API from RefStack project:
+https://refstack.openstack.org/api/v1/guidelines/2016.08/tests?target=compute&type=required&alias=true&flag=false
+
+Running methods
+^^^^^^^^^^^^^^^
+
+Two running methods are provided after refstack-client integrated into
+Functest, Functest command line and manually, respectively.
+
+By default, for Defcore test cases run by Functest command line,
+are run followed with automatically generated
+configuration file, i.e., refstack_tempest.conf. In some circumstances,
+the automatic configuration file may not quite satisfied with the SUT,
+Functest also inherits the refstack-client command line and provides a way
+for users to set its configuration file according to its own SUT manually.
+
+*command line*
+
+Inside the Functest container, first to prepare Functest environment:
+
+::
+
+ cd /home/opnfv/repos/functest
+ pip install -e .
+ functest env prepare
+
+then to run default defcore testcases by using refstack-client:
+
+::
+
+ functest testcase run refstack_defcore
+
+In OPNFV Continuous Integration(CI) system, the command line method is used.
+
+*manually*
+
+Inside the Functest container, first to prepare the refstack virtualenv:
+
+::
+
+ cd /home/opnfv/repos/refstack-client
+ source .venv/bin/activate
+
+then prepare the tempest configuration file and the testcases want to run with the SUT,
+run the testcases with:
+
+::
+
+ ./refstack-client test -c <Path of the tempest configuration file to use> -v --test-list <Path or URL of test list>
+
+using help for more information:
+
+::
+
+ ./refstack-client --help
+ ./refstack-client test --help
+
+Reference tempest configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+*command line method*
+
+When command line method is used, the default tempest configuration file
+is generated by Rally.
+
+*manually*
+
+When running manually is used, recommended way to generate tempest configuration
+file is:
+
+::
+
+ cd /home/opnfv/repos/functest/functest/opnfv_tests/openstack/refstack_client
+ python tempest_conf.py
+
+a file called tempest.conf is stored in the current path by default, users can do
+some adjustment according to the SUT:
+
+::
+
+ vim refstack_tempest.conf
+
+a reference article can be used `[15]`_.
+
+
snaps_smoke
------------
This test case contains tests that setup and destroy environments with VMs with
and without Floating IPs with a newly created user and project. Set the config
value snaps.use_floating_ips (True|False) to toggle this functionality. When
-the config value of snaps.use_keystone is True, functest must have access
+the config value of snaps.use_keystone is True, Functest must have access
the cloud's private network.
This suite consists in 38 tests (test duration < 10 minutes)
@@ -362,7 +474,7 @@ The test cases are described as follows:
Features
--------
-In Danube, functest supports the integration of:
+In Danube, Functest supports the integration of:
* barometer
* bgpvpn
@@ -520,10 +632,14 @@ References
.. _`[5]`: https://github.com/Orange-OpenSource/opnfv-cloudify-clearwater/blob/master/openstack-blueprint.yaml
.. _`[6]`: https://scap.nist.gov/
.. _`[7]`: https://github.com/OpenSCAP/openscap
+.. _`[8]`: https://github.com/openstack/refstack-client
+.. _`[9]`: https://github.com/openstack/defcore
+.. _`[10]`: https://github.com/openstack/interop/blob/master/2016.08/procedure.rst
.. _`[11]`: http://robotframework.org/
.. _`[12]`: http://artifacts.opnfv.org/parser/colorado/docs/userguide/index.html
.. _`[13]`: https://wiki.opnfv.org/display/PROJ/SNAPS-OO
.. _`[14]`: https://github.com/oolorg/opnfv-functest-vrouter
+.. _`[15]`: https://aptira.com/testing-openstack-tempest-part-1/
`OPNFV main site`_