refstack-client userguide

JIRA: FUNCTEST-758

container setup steps are skipped, since they already
included in the functest userguide documents.

Change-Id: I729d4c3174b03706624a7235da24cb3043b28c2c
Signed-off-by: MatthewLi <matthew.lijun@huawei.com>

2 files changed, 119 insertions, 2 deletions

diff --git a/docs/testing/user/configguide/configguide.rst b/docs/testing/user/configguide/configguide.rst
index e3485be4..9a174978 100644
--- a/docs/testing/user/configguide/configguide.rst
+++ b/docs/testing/user/configguide/configguide.rst
@@ -336,6 +336,7 @@ should now be in place::
             |-- parser
             |-- promise
             |-- rally
+            |-- refstack-client
             |-- releng
             |-- sdnvpn
             |-- securityscanning
diff --git a/docs/testing/user/userguide/index.rst b/docs/testing/user/userguide/index.rst
index 6c9c2de8..59004387 100644
--- a/docs/testing/user/userguide/index.rst
+++ b/docs/testing/user/userguide/index.rst
@@ -247,13 +247,125 @@ The Rally testcases are distributed accross two Tiers:
 NOTE: Test case 'rally_sanity' executes a limited number of Rally smoke test
 cases. Test case 'rally_full' executes the full defined set of Rally tests.
 
+
+Refstack-client to run Defcore testcases
+-----------------------------------------
+
+Refstack-client `[8]`_ is a command line utility that allows you to
+execute Tempest test runs based on configurations you specify.
+It is the official tool to run Defcore `[9]`_ testcases,
+which focuses on testing interoperability between OpenStack clouds.
+
+Refstack-client is integrated in Functest, consumed by Dovetail, which
+intends to define and provide a set of OPNFV related validation criteria
+that will provide input for the evaluation of the use of OPNFV trademarks.
+This progress is under the guideline of Compliance Verification Program(CVP).
+
+Defcore testcases
+^^^^^^^^^^^^^^^^^^
+
+*Danube Release*
+
+Set of DefCore tempest test cases not flagged and required.
+According to `[10]`_, some tests are still flagged due to outstanding bugs
+in the Tempest library, particularly tests that require SSH. Refstack developers
+are working on correcting these bugs upstream. Please note that although some tests
+are flagged because of bugs, there is still an expectation that the capabilities
+covered by the tests are available. It only contains Openstack core compute
+(no object storage). The approved guidelines (2016.08) are valid for Kilo,
+Liberty, Mitaka and Newton releases of OpenStack.
+The list can be generated using the Rest API from RefStack project:
+https://refstack.openstack.org/api/v1/guidelines/2016.08/tests?target=compute&type=required&alias=true&flag=false
+
+Running methods
+^^^^^^^^^^^^^^^
+
+Two running methods are provided after refstack-client integrated into
+Functest, Functest command line and manually, respectively.
+
+By default, for Defcore test cases run by Functest command line,
+are run followed with automatically generated
+configuration file, i.e., refstack_tempest.conf. In some circumstances,
+the automatic configuration file may not quite satisfied with the SUT,
+Functest also inherits the refstack-client command line and provides a way
+for users to set its configuration file according to its own SUT manually.
+
+*command line*
+
+Inside the Functest container, first to prepare Functest environment:
+
+::
+
+  cd /home/opnfv/repos/functest
+  pip install -e .
+  functest env prepare
+
+then to run default defcore testcases by using refstack-client:
+
+::
+
+  functest testcase run refstack_defcore
+
+In OPNFV Continuous Integration(CI) system, the command line method is used.
+
+*manually*
+
+Inside the Functest container, first to prepare the refstack virtualenv:
+
+::
+
+  cd /home/opnfv/repos/refstack-client
+  source .venv/bin/activate
+
+then prepare the tempest configuration file and the testcases want to run with the SUT,
+run the testcases with:
+
+::
+
+  ./refstack-client test -c <Path of the tempest configuration file to use> -v --test-list <Path or URL of test list>
+
+using help for more information:
+
+::
+
+  ./refstack-client --help
+  ./refstack-client test --help
+
+Reference tempest configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+*command line method*
+
+When command line method is used, the default tempest configuration file
+is generated by Rally.
+
+*manually*
+
+When running manually is used, recommended way to generate tempest configuration
+file is:
+
+::
+
+  cd /home/opnfv/repos/functest/functest/opnfv_tests/openstack/refstack_client
+  python tempest_conf.py
+
+a file called tempest.conf is stored in the current path by default, users can do
+some adjustment according to the SUT:
+
+::
+
+  vim refstack_tempest.conf
+
+a reference article can be used `[15]`_.
+
+
 snaps_smoke
 ------------
 
 This test case contains tests that setup and destroy environments with VMs with
 and without Floating IPs with a newly created user and project. Set the config
 value snaps.use_floating_ips (True|False) to toggle this functionality. When
-the config value of snaps.use_keystone is True, functest must have access
+the config value of snaps.use_keystone is True, Functest must have access
 the cloud's private network.
 This suite consists in 38 tests (test duration < 10 minutes)
 
@@ -362,7 +474,7 @@ The test cases are described as follows:
 Features
 --------
 
-In Danube, functest supports the integration of:
+In Danube, Functest supports the integration of:
 
  * barometer
  * bgpvpn
@@ -520,10 +632,14 @@ References
 .. _`[5]`: https://github.com/Orange-OpenSource/opnfv-cloudify-clearwater/blob/master/openstack-blueprint.yaml
 .. _`[6]`: https://scap.nist.gov/
 .. _`[7]`: https://github.com/OpenSCAP/openscap
+.. _`[8]`: https://github.com/openstack/refstack-client
+.. _`[9]`: https://github.com/openstack/defcore
+.. _`[10]`: https://github.com/openstack/interop/blob/master/2016.08/procedure.rst
 .. _`[11]`: http://robotframework.org/
 .. _`[12]`: http://artifacts.opnfv.org/parser/colorado/docs/userguide/index.html
 .. _`[13]`: https://wiki.opnfv.org/display/PROJ/SNAPS-OO
 .. _`[14]`: https://github.com/oolorg/opnfv-functest-vrouter
+.. _`[15]`: https://aptira.com/testing-openstack-tempest-part-1/
 
 `OPNFV main site`_