summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--docs/testing/user/userguide/testing_guide.rst27
-rw-r--r--dovetail/compliance/cvp.0.8.0.yml57
-rw-r--r--dovetail/conf/cmd_config.yml2
-rw-r--r--dovetail/conf/dovetail_config.yml1
4 files changed, 75 insertions, 12 deletions
diff --git a/docs/testing/user/userguide/testing_guide.rst b/docs/testing/user/userguide/testing_guide.rst
index 88c5eecc..42a9ad7f 100644
--- a/docs/testing/user/userguide/testing_guide.rst
+++ b/docs/testing/user/userguide/testing_guide.rst
@@ -397,7 +397,7 @@ Now you should be in the Dovetail container's prompt and ready to execute
test suites.
The Dovetail client CLI allows the tester to specify which test suite to run.
-You can refer to :ref:`cli-reference`.
+You can refer to :ref:`cli-reference`
for the details of the CLI.
By default the results are stored in a local file ``$DOVETAIL_HOME/results``.
@@ -407,12 +407,18 @@ By default the results are stored in a local file ``$DOVETAIL_HOME/results``.
Multiple test suites may be available. For the purpose of running
CVP test suite, the test suite name follows the following format,
-``CVP_<major>_<minor>_<patch>``
-For example, CVP_1_0_0.
+``cvp.<major>.<minor>.<patch>``.
+The latest and default test suite is cvp.0.8.0.
.. code-block:: bash
- $ dovetail run --testsuite CVP_1_0_0
+ $ dovetail run
+
+This command is equal to
+
+.. code-block:: bash
+
+ $ dovetail run --testsuite cvp.0.8.0
If you are not running the entire test suite, you can choose to run an
individual test area instead. The test area can be a single test area or
@@ -421,30 +427,29 @@ and "ha". The optional test areas are "ipv6", "sdnvpn" and "tempest".
.. code-block:: bash
- $ dovetail run --testsuite CVP_1_0_0 --testarea mandatory
+ $ dovetail run --testarea mandatory
You need to push the results to local DB if you want to store the results
or report the results to CVP.
.. code-block:: bash
- $ dovetail run --testsuite CVP_1_0_0 --report http://<test_host_ip>:<testapi_port>/api/v1/results
+ $ dovetail run --report http://<test_host_ip>:<testapi_port>/api/v1/results
If the Test Host is offline, ``--offline`` should be added to support running with
local resources.
.. code-block:: bash
- $ dovetail run --testsuite CVP_1_0_0 --offline --report http://<test_host_ip>:<testapi_port>/api/v1/results
+ $ dovetail run --offline --report http://<test_host_ip>:<testapi_port>/api/v1/results
-Until the official test suite is approved and released, you can use
-the *proposed_tests* for your trial runs, like this.
+Here is an example of running mandatory test areas.
.. code-block:: bash
- $ dovetail run --testsuite proposed_tests --testarea mandatory --report http://192.168.135.2:8000/api/v1/results
+ $ dovetail run --offline --testarea mandatory --report http://192.168.135.2:8000/api/v1/results
2017-09-29 07:00:55,718 - run - INFO - ================================================
- 2017-09-29 07:00:55,718 - run - INFO - Dovetail compliance: proposed_tests!
+ 2017-09-29 07:00:55,718 - run - INFO - Dovetail compliance: cvp.0.8.0!
2017-09-29 07:00:55,718 - run - INFO - ================================================
2017-09-29 07:00:55,719 - run - INFO - Build tag: daily-master-f0795af6-a4e3-11e7-acc5-0242ac110004
2017-09-29 07:00:55,956 - run - INFO - >>[testcase]: dovetail.osinterop.tc001
diff --git a/dovetail/compliance/cvp.0.8.0.yml b/dovetail/compliance/cvp.0.8.0.yml
new file mode 100644
index 00000000..942215b9
--- /dev/null
+++ b/dovetail/compliance/cvp.0.8.0.yml
@@ -0,0 +1,57 @@
+---
+cvp.0.8.0:
+ name: cvp.0.8.0
+ testcases_list:
+ # mandatory test cases
+ # osinterop
+ - dovetail.osinterop.tc001
+ # vping
+ - dovetail.vping.tc001
+ - dovetail.vping.tc002
+ # HA
+ - dovetail.ha.tc001
+ - dovetail.ha.tc002
+ - dovetail.ha.tc003
+ - dovetail.ha.tc004
+ - dovetail.ha.tc005
+ - dovetail.ha.tc006
+ - dovetail.ha.tc007
+ - dovetail.ha.tc008
+ # optional test cases
+ # ipv6
+ - dovetail.ipv6.tc001
+ - dovetail.ipv6.tc002
+ - dovetail.ipv6.tc003
+ - dovetail.ipv6.tc004
+ - dovetail.ipv6.tc005
+ - dovetail.ipv6.tc006
+ - dovetail.ipv6.tc007
+ - dovetail.ipv6.tc008
+ - dovetail.ipv6.tc009
+ - dovetail.ipv6.tc010
+ - dovetail.ipv6.tc011
+ - dovetail.ipv6.tc012
+ - dovetail.ipv6.tc013
+ - dovetail.ipv6.tc014
+ - dovetail.ipv6.tc015
+ - dovetail.ipv6.tc016
+ - dovetail.ipv6.tc017
+ - dovetail.ipv6.tc018
+ - dovetail.ipv6.tc019
+ - dovetail.ipv6.tc020
+ - dovetail.ipv6.tc021
+ - dovetail.ipv6.tc022
+ - dovetail.ipv6.tc023
+ - dovetail.ipv6.tc024
+ - dovetail.ipv6.tc025
+ # tempest
+ - dovetail.tempest.tc001
+ - dovetail.tempest.tc002
+ - dovetail.tempest.tc003
+ - dovetail.tempest.tc004
+ - dovetail.tempest.tc005
+ # sdnvpn
+ - dovetail.sdnvpn.tc001
+ - dovetail.sdnvpn.tc002
+ - dovetail.sdnvpn.tc004
+ - dovetail.sdnvpn.tc008
diff --git a/dovetail/conf/cmd_config.yml b/dovetail/conf/cmd_config.yml
index 470c7108..2742b8a3 100644
--- a/dovetail/conf/cmd_config.yml
+++ b/dovetail/conf/cmd_config.yml
@@ -40,7 +40,7 @@ cli:
testsuite:
flags:
- '--testsuite'
- default: 'compliance_set'
+ default: 'cvp.0.8.0'
help: 'compliance testsuite.'
testarea:
flags:
diff --git a/dovetail/conf/dovetail_config.yml b/dovetail/conf/dovetail_config.yml
index b95c6d37..25fd1c3b 100644
--- a/dovetail/conf/dovetail_config.yml
+++ b/dovetail/conf/dovetail_config.yml
@@ -27,6 +27,7 @@ testsuite_supported:
- compliance_set
- proposed_tests
- debug
+ - cvp.0.8.0
# testarea supported, should adjust accordingly
testarea_supported:
- osinterop