aboutsummaryrefslogtreecommitdiffstats
path: root/docs/testing/user/userguide/test_results.rst
diff options
context:
space:
mode:
Diffstat (limited to 'docs/testing/user/userguide/test_results.rst')
-rw-r--r--docs/testing/user/userguide/test_results.rst101
1 files changed, 69 insertions, 32 deletions
diff --git a/docs/testing/user/userguide/test_results.rst b/docs/testing/user/userguide/test_results.rst
index 2d36bdc33..40a7af33c 100644
--- a/docs/testing/user/userguide/test_results.rst
+++ b/docs/testing/user/userguide/test_results.rst
@@ -10,42 +10,79 @@ In manual mode test results are displayed in the console and result files
are put in /home/opnfv/functest/results.
If you want additional logs, you may configure the logging.ini under
-/usr/lib/python2.7/site-packages/functest/ci.
+/usr/lib/python2.7/site-packages/xtesting/ci.
Automated testing
-----------------
-In automated mode, test results are displayed in jenkins logs, a summary is
-provided at the end of the job and can be described as follow::
-
- +-------------------------+----------------------------------------------------------+
- | ENV VAR | VALUE |
- +-------------------------+----------------------------------------------------------+
- | INSTALLER_TYPE | daisy |
- | DEPLOY_SCENARIO | os-nosdn-nofeature-ha |
- | BUILD_TAG | jenkins-functest-daisy-baremetal-daily-master-67 |
- | CI_LOOP | daily |
- +-------------------------+----------------------------------------------------------+
-
-::
-
- +------------------------------+------------------+---------------------+------------------+----------------+
- | TEST CASE | PROJECT | TIER | DURATION | RESULT |
- +------------------------------+------------------+---------------------+------------------+----------------+
- | connection_check | functest | healthcheck | 00:08 | PASS |
- | api_check | functest | healthcheck | 04:22 | PASS |
- | snaps_health_check | functest | healthcheck | 00:35 | PASS |
- | vping_ssh | functest | smoke | 00:54 | PASS |
- | vping_userdata | functest | smoke | 00:27 | PASS |
- | tempest_smoke_serial | functest | smoke | 19:39 | FAIL |
- | rally_sanity | functest | smoke | 15:16 | PASS |
- | refstack_defcore | functest | smoke | 15:55 | PASS |
- | snaps_smoke | functest | smoke | 26:45 | FAIL |
- | cloudify_ims | functest | vnf | 23:56 | PASS |
- | orchestra_openims | orchestra | vnf | 15:07 | PASS |
- | orchestra_clearwaterims | orchestra | vnf | 19:10 | PASS |
- | vyos_vrouter | functest | vnf | 00:00 | SKIP |
- +------------------------------+------------------+---------------------+------------------+----------------+
+In automated mode, tests are run within split Alpine containers, and test
+results are displayed in jenkins logs. The result summary is provided at the
+end of each suite and can be described as follow.
+
+Healthcheck suite::
+
+ +----------------------------+------------------+---------------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +----------------------------+------------------+---------------------+------------------+----------------+
+ | connection_check | functest | healthcheck | 00:07 | PASS |
+ | api_check | functest | healthcheck | 07:46 | PASS |
+ | snaps_health_check | functest | healthcheck | 00:36 | PASS |
+ +----------------------------+------------------+---------------------+------------------+----------------+
+
+Smoke suite::
+
+ +------------------------------+------------------+---------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +------------------------------+------------------+---------------+------------------+----------------+
+ | vping_ssh | functest | smoke | 00:57 | PASS |
+ | vping_userdata | functest | smoke | 00:33 | PASS |
+ | tempest_smoke_serial | functest | smoke | 13:22 | PASS |
+ | rally_sanity | functest | smoke | 24:07 | PASS |
+ | refstack_defcore | functest | smoke | 05:21 | PASS |
+ | patrole | functest | smoke | 04:29 | PASS |
+ | snaps_smoke | functest | smoke | 46:54 | PASS |
+ | odl | functest | smoke | 00:00 | SKIP |
+ | neutron_trunk | functest | smoke | 00:00 | SKIP |
+ +------------------------------+------------------+---------------+------------------+----------------+
+
+Features suite::
+
+ +-----------------------------+------------------------+------------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +-----------------------------+------------------------+------------------+------------------+----------------+
+ | doctor-notification | doctor | features | 00:00 | SKIP |
+ | bgpvpn | sdnvpn | features | 00:00 | SKIP |
+ | functest-odl-sfc | sfc | features | 00:00 | SKIP |
+ | barometercollectd | barometer | features | 00:00 | SKIP |
+ | fds | fastdatastacks | features | 00:00 | SKIP |
+ +-----------------------------+------------------------+------------------+------------------+----------------+
+
+Components suite::
+
+ +-------------------------------+------------------+--------------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +-------------------------------+------------------+--------------------+------------------+----------------+
+ | tempest_full_parallel | functest | components | 48:28 | PASS |
+ | rally_full | functest | components | 126:02 | PASS |
+ +-------------------------------+------------------+--------------------+------------------+----------------+
+
+Vnf suite::
+
+ +----------------------+------------------+--------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +----------------------+------------------+--------------+------------------+----------------+
+ | cloudify_ims | functest | vnf | 28:15 | PASS |
+ | vyos_vrouter | functest | vnf | 17:59 | PASS |
+ | juju_epc | functest | vnf | 46:44 | PASS |
+ +----------------------+------------------+--------------+------------------+----------------+
+
+Parser testcase::
+
+ +-----------------------+-----------------+------------------+------------------+----------------+
+ | TEST CASE | PROJECT | TIER | DURATION | RESULT |
+ +-----------------------+-----------------+------------------+------------------+----------------+
+ | parser-basics | parser | features | 00:00 | SKIP |
+ +-----------------------+-----------------+------------------+------------------+----------------+
Results are automatically pushed to the test results database, some additional
result files are pushed to OPNFV artifact web sites.