diff options
Diffstat (limited to 'docs/testing/user/userguide/test_results.rst')
| -rw-r--r-- | docs/testing/user/userguide/test_results.rst | 101 |
1 files changed, 69 insertions, 32 deletions
diff --git a/docs/testing/user/userguide/test_results.rst b/docs/testing/user/userguide/test_results.rst index 2d36bdc33..40a7af33c 100644 --- a/docs/testing/user/userguide/test_results.rst +++ b/docs/testing/user/userguide/test_results.rst @@ -10,42 +10,79 @@ In manual mode test results are displayed in the console and result files are put in /home/opnfv/functest/results. If you want additional logs, you may configure the logging.ini under -/usr/lib/python2.7/site-packages/functest/ci. +/usr/lib/python2.7/site-packages/xtesting/ci. Automated testing ----------------- -In automated mode, test results are displayed in jenkins logs, a summary is -provided at the end of the job and can be described as follow:: - - +-------------------------+----------------------------------------------------------+ - | ENV VAR | VALUE | - +-------------------------+----------------------------------------------------------+ - | INSTALLER_TYPE | daisy | - | DEPLOY_SCENARIO | os-nosdn-nofeature-ha | - | BUILD_TAG | jenkins-functest-daisy-baremetal-daily-master-67 | - | CI_LOOP | daily | - +-------------------------+----------------------------------------------------------+ - -:: - - +------------------------------+------------------+---------------------+------------------+----------------+ - | TEST CASE | PROJECT | TIER | DURATION | RESULT | - +------------------------------+------------------+---------------------+------------------+----------------+ - | connection_check | functest | healthcheck | 00:08 | PASS | - | api_check | functest | healthcheck | 04:22 | PASS | - | snaps_health_check | functest | healthcheck | 00:35 | PASS | - | vping_ssh | functest | smoke | 00:54 | PASS | - | vping_userdata | functest | smoke | 00:27 | PASS | - | tempest_smoke_serial | functest | smoke | 19:39 | FAIL | - | rally_sanity | functest | smoke | 15:16 | PASS | - | refstack_defcore | functest | smoke | 15:55 | PASS | - | snaps_smoke | functest | smoke | 26:45 | FAIL | - | cloudify_ims | functest | vnf | 23:56 | PASS | - | orchestra_openims | orchestra | vnf | 15:07 | PASS | - | orchestra_clearwaterims | orchestra | vnf | 19:10 | PASS | - | vyos_vrouter | functest | vnf | 00:00 | SKIP | - +------------------------------+------------------+---------------------+------------------+----------------+ +In automated mode, tests are run within split Alpine containers, and test +results are displayed in jenkins logs. The result summary is provided at the +end of each suite and can be described as follow. + +Healthcheck suite:: + + +----------------------------+------------------+---------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +----------------------------+------------------+---------------------+------------------+----------------+ + | connection_check | functest | healthcheck | 00:07 | PASS | + | api_check | functest | healthcheck | 07:46 | PASS | + | snaps_health_check | functest | healthcheck | 00:36 | PASS | + +----------------------------+------------------+---------------------+------------------+----------------+ + +Smoke suite:: + + +------------------------------+------------------+---------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +------------------------------+------------------+---------------+------------------+----------------+ + | vping_ssh | functest | smoke | 00:57 | PASS | + | vping_userdata | functest | smoke | 00:33 | PASS | + | tempest_smoke_serial | functest | smoke | 13:22 | PASS | + | rally_sanity | functest | smoke | 24:07 | PASS | + | refstack_defcore | functest | smoke | 05:21 | PASS | + | patrole | functest | smoke | 04:29 | PASS | + | snaps_smoke | functest | smoke | 46:54 | PASS | + | odl | functest | smoke | 00:00 | SKIP | + | neutron_trunk | functest | smoke | 00:00 | SKIP | + +------------------------------+------------------+---------------+------------------+----------------+ + +Features suite:: + + +-----------------------------+------------------------+------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-----------------------------+------------------------+------------------+------------------+----------------+ + | doctor-notification | doctor | features | 00:00 | SKIP | + | bgpvpn | sdnvpn | features | 00:00 | SKIP | + | functest-odl-sfc | sfc | features | 00:00 | SKIP | + | barometercollectd | barometer | features | 00:00 | SKIP | + | fds | fastdatastacks | features | 00:00 | SKIP | + +-----------------------------+------------------------+------------------+------------------+----------------+ + +Components suite:: + + +-------------------------------+------------------+--------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-------------------------------+------------------+--------------------+------------------+----------------+ + | tempest_full_parallel | functest | components | 48:28 | PASS | + | rally_full | functest | components | 126:02 | PASS | + +-------------------------------+------------------+--------------------+------------------+----------------+ + +Vnf suite:: + + +----------------------+------------------+--------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +----------------------+------------------+--------------+------------------+----------------+ + | cloudify_ims | functest | vnf | 28:15 | PASS | + | vyos_vrouter | functest | vnf | 17:59 | PASS | + | juju_epc | functest | vnf | 46:44 | PASS | + +----------------------+------------------+--------------+------------------+----------------+ + +Parser testcase:: + + +-----------------------+-----------------+------------------+------------------+----------------+ + | TEST CASE | PROJECT | TIER | DURATION | RESULT | + +-----------------------+-----------------+------------------+------------------+----------------+ + | parser-basics | parser | features | 00:00 | SKIP | + +-----------------------+-----------------+------------------+------------------+----------------+ Results are automatically pushed to the test results database, some additional result files are pushed to OPNFV artifact web sites. |