aboutsummaryrefslogtreecommitdiffstats
path: root/docs/results
diff options
context:
space:
mode:
Diffstat (limited to 'docs/results')
-rw-r--r--docs/results/compass-os-nosdn-nofeature-ha.rst20
-rw-r--r--docs/results/compass-os-odl_l2-nofeature-ha.rst19
-rw-r--r--docs/results/compass-os-onos-nofeature-ha.rst10
-rw-r--r--docs/results/joid-os-odl_l2-nofeature-ha.rst18
-rw-r--r--docs/results/overview.rst83
5 files changed, 77 insertions, 73 deletions
diff --git a/docs/results/compass-os-nosdn-nofeature-ha.rst b/docs/results/compass-os-nosdn-nofeature-ha.rst
index e09597f17..294cb09f7 100644
--- a/docs/results/compass-os-nosdn-nofeature-ha.rst
+++ b/docs/results/compass-os-nosdn-nofeature-ha.rst
@@ -4,15 +4,13 @@
Detailed test results for compass-os-nosdn-nofeature-ha
-------------------------------------------------------
-The following section outlines the detailed functest results for the Brahmaputra scenario
-deploying OpenStack in a Pharos environment by the Compass installer.
-
VIM
---
vping_ssh
^^^^^^^^^
::
+
FUNCTEST.info: Running vPing-SSH test...
vPing_ssh- INFO - Creating image 'functest-vping' from '/home/opnfv/functest/data/cirros-0.3.4-x86_64-disk.img'...
vPing_ssh- INFO - Creating neutron network vping-net...
@@ -44,12 +42,12 @@ vping_ssh
vPing_ssh- INFO - vPing duration:'43.2' s.
vPing_ssh- INFO - Cleaning up...
vPing_ssh- INFO - vPing OK
-::
+
vping_userdata
^^^^^^^^^^^^^^
-
::
+
FUNCTEST.info: Running vPing-userdata test...
vPing_userdata- INFO - Creating image 'functest-vping' from '/home/opnfv/functest/data/cirros-0.3.4-x86_64-disk.img'...
vPing_userdata- INFO - Creating neutron network vping-net...
@@ -90,11 +88,12 @@ vping_userdata
vPing_userdata- INFO - vPing OK
vPing_userdata- INFO - Cleaning up...
vPing_userdata- INFO - Deleting network 'vping-net'...
-::
+
Tempest
^^^^^^^
::
+
FUNCTEST.info: Running Tempest tests...
run_tempest - INFO - Creating tenant and user for Tempest suite
2016-02-23 04:37:19.865 23855 INFO rally.verification.tempest.tempest [-] Starting: Creating configuration file for Tempest.
@@ -328,11 +327,12 @@ Tempest
run_tempest - INFO - Results: {'timestart': '2016-02-2304:37:25.793306', 'duration': 261, 'tests': 210, 'failures': 0}
run_tempest - INFO - Pushing results to DB: 'http://testresults.opnfv.org/testapi/results'.
run_tempest - INFO - Deleting tenant and user for Tempest suite)
-::
+
Rally
^^^^^
::
+
FUNCTEST.info: Running Rally benchmark suite...
run_rally - INFO - Starting test scenario "authenticate" ...
run_rally - INFO -
@@ -1622,15 +1622,16 @@ Rally
+===================+============+===============+===========+
| TOTAL: | 01:03:52 | 229 | 98.59% |
+===================+============+===============+===========+
-::
+
+
Feature tests
-------------
vIMS
^^^^
-
::
+
FUNCTEST.info: Running vIMS test...
vIMS - INFO - Prepare OpenStack plateform (create tenant and user)
vIMS - INFO - Update OpenStack creds informations
@@ -1664,4 +1665,3 @@ vIMS
vIMS - INFO - Cloudify-manager server has been successfully removed!
vIMS - INFO - Removing vIMS tenant ..
vIMS - INFO - Removing vIMS user ..
-::
diff --git a/docs/results/compass-os-odl_l2-nofeature-ha.rst b/docs/results/compass-os-odl_l2-nofeature-ha.rst
index 088097b2e..bed7f8646 100644
--- a/docs/results/compass-os-odl_l2-nofeature-ha.rst
+++ b/docs/results/compass-os-odl_l2-nofeature-ha.rst
@@ -10,6 +10,7 @@ VIM
vping_ssh
^^^^^^^^^
::
+
FUNCTEST.info: Running vPing-SSH test...
vPing_ssh- INFO - Creating image 'functest-vping' from '/home/opnfv/functest/data/cirros-0.3.4-x86_64-disk.img'...
vPing_ssh- INFO - Creating neutron network vping-net...
@@ -41,12 +42,12 @@ vping_ssh
vPing_ssh- INFO - vPing duration:'26.0' s.
vPing_ssh- INFO - Cleaning up...
vPing_ssh- INFO - vPing OK
-::
+
vping_userdata
^^^^^^^^^^^^^^
-
::
+
FUNCTEST.info: Running vPing-userdata test...
vPing_userdata- INFO - Creating image 'functest-vping' from '/home/opnfv/functest/data/cirros-0.3.4-x86_64-disk.img'...
vPing_userdata- INFO - Creating neutron network vping-net...
@@ -87,11 +88,12 @@ vping_userdata
vPing_userdata- INFO - vPing OK
vPing_userdata- INFO - Cleaning up...
vPing_userdata- INFO - Deleting network 'vping-net'...
-::
+
Tempest
^^^^^^^
::
+
FUNCTEST.info: Running Tempest tests...
run_tempest - INFO - Creating tenant and user for Tempest suite
INFO rally.verification.tempest.tempest [-] Starting: Creating configuration file for Tempest.
@@ -324,11 +326,12 @@ Tempest
run_tempest - INFO - Results: {'timestart': '2016-02-2222:27:42.722872', 'duration': 171, 'tests': 210, 'failures': 1}
run_tempest - INFO - Pushing results to DB: 'http://testresults.opnfv.org/testapi/results'.
run_tempest - INFO - Deleting tenant and user for Tempest suite)
-::
+
Rally
^^^^^
::
+
FUNCTEST.info: Running Rally benchmark suite...
run_rally - INFO - Starting test scenario "authenticate" ...
run_rally - INFO -
@@ -1617,7 +1620,7 @@ Rally
+===================+============+===============+===========+
| TOTAL: | 00:57:39 | 229 | 98.51% |
+===================+============+===============+===========+
-::
+
SDN Controller
--------------
@@ -1625,6 +1628,7 @@ SDN Controller
ODL
^^^
::
+
==============================================================================
Basic
==============================================================================
@@ -1721,15 +1725,15 @@ ODL
Neutron :: Test suite for Neutron Plugin | PASS |
18 critical tests, 18 passed, 0 failed
18 tests total, 18 passed, 0 failed
-::
+
Feature tests
-------------
vIMS
^^^^
-
::
+
FUNCTEST.info: Running vIMS test...
vIMS - INFO - Prepare OpenStack plateform (create tenant and user)
vIMS - INFO - Update OpenStack creds informations
@@ -1763,4 +1767,3 @@ vIMS
vIMS - INFO - Cloudify-manager server has been successfully removed!
vIMS - INFO - Removing vIMS tenant ..
vIMS - INFO - Removing vIMS user ..
-::
diff --git a/docs/results/compass-os-onos-nofeature-ha.rst b/docs/results/compass-os-onos-nofeature-ha.rst
index b9961fa0f..87103c307 100644
--- a/docs/results/compass-os-onos-nofeature-ha.rst
+++ b/docs/results/compass-os-onos-nofeature-ha.rst
@@ -10,6 +10,7 @@ VIM
vping_ssh
^^^^^^^^^
::
+
FUNCTEST.info: Running vPing-SSH test...
vPing_ssh- INFO - Creating image 'functest-vping' from '/home/opnfv/functest/data/cirros-0.3.4-x86_64-disk.img'...
vPing_ssh- INFO - Creating neutron network vping-net...
@@ -41,11 +42,11 @@ vping_ssh
vPing_ssh- INFO - vPing duration:'262.5' s.
vPing_ssh- INFO - Cleaning up...
vPing_ssh- INFO - vPing OK
-::
Tempest
^^^^^^^
::
+
FUNCTEST.info: Running Tempest tests...
run_tempest - INFO - Creating tenant and user for Tempest suite
2016-02-22 16:28:55.743 23855 INFO rally.verification.tempest.tempest [-] Starting: Creating configuration file for Tempest.
@@ -279,11 +280,12 @@ Tempest
run_tempest - INFO - Results: {'timestart': '2016-02-2216:29:01.778060', 'duration': 1082, 'tests': 210, 'failures': 11}
run_tempest - INFO - Pushing results to DB: 'http://testresults.opnfv.org/testapi/results'.
run_tempest - INFO - Deleting tenant and user for Tempest suite)
-::
+
Rally
^^^^^
::
+
FUNCTEST.info: Running Rally benchmark suite...
run_rally - INFO - Starting test scenario "authenticate" ...
run_rally - INFO -
@@ -1581,7 +1583,7 @@ Rally
+===================+============+===============+===========+
| TOTAL: | 00:55:38 | 229 | 99.15% |
+===================+============+===============+===========+
-::
+
SDN Controller
--------------
@@ -1589,6 +1591,7 @@ SDN Controller
ONOS
^^^^
::
+
FUNCTEST.info: Running ONOS test case...
Cloning into '/home/opnfv/repos/functest/testcases/Controllers/ONOS/Teston/CI/OnosSystemTest'...
FUNCvirNetNB - INFO - Creating component Handle: ONOSrest
@@ -2113,4 +2116,3 @@ ONOS
Success Percentage : 100%
Execution Result : 100%
Disconnecting from ONOSrest: <drivers.common.api.controller.onosrestdriver.OnosRestDriver object at 0x7fcc82fc7ad0>
-::
diff --git a/docs/results/joid-os-odl_l2-nofeature-ha.rst b/docs/results/joid-os-odl_l2-nofeature-ha.rst
index dc338ab42..29f4b37c1 100644
--- a/docs/results/joid-os-odl_l2-nofeature-ha.rst
+++ b/docs/results/joid-os-odl_l2-nofeature-ha.rst
@@ -10,6 +10,7 @@ VIM
vping_ssh
^^^^^^^^^
::
+
FUNCTEST.info: Running vPing-SSH test...
vPing_ssh- INFO - Creating image 'functest-vping' from '/home/opnfv/functest/data/cirros-0.3.4-x86_64-disk.img'...
vPing_ssh- INFO - Creating neutron network vping-net...
@@ -41,12 +42,12 @@ vping_ssh
vPing_ssh- INFO - vPing duration:'33.5' s.
vPing_ssh- INFO - Cleaning up...
vPing_ssh- INFO - vPing OK
-::
+
vping_userdata
^^^^^^^^^^^^^^
-
::
+
FUNCTEST.info: Running vPing-userdata test...
vPing_userdata- INFO - Creating image 'functest-vping' from '/home/opnfv/functest/data/cirros-0.3.4-x86_64-disk.img'...
vPing_userdata- INFO - Creating neutron network vping-net...
@@ -87,11 +88,12 @@ vping_userdata
vPing_userdata- INFO - vPing OK
vPing_userdata- INFO - Cleaning up...
vPing_userdata- INFO - Deleting network 'vping-net'...
-::
+
Tempest
^^^^^^^
::
+
FUNCTEST.info: Running Tempest tests...
run_tempest - INFO - Creating tenant and user for Tempest suite
2016-02-23 10:28:17.615 23826 INFO rally.verification.tempest.tempest [-] Starting: Creating configuration file for Tempest.
@@ -325,11 +327,12 @@ Tempest
run_tempest - INFO - Results: {'timestart': '2016-02-2310:28:27.307831', 'duration': 931, 'tests': 210, 'failures': 5}
run_tempest - INFO - Pushing results to DB: 'http://testresults.opnfv.org/testapi/results'.
run_tempest - INFO - Deleting tenant and user for Tempest suite)
-::
+
Rally
^^^^^
::
+
FUNCTEST.info: Running Rally benchmark suite...
run_rally - INFO - Starting test scenario "authenticate" ...
run_rally - INFO -
@@ -1630,7 +1633,7 @@ Rally
+===================+============+===============+===========+
| TOTAL: | 01:25:53 | 232 | 100.00% |
+===================+============+===============+===========+
-::
+
SDN Controller
--------------
@@ -1638,6 +1641,7 @@ SDN Controller
ODL
^^^
::
+
==============================================================================
Basic
==============================================================================
@@ -1734,7 +1738,7 @@ ODL
Neutron :: Test suite for Neutron Plugin | PASS |
18 critical tests, 18 passed, 0 failed
18 tests total, 18 passed, 0 failed
-::
+
Feature tests
-------------
@@ -1742,6 +1746,7 @@ Feature tests
Promise
^^^^^^^
::
+
FUNCTEST.info: Running PROMISE test case...
Promise- INFO - Creating tenant 'promise'...
Promise- INFO - Adding role 'eed9b04b536646c994679496e7b653cc' to tenant 'promise'...
@@ -2182,4 +2187,3 @@ Promise
****************************************
Promise- INFO - Pushing results to DB...
-::
diff --git a/docs/results/overview.rst b/docs/results/overview.rst
index 9a9b68d32..ea71426b0 100644
--- a/docs/results/overview.rst
+++ b/docs/results/overview.rst
@@ -14,54 +14,49 @@ considered to evaluate whether the test case can be run or not. That is why all
the number of test cases may vary from 1 installer to another and from 1
scenario to another.
-The list of scenario supported by installer can be described as follows::
+The list of scenario supported by installer can be described as follows:
- +================+=========+=========+=========+=========+
- | Scenario | Apex | Compass | Fuel | Joid |
- +================+=========+=========+=========+=========+
- | odl_l2 | X | X | X | X |
- +----------------+---------+---------+---------+---------+
- | odl_l3 | X | | X | |
- +----------------+---------+---------+---------+---------+
- | onos | X | X | X | X |
- +----------------+---------+---------+---------+---------+
- | nosdn | | X | X | X |
- +----------------+---------+---------+---------+---------+
- | odl_l2-sfc | X | | | |
- +----------------+---------+---------+---------+---------+
- | odl_l2-bgpvpn | | | X | |
- +----------------+---------+---------+---------+---------+
- | ovs (dpdk) | | | X | |
- +----------------+---------+---------+---------+---------+
++----------------+---------+---------+---------+---------+
+| Scenario | Apex | Compass | Fuel | Joid |
++================+=========+=========+=========+=========+
+| odl_l2 | X | X | X | X |
++----------------+---------+---------+---------+---------+
+| onos | | X | | |
++----------------+---------+---------+---------+---------+
+| nosdn | | X | X | |
++----------------+---------+---------+---------+---------+
+| ovs (dpdk) | | | X | |
++----------------+---------+---------+---------+---------+
The matrix below details the Functest runnable tests versus the installer and
-the scenario::
+the scenario:
- +================+=============+=============+=============+=============+
- | Test cases | Apex | Compass | Fuel | Joid |
- +================+=============+=============+=============+=============+
- | vPing |all scenario | all | all | all |
- +----------------+-------------+-------------+-------------+-------------+
- | vPing userdata | all except | all except | all except | all except |
- | | ONOS | ONOS | ONOS | ONOS |
- +----------------+-------------+-------------+-------------+-------------+
- | Tempest | all | all | all | all |
- +----------------+-------------+-------------+-------------+-------------+
- | Rally | all | all | all | all |
- +----------------+-------------+-------------+-------------+-------------+
- | ODL | all ODL | all ODL | all ODL | all ODL |
- +----------------+-------------+-------------+-------------+-------------+
- | ONOS | ONOS | ONOS | ONOS | ONOS |
- +----------------+-------------+-------------+-------------+-------------+
- | Promise | no | no | all | all |
- +----------------+-------------+-------------+-------------+-------------+
- | vIMS | all except | all except | all except | all except |
- | | ONOS | ONOS | ONOS | ONOS |
- +----------------+-------------+-------------+-------------+-------------+
- | Doctor | all | no | no | no |
- +----------------+-------------+-------------+-------------+-------------+
- | SDNVPN | no | no | bgpvpn | no |
- +----------------+-------------+-------------+-------------+-------------+
++----------------+-------------+-------------+-------------+-------------+
+| Test cases | Apex | Compass | Fuel | Joid |
++================+=============+=============+=============+=============+
+| vPing SSH | all | all | all | all |
++----------------+-------------+-------------+-------------+-------------+
+| vPing userdata | all except | all except | all except | all except |
+| | ONOS | ONOS | ONOS | ONOS |
++----------------+-------------+-------------+-------------+-------------+
+| Tempest | all | all | all | all |
++----------------+-------------+-------------+-------------+-------------+
+| Rally | all | all | all | all |
++----------------+-------------+-------------+-------------+-------------+
+| ODL | all ODL | all ODL | all ODL | all ODL |
++----------------+-------------+-------------+-------------+-------------+
+| ONOS | ONOS | ONOS | ONOS | ONOS |
++----------------+-------------+-------------+-------------+-------------+
+| Promise | no | no | all | all |
++----------------+-------------+-------------+-------------+-------------+
+| vIMS | all except | all except | all except | all except |
+| | ONOS | ONOS | ONOS | ONOS |
++----------------+-------------+-------------+-------------+-------------+
+| Doctor | all | no | no | no |
++----------------+-------------+-------------+-------------+-------------+
+
+all means that the test case is run on all the scenarios related to the
+installer.
Functest results from continuous integration can be found in:
* jenkins logs: https://build.opnfv.org/ci/view/functest/