diff options
| author |   xudan <xudan16@huawei.com> | 2019-12-11 03:13:57 -0500 |
|---|---|---|
| committer | Dan Xu <xudan16@huawei.com> | 2019-12-18 01:25:33 +0000 |
| commit | e7059465f95bdd8222f2547af802346a04c79362 (patch) | |
| tree | 51034d870ebbf21289df9b2077b254919e18ef59 | |
| parent | 2cc610ce13a0dde2e054a25597292d030aa27353 (diff) | |
Update all test specification docs
JIRA: DOVETAIL-795
Change-Id: I807a750c92aa8b50cf3af242b1766e7407063db2
Signed-off-by: xudan <xudan16@huawei.com>
23 files changed, 655 insertions, 466 deletions
diff --git a/docs/testing/developer/testcaserequirements/index.rst b/docs/testing/developer/testcaserequirements/index.rst index 6171a07f..b03a033c 100644 --- a/docs/testing/developer/testcaserequirements/index.rst +++ b/docs/testing/developer/testcaserequirements/index.rst @@ -23,7 +23,7 @@ portability across NFVI instances. All OVP tests are available in open source and are executed in open source test frameworks. -Test case requirements +Test Case Requirements ====================== The following requirements are mandatory for a test to be submitted for @@ -101,7 +101,7 @@ consideration in the OVP test suite: - Use case specification - Test preconditions - Basic test flow execution description and test assertions - - Pass fail criteria + - Pass/Fail criteria - The following things may be documented for the test case: @@ -120,9 +120,9 @@ Dovetail Test Suite Naming Convention Test case naming and structuring must comply with the following conventions. The fully qualified name of a test case must comprise three sections: -`<testproject>.<test_area>.<test_case_name>` +`<test_project>.<test_area>.<test_case_name>` -- **testproject**: The fully qualified test case name must identify the test +- **test_project**: The fully qualified test case name must identify the test project which developed and maintains the test case. - **test_area**: The fully qualified test case name must identify the test case diff --git a/docs/testing/user/testspecification/security_patrole/index.rst b/docs/testing/user/testspecification/security_patrole/index.rst index 3a80ba86..250d8bbd 100644 --- a/docs/testing/user/testspecification/security_patrole/index.rst +++ b/docs/testing/user/testspecification/security_patrole/index.rst @@ -45,7 +45,7 @@ by the tests, review the Python source code accessible via the following links. These tests cover the RBAC tests of image basic operations. Implementation: -`BasicOperationsImagesRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/image/test_images_rbac.py>`_ +`BasicOperationsImagesRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/image/test_images_rbac.py>`_ - patrole_tempest_plugin.tests.api.image.test_images_rbac.BasicOperationsImagesRbacTest.test_create_image - patrole_tempest_plugin.tests.api.image.test_images_rbac.BasicOperationsImagesRbacTest.test_create_image_tag @@ -66,10 +66,9 @@ Implementation: These tests cover the RBAC tests of image namespaces. Implementation: -`ImageNamespacesRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_rbac.py>`_ +`ImageNamespacesRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_rbac.py>`_ - patrole_tempest_plugin.tests.api.image.test_image_namespace_rbac.ImageNamespacesRbacTest.test_create_metadef_namespace -- patrole_tempest_plugin.tests.api.image.test_image_namespace_rbac.ImageNamespacesRbacTest.test_list_metadef_namespaces - patrole_tempest_plugin.tests.api.image.test_image_namespace_rbac.ImageNamespacesRbacTest.test_modify_metadef_namespace @@ -78,7 +77,7 @@ Implementation: These tests cover the RBAC tests of image namespaces objects. Implementation: -`ImageNamespacesObjectsRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_objects_rbac.py>`_ +`ImageNamespacesObjectsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_objects_rbac.py>`_ - patrole_tempest_plugin.tests.api.image.test_image_namespace_objects_rbac.ImageNamespacesObjectsRbacTest.test_create_metadef_object_in_namespace - patrole_tempest_plugin.tests.api.image.test_image_namespace_objects_rbac.ImageNamespacesObjectsRbacTest.test_list_metadef_objects_in_namespace @@ -91,7 +90,7 @@ Implementation: These tests cover the RBAC tests of image namespaces property. Implementation: -`NamespacesPropertyRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_property_rbac.py>`_ +`NamespacesPropertyRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_property_rbac.py>`_ - patrole_tempest_plugin.tests.api.image.test_image_namespace_property_rbac.NamespacesPropertyRbacTest.test_add_md_properties - patrole_tempest_plugin.tests.api.image.test_image_namespace_property_rbac.NamespacesPropertyRbacTest.test_get_md_properties @@ -104,7 +103,7 @@ Implementation: These tests cover the RBAC tests of image namespaces tags. Implementation: -`NamespaceTagsRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_tags_rbac.py>`_ +`NamespaceTagsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/image/test_image_namespace_tags_rbac.py>`_ - patrole_tempest_plugin.tests.api.image.test_image_namespace_tags_rbac.NamespaceTagsRbacTest.test_create_namespace_tag - patrole_tempest_plugin.tests.api.image.test_image_namespace_tags_rbac.NamespaceTagsRbacTest.test_create_namespace_tags @@ -118,7 +117,7 @@ Implementation: These tests cover the RBAC tests of image resource types. Implementation: -`ImageResourceTypesRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/image/test_image_resource_types_rbac.py>`_ +`ImageResourceTypesRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/image/test_image_resource_types_rbac.py>`_ - patrole_tempest_plugin.tests.api.image.test_image_resource_types_rbac.ImageResourceTypesRbacTest.test_add_metadef_resource_type - patrole_tempest_plugin.tests.api.image.test_image_resource_types_rbac.ImageResourceTypesRbacTest.test_get_metadef_resource_type @@ -130,7 +129,7 @@ Implementation: These tests cover the RBAC tests of image member. Implementation: -`ImagesMemberRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/image/test_images_member_rbac.py>`_ +`ImagesMemberRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/image/test_images_member_rbac.py>`_ - patrole_tempest_plugin.tests.api.image.test_images_member_rbac.ImagesMemberRbacTest.test_add_image_member - patrole_tempest_plugin.tests.api.image.test_images_member_rbac.ImagesMemberRbacTest.test_delete_image_member @@ -143,8 +142,8 @@ Implementation: These tests cover the RBAC tests of network agents. Implementation: -`AgentsRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_agents_rbac.py#L24>`_ and -`DHCPAgentSchedulersRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_agents_rbac.py#L147>`_. +`AgentsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_agents_rbac.py#L24>`_ and +`DHCPAgentSchedulersRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_agents_rbac.py#L147>`_. - patrole_tempest_plugin.tests.api.network.test_agents_rbac.AgentsRbacTest.test_show_agent - patrole_tempest_plugin.tests.api.network.test_agents_rbac.AgentsRbacTest.test_update_agent @@ -158,7 +157,7 @@ Implementation: These tests cover the RBAC tests of network floating ips. Implementation: -`FloatingIpsRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_floating_ips_rbac.py>`_ +`FloatingIpsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_floating_ips_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_create_floating_ip - patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_create_floating_ip_floatingip_address @@ -172,25 +171,24 @@ Implementation: These tests cover the RBAC tests of network basic operations. Implementation: -`NetworksRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_networks_rbac.py>`_ +`NetworksRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_networks_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network +- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network_is_default - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network_router_external - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network_shared -- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_subnet - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_delete_network -- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_delete_subnet - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_list_dhcp_agents_on_hosting_network - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_show_network - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_show_network_provider_network_type - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_show_network_provider_physical_network - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_show_network_provider_segmentation_id - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_show_network_router_external -- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_show_subnet - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_update_network +- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_update_network_provider_physical_network +- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_update_network_provider_segmentation_id - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_update_network_router_external - patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_update_network_shared -- patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_update_subnet **Network ports RBAC test:** @@ -198,14 +196,14 @@ Implementation: These tests cover the RBAC tests of network ports. Implementation: -`PortsRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_ports_rbac.py>`_ +`PortsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_ports_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_allowed_address_pairs - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_binding_host_id - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_binding_profile - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_device_owner -- patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_fixed_ips +- patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_fixed_ips_ip_address - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_mac_address - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_create_port_security_enabled - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_delete_port @@ -219,7 +217,7 @@ Implementation: - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_update_port_binding_host_id - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_update_port_binding_profile - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_update_port_device_owner -- patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_update_port_fixed_ips +- patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_update_port_fixed_ips_ip_address - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_update_port_mac_address - patrole_tempest_plugin.tests.api.network.test_ports_rbac.PortsRbacTest.test_update_port_security_enabled @@ -229,7 +227,7 @@ Implementation: These tests cover the RBAC tests of network routers. Implementation: -`RouterRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_routers_rbac.py>`_ +`RouterRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_routers_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_add_router_interface - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_create_router @@ -237,6 +235,7 @@ Implementation: - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_create_router_external_fixed_ips - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_delete_router - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_remove_router_interface +- patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_show_high_availability_router - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_show_router - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_update_router - patrole_tempest_plugin.tests.api.network.test_routers_rbac.RouterRbacTest.test_update_router_enable_snat @@ -250,7 +249,7 @@ Implementation: These tests cover the RBAC tests of network security groups. Implementation: -`SecGroupRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_security_groups_rbac.py>`_ +`SecGroupRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_security_groups_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_create_security_group - patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_create_security_group_rule @@ -259,7 +258,7 @@ Implementation: - patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_list_security_group_rules - patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_list_security_groups - patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_show_security_group_rule -- patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_show_security_groups +- patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_show_security_group - patrole_tempest_plugin.tests.api.network.test_security_groups_rbac.SecGroupRbacTest.test_update_security_group @@ -268,7 +267,7 @@ Implementation: These tests cover the RBAC tests of network service providers. Implementation: -`ServiceProvidersRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_service_providers_rbac.py>`_ +`ServiceProvidersRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_service_providers_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_service_providers_rbac.ServiceProvidersRbacTest.test_list_service_providers @@ -278,7 +277,7 @@ Implementation: These tests cover the RBAC tests of network subnetpools. Implementation: -`SubnetPoolsRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_subnetpools_rbac.py>`_ +`SubnetPoolsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_subnetpools_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_subnetpools_rbac.SubnetPoolsRbacTest.test_create_subnetpool - patrole_tempest_plugin.tests.api.network.test_subnetpools_rbac.SubnetPoolsRbacTest.test_create_subnetpool_shared @@ -293,10 +292,36 @@ Implementation: These tests cover the RBAC tests of network subnets. Implementation: -`SubnetsRbacTest <https://github.com/openstack/patrole/blob/0.2.0/patrole_tempest_plugin/tests/api/network/test_subnets_rbac.py>`_ +`SubnetsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_subnets_rbac.py>`_ - patrole_tempest_plugin.tests.api.network.test_subnets_rbac.SubnetsRbacTest.test_create_subnet - patrole_tempest_plugin.tests.api.network.test_subnets_rbac.SubnetsRbacTest.test_delete_subnet - patrole_tempest_plugin.tests.api.network.test_subnets_rbac.SubnetsRbacTest.test_list_subnets - patrole_tempest_plugin.tests.api.network.test_subnets_rbac.SubnetsRbacTest.test_show_subnet - patrole_tempest_plugin.tests.api.network.test_subnets_rbac.SubnetsRbacTest.test_update_subnet + + +**Network flavors RBAC test:** + +These tests cover the RBAC tests of network flavors. + +Implementation: +`FlavorsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_flavors_rbac.py>`_ + +- patrole_tempest_plugin.tests.api.network.test_flavors_rbac.FlavorsPluginRbacTest.test_create_flavor +- patrole_tempest_plugin.tests.api.network.test_flavors_rbac.FlavorsPluginRbacTest.test_delete_flavor +- patrole_tempest_plugin.tests.api.network.test_flavors_rbac.FlavorsPluginRbacTest.test_list_flavors +- patrole_tempest_plugin.tests.api.network.test_flavors_rbac.FlavorsPluginRbacTest.test_show_flavor +- patrole_tempest_plugin.tests.api.network.test_flavors_rbac.FlavorsPluginRbacTest.test_update_flavor + + +**Network segments RBAC test:** + +These tests cover the RBAC tests of network segments. + +Implementation: +`SegmentsRbacTest <https://github.com/openstack/patrole/blob/0.4.0/patrole_tempest_plugin/tests/api/network/test_network_segments_rbac.py>`_ + +- patrole_tempest_plugin.tests.api.network.test_network_segments_rbac.NetworkSegmentsRbacTest.test_create_network_segments +- patrole_tempest_plugin.tests.api.network.test_network_segments_rbac.NetworkSegmentsRbacTest.test_show_network_segments +- patrole_tempest_plugin.tests.api.network.test_network_segments_rbac.NetworkSegmentsRbacTest.test_update_network_segments diff --git a/docs/testing/user/testspecification/stress/index.rst b/docs/testing/user/testspecification/stress/index.rst index 74961fd1..5483fc93 100644 --- a/docs/testing/user/testspecification/stress/index.rst +++ b/docs/testing/user/testspecification/stress/index.rst @@ -71,7 +71,7 @@ Test Case 1 - Concurrent capacity based on life-cycle ping test Short name ---------- -dovetail.stress.ping +bottlenecks.stress.ping Use case specification ---------------------- @@ -133,7 +133,7 @@ Test execution * Test action 8: Go to *Test action 3* and do the test again to create *N2* VM pairs with PASS VM pairs counted as *S2* * Test action 9: If *S2<N3*, the SUT is marked with FAIL. Otherwise marked with PASS. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' Typical setting of *(N1, N2, N3, P1)* is *(5, 5, 5, 10)*. diff --git a/docs/testing/user/testspecification/tempest_compute/index.rst b/docs/testing/user/testspecification/tempest_compute/index.rst index bd1cace0..ee86ae77 100644 --- a/docs/testing/user/testspecification/tempest_compute/index.rst +++ b/docs/testing/user/testspecification/tempest_compute/index.rst @@ -2,9 +2,9 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) Ericsson AB -=========================================== +================================== Tempest Compute test specification -=========================================== +================================== Scope @@ -86,7 +86,7 @@ The test area is structured in individual tests as listed below. For detailed information on the individual steps and assertions performed by the tests, review the Python source code accessible via the following links: -All these test cases are included in the test case dovetail.tempest.compute of +All these test cases are included in the test case functest.tempest.compute of OVP test suite. @@ -110,8 +110,8 @@ by the tests, review the Python source code accessible via the following links: - `Security Groups test <https://github.com/openstack/tempest/blob/master/tempest/api/compute/security_groups/test_security_groups.py#L23>`_ - tempest.api.compute.security_groups.test_security_groups.SecurityGroupsTestJSON.test_security_groups_create_list_delete -- `Attach Interfaces test <https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_attach_interfaces.py#L32>`_ - - tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_add_remove_fixed_ip +- `Attach Interfaces test <https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_attach_interfaces.py#L347>`_ + - tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesUnderV243Test.test_add_remove_fixed_ip - `Server Addresses test <https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_server_addresses.py#L21>`_ diff --git a/docs/testing/user/testspecification/tempest_identity_v3/index.rst b/docs/testing/user/testspecification/tempest_identity_v3/index.rst index 12e8f308..599427a3 100644 --- a/docs/testing/user/testspecification/tempest_identity_v3/index.rst +++ b/docs/testing/user/testspecification/tempest_identity_v3/index.rst @@ -2,9 +2,9 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) Ericsson AB -=========================================== +====================================== Tempest Identity v3 test specification -=========================================== +====================================== Scope @@ -46,7 +46,7 @@ The test area is structured in individual tests as listed below. For detailed information on the individual steps and assertions performed by the tests, review the Python source code accessible via the following links: -All these test cases are included in the test case dovetail.tempest.identity_v3 of +All these test cases are included in the test case functest.tempest.identity_v3 of OVP test suite. - `Create, Get, Update and Delete Credentials <https://github.com/openstack/tempest/blob/12.2.0/tempest/api/identity/admin/v3/test_credentials.py#L21>`_ diff --git a/docs/testing/user/testspecification/tempest_image/index.rst b/docs/testing/user/testspecification/tempest_image/index.rst index 2b768692..121b38e6 100644 --- a/docs/testing/user/testspecification/tempest_image/index.rst +++ b/docs/testing/user/testspecification/tempest_image/index.rst @@ -2,9 +2,9 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) Ericsson AB -=========================================== +================================ Tempest Image test specification -=========================================== +================================ Scope @@ -32,7 +32,7 @@ The test area is structured in individual tests as listed below. For detailed information on the individual steps and assertions performed by the tests, review the Python source code accessible via the following links: -All these test cases are included in the test case dovetail.tempest.image of +All these test cases are included in the test case functest.tempest.image of OVP test suite. - `Register, Upload, Get Image and Get Image File API's <https://github.com/openstack/tempest/blob/18.0.0/tempest/api/image/v2/test_images.py#L32>`_ diff --git a/docs/testing/user/testspecification/tempest_ipv6/ipv6_api.rst b/docs/testing/user/testspecification/tempest_ipv6/ipv6_api.rst index 60a5633e..b3d2c9dc 100644 --- a/docs/testing/user/testspecification/tempest_ipv6/ipv6_api.rst +++ b/docs/testing/user/testspecification/tempest_ipv6/ipv6_api.rst @@ -9,7 +9,7 @@ Test Case 1 - Create and Delete Bulk Network, IPv6 Subnet and Port Short name ---------- -dovetail.tempest.ipv6_api.bulk_network_subnet_port_create_delete +functest.tempest.ipv6_api.bulk_network_subnet_port_create_delete Use case specification ---------------------- @@ -61,7 +61,7 @@ Test execution * Test action 18: List all networks, verifying the network ids are no longer present * **Test assertion 8:** The two "id" parameters are not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use bulk create commands to create networks, IPv6 subnets and ports on @@ -84,7 +84,7 @@ Test Case 2 - Create, Update and Delete an IPv6 Network and Subnet Short name ----------- -dovetail.tempest.ipv6_api.network_subnet_create_update_delete +functest.tempest.ipv6_api.network_subnet_create_update_delete Use case specification ---------------------- @@ -125,7 +125,7 @@ Test execution * **Test assertion 5:** The network "id" is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to create, update, delete network, IPv6 subnet on the @@ -148,7 +148,7 @@ Test Case 3 - Check External Network Visibility Short name ----------- -dovetail.tempest.ipv6_api.external_network_visibility +functest.tempest.ipv6_api.external_network_visibility Use case specification ---------------------- @@ -189,7 +189,7 @@ Test execution * **Test assertion 4:** There is no subnet of the external network with the configured public network id -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use list commands to list external networks, pre-configured @@ -211,7 +211,7 @@ Test Case 4 - List IPv6 Networks and Subnets Short name ----------- -dovetail.tempest.ipv6_api.network_subnet_list +functest.tempest.ipv6_api.network_subnet_list Use case specification ---------------------- @@ -248,7 +248,7 @@ Test execution * Test action 8: List all networks, verifying the network id is no longer present * **Test assertion 4:** The network "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to use create commands to create network, IPv6 subnet, list @@ -270,7 +270,7 @@ Test Case 5 - Show Details of an IPv6 Network and Subnet Short name ---------- -dovetail.tempest.ipv6_api.network_subnet_show +functest.tempest.ipv6_api.network_subnet_show Use case specification ---------------------- @@ -308,7 +308,7 @@ Test execution * Test action 8: List all networks, verifying the network id is no longer present * **Test assertion 4:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use create commands to create network, IPv6 subnet and show @@ -330,7 +330,7 @@ Test Case 6 - Create an IPv6 Port in Allowed Allocation Pools Short name ---------- -dovetail.tempest.ipv6_api.port_create_in_allocation_pool +functest.tempest.ipv6_api.port_create_in_allocation_pool Use case specification ---------------------- @@ -373,7 +373,7 @@ Test execution * Test action 12: List all networks, verifying the network id is no longer present * **Test assertion 5:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use create commands to create an IPv6 subnet within allowed @@ -395,7 +395,7 @@ Test Case 7 - Create an IPv6 Port with Empty Security Groups Short name ----------- -dovetail.tempest.ipv6_api.port_create_empty_security_group +functest.tempest.ipv6_api.port_create_empty_security_group Use case specification ---------------------- @@ -431,7 +431,7 @@ Test execution * Test action 10: List all networks, verifying the network id is no longer present * **Test assertion 4:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use create commands to create port with @@ -452,7 +452,7 @@ Test Case 8 - Create, Update and Delete an IPv6 Port Short name ---------- -dovetail.tempest.ipv6_api.port_create_update_delete +functest.tempest.ipv6_api.port_create_update_delete Use case specification ---------------------- @@ -489,7 +489,7 @@ Test execution * Test action 9: List all networks, verifying the network id is no longer present * **Test assertion 4:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to use create/update/delete commands to create/update/delete port @@ -511,7 +511,7 @@ Test Case 9 - List IPv6 Ports Short name ---------- -dovetail.tempest.ipv6_api.port_list +functest.tempest.ipv6_api.port_list Use case specification ---------------------- @@ -543,7 +543,7 @@ Test execution * Test action 7: List all networks, verifying the network id is no longer present * **Test assertion 3:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use list commands to list the networks and ports on @@ -564,7 +564,7 @@ Test Case 10 - Show Key/Valus Details of an IPv6 Port Short name ---------- -dovetail.tempest.ipv6_api.port_show_details +functest.tempest.ipv6_api.port_show_details Use case specification ---------------------- @@ -602,7 +602,7 @@ Test execution * Test action 8: List all networks, verifying the network id is no longer present * **Test assertion 4:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use show commands to show port details on the SUT API. @@ -625,7 +625,7 @@ Test Case 11 - Add Multiple Interfaces for an IPv6 Router Short name ----------- -dovetail.tempest.ipv6_api.router_add_multiple_interface +functest.tempest.ipv6_api.router_add_multiple_interface Use case specification ---------------------- @@ -667,7 +667,7 @@ Test execution * **Test assertion 3:** The interfaces, router, IPv6 subnets and networks ids are not present in the lists after deleting -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use bulk create commands to create networks, IPv6 subnets and ports on @@ -690,7 +690,7 @@ Test Case 12 - Add and Remove an IPv6 Router Interface with port_id Short name ---------- -dovetail.tempest.ipv6_api.router_interface_add_remove_with_port +functest.tempest.ipv6_api.router_interface_add_remove_with_port Use case specification ---------------------- @@ -728,7 +728,7 @@ Test execution ones are not found in the list. * **Test assertion 3:** interfaces, ports, routers, subnets and networks are not found in the lists after deleting -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to use add/remove commands to add/remove router interface to the port, @@ -750,7 +750,7 @@ Test Case 13 - Add and Remove an IPv6 Router Interface with subnet_id Short name ---------- -dovetail.tempest.ipv6_api.router_interface_add_remove +functest.tempest.ipv6_api.router_interface_add_remove Use case specification ---------------------- @@ -794,7 +794,7 @@ Test execution * Test action 13: List all networks, verifying the network id is no longer present * **Test assertion 7:** The network "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to add and remove router interface with the subnet id on the @@ -818,7 +818,7 @@ Test Case 14 - Create, Show, List, Update and Delete an IPv6 router Short name ---------- -dovetail.tempest.ipv6_api.router_create_show_list_update_delete +functest.tempest.ipv6_api.router_create_show_list_update_delete Use case specification ---------------------- @@ -856,7 +856,7 @@ Test execution * Test action 7: List all routers, verifying the router id is no longer present * **Test assertion 8:** The "id" parameter is not present in the router list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to create, show, list, update and delete router on @@ -882,7 +882,7 @@ Test Case 15 - Create, List, Update, Show and Delete an IPv6 security group Short name ---------- -dovetail.tempest.ipv6_api.security_group_create_list_update_show_delete +functest.tempest.ipv6_api.security_group_create_list_update_show_delete Use case specification ---------------------- @@ -917,7 +917,7 @@ Test execution * Test action 7: List all security groups, verifying the security group's id is no longer present * **Test assertion 6:** The "id" parameter is not present in the security group list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to create list, update, show and delete security group on @@ -940,7 +940,7 @@ Test Case 16 - Create, Show and Delete IPv6 security group rule Short name ---------- -dovetail.tempest.ipv6_api.security_group_rule_create_show_delete +functest.tempest.ipv6_api.security_group_rule_create_show_delete Use case specification ---------------------- @@ -977,7 +977,7 @@ Test execution * Test action 8: List all security groups, verifying the security group's id is no longer present * **Test assertion 4:** The security group "id" parameter is not present in the list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to create, show, list and delete security group rules on @@ -998,7 +998,7 @@ Test Case 17 - List IPv6 Security Groups Short name ---------- -dovetail.tempest.ipv6_api.security_group_list +functest.tempest.ipv6_api.security_group_list Use case specification ---------------------- @@ -1023,7 +1023,7 @@ Test execution if the default security group exists * **Test assertion 1:** The default security group is in the list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to list security groups on the SUT API. diff --git a/docs/testing/user/testspecification/tempest_ipv6/ipv6_scenario.rst b/docs/testing/user/testspecification/tempest_ipv6/ipv6_scenario.rst index f3a279f0..5871321d 100644 --- a/docs/testing/user/testspecification/tempest_ipv6/ipv6_scenario.rst +++ b/docs/testing/user/testspecification/tempest_ipv6/ipv6_scenario.rst @@ -9,7 +9,7 @@ Test Case 1 - IPv6 Address Assignment - Dual Stack, SLAAC, DHCPv6 Stateless Short name ---------- -dovetail.tempest.ipv6_scenario.dhcpv6_stateless +functest.tempest.ipv6_scenario.dhcpv6_stateless Use case specification ---------------------- @@ -60,7 +60,7 @@ Test execution * Test action 14: List all networks, verifying the id is no longer present * **Test assertion 6:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode @@ -84,7 +84,7 @@ Test Case 2 - IPv6 Address Assignment - Dual Net, Dual Stack, SLAAC, DHCPv6 Stat Short name ---------- -dovetail.tempest.ipv6_scenario.dualnet_dhcpv6_stateless +functest.tempest.ipv6_scenario.dualnet_dhcpv6_stateless Use case specification ---------------------- @@ -138,7 +138,7 @@ Test execution * Test action 16: List all networks, verifying the ids are no longer present * **Test assertion 6:** The two "id" parameters are not present in the network list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'dhcpv6_stateless' @@ -163,7 +163,7 @@ Test Case 3 - IPv6 Address Assignment - Multiple Prefixes, Dual Stack, SLAAC, DH Short name ---------- -dovetail.tempest.ipv6_scenario.multiple_prefixes_dhcpv6_stateless +functest.tempest.ipv6_scenario.multiple_prefixes_dhcpv6_stateless Use case specification ---------------------- @@ -216,7 +216,7 @@ Test execution * Test action 14: List all networks, verifying the id is no longer present * **Test assertion 6:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'dhcpv6_stateless' @@ -241,7 +241,7 @@ Test Case 4 - IPv6 Address Assignment - Dual Net, Multiple Prefixes, Dual Stack, Short name ---------- -dovetail.tempest.ipv6_scenario.dualnet_multiple_prefixes_dhcpv6_stateless +functest.tempest.ipv6_scenario.dualnet_multiple_prefixes_dhcpv6_stateless Use case specification ---------------------- @@ -296,7 +296,7 @@ Test execution * Test action 16: List all networks, verifying the ids are no longer present * **Test assertion 6:** The two "id" parameters are not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'dhcpv6_stateless' @@ -321,7 +321,7 @@ Test Case 5 - IPv6 Address Assignment - Dual Stack, SLAAC Short name ---------- -dovetail.tempest.ipv6_scenario.slaac +functest.tempest.ipv6_scenario.slaac Use case specification ---------------------- @@ -371,7 +371,7 @@ Test execution * Test action 14: List all networks, verifying the id is no longer present * **Test assertion 6:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' @@ -395,7 +395,7 @@ Test Case 6 - IPv6 Address Assignment - Dual Net, Dual Stack, SLAAC Short name ---------- -dovetail.tempest.ipv6_scenario.dualnet_slaac +functest.tempest.ipv6_scenario.dualnet_slaac Use case specification ---------------------- @@ -448,7 +448,7 @@ Test execution * Test action 16: List all networks, verifying the ids are no longer present * **Test assertion 6:** The two "id" parameters are not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' @@ -473,7 +473,7 @@ Test Case 7 - IPv6 Address Assignment - Multiple Prefixes, Dual Stack, SLAAC Short name ---------- -dovetail.tempest.ipv6_scenario.multiple_prefixes_slaac +functest.tempest.ipv6_scenario.multiple_prefixes_slaac Use case specification ---------------------- @@ -524,7 +524,7 @@ Test execution * Test action 14: List all networks, verifying the id is no longer present * **Test assertion 6:** The "id" parameter is not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' @@ -549,7 +549,7 @@ Test Case 8 - IPv6 Address Assignment - Dual Net, Dual Stack, Multiple Prefixes, Short name ---------- -dovetail.tempest.ipv6_scenario.dualnet_multiple_prefixes_slaac +functest.tempest.ipv6_scenario.dualnet_multiple_prefixes_slaac Use case specification ---------------------- @@ -602,7 +602,7 @@ Test execution * Test action 16: List all networks, verifying the ids are no longer present * **Test assertion 6:** The two "id" parameters are not present in the network list -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test evaluates the ability to assign IPv6 addresses in ipv6_ra_mode 'slaac' diff --git a/docs/testing/user/testspecification/tempest_multi_node_scheduling/index.rst b/docs/testing/user/testspecification/tempest_multi_node_scheduling/index.rst index abe4e88e..f414de61 100644 --- a/docs/testing/user/testspecification/tempest_multi_node_scheduling/index.rst +++ b/docs/testing/user/testspecification/tempest_multi_node_scheduling/index.rst @@ -53,7 +53,7 @@ on multiple nodes. Each test case is able to run independently, i.e. irrelevant the state created by a previous test. Specifically, every test performs clean-up operations which return the system to the same state as before the test. -All these test cases are included in the test case dovetail.tempest.multi_node_scheduling of +All these test cases are included in the test case functest.tempest.multi_node_scheduling of OVP test suite. Test Descriptions @@ -145,7 +145,7 @@ Test execution that every server ended up on a different host * Test action 6: Delete the created servers -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of VM resource scheduling. @@ -190,7 +190,7 @@ Test execution * Test action 5: List all server groups * **Test assertion 4:** SERG1 and SERG2 are not in the list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of creating and deleting server groups with the same name and policy. @@ -233,7 +233,7 @@ Test execution * Test action 3: Delete SERG1 and list all server groups * **Test assertion 3:** SERG1 is not in the list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of creating and deleting server group with affinity policy. @@ -275,7 +275,7 @@ Test execution * Test action 3: Delete SERG1 and list all server groups * **Test assertion 3:** SERG1 is not in the list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of creating and deleting server group with anti-affinity policy. @@ -316,7 +316,7 @@ Test execution * **Test assertion 1:** SERG1 is in the list * Test action 4: Delete SERG1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of listing server groups. @@ -358,7 +358,7 @@ Test execution * **Test assertion 1:** All values in D1 are the same as the values in D2 * Test action 4: Delete SERG1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of showing server group details. diff --git a/docs/testing/user/testspecification/tempest_network_api/index.rst b/docs/testing/user/testspecification/tempest_network_api/index.rst index 31ec6763..4fc47208 100644 --- a/docs/testing/user/testspecification/tempest_network_api/index.rst +++ b/docs/testing/user/testspecification/tempest_network_api/index.rst @@ -2,9 +2,9 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) Ericsson AB -=========================================== +====================================== Tempest Network API test specification -=========================================== +====================================== Scope @@ -82,7 +82,7 @@ The test area is structured in individual tests as listed below. For detailed information on the individual steps and assertions performed by the tests, review the Python source code accessible via the following links: -All these test cases are included in the test case dovetail.tempest.network of +All these test cases are included in the test case functest.tempest.network of OVP test suite. diff --git a/docs/testing/user/testspecification/tempest_network_scenario/index.rst b/docs/testing/user/testspecification/tempest_network_scenario/index.rst index afefe8c7..feee105d 100644 --- a/docs/testing/user/testspecification/tempest_network_scenario/index.rst +++ b/docs/testing/user/testspecification/tempest_network_scenario/index.rst @@ -58,7 +58,7 @@ test case is able to run independently, i.e. irrelevant of the state created by a previous test. Specifically, every test performs clean-up operations which return the system to the same state as before the test. -All these test cases are included in the test case dovetail.tempest.network_scenario of +All these test cases are included in the test case functest.tempest.network_scenario of OVP test suite. Test Descriptions @@ -149,7 +149,7 @@ Test execution * **Test assertion 5:** Ping FIP1 and SSH to VM2 via FIP1 successfully * Test action 8: Delete SG1, NET1, SUBNET1, R1, VM1, VM2 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of basic network operations. @@ -220,7 +220,7 @@ Test execution * **Test assertion 2:** Ping NET2's internal gateway successfully * Test action 8: Delete SG1, NET1, NET2, SUBNET1, SUBNET2, R1, NIC2, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of adding network to an active VM. @@ -278,7 +278,7 @@ Test execution retrieve the VM1's configured dns and verify it has been successfully updated * Test action 7: Delete SG1, NET1, SUBNET1, R1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the functionality of updating subnet's configurations. @@ -336,7 +336,7 @@ Test execution * **Test assertion 6:** Ping FIP1 via SSHCLNT1 successfully * Test action 10: Delete SG1, NET1, SUBNET1, R1, SSHCLNT1, VM1, VM2 and FIP1, FIP2 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the VM public and project connectivity status by changing VM port @@ -395,7 +395,7 @@ Test execution * **Test assertion 3:** Ping FIP1 and SSH to VM1 with FIP1 successfully * Test action 8: Delete SG1, NET1, SUBNET1, R1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the router public connectivity status by changing diff --git a/docs/testing/user/testspecification/tempest_network_security/index.rst b/docs/testing/user/testspecification/tempest_network_security/index.rst index 3a28801d..6934be1f 100644 --- a/docs/testing/user/testspecification/tempest_network_security/index.rst +++ b/docs/testing/user/testspecification/tempest_network_security/index.rst @@ -53,7 +53,7 @@ port security. Each test case is able to run independently, i.e. irrelevant of the state created by a previous test. Specifically, every test performs clean-up operations which return the system to the same state as before the test. -All these test cases are included in the test case dovetail.tempest.network_security of +All these test cases are included in the test case functest.tempest.network_security of OVP test suite. Test Descriptions @@ -145,7 +145,7 @@ Test execution * **Test assertion 2:** The ping operation is successful * Test action 16: Delete SG1, NET1, NET2, SUBNET1, SUBNET2, R1, VM1, VM2 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to prevent MAC spoofing by using port security. @@ -206,7 +206,7 @@ Test execution * Test action 14: Delete SG1, SG2, SG3, SG4, NET1, NET2, SUBNET1, SUBNET2, R1, R2, VM1, VM2, FIP1 and FIP2 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability of the security group to filter packets cross tenant. @@ -263,7 +263,7 @@ Test execution * **Test assertion 2:** The ping operation is successful * Test action 11: Delete SG1, SG2, NET1, SUBNET1, R1, VM1, VM2 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability of the security group to filter packets in one tenant. @@ -319,7 +319,7 @@ Test execution * **Test assertion 3:** Can SSH to VM1 successfully * Test action 11: Delete SG1, SG2, NET1, SUBNET1, R1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability of multiple security groups to filter packets. @@ -379,7 +379,7 @@ Test execution * **Test assertion 2:** The ping operation is successful * Test action 12: Delete SG1, SG2, NET1, SUBNET1, R1, VM1, VM2 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability of port security to disable security group. @@ -436,7 +436,7 @@ Test execution * **Test assertion 2:** The ping operation is successful * Test action 13: Delete SG1, SG2, SG3, NET1, SUBNET1, R1, VM1, VM2 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to update port with a new security group. diff --git a/docs/testing/user/testspecification/tempest_osinterop/index.rst b/docs/testing/user/testspecification/tempest_osinterop/index.rst index 6773275e..d2a54e86 100644 --- a/docs/testing/user/testspecification/tempest_osinterop/index.rst +++ b/docs/testing/user/testspecification/tempest_osinterop/index.rst @@ -3,11 +3,11 @@ .. (c) Huawei Technologies Co.,Ltd and others ============================================= -OpenStack Interoperability test specification +OpenStack Interoperability Test Specification ============================================= The test cases documented here are the API test cases in the OpenStack -Interop guideline 2017.09 as implemented by the RefStack client. +Interop guideline 2018.11 as implemented by the RefStack client. References ================ @@ -16,9 +16,9 @@ References - https://wiki.openstack.org/wiki/Governance/InteropWG -- OpenStack Interoperability guidelines (version 2017.09) +- OpenStack Interoperability guidelines (version 2018.11) - - https://github.com/openstack/interop/blob/master/2017.09.json + - https://github.com/openstack/interop/blob/master/2018.11.json - Refstack client diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_compute.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_compute.rst index 63175306..3e663d98 100644 --- a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_compute.rst +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_compute.rst @@ -11,7 +11,7 @@ Scope The VIM compute operations test area evaluates the ability of the system under test to support VIM compute operations. The test cases documented here are the -compute API test cases in the OpenStack Interop guideline 2017.09 as implemented +compute API test cases in the OpenStack Interop guideline 2018.11 as implemented by the RefStack client. These test cases will evaluate basic OpenStack (as a VIM) compute operations, including: @@ -22,7 +22,7 @@ compute operations, including: - Basic server operations - Volume management operations -Definitions and abbreviations +Definitions and Abbreviations ============================= The following terms and abbreviations are used in conjunction with this test area @@ -30,14 +30,15 @@ The following terms and abbreviations are used in conjunction with this test are - API - Application Programming Interface - NFVi - Network Functions Virtualization infrastructure - SUT - System Under Test -- UUID - Universally Unique Identifier +- UUID - Universally Unique IDentifier - VIM - Virtual Infrastructure Manager - VM - Virtual Machine System Under Test (SUT) ======================= -The system under test is assumed to be the NFVi and VIM deployed with a Pharos compliant infrastructure. +The system under test is assumed to be the NFVi and VIM deployed with a Pharos +compliant infrastructure. Test Area Structure ==================== @@ -50,7 +51,7 @@ the same state as before the test. For brevity, the test cases in this test area are summarized together based on the operations they are testing. -All these test cases are included in the test case dovetail.tempest.osinterop of +All these test cases are included in the test case functest.tempest.osinterop of OVP test suite. Test Descriptions @@ -759,3 +760,48 @@ Post conditions --------------- N/A + +-------------------------------------------------------- +Test Case 10 - Keypair operations within the Compute API +-------------------------------------------------------- + +Test case specification +----------------------- + +This test case evaluates the Compute API ability of creating keypair with type, +the reference is, + +tempest.api.compute.keypairs.test_keypairs_v22.KeyPairsV22TestJSON.test_keypairsv22_create_list_show_with_type + +Test preconditions +------------------ + +* Compute server extension API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create a keypair with type 'x509' and a random name +* **Test assertion 1:** The keypair type received in the response body is equal to 'x509' +* Test action 2: Show the details of this created keypair +* **Test assertion 2:** The keypair type received in the response body is equal to 'x509' +* Test action 3: List all keypairs and find the one with the same name as given in test action 1 +* **Test assertion 3:** The keypair type of this keypair is equal to 'x509' + +Pass / fail criteria +'''''''''''''''''''' + +This test evaluates the functionality of keypair operations within the Compute API. +Specifically, the test verifies that: + +* Can create keypair by specifying keypair type. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_identity.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_identity.rst index 6c0d23b7..34a71168 100644 --- a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_identity.rst +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_identity.rst @@ -14,7 +14,7 @@ support VIM identity operations. The tests in this area will evaluate API discovery operations within the Identity v3 API, auth operations within the Identity API. -Definitions and abbreviations +Definitions and Abbreviations ============================= The following terms and abbreviations are used in conjunction with this test area @@ -34,7 +34,7 @@ Test Area Structure The test area is structured based on VIM identity operations. Each test case is able to run independently, i.e. irrelevant of the state created by a previous test. -All these test cases are included in the test case dovetail.tempest.osinterop of +All these test cases are included in the test case functest.tempest.osinterop of OVP test suite. Dependency Description @@ -42,15 +42,15 @@ Dependency Description The VIM identity operations test cases are a part of the OpenStack interoperability tempest test cases. For Fraser based dovetail release, the -OpenStack interoperability guidelines (version 2017.09) is adopted, which is +OpenStack interoperability guidelines (version 2018.11) is adopted, which is valid for Mitaka, Newton, Ocata and Pike releases of Openstack. Test Descriptions ================= ----------------------------------------------------- -API discovery operations within the Identity v3 API ----------------------------------------------------- +----------------------------------------------------------------- +Test Case 1 - API discovery operations within the Identity v3 API +----------------------------------------------------------------- Use case specification ----------------------- @@ -79,7 +79,7 @@ Test execution 'stable', 'experimental', 'supported', 'deprecated' are all of the identity api 'status' values. -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test case passes if all test action steps execute successfully and all assertions @@ -91,14 +91,15 @@ Post conditions None ------------------------------------------- -Auth operations within the Identity API ------------------------------------------- +----------------------------------------------------- +Test Case 2 - Auth operations within the Identity API +----------------------------------------------------- Use case specification ----------------------- tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token +tempest.api.identity.v3.test_tokens.TokensV3Test.test_validate_token Test preconditions ------------------- @@ -119,8 +120,16 @@ Test execution passes if it is equal to the user_name which is used to get token. * Test action 4: Get the method in getting token response message, the test passes if it is equal to the password which is used to get token. +* Test action 5: Get the token by system credentials and show the token, + the test passes if the response bodies of the get and show operations are the same. +* Test action 6: Get the user_id in showing token response message, the test + passes if it is equal to the user_id which is used to get token. +* Test action 7: Get the username in showing token response message, the test + passes if it is equal to the username which is used to get token. +* Test action 8: Delete this token by non-admin compute client, the test passes + if it raises a NotFound exception. -Pass / fail criteria +Pass / Fail criteria ''''''''''''''''''''' This test case passes if all test action steps execute successfully and all assertions @@ -132,3 +141,39 @@ Post conditions None +-------------------------------------------------------- +Test Case 3 - Catalog operations within the Identity API +-------------------------------------------------------- + +Use case specification +----------------------- + +tempest.api.identity.v3.test_catalog.IdentityCatalogTest.test_catalog_standardization + +Test preconditions +------------------- + +None + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +''''''''''''''' + +* Test action 1: Show all catalogs by non-admin catalog client, the test passes + if the catalog types getting in the show response message equal to the + standard service values. Standard catalog types of 'keystone', 'nova', 'glance' and + 'swift' should be 'identity', 'compute', 'image' and 'object-store' respectively. + +Pass / Fail criteria +''''''''''''''''''''' + +This test case passes if all test action steps execute successfully and all assertions +are affirmed. If any test steps fails to execute successfully or any of the assertions +is not met, the test case fails. + +Post conditions +--------------- + +None diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_image.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_image.rst index c5926b02..b6cdb77f 100644 --- a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_image.rst +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_image.rst @@ -11,11 +11,11 @@ Scope The VIM image test area evaluates the ability of the system under test to support VIM image operations. The test cases documented here are the Image API test cases -in the Openstack Interop guideline 2017.09 as implemented by the Refstack client. +in the Openstack Interop guideline 2018.11 as implemented by the Refstack client. These test cases will evaluate basic Openstack (as a VIM) image operations including image creation, image list, image update and image deletion capabilities using Glance v2 API. -Definitions and abbreviations +Definitions and Abbreviations ============================= The following terms and abbreviations are used in conjunction with this test area @@ -40,7 +40,7 @@ to run independently, i.e. irrelevant of the state created by a previous test. For brevity, the test cases in this test area are summarized together based on the operations they are testing. -All these test cases are included in the test case dovetail.tempest.osinterop of +All these test cases are included in the test case functest.tempest.osinterop of OVP test suite. Test Descriptions @@ -62,9 +62,9 @@ Images: https://docs.openstack.org/api-ref/image/v2/ - add image tag - delete image tag ---------------------------------------- -Image get tests using the Glance v2 API ---------------------------------------- +----------------------------------------------------- +Test Case 1 - Image get tests using the Glance v2 API +----------------------------------------------------- Test case specification ----------------------- @@ -104,7 +104,7 @@ Test execution whether the 6 images' ids are not in the show list. * **Test assertion 6:** The 6 images' ids are not found in the show list. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' The first two test cases evaluate the ability to use Glance v2 API to show image @@ -124,9 +124,9 @@ Post conditions None --------------------------------------- -CRUD image operations in Images API v2 --------------------------------------- +---------------------------------------------------- +Test Case 2 - CRUD image operations in Images API v2 +---------------------------------------------------- Test case specification ----------------------- @@ -148,7 +148,7 @@ Test execution * Test action 2: List all images and check whether the ids listed are in the created images list. * **Test assertion 1:** The ids get from the list images API are in the created images list. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the ability to use Glance v2 API to list images. @@ -163,9 +163,9 @@ Post conditions None ----------------------------------------- -Image list tests using the Glance v2 API ----------------------------------------- +------------------------------------------------------ +Test Case 3 - Image list tests using the Glance v2 API +------------------------------------------------------ Test case specification ----------------------- @@ -224,7 +224,7 @@ Test execution the 6 ids are not in the show list. * **Test assertion 8:** The stored 6 ids are not found in the show list. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the ability to use Glance v2 API to list images with @@ -245,9 +245,9 @@ Post conditions None ------------------------------------------- -Image update tests using the Glance v2 API ------------------------------------------- +-------------------------------------------------------- +Test Case 4 - Image update tests using the Glance v2 API +-------------------------------------------------------- Test case specification ----------------------- @@ -291,7 +291,7 @@ Test execution and check whether the ids are not in the show list. * **Test assertion 6:** The two ids are not found in the show list. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the ability to use Glance v2 API to update images with @@ -308,9 +308,9 @@ Post conditions None --------------------------------------------- -Image deletion tests using the Glance v2 API --------------------------------------------- +---------------------------------------------------------- +Test Case 5 - Image deletion tests using the Glance v2 API +---------------------------------------------------------- Test case specification ----------------------- @@ -349,7 +349,7 @@ Test execution and check whether the ids are in the list. * **Test assertion 5:** The two ids are not found in the list. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' The first three test cases evaluate the ability to use Glance v2 API to delete images diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst index 4e3e5a95..d8e7413b 100644 --- a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_network.rst @@ -11,11 +11,11 @@ Scope The VIM network test area evaluates the ability of the system under test to support VIM network operations. The test cases documented here are the network API test cases -in the Openstack Interop guideline 2017.09 as implemented by the Refstack client. +in the Openstack Interop guideline 2018.11 as implemented by the Refstack client. These test cases will evaluate basic Openstack (as a VIM) network operations including basic CRUD operations on L2 networks, L2 network ports and security groups. -Definitions and abbreviations +Definitions and Abbreviations ============================= The following terms and abbreviations are used in conjunction with this test area @@ -42,7 +42,7 @@ the same state as before the test. For brevity, the test cases in this test area are summarized together based on the operations they are testing. -All these test cases are included in the test case dovetail.tempest.osinterop of +All these test cases are included in the test case functest.tempest.osinterop of OVP test suite. Test Descriptions @@ -84,9 +84,9 @@ Network: https://docs.openstack.org/api-ref/network/v2/index.html - show security group rule - delete security group rule ---------------------------------------------------------- -Basic CRUD operations on L2 networks and L2 network ports ---------------------------------------------------------- +----------------------------------------------------------------------- +Test Case 1 - Basic CRUD operations on L2 networks and L2 network ports +----------------------------------------------------------------------- Test case specification ----------------------- @@ -100,6 +100,7 @@ tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_without tempest.api.network.test_networks.NetworksTest.test_create_delete_subnet_all_attributes tempest.api.network.test_networks.NetworksTest.test_create_update_delete_network_subnet tempest.api.network.test_networks.NetworksTest.test_delete_network_with_subnet +tempest.api.network.test_networks.NetworksTest.test_external_network_visibility tempest.api.network.test_networks.NetworksTest.test_list_networks tempest.api.network.test_networks.NetworksTest.test_list_networks_fields tempest.api.network.test_networks.NetworksTest.test_list_subnets @@ -114,6 +115,7 @@ tempest.api.network.test_ports.PortsTestJSON.test_create_port_in_allowed_allocat tempest.api.network.test_ports.PortsTestJSON.test_create_update_delete_port tempest.api.network.test_ports.PortsTestJSON.test_list_ports tempest.api.network.test_ports.PortsTestJSON.test_list_ports_fields +tempest.api.network.test_ports.PortsTestJSON.test_port_list_filter_by_router_id tempest.api.network.test_ports.PortsTestJSON.test_show_port tempest.api.network.test_ports.PortsTestJSON.test_show_port_fields @@ -163,55 +165,64 @@ Test execution found after deletion * Test action 9: Create a network and create a subnet of this network, then delete this network * **Test assertion 9:** The subnet has also been deleted after deleting the network -* Test action 10: Create a network and list all networks -* **Test assertion 10:** The network created is found in the list -* Test action 11: Create a network and list networks with the id and name of the created network -* **Test assertion 11:** The id and name of the list network equal to the created network's id and name -* Test action 12: Create a network and create a subnet of this network, then list all subnets -* **Test assertion 12:** The subnet created is found in the list -* Test action 13: Create a network and create a subnet of this network, then list subnets with +* Test action 10: List all external networks, find the one with the same public_network_id + as defined in tempest.conf and list its subnets +* **Test assertion 10:** The external network can be found, no internal network got by the + list operation, if this external network is shared, the subnets list is not empty, + otherwise, it should be empty +* Test action 11: Create a network and list all networks +* **Test assertion 11:** The network created is found in the list +* Test action 12: Create a network and list networks with the id and name of the created network +* **Test assertion 12:** The id and name of the list network equal to the created network's id and name +* Test action 13: Create a network and create a subnet of this network, then list all subnets +* **Test assertion 13:** The subnet created is found in the list +* Test action 14: Create a network and create a subnet of this network, then list subnets with the id and network_id of the created subnet -* **Test assertion 13:** The id and network_id of the list subnet equal to the created subnet -* Test action 14: Create a network and show network's details with the id of the created network -* **Test assertion 14:** The id and name returned in the response equal to the created network's id and name -* Test action 15: Create a network and just show network's id and name info with the id of the created network -* **Test assertion 15:** The keys returned in the response are only id and name, and the values +* **Test assertion 14:** The id and network_id of the list subnet equal to the created subnet +* Test action 15: Create a network and show network's details with the id of the created network +* **Test assertion 15:** The id and name returned in the response equal to the created network's id and name +* Test action 16: Create a network and just show network's id and name info with the id of the created network +* **Test assertion 16:** The keys returned in the response are only id and name, and the values of all the keys equal to network's id and name -* Test action 16: Create a network and create a subnet of this network, then show subnet's details +* Test action 17: Create a network and create a subnet of this network, then show subnet's details with the id of the created subnet -* **Test assertion 16:** The id and cidr info returned in the response equal to the created +* **Test assertion 17:** The id and cidr info returned in the response equal to the created subnet's id and cidr -* Test action 17: Create a network and create a subnet of this network, then show subnet's id and +* Test action 18: Create a network and create a subnet of this network, then show subnet's id and network_id info with the id of the created subnet -* **Test assertion 17:** The keys returned in the response are just id and network_id, and the values +* **Test assertion 18:** The keys returned in the response are just id and network_id, and the values of all the keys equal to subnet's id and network_id -* Test action 18: Create a network and create a subnet of this network, then update subnet's +* Test action 19: Create a network and create a subnet of this network, then update subnet's name, host_routes, dns_nameservers and gateway_ip -* **Test assertion 18:** The name, host_routes, dns_nameservers and gateway_ip returned in the +* **Test assertion 19:** The name, host_routes, dns_nameservers and gateway_ip returned in the response equal to the values used to update the subnet -* Test action 19: Create 2 networks and bulk create 2 ports with the ids of the created networks -* **Test assertion 19:** The network_id of each port equals to the one used to create the port and +* Test action 20: Create 2 networks and bulk create 2 ports with the ids of the created networks +* **Test assertion 20:** The network_id of each port equals to the one used to create the port and the admin_state_up of each port is True -* Test action 20: Create a network and create a subnet of this network by setting allocation_pools, +* Test action 21: Create a network and create a subnet of this network by setting allocation_pools, then create a port with the created network's id -* **Test assertion 20:** The ip_address of the created port is in the range of the allocation_pools -* Test action 21: Create a network and create a port with its id, then update the port's name and +* **Test assertion 21:** The ip_address of the created port is in the range of the allocation_pools +* Test action 22: Create a network and create a port with its id, then update the port's name and set its admin_state_up to be False -* **Test assertion 21:** The name returned in the response equals to the name used to update +* **Test assertion 22:** The name returned in the response equals to the name used to update the port and the port's admin_state_up is False -* Test action 22: Create a network and create a port with its id, then list all ports -* **Test assertion 22:** The created port is found in the list -* Test action 23: Create a network and create a port with its id, then list ports with the id - and mac_address of the created port +* Test action 23: Create a network and create a port with its id, then list all ports * **Test assertion 23:** The created port is found in the list -* Test action 24: Create a network and create a port with its id, then show the port's details -* **Test assertion 24:** The key 'id' is in the details -* Test action 25: Create a network and create a port with its id, then show the port's id +* Test action 24: Create a network and create a port with its id, then list ports with the id + and mac_address of the created port +* **Test assertion 24:** The created port is found in the list +* Test action 25: Create a network and create a subnet, port with its id, create a router + and add this port as this router's interface, then list ports with this router id +* **Test assertion 25:** The number of the ports list is 1, the port id and device_id + getting with the list operation are the same as the ones got when creating them +* Test action 26: Create a network and create a port with its id, then show the port's details +* **Test assertion 26:** The key 'id' is in the details +* Test action 27: Create a network and create a port with its id, then show the port's id and mac_address info with the port's id -* **Test assertion 25:** The keys returned in the response are just id and mac_address, +* **Test assertion 27:** The keys returned in the response are just id and mac_address, and the values of all the keys equal to port's id and mac_address -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' These test cases evaluate the ability of basic CRUD operations on L2 networks and L2 network ports. @@ -232,9 +243,9 @@ Post conditions N/A ----------------------------------------- -Basic CRUD operations on security groups ----------------------------------------- +------------------------------------------------------ +Test Case 2 - Basic CRUD operations on security groups +------------------------------------------------------ Test case specification ----------------------- @@ -321,7 +332,7 @@ Test execution * Test action 19: Generate a random uuid and use this id to show security group rule * **Test assertion 19:** Failed to show security group rule because of nonexistent id of security group rule -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' These test cases evaluate the ability of Basic CRUD operations on security groups and security group rules. @@ -342,9 +353,9 @@ Post conditions N/A -------------------------------- -CRUD operations on subnet pools -------------------------------- +--------------------------------------------- +Test Case 3 - CRUD operations on subnet pools +--------------------------------------------- Test case specification ----------------------- @@ -371,7 +382,7 @@ Test execution * Test action 4: Delete SNP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' These test cases evaluate the ability of Basic CRUD operations on subnetpools. @@ -385,3 +396,131 @@ Post conditions --------------- N/A + +---------------------------------------- +Test Case 4 - CRUD operations on routers +---------------------------------------- + +Test case specification +----------------------- + +tempest.api.network.test_routers.RoutersTest.test_add_multiple_router_interfaces +tempest.api.network.test_routers.RoutersTest.test_add_remove_router_interface_with_port_id +tempest.api.network.test_routers.RoutersTest.test_add_remove_router_interface_with_subnet_id +tempest.api.network.test_routers.RoutersTest.test_create_show_list_update_delete_router +tempest.api.network.test_routers.RoutersTest.test_update_delete_extra_route +tempest.api.network.test_routers.RoutersTest.test_update_router_admin_state +tempest.api.network.test_routers_negative.RoutersNegativeTest.test_add_router_interfaces_on_overlapping_subnets_returns_400 +tempest.api.network.test_routers_negative.RoutersNegativeTest.test_delete_non_existent_router_returns_404 +tempest.api.network.test_routers_negative.RoutersNegativeTest.test_router_add_gateway_invalid_network_returns_404 +tempest.api.network.test_routers_negative.RoutersNegativeTest.test_router_add_gateway_net_not_external_returns_400 +tempest.api.network.test_routers_negative.RoutersNegativeTest.test_router_remove_interface_in_use_returns_409 +tempest.api.network.test_routers_negative.RoutersNegativeTest.test_show_non_existent_router_returns_404 +tempest.api.network.test_routers_negative.RoutersNegativeTest.test_update_non_existent_router_returns_404 + +Test preconditions +------------------ + +Neutron is available. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: Create 2 networks NET1 and NET2, create SubNet1 of NET1 and SubNet2 with cidr of NET2, + create a router R1, add SubNet1 and SubNet2 to be R1's interfaces and get port details +* **Test assertion 1:** The device_id and subnet_id of the port equals to the router id + and subnet id getting from the creating response body +* Test action 2: Create a network NET1, create SubNet1 of NET1, create P1 of NET1, create a router R1, + add P1 to be its interface, store the response body B1, show P1 and store the response body B2 and + remove the interface with port id +* **Test assertion 2:** B1 includes keys 'subnet_id' and 'port_id', 'device_id' of B2 equals to router id +* Test action 3: Create a network NET1, create SubNet1 of NET1, create a router R1, add SubNet1 to be its + interface, store the response body B1, show port details with 'port_id' in B1 +* **Test assertion 3:** B1 includes keys 'subnet_id' and 'port_id', 'device_id' equals to router id +* Test action 4: Create a router R1 with name, admin_state_up False and external_network_id Ext-Net, + store the request body B1, show R1 with 'id' in B1, list all routers, update R1's name +* **Test assertion 4:** 'name', 'admin_state_up' and 'network_id' in B1 equal to the name, False and Ext-Net, + 'name' in show details equals to 'name' in B1, 'network_id' in show details equals to Ext-Net, + R1 is in the router list, the 'name' has been updated +* Test action 5: Create a router R1 with admin_state_up=True, create 4 networks and 4 subnets with different + cidr, add these 4 subnets to R1 as its interfaces, update R1 by given routes and show R1, delete extra + route of R1 and then show R1 +* **Test assertion 5:** The number of routes queals to 4 and routes is empty after deletion +* Test action 6: Create a router R1, update 'admin_state_up' of R1 to be True +* **Test assertion 6:** 'admin_state_up' of R1 is False, 'admin_state_up' of R1 is True after updating +* Test action 7: Create 2 networks NET1 and NET2, create SubNet1 of NET1 and SubNet2 of NET2, + create a router R1, add SubNet1 to be R1's interface and then trying to add SubNet2 to be R1's interface +* **Test assertion 7:** It raises a BadRequest exception when trying to add SubNet2 +* Test action 8: Try to delete router with a random name +* **Test assertion 8:** It raises a NotFound exception when trying delete operation +* Test action 9: Create a router R1 and try to update it with wrong external_gateway_info +* **Test assertion 9:** It raises a NotFound exception when trying to update R1 +* Test action 10: Create an internal network NET1, create SubNet1 with cidr of NET1, + create a router R1 and try to update it with NET1 as external_gateway_info +* **Test assertion 10:** It raises a BadRequest exception when trying to update R1 +* Test action 11: Create a network NET1, create SubNet1 of NET1, create a router R1, add SubNet1 to + the interface of R1 and try to delete R1 +* **Test assertion 11:** It raises a Conflict exception when trying to delete R1 +* Test action 12: Try to show router with a random name +* **Test assertion 12:** It raises a NotFound exception when trying to show router +* Test action 13: Try to update router with a random name +* **Test assertion 13:** It raises a NotFound exception when trying to update router + + +Pass / Fail criteria +'''''''''''''''''''' + +These test cases evaluate the ability of Basic CRUD operations on routers. +Specifically it verifies that: + +* Routers can be created, updated, shown and deleted. +* Can not show, update and delete non existent router. +* Can not remove interface in use. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A + +---------------------------------------------- +Test Case 5 - List versions within Network API +---------------------------------------------- + +Test case specification +----------------------- + +tempest.api.network.test_versions.NetworksApiDiscovery.test_api_version_resources + +Test preconditions +------------------ + +Neutron is available. + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: List network API versions +* **Test assertion 1:** The network API version is 'v2.0' + +Pass / Fail criteria +'''''''''''''''''''' + +This test case evaluates the ability of listing network API versions. +Specifically it verifies that: + +* The network API version is 'v2.0'. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A diff --git a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_volume.rst b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_volume.rst index 191427de..263e87cc 100644 --- a/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_volume.rst +++ b/docs/testing/user/testspecification/tempest_osinterop/tempest_osinterop_volume.rst @@ -11,7 +11,7 @@ Scope The VIM volume operations test area evaluates the ability of the system under test to support VIM volume operations. The test cases documented here are the -volume API test cases in the OpenStack Interop guideline 2017.09 as implemented +volume API test cases in the OpenStack Interop guideline 2018.11 as implemented by the RefStack client. These test cases will evaluate basic OpenStack (as a VIM) volume operations, including: @@ -24,7 +24,7 @@ volume operations, including: - Volume metadata operations - Volume snapshot operations -Definitions and abbreviations +Definitions and Abbreviations ============================= The following terms and abbreviations are used in conjunction with this test area @@ -51,7 +51,7 @@ the same state as before the test. For brevity, the test cases in this test area are summarized together based on the operations they are testing. -All these test cases are included in the test case dovetail.tempest.osinterop of +All these test cases are included in the test case functest.tempest.osinterop of OVP test suite. Test Descriptions @@ -77,49 +77,8 @@ Block storage: https://docs.openstack.org/api-ref/block-storage/ - update snapshot - delete snapshot ------------------------------------------------------ -Test Case 1 - Upload volumes with Cinder v2 or v3 API ------------------------------------------------------ - -Test case specification ------------------------ - -tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_volume_upload - -Test preconditions ------------------- - -* Volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' -* Test action 1: Create a volume VOL1 -* Test action 2: Convert VOL1 and upload image IMG1 to the Glance -* Test action 3: Wait until the status of IMG1 is 'ACTIVE' and VOL1 is 'available' -* Test action 4: Show the details of IMG1 -* **Test assertion 1:** The name of IMG1 shown is the same as the name used to upload it -* **Test assertion 2:** The disk_format of IMG1 is the same as the disk_format of VOL1 - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of uploading images. -Specifically, the test verifies that: - -* The Volume can convert volumes and upload images. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - -------------------------------------------------------------------------------------- -Test Case 2 - Volume service availability zone operations with the Cinder v2 or v3 API +Test Case 1 - Volume service availability zone operations with the Cinder v2 or v3 API -------------------------------------------------------------------------------------- Test case specification @@ -140,7 +99,7 @@ Test execution * Test action 1: List all existent availability zones * **Test assertion 1:** Verify the availability zone list length is greater than 0 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of listing availability zones. @@ -156,7 +115,7 @@ Post conditions N/A -------------------------------------------------------------------- -Test Case 3 - Volume cloning operations with the Cinder v2 or v3 API +Test Case 2 - Volume cloning operations with the Cinder v2 or v3 API -------------------------------------------------------------------- Test case specification @@ -190,7 +149,7 @@ Test execution * Test action 7: Update the name of VOL3 and description with the original value * **Test assertion 6:** Verify VOL3's bootable flag is 'False' -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of creating a cloned volume from a source volume, @@ -210,7 +169,7 @@ Post conditions N/A -------------------------------------------------------------------------- -Test Case 4 - Image copy-to-volume operations with the Cinder v2 or v3 API +Test Case 3 - Image copy-to-volume operations with the Cinder v2 or v3 API -------------------------------------------------------------------------- Test case specification @@ -249,7 +208,7 @@ Test execution * Test action 11: Update the name of VOL3 and description with the original value * **Test assertion 8:** Verify VOL3's bootable flag is 'True' -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of updating volume's bootable flag and creating @@ -270,7 +229,7 @@ Post conditions N/A ---------------------------------------------------------------------------------- -Test Case 5 - Volume creation and deletion operations with the Cinder v2 or v3 API +Test Case 4 - Volume creation and deletion operations with the Cinder v2 or v3 API ---------------------------------------------------------------------------------- Test case specification @@ -320,7 +279,7 @@ Test execution * Test action 13: Create a volume with volume size '0' * **Test assertion 12:** Verify create volume failed, a bad request error is returned in the response -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of creating a volume, getting volume @@ -343,7 +302,7 @@ Post conditions N/A -------------------------------------------------------------------------------------- -Test Case 6 - Volume service extension listing operations with the Cinder v2 or v3 API +Test Case 5 - Volume service extension listing operations with the Cinder v2 or v3 API -------------------------------------------------------------------------------------- Test case specification @@ -366,7 +325,7 @@ Test execution * Test action 1: List all cinder service extensions * **Test assertion 1:** Verify all extensions are list in the extension list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of listing all existent volume service extensions. @@ -381,7 +340,7 @@ Post conditions N/A ---------------------------------------------------------------- -Test Case 7 - Volume GET operations with the Cinder v2 or v3 API +Test Case 6 - Volume GET operations with the Cinder v2 or v3 API ---------------------------------------------------------------- Test case specification @@ -409,7 +368,7 @@ Test execution * Test action 3: Retrieve a volume with a nonexistent volume ID * **Test assertion 3:** Verify retrieve volume failed, a 'Not Found' error is returned in the response -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of getting volumes. @@ -425,7 +384,7 @@ Post conditions N/A -------------------------------------------------------------------- -Test Case 8 - Volume listing operations with the Cinder v2 or v3 API +Test Case 7 - Volume listing operations with the Cinder v2 or v3 API -------------------------------------------------------------------- Test case specification @@ -510,7 +469,7 @@ Test execution * Test action 22: List all existent volumes and paginate the volume list by desired volume IDs * **Test assertion 24:** Verify only the desired volumes are listed in the filtered volume list -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of getting a list of volumes and filtering the volume list. @@ -529,7 +488,7 @@ Post conditions N/A --------------------------------------------------------------------- -Test Case 9 - Volume metadata operations with the Cinder v2 or v3 API +Test Case 8 - Volume metadata operations with the Cinder v2 or v3 API --------------------------------------------------------------------- Test case specification @@ -561,7 +520,7 @@ Test execution * Test action 6: Update one metadata item 'key3' of VOL2 * **Test assertion 5:** Verify the metadata of VOL2 is updated -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of creating metadata for a volume, getting the @@ -580,9 +539,9 @@ Post conditions N/A ---------------------------------------------------------------------------------------- -Test Case 10 - Verification of read-only status on volumes with the Cinder v2 or v3 API ---------------------------------------------------------------------------------------- +-------------------------------------------------------------------------------------- +Test Case 9 - Verification of read-only status on volumes with the Cinder v2 or v3 API +-------------------------------------------------------------------------------------- Test case specification ----------------------- @@ -605,7 +564,7 @@ Test execution * Test action 2: Update a provided volume VOL1's read-only access mode to 'False' * **Test assertion 2:** Verify VOL1 is not in read-only access mode -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of setting and updating volume read-only access mode. @@ -620,61 +579,8 @@ Post conditions N/A -------------------------------------------------------------------------- -Test Case 11 - Volume reservation operations with the Cinder v2 or v3 API -------------------------------------------------------------------------- - -Test case specification ------------------------ - -tempest.api.volume.test_volumes_actions.VolumesActionsTest.test_reserve_unreserve_volume -tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_reserve_volume_with_negative_volume_status -tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_reserve_volume_with_nonexistent_volume_id -tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_unreserve_volume_with_nonexistent_volume_id - -Test preconditions ------------------- - -* Volume extension API - -Basic test flow execution description and pass/fail criteria ------------------------------------------------------------- - -Test execution -'''''''''''''' - -* Test action 1: Update a provided volume VOL1 as reserved -* **Test assertion 1:** Verify VOL1 is in 'attaching' status -* Test action 2: Update VOL1 as un-reserved -* **Test assertion 2:** Verify VOL1 is in 'available' status -* Test action 3: Update a provided volume VOL2 as reserved -* Test action 4: Update VOL2 as reserved again -* **Test assertion 3:** Verify update VOL2 status failed, a bad request error is returned in the response -* Test action 5: Update VOL2 as un-reserved -* Test action 6: Update a non-existent volume as reserved by using an invalid volume ID -* **Test assertion 4:** Verify update non-existent volume as reserved failed, a 'Not Found' error is returned in the response -* Test action 7: Update a non-existent volume as un-reserved by using an invalid volume ID -* **Test assertion 5:** Verify update non-existent volume as un-reserved failed, a 'Not Found' error is returned in the response - -Pass / fail criteria -'''''''''''''''''''' - -This test case evaluates the volume API ability of reserving and un-reserving volumes. -Specifically, the test verifies that: - -* Volume can be reserved and un-reserved. -* Update a non-existent volume as reserved is not allowed. -* Update a non-existent volume as un-reserved is not allowed. - -In order to pass this test, all test assertions listed in the test execution above need to pass. - -Post conditions ---------------- - -N/A - ---------------------------------------------------------------------------------------- -Test Case 12 - Volume snapshot creation/deletion operations with the Cinder v2 or v3 API +Test Case 10 - Volume snapshot creation/deletion operations with the Cinder v2 or v3 API ---------------------------------------------------------------------------------------- Test case specification @@ -753,7 +659,7 @@ Test execution * Test action 27: Create a snapshot from a volume by using an empty volume ID * **Test assertion 21:** Verify create snapshot failed, a 'Not Found' error is returned in the response -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of managing snapshot and snapshot metadata. @@ -784,7 +690,7 @@ Post conditions N/A -------------------------------------------------------------------- -Test Case 13 - Volume update operations with the Cinder v2 or v3 API +Test Case 11 - Volume update operations with the Cinder v2 or v3 API -------------------------------------------------------------------- Test case specification @@ -812,7 +718,7 @@ Test execution * Test action 3: Update a non-existent volume by using a random generated volume ID * **Test assertion 3:** Verify update volume failed, a 'Not Found' error is returned in the response -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test case evaluates the volume API ability of updating volume attributes. @@ -828,3 +734,41 @@ Post conditions --------------- N/A + +------------------------------------------------------------------- +Test Case 12 - Volume list version operation with the Cinder v3 API +------------------------------------------------------------------- + +Test case specification +----------------------- + +tempest.api.volume.test_versions.VersionsTest.test_list_versions + +Test preconditions +------------------ + +* Volume API + +Basic test flow execution description and pass/fail criteria +------------------------------------------------------------ + +Test execution +'''''''''''''' + +* Test action 1: List volume API versions +* **Test assertion 1:** The list versions operation is successful executed + +Pass / Fail criteria +'''''''''''''''''''' + +This test case evaluates the volume API ability of listing volume API versions. +Specifically, the test verifies that: + +* Successfully listing volume API versions. + +In order to pass this test, all test assertions listed in the test execution above need to pass. + +Post conditions +--------------- + +N/A diff --git a/docs/testing/user/testspecification/tempest_trunk_ports/index.rst b/docs/testing/user/testspecification/tempest_trunk_ports/index.rst index 76ba9a79..f82fc468 100644 --- a/docs/testing/user/testspecification/tempest_trunk_ports/index.rst +++ b/docs/testing/user/testspecification/tempest_trunk_ports/index.rst @@ -35,9 +35,9 @@ is able to run independently, i.e. irrelevant of the state created by a previous test. For detailed information on the individual steps and assertions performed by the tests, review the Python source code accessible via the following links: -- `Neutron Trunk API tests <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk.py>`_ -- `Neutron Trunk API trunk details <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk_details.py>`_ -- `Neutron Trunk API negative tests <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk_negative.py>`_ +- `Neutron Trunk API tests <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/api/test_trunk.py>`_ +- `Neutron Trunk API trunk details <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/api/test_trunk_details.py>`_ +- `Neutron Trunk API negative tests <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/api/test_trunk_negative.py>`_ **Trunk port and sub-port CRUD operations:** @@ -46,38 +46,17 @@ These tests cover the CRUD (Create, Read, Update, Delete) life-cycle operations of trunk ports and subports. Implementation: -`TrunkTestInheritJSONBase <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk.py#L228>`_ -and -`TrunkTestJSON <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk.py#L83>`_. +`TrunkTestJSON <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/api/test_trunk.py#L71>`_ -- neutron.tests.tempest.api.test_trunk.TrunkTestInheritJSONBase.test_add_subport -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_add_subport -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_create_show_delete_trunk -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_create_trunk_empty_subports_list -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_create_trunk_subports_not_specified -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_create_update_trunk -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_create_update_trunk_with_description -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_delete_trunk_with_subport_is_allowed -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_get_subports -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_list_trunks -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_remove_subport -- neutron.tests.tempest.api.test_trunk.TrunkTestJSON.test_show_trunk_has_project_id - - -**MTU-related operations:** - -These tests validate that trunk ports and subports can be created and added -when specifying valid MTU sizes. These tests do not include negative tests -covering invalid MTU sizes. - -Implementation: -`TrunkTestMtusJSON <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk.py#L302>`_ - -- neutron.tests.tempest.api.test_trunk.TrunkTestMtusJSON.test_add_subport_with_mtu_equal_to_trunk -- neutron.tests.tempest.api.test_trunk.TrunkTestMtusJSON.test_add_subport_with_mtu_smaller_than_trunk -- neutron.tests.tempest.api.test_trunk.TrunkTestMtusJSON.test_create_trunk_with_mtu_equal_to_subport -- neutron.tests.tempest.api.test_trunk.TrunkTestMtusJSON.test_create_trunk_with_mtu_greater_than_subport +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_add_subports +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_create_show_delete_trunk +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_create_trunk_empty_subports_list +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_create_trunk_subports_not_specified +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_delete_trunk_with_subport_is_allowed +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_get_subports +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_list_trunks +- neutron_tempest_plugin.api.test_trunk.TrunkTestJSON.test_remove_subport **API for listing query results:** @@ -86,17 +65,15 @@ These tests verify that listing operations of trunk port objects work. This functionality is required for CLI and UI operations. Implementation: -`TrunksSearchCriteriaTest <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk.py#L346>`_ +`TrunksSearchCriteriaTest <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/api/test_trunk.py#L306>`_ -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_no_pagination_limit_0 -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_page_reverse_asc -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_page_reverse_desc -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_page_reverse_with_href_links -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_with_href_links -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_with_marker -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_sorts_asc -- neutron.tests.tempest.api.test_trunk.TrunksSearchCriteriaTest.test_list_sorts_desc +- neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_no_pagination_limit_0 +- neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination +- neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_page_reverse_asc +- neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_page_reverse_desc +- neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_pagination_with_marker +- neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_sorts_asc +- neutron_tempest_plugin.api.test_trunk.TrunksSearchCriteriaTest.test_list_sorts_desc **Query trunk port details:** @@ -104,11 +81,11 @@ Implementation: These tests validate that all attributes of trunk port objects can be queried. Implementation: -`TestTrunkDetailsJSON <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk_details.py#L20>`_ +`TestTrunkDetailsJSON <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/api/test_trunk_details.py#L20>`_ -- neutron.tests.tempest.api.test_trunk_details.TestTrunkDetailsJSON.test_port_resource_empty_trunk_details -- neutron.tests.tempest.api.test_trunk_details.TestTrunkDetailsJSON.test_port_resource_trunk_details_no_subports -- neutron.tests.tempest.api.test_trunk_details.TestTrunkDetailsJSON.test_port_resource_trunk_details_with_subport +- neutron_tempest_plugin.api.test_trunk_details.TestTrunkDetailsJSON.test_port_resource_empty_trunk_details +- neutron_tempest_plugin.api.test_trunk_details.TestTrunkDetailsJSON.test_port_resource_trunk_details_no_subports +- neutron_tempest_plugin.api.test_trunk_details.TestTrunkDetailsJSON.test_port_resource_trunk_details_with_subport **Negative tests:** @@ -117,30 +94,27 @@ These group of tests comprise negative tests which verify that invalid operation are handled correctly by the system under test. Implementation: -`TrunkTestNegative <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/api/test_trunk_negative.py#L24>`_ - -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_add_subport_duplicate_segmentation_details -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_add_subport_passing_dict -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_add_subport_port_id_disabled_trunk -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_add_subport_port_id_uses_trunk_port_id -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_subport_invalid_inherit_network_segmentation_type -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_subport_missing_segmentation_id -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_subport_nonexistent_port_id -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_subport_nonexistent_trunk -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_duplicate_subport_segmentation_ids -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_nonexistent_port_id -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_nonexistent_subport_port_id -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_with_subport_missing_port_id -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_with_subport_missing_segmentation_id -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_with_subport_missing_segmentation_type -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_delete_port_in_use_by_subport -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_delete_port_in_use_by_trunk -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_delete_trunk_disabled_trunk -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_remove_subport_not_found -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_remove_subport_passing_dict -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestJSON.test_remove_subport_port_id_disabled_trunk -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestMtusJSON.test_add_subport_with_mtu_greater_than_trunk -- neutron.tests.tempest.api.test_trunk_negative.TrunkTestMtusJSON.test_create_trunk_with_mtu_smaller_than_subport +`TrunkTestNegative <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/api/test_trunk_negative.py#L27>`_ + +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_add_subport_duplicate_segmentation_details +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_add_subport_passing_dict +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_add_subport_port_id_disabled_trunk +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_add_subport_port_id_uses_parent_port_id +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_subport_missing_segmentation_id +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_subport_nonexistent_port_id +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_subport_nonexistent_trunk +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_duplicate_subport_segmentation_ids +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_nonexistent_port_id +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_nonexistent_subport_port_id +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_with_subport_missing_port_id +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_with_subport_missing_segmentation_id +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_create_trunk_with_subport_missing_segmentation_type +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_delete_port_in_use_by_subport +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_delete_port_in_use_by_trunk +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_delete_trunk_disabled_trunk +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_remove_subport_not_found +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_remove_subport_passing_dict +- neutron_tempest_plugin.api.test_trunk_negative.TrunkTestJSON.test_remove_subport_port_id_disabled_trunk **Scenario tests (tests covering more than one functionality):** @@ -151,7 +125,6 @@ functions as intended. To this end, they boot up two VMs with trunk ports and sub ports and verify connectivity between those VMs. Implementation: -`TrunkTest <https://github.com/openstack/neutron/blob/stable/pike/neutron/tests/tempest/scenario/test_trunk.py#L45>`_ +`TrunkTest <https://github.com/openstack/neutron-tempest-plugin/blob/0.3.0/neutron_tempest_plugin/scenario/test_trunk.py#L41>`_ -- neutron.tests.tempest.scenario.test_trunk.TrunkTest.test_subport_connectivity -- neutron.tests.tempest.scenario.test_trunk.TrunkTest.test_trunk_subport_lifecycle +- neutron_tempest_plugin.scenario.test_trunk.TrunkTest.test_trunk_subport_lifecycle diff --git a/docs/testing/user/testspecification/tempest_vm_lifecycle/index.rst b/docs/testing/user/testspecification/tempest_vm_lifecycle/index.rst index 5618ebf8..1621d8f5 100644 --- a/docs/testing/user/testspecification/tempest_vm_lifecycle/index.rst +++ b/docs/testing/user/testspecification/tempest_vm_lifecycle/index.rst @@ -59,7 +59,7 @@ Each test case is able to run independently, i.e. irrelevant of the state created by a previous test. Specifically, every test performs clean-up operations which return the system to the same state as before the test. -All these test cases are included in the test case dovetail.tempest.vm_lifecycle of +All these test cases are included in the test case functest.tempest.vm_lifecycle of OVP test suite. Test Descriptions @@ -174,7 +174,7 @@ Test execution * **Test assertion 8:** Verify VM1's addresses have been refreshed after disassociating FIP1 * Test action 10: Delete SG1, IMG1, KEYP1, VOL1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates a minimum basic scenario. Specifically, the test verifies that: @@ -227,7 +227,7 @@ Test execution * **Test assertion 3:** Verify SRC_HOST does not equal to DST_HOST * Test action 11: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to cold migrate VMs. Specifically, the test verifies that: @@ -274,7 +274,7 @@ Test execution * **Test assertion 3:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 * Test action 8: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to pause and unpause VMs. Specifically, the test verifies that: @@ -319,7 +319,7 @@ Test execution * **Test assertion 1:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 * Test action 6: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to reboot servers. Specifically, the test verifies that: @@ -362,7 +362,7 @@ Test execution * **Test assertion 1:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 * Test action 6: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to rebuild servers. Specifically, the test verifies that: @@ -407,7 +407,7 @@ Test execution * **Test assertion 1:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 * Test action 8: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to resize servers. Specifically, the test verifies that: @@ -453,7 +453,7 @@ Test execution * **Test assertion 2:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 * Test action 8: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to stop and start servers. Specifically, the test verifies that: @@ -501,7 +501,7 @@ Test execution * **Test assertion 2:** Verify can ping FIP1 successfully and can SSH to VM1 via FIP1 * Test action 8: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to suspend and resume servers. Specifically, the test verifies that: @@ -553,7 +553,7 @@ Test execution * **Test assertion 4:** Verify VM1 status is 'ACTIVE' * Test action 10: Delete KEYP1, VM1 and FIP1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to suspend and resume servers in sequence. @@ -597,7 +597,7 @@ Test execution * **Test assertion 1:** VM1's status is 'ACTIVE' * Test action 6: Delete VM1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to resize volume backed servers. @@ -646,7 +646,7 @@ Test execution * **Test assertion 1:** Verify the values written and read are the same * Test action 9: Delete SG1, KEYP1 and VM1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to shelve and unshelve servers. @@ -694,7 +694,7 @@ Test execution * **Test assertion 1:** Verify T_STAMP1 equals to T_STAMP2 * Test action 9: Delete SG1, KEYP1 and VM1 -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates the ability to shelve and unshelve volume backed servers. diff --git a/docs/testing/user/testspecification/tempest_volume/index.rst b/docs/testing/user/testspecification/tempest_volume/index.rst index 7672e4de..64888c99 100644 --- a/docs/testing/user/testspecification/tempest_volume/index.rst +++ b/docs/testing/user/testspecification/tempest_volume/index.rst @@ -2,9 +2,9 @@ .. http://creativecommons.org/licenses/by/4.0 .. (c) Ericsson AB -=========================================== +================================= Tempest Volume test specification -=========================================== +================================= Scope @@ -35,7 +35,7 @@ The test area is structured in individual tests as listed below. For detailed information on the individual steps and assertions performed by the tests, review the Python source code accessible via the following links: -All these test cases are included in the test case dovetail.tempest.volume of +All these test cases are included in the test case functest.tempest.volume of OVP test suite. diff --git a/docs/testing/user/testspecification/vnf/index.rst b/docs/testing/user/testspecification/vnf/index.rst index e53596e9..d13634a1 100644 --- a/docs/testing/user/testspecification/vnf/index.rst +++ b/docs/testing/user/testspecification/vnf/index.rst @@ -82,14 +82,14 @@ return the system to the same state as before the test. Test Descriptions ================= ----------------------------------------------------------------- +------------------ Test Case 1 - vEPC ----------------------------------------------------------------- +------------------ Short name ---------- -dovetail.vnf.vepc +functest.vnf.vepc Use case specification @@ -136,7 +136,7 @@ Test execution * Test action 5: The deployed VMs are deleted. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' The VNF Manager (juju) should be deployed successfully @@ -153,14 +153,14 @@ Post conditions The clean-up operations are run. ----------------------------------------------------------------- +------------------ Test Case 2 - vIMS ----------------------------------------------------------------- +------------------ Short name ---------- -dovetail.vnf.vims +functest.vnf.vims Use case specification ---------------------- @@ -202,7 +202,7 @@ Test execution * Test action 5: The deployed VMs are deleted. -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' The VNF orchestrator (Cloudify) should be deployed successfully. diff --git a/docs/testing/user/testspecification/vping/index.rst b/docs/testing/user/testspecification/vping/index.rst index 6ba29a1d..355018c6 100644 --- a/docs/testing/user/testspecification/vping/index.rst +++ b/docs/testing/user/testspecification/vping/index.rst @@ -3,7 +3,7 @@ .. (c) Ericsson AB ======================== -Vping test specification +Vping Test Specification ======================== .. toctree:: @@ -34,7 +34,7 @@ References - https://pypi.org/project/scp/ -Definitions and abbreviations +Definitions and Abbreviations ============================= The following terms and abbreviations are used in conjunction with this test @@ -42,10 +42,10 @@ area - ICMP - Internet Control Message Protocol - L3 - Layer 3 -- NFVi - Network functions virtualization infrastructure -- SCP - Secure Copy -- SSH - Secure Shell -- VM - Virtual machine +- NFVi - Network Functions Virtualization infrastructure +- SCP - Secure CoPy +- SSH - Secure SHell +- VM - Virtual Machine System Under Test (SUT) @@ -73,7 +73,7 @@ Test Case 1 - vPing using userdata provided by nova metadata service Short name ---------- -dovetail.vping.userdata +functest.vping.userdata Use case specification @@ -106,39 +106,46 @@ checking the console output of the source VMs. Test execution '''''''''''''' -* Test action 1: +* **Test action 1:** * Create a private tenant network by using neutron client * Create one subnet and one router in the network by neutron client * Add one interface between the subnet and router * Add one gateway route to the router by neutron client * Store the network id in the response * **Test assertion 1:** The network id, subnet id and router id can be found in the response -* Test action 2: +* **Test action 2:** * Create an security group by using neutron client * Store the security group id parameter in the response * **Test assertion 2:** The security group id can be found in the response -* Test action 3: boot VM1 by using nova client with configured name, image, flavor, private tenant - network created in test action 1, security group created in test action 2 +* **Test action 3:** + * Boot VM1 by using nova client with configured name, image, flavor, private tenant + network created in test action 1 and security group created in test action 2 * **Test assertion 3:** The VM1 object can be found in the response -* Test action 4: Generate ping script with the IP of VM1 to be passed as userdata provided by - the **nova metadata service**. -* Test action 5: Boot VM2 by using nova client with configured name, image, flavor, private tenant - network created in test action 1, security group created in test action 2, userdata created - in test action 4 +* **Test action 4:** + * Generate ping script with the IP of VM1 to be passed as userdata provided by + the **nova metadata service** +* **Test action 5:** + * Boot VM2 by using nova client with configured name, image, flavor, private tenant + network created in test action 1, security group created in test action 2, and + userdata created in test action 4 * **Test assertion 4:** The VM2 object can be found in the response -* Test action 6: Inside VM2, the ping script is executed automatically when booted and it contains a - loop doing the ping until the return code is 0 or timeout reached. For each ping, when the return - code is 0, "vPing OK" is printed in the VM2 console-log, otherwise, "vPing KO" is printed. - Monitoring the console-log of VM2 to see the response generated by the script. +* **Test action 6:** + * Inside VM2, the ping script is executed automatically when booted and it contains a + loop doing the ping until the return code is 0 or timeout reached + * For each ping, when the return code is 0, "vPing OK" is printed in the VM2 console-log, + otherwise, "vPing KO" is printed + * Monitoring the console-log of VM2 to see the response generated by the script * **Test assertion 5:** "vPing OK" is detected, when monitoring the console-log in VM2 -* Test action 7: delete VM1, VM2 +* **Test action 7:** + * Delete VM1, VM2 * **Test assertion 6:** VM1 and VM2 are not present in the VM list -* Test action 8: delete security group, gateway, interface, router, subnet and network +* **Test action 8:** + * Delete security group, gateway, interface, router, subnet and network * **Test assertion 7:** The security group, gateway, interface, router, subnet and network are no longer present in the lists after deleting -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates basic NFVi capabilities of the system under test. @@ -173,7 +180,7 @@ Test Case 2 - vPing using SSH to a floating IP Short name ---------- -dovetail.vping.ssh +functest.vping.ssh Use case specification @@ -207,45 +214,55 @@ Test execution '''''''''''''' -* Test action 1: +* **Test action 1:** * Create a private tenant network by neutron client * Create one subnet and one router are created in the network by using neutron client * Create one interface between the subnet and router * Add one gateway route to the router by neutron client * Store the network id in the response * **Test assertion 1:** The network id, subnet id and router id can be found in the response -* Test action 2: +* **Test action 2:** * Create an security group by using neutron client * Store the security group id parameter in the response * **Test assertion 2:** The security group id can be found in the response -* Test action 3: Boot VM1 by using nova client with configured name, image, flavor, private tenant - network created in test action 1, security group created in test action 2 +* **Test action 3:** + * Boot VM1 by using nova client with configured name, image, flavor, private tenant + network created in test action 1 and security group created in test action 2 * **Test assertion 3:** The VM1 object can be found in the response -* Test action 4: Boot VM2 by using nova client with configured name, image, flavor, private tenant - network created in test action 1, security group created in test action 2 +* **Test action 4:** + * Boot VM2 by using nova client with configured name, image, flavor, private tenant + network created in test action 1 and security group created in test action 2 * **Test assertion 4:** The VM2 object can be found in the response -* Test action 5: create one floating IP by using neutron client, storing the floating IP address - returned in the response +* **Test action 5:** + * Create one floating IP by using neutron client, storing the floating IP address + returned in the response * **Test assertion 5:** Floating IP address can be found in the response -* Test action 6: Assign the floating IP address created in test action 5 to VM2 by using nova client +* **Test action 6:** + * Assign the floating IP address created in test action 5 to VM2 by using nova client * **Test assertion 6:** The assigned floating IP can be found in the VM2 console log file -* Test action 7: Establish SSH connection between the test host and VM2 through the floating IP +* **Test action 7:** + * Establish SSH connection between the test host and VM2 through the floating IP * **Test assertion 7:** SSH connection between the test host and VM2 is established within 300 seconds -* Test action 8: Copy the Ping script from the test host to VM2 by using SCPClient +* **Test action 8:** + * Copy the Ping script from the test host to VM2 by using SCPClient * **Test assertion 8:** The Ping script can be found inside VM2 -* Test action 9: Inside VM2, to execute the Ping script to ping VM1, the Ping script contains a - loop doing the ping until the return code is 0 or timeout reached, for each ping, when the return - code is 0, "vPing OK" is printed in the VM2 console-log, otherwise, "vPing KO" is printed. - Monitoring the console-log of VM2 to see the response generated by the script. +* **Test action 9:** + * Inside VM2, to execute the Ping script to ping VM1, the Ping script contains a + loop doing the ping until the return code is 0 or timeout reached + * For each ping, when the return code is 0, "vPing OK" is printed in the VM2 console-log, + otherwise, "vPing KO" is printed + * Monitoring the console-log of VM2 to see the response generated by the script * **Test assertion 9:** "vPing OK" is detected, when monitoring the console-log in VM2 -* Test action 10: delete VM1, VM2 +* **Test action 10:** + * Delete VM1, VM2 * **Test assertion 10:** VM1 and VM2 are not present in the VM list -* Test action 11: delete floating IP, security group, gateway, interface, router, subnet and network +* **Test action 11:** + * Delete floating IP, security group, gateway, interface, router, subnet and network * **Test assertion 11:** The security group, gateway, interface, router, subnet and network are no longer present in the lists after deleting -Pass / fail criteria +Pass / Fail criteria '''''''''''''''''''' This test evaluates basic NFVi capabilities of the system under test. |